Thursday 3 September 2026

The one idea that cracks it

Do not try to integrate over configurations of arrivals in $[0,T]$. Condition on the first inter-arrival time $X_1~\mathrm{E}\mathrm{x}\mathrm{p}(\lambda )$ and exploit the memorylessness (strong Markov / independent increments) of the Poisson process:

• If $X_1\ge \tau$, the interval $[0,\tau ]$ is empty, so the lull completes at time $\tau$ with no orders seen.
• If $X_1<\tau$, an order arrives at time $X_1$; by memorylessness the process restarts afresh from that instant, and everything we are asked about renews as an independent copy.

This renewal-at-each-arrival is the step a candidate skips. Everything below is just executing it.

Part 1: expected number of orders

Let $p=\mathbb{P}(X_1<\tau )=1-e^{-\lambda \tau }$. Conditioning on the first arrival:

$$𝔼[N]=p(1+𝔼[N])+(1-p)·0.$$

The $1$ counts the order that just arrived; the $𝔼[N]$ is the independent restart. Solving,

$$𝔼[N]=\frac{p}{1-p}=\frac{1-e^{-\lambda \tau }}{e^{-\lambda \tau }}=e^{\lambda \tau }-1.$$

Equivalently: $N$ is geometric, counting "short" gaps (each with success probability $1-p=e^{-\lambda \tau }$ of being the terminating long gap), so $𝔼[N]=\frac{1-e^{-\lambda \tau }}{e^{-\lambda \tau }}$.

Part 2: expected time

Let $m=𝔼[T]$. Condition on $X_1$ again. If $X_1=x\ge \tau$ the lull completes at $\tau$; if $x<\tau$ we have burned time $x$ and restart, so $T=x+T^{\prime }$ with $T^{\prime }$ an independent copy:

$$m=\tau e^{-\lambda \tau }+{\int }_0^{\tau }(x+m)\lambda e^{-\lambda x}dx.$$

Use ${\int }_0^{\tau }\lambda e^{-\lambda x}dx=1-e^{-\lambda \tau }$ and, by parts,

$${\int }_0^{\tau }x\lambda e^{-\lambda x}dx=\frac{1-e^{-\lambda \tau }}{\lambda }-\tau e^{-\lambda \tau }.$$

Substituting, the $\pm \tau e^{-\lambda \tau }$ terms cancel:

$$m=\frac{1-e^{-\lambda \tau }}{\lambda }+m(1-e^{-\lambda \tau }).$$

Hence $m{e}^{-\lambda \tau }={\displaystyle \frac{1-e^{-\lambda \tau }}{\lambda }}$, giving

$$𝔼[T]=\frac{e^{\lambda \tau }-1}{\lambda }.$$

Note the clean relation $𝔼[T]=𝔼[N]/\lambda$. That is not a coincidence: the wasted "short" gaps sum to $(e^{\lambda \tau }-1)/\lambda -\tau$ in expectation, and adding the final $\tau$ recovers $𝔼[T]$; the $\tau$'s cancel exactly.

Part 3: numbers

Here $\lambda \tau =6\times \frac{1}{3}=2$, so $e^{\lambda \tau }=e^2\approx 7.389$.

Closing note

The qualitative punchline worth internalizing: waiting for a fixed quiet window costs time that grows exponentially in $\lambda \tau$, not linearly. A tempting wrong answer is to reason "a gap of length $\tau$ is $e^{-\lambda \tau }$-likely, and I get a fresh gap every $1/\lambda$, so $𝔼[T]\approx \tau e^{\lambda \tau }$" — right order of magnitude in the blow-up but the wrong constant; the exact answer is $(e^{\lambda \tau }-1)/\lambda$.

Answers: $𝔼[N]=e^{\lambda \tau }-1,𝔼[T]={\displaystyle \frac{e^{\lambda \tau }-1}{\lambda }},$ numerically $\approx 6.39$ orders and $\approx 1.065$ minutes.

(a) The two decision rules

Under $M_1$ the MLE is $\widehat{\mu }=\overline{X}$, giving $-2{\ell }_1=n\log (2\pi )+{\sum }_i(X_i-\overline{X}{)}^2=n\log (2\pi )+S$. Under $M_0$, $\widehat{\mu }=0$, so $-2{\ell }_0=n\log (2\pi )+{\sum }_i{X}_i^2$. Using the decomposition ${\sum }_i{X}_i^2=S+n{\overline{X}}^2=S+T$,

$$-2{\ell }_0-(-2{\ell }_1)=(S+T)-S=T.$$

Thus AIC${}_1<$AIC${}_0$ iff $-2{\ell }_1+2<-2{\ell }_0+0$, i.e. $T>2(k_1-k_0)=2$. Likewise BIC selects $M_1$ iff $T>\log n$. The generic fact behind this: twice the fitted log-likelihood gap of a one-parameter extension equals the score/Wald statistic, here exactly $T=n{\overline{X}}^2$.

Distribution of $T$ under $\mu =0$. With $\mu =0$, $\overline{X}~N(0,1/n)$, so $T=n{\overline{X}}^2~{\chi }_1^2$ exactly (no asymptotics needed here). Hence

$$Pr(\text{AIC picks }{M}_1)=Pr({\chi }_1^2>2)=2(1-\Phi (\sqrt{2})).$$

Numerically $\sqrt{2}\approx 1.4142$, $\Phi (1.4142)\approx 0.92135$, so the probability is $2(0.07865)\approx 0.1573$. This does not shrink with $n$: AIC overfits by one parameter with probability $\approx 15.7\%$ regardless of sample size.

For BIC, $Pr({\chi }_1^2>\log n)\to 0$. Using the chi-square tail $Pr({\chi }_1^2>x)=2(1-\Phi (\sqrt{x}))~\sqrt{2/(\pi x)}{e}^{-x/2}$ with $x=\log n$ (so $e^{-x/2}=n^{-1/2}$),

$$Pr(\text{BIC picks }{M}_1)~\sqrt{\frac{2}{\pi \log n}}{n}^{-1/2}.$$

BIC is consistent: it discards the spurious parameter with probability $\to 1$.

(b) Exact LOOCV scores

For $M_0$ the prediction is the constant $0$ irrespective of which point is held out, so its LOOCV score is ${\mathrm{C}\mathrm{V}}_0={\sum }_i{X}_i^2=S+T$.

For $M_1$, the held-out mean is ${\widehat{\mu }}_{-i}=\frac{n\overline{X}-X_i}{n-1}$. The one-line computation people miss is the residual identity:

$$X_i-{\widehat{\mu }}_{-i}=X_i-\frac{n\overline{X}-X_i}{n-1}=\frac{n{X}_i-n\overline{X}}{n-1}=\frac{n}{n-1}(X_i-\overline{X}).$$

Therefore

$${\mathrm{C}\mathrm{V}}_1=\sum _i(X_i-{\widehat{\mu }}_{-i}{)}^2=\frac{n^2}{(n-1{)}^2}S.$$

LOOCV prefers $M_1$ iff ${\mathrm{C}\mathrm{V}}_1<{\mathrm{C}\mathrm{V}}_0$, i.e. $\frac{n^2}{(n-1{)}^2}S<S+T$, which rearranges to

$$T>S·\frac{n^2-(n-1{)}^2}{(n-1{)}^2}=S·\frac{2n-1}{(n-1{)}^2}.$$

So LOOCV uses the same statistic $T$ as AIC, but against a random, data-dependent threshold $S\frac{2n-1}{(n-1{)}^2}$ rather than the fixed value $2$.

(c) Exact overfit probability of LOOCV

Under $\mu =0$ we need $Pr(T>S\frac{2n-1}{(n-1{)}^2})$. Two facts make this exact:

1. $T=n{\overline{X}}^2~{\chi }_1^2$ and $S~{\chi }_{n-1}^2$;
2. $\overline{X}$ and $S$ are independent — this is the key step (Cochran's theorem, or Basu: the ancillary $S$ is independent of the complete sufficient $\overline{X}$ in the normal family). Without it the ratio would not have an $F$ distribution.

Hence ${\displaystyle \frac{T/1}{S/(n-1)}}~F_{1,n-1}$, and the event $T>S\frac{2n-1}{(n-1{)}^2}$ is

$$\frac{(n-1)T}{S}>(n-1)·\frac{2n-1}{(n-1{)}^2}=\frac{2n-1}{n-1}=2+\frac{1}{n-1}.$$

Therefore

$$\boxed{Pr(\text{LOOCV picks }{M}_1)=Pr(F_{1,n-1}>2+\frac{1}{n-1}).}$$

As $n\to \infty$, $F_{1,n-1}\Rightarrow {\chi }_1^2$ and the threshold $\to 2$, so this probability $\to Pr({\chi }_1^2>2)\approx 0.1573$ — exactly the AIC overfit rate.

Conclusion. LOOCV is asymptotically equivalent to AIC here: both keep a spurious parameter with limiting probability $\approx 15.7\%$, and neither is consistent. BIC alone drives the overfit probability to $0$ (at rate $~\sqrt{2/(\pi \log n)}{n}^{-1/2}$). The slight finite-$n$ conservatism of LOOCV — its expected threshold is $𝔼[S]\frac{2n-1}{(n-1{)}^2}=(n-1)\frac{2n-1}{(n-1{)}^2}=2+\frac{1}{n-1}$, just above AIC's $2$ — vanishes as $n\to \infty$.

Closing note

The tempting wrong answer is to declare cross-validation "honest" and therefore consistent for model selection. It is honest as an estimator of prediction error, but choosing the model with the smallest CV error is a different task: the effective complexity penalty LOOCV imposes is $O(1)$ per parameter, matching AIC, not the $O(\log n)$ needed for selection consistency. The $F_{1,n-1}$ identity above makes the equivalence exact rather than merely asymptotic in spirit, and it rests entirely on the independence of $\overline{X}$ and $S$.