Thursday 13 August 2026

1. The exit distribution

First, $\tau <\infty$ almost surely: Brownian motion has ${lim sup}_t{B}_t=+\infty$, so it cannot stay in a bounded interval forever.

Now the justification that matters. $B$ is a martingale, but optional stopping does not apply to an arbitrary unbounded stopping time. What rescues the argument is that the stopped process $B_{t\wedge \tau }$ never leaves $[-b,a]$, so it is uniformly bounded, hence uniformly integrable, and $B_{t\wedge \tau }\to B_{\tau }$ almost surely. Bounded convergence gives

$$𝔼[B_{\tau }]={lim}_{t\to \infty }𝔼[B_{t\wedge \tau }]=B_0=0.$$

Writing $p=\mathbb{P}(B_{\tau }=a)$ and using that $B_{\tau }\in \{a,-b\}$,

$$pa-(1-p)b=0⟹p=\frac{b}{a+b}.$$

2. The expected exit time

$M_t=B_t^2-t$ is a martingale, so $𝔼[B_{t\wedge \tau }^2]=𝔼[t\wedge \tau ]$ for every $t$. The left side is bounded by $max(a,b{)}^2$, so the right side is too, and letting $t\to \infty$ with monotone convergence on the right and bounded convergence on the left gives $𝔼[\tau ]=𝔼[B_{\tau }^2]<\infty$. Then

$$𝔼[B_{\tau }^2]=p{a}^2+(1-p)b^2=\frac{b{a}^2+a{b}^2}{a+b}=ab.$$

$$𝔼[\tau ]=ab$$

Note how the finiteness of $𝔼[\tau ]$ came out of the argument rather than being assumed; assuming it is the usual gap in a hurried solution.

3. Adding drift

$X$ itself is not a martingale, so exponentiate. For any $\theta$, $e^{\theta B_t-{\theta }^{2}t/2}$ is a martingale; take $\theta =-2\mu$, whose ${\theta }^2/2$ equals $2{\mu }^2$, and observe

$$e^{-2\mu X_t}=e^{-2\mu B_t-2{\mu }^{2}t},$$

so $M_t=e^{-2\mu X_t}$ is a martingale with $M_0=1$. On the stopped interval $X$ stays in $[-b,a]$, so $M_{t\wedge {\tau }_{\mu }}$ lies in $[e^{-2\mu a},e^{2\mu b}]$ — bounded again, which is what licenses optional stopping. With $p_{\mu }=\mathbb{P}(X_{{\tau }_{\mu }}=a)$,

$$1=p_{\mu }{e}^{-2\mu a}+(1-p_{\mu })e^{2\mu b}$$

$$p_{\mu }=\frac{e^{2\mu b}-1}{e^{2\mu b}-e^{-2\mu a}}=\frac{1-e^{-2\mu b}}{1-e^{-2\mu (a+b)}}$$

As $\mu \to 0$ both numerator and denominator vanish; to first order they are $2\mu b$ and $2\mu (a+b)$, so $p_{\mu }\to b/(a+b)$, matching part 1. As $\mu \to \infty$, $p_{\mu }\to 1$, as it must.

Note. The tempting move is to apply optional stopping to $X_t-\mu t$ and stop there, but that only reproduces part 1 in disguise: it gives one equation relating $𝔼[{\tau }_{\mu }]$ and $p_{\mu }$, two unknowns. The exponential martingale is the right tool precisely because it is bounded on the interval and eliminates ${\tau }_{\mu }$ from the equation entirely.

1. From posterior to penalty

Up to terms free of $\theta$,

$$-\log p(\theta \mid y)=\frac{(y-\theta {)}^2}{2{\sigma }^2}+\frac{|\theta |}{b}+\text{const},$$

so multiplying by ${\sigma }^2$ (which changes the objective but not its minimizer) the MAP estimator minimizes

$$\frac{1}{2}(y-\theta {)}^2+\lambda |\theta |,\lambda =\frac{{\sigma }^2}{b}.$$

This is the lasso objective in one dimension. The penalty weight is a ratio of scales: a tighter prior (small $b$) or noisier data (large ${\sigma }^2$) both penalize more.

2. Soft thresholding

The objective is convex but not differentiable at $0$, so use the subdifferential. Away from zero, stationarity reads $\theta -y+\lambda \mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}(\theta )=0$, giving $\theta =y-\lambda$ when $\theta >0$ (consistent only if $y>\lambda$) and $\theta =y+\lambda$ when $\theta <0$ (only if $y<-\lambda$). At zero, optimality requires $0\in -y+\lambda [-1,1]$, that is $|y|\le \lambda$. Collecting the cases,

$${\widehat{\theta }}_{\mathrm{M}\mathrm{A}\mathrm{P}}=\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}(y)(|y|-\lambda {)}^{+}.$$

3. The Gaussian prior, and the source of the difference

With $\theta ~N(0,{\tau }^2)$ the objective is $\frac{(y-\theta {)}^2}{2{\sigma }^2}+\frac{{\theta }^2}{2{\tau }^2}$, smooth everywhere, and setting the derivative to zero gives pure linear shrinkage:

$${\widehat{\theta }}_{\mathrm{r}\mathrm{i}\mathrm{d}\mathrm{g}\mathrm{e}}=\frac{{\tau }^2}{{\tau }^2+{\sigma }^2}y,$$

which is zero only when $y=0$.

The distinction is not about how heavy the prior's tails are; it is about the derivative at the origin. The Laplace penalty has a kink, so its subdifferential at $0$ is the whole interval $[-\lambda ,\lambda ]$, and any data gradient smaller in magnitude than $\lambda$ can be absorbed — zero is a genuine minimizer for a whole range of $y$. The Gaussian penalty is differentiable with derivative exactly $0$ at the origin, so it exerts no force there: the stationarity condition always balances at a nonzero $\theta$ whenever $y\ne 0$. Sparsity comes from non-differentiability at zero, and this is why ${\ell }_q$ penalties with $q\le 1$ threshold while $q>1$ does not.

Note. This is a statement about the posterior mode only. Under the Laplace prior the posterior mean is a smooth, strictly monotone function of $y$ and is never exactly zero, so the sparsity is an artifact of choosing the mode as the summary — worth remembering before describing the lasso as "the Bayesian estimator" for a Laplace prior.