Saturday 5 September 2026

Setting up the Lagrangian

Minimize $\frac{1}{2}{\sum }_i{w}_i^2{\sigma }_i^2$ (the $\frac{1}{2}$ is cosmetic) subject to the two linear constraints ${\sum }_i{w}_i=1$ and ${\sum }_i{w}_i{\mu }_i=m$. The objective is strictly convex in $w$ and the constraints are affine, so a stationary point of the Lagrangian is the global minimizer — Lagrange multipliers give a necessary and sufficient condition here.

The step a candidate skips is using two multipliers, one per constraint:

$$ℒ(w,\lambda ,\gamma )=\frac{1}{2}\sum _i{w}_i^2{\sigma }_i^2-\lambda (\sum _i{w}_i-1)-\gamma (\sum _i{w}_i{\mu }_i-m).$$

Stationarity $\partial ℒ/\partial w_i=0$ gives

$$w_i{\sigma }_i^2-\lambda -\gamma {\mu }_i=0,$$

so the optimal weight is

$$w_i=\frac{\lambda +\gamma {\mu }_i}{{\sigma }_i^2}.$$

Each weight is an affine function of ${\mu }_i$ scaled by the precision $1/{\sigma }_i^2$ — this is the two-fund structure. Now impose the constraints. Writing $A={\sum }_{i}1/{\sigma }_i^2$, $B={\sum }_i{\mu }_i/{\sigma }_i^2$, $C={\sum }_i{\mu }_i^2/{\sigma }_i^2$:

$$\sum _i{w}_i=\lambda A+\gamma B=1,$$

$$\sum _i{w}_i{\mu }_i=\lambda B+\gamma C=m.$$

Solve the $2\times 2$ system. With determinant $D=AC-B^2$ (positive by Cauchy–Schwarz unless all ${\mu }_i$ are equal),

$$\lambda =\frac{C-Bm}{D},\gamma =\frac{Am-B}{D}.$$

Hence the closed form:

$$\boxed{w_i=\frac{1}{{\sigma }_i^2}·\frac{(C-Bm)+(Am-B){\mu }_i}{AC-B^2}.}$$

Plugging in the numbers

The precisions are $1/{\sigma }_i^2=(100,50,25)$, so

$$A=100+50+25=175.$$

$$B=100(0.08)+50(0.12)+25(0.20)=8+6+5=19.$$

$$C=100(0.08{)}^2+50(0.12{)}^2+25(0.20{)}^2=0.64+0.72+1.00=2.36.$$

$$D=AC-B^2=175(2.36)-{19}^2=413-361=52.$$

For $m=0.12$:

$$\lambda =\frac{2.36-19(0.12)}{52}=\frac{0.08}{52},\gamma =\frac{175(0.12)-19}{52}=\frac{2}{52}.$$

Then $w_i=(1/{\sigma }_i^2)(\lambda +\gamma {\mu }_i)$:

$$w_1=100(\frac{0.08}{52}+\frac{2}{52}(0.08))=\frac{6}{13},$$

$$w_2=50(\frac{0.08}{52}+\frac{2}{52}(0.12))=\frac{4}{13},$$

$$w_3=25(\frac{0.08}{52}+\frac{2}{52}(0.20))=\frac{3}{13}.$$

These sum to $1$ and reproduce $\sum w_i{\mu }_i=0.12$, as they must. The variance is

$$\sum _i{w}_i^2{\sigma }_i^2=(\frac{6}{13}{)}^2(0.01)+(\frac{4}{13}{)}^2(0.02)+(\frac{3}{13}{)}^2(0.04)=\frac{2}{325}\approx 0.006154,$$

i.e. a return standard deviation of about $7.84\%$.

The efficient frontier and the GMV portfolio

At the optimum the variance can be read off without recomputing $\sum w_i^2{\sigma }_i^2$ directly. Since $w_i{\sigma }_i^2=\lambda +\gamma {\mu }_i$,

$${\sigma }^2(m)=\sum _i{w}_i^2{\sigma }_i^2=\sum _i{w}_i(\lambda +\gamma {\mu }_i)=\lambda \underset{=1}{\underbrace{{\sum }_i{w}_i}}+\gamma \underset{=m}{\underbrace{{\sum }_i{w}_i{\mu }_i}}=\lambda +\gamma m.$$

Substituting the solved multipliers,

$${\sigma }^2(m)=\frac{C-Bm}{D}+\frac{Am-B}{D}m=\frac{A{m}^2-2Bm+C}{AC-B^2}.$$

The minimum-variance frontier is therefore a parabola in the variance–return plane (a hyperbola in the $\sigma$–return plane). For the table,

$${\sigma }^2(m)=\frac{175m^2-38m+2.36}{52}.$$

Check: $m=0.12$ gives $(175·0.0144-38·0.12+2.36)/52=(2.52-4.56+2.36)/52=0.32/52=2/325$. Consistent.

The global minimum-variance (GMV) portfolio minimizes ${\sigma }^2(m)$ over $m$: setting the derivative $2Am-2B=0$ gives

$$m^{\star }=\frac{B}{A}=\frac{19}{175}\approx 0.1086,{\sigma }_{min}^2=\frac{1}{A}=\frac{1}{175}\approx 0.005714.$$

At $m=m^{\star }$ the return multiplier vanishes ($\gamma =0$), so its weights are simply proportional to the precisions:

$$w_i^{\text{GMV}}=\frac{1/{\sigma }_i^2}{A}=(\frac{100}{175},\frac{50}{175},\frac{25}{175})=(\frac{4}{7},\frac{2}{7},\frac{1}{7}).$$

Closing note

The clean route to part 3 is the identity ${\sigma }^2=\lambda +\gamma m$, which uses the constraints themselves rather than squaring the weights — a common time-saver that also generalizes verbatim to correlated assets by replacing $A,B,C$ with ${\mathbf{1}}^{\top }{\Sigma }^{-1}\mathbf{1}$, ${\mathbf{1}}^{\top }{\Sigma }^{-1}\mu$, ${\mu }^{\top }{\Sigma }^{-1}\mu$. The tempting error is to impose only the budget constraint and then be surprised the target return is not met; both constraints must carry their own multiplier.

Throughout write $r^2=\Vert {\beta }^{\backslash }*{\Vert }_2^2$.

The risk functional

For an independent test point, $$𝔼[(x_0^{\top }\widehat{\beta }-y_0{)}^2]=𝔼[(x_0^{\top }(\widehat{\beta }-{\beta }^{\backslash }*)-{ϵ}_0{)}^2].$$ Condition on $\widehat{\beta }$ (a function of the training data, independent of $(x_0,{ϵ}_0)$). Since $𝔼[{ϵ}_0]=0$ and $\mathrm{C}\mathrm{o}\mathrm{v}(x_0)=I_p$, $$𝔼[(x_0^{\top }v{)}^2]=v^{\top }𝔼[x_0{x}_0^{\top }]v=\Vert v{\Vert }_2^2,v=\widehat{\beta }-{\beta }^{\backslash }*.$$ The cross term vanishes and $𝔼[{ϵ}_0^2]={\sigma }^2$, so the prediction risk is $𝔼\Vert \widehat{\beta }-{\beta }^{\backslash }*{\Vert }_2^2+{\sigma }^2=R(p)+{\sigma }^2$. The isotropy of $x_0$ is what turns the test risk into a plain squared parameter error; this is the reason the problem is exactly solvable.

Part 1 — underparameterized regime

Here $p<n$, so $X^{\top }X$ is a.s. invertible and $\widehat{\beta }=(X^{\top }X{)}^{-1}{X}^{\top }y$. The model is well specified, so $$\widehat{\beta }-{\beta }^{\backslash }*=(X^{\top }X{)}^{-1}{X}^{\top }ϵ.$$ There is no bias: $𝔼[\widehat{\beta }\mid X]={\beta }^{\backslash }*$ because $X^{\top }y=X^{\top }X{\beta }^{\backslash }*+X^{\top }ϵ$. Conditioning on $X$, $$𝔼[\Vert \widehat{\beta }-{\beta }^{\backslash }*{\Vert }_2^2\mid X]={\sigma }^2\mathrm{t}\mathrm{r}((X^{\top }X{)}^{-1}{X}^{\top }X(X^{\top }X{)}^{-1})={\sigma }^2\mathrm{t}\mathrm{r}((X^{\top }X{)}^{-1}).$$ Now $X^{\top }X={\sum }_{i=1}^n{x}_i{x}_i^{\top }$ is Wishart with $k=p$, $m=n$. For $n>p+1$ the inverse-Wishart mean gives $𝔼[(X^{\top }X{)}^{-1}]=\frac{1}{n-p-1}{I}_p$, hence $$\boxed{R(p)=\frac{{\sigma }^{2}p}{n-p-1},p\le n-2.}$$ As $p\uparrow n$ the denominator $\to 0$ and $R(p)\to \infty$: the risk diverges at the interpolation threshold. This is the first descent's collapse.

Part 2 — overparameterized regime

Now $p>n$, the rows of $X$ span an $n$-dimensional subspace a.s., and $\widehat{\beta }=X^{+}y=X^{\top }(X{X}^{\top }{)}^{-1}y$. Substitute $y=X{\beta }^{\backslash }*+ϵ$ and let $$P:=X^{+}X=X^{\top }(X{X}^{\top }{)}^{-1}X$$ be the orthogonal projector onto the row space of $X$ (an $n$-dimensional subspace of ${ℝ}^p$). Then $$\widehat{\beta }=P{\beta }^{\backslash }*+X^{+}ϵ,\widehat{\beta }-{\beta }^{\backslash }*=-(I-P){\beta }^{\backslash }*+X^{+}ϵ.$$ Because $𝔼[ϵ\mid X]=0$, the cross term drops when we take expectations, leaving $$R(p)=\underset{\text{bias}}{\underbrace{𝔼\backslash |(I-P){\beta }^{\backslash }*{\backslash |}_2^2}}+\underset{\text{variance}}{\underbrace{𝔼\backslash |X^{+}ϵ{\backslash |}_2^2}}.$$

Bias. This is the step a candidate misses: the minimum-norm interpolator can only fit the component of ${\beta }^{\backslash }*$ lying in the row space; the orthogonal component $(I-P){\beta }^{\backslash }*$ is irrecoverable. Since $I-P$ is idempotent, $\Vert (I-P){\beta }^{\backslash }*{\Vert }_2^2={\beta }^{\backslash *\top }(I-P){\beta }^{\backslash }*$. The row space of a Gaussian $X$ is a uniformly random $n$-dimensional subspace (the standard Gaussian is rotationally invariant), so $𝔼[P]=\frac{n}{p}{I}_p$ and $𝔼[I-P]=\frac{p-n}{p}{I}_p$. Hence $$\text{bias}=\frac{p-n}{p}{r}^2.$$ In Part 1 the row space had dimension $p$ and equalled all of the column-relevant space, so $I-P=0$: no bias.

Variance. $𝔼\Vert X^{+}ϵ{\Vert }_2^2={\sigma }^2\mathrm{t}\mathrm{r}((X^{+}{)}^{\top }{X}^{+})$. With $X^{+}=X^{\top }(X{X}^{\top }{)}^{-1}$, $$(X^{+}{)}^{\top }{X}^{+}=(X{X}^{\top }{)}^{-1}X{X}^{\top }(X{X}^{\top }{)}^{-1}=(X{X}^{\top }{)}^{-1},$$ so the variance is ${\sigma }^2𝔼\mathrm{t}\mathrm{r}((X{X}^{\top }{)}^{-1})$. Now $X{X}^{\top }={\sum }_{j=1}^p({\text{col}}_j)({\text{col}}_j{)}^{\top }$ is Wishart with $k=n$, $m=p$; for $p>n+1$, $$𝔼[(X{X}^{\top }{)}^{-1}]=\frac{1}{p-n-1}{I}_n,\text{variance}=\frac{{\sigma }^{2}n}{p-n-1}.$$

Therefore $$\boxed{R(p)=(1-\frac{n}{p})r^2+\frac{{\sigma }^{2}n}{p-n-1},p\ge n+2.}$$ As $p\downarrow n+1$ the variance $\to \infty$ (the peak again); as $p\to \infty$ the variance $\to 0$ while the bias $\uparrow r^2$. The interpolator never forgets that it has thrown away a $\frac{p-n}{p}$ fraction of the signal.

Part 3 — the second descent

On $(n+1,\infty )$ differentiate: $$R^{\prime }(p)=\frac{n{r}^2}{p^2}-\frac{{\sigma }^{2}n}{(p-n-1{)}^2}.$$ Setting $R^{\prime }(p)=0$ gives ${\displaystyle \frac{r^2}{p^2}}={\displaystyle \frac{{\sigma }^2}{(p-n-1{)}^2}}$. Both sides positive, so take positive roots: ${\displaystyle \frac{r}{p}}={\displaystyle \frac{\sigma }{p-n-1}}$, i.e. $r(p-n-1)=\sigma p$, whence $$\boxed{p^{\backslash }*=\frac{(n+1)r}{r-\sigma }.}$$ Since $\frac{r}{r-\sigma }>1$ exactly when $r>\sigma >0$, an interior minimizer with $p^{\backslash }*>n+1$ exists iff $r>\sigma$ (signal norm exceeds the noise level). When $r\le \sigma$, $R^{\prime }(p)<0$ for all large $p$ and $R$ decreases monotonically to its infimum $r^2$: the best overparameterized model merely matches the null predictor.

Evaluate $R(p^{\backslash }*)$. Writing $a=n+1$, one has $p^{\backslash }*-a={\displaystyle \frac{a\sigma }{r-\sigma }}$, and $$\frac{n{r}^2}{p^{\backslash }*}=\frac{nr(r-\sigma )}{a},\frac{{\sigma }^{2}n}{p^{\backslash }*-a}=\frac{\sigma n(r-\sigma )}{a}.$$ Hence $$R(p^{\backslash }*)=r^2-\frac{nr(r-\sigma )}{a}+\frac{\sigma n(r-\sigma )}{a}=r^2-\frac{n(r-\sigma {)}^2}{a},$$ that is $$\boxed{R(p^{\backslash }*)=r^2-\frac{n}{n+1}(r-\sigma {)}^2,r>\sigma .}$$

Interpretation. The null predictor $\widehat{\beta }\equiv 0$ has excess risk $\Vert {\beta }^{\backslash }*{\Vert }_2^2=r^2$. The optimally overparameterized interpolator beats it by exactly $\frac{n}{n+1}(r-\sigma {)}^2>0$. So heavy overparameterization is genuinely useful, but only when $r>\sigma$; the benefit is the squared gap between signal and noise scales, discounted by $\frac{n}{n+1}$.

Closing notes

• The three regimes assemble into the classic double-descent curve: $R$ rises to $+\infty$ as $p\uparrow n$ (Part 1), the interpolation peak is another $+\infty$ as $p\downarrow n+1$ (Part 2), and then $R$ descends a second time to a minimum at $p^{\backslash }*$ before flattening toward $r^2$.
• The tempting error is to declare that more features always help. In isotropic features the interpolator's asymptote is the null risk $r^2$, not $0$: past $p^{\backslash }*$ additional useless features re-inject bias faster than they shrink variance. Whether the second descent dips below the best underparameterized risk depends entirely on the SNR $r/\sigma$.
• The slick sanity check on the bias: $𝔼[P]=\frac{n}{p}{I}_p$ follows in one line from $𝔼[\mathrm{t}\mathrm{r}(P)]=n$ (rank) plus rotational invariance forcing $𝔼[P]\propto I_p$.