Saturday 15 August 2026

1. Spectrum

$$\Sigma ={\sigma }^2[(1-\rho )I+\rho \mathbf{1}{\mathbf{1}}^{\top }]$$

Since $\mathbf{1}{\mathbf{1}}^{\top }$ has eigenvalue $n$ on $\mathbf{1}$ and $0$ on the orthogonal complement ${\mathbf{1}}^{⟂}$, the eigenvalues of $\Sigma$ are

$${\sigma }^2(1+(n-1)\rho )\text{once, on }\mathbf{1},{\sigma }^2(1-\rho )\text{with multiplicity }n-1,\text{ on }{\mathbf{1}}^{⟂}.$$

Both must be nonnegative, so

$$-\frac{1}{n-1}\le \rho \le 1.$$

The lower limit is the interesting one: $n$ assets cannot all be strongly negatively correlated with one another. At $\rho =-1/(n-1)$ the vector $\mathbf{1}$ is in the kernel, meaning the equally weighted portfolio is exactly riskless.

2. The equally weighted portfolio

With $w=\mathbf{1}/n$, the quadratic form picks out the $\mathbf{1}$ eigenvalue:

$$\mathrm{V}\mathrm{a}\mathrm{r}=\frac{{\mathbf{1}}^{\top }\Sigma \mathbf{1}}{n^2}=\frac{{\sigma }^2(1+(n-1)\rho )}{n}={\sigma }^2(\rho +\frac{1-\rho }{n})$$

$${lim}_{n\to \infty }\mathrm{V}\mathrm{a}\mathrm{r}={\sigma }^2\rho$$

Only the idiosyncratic part ${\sigma }^2(1-\rho )$ diversifies away, at rate $1/n$. The common part ${\sigma }^2\rho$ does not shrink no matter how many names you add.

3. Equal weights are optimal

For fully invested portfolios the minimizer of $w^{\top }\Sigma w$ subject to ${\mathbf{1}}^{\top }w=1$ is

$$w^{\star }=\frac{{\Sigma }^{-1}\mathbf{1}}{{\mathbf{1}}^{\top }{\Sigma }^{-1}\mathbf{1}},$$

from the Lagrangian $w^{\top }\Sigma w-2\lambda ({\mathbf{1}}^{\top }w-1)$, whose stationarity condition is $\Sigma w=\lambda \mathbf{1}$. Here $\mathbf{1}$ is an eigenvector of $\Sigma$, so

$${\Sigma }^{-1}\mathbf{1}=\frac{\mathbf{1}}{{\sigma }^2(1+(n-1)\rho )}⟹w^{\star }=\frac{\mathbf{1}}{n},$$

provided $1+(n-1)\rho >0$ so that $\Sigma$ is invertible and positive definite, which also makes the stationary point a genuine minimum. The minimum variance is therefore the quantity computed in part 2:

$${min}_{{\mathbf{1}}^{\top }w=1}{w}^{\top }\Sigma w=\frac{{\sigma }^2(1+(n-1)\rho )}{n}.$$

Note. The practical reading is that the floor on portfolio risk is set by correlation, not by breadth: with $\rho =0.3$ and $\sigma =20\%$, no number of equally risky names gets annualized volatility below $20\%\sqrt{0.3}\approx 11\%$. The tempting error is to treat $1/\sqrt{n}$ as the governing rate; it governs only the $(1-\rho )$ slice.

1. The decomposition

$\log p_{\theta }(x)$ does not depend on $z$, so it equals its own expectation under any $q$. Insert the definition of conditional probability and split the logarithm:

$$\log p_{\theta }(x)={𝔼}_q\left[\log \frac{p_{\theta }(x,z)}{p_{\theta }(z\mid x)}\right]={𝔼}_q\left[\log \frac{p_{\theta }(x,z)}{q(z)}\right]+{𝔼}_q\left[\log \frac{q(z)}{p_{\theta }(z\mid x)}\right]$$

The first term is $ℱ(q,\theta )$ and the second is $\mathrm{K}\mathrm{L}(q\Vert p_{\theta }(·\mid x))$, which proves the identity. Since a Kullback-Leibler divergence is nonnegative and vanishes only when its arguments agree almost everywhere, $ℱ(q,\theta )\le \log p_{\theta }(x)$ with equality exactly at $q=p_{\theta }(·\mid x)$. Note the identity is exact for every $q$: the bound's slack is not merely bounded by the KL term, it is the KL term.

2. Monotonicity in three steps

$$\log p_{{\theta }^{(t+1)}}(x)\ge ℱ(q^{(t)},{\theta }^{(t+1)})\ge ℱ(q^{(t)},{\theta }^{(t)})=\log p_{{\theta }^{(t)}}(x)$$

The first inequality is part 1 applied at ${\theta }^{(t+1)}$. The second is the M-step, which maximizes over $\theta$ and so cannot do worse than the incumbent. The final step is an equality, and it is the one that carries the argument: the E-step chose $q^{(t)}$ to be the posterior at ${\theta }^{(t)}$, so the bound is tight there.

Without that tightness you would only be comparing a bound to a bound, which proves nothing about the likelihood itself. This is the step a rushed proof skips.

3. What monotonicity actually gives

It does not give a global maximum. What follows is only that the sequence $\log p_{{\theta }^{(t)}}(x)$ is nondecreasing, hence convergent whenever the likelihood is bounded above.

Three gaps separate that from what one wants. The likelihood value can converge while the parameters do not; the limit point, if the parameters do converge, is in general only a stationary point, so a local maximum or even a saddle is possible; and establishing convergence of ${\theta }^{(t)}$ at all requires regularity conditions on the model rather than following from the iteration (this is the content of Wu's 1983 analysis, which corrected the original claim in Dempster, Laird and Rubin).

For a Gaussian mixture the likelihood is not concave, is invariant under relabeling the components — so every maximum comes with $k!$ copies — and is unbounded above as a component variance goes to zero with its mean pinned to a data point. Monotone ascent in that landscape guarantees you stop going down, nothing more, which is why EM is run from several random starts and the best run kept.