Covariance Selection by LASSO
Estimates a sparse inverse covariance matrix by ℓ1-penalized maximum likelihood — revealing which assets genuinely interact — using the GLASSOFAST algorithm, an order of magnitude faster than the original GLASSO.
The precision matrix Θ = Σ⁻¹ encodes conditional dependence: a zero entry means two assets are independent given the rest. Banerjee, El Ghaoui & d'Aspremont estimate it sparsely by maximizing log det X − tr(SX) − λ‖X‖₁, which stays invertible even when there are fewer observations than assets, and comes with a principled formula for choosing λ that controls the probability of false connections.
Their block coordinate descent algorithm solves one LASSO regression per row — the recursion later popularized as the graphical lasso. Applied to a stationary VAR model of prices, the sparse precision matrix reveals clusters of dependent assets and yields small, tradeable mean-reverting baskets.
We solve the problem with GLASSOFAST (Sustik & Calderhead), which fixes a non-termination bug in the original GLASSO, guarantees a symmetric solution, supports element-wise penalties λᵢⱼ, and runs roughly an order of magnitude faster than GLASSO 1.3/1.7 and QUIC.
Algorithm
- › Outer loop: convergence on the whole matrix
- › Middle loop: iterate through rows
- › Inner loops: per-row LASSO by coordinate descent
References
- Dempster, A. P. (1972). "Covariance Selection." Biometrics, 28(1), 157–175. https://websites.umich.edu/~jizhu/jizhu/covar/Dempster-Biometrics72.pdf
- Banerjee, O., El Ghaoui, L. & d'Aspremont, A. (2008). "Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data." Journal of Machine Learning Research, 9, 485–516. https://jmlr.org/papers/v9/banerjee08a.html
- Sustik, M. A. & Calderhead, B. (2012). "GLASSOFAST: An Efficient GLASSO Implementation." UTCS Technical Report TR-12-29. https://www.academia.edu/113950585/GLASSOFAST_An_efficient_GLASSO_implementation