Linear Algebra for Those Who Know Linear Algebra

You've reached the landing page for the eBook.

The book employs a novel approach to teaching linear algebra by centering on singular value decomposition (SVD). This vantage point provides a deeper understanding of algorithms, like PCA, quadratic programming, and regressions, and allows the reader to develop their intuition for how to modify and apply these algorithms in real world applications.

The book interweaves theory and application, beginning with SVD, from which PCA and collaborative filtering immediately follow. Regressions can be understood via projection matrices (matrices with singular values all equal to 1) or via pseudo-inverses (inverses on the non-zero singular values). Quadratic programming can be understood via positive definite matrices (matrices with an SVD having inverse isometries and all positive singular values). Recognizing that most applied linear algebra follows from a solid understanding of SVD makes it easier to understand these hard topics.

After establishing a solid theoretical basis, the book dives into application. It discusses which model to use when, how to interpret model outputs, and what modeling choices are available for each algorithm. It grounds this discussion in concrete applications, like the Netflix Prize for collaborative filtering. Further, the book provides real-world exercises, such as, “You and a coworker are building an insurance model on credit data. The model will be a logistic regression to predict the probability of an insurance claim. You have historical claim data. You think you should use only PLS, and your coworker thinks you should only use PCA. How can you decide with data which model is better?”

• 1 Introduction
• 2 Singular Value Decomposition
• 3 Direct Applications of SVD
• 4 [WIP] Regression
• 4.1 Projection matrices
• 4.2 Inverses and pseudo-inverses
• 4.3 Least Squares
• 4.4 Graham-Schmidt and QR factorizations
• 4.5 Legendre polynomials
• 5 [WIP] Generalized Regression
• 5.1 Polynomial Regression
• 5.2 Logistic
• 5.3 GLM
• 5.4 LOESS
• 5.5 Ridge Regression
• 6 [WIP] Definite Matrices
• 6.1 Definite matrices
• 6.3 Partial second derivative test
• 7 [WIP] Symmetric Matrices
• 7.1 Symmetric Matrices
• 7.2 Prove specific form
• 7.3 Corollaries
• 7.4 Use to calculate SVD
• 7.5 Inner products, covariance
• 7.6 Spectral Theorem
• 7.7 Taylor series and smooth functions
• 7.8 Functions on Non-symmetric matrices
• 8 [WIP] Stats
• 8.1 Covariance matrix
• 8.2 Multi-dimensional normal
• 8.3 Moment generating function
• 8.4 Markov Chain
• 8.5 PageRank
• 8.6 Linear discriminant analysis
• 9 [WIP] SVM
• 10 Review