MM Algorithm[unfinished]
MM algorithm 1. Overview of MM algorithm The MM algorithm is not an algorithm, but a prescription for constructing optimization algorithms. The EM algorithm from statistics is a special case. An MM algorithm operates by creating a surrogate function that minorizes or majorizes the objective function. When the surrogate function is optimized, the objective function is driven uphill or downhill as needed. In minimization MM stands for majorize/minimize, and in maximization MM stands for minorize/maximize. 2. Rationale for the MM principle It can generate an algorithm that avoids matrix inversion. It can separate the parameters of a problem. It can linearize an optimization problem. It can deal gracefully with equality and inequality constraints. It can turn a non-differentiable problem into a smooth problem. 3. Majorization and definition of algorithms A function $g(θ | θ^n)$ is said to majorize the function f(θ) ...