The Math Behind LLS¶
Linearly Constrained Least Squares¶
LLS solves linearly constrained least squares (or LCLS) problems, which have the form:
where the unknown variable \(x\) is a vector of size \(n\). The values for \(A\), \(b\), \(C\), and \(d\) are given and have sizes \(m\times n\), \(m\), \(p\times n\), and \(p\), respectively. LLS finds a value for \(x\) that satisfies the linear equality constraints \(Cx = d\) and minimizes the objective, the sum of the squares of the entries of \(Ax - b\).
When there are no equality constraints, LCLS reduces to the simple unconstrained least squares problem (LS):
When the objective is absent, LCLS reduces to finding \(x\) that satisfies \(Cx=d\), i.e., solving a set of linear equations.
Solving LCLS¶
There is a unique solution to the LCLS problem if and only if there is a unique solution to the following system of linear equations in the variable \(x\) and a new variable \(z\):
i.e., the matrix on the left is invertible. This occurs when the matrix \(C\) has independent rows, and the matrix \(\begin{bmatrix} A\\ C\end{bmatrix}\) has indepedent columns.
When there are no equality constraints, the unconstrained least squares problem has a unique solution if and only if the system of linear equations:
has a unique solution, which occurs when \(A^TA\) is invertible, i.e., the columns of \(A\) are independent.
When the objective is absent, the system of linear equations \(Cx = d\) has a unique solution if and only if \(C\) is invertible.
LLS allows you to specify an LCLS problem in a natural way. It translates your specification into the general form in this section, and then solves the appropriate set of linear equations.