Sequential linear-quadratic programming

Sequential linear-quadratic programming (SLQP) is an iterative method for nonlinear optimization problems where objective function and constraints are twice continuously differentiable. Similarly to sequential quadratic programming (SQP), SLQP proceeds by solving a sequence of optimization subproblems. The difference between the two approaches is that:

in SQP, each subproblem is a quadratic program, with a quadratic model of the objective subject to a linearization of the constraints
in SLQP, two subproblems are solved at each step: a linear program (LP) used to determine an active set, followed by an equality-constrained quadratic program (EQP) used to compute the total step

This decomposition makes SLQP suitable to large-scale optimization problems, for which efficient LP and EQP solvers are available, these problems being easier to scale than full-fledge quadratic programs.

Algorithm basics

Consider a nonlinear programming problem of the form:

\begin{array}{rl} \min\limits_{x} & f(x) \\ \mbox{s.t.} & b(x) \ge 0 \\ & c(x) = 0. \end{array}

The Lagrangian for this problem is^[1]

\mathcal{L}(x,\lambda,\sigma) = f(x) - \lambda^T b(x) - \sigma^T c(x),

where $\lambda$ and $\sigma$ are Lagrange multipliers.

LP phase

In the LP phase of SLQP, the following linear program is solved:

{\begin{array}{rl}\min \limits _{d}&f(x_{k})+\nabla f(x_{k})^{T}d\\\mathrm {s.t.} &b(x_{k})+\nabla b(x_{k})^{T}d\geq 0\\&c(x_{k})+\nabla c(x_{k})^{T}d=0.\end{array}}

Let ${\cal {A}}_{k}$ denote the active set at the optimum $d_{\text{LP}}^{*}$ of this problem, that is to say, the set of constraints that are equal to zero at $d_{\text{LP}}^{*}$ . Denote by $b_{{\cal {A}}_{k}}$ and $c_{{\cal {A}}_{k}}$ the sub-vectors of $b$ and $c$ corresponding to elements of ${\cal {A}}_{k}$ .

EQP phase

In the EQP phase of SLQP, the search direction $d_{k}$ of the step is obtained by solving the following quadratic program:

{\begin{array}{rl}\min \limits _{d}&f(x_{k})+\nabla f(x_{k})^{T}d+{\tfrac {1}{2}}d^{T}\nabla _{xx}^{2}{\mathcal {L}}(x_{k},\lambda _{k},\sigma _{k})d\\\mathrm {s.t.} &b_{{\cal {A}}_{k}}(x_{k})+\nabla b_{{\cal {A}}_{k}}(x_{k})^{T}d=0\\&c_{{\cal {A}}_{k}}(x_{k})+\nabla c_{{\cal {A}}_{k}}(x_{k})^{T}d=0.\end{array}}

Note that the term $f(x_{k})$ in the objective functions above may be left out for the minimization problems, since it is constant.

Notes

↑ Jorge Nocedal and Stephen J. Wright (2006). Numerical Optimization. Springer. ISBN 0-387-30303-0.

References

Jorge Nocedal and Stephen J. Wright (2006). Numerical Optimization. Springer. ISBN 0-387-30303-0.

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.

[1] Jorge Nocedal and Stephen J. Wright (2006). Numerical Optimization. Springer. ISBN 0-387-30303-0.