Farkas' lemma

Farkas' lemma is a solvability theorem for a finite system of linear inequalities in mathematics. It was originally proven by the Hungarian mathematician Gyula Farkas.^[1] Farkas' lemma is the key result underpinning the linear programming duality and has played a central role in the development of mathematical optimization (alternatively, mathematical programming). It is used amongst other things in the proof of the Karush–Kuhn–Tucker theorem in nonlinear programming.

Generalizations of the Farkas' lemma are about the solvability theorem for convex inequalities,^[2] i.e., infinite system of linear inequalities. Farkas' lemma belongs to a class of statements called "theorems of the alternative": a theorem stating that exactly one of two systems has a solution.

Statement of the lemma

There are a number of slightly different (but equivalent) formulations of the lemma in the literature. The one given here is due to Gale, Kuhn & Tucker (1951).

Farkas' lemma — Let $\mathbf {A} \in \mathbb {R} ^{m\times n}$ and $\mathbf {b} \in \mathbb {R} ^{m}$ . Then exactly one of the following two statements is true:

There exists an $\mathbf {x} \in \mathbb {R} ^{n}$ such that $\mathbf {Ax} =\mathbf {b}$ and $\mathbf {x} \geq 0$ .
There exists a $\mathbf {y} \in \mathbb {R} ^{m}$ such that $\mathbf {A} ^{\mathsf {T}}\mathbf {y} \geq 0$ and $\mathbf {b} ^{\mathsf {T}}\mathbf {y} <0$ .

Here, the notation $\mathbf {x} \geq 0$ means that all components of the vector $\mathbf {x}$ are nonnegative. Let the cone generated by the columns of $\mathbf {A}$ denoted by $C(\mathbf {A} )=\{\mathbf {A} \mathbf {x} |\mathbf {x} \geq 0\}$ . Then $C(\mathbf {A} )$ is a closed convex cone. The vector $\mathbf {x}$ proves that $\mathbf {b}$ lies in $C(\mathbf {A} )$ while the vector $\mathbf {y}$ gives a linear functional that separates $\mathbf {b}$ from $C(\mathbf {A} )$ .

Let $\mathbf {a} _{1},\dots ,\mathbf {a} _{n}\in \mathbb {R} ^{m}$ denote the columns of $\mathbf {A}$ . In terms of these vectors, Farkas' lemma states that exactly one of the following two statements is true:

There exist coefficients $x_{1},\dots ,x_{n}\in \mathbb {R} ,x_{1},\dots ,x_{n}\geq 0$ , such that $\mathbf {b} =x_{1}\mathbf {a} _{1}+\dots +x_{n}\mathbf {a} _{n}$ .
There exists a vector $\mathbf {y} \in \mathbb {R} ^{m}$ such that $\mathbf {a} _{i}^{\mathsf {T}}\mathbf {y} \geq 0$ for $i=1,\dots ,n$ and $\mathbf {b} ^{\mathsf {T}}\mathbf {y} <0$ .

The vectors $x_{1}\mathbf {a} _{1}+\dots +x_{n}\mathbf {a} _{n}$ with nonnegative coefficients constitute the convex cone of the set $\{\mathbf {a} _{1},\dots ,\mathbf {a} _{n}\}$ so the first statement says that $\mathbf {b}$ is in this cone.

The second statement says that there exists a vector $\mathbf {y}$ such that the angle of $\mathbf {y}$ with the vectors $\mathbf{a}_i$ is at most 90° while the angle of $\mathbf {y}$ with the vector $\mathbf {b}$ is more than 90°. The hyperplane normal to this vector has the vectors $\mathbf{a}_i$ on one side and the vector $\mathbf {b}$ on the other side. Hence, this hyperplane separates the vectors in the cone of $\{\mathbf {a} _{1},\dots ,\mathbf {a} _{n}\}$ and the vector $\mathbf {b}$ .

For example, let n,m=2 and a₁ = (1,0)^T and a₂ = (1,1)^T. The convex cone spanned by a₁ and a₂ can be seen as a wedge-shaped slice of the first quadrant in the x-y plane. Now, suppose b = (0,1). Certainly, b is not in the convex cone a₁x₁+a₂x₂. Hence, there must be a separating hyperplane. Let y = (1,−1)^T. We can see that a₁ · y = 1, a₂ · y = 0, and b · y = −1. Hence, the hyperplane with normal y indeed separates the convex cone a₁x₁+a₂x₂ from b.

Generalizations

Generalized Farkas' lemma — Let $\mathbf {A} \in \mathbb {R} ^{m\times n}$ , $\mathbf {b} \in \mathbb {R} ^{m}$ , ${\mathbf {S}}$ is a closed convex cone in $\mathbb {R} ^{n}$ and the dual cone of $\mathbf {S}$ is $\mathbf {S^{*}} =\{\mathbf {z} \in \mathbb {R} ^{n}|\mathbf {z} ^{\mathsf {T}}\mathbf {x} \geq 0,\forall \mathbf {x} \in \mathbf {S} \}$ . If convex cone $C(\mathbf {A} )=\{\mathbf {A} \mathbf {x} |\mathbf {x} \in \mathbf {S} \}$ is closed, then exactly one of the following two statements is true:

There exists an $\mathbf {x} \in \mathbb {R} ^{n}$ such that $\mathbf {Ax} =\mathbf {b}$ and $\mathbf {x} \in \mathbf {S}$ .
There exists a $\mathbf {y} \in \mathbb {R} ^{m}$ such that $\mathbf {A} ^{\mathsf {T}}\mathbf {y} \in \mathbf {S^{*}}$ and $\mathbf {b} ^{\mathsf {T}}\mathbf {y} <0$ .

Generalized Farkas' lemma can be interpreted geometrically as follows: a vector is either in a given closed convex cone or that there exists a hyperplane separating the vector from the cone—there are no other possibilities. The closedness condition is necessary, see Separation theorem I in Hyperplane separation theorem. For original Farkas' lemma, ${\mathbf {S}}$ is the nonnegative orthant $\mathbb {R} _{+}^{n}$ , hence the closedness condition holds automatically. Indeed, for polyhedral convex cone, i.e., there exists a $\mathbf {B} \in \mathbb {R} ^{n\times k}$ such that $\mathbf {S} =\{\mathbf {B} \mathbf {x} |\mathbf {x} \in \mathbb {R} _{+}^{k}\}$ , the closedness condition holds automatically. In convex optimization, various kinds of constraint qualification, e.g. Slater's condition, are responsible for closedness of the underlying convex cone $C(\mathbf {A} )$ .

By setting $\mathbf {S} =\mathbb {R} ^{n}$ and $\mathbf {S^{*}} =\{0\}$ in Generalized Farkas' lemma, we obtain the following corollary about the solvability for a finite system of linear equalities.

Corollary — Let $\mathbf {A} \in \mathbb {R} ^{m\times n}$ and $\mathbf {b} \in \mathbb {R} ^{m}$ . Then exactly one of the following two statements is true:

There exists an $\mathbf {x} \in \mathbb {R} ^{n}$ such that $\mathbf {Ax} =\mathbf {b}$ .
There exists a $\mathbf {y} \in \mathbb {R} ^{m}$ such that $\mathbf {A} ^{\mathsf {T}}\mathbf {y} =0$ and $\mathbf {b} ^{\mathsf {T}}\mathbf {y} \neq 0$ .

Further implications

Farkas's lemma can be varied to many further theorems of alternative by simple modifications, such as Gordan's theorem: Either $Ax < 0$ has a solution x, or $A^{\mathsf {T}}y=0$ has a nonzero solution y with y ≥ 0.

Common applications of Farkas' lemma include proving the strong and weak duality theorem associated with linear programming, game theory at a basic level and the Kuhn–Tucker constraints. An extension of Farkas' lemma can be used to analyze the strong duality conditions for and construct the dual of a semidefinite program. It is sufficient to prove the existence of the Kuhn–Tucker constraints using the Fredholm alternative but for the condition to be necessary, one must apply Von Neumann's minimax theorem to show the equations derived by Cauchy are not violated.

A particularly suggestive and easy-to-remember version is the following: if a set of inequalities has no solution, then a contradiction can be produced from it by linear combination with nonnegative coefficients. In formulas: if $Ax$ ≤ $b$ is unsolvable then $y^{\mathsf {T}}A=0$ , $y^{\mathsf {T}}b=-1$ , $y$ ≥ $0$ has a solution.^[3] (Note that $y^{\mathsf {T}}A$ is a combination of the left hand sides, $y^{\mathsf {T}}b$ a combination of the right hand side of the inequalities. Since the positive combination produces a zero vector on the left and a −1 on the right, the contradiction is apparent.)

Notes

↑ Farkas, Julius (Gyula) (1902), "Über die Theorie der Einfachen Ungleichungen", Journal für die Reine und Angewandte Mathematik, 124 (124): 1–27, doi:10.1515/crll.1902.124.1
↑ Dinh, N.; Jeyakumar, V. (2014), "Farkas' lemma three decades of generalizations for mathematical optimization", TOP, 22 (1.): 1–22, doi:10.1007/s11750-014-0319-y
↑ Boyd, Stephen P.; Vandenberghe, Lieven (2004), "Section 5.8.3", Convex Optimization (pdf), Cambridge University Press, ISBN 978-0-521-83378-3, retrieved October 15, 2011

References

Rockafellar, R. T. (1979), Convex Analysis, Princeton University Press, p. 200

Gale, David; Kuhn, Harold; Tucker, Albert W. (1951), "Linear Programming and the Theory of Games - Chapter XII", in Koopmans, Activity Analysis of Production and Allocation (PDF), Wiley . See Lemma 1 on page 318.

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.

[Farkas1902-1] Farkas, Julius (Gyula) (1902), "Über die Theorie der Einfachen Ungleichungen", Journal für die Reine und Angewandte Mathematik, 124 (124): 1–27, doi:10.1515/crll.1902.124.1

[Jeyakumar2014-2] Dinh, N.; Jeyakumar, V. (2014), "Farkas' lemma three decades of generalizations for mathematical optimization", TOP, 22 (1.): 1–22, doi:10.1007/s11750-014-0319-y

[3] Boyd, Stephen P.; Vandenberghe, Lieven (2004), "Section 5.8.3", Convex Optimization (pdf), Cambridge University Press, ISBN 978-0-521-83378-3, retrieved October 15, 2011