Navigation function

A potential function. Imagine dropping a marble on the surface. It will avoid the three obstacles and eventually reach the goal position in the center.

Potential functions assume that the environment or work space is known. Obstacles are assigned a high potential value, and the goal position is assigned a low potential. To reach the goal position, a robot only needs to follow the negative gradient of the surface.

We can formalize this concept mathematically as following: Let ${\displaystyle X}$ be the state space of all possible configurations of a robot. Let ${\displaystyle X_{g}\subset X}$ denote the goal region of the state space.

Then a potential function ${\displaystyle \phi (x)}$ is called a (feasible) navigation function if [1]

1. ${\displaystyle \phi (x)=0\ \forall x\in X_{g}}$
2. ${\displaystyle \phi (x)=\infty }$ if and only if no point in ${\displaystyle {X_{g}}}$ is reachable from ${\displaystyle x}$.
3. For every reachable state, ${\displaystyle x\in X\setminus {X_{g}}}$, the local operator produces a state ${\displaystyle x'}$ for which ${\displaystyle \phi (x')<\phi (x)}$.

While for certain applications, it suffices to have a feasible navigation function, in many cases it is desirable to have an optimal navigation function with respect to a given cost functional ${\displaystyle J}$. Formalized as an optimal control problem, we can write

${\displaystyle {\text{minimize }}J(x_{1:T},u_{1:T})=\int \limits _{T}L(x_{t},u_{t},t)dt}$

${\displaystyle {\text{subject to }}{\dot {x_{t}}}=f(x_{t},u_{t})}$

whereby ${\displaystyle x}$ is the state, ${\displaystyle u}$ is the control to apply, ${\displaystyle L}$ is a cost at a certain state ${\displaystyle x}$ if we apply a control ${\displaystyle u}$, and ${\displaystyle f}$ models the transition dynamics of the system.

Applying Bellman's principle of optimality the optimal cost-to-go function is defined as

${\displaystyle \displaystyle \phi (x_{t})=\min _{u_{t}\in U(x_{t})}{\Big \{}L(x_{t},u_{t})+\phi (f(x_{t},u_{t})){\Big \}}}$

Together with the above defined axioms we can define the optimal navigation function as

1. ${\displaystyle \phi (x)=0\ \forall x\in X_{g}}$
2. ${\displaystyle \phi (x)=\infty }$ if and only if no point in ${\displaystyle {X_{G}}}$ is reachable from ${\displaystyle x}$.
3. For every reachable state, ${\displaystyle x\in X\setminus {X_{G}}}$, the local operator produces a state ${\displaystyle x'}$ for which ${\displaystyle \phi (x')<\phi (x)}$.
4. ${\displaystyle \displaystyle \phi (x_{t})=\min _{u_{t}\in U(x_{t})}{\Big \{}L(x_{t},u_{t})+\phi (f(x_{t},u_{t})){\Big \}}}$

If we assume the transition dynamics of the system or the cost function as subjected to noise, we obtain a stochastic optimal control problem with a cost ${\displaystyle J(x_{t},u_{t})}$ and dynamics ${\displaystyle f}$. In the field of reinforcement learning the cost is replaced by a reward function ${\displaystyle R(x_{t},u_{t})}$ and the dynamics by the transition probabilities ${\displaystyle P(x_{t+1}|x_{t},u_{t})}$.

• Control Theory
• Optimal Control
• Robot Control
• Motion Planning
• Reinforcement Learning

Sources

• LaValle, Steven M. (2006), Planning Algorithms (First ed.), Cambridge University Press, ISBN 978-0-521-86205-9
• Laumond, Jean-Paul (1998), Robot Motion Planning and Control (First ed.), Springer, ISBN 3-540-76219-1

• NFsim: MATLAB Toolbox for motion planning using Navigation Functions.