ceres-solver / ceres-solver / e91995cce456d7edf404103bd3dc40794e13886e / . / docs / source / derivatives.rst

.. default-domain:: cpp | |

.. cpp:namespace:: ceres | |

.. _chapter-on_derivatives: | |

============== | |

On Derivatives | |

============== | |

Ceres Solver, like all gradient based optimization algorithms, depends | |

on being able to evaluate the objective function and its derivatives | |

at arbitrary points in its domain. Indeed, defining the objective | |

function and its `Jacobian | |

<https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant>`_ is | |

the principal task that the user is required to perform when solving | |

an optimization problem using Ceres Solver. The correct and efficient | |

computation of the Jacobian is the key to good performance. | |

Ceres Solver offers considerable flexibility in how the user can | |

provide derivatives to the solver. She can use: | |

#. :ref:`chapter-analytical_derivatives`: The user figures out the | |

derivatives herself, by hand or using a tool like `Maple | |

<https://www.maplesoft.com/products/maple/>`_ or `Mathematica | |

<https://www.wolfram.com/mathematica/>`_, and implements them in a | |

:class:`CostFunction`. | |

#. :ref:`chapter-numerical_derivatives`: Ceres numerically computes | |

the derivative using finite differences. | |

#. :ref:`chapter-automatic_derivatives`: Ceres automatically computes | |

the analytic derivative using C++ templates and operator | |

overloading. | |

Which of these three approaches (alone or in combination) should be | |

used depends on the situation and the tradeoffs the user is willing to | |

make. Unfortunately, numerical optimization textbooks rarely discuss | |

these issues in detail and the user is left to her own devices. | |

The aim of this article is to fill this gap and describe each of these | |

three approaches in the context of Ceres Solver with sufficient detail | |

that the user can make an informed choice. | |

For the impatient amongst you, here is some high level advice: | |

#. Use :ref:`chapter-automatic_derivatives`. | |

#. In some cases it maybe worth using | |

:ref:`chapter-analytical_derivatives`. | |

#. Avoid :ref:`chapter-numerical_derivatives`. Use it as a measure of | |

last resort, mostly to interface with external libraries. | |

For the rest, read on. | |

.. toctree:: | |

:maxdepth: 1 | |

spivak_notation | |

analytical_derivatives | |

numerical_derivatives | |

automatic_derivatives | |

interfacing_with_autodiff | |

inverse_and_implicit_function_theorems |