Documentation update.
Change-Id: I0a3c5ae4bc981a8f5bdd5a8905f923dc5f09a024
diff --git a/docs/build.tex b/docs/build.tex
index e8b564f..9794663 100644
--- a/docs/build.tex
+++ b/docs/build.tex
@@ -101,7 +101,7 @@
You can also try running the command line bundling application with one of the
included problems, which comes from the University of Washington's BAL dataset~\cite{Agarwal10bal}:
\begin{minted}{bash}
-examples/simple_bundle_adjuster \
+bin/simple_bundle_adjuster \
../ceres-solver-1.2.1/data/problem-16-22106-pre.txt \
\end{minted}
This runs Ceres for a maximum of 10 iterations using the \denseschur\ linear solver. The output should look something like this.
@@ -185,7 +185,7 @@
make -j3
make test
\end{minted}
-Like the Linux build, you should now be able to run \texttt{examples/simple\_bundle\_adjuster}.
+Like the Linux build, you should now be able to run \texttt{bin/simple\_bundle\_adjuster}.
\section{Building on Windows with Visual Studio}
@@ -231,7 +231,7 @@
To run the tests, select the \texttt{RUN\_TESTS} target and hit "Build RUN\_TESTS" from the build menu.
-Like the Linux build, you should now be able to run \texttt{examples/simple\_bundle\_adjuster}.
+Like the Linux build, you should now be able to run \texttt{bin/simple\_bundle\_adjuster}.
Notes:
\begin{itemize}
@@ -247,6 +247,12 @@
port.
\end{itemize}
+\section{Building on Android}
+\label{sec:android}
+Download the Android NDK. Run \texttt{ndk-build} from inside the \texttt{jni} directory. Use the \texttt{libceres.a} that gets created.
+
+TODO(keir): Expand this section further.
+
\section{Compiler Flags to use when building your own applications}
\label{sec:compiler-flags}
TBD
diff --git a/docs/ceres.bib b/docs/ceres-solver.bib
similarity index 91%
rename from docs/ceres.bib
rename to docs/ceres-solver.bib
index b26a984..09de30d 100644
--- a/docs/ceres.bib
+++ b/docs/ceres-solver.bib
@@ -1,3 +1,20 @@
+@book{conn2000trust,
+ title={Trust-region methods},
+ author={Conn, A.R. and Gould, N.I.M. and Toint, P.L.},
+ volume={1},
+ year={2000},
+ publisher={Society for Industrial Mathematics}
+}
+@article{byrd1988approximate,
+ title={Approximate solution of the trust region problem by minimization over two-dimensional subspaces},
+ author={Byrd, R.H. and Schnabel, R.B. and Shultz, G.A.},
+ journal={Mathematical programming},
+ volume={40},
+ number={1},
+ pages={247--263},
+ year={1988},
+ publisher={Springer}
+}
@article{stigler1981gauss,
title={Gauss and the invention of least squares},
author={Stigler, S.M.},
diff --git a/docs/ceres-solver.tex b/docs/ceres-solver.tex
index ca839ea..cf5e52d 100644
--- a/docs/ceres-solver.tex
+++ b/docs/ceres-solver.tex
@@ -128,5 +128,5 @@
\input{faq}
\input{further}
\bibliographystyle{plain}
-\bibliography{ceres}
+\bibliography{ceres-solver}
\end{document}
diff --git a/docs/changes.tex b/docs/changes.tex
index bc85ddb..8200eb6 100644
--- a/docs/changes.tex
+++ b/docs/changes.tex
@@ -11,17 +11,17 @@
\item New loss functions \texttt{ArcTanLossFunction,
TolerantLossFunction} and \texttt{ComposedLossFunction}. (James Roseborough).
\item New \texttt{DENSE\_NORMAL\_CHOLESKY} linear solver, which uses Eigen's
- \texttt{LDLT} factorization on the normal equations.
+ LDLT factorization on the normal equations.
\item The traditional Dogleg solver now uses an elliptical trust
region (Markus Moll)
\item Support for returning initial and final gradients \& Jacobians.
\item Gradient computation support in the evaluators, with an eye
towards developing first order/gradient based solvers.
\item A better way to compute \texttt{Solver::Summary::fixed\_cost}. (Markus Moll)
-\item Cmake support for building documentation, separate examples,
+\item \texttt{CMake} support for building documentation, separate examples,
installing and uninstalling the library and Gerrit hooks (Arnaud
Gelas)
-\item SuiteSparse4 support (Markus Moll)
+\item \texttt{SuiteSparse4} support (Markus Moll)
\item Support for building Ceres without \texttt{TR1} (This leads to
slightly slower \texttt{DENSE\_SCHUR} and \texttt{SPARSE\_SCHUR} solvers).
\item \texttt{BALProblem} can now write a problem back to disk.
@@ -66,7 +66,7 @@
\item Fixed Jet to integer comparison. (Keith Leung)
\item Fix use of uninitialized arrays. (Sebastian Koch \& Markus Moll)
\item Conditionally compile gflag dependencies.(Casey Goodlett)
-\item Add \texttt{data\_fitting.cc } to the examples CMake file.
+\item Add \texttt{data\_fitting.cc } to the examples \texttt{CMake} file.
\end{itemize}
\section*{1.2.3}
@@ -111,7 +111,7 @@
\texttt{CHOLMOD} significantly.
\item New Trust region loop with support for multiple
trust region step strategies. Currently only Levenberg-Marquardt is supported, but this refactoring opens the door for Dog-leg, Stiehaug and others.
-\item \texttt{Cmake} file restructuring. Builds in \texttt{Release} mode by default, and now has platform specific tuning flags.
+\item \texttt{CMake} file restructuring. Builds in \texttt{Release} mode by default, and now has platform specific tuning flags.
\item Re-organized documentation. No new content, but better organization.
\end{itemize}
diff --git a/docs/modeling.tex b/docs/modeling.tex
index 5cd199c..82080b7 100644
--- a/docs/modeling.tex
+++ b/docs/modeling.tex
@@ -2,8 +2,8 @@
\chapter{Modeling}
\label{chapter:api}
\section{\texttt{CostFunction}}
-Given parameter blocks $\left[x_{i_1}, \hdots , x_{i_k}\right]$, a \texttt{CostFunction} is responsible for computing
-a vector of residuals and if asked a vector of Jacobian matrices, i.e., given $\left[x_{i_1}, \hdots , x_{i_k}\right]$, compute the vector $f_i\left(x_{i_1},\hdots,x_{k_i}\right)$ and the matrices
+Given parameter blocks $\left[x_{i_1}, \hdots , x_{i_k}\right]$, a \texttt{CostFunction} is responsible for computing
+a vector of residuals and if asked a vector of Jacobian matrices, i.e., given $\left[x_{i_1}, \hdots , x_{i_k}\right]$, compute the vector $f_i\left(x_{i_1},\hdots,x_{k_i}\right)$ and the matrices
\begin{equation}
J_{ij} = \frac{\partial}{\partial x_{i_j}}f_i\left(x_{i_1},\hdots,x_{k_i}\right),\quad \forall j = i_1,\hdots, i_k
\end{equation}
@@ -13,7 +13,7 @@
virtual bool Evaluate(double const* const* parameters,
double* residuals,
double** jacobians) = 0;
- const vector<int16>& parameter_block_sizes();
+ const vector<int16>& parameter_block_sizes();
int num_residuals() const;
protected:
@@ -26,18 +26,18 @@
is stored in \texttt{parameter\_block\_sizes\_} and \texttt{num\_residuals\_} respectively. User
code inheriting from this class is expected to set these two members with the
corresponding accessors. This information will be verified by the Problem
-when added with \texttt{Problem::AddResidualBlock}.
+when added with \texttt{Problem::AddResidualBlock}.
The most important method here is \texttt{Evaluate}. It implements the residual and Jacobian computation.
-\texttt{parameters} is an array of pointers to arrays containing the various parameter blocks. parameters has the same number of elements as parameter\_block\_sizes\_. Parameter blocks are in the same order as parameter\_block\_sizes\_.
+\texttt{parameters} is an array of pointers to arrays containing the various parameter blocks. parameters has the same number of elements as parameter\_block\_sizes\_. Parameter blocks are in the same order as parameter\_block\_sizes\_.
-\texttt{residuals} is an array of size \texttt{num\_residuals\_}.
+\texttt{residuals} is an array of size \texttt{num\_residuals\_}.
-\texttt{jacobians} is an array of size \texttt{parameter\_block\_sizes\_} containing pointers to storage for Jacobian matrices corresponding to each parameter block. The Jacobian matrices are in the same order as \texttt{parameter\_block\_sizes\_}. \texttt{jacobians[i]} is an array that contains \texttt{num\_residuals\_} $\times$ \texttt{parameter\_block\_sizes\_[i]} elements. Each Jacobian matrix is stored in row-major order, i.e.,
-
+\texttt{jacobians} is an array of size \texttt{parameter\_block\_sizes\_} containing pointers to storage for Jacobian matrices corresponding to each parameter block. The Jacobian matrices are in the same order as \texttt{parameter\_block\_sizes\_}. \texttt{jacobians[i]} is an array that contains \texttt{num\_residuals\_} $\times$ \texttt{parameter\_block\_sizes\_[i]} elements. Each Jacobian matrix is stored in row-major order, i.e.,
+
\begin{equation}
\texttt{jacobians[i][r * parameter\_block\_size\_[i] + c]} =
%\frac{\partial}{\partial x_{ic}} f_{r}\left(x_{1},\hdots, x_{k}\right)
@@ -98,7 +98,7 @@
double k_;
};
\end{minted}
-
+
Note that in the declaration of \texttt{operator()} the input parameters \texttt{x} and \texttt{y} come
first, and are passed as const pointers to arrays of \texttt{T}. If there were three
input parameters, then the third input parameter would come after \texttt{y}. The
@@ -136,7 +136,7 @@
set the template parameters to (dimension of residual, number of parameters)
instead of passing a dimension parameter for {\em every parameter block}. In the
example above, that would be \texttt{<MyScalarCostFunction, 1, 2>}, which is missing
- the 2 as the last template argument.
+ the 2 as the last template argument.
\subsection{Theory \& Implementation}
TBD
@@ -206,7 +206,7 @@
public:
virtual void Evaluate(double s, double out[3]) const = 0;
};
-\end{minted}
+\end{minted}
The key method is \texttt{Evaluate}, which given a non-negative scalar \texttt{s}, computes
\begin{align}
@@ -227,7 +227,7 @@
\subsection{Scaling}
Given one robustifier $\rho(s)$
one can change the length scale at which robustification takes
- place, by adding a scale factor $a > 0$ which gives us $\rho(s,a) = a^2 \rho(s / a^2)$ and the first and second derivatives as $\rho'(s / a^2)$ and $(1 / a^2) \rho''(s / a^2)$ respectively.
+ place, by adding a scale factor $a > 0$ which gives us $\rho(s,a) = a^2 \rho(s / a^2)$ and the first and second derivatives as $\rho'(s / a^2)$ and $(1 / a^2) \rho''(s / a^2)$ respectively.
\begin{figure}[hbt]
@@ -238,7 +238,7 @@
The reason for the appearance of squaring is that $a$ is in the units of the residual vector norm whereas $s$ is a squared norm. For applications it is more convenient to specify $a$ than
-its square.
+its square.
Here are some common loss functions implemented in Ceres. For simplicity we described their unscaled versions. Figure~\ref{fig:loss} illustrates their shape graphically.
@@ -252,8 +252,36 @@
\rho(s) &= \log(1 + s) \tag{\texttt{CauchyLoss}}
\end{align}
+Ceres includes a number of other loss functions, the descriptions and
+documentation for which can be found in \texttt{loss\_function.h}.
+
\subsection{Theory \& Implementation}
-TBD
+Let us consider a problem with a single problem and a single parameter
+block.
+\begin{align}
+\min_x \frac{1}{2}\rho(f^2(x))
+\end{align}
+
+Then, the robustified gradient and the Gauss-Newton Hessian are
+\begin{align}
+ g(x) &= \rho'J^\top(x)f(x)\\
+ H(x) &= J^\top(x)\left(\rho' + 2 \rho''f(x)f^\top(x)\right)J(x)
+\end{align}
+where the terms involving the second derivatives of $f(x)$ have been ignored. Note that $H(x)$ is indefinite if $\rho''f(x)^\top f(x) + \frac{1}{2}\rho' < 0$. If this is not the case, then its possible to re-weight the residual and the Jacobian matrix such that the corresponding linear least squares problem for the robustified Gauss-Newton step.
+
+
+Let $\alpha$ be a root of
+\begin{equation}
+ \frac{1}{2}\alpha^2 - \alpha - \frac{\rho''}{\rho'}\|f(x)\|^2 = 0.
+\end{equation}
+Then, define the rescaled residual and Jacobian as
+\begin{align}
+ \tilde{f}(x) &= \frac{\sqrt{\rho'}}{1 - \alpha} f(x)\\
+ \tilde{J}(x) &= \sqrt{\rho'}\left(1 - \alpha \frac{f(x)f^\top(x)}{\left\|f(x)\right\|^2} \right)J(x)
+\end{align}
+In the case $2 \rho''\left\|f(x)\right\|^2 + \rho' \lesssim 0$, we limit $\alpha \le 1- \epsilon$ for some small $\epsilon$. For more details see Triggs et al~\cite{triggs-etal-1999}.
+
+With this simple rescaling, one can use any Jacobian based non-linear least squares algorithm to robustifed non-linear least squares problems.
\section{\texttt{LocalParameterization}}
Sometimes the parameters $x$ can overparameterize a problem. In
@@ -302,7 +330,7 @@
};
\end{minted}
-\texttt{GlobalSize} is the dimension of the ambient space in which the parameter block $x$ lives. \texttt{LocalSize} is the size of the tangent space that $\Delta x$ lives in. \texttt{Plus} implements $\boxplus(x,\Delta x)$ and $\texttt{ComputeJacobian}$ computes the Jacobian matrix
+\texttt{GlobalSize} is the dimension of the ambient space in which the parameter block $x$ lives. \texttt{LocalSize} is the size of the tangent space that $\Delta x$ lives in. \texttt{Plus} implements $\boxplus(x,\Delta x)$ and $\texttt{ComputeJacobian}$ computes the Jacobian matrix
\begin{equation}
J = \left . \frac{\partial }{\partial \Delta x} \boxplus(x,\Delta x)\right|_{\Delta x = 0}
\end{equation}
@@ -334,9 +362,9 @@
dimensional vector and define $\boxplus$ to be
\begin{equation}
- \boxplus(x, \Delta x) =
+ \boxplus(x, \Delta x) =
\left[
-\cos(|\Delta x|), \frac{\sin\left(|\Delta x|\right)}{|\Delta x|} \Delta x
+\cos(|\Delta x|), \frac{\sin\left(|\Delta x|\right)}{|\Delta x|} \Delta x
\right] * x
\label{eq:quaternion}
\end{equation}
@@ -413,13 +441,13 @@
blocks if they are not present, so calling \texttt{AddParameterBlock}
explicitly is not required.
-
+
\texttt{Problem} by default takes ownership of the
\texttt{cost\_function} and \texttt{loss\_function pointers}. These objects remain
live for the life of the \texttt{Problem} object. If the user wishes to
keep control over the destruction of these objects, then they can
do this by setting the corresponding enums in the \texttt{Options} struct.
-
+
Note that even though the Problem takes ownership of \texttt{cost\_function}
and \texttt{loss\_function}, it does not preclude the user from re-using
@@ -430,9 +458,9 @@
\texttt{AddParameterBlock} explicitly adds a parameter block to the \texttt{Problem}. Optionally it allows the user to associate a LocalParameterization object with the parameter block too. Repeated calls with the same arguments are ignored. Repeated
calls with the same double pointer but a different size results in undefined behaviour.
-You can set any parameter block to be constant using
+You can set any parameter block to be constant using
-\texttt{Problem::SetParameterBlockConstant}
+\texttt{Problem::SetParameterBlockConstant}
and undo this using
@@ -445,4 +473,4 @@
live for the life of the \texttt{Problem} object. If the user wishes to
keep control over the destruction of these objects, then they can
do this by setting the corresponding enums in the \texttt{Options} struct. Even though \texttt{Problem} takes ownership of these pointers, it does not preclude the user from re-using them in another residual or parameter block. The destructor takes care to call
- delete on each pointer only once.
\ No newline at end of file
+ delete on each pointer only once.
diff --git a/docs/solving.tex b/docs/solving.tex
index c2b7317..3e9f419 100644
--- a/docs/solving.tex
+++ b/docs/solving.tex
@@ -3,6 +3,7 @@
Effective use of Ceres requires some familiarity with the basic components of a nonlinear least squares solver, so before we describe how to configure the solver, we will begin by taking a brief look at how some of the core optimization algorithms in Ceres work and the various linear solvers and preconditioners that power it.
\section{Trust Region Methods}
+\label{sec:trust-region}
Let $x \in \mathbb{R}^{n}$ be an $n$-dimensional vector of variables, and
$ F(x) = \left[f_1(x), \hdots, f_{m}(x) \right]^{\top}$ be a $m$-dimensional function of $x$. We are interested in solving the following optimization problem~\footnote{At the level of the non-linear solver, the block and residual structure is not relevant, therefore our discussion here is in terms of an optimization problem defined over a state vector of size $n$.},
\begin{equation}
@@ -16,9 +17,14 @@
\min_{\Delta x} \frac{1}{2}\|J(x)\Delta x + F(x)\|^2
\label{eq:linearapprox}
\end{equation}
-Unfortunately, na\"ively solving a sequence of these problems and updating $x \leftarrow x+ \Delta x$ leads to an algorithm that may not converge. To get a convergent algorithm, we need to control the size of the step $\Delta x$. And this is where the idea of a trust-region comes in. The generic trust-region loop for non-linear least squares problems looks something like this
+Unfortunately, na\"ively solving a sequence of these problems and
+updating $x \leftarrow x+ \Delta x$ leads to an algorithm that may not
+converge. To get a convergent algorithm, we need to control the size
+of the step $\Delta x$. And this is where the idea of a trust-region
+comes in. Algorithm~\ref{alg:trust-region} describes the basic trust-region loop for non-linear least squares problems.
-
+\begin{algorithm}
+\caption{The basic trust-region algorithm.\label{alg:trust-region}}
\begin{algorithmic}
\REQUIRE Initial point $x$ and a trust region radius $\mu$.
\LOOP
@@ -36,6 +42,7 @@
\ENDIF
\ENDLOOP
\end{algorithmic}
+\end{algorithm}
Here, $\mu$ is the trust region radius, $D(x)$ is some matrix used to define a metric on the domain of $F(x)$ and $\rho$ measures the quality of the step $\Delta x$, i.e., how well did the linear model predict the decrease in the value of the non-linear objective. The idea is to increase or decrease the radius of the trust region depending on how well the linearization predicts the behavior of the non-linear objective, which in turn is reflected in the value of $\rho$.
@@ -46,7 +53,7 @@
\label{eq:trp}
\end{align}
-There are a number of different ways of solving this problem, each giving rise to a different concrete trust-region algorithm. Currently Ceres, implements two trust-region algorithms - Levenberg-Marquardt and Powell's Dogleg.
+There are a number of different ways of solving this problem, each giving rise to a different concrete trust-region algorithm. Currently Ceres, implements two trust-region algorithms - Levenberg-Marquardt and Dogleg.
\subsection{Levenberg-Marquardt}
The Levenberg-Marquardt algorithm~\cite{levenberg1944method, marquardt1963algorithm} is the most popular algorithm for solving non-linear least squares problems. It was also the first trust region algorithm to be developed~\cite{levenberg1944method,marquardt1963algorithm}. Ceres implements an exact step~\cite{madsen2004methods} and an inexact step variant of the Levenberg-Marquardt algorithm~\cite{wright1985inexact,nash1990assessing}.
@@ -79,20 +86,64 @@
Ceres supports both exact and inexact step solution strategies. When the user chooses a factorization based linear solver, the exact step Levenberg-Marquardt algorithm is used. When the user chooses an iterative linear solver, the inexact step Levenberg-Marquardt algorithm is used.
-\subsection{Powell's Dogleg}
+\subsection{Dogleg}
+\label{sec:dogleg}
Another strategy for solving the trust region problem~\eqref{eq:trp} was introduced by M. J. D. Powell. The key idea there is to compute two vectors
\begin{align}
\Delta x^{\text{Gauss-Newton}} &= \arg \min_{\Delta x}\frac{1}{2} \|J(x)\Delta x + f(x)\|^2.\\
\Delta x^{\text{Cauchy}} &= -\frac{\|g(x)\|^2}{\|J(x)g(x)\|^2}g(x).
\end{align}
-Note that the vector $\Delta x^{\text{Gauss-Newton}}$ is the solution to~\eqref{eq:linearapprox} and $\Delta x^{\text{Cauchy}}$ is the vector that minimizes the linear approximation if we restrict ourselves to moving along the direction of the gradient.
+Note that the vector $\Delta x^{\text{Gauss-Newton}}$ is the solution
+to~\eqref{eq:linearapprox} and $\Delta x^{\text{Cauchy}}$ is the
+vector that minimizes the linear approximation if we restrict
+ourselves to moving along the direction of the gradient. Dogleg methods finds a vector $\Delta x$ defined by $\Delta
+x^{\text{Gauss-Newton}}$ and $\Delta x^{\text{Cauchy}}$ that solves
+the trust region problem. Ceres supports two
+variants.
-Then Powell's Dogleg method finds a vector $\Delta x$ in the two dimensional subspace defined by $\Delta x^{\text{Gauss-Newton}}$ and $\Delta x^{\text{Cauchy}}$ that solves the trust region problem. For more details on the exact reasoning and computations, please see Madsen et al~\cite{madsen2004methods}.
+\texttt{TRADITIONAL\_DOGLEG} as described by Powell,
+constructs two line segments using the Gauss-Newton and Cauchy vectors
+and finds the point farthest along this line shaped like a dogleg
+(hence the name) that is contained in the
+trust-region. For more details on the exact reasoning and computations, please see Madsen et al~\cite{madsen2004methods}.
-The key advantage of the Dogleg over Levenberg Marquardt is that if the step computation for a particular choice of $\mu$ does not result in sufficient decrease in the value of the objective function, Levenberg-Marquardt solves the linear approximation from scratch with a small value of $\mu$. Dogleg on the other hand, only needs to compute the interpolation between the Gauss-Newton and the Cauchy vectors, as neither of them depend on the value of $\mu$.
+ \texttt{SUBSPACE\_DOGLEG} is a more sophisticated method
+that considers the entire two dimensional subspace spanned by these
+two vectors and finds the point that minimizes the trust region
+problem in this subspace\cite{byrd1988approximate}.
+
+The key advantage of the Dogleg over Levenberg Marquardt is that if the step computation for a particular choice of $\mu$ does not result in sufficient decrease in the value of the objective function, Levenberg-Marquardt solves the linear approximation from scratch with a smaller value of $\mu$. Dogleg on the other hand, only needs to compute the interpolation between the Gauss-Newton and the Cauchy vectors, as neither of them depend on the value of $\mu$.
The Dogleg method can only be used with the exact factorization based linear solvers.
+\subsection{Non-monotonic Steps}
+\label{sec:non-monotic}
+Note that the basic trust-region algorithm described in
+Algorithm~\ref{alg:trust-region} is a descent algorithm in that they
+only accepts a point if it strictly reduces the value of the objective
+function.
+
+Relaxing this requirement allows the algorithm to be more
+efficient in the long term at the cost of some local increase
+in the value of the objective function.
+
+This is because allowing for non-decreasing objective function
+values in a princpled manner allows the algorithm to ``jump over
+boulders'' as the method is not restricted to move into narrow
+valleys while preserving its convergence properties.
+
+Setting \texttt{Solver::Options::use\_nonmonotonic\_steps} to \texttt{true}
+enables the non-monotonic trust region algorithm as described by
+Conn, Gould \& Toint in~\cite{conn2000trust}.
+
+Even though the value of the objective function may be larger
+than the minimum value encountered over the course of the
+optimization, the final parameters returned to the user are the
+ones corresponding to the minimum cost over all iterations.
+
+The option to take non-monotonic is available for all trust region
+strategies.
+
\section{\texttt{LinearSolver}}
Recall that in both of the trust-region methods described above, the key computational cost is the solution of a linear least squares problem of the form
\begin{align}
@@ -113,19 +164,28 @@
\begin{align}
\Delta x^* = -R^{-1}Q^\top f
\end{align}
-Ceres uses \texttt{Eigen}'s dense QR decomposition routines.
+Ceres uses \texttt{Eigen}'s dense QR factorization routines.
-
-\subsection{\texttt{SPARSE\_NORMAL\_CHOLESKY}}
+\subsection{\texttt{DENSE\_NORMAL\_CHOLESKY} \& \texttt{SPARSE\_NORMAL\_CHOLESKY}}
Large non-linear least square problems are usually sparse. In such cases, using a dense QR factorization is inefficient. Let $H = R^\top R$ be the Cholesky factorization of the normal equations, where $R$ is an upper triangular matrix, then the solution to ~\eqref{eq:normal} is given by
\begin{equation}
\Delta x^* = R^{-1} R^{-\top} g.
\end{equation}
-The observant reader will note that the $R$ in the Cholesky factorization of $H$ is the same upper triangular matrix $R$ in the QR factorization of $J$. Since $Q$ is an orthonormal matrix, $J=QR$ implies that $J^\top J = R^\top Q^\top Q R = R^\top R$.
+The observant reader will note that the $R$ in the Cholesky
+factorization of $H$ is the same upper triangular matrix $R$ in the QR
+factorization of $J$. Since $Q$ is an orthonormal matrix, $J=QR$
+implies that $J^\top J = R^\top Q^\top Q R = R^\top R$. There are two variants of Cholesky factorization -- sparse and
+dense.
+\texttt{DENSE\_NORMAL\_CHOLESKY} as the name implies performs a dense
+Cholesky factorization of the normal equations. Ceres uses
+\texttt{Eigen}'s dense LDLT factorization routines.
-There are two variants of Cholesky factorization -- sparse and dense. \texttt{SPARSE\_NORMAL\_CHOLESKY}, as the name implies performs a sparse Cholesky factorization of the normal equations. This leads to substantial savings in time and memory for large sparse problems. We use the Professor Tim Davis' \texttt{CHOLMOD} library (part of the \texttt{SuiteSparse} package) to perform the sparse cholesky~\cite{chen2006acs}.
-
+\texttt{SPARSE\_NORMAL\_CHOLESKY}, as the name implies performs a
+sparse Cholesky factorization of the normal equations. This leads to
+substantial savings in time and memory for large sparse
+problems. Ceres uses the sparse Cholesky factorization routines in Professor Tim Davis' \texttt{SuiteSparse} or
+\texttt{CXSparse} packages~\cite{chen2006acs}.
\subsection{\texttt{DENSE\_SCHUR} \& \texttt{SPARSE\_SCHUR}}
While it is possible to use \texttt{SPARSE\_NORMAL\_CHOLESKY} to solve bundle adjustment problems, bundle adjustment problem have a special structure, and a more efficient scheme for solving~\eqref{eq:normal} can be constructed.
@@ -248,37 +308,77 @@
\section{\texttt{Solver::Options}}
-\texttt{Solver::Options} controls the overall behavior of the solver. We list the various settings and their default values below.
+\texttt{Solver::Options} controls the overall behavior of the
+solver. We list the various settings and their default values below.
\begin{enumerate}
-\item{\texttt{trust\_region\_strategy\_type }} (\texttt{LEVENBERG\_MARQUARDT}) The trust region step computation algorithm used by Ceres. Currently \texttt{LEVENBERG\_MARQUARDT } and \texttt{DOGLEG} are the two valid choices.
+\item{\texttt{trust\_region\_strategy\_type }}
+ (\texttt{LEVENBERG\_MARQUARDT}) The trust region step computation
+ algorithm used by Ceres. Currently \texttt{LEVENBERG\_MARQUARDT }
+ and \texttt{DOGLEG} are the two valid choices.
-\item{\texttt{max\_num\_iterations }}(\texttt{50}) Maximum number of iterations for Levenberg-Marquardt.
+\item{\texttt{dogleg\_type}} (\texttt{TRADITIONAL\_DOGLEG}) Ceres
+ supports two different dogleg strategies.
+ \texttt{TRADITIONAL\_DOGLEG} method by Powell and the
+ \texttt{SUBSPACE\_DOGLEG} method described by Byrd et al.
+~\cite{byrd1988approximate}. See Section~\ref{sec:dogleg} for more details.
-\item{\texttt{max\_solver\_time\_in\_seconds }} ($10^9$) Maximum amount of time for which the solver should run.
+\item{\texttt{use\_nonmonotoic\_steps}} (\texttt{false})
+Relax the requirement that the trust-region algorithm take strictly
+decreasing steps. See Section~\ref{sec:non-monotonic} for more details.
-\item{\texttt{num\_threads }}(\texttt{1})
-Number of threads used by Ceres to evaluate the Jacobian.
+\item{\texttt{max\_consecutive\_nonmonotonic\_steps}} (5)
+The window size used by the step selection algorithm to accept
+non-monotonic steps.
-\item{\texttt{initial\_trust\_region\_radius } ($10^4$)} The size of the initial trust region. When the \texttt{LEVENBERG\_MARQUARDT} strategy is used, the reciprocal of this number is the initial regularization parameter.
+\item{\texttt{max\_num\_iterations }}(\texttt{50}) Maximum number of
+ iterations for Levenberg-Marquardt.
-\item{\texttt{max\_trust\_region\_radius } ($10^{16}$)} The trust region radius is not allowed to grow beyond this value.
-\item{\texttt{max\_trust\_region\_radius } ($10^{-32}$)} The solver terminates, when the trust region becomes smaller than this value.
+\item{\texttt{max\_solver\_time\_in\_seconds }} ($10^9$) Maximum
+ amount of time for which the solver should run.
-\item{\texttt{min\_relative\_decrease }}($10^{-3}$) Lower threshold for relative decrease before a Levenberg-Marquardt step is acceped.
+\item{\texttt{num\_threads }}(\texttt{1}) Number of threads used by
+ Ceres to evaluate the Jacobian.
-\item{\texttt{lm\_min\_diagonal } ($10^6$)} The \texttt{LEVENBERG\_MARQUARDT} strategy, uses a diagonal matrix to regularize the the trust region step. This is the lower bound on the values of this diagonal matrix.
+\item{\texttt{initial\_trust\_region\_radius } ($10^4$)} The size of
+ the initial trust region. When the \texttt{LEVENBERG\_MARQUARDT}
+ strategy is used, the reciprocal of this number is the initial
+ regularization parameter.
-\item{\texttt{lm\_max\_diagonal } ($10^{32}$)} The \texttt{LEVENBERG\_MARQUARDT} strategy, uses a diagonal matrix to regularize the the trust region step. This is the upper bound on the values of this diagonal matrix.
+\item{\texttt{max\_trust\_region\_radius } ($10^{16}$)} The trust
+ region radius is not allowed to grow beyond this value.
-\item{\texttt{max\_num\_consecutive\_invalid\_steps } (5)} The step returned by a trust region strategy can sometimes be numerically invalid, usually because of conditioning issues. Instead of crashing or stopping the optimization, the optimizer can go ahead and try solving with a smaller trust region/better conditioned problem. This parameter sets the number of consecutive retries before the minimizer gives up.
+\item{\texttt{min\_trust\_region\_radius } ($10^{-32}$)} The solver
+ terminates, when the trust region becomes smaller than this value.
+
+\item{\texttt{min\_relative\_decrease }}($10^{-3}$) Lower threshold
+ for relative decrease before a Levenberg-Marquardt step is acceped.
+
+\item{\texttt{lm\_min\_diagonal } ($10^6$)} The
+ \texttt{LEVENBERG\_MARQUARDT} strategy, uses a diagonal matrix to
+ regularize the the trust region step. This is the lower bound on the
+ values of this diagonal matrix.
+
+\item{\texttt{lm\_max\_diagonal } ($10^{32}$)} The
+ \texttt{LEVENBERG\_MARQUARDT} strategy, uses a diagonal matrix to
+ regularize the the trust region step. This is the upper bound on the
+ values of this diagonal matrix.
+
+\item{\texttt{max\_num\_consecutive\_invalid\_steps } (5)} The step
+ returned by a trust region strategy can sometimes be numerically
+ invalid, usually because of conditioning issues. Instead of crashing
+ or stopping the optimization, the optimizer can go ahead and try
+ solving with a smaller trust region/better conditioned problem. This
+ parameter sets the number of consecutive retries before the
+ minimizer gives up.
\item{\texttt{function\_tolerance }}($10^{-6}$) Solver terminates if
\begin{align}
\frac{|\Delta \text{cost}|}{\text{cost}} < \texttt{function\_tolerance}
\end{align}
-where, $\Delta \text{cost}$ is the change in objective function value (up or down) in the current iteration of Levenberg-Marquardt.
+where, $\Delta \text{cost}$ is the change in objective function value
+(up or down) in the current iteration of Levenberg-Marquardt.
\item \texttt{Solver::Options::gradient\_tolerance } Solver terminates if
\begin{equation}
@@ -294,97 +394,163 @@
\item{\texttt{linear\_solver\_type }(\texttt{SPARSE\_NORMAL\_CHOLESKY})}
-\item{\texttt{linear\_solver\_type }}(\texttt{SPARSE\_NORMAL\_CHOLESKY}/\texttt{DENSE\_QR}) Type of linear solver used to compute the solution to the linear least squares problem in each iteration of the Levenberg-Marquardt algorithm. If Ceres is build with \suitesparse linked in then the default is \texttt{SPARSE\_NORMAL\_CHOLESKY}, it is \texttt{DENSE\_QR} otherwise.
+\item{\texttt{linear\_solver\_type
+ }}(\texttt{SPARSE\_NORMAL\_CHOLESKY}/\texttt{DENSE\_QR}) Type of
+ linear solver used to compute the solution to the linear least
+ squares problem in each iteration of the Levenberg-Marquardt
+ algorithm. If Ceres is build with \suitesparse linked in then the
+ default is \texttt{SPARSE\_NORMAL\_CHOLESKY}, it is
+ \texttt{DENSE\_QR} otherwise.
-\item{\texttt{preconditioner\_type }}(\texttt{JACOBI}) The preconditioner used by the iterative linear solver. The default is the block Jacobi preconditioner. Valid values are (in increasing order of complexity) \texttt{IDENTITY},\texttt{JACOBI}, \texttt{SCHUR\_JACOBI}, \texttt{CLUSTER\_JACOBI} and \texttt{CLUSTER\_TRIDIAGONAL}.
+\item{\texttt{preconditioner\_type }}(\texttt{JACOBI}) The
+ preconditioner used by the iterative linear solver. The default is
+ the block Jacobi preconditioner. Valid values are (in increasing
+ order of complexity) \texttt{IDENTITY},\texttt{JACOBI},
+ \texttt{SCHUR\_JACOBI}, \texttt{CLUSTER\_JACOBI} and
+ \texttt{CLUSTER\_TRIDIAGONAL}.
-\item{\texttt{sparse\_linear\_algebra\_library } (\texttt{SUITE\_SPARSE})} Ceres supports the use of two sparse linear algebra libraries, \texttt{SuiteSparse}, which is enabled by setting this parameter to \texttt{SUITE\_SPARSE} and \texttt{CXSparse}, which can be selected by setting this parameter to $\texttt{CX\_SPARSE}$. \texttt{SuiteSparse} is a sophisticated and complex sparse linear algebra library and should be used in general. If your needs/platforms prevent you from using \texttt{SuiteSparse}, consider using \texttt{CXSparse}, which is a much smaller, easier to build library. As can be expected, its performance on large problems is not comparable to that of \texttt{SuiteSparse}.
+\item{\texttt{sparse\_linear\_algebra\_library }
+ (\texttt{SUITE\_SPARSE})} Ceres supports the use of two sparse
+ linear algebra libraries, \texttt{SuiteSparse}, which is enabled by
+ setting this parameter to \texttt{SUITE\_SPARSE} and
+ \texttt{CXSparse}, which can be selected by setting this parameter
+ to $\texttt{CX\_SPARSE}$. \texttt{SuiteSparse} is a sophisticated
+ and complex sparse linear algebra library and should be used in
+ general. If your needs/platforms prevent you from using
+ \texttt{SuiteSparse}, consider using \texttt{CXSparse}, which is a
+ much smaller, easier to build library. As can be expected, its
+ performance on large problems is not comparable to that of
+ \texttt{SuiteSparse}.
-\item{\texttt{num\_linear\_solver\_threads }}(\texttt{1}) Number of threads used by the linear solver.
+\item{\texttt{num\_linear\_solver\_threads }}(\texttt{1}) Number of
+ threads used by the linear solver.
\item{\texttt{num\_eliminate\_blocks }}(\texttt{0})
For Schur reduction based methods, the first 0 to num blocks are
- eliminated using the Schur reduction. For example, when solving
- traditional structure from motion problems where the parameters are in
- two classes (cameras and points) then \texttt{num\_eliminate\_blocks} would be the
- number of points.
+eliminated using the Schur reduction. For example, when solving
+traditional structure from motion problems where the parameters are in
+two classes (cameras and points) then \texttt{num\_eliminate\_blocks}
+would be the number of points.
-\item{\texttt{ordering\_type }}(\texttt{NATURAL})
- Internally Ceres reorders the parameter blocks to help the
- various linear solvers. This parameter allows the user to
- influence the re-ordering strategy used. For structure from
- motion problems use \texttt{SCHUR}, for other problems \texttt{NATURAL} (default)
- is a good choice. In case you wish to specify your own ordering
- scheme, for example in conjunction with \texttt{num\_eliminate\_blocks},
- use \texttt{USER}.
+\item{\texttt{ordering\_type }}(\texttt{NATURAL}) Internally Ceres
+ reorders the parameter blocks to help the various linear
+ solvers. This parameter allows the user to influence the re-ordering
+ strategy used. For structure from motion problems use
+ \texttt{SCHUR}, for other problems \texttt{NATURAL} (default) is a
+ good choice. In case you wish to specify your own ordering scheme,
+ for example in conjunction with \texttt{num\_eliminate\_blocks}, use
+ \texttt{USER}.
-\item{\texttt{ordering }} The ordering of the parameter blocks. The solver pays attention
- to it if the \texttt{ordering\_type} is set to \texttt{USER} and the ordering vector is
- non-empty.
+\item{\texttt{ordering }} The ordering of the parameter blocks. The
+ solver pays attention to it if the \texttt{ordering\_type} is set to
+ \texttt{USER} and the ordering vector is non-empty.
-\item{\texttt{use\_block\_amd } (\texttt{true})} By virtue of the modeling layer in Ceres being block oriented,
-all the matrices used by Ceres are also block oriented.
+\item{\texttt{use\_block\_amd } (\texttt{true})} By virtue of the
+ modeling layer in Ceres being block oriented, all the matrices used
+ by Ceres are also block oriented.
When doing sparse direct factorization of these matrices, the
fill-reducing ordering algorithms can either be run on the
block or the scalar form of these matrices. Running it on the
block form exposes more of the super-nodal structure of the
matrix to the Cholesky factorization routines. This leads to
-substantial gains in factorization performance. Setting this parameter to true, enables the use of a block oriented Approximate Minimum Degree ordering algorithm. Settings it to \texttt{false}, uses a scalar AMD algorithm. This option only makes sense when using \texttt{sparse\_linear\_algebra\_library = SUITE\_SPARSE} as it uses the \texttt{AMD} package that is part of \texttt{SuiteSparse}.
+substantial gains in factorization performance. Setting this parameter
+to true, enables the use of a block oriented Approximate Minimum
+Degree ordering algorithm. Settings it to \texttt{false}, uses a
+scalar AMD algorithm. This option only makes sense when using
+\texttt{sparse\_linear\_algebra\_library = SUITE\_SPARSE} as it uses
+the \texttt{AMD} package that is part of \texttt{SuiteSparse}.
-\item{\texttt{linear\_solver\_min\_num\_iterations }}(\texttt{1}) Minimum number of iterations used by the linear solver. This only makes sense when the linear solver is an iterative solver, e.g., \texttt{ITERATIVE\_SCHUR}.
+\item{\texttt{linear\_solver\_min\_num\_iterations }}(\texttt{1})
+ Minimum number of iterations used by the linear solver. This only
+ makes sense when the linear solver is an iterative solver, e.g.,
+ \texttt{ITERATIVE\_SCHUR}.
-\item{\texttt{linear\_solver\_max\_num\_iterations }}(\texttt{500}) Minimum number of iterations used by the linear solver. This only makes sense when the linear solver is an iterative solver, e.g., \texttt{ITERATIVE\_SCHUR}.
+\item{\texttt{linear\_solver\_max\_num\_iterations }}(\texttt{500})
+ Minimum number of iterations used by the linear solver. This only
+ makes sense when the linear solver is an iterative solver, e.g.,
+ \texttt{ITERATIVE\_SCHUR}.
\item{\texttt{eta }} ($10^{-1}$)
- Forcing sequence parameter. The truncated Newton solver uses
- this number to control the relative accuracy with which the
- Newton step is computed. This constant is passed to ConjugateGradientsSolver which uses
- it to terminate the iterations when
+ Forcing sequence parameter. The truncated Newton solver uses this
+ number to control the relative accuracy with which the Newton step is
+ computed. This constant is passed to ConjugateGradientsSolver which
+ uses it to terminate the iterations when
\begin{equation}
- \frac{Q_i - Q_{i-1}}{Q_i} < \frac{\eta}{i}
+\frac{Q_i - Q_{i-1}}{Q_i} < \frac{\eta}{i}
\end{equation}
-\item{\texttt{jacobi\_scaling }}(\texttt{true}) \texttt{true} means that the Jacobian is scaled by the norm of its columns before being passed to the linear solver. This improves the numerical conditioning of the normal equations.
+\item{\texttt{jacobi\_scaling }}(\texttt{true}) \texttt{true} means
+ that the Jacobian is scaled by the norm of its columns before being
+ passed to the linear solver. This improves the numerical
+ conditioning of the normal equations.
\item{\texttt{logging\_type }}(\texttt{PER\_MINIMIZER\_ITERATION})
\item{\texttt{minimizer\_progress\_to\_stdout }}(\texttt{false})
-By default the Minimizer progress is logged to \texttt{STDERR} depending on the \texttt{vlog} level. If this flag is
-set to true, and \texttt{logging\_type } is not \texttt{SILENT}, the logging output
+By default the Minimizer progress is logged to \texttt{STDERR}
+depending on the \texttt{vlog} level. If this flag is
+set to true, and \texttt{logging\_type } is not \texttt{SILENT}, the
+logging output
is sent to \texttt{STDOUT}.
\item{\texttt{return\_initial\_residuals }}(\texttt{false})
\item{\texttt{return\_final\_residuals }}(\texttt{false})
-If true, the vectors \texttt{Solver::Summary::initial\_residuals } and \texttt{Solver::Summary::final\_residuals } are filled with the residuals before and after the optimization. The entries of these vectors are in the order in which ResidualBlocks were added to the Problem object.
-
+If true, the vectors \texttt{Solver::Summary::initial\_residuals } and
+\texttt{Solver::Summary::final\_residuals } are filled with the
+residuals before and after the optimization. The entries of these
+vectors are in the order in which ResidualBlocks were added to the
+Problem object.
+
\item{\texttt{return\_initial\_gradient }}(\texttt{false})
\item{\texttt{return\_final\_gradient }}(\texttt{false})
-If true, the vectors \texttt{Solver::Summary::initial\_gradient } and \texttt{Solver::Summary::final\_gradient } are filled with the gradient before and after the optimization. The entries of these vectors are in the order in which ParameterBlocks were added to the Problem object.
+If true, the vectors \texttt{Solver::Summary::initial\_gradient } and
+\texttt{Solver::Summary::final\_gradient } are filled with the
+gradient before and after the optimization. The entries of these
+vectors are in the order in which ParameterBlocks were added to the
+Problem object.
-Since \texttt{AddResidualBlock } adds ParameterBlocks to the \texttt{Problem } automatically if they do not already exist, if you wish to have explicit control over the ordering of the vectors, then use \texttt{Problem::AddParameterBlock } to explicitly add the ParameterBlocks in the order desired.
-
+Since \texttt{AddResidualBlock } adds ParameterBlocks to the
+\texttt{Problem } automatically if they do not already exist, if you
+wish to have explicit control over the ordering of the vectors, then
+use \texttt{Problem::AddParameterBlock } to explicitly add the
+ParameterBlocks in the order desired.
+
\item{\texttt{return\_initial\_jacobian }}(\texttt{false})
\item{\texttt{return\_initial\_jacobian }}(\texttt{false})
-If true, the Jacobian matrices before and after the optimization are returned in \texttt{Solver::Summary::initial\_jacobian } and \texttt{Solver::Summary::final\_jacobian } respectively.
+If true, the Jacobian matrices before and after the optimization are
+returned in \texttt{Solver::Summary::initial\_jacobian } and
+\texttt{Solver::Summary::final\_jacobian } respectively.
-The rows of these matrices are in the same order in which the ResidualBlocks were added to the Problem object. The columns are in the same order in which the ParameterBlocks were added to the Problem object.
-
-Since \texttt{AddResidualBlock } adds ParameterBlocks to the \texttt{Problem } automatically if they do not already exist, if you wish to have explicit control over the column ordering of the matrix, then use \texttt{Problem::AddParameterBlock } to explicitly add the ParameterBlocks in the order desired.
+The rows of these matrices are in the same order in which the
+ResidualBlocks were added to the Problem object. The columns are in
+the same order in which the ParameterBlocks were added to the Problem
+object.
-The Jacobian matrices are stored as compressed row sparse matrices. Please see \texttt{ceres/crs\_matrix.h } for more details of the format.
-
-\item{\texttt{lsqp\_iterations\_to\_dump }}
- List of iterations at which the optimizer should dump the
- linear least squares problem to disk. Useful for testing and
- benchmarking. If empty (default), no problems are dumped.
+Since \texttt{AddResidualBlock } adds ParameterBlocks to the
+\texttt{Problem } automatically if they do not already exist, if you
+wish to have explicit control over the column ordering of the matrix,
+then use \texttt{Problem::AddParameterBlock } to explicitly add the
+ParameterBlocks in the order desired.
+
+The Jacobian matrices are stored as compressed row sparse
+matrices. Please see \texttt{ceres/crs\_matrix.h } for more details of
+the format.
+
+\item{\texttt{lsqp\_iterations\_to\_dump }} List of iterations at
+ which the optimizer should dump the linear least squares problem to
+ disk. Useful for testing and benchmarking. If empty (default), no
+ problems are dumped.
\item{\texttt{lsqp\_dump\_directory }} (\texttt{/tmp})
- If \texttt{lsqp\_iterations\_to\_dump} is non-empty, then this setting determines the directory to which the files containing the linear least squares problems are written to.
+ If \texttt{lsqp\_iterations\_to\_dump} is non-empty, then this
+ setting determines the directory to which the files containing the
+ linear least squares problems are written to.
-\item{\texttt{lsqp\_dump\_format }}(\texttt{TEXTFILE}) The format in which linear least squares problems should be logged
+\item{\texttt{lsqp\_dump\_format }}(\texttt{TEXTFILE}) The format in
+ which linear least squares problems should be logged
when \texttt{lsqp\_iterations\_to\_dump} is non-empty. There are three options
\begin{itemize}
\item{\texttt{CONSOLE }} prints the linear least squares problem in a human readable format
@@ -396,7 +562,9 @@
pointed to by \texttt{lsqp\_dump\_directory} as a protocol
buffer. \texttt{linear\_least\_squares\_problems.h/cc} contains routines for
loading these problems. For details on the on disk format used,
- see \texttt{matrix.proto}. The files are named \texttt{lm\_iteration\_???.lsqp}. This requires that \texttt{protobuf} be linked into Ceres Solver.
+ see \texttt{matrix.proto}. The files are named
+ \texttt{lm\_iteration\_???.lsqp}. This requires that
+ \texttt{protobuf} be linked into Ceres Solver.
\item{\texttt{TEXTFILE }}
Write out the linear least squares problem to the directory
pointed to by \texttt{lsqp\_dump\_directory} as text files
@@ -429,33 +597,42 @@
values, \eg for forward differences, the numerical derivative is
\begin{align}
- \delta &= \texttt{numeric\_derivative\_relative\_step\_size}\\
- \Delta f &= \frac{f((1 + \delta) x) - f(x)}{\delta x}
+ \delta &= \texttt{numeric\_derivative\_relative\_step\_size}\\
+ \Delta f &= \frac{f((1 + \delta) x) - f(x)}{\delta x}
\end{align}
-
- The finite differencing is done along each dimension. The
- reason to use a relative (rather than absolute) step size is
- that this way, numeric differentiation works for functions where
- the arguments are typically large (e.g. $10^9$) and when the
- values are small (e.g. $10^{-5}$). It is possible to construct
- "torture cases" which break this finite difference heuristic,
- but they do not come up often in practice.
+The finite differencing is done along each dimension. The
+reason to use a relative (rather than absolute) step size is
+that this way, numeric differentiation works for functions where
+the arguments are typically large (e.g. $10^9$) and when the
+values are small (e.g. $10^{-5}$). It is possible to construct
+"torture cases" which break this finite difference heuristic,
+but they do not come up often in practice.
\item{\texttt{callbacks }}
Callbacks that are executed at the end of each iteration of the
- \texttt{Minimizer}. They are executed in the order that they are
- specified in this vector. By default, parameter blocks are
- updated only at the end of the optimization, i.e when the
- \texttt{Minimizer} terminates. This behavior is controlled by
- \texttt{update\_state\_every\_variable}. If the user wishes to have access
- to the update parameter blocks when his/her callbacks are
- executed, then set \texttt{update\_state\_every\_iteration} to true.
+\texttt{Minimizer}. They are executed in the order that they are
+specified in this vector. By default, parameter blocks are
+updated only at the end of the optimization, i.e when the
+\texttt{Minimizer} terminates. This behavior is controlled by
+\texttt{update\_state\_every\_variable}. If the user wishes to have access
+to the update parameter blocks when his/her callbacks are
+executed, then set \texttt{update\_state\_every\_iteration} to true.
- The solver does NOT take ownership of these pointers.
+The solver does NOT take ownership of these pointers.
\item{\texttt{update\_state\_every\_iteration }}(\texttt{false})
-Normally the parameter blocks are only updated when the solver terminates. Setting this to true update them in every iteration. This setting is useful when building an interactive application using Ceres and using an \texttt{IterationCallback}.
+Normally the parameter blocks are only updated when the solver
+terminates. Setting this to true update them in every iteration. This
+setting is useful when building an interactive application using Ceres
+and using an \texttt{IterationCallback}.
+
+\item{\texttt{solver\_log}} If non-empty, a summary of the execution of the solver is
+ recorded to this file. This file is used for recording and Ceres'
+ performance. Currently, only the iteration number, total
+ time and the objective function value are logged. The format of this
+ file is expected to change over time as the performance evaluation
+ framework is fleshed out.
\end{enumerate}
\section{\texttt{Solver::Summary}}
