Initial commit of Ceres Solver.
diff --git a/docs/api.tex b/docs/api.tex
new file mode 100644
index 0000000..2c36377
--- /dev/null
+++ b/docs/api.tex
@@ -0,0 +1,608 @@
+%!TEX root = ceres.tex
+\chapter{API Reference}
+\label{chapter:api}
+Ceres solves robustified non-linear least squares problems of the form
+\begin{equation}
+ \frac{1}{2}\sum_{i=1}^{k} \rho_i\left(\left\|f_i\left(x_{i_1},\hdots,x_{k_i}\right)\right\|^2\right).
+ \label{eq:ceresproblem}
+\end{equation}
+Where $f_i()$ is a cost function that depends on the parameter blocks $\left[x_{i_1}, \hdots , x_{i_k}\right]$ and $\rho_i$ is a loss function. In most optimization problems small groups of scalars occur together. For example the three components of a translation vector and the four components of the quaternion that define the pose of a camera. We refer to such a group of small scalars as a {\em Parameter Block}. Of course a parameter block can just have a single parameter.
+The term $ \rho_i\left(\left\|f_i\left(x_{i_1},\hdots,x_{k_i}\right)\right\|^2\right)$ is known as a residual block. A Ceres problem is a collection of residual blocks, each of which depends on a subset of the parameter blocks.
+
+
+\section{\texttt{CostFunction}}
+Given parameter blocks $\left[x_{i_1}, \hdots , x_{i_k}\right]$, a \texttt{CostFunction} is responsible for computing
+a vector of residuals and if asked a vector of Jacobian matrices, i.e., given $\left[x_{i1}, \hdots , x_{i_k}\right]$, compute the vector $f_i\left(x_{i_1},\hdots,x_{k_i}\right)$ and the matrices
+\begin{equation}
+J_{ij} = \frac{\partial}{\partial x_{i_j}}f_i\left(x_{i_1},\hdots,x_{k_i}\right),\quad \forall j = i_1,\hdots, i_k
+\end{equation}
+
+\begin{minted}{c++}
+class CostFunction {
+ public:
+ virtual bool Evaluate(double const* const* parameters,
+ double* residuals,
+ double** jacobians) = 0;
+ const vector<int16>& parameter_block_sizes();
+ int num_residuals() const;
+
+ protected:
+ vector<int16>* mutable_parameter_block_sizes();
+ void set_num_residuals(int num_residuals);
+};
+\end{minted}
+
+The signature of the function (number and sizes of input parameter blocks and number of outputs)
+is stored in \texttt{parameter\_block\_sizes\_} and \texttt{num\_residuals\_} respectively. User
+code inheriting from this class is expected to set these two members with the
+corresponding accessors. This information will be verified by the Problem
+when added with \texttt{Problem::AddResidualBlock}.
+
+The most important method here is \texttt{Evaluate}. It implements the residual and Jacobian computation.
+
+\texttt{parameters} is an array of pointers to arrays containing the various parameter blocks. parameters has the same number of elements as parameter\_block\_sizes\_. Parameter blocks are in the same order as parameter\_block\_sizes\_.
+
+
+\texttt{residuals} is an array of size \texttt{num\_residuals\_}.
+
+
+\texttt{jacobians} is an array of size \texttt{parameter\_block\_sizes\_} containing pointers to storage for Jacobian matrices corresponding to each parameter block. Jacobian matrices are in the same order as \texttt{parameter\_block\_sizes\_}, i.e., \texttt{jacobians[i]}, is an array that contains \texttt{num\_residuals\_} * \texttt{parameter\_block\_sizes\_[i]} elements. Each Jacobian matrix is stored in row-major order, i.e.,
+
+\begin{equation}
+\texttt{jacobians[i][r*parameter\_block\_size\_[i] + c]} = \frac{\partial \texttt{residual[r]}}{\partial \texttt{parameters[i][c]}}
+\end{equation}
+
+If \texttt{jacobians} is \texttt{NULL}, then no derivatives are returned; this is the case when computing cost only. If \texttt{jacobians[i]} is \texttt{NULL}, then the Jacobian matrix corresponding to the $i^{\textrm{th}}$ parameter block must not be returned, this is the case when the a parameter block is marked constant.
+
+\section{\texttt{SizedCostFunction}}
+If the size of the parameter blocks and the size of the residual vector is known at compile time (this is the common case), Ceres provides \texttt{SizedCostFunction}, where these values can be specified as template parameters.
+\begin{minted}{c++}
+template<int kNumResiduals,
+ int N0 = 0, int N1 = 0, int N2 = 0, int N3 = 0, int N4 = 0, int N5 = 0>
+class SizedCostFunction : public CostFunction {
+ public:
+ virtual bool Evaluate(double const* const* parameters,
+ double* residuals,
+ double** jacobians) = 0;
+};
+\end{minted}
+In this case the user only needs to implement the \texttt{Evaluate} method.
+
+\section{\texttt{AutoDiffCostFunction}}
+But even defining the \texttt{SizedCostFunction} can be a tedious affair if complicated derivative computations are involved. To this end Ceres provides automatic differentiation.
+
+To get an auto differentiated cost function, you must define a class with a
+ templated \texttt{operator()} (a functor) that computes the cost function in terms of
+ the template parameter \texttt{T}. The autodiff framework substitutes appropriate
+ \texttt{Jet} objects for T in order to compute the derivative when necessary, but
+ this is hidden, and you should write the function as if T were a scalar type
+ (e.g. a double-precision floating point number).
+
+ The function must write the computed value in the last argument (the only
+ non-\texttt{const} one) and return true to indicate success.
+
+ For example, consider a scalar error $e = k - x^\top y$, where both $x$ and $y$ are
+ two-dimensional vector parameters and $k$ is a constant. The form of this error, which is the
+ difference between a constant and an expression, is a common pattern in least
+ squares problems. For example, the value $x^\top y$ might be the model expectation
+ for a series of measurements, where there is an instance of the cost function
+ for each measurement $k$.
+
+ The actual cost added to the total problem is $e^2$, or $(k - x^\top y)^2$; however,
+ the squaring is implicitly done by the optimization framework.
+
+ To write an auto-differentiable cost function for the above model, first
+ define the object
+\begin{minted}{c++}
+class MyScalarCostFunction {
+ MyScalarCostFunction(double k): k_(k) {}
+ template <typename T>
+ bool operator()(const T* const x , const T* const y, T* e) const {
+ e[0] = T(k_) - x[0] * y[0] + x[1] * y[1]
+ return true;
+ }
+
+ private:
+ double k_;
+};
+\end{minted}
+
+Note that in the declaration of \texttt{operator()} the input parameters \texttt{x} and \texttt{y} come
+ first, and are passed as const pointers to arrays of \texttt{T}. If there were three
+ input parameters, then the third input parameter would come after \texttt{y}. The
+ output is always the last parameter, and is also a pointer to an array. In
+ the example above, \texttt{e} is a scalar, so only \texttt{e[0]} is set.
+
+ Then given this class definition, the auto differentiated cost function for
+ it can be constructed as follows.
+
+\begin{minted}{c++}
+CostFunction* cost_function
+ = new AutoDiffCostFunction<MyScalarCostFunction, 1, 2, 2>(
+ new MyScalarCostFunction(1.0)); ^ ^ ^
+ | | |
+ Dimension of residual ------+ | |
+ Dimension of x ----------------+ |
+ Dimension of y -------------------+
+\end{minted}
+
+In this example, there is usually an instance for each measumerent of k.
+
+In the instantiation above, the template parameters following
+ \texttt{MyScalarCostFunction}, \texttt{<1, 2, 2>} describe the functor as computing a
+ 1-dimensional output from two arguments, both 2-dimensional.
+
+ The framework can currently accommodate cost functions of up to 6 independent
+ variables, and there is no limit on the dimensionality of each of them.
+
+ \textbf{WARNING 1} Since the functor will get instantiated with different types for
+ \texttt{T}, you must convert from other numeric types to \texttt{T} before mixing
+ computations with other variables of type \texttt{T}. In the example above, this is
+ seen where instead of using \texttt{k\_} directly, \texttt{k\_} is wrapped with \texttt{T(k\_)}.
+
+ \textbf{WARNING 2} A common beginner's error when first using \texttt{AutoDiffCostFunction} is to get the sizing wrong. In particular, there is a tendency to
+ set the template parameters to (dimension of residual, number of parameters)
+ instead of passing a dimension parameter for {\em every parameter block}. In the
+ example above, that would be \texttt{<MyScalarCostFunction, 1, 2>}, which is missing
+ the 2 as the last template argument.
+
+\section{\texttt{NumericDiffCostFunction}}
+To get a numerically differentiated cost function, define a subclass of
+\texttt{CostFunction} such that the \texttt{Evaluate} function ignores the jacobian
+parameter. The numeric differentiation wrapper will fill in the jacobians array
+ if nececssary by repeatedly calling the \texttt{Evaluate} method with
+small changes to the appropriate parameters, and computing the slope. For
+performance, the numeric differentiation wrapper class is templated on the
+concrete cost function, even though it could be implemented only in terms of
+the virtual \texttt{CostFunction} interface.
+\begin{minted}{c++}
+template <typename CostFunctionNoJacobian,
+ NumericDiffMethod method = CENTRAL, int M = 0,
+ int N0 = 0, int N1 = 0, int N2 = 0, int N3 = 0, int N4 = 0, int N5 = 0>
+class NumericDiffCostFunction
+ : public SizedCostFunction<M, N0, N1, N2, N3, N4, N5> {
+};
+\end{minted}
+
+The numerically differentiated version of a cost function for a cost function
+can be constructed as follows:
+\begin{minted}{c++}
+CostFunction* cost_function
+ = new NumericDiffCostFunction<MyCostFunction, CENTRAL, 1, 4, 8>(
+ new MyCostFunction(...), TAKE_OWNERSHIP);
+\end{minted}
+where \texttt{MyCostFunction} has 1 residual and 2 parameter blocks with sizes 4 and 8
+respectively. Look at the tests for a more detailed example.
+
+The central difference method is considerably more accurate at the cost of
+twice as many function evaluations than forward difference. Consider using
+central differences begin with, and only after that works, trying forward
+difference to improve performance.
+
+\section{\texttt{LossFunction}}
+ For least squares problems where the minimization may encounter
+ input terms that contain outliers, that is, completely bogus
+ measurements, it is important to use a loss function that reduces
+ their influence.
+
+ Consider a structure from motion problem. The unknowns are 3D
+ points and camera parameters, and the measurements are image
+ coordinates describing the expected reprojected position for a
+ point in a camera. For example, we want to model the geometry of a
+ street scene with fire hydrants and cars, observed by a moving
+ camera with unknown parameters, and the only 3D points we care
+ about are the pointy tippy-tops of the fire hydrants. Our magic
+ image processing algorithm, which is responsible for producing the
+ measurements that are input to Ceres, has found and matched all
+ such tippy-tops in all image frames, except that in one of the
+ frame it mistook a car's headlight for a hydrant. If we didn't do
+ anything special the
+ residual for the erroneous measurement will result in the
+ entire solution getting pulled away from the optimum to reduce
+ the large error that would otherwise be attributed to the wrong
+ measurement.
+
+ Using a robust loss function, the cost for large residuals is
+ reduced. In the example above, this leads to outlier terms getting
+ down-weighted so they do not overly influence the final solution.
+
+\begin{minted}{c++}
+class LossFunction {
+ public:
+ virtual void Evaluate(double s, double out[3]) const = 0;
+};
+\end{minted}
+
+The key method is \texttt{Evaluate}, which given a non-negative scalar \texttt{s}, computes
+\begin{align}
+ \texttt{out} = \begin{bmatrix}\rho(s), & \rho'(s), & \rho''(s)\end{bmatrix}
+\end{align}
+
+Here the convention is that the contribution of a term to the cost function is given by $\frac{1}{2}\rho(s)$, where $s = \|f_i\|^2$. Calling the method with a negative value of $s$ is an error and the implementations are not required to handle that case.
+
+Most sane choices of $\rho$ satisfy:
+\begin{align}
+ \rho(0) &= 0\\
+ \rho'(0) &= 1\\
+ \rho'(s) &< 1 \text{ in the outlier region}\\
+ \rho''(s) &< 0 \text{ in the outlier region}
+\end{align}
+so that they mimic the squared cost for small residuals.
+
+\subsection{Scaling}
+Given one robustifier $\rho(s)$
+ one can change the length scale at which robustification takes
+ place, by adding a scale factor $a > 0$ which gives us $\rho(s,a) = a^2 \rho(s / a^2)$ and the first and second derivatives as $\rho'(s / a^2)$ and $(1 / a^2) \rho''(s / a^2)$ respectively.
+
+
+\begin{figure}[hbt]
+\includegraphics[width=\textwidth]{loss.pdf}
+\caption{Shape of the various common loss functions.}
+\label{fig:loss}
+\end{figure}
+
+
+The reason for the appearance of squaring is that $a$ is in the units of the residual vector norm whereas $s$ is a squared norm. For applications it is more convenient to specify $a$ than
+its square.
+
+Here are some common loss functions implemented in Ceres. For simplicity we described their unscaled versions. Figure~\ref{fig:loss} illustrates their shape graphically.
+
+\begin{align}
+ \rho(s)&=s \tag{\texttt{NullLoss}}\\
+ \rho(s) &= \begin{cases}
+ s & s \le 1\\
+ 2 \sqrt{s} - 1 & s > 1
+ \end{cases} \tag{\texttt{HuberLoss}}\\
+ \rho(s) &= 2 (\sqrt{1+s} - 1) \tag{\texttt{SoftLOneLoss}}\\
+ \rho(s) &= \log(1 + s) \tag{\texttt{CauchyLoss}}
+\end{align}
+
+\section{\texttt{LocalParameterization}}
+Sometimes the parameters $x$ can overparameterize a problem. In
+that case it is desirable to choose a parameterization to remove
+the null directions of the cost. More generally, if $x$ lies on a
+manifold of a smaller dimension than the ambient space that it is
+embedded in, then it is numerically and computationally more
+effective to optimize it using a parameterization that lives in
+the tangent space of that manifold at each point.
+
+For example, a sphere in three dimensions is a two dimensional
+manifold, embedded in a three dimensional space. At each point on
+the sphere, the plane tangent to it defines a two dimensional
+tangent space. For a cost function defined on this sphere, given a
+point $x$, moving in the direction normal to the sphere at that
+point is not useful. Thus a better way to parameterize a point on
+a sphere is to optimize over two dimensional vector $\Delta x$ in the
+tangent space at the point on the sphere point and then "move" to
+the point $x + \Delta x$, where the move operation involves projecting
+back onto the sphere. Doing so removes a redundant dimension from
+the optimization, making it numerically more robust and efficient.
+
+More generally we can define a function
+\begin{equation}
+ x' = \boxplus(x, \Delta x),
+\end{equation}
+where $x'$ has the same size as $x$, and $\Delta x$ is of size less
+than or equal to $x$. The function $\boxplus$, generalizes the
+definition of vector addition. Thus it satisfies the identity
+\begin{equation}
+ \boxplus(x, 0) = x,\quad \forall x.
+\end{equation}
+
+Instances of \texttt{LocalParameterization} implement the $\boxplus$ operation and its derivative with respect to $\Delta x$ at $\Delta x = 0$.
+
+\begin{minted}{c++}
+class LocalParameterization {
+ public:
+ virtual ~LocalParameterization() {}
+ virtual bool Plus(const double* x,
+ const double* delta,
+ double* x_plus_delta) const = 0;
+ virtual bool ComputeJacobian(const double* x, double* jacobian) const = 0;
+ virtual int GlobalSize() const = 0;
+ virtual int LocalSize() const = 0;
+};
+\end{minted}
+
+\texttt{GlobalSize} is the dimension of the ambient space in which the parameter block $x$ lives. \texttt{LocalSize} is the size of the tangent space that $\Delta x$ lives in. \texttt{Plus} implements $\boxplus(x,\Delta x)$ and $\texttt{ComputeJacobian}$ computes the Jacobian matrix
+\begin{equation}
+ J = \left . \frac{\partial }{\partial \Delta x} \boxplus(x,\Delta x)\right|_{\Delta x = 0}
+\end{equation}
+in row major form.
+
+A trivial version of $\boxplus$ is when delta is of the same size as $x$
+and
+
+\begin{equation}
+ \boxplus(x, \Delta x) = x + \Delta x
+\end{equation}
+
+A more interesting case if $x$ is a two dimensional vector, and the
+user wishes to hold the first coordinate constant. Then, $\Delta x$ is a
+scalar and $\boxplus$ is defined as
+
+\begin{equation}
+ \boxplus(x, \Delta x) = x + \left[ \begin{array}{c} 0 \\ 1
+ \end{array} \right] \Delta x
+\end{equation}
+
+\texttt{SubsetParameterization} generalizes this construction to hold any part of a parameter block constant.
+
+
+Another example that occurs commonly in Structure from Motion problems
+is when camera rotations are parameterized using a quaternion. There,
+it is useful only to make updates orthogonal to that 4-vector defining
+the quaternion. One way to do this is to let $\Delta x$ be a 3
+dimensional vector and define $\boxplus$ to be
+
+\begin{equation}
+ \boxplus(x, \Delta x) =
+\left[
+\cos(|\Delta x|), \frac{\sin\left(|\Delta x|\right)}{|\Delta x|} \Delta x
+\right] * x
+\label{eq:quaternion}
+\end{equation}
+The multiplication between the two 4-vectors on the right hand
+side is the standard quaternion product. \texttt{QuaternionParameterization} is an implementation of~\eqref{eq:quaternion}.
+
+\section{\texttt{Problem}}
+\begin{minted}{c++}
+class Problem {
+ public:
+ struct Options {
+ Options();
+ Ownership cost_function_ownership;
+ Ownership loss_function_ownership;
+ Ownership local_parameterization_ownership;
+ };
+
+ Problem();
+ explicit Problem(const Options& options);
+ ~Problem();
+
+ ResidualBlockId AddResidualBlock(CostFunction* cost_function,
+ LossFunction* loss_function,
+ const vector<double*>& parameter_blocks);
+
+ void AddParameterBlock(double* values, int size);
+ void AddParameterBlock(double* values,
+ int size,
+ LocalParameterization* local_parameterization);
+
+ void SetParameterBlockConstant(double* values);
+ void SetParameterBlockVariable(double* values);
+ void SetParameterization(double* values,
+ LocalParameterization* local_parameterization);
+
+ int NumParameterBlocks() const;
+ int NumParameters() const;
+ int NumResidualBlocks() const;
+ int NumResiduals() const;
+};
+\end{minted}
+
+The \texttt{Problem} objects holds the robustified non-linear least squares problem~\eqref{eq:ceresproblem}. To create a least squares problem, use the \texttt{Problem::AddResidualBlock} and \texttt{Problem::AddParameterBlock} methods.
+
+For example a problem containing 3 parameter blocks of sizes 3, 4 and 5
+respectively and two residual blocks of size 2 and 6:
+
+\begin{minted}{c++}
+double x1[] = { 1.0, 2.0, 3.0 };
+double x2[] = { 1.0, 2.0, 3.0, 5.0 };
+double x3[] = { 1.0, 2.0, 3.0, 6.0, 7.0 };
+
+Problem problem;
+problem.AddResidualBlock(new MyUnaryCostFunction(...), x1);
+problem.AddResidualBlock(new MyBinaryCostFunction(...), x2, x3);
+\end{minted}
+
+
+\texttt{AddResidualBlock} as the name implies, adds a residual block to the problem. It adds a cost function, an optional loss function, and connects the cost function to a set of parameter blocks.
+
+The cost
+ function carries with it information about the sizes of the
+ parameter blocks it expects. The function checks that these match
+ the sizes of the parameter blocks listed in \texttt{parameter\_blocks}. The
+ program aborts if a mismatch is detected. \texttt{loss\_function} can be
+ \texttt{NULL}, in which case the cost of the term is just the squared norm
+ of the residuals.
+
+ The user has the option of explicitly adding the parameter blocks
+ using \texttt{AddParameterBlock}. This causes additional correctness
+ checking; however, \texttt{AddResidualBlock} implicitly adds the parameter
+ blocks if they are not present, so calling \texttt{AddParameterBlock}
+ explicitly is not required.
+
+
+ \texttt{Problem} by default takes ownership of the
+ \texttt{cost\_function} and \texttt{loss\_function pointers}. These objects remain
+ live for the life of the \texttt{Problem} object. If the user wishes to
+ keep control over the destruction of these objects, then they can
+ do this by setting the corresponding enums in the \texttt{Options} struct.
+
+
+ Note that even though the Problem takes ownership of \texttt{cost\_function}
+ and \texttt{loss\_function}, it does not preclude the user from re-using
+ them in another residual block. The destructor takes care to call
+ delete on each \texttt{cost\_function} or \texttt{loss\_function} pointer only once,
+ regardless of how many residual blocks refer to them.
+
+\texttt{AddParameterBlock} explicitly adds a parameter block to the \texttt{Problem}. Optionally it allows the user to associate a LocalParameterization object with the parameter block too. Repeated calls with the same arguments are ignored. Repeated
+calls with the same double pointer but a different size results in undefined behaviour.
+
+You can set any parameter block to be constant using
+
+\texttt{Problem::SetParameterBlockConstant}
+
+and undo this using
+
+\texttt{Problem::SetParameterBlockVariable}.
+
+In fact you can set any number of parameter blocks to be constant, and Ceres is smart enough to figure out what part of the problem you have constructed depends on the parameter blocks that are free to change and only spends time solving it. So for example if you constructed a problem with a million parameter blocks and 2 million residual blocks, but then set all but one parameter blocks to be constant and say only 10 residual blocks depend on this one non-constant parameter block. Then the computational effort Ceres spends in solving this problem will be the same if you had defined a problem with one parameter block and 10 residual blocks.
+
+ \texttt{Problem} by default takes ownership of the
+ \texttt{cost\_function}, \texttt{loss\_function} and \\ \texttt{local\_parameterization} pointers. These objects remain
+ live for the life of the \texttt{Problem} object. If the user wishes to
+ keep control over the destruction of these objects, then they can
+ do this by setting the corresponding enums in the \texttt{Options} struct. Even though \texttt{Problem} takes ownership of these pointers, it does not preclude the user from re-using them in another residual or parameter block. The destructor takes care to call
+ delete on each pointer only once.
+
+\section{\texttt{Solver::Options}}
+\texttt{Solver::Options} controls the overall behavior of the solver. We list the various settings and their default values below.
+
+\begin{enumerate}
+\item{\texttt{minimizer\_type}}(\texttt{LEVENBERG\_MARQUARDT}) The minimization algorithm used by Ceres. \texttt{LEVENBERG\_MARQUARDT} is currently the only valid value.
+
+\item{\texttt{max\_num\_iterations}}(\texttt{50}) Maximum number of iterations for Levenberg-Marquardt.
+
+\item{\texttt{max\_solver\_time\_sec}}(\texttt{1e9}) Maximum amount of time (in seconds) for which the solver should run.
+
+\item{\texttt{num\_threads}}(\texttt{1})
+Number of threads used by Ceres to evaluate the Jacobian.
+
+\item{\texttt{tau}}(\texttt{1e-4}) Initial value of the regularization parameter $\mu$ used by the Levenberg-Marquardt algorithm. The size of this parameter indicate the user's guess of how far the initial solution is from the minimum. Large values indicates that the solution is far away.
+
+\item{\texttt{min\_relative\_decrease}}(\texttt{1e-3}) Lower threshold for relative decrease before a Levenberg-Marquardt step is acceped.
+
+
+\item{\texttt{function\_tolerance}}(\texttt{1e-6}) Solver terminates if
+\begin{align}
+\frac{|\Delta \text{cost}|}{\text{cost}} < \texttt{function\_tolerance}
+\end{align}
+where, $\Delta \text{cost}$ is the change in objective function value (up or down) in the current iteration of Levenberg-Marquardt.
+
+\item \texttt{Solver::Options::gradient\_tolerance} Solver terminates if
+\begin{equation}
+ \frac{\|g(x)\|_\infty}{\|g(x_0)\|_\infty} < \texttt{gradient\_tolerance}
+\end{equation}
+where $\|\cdot\|_\infty$ refers to the max norm, and $x_0$ is the vector of initial parameter values.
+
+\item{\texttt{parameter\_tolerance}}(\texttt{1e-8}) Solver terminates if
+\begin{equation}
+ \frac{\|\Delta x\|}{\|x\| + \texttt{parameter\_tolerance}} < \texttt{parameter\_tolerance}
+\end{equation}
+where $\Delta x$ is the step computed by the linear solver in the current iteration of Levenberg-Marquardt.
+
+\item{\texttt{linear\_solver\_type}}(\texttt{SPARSE\_NORMAL\_CHOLESKY}/\texttt{DENSE\_QR}) Type of linear solver used to compute the solution to the linear least squares problem in each iteration of the Levenberg-Marquardt algorithm. If Ceres is build with \suitesparse linked in then the default is \texttt{SPARSE\_NORMAL\_CHOLESKY}, it is \texttt{DENSE\_QR} otherwise.
+
+\item{\texttt{preconditioner\_type}}(\texttt{JACOBI}) The preconditioner used by the iterative linear solver. The default is the block Jacobi preconditioner. Valid values are (in increasing order of complexity) \texttt{IDENTITY},\texttt{JACOBI}, \texttt{SCHUR\_JACOBI}, \texttt{CLUSTER\_JACOBI} and \texttt{CLUSTER\_TRIDIAGONAL}.
+
+\item{\texttt{num\_linear\_solver\_threads}}(\texttt{1}) Number of threads used by the linear solver.
+
+\item{\texttt{num\_eliminate\_blocks}}(\texttt{0})
+For Schur reduction based methods, the first 0 to num blocks are
+ eliminated using the Schur reduction. For example, when solving
+ traditional structure from motion problems where the parameters are in
+ two classes (cameras and points) then \texttt{num\_eliminate\_blocks} would be the
+ number of points.
+
+\item{\texttt{ordering\_type}}(\texttt{NATURAL})
+ Internally Ceres reorders the parameter blocks to help the
+ various linear solvers. This parameter allows the user to
+ influence the re-ordering strategy used. For structure from
+ motion problems use \texttt{SCHUR}, for other problems \texttt{NATURAL} (default)
+ is a good choice. In case you wish to specify your own ordering
+ scheme, for example in conjunction with \texttt{num\_eliminate\_blocks},
+ use \texttt{USER}.
+
+\item{\texttt{ordering}} The ordering of the parameter blocks. The solver pays attention
+ to it if the \texttt{ordering\_type} is set to \texttt{USER} and the ordering vector is
+ non-empty.
+
+
+\item{\texttt{linear\_solver\_min\_num\_iterations}}(\texttt{1}) Minimum number of iterations used by the linear solver. This only makes sense when the linear solver is an iterative solver, e.g., \texttt{ITERATIVE\_SCHUR}.
+
+\item{\texttt{linear\_solver\_max\_num\_iterations}}(\texttt{500}) Minimum number of iterations used by the linear solver. This only makes sense when the linear solver is an iterative solver, e.g., \texttt{ITERATIVE\_SCHUR}.
+
+\item{\texttt{eta}}(\texttt{1e-1})
+ Forcing sequence parameter. The truncated Newton solver uses
+ this number to control the relative accuracy with which the
+ Newton step is computed. This constant is passed to ConjugateGradientsSolver which uses
+ it to terminate the iterations when
+\begin{equation}
+ \frac{Q_i - Q_{i-1}}{Q_i} < \frac{\eta}{i}
+\end{equation}
+
+\item{\texttt{jacobi\_scaling}}(\texttt{true}) \texttt{true} means that the Jacobian is scaled by the norm of its columns before being passed to the linear solver. This improves the numerical conditioning of the normal equations.
+
+\item{\texttt{logging\_type}}(\texttt{PER\_MINIMIZER\_ITERATION})
+
+
+\item{\texttt{minimizer\_progress\_to\_stdout}}(\texttt{false})
+By default the Minimizer progress is logged to \texttt{STDERR} depending on the \texttt{vlog} level. If this flag is
+set to true, and \texttt{logging\_type} is not \texttt{SILENT}, the logging output
+is sent to \texttt{STDOUT}.
+
+\item{\texttt{return\_initial\_residuals}}(\texttt{false})
+\item{\texttt{return\_final\_residuals}}(\texttt{false})
+
+
+\item{\texttt{lsqp\_iterations\_to\_dump}}
+ List of iterations at which the optimizer should dump the
+ linear least squares problem to disk. Useful for testing and
+ benchmarking. If empty (default), no problems are dumped.
+Note that this requires a version of Ceres built with protocol buffers.
+
+\item{\texttt{lsqp\_dump\_format}}(\texttt{"lm\_iteration\_\%03d.lsqp"})
+ Format string for the file name used for dumping the least
+ squares problem to disk. If the format is \texttt{ascii}, then the
+ problem is logged to the screen; don't try this with large
+ problems or expect a frozen terminal.
+
+\item{\texttt{crash\_and\_dump\_lsqp\_on\_failure}}(\texttt{false})
+ Dump the linear least squares problem to disk if the minimizer
+ fails due to \texttt{NUMERICAL\_FAILURE} and crash the process. This flag
+ is useful for generating debugging information. The problem is
+ dumped in a file whose name is determined by
+ \texttt{lsqp\_dump\_format}. Note that this requires a version of Ceres built with protocol buffers.
+
+\item{\texttt{check\_gradients}}(\texttt{false})
+ Check all Jacobians computed by each residual block with finite
+ differences. This is expensive since it involves computing the
+ derivative by normal means (e.g. user specified, autodiff,
+ etc), then also computing it using finite differences. The
+ results are compared, and if they differ substantially, details
+ are printed to the log.
+
+\item{\texttt{gradient\_check\_relative\_precision}}(\texttt{1e-8})
+ Relative precision to check for in the gradient checker. If the
+ relative difference between an element in a Jacobian exceeds
+ this number, then the Jacobian for that cost term is dumped.
+
+\item{\texttt{numeric\_derivative\_relative\_step\_size}}(\texttt{1e-6})
+ Relative shift used for taking numeric derivatives. For finite
+ differencing, each dimension is evaluated at slightly shifted
+ values, \eg for forward differences, the numerical derivative is
+
+\begin{align}
+ \delta &= \texttt{numeric\_derivative\_relative\_step\_size}\\
+ \Delta f &= \frac{f((1 + \delta) x) - f(x)}{\delta x}
+\end{align}
+
+
+ The finite differencing is done along each dimension. The
+ reason to use a relative (rather than absolute) step size is
+ that this way, numeric differentation works for functions where
+ the arguments are typically large (e.g. 1e9) and when the
+ values are small (e.g. 1e-5). It is possible to construct
+ "torture cases" which break this finite difference heuristic,
+ but they do not come up often in practice.
+
+\item{\texttt{callbacks}}
+ Callbacks that are executed at the end of each iteration of the
+ \texttt{Minimizer}. They are executed in the order that they are
+ specified in this vector. By default, parameter blocks are
+ updated only at the end of the optimization, i.e when the
+ \texttt{Minimizer} terminates. This behavior is controlled by
+ \texttt{update\_state\_every\_variable}. If the user wishes to have access
+ to the update parameter blocks when his/her callbacks are
+ executed, then set \texttt{update\_state\_every\_iteration} to true.
+
+ The solver does NOT take ownership of these pointers.
+
+\item{\texttt{update\_state\_every\_iteration}}(\texttt{false})
+Normally the parameter blocks are only updated when the solver terminates. Setting this to true update them in every iteration. This setting is useful when building an interactive application using Ceres and using an \texttt{IterationCallback}.
+\end{enumerate}
+
+\section{\texttt{Solver::Summary}}
+TBD
\ No newline at end of file
