tree 5cf1f2ebdaaf6d778ad6ed4792529467e104d9d7
parent 9102bdd831027f38e390d44c7cf95a3c97af0249
author Sameer Agarwal <sameeragarwal@google.com> 1393200633 -0800
committer Sameer Agarwal <sameeragarwal@google.com> 1393268086 -0800

Changes to how gradient based convergence is diagnosed.

The original implementation for computing the norm of the gradient was

gradient_norm = norm(gradient)

when the gradient vector lies in the same space as the parameter
vector, this value is meaningful. When there is a local parameterization
involved, interpreting this value and diagnosing convergence using it
is hard.

Further, this expression does not respect the bounds constraints
on the parmeters. Measuring the norm of the gradient only makes
sense when the optimization being performed is unconstrained.

A better solution, used by LANCELOT is the expression
gradient_norm = norm(x - P(x - gradient))

Here, P is the projection operator onto the bounds constraints.
x - gradient is computed by computing Plus(x, -gradient), thus the
actual expression becomes

gradient_norm = norm(x - P(Plus(x, -gradient)));

Which in the case where there are no bounds constraints, and there
are no local parameterizations, reduces to the usual Euclidean
expression from above, since

Plus(x, -gradient) = x - gradient

and P(x - gradient) = x - gradient.

This change implements this change. Further, the convergence
test using the gradient tolerance now uses an absolute measure
rather than a relative measure. This is a forward looking change
as we start implementing the Augmented Lagrangian solver.

Last but not the least, various "Terminating: Foo" messages
have been changed so that "Terminating: " is logged but is
not part of the Solver::Summary::message string as it is
pointless.

Change-Id: I943146f71a1da47c8c7592986039b4112781b99b