Changes to how gradient based convergence is diagnosed.
The original implementation for computing the norm of the gradient was
gradient_norm = norm(gradient)
when the gradient vector lies in the same space as the parameter
vector, this value is meaningful. When there is a local parameterization
involved, interpreting this value and diagnosing convergence using it
is hard.
Further, this expression does not respect the bounds constraints
on the parmeters. Measuring the norm of the gradient only makes
sense when the optimization being performed is unconstrained.
A better solution, used by LANCELOT is the expression
gradient_norm = norm(x - P(x - gradient))
Here, P is the projection operator onto the bounds constraints.
x - gradient is computed by computing Plus(x, -gradient), thus the
actual expression becomes
gradient_norm = norm(x - P(Plus(x, -gradient)));
Which in the case where there are no bounds constraints, and there
are no local parameterizations, reduces to the usual Euclidean
expression from above, since
Plus(x, -gradient) = x - gradient
and P(x - gradient) = x - gradient.
This change implements this change. Further, the convergence
test using the gradient tolerance now uses an absolute measure
rather than a relative measure. This is a forward looking change
as we start implementing the Augmented Lagrangian solver.
Last but not the least, various "Terminating: Foo" messages
have been changed so that "Terminating: " is logged but is
not part of the Solver::Summary::message string as it is
pointless.
Change-Id: I943146f71a1da47c8c7592986039b4112781b99b
3 files changed
