minor corrections to derivatives.rst Change-Id: I6bb5e3c507113c8873c4c516e9cae3239cca247c
diff --git a/docs/source/derivatives.rst b/docs/source/derivatives.rst index 3919f86..3838d43 100644 --- a/docs/source/derivatives.rst +++ b/docs/source/derivatives.rst
@@ -651,8 +651,8 @@ Forward Differences and the remarkable accuracy improvements of Ridders' method cost an order of magnitude more runtime. -Recommendation --------------- +Recommendations +--------------- Numeric differentiation should be used when you cannot compute the derivatives either analytically or using automatic differention. This @@ -929,6 +929,7 @@ Indeed, this is essentially how :class:`AutoDiffCostFunction` works. + Pitfalls -------- @@ -992,14 +993,12 @@ TODO ==== -#. Inverse function theorem -#. Add references in the various sections about the things to - do. NIST, RIDDER's METHOD, Numerical Recipes. -#. Calling iterative routines. +#. Why does the quality of derivatives matter? #. Discuss, forward v/s backward automatic differentiation and relation to backprop, impact of large parameter block sizes on differentiation performance. -#. Why does the quality of derivatives matter? +#. Inverse function theorem +#. Calling iterative routines. #. Reference to how numeric derivatives lead to slower convergence. #. Pitfalls of Numeric differentiation. #. Ill conditioning of numeric differentiation/dependence on curvature.