Changeset 4531 for trunk/doc/pslib/psLibADD.tex

Timestamp:

Jul 11, 2005, 9:36:18 AM (21 years ago)

Author:

eugene

Message:

clarified LMM description

File:

• trunk/doc/pslib/psLibADD.tex (modified) (9 diffs)

trunk/doc/pslib/psLibADD.tex

@@ -1 @@
-%%% $Id: psLibADD.tex,v 1.83 2005-07-11 19:17:38 eugene Exp $
+%%% $Id: psLibADD.tex,v 1.84 2005-07-11 19:36:18 eugene Exp $
 \documentclass[panstarrs]{panstarrs}
 
@@ -455 +455 @@
 
 Non-linear minimization techniques use an iterative approach to find a
-minimization since an analytical inversion is impractical or not
+minimization when an analytical inversion is impractical or not
 possible.  These techniques use a starting guess for the parameters of
 interest, and make a sequence of new guess parameters based on the
@@ -463 +463 @@
 must be modified and another attempt is made.  Convergence may be
 determined based on the absolute amount of change in the function
-value, or by comparison with the expectation for a linear system. 
+value, or by comparison with the expectation for a linear system.
 
 The two common techniques used to construct a guess parameter set are
-the 'steepest descent method' and the 'gauss newton method'.  In the
+the 'steepest descent method' and the 'Gauss-Newton method'.  In the
 first case, the guess is selected some distance along the local
 gradient.  In the second case, a local Taylor expansion of the
 function is used to construct a linear model for the function, and the
-new guess is chosen to minimize that model.  The methods discussed
-below make use of combinations of these two methods.  Aside from the
-differences in their guess steps, the two methods differ in using
-first derivatives of the function supplied by the user, or by locally
-calculating the first derivatives.  
+new guess is chosen to minimize that linear model.  The methods
+discussed below make use of combinations of these two methods.  Aside
+from the differences in their guess steps, the two methods differ in
+using first derivatives of the function supplied by the user, or by
+locally calculating the first derivatives.
 
 Mathematically, we would like to choose the parameter set $a_m$ to
@@ -485 +485 @@
 $\delta_m = -g_m$.  The Gauss-Newton method uses a Taylor expansion of
 the function to solve for the step: $(\nabla^2 F) \bar{\delta} =
--\nabla F$.  Defining $H_{m,n}$ as a component of the second
-derivatives, we can write the Gauss-Newton step as $\delta_m = -
-H_{m,n}^{-1} g_n$.
+-\nabla F$.  Defining an element of the Hessian matrix $H_{m,n}$ as a
+component of the second derivatives, we can write the Gauss-Newton
+step as $\delta_m = - H_{m,n}^{-1} g_n$.
 
 \subsubsection{Levenberg-Marquardt Method}
@@ -495 +495 @@
 interest, vary the parameters by a particular choice based on the
 gradient, evaluate the function again, and adjust the parameters and
-the parameter varient based on the results.  The LMM only works if the
-second derivative of the function can be considered negligible, as in
-the case of minimizing $\chi^2$.
-
-Consider the chi-square function, given some ordinates, $x_i$, we
+the parameter varient based on the results.  The LMM requires the
+second derivative of the function to be negligible, as in the case of
+minimizing $\chi^2$.
+
+Consider the chi-square function. Given some ordinates, $x_i$, we
 would like to find the parameters, $a_m$, of the function which
 minimize $\chi^2$ for some measurements, $y_i$ and associated errors,
 $\sigma_i$:
 \begin{eqnarray}
-\chi^2 (\bar{a})  & = & \sum_i \frac{1}{\sigma_i^2} \left( y_i - p(x_i;a_m) \right)^2  \\
+\chi^2 (\bar{a})  & = & \sum_i \frac{1}{\sigma_i^2} \left( y_i - p(x_i;a_m) \right)^2 
 \end{eqnarray}
 
@@ -511 +511 @@
 p_i (a_m)         & = & p(x_i;a_m) \\
 f_i (a_m)         & = & \frac{1}{\sigma_i} (y_i - p_i) \\
-\chi^2 (\bar{a})  & = & \sum_i f_i^2 \\
+\chi^2 (\bar{a})  & = & \sum_i f_i^2 
 \end{eqnarray}
 
 We write the minimization function $F(a_m) = \frac{1}{2} \chi^2$ to
-avoid various extra factors of 2.  We can now write out the needed
+avoid the various extra factors of 2.  We can now write out the needed
 derivatives in terms of $f_i$:
 \begin{eqnarray}
-F(a_m) = \frac{1}{2} \sum_i f_i^2 \\
-\nabla F(a_m) = \sum_i f_i \frac{\partial f_i}{\partial a_m} \\
-\nabla^2 F(a_m) = \sum_i \frac{\partial f_i}{\partial a_m} \frac{\partial f_i}{\partial a_n} \\
+F(a_m)            & = & \frac{1}{2} \sum_i f_i^2 \\
+\nabla F(a_m)     & = & \sum_i f_i \frac{\partial f_i}{\partial a_m} \\
+\nabla^2 F(a_m)   & = & \sum_i \frac{\partial f_i}{\partial a_m} \frac{\partial f_i}{\partial a_n}
 \end{eqnarray}
 
@@ -528 +528 @@
 a_m}$, we can write these in terms of the derivatives of $p_i$ only:
 \begin{eqnarray}
-\nabla F(a_m)   = -\sum_i \frac{f_i}{\sigma_i} \frac{\partial p_i}{\partial a_m} \\
-\nabla^2 F(a_m) = \sum_i \frac{1}{\sigma_i^2} \frac{\partial p_i}{\partial a_m} \frac{\partial p_i}{\partial a_n} \\
+\nabla F(a_m)     & = & -\sum_i \frac{f_i}{\sigma_i} \frac{\partial p_i}{\partial a_m} \\
+\nabla^2 F(a_m)   & = &  \sum_i \frac{1}{\sigma_i^2} \frac{\partial p_i}{\partial a_m} \frac{\partial p_i}{\partial a_n}
 \end{eqnarray}
 
 Writing these in matrix representation, and replacing $f_i$, we have:
 \begin{eqnarray}
--g_m    = \sum_i \frac{(y_i - p_i)}{\sigma_i^2} \frac{\partial p_i}{\partial a_m} \\
-H_{m,n} = \sum_i \frac{1}{\sigma_i^2} \frac{\partial p_i}{\partial a_m} \frac{\partial p_i}{\partial a_n} \\
+-g_m              & = & \sum_i \frac{(y_i - p_i)}{\sigma_i^2} \frac{\partial p_i}{\partial a_m} \\
+H_{m,n}           & = & \sum_i \frac{1}{\sigma_i^2} \frac{\partial p_i}{\partial a_m} \frac{\partial p_i}{\partial a_n}
 \end{eqnarray}
 
@@ -544 +544 @@
 
 \begin{eqnarray}
-A_{m,n} & = & H_{m,n} if (j \ne k) \\
-A_{m,n} & = & H_{m,n}(1 + \lambda) if (j = k)
+A_{m,n} & = & H_{m,n} \mbox{if} (j \ne k) \\
+A_{m,n} & = & H_{m,n}(1 + \lambda) \mbox{if} (j = k)
 \end{eqnarray}
 %
@@ -553 +553 @@
 \end{equation}
 %
-where $alpha^\prime_n = \alpha_n + \delta_n$ represents our new
-attempt at a parameter guess. We use this parameter set to evaluate
-the function.  
-
-To evaluate the new parameter set, we compare the change in $\chi^2$
-with the change expected from the linear model (the Taylor expansion).
-If the linear model were correct, we would have expected a change (a
-reduction) in $\chi^2$ of $\Delta = \frac{\lambda}{2}\sum\delta_m^2 +
-\frac{1}{2}\sum\delta_m g_m$.  We use the 'gain ratio' $\rho =
-\frac{\chi^2_{\rm old} - \chi^2_{\rm new}}{\Delta}$ to judge the new
-step. If $\rho > 0$, we accept this new set of parameters and decrease
-$\lambda$ by a factor of 10, otherwise we keep the old set, and
-increase the value of $\lambda$ by a factor of 10.  We repeat this
-process until the value of the function changes by much less than the
-tolerance.  The resulting values of $a_m$ are the best-fit parameters
-for the system.
+The new parameter guess is then found from this value with
+$\alpha^\prime_n = \alpha_n + \delta_n$. We use this parameter set to
+evaluate the function.
+
+To assess the quality of the new parameter set, we compare the change
+in $\chi^2$ with the change expected from the linear model (the Taylor
+expansion).  If the linear model were correct, we would have expected
+a change (a reduction) in $\chi^2$ of $\Delta =
+\frac{\lambda}{2}\sum\delta_m^2 + \frac{1}{2}\sum\delta_m g_m$.  We
+use the 'gain ratio' $\rho = \frac{\chi^2_{\rm old} - \chi^2_{\rm
+new}}{\Delta}$ to judge the new step. If $\rho > 0$, we accept this
+new set of parameters and decrease $\lambda$ by a factor of 10,
+otherwise we keep the old set, and increase the value of $\lambda$ by
+a factor of 10.  We repeat this process until the value of the
+function changes by much less than the tolerance.  The resulting
+values of $a_m$ are the best-fit parameters for the system.
 
 The covariance matrix, $C_{i,j}$, which is the inverse of the matrix
-$H_{m,n}$ allows simple calculation of the confidence limits of the
+$H_{m,n}$ provides an estimate of the confidence limits of the
 parameters.