id	summary	reporter	owner	description	type	status	priority	milestone	component	version	severity	resolution	keywords	cc
366	psStats robust stats not well designed	eugene	eugene	"I've been looking at how we have spec-ed psStats robust statistics functions,
and I am not happy with some of the parameters.  the resulting statistics are
somewhat sensitive to the choices of the bin size and smoothing scales (which
are also somewhat coupled) as well as the number of bins used in the fit (what
fraction of LQ to UQ is kept).  I will explore this problem and consult with
people here to fix this algorithm.

Meanwhile, two improvements can be made:

- the clipped statistic is used to make a quick guess of MEAN & SIGMA.  if the
dataset is large, this iteration can take some substantial amount of time.  If
the number of points in the vector is larger than some number (let's set it at
50000, but use a #define), a random fraction of the pixels should be dropped to
keep the total number at about 50000 (this is only for the vector for which
clipped stats are measured).

- the fit of a gaussian to the points of interest in the peak should not use the
LM non-linear fitting.  it is better to take the natural log of the ordinate and
fit a 2nd order polynomial (much faster).  The resulting terms relate directly
to the gaussian parameters:

  $SIGMA = sqrt (-0.5/$C2)
  $MEAN  = $C1 * $SIGMA^2
  $NORM  = exp ($C0 + $MEAN^2 / (2*$SIGMA^2))

where C0, C1, C2 are the coefficients of the 0, 1, 2 order terms.  the gaussian
which is fitted has the form:

f(x) = $NORM * exp(-0.5*(x - $MEAN)^2 / $SIGMA^2)"	defect	closed	high		types	0.5.0	normal	fixed