id	summary	reporter	owner	description	type	status	priority	milestone	component	version	severity	resolution	keywords	cc
1071	ppStats on compressed images yield different results than uncompressed	bills	Paul Price	"The following are some of the statistical measures for the residuals
for two different flat detrend runs with the same input exposures.

For run 20 the ppMerge output images were compressed, for run 20
they were not.

Note that the bg and bg_stdev values are somewhat different, and the
bin_stdev values are signficantly different.

Another experiement where the same ppMerge output was used and
only the residual calculations (which include the binning process)
were done on compresed images had similar results.

Perhaps this has something to do with how nan pixels are handled in
the compression process.

Note that the statistics are different. In the case of bin_stdev the values 
are very  different.
In the one case I looked at in detail, the cause of the high bin_stdev was a
huge outlier value from one chip.

mysql> select det_id, exp_id, bg, bg_stdev, bin_stdev from detResidImfile where det_id = 21 or det_id = 20 order by exp_id;
+--------+--------+------------------+------------------+------------------+
| det_id | exp_id | bg               | bg_stdev         | bin_stdev        |
+--------+--------+------------------+------------------+------------------+
|     20 |  50714 | 852.253815277778 | 20.2682251571526 | 253.213919128524 |
|     21 |  50714 | 852.315426388889 |  20.305931572922 | 2.27909648890663 |
|     21 |  50715 | 11407.4040277778 | 93.6562971802432 | 26.2769122627497 |
|     20 |  50715 | 11407.1608333333 | 93.7857139494351 |  3016.2631486739 |
|     21 |  50716 | 12253.7090277778 |   99.28685315589 | 29.1512105290881 |
|     20 |  50716 | 12253.4465277778 | 99.1299406999631 |  3045.1881543219 |
|     20 |  50717 |      12878.60625 | 103.696981163564 | 2525.55742331584 |
|     21 |  50717 | 12878.8972222222 | 103.714281568562 | 33.5483923541618 |
|     20 |  50718 |         11540.21 | 97.4627620960095 | 3940.88768852193 |
|     21 |  50718 | 11540.5791666667 | 97.3087913068767 | 24.9529588441886 |
|     20 |  50719 | 9865.94091666666 | 89.7526273109547 | 5647.43080833958 |
|     21 |  50719 | 9866.03281944444 | 89.3763152948967 | 46.0943350782944 |
|     20 |  50720 | 7346.96540277778 | 73.1578091868154 | 6054.02308449873 |
|     21 |  50720 | 7347.31634722222 | 72.5178396846033 | 31.5039281117779 |
+--------+--------+------------------+------------------+------------------+
14 rows in set (0.00 sec)"	defect	assigned	high		ppStats	2.5	normal