February 8, 2013
The issue with the cell-to-cell continuity is likely a result of the interaction between PATTERN.MEDIAN and the non-linearity correction. Disabling NONLIN allows for the 3-phase OTAs (which have fewer nonlinearity issues) to retain continuity. I am not fully clear on how this interaction is working, as NONLIN is applied with the other detrend steps (DARK/FLAT/etc) before the PATTERN functions are called. I would have not have expected that this would cause discontinuities to remain.
| SAS footprint test52 |
| Current trunk + PATTERN.MEDIAN - NONLIN |
|
February 7, 2013
Conclusions based on the initial ppImage implementation
- Cosmetic evaluation. OTA00 is in lower right corner. Click to zoom in to larger image.
- Pro:
- Butterfly effect around stars is removed.
- OTAs which were not being corrected with PATTERN.ROW (ie OTA45, OTA55) see improvement.
- Definite improvement in some "bad OTAs" (OTA67 in particular).
- Corner glows are mitigated (OTA12)
- Cross talk effect in OTA61 improved.
- Con:
- Print through of bright sources into median correction (ghost in OTA11, star in OTA24, all of OTA06, etc).
- Incomplete removal of row-to-row variations (OTA15, OTA27, OTA45, etc).
- Cell-to-cell continuity no longer maintained for unknown reason (OTA11, many others).
| SAS footprint test48. | SAS footprint test51 |
| Effectively current trunk | Current trunk + PATTERN.MEDIAN |
 | 
|
- Individual row-by-row solutions are not stable between exposures. The following image shows a series of median solution images for OTA56/row 0 for the SAS footprint and for a series of similar science exposures taken after that date (highlighted frame is the first non-SAS footprint exposure). There are clear differences between the row-to-row patterns even within the short time frame of the SAS footprint.
- Applying only the PATTERN.MEDIAN to exposures (without DARK, FLAT, or PATTERN.CONTINUITY) yields a clear x-pixel direction gradient curve. This is effectively a result of solving for the DARK contribution on a per exposure basis. Applying DARK + FLAT removes this gradient on OTAs that are well behaved. This illustrates that the DARK model is not a significant problem for these OTAs.
| Science images, PATTERN.MEDIAN only, OTA56/row 0 | 
|
| Dark images, PATTERN.MEDIAN only, OTA56/row 0 | 
|
| Science images, PATTERN.MEDIAN/DARK/FLAT, OTA56/row 0 | 
|
- Applying only PATTERN.MEDIAN to a series of bias frames helps explain why the cell-column structures are not well removed by this correction. The larger scale ripples have a period that is not constrained to a cell. For example, in the highlighted bias (OTA31, PATTERN.MEDIAN only) below, I count ~11 ripples in the 4th (from left) column of cells. Whatever is driving these ripples is independent of the cell structure, and as such, these ripples do not reinforce in the PATTERN.MEDIAN calculation.
February 5, 2013
I have an initial implementation in ppImage/psModules of this correction.
| Cell | Mode | Science | OTA20 | OTA27 | OTA67 | - | Dark | OTA20 | OTA27 | OTA67 | |||||||
| Pre | Post | Pre | Post | Pre | Post | Pre | Post | Pre | Post | Pre | Post | ||||||
| xy00 | 0 | - | 19.1360 | 16.2163 | 18.2377 | 12.3995 | 19.1727 | 14.1483 | - | 6.1285 | 4.4151 | 6.0682 | 3.7090 | 6.6669 | 4.6517 | - | |
| xy00 | 1 | - | 16.2163 | 17.1696 | 12.3995 | 15.3510 | 14.1483 | 9.2773 | - | 4.4151 | 3.6588 | 3.7090 | 3.2176 | 4.6517 | 3.0194 | - | |
| xy01 | 0 | - | 19.0460 | 14.9990 | 18.0620 | 13.3894 | 18.4362 | 13.6208 | - | 6.1873 | 4.3626 | 6.0671 | 4.1361 | 6.3822 | 4.3927 | - | |
| xy01 | 1 | - | 14.9990 | 15.1816 | 13.3894 | 15.1782 | 13.6208 | 9.1863 | - | 4.3626 | 3.5922 | 4.1361 | 3.4309 | 4.3927 | 2.8451 | - | |
| xy02 | 0 | - | 18.9513 | 15.8220 | 17.1856 | 12.7766 | 18.2383 | 12.7415 | - | 6.1805 | 4.5006 | 5.7957 | 3.9339 | 6.8098 | 4.2825 | - | |
| xy02 | 1 | - | 15.8220 | 13.9137 | 12.7766 | 10.9814 | 12.7415 | 8.2730 | - | 4.5006 | 3.6908 | 3.9339 | 3.2501 | 4.2825 | 2.8721 | - | |
| xy03 | 0 | - | 18.5051 | 15.0910 | 16.6074 | 12.9144 | 18.9350 | 15.1614 | - | 6.0329 | 4.1954 | 6.1399 | 4.2113 | 6.8538 | 4.3345 | - | |
| xy03 | 1 | - | 15.0910 | 11.8479 | 12.9144 | 10.4689 | 15.1614 | 9.3123 | - | 4.1954 | 3.4928 | 4.2113 | 3.5075 | 4.3345 | 2.7423 | - | |
| xy04 | 0 | - | 18.0283 | 14.2967 | 17.2791 | 13.1565 | 17.9581 | 10.2001 | - | 6.0211 | 4.2602 | 6.0691 | 4.2499 | 6.5565 | 3.5076 | - | |
| xy04 | 1 | - | 14.2967 | 11.4103 | 13.1565 | 10.6392 | 10.2001 | 10.0613 | - | 4.2602 | 3.5722 | 4.2499 | 3.4626 | 3.5076 | 2.3275 | - | |
| xy05 | 0 | - | 18.0424 | 13.6410 | 17.0563 | 12.8031 | 18.5023 | nan | - | 5.8812 | 4.0306 | 6.1444 | 4.1855 | 6.5560 | nan | - | |
| xy05 | 1 | - | 13.6410 | 12.0317 | 12.8031 | 12.7779 | 18.5023 | nan | - | 4.0306 | 3.3730 | 4.1855 | 3.4577 | 6.5560 | nan | - | |
| xy06 | 0 | - | 17.7460 | 12.4594 | 17.6375 | 15.2400 | 18.0741 | nan | - | 6.0368 | 3.9447 | 6.2297 | 4.3317 | 6.5286 | nan | - | |
| xy06 | 1 | - | 12.4594 | 14.6684 | 15.2400 | 12.3931 | 18.0741 | nan | - | 3.9447 | 3.3657 | 4.3317 | 3.5709 | 6.5286 | nan | - | |
| xy07 | 0 | - | 17.4767 | 11.9081 | 17.0315 | 12.6593 | 0.0000 | nan | - | 5.7820 | 3.7218 | 6.1509 | 4.1284 | 0.0000 | nan | - | |
| xy07 | 1 | - | 11.9081 | 14.7872 | 12.6593 | 15.5990 | 0.0000 | nan | - | 3.7218 | 3.1714 | 4.1284 | 3.4052 | 0.0000 | nan | - | |
| xy10 | 0 | - | 18.2514 | 14.1018 | 18.3373 | 11.8626 | 19.3578 | 18.0667 | - | 5.8554 | 4.2808 | 5.8761 | 3.4812 | 7.1169 | 4.9970 | - | |
| xy10 | 1 | - | 14.1018 | 16.0846 | 11.8626 | 15.5814 | 18.0667 | 14.8097 | - | 4.2808 | 3.5589 | 3.4812 | 3.0692 | 4.9970 | 3.2849 | - | |
| xy11 | 0 | - | 18.5462 | 13.9435 | 17.9536 | 12.7593 | 18.6224 | 16.8352 | - | 6.1133 | 4.3305 | 5.8202 | 3.9862 | 7.1274 | 4.8184 | - | |
| xy11 | 1 | - | 13.9435 | 14.1660 | 12.7593 | 14.4832 | 16.8352 | 13.8805 | - | 4.3305 | 3.5792 | 3.9862 | 3.2806 | 4.8184 | 3.1085 | - | |
| xy12 | 0 | - | 18.4375 | 15.0417 | 17.5258 | 12.4371 | 19.0663 | 15.5909 | - | 6.0537 | 4.3680 | 5.7809 | 4.0517 | 7.1838 | 4.4087 | - | |
| xy12 | 1 | - | 15.0417 | 12.8425 | 12.4371 | 10.8093 | 15.5909 | 8.5315 | - | 4.3680 | 3.6009 | 4.0517 | 3.2822 | 4.4087 | 2.9466 | - | |
| xy13 | 0 | - | 18.2929 | 14.1276 | 17.3519 | 13.8896 | 17.1998 | 13.0043 | - | 5.5887 | 3.8780 | 5.8373 | 4.2403 | 7.1461 | 4.2662 | - | |
| xy13 | 1 | - | 14.1276 | 12.2511 | 13.8896 | 11.6127 | 13.0043 | 14.0887 | - | 3.8780 | 3.2592 | 4.2403 | 3.5201 | 4.2662 | 2.7240 | - | |
| xy14 | 0 | - | 17.9325 | 13.1143 | 17.8327 | 13.3869 | 17.7430 | 14.2839 | - | 5.7602 | 4.0778 | 6.1397 | 4.5782 | 7.0328 | 3.6044 | - | |
| xy14 | 1 | - | 13.1143 | 10.8539 | 13.3869 | 10.8322 | 14.2839 | 9.6809 | - | 4.0778 | 3.4093 | 4.5782 | 3.6551 | 3.6044 | 2.4486 | - | |
| xy15 | 0 | - | 17.9928 | 12.8740 | 16.7583 | 12.2736 | 17.2278 | nan | - | 5.7530 | 3.9591 | 6.0022 | 4.3926 | 7.1338 | nan | - | |
| xy15 | 1 | - | 12.8740 | 12.1617 | 12.2736 | 13.1751 | 17.2278 | nan | - | 3.9591 | 3.2744 | 4.3926 | 3.5546 | 7.1338 | nan | - | |
| xy16 | 0 | - | 18.0439 | 12.0950 | 17.2752 | 12.8557 | 18.6109 | nan | - | 5.7624 | 3.6545 | 6.2035 | 4.4541 | 6.8946 | nan | - | |
| xy16 | 1 | - | 12.0950 | 13.6971 | 12.8557 | 14.2169 | 18.6109 | nan | - | 3.6545 | 3.1643 | 4.4541 | 3.6680 | 6.8946 | nan | - | |
| xy17 | 0 | - | 17.3371 | 11.5769 | 19.4375 | 14.8047 | 0.0000 | nan | - | 5.6314 | 3.5134 | 6.4868 | 4.6869 | 0.0000 | nan | - | |
| xy17 | 1 | - | 11.5769 | 14.2065 | 14.8047 | 16.9240 | 0.0000 | nan | - | 3.5134 | 2.9966 | 4.6869 | 3.9870 | 0.0000 | nan | - | |
| xy20 | 0 | - | 19.4929 | 15.1833 | 17.2820 | 12.2914 | 18.8305 | 14.4255 | - | 5.8749 | 4.1899 | 6.2832 | 3.6105 | 6.5996 | 4.3940 | - | |
| xy20 | 1 | - | 15.1833 | 16.8889 | 12.2914 | 14.7989 | 14.4255 | 9.1376 | - | 4.1899 | 3.4093 | 3.6105 | 3.1802 | 4.3940 | 2.8409 | - | |
| xy21 | 0 | - | 18.8188 | 14.3427 | 18.2276 | 12.8475 | 18.6053 | 13.7738 | - | 6.0194 | 4.0294 | 6.3746 | 4.1167 | 7.0766 | 4.7348 | - | |
| xy21 | 1 | - | 14.3427 | 14.8981 | 12.8475 | 14.4139 | 13.7738 | 8.4585 | - | 4.0294 | 3.4695 | 4.1167 | 3.4340 | 4.7348 | 3.0959 | - | |
| xy22 | 0 | - | 18.4371 | 14.4400 | 17.7528 | 12.8034 | 18.8758 | 13.2251 | - | 6.1783 | 4.2438 | 6.2803 | 4.0339 | 6.9602 | 4.0829 | - | |
| xy22 | 1 | - | 14.4400 | 13.7328 | 12.8034 | 11.1047 | 13.2251 | 9.4503 | - | 4.2438 | 3.5118 | 4.0339 | 3.3759 | 4.0829 | 2.7076 | - | |
| xy23 | 0 | - | 18.3282 | 13.5068 | 17.9319 | 13.1245 | 18.3186 | 13.5902 | - | 6.0390 | 4.0628 | 6.3250 | 4.2480 | 6.9023 | 4.2354 | - | |
| xy23 | 1 | - | 13.5068 | 11.3889 | 13.1245 | 10.8052 | 13.5902 | 8.1498 | - | 4.0628 | 3.3967 | 4.2480 | 3.4674 | 4.2354 | 2.7378 | - | |
| xy24 | 0 | - | 18.5331 | 13.9161 | 17.4129 | 12.5379 | 16.8786 | 10.5109 | - | 5.9910 | 4.0248 | 6.2760 | 4.1768 | 6.9114 | 3.3142 | - | |
| xy24 | 1 | - | 13.9161 | 11.7017 | 12.5379 | 10.3224 | 10.5109 | 12.8027 | - | 4.0248 | 3.3709 | 4.1768 | 3.3658 | 3.3142 | 2.2545 | - | |
| xy25 | 0 | - | 18.8492 | 13.7972 | 16.9552 | 12.2764 | 16.8643 | nan | - | 5.9284 | 3.9407 | 6.4132 | 4.1226 | 6.7820 | nan | - | |
| xy25 | 1 | - | 13.7972 | 12.2562 | 12.2764 | 12.7897 | 16.8643 | nan | - | 3.9407 | 3.3190 | 4.1226 | 3.3770 | 6.7820 | nan | - | |
| xy26 | 0 | - | 17.8476 | 12.0684 | 17.2714 | 11.9061 | 0.0000 | nan | - | 5.7852 | 3.5705 | 6.4087 | 4.1523 | 0.0000 | nan | - | |
| xy26 | 1 | - | 12.0684 | 14.4886 | 11.9061 | 13.5854 | 0.0000 | nan | - | 3.5705 | 3.0873 | 4.1523 | 3.4065 | 0.0000 | nan | - | |
| xy27 | 0 | - | 17.6855 | 11.6663 | 16.6421 | 12.2780 | 0.0000 | nan | - | 5.4959 | 3.7763 | 6.4091 | 4.1174 | 0.0000 | nan | - | |
| xy27 | 1 | - | 11.6663 | 14.5230 | 12.2780 | 14.9671 | 0.0000 | nan | - | 3.7763 | 3.0864 | 4.1174 | 3.4885 | 0.0000 | nan | - | |
| xy30 | 0 | - | 18.4763 | 13.6104 | 18.3649 | 12.0178 | 16.8053 | 14.8654 | - | 5.7268 | 4.2020 | 6.2051 | 4.4402 | 7.3537 | 4.9785 | - | |
| xy30 | 1 | - | 13.6104 | 15.7799 | 12.0178 | 15.0555 | 14.8654 | 8.9357 | - | 4.2020 | 3.4855 | 4.4402 | 3.8393 | 4.9785 | 3.1229 | - | |
| xy31 | 0 | - | 19.3674 | 14.7158 | 17.9415 | 13.2576 | 19.0805 | 14.7440 | - | 6.0188 | 4.2763 | 6.3727 | 4.8845 | 7.9630 | 5.5280 | - | |
| xy31 | 1 | - | 14.7158 | 16.1925 | 13.2576 | 14.1147 | 14.7440 | 10.2556 | - | 4.2763 | 3.5746 | 4.8845 | 3.9310 | 5.5280 | 3.6413 | - | |
| xy32 | 0 | - | 18.3466 | 13.6634 | 17.8306 | 13.2970 | 18.0555 | 10.9673 | - | 6.0750 | 4.4342 | 6.5418 | 4.8183 | 7.8396 | 4.9507 | - | |
| xy32 | 1 | - | 13.6634 | 12.6847 | 13.2970 | 12.2090 | 10.9673 | 8.8627 | - | 4.4342 | 3.5512 | 4.8183 | 3.9323 | 4.9507 | 3.1904 | - | |
| xy33 | 0 | - | 18.4171 | 12.9474 | 17.5620 | 14.0882 | 18.1302 | 13.0712 | - | 6.0303 | 4.2300 | 6.4342 | 4.9112 | 7.9225 | 4.9694 | - | |
| xy33 | 1 | - | 12.9474 | 10.6324 | 14.0882 | 11.1242 | 13.0712 | 8.0515 | - | 4.2300 | 3.5418 | 4.9112 | 3.9866 | 4.9694 | 3.1990 | - | |
| xy34 | 0 | - | 17.9496 | 13.0746 | 17.3083 | 13.1250 | 18.4269 | 10.6724 | - | 6.0228 | 4.1859 | 6.2401 | 4.8411 | 8.1413 | 4.5869 | - | |
| xy34 | 1 | - | 13.0746 | 10.9876 | 13.1250 | 11.4894 | 10.6724 | 8.4659 | - | 4.1859 | 3.4852 | 4.8411 | 3.8676 | 4.5869 | 2.9100 | - | |
| xy35 | 0 | - | 18.4303 | 13.0714 | 17.4326 | 15.6595 | 0.0000 | nan | - | 5.7687 | 3.9480 | 6.2028 | 4.8671 | 0.0000 | nan | - | |
| xy35 | 1 | - | 13.0714 | 12.0987 | 15.6595 | 12.2467 | 0.0000 | nan | - | 3.9480 | 3.2420 | 4.8671 | 3.8953 | 0.0000 | nan | - | |
| xy36 | 0 | - | 17.6417 | 11.8336 | 17.1183 | 15.5740 | 0.0000 | nan | - | 5.8034 | 3.7506 | 6.3231 | 4.6842 | 0.0000 | nan | - | |
| xy36 | 1 | - | 11.8336 | 14.3535 | 15.5740 | 13.2449 | 0.0000 | nan | - | 3.7506 | 3.1686 | 4.6842 | 3.9192 | 0.0000 | nan | - | |
| xy37 | 0 | - | 17.7989 | 11.8550 | 16.8804 | 13.4954 | 0.0000 | nan | - | 5.9150 | 3.8859 | 6.3897 | 4.7409 | 0.0000 | nan | - | |
| xy37 | 1 | - | 11.8550 | 13.3849 | 13.4954 | 15.3119 | 0.0000 | nan | - | 3.8859 | 3.3115 | 4.7409 | 4.0454 | 0.0000 | nan | - | |
| xy40 | 0 | - | 18.5319 | 14.1220 | 18.5072 | 11.5330 | 18.0607 | 13.3059 | - | 6.1715 | 4.4447 | 6.1677 | 3.6401 | 6.3359 | 4.4088 | - | |
| xy40 | 1 | - | 14.1220 | 15.5400 | 11.5330 | 15.7441 | 13.3059 | 10.8456 | - | 4.4447 | 3.6159 | 3.6401 | 3.2119 | 4.4088 | 3.0171 | - | |
| xy41 | 0 | - | 18.3665 | 14.5400 | 17.6085 | 12.7655 | 17.0868 | 13.1887 | - | 6.1858 | 4.2359 | 5.7857 | 3.6231 | 6.4787 | 4.3623 | - | |
| xy41 | 1 | - | 14.5400 | 14.7101 | 12.7655 | 14.0190 | 13.1887 | 13.4015 | - | 4.2359 | 3.4932 | 3.6231 | 2.9904 | 4.3623 | 3.0141 | - | |
| xy42 | 0 | - | 18.5093 | 13.6799 | 17.4195 | 12.3168 | 17.4092 | 15.9769 | - | 6.0979 | 4.3039 | 5.8623 | 3.7256 | 6.8841 | 4.4221 | - | |
| xy42 | 1 | - | 13.6799 | 12.2633 | 12.3168 | 11.5988 | 15.9769 | 10.0928 | - | 4.3039 | 3.4998 | 3.7256 | 3.0495 | 4.4221 | 3.0557 | - | |
| xy43 | 0 | - | 18.0734 | 12.9225 | 16.9227 | 14.3226 | 18.3046 | 14.0339 | - | 6.3452 | 4.4203 | 6.0097 | 3.9800 | 6.4185 | 3.9673 | - | |
| xy43 | 1 | - | 12.9225 | 10.4091 | 14.3226 | 11.9042 | 14.0339 | 8.8871 | - | 4.4203 | 3.6298 | 3.9800 | 3.2304 | 3.9673 | 2.7529 | - | |
| xy44 | 0 | - | 18.3202 | 13.5666 | 17.4369 | 13.0589 | 17.2433 | 12.3189 | - | 6.3044 | 4.4849 | 6.0454 | 3.9135 | 6.4449 | 3.2703 | - | |
| xy44 | 1 | - | 13.5666 | 10.8389 | 13.0589 | 10.4720 | 12.3189 | 10.9676 | - | 4.4849 | 3.6020 | 3.9135 | 3.2339 | 3.2703 | 2.3119 | - | |
| xy45 | 0 | - | 17.6807 | 12.9688 | 16.9089 | 12.3285 | 0.0000 | nan | - | 5.9951 | 4.0661 | 6.1289 | 3.9398 | 0.0000 | nan | - | |
| xy45 | 1 | - | 12.9688 | 13.4718 | 12.3285 | 13.3575 | 0.0000 | nan | - | 4.0661 | 3.3636 | 3.9398 | 3.1533 | 0.0000 | nan | - | |
| xy46 | 0 | - | 17.4889 | 11.7468 | 18.1185 | 13.6169 | 0.0000 | nan | - | 5.9795 | 3.8890 | 6.2004 | 3.9215 | 0.0000 | nan | - | |
| xy46 | 1 | - | 11.7468 | 14.0915 | 13.6169 | 12.7394 | 0.0000 | nan | - | 3.8890 | 3.1813 | 3.9215 | 3.2310 | 0.0000 | nan | - | |
| xy47 | 0 | - | 17.2549 | 12.2509 | 17.0664 | 13.1579 | 0.0000 | nan | - | 6.1473 | 3.9337 | 6.5688 | 4.2108 | 0.0000 | nan | - | |
| xy47 | 1 | - | 12.2509 | 14.2660 | 13.1579 | 13.8481 | 0.0000 | nan | - | 3.9337 | 3.3917 | 4.2108 | 3.7338 | 0.0000 | nan | - | |
| xy50 | 0 | - | 19.1672 | 14.3945 | 19.1031 | 13.6655 | 17.7989 | 13.4743 | - | 5.8081 | 4.2717 | 6.1104 | 3.8799 | 6.4406 | 4.2570 | - | |
| xy50 | 1 | - | 14.3945 | 16.4193 | 13.6655 | 16.6419 | 13.4743 | 10.1684 | - | 4.2717 | 3.3874 | 3.8799 | 3.2374 | 4.2570 | 3.3138 | - | |
| xy51 | 0 | - | 17.5836 | 13.4868 | 18.6964 | 13.2541 | 17.8646 | 13.0439 | - | 5.8572 | 4.1488 | 5.7090 | 3.8340 | 6.6715 | 4.3907 | - | |
| xy51 | 1 | - | 13.4868 | 11.8247 | 13.2541 | 15.9136 | 13.0439 | 10.6652 | - | 4.1488 | 3.2842 | 3.8340 | 3.1090 | 4.3907 | 3.4019 | - | |
| xy52 | 0 | - | 18.1610 | 13.1693 | 17.5781 | 12.6718 | 17.9637 | 12.2584 | - | 5.5796 | 3.9051 | 5.4741 | 3.4871 | 6.7154 | 3.9766 | - | |
| xy52 | 1 | - | 13.1693 | 11.1376 | 12.6718 | 13.0346 | 12.2584 | 10.1146 | - | 3.9051 | 3.1355 | 3.4871 | 2.8429 | 3.9766 | 3.3090 | - | |
| xy53 | 0 | - | 17.8415 | 13.2155 | 17.0598 | 12.4405 | 17.7597 | 13.9421 | - | 5.5153 | 3.7380 | 5.3750 | 3.5125 | 6.7748 | 4.2158 | - | |
| xy53 | 1 | - | 13.2155 | 10.8081 | 12.4405 | 9.7693 | 13.9421 | 10.6826 | - | 3.7380 | 2.9665 | 3.5125 | 2.9122 | 4.2158 | 3.2870 | - | |
| xy54 | 0 | - | 17.8132 | 12.8345 | 16.4835 | 13.2793 | 0.0000 | nan | - | 5.5682 | 3.8162 | 5.3923 | 3.5193 | 0.0000 | nan | - | |
| xy54 | 1 | - | 12.8345 | 11.0486 | 13.2793 | 11.0388 | 0.0000 | nan | - | 3.8162 | 3.0777 | 3.5193 | 2.8222 | 0.0000 | nan | - | |
| xy55 | 0 | - | 18.3738 | 13.1667 | 16.4079 | 12.1191 | 0.0000 | nan | - | 5.5168 | 3.6796 | 5.4315 | 3.4924 | 0.0000 | nan | - | |
| xy55 | 1 | - | 13.1667 | 12.1194 | 12.1191 | 11.7436 | 0.0000 | nan | - | 3.6796 | 3.0129 | 3.4924 | 2.8222 | 0.0000 | nan | - | |
| xy56 | 0 | - | 17.9210 | 12.8984 | 16.9084 | 11.9762 | 0.0000 | nan | - | 5.7195 | 3.9752 | 5.4183 | 3.4499 | 0.0000 | nan | - | |
| xy56 | 1 | - | 12.8984 | 14.2478 | 11.9762 | 13.0840 | 0.0000 | nan | - | 3.9752 | 3.2214 | 3.4499 | 2.8536 | 0.0000 | nan | - | |
| xy57 | 0 | - | 17.9643 | 12.5860 | 16.4967 | 12.6264 | 0.0000 | nan | - | 5.4871 | 3.6574 | 5.3740 | 3.4181 | 0.0000 | nan | - | |
| xy57 | 1 | - | 12.5860 | 14.3727 | 12.6264 | 13.5046 | 0.0000 | nan | - | 3.6574 | 3.0978 | 3.4181 | 2.7987 | 0.0000 | nan | - | |
| xy60 | 0 | - | 18.5754 | 14.3014 | 19.7871 | 16.1240 | 18.2467 | 13.6199 | - | 5.7715 | 4.1062 | 7.6085 | 5.8120 | 6.8863 | 4.9200 | - | |
| xy60 | 1 | - | 14.3014 | 15.4722 | 16.1240 | 16.7295 | 13.6199 | 3.5696 | - | 4.1062 | 3.1894 | 5.8120 | 4.3245 | 4.9200 | 0.8764 | - | |
| xy61 | 0 | - | 18.4149 | 14.1142 | 19.0320 | 13.4497 | 18.5325 | 13.8615 | - | 6.0880 | 4.1860 | 7.2236 | 4.9619 | 7.1891 | 5.1303 | - | |
| xy61 | 1 | - | 14.1142 | 12.1579 | 13.4497 | 16.1997 | 13.8615 | 7.5957 | - | 4.1860 | 3.1970 | 4.9619 | 3.7255 | 5.1303 | 1.1089 | - | |
| xy62 | 0 | - | 17.9743 | 13.8208 | 17.8738 | 12.7798 | 17.9593 | 12.1875 | - | 5.5695 | 3.8447 | 6.7363 | 5.0456 | 7.0439 | 4.6350 | - | |
| xy62 | 1 | - | 13.8208 | 10.9875 | 12.7798 | 15.3255 | 12.1875 | 4.8085 | - | 3.8447 | 3.0123 | 5.0456 | 3.6803 | 4.6350 | 0.9223 | - | |
| xy63 | 0 | - | 17.5187 | 14.5204 | 17.3614 | 14.7514 | 0.0000 | nan | - | 5.7046 | 3.7578 | 6.4055 | 4.6858 | 0.0000 | nan | - | |
| xy63 | 1 | - | 14.5204 | 11.6832 | 14.7514 | 11.0854 | 0.0000 | nan | - | 3.7578 | 2.9840 | 4.6858 | 3.6345 | 0.0000 | nan | - | |
| xy64 | 0 | - | 17.6017 | 14.6413 | 17.3987 | 16.2023 | 0.0000 | nan | - | 5.7881 | 3.8969 | 6.7611 | 4.9196 | 0.0000 | nan | - | |
| xy64 | 1 | - | 14.6413 | 12.5936 | 16.2023 | 11.4374 | 0.0000 | nan | - | 3.8969 | 3.0256 | 4.9196 | 3.7609 | 0.0000 | nan | - | |
| xy65 | 0 | - | 18.1412 | 13.8778 | 16.4766 | 14.6702 | 0.0000 | nan | - | 5.7909 | 3.8372 | 6.5445 | 4.5016 | 0.0000 | nan | - | |
| xy65 | 1 | - | 13.8778 | 12.7789 | 14.6702 | 12.5641 | 0.0000 | nan | - | 3.8372 | 3.1923 | 4.5016 | 3.4447 | 0.0000 | nan | - | |
| xy66 | 0 | - | 17.7994 | 13.3668 | 16.2286 | 14.9296 | 0.0000 | nan | - | 5.8586 | 3.9755 | 6.5579 | 4.5374 | 0.0000 | nan | - | |
| xy66 | 1 | - | 13.3668 | 13.7715 | 14.9296 | 14.7816 | 0.0000 | nan | - | 3.9755 | 3.2770 | 4.5374 | 3.5710 | 0.0000 | nan | - | |
| xy67 | 0 | - | 18.6031 | 12.7620 | 16.5695 | 12.3977 | 0.0000 | nan | - | 5.8040 | 3.6758 | 6.3274 | 4.2493 | 0.0000 | nan | - | |
| xy67 | 1 | - | 12.7620 | 13.5308 | 12.3977 | 13.6571 | 0.0000 | nan | - | 3.6758 | 3.1162 | 4.2493 | 3.5995 | 0.0000 | nan | - | |
| xy70 | 0 | - | 18.6224 | 16.8280 | 0.0000 | nan | 17.9232 | 13.5220 | - | 6.4643 | 4.8306 | 0.0000 | nan | 6.2814 | 4.7851 | - | |
| xy70 | 1 | - | 16.8280 | 10.8186 | 0.0000 | nan | 13.5220 | nan | - | 4.8306 | 2.2564 | 0.0000 | nan | 4.7851 | nan | - | |
| xy71 | 0 | - | 18.0627 | 16.2561 | 19.2449 | 14.9796 | 18.4076 | 13.6569 | - | 6.2000 | 4.3428 | 6.5232 | 4.8270 | 6.4697 | 4.7545 | - | |
| xy71 | 1 | - | 16.2561 | 11.4097 | 14.9796 | 16.7605 | 13.6569 | nan | - | 4.3428 | 3.1694 | 4.8270 | 3.5373 | 4.7545 | nan | - | |
| xy72 | 0 | - | 17.9131 | 16.1474 | 20.6421 | 19.7244 | 0.0000 | nan | - | 6.1307 | 4.2060 | 6.5394 | 4.7448 | 0.0000 | nan | - | |
| xy72 | 1 | - | 16.1474 | 10.5407 | 19.7244 | 22.3510 | 0.0000 | nan | - | 4.2060 | 2.9494 | 4.7448 | 3.5783 | 0.0000 | nan | - | |
| xy73 | 0 | - | 17.6573 | 15.5024 | 17.1247 | 15.9317 | 0.0000 | nan | - | 6.1636 | 4.9120 | 6.2946 | 5.0910 | 0.0000 | nan | - | |
| xy73 | 1 | - | 15.5024 | 12.1165 | 15.9317 | 17.2715 | 0.0000 | nan | - | 4.9120 | 3.9651 | 5.0910 | 3.4171 | 0.0000 | nan | - | |
| xy74 | 0 | - | 19.5564 | 16.1072 | 17.5456 | 14.3083 | 0.0000 | nan | - | 6.3147 | 4.3845 | 6.5893 | 5.2462 | 0.0000 | nan | - | |
| xy74 | 1 | - | 16.1072 | 13.0130 | 14.3083 | 13.3359 | 0.0000 | nan | - | 4.3845 | 3.5398 | 5.2462 | 3.7716 | 0.0000 | nan | - | |
| xy75 | 0 | - | 17.8748 | 15.3014 | 16.4173 | 12.4891 | 0.0000 | nan | - | 6.2100 | 4.3155 | 6.8408 | 5.2154 | 0.0000 | nan | - | |
| xy75 | 1 | - | 15.3014 | 13.1325 | 12.4891 | 9.7654 | 0.0000 | nan | - | 4.3155 | 3.4496 | 5.2154 | 3.5053 | 0.0000 | nan | - | |
| xy76 | 0 | - | 18.0350 | 13.6841 | 16.1565 | 14.2040 | 0.0000 | nan | - | 6.1736 | 4.1423 | 6.3203 | 5.1315 | 0.0000 | nan | - | |
| xy76 | 1 | - | 13.6841 | 12.0662 | 14.2040 | 14.8797 | 0.0000 | nan | - | 4.1423 | 3.3137 | 5.1315 | 3.4352 | 0.0000 | nan | - | |
| xy77 | 0 | - | 17.3479 | 16.0663 | 16.5455 | 15.2718 | 0.0000 | nan | - | 6.7330 | 4.7866 | 6.7884 | 5.4040 | 0.0000 | nan | - | |
| xy77 | 1 | - | 16.0663 | 16.7648 | 15.2718 | 18.0559 | 0.0000 | nan | - | 4.7866 | 3.7874 | 5.4040 | 3.9781 | 0.0000 | nan | - |
September 5, 2012
Based on Nigel's email this morning (details at http://ps1sc.ifa.hawaii.edu/PS1wiki/index.php/DRAVG_telecon#Correlated_bias_structure_between_cells), I spent part of the afternoon looking at this structure again, starting from Nigel's point that adjacent cells seemed to have the same structure.
The OTA chosen by Nigel is OTA57, which has PATTERN.ROW applied to columns 0,1,2,6,7, leaving 3,4,5 with no PATTERN.ROW correction. The oversubtraction PATTERN.ROW leaves is clearly visible in the first image, in the cell one up and right of the bottom left corner. As columns 345 had lower row-to-row offsets, these cells were left unmodified. It's also clear from this image (and other investigations) that the cells we do not apply PATTERN.ROW to still have some level of row-to-row variations.
My study used bias images from OTA67, as I had them on disk locally (an MHPCC power outage prevented grabbing new data). For this study, I constructed row models by subtracting the median value for each cell from that cell, and then calculating the median "stack" across a row of cells. This results in 8 cell sized models appropriate for removing the bias variations from a row of cells. Subtracting this row model from each cell should remove the row-to-row variations. The following image shows these models for one bias frame, showing the variations between different cell-rows for one image.
A similar process can be done on a science image, with two result cells shown below. Both images show the original raw image, the row model subtracted image, and a similar model generated from columns of cells.
This method seems to do a decent job of minimizing row-to-row variations, without the oversubtraction around bright stars that PATTERN.ROW introduces. However, there are still some issues that need to be considered before this can be implemented.
- The first science image shows a ghosting due to bright pixel columns that cause an oversubtraction in the background level in vertical bands. This can probably be mitigated by including the static mask data when making the median stack.
- This structure appears to change between each bias frame that I examined, suggesting that this is not something that can be precalculated. This variation makes sense, as any static feature would be included in the dark model.
- These tests were done on raw images, without any dark correction. Some thought needs to be done to determine where in the detrending process this kind of correction should be applied.
- Bright extended objects will need to be checked closely to ensure that this does not damage those objects.
Last modified
13 years ago
Last modified on Feb 8, 2013, 2:48:56 PM
Attachments (13)
- rowmodels.png (1.2 MB ) - added by 14 years ago.
- ghosting.png (1.7 MB ) - added by 14 years ago.
- star.png (1.3 MB ) - added by 14 years ago.
- quickmosaic.jpg (581.0 KB ) - added by 13 years ago.
- pattern_median.jpg (618.8 KB ) - added by 13 years ago.
- science_dark_flat_pattern_0.0.png (14.2 KB ) - added by 13 years ago.
- science_gradient_0.0.png (14.4 KB ) - added by 13 years ago.
- dark_gradient_0.0.png (12.8 KB ) - added by 13 years ago.
- column_ripples_in_bias.jpg (439.3 KB ) - added by 13 years ago.
- median_0.0.jpg (761.0 KB ) - added by 13 years ago.
- test48.jpg (1.5 MB ) - added by 13 years ago.
- test51.jpg (1.5 MB ) - added by 13 years ago.
- test52.jpg (1.5 MB ) - added by 13 years ago.
Note:
See TracWiki
for help on using the wiki.
