Changeset 40562 for trunk/doc/release.2015/ps1.detrend/detrend.tex
- Timestamp:
- Dec 3, 2018, 5:47:07 AM (8 years ago)
- File:
-
- 1 edited
-
trunk/doc/release.2015/ps1.detrend/detrend.tex (modified) (58 diffs)
Legend:
- Unmodified
- Added
- Removed
-
trunk/doc/release.2015/ps1.detrend/detrend.tex
r40559 r40562 136 136 (to create difference images), along with the resulting image products and their 137 137 properties. 138 \citet[][Paper I]{chambers2017} provide san overview of the Pan-STARRS System, the138 \citet[][Paper I]{chambers2017} provide an overview of the Pan-STARRS System, the 139 139 design and execution of the Surveys, the resulting image and catalog data 140 140 products, a discussion of the overall data quality and basic … … 143 143 %Pan-STARRS Data Processing Stages 144 144 \citet[][Paper II]{magnier2017.datasystem} 145 describe show the various data processing stages are organized and145 describe how the various data processing stages are organized and 146 146 implemented 147 147 in the Imaging Processing Pipeline (IPP), including details of the … … 154 154 %Pan-STARRS Pixel Analysis : Source Detection 155 155 \citet[][Paper IV]{magnier2017.analysis} 156 describe sthe details of the source detection and photometry, including156 describe the details of the source detection and photometry, including 157 157 point-spread-function and extended source fitting models, and the 158 158 techniques for ``forced'' photometry measurements. … … 160 160 %Pan-STARRS Photometric and Astrometric Calibration 161 161 \citet[][Paper V]{magnier2017.calibration} 162 describe sthe final calibration process, and the resulting photometric and162 describe the final calibration process, and the resulting photometric and 163 163 astrometric quality. 164 164 %Flewelling et al. 2017 (Paper VI) 165 165 %Pan-STARRS 1 Database and Data Products 166 166 \citet[][Paper VI]{flewelling2017} 167 describe sthe details of the resulting catalog data and its organization167 describe the details of the resulting catalog data and its organization 168 168 in the Pan-STARRS database. 169 169 % … … 171 171 \citet[][Paper VII]{huber2017} 172 172 %Huber et al. 2017 (Paper VII) 173 describe sthe Medium Deep Survey in detail, including the unique issues and173 describe the Medium Deep Survey in detail, including the unique issues and 174 174 data products specific to that survey. The Medium Deep Survey is not part 175 175 of Data Release 1. (DR1) … … 188 188 view. 189 189 190 \note{DS notes fonts are not consistent for keywords, etc} 191 192 \note{DS: captions need to be clear re: illustrated effect} 193 190 194 \note{need to define PV3 (and PV0-2) here. see datasystem.tx} 191 195 192 196 %The Processing Version 3 (PV3) reduction represents the third full 193 197 DR1 contains the results of the third full reduction of the Pan-STARRS 194 archival data. The first two reductions were used internally for 195 pipeline optimization and the development of the initial photometric 196 and astrometric reference catalog \citep{magnier2017.calibration}. 197 The products from these reductions were not publicly released, but 198 have been used to produce a wide range of scientific papers from the 199 Pan-STARRS 1 Science Consortium members. 198 archival data, idenfied as PV3. Previous reductions \citep[PV0, PV1, 199 PV2; see][]{magnier2017.datasystem} 200 were used internally for pipeline optimization and the development of 201 the initial photometric and astrometric reference catalog 202 \citep{magnier2017.calibration}. The products from these reductions 203 were not publicly released, but have been used to produce a wide range 204 of scientific papers from the Pan-STARRS 1 Science Consortium members 205 \citep{chambers2017}. 200 206 201 207 The Pan-STARRS image processing pipeline (IPP) is described elsewhere … … 232 238 are provided in \citet{magnier2017.analysis}. 233 239 234 A limited version of same reduction procedure described above is also240 A limited version of the same reduction procedure described above is also 235 241 performed in real time on new exposures as they are observed by the 236 242 telescope. This process is automatic, with new exposures being … … 301 307 302 308 These corrections assume that the detector response is linear across 303 the full range of values. This assumption is not universally true for 304 GPC1, and an additional set of detrending steps are required as a 309 the full dynamic range and that the pixels contain only signals coming 310 from the imaged portion of the sky, or from linear dark current 311 sources within the detector. This assumption is not universally true 312 for GPC1, and an additional set of detrending steps are required as a 305 313 result. The first of these is the \IPPprog{burntool} correction, 306 314 which removes the flux trails left by the incomplete transfer of 307 315 charge along the readout columns. These trails are generally only 308 316 evident for the brightest stars, as only pixels that are at or beyond 309 the saturation point of the detector leave residual charge. More 310 widespread is the non-linearity at the faint end of the pixel range. 311 Some readout cells and some readout cell edge pixels experience a sag 312 relative to linear at low illumination, such that faint pixels appear 313 fainter than expected. The correction to this requires amplifying the 314 pixel values in these regions to match the expected model. 317 the saturation point of the detector leave residual charge. A second 318 confounding effect is the non-linearity at the faint end of the pixel 319 range. Some readout cells and some readout cell edge pixels 320 experience a sag relative to the linear trend at low illumination, 321 such that faint pixels appear fainter than expected. The correction 322 to this requires amplifying the pixel values in these regions to match 323 the linear response. 315 324 316 325 Large regions of some OTA cells experience significant charge transfer … … 326 335 field of view. 327 336 328 For the PV3 processing, all detrending is done by the 329 \IPPprog{ppImage} program. This program applies the detrend 330 corrections to the individual cells, and then an OTA-level mosaic is 331 constructed for the signal image, the mask image, and the variance map 332 image. The single epoch photometry is done at this stage as well. 333 The following subsections (\ref{sec:overscan} - \ref{sec:background}) 334 det ail the detrending process used on GPC1 that are common to other335 detectors. The GPC1 specific detrending steps are included after, 336 explainingthese additional steps that remove the instrument signature.337 Within the IPP, all detrending is done by the \IPPprog{ppImage} 338 program. This program applies the detrend corrections to the 339 individual cells, and then an OTA-level mosaic is constructed for the 340 signal image, the mask image, and the variance map image. The single 341 epoch photometry is done at this stage as well. The following 342 subsections (\ref{sec:overscan} - \ref{sec:background}) detail the 343 detrending process used on GPC1 that are common to other detectors. 344 The GPC1 specific detrending steps are included after, explaining 345 these additional steps that remove the instrument signature. 337 346 338 347 \subsection{Overscan} … … 427 436 \centering 428 437 \begin{minipage}{0.45\hsize} 429 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_M_OS_NL_XY23 _b1.jpg}438 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_M_OS_NL_XY23.png} 430 439 \end{minipage}% 431 440 \begin{minipage}{0.45\hsize} 432 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_to_DARK_XY23 _b1.jpg}441 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_to_DARK_XY23.png} 433 442 \end{minipage} 434 443 \caption{An example of the dark model application to exposure o5677g0123o, OTA23 (2011-04-26, 43s \gps{} filter). The left panel shows the image data mosaicked to the OTA level, and has had the static mask applied, the overscan subtracted, and the detector non-linearity corrected. The right panel, shows the same exposure with the dark applied in addition to the processing shown on the left, removing the amplifier glows in the cell corners.} … … 482 491 \centering 483 492 \begin{minipage}{0.45\hsize} 484 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_VIDEODARK_VDim_Rdark_XY22 _b1.jpg}493 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_VIDEODARK_VDim_Rdark_XY22.png} 485 494 \end{minipage}% 486 495 \begin{minipage}{0.45\hsize} 487 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_VIDEODARK_VDim_VDdark_XY22 _b1.jpg}496 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_VIDEODARK_VDim_VDdark_XY22.png} 488 497 \end{minipage} 489 498 \caption{An example of the video dark model application to exposure o5677g0123o, OTA22 (2011-04-26, 43s \gps{} filter), which has a video cell located in cell xy16. The left panel shows the image data mosaicked to the OTA level, and has had the static mask applied, the overscan subtracted, the detector non-linearity corrected, and a regular dark applied. The right panel, shows the same exposure with a video dark applied instead of the standard dark. The main impact of this change is the improved correction of the corner glows, which are over subtracted with the standard dark.} … … 498 507 characteristics. Instead, there is a gradient along the pixel rows, 499 508 with the noise generally higher away from the read out amplifier 500 (higher cell xpixel positions). This is likely an effect of the509 (higher cell $x$ pixel positions). This is likely an effect of the 501 510 row-by-row bias issue discussed below (Section~\ref{sec:pattern.row}). 502 511 As a result of this increased noise, more sources are detected in the … … 526 535 dependent read noise. By binning the number of false positives 527 536 measured on the bias frames on the noisemap inputs using 20 pixel 528 boxes in the cell x-axis, and comparing this to the number expected537 boxes in the cell $x$-axis, and comparing this to the number expected 529 538 from random Gaussian noise, we estimated the true read noise level. 530 539 … … 574 583 575 584 In addition to this flat field applied to the individual images, the 576 ubercal process used to calibrate the database of all detections 577 \citep{2012ApJ...756..158S} constructs ``in catalog'' flat field 578 corrections. Although a single set of image flat fields was used for 579 the entire PV3 survey, five separate ``seasons'' of database flat 580 fields were needed to ensure proper calibration. This indicates that 581 the flat field response is not completely fixed in time. More details 582 on this process are contained in \citet{magnier2017.calibration}. 585 ``ubercal'' analysis -- in which photometric data are used define 586 image zero points 587 \citep[][]{2012ApJ...756..158S,magnier2017.calibration} and in turn 588 used used to calibrate the database of all detections -- constructs 589 ``in catalog'' flat field corrections. Although a single set of image 590 flat fields was used for the PV3 processing of the entire $3\pi$ 591 survey, five separate ``seasons'' of database flat fields were needed 592 to ensure proper calibration. This indicates that the flat field 593 response is not completely fixed in time. More details on this 594 process are contained in \citet{magnier2017.calibration}. 583 595 584 596 \subsection{Fringe correction} … … 619 631 \begin{figure} 620 632 \centering 621 \begin{minipage}{0. 5\hsize}622 \includegraphics[width= 1.5\hsize,angle=0,clip]{images/o5220g0025o_XY53_nofringe.png}633 \begin{minipage}{0.45\hsize} 634 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5220g0025o_nofringe_XY53.png} 623 635 \end{minipage}% 624 \begin{minipage}{0. 5\hsize}625 \includegraphics[width= 1.5\hsize,angle=0,clip]{images/o5220g0025o_XY53_fringe.png}636 \begin{minipage}{0.45\hsize} 637 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5220g0025o_fringe_XY53.png} 626 638 \end{minipage} 627 639 \caption{Example of the \yps{} filter fringe pattern on exposure o5220g0025o OTA53 (\yps{} filter 30s). The left panel shows the OTA mosaic with all detrending except the fringe correction, while the right shows the same including the fringe correction. Both images have been smoothed with a Gaussian with $\sigma = 3$ pixels to highlight the faint and large scale fringe patterns. … … 710 722 \tablewidth{0pc} 711 723 \tablecaption{GPC1 Mask Values} 712 \tablehead{\colhead{Mask Name} & \colhead{Mask Value} & \colhead{Description}} 724 \tablehead{\colhead{Mask Name} & \colhead{Mask Value} & 725 \colhead{Description (static values listed in bold)}} 713 726 \startdata 714 DETECTOR & 0x0001 & A detector defect is present.\\715 FLAT & 0x0002 & The flat field model does not calibrate the pixel reliably.\\716 DARK & 0x0004 & The dark model does not calibrate the pixel reliably.\\717 BLANK & 0x0008 & The pixel does not contain valid data.\\718 CTE & 0x0010 & The pixel has poor charge transfer efficiency.\\727 {\bf DETECTOR & 0x0001 & A detector defect is present.} \\ 728 {\bf FLAT & 0x0002 & The flat field model does not calibrate the pixel reliably.} \\ 729 {\bf DARK & 0x0004 & The dark model does not calibrate the pixel reliably.} \\ 730 {\bf BLANK & 0x0008 & The pixel does not contain valid data.} \\ 731 {\bf CTE & 0x0010 & The pixel has poor charge transfer efficiency.} \\ 719 732 SAT & 0x0020 & The pixel is saturated. \\ 720 733 LOW & 0x0040 & The pixel has a lower value than expected. \\ 721 SUSPECT & 0x0080 & The pixel is suspected of being bad . \\734 SUSPECT & 0x0080 & The pixel is suspected of being bad (overloaded with the BURNTOOL bit). \\ 722 735 BURNTOOL & 0x0080 & The pixel contain an burntool repaired streak. \\ 723 736 CR & 0x0100 & A cosmic ray is present. \\ … … 764 777 Table \ref{tab:crosstalk_rules} summarizes the list of known crosstalk 765 778 rules, with an estimate of the magnitude difference between the source 766 and ghost. For all of the rules, any source cell $v$ within the specified 767 column of cells on any of the OTAs in the specified column of OTAs $Y$ 768 can create a ghost in the same cell $v$ and OTA $Y$ in the target column of 769 cells and OTAs. In each of these cases, a source object with an 770 instrumental magnitude brighter than -14.47 creates a ghost object 771 many orders of magnitude fainter at the target location. The cell 772 (x,y) pixel coordinate is identical between source and ghost, as a 773 result of the transfer occurring as the devices are read. A circular 774 mask is added to the ghost location with radius $R = 3.44 \left(-14.47 775 - m_{source, instrumental}\right)$ pixels. Any objects in the 776 photometric catalog found at the location of the ghost mask have the 777 GHOST mask bit set, marking the object as a likely ghost. The 778 majority of the crosstalk rules are bi-directional, with a source in 779 either position creating a ghost at the corresponding crosstalk target 780 position. The two faintest rules are uni-directional, due to 781 differences in the electronic path for the crosstalk. 782 783 For the very brightest sources ($m_{instrumental} < -15$), there can 784 be crosstalk ghosts between all columns of cells during the readout. 779 and ghost. For all of the rules, any source cell $v$ within the 780 specified column of cells on any of the OTAs in the specified column 781 of OTAs $Y$ can create a ghost in the same cell $v$ and OTA $Y$ in the 782 target column of cells and OTAs. This effect depends on the number of 783 electrons detected for the star, thus the size of the ghost scales 784 with the instrumental magnitude ($m_{inst} = -2.5 \log_{10} (ADU)$) of 785 the star. In each of these cases, a source object with $m_{inst} < 786 -14.47$) (corresponding to $\rps \lesssim 14$ for the $3\pi$ survey) 787 creates a ghost object many orders of magnitude fainter at the target 788 location. The cell ($x,y$) pixel coordinate is identical between 789 source and ghost, as a result of the transfer occurring as the devices 790 are read. A circular mask is added to the ghost location with radius 791 $R = 3.44 \left(-14.47 - m_{inst,source}\right)$ pixels; only 792 positive radii are allowed. Any objects in the photometric catalog 793 found at the location of the ghost mask have the GHOST mask bit set, 794 marking the object as a likely ghost. The majority of the crosstalk 795 rules are bi-directional, with a source in either position creating a 796 ghost at the corresponding crosstalk target position. The two 797 faintest rules are uni-directional, due to differences in the 798 electronic path for the crosstalk. 799 800 For the very brightest sources ($m_{inst} < -15$), there can be 801 crosstalk ghosts between all columns of cells during the readout. 785 802 These ``bleed'' ghosts were originally identified as ghosts of the 786 803 saturation bleeds appearing in the neighboring cells, and as such, the … … 788 805 bottom of cells in all columns that are in the same row of cells as 789 806 the bright source. The width of this box is a function of the source 790 magnitude, with $W = 5 * \left(-15 - m_{source, instrumental}\right)$807 magnitude, with $W = 5 \times \left(-15 - m_{inst,source}\right)$ 791 808 pixels. 792 809 … … 824 841 extra travel distance, the resulting source is out of focus and 825 842 elongated along the radial direction of the camera focal plane. These 826 optical ghosts can be modeled in the focal plane coordinates ( L,M)843 optical ghosts can be modeled in the focal plane coordinates ($L,M$) 827 844 which has its origin at the center of the focal plane. In this 828 system, a bright object at location ( L,M) on the focal plane creates a845 system, a bright object at location ($L,M$) on the focal plane creates a 829 846 reflection ghost on the opposite side of the optical axis near 830 ( -L,-M). The exact location is fit as a third order polynomial in the831 focal plane L and Mdirections (as listed in Table847 ($-L,-M$). The exact location is fit as a third order polynomial in the 848 focal plane $L$ and $M$ directions (as listed in Table 832 849 \ref{tab:ghost_centers}). An elliptical annulus mask is constructed 833 850 at the expected ghost location, with the major and minor axes defined … … 842 859 \tablewidth{0pc} 843 860 \tablecaption{Optical Ghost Center Transformations} 844 \tablehead{\colhead{Polynomial Term}&\colhead{ L center}&\colhead{Mcenter}}861 \tablehead{\colhead{Polynomial Term}&\colhead{$L$ center}&\colhead{$M$ center}} 845 862 \startdata 846 863 $x^0 y^0$ & -1.215661e+02 & 2.422174e+01 \\ … … 898 915 Prior to 2010-08-24, a reflective surface at the edge of the camera 899 916 aperture was incompletely screened to light passing through the 900 telescope. Sources brighter than $m_{inst} = -21$ that fell on this901 reflective surface resulted in light being scattered across the 902 detector surface in a long narrow glint. This surface was physically 903 masked on 2010-08-24, removing the possibility of glints in subsequent 904 data, but images that were taken prior to this date have an advisory 905 d ynamic mask constructed when a reference source falls on the focal906 plane within one degree of the detector edge. This mask is 150 pixels907 wide, with length $L = 2500 \left(-20 - m_{inst}\right)$ pixels. 908 These glint masks are constructed by selecting sufficiently bright 909 s ources in the reference catalog that fall within rectangular regions910 around each edge of the GPC1 camera. These regions are separated from 911 the edge of the camera by 17 arcminutes, and extend outwards an 912 a dditional degree.917 telescope. Sources brighter than $m_{inst} = -21$ ($\rps \lesssim 918 7.5$) that fell on this reflective surface resulted in light being 919 scattered across the detector surface in a long narrow glint. This 920 surface was physically masked on 2010-08-24, removing the possibility 921 of glints in subsequent data, but images that were taken prior to this 922 date have an advisory dynamic mask constructed when a reference source 923 falls on the focal plane within one degree of the detector edge. This 924 mask is 150 pixels wide, with length $L = 2500 \left(-20 - 925 m_{inst}\right)$ pixels. These glint masks are constructed by 926 selecting sufficiently bright sources in the reference catalog that 927 fall within rectangular regions around each edge of the GPC1 camera. 928 These regions are separated from the edge of the camera by 17 929 arcminutes, and extend outwards an additional degree. 913 930 914 931 \begin{figure} … … 924 941 Bright sources also form diffraction spikes that are dynamically 925 942 masked. These are filter independent, and are modeled as rectangles 926 with length $L = 10^{0.096 * (7.35 - m_{instrumental})} - 200$ and927 width $W = 8 + (L - 200) *0.01$, with negative values indicating no943 with length $L = 10^{0.096 \times (7.35 - m_{inst})} - 200$ and 944 width $W = 8 + (L - 200) \times 0.01$, with negative values indicating no 928 945 mask is constructed, as the source is likely too faint to produce the 929 946 feature. These spikes are dependent on the camera rotation, and are 930 oriented based on the header keyword at $\theta = n *\frac{\pi}{2} -947 oriented based on the header keyword at $\theta = n \times \frac{\pi}{2} - 931 948 \mathrm{ROTANGLE} + 0.798$, for $n = {0,1,2,3}$. 932 949 933 950 The cores of stars that are saturated are masked as well, with a 934 circular mask radius $r = 10.15 * (-15 - m_{instrumental})$. An951 circular mask radius $r = 10.15 \times (-15 - m_{inst})$. An 935 952 example of a saturated star, with the masked regions for the 936 953 diffraction spikes and core saturation highlighted, is shown in Figure … … 939 956 \begin{figure} 940 957 \centering 941 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o6802g0338o_ XY51_b1.jpg}958 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o6802g0338o_SATSTAR_XY51.png} 942 959 \caption{Example of saturated star, with diffraction spikes extending from the core on exposure o6802g0338o, OTA51 (2014-05-25, 45s \gps{} filter).} 943 960 \label{fig:saturated star} … … 1047 1064 median of the pixel distribution, with the standard deviation of the 1048 1065 distribution set as the median of the $\sigma$ values calculated from 1049 the $0.5 *(\sigma_{+1} - \sigma_{-1})$, $\sigma_{+0.5} -1050 \sigma_{-0.5}$, and $0.25 *(\sigma_{+2} - \sigma_{-2})$ differences.1066 the $0.5 \times (\sigma_{+1} - \sigma_{-1})$, $\sigma_{+0.5} - 1067 \sigma_{-0.5}$, and $0.25 \times (\sigma_{+2} - \sigma_{-2})$ differences. 1051 1068 If this measured standard deviation is smaller than 3 times the bin 1052 1069 size, then all points more than 25 bins away from the calculated … … 1057 1074 are found by interpolating the 5 bins around the expected bin as well, 1058 1075 and the count of the number of input values within this inner 1059 50-percentile region, $N_{50}$ is calculated.1076 50-percentile region, $N_{50}$, is calculated. 1060 1077 1061 1078 These initial statistics are then used as the starting guesses for a … … 1191 1208 \centering 1192 1209 \begin{minipage}{0.45\hsize} 1193 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_ XY11_nobt.png}1210 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_nbt_XY11.png} 1194 1211 \end{minipage}% 1195 1212 \begin{minipage}{0.45\hsize} 1196 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0124o_ XY11_nobt.png}1213 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0124o_nbt_XY11.png} 1197 1214 \end{minipage} 1198 1215 \begin{minipage}{0.45\hsize} 1199 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_ XY11_bt.png}1216 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0123o_wbt_XY11.png} 1200 1217 \end{minipage}% 1201 1218 \begin{minipage}{0.45\hsize} 1202 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0124o_ XY11_bt.png}1219 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5677g0124o_wbt_XY11.png} 1203 1220 \end{minipage} 1204 1221 \caption{Example of OTA11 cell xy50 on exposures o5677g0123o (left) and o5677g0124o (right). The top panels show the image with all appropriate detrending steps, but without burntool, and the bottom show the same with burntool applied. There is some slight over subtraction in fitting the initial trail, but the impact of the trail is greatly reduced in both exposures.} … … 1246 1263 We store the average flux measurement and deviation from the linear 1247 1264 fit for each exposure time for each region on all detector cells in 1248 the linearity detrend look 1249 shown in figure\ref{fig: nonlinearity}. When this correction is1265 the linearity detrend look-up tables. An example of this data is 1266 shown in Figure~\ref{fig: nonlinearity}. When this correction is 1250 1267 applied to science data, these lookup tables are loaded, and a linear 1251 1268 interpolation is performed to determine the correction needed for the … … 1270 1287 \centering 1271 1288 \includegraphics[width=0.9\hsize,angle=0,clip]{images/linearity_XY27_xy16.png} 1272 \caption{Example plotof the linearity correction as a fraction of observed flux for OTA27, cell xy16.}1289 \caption{Example of the linearity correction as a fraction of observed flux for OTA27, cell xy16.} 1273 1290 \label{fig: nonlinearity} 1274 1291 \end{figure} … … 1293 1310 offsets increases as the distance from the readout amplifier and 1294 1311 overscan region increases, resulting in horizontal streaks that are 1295 more pronounced along the large xpixel edge of the cell. As the1312 more pronounced along the large $x$ pixel edge of the cell. As the 1296 1313 level of the offset is apparently random between exposures, the dark 1297 1314 correction cannot fully remove this structure from the images, and the … … 1299 1316 by these bias offsets. Therefore, we apply the PATTERN.ROW correction 1300 1317 in an attempt to mitigate the offsets and correct the image values. 1301 To force the rows to agree, a second order clipped polynomial is fit 1302 to each row in the cell. Four fit iterations are run, and pixels 1303 $2.5\sigma$ deviant are excluded from subsequent fits, to minimize the 1304 effect stars and other astronomical signals have. This final trend is 1305 then subtracted from that row. Simply doing this subtraction will 1306 also have the effect of removing the background sky level. To prevent 1318 To force the rows to agree, a second order clipped polynomial is 1319 fitted to each row in the cell. Four fit iterations are run and 1320 pixels $2.5\sigma$ deviant (chosen empirically) are excluded from 1321 subsequent fits in order to minimize the bias from stars and other 1322 astronomical sources in the pixels. This final trend is then 1323 subtracted from that row. Simply doing this subtraction will also 1324 have the effect of removing the background sky level. To prevent 1307 1325 this, the constant and linear terms for each row are stored, and 1308 1326 linear fits are made to these parameters as a function of row, … … 1368 1386 \centering 1369 1387 \begin{minipage}{0.45\hsize} 1370 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5379g0103o_ XY57_nopat.png}1388 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5379g0103o_npt_XY57.png} 1371 1389 \end{minipage}% 1372 1390 \begin{minipage}{0.45\hsize} 1373 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5379g0103o_ XY57_pat.png}1391 \includegraphics[width=0.9\hsize,angle=0,clip]{images/o5379g0103o_wpt_XY57.png} 1374 1392 \end{minipage} 1375 1393 \caption{Example of the PATTERN.ROW correction on exposure o5379g0103o OTA57 cell xy01 (\ips{} filter 45s). The left panel shows the cell with all appropriate detrending except the PATTERN.ROW, and the right shows the same cell with PATTERN.ROW applied. The correction reduces the correlated noise on the right side, which is most distant from the read out amplifier. There is a slight over subtraction along the rows near the bright star.} … … 1379 1397 \subsubsection{Pattern Continuity} 1380 1398 1381 The background levels of cells on a single OTA do not always have the 1382 same value. Even with dark and flat corrections applied, adjacent 1383 cells may not match. In addition, studies of the background level 1384 indicate that the row-by-row bias can introduce small background 1385 gradient variations along the rows of the cells that are not stable. 1386 This common feature across the columns of cells results in a ``saw 1387 tooth'' pattern horizontally across an the mosaicked OTA, and as the 1388 background model fits a smooth sky level, this induces over and under 1389 subtraction at the cell boundaries. 1399 The background sky levels of cells on a single OTA do not always have 1400 the same value. Despite having dark and flat corrections applied, 1401 adjacent cells may not match even for images of nominally empty sky. 1402 In addition, studies of the background level indicate that the 1403 row-by-row bias can introduce small background gradient variations 1404 along the rows of the cells that are not stable. This common feature 1405 across the columns of cells results in a ``saw tooth'' pattern 1406 horizontally across an the mosaicked OTA, and as the background model 1407 fits a smooth sky level, this induces over- and under subtraction at 1408 the cell boundaries. 1390 1409 1391 1410 The PATTERN.CONTINUITY correction, attempts to match the edges of a 1392 1411 cell to those of its neighbors. For each cell, a thin box 10 pixels 1393 1412 wide running the full length of each edge is extracted and the median 1394 value of unmasked values calculated for that box. These median values1413 of unmasked values is calculated for that box. These median values 1395 1414 are then used to construct a vector of the sum of the differences 1396 1415 between that cell's edges and the corresponding edge on any adjacent … … 1398 1417 constructed, with the diagonal containing the number of cells adjacent 1399 1418 to that cell, and the off-diagonal values being set to -1 for each 1400 pair of adjacent cells. The offsets needed for each chip, $ x$ can1401 then be found by solving the system $A x= \Delta$. A cell with the1419 pair of adjacent cells. The offsets needed for each chip, $\zeta$ can 1420 then be found by solving the system $A \zeta = \Delta$. A cell with the 1402 1421 maximum number of neighbors, usually cell xy11, the first cell not on 1403 1422 the edge of the OTA, is used to constrain the system, ensuring that … … 1496 1515 \tablewidth{0pc} 1497 1516 \tablecaption{PV3 Detrends} 1498 \tablehead{\colhead{Detrend Type} & \colhead{Detrend ID} & \colhead{Start Date} & \colhead{End Date} & \colhead{Note} } 1517 \tablehead{\colhead{Detrend Type} & \colhead{Detrend ID} & 1518 \colhead{Start Date (UT)} & \colhead{End Date (UT)} & \colhead{Note} } 1499 1519 \startdata 1500 1520 LINEARITY & 421 & 2009-01-01 00:00:00 & & \\ … … 1541 1561 To provide a consistent and uniform set of coordinates for image 1542 1562 combination (including stacking and differences), the individual 1543 mosaicked OTA images are projected onto acommon pixel grids, called1563 mosaicked OTA images are projected onto common pixel grids, called 1544 1564 tessellations. A tessellation can contain any number of tangent plane 1545 1565 projections, with those designed for single pointing surveys using … … 1554 1574 used for processing image data beyond the initial chip stage. The 1555 1575 coordinate system used for these images matches the parity of the sky, 1556 with north in the positive y direction and east to the negative x1576 with north in the positive $y$ direction and east to the negative $x$ 1557 1577 direction. 1558 1578 … … 1572 1592 output pixel has a unique sampling position on the input image 1573 1593 (although it may be off the image frame and therefore not populated), 1574 preventing gaps in the output image due to the spacing of the input 1575 pixels.1594 guaranteing that all output pixels are addressed, and thus preventing 1595 gaps in the output image due to the spacing of the input pixels. 1576 1596 1577 1597 With the locally linear grid defined, Lanczos interpolation 1578 \citep{lanczos1956applied} with filter size parameter $a = 3$ on the input 1579 image is used to determine the values to assign to the output pixel 1580 location. This process is repeated for all grid boxes, for all input 1581 images, and for each output image product: the science image, the 1582 variance, and the mask. The image values are scaled by the absolute 1583 value of the Jacobian determinant of the transformation for each grid 1584 box. This corrects the pixel values for the possible change in pixel 1585 area due to the transformation. Similarly, the variance image is 1586 scaled by the square of this value, again to correctly account for the 1587 pixel area change. 1598 \citep{lanczos1956applied} with filter size parameter $a = 3$ on the 1599 input image is used to determine the values to assign to the output 1600 pixel location. This interpolation kernel was chosen as a compromise 1601 between simple interpolations and higher-order Lanczos kernels, with 1602 the goal of limiting the smear in the output image while avoiding 1603 the high-frequency ringing generated by higher order kernels. This 1604 process is repeated for all grid boxes, for all input images, and for 1605 each output image product: the science image, the variance, and the 1606 mask. The image values are scaled by the absolute value of the 1607 Jacobian determinant of the transformation for each grid box. This 1608 corrects the pixel values for the possible change in pixel area due to 1609 the transformation. Similarly, the variance image is scaled by the 1610 square of this value, again to correctly account for the pixel area 1611 change. 1588 1612 1589 1613 The interpolation constructs the output pixels from more than one … … 1599 1623 to scale the position uncertainties based on the new orientation. 1600 1624 1601 The output image also contains header keywords SRC\_0000, SEC\_0000, 1602 MPX\_0000, and MPY\_0000 that define the mappings from the warped 1603 pixel space to the input image. The SRC keyword lists the input OTA 1604 name, and the SEC keyword lists the image section that the mapping 1605 covers. The MPX and MPY contain the back-transformation linearized 1606 across the full chip. These parameters are stored in a string listing 1607 the reference position in the chip coordinate frame, the slope of the 1608 relation in the warp x axis, and the slope of the relation in the warp 1609 y axis. From these keywords, any position in the warp can be mapped 1610 back to the location in any of the input OTA images, with some 1611 reduction in accuracy. 1625 The output image also contains header keywords SRC\_nnnn, SEC\_nnnn, 1626 MPX\_nnnn, and MPY\_nnnn that define the mappings from the warped 1627 pixel space to the input images. The 'nnnn' for each keyword has the 1628 values 0000, 0001, etc., up to the number of input images. The SRC 1629 keyword lists the input OTA name, and the SEC keyword lists the image 1630 section that the mapping covers. The MPX and MPY contain the 1631 back-transformation linearized across the full chip. These parameters 1632 are stored in a string listing the reference position in the chip 1633 coordinate frame, the slope of the relation in the warp $x$ axis, and 1634 the slope of the relation in the warp $y$ axis. From these keywords, 1635 any position in the warp can be mapped back to the location in any of 1636 the input OTA images, with some reduction in accuracy. 1612 1637 1613 1638 \begin{figure} 1614 1639 \centering 1615 \includegraphics[width=0.9\hsize,angle=0,clip]{images/warp_ 1046511_sci.jpg}1640 \includegraphics[width=0.9\hsize,angle=0,clip]{images/warp_2046019_sci.png} 1616 1641 \caption{Example of the warp image for skycell skycell.2047.005 1617 1642 centered at ($\alpha,\delta$) = (179.763, 32.1899) for exposure … … 1629 1654 \begin{figure} 1630 1655 \centering 1631 \includegraphics[width=0.9\hsize,angle=0,clip]{images/warp_ 1046511_wt.jpg}1656 \includegraphics[width=0.9\hsize,angle=0,clip]{images/warp_2046019_var.png} 1632 1657 \caption{Example of the warp variance image for skycell 1633 1658 skycell.2047.005 of exposure o4985g0073o, the same as in Figure … … 1644 1669 \begin{figure} 1645 1670 \centering 1646 \includegraphics[width=0.9\hsize,angle=0,clip]{images/warp_ 1046511_mask.jpg}1671 \includegraphics[width=0.9\hsize,angle=0,clip]{images/warp_2046019_mask.png} 1647 1672 \caption{Example of the warp mask image for skycell skycell.2047.005 1648 1673 of exposure o4985g0073o, the same as in Figure \ref{fig:warp … … 1675 1700 prepare the inputs and stack the frames. 1676 1701 1702 \note{need to point out that we are convolving to a matched PSF} 1703 1677 1704 Once all files are ingested, the first step is to measure the size and 1678 1705 shapes of the input image PSFs. We exclude images that have a PSF 1679 1706 FWHM greater than 10 pixels (2.5 arcseconds), as those images have the 1680 1707 seeing far worse than average, and would degrade the final output 1681 stack. For the PV3 $3\pi$ survey, this size represents a PSF larger1708 stack. For the PV3 processing of the $3\pi$ survey, this size represents a PSF larger 1682 1709 than the $97$th percentile in all filters. A target PSF for the stack 1683 1710 is constructed by finding the maximum envelope of all input PSFs, … … 1693 1720 airmass, image exposure time, and zeropoint. All output stacks are 1694 1721 constructed to a target zeropoint of 25.0 in all filters, and to have 1695 an airmass of 1.0. The output exposure time is set to the sum of the 1696 input exposure times, regardless of whether those inputs are rejected later 1697 in the combination process. We can determine the relative 1722 an airmass of 1.0. The target zeropoint is arbitrary; 25.0 was chosen 1723 to be roughly consistent with the PS1 zero points, while still being a 1724 simple number. The output exposure time is set to the sum of the 1725 input exposure times, {\em regardless of whether those inputs are rejected 1726 later in the combination process}. We can determine the relative 1698 1727 transparency for each input image by comparing the magnitudes of 1699 1728 matched sources between the different images. Each image then has a 1700 1729 normalization factor defined, equal to $\mathrm{norm}_{input} = 1701 1730 (ZP_\mathrm{input} - ZP_\mathrm{target}) - 1702 \mathrm{transparency}_\mathrm{input} - 2.5 *\log_{10}1703 (t_\mathrm{target} / t_\mathrm{input}) - \mathrm{F}_\mathrm{airmass} *1731 \mathrm{transparency}_\mathrm{input} - 2.5 \times \log_{10} 1732 (t_\mathrm{target} / t_\mathrm{input}) - \mathrm{F}_\mathrm{airmass} \times 1704 1733 (\mathrm{airmass}_\mathrm{input} - \mathrm{airmass}_\mathrm{target})$. 1705 1734 For the PV3 processing, the airmass factor … … 1715 1744 the entire region of the sky imaged. This further calibration is not 1716 1745 available at the time of stacking, and so there may be small residuals 1717 in the transparency values as a result of this \cite t{magnier2017.calibration}.1746 in the transparency values as a result of this \citep{magnier2017.calibration}. 1718 1747 1719 1748 With the flux normalization factors and target PSF chosen, the 1720 convolution kernels can be calculated for each image. ISIS kernels 1721 \citep{1998ApJ...503..325A} are used with FWHM values of 1.5, 3.0, and 6.0 1722 pixels and polynomial orders of 6, 4, and 2. Regions around the 1723 sources identified in the input images are extracted, convolved with 1724 the kernel, and the residual with the target PSF used to update the 1725 parameters of the kernel via least squares optimization. Stamps that 1726 significantly deviate are rejected, although the squared residual 1727 difference will increase with increasing source flux. To mitigate 1728 this effect, a parabola is fit to the distribution of squared 1729 residuals as a function of source flux. Stamps that deviate from this 1730 fit by more than $2.5\sigma$ are rejected, and not used on further 1731 kernel fit iterations. This process is repeated twice, and the final 1732 convolution kernel is returned. 1749 convolution kernels can be calculated for each image. To calculate 1750 the convolution kernels, we use the algorithm described by 1751 \cite{1998ApJ...503..325A} and \cite{2000.alard} to perform optimal 1752 image subtraction. These `ISIS' kernels \citep[named after the 1753 software package described by][]{1998ApJ...503..325A} are used with 1754 FWHM values of 1.5, 3.0, and 6.0 pixels and polynomial orders of 6, 4, 1755 and 2. Regions around the sources identified in the input images are 1756 extracted, convolved with the kernel, and the residual with the target 1757 PSF used to update the parameters of the kernel via least squares 1758 optimization. Stamps that significantly deviate are rejected, 1759 although the squared residual difference will increase with increasing 1760 source flux. To mitigate this effect, a parabola is fit to the 1761 distribution of squared residuals as a function of source flux. 1762 Stamps that deviate from this fit by more than $2.5\sigma$ are 1763 rejected, and not used on further kernel fit iterations. This process 1764 is repeated twice, and the final convolution kernel is returned. 1733 1765 1734 1766 This convolution may change the image flux scaling, so the kernel is … … 1759 1791 identify discrepant input values that should be excluded. 1760 1792 1793 \note{clarify 'should' below, e.g., with a histogram} 1794 1761 1795 If only a single input is available, the initial stack contains the 1762 1796 value from that single input. If there are only two inputs, the … … 1770 1804 1771 1805 \begin{eqnarray} 1772 \mathrm{Stack}_\mathrm{value} &=& \sum_i\left(\mathrm{value}_\mathrm{input} *W_\mathrm{input}\right) / \sum_\mathrm{inputs}\left(W_\mathrm{input}\right) \\1773 \mathrm{Stack}_\mathrm{exp weight} &=& \sum_i \left(\mathrm{exptime}_\mathrm{input} *W_\mathrm{input}\right) / \sum_\mathrm{inputs}\left(W_\mathrm{input}\right) \\1806 \mathrm{Stack}_\mathrm{value} &=& \sum_i\left(\mathrm{value}_\mathrm{input} \times W_\mathrm{input}\right) / \sum_\mathrm{inputs}\left(W_\mathrm{input}\right) \\ 1807 \mathrm{Stack}_\mathrm{exp weight} &=& \sum_i \left(\mathrm{exptime}_\mathrm{input} \times W_\mathrm{input}\right) / \sum_\mathrm{inputs}\left(W_\mathrm{input}\right) \\ 1774 1808 \end{eqnarray} 1775 1809 … … 1805 1839 attempt to identify outlier points. Again, if only one input is 1806 1840 available, that input is accepted. If there are two inputs, $A$ and 1807 $B$, then a check is made to see if $(0.5 *(\mathrm{value}_A -1808 \mathrm{value}_B))^2 > 16 *(\sigma^2_A + \sigma^2_B1809 + (0.1 * \mathrm{value}_A)^2 + (0.1 *\mathrm{value}_B)^2)$, such that1841 $B$, then a check is made to see if $(0.5 \times (\mathrm{value}_A - 1842 \mathrm{value}_B))^2 > 16 \times (\sigma^2_A + \sigma^2_B 1843 + (0.1 \times \mathrm{value}_A)^2 + (0.1 \times \mathrm{value}_B)^2)$, such that 1810 1844 the deviation of the inputs from their mean position is greater than 1811 1845 four times the sum of their measured uncertainties and a 10\% … … 1824 1858 distribution is likely to be unimodal), or if there are insufficient 1825 1859 inputs for this mixture model analysis, the input values are passed to 1826 an Olympic weighted mean calculation. We reject $20\%$ of the number1860 an Olympic \note{define} weighted mean calculation. We reject $20\%$ of the number 1827 1861 of inputs through this process. The number of bad inputs is set to 1828 $N_\mathrm{bad} = 0.2 *N_\mathrm{input} + 0.5$, with the 0.5 term1862 $N_\mathrm{bad} = 0.2 \times N_\mathrm{input} + 0.5$, with the 0.5 term 1829 1863 ensuring at least one input is rejected. This number is further 1830 1864 separated into the number of low values to exclude, $N_\mathrm{low} = … … 1843 1877 1844 1878 \begin{eqnarray} 1845 \mathrm{limit}_\mathrm{mixture\ model} &=& 4^2 *(\sigma^2_\mathrm{input} + \sigma_\mathrm{mixture\ model}^2) \\1846 \mathrm{limit}_\mathrm{default} &=& 4^2 * (\sigma^2_\mathrm{input} + (0.1 *\mathrm{value}_\mathrm{input})^2)1879 \mathrm{limit}_\mathrm{mixture\ model} &=& 4^2 \times (\sigma^2_\mathrm{input} + \sigma_\mathrm{mixture\ model}^2) \\ 1880 \mathrm{limit}_\mathrm{default} &=& 4^2 \times (\sigma^2_\mathrm{input} + (0.1 \times \mathrm{value}_\mathrm{input})^2) 1847 1881 \end{eqnarray} 1848 1882 … … 1869 1903 pixels. The ISIS kernel used in the previous step is again used to 1870 1904 determine the largest square box that does not exceed the limit of 1871 $0.25 *\sum_{x,y} kernel^2$. This square box is then convolved with1905 $0.25 \times \sum_{x,y} kernel^2$. This square box is then convolved with 1872 1906 the rejected pixel mask to reject the neighboring pixels. This final 1873 1907 list of rejected pixels is passed to the final combination, which … … 1924 1958 \begin{figure} 1925 1959 \centering 1926 \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3 775944_sci.jpg}1960 \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_sci.png} 1927 1961 \caption{Example of the stack image for skycell skycell.2047.005 1928 1962 centered at ($\alpha,\delta$) = (179.763, 32.1899) in the \zps{} … … 1940 1974 \begin{figure} 1941 1975 \centering 1942 \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3 775944_mask.jpg}1976 \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_mask.png} 1943 1977 \caption{Example of the stack mask image for skycell 1944 1978 skycell.2047.005 centered at ($\alpha,\delta$) = (179.763, … … 1954 1988 \begin{figure} 1955 1989 \centering 1956 \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3 775944_wt.jpg}1990 \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_var.png} 1957 1991 \caption{Example of the stack variance image for skycell 1958 1992 skycell.2047.005 centered at ($\alpha,\delta$) = (179.763, … … 1968 2002 \begin{figure} 1969 2003 \centering 1970 \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3 775944_num.jpg}2004 \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_num.png} 1971 2005 \caption{Example of the stack number image for skycell 1972 2006 skycell.2047.005 centered at ($\alpha,\delta$) = (179.763, … … 1982 2016 \begin{figure} 1983 2017 \centering 1984 \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3 775944_exp.jpg}2018 \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_exp.png} 1985 2019 \caption{Example of the stack exposure time image for skycell 1986 2020 skycell.2047.005 centered at ($\alpha,\delta$) = (179.763, … … 1995 2029 \begin{figure} 1996 2030 \centering 1997 \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3 775944_expwt.jpg}2031 \includegraphics[width=0.9\hsize,angle=0,clip]{images/stack_3956997_expwt.png} 1998 2032 \caption{Example of the stack weighted exposure image for skycell 1999 2033 skycell.2047.005 centered at ($\alpha,\delta$) = (179.763, … … 2071 2105 2072 2106 Although the detrending and image combination algorithms work well to 2073 produce a consistent and calibrated images, having the full PV3 data2074 set allows issues to be identified and solutions created for future 2075 improvements to the IPP pipeline. In addition, the existence ofthe2076 final calibrated catalog can be used to look for issues that appear2077 dependent on focal plane position.2107 produce consistent and calibrated images, having the PV3 processing of 2108 the full $3\pi$ data set allows issues to be identified and solutions 2109 created for future improvements to the IPP pipeline. In addition, the 2110 existence of the final calibrated catalog can be used to look for 2111 issues that appear dependent on focal plane position. 2078 2112 2079 2113 One obvious way to make use of the PV3 catalog is to do a statistical … … 2171 2205 University (ELTE), and the Los Alamos National Laboratory. 2172 2206 2207 \note{ApJ, etc latex macros have an extra comma} 2208 2173 2209 \bibliography{lib}{} 2174 2210 \bibliographystyle{apj} … … 2176 2212 2177 2213 \end{document} 2178 2179
Note:
See TracChangeset
for help on using the changeset viewer.
