- Timestamp:
- Dec 12, 2016, 6:03:54 PM (10 years ago)
- File:
-
- 1 edited
-
trunk/doc/release.2015/ps1.detrend/detrend.tex (modified) (19 diffs)
Legend:
- Unmodified
- Added
- Removed
-
trunk/doc/release.2015/ps1.detrend/detrend.tex
r39844 r39850 48 48 % list and (2) re-order the list at the bottom (and comment-out as needed) 49 49 \def\IfA{1} 50 \def\CfA{2} 51 \def\MPIA{3} 52 \def\Princeton{3} 53 \def\USNO{4} 54 \def\JHU{1} 50 \def\Princeton{2} 51 \def\STSCI{3} 52 \def\Pitt{4} 53 %\def\CfA{2} 54 %\def\MPIA{3} 55 %\def\USNO{4} 56 %\def\JHU{1} 55 57 56 58 % This example has a first author from UH: 57 59 \author{ 58 C. Z. Waters,\altaffilmark{\IfA} 59 IPP Team, 60 C.~Z. Waters,\altaffilmark{\IfA} 61 E.~A. Magnier,\altaffilmark{\IfA} 62 P.~A. Price,\altaffilmark{\Princeton} 63 H.~A. Flewelling,\altaffilmark{\IfA} 64 M.~E. Huber,\altaffilmark{\IfA} 65 W.~E. Sweeney,\altaffilmark{\IfA} 66 J.~L. Tonry, \altaffilmark{\IfA} 67 K.~C. Chambers,\altaffilmark{\IfA} 68 R.~H. Lupton,\altaffilmark{\Princeton} 69 A. Rest,\altaffilmark{\STSCI} 70 W.~M. Wood-Vasey,\altaffilmark{\Pitt} 71 PS1 Builders 60 72 %PS Builder List 61 73 % W.~S. Burgett,\altaffilmark{\IfA} 62 % K.~C. Chambers,\altaffilmark{\IfA}63 74 % L. Denneau,\altaffilmark{\IfA} 64 75 % P. Draper,\altaffilmark{\DUR} 65 % H.~A. Flewelling,\altaffilmark{\IfA}66 76 % T. Grav,\altaffilmark{\IfA} 67 77 % J. N. Heasley,\altaffilmark{\IfA} 68 78 % K. W. Hodapp,\altaffilmark{\IfA} 69 % M. E. Huber,\altaffilmark{\IfA}70 79 % R. Jedicke,\altaffilmark{\IfA} 71 80 % N. Kaiser,\altaffilmark{\IfA} 72 81 % R.-P. Kudritzki,\altaffilmark{\IfA} 73 82 % G. A. Luppino,\altaffilmark{\IfA} 74 % R. H. Lupton,\altaffilmark{\Princeton}75 % E. A. Magnier,\altaffilmark{\IfA}76 83 % N. Metcalfe,\altaffilmark{\DUH} 77 84 % D. G. Monet,\altaffilmark{\USNO} 78 85 % J.~S. Morgan,\altaffilmark{\IfA} 79 86 % P. M. Onaka,\altaffilmark{\IfA} 80 % P.~A. Price,\altaffilmark{\Princeton}81 87 % C.~W. Stubbs,\altaffilmark{\CfA} 82 % W.~E. Sweeney,\altaffilmark{\IfA}83 % J.~L. Tonry, \altaffilmark{\IfA}84 88 % R. J. Wainscoat,\altaffilmark{\IfA} and 85 89 % C. Z. Waters,\altaffilmark{\IfA} … … 89 93 \altaffiltext{\IfA}{Institute for Astronomy, University of Hawaii, 2680 Woodlawn Drive, Honolulu HI 96822} 90 94 % \altaffiltext{\CfA}{Harvard-Smithsonian Center for Astrophysics, 60 Garden Street, Cambridge, MA 02138} 91 % \altaffiltext{\Princeton}{Department of Astrophysical Sciences, Princeton University, Princeton, NJ 08544, USA} 95 \altaffiltext{\Princeton}{Department of Astrophysical Sciences, Princeton University, Princeton, NJ 08544, USA} 96 \altaffiltext{\STSCI}{Space Telescope Science Institute, 3700 San Martin Drive, Baltimore, MD 21218, USA} 97 \altaffiltext{\Pitt}{Department of Physics and Astronomy, University of Pittsburgh, Pittsburgh, PA 15260, USA} 92 98 % \altaffiltext{\USNO}{US Naval Observatory, Flagstaff Station, Flagstaff, AZ 86001, USA} 93 99 % \altaffiltext{\JHU}{Department of Physics and Astronomy, Johns Hopkins University, 3400 North Charles Street, Baltimore, MD 21218, USA} … … 95 101 \begin{abstract} 96 102 97 Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vestibulum 98 bibendum nisi id tristique posuere. Duis eu mollis nulla. Maecenas est 99 turpis, mattis tempor urna vitae, placerat rhoncus sem. Lorem ipsum 100 dolor sit amet, consectetur adipiscing elit. Sed quis velit 101 nisl. Aliquam erat volutpat. Cras lacinia, nisl tristique auctor 102 molestie, dolor nulla rhoncus purus, ac accumsan nunc nunc ac 103 nibh. Maecenas vitae mollis mauris. Ut sollicitudin pulvinar purus, 104 eget luctus lorem tincidunt vitae. Vestibulum eu mattis neque. Nulla 105 in tortor id urna dapibus gravida a vel leo. 103 the Pan-STARRS1 Science Consortium have carried out a set of imaging 104 surveys using the 1.4 giga-pixel GPC1 camera on the PS1 telescope. As 105 this camera is composed of many individual electronic readouts, and 106 covers a very large field of view, great care was taken to ensure that 107 the many instrumental effects were corrected to produce the most 108 uniform detector response possible. We present the image detrending 109 steps used as part of the processing of the data contained within the 110 public release of the Pan-STARRS1 Data Release 1 (DR1). In addition 111 to the single image processing, the methods used to transform the 112 375,573 individual exposures into a common sky-oriented grid are 113 discussed, as well as those used to produce both the image stack and 114 difference combination products. 106 115 107 116 \end{abstract} … … 131 140 Pan-STARRS 1 Science Consortium members. 132 141 133 \czwdraft{Nigel: you mention calibrating to the reference catalog without telling us134 what this is composed of (maybe this is in a different section, but would be135 nice to have some idea here).}136 137 \czwdraft{Can we get around this point by simply adding a reference to138 the paper describing the reference catalog? It's not really part of139 the detrending process, and is discussed here mostly to give an140 overview of the stages, and later to find sources of ghosts for141 masking.}142 %% \czwdraft{Nigel: you mention calibrating to the reference catalog without telling us 143 %% what this is composed of (maybe this is in a different section, but would be 144 %% nice to have some idea here).} 145 146 %% \czwdraft{Can we get around this point by simply adding a reference to 147 %% the paper describing the reference catalog? It's not really part of 148 %% the detrending process, and is discussed here mostly to give an 149 %% overview of the stages, and later to find sources of ghosts for 150 %% masking.} 142 151 143 152 The Pan-STARRS image processing pipeline (IPP) is described elsewhere … … 167 176 identified in the \ippstage{diff} stage, which takes input 168 177 \ippstage{warp} and/or \ippstage{stack} data and performs image 169 differencing \citep{HuberXXX}. Further photometry is performed in the170 \ippstage{staticsky} and \ippstage{skycal} stages, which add extended 171 source fitting to the point source photometry of objects detected in 172 the \ippstage{stack} images, and calibrate the results againstthe173 re ference catalog. The \ippstage{fullforce} stage takes the catalog174 output of the \ippstage{skycal} stage, and uses the objects detected 175 in that to perform forced photometry on the individual \ippstage{warp} 176 stage images. The details of these stages are provided in 177 \citet{MagnierXXY}.178 differencing (Section \ref{sec:diffs}). Further photometry is 179 performed in the \ippstage{staticsky} and \ippstage{skycal} stages, 180 which add extended source fitting to the point source photometry of 181 objects detected in the \ippstage{stack} images, and calibrate the 182 results against the reference catalog. The \ippstage{fullforce} stage 183 takes the catalog output of the \ippstage{skycal} stage, and uses the 184 objects detected in that to perform forced photometry on the 185 individual \ippstage{warp} stage images. The details of these stages 186 are provided in \citet{MagnierXXY}. 178 187 179 188 The same reduction procedure described above is also performed in real … … 192 201 details of the construction of those detrends in Section 193 202 \ref{sec:detrend construction}. An analysis of the algorithms used to 194 complete the \ippstage{warp} (section \ref{sec:warping}) and195 \ippstage{stack} (section \ref{sec:stacking}) stage transformations of196 the image data to from the detector frame to a common sky frame, and 197 the co-adding of those common sky frame images continues after the 198 list of detrend steps. Finally, a discussion of the remaining issues 199 and possible future improvements is presented in section 200 \ref{sec:discussion}.203 complete the \ippstage{warp} (section \ref{sec:warping}), 204 \ippstage{stack} (section \ref{sec:stacking}), and \ippstage{diff} 205 (section \ref{sec:diffs}) stage transformations of the image data to 206 from the detector frame to a common sky frame, and the co-adding of 207 those common sky frame images continues after the list of detrend 208 steps. Finally, a discussion of the remaining issues and possible 209 future improvements is presented in section \ref{sec:discussion}. 201 210 202 211 Image products presented in figures have been … … 218 227 and pixel $(1,1)$ to the top right of their position. 219 228 229 % Note taken verbatim from Ken's Paper 1. 230 \textit{Note: These papera are being placed on the arXiv.org to 231 provide crucial support information at the time of the public 232 release of Data Release 1 (DR1). We expect the arXiv versions to be 233 updated prior to submission and there could be significant 234 variations with the refereed papers. We apologize for the 235 inconvience.} 236 220 237 % Discuss 2-phase/3-phase device differnces 221 238 … … 227 244 \label{sec:detrending} 228 245 229 \czwdraft{Nigel: I forgot: when we are talking about the various bias corrections it might be230 worth pointing out that we expect these to be more of an issue in the g-band231 (and maybe r?) where read noise is a significant contributor.232 }246 %% \czwdraft{Nigel: I forgot: when we are talking about the various bias corrections it might be 247 %% worth pointing out that we expect these to be more of an issue in the g-band 248 %% (and maybe r?) where read noise is a significant contributor. 249 %% } 233 250 234 251 Ensuring a consistent and uniform detector response across the … … 727 744 better represents the true detector response. 728 745 729 \czwdraft{EAM: the flat-field construction part needs to make a clearer discussion of730 the skyflat vs the photometric correction (photflat) built initially for731 the survey vs the flat-field corrections determined in the database as part732 of ubercal (for the latter, you should just mention the concept -- it will733 also be mentioned in the calibration paper). The statement that the734 flat-field response was stable is not true since we did need 5 'seasons'.}746 %% \czwdraft{EAM: the flat-field construction part needs to make a clearer discussion of 747 %% the skyflat vs the photometric correction (photflat) built initially for 748 %% the survey vs the flat-field corrections determined in the database as part 749 %% of ubercal (for the latter, you should just mention the concept -- it will 750 %% also be mentioned in the calibration paper). The statement that the 751 %% flat-field response was stable is not true since we did need 5 'seasons'.} 735 752 736 753 In addition to this flat field applied to the individual images, the … … 740 757 survey, five separate ``seasons'' of database flat fields were needed 741 758 to ensure proper calibration. This indicates that the flat field 742 response is not completely fixed in time. 759 response is not completely fixed in time. More details on this 760 process are contained in \citet{calibration}. 743 761 744 762 \subsection{Pattern correction} … … 954 972 \includegraphics[width=1.5\hsize,angle=0,clip]{images/o5220g0025o_XY53_fringe.png} 955 973 \end{minipage} 956 \caption{Example of the \yps{} filter fringe pattern on exposure o5220g0025o OTA53 (\yps{} filter 30s). The left panel shows the OTA mosaic with all detrending except the fringe correction, while the right shows the same including the fringe correction. Both images have been smoothed with a Gaussian with $\sigma = 3$ pixels to highlight the faint and large scale fringe patterns. \czwdraft{See if there's a way to have mana produce images larger than the screen size.}} 974 \caption{Example of the \yps{} filter fringe pattern on exposure o5220g0025o OTA53 (\yps{} filter 30s). The left panel shows the OTA mosaic with all detrending except the fringe correction, while the right shows the same including the fringe correction. Both images have been smoothed with a Gaussian with $\sigma = 3$ pixels to highlight the faint and large scale fringe patterns. 975 %\czwdraft{See if there's a way to have mana produce images larger than the screen size.} 976 } 957 977 \label{fig: fringe example} 958 978 \end{figure} … … 1372 1392 \label{sec:background} 1373 1393 1374 \czwdraft{Nigel: 2.10 The background section is rather short, given all the fuss DRAVG made1375 about it. What is done to eliminate contamination by bright objects - isn't1376 there some sort of clipping? We also have a confusing number of ``bins'' in the1377 text (``These bins have 10000 .... a binned cumulative distribution is1378 generated. These bins are interpolated ... levels. Repeating this across all1379 bins ...''). There is no mention of the fact that this will subtract real1380 astrophysics backgrounds if they are on a suitably large scale, or of the fact1381 that the subtraction is not perfect (don't I remember that the stacks end up1382 with a non-zero background which scales with the number of input warps?).1383 }1384 1385 \czwdraft{Based on the wiki page on 2014-05-21 the stack background issue should be resolved.}1394 %% \czwdraft{Nigel: 2.10 The background section is rather short, given all the fuss DRAVG made 1395 %% about it. What is done to eliminate contamination by bright objects - isn't 1396 %% there some sort of clipping? We also have a confusing number of ``bins'' in the 1397 %% text (``These bins have 10000 .... a binned cumulative distribution is 1398 %% generated. These bins are interpolated ... levels. Repeating this across all 1399 %% bins ...''). There is no mention of the fact that this will subtract real 1400 %% astrophysics backgrounds if they are on a suitably large scale, or of the fact 1401 %% that the subtraction is not perfect (don't I remember that the stacks end up 1402 %% with a non-zero background which scales with the number of input warps?). 1403 %% } 1404 1405 %% \czwdraft{Based on the wiki page on 2014-05-21 the stack background issue should be resolved.} 1386 1406 1387 1407 Once all other detrending is done, the pixels from each cell are … … 1813 1833 assumed to be included in the zeropoint and transparency values. 1814 1834 1815 1816 \czwdraft{Nigel: 5. ``The ouput exposure time is set to the sum of the input exposure times.'' 1817 True, but we should note that as warps can be rejected later on in the 1818 stacking process this output time is notional in some sense. 1819 Calibration - for PV3 what photometric calibration has been used at this stage 1820 for the input warps? Should we make it clear here that pixels are not subject 1821 to the final (any?) ubercal? 1822 } 1835 The zeropoint calibration performed here uses the calibration of the 1836 individual input exposures against the reference catalog. Upon the 1837 conclusion of the survey, the entire set of detection catalogs is 1838 further re-calibrated in the ``ubercal'' process \citep{ubercal}. 1839 This produces a more consistent calibration of each exposure across 1840 the entire region of the sky imaged. This further calibration is not 1841 available at the time of stacking, and so there may be small residuals 1842 in the transparency values as a result of this \citet{calibration}. 1843 1844 %% \czwdraft{Nigel: 5. ``The ouput exposure time is set to the sum of the input exposure times.'' 1845 %% True, but we should note that as warps can be rejected later on in the 1846 %% stacking process this output time is notional in some sense. 1847 %% Calibration - for PV3 what photometric calibration has been used at this stage 1848 %% for the input warps? Should we make it clear here that pixels are not subject 1849 %% to the final (any?) ubercal? 1850 %% } 1823 1851 1824 1852 % PREPARE … … 2041 2069 2042 2070 \begin{eqnarray} 2043 \mathrm{limit}_\mathrm{mixture model} &=& 4^2 * (\sigma^2_\mathrm{input} + \sigma_\mathrm{mixturemodel}^2) \\2071 \mathrm{limit}_\mathrm{mixture\ model} &=& 4^2 * (\sigma^2_\mathrm{input} + \sigma_\mathrm{mixture\ model}^2) \\ 2044 2072 \mathrm{limit}_\mathrm{default} &=& 4^2 * (\sigma^2_\mathrm{input} + (0.1 * \mathrm{value}_\mathrm{input})^2) 2045 2073 \end{eqnarray} … … 2269 2297 \end{figure} 2270 2298 2271 2272 2273 2299 \section{Difference Images} 2300 \label{sec:diffs} 2301 2302 Constructing difference images is essentially the same as that used in 2303 the stacking process. An image is chosen as a template, another image 2304 as the input, and after matching sources to determine the scaling and 2305 transparency, convolution kernels are defined that are used to 2306 convolve one or both of the images to a target PSF. The images are 2307 then subtracted, and as they should now share a common PSF, static 2308 sources are largely subtracted (completely in an ideal case), whereas 2309 sources that are not static between the two images leave a significant 2310 remnant. More information on the difference image construction is 2311 contained in \citet{pauls_diff_paper}. The follow section contains a 2312 overview of the difference image construction used for the data in 2313 DR2. 2314 2315 The images used to construct difference images can be either 2316 individual warp skycell frames or stacked images, with support for 2317 either to be used as the template or input. In general, for 2318 differences using stacks, the deepest stack (or the only stack in the 2319 case of a warp-stack difference) is used as the template. The PV3 2320 processing used warp-stack differences of all input warps against the 2321 stack that was constructed from those inputs. The same ISIS kernels 2322 as were used in the stack image combination were again used to match 2323 the stack PSF to the input warp PSF. After convolution of the image 2324 products, the difference is constructed for both the positive (warp 2325 minus stack) and inverse (stack minus warp) to allow for the 2326 photometry of the difference image to detect sources that both rise 2327 and fall relative to the stack. Note that the convolution process 2328 grows the mask fraction of pixels relative to the warp (the largest 2329 source of masked pixels in these warp stack differences). Any pixel 2330 that after convolution has any contribution from a masked pixel is 2331 masked as well, ensuring only fully unmasked pixels are used. 2332 2333 For warp-warp differences, such as those used for the ongoing Solar 2334 System moving object search in nightly observations \citep{MOPS}, the 2335 warp that was taken first is used as the template. As there is less 2336 certainty in which of the two input images will have better seeing, a 2337 ``dual'' convolution method is used. Both inputs are convolved to a 2338 target PSF that is not identical to either input. This intermediate 2339 target is essential for the case in which the PSFs of the two inputs 2340 have been distorted in orthogonal directions. Simply convolving one 2341 to match the other would require some degree of deconvolution along 2342 one axis. As this convolution method by necessity uses more free 2343 parameters, the ISIS kernels used are chosen to be simpler than those 2344 used in the warp-stack differences. The ISIS widths are kept the same 2345 (1.5, 3.0, 6.0 pixel FWHMs), but each Gaussian kernel is constrained 2346 to only use a second-order polynomial. As with the warp-stack 2347 differences, the mask fraction grows between the input warp and the 2348 final difference image due to the convolution. For the warp-warp 2349 differences, each image mask grows based on the appropriate 2350 convolution kernel, so the final usable image area is highly dependent 2351 on ensuring that the telescope pointings are as close to identical as 2352 possible. The observing strategy to enable this is discussed in more 2353 detail in \citet{paper1}. 2274 2354 2275 2355 … … 2297 2377 There is some evidence that we have not fully identified all of these 2298 2378 crosstalk rules, based on a study of PV3 images. For example, 2299 extremely bright stars \czwdraft{exp o5677g0123o has this rule, find a 2300 magnitude} may be able to create crosstalk ghosts between the second 2379 extremely bright stars %\czwdraft{exp o5677g0123o has this rule, find a 2380 % magnitude} 2381 may be able to create crosstalk ghosts between the second 2301 2382 cell column of OTA01 and OTA21, with possibly fainter ghosts appearing 2302 2383 on OTA11. Despite the symmetry observed in the main ghost rules, … … 2329 2410 stacks if fewer pixels need to be rejected. 2330 2411 2412 % \czwdraft{one, I believe} 2331 2413 The fringe model used currently is based on only a limited number of 2332 days of data \czwdraft{one, I believe}. This means that the model 2333 calculated may not be fully sensitive to the exact spectrum of the 2334 sky. This may make the model quality differ based on the date and 2335 local time of observation. There is some evidence that the fringe 2336 model does fit some dates better than others, and so improving this by 2337 expanding the number of input exposures may improve a wider range of 2338 dates. 2414 days of data. This means that the model calculated may not be fully 2415 sensitive to the exact spectrum of the sky. This may make the model 2416 quality differ based on the date and local time of observation. There 2417 is some evidence that the fringe model does fit some dates better than 2418 others, and so improving this by expanding the number of input 2419 exposures may improve a wider range of dates. 2339 2420 % o5818g0349o is a good example of bad fringe correction. 2340 2421 … … 2349 2430 clip this peak to reduce the noise in the image space is not clear. 2350 2431 2432 2351 2433 \section{Conclusion} 2352 2434 2353 \czwdraft{Not happy with this.}2435 %\czwdraft{Not happy with this.} 2354 2436 2355 2437 The Pan-STARRS1 PV3 processing has reduced an unprecidented volume of 2356 image data, and has produced a catalog of \czwdraft{N} individual2357 measurements of \czwdraft{Y} astronomical objects. Accurately 2358 calibrating and detrending is essential to ensuring the quality of 2359 these results. The detrending process detailed here produces 2360 consistent data, despite the many individual detectors and their 2361 individual response functions.2438 image data, and has produced a catalog for the $3\Pi$ Survey 2439 containing hundreds of billions of individual measurements of 2440 \czwdraft{five billion} astronomical objects. Accurately calibrating 2441 and detrending is essential to ensuring the quality of these results. 2442 The detrending process detailed here produces consistent data, despite 2443 the many individual detectors and their individual response functions. 2362 2444 2363 2445 From these individual exposures, we are able to construct images on … … 2367 2449 data set that is ideal for use as a template for image differences. 2368 2450 2369 The Pan-STARRS1 Surveys (PS1) have been 2370 made possible through contributions by the Institute for Astronomy, the 2371 University of Hawaii, the Pan-STARRS Project Office, the Max-Planck 2372 Society and its participating institutes, the Max Planck Institute for 2373 Astronomy, Heidelberg and the Max Planck Institute for Extraterrestrial 2374 Physics, Garching, The Johns Hopkins University, Durham University, 2375 the University of Edinburgh, the Queen's University Belfast, the 2376 Harvard-Smithsonian Center for Astrophysics, the Las 2377 Cumbres Observatory Global Telescope Network Incorporated, the 2378 National Central University of Taiwan, the Space Telescope Science Institute, and the National 2379 Aeronautics and Space Administration under Grant No. NNX08AR22G issued 2380 through the Planetary Science Division of the NASA Science Mission 2381 Directorate, the National Science Foundation Grant No. AST-1238877, 2382 the University of Maryland, Eotvos Lorand University (ELTE), 2383 and the Los Alamos National Laboratory. 2451 The Pan-STARRS1 Surveys (PS1) have been made possible through 2452 contributions by the Institute for Astronomy, the University of 2453 Hawaii, the Pan-STARRS Project Office, the Max-Planck Society and its 2454 participating institutes, the Max Planck Institute for Astronomy, 2455 Heidelberg and the Max Planck Institute for Extraterrestrial Physics, 2456 Garching, The Johns Hopkins University, Durham University, the 2457 University of Edinburgh, the Queen's University Belfast, the 2458 Harvard-Smithsonian Center for Astrophysics, the Las Cumbres 2459 Observatory Global Telescope Network Incorporated, the National 2460 Central University of Taiwan, the Space Telescope Science Institute, 2461 and the National Aeronautics and Space Administration under Grant 2462 No. NNX08AR22G issued through the Planetary Science Division of the 2463 NASA Science Mission Directorate, the National Science Foundation 2464 Grant No. AST-1238877, the University of Maryland, Eotvos Lorand 2465 University (ELTE), and the Los Alamos National Laboratory. 2384 2466 2385 2467
Note:
See TracChangeset
for help on using the changeset viewer.
