Index: /trunk/doc/release.2015/ps1.analysis/Makefile =================================================================== --- /trunk/doc/release.2015/ps1.analysis/Makefile (revision 39821) +++ /trunk/doc/release.2015/ps1.analysis/Makefile (revision 39822) @@ -5,7 +5,8 @@ @echo " targets: all analysis" -all: analysis.pdf +all: analysis.pdf stages.pdf -ANALYSIS = analysis.tex +ANALYSIS = analysis.tex +STAGES = stages.tex # pics/Metadata.ps @@ -13,6 +14,8 @@ analysis.pdf: $(ANALYSIS) +stages.pdf: $(STAGES) analysis.ps: $(ANALYSIS) +stages.ps: $(STAGES) include ../Makefile.Common Index: /trunk/doc/release.2015/ps1.analysis/analysis.tex =================================================================== --- /trunk/doc/release.2015/ps1.analysis/analysis.tex (revision 39821) +++ /trunk/doc/release.2015/ps1.analysis/analysis.tex (revision 39822) @@ -1,8 +1,8 @@ -\documentclass[iop,floatfix]{emulateapj} +% \documentclass[iop,floatfix]{emulateapj} % \documentclass[iop,floatfix,onecolumn]{emulateapj} % \pdfoutput=1 % see latex.readme.txt for notes on using the PS1 template -%\documentclass[12pt,preprint]{aastex} +\documentclass[12pt,preprint]{aastex} %\documentclass[manuscript]{aastex} %\documentclass[preprint2]{aastex} Index: /trunk/doc/release.2015/ps1.analysis/stages.tex =================================================================== --- /trunk/doc/release.2015/ps1.analysis/stages.tex (revision 39821) +++ /trunk/doc/release.2015/ps1.analysis/stages.tex (revision 39822) @@ -299,11 +299,221 @@ monitoring system to visualize the data processing. +\section{Warp} + +Once astrometric and photometric calibrations have been performed, +images are geometrically transformed into a set of common pixel-grid +images with simple projections from the sky. These images, called +skycells, can then be used in subsequent stacking and difference image +analysis without concern about the astrometric transformation of an +exposure. This processing is called `warping'; the warp analysis +stage is run on all exposures before they are processed further. For +details on the warping algorithm, see \note{Waters et al paper}. + +The output products from the Warp stage consist of the skycell images +containing the signal, the variance, and the mask information. These +images have been shipped to STScI and \note{are available / will be + available} from the image extraction tools \note{in DR2}. + +\section{Stack} + +The skycell images generated by the Warp process are added together to +make deeper, higher signal-to-noise images in the Stack stage. The +stacks also fill in coverage gaps between different exposures, +resulting in an image of the sky with more uniform coverage than a +single exposure. See~\note{Waters paper} for details on the stack +combination algorithm. + +In the IPP processing, stacks may be made with various options for the +input images. During nightly science processing, the 8 exposures per +filter for each Medium Deep field are combined into a set of stacks +for that field. These so-called `nightly stacks' are used by the +transient survey projects to detect the faint supernovae, among other +transient events. For the PV3 $3\pi$ analysis, all filter images from +the $3\pi$ survey observation were stacked together to generate a +single set of images with $\sim 10 - 20\times$ the exposure of the +individual survey exposures. The signal, variance, and mask images +resulting from these deep stacks are part of the DR1 release and are +available from the image extraction tools. + +For the PV3 processing of the Medium Deep fields, stacks have been +generated for the nightly groups and for the full depth using all +exposures (deep stacks). In addition, a 'best seeing' set of stack +have been produced \note{using image quality cuts to be described}. +We have also generated out-of-season stacks for the Medium Deep +fields, in which all image not from a particular observing season for +a field are combined into a stack. These later stacks are useful as +deep templates when studying long-term transient events in the Medium +Deep fields as they are not (or less) contaminated by the flux of the +transients from a given season. + +\section{Stack Photometry} + +The stack images are generated in the Stack stage of the IPP, but the +source detection and extraction analysis of those images is deferred +until a separate stage, the Stack Photometry stage. This separation +is maintained because the stack photometry analysis is performed on +all 5 filter stack images at the same time. By deferring the +analysis, the processing system may decouple the generation of the +pixels from the source detection. This makes the sequencing of +analysis somewhat easier and less subject to blocks due to a failure +in the stacking analysis. + +The stack photometry algorithms are described in detail in +\note{Magnier et al}. In short, sources are detected in all 5 filter +images down to the $5\sigma$ significance. The collection of detected +sources is merged into a single master list. If a source is detected +in at least two bands, or only in $y$-band, then a PSF model is fitted +to the pixels of the other bands in which the source was not detected. +This forced photometry results in lower significance measurements of +the flux at the positions of objects which are thought to be real +sources, by virtue of triggering a detection in at least two bands. +The relaxed limit for $y$-band is included to allow for searches of +$y$-dropout objects: it is known that faint, high-redshift quasars may +be detected in $y$-band only. The casual user of the PV3 dataset +should be wary of sources detected only in $y$-band as these are +likely to have a higher false-positive rate than the other stack +sources. + +The stack photometry output files consist of a set of FITS tables in a +single file, with one file for each filter. Within one of these +files, the tables include: the measurements of sources based on the +PSF model; aperture like parameters such as the Petrosian flux and +radius; the convolved Galaxy model fits; the radial aperture +measurements. \note{is this list complete?} + +The stack photometry output catalogs are re-calibrated for both +photometry and astrometry in a process very similar to the Camera +calibration stage. In the case of the stack calibration, however, +each skycell is processed independently. The same reference catalog +is used for the Camera and Stack calibration stages. + +\section{Forced Warp Photometry} + +Traditionally, projects which use multiple exposures to increase the +depth and sensitivity of the observations have generated something +equivalent to the stack images produced by the IPP analysis. In +theory, the photometry of the stack images produces the `best' +photometry catalog, with best sensitivity and the best data quality at +all magnitudes (c.f, CFHT Legacy survey, COSMOS, etc). In practice, +the stack images have some significant limitations due to the +difficulty of modelling the PSF variations. This difficulty is +particularly severe for the Pan-STARRS $3\pi$ survey stacks due to the +combination of the substantial mask fraction of the individual +exposures, the large instrinsic image quality variations within a +single exposure, and the wide range of image quality conditions under +which data were obtained and used to generate the $3\pi$ PV3 stacks. + +For any specific stack, the point spread function at a particular +location is the result of the combination of the point spread +functions for those individual exposures which went into the stack at +that point. Because of the high mask fraction, the exposures which +contributed to pixels at one location may be somewhat different just a +few tens of pixels away. Because of the intrinsic variations in the +PSF across an exposure and because of the variations from exposure to +exposure, the distribution of point spread functions of the images +used at one position may be quite different from those at a nearby +location. In the end, the stack images have a effective point spread +function which is not just variable, but changing significantly on +small scales in a highly textured fashion. + +Any measurement which relies on a good knowledge of the PSF at the +location of an object either needs to determine the PSF variations +present in the stack, or the measurement will be somewhat degraded. +The highly textured PSF variations make this a very challenging +problem: not would such a PSF model require an unusually fine-grained +PSF model, there would likely not be enough PSF stars in an given +stack to determine the model at the resolution required. The IPP +photometry analysis code uses a PSF model with 2D variations using a +grid of at most $6\times 6$ samples per skycell, a number reasonably +well-matched to the density of stars at most moderate Galactic +latitudes. This scale is far too large to track the fine-grained +changes apparent in the stack images. + +Thus PSF photometry as well as convolved Galaxy models in the stack +are degraded by the PSF variations. Aperture-like measurements are in +general not as affected by the PSF variations, as long as the aperture +in question is large compared to the FWHM of the PSF. + +%% The IPP team initially explored the option of convolving each input +%% warp to a single target PSF chosen to match the worst of the input +%% images for a given stack. + +The PV3 $3\pi$ analysis solves this problem by using the sources +detected in the Stack images and performing forced photometry on the +individual warp images used to generate the stack. This analysis is +performed on all warps for a single filter as a single job, though +this is more of a bookkeepping aid as it is not necessary for the +analysis of the different warps to know about the results of the other +warps. + +In the forced warp photometry, the positions of sources are loaded +from the stack outputs. PSF stars are pre-identified and a PSF model +generated for each warp based on those stars, using the same stars for +all warps to the extent possible (PSF stars which are excessively +masked on a particular image are not used to model the PSF). The PSF +model is fitted to all of the known source positions in the warp +images. Aperture magnitudes, Kron magnitudes, and moments are also +measured at this stage for each warp. Note that the flux measurement +for a faint, but significant, source from the stack image may be at a +low significance ($< 5\sigma$) in any individual warp image; the flux +may even be negative for specific warps. When combined together, +these low-significance measurements will result in a signficant +measurement as the signal-to-noise increases by $\sqrt{N}$. + +\section{Forced Galaxy Models} + +The convolved galaxy models are also re-measured on the warp images by +the forced photometry analysis stage. In this analysis, the galaxy +models determined by the stack photometry analysis are used to seed +the analysis in the individual warps. The purpose of this analysis is +the same as the forced PSF photometry: the PSF of the stack is poorly +determined due to the masking and PSF variations in the inputs. +Without a good PSF model, the PSF-convolved galaxy models are of +limited accuracy. + +In the forced galaxy model analysis, we assume that the galaxy +position and position angle, along with the Sersic index if +appropriate, have been sufficiently well determined in the stack +analysis. In this case, the goal is to determine the best values for +the major and minor axis of the elliptical contour and at the same +time the best normalization corresponding to the best elliptical shape +(and thus the best galaxy magnitude value). + +For each warp image, the Stack value for the major and minor axis are +used as the center of a $7\times 7$ grid search of the major and minor +axis parameter values. The grid spacing is defined as a function of +the signal-to-noise of the galaxy in the stack image so that bright +galaxies are measured with a much finer grid spacing that faint +galaxies \note{need to quantify this}. For each grid point, the major +and minor axis values at that point are determined for the model. The +model is then generated and convolved with the PSF model for the warp +image at that point. The resulting model is then compared to the warp +pixel data values and the best fit normalization value is defined. +The normalization and the $\chi^2$ value for each grid point is +recorded. + +For a given galaxy, the result is a collection of $\chi^2$ values for +each of the grid points spanning all warp images. A single $\chi^2$ +grid can then be made from all warps by combining each grid point +across the warps. The combined $\chi^2$ for a single grid point is +simply the sum of all $\chi^2$ values at that point. If, for a single +warp image, the galaxy model is excessively masked, then that image +will be dropped for all grid points for that galaxy. The reduced +$\chi^2$ values can be determined by tracking the total number of warp +pixels used across all warps to generate the combined $\chi^2$ values. +From the combined grid of $\chi^2$ values, the point in the grid with +the minimum $\chi^2$ is found. Quadratic interpolation is used to +determine the major, minor axis values for the interpolated minimum +$\chi^2$ value. The errors on these two parameters is then found by +determining the contour at which the \note{reduced?} $\chi^2$ +increases by 1. + +Thus the Forced Galaxy Model analysis uses the PSF information from +each warp to determine a best set of convovled galaxy models for each +object in the stack images. \note{discuss the subset of galaxy models + and objects}. + \begin{verbatim} Outline: -Warp -Stack -Stack Photometry -Forced Warp Photometry -Forced Mean DVO Ingest Calibration