Changeset 39822

Timestamp:

Nov 22, 2016, 4:23:59 PM (10 years ago)

Author:

eugene

Message:

updates to stages.tex

Location:

trunk/doc/release.2015/ps1.analysis

Files:

• Makefile (modified) (2 diffs)
• analysis.tex (modified) (1 diff)
• stages.tex (modified) (1 diff)

trunk/doc/release.2015/ps1.analysis/Makefile

@@ -5 +5 @@
         @echo "  targets:  all analysis"
 
-all: analysis.pdf
+all: analysis.pdf stages.pdf
 
-ANALYSIS = analysis.tex
+ANALYSIS = analysis.tex 
+STAGES = stages.tex 
 
 #       pics/Metadata.ps 
@@ -13 +14 @@
 
 analysis.pdf: $(ANALYSIS)
+stages.pdf: $(STAGES)
 
 analysis.ps: $(ANALYSIS)
+stages.ps: $(STAGES)
 
 include ../Makefile.Common

trunk/doc/release.2015/ps1.analysis/analysis.tex

@@ -1 @@
-\documentclass[iop,floatfix]{emulateapj}
+% \documentclass[iop,floatfix]{emulateapj}
 % \documentclass[iop,floatfix,onecolumn]{emulateapj}
 % \pdfoutput=1
 
 % see latex.readme.txt for notes on using the PS1 template
-%\documentclass[12pt,preprint]{aastex}
+\documentclass[12pt,preprint]{aastex}
 %\documentclass[manuscript]{aastex}
 %\documentclass[preprint2]{aastex}

trunk/doc/release.2015/ps1.analysis/stages.tex

@@ -299 +299 @@
 monitoring system to visualize the data processing.
 
+\section{Warp}
+
+Once astrometric and photometric calibrations have been performed,
+images are geometrically transformed into a set of common pixel-grid
+images with simple projections from the sky.  These images, called
+skycells, can then be used in subsequent stacking and difference image
+analysis without concern about the astrometric transformation of an
+exposure.  This processing is called `warping'; the warp analysis
+stage is run on all exposures before they are processed further.  For
+details on the warping algorithm, see \note{Waters et al paper}.
+
+The output products from the Warp stage consist of the skycell images
+containing the signal, the variance, and the mask information.  These
+images have been shipped to STScI and \note{are available / will be
+  available} from the image extraction tools \note{in DR2}.
+
+\section{Stack}
+
+The skycell images generated by the Warp process are added together to
+make deeper, higher signal-to-noise images in the Stack stage.  The
+stacks also fill in coverage gaps between different exposures,
+resulting in an image of the sky with more uniform coverage than a
+single exposure.  See~\note{Waters paper} for details on the stack
+combination algorithm.
+
+In the IPP processing, stacks may be made with various options for the
+input images.  During nightly science processing, the 8 exposures per
+filter for each Medium Deep field are combined into a set of stacks
+for that field.  These so-called `nightly stacks' are used by the
+transient survey projects to detect the faint supernovae, among other
+transient events.  For the PV3 $3\pi$ analysis, all filter images from
+the $3\pi$ survey observation were stacked together to generate a
+single set of images with $\sim 10 - 20\times$ the exposure of the
+individual survey exposures.  The signal, variance, and mask images
+resulting from these deep stacks are part of the DR1 release and are
+available from the image extraction tools.
+
+For the PV3 processing of the Medium Deep fields, stacks have been
+generated for the nightly groups and for the full depth using all
+exposures (deep stacks).  In addition, a 'best seeing' set of stack
+have been produced \note{using image quality cuts to be described}.
+We have also generated out-of-season stacks for the Medium Deep
+fields, in which all image not from a particular observing season for
+a field are combined into a stack.  These later stacks are useful as
+deep templates when studying long-term transient events in the Medium
+Deep fields as they are not (or less) contaminated by the flux of the
+transients from a given season.
+
+\section{Stack Photometry}
+
+The stack images are generated in the Stack stage of the IPP, but the
+source detection and extraction analysis of those images is deferred
+until a separate stage, the Stack Photometry stage.  This separation
+is maintained because the stack photometry analysis is performed on
+all 5 filter stack images at the same time.  By deferring the
+analysis, the processing system may decouple the generation of the
+pixels from the source detection.  This makes the sequencing of
+analysis somewhat easier and less subject to blocks due to a failure
+in the stacking analysis.
+
+The stack photometry algorithms are described in detail in
+\note{Magnier et al}.  In short, sources are detected in all 5 filter
+images down to the $5\sigma$ significance.  The collection of detected
+sources is merged into a single master list.  If a source is detected
+in at least two bands, or only in $y$-band, then a PSF model is fitted
+to the pixels of the other bands in which the source was not detected.
+This forced photometry results in lower significance measurements of
+the flux at the positions of objects which are thought to be real
+sources, by virtue of triggering a detection in at least two bands.
+The relaxed limit for $y$-band is included to allow for searches of
+$y$-dropout objects: it is known that faint, high-redshift quasars may
+be detected in $y$-band only.  The casual user of the PV3 dataset
+should be wary of sources detected only in $y$-band as these are
+likely to have a higher false-positive rate than the other stack
+sources.
+
+The stack photometry output files consist of a set of FITS tables in a
+single file, with one file for each filter.  Within one of these
+files, the tables include: the measurements of sources based on the
+PSF model; aperture like parameters such as the Petrosian flux and
+radius; the convolved Galaxy model fits; the radial aperture
+measurements.  \note{is this list complete?}
+
+The stack photometry output catalogs are re-calibrated for both
+photometry and astrometry in a process very similar to the Camera
+calibration stage.  In the case of the stack calibration, however,
+each skycell is processed independently.  The same reference catalog
+is used for the Camera and Stack calibration stages.
+
+\section{Forced Warp Photometry}
+
+Traditionally, projects which use multiple exposures to increase the
+depth and sensitivity of the observations have generated something
+equivalent to the stack images produced by the IPP analysis.  In
+theory, the photometry of the stack images produces the `best'
+photometry catalog, with best sensitivity and the best data quality at
+all magnitudes (c.f, CFHT Legacy survey, COSMOS, etc).  In practice,
+the stack images have some significant limitations due to the
+difficulty of modelling the PSF variations.  This difficulty is
+particularly severe for the Pan-STARRS $3\pi$ survey stacks due to the
+combination of the substantial mask fraction of the individual
+exposures, the large instrinsic image quality variations within a
+single exposure, and the wide range of image quality conditions under
+which data were obtained and used to generate the $3\pi$ PV3 stacks.
+
+For any specific stack, the point spread function at a particular
+location is the result of the combination of the point spread
+functions for those individual exposures which went into the stack at
+that point.  Because of the high mask fraction, the exposures which
+contributed to pixels at one location may be somewhat different just a
+few tens of pixels away.  Because of the intrinsic variations in the
+PSF across an exposure and because of the variations from exposure to
+exposure, the distribution of point spread functions of the images
+used at one position may be quite different from those at a nearby
+location.  In the end, the stack images have a effective point spread
+function which is not just variable, but changing significantly on
+small scales in a highly textured fashion.  
+
+Any measurement which relies on a good knowledge of the PSF at the
+location of an object either needs to determine the PSF variations
+present in the stack, or the measurement will be somewhat degraded.
+The highly textured PSF variations make this a very challenging
+problem: not would such a PSF model require an unusually fine-grained
+PSF model, there would likely not be enough PSF stars in an given
+stack to determine the model at the resolution required.  The IPP
+photometry analysis code uses a PSF model with 2D variations using a
+grid of at most $6\times 6$ samples per skycell, a number reasonably
+well-matched to the density of stars at most moderate Galactic
+latitudes.  This scale is far too large to track the fine-grained
+changes apparent in the stack images.
+
+Thus PSF photometry as well as convolved Galaxy models in the stack
+are degraded by the PSF variations.  Aperture-like measurements are in
+general not as affected by the PSF variations, as long as the aperture
+in question is large compared to the FWHM of the PSF.
+
+%% The IPP team initially explored the option of convolving each input
+%% warp to a single target PSF chosen to match the worst of the input
+%% images for a given stack.  
+
+The PV3 $3\pi$ analysis solves this problem by using the sources
+detected in the Stack images and performing forced photometry on the
+individual warp images used to generate the stack.  This analysis is
+performed on all warps for a single filter as a single job, though
+this is more of a bookkeepping aid as it is not necessary for the
+analysis of the different warps to know about the results of the other
+warps.
+
+In the forced warp photometry, the positions of sources are loaded
+from the stack outputs.  PSF stars are pre-identified and a PSF model
+generated for each warp based on those stars, using the same stars for
+all warps to the extent possible (PSF stars which are excessively
+masked on a particular image are not used to model the PSF).  The PSF
+model is fitted to all of the known source positions in the warp
+images.  Aperture magnitudes, Kron magnitudes, and moments are also
+measured at this stage for each warp.  Note that the flux measurement
+for a faint, but significant, source from the stack image may be at a
+low significance ($< 5\sigma$) in any individual warp image; the flux
+may even be negative for specific warps.  When combined together,
+these low-significance measurements will result in a signficant
+measurement as the signal-to-noise increases by $\sqrt{N}$.  
+
+\section{Forced Galaxy Models}
+
+The convolved galaxy models are also re-measured on the warp images by
+the forced photometry analysis stage.  In this analysis, the galaxy
+models determined by the stack photometry analysis are used to seed
+the analysis in the individual warps.  The purpose of this analysis is
+the same as the forced PSF photometry: the PSF of the stack is poorly
+determined due to the masking and PSF variations in the inputs.
+Without a good PSF model, the PSF-convolved galaxy models are of
+limited accuracy.  
+
+In the forced galaxy model analysis, we assume that the galaxy
+position and position angle, along with the Sersic index if
+appropriate, have been sufficiently well determined in the stack
+analysis.  In this case, the goal is to determine the best values for
+the major and minor axis of the elliptical contour and at the same
+time the best normalization corresponding to the best elliptical shape
+(and thus the best galaxy magnitude value).
+
+For each warp image, the Stack value for the major and minor axis are
+used as the center of a $7\times 7$ grid search of the major and minor
+axis parameter values.  The grid spacing is defined as a function of
+the signal-to-noise of the galaxy in the stack image so that bright
+galaxies are measured with a much finer grid spacing that faint
+galaxies \note{need to quantify this}.  For each grid point, the major
+and minor axis values at that point are determined for the model.  The
+model is then generated and convolved with the PSF model for the warp
+image at that point.  The resulting model is then compared to the warp
+pixel data values and the best fit normalization value is defined.
+The normalization and the $\chi^2$ value for each grid point is
+recorded.  
+
+For a given galaxy, the result is a collection of $\chi^2$ values for
+each of the grid points spanning all warp images.  A single $\chi^2$
+grid can then be made from all warps by combining each grid point
+across the warps.  The combined $\chi^2$ for a single grid point is
+simply the sum of all $\chi^2$ values at that point.  If, for a single
+warp image, the galaxy model is excessively masked, then that image
+will be dropped for all grid points for that galaxy.  The reduced
+$\chi^2$ values can be determined by tracking the total number of warp
+pixels used across all warps to generate the combined $\chi^2$ values.
+From the combined grid of $\chi^2$ values, the point in the grid with
+the minimum $\chi^2$ is found.  Quadratic interpolation is used to
+determine the major, minor axis values for the interpolated minimum
+$\chi^2$ value.  The errors on these two parameters is then found by
+determining the contour at which the \note{reduced?} $\chi^2$
+increases by 1.  
+
+Thus the Forced Galaxy Model analysis uses the PSF information from
+each warp to determine a best set of convovled galaxy models for each
+object in the stack images.  \note{discuss the subset of galaxy models
+  and objects}.
+
 \begin{verbatim}
 Outline:
-Warp
-Stack
-Stack Photometry
-Forced Warp Photometry
-Forced Mean
 DVO Ingest
 Calibration