Index: trunk/doc/release.2015/ps1.analysis/Makefile =================================================================== --- trunk/doc/release.2015/ps1.analysis/Makefile (revision 39959) +++ trunk/doc/release.2015/ps1.analysis/Makefile (revision 39960) @@ -9,4 +9,6 @@ @echo "USAGE: make (target)" @echo " targets: all pdf tgz" + +test: test.pdf all: pdf tgz Index: trunk/doc/release.2015/ps1.analysis/analysis.tex =================================================================== --- trunk/doc/release.2015/ps1.analysis/analysis.tex (revision 39959) +++ trunk/doc/release.2015/ps1.analysis/analysis.tex (revision 39960) @@ -103,4 +103,19 @@ \section{INTRODUCTION}\label{sec:intro} +\begin{verbatim} +here is a list of things to do: +* clear out \note entries + * explain use of covariance + * add example for sky model + * Kaiser optimal detection reference + * find a brighter-fatter reference +* define more tests and generate examples + * simulation example of background subtraction at different densities + * real example of oversubtracted galaxy +* check all references +* fix the \code macro to work with alternate concepts + * DONE : the use of \textbf style formats was a problem : expects \textbf{foobar} +\end{verbatim} + This is the fourth in a series of seven papers describing the Pan-STARRS1 Surveys, the data reduction techiques and the resulting @@ -163,4 +178,6 @@ from early users of the data products are welcome during the submission and refereeing process.}} + +\end{document} \section{Background} @@ -227,31 +244,33 @@ When the IPP development was starting, the existing photometry -packages either did not meet the accuracy requirements or -required too much human intervention to be considered for the needs of -PS1. In the case of the SDSS Photo tool, the software was judged to -be too tightly integrated to the architecture of SDSS to be easily -re-integrated into the Pan-STARRS pipeline. A new photometry analysis -package was developed using lessons learned from the existing -photometry systems. In the process, the source analysis software was -written using the data analysis C-code library written for the IPP, -\code{psLib}. Components of the photometry code were integrated into +packages either did not meet the accuracy requirements or required too +much human intervention to be considered for the needs of PS1. In the +case of the SDSS Photo tool, the software was judged to be too tightly +integrated to the architecture of SDSS to be easily re-integrated into +the Pan-STARRS pipeline. A new photometry analysis package was +developed using lessons learned from the existing photometry systems. +In the process, the source analysis software was written using the +data analysis C-code library written for the IPP, \code{psLib} +\citep{psLib}. Components of the photometry code were integrated into the IPP's mid-level astronomy data analysis toolkit called -\code{psModules}. The resulting software, `\code{psphot}', can be used either -as a stand-alone C program, or as a set of library functions which may -be integrated into other programs - -\note{add refs to the psLib and psModules ADDs} - -The main version of \code{psphot} is a stand-alone program which is run on a -single image, or a group of related images representing the data read -from a camera in a single exposure. The images are expected to have -already been detrended so that pixel values are linearly related to -the flux. The gain may be specified by the configuration system, or a -variance image may be supplied. A mask may also be supplied to mark -good, bad, and suspect pixels. Several variants of psphot have also -been used in the PS1 PV3 analysis. \note{ppImage version is an - integrated library call} - -The version called \code{psphotStack} accepts a set of images, each +\code{psModules} \citep{psModules}. The resulting software, +`\code{psphot}', can be used either as a stand-alone C program, or as +a set of library functions which may be integrated into other programs + +% \note{add refs to the psLib and psModules ADDs} : ref to online docs? + +Several variants of \code{psphot} have been used in the PS1 PV3 +analysis. The main variant of \code{psphot} operates on a single +image, or a group of related images representing the data read from a +camera in a single exposure. The images are expected to have already +been detrended so that pixel values are linearly related to the flux. +The gain may be specified by the configuration system, or a variance +image may be supplied. A mask may also be supplied to mark good, bad, +and suspect pixels. This variant of \code{psphot} can be called as a +stand-alone program, also called \code{psphot}. In standard IPP +operations, this variant is used as a library call within the analysis +program \code{ppImage} during the \ippstage{chip} analysis stage. + +The variant called \code{psphotStack} accepts a set of images, each representing the same patch of sky in a different filter, nominally the full $grizy$ filter set for the analysis of the PS1 PV3 stack @@ -265,8 +284,8 @@ photometry. -Another version of \code{psphot} used in the PV3 analysis is called -\code{psphotFullForce}. In this version, a set of image all representing the +Another variant of \code{psphot} used in the PV3 analysis is called +\code{psphotFullForce}. In this variant, a set of image all representing the same pixels are processed together, with the positions of sources to -be analysed loaded from a supplied file. In this version of the +be analysed loaded from a supplied file. In this variant of the analysis, sources are not discovered -- only the supplied sources are considered. PSF models are determined for each exposure and the @@ -389,5 +408,5 @@ case the PSF modeling stage can be skipped. -{\bf A note on nomenclature:} +% {\bf A note on nomenclature: ???} \subsection{Image Preparation} @@ -415,20 +434,21 @@ saturated pixel. In addition, the mask pixels are used to define the pixels available during a model fit, and which should be ignored for -that specific fit (\code{MARK = 0x8000}). The initial mask, if not -supplied by the user, is constructed by default from the image by -applying three rules: 1) Pixels which are above a specified saturation -level are marked as saturated. The level is specified by the camera -format keyword \code{CELL.SATURATION}, which may specify a value or -define a header keyword which in turn specifies the value in the image -header. In the case of PS1 PV3, the header keyword \code{MAXLIN} -specifies the saturation level for each chip. \note{refer to detrend - paper here? what are GPC1 saturation levels?}. 2) Pixels which are -below a user-defined value are considered unresponsive and masked as -dead. (camera format keyword \code{CELL.BAD} = 0 for PS1 PV3). 3) -Pixels which lie outside of a user-defined coordinate window are -considered non-data pixels (eg, overscan) and are marked as invalid. -(psphot recipe keywords \code{XMIN}, \code{XMAX}, \code{YMIN}, -\code{YMAX}, all set to 0 for PS1 PV3 -- invalid pixels were specified -for PS1 PV3 with a supplied mask image, see \cite{waters2017}. +that specific fit by setting a special bit (\code{MARK = 0x8000}). +The initial mask, if not supplied by the user or library calls, is +constructed by default from the image by applying three rules: 1) +Pixels which are above a specified saturation level are marked as +saturated. The level is specified by the camera format keyword +\code{CELL.SATURATION}, which may specify a value or define a header +keyword which in turn specifies the value in the image header. In the +case of PS1 PV3, the header keyword \code{MAXLIN} specifies the +saturation level for each chip (see \cite{waters2017}). 2) Pixels +which are below a user-defined value are considered unresponsive and +masked as dead. (camera format keyword \code{CELL.BAD} = 0 for PS1 +PV3). 3) Pixels which lie outside of a user-defined coordinate window +are considered non-data pixels (eg, overscan) and are marked as +invalid. (psphot recipe keywords \code{XMIN}, \code{XMAX}, +\code{YMIN}, \code{YMAX}, all set to 0 for PS1 PV3 -- invalid pixels +were specified for PS1 PV3 with a supplied mask image, see +\cite{waters2017}. The library functions used by \code{psphot} understand two types of @@ -888,11 +908,5 @@ some of the observed PSF variations in the images -\note{need to describe fitting the pixel residual image} - \note{write up the fitting process to define the grid?} - -\notespecify the rule for the polynomial order and grid scale} - -\note{discuss the improvements in the astrometric modeling PV1 - PV3} Several analytical functions which are likely candidates to describe @@ -940,4 +954,42 @@ \end{center} \end{figure} + +Once the smooth component of the PSF has been fitted with an +analytical model, a pixel representation of the residuals is +generated. This representation is constructed as an image of the +expected residuals for any position in the image. The value of each +pixel in the image model is determined from 2D fits to the measured +residuals of the PSF stars. Pixel values in this model are only +defined for pixels with + +The residual model is calculated using the residuals for all PSF +stars. The residuals (and their errors) for each star are +renormalized by the flux of the star to put them on a consistent flux +scale. For each PSF star, all pixels within a user-specified radius +(PSF.RESIDUALS.RADIUS = 9) are selected for the measurement. For a +given pixel in the model, the pixel values from the PSF stars are +interpolated to the center of the model pixel. + +Pixels for a given star which are more than XX sigma +(PSF.RESIDUALS.NSIGMA = 3.0) deviant from the median value of the +pixels from all stars are rejected. + +If no spatial variation is allowed, the mean or median value is +calculated for the model pixel based on the user-specified mean +statistic (\code{PSF.RESIDUALS.STATISTIC = ROBUST_MEDIAN}). + +If spatial variation is requested, then the pixel values are fitted to +a linear model: +\[ +R[(x_{\rm mod},y_{\rm mod})][(x_{\rm ccd},y_{\rm ccd})] = R_o[(x_{\rm + mod},y_{\rm mod})] + R_x[(x_{\rm + mod},y_{\rm mod})] x_{\rm ccd} + R_y[(x_{\rm + mod},y_{\rm mod})] y_{\rm ccd} +\] +where $R[(x_{\rm mod},y_{\rm mod})][(x_{\rm ccd},y_{\rm ccd})]$ is the +value for model pixel $(x_{\rm mod},y_{\rm mod})$ for a star with +centroid at image pixel $(x_{\rm ccd},y_{\rm ccd})$. The parameters +$R_o, R_x, R_y$ are determined for each pixel in the model $[(x_{\rm + mod},y_{\rm mod})]$. \subsubsection{Candidate PSF Source Selection} @@ -969,10 +1021,16 @@ ignored. +% \note{is the pixel scale $0.1 \sigma_w$ or PSF_CLUMP_GRID_SCALE = 0.2?} +% psphotSourceStats sets PSF_CLUMP_GRID_SCALE to 0.1 \sigma_w^2, set +% to 0.2 by default (before \sigma_w is known). +% pmSource uses PSF_CLUMP_GRID_SCALE. note that the image is in Mxx +% (\sigma_x^2) not \sigma_x,\sigma_y) + +\note{re-work wording above reflecting comment above} + Once a peak has been detected in this plane, the centroid and second -moments of this peak are measured. All sources which land within XXX -$\sigma$ of this centroid are selected as likely PSF-like sources in -the image. - -\note{work out the logic for selecting the PSF stars} +moments of this peak are measured. All sources which land within 2 +pixels of this centroid are selected as candidate PSF sources in the +image. \begin{figure}[htbp] @@ -1021,11 +1079,30 @@ sources and ignored in the later PSF model fitting stages. -%% table of orders: -%% N stars | max order | max Ncells -%% 16 | 1; // 4 cells, 4 per cell -%% 54 | 2; // 9 cells, 6 per cell -%% 128 | 3; // 16 cells, 8 per cell -%% 300 | 4; // 25 cells, 12 per cell -%% 576 | 5; // 36 cells, 16 per cell +The order of the fit or number of grid samples is modified if the +number of stars available for the fit is insufficient to justify the +highest value. Regardness of the requested order, if the number of +stars is below the following limits, the order is limited as shown in +Table~\ref{tab:psf.order.nstars}. Note that the number of grid cells +in one dimension is one greater than the equivalent polynomial order. + +\begin{table} +\caption{\label{tab:psf.order.nstars} Minimum number of stars required + for a given order of the PSF 2D variations.}\vspace{-0.5cm} +\begin{center} +\begin{tabular}{lcl} +\hline +\hline +{\bf Minimum Number of Stars} & {\bf Order} & {\bf Number of Grid Cells} \\ +\hline + 16 & 1 & 4 & 4 \\ + 54 & 2 & 9 & 6 \\ +128 & 3 & 16 & 8 \\ +300 & 4 & 25 & 12 \\ +576 & 5 & 36 & 16 \\ +\hline +\end{tabular} +\end{center} +\end{table} + All of the PSF-candidate sources are then re-fitted using the PSF