Index: /trunk/doc/release.2015/ps1.analysis/analysis.tex =================================================================== --- /trunk/doc/release.2015/ps1.analysis/analysis.tex (revision 40593) +++ /trunk/doc/release.2015/ps1.analysis/analysis.tex (revision 40594) @@ -93,8 +93,8 @@ sources have been automatically detected and characterized by the Pan-STARRS Image Processing Pipeline photometry software, -\code{psphot}. This fast, automatic, and reliable software was +\ippprog{psphot}. This fast, automatic, and reliable software was developed for the Pan-STARRS project, but is easily adaptable to images from other telescopes. We describe the analysis of the -astronomical sources by \code{psphot} in general as well as for the +astronomical sources by \ippprog{psphot} in general as well as for the specific case of the 3rd processing version used for the first public release of the Pan-STARRS $3\pi$ survey data. @@ -108,13 +108,10 @@ % \begin{verbatim} % here is a list of things to do: -% * clear out \note entries -% * explain use of covariance -% * add example for sky model -% * Kaiser optimal detection reference -% * define more tests and generate examples +% * explain use of covariance +% * add example for sky model +% * define more tests and generate examples % * simulation example of background subtraction % at different densities % * real example of oversubtracted galaxy -% * check all references % \end{verbatim} @@ -155,5 +152,5 @@ Release 1 (DR1) on 16 December 2016. DR1 contains the results of the third full reduction of the Pan-STARRS $3\pi$ Survey archival data, -idenfied as PV3. Previous reductions \citep[PV0, PV1, PV2; +identified as PV3. Previous reductions \citep[PV0, PV1, PV2; see][]{magnier2017.datasystem} were used internally for pipeline optimization and the development of the initial photometric and @@ -169,5 +166,5 @@ This is the fourth in a series of seven papers describing the -Pan-STARRS1 Surveys, the data reduction techiques and the resulting +Pan-STARRS1 Surveys, the data reduction techniques and the resulting data products. This paper (Paper IV) describes the details of the source detection and photometry, including point-spread-function and @@ -188,5 +185,5 @@ %Pan-STARRS Data Processing Stages \citet[][Paper II]{magnier2017.datasystem} -describes how the various data processing stages are organised and implemented +describes how the various data processing stages are organized and implemented in the Imaging Processing Pipeline (IPP), including details of the the processing database which is a critical element in the IPP infrastructure . @@ -305,10 +302,10 @@ were integrated into the IPP's mid-level astronomy data analysis toolkit called \code{psModules} \citep{magnier2017.datasystem}. The -resulting software, `\code{psphot}', can be used either as a +resulting software, `\ippprog{psphot}', can be used either as a stand-alone C program, or as a set of library functions which may be integrated into other programs -Several variants of \code{psphot} have been used in the PS1 PV3 -analysis. The main variant of \code{psphot} operates on a single +Several variants of \ippprog{psphot} have been used in the PS1 PV3 +analysis. The main variant of \ippprog{psphot} operates on a single image, or a group of related images representing the data read from a camera in a single exposure. The images are expected to have already @@ -316,15 +313,15 @@ The gain may be specified by the configuration system, or a variance image may be supplied. A mask may also be supplied to mark good, bad, -and suspect pixels. This variant of \code{psphot} can be called as a -stand-alone program, also called \code{psphot}. In standard IPP +and suspect pixels. This variant of \ippprog{psphot} can be called as a +stand-alone program, also called \ippprog{psphot}. In standard IPP operations, this variant is used as a library call within the analysis -program \code{ppImage} during the \ippstage{chip} analysis stage. - -The variant called \code{psphotStack} accepts a set of images, each +program \ippprog{ppImage} during the \ippstage{chip} analysis stage. + +The variant called \ippprog{psphotStack} accepts a set of images, each representing the same patch of sky in a different filter, nominally the full $grizy$ filter set for the analysis of the PS1 PV3 stack images, though where insufficient data were available in a given filter, a subset of these filters was processed as a group. As -discussed in detail below, the \code{psphotStack} analysis includes the +discussed in detail below, the \ippprog{psphotStack} analysis includes the capability of measuring forced PSF photometry in some filter images based on the position of sources detected in the other filters. It @@ -333,8 +330,8 @@ photometry. -Another variant of \code{psphot} used in the PV3 analysis is called -\code{psphotFullForce}. In this variant, a set of image all representing the +Another variant of \ippprog{psphot} used in the PV3 analysis is called +\ippprog{psphotFullForce}. In this variant, a set of image all representing the same pixels are processed together, with the positions of sources to -be analysed loaded from a supplied file. In this variant of the +be analyzed loaded from a supplied file. In this variant of the analysis, sources are not discovered -- only the supplied sources are considered. PSF models are determined for each exposure and the @@ -346,43 +343,45 @@ \section{\nocode{psphot} Design Goals} -\code{psphot} has a number of important requirements that it must meet, and a -number of design goals which we believe will help to make usable in a -wide range of circumstances. The critical requirements of the -Pan-STARRS IPP which drive the requirements for \code{psphot}: - -\begin{itemize} -\item {\bf 10 millimagnitude photometric accuracy}. For \code{psphot}, this - implies that the measured photometry of stellar sources must be - substantially better than this 10 mmag since the photometry error - per image is combined with an error in the flat-field calibration - and an error in measuring the atmospheric effects. We have set a - goal for \code{psphot} of 3mmag photometric consistency for bright stars - between pairs of images obtained in photometric conditions at the - same pointing, ie to remove sensitivity to flat-field errors. This - goal splits the difference between the three main contributors and - still allows some leeway. This requirement must be met for - well-sampled images and images with only modest undersampling. - -\item {\bf 10 milliarcsecond astrometric accuracy}. Relative - astrometric calibration depends on the consistency of the individual - measurements. The measurements from \code{psphot} must be sufficiently - representative of the true source position to enable astrometric - calibration at the 10mas level. The error in the individual - measurements will be folded together with the errors introduced by - the optical system, the effects of seeing, and by the available - reference catalogs. We have set a goal for \code{psphot} of 5mas - consistency between the true source postion and the measured - position given reasonable PSF variations under simulations. This - level must be reached for images with 250 mas pixels, implying - \code{psphot} must introduce measurement errors less than 1/50th of a - pixel. The choice of 32 bit floating point data values for the - source centroids places a numerical limit of 1e-7 on the accuracy of - a pixel relative to the size of a chip (since a single data value is - used for X or Y). For the $4800^2$ GPC chips, this yields a limit - of about 0.25 milliarcsecond. -\end{itemize} - -The design goals for \code{psphot} are chosen to make the program flexible, -general, and able to meet the unknown usages cases future projects may +% \subsection{Astronomy Measurement Goals} + +\ippprog{psphot} has a number of important requirements that it must +meet, and a number of design goals which we believe will help to make +it usable in a wide range of circumstances. The critical +astronomy-driven measurement goals of the Pan-STARRS project, which +drive the design of \ippprog{psphot}, are the photometric accuracy +goal (10 millimagntudes) and the astrometric accuracy goal (10 +milliarcseconds). For \ippprog{psphot}, the photometry accuracy goal +implies that the measured photometry of stellar sources must be +substantially better than this 10 mmag goal since the photometry error +per image is combined with an error in the flat-field calibration and +an error in measuring the atmospheric effects. We have set a goal for +\ippprog{psphot} of 3mmag photometric consistency for bright stars +between pairs of images obtained in photometric conditions at the same +pointing, ie to remove sensitivity to flat-field errors. This goal +splits the difference between the three main contributors and still +allows some leeway. This requirement must be met for well-sampled +images and images with only modest undersampling. + +The relative astrometric calibration depends on the consistency of the +individual measurements. The measurements from \ippprog{psphot} must +be sufficiently representative of the true source position to enable +astrometric calibration at the 10mas level. The error in the +individual measurements will be folded together with the errors +introduced by the optical system, the effects of seeing, and by the +available reference catalogs. We have set a goal for \ippprog{psphot} +of 5mas consistency between the true source postion and the measured +position given reasonable PSF variations under simulations. This +level must be reached for images with 250 mas pixels, implying +\ippprog{psphot} must introduce measurement errors less than 1/50th of +a pixel. The choice of 32 bit floating point data values for the +source centroids places a numerical limit of 1e-7 on the accuracy of a +pixel relative to the size of a chip (since a single data value is +used for X or Y). For the $4800^2$ GPC chips, this yields a limit of +about 0.25 milliarcsecond. + +% \subsection{Software System Goals} + +The design goals for \ippprog{psphot} are chosen to make the program flexible, +general, and able to meet the unknown usage cases future projects may require: @@ -396,22 +395,22 @@ naturally incorporate 2-D variations. -\item {\bf Flexible non-PSF models} \code{psphot} must be able to represent +\item {\bf Flexible non-PSF models} \ippprog{psphot} must be able to represent PSF-like sources as well as non-PSF sources (e.g., galaxies). It must be easy to add new source models as interesting representations of sources are invented. -\item {\bf Clean code base} \code{psphot} should incorporate a high-degree of +\item {\bf Clean code base} \ippprog{psphot} should incorporate a high-degree of abstraction and encapsulation so that changes to the code structure can be performed without pulling the code apart and starting from scratch. -\item {\bf PSF validity tests} \code{psphot} should include the ability to - choose different types of PSF models for diffent situations, or to +\item {\bf PSF validity tests} \ippprog{psphot} should include the ability to + choose different types of PSF models for different situations, or to provide the user with methods for assessing the different PSF models. -\item {\bf Careful systematic corrections} \code{psphot} must carefully +\item {\bf Careful systematic corrections} \ippprog{psphot} must carefully measure and correct for the photometric and astrometric trends introduced by using analytical PSF models. -\item {\bf User Configurable} \code{psphot} should allow users to change the +\item {\bf User Configurable} \ippprog{psphot} should allow users to change the options easily and to allow different approaches to the analysis. @@ -422,5 +421,6 @@ \subsection{Overview} -The \code{psphot} analysis is divided into several major stages: +The \ippprog{psphot} analysis is divided into several major stages, as +listed below. \begin{enumerate} @@ -451,5 +451,13 @@ \end{enumerate} -\code{psphot} is highly configurable. Users may choose via the configuration +Table~\ref{tab:measurements} lists the types of +analyses performed by \ippprog{psphot}, specifying which of the +\ippprog{psphot} usage cases performs the given analysis. The table +also provides a reference to the section of this paper in which the +analysis is described. Not all analyses are relevant to all sources +in all images. The table identifies thoses cases where the analyses +are applied to only a subset of all sources. + +\ippprog{psphot} is highly configurable. Users may choose via the configuration system which of the above analyses are performed. This is useful for testing, but also allows for specialized use cases. For example, the @@ -457,7 +465,46 @@ case the PSF modeling stage can be skipped. -% {\bf A note on nomenclature: ???} +\begin{table*} +\begin{center} +\footnotesize +\caption{\label{tab:measurements} \nocode{psphot} measurements performed} % \vspace{-0.5cm} +\begin{tabular}{lccccll} +\hline +\hline +{\bf Measurement} & {\bf Camera} & {\bf Stack} & {\bf Forced Warp} & {\bf Diff} & {\bf Section} & {\bf Which} \\ +\hline + Background Subtraction & Y & Y & Y & N$^1$ & \ref{sec:image.preparation} & N/A \\ + Peaks & Y & Y & N & Y & \ref{sec:peaks} & All \\ + Footprints & Y & Y & N & Y & \ref{sec:footprints} & All \\ + Moments & Y & Y & Y & Y & \ref{sec:moments} & All \\ + PSF Model & Y & Y & Y & N$^2$ & \ref{sec:PSF.Model} & Uses bright, unsat. stars \\ + Bright Star Profile & Y & Y & N & Y & \ref{sec:very.bright.star} & Saturated Stars \\ + Radial Profiles v1 & Y & Y & N & Y & \ref{sec:radial.profile} & All \\ + Kron Fluxes & Y & Y & Y & Y & \ref{sec:kron.mags} & All \\ + Source-Size Tests & Y & Y & N & Y & \ref{sec:source.size} & All \\ + Non-Linear PSF Fits & Y & Y & N & N & \ref{sec:nonlinear.psf.model} & $S/N > 20$ \\ + Unconvolved Galaxy Model & Y & Y & N & N & \ref{sec:nonlinear.galaxy.model} & $S/N > 20$, extended \\ + Unconvolved Streak Model & N & N & N & Y & \ref{sec:nonlinear.galaxy.model} & $S/N > 20$, extended \\ + Linear PSF Fits & Y & Y & Y & Y & \ref{sec:faint.psf.model} & All \\ + Radial Profiles v2 & Y & Y & N & Y & \ref{sec:radial.profile.v2} & Gal. Latitude Cut \\ + Petrosian Fluxes & N & Y & Y & N & \ref{sec:petrosian} & Gal. Latitude Cut \\ + Convolved Galaxy Models & N & Y & N & N & \ref{sec:galaxy.conv.fit} & Gal. Latitude Cut, mag cut \\ + Fixed Aperture Photometry & N & Y & Y & N & \ref{sec:fixed.aperture.photom} & All \\ + Convolved, Fixed Apertures & N & Y & N & N & \ref{sec:fixed.aperture.photom} & All \\ + Aperture Corrections & Y & Y & Y & N & \ref{sec:aperture.correction} & All \\ + Forced PSF Fluxes & N & N & Y & N & \ref{sec:psf.forced.fit} & All \\ + Forced Galaxy Models & N & N & Y & N & \ref{sec:galaxy.forced.fit} & Have Stack Galaxy Models \\ + Lensing Parameters & N & Y & Y & N & & All \\ +\hline +\multicolumn{5}{l}{$^1$ Background subtraction is performed by {\tt ppSub} before calling {\tt psphot}} \\ +\multicolumn{5}{l}{$^2$ PSF modeling is perform by {\tt ppSub} on the input warps before calling {\tt psphot}} \\ +\end{tabular} +\end{center} +\end{table*} + +% \subsection{Output Formats} \subsection{Image Preparation} +\label{sec:image.preparation} The first step is to prepare the image for detection of the @@ -473,5 +520,5 @@ references to the mask and variance are provided in the configuration information. As in the stand-alone C-program, the variance and mask may -be constructed automatically by \code{psphot}. +be constructed automatically by \ippprog{psphot}. The mask is represented as a 16-bit integer image in which a value of @@ -482,6 +529,6 @@ other circumstances, it may be useful to know the flux value of the saturated pixel. In addition, the mask pixels are used to define the -pixels available during a model fit, and which should be ignored for -that specific fit by setting a special bit (\code{MARK = 0x8000}). +pixels available during a model fit; those which should be ignored for +that specific fit are `marked' by setting a special bit (\code{MARK = 0x8000}). The initial mask, if not supplied by the user or library calls, is constructed by default from the image by applying three rules: 1) @@ -495,15 +542,15 @@ masked as dead. (camera format keyword \code{CELL.BAD} = 0 for PS1 PV3). 3) Pixels which lie outside of a user-defined coordinate window -are considered non-data pixels (eg, overscan) and are marked as -invalid. (\code{psphot} recipe keywords \code{XMIN}, \code{XMAX}, +are considered non-data pixels (\eg, overscan) and are marked as +invalid. (\ippprog{psphot} recipe keywords \code{XMIN}, \code{XMAX}, \code{YMIN}, \code{YMAX}, all set to 0 for PS1 PV3 -- invalid pixels were specified for PS1 PV3 with a supplied mask image \citep[see][]{waters2017}. -The library functions used by \code{psphot} understand two types of +The library functions used by \ippprog{psphot} understand two types of masked pixels: ``bad'' and ``suspect''. Bad pixels are those which should not be used in any operations, while suspect pixels are those for which the reported signal may be contaminated or biased, but may -be useable in some contexts. For example, a pixel with poor charge +be usable in some contexts. For example, a pixel with poor charge transfer efficiency is likely to be too untrustworthy to use in any circumstance, while a pixel in which persistence ghosts have been @@ -528,5 +575,5 @@ SAT & 0x0020 & The pixel is saturated. \\ LOW & 0x0040 & The pixel has a lower value than expected. \\ - SUSPECT & 0x0080 & The pixel is suspected of being bad. \\ + SUSPECT & 0x0080 & The pixel is suspected of being bad$^1$. \\ BURNTOOL & 0x0080 & The pixel contain an burntool repaired streak. \\ CR & 0x0100 & A cosmic ray is present. \\ @@ -539,4 +586,7 @@ MARK & 0x8000 & An internal flag for temporarily marking a pixel. \\ \hline +\multicolumn{3}{l}{$^1$ The SUSPECT bit is generic and only + used if a specific reason cannot be identified.}\\ +\multicolumn{3}{l}{It is overloaded on the same bit as BURNTOOL.}\\ \end{tabular} \end{center} @@ -560,5 +610,5 @@ Some image processing steps introduce cross-correlation between pixel fluxes. An obvious case is smoothing, but geometric transformations -which redistibute fractional flux between neighboring pixels also +which redistribute fractional flux between neighboring pixels also introduces cross-correlations. In the noise model, it is necessary to track the impact of the cross correlations on the per-pixel variance. @@ -569,12 +619,12 @@ covariance image is prohibitive. -\note{describe the way we handle covariance} +% \note{describe the way we handle covariance} Before sources are detected in the image, a model of the background is subtracted. The image is divided into a grid of background points -with a spacing defined by the \code{psphot} recipe values +with a spacing defined by the \ippprog{psphot} recipe values \code{BACKGROUND.XBIN, BACKGROUND.YBIN}, set to 400 pixels for PS1 -PV3. Superpixels of size \code{BACKGROUND.XSAMPLE, - BACKGROUND.YSAMPLE} ($2 \times 2$ for PS1 PV3) times larger than +PV3. Superpixels of size \code{BACKGROUND.XSAMPLE, BACKGROUND.YSAMPLE} +($2 \times 2$ for PS1 PV3) times larger than this spacing are used to measure the local background for each background grid point, thus over-sampling the background spatial @@ -600,7 +650,8 @@ suffering bias from the stellar flux. We thus perform a second Gaussian fit using an asymmetric subset of the histogram pixels, -fitting those histogram bins which are left of the peak but above 25\% of -the peak value, or right of the peak but above 50\% of the peak -value. +fitting those histogram bins which are left of the peak but for which +the bin value is greater than 25\% of the peak bin, or right of the +peak but only using those bins for whch the bin value is greater than +50\% of the peak bin value. If the fit to the asymmetric lower fraction of the curve is less than @@ -614,7 +665,8 @@ standard deviation image are kept in memory from which the values of \code{SKY} and \code{SKY_SIGMA} are calculated for each source in the -output catalog. See also the discussion in \cite{waters2017}. - -\note{give examples with simulations and show examples of over-subtraction} +output catalog. For more details of the background subtraction, see +the discussion in Section~2.7 of \cite{waters2017}. + +% \note{give examples with simulations and show examples of over-subtraction} \subsection{Initial Source Detection} @@ -703,5 +755,5 @@ The resulting peak position, ($x_{min}, y_{min}$), is used as the default starting coordinate for the source. Later in the -\code{psphot} analysis, improved measurements of the source positions +\ippprog{psphot} analysis, improved measurements of the source positions are calculated as discussed below. @@ -716,8 +768,9 @@ \subsubsection{Footprints} +\label{sec:footprints} The peaks detected in the image may correspond to real sources, but they may also correspond to noise fluctuations, especially in the -wings of bright stars. \code{psphot} attempts to identify peaks which may be +wings of bright stars. \ippprog{psphot} attempts to identify peaks which may be formally significant, but are not locally significant. It first generates a set of ``footprints'', contiguous collections of pixels in @@ -823,6 +876,6 @@ To choose the value of $\sigma_w$, we try a sequence of values -spanning a range guaranateed to contain any reasonable seeing values. -The values are specified in the \code{psphot} recipe as +spanning a range guaranteed to contain any reasonable seeing values. +The values are specified in the \ippprog{psphot} recipe as \code{PSF.SIGMA.VALUES} and have the following values for PS1 PV3: (1, 2, 3, 4.5, 6, 9, 12, 18) pixels $\approx$ (0.26, 0.51, 0.77, 1.15, @@ -900,9 +953,10 @@ the first radial moment of the PSF stars, or $0.75\sigma_w$ if that cannot be determined. $R_{\rm max}$ is set to the size of the moments -aperture, $4\sigma_w$. At this stage, the measurement of the Kron -parameters are preliminary since the aperture has been chosen as a -fixed size relative to the size of the PSF. At a later stage, -higher-quality Kron parameters appropriate to galaxies are measured -with more care paid to the exact aperture used +aperture, $4\sigma_w$. These Kron measurements are performed for all +sources with a valid set of moments. At this stage, the measurement +of the Kron parameters are preliminary since the aperture has been +chosen as a fixed size relative to the size of the PSF. At a later +stage, higher-quality Kron parameters appropriate to galaxies are +measured with more care paid to the exact aperture used (Section~\ref{sec:kron.mags}). @@ -911,4 +965,5 @@ \subsection{PSF Determination} +\label{sec:PSF.Model} \subsubsection{PSF Model vs Source Model} @@ -921,5 +976,5 @@ which vary across the image. -The PSF used by \code{psphot} consists of an analytical function +The PSF used by \ippprog{psphot} consists of an analytical function combined with a pixelized representation of the residual differences between the analytical model and the true PSF. Both the shape @@ -927,5 +982,5 @@ differences are allowed to vary in two dimensions across the images. -Within \code{psphot}, several analytical models may be used to +Within \ippprog{psphot}, several analytical models may be used to describe the smooth portion of the PSF, but all share a few common characteristics. As an example, a simple model consists of a 2-D @@ -954,7 +1009,7 @@ \sigma_x & = & f_1(x_{\rm ccd},y_{\rm ccd}) \\ \sigma_y & = & f_2(x_{\rm ccd},y_{\rm ccd}) \\ -\sigma_{xy} & = & f_3(x_{\rm ccd},y_{\rm ccd}) \\ +\sigma_{xy} & = & f_3(x_{\rm ccd},y_{\rm ccd}). \end{eqnarray} -\code{psphot} represents the variation in the PSF parameters as a function of +\ippprog{psphot} represents the variation in the PSF parameters as a function of position in the image in two possible ways, specified by the configuration. The first option is to use a 2-D polynomial which is @@ -976,5 +1031,5 @@ Several analytical functions which are likely candidates to describe -the smooth portion of the PSF are available in \code{psphot}: +the smooth portion of the PSF are available in \ippprog{psphot}: \begin{itemize} \item Gaussian : $f = I_0 e^{-z}$ @@ -990,5 +1045,5 @@ A user may choose to try more than one analytical function for a given image. As discussed below (Section~\ref{sec:psf.model.choice}), -\code{psphot} can automatically choose the best model based on the +\ippprog{psphot} can automatically choose the best model based on the quality of the PSF fits. @@ -1001,5 +1056,5 @@ variable power-law exponent model. -The analytical models in \code{psphot} are written with a high degree +The analytical models in \ippprog{psphot} are written with a high degree of code abstraction making it relatively easy to add different analytical models to the software. The same portion of code used to @@ -1025,6 +1080,5 @@ expected residuals for any position in the image. The value of each pixel in the image model is determined from 2D fits to the measured -residuals of the PSF stars. Pixel values in this model are only -defined for pixels with +residuals of the PSF stars. The residual model is calculated using the residuals for all PSF @@ -1032,11 +1086,15 @@ renormalized by the flux of the star to put them on a consistent flux scale. For each PSF star, all pixels within a user-specified radius -(PSF.RESIDUALS.RADIUS = 9) are selected for the measurement. For a +(\code{PSF.RESIDUALS.RADIUS = 9}) are selected for the measurement. For a given pixel in the model, the pixel values from the PSF stars are -interpolated to the center of the model pixel. +interpolated to the center of the model pixel. Pixels may be used in +this analysis if their signal-to-noise exceeds a user-defined limit. +For the PV3 $3\pi$ analysis, we allowed all pixels within the +user-specified radius, not limiting on the basis of the +signal-to-noise. Pixels for a given star which are more than a number of sigmas -(PSF.RESIDUALS.NSIGMA = 3.0) deviant from the median value of the -pixels from all stars are rejected. +(\code{PSF.RESIDUALS.NSIGMA = 3.0}) deviant from the median value of +the pixels from all stars are rejected. If no spatial variation is allowed, the mean or median value is @@ -1063,5 +1121,5 @@ The first stage of determining the PSF model for an image is to identify a collection of sources in the image which are {\em likely} -to be unresolved (i.e., stars). \code{psphot} uses the source sizes as +to be unresolved (i.e., stars). \ippprog{psphot} uses the source sizes as estimated from the second moments to make the initial guess at a collection of unresolved sources. At this point, the program has @@ -1070,5 +1128,5 @@ bright threshold. All sources with a S/N ratio greater than a user-defined parameter (\code{PSF_SN_LIM} = 20.0 for PS1 PV3) are -selected by \code{psphot}, though sources which have more than a +selected by \ippprog{psphot}, though sources which have more than a certain number of saturated pixels are excluded at this stage. The program then examines the 2-D plane of $M_{x,x}, M_{y,y}$ in search @@ -1115,6 +1173,6 @@ model, allowing all of the parameters (PSF and independent) to vary in the fit. The software uses the Levenberg-Marquardt minimization -technique \citep{Press,Madsen} for the non-linear fitting. Non-linear -fitting can be very computationally intensive, particularly for if the +technique \citep{1992nrca.book.....P,Madsen} for the non-linear fitting. Non-linear +fitting can be very computationally intensive, particularly if the starting parameters are far from the minimization values. The first and second moments are used to make a good guess for the centroid and @@ -1126,5 +1184,5 @@ position using either the 2-D polynomial or the gridded superpixel representation. The maximum order of these fits depends on the number -of PSF sources (see Table~\ref{tab:order}). The fitting process for +of PSF sources (see Table~\ref{tab:psf.order.nstars}). The fitting process for these polynomials is iterative, and rejects the $3\sigma$ outliers in each of three passes. This fitting technique results in a robust @@ -1138,5 +1196,5 @@ The order of the fit or number of grid samples is modified if the number of stars available for the fit is insufficient to justify the -highest value. Regardness of the requested order, if the number of +highest value. Regardless of the requested order, if the number of stars is below the following limits, the order is limited as shown in Table~\ref{tab:psf.order.nstars}. Note that the number of grid cells @@ -1171,5 +1229,5 @@ the PSF model for this particular image. -The metric used by \code{psphot} to assess the PSF model is the +The metric used by \ippprog{psphot} to assess the PSF model is the scatter in the differences between the aperture and fit magnitudes for the PSF sources. This difference is a critical parameter for any PSF @@ -1189,5 +1247,5 @@ Once a PSF model has been determined, the brighter sources in the -image may be analysed in detail. The goals in this stage are (1) to +image may be analyzed in detail. The goals in this stage are (1) to determine the fluxes and positions of the bright stellar sources with high precision appropriate to their high signal-to-noise and (2) to @@ -1197,26 +1255,36 @@ several stages in which the 2D flux models for all sources are subtracted from the image, and individual sources are replaced in the -image for a particular analysis step and then removed again. - +image for a particular analysis step and then removed again. The flux +limit for this analysis stage is user-defined as a signal-to-noise +value. In the PV3 analysis of the $3\pi$ survey data, this limit was +set to a signal-to-noise ratio of 20.0. + +% maybe drop this discussion? too much detail? In order to allow for multiple threads to process a single image, the -pixels in an image are divided into a grid of superpixels (see -Figure~\ref{fig:threadgrid}). The superpixels are assigned to one of -four groups, as illustrated, so that each superpixel in a group is -well separated from the other superpixels of that group. The analysis -of the image proceeds in 4 steps, one for each of these groups. Each -of the superpixels in the first group is assigned to a single thread -until all threads are assigned. A single thread is responsible for -the analysis of sources which land within their current superpixel, as -determined by the centroid coordinates. As the threads complete their -analysis, they are assigned the next unfinished superpixel in the -active group. When all superpixels in one group have been processed, -then the superpixels in the next group can start. This strategy -allows the threading to process sources which may be extended without -the danger that two threads are actively touching the same pixels. -For the PV3 analysis, 4 threads were used for most processing tasks. +pixels in an image are divided into a grid of superpixels. The +superpixels are assigned to one of four groups so that each superpixel +in a group is well separated from the other superpixels of that group. +The analysis of the image proceeds in 4 steps, one for each of these +groups. Each of the superpixels in the first group is assigned to a +single thread until all threads are assigned. A single thread is +responsible for the analysis of sources which land within their +current superpixel, as determined by the centroid coordinates. Since +the superpixels in a given thread group are not contiguous by +construction, sources near the edge of a superpixel can be analysed by +considering the nearby pixels from neighboring superpixel (guaranteed +not to be in the current thread group). + +As the threads complete their analysis, they are assigned the next +unfinished superpixel in the active group. When all superpixels in +one group have been processed, then the superpixels in the next group +can start. This strategy allows the threading to process sources +which may be extended without the danger that two threads are actively +touching the same pixels. For the PV3 analysis, 4 threads were used +for most processing tasks. \subsubsection{Very Bright Stars} - -The standard \code{psphot} PSF modeling code fails to fit the wings of +\label{sec:very.bright.star} + +The standard \ippprog{psphot} PSF modeling code fails to fit the wings of highly saturated stars, especially if the core of the star is too contaminated by saturated pixels. For stars with more than a single @@ -1265,6 +1333,6 @@ diagonal; the guess is multiplied by $M_{i,j}$, and the result compared with the observed vector $\bar{F_j}$. The difference is used -to modify the initial guess. This proces is repeated several times to -achieve a good convergence. Convergence is quick (a few iterations) +to modify the initial guess. This process is repeated several times +to achieve convergence. Convergence is quick (a few iterations) because of the highly diagonal matrix with small off-diagonal terms: the dot product of source $i$ and source $j$ is 1 where $i = j$ and @@ -1332,5 +1400,5 @@ If the source has 180\degree\ symmetry, this operation has no impact. However, if one of the two pixels is unusually high, the value will be -surpressed by the matched pixel on the other side. This trick has the +suppressed by the matched pixel on the other side. This trick has the effect of reducing the impact of pixels which include flux from near neighbors. @@ -1342,11 +1410,11 @@ After the PSF model has been fitted to all sources, and the Kron flux -has been measured for all sources, \code{psphot} uses these two measurements, -along with some additional pixel-level analysis, to determine the size class -of the source. If the source is large compared to a PSF, it is -considered to be {\em extended} and will be -fitted with a galaxy model (or possibly another type of extended -source model in special cases). If the source is small compared to a -PSF, it is considered to be a {\em cosmic ray} and masked. +has been measured for all sources, \ippprog{psphot} uses these two +measurements, along with some additional pixel-level analysis, to +determine the size class of the source. Sources identified as +extended will be fitted with a galaxy model (or possibly another type +of extended source model in special cases). If the source is small +compared to a PSF, it is considered to be a {\em cosmic ray} and +masked. Extended sources are identified as those for which the Kron magnitude @@ -1361,5 +1429,5 @@ considered to be extended. -Cosmic Rays are identified by a combination of the Kron magnitude and +Cosmic rays are identified by a combination of the Kron magnitude and the second-moment width of the source in the minor axis direction. The second-moment in the minor axis direction is calculated from @@ -1380,4 +1448,5 @@ \subsubsection{Full PSF Model Fitting} +\label{sec:nonlinear.psf.model} % gaussSigma = MOMENTS_GAUSS_SIGMA from recipe (initially) @@ -1388,5 +1457,5 @@ % apScale = 4.5 -Once a PSF model has been selected for an image, \code{psphot} +Once a PSF model has been selected for an image, \ippprog{psphot} attempts to fit all of the detected sources, with signal-to-noise ratio greater than a user-defined limit, with the PSF model. In the @@ -1395,5 +1464,5 @@ the dependent parameters are fixed by the PSF model and only the 4 independent source model parameters are allowed to vary in the fit. -\code{psphot} again uses Levenberg-Marquardt minimization for the +\ippprog{psphot} again uses Levenberg-Marquardt minimization for the non-linear fitting. The sources are fitted in their S/N order, starting with the brightest and working down to the user-specified @@ -1420,5 +1489,5 @@ of blended peaks. -%% Once a solution has been achieved for a source, \code{psphot} attempts to +%% Once a solution has been achieved for a source, \ippprog{psphot} attempts to %% judge the quality of the PSF model as a representation of the source %% shape. To do this, it calculates the next step of the minimization @@ -1432,5 +1501,5 @@ %% $\sigma_y$. For a generic model, the shape parameters may be defined %% differently, but there should always be two parameters which scale the -%% source size in two dimensions. Currently, \code{psphot} requires the two +%% source size in two dimensions. Currently, \ippprog{psphot} requires the two %% relevant shape parameters to be the first two dependent parameters in %% the list of model parameters (ie, parameters 4 \& 5). @@ -1455,5 +1524,5 @@ %% as a likely defect. -After the PSF model is fitted to each object, \code{psphot} makes an +After the PSF model is fitted to each object, \ippprog{psphot} makes an assessment of the quality of the PSF fits. First, it checks that the non-linear fitting process has converged with a valid fit. The fit @@ -1467,5 +1536,5 @@ exists, with a lower nearby sky region. However, the fitted PSF model cannot converge on the peak because it is very poorly defined (perhaps -only existing in the smoothed image). In these cases, \code{psphot} +only existing in the smoothed image). In these cases, \ippprog{psphot} flags the object with the bad bit \code{PM_SOURCE_MODE_FAIL}. It is also possible in this type of case for the fit to result in a very low @@ -1486,7 +1555,11 @@ non-linear PSF model fit (\code{PM_SOURCE_MODE_SATSTAR}). Among these sources, those for which the peak flux is greater than the saturation -limit are marked as saturated stars (\code{PM_SOURCE_MODE_SATSTAR}). -These model fits should be consisdered with caution, but the fluxes -and positions may have some validity (see Section~\ref{Saturation}). +limit (see Section~\ref{sec:image.preparation}) are marked as +saturated stars (\code{PM_SOURCE_MODE_SATSTAR}). These model fits +should be considered with caution, but the fluxes and positions may +have some validity. + +% \citep[see the discussion in][regarding the masking of saturated +% pixels]{waters2017} As the sources are fitted to the PSF model, those which survive the @@ -1520,4 +1593,5 @@ \subsubsection{Non-PSF Sources} +\label{sec:nonlinear.galaxy.model} Once every source (above the S/N cutoff) has been confronted with the @@ -1528,5 +1602,5 @@ moments aperture) and working to a user defined S/N limit. -\code{psphot} will use the user-selected extended source model to +\ippprog{psphot} will use the user-selected extended source model to attempt these fits. In the configuration system, the keyword \code{EXT_MODEL} is set to the model of interest. All suspected @@ -1542,5 +1616,5 @@ For each type of extended source model (in fact for all source models), a function is defined which examines the fit results and -determines if the fit can be consider as a success or a failure. The +determines if the fit can be considered as a success or a failure. The exact criteria for this decision depends on the details of the model, and so this level of abstraction is needed. For example, in some @@ -1571,8 +1645,9 @@ \subsection{Faint Source Analysis} +\label{sec:faint.psf.model} After a first pass through the image, in which the brighter sources above a high threshold level have been detected, measured, and -subtracted, \code{psphot} optionally begins a second pass at the image. In +subtracted, \ippprog{psphot} optionally begins a second pass at the image. In this stage, the new peaks are detected on the image with the bright sources subtracted. In this pass, the peak detection process uses the @@ -1596,5 +1671,5 @@ stacks in the major reprocessings. -The extended souce analysis consists of the following types of +The extended source analysis consists of the following types of measurements: 1) an analysis of the radial profile of the surface brightness of the source; 2) measurement of the Petrosian radius and @@ -1613,5 +1688,5 @@ galaxies. Several restrictions are possible within the software. For example, it is possible to limit which objects are processed by their -aparent magnitudes, by their signal-to-noise, by an indication if they +apparent magnitudes, by their signal-to-noise, by an indication if they are in fact extended, by the local stellar density, or by the galactic latitude. Some of these selections may be defined differently for the @@ -1664,4 +1739,5 @@ \subsubsection{Radial Profiles} +\label{sec:radial.profile.v2} Galaxies with regular profiles, such as elliptical galaxies and @@ -1670,8 +1746,8 @@ perturbation on that profile. For many galaxies, the azimuthal shape at a given isophotal level may be described as an elliptical contour. -To first order, a galaxy may be well decribed with a single elliptical +To first order, a galaxy may be well described with a single elliptical contour and radial profile. -In order to facilitate the Petrosian photometry analysis below, \code{psphot} +In order to facilitate the Petrosian photometry analysis below, \ippprog{psphot} generates a radial profile for each suspected galaxy. This analysis starts by generating a radial profile in 24 azimuthal segments. Near @@ -1723,12 +1799,14 @@ \subsubsection{Petrosian Radii and Magnitudes} +\label{sec:petrosian} \cite{1976ApJ...209L...1P} defined an adaptive aperture based on a ratio of surface brightnesses. The motivation is to define an aperture which can be determined for galaxies without significant -biases as a function of distance. Since surface brightness in a -resolved source is conserved as a function of distance, using a ratio -of surface brightness to define a spatial scale results in a spatial -scale which is constant regardless of galaxy distance. +biases as a function of distance from the observer. Since surface +brightness in a resolved source is conserved as a function of +distance, using a ratio of surface brightness to define a spatial +scale results in a spatial scale which is constant regardless of +galaxy distance. To measure the Petrosian radius and flux, we start by defining a @@ -1742,5 +1820,5 @@ \beta r_{\rm min}$, the Petrosian Ratio for that annulus is defined as the ratio of the -surface brightness in the annulus to the average surface brigthness +surface brightness in the annulus to the average surface brightness within $r_{\rm max}$. The Petrosian Radius is defined to be $r_{\rm max}$ for the annulus for which the Petrosian Ratio = 0.2, i.e., the @@ -1770,7 +1848,7 @@ \label{sec:galaxy.conv.fit} -In the galaxy model fittting stage, sources which meet certain +In the galaxy model fitting stage, sources which meet certain criteria are fitted with analytical models for galaxies. Three -traditional analytical galaxy models are implemented in \code{psphot} +traditional analytical galaxy models are implemented in \ippprog{psphot} and used in the PV3 analysis: \begin{itemize} @@ -1811,5 +1889,5 @@ \ref{sec:moments}) is used to estimate the effective radius of the model based on the results of Graham \& Driver (2005, Table 1). They -quantive the relationships between the first radial moment used to +quantify the relationships between the first radial moment used to calculated a Kron Magnitude and the effective radius for different S\'ersic index values, $n$. Since the Exponential and DeVaucouleur @@ -1904,5 +1982,5 @@ values for $R_{\rm eff}$ based on the value of $R_1$, the first radial moment. For a given value of the S\'ersic index, the $R_{\rm eff}$ is -related to the 1st radial moment by the scale factor specificy by +related to the 1st radial moment by the scale factor specified by Graham \& Driver. We use the observed value of the 1st radial moment and try $R_{\rm eff}$ values of a factor of (0.8, 0.9, 1.0, 1.12, @@ -1925,5 +2003,5 @@ % Graham & Driver : Graham A. W., Driver S. P. 2005, PASA 22, 118 -% DOI: https://doi.org/10.1071/AS05001 +a% DOI: https://doi.org/10.1071/AS05001 The central pixel of the S\'ersic, DeVaucouleur, and Exponential @@ -1965,14 +2043,15 @@ any of the parameters. -\subsubsection{Convolved Radial Aperture Photometry} +\subsubsection{Fixed Aperture Photometry} +\label{sec:fixed.aperture.photom} For some science goals, a well-measured color of a galaxy is more important than an accurate total magnitude. In the case of PS1, the image quality variations for stacks of different filters presents a serious -challenge for the determination of precise colors. \code{psphot} determines +challenge for the determination of precise colors. \ippprog{psphot} determines a set of PSF-matched radial aperture flux measurements in order to minimize the impact of the stack image quality variations. -In \code{psphotStack}, the stack analysis version of \code{psphot}, +In \ippprog{psphotStack}, the stack analysis version of \ippprog{psphot}, the 5 filter images are processed together. After the PSF models have been fitted and a best set of galaxy models have been determined, @@ -2005,5 +2084,5 @@ wasteful. We only calculate the circular apertures out to the second aperture larger than the ``sky radius'' (defined in -Section~\label{sec:radial.profile}), but we calculate photometry for +Section~\ref{sec:radial.profile}), but we calculate photometry for at least the smallest 4 apertures. @@ -2061,5 +2140,5 @@ saturation. -In order to thread the needle between these effects, \code{psphot} +In order to thread the needle between these effects, \ippprog{psphot} measures the aperture photometry on a modest-sized aperture, and then uses the PSF model to extrapolate to a large aperture. When the PSF @@ -2125,5 +2204,5 @@ % magnitude}. -%%% \code{psphot} measures the aperture correction ({\em ApResid}) for every PSF +%%% \ippprog{psphot} measures the aperture correction ({\em ApResid}) for every PSF %%% candidate source, then calculates the trend of this correction as a %%% function of the magnitude. This trend is fitted with a line. The @@ -2140,8 +2219,8 @@ %%% term. -\code{psphot} allows a collection of PSF model functions to be tried on all +\ippprog{psphot} allows a collection of PSF model functions to be tried on all PSF candidate sources. For each model test, the above corrected ApResid scatter is measured. The PSF model function with the smallest -value for the ApResid scatter is then used by \code{psphot} as the best PSF +value for the ApResid scatter is then used by \ippprog{psphot} as the best PSF model for this image. The number of models to be tested is specified by the configuration keyword \code{PSF_MODEL_N}. The configuration @@ -2150,44 +2229,6 @@ tested. -\begin{table*} -\begin{center} -\caption{\label{tab:measurements} \nocode{psphot} measurements performed} % \vspace{-0.5cm} -\begin{tabular}{lcccc} -\hline -\hline -{\bf Measurement} & {\bf Camera} & {\bf Stack} & {\bf Forced Warp} & {\bf Diff} \\ -\hline - Background & Y & Y & Y & N$^1$ \\ - Peaks & Y & Y & N & Y \\ - Footprints & Y & Y & N & Y \\ - Moments & Y & Y & Y & Y \\ - PSF Model & Y & Y & Y & N$^2$ \\ - Bright Star Profile & Y & Y & N & Y \\ - Non-Linear PSF Fits & Y & Y & N & N \\ - Source-Size Tests & Y & Y & N & Y \\ - Unconvolved Galaxy Model & Y & Y & N & N \\ - Unconvolved Streak Model & N & N & N & Y \\ - Linear PSF Fits & Y & Y & Y & Y \\ - Radial Profiles & Y & Y & N & Y \\ - Petrosian Fluxes & N & Y & Y & N \\ - Kron Fluxes & Y & Y & Y & Y \\ - Convolved Galaxy Models & N & Y & N & N \\ - Fixed Aperture Photometry & N & Y & Y & N \\ - Convolved, Fixed Apertures & N & Y & N & N \\ - Aperture Corrections & Y & Y & Y & N \\ - Forced PSF Fluxes & N & N & Y & N \\ - Forced Galaxy Models & N & N & Y & N \\ - Lensing Parameters & N & Y & Y & N \\ -\hline -\hline -\multicolumn{5}{l}{$^1$ Background subtraction is performed by {\tt ppSub} before calling {\tt psphot}} \\ -\multicolumn{5}{l}{$^2$ PSF modeling is perfom by {\tt ppSub} on the input warps before calling {\tt psphot}} \\ -\end{tabular} -\end{center} -\end{table*} - -% \subsection{Output Formats} - \section{Forced Photometry Modes} +\label{sec:psf.forced.fit} Traditionally, projects which use multiple exposures to increase the @@ -2198,8 +2239,8 @@ with best sensitivity and the best data quality at all magnitudes. In practice, these images have some significant limitations due to the -difficulty of modelling the PSF variations. This difficulty is +difficulty of modeling the PSF variations. This difficulty is particularly severe for the Pan-STARRS $3\pi$ survey stacks due to the combination of the substantial mask fraction of the individual input -exposures, the large instrinsic image quality variations within a +exposures, the large intrinsic image quality variations within a single exposure, and the wide range of image quality conditions under which data were obtained and used to generate the $3\pi$ PV3 stacks. @@ -2257,5 +2298,5 @@ image; the measured flux may even be negative due to statistical fluctuations. When combined together, these low-significance -measurements result in a signficant measurement as the signal-to-noise +measurements result in a significant measurement as the signal-to-noise increases with the combination of more data. @@ -2341,5 +2382,5 @@ The analysis of the difference image follows the same basic steps as -other \ippprog{psphot} versions with some minor modifictions (see +other \ippprog{psphot} versions with some minor modifications (see Table~\ref{tab:measurements}), as follows. The background subtraction is performed before the PSF matching and image subtraction is @@ -2378,5 +2419,5 @@ motion. If the astrometric solution for one of the two images is insufficiently accurate, all stars in large portions of the images may -be noticably displaced. In both of these situations, the stars will +be noticeably displaced. In both of these situations, the stars will appear as PSF dipoles in the difference images. The positive and the negative images will have stellar profiles, but they will be offset @@ -2407,5 +2448,5 @@ context of the input images, both the positive (subtrahend) and negative (minuend) images. We identify the closest source in both the -postive and negative images to the detection in the difference image, +positive and negative images to the detection in the difference image, out to a maximum of \code{INPUT.MATCH.RADIUS} (= 50 pixels), but only if the source in those images has a signal-to-noise greater than @@ -2433,6 +2474,7 @@ power-law profile) and flux from the tail (with a more complex flux distribution). We use the Kron magnitudes to identify possibly -extended objects which may be cometary in nature. \note{need some - info from MOPS folks on what is used} +extended objects which may be cometary in nature. + +% \note{need some info from MOPS folks on what is used} For a difference image, both positive and negative sources will be @@ -2512,2 +2554,17 @@ * background model description (see waters) +% alternative version: +% @book{madsen2004methods, +% title={Methods for Non-linear Least Squares Problems}, +% author={Madsen, K. and Nielsen, H.B. and Tingleff, O. and Danmarks tekniske universitet. Informatik og Matematisk Modellering}, +% url={https://books.google.com/books?id=mhj4MgEACAAJ}, +% year={2004}, +% publisher={Informatics and Mathematical Modelling, Technical University of Denmark} +% } + +% programs mentioned in this text: +% psphot +% psphotStack +% psphotFullForce +% ppImage +% ppSub