Index: trunk/doc/release.2015/ps1.analysis/analysis.tex =================================================================== --- trunk/doc/release.2015/ps1.analysis/analysis.tex (revision 41129) +++ trunk/doc/release.2015/ps1.analysis/analysis.tex (revision 41307) @@ -29,6 +29,6 @@ %\def\picdir{/home/eugene/chipresid.20140404} -%\def\picdir{pics} -\def\picdir{.} +\def\picdir{pics} +%\def\picdir{.} % Pick a terse version of the title here; @@ -98,6 +98,6 @@ images from other telescopes. We describe the analysis of the astronomical sources by \ippprog{psphot} in general as well as for the -specific case of the 3rd processing version used for the first public -release of the Pan-STARRS $3\pi$ survey data. +specific case of the 3rd processing version used for the first \textmod{two public +releases} of the Pan-STARRS $3\pi$ survey data. \end{abstract} @@ -155,5 +155,5 @@ Pan-STARRS produced its first large-scale public data release, Data Release 1 (DR1) on 16 December 2016. DR1 contains the results of the -third full reduction of the Pan-STARRS $3\pi$ Survey archival data, +third full reduction of the Pan-STARRS $3\pi$ Surveyo archival data, identified as PV3. Previous reductions \citep[PV0, PV1, PV2; see][]{magnier2017.datasystem} were used internally for pipeline @@ -166,6 +166,8 @@ images obtained by the $3\pi$ Survey observations. A second data release, DR2, was made available 28 January 2019. DR2 provides -measurements from all of the individual exposures, and include an -improved calibration of the PV3 processing of that dataset. +measurements from all of the individual exposures, and includes an +improved \textmod{astrometric calibration as well as improvements to the + photometric calibration of the stack and 'forced warp' measurements +from} the PV3 processing of that dataset. This is the fourth in a series of seven papers describing the @@ -174,7 +176,12 @@ source detection and photometry, including point-spread-function and extended source model fitting, and the techniques for ``forced'' -photometry measurements. The software described here was used with a +photometry measurements. \textadd{The same analysis software is used + for individual images, image stacks, and difference images.} +The software described here was used with a single consistent set of parameters for the complete PV3 analysis, -used for both DR1 and DR2. +used for both DR1 and DR2. \textadd{The software was also used for the +analysis of the Medium Deep Survey data, though with a different +software version and some modifications of +the analysis parameters to better suite the longer exposures.} %Chambers et al. 2017 (Paper I) @@ -190,5 +197,5 @@ \citet[][Paper II]{magnier2017.datasystem} describe how the various data processing stages are organized and implemented -in the Imaging Processing Pipeline (IPP), including details of the +in the \textmod{Image Processing Pipeline} (IPP), including details of the the processing database which is a critical element in the IPP infrastructure . @@ -231,11 +238,28 @@ %% submission and refereeing process.}} +\textadd{In this article, we use the following type-faces to distinguish +different concepts:} +\begin{itemize} +\item \ippstage{Small caps} for the analysis stages. +\item \ippdbtable{Italics} for database tables and columns. +\item \ippprog{Fixed-width} font for program names, variables, and + miscellaneous constants. +\end{itemize} + +\textadd{ +The latter catagory includes a number of configuration parameters used +to define the \ippprog{psphot} analysis. In those cases, unless the +values used for the PV3 analysis are explicitly discussed, we include +the PV3 value immediately after the configuration variable name in parenthesis.} + \section{Background} The photometric and astrometric precision goals for the Pan-STARRS\,1 -surveys were quite stringent: photometric accuracy of 10 -millimagnitudes, relative astrometric accuracy of 10 milliarcseconds +surveys were quite stringent. The astrometric goals were relative astrometric accuracy of 10 milliarcseconds and absolute astrometric accuracy of 100 milliarcseconds with respect -to the ICRS reference stars. +to the ICRS reference stars. For photometry, the goal was 10 +millimagnitudes accuracy within the internal photometric system across +the sky, though the tie to an absolute standard was not required to +meet this standard. An additional constraint on the Pan-STARRS analysis system comes from @@ -311,6 +335,8 @@ Several variants of \ippprog{psphot} have been used in the PS1 PV3 analysis. The main variant of \ippprog{psphot} operates on a single -image, or a group of related images representing the data read from a -camera in a single exposure. The images are expected to have already +image, or a group of related images representing the data read from +\textmod{the multiple chips of a mosaic +camera from} a single exposure. \textadd{In the IPP sequencing, this step is +called the \ippstage{chip} stage.} The images are expected to have already been detrended so that pixel values are linearly related to the flux. The gain may be specified by the configuration system, or a variance @@ -322,6 +348,10 @@ The variant called \ippprog{psphotStack} accepts a set of images, each -representing the same patch of sky in a different filter, nominally -the full $grizy$ filter set for the analysis of the PS1 PV3 stack +representing the same patch of sky \textadd{(with pixels aligned)} in +a different \textmod{filter. This version was used for the analysis + of the deep ``stacks'' (co-added images combining multiple + observations of the same field) produced by the IPP \ippstage{stack} + stage. Nominally, +the full $grizy$ filter set was used for the analysis} of the PS1 PV3 stack images, though where insufficient data were available in a given filter, a subset of these filters was processed as a group. As @@ -329,5 +359,5 @@ capability of measuring forced PSF photometry in some filter images based on the position of sources detected in the other filters. It -also include an option to convolve the set of images to a single, +also includes an option to convolve the set of images to a single, common PSF size across the filters for the purpose of fixed aperture photometry. @@ -335,5 +365,5 @@ Another variant of \ippprog{psphot} used in the PV3 analysis is called \ippprog{psphotFullForce}. In this variant, a set of images all representing the -same pixels are processed together, with the positions of sources to +same \textadd{co-aligned} pixels are processed together, with the positions of sources to be analyzed loaded from a supplied file. In this variant of the analysis, sources are not discovered -- only the supplied sources are @@ -348,20 +378,32 @@ % \subsection{Astronomy Measurement Goals} -\ippprog{psphot} has a number of important requirements that it must -meet, and a number of design goals which we believe will help to make -it usable in a wide range of circumstances. The critical -astronomy-driven measurement goals of the Pan-STARRS project, which -drive the design of \ippprog{psphot}, are the photometric accuracy -goal (10 millimagntudes) and the astrometric accuracy goal (10 -milliarcseconds). For \ippprog{psphot}, the photometry accuracy goal -implies that the measured photometry of stellar sources must be -substantially better than this 10 mmag goal since the photometry error -per image is combined with an error in the flat-field calibration and -an error in measuring the atmospheric effects. We have set a goal for +\textadd{The top-level design goals of \ippprog{psphot} are to detect and +determine the instrumental positions and fluxes of astronomical +sources in the images. For extended sources, the goals also include +the measurement of a variety of morphological information, including +galaxy model parameters and non-parametric measurements of the sizes +and profiles of the galaxies to aid in classification and for +weak-lensing analysis. For trailed asteroids, the goal also includes +the measurement of the length and direction of the trail.} + +\textmod{Beyond these basic elements, \ippprog{psphot} has a number of + design goals} which we believe will help to make it usable in a wide +range of circumstances. The critical astronomy-driven measurement +goals of the Pan-STARRS project, which drive the design of +\ippprog{psphot}, are the photometric accuracy goal (10 +millimagnitudes) and the \textadd{relative} astrometric accuracy goal +(10 milliarcseconds) \textadd{for bright stars for which the photon + shot-noise is small compared to the systematic errors.} + +For \ippprog{psphot}, the photometry accuracy goal implies that the +measured photometry of stellar sources must be substantially better +than this 10 mmag goal since the photometry error per image is +combined with an error in the flat-field calibration and an error in +measuring the atmospheric effects. We have set a goal for \ippprog{psphot} of 3 mmag photometric consistency for bright stars between pairs of images obtained in photometric conditions at the same pointing, ie to remove sensitivity to flat-field errors. This goal splits the difference between the three main contributors and still -allows some leeway. This requirement must be met for well-sampled +allows some leeway. This goal must be met for well-sampled images and images with only modest undersampling. @@ -420,4 +462,11 @@ \end{itemize} +\note{get a better example of the psphot accuracy achieved} + +\textadd{The success of the \ippprog{psphot} implementation is meeting + the photometry and astrometry design requirements is demonstrated by + the achieved accuracy for the Pan-STARRS $3\pi$ Survey data. +} + \section{Basic Analysis} @@ -480,5 +529,6 @@ \hline \hline -{\bf Measurement} & {\bf Camera} & {\bf Stack} & {\bf Forced Warp} & {\bf Diff} & {\bf Section} & {\bf Which} \\ +{\bf Measurement} & {\sc \bf CHIP} & {\sc \bf STACK} & {\sc \bf FORCED + WARP} & {\sc \bf DIFF} & {\bf Section} & {\bf Which} \\ \hline Background Subtraction & Y & Y & Y & N$^1$ & \ref{sec:image.preparation} & N/A \\ @@ -524,9 +574,9 @@ field \ippmisc{FLAGS}. When data from \ippprog{psphot} is loaded into a DVO database \citep{magnier2017.calibration}, these values are -stored in the field \code{Measure.photFlags} and exposed in the public +stored in the field \ippdbtable{Measure.photFlags} and exposed in the public database \citep[PSPS][]{flewelling2017} in the fields -\code{Detection.infoFlag}, \code{StackObjectThin.XinfoFlag} (where -\code{X} is one of {$grizy$}), and -\code{ForcedWarpMeasurement.FinfoFlag}. +\ippdbtable{Detection.infoFlag}, \ippdbtable{StackObjectThin.XinfoFlag} (where +\ippdbtable{X} is one of {$grizy$}), and +\ippdbtable{ForcedWarpMeasurement.FinfoFlag}. % Table~\ref{tab:det_flag2_values} lists the flags recorded in the @@ -534,7 +584,7 @@ loaded into a DVO database \citep{magnier2017.calibration}, these values are not currently loaded, but they are exposed in PSPS in the fields -\code{Detection.infoFlag2}, \code{StackObjectThin.XinfoFlag2} (where -\code{X} is one of {$grizy$}), and -\code{ForcedWarpMeasurement.FinfoFlag2}. +\ippdbtable{Detection.infoFlag2}, \ippdbtable{StackObjectThin.XinfoFlag2} (where +\ippdbtable{X} is one of {$grizy$}), and +\ippdbtable{ForcedWarpMeasurement.FinfoFlag2}. \begin{table*} @@ -635,5 +685,10 @@ be provided by the user, or they may be automatically generated from the input image, based on configuration-defined values for the image -gain, read-noise, saturation, and so forth. For the function-call +gain, read-noise, saturation, and so forth. \textadd{Within the IPP analysis, +we normally use images which are equivalent to the digital numbers +(scaled by the detrend images), but as long as the variance image is +constructed in a consistent fashion, \ippprog{psphot} can use images +in electron, calibrated flux units or other conventions (though this would +require some tuning of configuration parameters).} For the function-call form of the program, the flux image is provided in the API, and references to the mask and variance are provided in the configuration @@ -643,5 +698,7 @@ The mask is represented as a 16-bit integer image in which a value of 0 represents a valid pixel. Each of the 16 bits define different -reasons a pixel should be ignored. This allows us to optionally +reasons a pixel should be ignored, \textadd{listed in + Table~\ref{tab:mask_values}}. +This allows us to optionally respect or ignore the mask depending on the circumstance. For example, in some cases, we ignore saturated pixels completely while in @@ -658,7 +715,6 @@ case of PS1 PV3, the header keyword \code{MAXLIN} specifies the saturation level for each chip \citep[see][]{waters2017}. 2) Pixels -which are below a user-defined value are considered unresponsive and -masked as dead. (camera format keyword \code{CELL.BAD} = 0 for PS1 -PV3). 3) Pixels which lie outside of a user-defined coordinate window +which are below a user-defined value (\code{CELL.BAD} = 0 for PV3) are considered unresponsive and +masked as dead. 3) Pixels which lie outside of a user-defined coordinate window are considered non-data pixels (\eg, overscan) and are marked as invalid. (\ippprog{psphot} recipe keywords \code{XMIN}, \code{XMAX}, @@ -744,14 +800,14 @@ subtracted. The image is divided into a grid of background points with a spacing defined by the \ippprog{psphot} recipe values -\code{BACKGROUND.XBIN, BACKGROUND.YBIN}, set to 400 pixels for PS1 -PV3. Superpixels of size \code{BACKGROUND.XSAMPLE, BACKGROUND.YSAMPLE} -($2 \times 2$ for PS1 PV3) times larger than -this spacing are used to measure the local background for each -background grid point, thus over-sampling the background spatial -variations. In the interest of speed, a subset of \code{IMSTATS_NPIX} -(10,000 for PS1 PV3) randomly selected {\em unmasked} pixels in these -regions are used to determine the background. The background value -for each superpixel is determined by fitting a Gaussian distribution -to the histogram of pixels values. +\code{BACKGROUND.XBIN, BACKGROUND.YBIN}, set to 400 pixels +\textadd{($\sim 100$ arcseconds)} for PV3. Superpixels of size +\code{BACKGROUND.XSAMPLE, BACKGROUND.YSAMPLE} ($2 \times 2$ for PV3) +times larger than this spacing are used to measure the local +background for each background grid point, thus over-sampling the +background spatial variations. In the interest of speed, a subset of +\code{IMSTATS_NPIX} (10,000 for PV3) randomly selected {\em unmasked} +pixels in these regions are used to determine the background. The +background value for each superpixel is determined by fitting a +Gaussian distribution to the histogram of pixels values. If the image were empty of stars and only contained flux from a @@ -788,4 +844,21 @@ the discussion in Section~3.11 of \cite{waters2017}. +\textadd{Since the subtraction of the sky model supresses larger-scale + structures, features such as large galaxies which are comparable to + the superpixel size are adversely affected by the subtraction. + Photometry for galaxies larger than $\sim 30$ arcseconds is + unreliable as a result. The superpixel size used for the sky model + in the PV3 analysis was chosen as compromise between the need to + follow bright features with small spatial scales and the desire to + measure photometry of galaxies of sizes up to at least 30 + arcseconds. Features which we wished to suppress include both + astronomical sources, such as bright nebulosity and the wings of + bright stars, and non-astronomical sources, such as moonlight and + other scattered light sources. In some contexts, we have used a + finer spacing for the background model, such as in the dedicated + analysis of the photometry of the Andromeda Galaxy, where we are + only interested in stellar sources and the analysis is otherwise + badly affected by the background from this galaxy.} + \subsection{Initial Source Detection} @@ -801,20 +874,31 @@ significance image in signal-to-noise units, including correction for the covariance, if known. At this stage, the goal is only to detect -the brighter sources, above a user defined S/N limit (configuration -keyword: \code{PEAKS_NSIGMA_LIMIT} = 20.0 for PS1 PV3). A maximum of -\code{PEAKS_NMAX} (5000 of PS1 PV3) are found at this stage. The +the brighter sources, above a user defined S/N limit +(\code{PEAKS_NSIGMA_LIMIT} = 20.0 for PV3). A maximum of +\code{PEAKS_NMAX} (5000 for PV3) are found at this stage. + +\textadd{For an image with a Gaussian PSF of the same size, this method + would represent the optimal detection algorithm, equivalent to a + matched filter \note{add ref}. At this stage, our goal is simply to + detect the brighter sources, so the exact size and shape of the PSF + is not critical. } +The detection efficiency for the brighter sources is not strongly -dependent on the form of this smoothing function. +dependent on the form of this smoothing function. \textadd{Instead, + our goal with the smoothing kernel is to reduce our sensitivity to + pixel-to-pixel fluctuations in the location of the peak of the + sources in the image.}. The local peaks in the smoothed image are found by first detecting local peaks in each row. For each peak, the neighboring pixels are then examined and the peak is accepted or rejected depending on a set -of simple rules. First, any peak which is greater than all 8 +of simple rules. \textadd{The rules are defined so that we choose a unique set +of peaks which are not immediately adjacent to other peaks.} First, any peak which is greater than all 8 neighboring pixels is kept. Any peak which is lower than any of the 8 neighboring pixels is rejected. Any peak which has the same value as -any of the other 8 pixels is kept if the pixel $X$ and $Y$ coordinates -are greater than or equal to the other equal value pixels. This -simple rule set means that a flat-topped region will result peaks at -the maximum $X$ and $Y$ corners of the region. +any of the other 8 pixels is kept {\em if} the pixel $X$ and $Y$ coordinates +are greater than or equal to the other equal-value pixels. \textmod{This +last rule means that a flat-topped region will result in peaks at +the maximum $X$ and $Y$ corners of the region.} We use the 9 pixels which include the source peak to fit for the @@ -882,5 +966,6 @@ \caption{\label{fig:peaks} Illustration of peak finding and culling peaks within a footprint. Insignificant peaks within the footprint of a brighter - peak are ignored in further processing. } + peak are ignored in further processing. \note{NOTE that the + diagram is a 1D rep of a 2D path.}} \end{center} \end{figure} @@ -897,18 +982,20 @@ (\code{PEAKS_NSIGMA_LIMIT}). These regions are grown by a small amount to avoid errors on rough edges -- an image of the footprints is -convolved with a disk of radius \code{FOOTPRINT_GROW_RADIUS} (= 3 -pixels for PS1 PV3). Peaks are assigned to the footprints in which +convolved with a disk of radius \code{FOOTPRINT_GROW_RADIUS} (3 +pixels for PV3). Peaks are assigned to the footprints in which they are contained (note by construction all peaks must be located in a footprint since the peaks must be above the detection threshold). For any peak which is not the brightest peak in that footprint it is -possible to reach the brightest peak by following the highest valued -pixels between the two peaks. The lowest pixel along this path is the +possible to reach the brightest peak by following a sequence of the highest valued +pixels between the two peaks. The lowest pixel along this +\textadd{(potentially meandering)} path is the {\em key col} for this peak (as used in topographic descriptions of a mountain). If the key col for a given peak is less than -\code{FOOTPRINT_CULL_NSIGMA_DELTA} (4.0 for PS1 PV3) sigmas below the +\code{FOOTPRINT_CULL_NSIGMA_DELTA} (4.0 for PV3) sigmas below the peak of interest, the peak is considered to be {\em locally insignificant} and removed from the list of possible detections (see -Figure~\ref{fig:peaks}). In the vicinity of a saturated star, the +Figure~\ref{fig:peaks}). \textadd{If more than one such path is possible, the +path with the highest key col is used for this test.} In the vicinity of a saturated star, the rule is somewhat more aggressive as the flat-topped or structured saturated top of a bright star may appear as multiple peaks with @@ -976,5 +1063,5 @@ and the aperture is an iterative process: for a given value of $\sigma_w$, the PSF stars will have a measured value of the PSF size, -$\sigma^{\prime}_{\rm PSF}$ which different from the true value due to +$\sigma^{\prime}_{\rm PSF}$ \textmod{which is different} from the true value due to the effect of the window function. The measured value of the PSF size will be biased high or low depending on both the signal-to-noise of @@ -992,6 +1079,6 @@ FWHM for faint stars rises, and then over-shoots the truth value, while the scatter increases. Thus, for large values of $\sigma_w$, -the result is both a poorly estimated FWHM for the image and a trend -this the signal-to-noise of the star. We attempt to minimize the +the result is both a poorly estimated FWHM for the image and a \textmod{trend +with the} signal-to-noise of the star. We attempt to minimize the scatter and trends with instrumental magnitude at the cost of overall bias. @@ -1057,5 +1144,5 @@ $S = \sum_i (f_i - s_i) w_i$ is the window-weighted sum of the source flux, used to re-normalize the moments; $r_i$ is the radius of a -pixel, $\sqrt{(x_i - x_0)^2 + (y_i - y_0)^2}$; The sums are performed +pixel, $\sqrt{(x_i - x_0)^2 + (y_i - y_0)^2}$. The sums are performed over all (unmasked) pixels in the aperture. For the centroid calculation ($x_0, y_0$), the peak coordinate (see~\ref{sec:peaks}) is used to define the @@ -1076,5 +1163,5 @@ If the measured centroid coordinates ($x_0, y_0$) differ from the peak -coordinates be a large amount (1.5$\sigma_w$), then the peak is +coordinates \textmod{by} a large amount (1.5$\sigma_w$), then the peak is identified as being of poor quality and is skipped in further analyses; the flag bit @@ -1161,5 +1248,6 @@ parameters would be the shape terms ($\sigma_x, \sigma_y, \sigma_{\rm xy}$) while the independent parameters would be the centroid, -normalization and local sky values ($x_o, y_o, I_o, S$). Thus the +normalization and local sky values ($x_o, y_o, I_o, S$). \note{we do + not fit sky as a free parametery, right?} Thus the shape parameters are each a function of the source centroid coordinates: @@ -1169,20 +1257,22 @@ \sigma_{xy} & = & f_3(x_{\rm ccd},y_{\rm ccd}). \end{eqnarray} -\ippprog{psphot} represents the variation in the PSF parameters as a function of -position in the image in two possible ways, specified by the -configuration. The first option is to use a 2-D polynomial which is -fitted to the measured parameter values across the image. The second -option is to use a grid of values which are measured for sources -within a subregion of the image. In the latter case, the value at a -specific coordinate in the image is determined by interpolation -between the nearest grid points. The order of the polynomial or the -sampling size of the grid is dynamically determined depending on the -number of available of PSF stars. In the case of the PV3 analysis, -the grid of values was used, with a maximum of $6\times 6$ samples per -GPC1 chip image. For the earlier PV2 analysis, the maximum grid -sampling was $3\times 3$ per GPC1 chip image. For the PV1 analysis, -the polynomial representation was used, with up to 3rd order terms. -The higher order representation was used for PV3 in order to follow -some of the observed PSF variations in the images +\ippprog{psphot} represents the variation in the PSF parameters as a +function of position in the image in two possible ways, specified by +the configuration. The first option is to use a 2-D polynomial which +is fitted to the measured parameter values across the image. The +second option is to use a grid of values which are measured for +sources within a subregion of the image. In the latter case, the +value at a specific coordinate in the image is determined \textmod{via + bi-linear} interpolation between the nearest grid points. The order +of the polynomial or the sampling size of the grid is dynamically +determined depending on the number of available of PSF stars. In the +case of the PV3 analysis, the grid of values was used, with a maximum +of $6\times 6$ samples per GPC1 chip image \textadd{(grid cells of + size $\sim 3.4$ arcminutes)}. For the earlier PV2 analysis, the +maximum grid sampling was $3\times 3$ per GPC1 chip image +\textadd{(grid cells of size $\sim 6.9$ arcminutes)}. For the PV1 +analysis, the polynomial representation was used, with up to 3rd order +terms. The higher order representation was used for PV3 in order to +follow some of the observed PSF variations in the images. % \note{write up the fitting process to define the grid?} @@ -1193,5 +1283,5 @@ \item Gaussian : $f = I_0 e^{-z}$ \item Pseudo-Gaussian : $f = I_0 (1 + z + \frac{1}{2} z^2 + \frac{1}{6} z^3)^{-1}$ \code{[PGAUSS]} -\item Variable Power-Law : $f = I_0 (1 + z + z^{\alpha})^{-1}$ \code{[RGAUSS]} +\item Variable Power-Law : $f = I_0 (1 + z + z^{\alpha})^{-1}$ \code{[RGAUSS]}, $\alpha > 1.25$ \item Steep Power-Law : $f = I_0 (1 + \kappa z + z^{2.25})^{-1}$ \code{[QGAUSS]} \item PS1 Power-Law : $f = I_0 (1 + \kappa z + z^{1.67})^{-1}$ \code{[PS1_V1]} @@ -1201,4 +1291,9 @@ similar to the Moffat profile form \citep{1969AA.....3..455M,1983AA...126..278B}, with small differences. +\textadd{For these PSF models, the functions are evaluated at the pixel center. +Unlike some galaxy model representations (see +Section~\label{sec:galaxy.conv.fit} ), the first derivatives of these +functions approach zero as the radius approaches zero, so sub-pixel +integration is not necessary.} A user may choose to try more than one analytical function for a given image. As discussed below (Section~\ref{sec:psf.model.choice}), @@ -1245,7 +1340,7 @@ renormalized by the flux of the star to put them on a consistent flux scale. For each PSF star, all pixels within a user-specified radius -(\code{PSF.RESIDUALS.RADIUS = 9}) are selected for the measurement. For a -given pixel in the model, the pixel values from the PSF stars are -interpolated to the center of the model pixel. Pixels may be used in +(\code{PSF.RESIDUALS.RADIUS = 9}) are selected for the measurement. \textmod{For a +given pixel in the model, the value is calculated from the 4 closest +pixels in the PSF stars via bi-linear interpolation.} Pixels may be used in this analysis if their signal-to-noise exceeds a user-defined limit. For the PV3 $3\pi$ analysis, we allowed all pixels within the @@ -1271,8 +1366,8 @@ \] where $R[(x_{\rm mod},y_{\rm mod})][(x_{\rm ccd},y_{\rm ccd})]$ is the -value for model pixel $(x_{\rm mod},y_{\rm mod})$ for a star with -centroid at image pixel $(x_{\rm ccd},y_{\rm ccd})$. The parameters -$R_o, R_x, R_y$ are determined for each pixel in the model $[(x_{\rm - mod},y_{\rm mod})]$. +\textmod{value of the residual for model} pixel $(x_{\rm mod},y_{\rm mod})$ for a star with +centroid at image pixel $(x_{\rm ccd},y_{\rm ccd})$. \textmod{The parameters +$R_o, R_x, R_y$ are the elements of the 2-D linear fit for each pixel $(x_{\rm mod},y_{\rm mod})$ +in the model. } \subsubsection{Candidate PSF Source Selection} @@ -1355,6 +1450,6 @@ For the resulting collection of source model parameters, the PSF-dependent parameters of the models are all fitted as a function of -position using either the 2-D polynomial or the gridded superpixel -representation. The maximum order of these fits depends on the number +position using either the 2-D polynomial or the gridded +representation described above. The maximum order of these fits depends on the number of PSF sources (see Table~\ref{tab:psf.order.nstars}). The fitting process for these polynomials is iterative, and rejects the $3\sigma$ outliers in @@ -1380,15 +1475,15 @@ for a given order of the PSF 2D variations.} % \vspace{-0.5cm} \begin{center} -\begin{tabular}{lll} +\begin{tabular}{llll} \hline \hline -{\bf Minimum Number} & {\bf Order} & {\bf Number of} \\ -{\bf of Stars} & & {\bf Grid Cells} \\ +{\bf Minimum } & {\bf Order} & {\bf Number of} & {\bf Cell Size} \\ +{\bf \# of Stars} & & {\bf Grid Cells} & {\bf (arcmin) } \\ \hline - 16 & 1 & 4 \\ - 54 & 2 & 9 \\ -128 & 3 & 16 \\ -300 & 4 & 25 \\ -576 & 5 & 36 \\ + 16 & 1 & 4 & 10.3 \\ + 54 & 2 & 9 & 6.9 \\ +128 & 3 & 16 & 5.1 \\ +300 & 4 & 25 & 4.1 \\ +576 & 5 & 36 & 3.4 \\ \hline \end{tabular} @@ -1405,26 +1500,38 @@ the PSF model for this particular image. -The metric used by \ippprog{psphot} to assess the PSF model is the -scatter in the differences between the aperture and fit magnitudes for -the PSF sources. This difference is a critical parameter for any PSF -modeling software as it is a measurement of how well the PSF model -captures the flux of the star. Aperture photometry is measured for a -circular aperture with a radius of \code{PSF_APERTURE_SCALE} (= 4.5 -for the PV3 $3\pi$ analysis) times $\sigma_w$ +% For each model test, the above +% corrected ApResid scatter is measured. The PSF model function with +% the smallest value for the ApResid scatter is then used by +% \ippprog{psphot} as the best PSF model for this image. + +{\bf \ippprog{psphot} allows a collection of PSF model functions to be +tried on all PSF candidate sources. The number of models to be tested +is specified by the configuration keyword \code{PSF_MODEL_N}. The +configuration variables \code{PSF_MODEL_0}, \code{PSF_MODEL_1}, +through \code{PSF_MODEL_N - 1} specify the names of the models which +should be tested. The metric used by \ippprog{psphot} to assess the +PSF model is the scatter in the differences between the aperture and +fit magnitudes for the PSF sources. This difference is a critical +parameter for any PSF modeling software as it is a measurement of how +well the PSF model captures the flux of the star. Aperture photometry +is measured for a circular aperture with a radius of +\code{PSF_APERTURE_SCALE} (4.5 for PV3) times $\sigma_w$ (Section~\ref{sec:moments}). The average aperture correction ($m_{\rm AP} - m_{\rm PSF}$) is measured and, if multiple PSF model types are selected, the PSF model with the minimum clipped scatter in this -statistic is chosen for the image. An approximate aperture correction -is measured here, with a more detailed correction measured after all -source analysis is performed (see -Section~\ref{sec:aperture.correction}). Sources for which the -aperture magnitude is measured have the flag bit +statistic is chosen for the image. For the PV3 analysis, however, only the +\code{PS1_V1} model function was used.} + +An approximate aperture correction is measured at this stage, with a +more detailed correction measured after all source analysis is +performed (see Section~\ref{sec:aperture.correction}). Sources for +which the aperture magnitude is measured have the flag bit \code{PM_SOURCE_MODE_AP_MAGS} set. These aperture magnitudes are -stored in the DVO field \code{Measure.Map} and exported to the PSPS as -a flux in Janskies in the field \code{Detection.apFlux}. The radius -(in arcseconds) -of the aperture used for each exposure is reported in PSPS as -\code{Detection.apRadius}, while the unmasked fraction of the aperture -is reported in PSPS as \code{Detection.apFillF}. +stored in the DVO field \ippdbtable{Measure.Map} and exported to the +PSPS as a flux in Janskies in the field \ippdbtable{Detection.apFlux}. +The radius (in arcseconds) of the aperture used for each exposure is +reported in PSPS as \ippdbtable{Detection.apRadius}, while the +unmasked fraction of the aperture is reported in PSPS as +\ippdbtable{Detection.apFillF}. When the PSF and aperture photometry for a source is measured, two @@ -1486,5 +1593,7 @@ % maybe drop this discussion? too much detail? In order to allow for multiple threads to process a single image, the -pixels in an image are divided into a grid of superpixels. The +pixels in an image are divided into a grid of superpixels \textadd{(note that +these superpixels are not the same as those used for either the +background model or the PSF parameter variations)}. The superpixels are assigned to one of four groups so that each superpixel in a group is well separated from the other superpixels of that group. @@ -1498,4 +1607,6 @@ considering the nearby pixels from neighboring superpixel (guaranteed not to be in the current thread group). + +\note{explain number of superpixels (psphotThreadTools.c)} As the threads complete their analysis, they are assigned the next @@ -1589,5 +1700,5 @@ one annulus to the next is less than a user-defined limit, then the annulus at which the slope reaches this limit is used to define the -sky radius. These values are saved in the output smf / cmf files, but +sky radius. These values are saved in the \textmod{output FITS catalog files}, but not sent to the PSPS. The sky radius value is used below in the calculation of the Kron magnitude. @@ -1625,5 +1736,5 @@ surface brightness. The aperture is constrained to be less than a maximum value defined such that the minimum surface brightness is -1/2$times$ the effective surface brightness of a point source detected at the +1/2$\times$ the effective surface brightness of a point source detected at the $5\sigma$ limit. @@ -1636,5 +1747,9 @@ suppressed by the matched pixel on the other side. This trick has the effect of reducing the impact of pixels which include flux from near -neighbors. +neighbors. \textadd{We found it necessary to apply this filter because, +although the source models have been subtracted, at this point in the +analysis, only PSF models have been used. Thus extended objects +(galaxies) can leave behind significant amounts of flux to contaminate +the neighbors.} % \note{give a test example} @@ -1645,10 +1760,10 @@ After the PSF model has been fitted to all sources, and the Kron flux has been measured for all sources, \ippprog{psphot} uses these two -measurements, along with some additional pixel-level analysis, to -determine the size class of the source. Sources identified as +measurements, along with some additional pixel-level analysis, \textmod{for +classification based on source size.} Sources identified as extended will be fitted with a galaxy model (or possibly another type -of extended source model in special cases). If the source is small +of extended source model in special cases). \textadd{If the source is small compared to a PSF, it is considered to be a {\em cosmic ray} and -masked. +masked.} Extended sources are identified as those for which the Kron magnitude @@ -1660,5 +1775,5 @@ star. The result is divided by the quadrature error of the PSF and Kron magnitudes and called \code{extNsigma}. If \code{extNsigma} is -larger than \code{PSPHOT.EXT.NSIGMA.LIMIT} (3.0), the source is +larger than the configuration value \code{PSPHOT.EXT.NSIGMA.LIMIT} (3.0 for PV3), the source is considered to be extended and the flag bit \code{PM_SOURCE_MODE_EXT_LIMIT} is set for the source. @@ -1830,11 +1945,13 @@ exclusion stage are subtracted from the image. The subtraction process modifies the image pixels (removing the fitted flux, though -not the locally fitted background) but does not modify the mask or the -variance images. The signal-to-noise ratio in the image after -subtraction represents the significance of the remaining flux. If the +not the locally fitted background)\note{is the background actually + fitted locally?} but does not modify the mask or the variance +images. The signal-to-noise ratio in the image after subtraction +represents the significance of the remaining flux. If the subtractions are sufficiently accurate models of the PSF flux -distribution, the remaining flux should be below 1 $\sigma$ -significance. In practice the cores of bright stars are poorly -represented and may have larger residual significance. +distribution, \textmod{the remaining flux should be normally distributed about +zero with a standard deviation of 1 $\sigma$}. In practice the cores +of bright stars are poorly represented and may have larger residual +significance. For sources in groups of blended stars, the resulting fits are @@ -1895,5 +2012,5 @@ image is not modified. -For the single exposure (\ippstage{camera}) and \ippstage{stack} image +For the single exposure (\ippstage{chip}) and \ippstage{stack} image analysis, these galaxy model fits are only used internally to generate a clean object-subtracted residual image. For the PV3 analysis of the @@ -1949,4 +2066,6 @@ on one image based on detections in other images have the flag bit \code{PM_SOURCE_MODE2_MATCHED} set. + +\note{need to discuss the injection \& recovery analysis of the completeness} \subsection{Aperture Correction and Total Aperture Fluxes} @@ -1979,12 +2098,16 @@ fraction of the total source flux. Even more importantly, as the image conditions change, the amount lost will change by an even -smaller fraction, at least for a large aperture. This can be seen by -the fact that the dominant variations in the image quality are in the -focus, tracking and seeing. All of these errors initially affect the -cores of the stellar images, rather than the wide wings. The wide -wings are largely dominated by scattering in the optics and scattering -in the atmosphere. The amplitude and distribution of these two -scattering functions do not change significantly or quickly for a -single telescope and site. Aperture photometry can then be used to +smaller fraction, at least for a large aperture. +% +% This can be seen by +% the fact that the dominant variations in the image quality are in the +% focus, tracking and seeing. All of these errors initially affect the +% cores of the stellar images, rather than the wide wings. The wide +% wings are largely dominated by scattering in the optics and scattering +% in the atmosphere. The amplitude and distribution of these two +% scattering functions do not change significantly or quickly for a +% single telescope and site. +% +Aperture photometry can then be used to correct the PSF photometry. @@ -2111,14 +2234,4 @@ %%% term. -\ippprog{psphot} allows a collection of PSF model functions to be tried on all -PSF candidate sources. For each model test, the above corrected -ApResid scatter is measured. The PSF model function with the smallest -value for the ApResid scatter is then used by \ippprog{psphot} as the best PSF -model for this image. The number of models to be tested is specified -by the configuration keyword \code{PSF_MODEL_N}. The configuration -variables \code{PSF_MODEL_0}, \code{PSF_MODEL_1}, through -\code{PSF_MODEL_N - 1} specify the names of the models which should be -tested. - \subsection{Stellar Photometry Example} @@ -2191,5 +2304,5 @@ %% step ($S/N > 20$, Section~\ref{sec:xxxx}). -The extended source analysis is not applied to all object which may be +The extended source analysis is not applied to all \textmod{objects} which may be galaxies. Several restrictions are possible within the software. For example, it is possible to limit which objects are processed by their @@ -2311,4 +2424,6 @@ output file FITS header (\code{RMIN_NN}, \code{RMAX_NN}). +\note{specify PV3 config values?} + % \note{these profiles are not saved in PSPS} @@ -2319,6 +2434,6 @@ ratio of surface brightnesses. The motivation is to define an aperture which can be determined for galaxies without significant -biases as a function of distance from the observer. Since surface -brightness in a resolved source is conserved as a function of +biases as a function of distance from the observer. \textmod{Since the surface +brightness profile} in a resolved source is conserved as a function of distance, using a ratio of surface brightness to define a spatial scale results in a spatial scale which is constant regardless of @@ -2421,5 +2536,5 @@ fewer. The 1st radial moment (see \ref{sec:moments}) is used to estimate the effective radius of the -model based on the results of Graham \& Driver (2005, Table 1). They +model based on the results of \cite[][Table1]{2005PASA...22..118G}. They quantify the relationships between the first radial moment used to calculated a Kron Magnitude and the effective radius for different @@ -2447,9 +2562,4 @@ with the PSF model. -We simplify this by defining: -\begin{eqnarray} -f_p (a_m) & = & \frac{1}{\sigma_p} (I_p - M_p \otimes \mbox{PSF}) \\ -\end{eqnarray} - To determine the minimization, we need the gradient and laplacian of $\chi^2$ with respect to the model parameters, $a_m$: @@ -2460,7 +2570,13 @@ 2 H_{m,n} & = & \sum_p \frac{\partial f_p}{\partial a_m} \frac{\partial f_p}{\partial a_n} \end{eqnarray} -where we have approximated the Laplacian with the Hessian matrix, +where we define +\begin{eqnarray} +f_p (a_m) & = & \frac{1}{\sigma_p} (I_p - M_p \otimes \mbox{PSF}) +\end{eqnarray} +and we have approximated the Laplacian with the Hessian matrix, $H_{m,n}$ by dropping the second-derivatives (which are assumed to be -a small perturbation). Since +a small perturbation). + +Since \[ \frac{\partial f_p}{\partial a_m} = -\frac{1}{\sigma_p}\frac{\partial M_p \otimes \mbox{PSF}}{\partial a_m} @@ -2486,7 +2602,7 @@ parameters compared to the local-linear expectation and small when the last change was small. The iteration ends when the change in the -parameters is small and/or the change in the $\chi^2$ value is small. - -In the analysis, convolved galaxy fit, the galaxy model image and the +parameters is small or the change in the $\chi^2$ value is small. + +In the analysis, convolved galaxy fits, the galaxy model image and the model derivative images must be convolved with the PSF at each iteration step. To save computation time, this convolution is @@ -2577,4 +2693,6 @@ additions, or up to $6 \times$ that number if we interpolate between any of the parameters. + +\note{how much error does this approximation introduce?} \subsection{Fixed Aperture Photometry} @@ -3162,7 +3280,7 @@ negative (minuend) images. We identify the closest source in both the positive and negative images to the detection in the difference image, -out to a maximum of \code{INPUT.MATCH.RADIUS} (= 50 pixels), but only +out to a maximum of \code{INPUT.MATCH.RADIUS} (50 pixels for PV3), but only if the source in those images has a signal-to-noise greater than -\code{INPUT.MATCH.MIN.SN} (= 10). If there is a close neighbor in the +\code{INPUT.MATCH.MIN.SN} (10 for PV3). If there is a close neighbor in the positive image, and the difference in the magnitudes of the source in that image and the source in the difference image is less than 5 @@ -3206,5 +3324,5 @@ \section{Conclusions} -The Pan-STARRS Image Processing Pipeline has used the \code{psphot} +The Pan-STARRS Image Processing Pipeline has used the \ippprog{psphot} software to detect and characterize astronomical sources in images from both the PS\,1 and PS\,2 telescopes since 2008. This software @@ -3238,6 +3356,6 @@ \bibliographystyle{apj} -%\bibliography{lib}{} -\input{analysis.bbl} +\bibliography{lib}{} +%\input{analysis.bbl} \end{document}