Index: trunk/doc/release.2015/ps1.analysis/analysis.tex =================================================================== --- trunk/doc/release.2015/ps1.analysis/analysis.tex (revision 41315) +++ trunk/doc/release.2015/ps1.analysis/analysis.tex (revision 41316) @@ -45,5 +45,6 @@ \def\Princeton{2} \def\DUR{3} -\def\CfA{2} +\def\MPIA{4} +\def\CfA{5} % This example has a first author from UH: @@ -60,4 +61,5 @@ L. Denneau,\altaffilmark{\IfA} P.~W. Draper,\altaffilmark{\DUR} +D. Farrow,\altaffilmark{\DUR,\MPIA} R. Jedicke,\altaffilmark{\IfA} K. W. Hodapp,\altaffilmark{\IfA} @@ -86,5 +88,5 @@ % \altaffiltext{\USNO}{US Naval Observatory, Flagstaff Station, Flagstaff, AZ 86001, USA} % \altaffiltext{\JHU}{Department of Physics and Astronomy, Johns Hopkins University, 3400 North Charles Street, Baltimore, MD 21218, USA} -% \altaffiltext{\MPIA}{Max Planck Institute for Astronomy, K\"onigstuhl 17, D-69117 Heidelberg, Germany} +\altaffiltext{\MPIA}{Max Planck Institute for Astronomy, K\"onigstuhl 17, D-69117 Heidelberg, Germany} \begin{abstract} @@ -105,6 +107,4 @@ \keywords{methods: data analysis -- Surveys:\PSONE -- techniques: image processing -- techniques: photometric} -\note{add Danny Farrow to author list} - \section{Introduction} \label{sec:intro} @@ -155,10 +155,9 @@ for hazardous asteroids, funded by the NASA NEO Program. Additional partners collaborate with the Pan-STARRS team to harvest the transient -sources such supernovae and graviational wave counterparts -\note{REFS}. A second Pan-STARRS telescope (PS2), generally matching -the PS1 design \citep{Morgan2012} has since been constructed and has -been producing science results since early 2018. - -%The Processing Version 3 (PV3) reduction represents the third full +sources such supernovae and graviational wave counterparts. A second +Pan-STARRS telescope \citep[PS2][]{chambers2017,chambers2020}, +generally matching the PS1 design \citep{Morgan2012} has since been +constructed and has been producing science results since early 2018. + Pan-STARRS produced its first large-scale public data release, Data Release 1 (DR1) on 16 December 2016. DR1 contains the results of the @@ -176,6 +175,21 @@ measurements from all of the individual exposures, and includes an improved \textmod{astrometric calibration as well as improvements to the - photometric calibration of the stack and 'forced warp' measurements + photometric calibration of the stack and `forced warp' measurements from} the PV3 processing of that dataset. + +\textadd{The Pan-STARRS public data releases are hosted by the {\em + Barbara A. Mikulski Archive for Space Telescopes} (MAST) at the + Space Telescope Science Institute (STScI). MAST provides access to + the image data products and a hierachical database of measurements + using a system developed specifically for the Pan-STARRS dataset. + Development of this database systems was the product of a + collaboration between the Pan-STARRS Project and Alex Szalay's + database development group at The Johns Hopkins University (JHU) + \citep{2008AIPC.1082..352H}. The resulting system, called the + {\em Published Science Products Subsystem}, or PSPS + \citep{Heasley2006}, was initially used within the Pan-STARRS + Science Consortium for large-scale data access. A duplicate + PSPS installation was created at MAST for the DR1 and DR2 + public releases.} This is the fourth in a series of seven papers describing the @@ -184,5 +198,5 @@ source detection and photometry, including point-spread-function and extended source model fitting, and the techniques for ``forced'' -photometry measurements. \textadd{The same analysis software is used +photometry measurements. \textadd{The same analysis software, called \ippprog{psphot}, is used for individual images, image stacks, and difference images.} The software described here was used with a @@ -191,5 +205,9 @@ analysis of the Medium Deep Survey data, though with a different software version and some modifications of -the analysis parameters to better suite the longer exposures.} +the analysis parameters to better suite the longer exposures. This +program as well as the rest of the Pan-STARRS Image Processing +Pipeline (IPP) software suite is available for download from \url{http:ipp.ifa.hawaii.edu}}. + +\note{Generate a tarball of just the programs (skip certain directories)} %Chambers et al. 2017 (Paper I) @@ -215,5 +233,4 @@ and resulting image products and their properties. - %Magnier et al. 2017 (Paper IV) %Pan-STARRS Pixel Analysis : Source Detection @@ -226,9 +243,9 @@ describe the final calibration process, and the resulting photometric and astrometric quality. - %Flewelling et al. 2017 (Paper VI) %Pan-STARRS 1 Database and Data Products \citet[][Paper VI]{flewelling2017} -describe the details of the resulting catalog data and its organization in the Pan-STARRS database. +describe the details of the resulting catalog data and its +organization in the Pan-STARRS database system, PSPS. %Huber et al. 2017 (Paper VII) @@ -286,5 +303,5 @@ efficient. Not only is it necessary to make a careful measurement of the flux of individual sources, it is also critical to characterize -the image point-spread-function, and its variations across the field +the image point spread function (PSF), and its variations across the field and from image to image. Since comparisons between images must be reliable, the measurements must be stable for both photometry and @@ -501,5 +518,6 @@ \end{itemize} -\note{get a better example of the psphot accuracy achieved} +\note{Discuss the psphot photometry accuracy and the ubercal solution, + etc. mention Paper V} \textadd{The success of the \ippprog{psphot} implementation is meeting @@ -520,5 +538,5 @@ \item {\bf Initial Source Detection} Smooth, find peaks, measure basic - properties. + properties with focus on the point sources to measure the PSF. \item {\bf PSF Determination} Select PSF candidates, perform model @@ -562,5 +580,7 @@ \begin{table*} -\caption{\label{tab:measurements} \nocode{psphot} measurements performed} % \vspace{-0.5cm} +\caption{\label{tab:measurements} Measurements performed by + \nocode{psphot}, and whether performed in each of the 4 IPP analysis + stages. The analysis is described in this article in the listed Sections. } % \vspace{-0.5cm} \begin{center} \footnotesize @@ -569,28 +589,28 @@ \hline {\bf Measurement} & {\sc \bf CHIP} & {\sc \bf STACK} & {\sc \bf FORCED - WARP} & {\sc \bf DIFF} & {\bf Section} & {\bf Which} \\ + WARP} & {\sc \bf DIFF} & {\bf Section} & {\bf Details} \\ \hline Background Subtraction & Y & Y & Y & N$^1$ & \ref{sec:image.preparation} & N/A \\ - Peaks & Y & Y & N & Y & \ref{sec:peaks} & All \\ - Footprints & Y & Y & N & Y & \ref{sec:footprints} & All \\ - Moments & Y & Y & Y & Y & \ref{sec:moments} & All \\ + Peaks & Y & Y & N & Y & \ref{sec:peaks} & All detections \\ + Footprints & Y & Y & N & Y & \ref{sec:footprints} & All detections \\ + Moments & Y & Y & Y & Y & \ref{sec:moments} & All detections \\ PSF Model & Y & Y & Y & N$^2$ & \ref{sec:PSF.Model} & Uses bright, unsat. stars \\ Bright Star Profile & Y & Y & N & Y & \ref{sec:very.bright.star} & Saturated Stars \\ - Radial Profiles v1 & Y & Y & N & Y & \ref{sec:radial.profile} & All \\ - Kron Fluxes & Y & Y & Y & Y & \ref{sec:kron.mags} & All \\ - Source-Size Tests & Y & Y & N & Y & \ref{sec:source.size} & All \\ + Radial Profiles v1 & Y & Y & N & Y & \ref{sec:radial.profile} & All detections \\ + Kron Fluxes & Y & Y & Y & Y & \ref{sec:kron.mags} & All detections \\ + Source-Size Tests & Y & Y & N & Y & \ref{sec:source.size} & All detections \\ Non-Linear PSF Fits & Y & Y & N & N & \ref{sec:nonlinear.psf.model} & $S/N > 20$ \\ Unconvolved Galaxy Model & Y & Y & N & N & \ref{sec:nonlinear.galaxy.model} & $S/N > 20$, extended \\ Unconvolved Streak Model & N & N & N & Y & \ref{sec:nonlinear.galaxy.model} & $S/N > 20$, extended \\ - Linear PSF Fits & Y & Y & Y & Y & \ref{sec:faint.psf.model} & All \\ + Linear PSF Fits & Y & Y & Y & Y & \ref{sec:faint.psf.model} & All detections \\ Radial Profiles v2 & Y & Y & N & Y & \ref{sec:radial.profile.v2} & Gal. Latitude Cut \\ Petrosian Fluxes & N & Y & Y & N & \ref{sec:petrosian} & Gal. Latitude Cut \\ Convolved Galaxy Models & N & Y & N & N & \ref{sec:galaxy.conv.fit} & Gal. Latitude Cut, mag cut \\ - Fixed Aperture Photometry & N & Y & Y & N & \ref{sec:fixed.aperture.photom} & All \\ - Convolved, Fixed Apertures & N & Y & N & N & \ref{sec:fixed.aperture.photom} & All \\ - Aperture Corrections & Y & Y & Y & N & \ref{sec:aperture.correction} & All \\ - Forced PSF Fluxes & N & N & Y & N & \ref{sec:psf.forced.fit} & All \\ - Forced Galaxy Models & N & N & Y & N & \ref{sec:galaxy.forced.fit} & Have Stack Galaxy Models \\ - Lensing Parameters & N & Y & Y & N & \ref{sec:lensing.params} & All \\ + Fixed Aperture Photometry & N & Y & Y & N & \ref{sec:fixed.aperture.photom} & All detections \\ + Convolved, Fixed Apertures & N & Y & N & N & \ref{sec:fixed.aperture.photom} & All detections \\ + Aperture Corrections & Y & Y & Y & N & \ref{sec:aperture.correction} & All detections \\ + Forced PSF Fluxes & N & N & Y & N & \ref{sec:psf.forced.fit} & All detections \\ + Forced Galaxy Models & N & N & Y & N & \ref{sec:galaxy.forced.fit} & Requires stack galaxy models \\ + Lensing Parameters & N & Y & Y & N & \ref{sec:lensing.params} & All detections \\ \hline \multicolumn{5}{l}{$^1$ Background subtraction is performed by {\tt ppSub} before calling {\tt psphot}} \\ @@ -609,4 +629,5 @@ these two fields. These informational and warning bits are described in more detail later in this article. + % Table~\ref{tab:det_flag_values} lists the flags recorded in the output @@ -622,5 +643,5 @@ output field \ippmisc{FLAGS2}. When data from \ippprog{psphot} is loaded into a DVO database \citep{magnier2017.calibration}, these -values are not currently loaded, but they are exposed in PSPS in the fields +values are stored in the field \ippdbtable{Measure.photFlags2}, and they are exposed in PSPS in the fields \ippdbtable{Detection.infoFlag2}, \ippdbtable{StackObjectThin.XinfoFlag2} (where \ippdbtable{X} is one of {$grizy$}), and @@ -628,6 +649,13 @@ \begin{table*} -\caption{\label{tab:det_flag_values} \nocode{psphot} Detection Flag Values \#1} % \vspace{-0.5cm} \begin{center} +\caption{\label{tab:det_flag_values} +Detection Flag +Values \#1 reported by \texttt{psphot}. These are saved in output catalogs as the field +\texttt{FLAGS}, in the DVO database as +\textit{Measure.photFlags}, and in the public database as +\textit{Detection.infoFlag}, +\textit{StackObjectThin.XinfoFlag} (where \textit{X} is one +of {$grizy$}), and \textit{ForcedWarpMeasurement.FinfoFlag}.} \footnotesize \begin{tabular}{lrl} @@ -636,10 +664,10 @@ {\bf Flag Name} & {\bf Flag Value} & {\bf Description} \\ \hline - PM\_SOURCE\_MODE\_PSFMODEL & 0x00000001 & Source fitted with a psf model (linear or non-linear) \\ + PM\_SOURCE\_MODE\_PSFMODEL & 0x00000001 & Source fitted with a PSF model (linear or non-linear) \\ PM\_SOURCE\_MODE\_EXTMODEL & 0x00000002 & Source fitted with an extended-source model \\ PM\_SOURCE\_MODE\_FITTED & 0x00000004 & Source fitted with non-linear model (PSF or EXT; good or bad) \\ PM\_SOURCE\_MODE\_FAIL & 0x00000008 & Fit (non-linear) failed (non-converge, off-edge, run to zero) \\ - PM\_SOURCE\_MODE\_POOR & 0x00000010 & Fit succeeds, but low-SN, high-Chisq, or large (for PSF -- drop?) \\ - PM\_SOURCE\_MODE\_PAIR & 0x00000020 & Source fitted with a double psf \\ + PM\_SOURCE\_MODE\_POOR & 0x00000010 & Fit succeeds, but low-S/N or high chi-square \\ + PM\_SOURCE\_MODE\_PAIR & 0x00000020 & Source fitted with a double PSF \\ PM\_SOURCE\_MODE\_PSFSTAR & 0x00000040 & Source used to define PSF model \\ PM\_SOURCE\_MODE\_SATSTAR & 0x00000080 & Source model peak is above saturation \\ @@ -675,5 +703,13 @@ \begin{table*} -\caption{\label{tab:det_flag2_values} \nocode{psphot} Detection Flag Values \#2} % \vspace{-0.5cm} +\caption{\label{tab:det_flag2_values} +Detection Flag Values \#2 reported by \nocode{psphot}. +These are saved in output catalogs as the field +\texttt{FLAGS2}, in the DVO database as +\textit{Measure.photFlags2}, and in the public database as +\textit{Detection.infoFlag2}, +\textit{StackObjectThin.XinfoFlag2} (where \textit{X} is one +of $grizy$), and \textit{ForcedWarpMeasurement.FinfoFlag2}. +} \begin{center} \footnotesize @@ -772,8 +808,33 @@ sources. Table~\ref{tab:mask_values} lists the 16 bit values used for PS1 mask images, along with their description \citep[see][for - additional information]{waters2017}. + additional information]{waters2017}. + +{\bf An important point to note is that \ippprog{psphot} does not + attempt to interpolate or replace bad pixel values in the images + before processing. The GPC1 images have quite extensive masking due + to both defects and natural gaps between detectors and amplifier + regions. On average, roughly 71\% of the full useable field-of-view + is covered with valid pixels (See Paper III for more discussion). + Any attempt to interpolate bad pixels would be quickly overwhelmed + by these extensive regions. Rather than attempt to fill in the bad + pixels, we rely in the PS1 PV3 processing on the fact that regions + on the sky were observed many times. Thus, it should be noted that + model-fitting measurements (which can naturally ignore masked + pixels) should generally be more reliable than aperture-like + measurements for single exposures. Aperture-like measurements from + the stacks do not suffer from this masking issue. See also the + discussion of the \ippmisc{PSF_QF} and \ippmisc{PSF_QF_PERFECT} + parameters for judging the impact of masking on a particular source + (Section~\ref{sec:psf.model.choice}).} \begin{table*} -\caption{\label{tab:mask_values} \nocode{psphot} / GPC1 Mask Image Pixel Values} % \vspace{-0.5cm} + +\caption{\label{tab:mask_values} Pixel values for input GPC1 mask + images used by \nocode{psphot}. The table gives the bit value used + to mark the listed effects. Bits marked as `dynamic' are set for + each image based on the contents, such as the locations of bright + stars. Bits marked as `suspect' represent effects which do not + definitely affect the photometry, but users should be careful. The + mask image headers also list these values.} % \vspace{-0.5cm} \begin{center} \footnotesize @@ -906,4 +967,10 @@ %% is there a ref I can use for the optimal detection? see SDSS docs? +The initial source detection step is focused on finding and +identifying the brighter point sources. The goals are two-fold: 1) to +select sources which can be used to model the PSF and 2) to subtract +the brighter sources so that fainter sources may be found throughout +the image . + The sources are initially detected by finding the location of local peaks in the image. The flux and variance images are smoothed with a @@ -947,5 +1014,5 @@ \[ f(x,y) = C_{00} + C_{10}x + C_{01} y + C_{11} x y + C_{20} x^2 + C_{02} y^2 \] -and write the Chi-Square equation: +and write the chi-square equation: \[ \chi^2 = \sum_{i,j} (F_{i,j} - f(x,y))^2 / \sigma_{i,j}^2 \] @@ -1004,6 +1071,7 @@ \caption{\label{fig:peaks} Illustration of peak finding and culling peaks within a footprint. Insignificant peaks within the footprint of a brighter - peak are ignored in further processing. \note{NOTE that the - diagram is a 1D rep of a 2D path.}} + peak are ignored in further processing. Note that this 1D + illustration is representative of the full 2D path which may be + followed from one peak to the next.} \end{center} \end{figure} @@ -1026,21 +1094,22 @@ For any peak which is not the brightest peak in that footprint it is -possible to reach the brightest peak by following a sequence of the highest valued -pixels between the two peaks. The lowest pixel along this -\textadd{(potentially meandering)} path is the -{\em key col} for this peak (as used in topographic descriptions of a -mountain). If the key col for a given peak is less than -\code{FOOTPRINT_CULL_NSIGMA_DELTA} (4.0 for PV3) sigmas below the -peak of interest, the peak is considered to be {\em locally - insignificant} and removed from the list of possible detections (see -Figure~\ref{fig:peaks}). \textadd{If more than one such path is possible, the -path with the highest key col is used for this test.} In the vicinity of a saturated star, the -rule is somewhat more aggressive as the flat-topped or structured -saturated top of a bright star may appear as multiple peaks with -highly significant cols between them. However, this is an artifact of -the proximity to saturation. Sources for which the peak is greater -than 50\% of the saturation value require the col to also be a fixed -fraction (5\%) of the saturation below the peak to avoid being marked -as locally insignificant. +possible to reach the brightest peak by following a sequence of the +highest valued pixels between the two peaks. The lowest pixel along +this \textadd{(potentially meandering)} path is the {\em key col} for +this peak (as used in topographic descriptions of a mountain). If the +key col for a given peak is less than +\code{FOOTPRINT_CULL_NSIGMA_DELTA} (4.0 for PV3) sigmas below the peak +of interest, the peak is considered to be {\em locally insignificant} +and removed from the list of possible detections (see +Figure~\ref{fig:peaks}). \textadd{If more than one such path is + possible, the path with the highest key col is used for this test.} +In the vicinity of a saturated star, the rule is somewhat more +aggressive as the flat-topped or structured saturated top of a bright +star may appear as multiple peaks with highly significant cols between +them. However, this is an artifact of the proximity to saturation. +Sources for which the peak is greater than 50\% of the saturation +value require the col to also be a fixed fraction (5\%) of the +saturation below the peak to avoid being marked as locally +insignificant. Sometimes it is useful to know if a source has a near neighbor which @@ -1064,5 +1133,5 @@ \begin{figure}[htbp] \begin{center} - \includegraphics[width=0.95\hsize]{{\picdir/FWHM.smooth.trend.ps1}.\plotext} + \includegraphics[width=0.95\hsize]{{\picdir/FWHM.smooth.trend.v1.ps1}.\plotext} \caption{\label{fig:moments.window} Example of the biases encountered when measuring the second moments. A simulated image @@ -1088,6 +1157,11 @@ Once a collection of peaks has been identified, a number of basic properties of the sources related to the first, second, and higher -moments are measured. Below, the second moments are used to select -candidate stellar sources to be used in modeling the PSF. +moments are measured. \textmod{These moments can be used for a crude + classification of the sources. As discussed below, the second + moments are used to select candidate stellar sources to be used in + modeling the PSF and the exclude `cosmic rays' and extended sources. + The radial moment is used in the measurement of the Kron magnitudes \citep{1980ApJS...43..305K}. + The higher-order moments are provided primarily for image quality + diagnostics.} In order to measure the moments, it is necessary to define an @@ -1188,4 +1262,17 @@ centroid is used to center the window function. +\textadd{The motivation of measuring these higher order moments was to + select exposures with image quality problems. For example, trefoil + caused by errors in the collimation and alignment can in principle + be detected with the third-order moments. In our experience, these + statistics can be used to select some images with such problems, but + we have not been able to use these values to exclude poor images + from the data processing. If we were to reject images based on + these moments, we would reject too many images with image quality + issues that are not so poor as to preclude a useful analysis. A + future machine-learning based analysis starting with these moments + might potentially provide a better rejection statistic, but such + work is beyond the scope of this article.} + For sources with peak flux above the saturation limit, the moments are generally poorly measured if the aperture defined by $\sigma_w$ is @@ -1219,4 +1306,5 @@ $M_r$ and $M_h$ as defined below, are calculated: \begin{eqnarray} +\label{eqn:first.radial.moment} M_r & = & \frac{1}{S} \sum_i (f_i - s_i)r_i \\ M_h & = & \frac{1}{S} \sum_i (f_i - s_i)\sqrt{r_i} @@ -1226,21 +1314,24 @@ With the first radial moment, we can calculate a preliminary Kron -radius and magnitude. The Kron radius \citep{1980ApJS...43..305K} is -defined the be 2.5$\times$ the first radial moment. The Kron flux is -the sum of (sky-subtracted) pixel fluxes within the Kron radius. We -also calculate the flux in two related annular apertures: the Kron -inner flux is the sum of pixel values for the annulus $R_1 < r < 2.5 -R_1$, while the Kron outer flux is the sum of pixel values for $2.5 -R_1 < r < 4 R_1$. The first radial moment is limited at the low and -high ends by $R_{\rm min} < M_r < R_{\rm max}$ where $R_{\rm min}$ is -the first radial moment of the PSF stars, or $0.75\sigma_w$ if that -cannot be determined. $R_{\rm max}$ is set to the size of the moments -aperture, $4\sigma_w$. These Kron measurements are performed for all -sources with a valid set of moments. At this stage, the measurement -of the Kron parameters are preliminary since the aperture has been -chosen as a fixed size relative to the size of the PSF. At a later -stage, higher-quality Kron parameters appropriate to galaxies are -measured with more care paid to the exact aperture used -(Section~\ref{sec:kron.mags}). +radius and magnitude. \textadd{Kron magnitudes are provided as an option for +galaxy photometry. In addition, the comparison of Kron and PSF +magnitudes is useful as a star-galaxy separator.} The Kron radius +\citep{1980ApJS...43..305K} is defined the be 2.5$\times$ the first + radial moment. The Kron flux is the sum of (sky-subtracted) pixel + fluxes within the Kron radius. We also calculate the flux in two + related annular apertures: the Kron inner flux is the sum of pixel + values for the annulus $R_1 < r < 2.5 R_1$, while the Kron outer + flux is the sum of pixel values for $2.5 R_1 < r < 4 R_1$. The + first radial moment is limited at the low and high ends by $R_{\rm + min} < M_r < R_{\rm max}$ where $R_{\rm min}$ is the first radial + moment of the PSF stars, or $0.75\sigma_w$ if that cannot be + determined. $R_{\rm max}$ is set to the size of the moments + aperture, $4\sigma_w$. These Kron measurements are performed for + all sources with a valid set of moments. At this stage, the + measurement of the Kron parameters are preliminary since the + aperture has been chosen as a fixed size relative to the size of the + PSF. At a later stage, higher-quality Kron parameters appropriate + to galaxies are measured with more care paid to the exact aperture + used (Section~\ref{sec:kron.mags}). % $\sigma_w$ is saved as MOMENTS_GAUSS_SIGMA @@ -1253,5 +1344,5 @@ \label{sec:Source.Model} -The point-spread-function (PSF) of an image describes the shape of all +The point spread function (PSF) of an image describes the shape of all unresolved sources in the image. In a typical wide-field image, the shape of unresolved sources varies as a function of position in the @@ -1270,4 +1361,5 @@ elliptical Gaussian: \begin{eqnarray} +\label{eqn:2d.gaussian} f(x,y) & = & I_o e^{-z} + S \\ z & = & \frac{x^2}{2\sigma_x^2} + \frac{y^2}{2\sigma_y^2} + \sigma_{\rm xy} x y \\ @@ -1360,8 +1452,12 @@ \begin{center} \includegraphics[width=\hsize]{{\picdir/radial.profiles}.\plotext} - \caption{\label{fig:radial.profiles} Radial profiles of stellar images from PS1. These two - profiles illustrate the radial trend of the PS1 PSFs for a star - with FWHM 0.9 arcsec (red) and 2.2 arcsec (blue). The black line - shows the PSF model with radial trend of the form $(1 + \kappa r^2 + r^{3.33})^{-1}$.} + + \caption{\label{fig:radial.profiles} Radial profiles of stellar + images from PS1. These two profiles illustrate the radial trend + of the PS1 PSFs for a star with FWHM 0.9 arcsec (red) and 2.2 + arcsec (blue). The red and blue points are individual pixel + values. The black line shows the PSF model with radial trend of + the form $(1 + \kappa r^2 + r^{3.33})^{-1}$.} + \end{center} \end{figure} @@ -1479,5 +1575,5 @@ model, allowing all of the parameters (PSF and independent) to vary in the fit. The software uses the Levenberg-Marquardt minimization -technique \citep{1992nrca.book.....P,Madsen} for the non-linear fitting. Non-linear +technique \citep[e.g.,][]{1992nrca.book.....P,Madsen} for the non-linear fitting. Non-linear fitting can be very computationally intensive, particularly if the starting parameters are far from the minimization values. The first @@ -1485,4 +1581,56 @@ shape parameters for the PSF models. Any sources which fail to converge in the fit are flagged as invalid. + +{\bf +To generate the initial guess, the second moments are converted to the equivalent sigma values for a +2D elliptical Gaussian contour using the following transformations +inspired by \cite{sextractor,1980JBIS...33..323S}. +First, we calculate the sigma values in the major ($\sigma_a$) and +minor ($\sigma_b$) axis directions, along with the position angle +$\theta$ from the moments using: +\begin{eqnarray} +\theta & = & \frac{1}{2} \arctantwo (2 M_{xy}, g_2) \\ +\sigma_a & = & \sqrt{\frac{g_1 + g_3}{2}} \\ +\sigma_b & = & \sqrt{\frac{g_1 - g_3}{2}} +\end{eqnarray} +where the function $\arctantwo (y,x)$ returns the arctangent in the +proper quadrant (e.g,. as implemented by the \code{atan2(y,x)} +function in C) and the intermediate values $g_1$, $g_2$, $g_3$ are given by: +\begin{eqnarray} +g_1 & = & M_{xx} + M_{yy} \\ +g_2 & = & M_{xx} - M_{yy} \\ +g_3 & = & \sqrt{g_2^2 + 4 M_{xy}^2} +\end{eqnarray} +Since the moments may be noisy, the calculated value of $\sigma_b$ can +be numerically invalid if $g_3 > g_1$, a situation which is especially +likely for highly ellongated sources. We avoid this situation by +limiting the axial ratio to a maximum of 20 (setting $\sigma_b$ to +$\sigma_a / 20$ if the expected axial ratio would be greater than this +limit). The selected value of 20 is somewhat ad-hoc, chosen based on +failures in real images. A more careful examination of the trade-off +space would be worthwhile in the future. + +With $\sigma_a$, $\sigma_b$, $\theta$ in hand, we can now transform +these values to the parameters of our fits, $\sigma_x$, $\sigma_y$, +$\sigma_{\rm xy}$ (Eqn~\label{eqn:2d.gaussian} above). This transformation +can be determined by rotating the 2D Gaussian equation, yielding: +\begin{eqnarray} +\sigma_x^{-2} & = & \sigma_a^{-2} \cos^2 \theta + \sigma_b^{-2}\sin^2 \theta \\ +\sigma_y^{-2} & = & \sigma_b^{-2} \cos^2 \theta + \sigma_a^{-2}\sin^2 \theta \\ +\sigma_{\rm xy} & = & \frac{1}{2} \sin (2 \theta) (\sigma_b^{-2} - \sigma_a^{-2}) +\end{eqnarray} +In fact, since the calculated second moments have been measured with a +window function applied (see discussion in Section~\ref{sec:moments}), we instead +use the measured value of $M_r$ (Eqn~\ref{eqn:first.radial.moment}), the first +radial moment as the major axis size for the Gaussian ($\sigma_a$), retaining +the position angle and axial ratio from the calculation above. We use +these guess parameters for all version of the PSF analytical models, +despite the fact that for the versions which are not approximations of +Gaussians these guess values will be systematically incorrect. +It would be worthwhile in the future to tweak the guesses for +the different model version to speed up the convergence. +} + +% https://www.astromatic.net/pubsvn/software/sextractor/trunk/doc/sextractor.pdf For the resulting collection of source model parameters, the @@ -1511,5 +1659,5 @@ \begin{table} \caption{\label{tab:psf.order.nstars} Minimum number of stars required - for a given order of the PSF 2D variations.} % \vspace{-0.5cm} + for a given order of the PSF 2D variations, or for the given number of grid cells.} % \vspace{-0.5cm} \begin{center} \begin{tabular}{llll} @@ -1573,20 +1721,25 @@ \ippdbtable{Detection.apFillF}. -When the PSF and aperture photometry for a source is measured, two -additional quantities are measured which are useful to assess the -quality of the measurements. First, the mask image is examined and the -number of unmasked pixels is summed, weighted by the normalized PSF -model. The resulting quantity, \code{PSF_QF} has a value between 0.0 -(totally masked) and 1.0 (totally unmasked). Elsewhere in the IPP -system, we use this value to filter out detections which are -unreliable due to the masking. For a generous cut, leaning toward -completeness at the cost of some lower quality measurements, -\code{PSF_QF} $> 0.85$ is used in some contexts; in other cases, we -require \code{PSF_QF} $> 0.95$ to ensure a high-quality measurement -\citep[see for example the calculation of average photometry - in][]{magnier2017.calibration}. The second quantity is related to -the first: \code{PSF_QF_PERFECT} uses all mask values to assess the -quality factor, while \code{PSF_QF} uses only the ``bad'' mask bit -values (see Section~\ref{sec:image.preparation}). +\textmod{As noted above (Section~\ref{sec:image.preparation}), we do not +attempt to replace or interpolate masked pixel values. Aperture +photometry measurements of objects which include masked pixels are +thus inaccurate. For a stellar object, the amount of error is a +function of how close the masked pixels are to the core of the PSF. +To provide guidance, when the PSF and aperture photometry for a source +is measured, two additional quantities are measured which are useful +to assess the impact of masking.} First, the mask image is examined +and the number of unmasked pixels is summed, weighted by the +normalized PSF model. The resulting quantity, \code{PSF_QF} has a +value between 0.0 (totally masked) and 1.0 (totally unmasked). +Elsewhere in the IPP system, we use this value to filter out +detections which are unreliable due to the masking. For a generous +cut, leaning toward completeness at the cost of some lower quality +measurements, \code{PSF_QF} $> 0.85$ is used in some contexts; in +other cases, we require \code{PSF_QF} $> 0.95$ to ensure a +high-quality measurement \citep[see for example the calculation of + average photometry in][]{magnier2017.calibration}. The second +quantity is related to the first: \code{PSF_QF_PERFECT} uses all mask +values to assess the quality factor, while \code{PSF_QF} uses only the +``bad'' mask bit values (see Section~\ref{sec:image.preparation}). Several flag bits are raised based on statistics which are similar to @@ -1668,5 +1821,5 @@ distribution. Note that in the case of very saturated stars, pixels in the central regions are largely masked, because they are -saturated. Thus in these cases, the psf-weighted masked fraction (see +saturated. Thus in these cases, the PSF-weighted masked fraction (see Section~\ref{sec:psf.model.choice}) is generally quite low or 0.0. Sources for which this radial profile is subtracted have the flag bit @@ -1808,8 +1961,8 @@ Extended sources are identified as those for which the Kron magnitude is significantly brighter than the PSF magnitude when compared to a -PSF star. The value $\delta M_{rm KP} = m_{\rm Kron} - m_{\rm PSF}$, +PSF star. The value $\delta M_{\rm KP} = m_{\rm Kron} - m_{\rm PSF}$, the difference between the PSF and Kron magnitudes, is calculated for -each source. The median of $\delta M_{rm KP}$ is calculated for the -PSF stars. This median is subtracted from $\delta M_{rm KP}$ for each +each source. The median of $\delta M_{\rm KP}$ is calculated for the +PSF stars. This median is subtracted from $\delta M_{\rm KP}$ for each star. The result is divided by the quadrature error of the PSF and Kron magnitudes and called \code{extNsigma}. If \code{extNsigma} is @@ -1817,4 +1970,12 @@ considered to be extended and the flag bit \code{PM_SOURCE_MODE_EXT_LIMIT} is set for the source. + +\textmod{We decided to use $\delta M_{\rm KP}$ metric for this + assessment after we tested several possible star-galaxy separation + statistics. We found that the Kron-PSF comparison was more reliable + than second-moment and first-radial-moment based measurements. In + addition, since we needed a statistic which could be calculated + relatively quickly on every detected source, we rejected using a + galaxy model fit for the star-galaxy separator.} Cosmic rays are identified by a combination of the Kron magnitude and @@ -1938,5 +2099,5 @@ %% than the PSF (ie, a cosmic ray or other defect). A user-defined %% number of standard deviations is used to select these two cases, and -%% to flag the source as a likely galaxy (really meaning 'extended') or +%% to flag the source as a likely galaxy (really meaning `extended') or %% as a likely defect. @@ -1964,5 +2125,5 @@ than a user-defined cutoff (set to 2.0 for the PV3 analysis of the $3\pi$ survey), the non-linear PSF fit will be rejected. If the -Chi-Square per degree of freedom is greater than a user-defined limit +$\chi^2$ per degree of freedom is greater than a user-defined limit (set to 50.0 for the PV3 analysis of the $3\pi$ survey), the non-linear PSF fit will be rejected. These sources are marked with @@ -2036,4 +2197,6 @@ comparing the ratio to that expected. +\note{more on the parameter guess} + For each type of extended source model (in fact for all source models), a function is defined which examines the fit results and @@ -2043,5 +2206,5 @@ case, the range of valid values for each of the parameters must be considered in the fit assessment. In other cases, we may choose to -use only the parameter errors and the fit Chi-Square value. +use only the parameter errors and the fit chi-square value. All extended source model fits which are successful are then @@ -2164,17 +2327,18 @@ \begin{figure*}[htbp] \begin{center} - \includegraphics[width=\hsize,clip]{\picdir/{mag.resid.psf}.\plotext} + \includegraphics[width=\hsize,clip]{\picdir/{mag.resid.psf.v1}.\plotext} \caption{\label{fig:mag.resid.psf} PSF Photometry demonstration. - The bottom panel shows the difference of the measured PSF - photometry for stars in the first image of the STS sequence - compared to the next 17 images, after correction for a relative - zero point. Black dots are from stars for which both measurements - have {\tt PSF\_QF} $> 0.95$, while grey dots have lower {\tt - PSF\_QF} values. The top three panels show histograms in three - instrumental magnitude ranges for the magnitude difference divided - by the reported measurement error: $N\sigma = (m_0 - m_1) / - \sqrt{\sigma_0^2 + \sigma_1^2}$. The red curves are Gaussian fits - to these histograms, with the measured standard deviations in the - upper-right corners of the plots. The instrumental magnitude + Panel (d) shows the difference of the measured PSF photometry for + stars in the first image of an image sequence with constant + pointing compared to the next 17 images, after correction for a + relative zero point, as a function of the instrumental magnitudes + above the detection threshold. Black dots are from stars for + which both measurements have {\tt PSF\_QF} $> 0.95$, while grey + dots have lower {\tt PSF\_QF} values. The top three panels (a) - + (c) show histograms in three magnitude ranges for the magnitude + difference divided by the reported measurement error: $N\sigma = + (m_0 - m_1) / \sqrt{\sigma_0^2 + \sigma_1^2}$. The red curves are + Gaussian fits to these histograms, with the measured standard + deviations in the upper-right corners of the plots. The magnitude ranges are listed in the upper-left corners of the three plots and the boundaries are marked as vertical red lines in the lower plot. @@ -2186,5 +2350,5 @@ \begin{figure*}[htbp] \begin{center} - \includegraphics[width=\hsize,clip]{\picdir/{mag.resid.aper}.\plotext} + \includegraphics[width=\hsize,clip]{\picdir/{mag.resid.aper.v1}.\plotext} \caption{\label{fig:mag.resid.aper} Aperture Photometry demonstration. The plots show identical measurements to those in @@ -2274,4 +2438,5 @@ \subsection{Stellar Photometry Example} +\label{sec:phot.example} To illustrate the quality of the stellar photometry as measured with @@ -2334,5 +2499,9 @@ magnitude; 3) convolved galaxy model fits; and 4) photometry in several fixed-sized apertures, both raw and convolved to a defined -PSF size. +PSF size. \textadd{The motivation for these measurements is to + provide options to the end users for galaxy photometry and reliable + galaxy colors. The photometric redshift analysis of + \cite{2012ApJ...746..128S}, for example, uses the convolved, + fixed-size aperture photometry.} %% NOTE: This is NOT true: extended source analysis applied to both @@ -2340,5 +2509,5 @@ %% %% In order for a source to be included in the extended source -%% analysis, it much have been detected in the 'bright source' analysis +%% analysis, it much have been detected in the `bright source' analysis %% step ($S/N > 20$, Section~\ref{sec:xxxx}). @@ -2363,9 +2532,19 @@ cut was defined by $|b| > b_{\rm min}$ where $b_{\rm min} = b_0 + r_b e^{\frac{-l^2}{2 \sigma_b^2}}$. For the PV3 analysis, $b_0 = -$20\degree, $r_b = $15\degree, $\sigma_b = $50\degree. This contour +$20\degree, $r_b = $15\degree, $\sigma_b = $50\degree. \textadd{The Galactic plane cut is made on an object-by-object basis.} This contour avoids the denser portions of the Galactic plane and bulge, limiting the total time spent on the galaxy modeling analysis at the expense of galaxy photometry in the plane (though Kron photometry is available -for those sources). +for those sources). + +% uses plots.sh in this directory +\begin{figure}[htbp] + \begin{center} + \includegraphics[width=\hsize,clip]{\picdir/galplanecut.pdf} + \caption{\label{fig:galplanecut} Illustration of the Galactic Plane + cut used for PV3, in Galactic coordinates. Objects within the red + contours are skipped for galaxy model fits and Petrosian parameters.} + \end{center} +\end{figure} % galaxy model fits performed based on limits set in psphotChooseAnalysisOptions.c @@ -2583,5 +2762,6 @@ radius values for all 3 model types. Once the effective radius is chosen, the second moments are used to define the aspect ratio and -position angle of the elliptical contour. The Kron flux is used to +position angle of the elliptical contour, \textadd{as described for PSF sources +in Section~\ref{sec:psf.model.choice}}. The Kron flux is used to generate a guess for the normalization, applying an appropriate scale factor based on the ($R_{xx}$, $R_{yy}$ , $R_{xy}$) values, generated @@ -2758,5 +2938,5 @@ of the same galaxy for all 5 filters. In this analysis, the best model for each source is subtracted from the image pixels for all -sources excluding the source in consideration. The 'best model' is +sources excluding the source in consideration. The `best model' is determined based on the minimum $\chi^2$ value for the model fits. @@ -2856,5 +3036,5 @@ figures may be compared with the reported detection limits from the PS1 $3\pi$ survey. Note for reference that the typical stellar -detection limits in the PS1 $3\pi$ stack images are (\grizy) = (23.3, +detection limits in the PS1 $3\pi$ stack images (Paper I) are (\grizy) = (23.3, 23.2, 23.1, 22.3, 21.4). The minimum Kron magnitudes for which galaxy model fits were performed for the PV3 analysis @@ -3036,4 +3216,7 @@ recalibration of the zero points for the individual warp. +\note{discuss the relative quality of average exposure, forced warp + average, and stack photometry. reference to Best et al} + \subsection{Forced Galaxy Models} \label{sec:galaxy.forced.fit} @@ -3102,7 +3285,7 @@ lensing, and thus directly measure mass distributions in the Universe. The classic approach was originally described by -\cite{1995ApJ...449..460K} and applied to a set of deep HST +\cite[KSB]{1995ApJ...449..460K} and applied to a set of deep HST observations. The details of the technique were further refined by -\cite{1998ApJ...504..636H}; in the discussion below we primarily use +\cite[HFK]{1998ApJ...504..636H}; in the discussion below we primarily use their notation, though we explicitly cast their integrals as sums over discrete pixels. @@ -3291,19 +3474,26 @@ galaxies. In the Pan-STARRS system, difference images are generated using the PSF-matching technique described by -\citep[e.g.,][]{1998ApJ...503..325A}. The description of the -Pan-STARRS implementation is given by \cite{price2017}. The analysis -of the sources detected in these difference images uses a portion of -the \ippprog{psphot} code embedded in the program, \ippprog{ppSub}, -which generates those image. - -\note{Note that this article is limited to the analysis of the - difference image detections, and that additional work is needed to - filter real/bogus. Refer to Denneau et al 2013 PASP for the MOPS analysis. Refer - to the Wright et al papers for the SNe classifications (& other - papers?). Mention Yuan \& Akerloff 2008.} - -\note{mention the 3 difference image modes (WW, WS, SS)} - -% https://ui.adsabs.harvard.edu/abs/2013PASP..125..357D/abstract +\citep[e.g.,][]{1998ApJ...503..325A}. \textmod{The description of the +Pan-STARRS implementation is given by \cite{price2017} and uses an +implementation of cross-convolution based on the description of +\cite{2008ApJ...677..808Y}. The analysis of the sources detected in +these difference images uses a portion of the \ippprog{psphot} code +embedded in the program, \ippprog{ppSub}, which generates those image. +Difference images are generated from three different possible image +combinations: 1) pairs of individual exposures are differenced using +the warp images; 2) warps for individual exposures +are differenced against deep stacks; 3) stacks made from multiple +exposures of the same field within a night are differenced against +deep stacks. Note that this article is limited to the analysis of the +difference image detections, and that significant additional work is +needed to distinguish real detections from false positives, and +further to classify the detections as objects of scientific interest. +Within the Pan-STARRS science community, the Moving Object Processing +System \citep[MOPS][]{2013PASP..125..357D} is dedicated to the effort +of identifying asteroids and other solar system objects. Multiple +teams have focused on the identification of supernovae +\citep{2014ApJ...795...44R,2015MNRAS.449..451W}, including the use of +machine-learning techniques to filter the good detections from the bad +detections.} The analysis of the difference image follows the same basic steps as