Index: /trunk/doc/release.2015/ps1.analysis/analysis.tex =================================================================== --- /trunk/doc/release.2015/ps1.analysis/analysis.tex (revision 40608) +++ /trunk/doc/release.2015/ps1.analysis/analysis.tex (revision 40609) @@ -545,33 +545,36 @@ * PM\_SOURCE\_MODE\_PSFMODEL & 0x00000001 & Source fitted with a psf model (linear or non-linear) \\ * PM\_SOURCE\_MODE\_EXTMODEL & 0x00000002 & Source fitted with an extended-source model \\ - PM\_SOURCE\_MODE\_FITTED & 0x00000004 & Source fitted with non-linear model (PSF or EXT; good or bad) \\ - PM\_SOURCE\_MODE\_FAIL & 0x00000008 & Fit (non-linear) failed (non-converge, off-edge, run to zero) \\ - PM\_SOURCE\_MODE\_POOR & 0x00000010 & Fit succeeds, but low-SN, high-Chisq, or large (for PSF -- drop?) \\ - PM\_SOURCE\_MODE\_PAIR & 0x00000020 & Source fitted with a double psf \\ - PM\_SOURCE\_MODE\_PSFSTAR & 0x00000040 & Source used to define PSF model \\ - PM\_SOURCE\_MODE\_SATSTAR & 0x00000080 & Source model peak is above saturation \\ - PM\_SOURCE\_MODE\_BLEND & 0x00000100 & Source is a blend with other sources \\ - PM\_SOURCE\_MODE\_EXTERNAL & 0x00000200 & Source based on supplied input position \\ - PM\_SOURCE\_MODE\_BADPSF & 0x00000400 & Failed to get good estimate of object's PSF \\ - PM\_SOURCE\_MODE\_DEFECT & 0x00000800 & Source is thought to be a defect \\ - PM\_SOURCE\_MODE\_SATURATED & 0x00001000 & Source is thought to be saturated pixels (bleed trail) \\ - PM\_SOURCE\_MODE\_CR\_LIMIT & 0x00002000 & Source has crNsigma above limit \\ - PM\_SOURCE\_MODE\_EXT\_LIMIT & 0x00004000 & Source has extNsigma above limit \\ - PM\_SOURCE\_MODE\_MOMENTS\_FAILURE & 0x00008000 & could not measure the moments \\ - PM\_SOURCE\_MODE\_SKY\_FAILURE & 0x00010000 & could not measure the local sky \\ - PM\_SOURCE\_MODE\_SKYVAR\_FAILURE & 0x00020000 & could not measure the local sky variance \\ - PM\_SOURCE\_MODE\_BELOW\_MOMENTS\_SN & 0x00040000 & moments not measured due to low S/N \\ - PM\_SOURCE\_MODE\_BIG\_RADIUS & 0x00100000 & poor moments for small radius, try large radius \\ - PM\_SOURCE\_MODE\_AP\_MAGS & 0x00200000 & source has an aperture magnitude \\ - PM\_SOURCE\_MODE\_BLEND\_FIT & 0x00400000 & source was fitted as a blend \\ - PM\_SOURCE\_MODE\_EXTENDED\_FIT & 0x00800000 & full extended fit was used \\ - PM\_SOURCE\_MODE\_EXTENDED\_STATS & 0x01000000 & extended aperture stats calculated \\ - PM\_SOURCE\_MODE\_LINEAR\_FIT & 0x02000000 & source fitted with the linear fit \\ - PM\_SOURCE\_MODE\_NONLINEAR\_FIT & 0x04000000 & source fitted with the non-linear fit \\ - PM\_SOURCE\_MODE\_RADIAL\_FLUX & 0x08000000 & radial flux measurements calculated \\ - PM\_SOURCE\_MODE\_SIZE\_SKIPPED & 0x10000000 & size could not be determined \\ +* PM\_SOURCE\_MODE\_FITTED & 0x00000004 & Source fitted with non-linear model (PSF or EXT; good or bad) \\ +* PM\_SOURCE\_MODE\_FAIL & 0x00000008 & Fit (non-linear) failed (non-converge, off-edge, run to zero) \\ +* PM\_SOURCE\_MODE\_POOR & 0x00000010 & Fit succeeds, but low-SN, high-Chisq, or large (for PSF -- drop?) \\ +* PM\_SOURCE\_MODE\_PAIR & 0x00000020 & Source fitted with a double psf \\ +* PM\_SOURCE\_MODE\_PSFSTAR & 0x00000040 & Source used to define PSF model \\ +* PM\_SOURCE\_MODE\_SATSTAR & 0x00000080 & Source model peak is above saturation \\ +* PM\_SOURCE\_MODE\_BLEND & 0x00000100 & Source is a blend with other sources$^1$ \\ +* PM\_SOURCE\_MODE\_EXTERNAL & 0x00000200 & Source based on supplied input position \\ +* PM\_SOURCE\_MODE\_BADPSF & 0x00000400 & Failed to get good estimate of object's PSF \\ +* PM\_SOURCE\_MODE\_DEFECT & 0x00000800 & Source is thought to be a defect \\ +* PM\_SOURCE\_MODE\_SATURATED & 0x00001000 & Source is thought to be saturated pixels (bleed trail) \\ +* PM\_SOURCE\_MODE\_CR\_LIMIT & 0x00002000 & Source has crNsigma above limit \\ +* PM\_SOURCE\_MODE\_EXT\_LIMIT & 0x00004000 & Source has extNsigma above limit \\ +* PM\_SOURCE\_MODE\_MOMENTS\_FAILURE & 0x00008000 & could not measure the moments \\ +* PM\_SOURCE\_MODE\_SKY\_FAILURE & 0x00010000 & could not measure the local sky \\ +* PM\_SOURCE\_MODE\_SKYVAR\_FAILURE & 0x00020000 & could not measure the local sky variance \\ +x PM\_SOURCE\_MODE\_BELOW\_MOMENTS\_SN & 0x00040000 & moments not measured due to low S/N.$^1$ \\ +* PM\_SOURCE\_MODE\_BIG\_RADIUS & 0x00100000 & poor moments for small radius, try large radius \\ +* PM\_SOURCE\_MODE\_AP\_MAGS & 0x00200000 & source has an aperture magnitude \\ +* PM\_SOURCE\_MODE\_BLEND\_FIT & 0x00400000 & source was fitted as a blend \\ +* PM\_SOURCE\_MODE\_EXTENDED\_FIT & 0x00800000 & full extended fit was used \\ +* PM\_SOURCE\_MODE\_EXTENDED\_STATS & 0x01000000 & extended aperture stats calculated \\ +* PM\_SOURCE\_MODE\_LINEAR\_FIT & 0x02000000 & source fitted with the linear fit \\ +* PM\_SOURCE\_MODE\_NONLINEAR\_FIT & 0x04000000 & source fitted with the non-linear fit \\ +* PM\_SOURCE\_MODE\_RADIAL\_FLUX & 0x08000000 & radial flux measurements calculated \\ +x PM\_SOURCE\_MODE\_SIZE\_SKIPPED & 0x10000000 & size could not be determined$^2$ \\ PM\_SOURCE\_MODE\_ON\_SPIKE & 0x20000000 & peak lands on diffraction spike \\ PM\_SOURCE\_MODE\_ON\_GHOST & 0x40000000 & peak lands on ghost or glint \\ PM\_SOURCE\_MODE\_OFF\_CHIP & 0x80000000 & peak lands off edge of chip \\ +\hline +\multicolumn{3}{l}{$^1$ not used for DR1 or DR2.} \\ +\multicolumn{3}{l}{$^2$ should not normally occur, does it?} \\ \hline \end{tabular} @@ -1038,10 +1041,31 @@ centroid is used to center the window function. +For sources with peak flux above the saturation limit, the moments are +generally poorly measured if the aperture defined by $\sigma_w$ is +used. For these sources, the quality of the measurment is compromised +by the saturation. However, it is still useful to estimate the first +and second moments of the source in order to allow a crude measurement +of the brightness from the wings of the source. In this case, a +larger aperture, 3 times the standard aperture, is used to make a +crude estimate. For such sources, the flag bit +\code{PM_SOURCE_MODE_BIG_RADIUS} is set and the source is ignored in +all analyses below except for the analysis applied to very bright +stars (Section~\ref{sec:very.bright.star}). + If the measured centroid coordinates ($x_0, y_0$) differ from the peak coordinates be a large amount (1.5$\sigma_w$), then the peak is -identified as being of poor quality (\code{infoFlag} bit -\code{MOMENTS_FAILURE}) and is skipped in further analyses. In such -as case, it is likely that the `peak' was identified in a region of -flat flux distribution or many saturated or edge pixels. +identified as being of poor quality and is skipped in further +analyses; the flag bit +\code{PM_SOURCE_MOMENTS_FAILURE} is set for such sources. In such +a case, it is likely that the `peak' was identified in a region of +flat flux distribution or many saturated or edge pixels. During the +analysis of the moments, the background (``sky'') model is also examined for the +location of each source. The value of the background and the variance +of the background are recorded for each source. In some cases, the +sky model or the variance is not well defined at the location of a +specific sources (e.g., due to an extrapolation failure). In these +cases, the flag bits \code{PM_SOURCE_SKY_FAILURE} or +\code{PM_SOURCE_SKYVAR_FAILURE} are set as appropriate and the +measurement of the moments is skipped. In addition to the moments above, the 1st and half-radial moments, @@ -1254,5 +1278,6 @@ produce a single peak. To avoid detecting a peak from the unresolved cosmic rays, sources which have second-moments very close to 0 are -ignored. +ignored. For these sources, the flag bit \code{PM_SOURCE_MODE_DEFECT} +is set. Once a peak has been detected in this plane, the centroid and second @@ -1260,4 +1285,16 @@ pixels of this centroid are selected as candidate PSF sources in the image. + +When the second moments are measured, \ippprog{psphot} also counts the +number of saturated pixels within the analysis aperture. If more than +a single saturated pixel is found, and if the second moments of that +object are more than one standard deviation larger than the clump +identified above, this source is identified as a highly saturated star +and marked with the flag bit \code{PM_SOURCE_MODE_SATSTAR}. Sources +which have more than a single saturated pixel, but for which the +second moments do not exceed the above limits are marked as likely +saturated regions (e.g., bleed trails). These sources are skipping in +most additional analyses and are marked with the flag bit +\code{PM_SOURCE_MODE_SATURATED}. \begin{figure}[htbp] @@ -1304,5 +1341,7 @@ snuck into the sample). Sources whose model parameters are rejected by this iterative fitting technique are also marked as invalid PSF -sources and ignored in the later PSF model fitting stages. +sources and ignored in the later PSF model fitting stages. Sources +which are actually used to define the PSF model for a given image have +the flag bit \code{PM_SOURCE_MODE_PSFSTAR} set. The order of the fit or number of grid samples is modified if the @@ -1354,5 +1393,30 @@ is measured here, with a more detailed correction measured after all source analysis is performed (see -Section~\ref{sec:aperture.correction}). +Section~\ref{sec:aperture.correction}). Sources for which the +aperture magnitude is measured have the flag bit +\code{PM_SOURCE_MODE_AP_MAGS} set. These aperture magnitudes are +stored in the DVO field \code{Measure.Map} and exported to the PSPS as +a flux in Janskies in the field \code{Detection.apFlux}. The radius +(in arcseconds) +of the aperture used for each exposure is reported in PSPS as +\code{Detection.apRadius}, while the unmasked fraction of the aperture +is reported in PSPS as \code{Detection.apFillF}. + +When the PSF and aperture photometry for a source is measured, two +additional quantities are measured which are useful to assess the +quality of the measurements. First, the mask image is examined and the +number of unmasked pixels is summed, weighted by the normalized PSF +model. The resulting quantity, \code{PSF_QF} has a value between 0.0 +(totally masked) and 1.0 (totally unmasked). Elsewhere in the IPP +system, we use this value to filter out detections which are +unreliable due to the masking. For a generous cut, leaning toward +completeness at the cost of some lower quality measurements, +\code{PSF_QF} $> 0.85$ is used in some contexts; in other cases, we +require \code{PSF_QF} $> 0.95$ to ensure a high-quality measurement +\citep[see for example the calculation of average photometry + in][]{magnier2017.calibration}. The second quantity is related to +the first: \code{PSF_QF_PERFECT} uses all mask values to assess the +quality factor, while \code{PSF_QF} uses only the ``bad'' mask bit +values (see Section~\ref{sec:image.preparation}). \subsection{Bright Source Analysis} @@ -1458,5 +1522,8 @@ PSF. Sources for which a PSF model has been fitted (whether or not this is retained as the best model in the end) has the flag field -\code{PM_SOURCE_MODE_PSFMODEL} set. +\code{PM_SOURCE_MODE_PSFMODEL} set. All sources which are included in +the ensemble linear fit have the flag bit +\code{PM_SOURCE_MODE_LINEAR_FIT} set, including those for which the +model is not the PSF. \subsubsection{Radial Profile Wings} @@ -1544,5 +1611,6 @@ Kron magnitudes and called \code{extNsigma}. If \code{extNsigma} is larger than \code{PSPHOT.EXT.NSIGMA.LIMIT} (3.0), the source is -considered to be extended. +considered to be extended and the flag bit +\code{PM_SOURCE_MODE_EXT_LIMIT} is set for the source. Cosmic rays are identified by a combination of the Kron magnitude and @@ -1557,5 +1625,14 @@ identified as a cosmic ray and the associated pixels are masked. These values are tuned empirically for the PV3 analysis based on -cosmic rays identified in the GPC1 images. +cosmic rays identified in the GPC1 images. Sources which are +determined to the a cosmic ray in this manner have the flag bit +\code{PM_SOURCE_MODE_DEFECT} set. + +The pixels of any suspected cosmic ray identified above are examined +in additional detail to make a final judgement. The Laplacian edge +detection algorithm based on \cite{2001PASP..113.1420V} is used to +check for sharp edges in the flux distribution. If the sharpness +exceeds a defined limit, then the pixels are masked and the flag bit +\code{PM_SOURCE_MODE_CR_LIMIT} is set for the source. % Mminor < 0.8 && SN > 7 @@ -1584,5 +1661,18 @@ non-linear fitting. The sources are fitted in their S/N order, starting with the brightest and working down to the user-specified -limit, with the other sources subtracted as discussed above. +limit, with the other sources subtracted as discussed above. All +sources for which a non-linear PSF model has been attempted have the +flag bit \code{PM_SOURCE_MODE_FITTED} set, regardless of the quality +of that fit. + +Since the PSF model describes the variation of the PSF across the +image, the parameters used to fit a specific object are drawn from the +model at the position corresponding to the object centroid. +Occasionally, a PSF model for an image may not be well determined in +all regions of the image. For example, not enough bright stars were +available across the full range of the image to model the PSF and the +resulting fitted parameters yield non-sensical solutions in areas +where detected (fainter) sources are found. In such cases, the PSF +fitting is skipped and the flag bit \code{PM_SOURCE_MODE_BADPSF} is set. For the PSF model fitting, only pixels within a circular aperture @@ -1594,5 +1684,5 @@ window radius is $7 \times \sigma_w$. -Sources which are blended with other sources are fitted together as a +Sources which are blended with other sources may be fitted together as a set of PSFs. Blended objects are identified by first searching for objects for which the PSF fit windows overlap. For a group of such @@ -1604,5 +1694,11 @@ blends of this next object. All objects in the image are tested as possible blends. A single multi-source fit is performed on each group -of blended peaks. +of blended peaks. Sources which are identified as members of a +blended group have the flag bit \code{PM_SOURCE_MODE_BLEND} set, while +those for which a blended PSF fit succeeds have the flag bit +\code{PM_SOURCE_MODE_BLEND_FIT} set. {\em Note that for DR1 \& DR2, + this option was not used because it tended to prevent galaxies from + being fitted as extended objects; the rules for identifying blended + stars and galaxies will be revisited in future re-analyses.} %% Once a solution has been achieved for a source, \ippprog{psphot} attempts to @@ -1670,10 +1766,11 @@ Sources which are pass the above tests are marked as having a valid -non-linear PSF model fit (\code{PM_SOURCE_MODE_SATSTAR}). Among these -sources, those for which the peak flux is greater than the saturation -limit (see Section~\ref{sec:image.preparation}) are marked as -saturated stars (\code{PM_SOURCE_MODE_SATSTAR}). These model fits -should be considered with caution, but the fluxes and positions may -have some validity. +non-linear PSF model fit with the flag bit +\code{PM_SOURCE_MODE_NONLINEAR_FIT}. Among these sources, those for +which the peak flux is greater than the saturation limit (see +Section~\ref{sec:image.preparation}) are marked as saturated stars +(\code{PM_SOURCE_MODE_SATSTAR}). These model fits should be +considered with caution, but the fluxes and positions may have some +validity. % \citep[see the discussion in][regarding the masking of saturated @@ -1708,4 +1805,6 @@ as a valid PSF source. Otherwise, the double-PSF model is rejected and the source is fitted with the available non-PSF model or models. +Sources for which a double-PSF model is fitted have the flag bit +\code{PM_SOURCE_MODE_PAIR} set. \subsubsection{Non-PSF Sources} @@ -1914,5 +2013,5 @@ saved as equal-length vectors in the FITS table (\code{PROF_FLUX} and \code{PROF_FILL}). The values of the radial bins are saved in the -output file FITS header (\code{RMIN_NN}, \code{RMAX_NN}). +output file FITS header (\code{RMIN_NN}, \code{RMAX_NN}). % \note{these profiles are not saved in PSPS} @@ -1963,5 +2062,8 @@ Petrosian Flux, we can calculate two other interesting radii: $R_{50}$ and $R_{90}$, the radii inside which 50\% and 90\% of the total -Petrosian flux is contained. +Petrosian flux is contained. Sources for which the Petrosian +parameters are successfully measured have the flag bit +\code{PM_SOURCE_MODE_EXTENDED_STATS} set. + \subsubsection{Convolved Galaxy Model Fits} @@ -2111,14 +2213,16 @@ We next perform 3 Levenberg-Marquardt minimization fits allowing the shape parameters ($R_{xx}$, $R_{yy}$ , $R_{xy}$) and the normalization -to be fitted, holding the centroid ($x_0, y_0$), S\'ersic index $n$, and -sky constant. In these fits, the index $n$ is set to the minimum +to be fitted, holding the centroid ($x_0, y_0$), S\'ersic index $n$, +and sky constant. In these fits, the index $n$ is set to the minimum value previously calculated as well as values halfway to the next, and previous, values in the grid above. E.g., if the minimum fitted index -value is 3.0, then the LMM fits are performed using $n$ = 2.5, 3.0, 3.5. -The resulting $\chi^2$ values are then used to perform quadratic +value is 3.0, then the LMM fits are performed using $n$ = 2.5, 3.0, +3.5. The resulting $\chi^2$ values are then used to perform quadratic interpolation to find the index $n$ which produces the locally minimum $\chi^2$ value. Finally, this best-fit index value is held constant while Levenberg-Marquardt minimization is used to find the best fit -values of all other parameters. +values of all other parameters. Sources for which a convolved galaxy +model fit was successful have the flag bit +\code{PM_SOURCE_MODE_EXTENDED_FIT} set. % Graham & Driver : Graham A. W., Driver S. P. 2005, PASA 22, 118 @@ -2204,6 +2308,8 @@ wasteful. We only calculate the circular apertures out to the second aperture larger than the ``sky radius'' (defined in -Section~\ref{sec:radial.profile}), but we calculate photometry for -at least the smallest 4 apertures. +Section~\ref{sec:radial.profile}), but we calculate photometry for at +least the smallest 4 apertures. Sources for which photometry in these +fixed aperture are calculated have the flag bit +\code{PM_SOURCE_MODE_RADIAL_FLUX} set. % at least out to aperture # RADIAL_AP_MIN (= 4), but no further than @@ -2422,12 +2528,15 @@ Individual warp images are processed independently with separate -executions of the \ippprog{psphotFullForce} program. Once all of the -forced photometry measurements (for point sources as well as galaxies, -discussed below) are completed for all of the warps which contributed -to a stack image, the measurements are combined together by other -portions of the IPP system. The PSF photometry measurements are -combined in the context of the DVO database system -\citep{magnier2017.datasystem}, including recalibration of the zero -points for the individual warp. +executions of the \ippprog{psphotFullForce} program. Sources which +are loaded by \ippprog{psphotFullForce} for analysis are marked with +the flag bit \code{PM_SOURCE_MODE_EXTERNAL}. This bit is also used to +mark user-supplied sources loaded for analysis by the regular version +of \ippprog{psphot}. Once all of the forced photometry measurements +(for point sources as well as galaxies, discussed below) are completed +for all of the warps which contributed to a stack image, the +measurements are combined together by other portions of the IPP +system. The PSF photometry measurements are combined in the context +of the DVO database system \citep{magnier2017.datasystem}, including +recalibration of the zero points for the individual warp. \subsection{Forced Galaxy Models}