Index: /trunk/doc/release.2015/ps1.calibration/Makefile =================================================================== --- /trunk/doc/release.2015/ps1.calibration/Makefile (revision 40596) +++ /trunk/doc/release.2015/ps1.calibration/Makefile (revision 40597) @@ -4,5 +4,5 @@ # remember to set \pdfoutput at the top -DO_BIBTEX = 0 +DO_BIBTEX = 1 # remember to change from \bibliography to \input{.bbl} at the bottom @@ -14,4 +14,6 @@ pdf: calibration.pdf tgz: calibration.tgz + +quick: calibration.quick.pdf FILES = \ Index: /trunk/doc/release.2015/ps1.calibration/calibration.tex =================================================================== --- /trunk/doc/release.2015/ps1.calibration/calibration.tex (revision 40596) +++ /trunk/doc/release.2015/ps1.calibration/calibration.tex (revision 40597) @@ -98,7 +98,14 @@ \begin{abstract} -The Pan-STARRS\,1 $3\pi$ survey has produced photometry and astrometry -covering the \approx 30,000 square degrees $\delta > -30$\degrees. -This article describes the photometric and astrometric calibration of this survey. +We present the details of the photometric and astrometric calibration +of the Pan-STARRS\,1 $3\pi$ Survey. The photometric goals were to +reduce the systematic effects introduced by the camera and detectors, +and to place all of the observations into a photometric system with +consistent zero points over the entire area surveyed, the \approx +30,000 square degrees north of $\delta = -30$\degrees. The +astrometric calibration compensates for similar systematic effects so +that positions, proper motions, and parallaxes are reliable as well. +The Pan-STARRS Data Release 2 (DR2) astrometry is tied to the Gaia DR1 +release. \end{abstract} @@ -108,65 +115,4 @@ \section{Introduction}\label{sec:intro} - -This is the fifth in a series of seven papers describing the -Pan-STARRS1 Surveys, the data reduction techiques and the resulting -data products. This paper (Paper V) describes the final calibration -process, and the resulting photometric and astrometric quality. - -%Chambers et al. 2017 (Paper I) -%The Pan-STARRS\,1 Surveys -\citet[][Paper I]{chambers2017} -provides an overview of the Pan-STARRS System, the design and -execution of the Surveys, the resulting image and catalog data -products, a discussion of the overall data quality and basic -characteristics, and a brief summary of important results. - -%Magnier et al. 2017 (Paper II) -%Pan-STARRS Data Processing Stages -\citet[][Paper II]{magnier2017c} -describes how the various data processing stages are organised and implemented -in the Imaging Processing Pipeline (IPP), including details of the -the processing database which is a critical element in the IPP infrastructure . - -%Waters et al. 2017 (Paper III) -%Pan-STARRS Pixel Processing : Detrending, Warping, Stacking -\citet[][Paper III]{waters2017} -describes the details of the pixel processing algorithms, including detrending, warping, and adding (to create stacked images) and subtracting (to create difference images) and resulting image products and their properties. - - -%Magnier et al. 2017 (Paper IV) -%Pan-STARRS Pixel Analysis : Source Detection -\citet[][Paper IV]{magnier2017a} -describes the details of the source detection and photometry, including point-spread-function and extended source fitting models, and the techniques for ``forced" photometry measurements. - -%Magnier et al. 2017 (Paper V) -%Pan-STARRS Photometric and Astrometric Calibration -%\citet[][Paper V]{magnier2017b} -%describes the final calibration process, and the resulting photometric and astrometric quality. - - -%Flewelling et al. 2017 (Paper VI) -%Pan-STARRS 1 Database and Data Products -\citet[][Paper VI]{flewelling2017} -describes the details of the resulting catalog data and its organization in the Pan-STARRS database. -% -% -\citet[][Paper VII]{huber2017} -%Huber et al. 2017 (Paper VII) -describes the Medium Deep Survey in detail, including the unique issues and data products specific to that survey. The Medium Deep Survey is not part of Data Release 1. (DR1) - -% -The Pan-STARRS1 filters and photometric system have already been -described in detail in \cite{2012ApJ...750...99T}. - -{\color{red} {\em Note: These papers are being placed on arXiv.org to - provide crucial support information at the time of the public - release of Data Release 1 (DR1). We expect the arXiv versions to - be updated prior to submission to the Astrophysical Journal in - January 2017. Feedback and suggestions for additional information - from early users of the data products are welcome during the - submission and refereeing process.}} - -\section{Pan-STARRS\,1} From May 2010 through March 2014, the Pan-STARRS Science Consortium @@ -176,5 +122,13 @@ formation and architecture of the Milky Way galaxy, and the search for Type Ia supernovae to measure the history of the expansion of the -universe. +universe. The majority of the time (56\%) was spent on surveying the +$\frac{3}{4}$ of the sky north of $-30$ Declination with +\grizy\ filters in the so-called $3\pi$ Survey. Another $\sim 25\%$ +of the time was concentrated on repeated deep observations of 10 +specific fields in the Medium-Deep Survey. The rest of the time was +used for several other surveys, including a search for potentially +hazardous asteroids in our solar system. The details of the +telescope, surveys, and resulting science publications are described +by \cite{chambers2017}. The wide-field \PSONE\ telescope consists of a 1.8~meter diameter @@ -205,6 +159,89 @@ Maui. +%The Processing Version 3 (PV3) reduction represents the third full +Pan-STARRS produced its first large-scale public data release, Data +Release 1 (DR1) on 16 December 2016. DR1 contains the results of the +third full reduction of the Pan-STARRS $3\pi$ Survey archival data, +identified as PV3. Previous reductions \citep[PV0, PV1, PV2; + see][]{magnier2017.datasystem} were used internally for pipeline +optimization and the development of the initial photometric and +astrometric reference catalog \citep{magnier2017.calibration}. The +products from these reductions were not publicly released, but have +been used to produce a wide range of scientific papers from the +Pan-STARRS 1 Science Consortium members \citep{chambers2017}. DR1 +contained only average information resulting from the many individual +images obtained by the $3\pi$ Survey observations. A second data +release, DR2, was made available \note{20 January 2019}. DR2 provides +measurements from all of the individual exposures, and include an +improved calibration of the PV3 processing of that dataset. + +This is the fifth in a series of seven papers describing the +Pan-STARRS1 Surveys, the data reduction techiques and the resulting +data products. This paper (Paper V) describes the final calibration +process, and the resulting photometric and astrometric quality. + +%Chambers et al. 2017 (Paper I) +%The Pan-STARRS\,1 Surveys +\citet[][Paper I]{chambers2017} +provides an overview of the Pan-STARRS System, the design and +execution of the Surveys, the resulting image and catalog data +products, a discussion of the overall data quality and basic +characteristics, and a brief summary of important results. + +%Magnier et al. 2017 (Paper II) +%Pan-STARRS Data Processing Stages +\citet[][Paper II]{magnier2017.datasystem} +describes how the various data processing stages are organised and implemented +in the Imaging Processing Pipeline (IPP), including details of the +the processing database which is a critical element in the IPP infrastructure . + +%Waters et al. 2017 (Paper III) Pan-STARRS Pixel Processing : +%Detrending, Warping, Stacking +\citet[][Paper III]{waters2017} describes the details of the pixel +processing algorithms, including detrending, warping, and adding (to +create stacked images) and subtracting (to create difference images) +and resulting image products and their properties. + + +%Magnier et al. 2017 (Paper IV) +%Pan-STARRS Pixel Analysis : Source Detection +\citet[][Paper IV]{magnier2017.analysis} describes the details of the source +detection and photometry, including point-spread-function and extended +source fitting models, and the techniques for ``forced" photometry +measurements. + +%Magnier et al. 2017 (Paper V) +%Pan-STARRS Photometric and Astrometric Calibration +%\citet[][Paper V]{magnier2017.calibration} +%describes the final calibration process, and the resulting photometric and astrometric quality. +% THIS PAPER + +%Flewelling et al. 2017 (Paper VI) +%Pan-STARRS 1 Database and Data Products +\citet[][Paper VI]{flewelling2017} +describes the details of the resulting catalog data and its organization in the Pan-STARRS database. + +%Huber et al. 2017 (Paper VII) +\citet[][Paper VII]{huber2017} describes the Medium Deep Survey in +detail, including the unique issues and data products specific to that +survey. The Medium Deep Survey is not part of Data Releases 1 or 2 and +will be made available in a future data release. + +% +The Pan-STARRS1 filters and photometric system have already been +described in detail in \cite{2012ApJ...750...99T}. + +%% {\color{red} {\em Note: These papers are being placed on arXiv.org to +%% provide crucial support information at the time of the public +%% release of Data Release 1 (DR1). We expect the arXiv versions to +%% be updated prior to submission to the Astrophysical Journal in +%% January 2017. Feedback and suggestions for additional information +%% from early users of the data products are welcome during the +%% submission and refereeing process.}} + +\section{Pan-STARRS\,1 Data Analysis} + Images obtained by \PSONE\ are automatically processed in real time by -the \PSONE\ Image Processing Pipeline \citep[IPP,][]{magnier2017a}. +the \PSONE\ Image Processing Pipeline \citep[IPP,][]{magnier2017.datasystem}. Real-time analysis goals are aimed at feeding the discovery pipelines of the asteroid search and supernova search teams. The data obtained @@ -212,9 +249,12 @@ complete re-processing of the data: Processing Versions 1, 2, and 3 (PV1, PV2, and PV3). The real-time processing of the data is -considered ``PV0''. Except as otherwise noted, the PV3 analysis of -the data is used for the purpose of this article. +considered ``PV0''. Except as otherwise noted, this article describes +the calibration of the PV3 analysis of the data. Between the first +(DR1) and second (DR2) data releases, improvements were made to the +calibration of both the photometry and astrometry, as described in +this article. The data processing steps are described in detail by \cite{waters2017} -and \cite{magnier2017a,magnier2017b}. In summary, individual images +and \cite{magnier2017.datasystem,magnier2017.analysis}. In summary, individual images are detrended: non-linearity and bias corrections are applied, a dark current model is subtracted and flat-field corrections are applied. @@ -226,4 +266,6 @@ discussed below, preliminary astrometric and photometric calibrations are performed for all chips in a single exposure in a single analysis. +We refer to these measurements as the ``chip'' photometry and +astrometry products. Chip images are geometrically transformed based on the astrometric @@ -241,7 +283,7 @@ % from images for a single night (nightly stacks). -Astronomical objects are detected and characterized in the stacks +Astronomical objects are detected and characterized in the stack images. The details of the analysis of the sources in the stack -images are discussed in \cite{magnier2017b}, but in brief these include +images are discussed in \cite{magnier2017.analysis}, but in brief these include PSF photometry, along with a range of measurements driven by the goals of understanding the galaxies in the images. Because of the @@ -256,5 +298,5 @@ To recover most of the photometric quality of the individual chip images, while also exploiting the depth afforded by the stacks, the -PV3 analysis make use of forced photometry on the individual warp +PV3 analysis makes use of forced photometry on the individual warp images. PSF photometry is measured on the warp images for all sources which are detected in the stack images images. The positions @@ -267,10 +309,14 @@ measurement of the faint source flux is determined. The details of this analysis are described in detail in Magnier et al -\cite{magnier2017b}. - -In this article, we discuss the photometric calibration of the -individual exposures, the stacks, and the warp imags. We also discuss -the astrometric calibration of the individual exposures and the stack -images. +\cite{magnier2017.analysis}. + +The data products from the chip photometry, stack photometry, and +forced-warp photometry analysis stages are ingested into the internal +calibration database called the Desktop Virtual Observatory, or DVO +\citep[see Section~4 in][]{magnier2017.datasystem} and used for +photometric and astrometric calibrations. In this article, we discuss +the photometric calibration of the individual exposures, the stacks, +and the warp imags. We also discuss the astrometric calibration of +the individual exposures and the stack images. \section{Astrometric Models} @@ -293,5 +339,5 @@ where $P,Q$ are the tangent plane coordinates, $X_{\rm chip}, Y_{\rm chip}$ are the coordinates on the 60 GPC1 chips, and $C^P_{i,j}, C^Q_{i,j}$ -are the polynomial coefficients for each order. In the \code{psastro} +are the polynomial coefficients for each order. In the \ippprog{psastro} analysis, $i + j <= N_{\rm order}$ where the order of the fit, $N_{\rm order}$, may be 1 to 3, under the restriction that sufficient stars @@ -305,14 +351,14 @@ sky coordinates to a locally cartesian tangent plane coordinate system. A set of polynomials is then used to relate the tangent plane -coordinates to a 'focal plane' coordinate system, $L,M$: +coordinates to a `focal plane' coordinate system, $L,M$: \begin{eqnarray} P & = & \sum_{i,j} C^P_{i,j} L^i M^j \\ Q & = & \sum_{i,j} C^Q_{i,j} L^i M^j \end{eqnarray} -This set of polynomial accounts for effects such as optical distortion +This set of polynomials accounts for effects such as optical distortion in the camera and distortions due to changing atmospheric refraction across the field of the camera. Since these effects are smooth across the field of the camera, a single pair of polynomials can be used for -each exposure. Like in the chip analysis about, the \code{psastro} +each exposure. Like in the chip analysis about, the \ippprog{psastro} code restricts the exponents with the rule $i + j <= N_{\rm order}$ where the order of the fit, $N_{\rm order}$, may be 1 to 3, under the @@ -331,5 +377,5 @@ tangent plane), but the relationship between the chip and focal plane is represented with only the linear terms in the polynomial, -supplemented by a course grid of displacements, $\delta L, \delta M$ sampled +supplemented by a coarse grid of displacements, $\delta L, \delta M$ sampled across the coordinate range of the chip. This displacement grid may have a resolution of up to @@ -343,12 +389,16 @@ \end{eqnarray} -{\bf WCS Keywords} When this polynomial representation is written to -the output files, a set of WCS keywords are used to define the -astrometric transformation elements. It is necessary to transform the -simply polynomials above into an alternate form: -\begin{eqnarray} - P & = & \sum_{i,j} C^P_{i,j} (X_{\rm chip} - X_0)^i (Y_{\rm chip} - Y_0)^j \\ - Q & = & \sum_{i,j} C^Q_{i,j} (X_{\rm chip} - X_0)^i (Y_{\rm chip} - Y_0)^j -\end{eqnarray} +\note{does this section need more? does this section need to be moved?} + +%% Include a description of the WCS keywords used to represent the fit elements? + +%% {\bf WCS Keywords} When this polynomial representation is written to +%% the output files, a set of WCS keywords are used to define the +%% astrometric transformation elements. It is necessary to transform the +%% simply polynomials above into an alternate form: +%% \begin{eqnarray} +%% P & = & \sum_{i,j} C^P_{i,j} (X_{\rm chip} - X_0)^i (Y_{\rm chip} - Y_0)^j \\ +%% Q & = & \sum_{i,j} C^Q_{i,j} (X_{\rm chip} - X_0)^i (Y_{\rm chip} - Y_0)^j +%% \end{eqnarray} %% \note{need to complete this discussion of the WCS keywords, both @@ -383,12 +433,12 @@ ensure the warps are combined using consistent flux units. -The program used for the real-time calibration, \code{psastro}, loads -the measurements of the chip detections from their individual -\code{cmf}-format files. It uses the header information populated at -the telescope to determine an initial astrometric calibration guess -based on the position of the telescope boresite right ascension, -declination and position angle as reported by the telescope \& camera -subsystems. Using the initial guess, \code{psastro} loads astrometric -and photometric data from the reference database. +The program used for the real-time calibration, \ippprog{psastro}, +loads the measurements of the chip detections from their individual +output catalog files. It uses the header information populated at the +telescope to determine an initial astrometric calibration guess based +on the position of the telescope boresite right ascension, declination +and position angle as reported by the telescope \& camera subsystems. +Using the initial guess, \ippprog{psastro} loads astrometric and +photometric data from the reference database. \subsection{Reference Catalogs} @@ -396,12 +446,13 @@ During the course of the PS1SC Survey, several reference databases -have been used. For the first 20 months of the survey, \code{psastro} -used a reference catalog with synthetic PS1 \grizy\ photometry -generated by the Pan-STARRS IPP team based on based combined -photometry from Tycho (B, V), USNO (red, blue, IR), and 2MASS $J, H, -K$. The astrometry in the database was from 2MASS. After 2012 May, a -reference catalog generated from internal re-calibration of the PV0 -analysis of PS1 photometry and astrometry was used for the reference -catalog. +have been used. For the first 20 months of the survey, +\ippprog{psastro} used a reference catalog with synthetic PS1 +\grizy\ photometry generated by the Pan-STARRS IPP team based on based +combined photometry from Tycho (B, V), USNO \citep[red, blue, + IR][]{2003AJ....125..984M}, and 2MASS +$J, H, K$ \citep{2006AJ....131.1163S}. The astrometry in the database was from 2MASS +\citep{2006AJ....131.1163S}. After 2012 May, a reference catalog +generated from internal re-calibration of the PV0 analysis of PS1 +photometry and astrometry was used for the reference catalog. % \note{discuss history of the different refcats?} @@ -423,5 +474,5 @@ false-positive match, especially as many of the reference stars may not be detected in the GPC1 image. The seletion of the reference -stars includes a limit on the brightest and fainted magnitude of the +stars includes a limit on the brightest and faintest magnitudes of the stars selected. @@ -443,5 +494,5 @@ The first step of the analysis is to attempt to find the match between -the reference stars and the detected objects. \code{psastro} uses 2D +the reference stars and the detected objects. \ippprog{psastro} uses 2D cross correlation to search for the match. The guess astrometry calibration is used to define a predicted set of $X^{\rm ref}_{\rm @@ -468,5 +519,5 @@ value by a small amount. For each trial, the peak pixel is found and a figure of merit is measured. The figure of merit is defined as -$\frac{\sigma^2_x + \sigma^2_y}{N_p^4}$ where $\sigma^2_{x,y}$ are the +$\frac{\sigma^2_x + \sigma^2_y}{N_p^4}$ where $\sigma^2_{x,y}$ is the second moment of $\Delta X,Y$ for the star pairs associated with the peak pixel, and $N_p$ is the number of star pairs in the peak. This @@ -510,7 +561,7 @@ distortion, we choose a single common plate scale for the set of chips and re-define the chip $\rightarrow$ sky calibrations as a set of chip -$\rightarrow$ focal plane transformation using that common pixel +$\rightarrow$ focal plane transformations using that common pixel scale. We can now compare the observed focal plane coordinates, -derived from the chip coordinates, and the tangent plane coordiantes, +derived from the chip coordinates, and the tangent plane coordinates, derived from the projection of the reference coordinates. One caveat is that the chip reference coordinates are also degenerate with the @@ -526,5 +577,5 @@ Once the common distortion coming from the optics and atmosphere have -been modeled, \code{psastro} determines polynomial transformations +been modeled, \ippprog{psastro} determines polynomial transformations from the 60 chips to the focal plane coordinate system. In this stage, 5 iterations of the chip fits are performed. Before each @@ -542,43 +593,45 @@ After the astrometric calibration has finished, the photometric -calibration is performed by \code{psastro}. When the reference stars -are loaded, the apparent magnitude in the filter of interest is also -loaded. Stars for which the reference magnitude is brighter than +calibration is performed by \ippprog{psastro}. When the reference +stars are loaded, the apparent magnitude in the filter of interest is +also loaded. Stars for which the reference magnitude is brighter than (\grizy) = (19, 19, 18.5, 18.5, 17.5) are used to determine the zero points by comparison with the instrumental magnitudes. For the PV3 analysis, an outlier-rejecting median is used to measure the zero -point. For early versions of the analysis, when the reference catalog -used synthetic magnitudes, it was necessary to search for the blue -edge of the distribution: the synthetic magnitude poorly predicted the -magnitudes of stars in the presence of significant extinction or for -the very red stars, making the blue edge somewhat more reliable. Note -that we do not include an airmass correction in this zero point -analysis: the airmass correction is folded into the observed zero -point. The zero point may be measured separately for each chip or as -a single value for the entire exposure; the latter option was used for -the PV3 analysis. +point. For early versions of the real-time analysis, when the +reference catalog used synthetic magnitudes, it was necessary to +search for the blue edge of the distribution: the synthetic magnitude +poorly predicted the magnitudes of stars in the presence of +significant extinction or for the very red stars, making the blue edge +somewhat more reliable as a reference than the mean. Once the +calibration was based on a reference catalog generated from +\PSONE\ photometry, this methods was no longer needed. Note that we +do not include an airmass correction in this zero point analysis: the +airmass correction is folded into the observed zero point. The zero +point may be measured separately for each chip or as a single value +for the entire exposure; the latter option was used for the PV3 +analysis. \subsection{Real-time outputs} -The calibrations determined by \code{psastro} as saved as part of the -header information in the output FITS tables. A single -multi-extension FITS table is written using the \code{smf} format. In -these files, the measurements from each chip are written as a separate -FITS table. A second FITS extension for each chip is used to store -the header information from the original chip image. The original -chip header is modified so that the extension corresponds to an image -with no pixels data: \code{NAXIS} is set to 0, even though -\code{NAXIS1} and \code{NAXIS2} are retained with the original -dimensions of the chip. A pixel-less primary header unit (PHU) is -generated with a summary of some of the important and common -chip-level keywords (e.g., \code{DATE-OBS}). The astrometric -transformation information for each chip is saved in the corresponding -header using standard (and some non-standard) WCS keywords. For the -two-level astrometric model, the PHU header carries the astrometric -transformation related to the projection and the camera-wide -distortions. Photometric calibrations are written as a set of -keywords to individual chip headers, and if the calibration is -performed at the exposure-level, to the PHU. The photometry -calibration keywords are: +The calibrations determined by \ippprog{psastro} are saved as part of +the header information in the output FITS tables. For each exposure, +a single multi-extension FITS table is written. In these files, the +measurements from each chip are written as a separate FITS table. A +second FITS extension for each chip is used to store the header +information from the original chip image. The original chip header is +modified so that the extension corresponds to an image with no pixel +data: \code{NAXIS} is set to 0, even though \code{NAXIS1} and +\code{NAXIS2} are retained with the original dimensions of the chip. +A pixel-less primary header unit (PHU) is generated with a summary of +some of the important and common chip-level keywords (e.g., +\code{DATE-OBS}). The astrometric transformation information for each +chip is saved in the corresponding header using standard (and some +non-standard) WCS keywords. For the two-level astrometric model, the +PHU header carries the astrometric transformation related to the +projection and the camera-wide distortions. Photometric calibrations +are written as a set of keywords to individual chip headers, and if +the calibration is performed at the exposure-level, to the PHU. The +photometry calibration keywords are: \begin{itemize} \item \code{ZPT_REF} : the nominal zero point for this filter @@ -596,16 +649,17 @@ Data from the GPC1 chip images, the stack images, and the warp images -are loaded into DVO using the real-time analysis astrometric -calibration to guide the association of detections into objects. -After the full PV3 DVO database was constructed, including all of the -chip, stack, and warp detections, several external catalogs were -merged into the database. First, the complete 2MASS PSC was loaded -into a stand-alone DVO database, which was then merged into the PV3 -master database. Next the DVO database of synthetic photometry in the -PS1 bands (see Section~\ref{sec:synthdb}) was merged in. Next, the -full Tycho database was added, followed by the AllWISE database. -After the Gaia release in August 2016 \citep{2016AA...595A...2G}, we -generated a DVO database of the Gaia positional and photometric -information and merged that into the master DVO database. +are loaded into the DVO calibration database using the real-time +analysis astrometric calibration to guide the association of +detections into objects. After the full PV3 DVO database was +constructed, including all of the chip, stack, and warp detections, +several external catalogs were merged into the database. First, the +complete 2MASS PSC was loaded into a stand-alone DVO database, which +was then merged into the PV3 master database. Next the DVO database +of synthetic photometry in the PS1 bands (see +Section~\ref{sec:synthdb}) was merged in. Next, the full Tycho +database was added, followed by the AllWISE database. After the Gaia +release in August 2016 \citep{2016AA...595A...2G}, we generated a DVO +database of the Gaia positional and photometric information and merged +that into the master PV3 $3\pi$ DVO database. %% \note{need to describe the assignment of flags, etc, for the external data sources}. @@ -672,5 +726,5 @@ on the reference photometric night of MJD 55744 (UT 02 July 2011). \cite{2014ApJ...795...45S} and \cite{2015ApJ...815..117S} have -re-examined the photometry of Calspec standards %% XXX FIX: \citep{Bohlin.1996} as +re-examined the photometry of Calspec standards \citep{1996AJ....111.1743B} as observed by PS1. \cite{2014ApJ...795...45S} reject 2 of the 7 stars used by \cite{2012ApJ...750...99T} and add photometry of 5 additional @@ -704,7 +758,7 @@ split into three main components: \[ -zp_{\rm total} = zp_{\rm nominal} + M_{cal} + K_{rm \lambda}(sec \zeta - 1) +zp_{\rm total} = zp_{\rm nominal} + M_{cal} + K_{\rm \lambda}(\sec \zeta - 1) \] -where $zp_{\rm nominal}$ and $K_{rm \lambda}$ are static values for +where $zp_{\rm nominal}$ and $K_{\rm \lambda}$ are static values for each filter representing respectively the nominal zero point and the slope of the trend with respect to the airmass ($\zeta$) for each @@ -756,7 +810,8 @@ camera with the field of view of the PS1 GPC1, the airmass may vary significantly within the field of view, especially at low elevations. -In the worst cases, at the celestial pole, the airmass range within a -single exposure is XXX - XXX. The complete calibrated (`relative') -magnitude is determined from the stored database values as: +In the worst cases, at the celestial pole, the airmass within a single +exposure may span a range of 2.56 - 2.93. The complete calibrated +(`relative') magnitude is determined from the stored database values +as: \[ M_{\rm rel} = M_{\rm inst} - 25.0 + zp_{\rm ref} + M_{\rm cal} + M_{\rm flat} + K_\lambda (sec \zeta - 1). @@ -803,9 +858,8 @@ \[ M_{ave} = \frac{\sum_i M_{rel,i} w_i}{\sum_i w_i} \] We find that the color difference of the different chips can be -ignored, and set the value of $A$ to 0.0. -Note that we only use a single mean airmass extinction term for all -exposures -- the difference between the mean and the specific value -for a given night is taken up as an additional element of the -atmospheric attenuation. +ignored, and set the color-trend slope to 0.0. Note that we only use +a single mean airmass extinction term for all exposures -- the +difference between the mean and the specific value for a given night +is taken up as an additional element of the atmospheric attenuation. %% \note{color-color terms between chips?} @@ -843,5 +897,5 @@ rejections do not catch all cases of bad measurements. -%% \citep[\code{PSF_QF} $< 0.85$, see][]{magnier2017b}; +%% \citep[\code{PSF_QF} $< 0.85$, see][]{magnier2017.analysis}; %% \note{refer to the PSPHOT bad and poor psphot bits?} @@ -855,5 +909,5 @@ from the recalculated mean. -Suspicious stars are also exclude from the analsis. We exclude stars +Suspicious stars are also excluded from the analysis. We exclude stars with reduced $\chi^2$ values more than 20.0, or more than 2$\times$ the median, whichever is larger. We also exclude stars with standard @@ -893,5 +947,5 @@ IPP cluster: for PV3, 100 parallel hosts are used. These machines by design control data from a large number of unconnected small patches -on the sky, with the goal of speeding queries for arbitrary chunks of +on the sky, with the goal of speeding queries for arbitrary regions of the sky. As a result, this parallelization is entirely inappropriate as the basis of the relative photometry analysis. For the relative @@ -931,5 +985,5 @@ region host may be updated. -The completely work flow of the all-sky relative photometry analysis +The complete work flow of the all-sky relative photometry analysis starts with an instance of the program running on a master computer. This machine loads the image database table and assigns the images to @@ -979,4 +1033,5 @@ \subsubsection{Photometric Flat-field} +\label{sec:phot.flat} For PV3, the relphot analysis was performed two times. The first @@ -1020,13 +1075,15 @@ Especially notable in the bluer filters is a pattern of quarter circles centered on the corners of the chips. These patterns are -similar to the ``tree rings'' reported by the DES team and others -(G. Berstein REF \& REFS). The details of these tree rings are beyond -the scope of this article, and will be explored in future work. -Unlike the tree ring features discussed by these other authors, the -features observed in the GPC1 photometry are not caused by an -interaction of the flat-field with the effective pixel geometry. -Instead, these photometric features are due to low-level changes in -the PSF size which we attribute to variable charge diffusion (Magnier -in prep). +similar to the ``tree rings'' reported by the Dark Energy Survey team +\citep{2014PASP..126..750P} and identified as a result of lateral +migration of electrons in the detectors due to electric fields due to +dopant variations. Unlike the tree ring features discussed by these +other authors, the strong features observed in the GPC1 photometry are +not caused by lateral electric fields, but rather by variations in the +vertical electron diffusion rate due to electric field variations +perpendicular to the plane of the detector. This effect is discussed +in detail by \cite{2018PASP..130f5002M}. The photometric features are +due to low-level changes in the PSF size which we attribute to the +variable charge diffusion. Other features include some poorly responding cells (e.g., in XY14) @@ -1052,4 +1109,24 @@ the bright end. +For the stack calibration, we calculate two separate zero points: one +for photometry tied to the PSF model and a second for the +aperture-like measurements (total aperture magnitudes, Kron magnitude, +cicular fixed-radius aperture magnitudes). This split is needed +because of the limited quality of the stack PSF photometry due to the +highly variable PSF in the stacks. Aperture magnitudes, however, are +not significantly affected by the PSF variations. We therefore tie +the PSF magnitudes to the average of the chip photometry PSF +magnitudes, but the aperture-like magnitudes are tied by equating the +stack Kron magnitudes to the average chip Kron magnitudes. {\em Note + that for DR1, this split zero point calibration was used; instead + all stack photometry was tied to the average chip photometry via the + PSF magnitudes.} The result of using a single zero point is that +the stack PSF magnitudes are consistent across the sky with the chip +PSF magnitudes, but the aperture-like magnitudes show significant +spatial variations. Figure~\ref{fig:stack.bad.kron} illustrates the +impact of using a single PSF zero point for the stack photometry. +This split is not needed for the forced-warp photometry since the +individual warps have well-defined PSfs. + \subsection{Photometry Calibration Quality} @@ -1061,5 +1138,5 @@ reject artifacts detected in a pair of exposures from the same night), with \code{PSF_QF} $> 0.85$ (to reject excessively-masked objects), -and with $mag_{\rm PSF} - mag_{rm Kron} < 0.1$ (to reject galaxies). +and with $mag_{\rm PSF} - mag_{\rm Kron} < 0.1$ (to reject galaxies). We then generated histograms of the difference between the average magnitude and the apparent magnitude in an individual image for each @@ -1092,11 +1169,364 @@ \subsection{Calculation of Object Photometry} -\subsubsection{Iteratively Reweighted Least Squares Fitting (1D)} +Once the image photometric calibrations (zero points and flat-field +corrections) have been determined and applied to the measuremetns from +each image, we can calculate the best average photometry for each +object. We calculate average magnitudes for the chip photometry; for +the forced-warp photometry, we calculate the average of the fluxes and +report both average fluxes and the equivalent average magnitudes. +Since the chip photometry requires signal-to-noise of 5 for a +detection, the bias introduced by averaging magnitudes is small. +Since the forced-warp photometry measurements are low signal-to-noise, +with potentially negative flux values, it is necessary to average the +fluxes. + +The first challenge is to select which measurements to use in +the calculation of the average photometry. For the $3\pi$ Survey +data, a single object may have anywhere from zero to roughly twenty +measurements in a given filter. Not all measurements are of equal +value, but we need a process which assigns an average photometry value +in all cases (and a way for the user to recognize average values which +should be treated with care). As discussed in more detail below, we +have defined a triage process to select the ``best'' set of +measurements available in each filter for each object. Once the set +of measurements to be used in the analysis is determined, we use the +Iteratively Reweighted Least Squares (IRLS) technique to determine the +average photometry given the possible presence of non-Gaussian +outliers even within the best subset of measurements. \subsubsection{Selection of Measurements} +To choose the measurements which will be used in the analysis, we +give each measurement a rank value based on a variety of tests of the +quality of the measurement, with lower values being better quality. +In the description below +The ranking values are defined as follows: +\begin{itemize} +\item {\bf rank 0 :} perfect measurment (no quality concerns) +\item {\bf rank 1 :} PSF ``perfect pixel'' quality factor (\code{PSF_QF_PERFECT}) $< 0.85$. \code{PSF_QF_PERFECT} measures the PSF-weighted fraction of pixels which are not masked \citep[see][]{magnier2017.analysis}. +\item {\bf rank 2 :} Photometry analysis flag field (\code{photFlags}) has one of the ``poor quality'' bits raised. These bits are listed below; OR-ed together they have the hexideciaml value \code{0xe0440130} +\begin{itemize} + \item {\tt PM\_SOURCE\_MODE\_POOR = 0x00000010} : Fit succeeded, but with low-S/N or high-Chisq + \item {\tt PM\_SOURCE\_MODE\_PAIR = 0x00000020} : Source fitted with a double psf + \item {\tt PM\_SOURCE\_MODE\_BLEND = 0x00000100} : Source is a blend with other sources + \item {\tt PM\_SOURCE\_MODE\_BELOW\_MOMENTS\_SN = 0x00040000} : Moments not measured due to low S/N + \item {\tt PM\_SOURCE\_MODE\_BLEND\_FIT = 0x00400000} : Source was fitted as a blended object + \item {\tt PM\_SOURCE\_MODE\_ON\_SPIKE = 0x20000000} : Peak lands on diffraction spike + \item {\tt PM\_SOURCE\_MODE\_ON\_GHOST = 0x40000000} : Peak lands on ghost or glint + \item {\tt PM\_SOURCE\_MODE\_OFF\_CHIP = 0x80000000} : peak lands off edge of chip +\end{itemize} +\item {\bf rank 3 :} Poor measurement as defined by relphot. This may be due to a fixed allowed region on the detector, or due to an outlier clipped analysis. In the $3\pi$ PV3 calibration, these tests were not applied. +%% ID_MEAS_POOR_PHOTOM : > 5 sigma outlier, using sigma of 3 sigma inner subset +%% ID_MEAS_AREA : outside of valid pixel window on chip +%% neither of these are used for PV3 3pi (POOR is replaced by IRLS; +%% AREA is replaced by masking) +\item {\bf rank 4 :} PSF quality factor (\code{PSF_QF}) $< 0.85$. + \code{PSF_QF} measures the PSF-weighted fraction of pixels which are + not masked as ``bad'', but may be ``suspect''. Bad values are + blank, highly non-linear or non-responsibe; suspect pixels include + those pixels on ghosts, diffraction spikes, bright star bleeds, and + the mildly-saturated cores of bright stars. Suspect values may have + some use in measuring a flux, but with caution + \citep[see][]{magnier2017.analysis,waters2017}. +\item {\bf rank 5 :} Photometric calibration of the GPC1 exposure is + determined by relphot to be poor. This situation occurs if there + are too few stars available for the calibration ($< 10$ selected + stars, or if the selected stars account for $< 5\%$ of all stars in + the exposure). An exposure may also be identified as poor if the + zero point is excessively deviant ($> 2$ magnitudes from the nominal + value) or if the standard deviation of the calibration residuals is + more than $2\times$ the median standard deviation for all exposures. +%% IMAGE_POOR : ID_IMAGE_PHOTOM_POOR | ID_IMAGE_PHOTOM_FEW | ID_IMAGE_PHOTOM_SKIP +%% ID_IMAGE_PHOTOM_SKIP : not set? +%% ID_IMAGE_PHOTOM_FEW : < 10 or (Ngood < 0.05 Nstars) +%% ID_IMAGE_PHOTOM_POOR : (scatter > MaxScatter) or (Mcal - MedOffset) > MaxOffset +%% MaxScatter = MAX (IMAGE_SCATTER, 2*MEDIAN(sigma)) +%% MaxOffset = MAX (IMAGE_OFFSET, 3*STDEV(Mcal)) +%% IMAGE_OFFSET = 2.0 mag +%% IMAGE_SCATTER = 0.075 mag +\item {\bf rank 6 :} Photometry analysis flag field (\code{photFlags}) has one of the ``bad quality'' bits raised. These bits are listed below; OR-ed together they have the hexideciaml value \code{0x1003bc88} +\begin{itemize} + \item {\tt PM\_SOURCE\_MODE\_FAIL = 0x00000008} : Non-linear fit failed (non-converge, off-edge, run to zero) + \item {\tt PM\_SOURCE\_MODE\_SATSTAR = 0x00000080} : Source model peak is above saturation + \item {\tt PM\_SOURCE\_MODE\_BADPSF = 0x00000400} : Failed to get good estimate of object's PSF + \item {\tt PM\_SOURCE\_MODE\_DEFECT = 0x00000800} : Source is thought to be a defect + \item {\tt PM\_SOURCE\_MODE\_SATURATED = 0x00001000} : Source is thought to be saturated pixels (bleed trail) + \item {\tt PM\_SOURCE\_MODE\_CR\_LIMIT = 0x00002000} : Source has crNsigma above limit + \item {\tt PM\_SOURCE\_MODE\_MOMENTS\_FAILURE = 0x00008000} : Could not measure the moments + \item {\tt PM\_SOURCE\_MODE\_SKY\_FAILURE = 0x00010000} : Could not measure the local sky + \item {\tt PM\_SOURCE\_MODE\_SKYVAR\_FAILURE = 0x00020000} : Could not measure the local sky variance + \item {\tt PM\_SOURCE\_MODE\_SIZE\_SKIPPED = 0x10000000} : Size could not be determined +\end{itemize} +\item {\bf rank 7 :} Measurement is from an invalid time period or + photometry code. This rank level is not used in the $3\pi$ PV3 + calibration. Measurements were not restricted on the basis of the + time of the observation, and only GPC1 measurements were explicitly + included. +%% MEAS_BAD = ID_MEAS_NOCAL | ID_MEAS_SKIP_PHOTOM +%% ID_MEAS_NOCAL : excluded by time range, not a relevant photcode +%% (only relevant photcodes are considered) +%% ID_MEAS_SKIP_PHOTOM : not used +\item {\bf rank 8 :} Instrumental magnitude out of range. This rank level was not used in the $3\pi$ PV3 calibration. + % (not used, IMAG_MIN, IMAG_MAX = NAN) +\end{itemize} +%% rank 9 : IMAGE_BAD = ID_IMAGE_PHOTOM_NOCAL (not used) +%% rank 10 : measurement out of time range (not used) + +Rank values are assigned exclusively starting from the highest values: +if a measurements satisfieds the rule for \eg, rank 6, it will not be +tested for ranks 5 and lower. After all measurements have been +assigned a ranking value, the set of all measurements with the common +lowest value are selected to be used for the average photometry +analysis. If measurements from ranks 0 through 4 were used for the +average photometry for a given filter, a per-filter mask bit value is +raised identifying which rank was used. These bit are called +\code{ID_SECF_RANK_0} through \code{ID_SECF_RANK_4} (see +Table~\ref{tab:secf_mask_values}). + +\begin{table*} +\begin{center} +\footnotesize +\caption{\label{tab:secf_mask_values} Relphot Per-Filter Info Flag Bit Values} % \vspace{-0.5cm} +\begin{tabular}{lcl} +\hline +\hline +{\bf Bit Name} & {\bf Bit Value} & {\bf Description} \\ +\hline +ID\_SECF\_STAR\_FEW & 0x00000001 & Used within relphot: skip star \\ +ID\_SECF\_STAR\_POOR & 0x00000002 & Used within relphot: skip star \\ +ID\_SECF\_USE\_SYNTH & 0x00000004 & Synthetic photometry used in average measurement \\ +ID\_SECF\_USE\_UBERCAL & 0x00000008 & Ubercal photometry used in average measurement \\ +ID\_SECF\_HAS\_PS1 & 0x00000010 & PS1 photometry used in average measurement \\ +ID\_SECF\_HAS\_PS1\_STACK & 0x00000020 & PS1 stack photometry exists \\ +ID\_SECF\_HAS\_TYCHO & 0x00000040 & Tycho photometry used for synth mags \\ +ID\_SECF\_FIX\_SYNTH & 0x00000080 & Synth mags repaired with zpt map \\ +ID\_SECF\_RANK\_0 & 0x00000100 & Average magnitude uses rank 0 values \\ +ID\_SECF\_RANK\_1 & 0x00000200 & Average magnitude uses rank 1 values \\ +ID\_SECF\_RANK\_2 & 0x00000400 & Average magnitude uses rank 2 values \\ +ID\_SECF\_RANK\_3 & 0x00000800 & Average magnitude uses rank 3 values \\ +ID\_SECF\_RANK\_4 & 0x00001000 & Average magnitude uses rank 4 values \\ +ID\_SECF\_OBJ\_EXT\_PSPS & 0x00002000 & In PSPS ID\_SECF\_OBJ\_EXT is saved here so it fits within 16 bits \\ +ID\_SECF\_STACK\_PRIMARY & 0x00004000 & PS1 stack photometry includes a primary skycell \\ +ID\_SECF\_STACK\_BESTDET & 0x00008000 & PS1 stack best measurement is a detection (not forced) \\ +ID\_SECF\_STACK\_PRIMDET & 0x00010000 & PS1 stack primary measurement is a detection (not forced) \\ +ID\_SECF\_STACK\_PRIMARY\_MULTIPLE & 0x00020000 & PS1 stack object has multiple primary measurements \\ +ID\_SECF\_HAS\_SDSS & 0x00100000 & This photcode has SDSS photometry \\ +ID\_SECF\_HAS\_HSC & 0x00200000 & This photcode has HSC photometry \\ +ID\_SECF\_HAS\_CFH & 0x00400000 & This photcode has CFH photometry (mostly Megacam) \\ +ID\_SECF\_HAS\_DES & 0x00800000 & This photcode has DES photometry \\ +ID\_SECF\_OBJ\_EXT & 0x01000000 & Extended in this band \\ +\hline +\end{tabular} +\end{center} +\end{table*} + +\subsubsection{Iteratively Reweighted Least Squares Fitting} + +With an automatic process applied to hundreds of millions of stars, it +is important for the analysis to provide a measurement of the +photometry of each object which is robust against failures. The +Pan-STARRS\,1 detections have a relatively high rate of non-Gaussian +outliers, partly because of the wide range of instrumental features +affecting the data \citep[see][]{waters2017}. We have used a +technique called Iteratively Reweighted Least Squares (IRLS) fitting +to reduce the sensitivity of the fits to outlier measurements. We +have also used bootstrap resampling to determine confidence limits on +our fits given the observed collection of photometry measurements. In +this case, the analysis is fitting the trivial model that the +photometry measurements are derived from a population with an +underlying constant value. The discussion below applies to both the +average of the chip photometry magnitudes and the forced-warp +photometry fluxes. + +The IRLS analysis starts with an ordinary least squares fit, using the +weights for each measurement as determined from Poisson statistics. +Since our model is a constant flux, this step is equivalent to +calculating a simple weighted average. + +Next, the deviations from the average value for each photometry +measurement are calculated. The deviation, normalized by the Poisson +error, is used to modify the standard weight. We use a Cauchy +function to define a new weight: +\[ +\omega^\prime = \frac{\omega}{1 + r^2} +\] +using +\[ +r = \frac{F_o - F_i}{\sigma} +\] +where $F_o$ is the average magnitude (or flux for forced-warp +photometry), $F_i$ is the measured magnitude (or flux), $\sigma$ is +the standard Poisson-based error on the photometry measurement, and +$\omega$ is the ordinary Poisson weight ($\sigma^{-2}$). This +modified weight has the behavior that if the observed photometry +differs from the model by a substantial amount, the weight is greatly +reduced, while the weight approaches the standard weight if the model +and observed positions agree well. Thus, this procedure is equivalent +to sigma clipping, but allows the outliers to be reduced in impact in +a continuous way, rather than rigidly accepting or rejecting them. + +The weighted average photometry is re-calculated with these modified +weights. New values for $\omega$ are calculated, and the weighted +average is calculated again. On each iteration, the weighted average +photometry values are compared to the values from the previous +iteration. If they have not changed significantly ($< 10^{-6}$) or if +the fractional change is less than some tolerance ($10^{-4}$), then +iterations are halted and the last weighted average values are used. +If convergence is not reached in 10 iterations, the process is halted +in any case and a flag raised for the object to note that IRLS did not +converge. + +% \note{did this happen for any of our targets?} + +To calculate a fit $\chi^2$ value and to determine an appropriate set +of errors for the model parameters, it is necessary to transform the +modified weights into explicit cuts. We have used the rubric that if +the modified weight is less than 30\% of the median weight +($\omega^\prime < 0.3 <\omega>$) then the point is treated as clipped. +The $\chi^2$ is determined from the {\em unclipped} points using the +standard Poisson errors. + +Bootstrap-resampling analysis is used to assess the errors on the fit +parameters: A number of measurements equal to the number of {\em + unclipped} data points are randomly selected from the set of +unclipped data points, with replacement after each selection. These +data points are then used to calculate the weighted average +photometry. The average values is recorded and the process re-run 100 +times. The error on the photometry value is determined as half of the +68\% confidence range for the distribution of average values. +However, if the number of measurements is small, the +bootstrap-resampled measurement of the error may be artificially +small. We record the maximum of the bootstrap-sampling error and the +formal error from the weighted average calculation. The minimumn and +maximum of the unclipped values are also recorded for the chip +photometry. + +% mask values for which wt < threshold (0.3 * median wt) +% we record the min and max values of the unmasked / unclipped subset +% chisq uses only the unmasked +% bootstrap: use only unclipped subset and raw weights to estimate errors + +% \note{bootstrap uses unclipped values and the raw weights? confirmed} + +% \note{reported error is max of bootstrap and formal error? confirmed} + \subsubsection{Stack Photometry} +For the stack photometry, the assessment is different from the chip +and forced-warp photometry: multiple measurements are not used to +calculate an average value. For most of the sky, only a single set of +stack pixels exist for each filter. Ideally, a unique astronomical +object would only be detected once in a given filter, resulting in +only a single measurement of that object from that filter's stack in +the database. In practice, objects within a single stack image are +occasionally split by the analysis code, resulting in multiple +detections of the same object. This situation is discussed in more +detail below. + +\begin{figure*}[htbp] + \begin{center} + \includegraphics[width=\hsize,clip]{{pics/rings.v3.example}.png} + \caption{\label{fig:rings.v3.example} Illustration of overlapping + skycells and the identification of the ``primary'' detections.} + \end{center} +\end{figure*} + +In addition to the these relatively rare failure cases, the objects +detected in the stacks are more likely to have multiple measurements +due to the overlap between neighboring stack images. The skycells +(within which the stacks are generated) for a given projection cell +are defined to have significant overlap between neighbors to ensure a +modestly-extended object can be measured completely on the pixels in a +single skycell image. For the \ippmisc{RINGS.V3} skycell tessellation +used for the $3\pi$ PV3 analysis, this overlap was set to be 60 +arcseconds, \ie, 240 extra pixels on each edge. Within +\ippmisc{RINGS.V3}, projection cells themselves are defined to have an +overlap with neighboring projection cells to avoid gaps due to the +process of tiling the spherical sky with a series of flat +projections. Due to the curved surface of the sky, the amount of +overlap between projection cells increases away from the celestial +equator. Figure~\ref{fig:rings.v3.example} illustrates both skycell +and projection cell overlaps. + +Overlapping stack regions are not statistically independent. In the +typical circumstance, the same raw chip images are used to generate +the input warp images for the skycell on either side of the overlap. +Except for rare edge cases (\eg, an input warp which was rejected from +the stack for one side but not the other), exactly the same input raw +chip pixels contribute to all sets of stack pixels which overlap. It +would therefore be statistically inappropriate to average the multiple +stack measurements from different overlapping skycells. Instead, we +identify a unique set of stack measurements for the end user. + +We identify two different ways in which an appropriate set of unique +stack measurements can be selected. In the first case, if multiple +overlapping skycells contribute measurements to an object, we choose +the representative measurement based on their location in the skycell. +This selection is purely a function of the geometry of the skycells +and the coordinate of the object. We first identify the primary +projection cells, those for which the overlapping regions are closest +to the projection cell center. For regions in the primary projection +cell, we then identify the primary skycells, those for which the +overlapping regions are closest to the center of the skycell. For a +given object, the identification of the primary projection cell and +skycell is calculated based on that the coordinates of the object. We +then find the measurements for the object which came from the primary +projection cell and skycell and identify this set of measurements +(\grizy) as the ``primary'' set. Note that we use the average +position of the object to define the ``primary'' measurements, forcing +measurements from all filters for the same skycell to be ``primary'' +measurements, even if small deviations in the stack positions would +result in one of the filter detections falling on the other side of +the skycell ``primary'' boundary. Thus, for a given object in the +database, we expect all 5 filters to provide a ``primary'' measurement +from the same skycell for each object. + +Since the ``primary'' identification is purely based on the skycell +geometry and the coordinate of the object, there is no guarantee that +any primary measurement is in fact a good or even the best measurement +of the object. While the different overlapping pixels should be +essentially identical, it is possible (due to some of the edge cases +mentioned above) that one of the two sets of pixels is more heavily +masked than the other (\eg., more rejected inputs to the stack). +Thus, it is possible that one of the measurements is valid while the +other is not. To address this possibility, we also identify a set of +``best'' measurements for each object. + +For the stack measurements of an object in a specific filter, if there +are ``primary'' measurements with finite signal-to-noise and PSF +``perfect pixel'' quality factor (\code{PSF_QF_PERFECT}) $> 0.95$, the +measurement with the highest signal-to-noise is marked as ``best''. +If no primary measurement has \code{PSF_QF_PERFECT} $> 0.95$, but a +secondary measurement does, then the secondary measurement with the +highest signal-to-noise is chosen as ``best''. If neither of the +first two cases hold, but there exist primary measurements with lower +\code{PSF_QF_PERFECT} values, the measurement with the highest +\code{PSF_QF_PERFECT} value is chosen as ``best''. Finally, if no +``best'' value has yet been identified, the secondary measurement with +the highest value of \code{PSF_QF_PERFECT} is chosen as ``best''. +Note that the above rules allow for multiple measurements of the same +object from the same skycell pixels. This may occur if the object was +split due to, \eg, saturation or complex morphology. This type of +split should not be common (and in fact reflects a failure of the +algorithm), but we have defined the rules to allows us to choose an +acceptable measurement even in these cases. + \subsubsection{Warp Photometry} + +The calculation of the average forced-warp photometry is performed +very similarly to the average of the chip photometry, with two +important exceptions. First, as discussed above, the forced-warp {\em + fluxes} are averaged, rather than the magnitudes. Second, only the +warp measurements from the skycell which provided the ``best'' stack +measurements are used to calculate the average. Just as the +overlapping stack pixels are not statistically independent, +overlapping warp pixels from the same exposure are also not +statistically independent. It is critical to use only a single +measurement from each input exposure. We choose to use those from the +``best'' stack skycell rather than the ``primary'' stack skycell to +ensure the forced-warp photometry represents the highest quality set +of measurements. Once the measurements from the chosen skycell have +been selected, the same quality cuts are applied to the measurements +as are applied to the chip measurements, as discussed above. \begin{figure*}[htbp] @@ -1106,8 +1536,8 @@ on chip XY04. In each plot, the solid line shows the measured mean residual for stars detected on this chip as a function of the - instrumental magnitude / FWHM$^2$. {\bf top left} X-direction before correction. -{\bf top right} Y-direction before correction. -{\bf bottom left} X-direction after correction. -{\bf bottom right} Y-direction after correction. } + instrumental magnitude / FWHM$^2$. {\bf bottom left} X-direction before correction. +{\bf bottom right} Y-direction before correction. +{\bf top left} X-direction after correction. +{\bf top right} Y-direction after correction. } \end{center} \end{figure*} @@ -1122,5 +1552,5 @@ correction. {\bf bottom right} Y-direction before correction. {\bf top left} X-direction after correction. {\bf top right} - Y-direction after correction. } + Y-direction after correction.} \end{center} \end{figure} @@ -1208,24 +1638,25 @@ Differential Chromatic Refraction (DCR) affects astrometry because the -reference stars used the calibrate the images are not the same color -(SED) as the rest of the stars in the image. For a given star of a -color different from the reference stars, as exposures are taken at -higher airmass, the apparent position of the star will be biased along -the parallactic angle. While it is possible to build a model for the -DCR impact based on the filter response functions and atmospheric -refraction, we have instead elected to use an empirical correction for -the DCR present in the PV3 database. We have measured the DCR trend -using the astrometric residuals of millions of stars after performing -an initial relative astrometry calibration. We define a blue DCR -color ($g-i$) to be used when correcting the filters \gps,\rps,\ips, and a red -DCR color ($z - y$) to be used when correcting the filters $zy$. In -the process of performing the relative astrometry calibration, we -record the median red and blue colors of the reference stars used to -measure the astrometry calibration for each image. As we determine -the astrometry parameters for each object in the database, we record -the median red and blue reference star colors for all images used to -determine the astrometry for a given object. For each star in the -database, we know both the color of the star and the typical color of -the reference stars used to calibrate the astrometry for that star. +reference stars used to the calibrate the images are not the same +color (SED) as the rest of the stars in the image. For a given star +of a color different from the reference stars, as exposures are taken +at higher airmass, the apparent position of the star will be biased +along the parallactic angle. While it is possible to build a model +for the DCR impact based on the filter response functions and +atmospheric refraction, we have instead elected to use an empirical +correction for the DCR present in the PV3 database. We have measured +the DCR trend using the astrometric residuals of millions of stars +after performing an initial relative astrometry calibration. We +define a blue DCR color ($g-i$) to be used when correcting the filters +\gps,\rps,\ips, and a red DCR color ($z - y$) to be used when +correcting the filters $zy$. In the process of performing the +relative astrometry calibration, we record the median red and blue +colors of the reference stars used to measure the astrometry +calibration for each image. As we determine the astrometry parameters +for each object in the database, we record the median red and blue +reference star colors for all images used to determine the astrometry +for a given object. For each star in the database, we know both the +color of the star and the typical color of the reference stars used to +calibrate the astrometry for that star. We measure the mean deviation of the residuals in the parallactic @@ -1286,21 +1717,26 @@ features. +% http://adsabs.harvard.edu/abs/2008SPIE.7021E..05T +% http://adsabs.harvard.edu/abs/2010SPIE.7733E..0EK +% http://adsabs.harvard.edu/abs/2012SPIE.8453E..0KO + The dominant pattern in the astrometric residual is roughly a series of concentric rings. The pattern is similar to the pattern of the -focal surface residuals measured by (REF), which also has a concentric -series of rings with similar spacing. The ``tent'' in the center of -the focal surface reflected in these astrometry residual plots. Our -interpretation of the structure is that the deviations of the focal -plane from the ideal focal surface introduces small-scale PSF changes, -presumably coupled to the optical aberrations, which result in small -changes in the centroid of the object relative to the PSF model at -that location. Since the PSF model shape parameters are only able to -vary at the level of a 6x6 grid per chips, the finer structures are -not included in the PSF model. The PV2 analysis shows the ring -structure more clearly, with a pattern much more closely following the -focal surface deviations. In the PV2 analysis, the PSF model used at -most a 3x3 grid per chip to follow the shape variations, so any -changes caused by the optical aberrations would be less well modeled in -the PV2 analysis, as we observe. +focal surface residuals measured by \cite{onaka.spie}, which also has +a concentric series of rings with similar spacing. The ``tent'' in +the center of the focal surface is reflected in these astrometry +residual plots. Our interpretation of the structure is that the +deviations of the focal plane from the ideal focal surface introduces +small-scale PSF changes, presumably coupled to the optical +aberrations, which result in small changes in the centroid of the +object relative to the PSF model at that location. Since the PSF +model shape parameters are only able to vary at the level of a 6x6 +grid per chips, the finer structures are not included in the PSF +model. The PV2 analysis shows the ring structure more clearly, with a +pattern much more closely following the focal surface deviations. In +the PV2 analysis, the PSF model used at most a 3x3 grid per chip to +follow the shape variations, so any changes caused by the optical +aberrations would be less well modeled in the PV2 analysis, as we +observe. A second pattern which is weakly seen in several chips consists of @@ -1319,7 +1755,28 @@ {\em not} visible at the resolution of these astrometric flat-field images. Fine structures are observed at the \approx 10 pixel scale -similar to the ``tree rings'' reported by the DES team and others -(G. Berstein REF \& REFS). The details of these tree rings are beyond -the scope of this article, and will be explored in future work. +similar to the ``tree rings'' reported by the Dark Energy Survey team +\citep{2014PASP..126..750P} and identified as a result of lateral +diffusion of electrons in the detectors due to electric fields due to +dopant variations. Unlike the photometric tree ring features +discussed above (Section~\ref{sec:phot.flat}), these astrometric tree +rings appear to correspond to the features identified by the DES team. +Lateral electric fields in the detector silicon, caused by variations +in the dopant density, cause the photoelectrons to migrate laterally +in the detector silicon before landing in the pixel wells. This +migration affects the apparent position of the stars, thus affecting +the observed astrometry. A simple lateral translation of the +effective pixel locations would not be detected as it would be +degenerate with the astrometric solution. However, since the lateral +electric fields, and thus the electron migration, vary with position, +the astrometric displacement changes on small scales relative to the +average solution, resulting in residual astrometric structures. The +gradient of the astrometric displacement results in an apparent +expansion or compression of the pixel sizes, resulting in a signal +which can be observed in the flat-field images. For GPC1, unlike the +DES detectors, the amplitude of these flat-field variations are much +smaller than the photometric variations caused by the changing PSF +sized, caused in turn by varying electron diffusion rates. These +features, and the related vertical electron diffusion variations are +discussed in detail in \cite{2018PASP..130f5002M}. Unfortunately, we discovered a problem with the astrometric flat-field @@ -1328,5 +1785,5 @@ \ref{fig:astroflat.zy}, there is significant pixel-to-pixel noise in the the astrometric flat-field images. This pixel-to-pixel noise is -caused by too few stars used in the measuremnt of the flat-field +caused by too few stars used in the measurement of the flat-field structure for the high-resolution sampling. As a result, the astrometric flat-field correction reduces systematic structures on @@ -1342,6 +1799,6 @@ measurements in $i$-band (to reject artifacts detected in a pair of exposures from the same night), with \code{PSF_QF} $> 0.85$ (to reject -excessively-masked objects), and with $mag_{\rm PSF} - mag_{rm Kron} < -0.1$ (to reject galaxies). We then generated histograms of the +excessively-masked objects), and with $mag_{\rm PSF} - mag_{\rm Kron} +< 0.1$ (to reject galaxies). We then generated histograms of the difference between the object position predicted for the epoch of each measurement (based on the proper motion and parallax fit) and the @@ -1350,9 +1807,10 @@ given pixel in the images. From these residual histograms, we can then determine the median and the 68\%-ile range to calculate a robust -standard deviation. This represents the bright-end systematic error -floor for a measurement from a single exposure. The standard -deviations are then plotted in Figure~\ref{fig:allsky.photom.sigma}. -The median value of the standard deviations across the sky is -$(\sigma_\alpha, \sigma_\delta) = (22, 23)$ milliarcseconds. +version of the standard deviation. This represents the bright-end +systematic error floor for a measurement from a single exposure. The +standard deviations are then plotted in +Figure~\ref{fig:allsky.photom.sigma}. The median value of the +standard deviations across the sky is $(\sigma_\alpha, \sigma_\delta) += (22, 23)$ milliarcseconds. The Galactic plane is clearly apparently in these images. Like @@ -1361,5 +1819,5 @@ errors in both R.A. and DEC. This may be due to the larger typical seeing at these high airmass regions, but without further exploration -this is interpretation uncertain. Several features can be seen which +this interpretation is uncertain. Several features can be seen which appear to be an effect of the tie to the Gaia astrometry: the stripes near the center of the DEC image and the right side of the R.A. image. @@ -1371,6 +1829,8 @@ than the \approx 17 mas value in that earlier analysis. We attribute this degradation to the noise introduced by the astrometric -flat-field. This noise can likely be addressed before the DR2 release -of the individual measurement data. +flat-field. + +\note{This noise has been addressed for the DR2 release of the + individual measurement data. show updated maps and residuals} \begin{figure}[htbp] @@ -1475,7 +1935,7 @@ rotation parameters ($A,B$) = (14.82,-12.37) km sec$^{-1}$ pc$^{-1}$ and Solar motion parameters ($U_{\rm sol}, V_{\rm sol}, W_{\rm sol}$) -= (9.32, 11.18, 7.61) km sec$^{-1}$ as determined by Feast \& -Whitelock (REF) using Hipparchos data. Proper motions are determined -from the following: += (9.32, 11.18, 7.61) km sec$^{-1}$ as determined by +\cite{1997MNRAS.291..683F} using Hipparcos data. Proper motions are +determined from the following: \begin{eqnarray} \mu^{\rm gal}_{l} & = & (A \cos (2 l) + B) \cos (b) \\ @@ -1624,17 +2084,19 @@ \subsubsection{Iteratively Reweighted Least Squares Fitting} -After the entire database has been calibrated using the relative -astrometric analysis, we attempt to determine parallax and proper -motions for all objects in the database. We require a minimum of 5 +After the image astrometric parameters have been determined and +applied to the measurements from each image, we attempt to find +the best astrometric parameters (position, parallax and proper +motions) for all objects in the database. We require a minimum of 5 detections and 1 year of data for any object in order for it to be -fitted for proper motion. For a parallax fit, we require at least 7 -detections, 1 year of data, and a parallax factor range of at least -0.25; no object is fitted to parallax without proper motion as well. -If an object is fitted for parallax, it is also fitted with a model -including only proper motion and only a mean position. The chisq for -all three fits is saved. Currently, the highest order fit allowed is -saved in the database. The resulting parallax and proper motion -measurements are inserted back into the DVO database for use by -science queries. +fitted for just proper motion. For a parallax and proper-motion fit, +we require at least 7 detections, 1 year of data, and a parallax +factor range of at least 0.25; no object is fitted to parallax without +proper motion as well. If an object is fitted for parallax, it is +also fitted with a model including only proper motion and only a mean +position. The chisq for all three fits is saved. Currently, the +highest order fit allowed is saved in the database, regardless of the +significance of the improvement in adding parameters. The resulting +parallax and proper motion measurements are inserted back into the DVO +database for use by science queries. With an automatic process applied to hundreds of millions of stars, it @@ -1746,6 +2208,6 @@ \bibliographystyle{apj} -%\bibliography{lib}{} -\input{calibration.bbl} +\bibliography{lib}{} +% \input{calibration.bbl} \end{document}