IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changeset 40597


Ignore:
Timestamp:
Jan 9, 2019, 4:34:33 PM (8 years ago)
Author:
eugene
Message:

update calibration text

Location:
trunk/doc/release.2015/ps1.calibration
Files:
2 edited

Legend:

Unmodified
Added
Removed

  • trunk/doc/release.2015/ps1.calibration/Makefile

    r39893 r40597  
    44# remember to set \pdfoutput at the top
    55
    6 DO_BIBTEX = 0
     6DO_BIBTEX = 1
    77# remember to change from \bibliography to \input{.bbl} at the bottom
    88
    … …  
    1414pdf: calibration.pdf
    1515tgz: calibration.tgz
     16
     17quick: calibration.quick.pdf
    1618
    1719FILES = \

  • trunk/doc/release.2015/ps1.calibration/calibration.tex

    r40079 r40597  
    9898\begin{abstract}
    9999
    100 The Pan-STARRS\,1 $3\pi$ survey has produced photometry and astrometry
    101 covering the \approx 30,000 square degrees $\delta > -30$\degrees. 
    102 This article describes the photometric and astrometric calibration of this survey.
     100We present the details of the photometric and astrometric calibration
     101of the Pan-STARRS\,1 $3\pi$ Survey.  The photometric goals were to
     102reduce the systematic effects introduced by the camera and detectors,
     103and to place all of the observations into a photometric system with
     104consistent zero points over the entire area surveyed, the \approx
     10530,000 square degrees north of $\delta = -30$\degrees.  The
     106astrometric calibration compensates for similar systematic effects so
     107that positions, proper motions, and parallaxes are reliable as well.
     108The Pan-STARRS Data Release 2 (DR2) astrometry is tied to the Gaia DR1
     109release.
    103110
    104111\end{abstract}
    … …  
    108115
    109116\section{Introduction}\label{sec:intro}
    110 
    111 This is the fifth in a series of seven papers describing the
    112 Pan-STARRS1 Surveys, the data reduction techiques and the resulting
    113 data products.  This paper (Paper V) describes the final calibration
    114 process, and the resulting photometric and astrometric quality.
    115 
    116 %Chambers et al. 2017 (Paper I)
    117 %The Pan-STARRS\,1 Surveys
    118 \citet[][Paper I]{chambers2017}
    119 provides an overview of the Pan-STARRS System, the design and
    120 execution of the Surveys, the resulting image and catalog data
    121 products, a discussion of the overall data quality and basic
    122 characteristics, and a brief summary of important results.
    123 
    124 %Magnier et al. 2017 (Paper II)
    125 %Pan-STARRS Data Processing Stages
    126 \citet[][Paper II]{magnier2017c}
    127 describes how the various data processing stages are organised and implemented
    128 in the Imaging Processing Pipeline (IPP), including details of the
    129 the processing database which is a critical element in the IPP infrastructure .
    130 
    131 %Waters et al. 2017 (Paper III)
    132 %Pan-STARRS Pixel Processing : Detrending, Warping, Stacking
    133 \citet[][Paper III]{waters2017}
    134 describes the details of the pixel processing algorithms, including detrending, warping, and adding (to create stacked images) and subtracting (to create difference images) and resulting image products and their properties.
    135 
    136 
    137 %Magnier et al. 2017 (Paper IV)
    138 %Pan-STARRS Pixel Analysis : Source Detection
    139 \citet[][Paper IV]{magnier2017a}
    140 describes the details of the source detection and photometry, including point-spread-function and extended source fitting models, and the techniques for ``forced" photometry measurements.
    141 
    142 %Magnier et al. 2017 (Paper V)
    143 %Pan-STARRS Photometric and Astrometric Calibration
    144 %\citet[][Paper V]{magnier2017b}
    145 %describes the final calibration process, and the resulting photometric and astrometric quality. 
    146 
    147 
    148 %Flewelling et al. 2017 (Paper VI)
    149 %Pan-STARRS 1 Database and Data Products
    150 \citet[][Paper VI]{flewelling2017}
    151 describes  the details of the resulting catalog data and its organization in the Pan-STARRS database.
    152 %
    153 %
    154 \citet[][Paper VII]{huber2017}
    155 %Huber et al. 2017 (Paper VII)
    156 describes the Medium Deep Survey in detail, including the unique issues and data products specific to that survey. The Medium Deep Survey is not part of Data Release 1. (DR1)
    157 
    158 %
    159 The Pan-STARRS1 filters and photometric system have already been
    160 described in detail in \cite{2012ApJ...750...99T}.
    161 
    162 {\color{red} {\em Note: These papers are being placed on arXiv.org to
    163     provide crucial support information at the time of the public
    164     release of Data Release 1 (DR1). We expect the arXiv versions to
    165     be updated prior to submission to the Astrophysical Journal in
    166     January 2017. Feedback and suggestions for additional information
    167     from early users of the data products are welcome during the
    168     submission and refereeing process.}}
    169 
    170 \section{Pan-STARRS\,1}
    171117
    172118From May 2010 through March 2014, the Pan-STARRS Science Consortium
    … …  
    176122formation and architecture of the Milky Way galaxy, and the search for
    177123Type Ia supernovae to measure the history of the expansion of the
    178 universe. 
     124universe.  The majority of the time (56\%) was spent on surveying the
     125$\frac{3}{4}$ of the sky north of $-30$ Declination with
     126\grizy\ filters in the so-called $3\pi$ Survey.  Another $\sim 25\%$
     127of the time was concentrated on repeated deep observations of 10
     128specific fields in the Medium-Deep Survey.  The rest of the time was
     129used for several other surveys, including a search for potentially
     130hazardous asteroids in our solar system.  The details of the
     131telescope, surveys, and resulting science publications are described
     132by \cite{chambers2017}.
    179133
    180134The wide-field \PSONE\ telescope consists of a 1.8~meter diameter
    … …  
    205159Maui.
    206160
     161%The Processing Version 3 (PV3) reduction represents the third full
     162Pan-STARRS produced its first large-scale public data release, Data
     163Release 1 (DR1) on 16 December 2016.  DR1 contains the results of the
     164third full reduction of the Pan-STARRS $3\pi$ Survey archival data,
     165identified as PV3.  Previous reductions \citep[PV0, PV1, PV2;
     166 see][]{magnier2017.datasystem} were used internally for pipeline
     167optimization and the development of the initial photometric and
     168astrometric reference catalog \citep{magnier2017.calibration}.  The
     169products from these reductions were not publicly released, but have
     170been used to produce a wide range of scientific papers from the
     171Pan-STARRS 1 Science Consortium members \citep{chambers2017}.  DR1
     172contained only average information resulting from the many individual
     173images obtained by the $3\pi$ Survey observations.  A second data
     174release, DR2, was made available \note{20 January 2019}.  DR2 provides
     175measurements from all of the individual exposures, and include an
     176improved calibration of the PV3 processing of that dataset.
     177
     178This is the fifth in a series of seven papers describing the
     179Pan-STARRS1 Surveys, the data reduction techiques and the resulting
     180data products.  This paper (Paper V) describes the final calibration
     181process, and the resulting photometric and astrometric quality.
     182
     183%Chambers et al. 2017 (Paper I)
     184%The Pan-STARRS\,1 Surveys
     185\citet[][Paper I]{chambers2017}
     186provides an overview of the Pan-STARRS System, the design and
     187execution of the Surveys, the resulting image and catalog data
     188products, a discussion of the overall data quality and basic
     189characteristics, and a brief summary of important results.
     190
     191%Magnier et al. 2017 (Paper II)
     192%Pan-STARRS Data Processing Stages
     193\citet[][Paper II]{magnier2017.datasystem}
     194describes how the various data processing stages are organised and implemented
     195in the Imaging Processing Pipeline (IPP), including details of the
     196the processing database which is a critical element in the IPP infrastructure .
     197
     198%Waters et al. 2017 (Paper III) Pan-STARRS Pixel Processing :
     199%Detrending, Warping, Stacking
     200\citet[][Paper III]{waters2017} describes the details of the pixel
     201processing algorithms, including detrending, warping, and adding (to
     202create stacked images) and subtracting (to create difference images)
     203and resulting image products and their properties.
     204
     205
     206%Magnier et al. 2017 (Paper IV)
     207%Pan-STARRS Pixel Analysis : Source Detection
     208\citet[][Paper IV]{magnier2017.analysis} describes the details of the source
     209detection and photometry, including point-spread-function and extended
     210source fitting models, and the techniques for ``forced" photometry
     211measurements.
     212
     213%Magnier et al. 2017 (Paper V)
     214%Pan-STARRS Photometric and Astrometric Calibration
     215%\citet[][Paper V]{magnier2017.calibration}
     216%describes the final calibration process, and the resulting photometric and astrometric quality. 
     217% THIS PAPER
     218
     219%Flewelling et al. 2017 (Paper VI)
     220%Pan-STARRS 1 Database and Data Products
     221\citet[][Paper VI]{flewelling2017}
     222describes  the details of the resulting catalog data and its organization in the Pan-STARRS database.
     223
     224%Huber et al. 2017 (Paper VII)
     225\citet[][Paper VII]{huber2017} describes the Medium Deep Survey in
     226detail, including the unique issues and data products specific to that
     227survey. The Medium Deep Survey is not part of Data Releases 1 or 2 and
     228will be made available in a future data release.
     229
     230%
     231The Pan-STARRS1 filters and photometric system have already been
     232described in detail in \cite{2012ApJ...750...99T}.
     233
     234%% {\color{red} {\em Note: These papers are being placed on arXiv.org to
     235%%     provide crucial support information at the time of the public
     236%%     release of Data Release 1 (DR1). We expect the arXiv versions to
     237%%     be updated prior to submission to the Astrophysical Journal in
     238%%     January 2017. Feedback and suggestions for additional information
     239%%     from early users of the data products are welcome during the
     240%%     submission and refereeing process.}}
     241
     242\section{Pan-STARRS\,1 Data Analysis}
     243
    207244Images obtained by \PSONE\ are automatically processed in real time by
    208 the \PSONE\ Image Processing Pipeline \citep[IPP,][]{magnier2017a}.
     245the \PSONE\ Image Processing Pipeline \citep[IPP,][]{magnier2017.datasystem}.
    209246Real-time analysis goals are aimed at feeding the discovery pipelines
    210247of the asteroid search and supernova search teams.  The data obtained
    … …  
    212249complete re-processing of the data: Processing Versions 1, 2, and 3
    213250(PV1, PV2, and PV3).  The real-time processing of the data is
    214 considered ``PV0''.  Except as otherwise noted, the PV3 analysis of
    215 the data is used for the purpose of this article.
     251considered ``PV0''.  Except as otherwise noted, this article describes
     252the calibration of the PV3 analysis of the data.  Between the first
     253(DR1) and second (DR2) data releases, improvements were made to the
     254calibration of both the photometry and astrometry, as described in
     255this article.
    216256
    217257The data processing steps are described in detail by \cite{waters2017}
    218 and \cite{magnier2017a,magnier2017b}.  In summary, individual images
     258and \cite{magnier2017.datasystem,magnier2017.analysis}.  In summary, individual images
    219259are detrended: non-linearity and bias corrections are applied, a dark
    220260current model is subtracted and flat-field corrections are applied.
    … …  
    226266discussed below, preliminary astrometric and photometric calibrations
    227267are performed for all chips in a single exposure in a single analysis.
     268We refer to these measurements as the ``chip'' photometry and
     269astrometry products.
    228270
    229271Chip images are geometrically transformed based on the astrometric
    … …  
    241283% from images for a single night (nightly stacks). 
    242284
    243 Astronomical objects are detected and characterized in the stacks
     285Astronomical objects are detected and characterized in the stack
    244286images.  The details of the analysis of the sources in the stack
    245 images are discussed in \cite{magnier2017b}, but in brief these include
     287images are discussed in \cite{magnier2017.analysis}, but in brief these include
    246288PSF photometry, along with a range of measurements driven by the goals
    247289of understanding the galaxies in the images.  Because of the
    … …  
    256298To recover most of the photometric quality of the individual chip
    257299images, while also exploiting the depth afforded by the stacks, the
    258 PV3 analysis make use of forced photometry on the individual warp
     300PV3 analysis makes use of forced photometry on the individual warp
    259301images.  PSF photometry is measured on the warp images for all sources
    260302which are detected in the stack images images.  The positions
    … …  
    267309measurement of the faint source flux is determined.  The details of
    268310this analysis are described in detail in Magnier et al
    269 \cite{magnier2017b}.
    270 
    271 In this article, we discuss the photometric calibration of the
    272 individual exposures, the stacks, and the warp imags.  We also discuss
    273 the astrometric calibration of the individual exposures and the stack
    274 images.
     311\cite{magnier2017.analysis}.
     312
     313The data products from the chip photometry, stack photometry, and
     314forced-warp photometry analysis stages are ingested into the internal
     315calibration database called the Desktop Virtual Observatory, or DVO
     316\citep[see Section~4 in][]{magnier2017.datasystem} and used for
     317photometric and astrometric calibrations.  In this article, we discuss
     318the photometric calibration of the individual exposures, the stacks,
     319and the warp imags.  We also discuss the astrometric calibration of
     320the individual exposures and the stack images.
    275321
    276322\section{Astrometric Models}
    … …  
    293339where $P,Q$ are the tangent plane coordinates, $X_{\rm chip}, Y_{\rm
    294340  chip}$ are the coordinates on the 60 GPC1 chips, and $C^P_{i,j}, C^Q_{i,j}$
    295 are the polynomial coefficients for each order.  In the \code{psastro}
     341are the polynomial coefficients for each order.  In the \ippprog{psastro}
    296342analysis, $i + j <= N_{\rm order}$ where the order of the fit, $N_{\rm
    297343  order}$, may be 1 to 3, under the restriction that sufficient stars
    … …  
    305351sky coordinates to a locally cartesian tangent plane coordinate system.
    306352A set of polynomials is then used to relate the tangent plane
    307 coordinates to a 'focal plane' coordinate system, $L,M$:
     353coordinates to a `focal plane' coordinate system, $L,M$:
    308354\begin{eqnarray}
    309355P & = & \sum_{i,j} C^P_{i,j} L^i M^j \\
    310356Q & = & \sum_{i,j} C^Q_{i,j} L^i M^j
    311357\end{eqnarray}
    312 This set of polynomial accounts for effects such as optical distortion
     358This set of polynomials accounts for effects such as optical distortion
    313359in the camera and distortions due to changing atmospheric refraction
    314360across the field of the camera.  Since these effects are smooth across
    315361the field of the camera, a single pair of polynomials can be used for
    316 each exposure.  Like in the chip analysis about, the \code{psastro}
     362each exposure.  Like in the chip analysis about, the \ippprog{psastro}
    317363code restricts the exponents with the rule $i + j <= N_{\rm order}$
    318364where the order of the fit, $N_{\rm order}$, may be 1 to 3, under the
    … …  
    331377tangent plane), but the relationship between the chip and focal plane
    332378is represented with only the linear terms in the polynomial,
    333 supplemented by a course grid of displacements, $\delta L, \delta M$ sampled
     379supplemented by a coarse grid of displacements, $\delta L, \delta M$ sampled
    334380across the coordinate range
    335381of the chip.  This displacement grid may have a resolution of up to
    … …  
    343389\end{eqnarray}
    344390
    345 {\bf WCS Keywords} When this polynomial representation is written to
    346 the output files, a set of WCS keywords are used to define the
    347 astrometric transformation elements.  It is necessary to transform the
    348 simply polynomials above into an alternate form:
    349 \begin{eqnarray}
    350   P & = & \sum_{i,j} C^P_{i,j} (X_{\rm chip} - X_0)^i (Y_{\rm chip} - Y_0)^j \\
    351   Q & = & \sum_{i,j} C^Q_{i,j} (X_{\rm chip} - X_0)^i (Y_{\rm chip} - Y_0)^j
    352 \end{eqnarray}
     391\note{does this section need more? does this section need to be moved?}
     392
     393%% Include a description of the WCS keywords used to represent the fit elements?
     394
     395%% {\bf WCS Keywords} When this polynomial representation is written to
     396%% the output files, a set of WCS keywords are used to define the
     397%% astrometric transformation elements.  It is necessary to transform the
     398%% simply polynomials above into an alternate form:
     399%% \begin{eqnarray}
     400%%   P & = & \sum_{i,j} C^P_{i,j} (X_{\rm chip} - X_0)^i (Y_{\rm chip} - Y_0)^j \\
     401%%   Q & = & \sum_{i,j} C^Q_{i,j} (X_{\rm chip} - X_0)^i (Y_{\rm chip} - Y_0)^j
     402%% \end{eqnarray}
    353403
    354404%% \note{need to complete this discussion of the WCS keywords, both
    … …  
    383433ensure the warps are combined using consistent flux units.
    384434
    385 The program used for the real-time calibration, \code{psastro}, loads
    386 the measurements of the chip detections from their individual
    387 \code{cmf}-format files.  It uses the header information populated at
    388 the telescope to determine an initial astrometric calibration guess
    389 based on the position of the telescope boresite right ascension,
    390 declination and position angle as reported by the telescope \& camera
    391 subsystems.  Using the initial guess, \code{psastro} loads astrometric
    392 and photometric data from the reference database. 
     435The program used for the real-time calibration, \ippprog{psastro},
     436loads the measurements of the chip detections from their individual
     437output catalog files.  It uses the header information populated at the
     438telescope to determine an initial astrometric calibration guess based
     439on the position of the telescope boresite right ascension, declination
     440and position angle as reported by the telescope \& camera subsystems.
     441Using the initial guess, \ippprog{psastro} loads astrometric and
     442photometric data from the reference database.
    393443
    394444\subsection{Reference Catalogs}
    … …  
    396446
    397447During the course of the PS1SC Survey, several reference databases
    398 have been used.  For the first 20 months of the survey, \code{psastro}
    399 used a reference catalog with synthetic PS1 \grizy\ photometry
    400 generated by the Pan-STARRS IPP team based on based combined
    401 photometry from Tycho (B, V), USNO (red, blue, IR), and 2MASS $J, H,
    402 K$.  The astrometry in the database was from 2MASS.  After 2012 May, a
    403 reference catalog generated from internal re-calibration of the PV0
    404 analysis of PS1 photometry and astrometry was used for the reference
    405 catalog. 
     448have been used.  For the first 20 months of the survey,
     449\ippprog{psastro} used a reference catalog with synthetic PS1
     450\grizy\ photometry generated by the Pan-STARRS IPP team based on based
     451combined photometry from Tycho (B, V), USNO \citep[red, blue,
     452  IR][]{2003AJ....125..984M}, and 2MASS
     453$J, H, K$ \citep{2006AJ....131.1163S}.  The astrometry in the database was from 2MASS
     454\citep{2006AJ....131.1163S}.  After 2012 May, a reference catalog
     455generated from internal re-calibration of the PV0 analysis of PS1
     456photometry and astrometry was used for the reference catalog.
    406457
    407458% \note{discuss history of the different refcats?} 
    … …  
    423474false-positive match, especially as many of the reference stars may
    424475not be detected in the GPC1 image.  The seletion of the reference
    425 stars includes a limit on the brightest and fainted magnitude of the
     476stars includes a limit on the brightest and faintest magnitudes of the
    426477stars selected.
    427478
    … …  
    443494
    444495The first step of the analysis is to attempt to find the match between
    445 the reference stars and the detected objects.  \code{psastro} uses 2D
     496the reference stars and the detected objects.  \ippprog{psastro} uses 2D
    446497cross correlation to search for the match.  The guess astrometry
    447498calibration is used to define a predicted set of $X^{\rm ref}_{\rm
    … …  
    468519value by a small amount.  For each trial, the peak pixel is found and
    469520a figure of merit is measured.  The figure of merit is defined as
    470 $\frac{\sigma^2_x + \sigma^2_y}{N_p^4}$ where $\sigma^2_{x,y}$ are the
     521$\frac{\sigma^2_x + \sigma^2_y}{N_p^4}$ where $\sigma^2_{x,y}$ is the
    471522second moment of $\Delta X,Y$ for the star pairs associated with the
    472523peak pixel, and $N_p$ is the number of star pairs in the peak.  This
    … …  
    510561distortion, we choose a single common plate scale for the set of chips
    511562and re-define the chip $\rightarrow$ sky calibrations as a set of chip
    512 $\rightarrow$ focal plane transformation using that common pixel
     563$\rightarrow$ focal plane transformations using that common pixel
    513564scale.  We can now compare the observed focal plane coordinates,
    514 derived from the chip coordinates, and the tangent plane coordiantes,
     565derived from the chip coordinates, and the tangent plane coordinates,
    515566derived from the projection of the reference coordinates.  One caveat
    516567is that the chip reference coordinates are also degenerate with the
    … …  
    526577
    527578Once the common distortion coming from the optics and atmosphere have
    528 been modeled, \code{psastro} determines polynomial transformations
     579been modeled, \ippprog{psastro} determines polynomial transformations
    529580from the 60 chips to the focal plane coordinate system.  In this
    530581stage, 5 iterations of the chip fits are performed.  Before each
    … …  
    542593
    543594After the astrometric calibration has finished, the photometric
    544 calibration is performed by \code{psastro}.  When the reference stars
    545 are loaded, the apparent magnitude in the filter of interest is also
    546 loaded.  Stars for which the reference magnitude is brighter than
     595calibration is performed by \ippprog{psastro}.  When the reference
     596stars are loaded, the apparent magnitude in the filter of interest is
     597also loaded.  Stars for which the reference magnitude is brighter than
    547598(\grizy) = (19, 19, 18.5, 18.5, 17.5) are used to determine the zero
    548599points by comparison with the instrumental magnitudes.  For the PV3
    549600analysis, an outlier-rejecting median is used to measure the zero
    550 point. For early versions of the analysis, when the reference catalog
    551 used synthetic magnitudes, it was necessary to search for the blue
    552 edge of the distribution: the synthetic magnitude poorly predicted the
    553 magnitudes of stars in the presence of significant extinction or for
    554 the very red stars, making the blue edge somewhat more reliable.  Note
    555 that we do not include an airmass correction in this zero point
    556 analysis: the airmass correction is folded into the observed zero
    557 point.  The zero point may be measured separately for each chip or as
    558 a single value for the entire exposure; the latter option was used for
    559 the PV3 analysis.
     601point. For early versions of the real-time analysis, when the
     602reference catalog used synthetic magnitudes, it was necessary to
     603search for the blue edge of the distribution: the synthetic magnitude
     604poorly predicted the magnitudes of stars in the presence of
     605significant extinction or for the very red stars, making the blue edge
     606somewhat more reliable as a reference than the mean.  Once the
     607calibration was based on a reference catalog generated from
     608\PSONE\ photometry, this methods was no longer needed.  Note that we
     609do not include an airmass correction in this zero point analysis: the
     610airmass correction is folded into the observed zero point.  The zero
     611point may be measured separately for each chip or as a single value
     612for the entire exposure; the latter option was used for the PV3
     613analysis.
    560614
    561615\subsection{Real-time outputs}
    562616
    563 The calibrations determined by \code{psastro} as saved as part of the
    564 header information in the output FITS tables.  A single
    565 multi-extension FITS table is written using the \code{smf} format.  In
    566 these files, the measurements from each chip are written as a separate
    567 FITS table.  A second FITS extension for each chip is used to store
    568 the header information from the original chip image.  The original
    569 chip header is modified so that the extension corresponds to an image
    570 with no pixels data: \code{NAXIS} is set to 0, even though
    571 \code{NAXIS1} and \code{NAXIS2} are retained with the original
    572 dimensions of the chip.  A pixel-less primary header unit (PHU) is
    573 generated with a summary of some of the important and common
    574 chip-level keywords (e.g., \code{DATE-OBS}).  The astrometric
    575 transformation information for each chip is saved in the corresponding
    576 header using standard (and some non-standard) WCS keywords.  For the
    577 two-level astrometric model, the PHU header carries the astrometric
    578 transformation related to the projection and the camera-wide
    579 distortions.  Photometric calibrations are written as a set of
    580 keywords to individual chip headers, and if the calibration is
    581 performed at the exposure-level, to the PHU.  The photometry
    582 calibration keywords are:
     617The calibrations determined by \ippprog{psastro} are saved as part of
     618the header information in the output FITS tables.  For each exposure,
     619a single multi-extension FITS table is written.  In these files, the
     620measurements from each chip are written as a separate FITS table.  A
     621second FITS extension for each chip is used to store the header
     622information from the original chip image.  The original chip header is
     623modified so that the extension corresponds to an image with no pixel
     624data: \code{NAXIS} is set to 0, even though \code{NAXIS1} and
     625\code{NAXIS2} are retained with the original dimensions of the chip.
     626A pixel-less primary header unit (PHU) is generated with a summary of
     627some of the important and common chip-level keywords (e.g.,
     628\code{DATE-OBS}).  The astrometric transformation information for each
     629chip is saved in the corresponding header using standard (and some
     630non-standard) WCS keywords.  For the two-level astrometric model, the
     631PHU header carries the astrometric transformation related to the
     632projection and the camera-wide distortions.  Photometric calibrations
     633are written as a set of keywords to individual chip headers, and if
     634the calibration is performed at the exposure-level, to the PHU.  The
     635photometry calibration keywords are:
    583636\begin{itemize}
    584637\item \code{ZPT_REF} : the nominal zero point for this filter
    … …  
    596649
    597650Data from the GPC1 chip images, the stack images, and the warp images
    598 are loaded into DVO using the real-time analysis astrometric
    599 calibration to guide the association of detections into objects.
    600 After the full PV3 DVO database was constructed, including all of the
    601 chip, stack, and warp detections, several external catalogs were
    602 merged into the database.  First, the complete 2MASS PSC was loaded
    603 into a stand-alone DVO database, which was then merged into the PV3
    604 master database.  Next the DVO database of synthetic photometry in the
    605 PS1 bands (see Section~\ref{sec:synthdb}) was merged in.  Next, the
    606 full Tycho database was added, followed by the AllWISE database.
    607 After the Gaia release in August 2016 \citep{2016AA...595A...2G}, we
    608 generated a DVO database of the Gaia positional and photometric
    609 information and merged that into the master DVO database.
     651are loaded into the DVO calibration database using the real-time
     652analysis astrometric calibration to guide the association of
     653detections into objects.  After the full PV3 DVO database was
     654constructed, including all of the chip, stack, and warp detections,
     655several external catalogs were merged into the database.  First, the
     656complete 2MASS PSC was loaded into a stand-alone DVO database, which
     657was then merged into the PV3 master database.  Next the DVO database
     658of synthetic photometry in the PS1 bands (see
     659Section~\ref{sec:synthdb}) was merged in.  Next, the full Tycho
     660database was added, followed by the AllWISE database.  After the Gaia
     661release in August 2016 \citep{2016AA...595A...2G}, we generated a DVO
     662database of the Gaia positional and photometric information and merged
     663that into the master PV3 $3\pi$ DVO database.
    610664
    611665%% \note{need to describe the assignment of flags, etc, for the external data sources}.
    … …  
    672726on the reference photometric night of MJD 55744 (UT 02 July 2011).
    673727\cite{2014ApJ...795...45S} and \cite{2015ApJ...815..117S} have
    674 re-examined the photometry of Calspec standards %% XXX FIX: \citep{Bohlin.1996} as
     728re-examined the photometry of Calspec standards \citep{1996AJ....111.1743B} as
    675729observed by PS1.  \cite{2014ApJ...795...45S} reject 2 of the 7 stars
    676730used by \cite{2012ApJ...750...99T} and add photometry of 5 additional
    … …  
    704758split into three main components:
    705759\[
    706 zp_{\rm total} = zp_{\rm nominal} + M_{cal} + K_{rm \lambda}(sec \zeta - 1)
     760zp_{\rm total} = zp_{\rm nominal} + M_{cal} + K_{\rm \lambda}(\sec \zeta - 1)
    707761\]
    708 where $zp_{\rm nominal}$ and $K_{rm \lambda}$ are static values for
     762where $zp_{\rm nominal}$ and $K_{\rm \lambda}$ are static values for
    709763each filter representing respectively the nominal zero point and the
    710764slope of the trend with respect to the airmass ($\zeta$) for each
    … …  
    756810camera with the field of view of the PS1 GPC1, the airmass may vary
    757811significantly within the field of view, especially at low elevations.
    758 In the worst cases, at the celestial pole, the airmass range within a
    759 single exposure is XXX - XXX.  The complete calibrated (`relative')
    760 magnitude is determined from the stored database values as:
     812In the worst cases, at the celestial pole, the airmass within a single
     813exposure may span a range of 2.56 - 2.93.  The complete calibrated
     814(`relative') magnitude is determined from the stored database values
     815as:
    761816\[
    762817M_{\rm rel} = M_{\rm inst} - 25.0 + zp_{\rm ref} + M_{\rm cal} + M_{\rm flat} + K_\lambda (sec \zeta - 1).
    … …  
    803858\[ M_{ave} = \frac{\sum_i M_{rel,i} w_i}{\sum_i w_i} \]
    804859We find that the color difference of the different chips can be
    805 ignored, and set the value of $A$ to 0.0.
    806 Note that we only use a single mean airmass extinction term for all
    807 exposures -- the difference between the mean and the specific value
    808 for a given night is taken up as an additional element of the
    809 atmospheric attenuation.
     860ignored, and set the color-trend slope to 0.0.  Note that we only use
     861a single mean airmass extinction term for all exposures -- the
     862difference between the mean and the specific value for a given night
     863is taken up as an additional element of the atmospheric attenuation.
    810864
    811865%% \note{color-color terms between chips?}
    … …  
    843897rejections do not catch all cases of bad measurements.
    844898
    845 %% \citep[\code{PSF_QF} $< 0.85$, see][]{magnier2017b};
     899%% \citep[\code{PSF_QF} $< 0.85$, see][]{magnier2017.analysis};
    846900%% \note{refer to the PSPHOT bad and poor psphot bits?} 
    847901
    … …  
    855909from the recalculated mean. 
    856910
    857 Suspicious stars are also exclude from the analsis.  We exclude stars
     911Suspicious stars are also excluded from the analysis.  We exclude stars
    858912with reduced $\chi^2$ values more than 20.0, or more than 2$\times$
    859913the median, whichever is larger.  We also exclude stars with standard
    … …  
    893947IPP cluster: for PV3, 100 parallel hosts are used.  These machines by
    894948design control data from a large number of unconnected small patches
    895 on the sky, with the goal of speeding queries for arbitrary chunks of
     949on the sky, with the goal of speeding queries for arbitrary regions of
    896950the sky.  As a result, this parallelization is entirely inappropriate
    897951as the basis of the relative photometry analysis.  For the relative
    … …  
    931985region host may be updated.
    932986
    933 The completely work flow of the all-sky relative photometry analysis
     987The complete work flow of the all-sky relative photometry analysis
    934988starts with an instance of the program running on a master computer.
    935989This machine loads the image database table and assigns the images to
    … …  
    9791033
    9801034\subsubsection{Photometric Flat-field}
     1035\label{sec:phot.flat}
    9811036
    9821037For PV3, the relphot analysis was performed two times.  The first
    … …  
    10201075Especially notable in the bluer filters is a pattern of quarter
    10211076circles centered on the corners of the chips.  These patterns are
    1022 similar to the ``tree rings'' reported by the DES team and others
    1023 (G. Berstein REF \& REFS).  The details of these tree rings are beyond
    1024 the scope of this article, and will be explored in future work.
    1025 Unlike the tree ring features discussed by these other authors, the
    1026 features observed in the GPC1 photometry are not caused by an
    1027 interaction of the flat-field with the effective pixel geometry.
    1028 Instead, these photometric features are due to low-level changes in
    1029 the PSF size which we attribute to variable charge diffusion (Magnier
    1030 in prep).
     1077similar to the ``tree rings'' reported by the Dark Energy Survey team
     1078\citep{2014PASP..126..750P} and identified as a result of lateral
     1079migration of electrons in the detectors due to electric fields due to
     1080dopant variations.  Unlike the tree ring features discussed by these
     1081other authors, the strong features observed in the GPC1 photometry are
     1082not caused by lateral electric fields, but rather by variations in the
     1083vertical electron diffusion rate due to electric field variations
     1084perpendicular to the plane of the detector.  This effect is discussed
     1085in detail by \cite{2018PASP..130f5002M}.  The photometric features are
     1086due to low-level changes in the PSF size which we attribute to the
     1087variable charge diffusion.
    10311088
    10321089Other features include some poorly responding cells (e.g., in XY14)
    … …  
    10521109the bright end.
    10531110
     1111For the stack calibration, we calculate two separate zero points: one
     1112for photometry tied to the PSF model and a second for the
     1113aperture-like measurements (total aperture magnitudes, Kron magnitude,
     1114cicular fixed-radius aperture magnitudes).  This split is needed
     1115because of the limited quality of the stack PSF photometry due to the
     1116highly variable PSF in the stacks.  Aperture magnitudes, however, are
     1117not significantly affected by the PSF variations.  We therefore tie
     1118the PSF magnitudes to the average of the chip photometry PSF
     1119magnitudes, but the aperture-like magnitudes are tied by equating the
     1120stack Kron magnitudes to the average chip Kron magnitudes.  {\em Note
     1121  that for DR1, this split zero point calibration was used; instead
     1122  all stack photometry was tied to the average chip photometry via the
     1123  PSF magnitudes.}  The result of using a single zero point is that
     1124the stack PSF magnitudes are consistent across the sky with the chip
     1125PSF magnitudes, but the aperture-like magnitudes show significant
     1126spatial variations.  Figure~\ref{fig:stack.bad.kron} illustrates the
     1127impact of using a single PSF zero point for the stack photometry.
     1128This split is not needed for the forced-warp photometry since the
     1129individual warps have well-defined PSfs.
     1130
    10541131\subsection{Photometry Calibration Quality}
    10551132
    … …  
    10611138reject artifacts detected in a pair of exposures from the same night),
    10621139with \code{PSF_QF} $> 0.85$ (to reject excessively-masked objects),
    1063 and with $mag_{\rm PSF} - mag_{rm Kron} < 0.1$ (to reject galaxies).
     1140and with $mag_{\rm PSF} - mag_{\rm Kron} < 0.1$ (to reject galaxies).
    10641141We then generated histograms of the difference between the average
    10651142magnitude and the apparent magnitude in an individual image for each
    … …  
    10921169\subsection{Calculation of Object Photometry}
    10931170
    1094 \subsubsection{Iteratively Reweighted Least Squares Fitting (1D)}
     1171Once the image photometric calibrations (zero points and flat-field
     1172corrections) have been determined and applied to the measuremetns from
     1173each image, we can calculate the best average photometry for each
     1174object.  We calculate average magnitudes for the chip photometry; for
     1175the forced-warp photometry, we calculate the average of the fluxes and
     1176report both average fluxes and the equivalent average magnitudes.
     1177Since the chip photometry requires signal-to-noise of 5 for a
     1178detection, the bias introduced by averaging magnitudes is small.
     1179Since the forced-warp photometry measurements are low signal-to-noise,
     1180with potentially negative flux values, it is necessary to average the
     1181fluxes.
     1182
     1183The first challenge is to select which measurements to use in
     1184the calculation of the average photometry.  For the $3\pi$ Survey
     1185data, a single object may have anywhere from zero to roughly twenty
     1186measurements in a given filter.  Not all measurements are of equal
     1187value, but we need a process which assigns an average photometry value
     1188in all cases (and a way for the user to recognize average values which
     1189should be treated with care).  As discussed in more detail below, we
     1190have defined a triage process to select the ``best'' set of
     1191measurements available in each filter for each object.  Once the set
     1192of measurements to be used in the analysis is determined, we use the
     1193Iteratively Reweighted Least Squares (IRLS) technique to determine the
     1194average photometry given the possible presence of non-Gaussian
     1195outliers even within the best subset of measurements. 
    10951196
    10961197\subsubsection{Selection of Measurements}
    10971198
     1199To choose the measurements which will be used in the analysis, we
     1200give each measurement a rank value based on a variety of tests of the
     1201quality of the measurement, with lower values being better quality.
     1202In the description below
     1203The ranking values are defined as follows:
     1204\begin{itemize}
     1205\item {\bf rank 0 :} perfect measurment (no quality concerns)
     1206\item {\bf rank 1 :} PSF ``perfect pixel'' quality factor (\code{PSF_QF_PERFECT}) $< 0.85$.  \code{PSF_QF_PERFECT} measures the PSF-weighted fraction of pixels which are not masked \citep[see][]{magnier2017.analysis}.
     1207\item {\bf rank 2 :} Photometry analysis flag field (\code{photFlags}) has one of the ``poor quality'' bits raised.  These bits are listed below; OR-ed together they have the hexideciaml value \code{0xe0440130}
     1208\begin{itemize}
     1209  \item {\tt PM\_SOURCE\_MODE\_POOR = 0x00000010} : Fit succeeded, but with low-S/N or high-Chisq
     1210  \item {\tt PM\_SOURCE\_MODE\_PAIR = 0x00000020} : Source fitted with a double psf
     1211  \item {\tt PM\_SOURCE\_MODE\_BLEND = 0x00000100} : Source is a blend with other sources
     1212  \item {\tt PM\_SOURCE\_MODE\_BELOW\_MOMENTS\_SN = 0x00040000} : Moments not measured due to low S/N
     1213  \item {\tt PM\_SOURCE\_MODE\_BLEND\_FIT = 0x00400000} : Source was fitted as a blended object
     1214  \item {\tt PM\_SOURCE\_MODE\_ON\_SPIKE = 0x20000000} : Peak lands on diffraction spike
     1215  \item {\tt PM\_SOURCE\_MODE\_ON\_GHOST = 0x40000000} : Peak lands on ghost or glint
     1216  \item {\tt PM\_SOURCE\_MODE\_OFF\_CHIP = 0x80000000} : peak lands off edge of chip
     1217\end{itemize}
     1218\item {\bf rank 3 :} Poor measurement as defined by relphot.  This may be due to a fixed allowed region on the detector, or due to an outlier clipped analysis.  In the $3\pi$ PV3 calibration, these tests were not applied.
     1219%%       ID_MEAS_POOR_PHOTOM : > 5 sigma outlier, using sigma of 3 sigma inner subset
     1220%%       ID_MEAS_AREA : outside of valid pixel window on chip
     1221%%       neither of these are used for PV3 3pi (POOR is replaced by IRLS;
     1222%%       AREA is replaced by masking)
     1223\item {\bf rank 4 :} PSF quality factor (\code{PSF_QF}) $< 0.85$.
     1224  \code{PSF_QF} measures the PSF-weighted fraction of pixels which are
     1225  not masked as ``bad'', but may be ``suspect''.  Bad values are
     1226  blank, highly non-linear or non-responsibe; suspect pixels include
     1227  those pixels on ghosts, diffraction spikes, bright star bleeds, and
     1228  the mildly-saturated cores of bright stars.  Suspect values may have
     1229  some use in measuring a flux, but with caution
     1230  \citep[see][]{magnier2017.analysis,waters2017}.
     1231\item {\bf rank 5 :} Photometric calibration of the GPC1 exposure is
     1232  determined by relphot to be poor.  This situation occurs if there
     1233  are too few stars available for the calibration ($< 10$ selected
     1234  stars, or if the selected stars account for $< 5\%$ of all stars in
     1235  the exposure).  An exposure may also be identified as poor if the
     1236  zero point is excessively deviant ($> 2$ magnitudes from the nominal
     1237  value) or if the standard deviation of the calibration residuals is
     1238  more than $2\times$ the median standard deviation for all exposures.
     1239%% IMAGE_POOR : ID_IMAGE_PHOTOM_POOR | ID_IMAGE_PHOTOM_FEW | ID_IMAGE_PHOTOM_SKIP
     1240%%   ID_IMAGE_PHOTOM_SKIP : not set?
     1241%%   ID_IMAGE_PHOTOM_FEW : < 10 or (Ngood < 0.05 Nstars)
     1242%%   ID_IMAGE_PHOTOM_POOR : (scatter > MaxScatter) or (Mcal - MedOffset) > MaxOffset     
     1243%%   MaxScatter = MAX (IMAGE_SCATTER, 2*MEDIAN(sigma))
     1244%%   MaxOffset  = MAX (IMAGE_OFFSET, 3*STDEV(Mcal))
     1245%%   IMAGE_OFFSET = 2.0 mag
     1246%%   IMAGE_SCATTER = 0.075 mag
     1247\item {\bf rank 6 :} Photometry analysis flag field (\code{photFlags}) has one of the ``bad quality'' bits raised.  These bits are listed below; OR-ed together they have the hexideciaml value \code{0x1003bc88}
     1248\begin{itemize}
     1249  \item {\tt PM\_SOURCE\_MODE\_FAIL = 0x00000008} : Non-linear fit failed (non-converge, off-edge, run to zero)
     1250  \item {\tt PM\_SOURCE\_MODE\_SATSTAR = 0x00000080} : Source model peak is above saturation
     1251  \item {\tt PM\_SOURCE\_MODE\_BADPSF = 0x00000400} : Failed to get good estimate of object's PSF
     1252  \item {\tt PM\_SOURCE\_MODE\_DEFECT = 0x00000800} : Source is thought to be a defect
     1253  \item {\tt PM\_SOURCE\_MODE\_SATURATED = 0x00001000} : Source is thought to be saturated pixels (bleed trail)
     1254  \item {\tt PM\_SOURCE\_MODE\_CR\_LIMIT = 0x00002000} : Source has crNsigma above limit
     1255  \item {\tt PM\_SOURCE\_MODE\_MOMENTS\_FAILURE = 0x00008000} : Could not measure the moments
     1256  \item {\tt PM\_SOURCE\_MODE\_SKY\_FAILURE = 0x00010000} : Could not measure the local sky
     1257  \item {\tt PM\_SOURCE\_MODE\_SKYVAR\_FAILURE = 0x00020000} : Could not measure the local sky variance
     1258  \item {\tt PM\_SOURCE\_MODE\_SIZE\_SKIPPED = 0x10000000} : Size could not be determined
     1259\end{itemize}
     1260\item {\bf rank 7 :} Measurement is from an invalid time period or
     1261  photometry code.  This rank level is not used in the $3\pi$ PV3
     1262  calibration.  Measurements were not restricted on the basis of the
     1263  time of the observation, and only GPC1 measurements were explicitly
     1264  included.
     1265%% MEAS_BAD = ID_MEAS_NOCAL | ID_MEAS_SKIP_PHOTOM
     1266%%   ID_MEAS_NOCAL : excluded by time range, not a relevant photcode
     1267%%   (only relevant photcodes are considered)
     1268%%   ID_MEAS_SKIP_PHOTOM : not used
     1269\item {\bf rank 8 :} Instrumental magnitude out of range.  This rank level was not used in the $3\pi$ PV3 calibration.
     1270    % (not used, IMAG_MIN, IMAG_MAX = NAN)
     1271\end{itemize}
     1272%% rank 9 : IMAGE_BAD = ID_IMAGE_PHOTOM_NOCAL (not used)
     1273%% rank 10 : measurement out of time range  (not used)
     1274
     1275Rank values are assigned exclusively starting from the highest values:
     1276if a measurements satisfieds the rule for \eg, rank 6, it will not be
     1277tested for ranks 5 and lower.  After all measurements have been
     1278assigned a ranking value, the set of all measurements with the common
     1279lowest value are selected to be used for the average photometry
     1280analysis.  If measurements from ranks 0 through 4 were used for the
     1281average photometry for a given filter, a per-filter mask bit value is
     1282raised identifying which rank was used.  These bit are called
     1283\code{ID_SECF_RANK_0} through \code{ID_SECF_RANK_4} (see
     1284Table~\ref{tab:secf_mask_values}). 
     1285
     1286\begin{table*}
     1287\begin{center}
     1288\footnotesize
     1289\caption{\label{tab:secf_mask_values} Relphot Per-Filter Info Flag Bit Values} % \vspace{-0.5cm}
     1290\begin{tabular}{lcl}
     1291\hline
     1292\hline
     1293{\bf Bit Name} & {\bf Bit Value} & {\bf Description} \\
     1294\hline
     1295ID\_SECF\_STAR\_FEW                & 0x00000001 & Used within relphot: skip star \\
     1296ID\_SECF\_STAR\_POOR               & 0x00000002 & Used within relphot: skip star \\
     1297ID\_SECF\_USE\_SYNTH               & 0x00000004 & Synthetic photometry used in average measurement \\
     1298ID\_SECF\_USE\_UBERCAL             & 0x00000008 & Ubercal photometry used in average measurement \\
     1299ID\_SECF\_HAS\_PS1                 & 0x00000010 & PS1 photometry used in average measurement \\
     1300ID\_SECF\_HAS\_PS1\_STACK          & 0x00000020 & PS1 stack photometry exists \\
     1301ID\_SECF\_HAS\_TYCHO               & 0x00000040 & Tycho photometry used for synth mags \\
     1302ID\_SECF\_FIX\_SYNTH               & 0x00000080 & Synth mags repaired with zpt map \\
     1303ID\_SECF\_RANK\_0                  & 0x00000100 & Average magnitude uses rank 0 values \\
     1304ID\_SECF\_RANK\_1                  & 0x00000200 & Average magnitude uses rank 1 values \\
     1305ID\_SECF\_RANK\_2                  & 0x00000400 & Average magnitude uses rank 2 values \\
     1306ID\_SECF\_RANK\_3                  & 0x00000800 & Average magnitude uses rank 3 values \\
     1307ID\_SECF\_RANK\_4                  & 0x00001000 & Average magnitude uses rank 4 values \\
     1308ID\_SECF\_OBJ\_EXT\_PSPS           & 0x00002000 & In PSPS ID\_SECF\_OBJ\_EXT is saved here so it fits within 16 bits  \\
     1309ID\_SECF\_STACK\_PRIMARY           & 0x00004000 & PS1 stack photometry includes a primary skycell \\
     1310ID\_SECF\_STACK\_BESTDET           & 0x00008000 & PS1 stack best measurement is a detection (not forced) \\
     1311ID\_SECF\_STACK\_PRIMDET           & 0x00010000 & PS1 stack primary measurement is a detection (not forced) \\
     1312ID\_SECF\_STACK\_PRIMARY\_MULTIPLE & 0x00020000 & PS1 stack object has multiple primary measurements \\
     1313ID\_SECF\_HAS\_SDSS                & 0x00100000 & This photcode has SDSS photometry \\
     1314ID\_SECF\_HAS\_HSC                 & 0x00200000 & This photcode has HSC  photometry \\
     1315ID\_SECF\_HAS\_CFH                 & 0x00400000 & This photcode has CFH  photometry (mostly Megacam) \\
     1316ID\_SECF\_HAS\_DES                 & 0x00800000 & This photcode has DES  photometry \\
     1317ID\_SECF\_OBJ\_EXT                 & 0x01000000 & Extended in this band \\
     1318\hline
     1319\end{tabular}
     1320\end{center}
     1321\end{table*}
     1322
     1323\subsubsection{Iteratively Reweighted Least Squares Fitting}
     1324
     1325With an automatic process applied to hundreds of millions of stars, it
     1326is important for the analysis to provide a measurement of the
     1327photometry of each object which is robust against failures.  The
     1328Pan-STARRS\,1 detections have a relatively high rate of non-Gaussian
     1329outliers, partly because of the wide range of instrumental features
     1330affecting the data \citep[see][]{waters2017}.  We have used a
     1331technique called Iteratively Reweighted Least Squares (IRLS) fitting
     1332to reduce the sensitivity of the fits to outlier measurements.  We
     1333have also used bootstrap resampling to determine confidence limits on
     1334our fits given the observed collection of photometry measurements.  In
     1335this case, the analysis is fitting the trivial model that the
     1336photometry measurements are derived from a population with an
     1337underlying constant value.  The discussion below applies to both the
     1338average of the chip photometry magnitudes and the forced-warp
     1339photometry fluxes.
     1340
     1341The IRLS analysis starts with an ordinary least squares fit, using the
     1342weights for each measurement as determined from Poisson statistics.
     1343Since our model is a constant flux, this step is equivalent to
     1344calculating a simple weighted average. 
     1345
     1346Next, the deviations from the average value for each photometry
     1347measurement are calculated.  The deviation, normalized by the Poisson
     1348error, is used to modify the standard weight.  We use a Cauchy
     1349function to define a new weight:
     1350\[
     1351\omega^\prime = \frac{\omega}{1 + r^2}
     1352\]
     1353using
     1354\[
     1355r = \frac{F_o - F_i}{\sigma}
     1356\]
     1357where $F_o$ is the average magnitude (or flux for forced-warp
     1358photometry), $F_i$ is the measured magnitude (or flux), $\sigma$ is
     1359the standard Poisson-based error on the photometry measurement, and
     1360$\omega$ is the ordinary Poisson weight ($\sigma^{-2}$).  This
     1361modified weight has the behavior that if the observed photometry
     1362differs from the model by a substantial amount, the weight is greatly
     1363reduced, while the weight approaches the standard weight if the model
     1364and observed positions agree well.  Thus, this procedure is equivalent
     1365to sigma clipping, but allows the outliers to be reduced in impact in
     1366a continuous way, rather than rigidly accepting or rejecting them.
     1367
     1368The weighted average photometry is re-calculated with these modified
     1369weights.  New values for $\omega$ are calculated, and the weighted
     1370average is calculated again.  On each iteration, the weighted average
     1371photometry values are compared to the values from the previous
     1372iteration.  If they have not changed significantly ($< 10^{-6}$) or if
     1373the fractional change is less than some tolerance ($10^{-4}$), then
     1374iterations are halted and the last weighted average values are used.
     1375If convergence is not reached in 10 iterations, the process is halted
     1376in any case and a flag raised for the object to note that IRLS did not
     1377converge.
     1378
     1379% \note{did this happen for any of our targets?}
     1380
     1381To calculate a fit $\chi^2$ value and to determine an appropriate set
     1382of errors for the model parameters, it is necessary to transform the
     1383modified weights into explicit cuts.  We have used the rubric that if
     1384the modified weight is less than 30\% of the median weight
     1385($\omega^\prime < 0.3 <\omega>$) then the point is treated as clipped.
     1386The $\chi^2$ is determined from the {\em unclipped} points using the
     1387standard Poisson errors.
     1388
     1389Bootstrap-resampling analysis is used to assess the errors on the fit
     1390parameters: A number of measurements equal to the number of {\em
     1391  unclipped} data points are randomly selected from the set of
     1392unclipped data points, with replacement after each selection.  These
     1393data points are then used to calculate the weighted average
     1394photometry.  The average values is recorded and the process re-run 100
     1395times.  The error on the photometry value is determined as half of the
     139668\% confidence range for the distribution of average values.
     1397However, if the number of measurements is small, the
     1398bootstrap-resampled measurement of the error may be artificially
     1399small.  We record the maximum of the bootstrap-sampling error and the
     1400formal error from the weighted average calculation.  The minimumn and
     1401maximum of the unclipped values are also recorded for the chip
     1402photometry.
     1403
     1404% mask values for which wt < threshold (0.3 * median wt)
     1405% we record the min and max values of the unmasked / unclipped subset
     1406% chisq uses only the unmasked
     1407% bootstrap: use only unclipped subset and raw weights to estimate errors
     1408
     1409% \note{bootstrap uses unclipped values and the raw weights? confirmed}
     1410
     1411% \note{reported error is max of bootstrap and formal error?  confirmed}
     1412
    10981413\subsubsection{Stack Photometry}
    10991414
     1415For the stack photometry, the assessment is different from the chip
     1416and forced-warp photometry: multiple measurements are not used to
     1417calculate an average value.  For most of the sky, only a single set of
     1418stack pixels exist for each filter.  Ideally, a unique astronomical
     1419object would only be detected once in a given filter, resulting in
     1420only a single measurement of that object from that filter's stack in
     1421the database.  In practice, objects within a single stack image are
     1422occasionally split by the analysis code, resulting in multiple
     1423detections of the same object.  This situation is discussed in more
     1424detail below. 
     1425
     1426\begin{figure*}[htbp]
     1427  \begin{center}
     1428 \includegraphics[width=\hsize,clip]{{pics/rings.v3.example}.png}
     1429  \caption{\label{fig:rings.v3.example} Illustration of overlapping
     1430    skycells and the identification of the ``primary'' detections.}
     1431  \end{center}
     1432\end{figure*}
     1433
     1434In addition to the these relatively rare failure cases, the objects
     1435detected in the stacks are more likely to have multiple measurements
     1436due to the overlap between neighboring stack images.  The skycells
     1437(within which the stacks are generated) for a given projection cell
     1438are defined to have significant overlap between neighbors to ensure a
     1439modestly-extended object can be measured completely on the pixels in a
     1440single skycell image.  For the \ippmisc{RINGS.V3} skycell tessellation
     1441used for the $3\pi$ PV3 analysis, this overlap was set to be 60
     1442arcseconds, \ie, 240 extra pixels on each edge.  Within
     1443\ippmisc{RINGS.V3}, projection cells themselves are defined to have an
     1444overlap with neighboring projection cells to avoid gaps due to the
     1445process of tiling the spherical sky with a series of flat
     1446projections.  Due to the curved surface of the sky, the amount of
     1447overlap between projection cells increases away from the celestial
     1448equator.  Figure~\ref{fig:rings.v3.example} illustrates both skycell
     1449and projection cell overlaps.
     1450
     1451Overlapping stack regions are not statistically independent.  In the
     1452typical circumstance, the same raw chip images are used to generate
     1453the input warp images for the skycell on either side of the overlap.
     1454Except for rare edge cases (\eg, an input warp which was rejected from
     1455the stack for one side but not the other), exactly the same input raw
     1456chip pixels contribute to all sets of stack pixels which overlap.  It
     1457would therefore be statistically inappropriate to average the multiple
     1458stack measurements from different overlapping skycells.  Instead, we
     1459identify a unique set of stack measurements for the end user.
     1460
     1461We identify two different ways in which an appropriate set of unique
     1462stack measurements can be selected.  In the first case, if multiple
     1463overlapping skycells contribute measurements to an object, we choose
     1464the representative measurement based on their location in the skycell.
     1465This selection is purely a function of the geometry of the skycells
     1466and the coordinate of the object.  We first identify the primary
     1467projection cells, those for which the overlapping regions are closest
     1468to the projection cell center.  For regions in the primary projection
     1469cell, we then identify the primary skycells, those for which the
     1470overlapping regions are closest to the center of the skycell.  For a
     1471given object, the identification of the primary projection cell and
     1472skycell is calculated based on that the coordinates of the object.  We
     1473then find the measurements for the object which came from the primary
     1474projection cell and skycell and identify this set of measurements
     1475(\grizy) as the ``primary'' set.  Note that we use the average
     1476position of the object to define the ``primary'' measurements, forcing
     1477measurements from all filters for the same skycell to be ``primary''
     1478measurements, even if small deviations in the stack positions would
     1479result in one of the filter detections falling on the other side of
     1480the skycell ``primary'' boundary.  Thus, for a given object in the
     1481database, we expect all 5 filters to provide a ``primary'' measurement
     1482from the same skycell for each object.
     1483
     1484Since the ``primary'' identification is purely based on the skycell
     1485geometry and the coordinate of the object, there is no guarantee that
     1486any primary measurement is in fact a good or even the best measurement
     1487of the object.  While the different overlapping pixels should be
     1488essentially identical, it is possible (due to some of the edge cases
     1489mentioned above) that one of the two sets of pixels is more heavily
     1490masked than the other (\eg., more rejected inputs to the stack).
     1491Thus, it is possible that one of the measurements is valid while the
     1492other is not.  To address this possibility, we also identify a set of
     1493``best'' measurements for each object.
     1494
     1495For the stack measurements of an object in a specific filter, if there
     1496are ``primary'' measurements with finite signal-to-noise and PSF
     1497``perfect pixel'' quality factor (\code{PSF_QF_PERFECT}) $> 0.95$, the
     1498measurement with the highest signal-to-noise is marked as ``best''.
     1499If no primary measurement has \code{PSF_QF_PERFECT} $> 0.95$, but a
     1500secondary measurement does, then the secondary measurement with the
     1501highest signal-to-noise is chosen as ``best''.  If neither of the
     1502first two cases hold, but there exist primary measurements with lower
     1503\code{PSF_QF_PERFECT} values, the measurement with the highest
     1504\code{PSF_QF_PERFECT} value is chosen as ``best''.  Finally, if no
     1505``best'' value has yet been identified, the secondary measurement with
     1506the highest value of \code{PSF_QF_PERFECT} is chosen as ``best''.
     1507Note that the above rules allow for multiple measurements of the same
     1508object from the same skycell pixels.  This may occur if the object was
     1509split due to, \eg, saturation or complex morphology.  This type of
     1510split should not be common (and in fact reflects a failure of the
     1511algorithm), but we have defined the rules to allows us to choose an
     1512acceptable measurement even in these cases.
     1513
    11001514\subsubsection{Warp Photometry}
     1515
     1516The calculation of the average forced-warp photometry is performed
     1517very similarly to the average of the chip photometry, with two
     1518important exceptions.  First, as discussed above, the forced-warp {\em
     1519  fluxes} are averaged, rather than the magnitudes.  Second, only the
     1520warp measurements from the skycell which provided the ``best'' stack
     1521measurements are used to calculate the average.  Just as the
     1522overlapping stack pixels are not statistically independent,
     1523overlapping warp pixels from the same exposure are also not
     1524statistically independent. It is critical to use only a single
     1525measurement from each input exposure.  We choose to use those from the
     1526``best'' stack skycell rather than the ``primary'' stack skycell to
     1527ensure the forced-warp photometry represents the highest quality set
     1528of measurements.  Once the measurements from the chosen skycell have
     1529been selected, the same quality cuts are applied to the measurements
     1530as are applied to the chip measurements, as discussed above.
    11011531
    11021532\begin{figure*}[htbp]
    … …  
    11061536    on chip XY04.  In each plot, the solid line shows the measured
    11071537    mean residual for stars detected on this chip as a function of the
    1108     instrumental magnitude / FWHM$^2$.  {\bf top left} X-direction before correction. 
    1109 {\bf top right} Y-direction before correction. 
    1110 {\bf bottom left} X-direction after correction. 
    1111 {\bf bottom right} Y-direction after correction.  }
     1538    instrumental magnitude / FWHM$^2$.  {\bf bottom left} X-direction before correction. 
     1539{\bf bottom right} Y-direction before correction. 
     1540{\bf top left} X-direction after correction. 
     1541{\bf top right} Y-direction after correction.  }
    11121542  \end{center}
    11131543\end{figure*}
    … …  
    11221552    correction.  {\bf bottom right} Y-direction before correction.  {\bf
    11231553      top left} X-direction after correction.  {\bf top right}
    1124     Y-direction after correction.  }
     1554    Y-direction after correction.}
    11251555  \end{center}
    11261556\end{figure}
    … …  
    12081638
    12091639Differential Chromatic Refraction (DCR) affects astrometry because the
    1210 reference stars used the calibrate the images are not the same color
    1211 (SED) as the rest of the stars in the image.  For a given star of a
    1212 color different from the reference stars, as exposures are taken at
    1213 higher airmass, the apparent position of the star will be biased along
    1214 the parallactic angle.  While it is possible to build a model for the
    1215 DCR impact based on the filter response functions and atmospheric
    1216 refraction, we have instead elected to use an empirical correction for
    1217 the DCR present in the PV3 database.  We have measured the DCR trend
    1218 using the astrometric residuals of millions of stars after performing
    1219 an initial relative astrometry calibration.  We define a blue DCR
    1220 color ($g-i$) to be used when correcting the filters \gps,\rps,\ips, and a red
    1221 DCR color ($z - y$) to be used when correcting the filters $zy$.  In
    1222 the process of performing the relative astrometry calibration, we
    1223 record the median red and blue colors of the reference stars used to
    1224 measure the astrometry calibration for each image.  As we determine
    1225 the astrometry parameters for each object in the database, we record
    1226 the median red and blue reference star colors for all images used to
    1227 determine the astrometry for a given object.  For each star in the
    1228 database, we know both the color of the star and the typical color of
    1229 the reference stars used to calibrate the astrometry for that star. 
     1640reference stars used to the calibrate the images are not the same
     1641color (SED) as the rest of the stars in the image.  For a given star
     1642of a color different from the reference stars, as exposures are taken
     1643at higher airmass, the apparent position of the star will be biased
     1644along the parallactic angle.  While it is possible to build a model
     1645for the DCR impact based on the filter response functions and
     1646atmospheric refraction, we have instead elected to use an empirical
     1647correction for the DCR present in the PV3 database.  We have measured
     1648the DCR trend using the astrometric residuals of millions of stars
     1649after performing an initial relative astrometry calibration.  We
     1650define a blue DCR color ($g-i$) to be used when correcting the filters
     1651\gps,\rps,\ips, and a red DCR color ($z - y$) to be used when
     1652correcting the filters $zy$.  In the process of performing the
     1653relative astrometry calibration, we record the median red and blue
     1654colors of the reference stars used to measure the astrometry
     1655calibration for each image.  As we determine the astrometry parameters
     1656for each object in the database, we record the median red and blue
     1657reference star colors for all images used to determine the astrometry
     1658for a given object.  For each star in the database, we know both the
     1659color of the star and the typical color of the reference stars used to
     1660calibrate the astrometry for that star.
    12301661
    12311662We measure the mean deviation of the residuals in the parallactic
    … …  
    12861717features.
    12871718
     1719% http://adsabs.harvard.edu/abs/2008SPIE.7021E..05T
     1720% http://adsabs.harvard.edu/abs/2010SPIE.7733E..0EK
     1721% http://adsabs.harvard.edu/abs/2012SPIE.8453E..0KO
     1722
    12881723The dominant pattern in the astrometric residual is roughly a series
    12891724of concentric rings. The pattern is similar to the pattern of the
    1290 focal surface residuals measured by (REF), which also has a concentric
    1291 series of rings with similar spacing.  The ``tent'' in the center of
    1292 the focal surface reflected in these astrometry residual plots.  Our
    1293 interpretation of the structure is that the deviations of the focal
    1294 plane from the ideal focal surface introduces small-scale PSF changes,
    1295 presumably coupled to the optical aberrations, which result in small
    1296 changes in the centroid of the object relative to the PSF model at
    1297 that location.  Since the PSF model shape parameters are only able to
    1298 vary at the level of a 6x6 grid per chips, the finer structures are
    1299 not included in the PSF model.  The PV2 analysis shows the ring
    1300 structure more clearly, with a pattern much more closely following the
    1301 focal surface deviations.  In the PV2 analysis, the PSF model used at
    1302 most a 3x3 grid per chip to follow the shape variations, so any
    1303 changes caused by the optical aberrations would be less well modeled in
    1304 the PV2 analysis, as we observe.
     1725focal surface residuals measured by \cite{onaka.spie}, which also has
     1726a concentric series of rings with similar spacing.  The ``tent'' in
     1727the center of the focal surface is reflected in these astrometry
     1728residual plots.  Our interpretation of the structure is that the
     1729deviations of the focal plane from the ideal focal surface introduces
     1730small-scale PSF changes, presumably coupled to the optical
     1731aberrations, which result in small changes in the centroid of the
     1732object relative to the PSF model at that location.  Since the PSF
     1733model shape parameters are only able to vary at the level of a 6x6
     1734grid per chips, the finer structures are not included in the PSF
     1735model.  The PV2 analysis shows the ring structure more clearly, with a
     1736pattern much more closely following the focal surface deviations.  In
     1737the PV2 analysis, the PSF model used at most a 3x3 grid per chip to
     1738follow the shape variations, so any changes caused by the optical
     1739aberrations would be less well modeled in the PV2 analysis, as we
     1740observe.
    13051741
    13061742A second pattern which is weakly seen in several chips consists of
    … …  
    13191755{\em not} visible at the resolution of these astrometric flat-field
    13201756images.  Fine structures are observed at the \approx 10 pixel scale
    1321 similar to the ``tree rings'' reported by the DES team and others
    1322 (G. Berstein REF \& REFS).  The details of these tree rings are beyond
    1323 the scope of this article, and will be explored in future work.
     1757similar to the ``tree rings'' reported by the Dark Energy Survey team
     1758\citep{2014PASP..126..750P} and identified as a result of lateral
     1759diffusion of electrons in the detectors due to electric fields due to
     1760dopant variations.  Unlike the photometric tree ring features
     1761discussed above (Section~\ref{sec:phot.flat}), these astrometric tree
     1762rings appear to correspond to the features identified by the DES team.
     1763Lateral electric fields in the detector silicon, caused by variations
     1764in the dopant density, cause the photoelectrons to migrate laterally
     1765in the detector silicon before landing in the pixel wells.  This
     1766migration affects the apparent position of the stars, thus affecting
     1767the observed astrometry.  A simple lateral translation of the
     1768effective pixel locations would not be detected as it would be
     1769degenerate with the astrometric solution.  However, since the lateral
     1770electric fields, and thus the electron migration, vary with position,
     1771the astrometric displacement changes on small scales relative to the
     1772average solution, resulting in residual astrometric structures.  The
     1773gradient of the astrometric displacement results in an apparent
     1774expansion or compression of the pixel sizes, resulting in a signal
     1775which can be observed in the flat-field images.  For GPC1, unlike the
     1776DES detectors, the amplitude of these flat-field variations are much
     1777smaller than the photometric variations caused by the changing PSF
     1778sized, caused in turn by varying electron diffusion rates.  These
     1779features, and the related vertical electron diffusion variations are
     1780discussed in detail in \cite{2018PASP..130f5002M}.
    13241781
    13251782Unfortunately, we discovered a problem with the astrometric flat-field
    … …  
    13281785\ref{fig:astroflat.zy}, there is significant pixel-to-pixel noise in
    13291786the the astrometric flat-field images.  This pixel-to-pixel noise is
    1330 caused by too few stars used in the measuremnt of the flat-field
     1787caused by too few stars used in the measurement of the flat-field
    13311788structure for the high-resolution sampling.  As a result, the
    13321789astrometric flat-field correction reduces systematic structures on
    … …  
    13421799measurements in $i$-band (to reject artifacts detected in a pair of
    13431800exposures from the same night), with \code{PSF_QF} $> 0.85$ (to reject
    1344 excessively-masked objects), and with $mag_{\rm PSF} - mag_{rm Kron} <
    1345 0.1$ (to reject galaxies).  We then generated histograms of the
     1801excessively-masked objects), and with $mag_{\rm PSF} - mag_{\rm Kron}
     1802< 0.1$ (to reject galaxies).  We then generated histograms of the
    13461803difference between the object position predicted for the epoch of each
    13471804measurement (based on the proper motion and parallax fit) and the
    … …  
    13501807given pixel in the images.  From these residual histograms, we can
    13511808then determine the median and the 68\%-ile range to calculate a robust
    1352 standard deviation.  This represents the bright-end systematic error
    1353 floor for a measurement from a single exposure.  The standard
    1354 deviations are then plotted in Figure~\ref{fig:allsky.photom.sigma}.
    1355 The median value of the standard deviations across the sky is
    1356 $(\sigma_\alpha, \sigma_\delta) = (22, 23)$ milliarcseconds.
     1809version of the standard deviation.  This represents the bright-end
     1810systematic error floor for a measurement from a single exposure.  The
     1811standard deviations are then plotted in
     1812Figure~\ref{fig:allsky.photom.sigma}.  The median value of the
     1813standard deviations across the sky is $(\sigma_\alpha, \sigma_\delta)
     1814= (22, 23)$ milliarcseconds.
    13571815
    13581816The Galactic plane is clearly apparently in these images.  Like
    … …  
    13611819errors in both R.A. and DEC.  This may be due to the larger typical
    13621820seeing at these high airmass regions, but without further exploration
    1363 this is interpretation uncertain.  Several features can be seen which
     1821this interpretation is uncertain.  Several features can be seen which
    13641822appear to be an effect of the tie to the Gaia astrometry: the stripes
    13651823near the center of the DEC image and the right side of the R.A. image.
    … …  
    13711829than the \approx 17 mas value in that earlier analysis.  We attribute
    13721830this degradation to the noise introduced by the astrometric
    1373 flat-field.  This noise can likely be addressed before the DR2 release
    1374 of the individual measurement data.
     1831flat-field.
     1832
     1833\note{This noise has been addressed for the DR2 release of the
     1834  individual measurement data.  show updated maps and residuals}
    13751835
    13761836\begin{figure}[htbp]
    … …  
    14751935rotation parameters ($A,B$) = (14.82,-12.37) km sec$^{-1}$ pc$^{-1}$
    14761936and Solar motion parameters ($U_{\rm sol}, V_{\rm sol}, W_{\rm sol}$)
    1477 = (9.32, 11.18, 7.61) km sec$^{-1}$ as determined by Feast \&
    1478 Whitelock (REF) using Hipparchos data.  Proper motions are determined
    1479 from the following:
     1937= (9.32, 11.18, 7.61) km sec$^{-1}$ as determined by
     1938\cite{1997MNRAS.291..683F} using Hipparcos data.  Proper motions are
     1939determined from the following:
    14801940\begin{eqnarray}
    14811941\mu^{\rm gal}_{l} & = & (A \cos (2 l) + B) \cos (b) \\
    … …  
    16242084\subsubsection{Iteratively Reweighted Least Squares Fitting}
    16252085
    1626 After the entire database has been calibrated using the relative
    1627 astrometric analysis, we attempt to determine parallax and proper
    1628 motions for all objects in the database.  We require a minimum of 5
     2086After the image astrometric parameters have been determined and
     2087applied to the measurements from each image, we attempt to find
     2088the best astrometric parameters (position, parallax and proper
     2089motions) for all objects in the database.  We require a minimum of 5
    16292090detections and 1 year of data for any object in order for it to be
    1630 fitted for proper motion.  For a parallax fit, we require at least 7
    1631 detections, 1 year of data, and a parallax factor range of at least
    1632 0.25; no object is fitted to parallax without proper motion as well.
    1633 If an object is fitted for parallax, it is also fitted with a model
    1634 including only proper motion and only a mean position.  The chisq for
    1635 all three fits is saved.  Currently, the highest order fit allowed is
    1636 saved in the database.  The resulting parallax and proper motion
    1637 measurements are inserted back into the DVO database for use by
    1638 science queries.
     2091fitted for just proper motion.  For a parallax and proper-motion fit,
     2092we require at least 7 detections, 1 year of data, and a parallax
     2093factor range of at least 0.25; no object is fitted to parallax without
     2094proper motion as well.  If an object is fitted for parallax, it is
     2095also fitted with a model including only proper motion and only a mean
     2096position.  The chisq for all three fits is saved.  Currently, the
     2097highest order fit allowed is saved in the database, regardless of the
     2098significance of the improvement in adding parameters.  The resulting
     2099parallax and proper motion measurements are inserted back into the DVO
     2100database for use by science queries.
    16392101
    16402102With an automatic process applied to hundreds of millions of stars, it
    … …  
    17462208
    17472209\bibliographystyle{apj}
    1748 %\bibliography{lib}{}
    1749 \input{calibration.bbl}
     2210\bibliography{lib}{}
     2211% \input{calibration.bbl}
    17502212
    17512213\end{document}
Note: See TracChangeset for help on using the changeset viewer.