Index: trunk/doc/release.2015/ps1.calibration/Makefile =================================================================== --- trunk/doc/release.2015/ps1.calibration/Makefile (revision 39892) +++ trunk/doc/release.2015/ps1.calibration/Makefile (revision 39893) @@ -4,5 +4,5 @@ # remember to set \pdfoutput at the top -DO_BIBTEX = 1 +DO_BIBTEX = 0 # remember to change from \bibliography to \input{.bbl} at the bottom @@ -20,4 +20,14 @@ ../inputs/apj.bst \ ../inputs/lib.bib \ +pics/photflat.example.png \ +pics/allsky.photom.sigma.png \ +pics/KHexample.png \ +pics/KHmap.png \ +pics/dcr.r2.g.png \ +pics/astroflat.gri.png \ +pics/astroflat.zy.png \ +pics/allsky.astrom.sigma.png \ +pics/gaia.photom.png \ +pics/gaia.astrom.png \ calibration.tex Index: trunk/doc/release.2015/ps1.calibration/calibration.tex =================================================================== --- trunk/doc/release.2015/ps1.calibration/calibration.tex (revision 39892) +++ trunk/doc/release.2015/ps1.calibration/calibration.tex (revision 39893) @@ -62,4 +62,5 @@ K. W. Hodapp,\altaffilmark{\IfA} R. Jedicke,\altaffilmark{\IfA} +N. Kaiser,\altaffilmark{\IfA} R.-P. Kudritzki,\altaffilmark{\IfA} N. Metcalfe,\altaffilmark{\DUR} @@ -68,5 +69,4 @@ % T. Grav,\altaffilmark{\IfA} % J. N. Heasley,\altaffilmark{\IfA} -% N. Kaiser,\altaffilmark{\IfA} % G. A. Luppino,\altaffilmark{\IfA} % R. H. Lupton,\altaffilmark{\Princeton} @@ -591,4 +591,22 @@ the data from the exposure are loaded into the DVO database. +\section{PV3 DVO Master Database} + +Data from the GPC1 chip images, the stack images, and the warp images +are loaded into DVO using the real-time analysis astrometric +calibration to guide the association of detections into objects. +After the full PV3 DVO database was constructed, including all of the +chip, stack, and warp detections, several external catalogs were +merged into the database. First, the complete 2MASS PSC was loaded +into a stand-alone DVO database, which was then merged into the PV3 +master database. Next the DVO database of synthetic photometry in the +PS1 bands (see Section~\ref{sec:synthdb}) was merged in. Next, the +full Tycho database was added, followed by the AllWISE database. +After the Gaia release in August 2016 \citep{2016AA...595A...2G}, we +generated a DVO database of the Gaia positional and photometric +information and merged that into the master DVO database. + +%% \note{need to describe the assignment of flags, etc, for the external data sources}. + \section{Photometry Calibration} @@ -868,4 +886,17 @@ of responsibility. +\begin{figure*}[htbp] + \begin{center} + \begin{minipage}{0.85\linewidth} + \includegraphics[width=\textwidth,clip]{{pics/photflat.example}.png} + \end{minipage} + \hspace{-3.0in} + \begin{minipage}{0.4\linewidth} + \vspace{3.25in} + \caption{\label{fig:photflat} High-resolution flat-field correction images for the 5 filters $grizy$.} + \end{minipage} + \end{center} +\end{figure*} + The iterations described above (calculate mean magnitudes, calculate zero points, calculate new measurements) are @@ -900,5 +931,5 @@ back to all measurements in the database, updating the mean photometry as part of this process. The calculations for this last step are -performed in parallel on the DVO parition machines. +performed in parallel on the DVO partition machines. With the above software, we are able to perform the entire relphot @@ -911,5 +942,20 @@ analysis. +\begin{figure}[htbp] + \begin{center} + \includegraphics[width=\hsize,clip]{{pics/allsky.photom.sigma}.png} + \caption{\label{fig:allsky.photom.sigma} Consistency of photometry + measurements across the sky. Each panel shows a map of the + standard deviation of photometry residuals for stars in each + pixel. The median value of the measure standard deviations across + the sky is $(\sigma_g, \sigma_r, \sigma_i, \sigma_z, \sigma_y) = + (14, 14, 15, 15, 18)$ millimags. These values reflect the typical + single-measurement errors for bright stars.} + \end{center} +\end{figure} + %% \note{need to discuss the process of setting the final mean magnitudes} + +\subsubsection{Photometric Flat-field} For PV3, the relphot analysis was performed two times. The first @@ -927,8 +973,50 @@ and to set the average magnitudes. +Figure~\ref{fig:photflat} shows the high-resolution photometric +flat-field corrections applied to the measurements in the DVO +database. These flat-fields make low-level corrections of up to +\approx 0.03 magnitudes. Several features of interest are apparent in +these images. + +First, at the center of the camera is an important structure caused by +the telescope optics which we call the ``tent''. In this portion of +the focal plane, the image quality degrades very quickly. The +photometry is systematically biased because the point spread function +model cannot follow the real changes in the PSF shape on these small +scales. As is evident in the image, the effect is such that the flux +measured using a PSF model is systematically low, as expected if the +PSF model is too small. + +The square outline surrounding the ``tent'' is due to the 2$\times$2 +sampling per chip used for the Ubercal flat-field corrections. The +imprint of the Ubercal flat-field is visible throughout this +high-resolution flat-field: in regions where the underlying flat-field +structure follows a smooth gradient across a chip, the Ubercal +flat-field partly corrects the structure, leaving behind a saw-tooth +residual. The high-resolution flat-field corrects the residual +structures well. + +Especially notable in the bluer filters is a pattern of quarter +circles centered on the corners of the chips. These patterns are +similar to the ``tree rings'' reported by the DES team and others +(G. Berstein REF \& REFS). The details of these tree rings are beyond +the scope of this article, and will be explored in future work. +Unlike the tree ring features discussed by these other authors, the +features observed in the GPC1 photometry are not caused by an +interaction of the flat-field with the effective pixel geometry. +Instead, these photometric features are due to low-level changes in +the PSF size which we attribute to variable charge diffusion (Magnier +in prep). + +Other features include some poorly responding cells (e.g., in XY14) +and effects at the edges of chips, possibly where the PSF model fails +to follow the changes in the PSF. + +%% XXX : need to refer to system paper on the central tent? + %% \note{show the flat-field residual images, discuss the features?}. For stacks and warps, the image calibrations were determined after the -relative astrometry was performed on the individual chips. Each stack +relative photometry was performed on the individual chips. Each stack and each warp was tied via relative photometry to the average magnitudes from the chip photometry. In this case, no flat-field @@ -941,5 +1029,41 @@ appropriate for a given warp. This latter effect is one of several which degrade the warp photometry compared to the chip photometry at -the bright end. +the bright end. + +\subsection{Photometry Calibration Quality} + +Figure~\ref{fig:allsky.photom.sigma} shows the standard devitions of +the mean residual photometry for bright stars as a function of +position across the sky. For each pixel in these images, we selected +all objects with (14.5, 14.5, 14.5, 14.0, 13.0) $<$ ($g,r,i,z,y$) $<$ +(17, 17, 17, 16.5, 15.5), with at least 3 measurements in $i$-band (to +reject artifacts detected in a pair of exposures from the same night), +with \code{PSF_QF} $> 0.85$ (to reject excessively-masked objects), +and with $mag_{\rm PSF} - mag_{rm Kron} < 0.1$ (to reject galaxies). +We then generated histograms of the difference between the average +magnitude and the apparent magnitude in an individual image for each +filter for all stars in a given pixel in the images. From these +residual histograms, we can then determine the median and the 68\%-ile +range to calculate a robust standard deviation. This represents the +bright-end systematic error floor for a measurement from a single +exposure. The standard deviations are then plotted in +Figure~\ref{fig:allsky.photom.sigma}. + +The 5 panels in Figure~\ref{fig:allsky.photom.sigma} show several +features. The Galactic bulge is clearly seen in all five filters, +with the impact strongest in the reddest bands. We attribute this to +the effects of crowding and contamination of the photometry by +neighbors. Large-scale, roughly square features \approx 10 degrees on +a side in these images can be attributed to the vagaries of weather: +these patches correspond to the observing chunks. These images +include both photometric and non-photometric exposures. It seems +plausible that the non-photometric images from relatively poor quality +nights elevate the typical errors. On small scales, there are +circular patterns \approx 3 degrees in diameter corresponding to +individual exposures; these represent residual flat-fields structures +not corrected by our stellar flat-fielding. The median of the +standard deviations in the five filters are +$(\sigma_g,\sigma_r,\sigma_i,\sigma_z,\sigma_y) = (14, 14, 15, 15, +18)$ millimagnitudes. %% \note{recommendation} @@ -949,5 +1073,5 @@ \subsubsection{Iteratively Reweighted Least Squares Fitting (1D)} -\subsubsection{Seletion of Measurements} +\subsubsection{Selection of Measurements} \subsubsection{Stack Photometry} @@ -957,9 +1081,6 @@ \begin{figure*}[htbp] \begin{center} - \includegraphics[width=0.48\hsize,clip]{{pics/DXT0.mean}.png} - \includegraphics[width=0.48\hsize,clip]{{pics/DXT1.mean}.png} - \includegraphics[width=0.48\hsize,clip]{{pics/DYT0.mean}.png} - \includegraphics[width=0.48\hsize,clip]{{pics/DYT1.mean}.png} - \caption{\label{fig:KHchip} Illustration of the Koppenh\"ofer Effect + \includegraphics[width=\hsize,clip]{{pics/KHexample}.png} + \caption{\label{fig:KHexample} Illustration of the Koppenh\"ofer Effect on chip XY04. In each plot, the solid line shows the measured mean residual for stars detected on this chip as a function of the @@ -984,23 +1105,5 @@ \end{figure} -\section{PV3 DVO Master Database} - -Data from the GPC1 chip images, the stack images, and the warp images -are loaded into DVO using the real-time analysis astrometric -calibration to guide the association of detections into objects. -After the full PV3 DVO database was constructed, including all of the -chip, stack, and warp detections, several external catalogs were -merged into the database. First, the complete 2MASS PSC was loaded -into a stand-alone DVO database, which was then merged into the PV3 -master database. Next the DVO database of synthetic photometry in -the PS1 bands (see Section~\ref{sec:synthdb}) was merged in. Next, -the full Tycho database was added, followed by the AllWISE database. -After the Gaia release in August 2016, we generated a DVO database of -the Gaia positional and photometric information and merged that into -the master DVO database. - -%% \note{need to describe the assignment of flags, etc, for the external data sources}. - -\section{Astrometry Analysis} +\section{Astrometry Calibration} Once the full PV3 dataset loaded into the master PV3 DVO database, @@ -1081,17 +1184,4 @@ form which can be applied to the database measurements. -\begin{figure}[htbp] - \begin{center} - \includegraphics[width=\hsize,clip]{{pics/pv3.v1.dmag_g.sigma}.png} - \includegraphics[width=\hsize,clip]{{pics/pv3.v1.dmag_r.sigma}.png} - \includegraphics[width=\hsize,clip]{{pics/pv3.v1.dmag_i.sigma}.png} - \includegraphics[width=\hsize,clip]{{pics/pv3.v1.dmag_z.sigma}.png} - \includegraphics[width=\hsize,clip]{{pics/pv3.v1.dmag_y.sigma}.png} - \caption{\label{fig:dmag.measure} Consistency of photometry - measurements across the sky. Each panel shows a map of the - standard deviation of photometry residuals for stars in each pixel.} - \end{center} -\end{figure} - \subsubsection{Differential Chromatic Refraction} @@ -1128,6 +1218,36 @@ We represent the trend with a spline fitted to this dataset. -%% The DCR trend has an amplitude of \note{XXX - XXX} in the five filters. +\begin{figure}[htbp] + \begin{center} + \includegraphics[width=\hsize,clip]{{pics/dcr.r2.g}.png} + \caption{\label{fig:DCRexample} Example of the DCR trend in the + g-band. {\bf top:} DCR trend in the parallactic direction {\bf + bottom:} DCR trend perpendicular to the parallactic angle.} + \end{center} +\end{figure} + +The amplitude of the DCR trend in the five filters is $(g,r,i,z,y) = +(0.010, 0.001, -0.003, -0.017, -0.021)$ arcsec airmass$^{-1}$ +magntiude$^{-1}$. We saturate the DCR correction if the term $color +TAN (\zeta)$ for a given measurement is outside a range where the +DCR correction is well measured. The maximum DCR correction applied +to the five filters is $(g,r,i,z,y) = (0.019, 0.002, 0.003, 0.006, +0.008)$ arcseconds. + %% \note{write down the DCR formalae for reference}. + +\begin{figure*}[htbp] + \begin{center} + \includegraphics[width=0.85\textwidth,clip]{{pics/astroflat.gri}.png} + \caption{\label{fig:astroflat.gri} High-resolution astrometric flat-field correction images for $gri$.} + \end{center} +\end{figure*} + +\begin{figure*}[htbp] + \begin{center} + \includegraphics[width=0.85\textwidth,clip]{{pics/astroflat.zy}.png} + \caption{\label{fig:astroflat.zy} High-resolution astrometric flat-field correction images for $zy$.} + \end{center} +\end{figure*} \subsubsection{Astrometric Flat-field} @@ -1140,7 +1260,8 @@ astrometric flat using a sampling resolution of 40x40 pixels, matching the photometric flat-field correction images. -Figure~\ref{fig:astroflat} shows the astrometric flat-field images for -the five filters \grizy\ in each of the two coordinate directions. -These plots show several types of features. +Figures~\ref{fig:astroflat.gri} and \ref{fig:astroflat.zy} show the +astrometric flat-field images for the five filters \grizy\ in each of +the two coordinate directions. These plots show several types of +features. The dominant pattern in the astrometric residual is roughly a series @@ -1174,9 +1295,77 @@ of this is unclear, but likely caused by the astrometry model failing to follow the underlying variations because of the need to extrapolate -to the edge pixels. Finally, we also identify an interesting effect +to the edge pixels. Finally, we also mention an interesting effect {\em not} visible at the resolution of these astrometric flat-field images. Fine structures are observed at the \approx 10 pixel scale similar to the ``tree rings'' reported by the DES team and others -(G. Berstein REF \& REFS). We explore these tree rings in detail in +(G. Berstein REF \& REFS). The details of these tree rings are beyond +the scope of this article, and will be explored in future work. + +Unfortunately, we discovered a problem with the astrometric flat-field +correction too late to be repaired for DR1. As can be seen by +inspection of Figures~\ref{fig:astroflat.gri} and +\ref{fig:astroflat.zy}, there is significant pixel-to-pixel noise in +the the astrometric flat-field images. This pixel-to-pixel noise is +caused by too few stars used in the measuremnt of the flat-field +structure for the high-resolution sampling. As a result, the +astrometric flat-field correction reduces systematic structures on +large spatial scales, but at the expense of degrading the quality of +an individual measurement. Only $i$-band has sufficient +signal-to-noise per pixel to avoid significantly increasing the +per-measurement position errors. + +Figure~\ref{fig:allsky.astrom.sigma} shows the standard devitions of +the mean residual astrometry in $(\alpha,\delta)$ for bright stars as +a function of position across the sky. For each pixel in these +images, we selected all objects with $15 < i < 17$, with at least 3 +measurements in $i$-band (to reject artifacts detected in a pair of +exposures from the same night), with \code{PSF_QF} $> 0.85$ (to reject +excessively-masked objects), and with $mag_{\rm PSF} - mag_{rm Kron} < +0.1$ (to reject galaxies). We then generated histograms of the +difference between the object position predicted for the epoch of each +measurement (based on the proper motion and parallax fit) and the +observed position of that measurement, in both the Right Ascension and +Declination directions (in linear arcseconds), for all stars in a +given pixel in the images. From these residual histograms, we can +then determine the median and the 68\%-ile range to calculate a robust +standard deviation. This represents the bright-end systematic error +floor for a measurement from a single exposure. The standard +deviations are then plotted in Figure~\ref{fig:allsky.photom.sigma}. +The median value of the standard deviations across the sky is +$(\sigma_\alpha, \sigma_\delta) = (22, 23)$ milliarcseconds. + +The Galactic plane is clearly apparently in these images. Like +photometry, we attribute this to failure of the PSF fitting due to +crowding. The celestial North pole regions have somewhat elevated +errors in both R.A. and DEC. This may be due to the larger typical +seeing at these high airmass regions, but without further exploration +this is interpretation uncertain. Several features can be seen which +appear to be an effect of the tie to the Gaia astrometry: the stripes +near the center of the DEC image and the right side of the R.A. image. +The mesh of circular outlines is due to the outer edge of the focal +plane where the astrometric calibration is poorly determined. As +discussed above, the median values in the images are higher than +expected based on our PV2 analysis of the astrometry: the median +per-measurement error floor of \approx 22 mas is significantly worse +than the \approx 17 mas value in that earlier analysis. We attribute +this degradation to the noise introduced by the astrometric +flat-field. This noise can likely be addressed before the DR2 release +of the individual measurement data. + +\begin{figure}[htbp] + \begin{center} + \includegraphics[width=\hsize,clip]{{pics/allsky.astrom.sigma}.png} + \caption{\label{fig:allsky.astrom.sigma} Consistency of photometry + measurements across the sky. Each panel shows a map of the + standard deviation of astrometry residuals for stars in each + pixel. The median value of the standard deviations across the sky + is $(\sigma_\alpha, \sigma_\delta) = (22, 23)$ milliarcseconds. + These values reflect the typical single-measurement errors for + bright stars. See discussion regarding the astrometric flat which + is likely responsible for these elevated value. } + \end{center} +\end{figure} + +% plot of the astrometric error floor per filter? % \note{SECTION or REF?}. @@ -1295,5 +1484,6 @@ After the full relative astrometry analysis was performed for the PV3 -database, the Gaia Data Release 1 became available. This afforded us +database, the Gaia Data Release 1 became available +\citep{2016A&A...595A...2G, 2016A&A...595A...4L}. This afforded us the opportunity to constrain the astrometry on the basis of the Gaia observations. Gaia DR1 objects which are bright enough to have proper @@ -1320,4 +1510,93 @@ %% \note{Figures showing the Gaia residuals} +\begin{figure*}[htbp] + \begin{center} + \includegraphics[width=\hsize,clip]{{pics/gaia.photom}.png} + \caption{\label{fig:gaia.photom} Comparison with Gaia + photometry. {\bf Left} Mean of PS1 - Gaia, {\bf Right} Standard + deviation of PS1 - Gaia. For pixels with $|b| > 30$ and $\delta > + -30$, the standard deviation of the PS1 - Gaia mean values is 7 + millimagnitudes, while the median of the standard deviations is 12 + millimagnitudes. The former is a statement about the consistency + of the Gaia and Pan-STARRS\,1 photometry, while the latter + reflects the combined bright-end errors for both systems. } + \end{center} +\end{figure*} + +Figure~\ref{fig:gaia.photom} shows a comparison between the Pan-STARRS +photometry in $g,r,i$ and the Gaia photometry in the $G$-band. To +compare the PS1 photometry to the very broadband Gaia G filter, we +have determined a transformation based on a 3rd order polynomial fit +to $g-r$ and $g-i$ colors. This transformation reproduces Gaia +photometry reasonably well for stars which are not too red. For a +comparison, we have seleted all PS1 stars with Gaia measurements +meeting the following criteria: $14 < i < 19$, with at least 10 total +measurements, within a modest color range $0.2 < g - r < 0.9$. We +also restricted to objects with $i_{\rm PSF} - i_{\rm Kron} < 0.1$, +using the average $i$ magnitudes determined from the individual +exposures. + +For Figure~\ref{fig:gaia.photom}, we calculate the difference between +the estimated $G$-band magnitude based on PS1 $g,r,i$ photometry and +the $G$-band photometry reported by Gaia. For each pixel, we +determine the histogram of these differences and calculate the median +and the 68\%-ile range. In Figure~\ref{fig:gaia.photom}, these +values are plotted as a color scale. + +The Galactic plane is clearly poorly matched between the two +photometry systems. This may in part be due to the difficulty of +predicting $G$-band magnitudes for stars which are significantly +extincted: the $G$-band includes significant flux from the PS1 +$z$-band which was not used in our transformation. Many other large +scale feature in the median differences have structures similar to the +Gaia scanning pattern (large arcs and long parallel lines. There are +also structures related to the PS1 exposure footprint. These show up +as a mottling on the \approx 3 degree scale (e.g., lower right below +the Galactic plane). The amplitude of the residual structures is +fairly modest. The standard devition of the median difference values +is 7 millimagnitudes. This number gives an indication of the overall +photometric consistency of both Gaia and PS1 and implies that the +systematic error floor for each survey is less than 7 millimags. + +% set Gr = -0.090 + gr*gi*0.229 + gi*(-0.207+gi*(gi*0.015 - 0.250)) + gr*(0.491+gr*(-0.021*gr - 0.052)) + +%\[ +%G - r = -0.09 + 0.229(g-r)(g-r) + (g-i)(( + +\begin{figure*}[htbp] + \begin{center} + \includegraphics[width=\hsize,clip]{{pics/gaia.astrom}.png} + \caption{\label{fig:gaia.astrom} Comparison with Gaia + astrometry. {\bf Left} Mean of PS1 - Gaia, {\bf Right} Standard + deviation of PS1 - Gaia. The median value of the standard + deviations is $(\sigma_\alpha, \sigma_\delta) = (4, 3)$ + milliarcseconds. } + \end{center} +\end{figure*} + +Figure~\ref{fig:gaia.astrom} shows a comparison between the Pan-STARRS +mean astrometry positions in $\alpha,\delta$ and the Gaia astrometry. +For this comparison, we have seleted all PS1 stars with Gaia +measurements with $14 < i < 19$ and with at least 10 total +measurements. For Figure~\ref{fig:gaia.astrom}, we calculate the +difference between the position predicted by PS1 at the Gaia epoch +(using the proper motion and parallax fit) and the position reported +by Gaia. For each pixel, we determine the histogram of these +differences in the R.A\. and DEC directions, and calculate the median +and the 68\%-ile range. In Figure~\ref{fig:gaia.astrom}, these +values are plotted as a color scale. + +There is good consistency between the PS1 and Gaia astrometry. There +are patterns from the Galactic plane (though not very strongly at the +bulge). There are also clear features due to the PS1 exposure +footprint (ring structure on \approx 3 degree scales). In the plots +of the scatter, there are patterns which are related to the Gaia +scanning rule. These are presumably regions with relatively low +signal to noise in Gaia; they were also apparent in the plots of the +statisics of the per-exposure measurement residuals +(Figure~\ref{fig:allsky.astrom.sigma}. The standard deviations of the +median differences are ($\sigma_\alpha, \sigma_\delta) = (4, 3)$ +milliarcseconds. + \subsection{Calculation of Object Astrometry} @@ -1349,6 +1628,6 @@ \bibliographystyle{apj} -\bibliography{lib}{} -%\input{calibration.bbl} +%\bibliography{lib}{} +\input{calibration.bbl} \end{document} @@ -1372,2 +1651,37 @@ \end{verbatim} +List of Figures and their sources: + +* KH example & map: + * kukui:/data/kukui.3/eugene/pv3.stats.20161202 + * kh.data.20151203.v1/spline.final.fits : spline fits to the KH data + * kh.data.20151203.v1.fits : densify images of residuals per chip : (dX,dY) & (T0, T1) = (pre fix, post fix) + * mana.sh : kh.example - plot of XY04 + * mana.sh : khmap (needs cleanup) + * ipp094:/data/ipp094.0/eugene/pv3.cam.20150607/astrom.corrections : extractions and original scripts to make spline, etc + +* DCR plots: + * need to rebuild density plots (density images used to make splines are poor for plots) + * old examples: + * /data/kukui.3/eugene/dcr.20141205 + * dcr.r2.g.png + * spline fits (DCR.example) + * g : dP/dQ = 0.010, dPmax = 0.019 + * r : dP/dQ = 0.001, dPmax = 0.002 + * i : dP/dQ = -0.003, dPmax = -0.003 + * z : dP/dQ = -0.017, dPmax = -0.006 + * y : dP/dQ = -0.021, dPmax = -0.008 + +* astroflats: + * kukui:/data/kukui.3/eugene/pv3.cam.20150607 + * plots.sh : + * photflat.20151127.fix.fits was made in: + * kukui:/data/kukui.3/eugene/setphot.20151213 + +* Gaia comparisons: + * ipp094:/data/ipp094.0/eugene/pv3.stats.20161022 + * kukui:/data/kukui.3/eugene/pv3.stats.20161022 + +* photom & astrom residuals: + kukui:/data/kukui.3/eugene/pv3.stats.20161202/maps.measure +