Index: trunk/doc/release.2015/systematics.20140411/systematics.tex =================================================================== --- trunk/doc/release.2015/systematics.20140411/systematics.tex (revision 39833) +++ trunk/doc/release.2015/systematics.20140411/systematics.tex (revision 40096) @@ -104,211 +104,125 @@ \section{Pan-STARRS1} -The Pan-STARRS\,1 telescope (PS1) has recently completed observations -for its first survey mission (REF). PS1 is 1.8m telescope located on -the summit of Haleakala on the Hawaiian island of Maui (REF). This -wide-field telescope images a 3.3 degree field of view on a 1.4 -gigapixel camera, with low distortion and generally good image quality -(median \approx \note{1.1 arcseconds} in all filters except for g, -with 1.3 arcseconds). Available filters are \gps,\rps,\ips with -bandpasses similar to the equivalent SDSS filters; \zps is somewhat -bluer than the SDSS $z$, while \yps is somewhat redder (see, REF, for -a complete specification). - -The first PS1 science survey mission began 2010 May and completed 31 -March 2014. The bulk of the observing time (56\%) goes to the -``3$\pi$ Survey'', in which the \approx 30,000 square degrees -observable from Hawaii (north of Dec = -30) were repeatedly observed -over the nearly 4 year survey period. The typical coverage at the end -of the survey is \approx 10 observations per filter per point on the -sky in the 3$\pi$ region. The repeated observations allows for good -characterization of systematic sources of uncertainty. - -The PS1 data will be released to the public in the Spring of 2015 via -the Mikulsky Archive to Space Telescopes (MAST) at the Space Telescope -Science Institute (STScI). - -\subsection{GigaPixel Camera\,1} - -The PS1 GigaPixel Camera\,1 (GPC1) was the largest astronomical camera -until HypersuprimeCam was completed. GPC1 is a mosaic camera -consisting of 60 detectors (``chips'') arranged in an 8x8 grid with -the 4 corners missing (FIGURE?). The detectors in this camera are -othogonal transfer array devices (REF), in which the single silicon -chip is sub-divided into a checkerboard of 64 sub-arrays (``cells''). -Each cell may be independently addressed and read, and the charge -accumulated in the pixels may be moved in either the $x$ or $y$ -directions (thus the term ``orthogonal transfer''). The full GPC1 -thus consists of a total of 3840 cells, each \note{608 x 610} pixels. - -\subsection{Data Processing} - -Images obtained from PS1 are written to a set of computers located in -the observatory dome (``pixel servers''), and notification is then -sent via a web service to the Image Processing Pipeline (IPP) -computers in Kihei. The IPP cluster retrieves images from the summit -pixel servers as they are available, though no summit hand-shaking -takes place. The summit computers have a buffer for data from several -days' of observing so that data is not lost if the link to Kihei is -down or the IPP computer cluster is offline. - -Raw images are saved on the IPP cluster and processed as needed. In -general, every science exposure is processed as soon as possible in a -stream called the ``nightly science processing''. As the survey has -proceeded and improvements have been made in the analysis software, -images have been reprocessed. To date (2015 Jan), there have been 2 -complete reprocesssing runs (identified PV1 and PV2), and a third -(PV3) is in progress. As the software has been improved, the nightly -science processing software has been updated to reflect those -improvements. The nightly science processing thus is heterogenous in -the analysis and algorithms used. The software for the PVx -reprocessing is more homogenous. - -Raw images are stored as multi-extension FITS images, with each cell -in a given chip stored as a separate image extension. The raw data are -generally stored with lossless compression applied to the pixel -arrays \note{CFITSIO REF}. The data for the 60 chips in a given -exposure are stored as separate FITS files. - -The data processing system performs a number of steps distributed -across a cluster of linux-based computers. The first stage (``chip -processing'') is performed on individual chip image files -independently in parallel. At this stage, the images are detrended, -including overscan, bias, dark, flat-field, and (for $y$-band) -fringing corrections. Areas of bad data (e.g., regions of poor charge -transfer efficiency or non-linear dark glows) are masked. Within the -IPP processing code, masked pixels are identified as non-zero values -in a separate 16-bit mask image, which is then carried as part of the -processed image data products. The chip analysis also generates a -variance image which is used along with the signal image to specify -the noise properities as a function of pixel. Details of the -chip-stage processing can be found in \note{Waters et al REF}. - -The chip-stage analysis also includes source detection and basic -characterization. After a sky background model is subtracted from the -image, individual sources are detected via cross-correlation with a -PSF model. Sources detected in the image are used to define the PSF -model. Simple aperture based measurements (total flux, 2nd and -higher-order moments, etc) are measured for all detections. An -estimate of the extendedness (non-PSF-nature) of the sources is used -to distinguish ``stellar'' detections from ``non-stellar'' detections. -Non-stellar detections above a minimum signal-to-noise \note{of 20} -are fitted with a galaxy surface brightness model, while remaining -sources are fitted with a PSF model, or a cluster of PSF models. The -end result of the chip-stage analysis is a collection of parameters -for each detection, including the positions ($X$, $Y$), PSF model -flux, aperture fluxes, 2nd moments. Details of the source analysis -and characterization can be found in \note{Magnier et al REF}. - -\section{Photometry Analysis} - -Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vestibulum -bibendum nisi id tristique posuere. Duis eu mollis nulla. Maecenas est -turpis, mattis tempor urna vitae, placerat rhoncus sem. Lorem ipsum -dolor sit amet, consectetur adipiscing elit. Sed quis velit -nisl. Aliquam erat volutpat. Cras lacinia, nisl tristique auctor -molestie, dolor nulla rhoncus purus, ac accumsan nunc nunc ac -nibh. Maecenas vitae mollis mauris. Ut sollicitudin pulvinar purus, -eget luctus lorem tincidunt vitae. Vestibulum eu mattis neque. Nulla -in tortor id urna dapibus gravida a vel leo. - -% refer back to refcat, ubercal, tonry -% discuss the PSF modeling -% what are the limitations of the current photometric precision - -\section{Astrometry Analysis} - -Astrometric calibration defines a transformation from the raw -instrumental positions (X,Y on a chip) to RA,DEC coordinates on the -sky. Astrometry calibration is performed at two separate stages in -the IPP analysis. First, every exposure is calibrated independently -during the processing by comparison to a reference catalog. This -astrometric analysis is performed as part of the second processing -stage, the ``camera'' analysis stage. The output from the chip stage -for all chips in an exposure are processed as a group in this -analysis. The initial guess for the astrometry comes from the -telescope coordinates provided in the image headers. - -The RA, DEC, and position angle of the telescope boresite are provided -as part of the image metadata. The telescope and camera systems also -provide WCS header keywords for each chip separately. However, in the -early stages of the telescope commissioning, these values were not -reliable. Thus, within the IPP, and model for the camera layout is -used so that the single triplet (RA,DEC,PA) can be used to predict the -astrometric calibration of each chip. This guess is generally good to -10s of pixels, and need only be accurate to \approx 1/2 of a chip for -the software to discover the true coordinates. - -The guess astrometry is used to select a set of likely reference stars -from the reference database. These stars are the cross-correlated -with the measured source positions to find the likely match between -stars in the reference catalog and detected sources in the image. - -Once a reliable match has been determined, astrometric transformations -are determined to match the raw (X,Y) coordinates of the detections to -sky coordinates of the corresponding reference stars. - -Within the IPP, two main classes of transformations may be used. For -single chips from non-mosaic cameras (or for the warp \& stack -skycells), the astrometry calibration consists of a projection from the -celestial sphere to a linearized coordinate system followed by a -transformation from the projection coordiante frame to the pixel -coordinate frame. The projection may be one of several possible -options (see Calabria et al REF), though in general the IPP uses the -SIN (or TAN) projection (define?). The transformation from the -projection coordiates to the pixel coordinate may use an afine -transformation or higher order polynomials (up to 3rd order). - -For chips from a mosaic camera, the astrometric transformation is -defined as a multi-level operation. As in single-chip astrometry, a -projection (SIN or TAN) is used to convert the spherical celestial -coordiates to a local linear system (the 'tanget plane' coorinate -system). There are then two level of cartesian transformations: a -first set of polynomials (up to 3rd order) are used to transform the -tangent plane coordinates to the ``focal plane'' coordinates. This, -relatively low-order, correction accounts for rotation of the camera -and basic optical distortion coming from the optics and the -atmosphere. A second transformation is used to convert the focal -plane coordinate to the pixel coordinates for each chip. For the -on-the-fly calibration, these -transformations may consist of a set of polynomials up -to 3rd order in X and/or Y (i.e, $x^i y^j$ where $i + j <= 3$) - -\note{add details on the transformation from focal-plane to tangent - plane: fit is done on the gradient}. - -The on-the-fly astrometric calibration determines the astrometric -transformation to an accuracy of at least 0.3 arcsec (chips with worse -astrometry are rejected). Data from these images may now be ingested -into our internal database software for astronomical objects, the -Desktop Virtual Observatory \citep[DVO,][]{PS1.IPP}. This database -software associates detections from images together based on their -positions in the sky to define astrnomical ``objects''. The database -includes metadata to describe the links between objects and their -multiple detections as well as metadata defining the images and their -astrometric transformations. - -After images have been ingested into the DVO software, calibration -improvements may be made within the database. A program called -``relastro'' is used to improve the astrometry iterative. In one -mode, relastro loops between improving the mean positions of objects -and using the improved mean positions to improve the astrometric -calibrations of the images. In normal usage, relastro is only allowed -to update the chip-to-focal plane transformations; it is not normally -used to improve the focal-plane to tangent plane transformations. -Also, within relastro, the transformation from chip-to-focal plane may -be represented using either the polynomials as above or with a linear -transformation plus bilinear interpolation of a grid of correction -cells. In the latter case, the grid may be sampled as finely as 6x6 -positions per chip. - -% precision demonstration -% koppenhoefer effect -% MISSING: DCR! - -\section{Systematic Residuals} - -\subsection{Camera-Scale Trends} - -\subsection{Tree-Rings : An Example Chip} +\note{tidy up this section} + +The 1.8m Pan-STARRS\,1 telescope (PS1), located on the summit of +Haleakala on the Hawaiian island of Maui, has been surveying the sky +regularly since May 2010 \citep{chambers.2017}. From May 2010 through +March 2014, PS1 was run under the aegis of the Pan-STARRS Science +Consortium to perform a set of wide-field science surveys; since March +2014, the telescope is operated by the Pan-STARRS New Science +Consortium (PSNSC). Under the PS1SC, the largest survey, both in +terms of area of the sky covered and fraction of observing time +(56\%), was the \TPS\ in which the entire sky north of Declination +$-30$\degrees\ was imaged up \approx 80 times over the 4 years. These +observations were distributed over five filters, \grizy, and have been +astrometrically and photometrically calibrated to good precision +\citep{magnier.2017.calibration}. + +The wide-field PS1 telescope optics \citep{PS1.optics} image a 3.3 +degree field of view on a 1.4 gigapixel camera \citep[GPC1][]{PS1.GPC1}, with +low distortion and generally good image quality. The median seeing +for the \TPS\ data vary somewhat by filter, with (\grizy) = (XXXX). +Routine observations are conducted remotely from the Advanced +Technology Research Center in Kula, the main facility of the +University of Hawaii's Institute for Astronomy operations on Maui. + +GPC1 \citep{PS1.GPCA}, currently the largest astronomical camera in +terms of number of pixels, consists of a mosaic of 60 edge-abutted +$4800\times4800$ pixel detectors, with 10~$\mu$m pixels subtending +0.258~arcsec. These \note{OTA51} detectors, manufactured by Lincoln +Laboratory, are \note{75$\mu$m}-thick back-illuminated CCDs with a +readout time of 7 seconds for a full unbinned image. \note{details + about the voltages?} Initial performance assessments are presented +in \cite{PS1.GPCB}. The active, usable pixels cover $\sim 80$\% of the +FOV. + +\subsection{Data Processing and Calibration} + +Images obtained by PS1 are processed by the Pan-STARRS Image +Processing Pipeline (IPP; \citealp{PS1_IPP, + magnier.etal.2016.datasystem}). All observations are processed +nightly, with results sent to groups within the science consortium +(i.e., PS1SC during the \TPS) performing short-term science projects +(e.g., searching for transient and moving objects). In addition, the +\TPS\ dataset has been re-processed several times with improved +calibration and analysis techniques. To date (2017 July), 3 +re-processings starting from raw pixel data have been performed. The +labels PV0, PV1, PV2, PV3 are used identify the nightly processing and +successive re-processing versions. PV3 has been used for the public +release of the Pan-STARRS \TPS\ data via the {\it Barbara A. Mikulski + Archive for Space Telescopes} (MAST) at the Space Telescope Science +Institute.\footnote{http//panstarrs.stci.edu} + +The data processing and calibration operations are discussed in detail +in elsewhere +\citep{magnier.etal.2017.analysis,magnier.etal.2017.calibration,waters.2017}. +We re-visit here a number of points that are of significance to this +study. Images are processed following a fairly standard sequence of +image detrending, source detection, and initial calibration +(astrometric and photometric) of those detected sources. Additional +standard processing critical to PS1 science operations includes +geometric transformation (`warping') and image combinations (summed +stacks and differences). For the purposes of this analysis, we are +only considering the sources detected in the individual exposures from +the initial analysis steps. + +As discussed in \cite{waters.2017}, image detrending includes +flat-field processing with a single epoch flat-field image for each +filter. The flat-field image used for this analysis has been +generated by median-combining dome flat-field images (after +pre-processing and pixel outlier rejections) and then multiplying by a +photometric flat-field correction image generated by the analysis of a +grid of images of a dense stellar field. The purpose of this second +step is to correct the basic flat-field image for errors arising from +the non-uniformity of the illumination, from non-pixel uniformity due +to the varying optical distorition across the field, and any other +factors which may make the flat-field image inconsistent with stellar +photometry, e.g., SED, filter band-pass variations, etc +\citep[see][]{waters.2017,magnier.cuillandre,magnier.belgium}. This +correction was made on a relatively coarse grid across the focal plane +in order to accumulate sufficient statistics from the stars in the +relatively small number of images available at the time. We have +found that a single flat-field set can be used for all PS1 +observations to yield photometric consistency at the level of \approx +2\% \note{use the ubercal flat stdev as a statistic}. PS1 benefits in +this regard from the stability of having a single instrument which is +rarely removed. + +Photometry of the PS1 images is performed using a +point-spread-function (PSF) model as well as multiple kinds of +apertures \citep{magnier.etal.2017.analysis}. In this analysis, we +refer to aperture photometry performed using an aperture defined based +on the image quality observed for a given chip. The aperture diameter +is set to be \note{XXX} times the FWHM for the image. + +To improve the photometric systematic errors beyond the level achieved +with a single (photometrically corrected) flat-field set, the PS1 +photometry is re-calibrated within the databasing system based on the +properties of the measured photometry. The calibration process is +discussed by \cite{ubercal,photladder,magnier.etal.2017.calibration}. +As part of this process, several flat-field corrections have been +determined. For the PV2 analysis discussed here, a flat-field +correction determined during the ubercal analysis +\citep[see][]{ubercal} consisted of an $8\times 8$ grid of corrections +for each GPC1 chip and filter for each of 4 seasons. The boundaries +of those seasons are \note{tentatively} identified with modifications +to the baffle structures or the system optics. The critical point +here is that the final effective flat-field image for the PV2 dataset +is based on a dome-flat at the highest resolution, with very low +resolution corrections based on photometry, resulting in photometric +calibration with roughly 1 millimag consistency for each measurement +\note{better number from ubercal?}. + +For all objects, positions are measured from the PSF model for the +brighter sources (using a non-linear fitting process) and from a +simple centroid (1st moment) for the fainter source +\citep{magnier.etal.2017.analysis}. These position measurements are +used in the astrometric analysis. The astrometric calibration is +discussed by \cite{magnier.etal.2017.calibration}; for the PV2 +dataset, the typical systematic floor is \approx 15 - 20 +milliarcsecond for individual measurements of brighter stars. + +\section{Tree-Ring-Like Patterns} \begin{table} @@ -331,33 +245,52 @@ \end{table} -We observe a number of low-level effects in different types of -measurements which have a similar spatial structure on individual -chips. These structures have a circular pattern centered one corner -of the affected chips. \note{do all chips show all effects? is the - amplitude very different from chip to chip?} We use measurements -from chip XY40 to illustrate the spatial patterns and relationships -between the different effects. For all effects, we are measuring the -mean value of the effect in 10x10 pixel boxes. The resulting images -are all constructed so that a given superpixel represents the same -range of true GPC1 XY40 pixels. Measurements were extracted from the -``nightly science'' DVO database for observations covering the region -($\alpha$,$\delta$) = (90\degree\ -- 150\degree, -25\degree\ -- -10\degree). This region avoids the Galactic Plane where astrometric -outliers have been more common. We limit the analysis to good -measurements (PSF\_QF $>$ 0.85) of likely stars ($|m_{psf} - m_{aper}| -< 0.2$). Only measurements with instrumental magnitude $< -8.0$ -($-2.5\log \mbox{cts sec}^{-1} < -8.0$) are include to ensure +For many of the GPC1 OTA CCDs, we observe a pattern in the photometric +residuals which is similar in appearence to the Tree Rings described +in the Dark Energy Camera (DECam) by \cite{plazas.2014}. This pattern +consists of systematic deviations which are consistent in a set of +circular arcs centered on the corner of the CCD, as shown in +Figure~\ref{fig:psfmags.by.filter}. The details of the analysis used +to generate Figure~\ref{fig:psfmags.by.filter} are given below. For +now, we note that the GPC1 CCDs are constructed by dividing the +circular silicon wafer into 4 inscribed squares. Thus the corners of +the CCDs lie in the center of the silicon boule, just as the center of +the circular Tree Rings described by \cite{plazas.2014} match the +center of the boule from which they came. This gives the impression +that a similar mechanism is responsible for the pattern observed in +the PS1 photometry and the DECam photometry, namely the diffusive +effects of lateral electric field variations in the detectors. In the +next section, we will make the case that the patterns observed in the +PS1 residuals are {\em not} caused by this mechanism, but are instead +caused by variations in the {\em vertical} electric field (the field +direction perpendicular to the CCD surface). + +First, in this section, we will describe how we have measured the +presence or absence of these tree-ring patterns in 5 types of data. +For all of these examples, we use a single GPC1 CCD (XY40) to +illustrate the effects in detail, but a similar set of effects are +seen in \note{many? most?} GPC1 detectors. First, we show the +residual PSF photometry. Second, we show the residual Aperture +photometry. Third, we show the astrometric residual patterns. +Fourth, we show the patterns observed in the flat-field images. +Finally, we show measurements derived from the second-moments of the +stars. + +For all effects discussed below, we are measuring the mean value of +the effect in 10x10 pixel superpixels across the detector. The +resulting images are all constructed so that a given superpixel +represents the same range of true GPC1 XY40 pixels regardless of the +type of measurement. To generate the photometry, astrometry, or +second-moment measurements were extracted from the \note{PV0} DVO +database for observations covering the region ($\alpha$,$\delta$) = +(90\degree\ -- 150\degree, -25\degree\ -- 10\degree). This region of +the sky provides a fairly high density of stars, but avoids the +Galactic Plane where confusion may potentially contaminate the +measurement. We limit the analysis to good measurements +(\ippmisc{PSF_QF} $>$ 0.85) of likely stars ($|m_{psf} - m_{aper}| < +0.2$). Only measurements with instrumental magnitude $< -8.0$ +($-2.5\log \mbox{cts sec}^{-1} < -8.0$) are included to ensure reasonable signal-to-noise per measurement. We require at least 2 measurements in a given filter and 5 measurements total for any star included in the analysis. - -The following four different measurements show tree-ring structures -(a) photometric residuals, (b) astrometric residuals, (c) a portion of -the flat-field structure, and (d) variations in the second-moment of -stars. In the following section, we show the spatial patterns for -these features and measure their intensity as a function of the -different filters. By comparing the spatial structures, we show that -these effects are directly related. We defer for now discussion of -any causes of the observed effects. \subsubsection{Photometric Residuals} @@ -397,15 +330,15 @@ \end{figure*} -The tree-ring structure is clearly seen in the PSF magnitude -residuals. In this case, we select PSF magnitude measurements for -detections which fall in the given superpixel. We subtract each -measurement from the average magnitude for the object in the selected -filter ($\delta m_{psf} = \overline{m}_{psf} - m_{psf}$) to determine the -residual magnitude, excluding as bad any measurement with $|\delta -m_{psf}| > 0.5$. For a given superpixel, we measure the median of the -$\delta m_{psf}$ distribution. Figure~\ref{fig:psfmags.by.filter} -shows the 2D patterns of $\delta m_{psf}$ for each filter (\grizy). -The dynamic range of the color scale is from -20 to +20 -millimagnitudes for all 5 plots. +Figure~\ref{fig:psfmags.by.filter} shows the 2D patterns of PSF +photometric residuals. In this case, we select PSF magnitude +measurements for detections of stars which fall in the given +superpixel. We subtract each measurement from the average magnitude +for the object in the selected filter ($\delta m_{psf} = +\overline{m}_{psf} - m_{psf}$) to determine the residual magnitude, +excluding as an outlier any measurement with $|\delta m_{psf}| > 0.5$. +For a given superpixel, we measure the median of the $\delta m_{psf}$ +distribution. The figure shows $\delta m_{psf}$ for each filter +(\grizy). The dynamic range of the color scale is from -20 to +20 +millimagnitudes for all 5 plots. The tree-ring pattern is clearly visible for the four blue filters, @@ -418,9 +351,8 @@ is comparable to the amplitude of the correlated structures, so we need to integrate along the radial structures to make stronger -statements about these patterns. - -We have also performed the same measurement for aperture magnitudes, -using the same selections. The 2D patterns for the aperture -magnitudes is shown in Figure~\ref{fig:apmags.by.filter}. The finging +statements about these patterns. \note{hanging statement?} + +Figure~\ref{fig:apmags.by.filter} shows the equivalent measurement for +aperture photometry instead of PSF photometry. The finging pattern again dominates the plot for \yps, but the tree-rings are not seen in any of the filters. A diagonal pattern is visible in \gps @@ -450,27 +382,30 @@ \end{figure*} -The tree-ring structure is also clearly seen in the astrometric -residuals. We use the same selection of measurements for astrometry -as for photometry. In this case, we extract the residual in both the -RA and DEC directions ($\delta RA = \overline{RA} - RA_i$, $\delta DEC -= \overline{DEC} - DEC_i$) and rotate these values to the chip -coordinate system ($\delta X,\delta Y$). We again exclude as bad any -measurement with $|\delta X|$ or $|\delta Y| > 0.5$ arcsec before -measuring the median values for each superpixel. We have determined -the approximate center of the circular tree-ring pattern as (-5,4960) -for this particular chip. Using this coordinate as the center, we -have converted the $\delta X,\delta Y$ offsets into $\delta R,\delta -\theta$ measurements ($\delta R$ : radial component away from the -center, $\delta \theta$ : tangential component). +Figure~\ref{fig:astrom.by.filter} shows a similar type of measurement +for astrometric residuals. To generate this plot, we use the same +selection of measurements for astrometry as for photometry. In this +case, we extract the residual in both the RA and DEC directions +($\delta RA = \overline{RA} - RA_i$, $\delta DEC = \overline{DEC} - +DEC_i$) and rotate these values to the chip coordinate system ($\delta +X,\delta Y$) using our knowledge of the chip orientation on the sky. +We again exclude as bad any measurement with $|\delta X|$ or $|\delta +Y| > 0.5$ arcsec before measuring the median values for each +superpixel. We have determined the approximate center of the circular +tree-ring pattern as (-5,4960) for this particular chip. Using this +coordinate as the center of the pattern, we have converted the $\delta +X,\delta Y$ offsets into $\delta R,\delta \theta$ measurements +($\delta R$ : radial component away from the center, $\delta \theta$ : +tangential component). Figure~\ref{fig:astrom.by.filter} shows the 2D patterns of $\delta R$ for each filter (\grizy). The dynamic range of the color scale is -from -20 to +20 milliarcseconds for all 5 plots. The tree-ring -pattern is visible for all five filters; the finging pattern is not -apparent in the \yps\ astrometry. \note{low-frequency structures? did - that take off fringing?} The per-pixel standard deviations of these -plots is listed in Table~\ref{table:sigmas.by.filter}. The -signal-to-noise of these structures is again somewhat weak, but the -pattern is clearly visible in these figures. +from -20 to +20 milliarcseconds for all 5 plots. A tree-ring-like +pattern is visible for all five filters, with systematic structures +following a circular pattern centered on the chip corner.; the finging +pattern is not apparent in the \yps\ astrometry. The per-pixel +standard deviations of these plots is listed in +Table~\ref{table:sigmas.by.filter}. The signal-to-noise of these +structures is again somewhat weak, but the pattern is clearly visible +in these figures. \subsubsection{Flat-field Structures} @@ -493,13 +428,22 @@ \end{figure*} -The tree-ring structure is also clearly seen in the flat-field -pattern. For this measurement, we have used a set of monochromatic -flat-field images obtained with a tunable laser. The laser is used to -illuminate our flat-field screen which is then observed by the PS1 -telescope. These flat-field images were obtained 2011 Feb 09 as part -of a campaign to study the PS1 system response (Tonry et al REF). -Flats were obtain in a set of 4nm steps, with XXnm band-pass. To -enhance the signal-to-noise, we have combined a set of 6 flats at the -center of the corresponding filter. \note{high-pass filtering}. +Figure~\ref{fig:flats.by.filter} shows the high-spatial-frequency +structures in the flat-field images. For this measurement, we have +used a set of monochromatic flat-field images obtained with a tunable +laser. The laser is used to illuminate our flat-field screen which is +then observed by the PS1 telescope. These flat-field images were +obtained 2011 Feb 09 as part of a campaign to study the PS1 system +response \citep{tonry.phot}. Flats were obtain in a set of 4nm steps, +with \note{XXnm} band-pass. To enhance the signal-to-noise, we have +median-combined a set of 6 flats at the center of the corresponding filter. + +In order to mask pixels which do not flatten well, we generate a +a copy of the image smoothed with a Gaussian kernel with +$\sigma = 1.5 pixels$. Any pixels in the smoothed image which deviate +from the median value in the image by more than 4 standard deviations +is masked. We generate the superpixel image by averaging the unmasked +pixels associated with each superpixel. We then high-pass filter the +superpixel image by subtracting a copy smoothed with a Gaussian of +$\sigma = 3.0$. Figure~\ref{fig:flats.by.filter} shows the remaining high-frequency @@ -513,5 +457,5 @@ measured flux in those pixels, and thus a {\em negative} deviation in $\delta m_{psf}$ as defined above. The dynamic range of the color -scale in these plots is -0.01 to +0.01. The tree-ring pattern is +scale in these plots is -0.01 to +0.01. The tree-ring-like pattern is strong in the (\gps,\rps,\ips) images, but nearly swamped by fringing in \zps, and completely lost to finging in \yps. A diagonal banding @@ -556,19 +500,32 @@ \end{figure*} -The tree-ring structure is also seen in the changes of the image size. -To measure this effect, we extract the second moments for all -detections, subject to the same selections as for the photometry and -astrometry residuals (good stars, multiple detections). The second -moments are measured with a Gaussian weighting function, with the -$\sigma_{w}$ scaled by the PSF size so that the $\sigma$ measured for -PSF stars is \approx 60\% of $\sigma_{w}$. (Note that, since the -measured $\sigma$ of stellar objects is biased down by the weighting -function, this is not quite the same as having $\sigma_{w} = 1.6$ -times the true PSF $\sigma$). For each detection, we measure -$M_{xx,xy,yy} = \sum F_i w_i (x^2, x y, y^2) / \sum F_i w_i$. For -each exposure, we find the mean second moments for PSF objects and +During the image analysis, the second moments are measured for all +stars. The values can be used to assess changes in the shape of stars +on the image. To measure changes in the shapes, we have extracted the +second moments for all stellar detections, subject to the same +selections as for the photometry and astrometry residuals (good stars, +multiple detections). The second moments are measured with a Gaussian +weighting function, with the $\sigma_{w}$ scaled by the PSF size so +that the $\sigma$ measured for PSF stars is \approx 60\% of +$\sigma_{w}$. (Note that, since the measured $\sigma$ of stellar +objects is biased down by the weighting function, this is not quite +the same as having $\sigma_{w} = 1.6$ times the true PSF $\sigma$, see +discussion in \citealt{magnier.etal.2017.analysis}). For each stellar +detection, we extract the values $M_{xx,xy,yy} = \sum F_i w_i (x^2, x +y, y^2) / \sum F_i w_i$. For each exposure, we find the mean second +moments ($\bar{M_{xx,xy,yy}}$) for PSF objects on this chip (XY40) and subtract that mean value from the instantaneous measurements of $M_{xx,xy,yy}$. We then determine the median of the residual second -moments for each superpixel. +moments for each superpixel, resulting in 3 images for each filter. + +\note{write out this math, check out psLibADD} + +Using the second moment images, we can construct certain interesting +combinations, inspired by discussions of lensing measurements \citep{kaiser.1995}: +\begin{eqnarray} +R^2 & = & \delta M_{xx} + \delta M_{yy} \\ +e_1 & = & \delta M_{xx} + \delta M_{yy} \\ +e_2 & = & 2 \delta M_{xy} +\end{eqnarray} Figure~\ref{fig:smear.by.filter} shows the spatial trend of the {\em @@ -576,8 +533,8 @@ \delta M_{yy}$. This value corresponds to the increase or decrease in the circularly-symmetric component of the image size. The dynamic -range of these images is -0.3 to +0.3 pixel$^2$. The tree-ring pattern -is visible for all 5 filters, though \yps is dominated by the fringing -pattern. Structures with relatively low spatial frequencies can also -be seen. +range of these images is -0.3 to +0.3 pixel$^2$. A tree-ring-like +pattern is visible for all 5 filters, though \yps is dominated by the +fringing pattern. Structures with relatively low spatial frequencies +can also be seen. We can also construct a measurement of the change in ellipticity @@ -590,9 +547,9 @@ ellipse orientation as a function of postion. The length of the vectors corresponds to the value of $\sigma^2_{major} - -\sigma^2_{minor}$. The tree-ring structure is not apparent in this +\sigma^2_{minor}$. The tree-ring-like structure is {\em not} apparent in this figure for any filter. The spatial variations are low-frequency and unrelated to the radial trend from the upper-left corner. -\subsubsection{Correlations Between Systematic Trends} +\subsubsection{Correlations Between Tree-Ring-Like Patterns} \begin{table} @@ -614,10 +571,14 @@ \end{table} -As discussed above, the signal-to-noise per pixel in the plots of the -systematic trends is relatively low (\approx 1.0). While the tree -rings are apparent in many of these figures, there are also -some other systematic structures which may degrade the signal -further. To quantatatively compare the tree-ring trends between -filters and between systematic effects, we need to measure the +Tree-ring-like patterns are clearly seen in 4 of the measurement types +above: the PSF photometry, the astrometry, the flat-field, and the +smear terms. As discussed above, the signal-to-noise per pixel in the +plots of the systematic trends is relatively low (\approx 1.0). While +the tree-ring-like patterns are apparent in many of these figures, +there are also some other systematic structures which may degrade the +signal further. + +To quantatatively compare the tree-ring-like trends between +filters and between the types of measurements, we need to measure the tree-ring structure explicitly and filter out the other effects if possible. To do this, we have applied a high-pass filter to all of @@ -630,9 +591,9 @@ the arc to minimize the error associated with the choice of the pattern center and to avoid several bad cells near the bottom of the -chip. \note{draw the arcs?} - -For a given trend, the systematic effect is strongly correlated -between filters. The strongest correlation is the smear term: -Figure~\ref{fig:smear.trends} shows the correlation of the smear +chip. + +For a given type of measurement, the systematic effect is strongly +correlated between filters. The strongest correlation is the smear +term: Figure~\ref{fig:smear.trends} shows the correlation of the smear pattern between \gps\ and the other four filters. Even \yps\ is strongly correlated with \gps\ despite the presence of the fringe @@ -645,12 +606,13 @@ flat-field residuals are generally correlated between filters, but both \zps\ and \yps\ are affected by fringing. For \yps, the -correlation is completely washed out by the very strong fringing pattern. - -For all four measurements, the slope of the fitted lines are listed in -Table~\ref{table:correlation.by.filter}. There is a consistency in -the trend from \gps, with the strongest systematic tree-ring effects -to \yps, with the weakest effects. Note that the second moment smear -and astrometry terms have different relative strength in -\yps\ compared with \gps. +correlation is completely washed out by the very strong fringing +pattern. + +For all four types of measurements, the slope of the fitted lines are +listed in Table~\ref{table:correlation.by.filter}. There is a +consistency in the trend from \gps, with the strongest systematic +tree-ring effects to \yps, with the weakest effects. Note that the +second moment smear and astrometry terms have different relative +strength in \yps\ compared with \gps. % smear trends by filter @@ -694,30 +656,38 @@ \end{figure*} -An important question is the relationship between the different types -of systematic effects. Different models for the tree-ring structures -will make different predictions about the correlations between -different effects. We find the following relationships hold. First, -the PSF magnitude residuals and the second-moment smear trends are -strongly anti-correlated: regions which have larger PSFs than the mean -tend to have smaller measured PSF fluxes than the mean (note that +An important question is the relationship of the tree-ring-like +pattern between the different types of measurements. Different models +for the tree-ring structures make different predictions about the +correlations between different effects. Note the very different +spatial structure between the different measurements in a given +filter: the radial variations do not all follow the same patterns. +Instead, we find the following relationships hold: + +First, the PSF magnitude residuals and the second-moment smear trends +are strongly anti-correlated: regions which have larger PSFs than the +mean tend to have smaller measured PSF fluxes than the mean (note that $\delta m_{psf}$ is defined so that positive values correspond to larger fluxes). These trends are shown in -Figure~\ref{fig:smear.vs.psfmag}. Second, the radial derivative of -the smear is anti-correlated with the radial component of the -astrometric residuals: $\frac{\partial (\sigma^2_{major} + - \sigma^2_{minor})}{\partial radius} \sim \delta R$ (see -Figure~\ref{fig:dsmear.vs.astrom}. Finally, the radial derivative of -the radial component of the astrometric residual is anti-correlated -with the flat-field residual errors: $\frac{\partial \delta - R}{\partial radius} \sim \delta flat$ (see -Figure~\ref{fig:dastrom.vs.flat}. This last relationship is somewhat -weakly measured. Because of the periodic nature of the tree rings, it -is also difficult to completely certain that the flat-field is -proportional to the derivative of the astrometry residual, and not the -other way around. The correlation is somewhat weaker for derivative -of the flat-field vs astrometry residual. The correlation is very -weak between the flat-field and the astrometry directly. We are -convinced that we have the sense of the derivative correct by the -details of specific features. +Figure~\ref{fig:smear.vs.psfmag}. + +Second, the radial derivative of the smear is anti-correlated with the +radial component of the astrometric residuals: $\frac{\partial + (\sigma^2_{major} + \sigma^2_{minor})}{\partial radius} \sim \delta +R$ (see Figure~\ref{fig:dsmear.vs.astrom}. + +Finally, the radial derivative of the radial component of the +astrometric residual is anti-correlated with the flat-field residual +errors: $\frac{\partial \delta R}{\partial radius} \sim \delta flat$ +(see Figure~\ref{fig:dastrom.vs.flat}. This last relationship is +somewhat weakly measured. Because of the periodic nature of the tree +rings, it is also difficult to be completely certain that the +flat-field is proportional to the derivative of the astrometry +residual, rather than the astrometry residual being proportional to +the derivative of the flat-field. The correlation is somewhat weaker +for derivative of the flat-field vs astrometry residual. The +correlation is very weak between the flat-field and the astrometry +residual values without a derivative. We are convinced that we have +the sense of the derivative correct by examination of specific +features in each imaage (e.g., \note{give example}). \begin{table} @@ -772,33 +742,47 @@ These trends help to illuminate the underlying causes of these -different effects. First, we can easily explain the relationship -between the PSF photometry residuals and the observed smear. In the -photometry analysis, we model the PSF allowing for some spatial -variation in the shape. However, we limit the 2D variation to a 3x3 -grid for each chip: the PSF parameters may vary smoothly across the -chip following the bilinear interpolation between the 3x3 grid points. +different effects. + +\note{summarize what pure lateral electric fields would do} + +First, if we consider the smear pattern +(Figure~\ref{fig:smear.by.filter}), the measurement shows that the +intrinsic size of the stellar images is varying in a radial sense +between the different tree-ring regions. Although images experience +an average image quality (due to seeing and focus) across the chip +which may vary substantially from exposure to exposure, stars landing +in the different tree-ring-like regions are consistently somewhat +larger or somewhat smaller than that average. + +Next, we can explain the relationship between the PSF photometry +residuals and the observed smear: In the photometry analysis, we model +the PSF, allowing for some spatial variation in the shape. However, +we have a limited number of stars to measure any spatial variation. +Thus the 2D variation are sampled on a very coarse (e.g., 3x3) grid +for each chip: the PSF parameters may vary smoothly across the chip +following the bilinear interpolation between the 3x3 grid points. Thus, the spatial scale on which we model PSF variations is much larger than the spatial scale on which PSF variations are apparently -occuring. When the true PSF is larger than the model PSF, our model -fits systematically underestimate the amount of flux in a given -object. Conversely, when the PSF is smaller, we overestimate the -flux. The slope of the trend depends on the mean typical seeing for -the given filter. For example, the \gps\ seeing is typically -1.3\arcsec, corresponding to a Gaussian $\sigma$ of 2.15 pixels. A -smearing of $\sigma^2_{major} + \sigma^2_{minor} = 0.1$ pixels$^2$ -would increase the size by about 0.02 pixels, or 1\%, so roughly -consistent with the observed photometric deviation of about 5 to 10 -millimags for this amount of smearing. \note{model the 2D effect more - explicitly}. - -Second, the relationship between the flat-field residual and the -astrometric gradient is consistent with radial variations in the -plate-scale. The tree-rings observed by DES are completely attributed -to effective plate scale changes. Effective plate scale changes would -result in flat-field deviations since the flat-field illumination is a -source of constant surface brightness. Pixels see a varying amount of -flux depending on their effective area. This changing plate scale -also affects the astrometry since these variations occur on spatial -scales much smaller than the astrometric model. In such a model, the +occuring, as illustrated by the changes in the smear plot. When the +true PSF is larger than the model PSF, our model fits systematically +underestimate the amount of flux in a given object. Conversely, when +the PSF is smaller, we overestimate the flux -- this type of offset is +a typical effect when mis-estimating the PSF size. The slope of the +trend depends on the mean typical seeing for the given filter. For +example, the \gps\ seeing is typically 1.3\arcsec, corresponding to a +Gaussian $\sigma$ of 2.15 pixels. A smearing of $\sigma^2_{major} + +\sigma^2_{minor} = 0.1$ pixels$^2$ would increase the size by about +0.02 pixels, or 1\%, roughly consistent with the observed photometric +deviation of about 5 to 10 millimags for this amount of smearing. + +The relationship between the flat-field residual and the astrometric +gradient is consistent with radial variations in the plate-scale. The +tree-rings observed by DES are completely attributed to effective +plate scale changes. Effective plate scale changes would result in +flat-field deviations since the flat-field illumination is a source of +constant surface brightness. Pixels see a varying amount of flux +depending on their effective area. This changing plate scale also +affects the astrometry since these variations occur on spatial scales +much smaller than the astrometric model. In such a model, the flat-field deviations are $-1 \times \frac{\partial Pos}{\partial R}$. The slope of our relationship is \approx 0.5 in normalized units. @@ -806,4 +790,8 @@ 2. \note{looks like a slope of 1.0 would not be excluded by these plots} + +\note{I need to use the relationship between the astrometry and the + flat-field to calculate the amplitude of the lateral electric + fields.} The fact that the PSF ellipticity changes are {\em not} correlated @@ -841,2 +829,45 @@ \end{document} + +Notes for paper re-work: + +* Paper focus is now only on the diffusion variations + * strip out the discussion of other systematic effects + * strip down the PS1 introduction discussion + +* tentative title: + Evidence for Small-Scale Charge-Diffusion Variations in Pan-STARRS CCDs + +* outline + + 1. introduction + * thick CCDs + * tree rings == transverse field effects (see Plazas et al) + * we see something else + + 4 model : diffusion variations due to E|| field variations + + 5 discussion (how to treat in calibration / analysis) + + 6 conclusions + +some possible refs to tree rings / charge diff: + +* http://adsabs.harvard.edu/abs/2016SPIE.9904E..2CW (Woods et al 2016; TESS) +* https://arxiv.org/pdf/1605.01001.pdf : plazas et al +* http://ieeexplore.ieee.org/document/1225293/?part=1 Altmannshofer et al 2003 (about thick Si) + +* plazas et al 2014 outline + + 1. intro: thick CCDs, transverse electric fields + 2. DES / DECam + + 2.1 flat-field tree rings (discussion of flat-field tree rings + starting from the premise that they know the answer). + + 3 impact on astrometry and photometry + + 4 improving calibrations given tree rings + + 5 summary and conclusions +