IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Ignore:
Timestamp:
Jun 22, 2017, 3:35:25 PM (9 years ago)
Author:
eugene
Message:

update references, clean up some of the \notes

File:
1 edited

Legend:

Unmodified
Added
Removed
  • trunk/doc/release.2015/ps1.datasystem/datasystem.tex

    r40032 r40065  
    9595The 1.8m Pan-STARRS\,1 telescope is located on the summit of Haleakala
    9696on the Hawaiian island of Maui.  The wide-field optical design of the
    97 telescope \citep{PS1.optics} produces a 3.3 degree field of view with
     97telescope \citep{2004SPIE.5489..667H} produces a 3.3 degree field of view with
    9898low distortion and minimal vignetting even at the edges of the
    9999illuminated region.  The optics and natural seeing combine to yield
     
    102102a floor of $\sim 0.7$ arcseconds.
    103103
    104 The \PSONE\ camera \citep{PS1.GPCA}, known as GPC1, consists of a
     104The \PSONE\ camera \citep{2009amos.confE..40T}, known as GPC1, consists of a
    105105mosaic of 60 back-illuminated CCDs manufactured by Lincoln Laboratory.
    106106The CCDs each consist of an $8\times8$ grid of $\sim 600\times 600$
    107107pixel readout regions, yielding an effective $4800\times4800$
    108108detector.  Initial performance assessments are presented in
    109 \cite{PS1.GPCB}.  Routine observations are conducted remotely from the
     109\cite{2008SPIE.7014E..0DO}.  Routine observations are conducted remotely from the
    110110Advanced Technology Research Center in Kula, the main facility of the
    111111University of Hawaii's Institute for Astronomy operations on Maui.
     
    123123search for potentially hazardous asteroids in our solar system.  The
    124124details of the telescope, surveys, and resulting science publications
    125 are described by \cite{Chambers}.
     125are described by \cite{chambers2017}.
    126126
    127127This is the second in a series of seven papers describing the
     
    153153%Magnier et al. 2017 (Paper II)
    154154%Pan-STARRS Data Processing Stages
    155 %\citet[][Paper II]{magnier2017c}
     155%\citet[][Paper II]{magnier2017.datasystem}
    156156%describes how the various data processing stages are organised and implemented
    157157%in the Imaging Processing Pipeline (IPP), including details of the
     
    166166%Magnier et al. 2017 (Paper IV)
    167167%Pan-STARRS Pixel Analysis : Source Detection
    168 \citet[][Paper IV]{magnier2017a}
     168\citet[][Paper IV]{magnier2017.analysis}
    169169describes the details of the source detection and photometry, including point-spread-function and extended source fitting models, and the techniques for ``forced" photometry measurements.
    170170
    171171%Magnier et al. 2017 (Paper V)
    172172%Pan-STARRS Photometric and Astrometric Calibration
    173 \citet[][Paper V]{magnier2017b}
     173\citet[][Paper V]{magnier2017.calibration}
    174174describes the final calibration process, and the resulting photometric and astrometric quality. 
    175175
     
    190190detail each of the analysis steps which may be applied to the images
    191191and resulting catalogs of detected sources.
    192 Section~\ref{sec:postprocessing} discusses the calibration operations
    193 and database used for calibration.  Section~\ref{sec:operations}
    194 discusses the operational infrastructure of the IPP.
    195 Section~\ref{sec:hardware} discusses the hardware systems used by the
    196 IPP for regular nightly operations and for processing the PV3 data
    197 release, with some details on the scale of computing needed to reduce
    198 this large number of exposures.  Finally, Section~\ref{sec:discussion}
    199 presents a discussion of some of the lessons learned in the creation
    200 of the IPP, and its utility in reducing data from other cameras and
    201 telescopes.
     192Section~\ref{sec:postprocessing} discusses the databasing system used
     193for calibration, the calibration operations, and summarizes the
     194construction of the public release database.
     195Section~\ref{sec:operations} discusses the operational infrastructure
     196of the IPP.  Section~\ref{sec:hardware} discusses the hardware systems
     197used by the IPP for regular nightly operations and for processing the
     198PV3 data release, with some details on the scale of computing needed
     199to reduce this large number of exposures.  Finally,
     200Section~\ref{sec:discussion} presents a discussion of some of the
     201lessons learned in the creation of the IPP, and its utility in
     202reducing data from other cameras and telescopes.
    202203
    203204{\color{red} {\em Note: These papers are being placed on arXiv.org to
     
    426427glitches or hardware crashes.
    427428
    428 % \note{start of section needed a re-read}
    429 
    430429\subsection{Summit copy}
    431430\label{sec:summitcopy}
     
    470469is ready to be registered.  In this context, `registration' refers to
    471470the process of adding them to the database listing of known, raw
    472 exposures (not to be confused with 'registration' in the sense of
     471exposures (not to be confused with `registration' in the sense of
    473472pixel re-alignment).  The result of the registration analysis is an
    474473entry for each exposure in the \ippdbtable{rawExp} table, and one for
     
    525524(with the \ippdbcolumn{state} column indicating it needs processing),
    526525and the associated information listed in the \ippdbtable{rawImfile},
    527 jobs can be spawned for each component OTA.  The \ippprog{pantasks}
    528 environment managing the jobs attempts to target the processing host
    529 to one that should host the OTA, to reduce number of operations done
    530 on remote data.  In practice, this targeted processing has not had as
    531 large of an effect as was originally intended, as the data volume has
     526jobs can be spawned for each component OTA. 
     527
     528The \ippstage{chip} stage is naturally parallelized by processing data
     529from each of the 60 OTAs independently.  Several stages in the IPP
     530analysis are parallelized in a similar fashion; although there are
     531multiple stages that operate on an entire exposure at once, the
     532majority of stages operate on smaller segments of a full exposure,
     533allowing the processing tasks to be spread over the machines in the
     534processing cluster.  The \ippprog{pantasks} environment, which manages
     535the jobs, attempts to target the processing to a computer which is
     536assigned to host data for the particular OTA.  This capability is
     537implemented to reduce the network I/O load by minimizing the number of
     538operations done on non-local data.  In practice, this targeted
     539processing has not had as large of an impact as was originally
     540intended: the data volume and operational details of the hardware has
    532541reduced the ability of any one node to reliably contain a particular
    533542OTA.  The targeted processing has probably reduced the network load
    534543somewhat but it has not been as critical of a requirement as
    535544originally expected.
    536 
    537 \note{keep this paragraph?}
    538 
    539 Part of this parallelization is derived from the fact that this camera
    540 consists of 60 independent orthogonal transfer array (OTA) devices,
    541 and can therefore be processed simultaneously.  Although there are
    542 multiple stages that operate on an entire exposure at once, the
    543 majority of stages operate only on smaller segments of a full exposure
    544 to allow the processing tasks to be spread over the machines in the
    545 processing cluster.
    546545
    547546%% In the \ippstage{chip} stage,
     
    581580this analysis, removing the need to write out and re-read the image
    582581data.  The details of the detection and characterization of the
    583 sources in the image are provided in \citet{magnier2017b}. 
     582sources in the image are provided in \citet{magnier2017.analysis}. 
    584583
    585584The results of the image processing are then written to disk,
     
    657656used to generate synthetic w-band photometry for areas where no
    658657PS1-based calibrated w-band photometry is available.  For more
    659 details, see \cite{magnier2017c}.  The result of these calibrations is
     658details, see \cite{magnier2017.calibration}.  The result of these calibrations is
    660659stored as a single multi-extension FITS table containing the results
    661660from each OTA as a separate extension.
     
    718717pixels.  These projections are further broken down into ``skycells''
    719718that form a $10\times{}10$ grid within the projection, with an overlap
    720 region of 60" between adjacent skycells to ensure that objects are not
     719region of 60\arcsec\ between adjacent skycells to ensure that objects are not
    721720split on all images.
    722721
     
    778777For the PV3 processing of the Medium Deep fields, stacks have been
    779778generated for the nightly groups and for the full depth using all
    780 exposures, producing ``deep stacks''.  In addition, a 'best seeing'
     779exposures, producing ``deep stacks''.  In addition, a `best seeing'
    781780set of stacks have been produced \note{using image quality cuts to be
    782781  described}.  We have also generated out-of-season stacks for the
     
    816815\ippstage{stack} processing.  As this completes all processing for the
    817816entry, no \ippmisc{advance} job is required.
    818 
    819 % \note{end of section needed a re-read}
    820817
    821818\subsection{Stack Photometry}
     
    839836The input images are passed to the \ippprog{psphotStack} program,
    840837which does the analysis.  The stack photometry algorithms are
    841 described in detail in \cite{magnier2017b}.  In short, sources are
     838described in detail in \cite{magnier2017.analysis}.  In short, sources are
    842839detected in all 5 filter images down to the $5\sigma$ significance.
    843840The collection of detected sources is merged into a single master
     
    859856sources based on the PSF model; aperture like parameters such as the
    860857Petrosian flux and radius; the convolved galaxy model fits; and the
    861 radial aperture measurements.  \note{is this list complete?}  Once the
    862 photometry is complete, a row is added to the
    863 \ippdbtable{staticskyResult} table with basic statistics from the
    864 analysis.
     858radial aperture measurements.  Once the photometry is complete, a row
     859is added to the \ippdbtable{staticskyResult} table with basic
     860statistics from the analysis.
    865861
    866862The stack photometry output catalogs are re-calibrated for both
     
    877873for the \ippstage{camera} and \ippstage{stack} calibration stages.
    878874Upon completion, the analysis statistics are written to the
    879 \ippdbtable{skycalResult} table. \note{Any difference in output formats?}
     875\ippdbtable{skycalResult} table.
    880876
    881877\subsection{Forced Warp Photometry}
     
    951947
    952948In this program, the positions of sources are loaded from the input
    953 catalog.  PSF stars are pre-identified \note{how?} and a PSF model
    954 generated for each \ippstage{warp} image based on those stars, using
    955 the same stars for all warps to the extent possible (PSF stars which
    956 are excessively masked on a particular image are not used to model the
    957 PSF).  The PSF model is fitted to all of the known source positions in
    958 the warp images.  Aperture magnitudes, Kron magnitudes, and moments
    959 are also measured at this stage for each warp.  Note that the flux
    960 measurement for a faint, but significant, source from the stack image
    961 may be at a low significance (less than the $5\sigma$ criterion used
    962 when the photometry is not run in this forced mode) in any individual
    963 warp image; the flux may even be negative for specific warps.  When
    964 combined together, these low-significance measurements will result in
    965 a signficant measurement as the signal-to-noise increases by the
    966 square root of the number of measurements.  \note{The individual warp
    967 measurements are combined together to generate averages values within
    968 DVO.}
     949catalog.  PSF stars are pre-identified from the stack image and a PSF
     950model generated for each \ippstage{warp} image based on those stars,
     951using the same stars for all warps to the extent possible (PSF stars
     952which are excessively masked on a particular image are not used to
     953model the PSF).  The PSF model is fitted to all of the known source
     954positions in the warp images.  Aperture magnitudes, Kron magnitudes,
     955and moments are also measured at this stage for each warp.  Note that
     956the flux measurement for a faint, but significant, source from the
     957stack image may be at a low significance (less than the $5\sigma$
     958criterion used when the photometry is not run in this forced mode) in
     959any individual warp image; the flux may even be negative for specific
     960warps.  When combined together, these low-significance measurements
     961will result in a signficant measurement as the signal-to-noise
     962increases by the square root of the number of measurements.  The
     963individual warp measurements are combined together to generate
     964averages values within DVO.
    969965
    970966Upon completion of the forced photometry (for point sources as well as
     
    976972analysis measurements into a single value.  The output catalogs listed
    977973in the \ippdbtable{fullForceResult} table are passed to the
    978 \ippprog{psphotFullForceSummary} to do this averaging.  \note{describe
    979   what is done} When this completes, an entry is added to the
    980 \ippdbtable{fullForceSummary}, and the \ippdbtable{fullForceRun} entry
    981 is marked as completed.
     974\ippprog{psphotFullForceSummary} to do this averaging.  When this
     975completes, an entry is added to the \ippdbtable{fullForceSummary}, and
     976the \ippdbtable{fullForceRun} entry is marked as completed.
    982977
    983978\subsubsection{Forced Galaxy Models}
    984 \note{CZW: is this the appropriate place for this section?}
    985979\note{too much detail in this section; balance relative to psphot}
    986980
     
    10201014$\chi^2$ grid can then be made by combining each grid point across the
    10211015inputs.  The combined $\chi^2$ for a single grid point is simply the
    1022 sum of all $\chi^2$ values at that point.  If, for a single \ippstage{warp}
    1023 image, the galaxy model is excessively masked, then that image will be
    1024 dropped for all grid points for that galaxy.  The reduced $\chi^2$
    1025 values can be determined by tracking the total number of pixels
    1026 used across all inputs to generate the combined $\chi^2$ values.  From
    1027 the combined grid of $\chi^2$ values, the point in the grid with the
    1028 minimum $\chi^2$ is found.  Quadratic interpolation is used to
    1029 determine the major, minor axis values for the interpolated minimum
    1030 $\chi^2$ value.  The errors on these two parameters is then found by
    1031 determining the contour at which the \note{reduced?} $\chi^2$
    1032 increases by 1.
     1016sum of all $\chi^2$ values at that point.  If, for a single
     1017\ippstage{warp} image, the galaxy model is excessively masked, then
     1018that image will be dropped for all grid points for that galaxy.  The
     1019reduced $\chi^2$ values can be determined by tracking the total number
     1020of pixels used across all inputs to generate the combined $\chi^2$
     1021values.  From the combined grid of $\chi^2$ values, the point in the
     1022grid with the minimum $\chi^2$ is found.  Quadratic interpolation is
     1023used to determine the major, minor axis values for the interpolated
     1024minimum $\chi^2$ value.  The errors on these two parameters is then
     1025found by determining the contour at which the $\chi^2$ increases by 1.
    10331026
    10341027Thus the \ippstage{fullforce} galaxy analysis uses the PSF information
    10351028from each \ippstage{warp} to determine a best set of convovled galaxy
    10361029models for each object in the \ippstage{skycal} catalog.
     1030
    10371031\note{discuss the subset of galaxy models and objects}.
    10381032
     
    11101104\begin{table}[hb]
    11111105\begin{center}
    1112 \caption{DVO Database Tables\label{tab:DVOtables}}
     1106\caption{DVO Database Tables\label{tab:DVO_schema}}
    11131107\begin{tabular}{ll}
    11141108\hline
     
    12381232processed by the IPP may also be included similarly in a DVO database.
    12391233Measurements from other sources, such as SDSS, 2MASS, or WISE, can
    1240 also be included in this table (see \S\ref{sec:other.photometry}.
     1234also be included in this table.
    12411235
    12421236The \ippdbtable{Measure} table includes the instrumental magnitudes
     
    12781272
    12791273\subsubsubsection{Object Tables}
     1274\label{sec:object}
    12801275
    12811276% object -> detection
     
    13531348these photometric distance modulus measurements are not extremely
    13541349precise (see below), they provide a constraint on the distance is used
    1355 in our analysis of the astrometry \citep[][see]{magnier2017a}.
     1350in our analysis of the astrometry \citep[][see]{magnier2017.calibration}.
    13561351
    13571352In the \ippdbtable{Measure} table, there are three fields which
     
    14101405determined by the photometry calibration analysis and the astrometric
    14111406flat-field corrections determined by the astrometry calibration
    1412 analysis \citep[][see]{magnier2017a}.
     1407analysis \citep[][see]{magnier2017.calibration}.
    14131408
    14141409\subsubsection{Sky Partition}
     
    14571452machines that contain partition data.
    14581453
    1459 \note{is the use of the term 'partition host' consistent in this paper
     1454\note{is the use of the term `partition host' consistent in this paper
    14601455  and the calibration paper?}
    14611456
     
    14991494The FITS binary table compression scheme uses a strategy similar to
    15001495that used for FITS image compression (\note{REF}).  The binary tabular
    1501 data is compressed and stored in the 'HEAP' section of the FITS table
     1496data is compressed and stored in the `HEAP' section of the FITS table
    15021497extension, with pointers to the compressed data stored in the regular
    15031498data section.  Each column in the FITS table is compressed as one (or
     
    15051500column format (e.g., TFORM1) are replaced with keywords which describe
    15061501the location and size of the compressed data in the HEAP section; the
    1507 information about the uncompressed data is moved to a keyword with 'Z'
     1502information about the uncompressed data is moved to a keyword with `Z'
    15081503prepended (e.g., ZFORM1) and an additional field is added to define
    15091504the compression algorithm (e.g., ZCTYP1).  The column names (e.g.,
     
    15941589astrometric and photometric calibrations can be calculated.  The
    15951590details of the calibration analysis are discussed in
    1596 \cite{magnier2017c}.  We present a brief summary here.
     1591\cite{magnier2017.calibration}.  We present a brief summary here.
    15971592
    15981593Astrometric calibration consists of measuring and correcting
     
    16071602a function of position in the camera (essentially an astrometric
    16081603flat-field correction), as a function of the brightness of the star
    1609 (the so-called Koppenh\"offer effect, see~\ref{magnier2017c}), and as
     1604(the so-called Koppenh\"offer effect, see~\ref{magnier2017.calibration}), and as
    16101605a function of airmass and color (Differential chromatic refraction).
    16111606Once the systematic errors have been measured, they are applied back
     
    16261621exposures which were believed to be obtained in photometric
    16271622conditions.  This process, called `\"ubercal', is described in detail
    1628 by \cite{ubercal} for the first (PV1) version.  In brief, photometric
     1623by \cite{2012ApJ...756..158S} for the first (PV1) version.  In brief, photometric
    16291624periods, with time-scales of at least \note{half of a night}, are
    16301625identified by a combination of automatic analysis and manual
     
    16581653flat-field correction addresses photometric variations due to spatial
    16591654variations in the PSF due to the optics and low-level effects on the
    1660 chips \citep[see][]{magnier2017c}.  After the systematic corrections
     1655chips \citep[see][]{magnier2017.calibration}.  After the systematic corrections
    16611656have been determined and applied back to the database, a final
    16621657relative photometry analysis pass is performed.
     
    17561751
    17571752Pantasks repeatedly checks each task in an attempt to generate a new
    1758 command: we say pantasks attempts to 'execute' the task in each of
     1753command: we say pantasks attempts to `execute' the task in each of
    17591754these attempts.  Tasks may specify the time between execution
    17601755attempts, with a 1 second default.
     
    17771772
    17781773Within the \ippprog{task.exec} macro, the command to be run must be
    1779 defined with the function 'command'.  Once the \ippprog{task.exec}
     1774defined with the function `command'.  Once the \ippprog{task.exec}
    17801775macro exits successfully, the defined command is the added to the list of jobs
    17811776to be run within the UNIX environment.  Jobs may be run in one of two
    17821777ways: locally or via the parallel processing system.  The task, or the
    1783 \ippprog{task.exec} macro, uses the 'host' command to define how to
    1784 run the job.  If the host is set to 'local', then the job is run in
     1778\ippprog{task.exec} macro, uses the `host' command to define how to
     1779run the job.  If the host is set to `local', then the job is run in
    17851780the background by pantasks itself (using the C \code{execvp}
    17861781function).  Otherwise, the job is sent to the parallel processing
    17871782system to be run on another machine within the cluster.  If the host
    1788 is set to the special value 'anyhost', then the parallel processing
     1783is set to the special value `anyhost', then the parallel processing
    17891784system is allowed to choose the processing computer arbitrarily.  Any
    17901785other value is taken to be the DNS name of the computer on which this
     
    17981793When the \ippprog{task.exec} macro is run, the code may choose (e.g.,
    17991794based on tests of some global variables) to exit the macro with an
    1800 error condition, e.g., with the 'break' command.  In this
     1795error condition, e.g., with the `break' command.  In this
    18011796circumstance, no job is produced by the task.  The task will be tried
    18021797again the next time it is executed.  This feature allows for the user
     
    18131808  online user guide?}
    18141809
    1815 The option 'npending' may be used to limit the number of jobs which
     1810The option `npending' may be used to limit the number of jobs which
    18161811are simultaneously executed for a specific task.  For example, some
    18171812classes of jobs should only be run one-at-a-time because they are not
    18181813protected against collisions or they may overload a resource.  The use
    1819 of 'npending' allows these situations to be handled cleanly within
     1814of `npending' allows these situations to be handled cleanly within
    18201815pantasks (avoiding cumbersome coding within with program or supporting
    18211816script).
    18221817
    1823 The option 'nmax' limits the total number of jobs which a task
     1818The option `nmax' limits the total number of jobs which a task
    18241819generates.  This option may be useful in cases where
    18251820\ippprog{pantasks} is used to perform a limited set of operations.
    18261821\note{do we actually use this in IPP?}
    18271822
    1828 The option 'trange' allows the user to restrict the time period during
     1823The option `trange' allows the user to restrict the time period during
    18291824which the specific tasks is executed.  This option is given with a
    18301825start and an end time for the limiting time range.  These times may be
     
    18411836ranges may be specified \note{how are they evaluated?}
    18421837
    1843 The option \code{nice} specifies the 'nice' level at which the job is
     1838The option \code{nice} specifies the `nice' level at which the job is
    18441839run when it is executed.  The parallel processing system must respect
    18451840this concept.
     
    19181913pantasks receives this completion information, the jobs are removed
    19191914from the list managed by pcontrol.  Thus pcontrol maintains at most a
    1920 modest list of jobs which are 'in flight', leaving all interpretation
     1915modest list of jobs which are `in flight', leaving all interpretation
    19211916work to pantasks.
    19221917
     
    24612456isolation of source objects is included, providing the organization of
    24622457detections that is used in the \ippprog{psphot} photometry analysis
    2463 \citep{magnier2017c}.  The PSF matching required for \ippstage{stack}
     2458\citep{magnier2017.analysis}.  The PSF matching required for \ippstage{stack}
    24642459and \ippstage{diff} stage image combinations is as well.  The
    24652460unification of configuration options between config files on disk and
     
    26852680column that link the tables together.
    26862681
    2687 \note{logical or alphabetical sequence?  alignment is broken}
     2682\note{logical or alphabetical sequence?}
    26882683
    26892684\begin{center}
     
    27392734\begin{verbatim}
    27402735MAJOR TODO ITEMS:
     2736* add figure showing DVO schema relationships
    27412737* re-read and trim details as needed (referring to the other papers)
    27422738* add some specific numbers (data volume, processing times, etc)
     2739* where is the smf/cmf format defined?  psphot?
     2740* where is the GPC1 naming convention discussed?
     2741* where are the flat-field seasons listed (magnier2017.calibration?)
    27432742\end{verbatim}
    27442743
    27452744\end{document}
    2746 
    2747 Figures needed for this document:
    2748 
    2749 *
Note: See TracChangeset for help on using the changeset viewer.