Changeset 41206 for trunk/doc/release.2015/ps1.datasystem/datasystem.tex
- Timestamp:
- Dec 16, 2019, 3:52:41 PM (7 years ago)
- File:
-
- 1 edited
Legend:
- Unmodified
- Added
- Removed
-
trunk/doc/release.2015/ps1.datasystem/datasystem.tex
r41204 r41206 288 288 \begin{figure*}[htbp] 289 289 \begin{center} 290 \includegraphics[width=\hsize,clip]{ PS1_Data_Analysis_System_Overview.pdf}290 \includegraphics[width=\hsize,clip]{{flowchart.v1}.pdf} 291 291 \caption{\label{fig:analysis.elements} Elements of the Pan-STARRS\,1 292 292 Data Analysis System. Rectangles represent data analysis steps; … … 574 574 For GPC1, the \ippstage{registration} stage is also the stage at which 575 575 the \ippprog{burntool} analysis is run. This analysis is more 576 completely described in Paper III. In brief, the 577 \ippprog{burntool} program identifies bright sources on the image, and 578 identifies persistence trails that result from the incomplete transfer 579 of charge. As this charge can leak out in subsequent exposures, the 580 burntoolanalysis is run sequentially on the exposures, based on the576 completely described in Paper III. In brief, the \ippprog{burntool} 577 program identifies bright sources on the image, and identifies 578 persistence trails that result from the incomplete transfer of charge. 579 As this charge can leak out in subsequent exposures, the burntool 580 analysis is run sequentially on the exposures, based on the 581 581 observation date and time listed in the headers, with the results 582 582 stored on disk. As a result of the sequential nature of this 583 583 analysis, the \ippstage{registration} of exposures is blocked until 584 584 the \ippprog{burntool} has been run on the previous exposures. 585 \textadd{Because this stage is only run once per exposure, changes to 586 the burntool code require a semi-manual re-running of the analysis 587 outside of the regular processing sequence. Since this is a rare 588 event, a standardized pipeline infrastructure was not developed for 589 this circumstance. In a future re-organization, a standard 590 serialized pre-processing step may be needed in the pipeline. } 585 591 586 592 Once the \ippstage{registration} process has finished, new science … … 700 706 individual chips is performed, including a fit to a single model for 701 707 the distortion introduced by the camera optics. The astrometric model 702 includes a set of 3rd order polynomials for the transformations from the chip 703 coordinate system to the camera focal plane coordinate system and a 704 single additional 3rd order polynomial transformation from the camera focal 705 plane coordinate system to the tangent plane of a tangent projection. 708 includes a set of 3rd order polynomials for the transformations from 709 the chip coordinate system to the camera focal plane coordinate system 710 and a single additional 3rd order polynomial transformation from the 711 camera focal plane coordinate system to the tangent plane of a tangent 712 projection. 713 714 \textadd{As discussed in detail in Paper V, We find that, for the PS1 715 images, small-scale structures are present in the astrometric 716 transformation. Some of these are due to ripples in the focal 717 surface, while others may be caused by the atmosphere. We find that 718 including higher-order terms in both the chip-to-focal plane and 719 focal-plane to sky are necessary to capture significant astrometric 720 signals. Some care must be taken in the fitting process to avoid 721 degeneracies between terms on different scales.} 722 706 723 For the $3\pi$ PV3 analysis, the typical astrometric residuals are in 707 724 the range of 20 - 30 milliarcseconds, sufficient to match observations … … 728 745 \ippstage{camera} stage also generates the dynamic masks for the 729 746 images. These include masking for optical ghosts, glints, saturated 730 stars, diffraction spikes, and electronic crosstalk. The mask images 747 stars, diffraction spikes, and electronic crosstalk. \textadd{The mask 748 information is generated based on the reference star catalog, along 749 with models for the various effects. Note however that this analysis does not 750 go back to the pixels to validate the prediction.} The mask images 731 751 generated by the \ippstage{chip} stage are updated with these dynamic 732 752 masks and a new set of files are saved for the downstream analysis … … 847 867 In the IPP processing, stacks may be made with various options for the 848 868 input images. During nightly science processing, the 8 exposures per 849 filter for each Medium Deep field are combined into a set of stacks850 for that field. These so-called ``nightly stacks'' are used by the 851 transient survey projects to detect faint supernovae, among other 852 transient events. For the PV3 $3\pi$ analysis, all images in each 853 filter from the observations for this survey were stacked together to 854 generate a single set of images with $\sim 10 - 20\times$ the exposure 855 of the individual survey exposures. 869 filter for each Medium Deep field are \textadd{automatically} combined 870 into a set of stacks for that field. These so-called ``nightly 871 stacks'' are used by the transient survey projects to detect faint 872 supernovae, among other transient events. For the PV3 $3\pi$ 873 analysis, all images in each filter from the observations for this 874 survey were stacked together to generate a single set of images with 875 $\sim 10 - 20\times$ the exposure of the individual survey exposures. 856 876 857 877 For the PV3 processing of the Medium Deep fields, stacks have been … … 868 888 When a given set of \ippstage{stack} stage processing is defined, 869 889 exposures with existing \ippstage{warp} entries that match the filter, 870 position, and other criteria such as seeing are identified. An entry 890 position, and other criteria such as seeing are identified \textadd{(see 891 Section~\ref{sec:automation} to see how this is automated)}. An entry 871 892 is then added for each skycell in the \ippdbtable{stackRun} table, 872 893 with the \ippdbcolumn{warp_id} entries for the exposures added to the … … 1103 1124 \subsection{Processing Failure Rates} 1104 1125 1105 Table~\ref{tab:failure_rates} lists the unrecoverable failure rates1126 \textadd{Table~\ref{tab:failure_rates} lists the unrecoverable failure rates 1106 1127 for several of the major IPP stages for both the regular nightly 1107 1128 processing and the PV3 analysis of the $3\pi$ dataset. The table 1108 1129 gives the rate per 100,000 of the item processed. In the case of the 1109 \ippstage{camera} stage, the items correspond to complete exposures, 1110 while for \ippstage{chip} and \ippstage{warp}, the items correspond to 1111 individual chips and skycells, respectively. For \ippstage{stack}, 1112 items are the full stack. For the \ippstage{camera} stage, the entire 1113 exposure fails only in extreme cases. The astrometric calibration of 1114 individual chips may fail if there are not enough stars in the image, 1115 but the rest of the exposure may then still succeed. 1116 1117 For the warp analysis, the apparent high failure rate is an artifact 1118 of two features. First, 1119 1120 1121 \begin{table*} 1130 \ippstage{chip} and \ippstage{warp} stages, the items correspond to 1131 individual chips and skycells, respectively, while for the 1132 \ippstage{stack} stage, items are the stack skycells. For the 1133 \ippstage{camera} stage, the items correspond to complete exposures. 1134 The entire exposure fails for \ippstage{camera} only in extreme cases. 1135 The astrometric calibration of individual chips may fail if there are 1136 not enough stars in the image, but the rest of the exposure may then 1137 still succeed. Chips which formally succeed in the astrometry 1138 analysis but which have an astrometric calibration quality worse than 1139 our specification will also be excluded from ingest into the DVO 1140 database (see below). We list the astrometry failure rate for chips 1141 based on their absence from the DVO database.} 1142 1143 \textadd{For the warp analysis, the apparent high failure rate is something of 1144 an artifact. Target output skycells are defined based on 1145 conservatively generous boundaries for the corresponding chips. This 1146 results in a number of skycells with only a small fraction of valid 1147 pixels, for which there are likely few stars to measure the PSF. In 1148 the processing, any warp skycell with less than 10\% of its pixels 1149 unmasked in the output are automatically rejected. In addition, the 1150 analysis will register a poor quality if too few stars are available 1151 for the PSF modelling. To judge the rate at which the warp stage is 1152 losing pixels, either due to this effect or veritable analysis 1153 failures, we compare the total area of good (unmasked) pixels in the 1154 warp skyfiles to the total number of expected unmasked pixels from the 1155 corresponding input exposures using the masking fractions and total 1156 detector areas reported in Paper III. The result is that roughly 1157 3.9\% of the good input pixels are lost to the warp processing.} 1158 1159 \begin{table} 1122 1160 \begin{center} 1123 1161 \caption{Processing Failure Rates per 100,000 Items\label{tab:failure_rates}} … … 1129 1167 Chip & 48 & 34 \\ 1130 1168 Camera & 262 & 280 \\ 1131 Chip Astrometry& N/A & 307 \\1169 ~~~Chip Astrom & N/A & 307 \\ 1132 1170 Warp & 14244 & 13835 \\ 1171 ~~~Warp Pixels & N/A & 3900 \\ 1133 1172 Stack & N/A & 5 \\ 1134 1173 \hline … … 2762 2801 2763 2802 \bibliographystyle{apj} 2764 %\bibliography{lib}{}2765 \input{datasystem.bbl}2803 \bibliography{lib}{} 2804 %\input{datasystem.bbl} 2766 2805 2767 2806 \end{document}
Note:
See TracChangeset
for help on using the changeset viewer.
