Changeset 6054
- Timestamp:
- Jan 18, 2006, 8:50:18 PM (21 years ago)
- Location:
- trunk/doc/design
- Files:
-
- 2 added
- 4 edited
-
Makefile (modified) (2 diffs)
-
dataflow.sxd (modified) ( previous)
-
ippSSDD.tex (modified) (12 diffs)
-
pics/phase4.ps (modified) ( previous)
-
pics/phase4a.ps (added)
-
pics/phase4b.ps (added)
Legend:
- Unmodified
- Added
- Removed
-
trunk/doc/design/Makefile
r6018 r6054 1 # $Id: Makefile,v 1.1 3 2006-01-16 18:58:45eugene Exp $1 # $Id: Makefile,v 1.14 2006-01-19 06:49:50 eugene Exp $ 2 2 3 3 PDFLATEX = env TEXINPUTS=../../latex/inputs:$(TEXINPUTS):.: pdflatex … … 8 8 @echo " targets: srs ssdd scd all" 9 9 10 cdr: ippCDR.pdf 10 11 srs: ippSRS.pdf 11 12 ssdd: ippSSDD.pdf -
trunk/doc/design/ippSSDD.tex
r6049 r6054 1 %%% $Id: ippSSDD.tex,v 1. 4 2006-01-19 03:51:41eugene Exp $1 %%% $Id: ippSSDD.tex,v 1.5 2006-01-19 06:49:50 eugene Exp $ 2 2 \documentclass[panstarrs]{panstarrs} 3 3 … … 137 137 PSDC-430-012 & Pan-STARRS IPP Modules Supplementary Design Requirements Specification \\ \hline 138 138 PSDC-430-014 & Pan-STARRS IPP PS-1 Cluster Support \\ \hline 139 \tbd{add the other subsystem SDDs}140 139 \DocumentsExternalSection 141 140 Posix Standard & Open Group Based Specifications Issue 6, IEEE Std 1003.1, 2003 \\ 142 141 \DocumentsEnd 142 143 \tbd{add the other subsystem SDDs} 143 144 144 145 \section{Subsystem Overview} … … 605 606 names. 606 607 607 \ subsubsubsection{House keeping}608 609 \ paragraph{Lock sweeping} In the event that a Storage Object operation fails to complete successfully608 \paragraph{House keeping} 609 610 \subparagraph{Lock sweeping} In the event that a Storage Object operation fails to complete successfully 610 611 stale locks will have to be identified and removed from the IPP Pixel 611 612 Data Server Database. This should be done periodically by comparing … … 622 623 table. 623 624 624 \ paragraph{Consistency sweeping} Periodically the IPP Pixel Data Server meta-data and Storage Object will need625 \subparagraph{Consistency sweeping} Periodically the IPP Pixel Data Server meta-data and Storage Object will need 625 626 to be checked for sanity. This would be similar to running fsck on a 626 627 modern filesystem. Consistency sweeping should include Lock sweeping … … 2031 2032 Phase 2. 2032 2033 2034 \subsection{Summary of the Phase 1-3 Data Products} 2035 2036 2033 2037 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 2034 2038 … … 2042 2046 sky, the sky cell, along with the associated pixels from a collection 2043 2047 of images which have been processed through phases 1--3. The size and 2044 exact representation of a static sky cell are yet to be determined. 2045 The working concept is that the static sky cells contain roughly the 2046 same number of pixels as an OTA (4k x 4k) and represent a portion of a 2047 local tangent plane projection. In order to meet the image 2048 degradation requirements, the pixel scale of the static sky is planned 2049 to be 0.2\arcsec, somewhat smaller than the 0.3\arcsec\ raw image 2050 pixel scale. 2051 2052 For each sky cell, the corresponding pixels are extracted from the 2053 exposures being processed and mapped to the projection of the sky 2048 exact representation of a static sky cell are configurable parameters 2049 which may vary between surveys. In some cases, it may be feasible to 2050 use a single large rectangular tangent plane image for the static sky 2051 and accept the distortion near the edges. In other cases, the static 2052 sky cells may be more appropriate defined to represent a relatively 2053 small patch over which the distortion is known to be small. The 2054 details of the static sky cell definition should not be a driver for 2055 the Phase 4 analysis. In order to meet the image degradation 2056 requirements, the pixel scale of the static sky is planned to be 2057 0.2\arcsec, somewhat smaller than the 0.25\arcsec\ raw image pixel 2058 scale; the choice of this scale must be determined with some caution. 2059 If the pixels are too large, the degredation will be excessive; too 2060 small, and the total volume of the static sky image data will be 2061 excessively costly. 2062 2063 \begin{figure} 2064 \begin{center} 2065 \resizebox{6in}{!}{\includegraphics{pics/phase4}} 2066 \caption{ \label{fig:phase2} Phase 2 dataflow - this diagram is old: update} 2067 \end{center} 2068 \end{figure} 2069 2070 In the basic concept of the Phase 4 analysis, each sky cell is 2071 examined independently. The corresponding pixels are extracted from 2072 the exposures being processed and mapped to the projection of the sky 2054 2073 cell. The pixels from the multiple input processed images are combined 2055 2074 into a single, cleaned image. Outlier pixels may be optionally … … 2061 2080 The remaining pixels are added to the existing static sky image. 2062 2081 Object detection must be performed on the difference ($P4\Delta$) and 2063 cleaned ($P4\Sigma$) images. 2082 cleaned ($P4\Sigma$) images. This process is illustrated graphically 2083 in Figure~\ref{BasicP4} 2064 2084 2065 2085 \subsubsection{Image Warping and Matching} … … 2154 2174 a time when the computing infrastructure is not under significant load. 2155 2175 2156 %\begin{figure} 2157 %\begin{center} 2158 %\resizebox{6in}{!}{\includegraphics{pics/phase4}} 2159 %\caption{ \label{fig:phase4} Phase 4 dataflow} 2160 %\end{center} 2161 %\end{figure} 2176 \subsubsection{Magic and Phase 4 Modifications} 2177 2178 \begin{figure} 2179 \begin{center} 2180 \resizebox{6in}{!}{\includegraphics{pics/phase4a}} 2181 \caption{ \label{fig:phase4a} Phase 4a} 2182 \end{center} 2183 \end{figure} 2184 2185 The Pan-STARRS relationship with the U.S. Air Force has some 2186 implications for the data processing which place some interesting 2187 constraints on the IPP implementation. The U.S. Air Force is funding 2188 the construction of PS-1, and we are thus subject to restrictions 2189 under which the Air Force must operate telescopes. The Air Force has 2190 diplomatic and security concerns about publically releasing images in 2191 which artificial satelites are detected, particularly if those 2192 satelites are identified. Historically, Air Force projects have been 2193 restricted from releasing images with identified satelites, 2194 considering the data to be of a sensitive nature. 2195 2196 The language which governs such Air Force projects is in the process 2197 of being modified so that images with identifable satelites can be 2198 treated under the classification of 'For Official Use Only'. Even 2199 under such an arrangement, however, the Air Force requires that the 2200 satelites which appear in the PS-1 images be excised in such a way 2201 that their orbits cannot be reliably identified. 2202 2203 Satelite detected in the Pan-STARRS images will appear as long ($> 2204 100$ pixels) streaks. The natural, and most effective, way to 2205 identify such streaks is to search for them in the images after they 2206 have been processed using the difference image processing in Phase 4. 2207 A team from Boeing has been contracted to develop a software module to 2208 idenfity streaks in these images. They delivered the initial release 2209 of their software, which is currently called {\em Magic}, in the 2210 beginning of January 2006. Further effort on that software will be 2211 required to confront it with the real PS-1 image parameters. 2212 2213 In order to mesh the operation of {\em Magic} with the IPP Phase 4 2214 analysis for PS-1, some modifications must be made to the operation 2215 sequence. In the case of PS-1, unlike PS-4, the basic set of images 2216 which confronted with the Static Sky image are obtained in sequence, 2217 not essentially simultaneously. This is important for Magic because 2218 the satelite streak in one image will not appear in the other three. 2219 Thus the Magic operation must be performed on intermediate difference 2220 images for individual exposures. To facilitate this, the basic Phase 2221 4 described above is divided into three stages: Phase 4a, in which the 2222 individual difference frames are constructed; Magic; and Phase 4b, in 2223 which the results of Magic are used to filter the images before the 2224 final stack and difference images are generated. 2225 2226 Figure~\ref{phase4a} illustrates the operation of Phase 4a and Magic. 2227 The individual warped image (P4$_W$) is constructed and differenced 2228 against the corresponding Static Sky image. In this analysis, poisub, 2229 which performs the kernel matching and difference image construction, 2230 is run with somewhat looser constraints on the image difference 2231 kernel. The lower-accuracy difference kernel will make the difference 2232 image noisier in the vicinity of galaxies and bright stars. Unlike 2233 the science transients, however, most of the pixels involved in a 2234 satelite streak are not in close proximity to another objects; for SNe 2235 and GRBs, the objects are preferentially located near host galaxies 2236 where it is most critical to have a high-quality subtraction. 2237 Allowing poisub a more relaxed image difference kernel results in a 2238 substantial improvement in the convolution speed. The resulting 2239 difference image is called P4$\Delta$'. 2240 2241 {\em Magic} searches for streakes in the difference images by using a 2242 heirarchy of Huff transforms. A single run of {\em Magic} expects to 2243 have access to all of the P4$\Delta$' images from a single FPA image. 2244 It starts with independent Huff transforms of the individual cells, 2245 which can be performed in parallel under the PanTasks paralleliztion 2246 scheme. The results of these Huff transforms are then merged to yield 2247 the equivalent of a larger Huff-transformed image. The heirarchy 2248 searches for significant streaks from the individual cells, the 2249 equivalent 2x2 cell sets, then 2x2 of those, etc, until the entire 2250 array has been searched. Significant peaks detected from the Huff 2251 transforms are then used to restrict the pixels in the image space. 2252 These pixels are examined and streaks detected by requiring a certain 2253 fraction of the pixels along the purported streak to be lit. 2254 Filtering using the cross-streak PSF can also be applied to enhance 2255 the detection and minimize the false positives. Any streaks which are 2256 detected are then excisized. In order to obfuscate the idenification 2257 of the satelite, this step masks the streak pixels, with a box 2258 somewhat wider than the streak (10-20 pixel wide), displaced by an 2259 unknown, random amount relative to the streak center-line. This strip 2260 is extended to the ends of the FPA array. Analysis of the expected 2261 density of streaks by the Boeing team, using known satelites as the 2262 input to the simulation, shows that we should expect to lose less than 2263 1\% of the pixels due to these long excised regions. 2264 2265 The result of the Magic process is a set of masks for each raw PS-1 2266 image. Any images which are released beyond the IPP cluster must have 2267 these masks applied, or with manual inspection for visible streaks. 2268 \tbd{time-stamps? ok to release without the valid time? or only with } 2269 2270 The final portion of the analysis, Phase 4b, is illustrated in 2271 Figure~\ref{phase4b}. In this stage, the intermediate result images 2272 from Phase 4a can be used. The P4$_W$ images are masked with the 2273 corresponding masks from Magic. These images are then stacked as 2274 normal, and the resulting summed image is differenced with poisub, 2275 using a more stringent limit on the image difference kernel. 2276 2277 \begin{figure} 2278 \begin{center} 2279 \resizebox{6in}{!}{\includegraphics{pics/phase4b}} 2280 \caption{ \label{fig:phase4b} Phase 4b} 2281 \end{center} 2282 \end{figure} 2162 2283 2163 2284 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% … … 2748 2869 \item {\bf output stage} as a stand-alone program, psphot will produce 2749 2870 output tables in various formats. 2871 \end{itemize} 2750 2872 2751 2873 \subsection{psastro} … … 2773 2895 and 3. 2774 2896 2775 In the conceptual y most straightforeward mode, each readout contained2897 In the conceptually most straightforeward mode, each readout contained 2776 2898 in the incoming FPA structure is treated independently. The metadata 2777 2899 describing the approximate astrometry of the readout are used to guess … … 2811 2933 basic description of the astrometry is the collection of header 2812 2934 keywords which define the boresite center coordinates (RA, DEC), the 2813 rotation and platescales 2935 location of the chip data arrays within the full mosaic (the IRAF 2936 DATASEC keywords). More detailed astrometric information may be 2937 defined using the WCS keywords. These unfortunately do not have a 2938 standard representation of higher-order terms. Furthermore, the two 2939 competing systems which have been proposed define only a single 2940 transformation frame. In the IPP, it is important to carry around 2941 more information which can be used to improve our astrometric 2942 solutions in the future. Specifically, we would like to maintain at 2943 least the transformation for the telescope optics independent from the 2944 individual chip warps or tilts. Even more, we would like to have a 2945 flexible astrometry definition format which can be extended in a 2946 flexible fashion. We have defined a FITS table convention to carry 2947 all of the elements of the astrometric transformation of a full FPA. 2948 Within the table, transformations are generally defined to convert one 2949 layer (eg, focal plane) to another. The form and the parameters of 2950 the conversion make up the columns of the table. With this structure, 2951 it is possible to add arbitrary layers as needed. The IPP, and 2952 portions of the project (particularly Otis), will share a common 2953 default astrometry model for the telescope. This will be defined on 2954 the basis of measurements over the first weeks of observations. The 2955 IPP will perform the astrometric analysis of individual images with 2956 psastro, and the results can be saved in this tabular format. Over 2957 time, these result tables can be used to improve the astrometric model 2958 for the telescope, and to improve the astrometric reference catalog. 2959 2960 \subsection{poisub} 2961 2962 Poisub is the image difference analysis program. \tbd{Paul: please 2963 flesh this out!}. 2964 2965 \subsection{stac} 2966 2967 STAC is the program which warps and optimally combines images from the 2968 same region of the sky. It consists of two major stages: the warping 2969 stage and the image combination stage with robust outlier rejection. 2970 \tbd{Paul: flesh this out!} 2814 2971 2815 2972 \section{Interfaces} … … 3817 3974 ------ 3818 3975 3819 * poisub / stack 3820 * re-org the Phase 4 stuff to discuss Magic 3821 * astrometry calibration data formats 3976 * output data products 3977 * DVO 3978 * PanTasks 3979 * ipptools / ippMonitor? 3822 3980 * analysis stages, versions and iterations 3823 * output data products
Note:
See TracChangeset
for help on using the changeset viewer.
