Index: /trunk/doc/design/Makefile =================================================================== --- /trunk/doc/design/Makefile (revision 2185) +++ /trunk/doc/design/Makefile (revision 2186) @@ -1,13 +1,14 @@ -# $Id: Makefile,v 1.7 2004-08-06 19:06:01 eugene Exp $ +# $Id: Makefile,v 1.8 2004-10-21 03:55:59 eugene Exp $ PDFLATEX = pdflatex PSLATEX = latex +PSLATEX =env TEXINPUTS=../../latex/inputs:$(TEXINPUTS):.: latex -srs : +srs : ippSRS.tex trace.pl ippSRS.tex ippSRSout.tex ippSRStrace.tex ltx ippSRSout + mv ippSRSout.pdf ippSRS.pdf -all : ippSCD.pdf ippSRS.pdf ippSDRS.pdf -ps : ippSCD.ps ippSRS.ps ippSDRS.ps +all : ippSRS.pdf ippSDRS.pdf %.pdf : %.tex Index: /trunk/doc/design/ippSDRS.tex =================================================================== --- /trunk/doc/design/ippSDRS.tex (revision 2185) +++ /trunk/doc/design/ippSDRS.tex (revision 2186) @@ -1,3 +1,3 @@ -%%% $Id: ippSDRS.tex,v 1.9 2004-10-20 08:17:36 eugene Exp $ +%%% $Id: ippSDRS.tex,v 1.10 2004-10-21 03:55:59 eugene Exp $ \documentclass[panstarrs]{panstarrs} @@ -2370,28 +2370,96 @@ \section{Computer Hardware} -\subsection{PS-1 Cluster requirements} - - \begin{itemize} - \item CPU requirements - \item per-node I/O requirements - \item switch throughput requirements - \item storage profile - \end{itemize} - -\subsection{PS-1 Cluster Hardware Plan} - - \begin{itemize} - \item COTS equipment - \item number of processors needed - \item number of I/O ports needed - \item number of disk slots needed - \item switch choice - \item design choice for computer nodes - \item total rack space - \end{itemize} +\subsection{PS-1 Cluster Design} + +The PS-1 IPP computer system is designed as a cluster of 'fat bricks': +computers with both processing power and large amounts of local disk +storage. These computers are large rack-mount boxes with space for +10s of disks (24 and 36 disk cases are available) and a motherboard +with two CPUs and two Gig-E ethernet ports. One set of machines is +specified for storage and processing of the individual OTAs up through +Phase 2 (the `OTA nodes'), another set of machines are specified for +storage of the Static Sky and processing of data from Phase 3 and +Phase 4 (the `Sky nodes'). Other machines will be necessary to +support the Metadata DB and the AP DB. + +The IPP PS-1 SRS (PSDC-xxx) specifies the processing throughput +requirements for the IPP. We have performed benchmark tests of the +processing needs in order to achieve this throughput. The details of +this study are presented in the IPP Hardware Analysis (PSDC-xxx), +which we summarize here. The analysis measures the processing time +(excluding I/O) for both Phase 2 and Phase 4 on an Intel Pentium 4 +processor, and expresses the processing time in GHz-seconds, under the +assumption that a machine with the same architecture and twice the +processor speed with perform the same analysis in half the time. This +is probably a valid assumption within a limited range on hardware +using the same architecture. We independently find that 32-bit Pentium +processors perform somewhat slower (up to a factor of 2) than +equivalently rated 64 bit Opeteron processors. This discrepancy makes +our numbers somewhat conservative, but may only compensate for the +simplistic analysis we have performed. + +Our benchmarks show that the Phase 2 analysis takes 12000 GHz-seconds +for a complete major frame (4 FPAs) while the Phase 4 analysis takes +7800 GHz-seconds for the same major frame. We also examine the total +data I/O required for each processing node both locally to disk and +across the network to other machines. These numbers in turn depend on +whether the data is optimally stored on the OTA nodes (raw images +matched to their calibration images) or if the data are randomized. +There are also differences in the analysis for how many bits and +images are used in the processing. For PS-1, the `minimal' data set +is approrpiate, resulting in a total Phase 2 I/O of 21 GBs per major +frame and a total Phase 4 I/O of 36 GBs. We will use the randomized +numbers as a conservative estimate, and assume the network is the +dominant I/O bottleneck. + +The analysis assumes each CPU is associated with one RAID array +(maximum throughput 110 MB/sec) and one network controller (maximum +throughput 70 MB/s) and that each one is a 2.2 GHz processor. In this +case, given the CPU load and I/O throughput above, the Phase 2 will +require a total of 190 seconds of I/O and 5500 seconds of processing +distributed across the cluster. Likewise, the Phase 4 analysis will +require a total of 330 sec of I/O and 3500 seconds of processing. +Given the 160 seconds available per major frame, these numbers imply a +total of 63 processors are needed to keep up with the processing and +I/O load. + +The other major driver on the IPP PS-1 cluster are the data storage +requirements. We are required to store the entire AP Survey data and +the IVP data, and to have storage enough to represent the Static Sky +by the end of the two year mission. These storage requirements as a +function of time are shown in Figure~\ref{StorageProfile}. Based on +the PS-1 Design Reference Mission (PSDC-xxx), by the end of the +second year, we will have total storage needs of 850 TB for raw images +and the Static Sky, and an additional \tbd{XXX} TB for the AP DB +storage. + +To meet these requirements, we have designed the IPP cluster to use +fat bricks which will be capable of holding 24 disks each. Before +PS-1 goes on line, we will purchase enough disks to fill 1/3 of the +disk slots. After 9 months (2006 Sept), we will purchase the next 1/3 +of the disks, and the remaining disks 9 months after that (2007 June). +We have made conservative estimates of the available disk sizes at +these purchase dates (400 GB, 600 GB, and 900 GB), allowing us to +determine the number of computers needed to meet the storage +specification. We will purchase 80 computers, with the storage +profile shown in the figure, ending at a total capacity (after +discounting volume for RAID overhead and binary vs digital terabytes) +of 950 TBs. The 80 computers will easily meet the processing and I/O +requirements given the above need to 63 processors. + +\tbd{switch information} + +\tbd{RAID and compression / duplication plan} \subsection{PS-1 Cluster Expected Reliability} \subsection{PS-1 Cluster Support} + +\begin{figure} +\begin{center} +\resizebox{6in}{!}{\includegraphics{pics/ps1_ipp_storage.ps}} +\caption{ \label{StorageProfile} Storage Profile} +\end{center} +\end{figure} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Index: /trunk/doc/design/ippSRS.tex =================================================================== --- /trunk/doc/design/ippSRS.tex (revision 2185) +++ /trunk/doc/design/ippSRS.tex (revision 2186) @@ -1,3 +1,3 @@ - %%% $Id: ippSRS.tex,v 1.9 2004-10-18 22:06:13 eugene Exp $ + %%% $Id: ippSRS.tex,v 1.10 2004-10-21 03:55:59 eugene Exp $ \documentclass[panstarrs,spec]{panstarrs} @@ -23,4 +23,5 @@ \newcommand\VER[2]{\\ {\scriptsize QUALIFICATION METHOD: #1, TRACE: #2}} \newcommand\TASK{\\ {\scriptsize TASK}} +\newcommand\grizy{$grizy$} % allow paragraphs to be listed in TOC for now @@ -40,10 +41,8 @@ \RevisionsEnd -\TBDsStart -Section & Page & Number & Description \\ \hline - & & & choice of scripting language \\ \hline - & & & coding standards for scripting language \\ \hline - & & & low-spatial frequency sky model details \\ \hline -\TBDsEnd +\inserttbd + +\inserttbr +\pagebreak \tableofcontents @@ -216,6 +215,6 @@ \item In photometric weather, produce reduced science images for each full camera exposure which have background deviations from the - static sky in the same filter of less than \tbd{1\%} for the median - in large ($> 30$ pixels diameter) regions.\VER{ANALYSIS}{SCD:3.5.12} + static sky in the same filter of less than 1\% for the median in + large ($> 30$ pixels diameter) regions.\VER{ANALYSIS}{SCD:3.5.12} \label{TLR:5a} @@ -255,5 +254,5 @@ \item Degrade the stacked image by no more than \tbr{10 - milliarcseconds} (FWHM added in quadrature) over the theoretical + milliarcseconds (FWHM added in quadrature)} over the theoretical limit for the stack under infinite sampling.\VER{ANALYSIS}{SCD:3.5.2} @@ -262,10 +261,10 @@ \item Perform the processing of science images to the level of transient detection and static sky inclusion at a rate such that - exposures taken at an average cadence of \tbr{40} seconds do not + exposures taken at an \tbr{average cadence of 40 seconds} do not accumulate in the processing buffer (average throughput requirement).\VER{TEST}{SCD:3.2.2.3} \label{TLR:17} -\item Limit the false alarm rate (FAR) to less than \tbr{5\%} for +\item Limit the \tbr{false alarm rate (FAR) to less than 5\%} for transient detections $> 5\sigma$ sent to the preferred client science pipelines.\footnote{note difference with PS-4: 1\%} @@ -273,5 +272,5 @@ \label{TLR:18} -\item Perform transient detection to a completeness of \tbr{99\%} at +\item Perform \tbr{transient detection to a completeness of 99\%} at the completeness for transient detections with a significant $> 5\sigma$.\VER{ANALYSIS}{SCD:xxx} @@ -287,5 +286,6 @@ \item Send the IPP metadata and received OTIS metadata to the - Pan-STARRS Published Science Products Subsystem (PSPS) weekly.\VER{TASK}{SCD:3.2.2.18} + Pan-STARRS Published Science Products Subsystem (PSPS) + weekly.\VER{TASK}{SCD:3.2.2.18} \label{TLR:21} @@ -294,17 +294,17 @@ \label{TLR:22} -\item Provide sufficent storage volume for \tbr{1 year} of raw images - from the AP and IVP Surveys.\footnote{note difference with PS-4: 1 - month of raw images} \VER{INSPECT}{allocated} +\item Provide sufficent storage volume for raw images from the AP and + IVP Surveys and the \grizy\ Static Sky.\footnote{note difference with + PS-4: 1 month of raw images} \VER{INSPECT}{allocated} \label{TLR:23} \item Provide sufficient storage volume for all detections from the - AP, IVP, and MVP Surveys.\footnote{note difference with PS-4: 1 - year of detections}\VER{INSPECT}{allocated} + AP, IVP, and MVP Surveys.\footnote{note difference with PS-4: 1 year + of detections}\VER{INSPECT}{allocated} \label{TLR:24} -\item Provide sufficient storage volume for \tbr{2 year} of - metadata.\footnote{note difference with PS-4: 10 - years of metadata}\VER{INSPECT}{allocated} +\item Provide sufficient storage volume for 2 years of + metadata.\footnote{note difference with PS-4: 10 years of + metadata}\VER{INSPECT}{allocated} \label{TLR:25} \end{enumerate} @@ -366,5 +366,5 @@ \item Source code files shall use the UNIX line-break convention (line-feed only). \VER{INSPECT}{allocated} \item C coding style shall adhere to the standard defined in the document `Pan-STARRS C-coding standard' (PSDC-430-004). \VER{INSPECT}{allocated} -\item Perl coding shall follow the standard defined in the document `Pan-STARRS Perl-coding standard' \tbd{(PSDC-430-0XX)}.\VER{INSPECT}{allocated} +\item Perl coding shall follow the standard defined in the document \tbd{`Pan-STARRS Perl-coding standard' (PSDC-430-0XX)}.\VER{INSPECT}{allocated} \end{enumerate} @@ -630,7 +630,11 @@ reference to the specified image.\VER{TEST}{allocated} -\item \tbd{total volume} - -\item \tbd{total number of entries} +\item The IPP Image Server shall provide a total data capacity of 300 + TB after the first year of PS-1 operations and 900 TB after the + second year of operations.\VER{INSPECT}{} + +\item The IPP Image Server shall provide storage for a total of $2 + \times 10^8$ data objects by the end of the second year of PS-1 + operations. \end{enumerate} @@ -1206,5 +1210,5 @@ \item The background residuals shall have a scatter of less than \tbr{1\%} of the input background amplitude for apertures of less - than \tbr{10~arcsec}.\VER{ANALYSIS}{TLR:1} + than \tbr{10 arcsec}.\VER{ANALYSIS}{TLR:1} \item The Phase 2 analysis shall detect cosmic rays with flux $> 5\sigma$ by Index: /trunk/doc/design/trace.pl =================================================================== --- /trunk/doc/design/trace.pl (revision 2185) +++ /trunk/doc/design/trace.pl (revision 2186) @@ -21,5 +21,5 @@ $N = @words; - printf OUT "$start\{"; + print OUT "$start\{"; for ($i = 0; $i < @words; $i++) { $word = $words[$i];