Changeset 9673
- Timestamp:
- Oct 19, 2006, 5:51:59 PM (20 years ago)
- File:
-
- 1 edited
-
trunk/doc/config/config.tex (modified) (9 diffs)
Legend:
- Unmodified
- Added
- Removed
-
trunk/doc/config/config.tex
r9622 r9673 1 %%% $Id: config.tex,v 1. 1 2006-10-18 04:19:54price Exp $1 %%% $Id: config.tex,v 1.2 2006-10-20 03:51:59 price Exp $ 2 2 \documentclass[panstarrs,spec]{panstarrs} 3 3 … … 100 100 provided; 101 101 \item The environment variable \code{PS_SITE}, if defined; or 102 \item \code{ \$HOME/.ipprc} otherwise.102 \item \code{$HOME/.ipprc} otherwise. %$ 103 103 \end{enumerate} 104 104 … … 183 183 ### Example .ipprc file 184 184 185 PATH STR .:/my/home/.ipp # Default search path for configuration files186 WORKDIR STR /my/data/disk/# Top-level working directory185 PATH STR .:/my/home/.ipp # Default search path for configuration files 186 WORKDIR STR /my/data/disk/ # Top-level working directory 187 187 188 188 ### Database configuration 189 DBSERVER STR localhost# Database host name (for psDBInit)190 DBNAME STR my_database# Database name (for psDBInit)191 DBUSER STR my_name# Database user name (for psDBInit)192 DBPASSWORD STR my_password# Database password (for psDBInit)189 DBSERVER STR localhost # Database host name (for psDBInit) 190 DBNAME STR my_database # Database name (for psDBInit) 191 DBUSER STR my_name # Database user name (for psDBInit) 192 DBPASSWORD STR my_password # Database password (for psDBInit) 193 193 194 194 ### Setups for each camera system 195 CAMERAS METADATA196 MCSHORT STRmcshort/camera.config197 MCSHORT_CHIP STRmcshort_chip/camera.config198 MCSHORT_FPA STRmcshort_fpa/camera.config199 MEGACAM STRmegacam/camera.config200 MEGACAM_CHIP STRmegacam_chipmosaic/camera.config201 MEGACAM_FPA STRmegacam_fpamosaic/camera.config202 MEGACAM_DET STRmegacam_detrended/camera.config203 UCAM STRucam/camera.config204 UCAM_MOSAIC STRucam_mosaic/camera.config205 GPC1 STRgpc1/camera.config206 LRIS_BLUE STRlris_blue/camera.config207 LRIS_RED STRlris_red/camera.config208 ISP STRisp/camera.config209 SIMPLE STRsimple/camera.config195 CAMERAS METADATA 196 MCSHORT STR mcshort/camera.config 197 MCSHORT_CHIP STR mcshort_chip/camera.config 198 MCSHORT_FPA STR mcshort_fpa/camera.config 199 MEGACAM STR megacam/camera.config 200 MEGACAM_CHIP STR megacam_chipmosaic/camera.config 201 MEGACAM_FPA STR megacam_fpamosaic/camera.config 202 MEGACAM_DET STR megacam_detrended/camera.config 203 UCAM STR ucam/camera.config 204 UCAM_MOSAIC STR ucam_mosaic/camera.config 205 GPC1 STR gpc1/camera.config 206 LRIS_BLUE STR lris_blue/camera.config 207 LRIS_RED STR lris_red/camera.config 208 ISP STR isp/camera.config 209 SIMPLE STR simple/camera.config 210 210 END 211 211 212 212 ### psLib setup 213 TIME STR pslib/psTime.config# Time configuration file214 LOGLEVEL S32 9# Logging level; 3=INFO215 LOGFORMAT STR THLNM# Log format216 LOGDEST STR STDERR# Log destination217 TRACEDEST STR STDERR# Trace destination213 TIME STR pslib/psTime.config # Time configuration file 214 LOGLEVEL S32 9 # Logging level; 3=INFO 215 LOGFORMAT STR THLNM # Log format 216 LOGDEST STR STDERR # Log destination 217 TRACEDEST STR STDERR # Trace destination 218 218 TRACEFORMAT STR THLNM # Trace format 219 TRACE METADATA# Trace levels220 err S3210221 END 222 223 RECIPES METADATA# Site-level recipes224 PPMERGE STR ppMerge_template.config# Recipe for combination225 PPSTATS_PHASE0 STR ppStats_phase0.config# Recipe for phase 0 processing219 TRACE METADATA # Trace levels 220 err S32 10 221 END 222 223 RECIPES METADATA # Site-level recipes 224 PPMERGE STR ppMerge_template.config # Recipe for combination 225 PPSTATS_PHASE0 STR ppStats_phase0.config # Recipe for phase 0 processing 226 226 END 227 227 \end{verbatim} … … 330 330 be useful to define a type: 331 331 \begin{verbatim} 332 TYPE LIMITS FILTER EXPECTED IMFILE.MEAN IMFILE.STDEV EXP.MEAN EXP.STDEV EXP.MEANSTDEV ENSEMBLE.MEAN ENSEMBLE.STDEVENSEMBLE.MEANSTDEV332 TYPE LIMITS FILTER EXPECTED IMFILE.MEAN IMFILE.STDEV EXP.MEAN EXP.STDEV EXP.MEANSTDEV ENSEMBLE.MEAN ENSEMBLE.STDEV ENSEMBLE.MEANSTDEV 333 333 \end{verbatim} 334 334 … … 344 344 345 345 # File formats that we know about 346 FORMATS METADATA347 RAW STRmcshort/format_raw.config348 SPLICE STRmcshort/format_spliced.config349 SPLIT STRmcshort/format_split.config346 FORMATS METADATA 347 RAW STR mcshort/format_raw.config 348 SPLICE STR mcshort/format_spliced.config 349 SPLIT STR mcshort/format_split.config 350 350 END 351 351 352 352 # Description of camera --- all the chips and the cells that comprise them 353 FPA METADATA354 ccd12 STRLeftAmp RightAmp355 ccd13 STRLeftAmp RightAmp356 ccd14 STRLeftAmp RightAmp357 ccd21 STRLeftAmp RightAmp358 ccd22 STRLeftAmp RightAmp359 ccd23 STRLeftAmp RightAmp353 FPA METADATA 354 ccd12 STR LeftAmp RightAmp 355 ccd13 STR LeftAmp RightAmp 356 ccd14 STR LeftAmp RightAmp 357 ccd21 STR LeftAmp RightAmp 358 ccd22 STR LeftAmp RightAmp 359 ccd23 STR LeftAmp RightAmp 360 360 END 361 361 … … 374 374 Haalpha.on STR Ha 375 375 HaOFF.MP7604 STR HaOff 376 377 CN.MP780 STR CN378 cn.MP7803 STR CN379 CN.MP7803 STR CN380 381 TiO.MP77 STR TiO382 tio.MP7701 STR TiO383 TiO.MP7701 STR TiO384 NB920 STR NB920385 386 B2F STR B2F387 Bj STR Bj388 Vj STR Vj389 Rj STR Rj390 Ij STR Ij391 Hb STR Hb392 HbOff STR HbOff393 376 END 394 377 395 378 396 379 # Recipe options 397 RECIPES METADATA398 # Recipes for ppImage380 RECIPES METADATA 381 # Recipes for ppImage 399 382 PPIMAGE STR megacam/ppImage.config # Default: all (normal) options on 400 PPIMAGE_O STR megacam/ppImage_o.config# Overscan only401 PPIMAGE_OB STR megacam/ppImage_ob.config# Overscan, bias only402 PPIMAGE_OBD STR megacam/ppImage_obd.config# Overscan, bias, dark only403 PPIMAGE_OBDF STR megacam/ppImage_obdf.config# Overscan, bias, dark, flat only404 PPIMAGE_B STR megacam/ppImage_b.config# Bias only405 PPIMAGE_D STR megacam/ppImage_d.config# Dark only406 PPIMAGE_F STR megacam/ppImage_f.config# Flat only407 PPIMAGE_J1 STR megacam/ppImage_j1.config# JPEG only; binning 1408 PPIMAGE_J2 STR megacam/ppImage_j2.config# JPEG only; binning 2409 PPIMAGE_N STR megacam/ppImage_n.config# Nothing significant; binning only410 411 # Recipes for ppMerge383 PPIMAGE_O STR megacam/ppImage_o.config # Overscan only 384 PPIMAGE_OB STR megacam/ppImage_ob.config # Overscan, bias only 385 PPIMAGE_OBD STR megacam/ppImage_obd.config # Overscan, bias, dark only 386 PPIMAGE_OBDF STR megacam/ppImage_obdf.config # Overscan, bias, dark, flat only 387 PPIMAGE_B STR megacam/ppImage_b.config # Bias only 388 PPIMAGE_D STR megacam/ppImage_d.config # Dark only 389 PPIMAGE_F STR megacam/ppImage_f.config # Flat only 390 PPIMAGE_J1 STR megacam/ppImage_j1.config # JPEG only; binning 1 391 PPIMAGE_J2 STR megacam/ppImage_j2.config # JPEG only; binning 2 392 PPIMAGE_N STR megacam/ppImage_n.config # Nothing significant; binning only 393 394 # Recipes for ppMerge 412 395 PPMERGE STR ppMerge_template.config # ppMerge recipe 413 PPMERGE_BIAS STRmegacam/ppMerge_bias.config414 PPMERGE_DARK STRmegacam/ppMerge_dark.config415 PPMERGE_FLAT STRmegacam/ppMerge_flat.config416 417 # Other recipes396 PPMERGE_BIAS STR megacam/ppMerge_bias.config 397 PPMERGE_DARK STR megacam/ppMerge_dark.config 398 PPMERGE_FLAT STR megacam/ppMerge_flat.config 399 400 # Other recipes 418 401 PSPHOT STR megacam/psphot.config # psphot details 419 402 PSASTRO STR megacam/psastro.config # psastro details 420 PPSTATS STR megacam/ppStats.config# ppStats recipe403 PPSTATS STR megacam/ppStats.config # ppStats recipe 421 404 END 422 405 423 406 424 407 # Rejection levels for detrend creation 425 REJECTION METADATA426 TYPE LIMITS FILTER EXPECTED IMFILE.MEAN IMFILE.STDEV EXP.MEAN EXP.STDEV EXP.MEANSTDEV ENSEMBLE.MEAN ENSEMBLE.STDEVENSEMBLE.MEANSTDEV427 FLATMULTI428 429 BIAS LIMITS * 0 0 15 0 15 0 0 00430 DARK LIMITS * 0 0 0 0 0 0 0 00431 FLAT LIMITS * 0 0 0 0 0 0 0 00432 FLAT LIMITS u 0 0 0 0 0 0 0 00433 FLAT LIMITS g 0 0 0 0 0 0 0 00434 FLAT LIMITS r 0 0 0 0 0 0 0 00435 FLAT LIMITS i 0 0 0 0 0 0 0 00436 FLAT LIMITS z 0 0 0 0 0 0 0 00437 438 END 439 408 REJECTION METADATA 409 TYPE LIMITS FILTER EXPECTED IMFILE.MEAN IMFILE.STDEV EXP.MEAN EXP.STDEV EXP.MEANSTDEV ENSEMBLE.MEAN ENSEMBLE.STDEV ENSEMBLE.MEANSTDEV 410 FLAT MULTI 411 412 BIAS LIMITS * 0 0 15 0 15 0 0 0 0 413 DARK LIMITS * 0 0 0 0 0 0 0 0 0 414 FLAT LIMITS * 0 0 0 0 0 0 0 0 0 415 FLAT LIMITS u 0 0 0 0 0 0 0 0 0 416 FLAT LIMITS g 0 0 0 0 0 0 0 0 0 417 FLAT LIMITS r 0 0 0 0 0 0 0 0 0 418 FLAT LIMITS i 0 0 0 0 0 0 0 0 0 419 FLAT LIMITS z 0 0 0 0 0 0 0 0 0 420 421 END 422 440 423 441 424 FILERULES METADATA … … 511 494 \subsection{Contents} 512 495 513 514 \subsection{Example} 515 516 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 517 518 \section{Recipes} 519 520 \subsection{Locations} 521 522 Recipes may be specified in a number of locations. Firstly, they may 523 be specified on the command line with the \code{-recipe} option, 524 giving a symbolic name and a filename or another symbolic name to link 525 to. In addition, they may be specified in the site configuration and 526 the camera configuration under the \code{RECIPES} metadata. Note that 527 the \code{PATH(STR)} in the site configuration defines the search paths for 528 these files. 529 530 \subsection{Contents} 531 532 \subsection{Example} 533 534 535 536 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 537 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 538 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 539 540 \section{Revision Change Log} 541 %\input{ChangeLog.tex} 542 543 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 544 545 %\bibliographystyle{plain} 546 %\bibliography{panstarrs} 547 548 \end{document} 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 \subsubsection{Camera configuration} 583 584 The camera configuration file is a fairly simple configuration file 585 containing information particular to a particular camera, regardless 586 of the file format used to represent that camera. The camera configuration 587 consists of the following elements: 496 The camera format specifies how a FITS file from a particular camera 497 is to be read. Different formats may be defined for a single camera 498 (e.g., one amplifier per extension, vs all amplifiers spliced together 499 in the PHU). The camera format configuration file contains the rules 500 for recognising the format, how to read the file, the contents of a 501 FITS file, data appropriate to different types of cells, information 502 on how to determine the concepts from the headers, default values, or 503 database, and expected formats for certain concepts. 504 505 \subsubsection{Rules for recognising} 506 507 \code{RULE(METADATA)} contains a list of telescope headers with 508 expected values (of the appropriate type) for this particular 509 combination of the camera and format. It is often useful to include 510 \code{TELESCOP} and \code{DETECTOR}, if possible, along with any other 511 headers that uniquely identify the camera and format. Note that all 512 of the headers must match exactly (modulo leading and trailing spaces 513 for strings), including the data type and value, for the rule to 514 match, and that the first format's rule to match is accepted. If a 515 rule doesn't match the header, try adjusting the types (especially for 516 numerical types; try S32 for integers, F32 and F64 for floats). 517 518 \subsubsection{How to read the file} 519 520 \code{FILE(METADATA)} contains information on how to read the FITS 521 file for this format. The contents are: 588 522 \begin{itemize} 589 \item \code{FORMATS} of type \code{METADATA}: this contains a list of 590 known FITS file formats with the file names (of type \code{STR}) of 591 the configuration files; 592 \item \code{FPA} of type \code{METADATA}: this contains a list of 593 chips, each with a string list (type \code{STR} of the component 594 cells; and 595 \item \code{RECIPES} of type \code{METADATA}: this contains a list of 596 recipes used for the camera with the file names (of type \code{STR} 597 of the configuration files. 523 \item \code{PHU(STR)} identifies the class of the file --- what level 524 in the focal plane hierarchy the primary header unit (PHU) of this 525 file belongs. Legal values are \code{FPA}, \code{CHIP} or 526 \code{CELL}. 527 \item \code{EXTENSIONS(STR)} identifies what level in the focal plane 528 hierarchy the extensions belong. Legal values are \code{CHIP}, 529 \code{CELL} or \code{NONE} (if there are no extensions). 530 \item \code{FPA.NAME(STR)} specifies a PHU header keyword for a unique 531 identifier for the FPA. This is usually an exposure number, or 532 similar. The purpose is to identify the FPA, so that only files 533 with the same value of \code{FPA.NAME} can be admitted to the same 534 FPA structure. 535 \item \code{CHIP.NAME(STR)} (necessary if \code{PHU} is \code{CHIP} or 536 \code{CELL}) specifies a PHU header keyword that identifies the name 537 of the chip. The purpose is to identify to which chip in the 538 hierarchy the file belongs. 539 \item \code{CELL.NAME(STR)} (necessary if \code{PHU} is \code{CELL}) 540 specifies a PHU header keyword that identifies the name of the cell 541 within the chip. The purpose is to identify to which cell in the 542 hierarchy the file belongs. 543 \item \code{CONTENT(STR)} (necessary if \code{EXTENSIONS} is 544 \code{NONE} and \code{PHU} is \code{CHIP} or \code{CELL}) specifies 545 a key to the \code{CONTENTS} menu (see below). The purpose is to 546 identify the contents of the file (in terms of its FPA hierarchy 547 components). The string has concepts interpolated, where these are 548 enclosed in curly brackets (currently \code{CHIP.NAME} and 549 \code{CELL.NAME} only; \tbd{future concepts may be permitted in the 550 future if there exists sufficient demand}. This allows such a 551 construct as \code{\{CHIP.NAME\}_\{CELL.NAME\}} to identify a 552 combination of chip and cell. 598 553 \end{itemize} 599 554 600 An example camera configuration file: 555 \subsubsection{File contents} 556 557 The exact meaning of the \code{CONTENTS} (as well as the type) depends 558 on the value of \code{PHU} and \code{EXTENSIONS} in the \code{FILE} 559 metadata. In each case, we rely on the use of \code{chip:cell:type} 560 triplets to identify the contents. These are used to identify the 561 contents of an extension: the chip and cell to which a component 562 belongs, and the type of the cell (see \S\ref{sec:cell_data} for cell 563 types), with the symbolic names separated by colons. The triplets may 564 be listed one after the other, separated by whitespace, where an 565 extension contains more than one cell. 566 567 \begin{itemize} 568 \item If \code{PHU} is \code{FPA} and \code{EXTENSIONS} is 569 \code{NONE}, then \code{CONTENTS} is of type \code{STR}, and 570 contains a string of \code{chip:cell:type} triplets. 571 \item If \code{PHU} is \code{CHIP} or \code{CELL} and 572 \code{EXTENSIONS} is \code{NONE}, then \code{CONTENTS} is of type 573 \code{METADATA}, and contains a menu of possible contents. Each 574 menu item is of type \code{STR}, and consists of a string of 575 \code{chip:cell:type} triplets. The menu key is provided by the 576 interpolated \code{CONTENT} value within the \code{FILE} metadata. 577 \item In all other cases, \code{CONTENTS} is of type \code{METADATA}, 578 and contains a list of extension names within the file, with the 579 values of type \code{STR} consisting of a string of 580 \code{chip:cell:type} triplets. 581 \end{itemize} 582 583 \subsubsection{Cell data} 584 \label{sec:cell_data} 585 586 \code{CELLS(METADATA)} contains a list of cell types, with concepts 587 particular to those types. Each type, which corresponds to a type 588 specified in the \code{CONTENTS}, is of type \code{METADATA}. The 589 contents of these metadata are values for concepts that are particular 590 to that cell type (e.g., left amplifier vs right amplifier). Usually 591 \code{CELL.TRIMSEC(STR)} and \code{CELL.BIASSEC(STR)} will be listed 592 here, since these differ according to the cell type. Since there is 593 ambiguity in what the values here refer to (if the concept is of type 594 \code{STR}), we also require an additional entry with \code{.SOURCE} 595 suffixed to the concept name, with the value (of type \code{STR}) 596 being \code{VALUE} to indicate that the concept is specified by value, 597 or \code{HEADER} to indicate that the concept is specified in the 598 header of the given name. 599 600 [It might be thought that there is no need to provide the ability to 601 look up headers here, since it is provided below. However, the header 602 name may vary depending on the cell type. For example, the Megacam 603 spliced format uses \code{TSECA} and \code{TSECB} to specify the trim 604 sections for the left and right amplifiers.] 605 606 \subsubsection{Concepts from headers} 607 608 \code{TRANSLATION(METADATA)} contains a list of concepts that have 609 their values ingested from the FITS headers. Each concept name should 610 have type \code{STR}, with the value being the header name from which 611 the concept is ingested. No distinction is made between the PHU and 612 extension headers, but inheritance (look at the PHU if it's not in the 613 extension header) should be the normal behaviour. Multiple headers 614 may be given for certain concepts: 615 \begin{itemize} 616 \item \code{FPA.TIME} and \code{CELL.TIME} to specify the date and 617 time in separate headers 618 \item \code{CELL.BIASSEC} to specify multiple bias regions (e.g., a 619 prescan and an overscan). 620 \end{itemize} 621 622 \tbd{TRANSLATION is a poor name (it's supposed to be a header 623 translation table); HEADERS would be better.} 624 625 \subsubsection{Concepts from default values} 626 627 \code{DEFAULTS(METADATA)} contains a list of concepts with their 628 default values (of the appropriate types). A concept may have type 629 \code{METADATA}, in which case the metadata acts as a menu. The menu 630 key is determined from an additional entry in the \code{DEFAULTS}, 631 formed from the concept name suffixed with \code{.DEPEND}, which must 632 be of type \code{STR} and contain a concept name. The value of this 633 extra concept determines the menu key. This allows dependence on the 634 chip (e.g., depending on \code{CHIP.NAME}) or cell (\code{CELL.NAME}), 635 which is useful for setting things such as \code{CHIP.X0} when it is 636 not contained in the header. 637 638 \subsubsection{Concepts from database} 639 640 \tbd{Database lookup for concepts has never been tested. In fact, the 641 current implementation probably doesn't even match this description.} 642 643 \code{DATABASE(METADATA)} contains a list of concepts whose values are 644 determined from database lookup. Each concept is of type 645 \code{METADATA}. Each concept metadata must contain the entries 646 \code{TABLE(STR)} and \code{COLUMN(STR)}, which specify the database 647 table to use, and the column within that table. Additional entries 648 provide the \code{WHERE} part of the database query. 649 650 651 \subsubsection{Formats for concepts} 652 653 \code{FORMATS(METADATA)} contains a list of concepts that require 654 additional information in order to parse. Each concept name contains 655 a value of type \code{STR} which is a list of options for parsing the 656 concept. 657 658 Concepts which require formats: 659 \begin{itemize} 660 \item \code{FPA.RA} and \code{FPA.DEC}: the format specifies the units 661 --- \code{HOURS}, \code{DEGREES} or \code{RADIANS}. \code{FPA.RA} 662 defaults to \code{HOURS}, and \code{FPA.DEC} defaults to 663 \code{DEGREES}. 664 \item \code{FPA.TIME} and \code{CELL.TIME}: \code{USA} indicates that 665 the date format is mm-dd-yyyy; \code{BACKWARDS} indicates that the 666 date format is dd-mm-yyyy; \code{PRE2000} indicates that a two-digit 667 date is used (1900 years is added if the year is less than 100); 668 \code{MJD} indicates the date is a modified julian date; \code{JD} 669 indicates the date is a julian date. 670 \item \code{CELL.X0}, \code{CELL.Y0}, \code{CHIP.X0} and 671 \code{CHIP.Y0}: \code{FORTRAN} indicates that the corner corresponds 672 to corner (1,1); if missing, assumes that the corner is at (0,0). 673 \end{itemize} 674 675 \subsubsection{Default concepts} 676 677 Default concepts that should be included in each camera format file, 678 either in the \code{CELLS}, \code{TRANSLATION}, \code{DEFAULTS} or 679 \code{DATABASE}: 680 \begin{itemize} 681 \item \code{FPA.CAMERA}: Camera used 682 \item \code{FPA.FOCUS}: Telescope focus 683 \item \code{FPA.AIRMASS}: Airmass at boresight 684 \item \code{FPA.FILTER}: Filter used 685 \item \code{FPA.POSANGLE}: Position angle of instrument 686 \item \code{FPA.RADECSYS}: Celestial coordinate system 687 \item \code{FPA.RA}: Right Ascension of boresight 688 \item \code{FPA.DEC}: Declination of boresight 689 \item \code{FPA.OBSTYPE}: Type of observation 690 \item \code{FPA.OBJECT}: Object of observation 691 \item \code{FPA.ALT}: Altitude of telescope 692 \item \code{FPA.AZ}: Azimuth of telescope 693 \item \code{FPA.TIMESYS}: Time system 694 \item \code{FPA.TIME}: Time of exposure 695 \item \code{CHIP.XPARITY}: Orientation in x compared to the rest of the FPA 696 \item \code{CHIP.YPARITY}: Orientation in y compared to the rest of the FPA 697 \item \code{CHIP.X0}: Position of (0,0) on the FPA 698 \item \code{CHIP.Y0}: Position of (0,0) on the FPA 699 \item \code{CHIP.TEMP}: Temperature of chip 700 \item \code{CELL.GAIN}: CCD gain (e/count) 701 \item \code{CELL.READNOISE}: CCD read noise (e) 702 \item \code{CELL.SATURATION}: Saturation level (counts) 703 \item \code{CELL.BAD}: Bad level (counts) 704 \item \code{CELL.XPARITY}: Orientation in x compared to the rest of the chip 705 \item \code{CELL.YPARITY}: Orientation in y compared to the rest of the chip 706 \item \code{CELL.READDIR}: Read direction, rows=1, cols=2 707 \item \code{CELL.EXPOSURE}: Exposure time (sec) 708 \item \code{CELL.DARKTIME}: Time since flush (sec) 709 \item \code{CELL.TRIMSEC}: Trim section 710 \item \code{CELL.BIASSEC}: Bias sections 711 \item \code{CELL.XBIN}: Binning in x 712 \item \code{CELL.YBIN}: Binning in y 713 \item \code{CELL.TIMESYS}: Time system 714 \item \code{CELL.TIME}: Time of exposure 715 \item \code{CELL.X0}: Position of (0,0) on the chip 716 \item \code{CELL.Y0}: Position of (0,0) on the chip 717 \end{itemize} 718 719 In addition, \code{FPA.NAME}, \code{CHIP.NAME} and \code{CELL.NAME} 720 are included automatically, based on the \code{FILE} and 721 \code{CONTENTS} metadatas. 722 723 \subsection{Examples} 724 725 \subsubsection{Megacam (short) raw} 601 726 602 727 \begin{verbatim} 603 # Camera configuration file for MegaCam: describes the camera 604 605 # File formats that we know about 606 FORMATS METADATA 607 RAW STR megacam_raw.config 608 SPLICE STR megacam_splice.config 609 SPLIT STR megacam_split.config 610 END 611 612 613 # Description of camera --- all the chips and the cells that comprise them 614 FPA METADATA 615 ccd00 STR left right 616 ccd01 STR left right 617 ccd02 STR left right 618 ccd03 STR left right 619 ccd04 STR left right 620 ccd05 STR left right 621 ccd06 STR left right 622 ccd07 STR left right 623 ccd08 STR left right 624 ccd09 STR left right 625 ccd10 STR left right 626 ccd11 STR left right 627 ccd12 STR left right 628 ccd13 STR left right 629 ccd14 STR left right 630 ccd15 STR left right 631 ccd16 STR left right 632 ccd17 STR left right 633 ccd18 STR left right 634 ccd19 STR left right 635 ccd20 STR left right 636 ccd21 STR left right 637 ccd22 STR left right 638 ccd23 STR left right 639 ccd24 STR left right 640 ccd25 STR left right 641 ccd26 STR left right 642 ccd27 STR left right 643 ccd28 STR left right 644 ccd29 STR left right 645 ccd30 STR left right 646 ccd31 STR left right 647 ccd32 STR left right 648 ccd33 STR left right 649 ccd34 STR left right 650 ccd35 STR left right 651 END 652 653 654 # Recipe options 655 RECIPES METADATA 656 PHASE2 STR phase2.config # Phase 2 recipe details 657 PSPHOT STR psphot.config # psphot details 728 # "mcshort" is a MegaCam camera with only the central six chips --- it's faster than the entire FPA. 729 # The raw MegaCam data comes off the telescope with each of the chips stored in extensions of a MEF file. 730 731 # How to identify this type 732 RULE METADATA 733 TELESCOP STR CFHT 3.6m 734 DETECTOR STR MegaCam 735 EXTEND BOOL T 736 NEXTEND S32 72 737 END 738 739 # How to read this data 740 FILE METADATA 741 PHU STR FPA # The FITS file represents an entire FPA 742 EXTENSIONS STR CELL # The extensions represent cells 743 FPA.NAME STR EXPNUM # A PHU keyword for unique identifier within the hierarchy level 744 END 745 746 # What's in the FITS file? 747 CONTENTS METADATA 748 # Extension name, chip:cell:type 749 amp24 STR ccd12:LeftAmp:left 750 amp25 STR ccd12:RightAmp:right 751 amp26 STR ccd13:LeftAmp:left 752 amp27 STR ccd13:RightAmp:right 753 amp28 STR ccd14:LeftAmp:left 754 amp29 STR ccd14:RightAmp:right 755 amp42 STR ccd21:LeftAmp:left 756 amp43 STR ccd21:RightAmp:right 757 amp44 STR ccd22:LeftAmp:left 758 amp45 STR ccd22:RightAmp:right 759 amp46 STR ccd23:LeftAmp:left 760 amp47 STR ccd23:RightAmp:right 761 END 762 763 # Specify the cell data 764 CELLS METADATA 765 left METADATA # Left amplifier 766 CELL.BIASSEC.SOURCE STR HEADER 767 CELL.TRIMSEC.SOURCE STR HEADER 768 CELL.BIASSEC STR BIASSEC 769 CELL.TRIMSEC STR DATASEC 770 CELL.XPARITY S32 1 # We could have specified this as a DEFAULT, but this works 771 CELL.X0 S32 1 772 END 773 right METADATA # Right amplifier 774 CELL.BIASSEC.SOURCE STR HEADER 775 CELL.TRIMSEC.SOURCE STR HEADER 776 CELL.BIASSEC STR BIASSEC 777 CELL.TRIMSEC STR DATASEC 778 CELL.XPARITY S32 -1 # This cell is read out in the opposite direction 779 CELL.X0 S32 2048 780 END 781 END 782 783 # How to translate PS concepts into FITS headers 784 TRANSLATION METADATA 785 FPA.NAME STR EXPNUM 786 FPA.AIRMASS STR AIRMASS 787 FPA.FILTER STR FILTER 788 FPA.POSANGLE STR ROTANGLE 789 FPA.RA STR RA 790 FPA.DEC STR DEC 791 FPA.RADECSYS STR RADECSYS 792 FPA.OBSTYPE STR OBSTYPE 793 FPA.OBJECT STR CMMTOBS 794 FPA.TIME STR MJD-OBS 795 FPA.TIMESYS STR TIMESYS 796 FPA.ALT STR TELALT 797 FPA.AZ STR TELAZ 798 CHIP.TEMP STR DETTEM 799 CELL.EXPOSURE STR EXPTIME 800 CELL.DARKTIME STR DARKTIME 801 CELL.GAIN STR GAIN 802 CELL.READNOISE STR RDNOISE 803 CELL.SATURATION STR SATURATE 804 CELL.TIME STR MJD-OBS 805 CELL.TIMESYS STR TIMESYS 806 CELL.XBIN STR CCDBIN1 807 CELL.YBIN STR CCDBIN2 808 END 809 810 # Default PS concepts that may be specified by value 811 DEFAULTS METADATA 812 CELL.READDIR S32 1 # Cell is read in x direction 813 CELL.BAD S32 0 814 CELL.YPARITY S32 1 815 CELL.Y0 S32 1 816 817 CHIP.X0.DEPEND STR CHIP.NAME 818 CHIP.X0 METADATA 819 ccd12 S32 6144 820 ccd13 S32 8192 821 ccd14 S32 10240 822 ccd21 S32 6144 823 ccd22 S32 8192 824 ccd23 S32 10240 825 END 826 CHIP.Y0.DEPEND STR CHIP.NAME 827 CHIP.Y0 METADATA 828 ccd12 S32 13835 829 ccd13 S32 13835 830 ccd14 S32 13835 831 ccd21 S32 4612 832 ccd22 S32 4612 833 ccd23 S32 4612 834 END 835 CHIP.XPARITY.DEPEND STR CHIP.NAME 836 CHIP.XPARITY METADATA 837 ccd12 S32 1 838 ccd13 S32 1 839 ccd14 S32 1 840 ccd21 S32 1 841 ccd22 S32 1 842 ccd23 S32 1 843 END 844 CHIP.YPARITY.DEPEND STR CHIP.NAME 845 CHIP.YPARITY METADATA 846 ccd12 S32 -1 847 ccd13 S32 -1 848 ccd14 S32 -1 849 ccd21 S32 1 850 ccd22 S32 1 851 ccd23 S32 1 852 END 853 END 854 855 # How to translate PS concepts into database lookups 856 DATABASE METADATA 857 TYPE dbLookup TABLE COLUMN chipId cellId 858 # CHIP.TEMP METADATA 859 # TABLE STR Cryostat 860 # COLUMN STR temp 861 # chipId STR {CHIP.NAME} 862 # time STR {CELL.TIME} 863 # END 864 # CELL.GAIN dbLookup Camera gain CHIP.NAME CELL.NAME 865 # CELL.READNOISE dbLookup Camera readNoise CHIP.NAME CELL.NAME 866 END 867 868 869 # Where there might be some ambiguity, specify the format 870 FORMATS METADATA 871 FPA.RA STR HOURS 872 FPA.DEC STR DEGREES 873 FPA.TIME STR MJD 874 CELL.TIME STR MJD 875 CELL.X0 STR FORTRAN 876 CELL.Y0 STR FORTRAN 658 877 END 659 878 \end{verbatim} 660 879 661 662 \subsubsection{FITS file format} 663 664 The FITS file format configuration files are somewhat complicated and 665 involved, since they must not only specify how to translate the pixels 666 from a FITS file into a focal plane hierarchy 667 (\S\ref{sec:focalplane}), but must also specify how to derive the 668 various values the IPP needs (\S\ref{sec:concepts}). Moreover, they 669 must be able to do these for the great variety of cameras in use in 670 the astronomical community. 671 672 Example camera configuration files are included in an appendix, but 673 below we explain the components. 674 675 \paragraph{FITS File to Focal Plane Hierarchy} 676 677 The Focal Plane hierarchy (\code{pmFPA, pmChip, pmCell, pmReadout}) is 678 explained in more detail in \S\ref{sec:focalplane}. The top level, an 679 FPA contains one or more chips, which correspond to a contiguous piece 680 of silicon. A chip contains one or more cells, which correspond to a 681 single amplifier. A cell contains one or more readouts, which 682 correspond to individual reads of the detector. 683 684 The FITS data storage formation is a standard in the astronomical 685 community for storing astronomical images. A FITS file consists of an 686 arbitrary number of coupled human readable \code{ASCII} header 687 segments and binary data segments. The headers describe the format 688 and layout of the data segments. The first of these groups is 689 traditionally called the ``primary header unit'' (PHU) and the rest are 690 referred to as ``extensions''. The header segments may contain 691 extensive documentary information related to the interpretation of the 692 data. Although the FITS format defines a standard representation of 693 the data, the header metadata is not so consistently defined within 694 the astronomical community. Also, the flexibility of the data format 695 means that different representations are possible for the same 696 fundamental collection of data. The tools presented in this section 697 provide a method to define and constrain the wide range of possible 698 FITS representations of astronomical images. 699 700 Within the FITS data representation, there are various choices which 701 can and have been made for the placement of the pixels in the file. 702 In the simplest case, the camera consists of a single chip consisting 703 of a single cell always read with a single readout. In this case, the 704 image data could be written as part of the primary header unit. In a 705 more complex case with multiple chips and multiple cells, the data may 706 be organized in several ways. The data may be distributed into 707 multiple files or in multiple FITS data extensions. A single camera 708 image may be written as a collection of files for individual chips 709 with separate extensions for each cell (CFH12K.split, GPC). Another 710 camera may write a single file with multiple extensions for each cell 711 (Megacam.raw), or multiple extensions per chip, with each cell 712 representing portions of the chip image (Megacam.splice, CFHT-IR). 713 714 In all of these representations, there are only two basic distinctions 715 in how the pixel data is stored: what level in the hierarchy the 716 entire FITS file corresponds to (FPA, chip, or cell), and what level 717 the extensions correspond to (chip, cell or no extensions at all). 718 Knowing these, and having a list of the extensions, we can construct 719 the focal plane hierarchy. 720 721 Note that a single data extension, consisting of a uniform grid of 722 pixels, can only naturally represent a cell or a chip. In order to 723 represent the entire focal plane array as a single grid, some 724 artificial choices would be made to fill-in or ignore the gaps between 725 chips and their relative rotations. Within our framework, a complete 726 focal plane mosaic of multiple chips could be represented as a single 727 extension by treating the collection of pixels as if they were from a 728 single chip. 729 730 To define the hierarchy, we specify the following keywords: 731 \begin{itemize} 732 \item \code{RULE} of type \code{METADATA}: contains headers with their 733 respective values that are required to be in the PHU of any FITS 734 file of this type. 735 736 \item \code{FILE} of type \code{METADATA}: contains information on 737 the global format of the FITS file with the following entries: 738 \begin{itemize} 739 \item \code{PHU} of type \code{STR}: May be one of \code{FPA}, 740 \code{CHIP}, or \code{CHIP}. This specifies the focal plane level 741 of the Primary Header Unit, and hence the entire FITS file (the 742 'class' of the file). 743 744 \item \code{EXTENSIONS} of type \code{STR}: May be one of 745 \code{CHIP}, \code{CELL} or \code{NONE}, though not of a level 746 higher than that specified by the \code{PHU}. This specifies what 747 each extension represents. 748 749 \item \code{FPA.NAME} of type \code{STR}: Specifies a header keyword 750 in the primary header for a unique identifier for the FPA name 751 (e.g., an observation number). 752 753 \item \code{CHIP.NAME} of type \code{STR}: Need only be included if 754 \code{PHU} is \code{CHIP} or \code{CELL}. Specifies a header 755 keyword in the primary header for a unique identifier for the chip 756 name (e.g., the CCD identification number or name). 757 758 \item \code{CELL.NAME} of type \code{STR}: Need only be included if 759 \code{PHU} is \code{CELL}. Specifies a header keyword in the 760 primary header for a unique identifier for the cell name (e.g., 761 the amplifier identification). 762 \end{itemize} 763 764 \item \code{CONTENTS} of type \code{METADATA}: Specifies what the 765 contents of the FITS file are. Each entry is an extension name with 766 the corresponding value being a string listing the source and the 767 cell type, separated by a colon (e.g., \code{ccd01:left 768 ccd01:right}). If \code{EXTENSIONS=NONE} then the \code{CONTENTS} 769 is ignored (since there are no extensions to list). 770 771 \item \code{CELLS} of type \code{METADATA}: specifies the cell types. 772 Entries are the cell types, each of type \code{METADATA}, with the 773 values being PS concept values appropriate for each cell type (more 774 detail later) \tbd{link to more detail}. In the event that 775 \code{EXTENSIONS=NONE}, the \code{CELLS} is used as a list of all 776 cells present in the file. 777 778 \item \code{TRANSLATION} of type \code{METADATA} 779 780 \item \code{DEFAULTS} of type \code{METADATA} 781 782 \item \code{DATABASE} of type \code{METADATA} 783 784 \item \code{FORMATS} of type \code{METADATA} 785 786 \end{itemize} 787 788 An example: 880 \subsubsection{Megacam (short) split} 789 881 790 882 \begin{verbatim} 791 # The raw MegaCam data comes off the telescope with each of the chips stored in extensions of a MEF file. 792 793 # How to identify this type 794 RULE METADATA 795 TELESCOP STR CFHT 3.6m 796 DETECTOR STR MegaCam 797 EXTEND BOOL T 798 NEXTEND S32 72 799 END 800 801 # How to read this data 802 FORMAT METADATA 803 PHU STR FPA # The FITS file represents an entire FPA 804 EXTENSIONS STR CELL # The extensions represent cells 805 FPA.NAME STR EXPNUM # A PHU keyword for unique identifier within the hierarchy level 883 # "mcshort" is a MegaCam camera with only the central six chips --- it's faster than the entire FPA. 884 # The spliced MecaCam data is stored in single extensions for each chip 885 886 # How to recognise this type 887 RULE METADATA 888 TELESCOP STR CFHT 3.6m 889 DETECTOR STR MegaCam 890 # No particular distinguishing features apart from these, so we list this format last 891 # in the camera configuration file. 892 END 893 894 FILE METADATA 895 # How to read this data 896 PHU STR CHIP # The FITS file represents an entire FPA 897 EXTENSIONS STR NONE # The extensions represent chips 898 FPA.NAME STR EXPNUM # A PHU keyword for unique identifier 899 CHIP.NAME STR EXTNAME # An extension keyword for unique identifie 900 CONTENT STR {CHIP.NAME} # Key to the CONTENTS menu 806 901 END 807 902 808 903 # What's in the FITS file? 809 CONTENTS METADATA 810 # Extension name, chip name:type 811 amp00 STR ccd00:left 812 amp01 STR ccd00:right 813 amp02 STR ccd01:left 814 amp03 STR ccd01:right 815 amp04 STR ccd02:left 816 amp05 STR ccd02:right 817 amp06 STR ccd03:left 818 amp07 STR ccd03:right 819 amp08 STR ccd04:left 820 amp09 STR ccd04:right 821 amp10 STR ccd05:left 822 amp11 STR ccd05:right 823 amp12 STR ccd06:left 824 amp13 STR ccd06:right 825 amp14 STR ccd07:left 826 amp15 STR ccd07:right 827 amp16 STR ccd08:left 828 amp17 STR ccd08:right 829 amp18 STR ccd09:left 830 amp19 STR ccd09:right 831 amp20 STR ccd10:left 832 amp21 STR ccd10:right 833 amp22 STR ccd11:left 834 amp23 STR ccd11:right 835 amp24 STR ccd12:left 836 amp25 STR ccd12:right 837 amp26 STR ccd13:left 838 amp27 STR ccd13:right 839 amp28 STR ccd14:left 840 amp29 STR ccd14:right 841 amp30 STR ccd15:left 842 amp31 STR ccd15:right 843 amp32 STR ccd16:left 844 amp33 STR ccd16:right 845 amp34 STR ccd17:left 846 amp35 STR ccd17:right 847 amp36 STR ccd18:left 848 amp37 STR ccd18:right 849 amp38 STR ccd19:left 850 amp39 STR ccd19:right 851 amp40 STR ccd20:left 852 amp41 STR ccd20:right 853 amp42 STR ccd21:left 854 amp43 STR ccd21:right 855 amp44 STR ccd22:left 856 amp45 STR ccd22:right 857 amp46 STR ccd23:left 858 amp47 STR ccd23:right 859 amp48 STR ccd24:left 860 amp49 STR ccd24:right 861 amp50 STR ccd25:left 862 amp51 STR ccd25:right 863 amp52 STR ccd26:left 864 amp53 STR ccd26:right 865 amp54 STR ccd27:left 866 amp55 STR ccd27:right 867 amp56 STR ccd28:left 868 amp57 STR ccd28:right 869 amp58 STR ccd29:left 870 amp59 STR ccd29:right 871 amp60 STR ccd30:left 872 amp61 STR ccd30:right 873 amp62 STR ccd31:left 874 amp63 STR ccd31:right 875 amp64 STR ccd32:left 876 amp65 STR ccd32:right 877 amp66 STR ccd33:left 878 amp67 STR ccd33:right 879 amp68 STR ccd34:left 880 amp69 STR ccd34:right 881 amp70 STR ccd35:left 882 amp71 STR ccd35:right 883 END 884 885 # Specify the cell data 886 CELLS METADATA 887 left METADATA # Left amplifier 888 CELL.NAME STR LeftSide 889 CELL.BIASSEC.SOURCE STR HEADER 890 CELL.TRIMSEC.SOURCE STR HEADER 891 CELL.BIASSEC STR BIASSEC 892 CELL.TRIMSEC STR DATASEC 893 CELL.XPARITY S32 1 # We could have specified this as a DEFAULT, but this works 894 CELL.X0 S32 1 895 CELL.Y0 S32 1 896 END 897 right METADATA # Right amplifier 898 CELL.NAME STR RightSide 899 CELL.BIASSEC.SOURCE STR HEADER 900 CELL.TRIMSEC.SOURCE STR HEADER 901 CELL.BIASSEC STR BIASSEC 902 CELL.TRIMSEC STR DATASEC 903 CELL.XPARITY S32 -1 # This cell is read out in the opposite direction 904 CELL.X0 S32 2048 905 CELL.Y0 S32 1 906 END 907 END 908 909 # How to translate PS concepts into FITS headers 910 TRANSLATION METADATA 911 FPA.NAME STR EXPNUM 912 FPA.AIRMASS STR AIRMASS 913 FPA.FILTER STR FILTER 914 FPA.POSANGLE STR ROTANGLE 915 FPA.RA STR RA 916 FPA.DEC STR DEC 917 FPA.RADECSYS STR RADECSYS 918 CELL.EXPOSURE STR EXPTIME 919 CELL.DARKTIME STR DARKTIME 920 CELL.GAIN STR GAIN 921 CELL.READNOISE STR RDNOISE 922 CELL.SATURATION STR SATURATE 923 CELL.TIME STR MJD-OBS 924 CELL.XBIN STR CCDBIN1 925 CELL.YBIN STR CCDBIN2 926 END 927 928 # Default PS concepts that may be specified by value 929 DEFAULTS METADATA 930 CELL.READDIR S32 1 # Cell is read in x direction 931 CELL.BAD S32 0 932 CELL.TIMESYS STR UTC 933 CELL.YPARITY S32 1 934 END 935 936 # How to translation PS concepts into database lookups 937 DATABASE METADATA 938 TYPE dbEntry TABLE COLUMN GIVENDBCOL GIVENPS 939 # FPA.BIAS METADATA 940 # TABLE STR Camera 941 # COLUMN STR gain 942 # chipId STR {CHIP.NAME} 943 # cellId STR {CELL.NAME} 944 # time STR {CELL.TIME} 945 # END 946 # CELL.GAIN dbEntry Camera gain chipId,cellId CHIP.NAME,CELL.NAME 947 # CELL.READNOISE dbEntry Camera readNoise chipId,cellId CHIP.NAME,CELL.NAME 948 949 # A database entry refers to a particular column (COLUMN) in a 950 # particular table (TABLE), given certain PS concepts (GIVENPS) that 951 # match certain database columns (GIVENDBCOL). 952 END 953 954 955 # Where there might be some ambiguity, specify the format 956 FORMATS METADATA 957 FPA.RA STR HOURS 958 FPA.DEC STR DEGREES 959 CELL.TIME STR MJD 960 # CELL.BINNING STR SEPARATE 961 CELL.X0 STR FORTRAN 962 CELL.Y0 STR FORTRAN 963 END 964 \end{verbatim} 965 966 Observe how the \code{CONTENTS} specifies the extension name, which we 967 know from the \code{EXTENSIONS} is a cell, and that each extension is 968 associated with a chip, and has a cell type. 969 970 \paragraph{Deriving concept values} 971 \label{sec:derivingconcepts} 972 973 The PS concepts are described in more detail in \S\ref{sec:concepts}. 974 Basically, astronomical cameras generally store the important details 975 (``concepts'') in different ways. This is generally manifested in the 976 choice of different FITS header keywords to describe the same concept, 977 but one can also imagine deriving values from a database or a known 978 default. 979 980 We therefore specify the following keywords: 981 \begin{itemize} 982 \item \code{TRANSLATION} of type \code{METADATA} is a translation 983 table for understanding PS concepts in terms of FITS headers. The 984 PS concept (keyword) is derived from the FITS header given in the 985 value. 986 \item \code{DATABASE} of type \code{METADATA} is a formula for 987 obtaining a PS concept from the database. Each component is of a 988 user-specified type containing \code{TABLE}, \code{COLUMN}, 989 \code{GIVENDBCOL} and \code{GIVENPS}. The idea is that to obtain 990 the value of a PS concept, one refers to a particular \code{COLUMN} 991 in a particular \code{TABLE}, where the value of certain PS concepts 992 (\code{GIVENPS}; multiple values separated by a comma or semicolon) 993 match certain database columns (\code{GIVENDBCOL}; multiple values 994 separated by a comma or semicolon). 995 \item \code{DEFAULTS} of type \code{METADATA} is a set of default 996 values of PS concepts for the camera. The PS concept (keyword) is 997 assigned the value. There is also limited dependency allowed; see 998 \S\ref{sec:concepts}. 999 \end{itemize} 1000 1001 An example: 1002 \begin{verbatim} 904 CONTENTS METADATA 905 # Extension name, chip:cell:type 906 ccd12 STR ccd12:LeftAmp:left ccd12:RightAmp:right 907 ccd13 STR ccd13:LeftAmp:left ccd13:RightAmp:right 908 ccd14 STR ccd14:LeftAmp:left ccd14:RightAmp:right 909 ccd21 STR ccd21:LeftAmp:left ccd21:RightAmp:right 910 ccd22 STR ccd22:LeftAmp:left ccd22:RightAmp:right 911 ccd23 STR ccd23:LeftAmp:left ccd23:RightAmp:right 912 END 913 914 # Specify the cells 915 CELLS METADATA 916 left METADATA 917 CELL.BIASSEC.SOURCE STR HEADER 918 CELL.TRIMSEC.SOURCE STR HEADER 919 CELL.BIASSEC STR BSECA 920 CELL.TRIMSEC STR TSECA 921 CELL.X0 S32 0 922 CELL.GAIN.SOURCE STR HEADER 923 CELL.GAIN STR GAINA 924 END 925 926 right METADATA 927 CELL.BIASSEC.SOURCE STR HEADER 928 CELL.TRIMSEC.SOURCE STR HEADER 929 CELL.BIASSEC STR BSECB 930 CELL.TRIMSEC STR TSECB 931 CELL.X0 S32 1024 932 CELL.GAIN.SOURCE STR HEADER 933 CELL.GAIN STR GAINB 934 END 935 END 936 1003 937 # How to translate PS concepts into FITS headers 1004 938 TRANSLATION METADATA … … 1010 944 FPA.DEC STR DEC 1011 945 FPA.RADECSYS STR RADECSYS 1012 FPA.MJD STR MJD-OBS 946 FPA.OBSTYPE STR OBSTYPE 947 FPA.OBJECT STR CMMTOBS 948 FPA.TIME STR MJD-OBS 949 FPA.TIMESYS STR TIMESYS 950 FPA.ALT STR TELALT 951 FPA.AZ STR TELAZ 952 CHIP.TEMP STR DETTEM 1013 953 CELL.EXPOSURE STR EXPTIME 1014 954 CELL.DARKTIME STR DARKTIME 955 CELL.READNOISE STR RDNOISE 956 CELL.SATURATION STR SATURATE 957 CELL.TIME STR MJD-OBS 958 CELL.TIMESYS STR TIMESYS 1015 959 CELL.XBIN STR CCDBIN1 1016 960 CELL.YBIN STR CCDBIN2 1017 CELL.SATURATION STR SATURATE1018 961 END 1019 962 1020 963 # Default PS concepts that may be specified by value 1021 964 DEFAULTS METADATA 965 CELL.READDIR S32 1 # Cell is read in x direction 1022 966 CELL.BAD S32 0 1023 CELL.PARITY.DEPEND STR CHIP.NAME 1024 CELL.PARITY METADATA 1025 amp00 S32 1 1026 amp01 S32 -1 1027 amp02 S32 1 1028 amp03 S32 -1 967 CELL.XPARITY S32 1 968 CELL.YPARITY S32 1 969 CELL.Y0 S32 0 970 # PPMERGE.SCALE F32 1.0 971 # PPMERGE.ZERO F32 0.0 972 CHIP.X0.DEPEND STR CHIP.NAME 973 CHIP.X0 METADATA 974 ccd12 S32 0 975 ccd13 S32 2048 976 ccd14 S32 4096 977 ccd21 S32 0 978 ccd22 S32 2048 979 ccd23 S32 4096 1029 980 END 1030 END 1031 1032 # How to translate PS concepts into database lookups 981 CHIP.Y0.DEPEND STR CHIP.NAME 982 CHIP.Y0 METADATA 983 ccd12 S32 9223 984 ccd13 S32 9223 985 ccd14 S32 9223 986 ccd21 S32 0 987 ccd22 S32 0 988 ccd23 S32 0 989 END 990 CHIP.XPARITY.DEPEND STR CHIP.NAME 991 CHIP.XPARITY METADATA 992 ccd12 S32 1 993 ccd13 S32 1 994 ccd14 S32 1 995 ccd21 S32 1 996 ccd22 S32 1 997 ccd23 S32 1 998 END 999 CHIP.YPARITY.DEPEND STR CHIP.NAME 1000 CHIP.YPARITY METADATA 1001 ccd12 S32 -1 1002 ccd13 S32 -1 1003 ccd14 S32 -1 1004 ccd21 S32 1 1005 ccd22 S32 1 1006 ccd23 S32 1 1007 END 1008 END 1009 1010 1011 # How to translation PS concepts into database lookups 1033 1012 DATABASE METADATA 1034 TYPE dbEntry TABLE COLUMN GIVENDBCOL GIVENPS 1035 CELL.GAIN dbEntry Camera gain chipId,cellId CHIP.NAME,CELL.NAME 1036 CELL.READNOISE dbEntry Camera readNoise chipId,cellId CHIP.NAME,CELL.NAME 1013 # None 1014 END 1015 1016 1017 # Where there might be some ambiguity, specify the format 1018 FORMATS METADATA 1019 FPA.RA STR HOURS 1020 FPA.DEC STR DEGREES 1021 FPA.TIME STR MJD 1022 CELL.TIME STR MJD 1037 1023 END 1038 1024 \end{verbatim} 1039 1025 1040 The \code{.DEPEND} entry in the \code{DEFAULTS} will be explained in 1041 \S\ref{sec:concepts}. 1042 1043 \paragraph{Indentification by rule} 1044 \label{sec:camerarule} 1045 1046 The function \code{pmConfigCameraFromHeader} requires that the camera 1047 configuration also contains a rule on how to recognise that a FITS 1048 header comes from that camera. 1049 1050 We therefore specify another keyword: \code{RULE} of type 1051 \code{METADATA}: Contains a list of FITS headers keywords and values 1052 (of the appropriate type) against which actual headers are compared to 1053 determine if it matches the camera type. 1054 1055 An example is: 1026 \subsubsection{Imaging Sky Probe} 1027 1056 1028 \begin{verbatim} 1029 # Pan-STARRS Imaging Sky Probe 1030 1057 1031 # How to identify this type 1058 1032 RULE METADATA 1059 TELESCOP STR CFHT 3.6m 1060 DETECTOR STR MegaCam 1061 EXTEND BOOL T 1062 NEXTEND S32 72 1063 END 1064 \end{verbatim} 1065 1066 \paragraph{Recipes} 1067 1068 The camera configuration file must also contain filenames for the 1069 recipe configuration files. We include \code{RECIPES} of type 1070 \code{METADATA} with component keywords being the various recipe names 1071 and the values (of type \code{STR}) the corresponding recipe 1072 configuration filename. 1073 1074 An example: 1075 \begin{verbatim} 1076 # Recipes for LRIS 1077 RECIPES METADATA 1078 PHASE1 STR lris_phase1.config 1079 PHASE2 STR lris_phase2.config 1080 PHASE4 STR lris_phase4.config 1081 END 1082 \end{verbatim} 1083 1084 \subsubsection{Recipe Configuration} 1085 1086 \tbd{The contents of the recipe configuration file are dependent upon 1087 the particular module, and hence are not specified here at this time.} 1088 1089 1090 \subsection{PS Concepts} 1091 1092 \subsubsection{Ingest} 1093 1094 For different camera systems, these concepts are not always known by 1095 the same name, nor are they generally obtained in the same manner, and 1096 so their source or value must be specified in the camera configuration 1097 file. At ingest, the value of a concept shall be found by searching in 1098 the following order: 1099 \begin{itemize} 1100 \item The cell data from the \code{CELLS} metadata in the camera configuration. 1101 \item The FITS header via the \code{TRANSLATION} table. 1102 \item The \code{DATABASE} lookup. 1103 \item The \code{DEFAULTS} value. 1104 \end{itemize} 1105 1106 \subsubsection{Dependencies for defaults} 1107 1108 In the \code{DEFAULTS} table in the camera configuration, we allow the 1109 specification of the concept with an additional suffix, \code{DEPEND}. 1110 The value (of type \code{STR}) of the \code{CONCEPT.DEPEND} is the 1111 name of a concept on which the first concept depends. For example, it 1112 might depend on the chip name. Then the first concept becomes of type 1113 \code{METADATA}, with the component keywords being the value of the 1114 second concept (on which the first depends). To avoid infinite 1115 recursion, no further dependency is permitted. We also allow an entry 1116 \code{CONCEPT.DEFAULT} specifiying the default value of the concept if 1117 a match is not made with the dependcency list. An example of the 1118 dependency: 1119 1120 \begin{verbatim} 1121 # Default PS concepts that may be specified by value 1122 DEFAULTS METADATA 1123 CELL.GAIN.DEPEND STR CHIP.NAME 1124 CELL.GAIN.DEFAULT STR 1.0 1125 CELL.GAIN METADATA 1126 ccd00 F32 1.2 1127 ccd01 F32 3.4 1128 ccd02 F32 5.6 1129 END 1130 END 1131 \end{verbatim} 1132 1133 \subsubsection{FORMATS} 1134 1135 Because of the variety of methods for specifying these concepts 1136 (especially in FITS headers), we must also specify additional 1137 information in the camera configuration that specifies how to 1138 interpret the data provided. These are provided in an entry 1139 \code{FORMATS} (of type \code{METADATA}) in the camera configuration. 1140 Within the \code{FORMATS} metadata, there is a string for each of the 1141 concepts that requires a format to be specified. 1142 1143 \paragraph{CELL.TIME} 1144 1145 The time at which the shutter opens is represented in a variety of 1146 ways in FITS files, so care must be taken to specify what the format 1147 is in the file under consideration. Permitted values of 1148 \code{CELL.TIME.FORMAT} are: 1149 1150 \begin{itemize} 1151 \item \code{JD}: The value pointed to by \code{CELL.TIME} is to be 1152 interpreted as a Julian Date. 1153 \item \code{MJD}: The value pointed to by \code{CELL.TIME} is to be 1154 interpreted as a Modified Julian Date. 1155 \item \code{ISO}: The value pointed to by \code{CELL.TIME} is to be 1156 interpreted as an ISO date-time (yyyy-mm-ddThh:mm:ss.ss). 1157 \item \code{SEPARATE}: The date and time are specified separately, and 1158 the \code{CELL.TIME} contains the headers for the date and the time 1159 separated by whitespace or a comma. Then it is necessary to add 1160 additional qualifiers to specify the formats of these: 1161 \begin{itemize} 1162 \item \code{PRE2000}: The year is in the old style two-digit format 1163 popular before the year 2000, and it should be assumed that the 1164 date is in the twentieth century. 1165 \item \code{BACKWARDS}: The date is in the format dd-mm-yyyy or 1166 dd/mm/yyyy. 1167 \item \code{SOD}: The time is specified as seconds-of-day. 1168 \end{itemize} 1169 \end{itemize} 1170 1171 Note that the FITS standard is that the time in the header refers to 1172 the {\it start} of the observation. 1173 1174 \tbd{the PRE2000 and BACKWARDS qualifiers should be replace with 1175 explicit format definitions in the form YYYY/MM/DD} 1176 1177 \tbd{In the future, we might add additional qualifiers that calculate 1178 the start time of the observation based on someone foolishly putting 1179 the end- or mid-time in the header.} 1180 1181 \tbd{Should we move CELL.TIMESYS into the format as well?} 1182 1183 \paragraph{FPA.RA and FPA.DEC} 1184 1185 The RA and Declination of the boresight might be specified in a few 1186 ways. We need to specify both how the value is interpreted and the 1187 units. \code{FPA.RA.FORMAT} and \code{FPA.DEC.FORMAT} should be one 1188 of the following: 1189 1190 \begin{itemize} 1191 \item \code{HOURS}: The value pointed to by the concept should be 1192 interpreted as being in hours. 1193 \item \code{DEGREES}: The value pointed to by the concept should be 1194 interpreted as being in degrees. 1195 \item \code{RADIANS}: The value pointed to by the concept should be 1196 interpreted as being in radians. 1197 \end{itemize} 1198 1199 How the value is interpreted can be determined from the type of the 1200 header: if it is of type \code{STR}, then we can reasonably assume 1201 that it is in sexagesimal format with colons or spaces as separators; 1202 and if it is of type \code{F32} (or \code{F64}), then we can assume 1203 that it is in decimal format. 1204 1205 \subsubsection{Implicit format information} 1206 1207 While details like the units of the right ascension in the header must 1208 be specified explicitly, some other details can be determined from 1209 implicit information. 1210 1211 \begin{itemize} 1212 \item \code{FPA.RA} and \code{FPA.DEC}: if the value on ingest is of 1213 type \code{STRING}, then it may be interpreted as sexagesimal 1214 notation, ``\code{dd:mm:ss.ss}'', or ``\code{dd:mm.mmm}''. A space 1215 may be used instead of a colon to separate the values. Otherwise, if 1216 the value is of a numerical type (\code{F32} or \code{F64}), then that 1217 is the appropriate value. 1218 \item \code{CELL.XBIN} and \code{CELL.YBIN}: if the value on ingest is 1219 of type \code{STRING}, then it may be interpreted as ``\code{x,y}'', 1220 where \code{x} is the binning in x, and \code{y} is the binning in y. 1221 A space may be used instead of a comma, and there may even be a space 1222 before or after the comma (or both). Otherwise, if the value is of a 1223 numerical type (\code{S32}, etc), then that is the appropriate value. 1224 \item \code{CELL.BIASSEC} and \code{CELL.TRIMSEC}: These values on 1225 ingest should always be of type \code{STRING}. If they contain a 1226 square bracket, then they may be interpreted as a list of standard 1227 region specifications, ``\code{[x0:x1,y0:y1];[x2:x3,y2:y3];...}'', 1228 where the semi-colon may be replaced by spaces. Otherwise, the string 1229 may be interpreted as a FITS header (or headers, separated by spaces, 1230 commas or semi-colons) that contains the appropriate values. 1231 \end{itemize} 1232 1233 \tbd{the use of implicit interpretation of formats should be 1234 discouraged: format interpretation guides should be provided} 1235 1236 \subsection{Configuration APIs} 1237 1238 \begin{prototype} 1239 bool pmConfigRead(psMetadata **site, psMetadata **camera, psMetadata **recipe, 1240 int *argc, char **argv, const char *recipeName); 1241 psMetadata *pmConfigCameraFromHeader(const psMetadata *site, const psMetadata *header); 1242 psMetadata *pmConfigRecipeFromCamera(const psMetadata *camera, const char *recipeName); 1243 \end{prototype} 1244 1245 \code{pmConfigRead} shall load the \code{site} configuration 1246 (according to the above rule for determining the source). The 1247 \code{camera} configuration shall also be loaded if it is specified on 1248 the command line (\code{argc, argv}); otherwise it shall be set to 1249 \code{NULL}. The \code{recipe} shall also be loaded from the command 1250 line (if specified) or, if the camera configuration has been loaded, 1251 from the camera configuration and recipe specification therein (see 1252 below). In dealing with the command line parameters, the functions 1253 shall use the appropriate functions in psLib to retrieve and remove 1254 the relevant options from the argument list; this simplifies 1255 assignment of the mandatory arguments, since all the optional command 1256 line arguments are removed leaving only the mandatory arguments. The 1257 following psLib setups shall also be performed if they are specified 1258 in the site configuration: 1259 \begin{itemize} 1260 \item the function shall call \code{psTimeInitialize} with the 1261 configuration file specified by \code{TIME}. 1262 \item the function shall call \code{psLogSetLevel} with the logging 1263 level specified by \code{LOGLEVEL}. 1264 \item the function shall call \code{psLogSetFormat} with the log 1265 format specified by \code{LOGFORMAT}. 1266 \item the function shall call \code{psTraceSetLevel} with the component names and 1267 trace levels specified by the \code{TRACE}. 1268 \end{itemize} 1269 Note that additional log/trace command-line options may be specified 1270 and interpretted using the \code{psArgumentVerbosity} function from 1271 psLib. These options should (in the case of logging) override the 1272 configuration-supplied information or (in the case of tracing) 1273 supplement it. 1274 1275 \code{pmConfigCameraFromHeader} shall load the \code{camera} 1276 configuration based on the contents of the FITS \code{header}, using 1277 the list of known cameras contained in the \code{site} configuration. 1278 If more than one camera matches the FITS header, a warning shall be 1279 generated and the first matching camera returned. 1280 1281 \code{pmConfigRecipeFromCamera} shall load the \code{recipe} 1282 configuration based on the \code{recipeName} and the list of known 1283 recipes contained in the \code{camera} configuration (details below). 1284 1285 \begin{prototype} 1286 bool pmConfigValidateCamera(const psMetadata *camera, const psMetadata *header); 1287 \end{prototype} 1288 1289 This function, used by \code{pmConfigCameraFromHeader}, shall return 1290 \code{true} if the FITS \code{header} matches the rule contained in 1291 the \code{camera} configuration (see \S\ref{sec:camerarule}); 1292 otherwise it shall return \code{false}. 1293 1294 \begin{prototype} 1295 psDB *pmConfigDB(psMetadata *site); 1296 \end{prototype} 1297 1298 \code{pmConfigDB} shall use the \code{site} configuration data to open 1299 a database handle. \tbd{This is fairly straightforward at the moment, 1300 but will change when we beef up security.} 1301 1302 \subsubsection{Example usage} 1303 1304 The following is provided as an example of how the above functions 1305 are envisioned in use. 1306 1307 \begin{verbatim} 1308 int main(int argc, char *argv[]) 1309 { 1310 // Parse other command-line arguments here 1311 psMetadata *site = NULL; // Site configuration 1312 psMetadata *camera = NULL; // Camera configuration 1313 psMetadata *recipe = NULL; // Recipe configuration 1314 if (! pmConfigRead(&site, &camera, &recipe, &argc, argv, "moduleName")) { 1315 psLogMsg("moduleName", PS_LOG_ERROR, "Can't find site configuration!\n"); 1316 exit(EXIT_FAILURE); 1317 } 1318 // Parse other command-line arguments here 1319 1320 // The command-line argument list now contains only mandatory arguments 1321 // Assume the first of these is an input image 1322 char *imageName = argv[1]; // Name of FITS file 1323 psFits *imageFH = psFitsOpen(imageName, "r"); // File handle for FITS file 1324 if (! imageFH) { 1325 psLogMsg("moduleName", PS_LOG_ERROR, "Can't open input image %s\n", imageName); 1326 exit(EXIT_FAILURE); 1327 } 1328 psMetadata *header = psFitsReadHeader(NULL, imageFH); // FITS header 1329 1330 if (!camera && !(camera = pmConfigCameraFromHeader(site, header))) { 1331 psLogMsg("moduleName", PS_LOG_ERROR, "Can't find camera configuration!\n"); 1332 exit(EXIT_FAILURE); 1333 } 1334 1335 if (! recipe && !(recipe = pmConfigRecipeFromCamera(camera, "moduleName"))) { 1336 psLogMsg("moduleName", PS_LOG_ERROR, "Can't find recipe configuration!\n"); 1337 exit(EXIT_FAILURE); 1338 } 1339 1340 // Now go on and do stuff 1341 .... 1342 } 1343 \end{verbatim} 1344 1345 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 1346 1347 \section{``Concepts''} 1348 \label{sec:concepts} 1349 1350 Each image from an astronomical instrument has associated with it what 1351 we will call {\it concepts} (for want of a better word; \tbd{we would 1352 like to call this ``metadata'', but unfortunately that name is already 1353 taken}). These are values corresponding to general quantities and 1354 qualities necessary to understand and interpret the data, such as 1355 airmass, date, read noise and filter. The values of each of the below 1356 concepts shall be determined when the FPA is read into memory (via 1357 \code{pmFPARead}), and stored at the appropriate level in the focal 1358 plane hierarchy. 1359 1360 After ingest (performed in \code{pmFPARead}, the user may safely 1361 assume that all of the above concepts exist at the appropriate level 1362 (meaning the user needn't be hampered by excessive error checking), is 1363 of the specified type (meaning the user doesn't need to worry about 1364 whether the value of interest is stored in, e.g., floating point or 1365 double precision or even a colon-delimited string) and in the 1366 specified format (meaning the user doesn't need to know, e.g., whether 1367 the right ascension is in radians or degrees) --- all the conversions 1368 are handled by the ``concepts'' functions at ingest. 1369 1370 Most of the structures and functions in this section are intended to 1371 be ``private'', since there is no need envisioned for the user to call 1372 them directly. 1373 1374 \subsection{Specifying a concept} 1375 1376 Specifying a ``concept'' requires a (meaningful) name (preferably with 1377 the level in the name, e.g., \code{CELL.EXPOSURE}), a 1378 comment/description, a type, a default or blank value, functions to 1379 read and write, and a level that the concept applies to 1380 (FPA/Chip/Cell). 1381 1382 \begin{datatype} 1383 typedef psMetadataItem* (*p_pmConceptReadFunc)(pmFPA *fpa, pmChip *chip, pmCell *cell, psDB *db); 1384 typedef bool (*p_pmConceptWriteFunc)(pmFPA *fpa, pmChip *chip, pmCell *cell, psDB *db); 1385 typedef struct { 1386 psMetadataItem *blank; // Blank value of concept; also contains the name 1387 p_pmConceptReadFunc read; // Function to call to read the concept 1388 p_pmConceptWriteFunc write; // Function to call to write the concept 1389 } p_pmConceptSpec; 1390 \end{datatype} 1391 1392 \code{blank} is a \code{psMetadataItem} that provides the name, type 1393 and default/blank value for the concept. \code{read} and \code{write} 1394 provide the functions to read and write. 1395 1396 A concept specification may be allocated: 1397 \begin{prototype} 1398 p_pmConceptSpec *p_pmConceptSpecAlloc(psMetadataItem *blank, pmConceptReadFunc read, 1399 pmConceptWriteFunc write); 1400 \end{prototype} 1401 1402 \subsection{Registering a concept} 1403 1404 The concept specifications that have been registered shall be stored in 1405 three \code{psMetadata}s, one for each level (FPA, chip, cell). 1406 1407 Registering a concept is achieved by: 1408 \begin{prototype} 1409 bool pmConceptRegister(psMetadataItem *blank, pmConceptReadFunc read, 1410 pmConceptWriteFunc write, pmConceptLevel level); 1411 \end{prototype} 1412 1413 \code{pmConceptRegister} shall generate a concept specification from 1414 the provided \code{blank}, and \code{read} and \code{write} functions, 1415 and register it in the metadata specified by the \code{level}. 1416 1417 \code{pmConceptLevel} simply specifies which level in the focal plane 1418 hiearchy the concept applies to: 1419 \begin{datatype} 1420 typedef enum { 1421 PM_CONCEPT_LEVEL_FPA, // Store in the FPA 1422 PM_CONCEPT_LEVEL_CHIP, // Store in the chip 1423 PM_CONCEPT_LEVEL_CELL // Store in the cell 1424 } pmConceptLevel; 1425 \end{datatype} 1426 1427 A \code{read} function of \code{NULL} indicates that there is no 1428 special interpretation of the concept required, and that it can be 1429 used as read. A \code{write} function of \code{NULL} indicates that 1430 no special formatting of the concept is required, and that it can be 1431 written as is. 1432 1433 1434 \subsection{Default concepts} 1435 1436 Below is a list of concepts that the IPP will use, with the expected 1437 type and a short description. 1438 1439 \begin{itemize} 1440 \item \code{FPA.NAME} (\code{psString}): An identifier (e.g., observation number) for the FPA instance 1441 \item \code{FPA.AIRMASS} (F32): Airmass at which the observation is made (boresight) 1442 \item \code{FPA.FILTER} (\code{psString}): Filter used in observation 1443 \item \code{FPA.POSANGLE} (F32): Position angle for camera 1444 \item \code{FPA.RADECSYS} (\code{psString}): System of RA,Dec (e.g., J2000 or ICRS) 1445 \item \code{FPA.RA} (F64): Right Ascension of boresight in radians 1446 \item \code{FPA.DEC} (F64): Declination of boresight in radians 1447 \item \code{CHIP.NAME} (\code{psString}): The name of the chip (unique within the FPA) --- set at FITS read 1448 \item \code{CELL.NAME} (\code{psString}): The name of the cell (unique within the parent chip) --- set at FITS read 1449 \item \code{CELL.GAIN} (F32): CCD gain (e/ADU) 1450 \item \code{CELL.READNOISE} (F32): CCD read noise (e) 1451 \item \code{CELL.SATURATION} (F32): CCD saturation point (ADU) 1452 \item \code{CELL.BAD} (F32): CCD bad pixel point (ADU) 1453 \item \code{CELL.XPARITY} (S32): Direction of CCD readout in x relative to the rest of the chip 1454 \item \code{CELL.YPARITY} (S32): Direction of CCD readout in y relative to the rest of the chip 1455 \item \code{CELL.READDIR} (S32): Read direction: line (1) or column (2) 1456 \item \code{CELL.EXPOSURE} (F32): Exposure time of image (sec) 1457 \item \code{CELL.DARKTIME} (F32): Dark time for image (sec) 1458 \item \code{CELL.TRIMSEC} (\code{psRegion*}): Trim region 1459 \item \code{CELL.BIASSEC} (\code{psList*} of \code{psRegion*}): Overscan region(s) 1460 \item \code{CELL.XBIN} (S32): CCD binning in x 1461 \item \code{CELL.YBIN} (S32): CCD binning in y 1462 \item \code{CELL.TIMESYS} (\code{psTimeType}): Time system in use 1463 \item \code{CELL.TIME} (\code{psTime*}): Time of observation start 1464 \item \code{CELL.X0} (S32): x position of cell (0,0) on the chip 1465 \item \code{CELL.Y0} (S32): y position of cell (0,0) on the chip 1466 \end{itemize} 1467 1468 \tbd{CELL.EXPOSURE, CELL.DARKTIME and CELL.TIME should actually be 1469 specified at the readout level. However, at this present time, we're 1470 not sure how these should be specified, and so we move them up to the 1471 cell level and assume that all readouts are of the same exposure and 1472 dark time.} 1473 1474 The concept specifications for the above shall be registered by 1475 \code{pmConceptsInit}: 1476 \begin{prototype} 1477 bool pmConceptsInit(void); 1478 \end{prototype} 1479 1480 Since defined concept specifications are required before any concept 1481 ingest can take place, all functions that work with the concepts must 1482 call \code{pmConceptsInit} first. 1483 1484 The concept specification metadata containers and the concept 1485 specifications that have been registered shall all be freed by 1486 \code{pmConceptsDone}: 1487 \begin{prototype} 1488 void pmConceptsDone(void); 1489 \end{prototype} 1490 Calling \code{pmConceptsDone} is required in order to avoid a memory 1491 leak, since the metadata containers are defined \code{static}. 1492 1493 \subsection{Reading, Writing and Blanking} 1494 1495 Reading concepts is the act of determining their values and setting 1496 them in the \code{concepts} metadata in the focal plane hierarchy. 1497 Writing concepts is the act of taking the \code{concepts} metadata 1498 which is in the focal plane hierarchy and preparing them for output. 1499 By ``blanking'', we mean setting the concepts to a default or blank 1500 value (e.g., \code{NaN} for floating point); this takes place before 1501 reading, and can be used to set up a focal plane hierarchy without 1502 reading from any particular source. 1503 1504 The following functions shall read, write or blank (as appropriate) 1505 the concepts at the appropriate level in the focal plane hierarchy: 1506 \begin{prototype} 1507 bool p_pmConceptsReadFPA(pmFPA *fpa); 1508 bool p_pmConceptsReadChip(pmChip *chip); 1509 bool p_pmConceptsReadCell(pmCell *cell); 1510 bool p_pmConceptsWriteFPA(pmFPA *fpa); 1511 bool p_pmConceptsWriteChip(pmChip *chip); 1512 bool p_pmConceptsWriteCell(pmCell *cell); 1513 bool p_pmConceptsBlankFPA(pmFPA *fpa); 1514 bool p_pmConceptsBlankChip(pmChip *chip); 1515 bool p_pmConceptsBlankCell(pmCell *cell); 1516 \end{prototype} 1517 1518 Under ordinary circumstances, these functions will be called by 1519 \code{pmFPARead}, \code{pmFPAWrite} and \code{pmFPAConstruct}. 1520 1521 1522 \subsection{Copying concepts} 1523 1524 The values of concepts may be copied from one source to another: 1525 \begin{prototype} 1526 bool pmFPACopyConcepts(pmFPA *target, pmFPA *source); 1527 \end{prototype} 1528 1529 \code{pmFPACopyConcepts} shall iterate through the focal plane 1530 hierarchy, copying the values of the concepts from the \code{source} 1531 to the \code{target}. 1532 1533 1534 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 1535 1536 %\input{CameraImages.tex} 1537 1538 %\input{CameraGeometry.tex} 1539 1540 \section{Photometry} 1541 1542 \tbd{This section is to be deferred, and for now consists only of 1543 place holders, with no functional items.} 1544 1545 Photometric observations are performed in an instrumental photometric 1546 system, and must be related to other photometric systems. We 1547 require a data structure which defines a photometric system, as well 1548 as a structure to define the transformation between photometric 1549 systems. 1550 1551 The photometric system is defined by the psPhotSystem structure. 1552 A photometric system is identified by a human-readable \code{name} 1553 (ie, SDSS.g, Landolt92.B, GPC1.OTA32.r). Each photometric system is 1554 given a unique identifier \code{ID}. Observations taken with a 1555 specific camera, detector, and filter represent their own photometric 1556 system, and it may be necessary to perform transformations between 1557 these systems. Photometric systems associated with observations from 1558 a specific camera/detector/filter combination can be associated with 1559 those components. 1560 \begin{datatype} 1561 typedef struct { 1562 const int ID; ///< ID number for this photometric system 1563 const char *name; ///< Name of photometric system 1564 const char *camera; ///< Camera for photometric system 1565 const char *filter; ///< Filter used for photometric system 1566 const char *detector; ///< Detector used for photometric system 1567 } psPhotSystem; 1568 \end{datatype} 1569 1570 The following structure defines the transformation between two 1571 photometric systems. 1572 \begin{datatype} 1573 typedef struct { 1574 psPhotSystem src; ///< Source photometric system 1575 psPhotSystem dst; ///< Destination photometric system 1576 psPhotSystem pP, pM; ///< Primary color reference 1577 psPhotSystem sP, sM; ///< Secondary color reference 1578 float pA, sA; ///< Color offset for references 1579 psPolynomial3D transform; ///< Transformation from source to destination 1580 } psPhotTransform; 1581 \end{datatype} 1582 1583 The transformation between two photometric systems may depend on the 1584 airmass of the observation and on the colors of the object of 1585 interest. For a specific observation, such a transformations can be 1586 defined as a polynomial function of the color of the star and the 1587 airmass of the observations. If sufficient data exists, the 1588 transformation between the photometric systems may include more than 1589 one color, constraining the curvature of the stellar spectral energy 1590 distributions. This latter term may be significant for stars which 1591 are highly reddened, for example. Derived photometric quantities may 1592 have been corrected for airmass variations, in which case only color 1593 terms may be measurable. The structure defines the transformation 1594 between a source photometric system (\code{src}) and a target 1595 photometric system (\code{dst}). The photometric system of a primary 1596 color is defined by \code{pP, pM} such that the color is constructed 1597 as $pP - pM$. A secondary color is defined by \code{sP, sM}. For 1598 both, a reference color is specified (\code{pA, sA}): the polynomial 1599 transformation terms refer to colors in the form $pP - pM - pA$. The 1600 transformation is specified as a 3D polynomial. For a star of 1601 magnitude $M_{\rm src}$ in the source photometric system, with 1602 additional magnitude information in the other systems $M_{\rm pP}$, 1603 $M_{\rm pM}$, $M_{\rm sP}$, $M_{\rm sM}$, observed at an airmass of 1604 $z$, the magnitude of the star in the target system $M_{\rm dst}$ is 1605 given by: $M_{\rm dst} = M_{\rm src} + transform(z, M_{\rm pP} - 1606 M_{\rm pM} - pA, M_{\rm sP} - M_{\rm sM} - sA)$. 1607 1608 \section{Image Detrending} 1609 1610 Image Detrending is the image analysis process wherein the 1611 instrumental signatures are removed from the individual images. This 1612 section discusses the modules used for image detrending. The basic 1613 image detrending steps are: 1614 \begin{itemize} 1615 \item Subtract bias; 1616 \item Correct for non-linearity; 1617 \item Flat-field; 1618 \item Mask bad pixels; 1619 \item Subtract the background; 1620 \item Mask cosmic rays; 1621 \item Mask optical defects; 1622 \end{itemize} 1623 1624 \subsection{Bias subtraction} 1625 \label{sec:bias} 1626 1627 The bias subtraction module provides a facility to correct detector 1628 images for the electronic pedestal introduced by the readout 1629 electronics. 1630 1631 Given an input image and various other parameters, 1632 \code{pmSubtractBias} shall subtract the bias from the image: 1633 1634 \begin{prototype} 1635 pmReadout *pmSubtractBias(pmReadout *in, pmOverscanOptions *overscanOpts, 1636 psRegion imageRegion, psList *overscanRegions, 1637 const pmReadout *bias, const pmReadout *dark); 1638 \end{prototype} 1639 1640 Three types of bias correction may optionally be performed on the 1641 input image, \code{in}. The first is the subtraction of an overscan. 1642 Multiple overscan regions may be specified and fit as a function of 1643 row (or column). The second is the subtraction of a full-frame bias 1644 image. The third is the subtraction of a suitably scaled full-frame 1645 dark image. 1646 1647 The input image, \code{in}, shall have the bias subtracted in-place. 1648 The input image may be of type U16, S32, or F32. The region of the 1649 input image that shall have the overscan or full-frame subtractions 1650 applied is specified by \code{imageRegion}. 1651 1652 Overscan subtraction is performed if \code{overscanOpts} is 1653 non-\code{NULL} (see \S\ref{sec:overscan}). \code{overscanRegions} 1654 shall be a list of \code{psRegion}s that specify the regions that 1655 comprise the overscans. 1656 1657 A \code{bias} frame shall be subtracted pixel-by-pixel from the input 1658 image if \code{bias} is non-NULL. If \code{dark} is non-\code{NULL}, 1659 then the dark image, scaled by the ratio of dark times (from 1660 \code{CELL.DARKTIME}) shall be subtracted pixel-by-pixel from the 1661 input image. The full-frame subtractions (both bias and dark) should 1662 only be performed on the image region specified by 1663 \code{CELL.TRIMSEC}. Note that the input image, \code{in}, and the 1664 \code{bias} and \code{dark} frames need not be the same size, but the 1665 function shall use the offsets in the image (\code{in->x0} and 1666 \code{in->y0}) to determine the appropriate offsets to obtain the 1667 correct pixel on the \code{bias}. In the event that the \code{bias} 1668 image is too small (i.e., pixels on the input image refer to pixels 1669 outside the range of the \code{bias} image), the function shall 1670 generate an error. Any pixels masked in the \code{bias} or 1671 \code{dark} shall also be masked in the output. The bias and dark 1672 images may be copied to the same type as the input image if required. 1673 1674 1675 \subsubsection{Overscan subtraction} 1676 \label{sec:overscan} 1677 1678 The options for performing the overscan subtraction are bundled in a 1679 \code{pmOverscanOptions}: 1680 1681 \begin{datatype} 1682 typedef struct { 1683 // Inputs 1684 bool single; // Reduce all overscan regions to a single value? 1685 bool scanRows; // Scan direction was rows? (otherwise columns) 1686 pmFit fitType; // Type of fit to overscan 1687 unsigned int order; // Order of polynomial, or number of spline pieces 1688 psStats *stat; // Statistic to use when reducing the minor direction 1689 // Outputs 1690 psPolynomial1D *poly; // Result of polynomial fit 1691 psSpline1D *spline; // Result of spline fit 1692 } pmOverscanOptions; 1693 \end{datatype} 1694 1695 The mode in which the overscan is subtracted is specified by the 1696 \code{single} boolean. If \code{single} is \code{true}, then the 1697 entire overscan region is reduced to a single value using the 1698 \code{stat}. If \code{single} is \code{false}, the overscan shall be 1699 reduced along the dimension specified by \code{scanRows} (rows if 1700 \code{scanRows} is true; otherwise columns). 1701 1702 If the overscan is not defined for each row/column, 1703 \code{pmSubtractBias} shall generate an error if \code{fitType} is 1704 \code{PM_FIT_NONE}; otherwise, the function shall shall generate a 1705 warning and the undefined values shall be interpolated using the 1706 provided functional form. 1707 1708 The statistic to use in combining multiple pixels in the 1709 prescan/overscan regions is specified by \code{stat}. \code{stat} is 1710 of type \code{psStats} instead of simply \code{psStatsOptions} so that 1711 clipping levels may be specified, if desired. In the event that 1712 multiple options are specified by \code{stats}, a warning shall be 1713 generated, and the option with the highest priority shall be used, 1714 according to the following priority order: \code{PS_STAT_SAMPLE_MEAN}, 1715 \code{PS_STAT_SAMPLE_MEDIAN}, \code{PS_STAT_CLIPPED_MEAN}, 1716 \code{PS_STAT_ROBUST_MEAN}, \code{PS_STAT_ROBUST_MEDIAN}, 1717 \code{PS_STAT_ROBUST_MODE}. 1718 1719 \code{fitType} is an enumerated type which specifies the type of fit 1720 to employed on the overscan vector: 1721 \begin{datatype} 1722 typedef enum { 1723 PM_FIT_NONE, ///< No fit 1724 PM_FIT_POLY_ORD, ///< Fit ordinary polynomial 1725 PM_FIT_POLY_CHEBY, ///< Fit Chebyshev polynomial 1726 PM_FIT_SPLINE ///< Fit cubic splines 1727 } pmFit; 1728 \end{datatype} 1729 1730 If \code{fitType} is \code{PM_FIT_NONE}, then the overscan vector is 1731 subtracted from the image without fitting. Otherwise, the overscan 1732 vector is fit using the specified functional form, the fit is 1733 subtracted from the image, and the \code{poly} or \code{spline} is 1734 allocated and updated with the results of the fit. 1735 1736 The allocator for a \code{pmOverscanOptions} shall be: 1737 \begin{prototype} 1738 pmOverscanOptions *pmOverscanOptionsAlloc(bool single, bool scanRows, 1739 pmFit fitType, unsigned int order, 1740 psStats *stat); 1741 \end{prototype} 1742 1743 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 1744 1745 \subsection{Non-linearity} 1746 1747 We here specify two functions to perform the non-linearity correction, 1748 since either (or both) might be used to specify the correction. 1749 1750 These operations act only on the region of the readout specified by 1751 \code{CELL.TRIMSEC}. 1752 1753 The first, \code{pmNonLinearityPolynomial} shall correct the input 1754 image for non-linearity by replacing the flux in each pixel of the 1755 input image, \code{in}, with the result of the specified polynomial, 1756 \code{coeff}, acting on the flux. The API shall be the following: 1757 1758 \begin{prototype} 1759 pmReadout *pmNonLinearityPolynomial(pmReadout *in, const psPolynomial1D *coeff); 1760 \end{prototype} 1761 1762 The polynomial coefficients, \code{coeff}, will be supplied by the 1763 caller, likely from the image metadata. 1764 1765 The second function, \code{pmNonLinearityLookup} shall correct 1766 the input image for non-linearity by using a lookup table. The API 1767 shall be the following: 1768 1769 \begin{prototype} 1770 pmReadout *pmNonLinearityLookup(pmReadout *in, const char *filename); 1771 \end{prototype} 1772 1773 For each pixel in the input image, the function shall replace the flux 1774 with the corresponding value from the supplied lookup table, specified 1775 by the \code{filename}. The lookup table file shall consist of two 1776 columns of data, the first being the original flux value and the 1777 second being the replaced flux value. The file shall be in a format 1778 suitable for reading by \code{psLookupTableRead}. 1779 1780 Both \code{pmNonLinearityPolynomial} and \code{pmNonLinearityLookup} 1781 shall modify the input image in-place. The input image may be of 1782 type U16, S32, or F32. 1783 1784 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 1785 1786 \subsection{Flat-fielding} 1787 1788 Given an input image and a flat-field image, \code{pmFlatField} shall 1789 divide the input image by the flat-field image and return it in place, 1790 updating the mask contained within the input image as appropriate. 1791 The API shall be the following: 1792 \begin{prototype} 1793 bool pmFlatField(pmReadout *in, const pmReadout *flat); 1794 \end{prototype} 1795 1796 Note that the input image, \code{in}, and the flat-field image, 1797 \code{flat}, need not be the same size, since the input image may 1798 already have been trimmed (following overscan subtraction), but the 1799 function shall use the offsets of the readout (\code{in->col0, 1800 in->row0}) and the image subarray (\code{in->image->x0, 1801 in->image->y0}) to determine the appropriate offsets to obtain the 1802 correct detector pixels in the flat-field image. Note that the image 1803 offset is relative to its parent, so this offset must be followed to 1804 the top level image which is not a child of another image and the 1805 offsets summed. The detector pixel coordinates of pixel \code{x,y} in 1806 a top-level image are thus \code{x + in->image->x0 + in->col0, y + 1807 in->image->y0 + in->row0}. In the event that the \code{flat} image is 1808 too small (i.e., pixels on the input image refer to pixels outside the 1809 range of the \code{flat} image), the function shall generate an error. 1810 1811 Pixels which are negative or zero in the \code{flat} shall be masked 1812 in the input image with the value \code{PM_MASK_FLAT} (see 1813 \S\ref{sec:maskValues}). Negative pixels in the \code{flat} may be 1814 set to zero so that they are treated identically to zeroes. Any 1815 pixels masked in the \code{flat} shall be masked with corresponding 1816 values in the \code{output}. 1817 1818 The function shall not normalize the \code{flat}; this responsibility 1819 is left to the caller. This function is basically equivalent to a 1820 divide (with \code{psImageOp}), but with care for the region that is 1821 divided, checking for zero and negative pixels, and copying of the 1822 mask from the \code{flat} to the output. 1823 1824 The images in the input and flat-field readouts must both be of type 1825 F32. 1826 1827 This operation acts only on the region of the readout specified by 1828 \code{CELL.TRIMSEC}. 1829 1830 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 1831 1832 \subsection{Masking} 1833 1834 \subsubsection{Mask values} 1835 \label{sec:maskValues} 1836 1837 We define several mask values for use in the detrend processing: 1838 \begin{datatype} 1839 /** Mask values */ 1840 typedef enum { 1841 PM_MASK_TRAP = 0x0001, ///< The pixel is a charge trap 1842 PM_MASK_BADCOL = 0x0002, ///< The pixel is a bad column 1843 PM_MASK_SAT = 0x0004, ///< The pixel is saturated 1844 PM_MASK_FLAT = 0x0008 ///< The pixel is non-positive in the flat-field 1845 } pmMaskValue; 1846 \end{datatype} 1847 1848 Of these, masks for the charge traps need to be grown by the extent of 1849 the OT convolution kernel. For other pixel types, orthogonal transfer 1850 of the flux in this pixel will not (necessarily) affect the flux in 1851 neighbouring pixels. 1852 1853 \subsubsection{Bad pixels} 1854 1855 Given an input image, \code{in}, a bad pixel \code{mask}, a 1856 corresponding value in the bad pixel mask to mask in the input image, 1857 \code{maskVal}, a saturation level, and a growing radius, 1858 \code{pmMaskBadPixels} shall mask in the input image those 1859 pixels in the bad pixel mask that match the value to mask. The API 1860 shall be the following: 1861 \begin{prototype} 1862 pmReadout *pmMaskBadPixels(pmReadout *in, const pmReadout *mask, unsigned int maskVal, 1863 float sat, unsigned int growVal, int grow); 1864 \end{prototype} 1865 1866 Note that the input image, \code{in}, is modified in-place. All 1867 pixels in the \code{mask} which satisfy the \code{maskVal} shall have 1868 their corresponding pixels masked in the input image, \code{in}. All 1869 pixels which satisfy the \code{growVal} shall have their corresponding 1870 pixels, along with all pixels within the \code{grow} radius masked. 1871 Pixels which have flux greater than \code{sat} shall also be masked, 1872 and grown by a single pixel (in addition to the \code{grow} done on 1873 the \code{growVal}). 1874 1875 \tbd{In the future, may change {\tt grow} to a convolution kernel}. 1876 1877 Note that the input image, \code{in}, and the \code{mask} need not be 1878 the same size, since the input image may already have been trimmed 1879 (following overscan subtraction), but the function shall use the 1880 offsets in the image (\code{in->x0} and \code{in->y0}) to determine 1881 the appropriate offsets to obtain the correct pixel on the mask. In 1882 the event that the \code{mask} image is too small (i.e., pixels on the 1883 input image correspond to pixels outside the range of the \code{mask} 1884 image), the function shall generate an error. 1885 1886 The input image may be of type U16, S32 or F32. The mask image 1887 must be of type U8. 1888 1889 This operation acts only on the region of the readout specified by 1890 \code{CELL.TRIMSEC}. 1891 1892 \subsection{Subtract sky} 1893 1894 \tbd{This may be deferred.} 1895 1896 Given an input image, a polynomial or spline specifying the order of a 1897 desired fit, a binning factor and statistics to use for the binning, 1898 along with a clipping level, \code{pmSubtractSky} shall fit and 1899 subtract a model for the background of the image. The API shall be 1900 the following: 1901 \begin{prototype} 1902 pmReadout *pmSubtractSky(pmReadout *in, psPolynomial2D *poly, psImage *mask, psU8 maskVal, 1903 int binFactor, psStats *stats, float clipSD); 1904 \end{prototype} 1905 1906 Note that the input image, \code{in}, shall be subtracted in-place. 1907 The function shall return the subtracted image, and also update the 1908 polynomial, Chebyshev or spline specified by \code{fitSpec}, to hold 1909 the coefficients used in the subtraction. 1910 1911 The polynomial, \code{poly}, specifies the order of the polynomial, 1912 and on return shall contain the coefficients of the fit. If 1913 \code{poly} is \code{NULL}, then no fit shall be performed, and the 1914 function shall generate a warning and return. 1915 1916 When fitting the polynomial, the function shall first bin the input 1917 image by \code{binFactor} in order to reduce the required processing 1918 time. In the binning, pixels in the \code{mask} (if non-\code{NULL}) 1919 which satisfy the \code{maskVal} shall be excluded. The statistic to 1920 use in this binning is specified by \code{stat}. \code{stat} is of 1921 type \code{psStats} instead of simply \code{psStatsOptions} so that 1922 clipping levels may be specified, if desired. In the event that 1923 multiple options are specified by \code{stats}, a warning shall be 1924 generated, and the option with the highest priority shall be used, 1925 according to the following priority order: \code{PS_STAT_SAMPLE_MEAN}, 1926 \code{PS_STAT_SAMPLE_MEDIAN}, \code{PS_STAT_CLIPPED_MEAN}, 1927 \code{PS_STAT_ROBUST_MEAN}, \code{PS_STAT_ROBUST_MEDIAN}, 1928 \code{PS_STAT_ROBUST_MODE}. If the \code{binFactor} is non-positive, 1929 or \code{stats} is \code{NULL} or fails to specify an option, a 1930 warning shall be generated, and the fit shall be performed on the 1931 entire image. 1932 1933 Binned pixels deviating more than \code{clipSD} standard deviations 1934 from the mean of the binned pixels shall be clipped in a single 1935 clipping iteration before polynomial fitting. These pixels may be 1936 interpolated over, or may be simply ignored in the fitting, according 1937 to the choice of algorithm. If the \code{clipSD} is non-positive, 1938 then the function shall generate a warning and not perform any 1939 clipping. 1940 1941 The \code{mask} shall be of type U8, and the input image, 1942 \code{in}, must be of type F32. 1943 1944 This operation acts only on the region of the readout specified by 1945 \code{CELL.TRIMSEC}. 1946 1947 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 1948 1949 \subsection{Paper Trail} 1950 1951 The elements of the focal plane hierarchy each contain an 1952 \code{analysis} member, intended to log the results of the detrend 1953 tasks. The detrend tasks shall add to the \code{analysis} members as 1954 follows: 1955 1956 \begin{itemize} 1957 \item \code{pmMaskBadPixels}: 1958 \begin{itemize} 1959 \item \code{MASK.DONE} (STR): The time at which masking was 1960 completed. 1961 \item \code{MASK.SAT} (S32): The number of saturated pixels masked 1962 in the image 1963 \item \code{MASK.SAT.GROW} (S32): The number of additional pixels 1964 masked by growing the saturated pixels. 1965 \item \code{MASK.BAD} (S32): The number of pixels masked in the 1966 image 1967 \item \code{MASK.BAD.GROW} (S32): The number of additional pixels 1968 masked by growing the specified bad pixels. 1969 \end{itemize} 1970 \item \code{pmNonLinearityPolynomial} and \code{pmNonLinearityLookup}: 1971 \begin{itemize} 1972 \item \code{NONLIN.DONE} (STR): The time at which the non-linearity 1973 correction was completed. 1974 \item \code{NONLIN.POLY} (STR): The polynomial coefficients used (if 1975 applicable). 1976 \item \code{NONLIN.LOOKUP} (STR): The filename for the lookup table 1977 (if applicable). 1978 \end{itemize} 1979 \item \code{pmSubtractBias}: 1980 \begin{itemize} 1981 \item \code{BIAS.DONE} (STR): The time at which the bias-subtraction 1982 was completed. 1983 \item \code{BIAS.OVERSCAN.AXIS} (STR): Overscan axis used. 1984 \item \code{BIAS.OVERSCAN.FIT.TYPE} (STR): Fit type applied to 1985 overscan. 1986 \item \code{BIAS.OVERSCAN.FIT.COEFF} (STR): Coefficients of overscan 1987 fit. 1988 \item \code{BIAS.OVERSCAN.REGION} (STR): Overscan regions (from 1989 \code{x0,y0,numCols,numRows}). 1990 \item \code{BIAS.OVERSCAN.BIN} (S32): Number of pixels per bin used 1991 in overscan. 1992 \item \code{BIAS.OVERSCAN.MEAN} (F32): The mean of the binned 1993 overscan pixels after subtracting the fit. 1994 \item \code{BIAS.OVERSCAN.SD} (F32): The standard deviation of the 1995 binned overscan pixels after subtracting the fit. 1996 \end{itemize} 1997 \item \code{pmFlatField}: 1998 \begin{itemize} 1999 \item \code{FLAT.DONE} (STR): The time at which the flat-fielding 2000 was completed. 2001 \item \code{FLAT.BAD} (S32): Number of non-positive flat-field 2002 pixels. 2003 \end{itemize} 2004 \end{itemize} 2005 2006 To be added by higher-levels: 2007 \begin{itemize} 2008 \item \code{BIAS.NAME} (STR): Name of bias image 2009 \item \code{DARK.NAME} (STR): Name of dark image 2010 \item \code{FLAT.NAME} (STR): Name of flat image 2011 \item \code{MASK.NAME} (STR): Name of mask image 2012 \end{itemize} 2013 2014 \subsection{Detrend Lookups} 2015 2016 When it comes time to perform a detrend operation on an image, it is 2017 necessary to determine {\em which} detrend image should be used. The 2018 Pan-STARRS Image Processing Pipeline uses the concept of a detrend 2019 image database table, or set of tables (part of the Metadata 2020 Database), to store the known master detrend images. These tables can 2021 be accessed though the basic query functions specified for the master 2022 detrend database. To simplify the interaction for the case of the 2023 detrend images, the following function allows the user to explicitly 2024 search the detrend database table or tables for detrend images which 2025 satisfy a set of characteristics. 2026 2027 \begin{prototype} 2028 psArray *pmDetrendLookup (psMetadata *constraints, psMetadata *tableDefs); 2029 \end{prototype} 2030 This function accepts a metadata structure which restricts the 2031 selected detrend images. This metadata structure may contain any of 2032 the following entries: 2033 \begin{verbatim} 2034 TYPE type of detrend data (eg, flat, bias) 2035 CAMERA name of desired camera (eg, GPC, MEGACAM) 2036 CHIP chip identifier (eg., ccd00) 2037 FILTERNAME name of specific filter hardware (eg, r.GPC01) 2038 FILTERTYPE conceptual name of filter (eg., r) 2039 TIME_MIN lower bound on valid time range 2040 TIME_MAX upper bound on valid time range 2041 LABEL match the entry label 2042 RECIPE recipe used to build detrend image 2043 EXPTIME exposure time 2044 AIRMASS airmass 2045 \end{verbatim} 2046 Any detrend images which match the provided constraints are returned 2047 as an array of \code{psMetadata} elements corresponding to the columns 2048 of the detrend database table. The additional input parameter 2049 specifies additional information to define the detrend database 2050 tables. This may include the access information (IP, Username, 2051 Password), as well as names for the table and the columns which 2052 correspond to the constraint names. 2053 2054 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 2055 2056 \section{Detrend Creation} 2057 2058 In the detrend creation process, a collection of raw images are 2059 combined to produce a clean, high-quality master image for correcting 2060 the effect of interest. The input images may potentially be processed 2061 and scaled in some way. The resulting output images may be to be 2062 re-scaled to have a consistent signal for all chips in the mosaic. 2063 The simplest example is the construction of a bias image, in the case 2064 where there is signficant 2-D bias structure. In this case, the input 2065 raw bias images are probably combined without any additional 2066 processing. In another example, flat-field image must be 2067 bias-corrected and scaled to a consistent normalization before being 2068 combined, and the flat-field images from the different chips must be 2069 normalized so that each chip will be flattened consistently across the 2070 mosaic. A complex example is the fringe pattern, in which the input 2071 images must be bias-corrected and flattened, and the resulting images 2072 must be scaled by the amplitude of the fringe pattern on each image, 2073 rather than by the average flux level. In this section, we define the 2074 tools necessary to perform the detrend creation process. 2075 2076 \subsection{Image Stacking} 2077 2078 A basic operation in generating the master detrend images is using a 2079 stack of many input images of a particular type and combining them, 2080 with perhaps some additional scaling, in order to build up 2081 signal-to-noise and to reject deviant pixel. For this, we require a 2082 general purpose image combination module. We forsee this module as 2083 only acting upon data from the same detector, and so each input image 2084 will have the same noise characteristics. 2085 2086 \begin{datatype} 2087 typedef struct { 2088 psStats *stats; // Statistics to use in combining pixels 2089 unsigned int maskVal, // Mask pixels where mask & maskVal == 1 2090 float fracHigh; // Fraction of high pixels to throw 2091 float fracLow; // Fraction of low pixels to throw 2092 int nKeep; // Number of pixels to be sure to keep 2093 } pmCombineParams; 2094 \end{datatype} 2095 2096 \begin{prototype} 2097 psImage * 2098 pmReadoutCombine(psImage *output, // Output image, or NULL 2099 const psList *inputs, // List of input readouts 2100 pmCombineParams *params, // Combination parameters 2101 const psVector *zero, // Offsets to apply for each image 2102 const psVector *scale, // Scales to apply for each image 2103 bool applyZeroScale, // Are zero and scale for application, or only noise properties? 2104 float gain, // Gain in e/ADU 2105 float readnoise // Read noise in e 2106 ); 2107 \end{prototype} 2108 2109 \code{pmReadoutCombine} combines input images pixel by pixel --- for 2110 each pixel of the output image, a stack of contributing input pixels 2111 is formed and combined. Several of its input parameters are lists or 2112 vectors, and if these are not all of the same length (or \code{NULL}), 2113 the module shall generate an error and return \code{NULL}. 2114 2115 If the provided \code{output} is \code{NULL}, then the module shall 2116 allocate a new image of sufficient size for the input images. If the 2117 \code{output} image is non-\code{NULL} and is not of sufficient size 2118 for the combined image, the module shall generate an error and return 2119 \code{NULL}. 2120 2121 If the \code{inputs} is \code{NULL}, the module shall generate an 2122 error and return \code{NULL}. Otherwise, the \code{inputs} shall be a 2123 list of \code{pmReadout}s. The images contained within the 2124 \code{pmReadout}s need not all be of the same size, but the module 2125 shall take into account the offsets (\code{col0,row0}) from the corner 2126 of the detector when comparing pixels, so that it is the same 2127 \textit{physical} pixels that are combined. 2128 2129 The parameters used in the combination, including how the pixels are 2130 to be combined, and how the rejection is performed is contained within 2131 the \code{params}, which may not be \code{NULL} (otherwise the module 2132 shall generate an error and return \code{NULL}). We choose to use 2133 this structure instead of supplying the values separately in order to 2134 keep down the number of parameters to \code{pmReadoutCombine}; the 2135 \code{pmCombineParams} may be recycled for subsequent calls to 2136 \code{pmReadoutCombine} since the values are not dependent upon the 2137 choice of inputs, but merely specify how the combination is to be 2138 performed. 2139 2140 The particular statistic specified by \code{stats} shall be used to 2141 combine each stack of pixels from the input images. Only one of the 2142 statistics choices may be specified, otherwise the module shall 2143 generate an error and return \code{NULL}. 2144 2145 If the \code{maskVal} is non-zero, then pixels in the \code{mask} of 2146 each \code{pmReadout} in the \code{inputs} which satisfy the 2147 \code{maskVal} shall not have the corresponding pixels placed in the 2148 stack for combination. 2149 2150 After masking, but before performing the combination, the highest 2151 \code{fracHigh} fraction and lowest \code{fracLow} fraction of pixels 2152 in the stack are immediately rejected, unless this would leave less 2153 than \code{nKeep} pixels in the stack, in which case no immediate 2154 rejection is performed. 2155 2156 If the \code{zero} vector is non-\code{NULL} and \code{applyZeroScale} 2157 is \code{true}, then the appropriate values shall be added to the 2158 \code{inputs} before rejection is performed. If \code{zero} is 2159 non-\code{NULL} and \code{applyZeroScale} is false, then the values 2160 shall only be used in calculating the Poisson variances. 2161 2162 If the \code{scale} vector is non-\code{NULL} and 2163 \code{applyZeroScale} is \code{true}, then the appropriate values 2164 shall multiply the \code{inputs} before rejection is performed. If 2165 \code{scale} is non-\code{NULL} and \code{applyZeroScale} is false, 2166 then the values shall only be used in calculating the Poisson 2167 variances. 2168 2169 The purpose of \code{applyZeroScale} is to allow combination of fringe 2170 frames, where the frames have been deliberately sky-subtracted and 2171 rescaled (to get the fringes amplitudes running from -1 to 1), which 2172 actions should not be undone when combining, but yet it is desirable 2173 to provide the \code{zero} and \code{scale} values so that the correct 2174 noise properties are used in the combination. 2175 2176 If the \code{gain} and \code{readnoise} are positive and non-negative 2177 (respectively), then these shall be used to provide weights for the 2178 combination using Poisson statistics ($\sigma_i$ below). 2179 2180 In summary, pixels corresponding to the same physical pixel are 2181 combined, having values $x_i \pm \sigma_i$. In the case that 2182 \code{applyZeroScale} is \code{true}, then: 2183 \begin{eqnarray} 2184 x_i & = & s_i f_i + z_i \\ 2185 \sigma_i & = & [g x_i + r^2]^{1/2} / g 2186 \end{eqnarray} 2187 Where $f_i$ is the value of the pixel in image $i$, $s_i$ is the scale 2188 applied to image $i$, $z_i$ is the zero offset applied to image $i$, 2189 $g$ is the gain, and $r$ is the read noise. If scales are not 2190 provided, they are set to unity; if zero offsets are not provided, 2191 they are set to zero. 2192 2193 If \code{applyZeroScale} is \code{false}, then the values are: 2194 \begin{eqnarray} 2195 x_i & = & f_i \\ 2196 \sigma_i & = & [g (s_i f_i + z_i) + r^2]^{1/2} / g 2197 \end{eqnarray} 2198 where the same symbols are used as above. 2199 2200 The \code{inputs, zero} and \code{scale} may be of U16, S32 and F32 2201 types, and must all be of the same type. The \code{output} shall be 2202 of the same type. 2203 2204 \subsection{Fringe Amplitude} 2205 2206 Some images contain a signal caused by thin-film interference in the 2207 device due to strong emission lines. The resulting instrumental 2208 effect consists of a pattern (the ``fringe pattern'') of bright and 2209 dark bands corresponding to the constructive and destructive 2210 interference of the emission lines. In the case that a single 2211 emission line causes the line structure, the resulting pattern can be 2212 described by two independent parameters: First, the amplitude of the 2213 emission line determines the overall amplitude of the pattern. 2214 Second, the three-dimensional surface structure of the device 2215 determines the shape of the pattern. In a typical situation, the 2216 device is illuminated by multiple emission lines, as well as a 2217 continuum spectral source, which contributes to the overall light 2218 detected by the device without following the fringe pattern. The 2219 relative intensities of the continuum background and the fringe 2220 pattern depend on the device structure (thickness) and on the ratio of 2221 the continuum and line emission fluxes. 2222 2223 A simple approach to the fringe pattern is to subtract a master fringe 2224 frame scaled by the amplitude of the fringe pattern. The amplitude of 2225 the fringe pattern is used both in the process of constructing the 2226 master image and in scaling the master image when it is applied to 2227 science image. This is the method currently in use at CFHT and it 2228 usually performs well. However, the fringe signal can vary as the 2229 emission lines in the atmosphere change, and the above method breaks 2230 down unless different fringe images corresponding to different 2231 atmospheric conditions are constructed. 2232 2233 An alternative technique is to use multiple master fringe images at 2234 the same time, each representing different atmospheric conditions. 2235 The observed fringe frame can be considered as a linear combination of 2236 different fringe patterns, depending on the relative strengths of the 2237 lines active in creating each of the fringe masters. It is not 2238 critical that the fringe master images represent completely orthogonal 2239 fringe patterns, they need only sample sufficiently different 2240 conditions to provide a handle on the underlying fringe signals. 2241 2242 We define a method of measuring the fringe pattern which is robust in 2243 the presence of stars and which is fast. We implement a varient on 2244 the method used at CFHT in which the fringe pattern is mapped by a 2245 series of points distributed across the image. At CFHT, manual effort 2246 is used to carefully define point pairs which correspond to peaks and 2247 valleys of the fringe pattern. We implement a different approach in 2248 which the fringe points are randomly chosen across the image. At each 2249 point in the image, the median flux is measured in a box of specified 2250 size. A low-frequency spatial filter is then applied to these 2251 measurements. The resulting array of points and fluxes then 2252 represents the strength of the fringe pattern on that image. The 2253 comparison between any two fringe images is then just a linear fit 2254 between these fringe statistics vectors, as follows: 2255 \[ 2256 S_i = C_0 + C_1 F_i 2257 \] 2258 where $S_i$ is the fringe statistic on the science image and $F_i$ is 2259 the fringe statistic on the reference fringe image. Extending this 2260 logic to any number of reference fringe images results in the 2261 following relationship: 2262 \[ 2263 S_i = C_0 + \sum_j C_j F_j 2264 \] 2265 2266 In order to correct a single science image, the collection of fringe 2267 statistics ($S_i$) are used to measure the coefficients $C_0$, $C_j$. 2268 The linear combination of the reference fringe images is then used to 2269 build a master image which is subtracted from the science image. The 2270 following structures and functions implement the above concepts. 2271 2272 The \code{pmFringeStats} structure represents the fringe statistics 2273 for a given image. 2274 \begin{datatype} 2275 typedef struct { 2276 psU32 nRequested; // number of fringe points selected 2277 psU32 nAccepted; // number of fringe points not masked 2278 psU32 dX; // median box half-width 2279 psU32 dY; // median box half-height 2280 psU32 nX; // large-scale smoothing in x (col) 2281 psU32 nY; // large-scale smoothing in y (row) 2282 psVector x; // fringe point coordinates (col) 2283 psVector y; // fringe point coordinates (row) 2284 psVector f; // fringe point median 2285 psVector df; // fringe point stdev 2286 psVector mask; // fringe point on/off mask 2287 } pmFringeStats; 2288 \end{datatype} 2289 2290 The \code{pmFringeStats} structure is allocated with the following 2291 function: 2292 \begin{prototype} 2293 pmFringeStats *pmFringeStatsAlloc ( 2294 int nPts, // number of points to create 2295 int dX, // half-width of fringe boxes 2296 int dY, // half-height of fringe boxes 2297 int nX, // smoothing scale in x 2298 int nY // smoothing scale in y 2299 ); 2300 \end{prototype} 2301 2302 A set of fringe points appropriate to the dimensions of a specific 2303 image are created with the following function: 2304 \begin{prototype} 2305 bool pmFringeStatsCreatePoints (pmFringeStats *fringe, psImage *image); 2306 \end{prototype} 2307 2308 In general, \code{pmFringeStatsCreatePoints} should only be needed 2309 when a new chip and filter are first use for analysis. Multiple 2310 fringe images with the same chip and filter need to be examined with 2311 the same fringe points in order for the statistical comparison to be 2312 meaningful. The constructed fringe points should be saved and loaded 2313 as a FITS table using the following function: 2314 \begin{prototype} 2315 bool pmFringeStatsWriteFits (psFits *fits, pmFringeStats *fringe); 2316 bool pmFringeStatsReadFits (psFits *fits, pmFringeStats *fringe); 2317 \end{prototype} 2318 2319 In order to measure the fringe statistics for a given image, the 2320 following function is defined: 2321 \begin{prototype} 2322 bool pmFringeStatsMeasure(pmFringeStats *fringe, pmReadout *readout) 2323 \end{prototype} 2324 This function measures the robust median at each of the fringe points 2325 and saves the median values in \code{fringe->f} and the scatter in 2326 \code{fringe->df}. 2327 2328 Given the fringe statistics for a science image, and the fringe 2329 statistics for a set of reference fringe images, the following 2330 function can be used to measure the scaling coefficients of the 2331 reference fringe frames which best fit the science image fringe 2332 pattern: 2333 \begin{prototype} 2334 pmFringeScale *pmFringeScaleMeasure (pmFringeStats *science, psArray *fringes) 2335 \end{prototype} 2336 2337 Given a science image, a set of master fringe images, and a the set of 2338 fringe statistics for the reference fringe images, the following 2339 function can be used to correct the science image for the fringe pattern: 2340 \begin{prototype} 2341 psImage *pmFringeCorrect(psImage *out, psMetadata *info, psImage *science, psArray *fringeImage, psArray *fringeStats); 2342 \end{prototype} 2343 2344 \subsection{Flat-field Re-Normalization} 2345 2346 Consider a collection of $N_i$ flat-field images obtained with a 2347 mosaic camera consisting of $N_j$ chips. Each image is exposed to an 2348 illumination source which should be a uniform surface 2349 brightness\footnote{This is likely a false assumption: the 2350 illumination source likely has spatial variations. However, for the 2351 purposes of this discussion, it only matters that such spatial 2352 variations scale consistently as a function of illumination intensity. 2353 The spatial errors are corrected by the photometric flat-field 2354 correction technique (eg., Magnier \& Cuillandre 2004).} Two factors 2355 determine the actual measured flux level (in Digital Numbers) on each 2356 of the chips in each image: the gain of each chip ($\mbox{gain}_j$) 2357 and the flux level from the illumination source ($\mbox{source}_i$). 2358 When the images are combined, the input images must be scaled so that 2359 the flux levels can be consistently compared. After combining the 2360 collection of images, it is necessary to determine an appropriate 2361 re-normalization for the resulting flat-field images. In effect, the 2362 individual chips must be adjusted so that the master flat-field image 2363 has a flux level which varies from chip to chip in proportion to the 2364 actual chip gain. In this case, if a uniform illumination source 2365 illuminates the mosaic, the resulting flux levels will be corrected by 2366 the flat-field to a single, consistent flux level. 2367 2368 In order to determine the correct relative scaling between the 2369 devices, it is thus necessary to know the individual chip gains, or at 2370 least the gain ratios. A typical technique scaled all chips relative 2371 to a reference chip, or by a statistic measured for the complete 2372 collection. These techniques fail if the input collection of images 2373 does not always consist of the same set of chips; for the GPC on 2374 Pan-STARRS, we must expect that individual cells or even chips may be 2375 disabled on a frequent basis, so our algorithms must not be limited by 2376 the assumption that all chips are available in all images. We 2377 therefore define the following algorithm to measure the relative chip 2378 gains for a collection of input flat-field images, each with a 2379 measured flux $\mbox{flux}_{i,j}$. We want to solve for the chip 2380 gains and the source illumination fluxes which would make the best 2381 prediction of the measured input image fluxes: 2382 \[ 2383 \mbox{flux}^{\rm pred}_{i,j} = \mbox{gain}_j \times \mbox{source}_i 2384 \] 2385 This relationship is easiest to determine if we take the logarithm of 2386 both sides of the equation: 2387 \[ 2388 M^{\rm pred}_{i,j} = G_j + S_i 2389 \] 2390 where $M^{\rm pred}_{i,j} = \log (\mbox{flux}^{\rm pred}_{i,j})$, $G_j 2391 = \log (\mbox{gain}_j)$, and $S_i = \log (\mbox{source}_i)$. We can 2392 then write the chi-square which we want to minimize as: 2393 \[ 2394 \chi^2 = \sum_{i,j} (M^{\rm obs}_{i,j} - G_j - S_i)^2 2395 \] 2396 where we ignore the weights of the different measured flux levels. 2397 Taking the derivatives with respect to the parameters of interest 2398 ($G_j, S_i$), and setting them to 0, we determine the following set of 2399 equations which must be solved: 2400 \[ 2401 G_j \times N_i = \sum_i M^{\rm obs}_{i,j} - \sum_i S_i 2402 \] 2403 \[ 2404 S_j \times N_j = \sum_j M^{\rm obs}_{i,j} - \sum_j G_j \\ 2405 \] 2406 This set of equations can be solved iteratively, starting from the 2407 assumption that all chip gains are 1.0, ($G_j = 0$), or by supplying 2408 a guess for the chip gains. The result of this analysis is the 2409 measured chip gains and the measured source illumination levels for 2410 each of the input flat-field images. The chip gains can then be used 2411 to modify the flux levels on the master flat-field images. 2412 2413 We define the following function to perform the analysis discussed 2414 above: 2415 \begin{prototype} 2416 bool pmFlatNormalization (psVector *sourceFlux, psVector *chipGains, psArray *fluxLevels); 2417 \end{prototype} 2418 The input array \code{fluxLevels} consists of $N_i$ vectors, one per 2419 mosaic image. Each vector consists of $N_j$ elements, each a 2420 measurement of the input flat-field image flux levels. All of these 2421 vectors must be constructed with the same number of elements, or the 2422 function will return an error. If a chip is missing from a particular 2423 image, that element should be set to \code{NaN}. The vector 2424 \code{chipGains} supplies initial guesses for the chip gains. If the 2425 vector contains the values 0.0 or \code{NaN} for any of the elements, 2426 the gain is set to the mean of the valid values. If the vector length 2427 does not match the number of chips, an warning is raised, all chip 2428 gain guesses will be set to 1.0, and the vector length modified to 2429 match the number of chips defined by the supplied \code{fluxLevels}. 2430 The \code{sourceFlux} input vector must be allocated (not 2431 \code{NULL}), but the routine will set the vector length to the number 2432 of source images regardless of the initial state of the vector. All 2433 vectors used by this function must be of type \code{PS_DATA_F64}. 2434 2435 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 2436 2437 \section{Objects on Images} 2438 2439 \subsection{Overview} 2440 2441 The process of finding, measuring, and classifying astronomical 2442 sources on images is one of the critical tasks of the IPP or any 2443 astronomical software system. In this section, we define structures 2444 and functions related to the task of source detection and measurement. 2445 The elements defined in this section are generally low-level 2446 components which can be connected together to construct a complete 2447 object measurement suite. 2448 2449 We first define the collection of structures needed to carry 2450 information about the detected sources. A major challenge is to 2451 define what we mean by an astronomical object in the context of image 2452 source detection. An astronomical object may be as simple as a 2453 stellar point source, or it may consist of a galaxy which has smooth 2454 extended structure; it may consist of an irregular galaxy or galaxy 2455 group with substantial and complex sub-structure, or it may consist of 2456 complex non-stellar structures such as planetary nebulae, reflection 2457 nebulae, outflows and jets. 2458 2459 The simplest objects (ie, stars) can be sufficiently modeled by the 2460 point-source function (PSF). More complex objects (such as simple, 2461 smooth galaxies), may have approximate analytical models which 2462 represent their morphology with more-or-less accuracy. In the extreme 2463 cases, the objects are not well modeled at all and must be represented 2464 in other ways. Thus, one aspect of our data structures must be 2465 elements to specify if an object has been represented by a model, what 2466 the model parameters are, and how well it is represented by the model. 2467 Another aspect of the data structures must be a representation of the 2468 pixels associated with the object so complex structures may be 2469 referenced without attempting to supply an analytical model. Finally, 2470 it is often useful to allow a single complex model to be represented 2471 as a collection of simpler contained structures which may be modeled. 2472 Thus, the representation of an object must be capable of identifying 2473 children, or substructures, of that object. 2474 2475 Two additional aspects must be considered. First, source detection 2476 need not be performed on a single image in isolation: it is necessary 2477 for multiple realizations of the same source in multiple images to be 2478 measured together (whether or not through simultaneous fitting in 2479 multiple bands or via application of the results from one image to 2480 another image). Second, it will be necessary to performed object 2481 measurements on pixels in which no source is actually detected. For 2482 example, this is a convenient way to provide flux upper limits at the 2483 locations of known objects. 2484 2485 In the discussion that follows, images are of type F32 and masks are 2486 of type U8. 2487 2488 \subsection{Structures to Describe Sources} 2489 2490 In the object analysis process, we will use specific mask values to 2491 mark the image pixels. The following structure defines the relevant 2492 mask values. 2493 \begin{datatype} 2494 typedef enum { 2495 PSPHOT_MASK_CLEAR = 0x00, 2496 PSPHOT_MASK_INVALID = 0x01, 2497 PSPHOT_MASK_SATURATED = 0x02, 2498 PSPHOT_MASK_MARKED = 0x08, 2499 } psphotMaskValues; 2500 \end{datatype} 2501 2502 \subsubsection{pmSource and pmPeak} 2503 2504 We define the following structure to represent a single source 2505 detected in a single image. 2506 \begin{datatype} 2507 typedef struct { 2508 pmPeak *peak; // description of peak pixel 2509 psImage *pixels; // rectangular region including object pixels 2510 psImage *weight; // Image variance. 2511 psImage *mask; // Mask which marks pixels associated with objects. 2512 pmMoments *moments; // basic moments measure for the object 2513 pmModel *modelPSF; // PSF model parameters and type 2514 pmModel *modelEXT; // FLT model parameters and type 2515 pmSourceType type; // Best identification of object 2516 pmSourceMode mode; // flags describing the model quality 2517 psArray *blends; // array of other sources blended with this source 2518 float apMag; // measured aperture magnitude for source 2519 float fitMag; // measured model magnitude for source 2520 psRegion region; // area on image covered by selected pixels 2521 } pmSource; 2522 \end{datatype} 2523 2524 A source has the capacity for several types of measurements. The 2525 simplest measurement of a source is the location and flux of the peak 2526 pixel associated with the source: 2527 \begin{datatype} 2528 typedef struct { 2529 int x; // x-coordinate of peak pixel 2530 int y; // y-coordinate of peak pixel 2531 float counts; // value of peak pixel (above sky?) 2532 pmPeakType class; // description of peak 2533 } pmPeak; 2534 \end{datatype} 2535 2536 A peak pixel may have several features which may be determined when 2537 the peak is found or measured. These are specified by the 2538 \code{pmPeakType} enum. \code{PM_PEAK_LONE} represents a single pixel 2539 which is higher than its 8 immediate neighbors. The 2540 \code{PM_PEAK_EDGE} represents a peak pixel which touching the image 2541 edge. The \code{PM_PEAK_FLAT} represents a peak pixel which has more 2542 than a specific number of neighbors at the same value, within some 2543 tolerance: 2544 \begin{datatype} 2545 typedef enum { 2546 PM_PEAK_LONE, // isolated peak 2547 PM_PEAK_EDGE, // peak on edge 2548 PM_PEAK_FLAT // peak has equal-value neighbors 2549 PM_PEAK_UNDEF // Undefined. 2550 } pmPeakType; 2551 \end{datatype} 2552 2553 \subsubsection{pmMoments and source description} 2554 2555 The pixels which contain the source are specified with the 2556 \code{psImage *pixels} element, a subimage of the image being 2557 analysed. Similarly, the \code{mask} element is a subimage of the 2558 corresponding mask image and the \code{weight} element is a subimage 2559 of the corresponding weight image (image varience). Since these are 2560 subimages, a collection of many objects may include overlapping 2561 pixels; care must be taken that pixel manipulations for one source do 2562 not unintentionally interfere with the other source pixels. The 2563 \code{mask} may be used to exclude any pixels which are not considered 2564 part of the source. Along with these pixel structures, we include the 2565 \code{psRegion region} element which defines the boundaries of the 2566 current associated subimages. 2567 2568 One of the simplest measurements which can be made quickly for an 2569 object are the object moments. We specify a structure to carry the 2570 moment information for a specific source: 2571 2572 \begin{datatype} 2573 typedef struct { 2574 float x; // x-coord of centroid 2575 float y; // y-coord of centroid 2576 float Sx; // x-second moment 2577 float Sy; // y-second moment 2578 float Sxy; // xy cross moment 2579 float Sum; // pixel sum above sky (background) 2580 float Peak; // peak counts above sky 2581 float Sky; // sky level (background) 2582 float SN; // approx signal-to-noise 2583 int nPixels; // number of pixels used 2584 } pmMoments; 2585 \end{datatype} 2586 2587 A collection of object moment measurements can be used to determine 2588 approximate object classes. The key to this analysis is the location 2589 and statistics (in the second-moment plane, $\sigma_x$ vs $\sigma_y$) 2590 of the group of objects which are likely PSF objects. We define the 2591 following structure to identify the location and size of the psf clump 2592 in the second-moment plane. 2593 \begin{datatype} 2594 typedef struct { 2595 float X; 2596 float dX; 2597 float Y; 2598 float dY; 2599 } pmPSFClump; 2600 \end{datatype} 2601 2602 A given source may be identified as most-likely to be one of several 2603 source types. The \code{pmSource} entry \code{pmSourceType} defines 2604 the current best-guess for this source. 2605 2606 \begin{datatype} 2607 typedef enum { 2608 PM_SOURCE_UNKNOWN, ///< no guess yet made 2609 PM_SOURCE_DEFECT, ///< a cosmic-ray 2610 PM_SOURCE_SATURATED, ///< random saturated pixels 2611 PM_SOURCE_STAR, ///< a good-quality star 2612 PM_SOURCE_EXTENDED, ///< an extended object (eg, galaxy) 2613 } pmSourceType; 2614 \end{datatype} 2615 2616 The related element, \code{pmSourceMode mode}, holds a collection of flags which 2617 are used to indicate the status of the analysis for a source. These 2618 are defined below: 2619 \begin{datatype} 2620 typedef enum { 2621 PM_SOURCE_DEFAULT = 0x0000, ///< no flags are set 2622 PM_SOURCE_PSFMODEL = 0x0001, ///< flags refer to the PSF model 2623 PM_SOURCE_EXTMODEL = 0x0002, ///< flags refer to the EXT model 2624 PM_SOURCE_SUBTRACTED = 0x0004, ///< the model has been subtracted from the image 2625 PM_SOURCE_FITTED = 0x0008, ///< the source has been fitted with a model 2626 PM_SOURCE_FAIL = 0x0010, ///< the model fit failed 2627 PM_SOURCE_POOR = 0x0020, ///< the model fit was poor (low S/N, etc) 2628 PM_SOURCE_PAIR = 0x0040, ///< the model fit is one of a paired source 2629 PM_SOURCE_PSFSTAR = 0x0080, ///< the source was used to construct the image PSF model 2630 PM_SOURCE_SATSTAR = 0x0100, ///< the source is saturated 2631 PM_SOURCE_BLEND = 0x0200, ///< the source is a blend with another source 2632 PM_SOURCE_LINEAR = 0x0400, ///< the source was fitted with the linear PSF model 2633 PM_SOURCE_TEMPSUB = 0x0800, ///< the source has been subtracted, but should be replaced 2634 } pmSourceMode; 2635 \end{datatype} 2636 2637 \subsubsection{pmModel Source Model and Abstraction} 2638 2639 An object's flux distribution may be modeled with some analytical 2640 function. The description of the model includes the model parameters 2641 and their errors, along with the fit $\chi^2$. The model type is 2642 identified by code \code{type}, dynamically assigned based on the 2643 available models (see below). We discuss the details of these models 2644 in section~\ref{ObjectModels}. The model parameters have 4 special 2645 elements. The first four elements represent aspects of the source 2646 which are not specified by the image PSF, even for point sources. 2647 These consist of, in order: 2648 \begin{itemize} 2649 \item the local sky 2650 \item the object normalization 2651 \item the x-coordinate 2652 \item the y-coordinate 2653 \end{itemize} 2654 2655 \tbd{should be include utility pointers to these parameters so that 2656 functions do not need to know the parameter sequence?} 2657 2658 The structure which carries the information about a given source model 2659 is defined below: 2660 \begin{datatype} 2661 typedef struct { 2662 pmModelType type; // model to be used 2663 psVector *params; // parameter values 2664 psVector *dparams; // parameter errors 2665 psF32 chisq; // fit chisq 2666 psS32 nDOF; // number of degrees of freedom 2667 psS32 nIter; // number of iterations 2668 pmModelStatus status; // fit status 2669 float radius; // fit radius actually used 2670 } pmModel; 2671 \end{datatype} 2672 2673 The \code{status} element carries the resulting success/failure status 2674 of an attempt to fit the model to the source: 2675 \begin{datatype} 2676 typedef enum { 2677 PM_MODEL_UNTRIED, ///< model fit not yet attempted 2678 PM_MODEL_SUCCESS, ///< model fit succeeded 2679 PM_MODEL_NONCONVERGE, ///< model fit did not converge 2680 PM_MODEL_OFFIMAGE, ///< model fit drove out of range 2681 PM_MODEL_BADARGS ///< model fit called with invalid args 2682 } pmModelStatus; 2683 \end{datatype} 2684 2685 We distinguish several ways in which an analytical model may be 2686 applied to a source. The PSF model represents the best fit of the 2687 image PSF to the specific object. In this case, the PSF-dependent 2688 parameters are specified for the object by the PSF, not by the fit. 2689 The EXT model represents the best fit of the given model to the 2690 object, with all parameters floating in the fit. Such a model would 2691 typically be used to represent and extended object, hence the 2692 abbreviation EXT. In some circumstances, a source may be fitted with 2693 a PSF model in which the position is held fixed, and not allowed to 2694 vary in the model fitting process. We identify such a model as FIX. 2695 Finally, we allow for the case in which two nearly-merged PSFs are 2696 fitted with a single 2-PSF model. We identify such a model as DBL. 2697 The \code{pmSource} structure contains a pointer to both a PSF and an 2698 EXT model, allowing any source to carry information about both 2699 possible fitting modes \tbd{not clear that we actually use this 2700 information; we might be better off simply distinguishing with one of 2701 the pmSourceMode flags}. The value of the model at a specific 2702 coordinate can be determined by calling the function: 2703 \begin{prototype} 2704 psF32 pmModelEval(pmModel *model, psImage *image, psS32 col, psS32 row); 2705 \end{prototype} 2706 For this function, the values of \code{col,row} are in the 2707 \code{image} coordinates, which may be a subimage, while the reference 2708 coordinate for the model is in the parent image coordinates. 2709 2710 In the \code{pmSource} structure, the elements \code{apMag} and 2711 \code{fitMag} are used to carry the measured magnitude of the source 2712 determined either from aperture photometry or from the integral of the 2713 fitted model function. The element \code{blends} is used to carry 2714 pointers to the collection of sources which were found to be blended 2715 with this source. Only the primary source of a blend group carries 2716 this information.%%% (see Section~\ref{blends}). 2717 2718 Every model instance belongs to a class of models, defined by the 2719 value of the \code{pmModelType type} entry. Various functions need 2720 access to information about each of the models. Some of this 2721 information varies from model to model, and may depend on the current 2722 parameter values or other data quantities. In order to keep the code 2723 from requiring the information about each model to be coded into the 2724 low-level fitting routines, we define a collection of functions which 2725 allow us to abstract this type of model-dependent information. These 2726 generic functions take the model type and return the corresponding 2727 function pointer for the specified model. Each 2728 model is defined by creating this collection of specific functions, 2729 and placing them in a single file for each model. We define the 2730 following structure to carry the collection of information about the 2731 models. 2732 2733 \begin{datatype} 2734 typedef struct { 2735 char *name; 2736 int nParams; 2737 pmModelFunc modelFunc; 2738 pmModelFlux modelFlux; 2739 pmModelRadius modelRadius; 2740 pmModelLimits modelLimits; 2741 pmModelGuessFunc modelGuessFunc; 2742 pmModelFromPSFFunc modelFromPSFFunc; 2743 pmModelFitStatusFunc modelFitStatusFunc; 2744 } pmModelGroup; 2745 \end{datatype} 2746 2747 Each entry in the \code{pmModelGroup} defines the information needed 2748 by the system to specify a model. The function types define above are 2749 \begin{prototype} 2750 typedef psMinimizeLMChi2Func pmModelFunc; 2751 typedef psF64 (*pmModelFlux)(const psVector *params); 2752 typedef psF64 (*pmModelRadius)(const psVector *params, double flux); 2753 typedef bool (*pmModelLimits)(psVector **beta_lim, psVector **params_min, psVector **params_max); 2754 typedef bool (*pmModelGuessFunc)(pmModel *model, pmSource *source); 2755 typedef bool (*pmModelFromPSFFunc)(pmModel *modelPSF, pmModel *modelFLT, pmPSF *psf); 2756 typedef bool (*pmModelFitStatusFunc)(pmModel *model); 2757 \end{prototype} 2758 2759 Each of these functions is found for a given model by calling the 2760 corresponding lookup function: 2761 \begin{prototype} 2762 pmModelFunc pmModelFunc_GetFunction (pmModelType type); 2763 pmModelFlux pmModelFlux_GetFunction (pmModelType type); 2764 pmModelRadius pmModelRadius_GetFunction (pmModelType type); 2765 pmModelLimits pmModelLimits_GetFunction (pmModelType type); 2766 pmModelGuessFunc pmModelGuessFunc_GetFunction (pmModelType type); 2767 pmModelFromPSFFunc pmModelFromPSFFunc_GetFunction (pmModelType type); 2768 pmModelFitStatusFunc pmModelFitStatusFunc_GetFunction (pmModelType type); 2769 \end{prototype} 2770 2771 \code{pmModelFunc} is the function used to determine the value of the 2772 model at a specific coordinate, and is the one used by 2773 \code{psMinimizeLMChi2}. 2774 2775 \code{pmModelFlux} returns the total integrated flux for the given 2776 input parameters. 2777 2778 \code{pmModelRadius} returns the scaling radius at which the flux of 2779 the model matches the specified flux. This presumes that the model is 2780 a function of an elliptical contour. 2781 2782 \code{pmModelLimits} sets the parameter limit vectors for the 2783 function. 2784 2785 \code{pmModelGuessFunc} generates an initial guess for the model based 2786 on the provided source statistics (moments and pixel values as 2787 needed). 2788 2789 \code{pmModelFromPSFFunc} takes as input a representation of the psf 2790 and a value for the model and fills in the PSF parameters of the 2791 model. The input primarily relies upon the centroid coordinates of 2792 the input model, thought the normalization may potentially be used. 2793 2794 \code{pmModelFitStatusFunc} returns a true or false values based on 2795 the success or failure of a model fit. the success is determined by 2796 quantities such as the chisq or the signal-to-noise. 2797 2798 In addition, the following functions are useful for interacting with 2799 the collection of models: 2800 \begin{prototype} 2801 int pmModelParameterCount (pmModelType type); 2802 \end{prototype} 2803 This function returns the number of parameters used by the listed 2804 function. 2805 2806 \begin{prototype} 2807 char *pmModelGetType (pmModelType type); 2808 pmModelType pmModelSetType (char *name); 2809 \end{prototype} 2810 These two functions provide translations between the user-space model 2811 names and the internal model type codes. The model type codes are not 2812 necessarily maintained between compilations of the program; the name 2813 should be used to transfer models between programs or systems. 2814 2815 \subsubsection{pmGrowthCurve} 2816 2817 When the photometry of source is measured in a fixed aperture, there 2818 is always a fraction of the source light which falls outside of the 2819 aperture. The resulting aperture magnitude is thus larger (ie, 2820 fainter) than the actual source. As the aperture is increased, the 2821 amount of loss decreases and the measured magnitude increases. This 2822 trend is the curve of growth for the source. We use the following 2823 structure to carry information about the curve of growth. We use the 2824 PSF model to measure the curve of growth for an image. 2825 2826 \begin{datatype} 2827 typedef struct { 2828 psVector *radius; 2829 psVector *apMag; 2830 psF32 refRadius; 2831 psF32 maxRadius; 2832 psF32 fitMag; 2833 psF32 apRef; // apMag[refRadius] 2834 psF32 apLoss; // fitMag - apRef 2835 } pmGrowthCurve; 2836 \end{datatype} 2837 In this structure, \code{radius} is a monotonically increasing 2838 sequence of radius values (in pixels). The \code{apMag} vector 2839 contains the measured magnitude at any of these radius: this is the 2840 curve-of-growth trend. The remaining entries summaries the 2841 relationship: \code{refRadius} is the global reference radius used for 2842 this image; \code{maxRadius} is the outermost radius at which the 2843 curve of growth was measured; \code{fitMag} is the fitted PSF model 2844 magnitude integrated to infinity; \code{apRef} is the aperture 2845 magnitude at the reference radius; \code{apLoss} is the difference 2846 between the aperture magnitude at the reference radius and the fitted 2847 model magnitude. A few related functions are specified to interact 2848 with the curve of growth: 2849 2850 \begin{prototype} 2851 pmGrowthCurve *pmGrowthCurveAlloc (psF32 minRadius, psF32 maxRadius, psF32 dRadius); 2852 \end{prototype} 2853 This function allocates a \code{pmGrowthCurve} structure and fills in 2854 the \code{radius} vector (see psLib SDRS \code{psVectorCreate}). It 2855 does {\em not} allocate the \code{apMag} vector. 2856 2857 \begin{prototype} 2858 psF32 pmGrowthCurveCorrect (pmGrowthCurve *growth, psF32 radius); 2859 \end{prototype} 2860 This function accepts a \code{growth} curve structure and returns the 2861 correction between the specified radius and the reference radius 2862 ($apMag(refRadius) - apMag(radius)$). 2863 2864 The following two functions are used to search the growth curve to the 2865 corresponding radius entry: 2866 \begin{prototype} 2867 int psVectorBracket (psVector *index, psF32 key, bool above); 2868 psF32 psVectorInterpolate (psVector *index, psVector *value, psF32 key); 2869 \end{prototype} 2870 2871 \subsubsection{Aperture Trends} 2872 2873 With PSF model fitting, there is always some discrepancy between the 2874 model of the PSF and the actual PSF. As a result, the measured flux 2875 from the model will not represent exactly the flux of the source. It 2876 is necessary to measure the correction between the model and the 2877 actual source flux. One way to perform this measurement is to compare 2878 the model flux with the flux measured for bright stars within a fixed 2879 aperture. The quantity to be measured is $dA = m_{\rm aperture} - 2880 m_{\rm fit}$. In practice, $dA$ exhibits variations as a function of 2881 the source position ($x,y$) and the source flux. The variations as a 2882 function of source position can be understood as a change in the PSF 2883 model error as a function of position due to the changing shape of the 2884 PSF (despite the varying PSF model, it is possible that the fitted 2885 model yields positional variations in the residual flux). The 2886 variations in $dA$ as a function of magnitude can be understood as the 2887 result of a bias in the local background measurement (for the fainter 2888 sources) and as a result of non-linearity in the detector setting on 2889 the bright end. We use a 4D polynomial to represent these trends. 2890 The first two dimensions of the polynomial represent the variation of 2891 $dA$ as a function of $x,y$; we provide helper functions to define 1st 2892 and 2nd order polynomials in $x,y$. The next two dimensions are 2893 fitted independently (no cross terms). The first represents the 2894 variation as a function of $r^2 / flux$, where $r$ is the aperture 2895 radius used to measure $dA$; this is the scaling of a magnitude error 2896 in the presence of a constant error in the sky level. The last 2897 dimension represents the variation of $dA$ as a function of the 2898 stellar flux. 2899 2900 The following forms of the aperture correction model may be selected 2901 by the user: 2902 \begin{datatype} 2903 typedef enum { 2904 PM_PSF_NONE, 2905 PM_PSF_CONSTANT, 2906 PM_PSF_SKYBIAS, 2907 PM_PSF_SKYSAT, 2908 PM_PSF_XY_LIN, 2909 PM_PSF_XY_QUAD, 2910 PM_PSF_SKY_XY_LIN, 2911 PM_PSF_SKYSAT_XY_LIN, 2912 PM_PSF_ALL 2913 } pmPSF_ApTrendOptions; 2914 \end{datatype} 2915 2916 The following utility function sets the aperture correction model 2917 coefficient masks to select the specific desired coefficients: 2918 \begin{prototype} 2919 bool pmPSF_MaskApTrend (pmPSF *psf, pmPSF_ApTrendOptions option); 2920 \end{prototype} 2921 2922 \subsubsection{pmPSF, pmPSFtry, and PSF model} 2923 2924 It is useful to generate a model to define the point-spread-function 2925 which describes the flux distribution for unresolved sources in an 2926 image. In general, the PSF varies with position in the image. We 2927 allow any of the source models defined for the \code{pmModel} to 2928 represent the PSF. For a given source model, the 2D spatial variation 2929 of all of the source parameters, except the first four PSF-independent 2930 parameters, are represented as polynomial, stored in a \code{psArray}. 2931 The structure also contains the aperture correction model 2932 (\code{ApTrend}) and the curve-of-growth model (\code{growth}). The 2933 additional elements are: \code{ApResid}, the constant term in the 2934 aperture correction model; \code{dApResid}, the residual scatter for 2935 bright sources ($S/N > 100$) after applying the aperture correction; 2936 \code{skyBias}, the measured average bias in the sky measurement; 2937 \code{skySat}, the scaling of the flux-dependent portion of the 2938 correction. 2939 2940 The other elements of the structure define the quality of the PSF 2941 determination. 2942 2943 \begin{datatype} 2944 typedef struct { 2945 pmModelType type; ///< PSF Model in use 2946 psArray *params; ///< Model parameters (psPolynomial2D) 2947 psPolynomial4D *ApTrend; ///< ApResid vs (x,y,rflux) (rflux = ten(0.4*mInst) 2948 pmGrowthCurve *growth; ///< apMag vs Radius 2949 float ApResid; ///< ??? 2950 float dApResid; ///< ??? 2951 float skyBias; ///< ??? 2952 float skySat; ///< ??? 2953 float chisq; ///< PSF goodness statistic 2954 int nPSFstars; ///< number of stars used to measure PSF 2955 int nApResid; ///< number of stars used to measure ApResid 2956 } pmPSF; 2957 \end{datatype} 2958 2959 \begin{prototype} 2960 pmModel *pmModelFromPSF (pmModel *model, pmPSF *psf); 2961 \end{prototype} 2962 This function constructs a \code{pmModel} instance based on the 2963 \code{pmPSF} description of the PSF. The input is a \code{pmModel} 2964 with at least the values of the centroid coordinates (possibly 2965 normalization if this is needed) defined. The values of the 2966 PSF-dependent parameters are specified for the specific realization 2967 based on the coordinates of the object. 2968 2969 \begin{prototype} 2970 bool pmPSFFromModels (pmPSF *psf, psArray *models, psVector *mask); 2971 \end{prototype} 2972 This function takes a collection of \code{pmModel} fitted models from 2973 across a single image and builds a \code{pmPSF} representation of the 2974 PSF. The input array of model fits may consist of entries to be 2975 ignored (noted by a non-zero \code{mask} entry). The analysis of the 2976 models fits a 2D polynomial for each parameter to the collection of 2977 model parameters as a function of position (and normalization?). In 2978 this process, some of the input models may be marked as outliers and 2979 excluded from the fit. These elements will be marked with a specific 2980 mask value (1 == \code{PSFTRY_MASK_OUTLIER}). 2981 2982 We definet he following two functions to convert the PSF model 2983 parameters into a collection of elements on a metadata structure, and 2984 vice versa. These can be used to read and write PSFs to a file and or 2985 a database. 2986 \begin{prototype} 2987 psMetadata *pmPSFtoMD (psMetadata *metadata, pmPSF *psf); 2988 pmPSF *pmPSFfromMD (psMetadata *metadata); 2989 \end{prototype} 2990 2991 We have the capability to test several different model functions in an 2992 attempt to build an accurate PSF for an image. The complete set of 2993 data needed to build and test as specific PSF model is carried by the 2994 \code{pmPSFtry} structure: 2995 \begin{datatype} 2996 typedef struct { 2997 pmModelType modelType; 2998 pmPSF *psf; 2999 psArray *sources; // pointers to the original sources 3000 psArray *modelEXT; // model fits, floating parameters 3001 psArray *modelPSF; // model fits, PSF parameters 3002 psVector *mask; 3003 psVector *metric; 3004 psVector *fitMag; 3005 } pmPSFtry; 3006 \end{datatype} 3007 This structure contains a pointer to the collection of \code{sources} 3008 which will be used to test the PSF model form. It lists the 3009 \code{pmModelType type} of model being tests, and contains an element 3010 to store the resulting \code{psf} representation. In addition, this 3011 structure carries the complete collection of FLT (floating parameter) 3012 and PSF (fixed parameter) model fits to each of the sources 3013 \code{modelFLT} and \code{modelPSF}. It also contains a mask which is 3014 set by the model fitting and psf fitting steps. For each model, the 3015 value of the quality metric is stored in the vector \code{metric} and 3016 the fitted instrumental magnitude is stored in \code{fitMag}. The 3017 quality metric for the PSF model is the aperture magnitude minus the 3018 fitted magnitude for each source. 3019 3020 This collection of aperture residuals is examined in the analysis 3021 process, and a linear trend of the residual with the inverse object 3022 flux (ie, $10^{0.4*mag}$) is fitted. The result of this fit is a 3023 measured sky bias (systematic error in the sky measured by the fits), 3024 an effective infinite-magnitude aperture correction (\code{ApResid}), 3025 and the scatter of the aperture correction for the ensemble of PSF 3026 stars (\code{dApResid}). The ultimate metric to intercompare multiple 3027 types of PSF models is the value of the aperture correction scatter. 3028 3029 The following functions are used to try out a single PSF model. 3030 \begin{prototype} 3031 pmPSFtry *pmPSFtryModel (psArray *sources, char *modelName, float RADIUS); 3032 \end{prototype} 3033 This function takes the input collection of sources and performs a 3034 complete analysis to determine a PSF model of the given type 3035 (specified by model name). The result is a \code{pmPSFtry} with the 3036 results of the analysis. 3037 3038 \begin{prototype} 3039 bool pmPSFtryMetric (pmPSFtry *try, float RADIUS); 3040 \end{prototype} 3041 This function is used to measure the PSF model metric for the set of 3042 results contained in the \code{pmPSFtry} structure. 3043 3044 The following datatype defines the masks used by the \code{pmPSFtry} 3045 analysis to identify sources which should or should not be included in 3046 the analysis. 3047 \begin{datatype} 3048 enum { 3049 PSFTRY_MASK_CLEAR = 0x00, 3050 PSFTRY_MASK_OUTLIER = 0x01, // 1: outlier in psf polynomial fit (provided by psPolynomials) 3051 PSFTRY_MASK_EXT_FAIL = 0x02, // 2: ext model failed to converge 3052 PSFTRY_MASK_PSF_FAIL = 0x04, // 3: psf model failed to converge 3053 PSFTRY_MASK_BAD_PHOT = 0x08, // 4: invalid source photometry 3054 PSFTRY_MASK_ALL = 0x0f, 3055 } pmPSFtryMaskValues; 3056 \end{datatype} 3057 3058 3059 \begin{datatype} 3060 typedef enum { 3061 PM_CONTOUR_CRUDE 3062 } pmContourType; 3063 \end{datatype} 3064 3065 Allocators for the above structures are defined as follows: 3066 \begin{prototype} 3067 pmSource *pmSourceAlloc (); 3068 pmPeak *pmPeakAlloc (int x, int y, float counts, psPeakType class); 3069 pmMoments *pmMomentsAlloc (); 3070 pmModel *pmModelAlloc (pmModelType type); 3071 \end{prototype} 3072 3073 \subsection{Basic Object Detection APIs} 3074 3075 In this section, we specify a collection of basic functions which 3076 operate on images and sources. We define them roughly in order in 3077 which we expect to use them in a basic object detection process. 3078 3079 \begin{prototype} 3080 psVector *pmFindVectorPeaks(const psVector *vector, float threshold); 3081 \end{prototype} 3082 3083 Find all local peaks in the given vector above the given threshold. A 3084 peak is defined as any element with a value greater than its two 3085 neighbors and with a value above the threshold. Two types of special 3086 cases must be addressed. Equal value elements: If an element has the 3087 same value as the following element, it is not considered a peak. If 3088 an element has the same value as the preceding element (but not the 3089 following), then it is considered a peak. Note that this rule 3090 (arbitrarily) identifies flat regions by their trailing edge. Edge 3091 cases: At start of the vector, the element must be higher than its 3092 neighbor. At the end of the vector, the element must be higher or 3093 equal to its neighbor. These two rules again places the peak 3094 associated with a flat region which touches the image edge at the 3095 image edge. The result of this function is a vector containing the 3096 coordinates (element number) of the detected peaks (type 3097 \code{psU32}). 3098 3099 \begin{prototype} 3100 psArray *pmFindImagePeaks(const psImage *image, float threshold); 3101 \end{prototype} 3102 3103 Find all local peaks in the given image above the given threshold. 3104 This function should find all row peaks using 3105 \code{pmFindVectorPeaks}, then test each row peak and exclude peaks 3106 which are not local peaks. A peak is a local peak if it has a higher 3107 value than all 8 neighbors. If the peak has the same value as its +y 3108 neighbor or +x neighbor, it is NOT a local peak. If any other 3109 neighbors have an equal value, the peak is considered a valid peak. 3110 Note two points: first, the +x neighbor condition is already enforced 3111 by \code{pmFindVectorPeaks}. Second, these rules have the effect of 3112 making flat-topped regions have single peaks at the (+x,+y) corner. 3113 When selecting the peaks, their type must also be set. The result of 3114 this function is an array of \code{pmPeak} entries. The resulting set 3115 of peaks should be considered a starting point, not an unambiguous 3116 sample of the only real peaks. If the input image is a subimage, the 3117 output peak coordinates should be in the {\em parent} coordinate 3118 frame. 3119 3120 \begin{prototype} 3121 psArray *pmPeaksSubset(psArray *peaks, float maxvalue, const psRegion valid); 3122 \end{prototype} 3123 3124 Create a new peaks array, removing certain types of peaks from the 3125 input array of peaks based on the given criteria. Peaks should be 3126 eliminated if they have a peak value above the given maximum value 3127 limit or if the fall outside the valid region. The result of the 3128 function is a new array with a reduced number of peaks. 3129 3130 \begin{prototype} 3131 bool pmSourceDefinePixels(pmSource *mySource, 3132 pmReadout *readout, 3133 psF32 x, 3134 psF32 y, 3135 psF32 Radius) 3136 3137 bool pmSourceRedefinePixels(pmSource *mySource, 3138 pmReadout *readout, 3139 psF32 x, 3140 psF32 y, 3141 psF32 Radius) 3142 \end{prototype} 3143 3144 The first form defines \code{psImage} subarrays (pixel, weight, and 3145 mask) for the source located at coordinates \code{x,y} on the image 3146 set defined by \code{readout} (in parent coords). The pixels defined 3147 by this operation consist of a square window (of full width $2 Radius 3148 + 1$) centered on the pixel which contains the given coordinate, in 3149 the frame of the readout. The window is defined to have limits which 3150 are valid within the boundary of the \code{readout} image, thus if the 3151 radius would fall outside the image pixels, the subimage is truncated 3152 to only consist of valid pixels. If \code{readout->mask} or 3153 \code{readout->weight} are not \code{NULL}, matching subimages are 3154 defined for those images as well. This function fails if no valid 3155 pixels can be defined (x or y less than Radius, for example). This 3156 function should be used to define a region of interest around a 3157 source, including both source and sky pixels. The second form accepts 3158 an existing source and redefines the pixels if the requested radius 3159 encompasses more pixels than the existing images. 3160 3161 \begin{prototype} 3162 pmSource *pmSourceLocalSky(pmSource *source, 3163 psStatsOptions statsOptions, 3164 float Radius) 3165 \end{prototype} 3166 3167 Measure the local sky in the vicinity of the given \code{source}. The 3168 \code{Radius} defines the square aperture in which the moments will be 3169 measured. This function assumes the source pixels have been defined, 3170 and that the value of \code{Radius} here is smaller than the value of 3171 \code{Radius} used to define the pixels. The annular region not 3172 contained within the radius defined here is used to measure the local 3173 background in the vicinity of the source. The local background 3174 measurement uses the specified statistic passed in via the 3175 \code{statsOptions} entry. This function allocates the 3176 \code{pmMoments} structure. The resulting sky is used to set the 3177 value of the \code{pmMoments.sky} element of the provided 3178 \code{pmSource} structure. 3179 3180 \begin{prototype} 3181 bool pmSourceMoments(pmSource *source, float radius); 3182 \end{prototype} 3183 3184 Measure source moments for the given \code{source}, using the value of 3185 \code{source.moments.sky} provided as the local background value and 3186 the peak coordinates as the initial source location. The resulting 3187 moment values are applied to the \code{source.moments} entry, and the 3188 source is returned. The moments are measured within the given 3189 circular radius of the \code{source.peak} coordinates. The return 3190 value indicates the success (TRUE) of the operation. This function 3191 also measures the approximate signal-to-noise ratio of the source 3192 (\code{source.SN}) based on the total number of source counts divided 3193 by the square-root of the total source variance, as determined from 3194 the weight image. 3195 3196 \begin{prototype} 3197 pmPSFClump pmSourcePSFClump(psArray *sources, psMetadata *metadata); 3198 \end{prototype} 3199 3200 We use the source moments to make an initial, approximate source 3201 classification, and as part of the information needed to build a PSF 3202 model for the image. As long as the PSF shape does not vary 3203 excessively across the image, the sources which are represented by a 3204 PSF (the start) will have very similar second moments. The function 3205 \code{pmSourcePSFClump} searches a collection of \code{sources} with 3206 measured moments for a group with moments which are all very similar. 3207 The function returns a \code{pmPSFClump} structure, representing the 3208 centroid and size of the clump in the $\sigma_x$, $\sigma_y$ 3209 second-moment plane. 3210 3211 The goal is to identify and characterize the stellar clump within the 3212 $\sigma_x, \sigma_y$ plane. To do this, an image is constructed to 3213 represent this plane. The units of $\sigma_x$ and $\sigma_y$ are in 3214 image pixels. A pixel in this analysis image represents 0.1 pixels in 3215 the input image. The dimensions of the image need only be 10 pixels. 3216 The peak pixel in this image (above a threshold of half of the image 3217 maximum) is found. The coordinates of this peak pixel represent the 3218 2D mode of the $\sigma_x, \sigma_y$ distribution. The sources with 3219 $\sigma_x, \sigma_y$ within 0.2 pixels of this value are then used to 3220 calculate the median and standard deviation of the $\sigma_x, 3221 \sigma_y$ values. These resulting values are returned via the 3222 \code{pmPSFClump} structure. 3223 3224 The return value indicates the success (TRUE) of the operation. 3225 3226 \tbd{limit the S/N of the candidate sources (part of Metadata)?} 3227 3228 \tbd{save the clump parameters on the Metadata} 3229 3230 \begin{prototype} 3231 bool pmSourceRoughClass(psArray *sources, psMetadata *metadata, pmPSFClump clump) 3232 \end{prototype} 3233 3234 Based on the specified data values, make a guess at the source 3235 classification. The sources are provides as a \code{psArray} of 3236 \code{pmSource} entries. Definable parameters needed to make the 3237 classification are provided to the routine with the \code{psMetadata} 3238 structure. The rules below refer to values which can be extracted 3239 from the metadata using the given keywords. Except as noted, the data 3240 type for these parameters are \code{psF32}. 3241 3242 The following rules are used to make the classification. The number 3243 of saturated pixels are counted, based on the mask having the 3244 \code{PSPHOT_MASK_SATURATED} bit set. Sources which are greater than 3245 1$\sigma$ larger than the \code{pmPSFClump} center in both dimensions 3246 and which have more than a single saturated pixel are identified as 3247 being a likely saturated star (\code{type = PM_SOURCE_STAR, mode = 3248 PM_SOURCE_SATSTAR}). Sources which are not so large but which have 3249 multiple saturated pixels are identified as saturated regions, ie 3250 bleed trails or hot columns (\code{type = PM_SOURCE_SATURATED}). 3251 3252 Sources with 3253 \[ \sigma_x < 0.05 \] 3254 or 3255 \[ \sigma_y < 0.05\] 3256 should be identified as type \code{PM_SOURCE_DEFECT} (likely cosmic 3257 ray pixel). 3258 3259 Sources with 3260 \[ \sigma_x > \mbox{CLUMP}_{x} + 3\mbox{CLUMP}_{dx}\] 3261 and 3262 \[ \sigma_y > \mbox{CLUMP}_{y} + 3\mbox{CLUMP}_{dy}\] 3263 should be identified as type \code{PM_SOURCE_EXTENDED}. 3264 3265 All other sources should be identified as type \code{PM_SOURCE_STAR}. 3266 Of these sources, the mode should be set to \code{PM_SOURCE_PSFSTAR} 3267 for any sources with $SN$ greater than \code{PSF_SN_LIM} which are 3268 within 1.5$\sigma$ of the PSF clump center. These sources are used to 3269 determine a guess at the shape of the PSF, based on the collection of 3270 $\sigma_x$ and $\sigma_y$ values. 3271 3272 \subsection{Object Fitting} 3273 3274 We need a way to fit a particular functional model to an object. 3275 PSLib includes the \code{psMinimizeLMChi2} and \code{psMinimizePowell} 3276 functions, which form the core of this processes. However, additional 3277 support functions and wrapping functions are necessary for the 3278 specific case of source fitting. The operations can be broken down 3279 into discrete steps: 3280 3281 \begin{enumerate} 3282 \item Identify the pixels of interest 3283 3284 \item Make a guess at the model parameters. For some models, the 3285 parameters may be guessed based on only the moments. For others, 3286 additional measurements must be made. 3287 3288 \item Construct the input vectors from the pixels of interest. 3289 3290 \item Apply fitting function \code{psMinimizeLMChi2()} 3291 3292 \item Construct model image. 3293 3294 \item Subtract model from image. 3295 \end{enumerate} 3296 3297 \begin{prototype} 3298 bool pmSourceModelGuess(pmSource *source, const psImage *image, pmModelType model); 3299 \end{prototype} 3300 3301 Convert available data to an initial guess for the given model. This 3302 function allocates a \code{pmModel} entry for the \code{pmSource} 3303 structure based on the provided model selection. The method of 3304 defining the model parameter guesses are determined by using 3305 \code{pmModelGuessFunc_GetFunction} to determine the guess function 3306 for the model of interest. The returned function is called and the 3307 guess values are used to set the model parameters. The function 3308 returns \code{TRUE} on success or \code{FALSE} on failure. 3309 3310 \begin{prototype} 3311 psArray *pmSourceContour(const pmSource *source, const psImage *image, float level, pmContourType type); 3312 \end{prototype} 3313 3314 Find points in a contour for the given source at the given level. If 3315 \code{type} is \code{PM_CONTOUR_CRUDE}, the contour is found by starting at 3316 the source peak, running along each pixel row until the level is 3317 crossed, then interpolating to the level coordinate for that row. 3318 This is done for each row, with the starting point determined by the 3319 midpoint of the previous row, until the starting point has a value 3320 below the contour level. The returned contour consists of two vectors 3321 giving the x and y coordinates of the contour levels. This function 3322 may be used as part of the model guess inputs. 3323 3324 \tbd{Other contour types may be specified in the future for more refined contours} 3325 3326 \begin{prototype} 3327 bool pmSourceFitModel(pmSource *source, psImage *image); 3328 \end{prototype} 3329 3330 Fit the requested model to the specified source. The starting guess 3331 for the model is given by the input \code{source.model} parameter 3332 values. The pixels of interest are specified by the 3333 \code{source.pixels} and \code{source.mask} entries. This function 3334 calls \code{psMinimizeLMChi2()} on the image data. The function 3335 returns \code{TRUE} on success or \code{FALSE} on failure. 3336 3337 \begin{prototype} 3338 bool pmModelFitStatus (pmModel *model); 3339 \end{prototype} 3340 3341 This function wraps the call to the model-specific function returned 3342 by \code{pmModelFitStatusFunc_GetFunction}. The model-specific 3343 function examines the model parameters, parameter errors, Chisq, S/N, 3344 and other parameters available from \code{model} to decide if the 3345 particular fit was successful or not. 3346 3347 \begin{prototype} 3348 bool pmSourceAddModel(psImage *image, pmSource *source, bool center, bool sky); 3349 bool pmSourceSubModel(psImage *image, pmSource *source, bool center, bool sky); 3350 \end{prototype} 3351 3352 Add or subtract the given source model flux to/from the provided 3353 image. The boolean option \code{center} selects if the source is 3354 re-centered to the image center or if it is placed at its centroid 3355 location. The boolean option \code{sky} selects if the background sky 3356 is applied (\code{TRUE}) or not. The pixel range in the target image 3357 is at most the pixel range specified by the \code{source.pixels} 3358 image. The success status is returned. 3359 3360 \begin{prototype} 3361 bool pmSourcePhotometry (float *fitMag, // integrated fit magnitude 3362 float *obsMag, // aperture flux magnitude 3363 pmModel *model, // model used for photometry 3364 psImage *image, // image pixels to be used 3365 psImage *mask // mask of pixels to ignore 3366 ); 3367 \end{prototype} 3368 3369 The function returns both the magnitude of the fit, defined as $-2.5 3370 \log{\rm flux}$, where the flux is integrated under the model, 3371 theoretically from a radius of 0 to infinity. In practice, we 3372 integrate the model beyond $50 \sigma$. The aperture magnitude is 3373 defined as $-2.5 \log{\rm flux}$, where the flux is summed for all 3374 pixels which are not excluded by the aperture mask. The model flux is 3375 calculated by calling the model-specific function provided by 3376 \code{pmModelFlux_GetFunction}. 3377 3378 \begin{prototype} 3379 int pmSourceDophotType (pmSource *source); 3380 \end{prototype} 3381 This function converts the source classification into the closest 3382 available approximation to the Dophot classification scheme. The 3383 following list gives the correspondence: 3384 \begin{verbatim} 3385 PM_SOURCE_DEFECT: 8 3386 PM_SOURCE_SATURATED: 8 3387 PM_SOURCE_SATSTAR: 10 3388 PM_SOURCE_PSFSTAR: 1 3389 PM_SOURCE_GOODSTAR: 1 3390 PM_SOURCE_POOR_FIT_PSF: 7 3391 PM_SOURCE_FAIL_FIT_PSF: 4 3392 PM_SOURCE_FAINTSTAR: 4 3393 PM_SOURCE_GALAXY: 2 3394 PM_SOURCE_FAINT_GALAXY: 2 3395 PM_SOURCE_DROP_GALAXY: 2 3396 PM_SOURCE_FAIL_FIT_GAL: 2 3397 PM_SOURCE_POOR_FIT_GAL: 2 3398 PM_SOURCE_OTHER: ? 3399 \end{verbatim} 3400 3401 \begin{prototype} 3402 int pmSourceSextractType (pmSource *source); 3403 \end{prototype} 3404 This function converts the source classification into the closest 3405 available approximation to the Sextractor classification scheme. 3406 \tbd{the correspondence is not yet defined}. 3407 3408 \subsection{Object List Input/Output} 3409 3410 We support several object catalog formats. Some of these mimic the 3411 formats used by the Elixir system to support testing with existing 3412 data and software. Some of these are for use by the Pan-STARRS 3413 project for testing. 3414 3415 \subsubsection{OBJ Format} 3416 3417 This format is produced by versions of DoPhot and is used by the 3418 Elixir system as an intermediate output data product. The objects are 3419 written to a text file with fixed line-length and with fixed column 3420 positions. The file has no header associated with it. This is only 3421 an output format, and should be used just for testing and comparison 3422 with the Elixir tools. 3423 3424 \subsubsection{SX Format} 3425 3426 This format is produced by versions of Sextractor and is used by the 3427 Elixir system as an intermediate output data product. The objects are 3428 written to a text file with fixed line-length and with fixed column 3429 positions. The file has no header associated with it. This is only 3430 an output format, and should be used just for testing and comparison 3431 with the Elixir tools. The SX and OBJ formats are similar, but use a 3432 somewhat different definition of the columns. 3433 3434 \subsubsection{CMP Format} 3435 3436 This format is used extensively by the Elixir system, and many data 3437 files are available in this format. The format is a pseudo-FITS 3438 format, consisting of a FITS header (with NAXIS=2) and a text data 3439 segment with fixed line length. The CMP files are always in SPLIT 3440 format in the sense that each object table is a single file. 3441 3442 \subsubsection{CMF Format} 3443 3444 This format is a true FITS table format. The object data is stored 3445 for each readout in a separate extension. In addition, the Cell 3446 headers are stored in their own extensions (with NAXIS=0). In SPLIT 3447 format, the Cell header is the PHU header. 3448 3449 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 3450 3451 \section{Image Combination} 3452 3453 The image combination for \PS{} will employ an iterative approach, in 3454 order to identify cosmic rays. The first pass involves transforming 3455 and combining the input images, and noting pixels which are apparently 3456 deviant. These pixels are examined in further detail, before a subset 3457 of them are declared to be bad, whereupon these pixels are 3458 re-transformed, and the images are combined properly. Here we 3459 introduce two functions which will perform the combination and 3460 examination steps. Prototype code exists for each of these functions. 3461 \tbd{For further details, see the document about image combination for 3462 \PS{}.} 3463 3464 \subsection{Combining images} 3465 3466 \begin{prototype} 3467 psImage *pmCombineImages(psImage *combined, // Combined image 3468 psArray **questionablePixels, // Array of rejection masks 3469 const psArray *images, // Array of input images 3470 const psArray *errors, // Array of input error images 3471 const psArray *masks,// Array of input masks 3472 unsigned int maskVal, // Mask value 3473 const psPixels *pixels, // Pixels to combine 3474 int numIter, // Number of rejection iterations 3475 float sigmaClip, // Number of standard deviations at which to reject 3476 const psStats *stats // Statistics to use in the combination 3477 ); 3478 \end{prototype} 3479 3480 \code{pmCombineImages} shall combine the input \code{images}, 3481 returning the \code{combined} image and a list of 3482 \code{questionablePixels} in each input image. The array of error 3483 images, \code{errors}, shall be used to calculate the value in the 3484 combined image and the list of questionable pixels, if 3485 non-\code{NULL}. Pixels whose corresponding value in the array of 3486 mask images, \code{masks}, matches \code{maskVal} shall be masked from 3487 the combination. The \code{images}, \code{errors} and \code{masks} 3488 arrays, if non-\code{NULL}, shall all carry the same number of images; 3489 otherwise the function shall generate an error and return \code{NULL}. 3490 The sizes of all images in the \code{images}, \code{errors} and 3491 \code{masks} arrays shall be identical; otherwise the function shall 3492 generate an error and return \code{NULL}. 3493 3494 If \code{pixels} is non-\code{NULL}, only those pixels specified shall 3495 be combined. The combination consists of \code{numIter} iterations in 3496 which a stack of pixels is combined using the specified \code{stats}. 3497 In each iteration, questionable pixels are identified as lying more 3498 than \code{sigmaClip} standard deviations from the combined value; 3499 these pixels are excluded from the stack for the next iteration. The 3500 value for the combined image is that produced by the \textit{first} 3501 iteration (i.e., with no pixels excluded except those which have their 3502 corresponding mask match the \code{maskVal}); this allows subsequent 3503 calls to the function to only act on a small fraction of the pixels, 3504 since questionable pixels identified in the first call of the function 3505 will be properly rejected at a later point (see the example, below). 3506 3507 In the event that \code{images} or \code{stats} are \code{NULL}, the 3508 function shall generate an error and return \code{NULL}. 3509 3510 \subsection{Rejecting pixels} 3511 3512 \begin{prototype} 3513 psArray *pmRejectPixels(const psArray *images, // Array of input images 3514 const psArray *masks, // Array of masks for input images 3515 const psArray *pixels, // These are the pixels which were rejected in the combination 3516 const psArray *inToOut, // Transformations from input to output system 3517 const psArray *outToIn, // Transformations from output to input system 3518 float rejThreshold, // Rejection threshold 3519 float gradLimit // Gradient limit 3520 ); 3521 \end{prototype} 3522 3523 \tbd{This algorithm will change: an addition will be made to avoid 3524 masking pixels in the wings of a star when combining images taken in 3525 different seeing, and the gradient limit criteria will be changed.} 3526 3527 \code{pmRejectPixels} inspects those questionable \code{pixels} 3528 identified by \code{pmCombineImages} to determine if they are truly 3529 discrepant. This inspection is performed in the coordinate frame of 3530 the detector, where the pixels haven't been smeared by transformation. 3531 Two tests are applied to each of the \code{images}: 3532 \begin{enumerate} 3533 \item The list of questionable pixels for an image is converted to an 3534 image which is transformed back to the coordinate frame of the 3535 detector. Those pixels in the detector frame which have a value 3536 exceeding \code{rejThreshold} are suspected cosmic rays and 3537 subjected to the next test. Depending on the value of the 3538 \code{rejThreshold}, this test basically amounts to demanding that 3539 questionable pixels neighbor each other in the transformed image. 3540 \item The cores of point sources may mimic a cosmic ray, especially in 3541 under-sampled images. To minimize flagging stars as cosmic rays, we 3542 determine the gradient around the pixel of interest; if the gradient 3543 is large, then the pixel is likely the core of a point source. In 3544 order to reliably measure the gradient in the presence of a 3545 suspected cosmic ray, we use the companion images --- the gradient 3546 is the mean gradient at the corresponding position on the other 3547 images. In order to calculate the corresponding positions, the 3548 \code{inToOut} and \code{outToIn} transformations are required. If 3549 the gradient is less than \code{gradLimit}, then the pixel is 3550 identified as a cosmic ray. 3551 \end{enumerate} 3552 3553 The function shall return an array of \code{psPixels}, one for each of 3554 the input \code{images}, containing pixels that have been identified 3555 as cosmic rays according to the above criteria. 3556 3557 If any of the input pointers are \code{NULL}, then the function shall 3558 generate an error and return \code{NULL}. 3559 3560 \subsection{Example} 3561 3562 Here is an example of what the image combination routine looks like, 3563 demonstrating how the various pieces fit together. The inputs are: 3564 \begin{itemize} 3565 \item \code{psArray *inputs}: Input detector images, each a 3566 \code{psImage} of type \code{psF32} 3567 \item \code{psArray *inputMask}: Input mask images, each a 3568 \code{psImage} of type \code{psU8} 3569 \item \code{psArray *inputsErr}: Input error images, each a 3570 \code{psImage} of type \code{psF32} 3571 \item \code{psPlaneTransform *skyToDetector}: Maps from sky 3572 coordinates to detector coordinates, each a \code{psPlaneTransform} 3573 \item \code{psRegion *combineRegion}: Sky coordinate pixels to combine 3574 \item \code{int numIter}: Number of iterations in combination 3575 \item \code{float rejThreshold}: Threshold for rejection 3576 \item \code{float gradLimit}: Limit for gradient 3577 \end{itemize} 3578 3579 The output is the combined image. 3580 3581 \begin{verbatim} 3582 psArray *transformed = psArrayAlloc(nImages); // Array of transformed images 3583 psArray *transformedErr = psArrayAlloc(nImages); // Array of transformed error images 3584 psArray *transformedMask = psArrayAlloc(nImages); // Array of masks for transformed images 3585 3586 for (int i = 0; i < nImages; i++) { 3587 psPixels *blanks = NULL; // List of blank pixels 3588 transformed->data[i] = psImageTransform(NULL, &blanks, inputs->data[i], 3589 inputMask->data[i], inputMaskVal, NAN, skyToDetector, 3590 combineRegion, NULL, PS_INTERPOLATE_BILINEAR); 3591 transformedErr->data[i] = psImageTransform(NULL, NULL, inputsErr->data[i], inputMask->data[i], 3592 inputMaskVal, NAN, skyToDetector, combineRegion, NULL, 3593 PS_INTERPOLATE_BILINEAR_VARIANCE); 3594 psImage *skyImage = transformed->data[i]; // Dereference the transformed image 3595 psRegion *blankRegion = psRegionAlloc(0, 0, skyImage->numCols, skyImage->numRows); // Size of 3596 // transformed 3597 // image 3598 transformedMask->data[i] = psPixelsToMask(NULL, blanks, *blankRegion, PS_MASK_BLANK); 3599 psFree(blankRegion); 3600 psFree(blanks); 3601 } 3602 3603 psArray *rejected = NULL; // Array of rejected pixel lists 3604 psStats *combineStats = psStatsAlloc(PS_STAT_SAMPLE_MEAN); // Statistic to use in doing the combination 3605 psImage *combined = pmCombineImages(NULL, &rejected, transformed, transformedErr, transformedMask, 0, 3606 NULL, numIter, sigmaClip, combineStats); // Combined image 3607 psArray *bad = pmRejectPixels(inputs, rejected, NULL, skyToDetector, rejThreshold, gradLimit); // Bad pix 3608 psPixels *combinePixels = NULL; // Pixels to combine 3609 for (int i = 0; i < nImages; i++) { 3610 psPixels *badSource = psPixelsTransform(NULL, bad->data[i], skyToDetector); // Bad pixels on the input 3611 psImage *badMask = psPixelsToMask(NULL, badSource, PS_MASK_COSMICRAY); // Mask image for the input 3612 (void)psBinaryOp(inputMask->data[i], inputMask->data[i], "|", badMask); // Put CRs into original mask 3613 psFree(badSource); 3614 psFree(badMask); 3615 3616 combinePixels = psPixelsConcatenate(redo, bad->data[i]); 3617 3618 // Update transformed image 3619 psPixels *blanks = NULL; // List of blank pixels 3620 transformed->data[i] = psImageTransform(transformed->data[i], &blanks, inputs->data[i], 3621 inputMask->data[i], inputMaskVal | PS_MASK_COSMICRAY, NAN, 3622 skyToDetector, combineRegion, bad->data[i], 3623 PS_INTERPOLATE_BILINEAR); 3624 transformedErr->data[i] = psImageTransform(transformedErr->data[i], NULL, inputsErr->data[i], 3625 inputMask->data[i], inputMaskVal | PS_MASK_COSMICRAY, 3626 NAN, skyToDetector, combineRegion, bad->data[i], 3627 PS_INTERPOLATE_BILINEAR_VARIANCE); 3628 psImage *skyImage = transformed->data[i]; // Dereference the transformed image 3629 psRegion *blankRegion = psRegionAlloc(0, 0, skyImage->numCols, skyImage->numRows); // Size of 3630 // transformed 3631 // image 3632 transformedMask->data[i] = psPixelsToMask(transformedMask->data[i], blanks, *blankRegion, 3633 PS_MASK_BLANK); 3634 psFree(blankRegion); 3635 psFree(blanks); 3636 } 3637 psFree(bad); 3638 3639 // Combine with no rejection 3640 combined = pmCombineImages(combined, NULL, transformed, transformedErr, transformedMask, 3641 PS_MASK_BLANK, combinePixels, 0, 0.0, combineStats); 3642 psFree(combineStats); 3643 psFree(combinePixels); 3644 psFree(transformed); 3645 psFree(transformedErr); 3646 psFree(transformedMask); 3647 \end{verbatim} 3648 3649 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 3650 3651 \section{Image Subtraction} 3652 3653 Image subtraction is arguably the best method of identifying faint 3654 variable sources in images with different point-spread functions. It 3655 relies on fitting for a convolution kernel that minimizes the 3656 residuals in subtracting small regions of the image. The use of a 3657 convolution kernel consisting of a linear combination of basis 3658 functions allows the problem to be solved with only modest computing 3659 power. 3660 3661 \subsection{The kernels} 3662 3663 We will allow for the use of two convolution kernels. The first is 3664 that employed by the popular image subtraction program, 3665 \href{http://www2.iap.fr/users/alard/package.html}{ISIS}, consisting 3666 of Gaussians modified by polynomials: 3667 \begin{equation} 3668 B_{ijk}(u,v) = e^{-(u^2 + v^2)/2\sigma_i^2} u^j v^k 3669 \end{equation} 3670 The second simply consists of delta functions, which we refer to as 3671 POIS (Pan-STARRS Optimal Image Subtraction): 3672 \begin{equation} 3673 B_{ij}(u,v) = \delta(u - i)\ \delta(v - j) 3674 \end{equation} 3675 \tbd{For further details, see the document about image subtraction for 3676 \PS{}.} The former is widely used, while the second appears to be 3677 equally useful and faster, though not as tried and proven. 3678 3679 \begin{datatype} 3680 typedef enum { 3681 PM_SUBTRACTION_KERNEL_POIS, // POIS kernel --- delta functions 3682 PM_SUBTRACTION_KERNEL_ISIS // ISIS kernel --- gaussians modified by polynomials 3683 } pmSubtractionKernelsType; 3684 \end{datatype} 3685 3686 In order to simplify the book-keeping for the kernels, we will define 3687 a \code{pmSubtractionKernels}, which keeps track of the details of the 3688 each of the kernel basis functions: 3689 3690 \begin{datatype} 3691 typedef struct { 3692 pmSubtractionKernelType type; // Type of kernels --- allowing the use of multiple kernels 3693 int size; // Size of kernel in x and y 3694 int spatialOrder; // Maximum order of spatial variations 3695 psVector *u, *v; // Offset (for POIS) or polynomial order (for ISIS) 3696 psVector *sigma; // Width of Gaussian (for ISIS) 3697 psVector *xOrder, *yOrder; // Spatial polynomial order (for all) 3698 int subIndex; // Index of kernel to be subtracted to maintain flux conservation 3699 psArray *preCalc; // Array of images containing pre-calculated kernel (to 3700 // accelerate ISIS; don't use for POIS) 3701 } pmSubtractionKernels; 3702 \end{datatype} 3703 3704 This structure caters for both choices of kernel type. For a POIS 3705 kernel, the \code{u} and \code{v} vectors shall be set to the 3706 coordinates for the delta functions for the corresponding kernel. For 3707 an ISIS kernel, the \code{sigma} vector shall be set to the Gaussian 3708 widths and the \code{u} and \code{v} vectors shall be set to the 3709 orders of the modifying polynomials for the corresponding kernel. For 3710 both choices of kernel, the \code{xOrder} and \code{yOrder} vectors 3711 specify the order of the spatial variation. 3712 3713 In order to maintain flux conservation when the kernel is spatially 3714 variable, we need to treat one kernel in the set differently. The 3715 convolutions for this kernel, identified by the \code{subIndex}, are 3716 calculated in the usual way, while all others have the \code{subIndex} 3717 kernel subtracted from them. For details, see the 3718 \href{http://www.edpsciences.org/journal/index.cfm?v_url=aas/full/2000/11/ds8706/ds8706.html}{paper 3719 by Alard (2000, A\&AS, 144, 363)}. 3720 3721 Since the ISIS kernels are continuous functions, it is worth 3722 pre-calculating them instead of calculating them each time they are 3723 required. The \code{preCalc} array, consisting of \code{psImage}s is 3724 provided for this purpose. 3725 3726 The \code{pmSubtractionKernels} are generated by the following functions: 3727 3728 \begin{prototype} 3729 pmSubtractionKernels *pmSubtractionKernelsAllocPOIS(int size, int spatialOrder); 3730 pmSubtractionKernels *pmSubtractionKernelsAllocISIS(const psVector *sigmas, const psVector *orders, 3731 int size, int spatialOrder); 3732 \end{prototype} 3733 3734 \code{pmSubtractionKernelsAllocPOIS} shall generate the 3735 \code{pmSubtractionKernels} suitable for the POIS kernel basis set. 3736 This involves setting the \code{u}, \code{v}, \code{xOrder} and 3737 \code{yOrder} to the appropriate values. \code{size} is the half-size 3738 of the kernel, and \code{spatialOrder} is the maximum spatial order 3739 (the spatial variation is $x^i y^j$ with $i+j <$ \code{spatialOrder}). 3740 The \code{subIndex} is set to the kernel which has \code{u = 0}, 3741 \code{v = 0}, \code{xOrder = 0} and \code{yOrder = 0}. There should 3742 be \code{(2 * size + 1) * (2 * size + 1) * (spatialOrder + 1) * 3743 (spatialOrder + 2) / 2} kernels. 3744 3745 \code{pmSubtractionKernelsAllocISIS} shall generate the 3746 \code{pmSubtractionKernels} suitable for the ISIS kernel basis set. 3747 This involves setting the \code{sigma}, \code{u}, \code{v}, 3748 \code{xOrder} and \code{yOrder} to the appropriate values, as well as 3749 generating the \code{preCalc} images. Note that the \code{sigma} 3750 vector contained within the \code{pmSubtractionKernels} is not the 3751 same as the input \code{sigmas} vector, but contains repeated entries. 3752 \code{size} is the half-size of the kernel, which specifies the size 3753 of the \code{preCalc} images. The \code{spatialOrder} is the maximum 3754 spatial order (the spatial variation is $x^i y^j$ with $i+j <$ 3755 \code{spatialOrder}). The \code{subIndex} is set to the kernel which 3756 has \code{u = 0}, \code{v = 0}, \code{xOrder = 0} and \code{yOrder = 3757 0}, for the first of the Gaussian widths in the \code{sigmas} vector. 3758 3759 \subsection{Stamps} 3760 3761 Sub-regions on an image which are used to derive the best-fit 3762 convolution kernel are referred to as ``stamps''. 3763 3764 \begin{datatype} 3765 typedef struct { 3766 int x, y; // Position 3767 psImage *matrix; // Associated matrix 3768 psVector *vector; // Associated vector 3769 pmStampStatus status; // Status of stamp 3770 } pmStamp; 3771 \end{datatype} 3772 3773 A stamp is the region around a central pixel, \code{x,y}. The 3774 \code{matrix} and \code{vector} are generated in the process of 3775 solving for the best-fit convolution kernel; each of these will likely 3776 be of type \code{psF64} in order to maintain the best possible 3777 precision (we will be summing squares). In order to allow us to throw 3778 out stamps without having to laboriously recompute the total 3779 least-squares matrix and vector, we use a separate matrix and vector 3780 for each stamp. 3781 3782 To allow iteration on the choice of stamps, a stamp contains a 3783 \code{status}, an enumerated type: 3784 3785 \begin{datatype} 3786 typedef enum { 3787 PM_STAMP_USED, // Use this stamp 3788 PM_STAMP_REJECTED, // This stamp has been rejected 3789 PM_STAMP_RECALC, // Having been reset, this stamp needs to be recalculated 3790 PM_STAMP_NONE // No stamp in this region 3791 } pmStampStatus; 3792 \end{datatype} 3793 3794 \begin{prototype} 3795 psArray *pmSubtractionFindStamps(psArray *stamps, // Output stamps, or NULL 3796 const psImage *image, // Image for which to find stamps 3797 const psImage *mask, // Mask 3798 unsigned int maskVal, // Value for mask 3799 float threshold, // Threshold for stamps in the image 3800 int xNum, int yNum, // Number of stamps in x and y 3801 int border // Border around image to ignore (should be size of kernel) 3802 ); 3803 \end{prototype} 3804 3805 \code{pmSubtractionFindStamps} returns an array of stamps on the 3806 \code{image} suitable for use in calculating the best-fit convolution 3807 kernel. Except for a \code{border} all the way around, the 3808 \code{image} is broken into \code{xNum} $\times$ \code{yNum} 3809 rectangles; there will be a stamp within each rectangle. If 3810 \code{stamps} is non-\code{NULL}, then the function shall only attempt 3811 to identify a new stamp in a particular rectangle if the corresponding 3812 stamp \code{status} is \code{PM_STAMP_REJECTED}. 3813 3814 A stamp shall be recognized as the pixel with the greatest value that 3815 does not have the corresponding pixel in the \code{mask} matching 3816 \code{maskVal}. If the value of the this pixel does not exceed 3817 \code{threshold}, then the stamp \code{status} shall be marked as 3818 \code{PM_STAMP_NONE}, which means that the stamp will be ignored in 3819 future iterations. If a legitimate stamp is found within the region, 3820 then its status shall be changed to \code{PM_STAMP_RECALC}. 3821 3822 3823 \subsection{Solving for the kernel} 3824 3825 Calculating the best-fit convolution kernel requires solving a matrix 3826 equation, the elements of which are obtained by applying the kernel 3827 basis functions to the stamps. The final matrix and vector are the 3828 sum of the matrices and vectors obtained for each of the individual 3829 stamps. 3830 3831 \begin{prototype} 3832 bool pmSubtractionCalculateEquation(psArray *stamps, // The stamps for which to calculate the equation 3833 const psImage *reference, // Reference image 3834 const psImage *input, // Input image 3835 const psSubtractionKernels *kernels, // The kernel basis functions 3836 int footprint // Half-size of region over which to calculate equation 3837 ); 3838 \end{prototype} 3839 3840 \code{pmSubtractionCalculateEquation} shall calculate the 3841 \code{matrix} and \code{vector} for each of the \code{stamps} which 3842 have \code{status} set to \code{PM_STAMP_RECALC}. The calculation is 3843 made over a region with a half size of \code{footprint} on the 3844 \code{reference} and \code{input} images, using each of the 3845 \code{kernels}. In the event that any of the input pointers are 3846 \code{NULL}, the function shall generate an error and return 3847 \code{false}; otherwise, the function shall return \code{true}. 3848 3849 The vector is: 3850 \begin{equation} 3851 v_i = \sum_{x,y} I(x,y) [ R(x,y) \otimes B_i(u,v) ] / \sigma(x,y)^2 3852 \end{equation} 3853 and the matrix is: 3854 \begin{equation} 3855 M_{ij} = \sum_{x,y} \left[ R(x,y) \otimes B_i(u,v) \right] \ \left[ R(x,y) \otimes B_j(u,v) \right] / \sigma(x,y)^2 3856 \end{equation} 3857 where $I(x,y)$ is the input image, $R(x,y)$ is the reference image, 3858 $B_i(u,v)$ is the $i$-th kernel basis function, $\otimes$ denotes 3859 convolution, $\sigma(x,y) = R(x,y)^{1/2}$ is an estimate of the error, 3860 and the sum over $x,y$ indicates summing over the stamp regions. 3861 3862 In addition to the each of the \code{kernels}, an additional parameter 3863 for which we must solve is the difference in the background level 3864 between the \code{reference} and \code{input} images. The appropriate 3865 term shall be added to the \code{matrix} and \code{vector}. 3866 3867 In order to maintain flux conservation when the kernel is spatially 3868 variable, for each of the kernel basis functions apart from the first, 3869 the kernel actually employed shall be the first kernel function 3870 subtracted from the original kernel function. 3871 3872 Having calculated the matrix equation for a stamp, its \code{status} 3873 is set to \code{PM_STAMP_USED}. 3874 3875 Since this step is one of the major rate-limiting factors in image 3876 subtraction, care should be taken with optimization. 3877 3878 \begin{prototype} 3879 psVector *pmSubtractionSolveEquation(psVector *solution, // Solution vector, or NULL 3880 const psArray *stamps // Array of stamps 3881 ); 3882 \end{prototype} 3883 3884 \code{pmSubtractionSolveEquation} shall solve the matrix equation 3885 provided by each of the \code{stamps}, returning the \code{solution} 3886 vector. This involves summing the \code{matrix} and \code{vector} of 3887 each of the stamps which have \code{status} set to 3888 \code{PM_STAMP_USED}, and multiplying the inverse of the matrix by the 3889 \code{vector}. If the \code{solution} is \code{NULL}, then the 3890 function shall allocate and return a new vector; otherwise, the 3891 \code{solution} vector shall be modified in-place. If \code{stamps} 3892 is \code{NULL}, then the function shall generate an error and return 3893 \code{NULL}. The type of the \code{solution} vector should be 3894 \code{psF64}, since the matrix equation involves summing squares. 3895 3896 3897 \subsection{Rejection of stamps} 3898 3899 \begin{prototype} 3900 bool pmSubtractionRejectStamps(psArray *stamps, // Array of stamps to check for rejection 3901 psImage *mask, // Mask image 3902 unsigned int badStampMaskVal, // Value to use in mask for bad stamp 3903 int footprint, // Region to mask if stamp is bad 3904 float sigmaRej, // Number of RMS deviations above zero at which to reject 3905 const psImage *refImage, // Reference image 3906 const psImage *inImage, // Input image 3907 const psVector *solution, // Solution vector 3908 const pmSubtractionKernels *kernels // Array of kernel parameters 3909 ); 3910 \end{prototype} 3911 3912 \code{pmSubtractionRejectStamps} shall apply the \code{solution} to 3913 the \code{stamps}, rejecting stamps for which the mean square 3914 residuals exceed \code{sigmaRej} RMS deviations from zero. 3915 \code{stamps} which are rejected have their \code{status} set to 3916 \code{PM_STAMP_REJECTED}, and have pixels within \code{footprint} of 3917 the corresponding position in the \code{mask} set to 3918 \code{badStampMaskVal} so they will not be used again. 3919 3920 The deviations are calculated through extracting the stamps from the 3921 \code{refImage} and \code{inImage}, convolving the reference stamp by 3922 the best-fit kernel (derived from the \code{solutions} vector and the 3923 \code{kernels}), subtracting and then dividing by the stamp from the 3924 input image, and then squaring to obtain the mean square residual. 3925 3926 \subsection{Visualization of kernel} 3927 3928 Having solved for the best-fit kernel, it is often useful to visualize 3929 it. 3930 3931 \begin{prototype} 3932 psImage *pmSubtractionKernelImage(psImage *out, const psVector *solution, 3933 const pmSubtractionKernels *kernels, float x, float y); 3934 \end{prototype} 3935 3936 \code{pmSubtractionKernelImage} shall create an image of the kernel 3937 from the \code{solution} vector and the \code{kernels}. The relative 3938 position (between -1 and +1) on the image at which to evaluate the 3939 kernel (important if the kernel is spatially variable) is specified by 3940 \code{x} and \code{y}. If \code{out} is \code{NULL}, then the 3941 function shall allocate a new image of sufficient size (matching the 3942 \code{precalc} images), and return the result; otherwise, \code{out} 3943 shall be modified in-place. 3944 3945 3946 \subsection{Example} 3947 3948 Here is an example of what the image subtraction routine looks like, 3949 demonstrating how the various pieces fit together. The inputs are: 3950 \begin{itemize} 3951 \item \code{psImage *reference}: Reference image 3952 \item \code{psImage *refMask}: Mask for reference image 3953 \item \code{psImage *input}: Input image 3954 \item \code{psImage *inMask}: Mask for input image 3955 \item \code{unsigned int maskVal}: Value to be masked 3956 \item \code{pmSubtractionKernelType kernelType}: Type of kernel to use 3957 \item \code{int kernelHalfSize}: Half the kernel size (full size is \code{2*kernelHalfSize + 1}) 3958 \item \code{psVector *sigmas}: Widths for the ISIS Gaussians 3959 \item \code{psVector *polyOrders}: Polynomial orders for ISIS Gaussians 3960 \item \code{int spatialOrder}: Maximum spatial order for spatially variable kernel 3961 \item \code{float stampThreshold}: Threshold for finding stamps 3962 \item \code{int nStampsX, nStampsY}: Number of stamps in x and y 3963 \item \code{int stampSize}: Half size of stamp footprint 3964 \item \code{int numIter}: Number of iterations on the stamps 3965 \item \code{float sigmaRej}: Rejection threshold for stamps 3966 \end{itemize} 3967 3968 The output is the subtracted image and the corresponding mask. 3969 3970 \begin{verbatim} 3971 // Mask around bad pixels in the reference image. There are two cases to worry about: 3972 // 1. Bad pixels within the kernel, which will affect the subtracted image 3973 // 2. Bad pixels within the stamp, which affects the calculation of the kernel 3974 psImage *subMask = psImageGrowMask(NULL, refMask, maskVal, kernelHalfSize, PS_MASK_NEAR_BAD); 3975 (void)psImageGrowMask(subMask, refMask, maskVal, stampSize, PS_MASK_BAD_STAMP); 3976 // Add in the mask for the input image. Don't need to grow this, since it isn't convolved. 3977 (void)psBinaryOp(subMask, subMask, "|", inMask); 3978 3979 // Generate kernel basis functions 3980 psArray *kernels = NULL; // Array of kernel basis functions 3981 switch (kernelType) { 3982 case PM_SUBTRACTION_KERNEL_POIS: 3983 // Create the kernel basis functions 3984 kernels = pmSubtractionKernelsGeneratePOIS(kernelHalfSize, spatialOrder); 3985 break; 3986 case PM_SUBTRACTION_KERNEL_ISIS: 3987 kernels = pmSubtractionKernelsGenerateISIS(sigmas, polyOrders, kernelHalfSize, spatialOrder); 3988 break; 3989 default: 3990 barf(); 3991 } 3992 3993 psArray *stamps = NULL; // Array of stamps 3994 psVector *kernelCoeffs = NULL; // Coefficients for the kernels 3995 bool rejected = true; // Did we reject a stamp in the last iteration? 3996 3997 // Iterate for a solution 3998 for (int iter = 0; iter < numIter && rejected; iter++) { 3999 4000 // Find stamps 4001 stamps = pmSubtractionFindStamps(stamps, reference, subMask, maskVal | PS_MASK_BAD_STAMP, 4002 stampThreshold, nStampsX, nStampsY, stampSize, kernelHalfSize); 4003 4004 // Generate and solve matrix equations 4005 (void)pmSubtractionCalculateEquation(stamps, reference, input, kernels, stampSize); 4006 kernelCoeffs = pmSubtractionSolveEquation(kernelCoeffs, stamps); 4007 4008 // Reject bad stamps 4009 rejected = pmSubtractionRejectStamps(stamps, subMask, PS_MASK_BAD_STAMP, stampSize, sigmaRej, 4010 reference, input, kernelCoeffs, kernels); 4011 } 4012 4013 // Convolve the reference image 4014 psImage *referenceConvolved = pmSubtractionConvolveImage(NULL, reference, subMask, kernelCoeffs, kernels); 4015 // Subtract 4016 psImage *subtracted = (psImage*)psBinaryOp(NULL, input, "-", referenceConvolved); 4017 4018 // What does the kernel look like? 4019 psImage *kernelImage = pmSubtractionKernelImage(NULL, kernelCoeffs, kernels, 0.0, 0.0); 4020 // Check/save kernel image, print statistics.... 4021 4022 psFree(referenceConvolved); 4023 psFree(stamps); 4024 psFree(kernels); 4025 psFree(kernelCoeffs); 4026 \end{verbatim} 4027 4028 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 4029 4030 \appendix 4031 4032 \section{Basic Object Models} 4033 \label{ObjectModels} 4034 4035 We specify a variety of basic object models which are required. 4036 Details of the model functional forms, parameters, and the derivatives 4037 are specified in the ADD. 4038 4039 \subsubsection{Real 2D Gaussian} 4040 4041 \begin{prototype} 4042 float pmMinLM_Gauss2D(psVector *deriv, psVector *params, psVector *x); 4043 \end{prototype} 4044 4045 This function is a two-dimensional Gaussian with an elliptical 4046 cross-section and a constant local background. 4047 4048 The initial guess for the Gaussian parameters may be taken from the 4049 moments, peak value, and local sky. 4050 4051 \subsubsection{Pseudo-Gaussian} 4052 4053 \begin{prototype} 4054 float pmMinLM_PseudoGauss2D(psVector *deriv, psVector *params, psVector *x); 4055 \end{prototype} 4056 4057 This function is a polynomial approximation of a 2D Gaussian otherwise 4058 very similar to the real Gaussian. It is used in place of a real 4059 Gaussian for speed. 4060 4061 The initial guess for the Gaussian parameters may be taken from the 4062 moments, peak value, and local sky. 4063 4064 \subsubsection{Waussian} 4065 4066 \begin{prototype} 4067 float pmMinLM_Wauss2D(psVector *deriv, psVector *params, psVector *x); 4068 \end{prototype} 4069 4070 The Waussian is a modified polynomial approximation of a 2D Gaussian, 4071 with non-linear polynomial terms having variable coefficients, rather 4072 than the Taylor series values of 1/2 and 1/6. 4073 4074 \subsubsection{Twisted Gaussian} 4075 4076 \begin{prototype} 4077 float pmMinLM_TwistGauss2D(psVector *deriv, psVector *params, psVector *x); 4078 \end{prototype} 4079 4080 This function describes an object with power-law wings and a flattened 4081 core, where the core has a different contour from the wings. 4082 4083 The initial guess for the Gaussian parameters may be taken from the 4084 moments, peak value, and local sky. 4085 4086 \tbd{future galaxy models to be implemented} 4087 4088 \subsubsection{Sersic Galaxy Model} 4089 4090 \begin{prototype} 4091 float pmMinLM_Sersic(psVector *deriv, psVector *params, psVector *x); 4092 \end{prototype} 4093 4094 \subsubsection{Sersic with Core Galaxy Model} 4095 4096 \begin{prototype} 4097 float pmMinLM_SersicCore(psVector *deriv, psVector *params, psVector *x); 4098 \end{prototype} 4099 4100 \subsubsection{Pseudo Sersic Galaxy Model} 4101 4102 \begin{prototype} 4103 float pmMinLM_PseudoSersic(psVector *deriv, psVector *params, psVector *x); 4104 \end{prototype} 4105 4106 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 4107 4108 \section{Example Camera Configuration Files} 4109 4110 \tbd{Some of these don't exactly match the specifications of this 4111 document yet, because they have been changed from the prototype, but 4112 it is hoped that they will be useful. Questions are welcome.} 4113 4114 \subsection{MegaCam Raw} 4115 4116 \begin{verbatim} 4117 # The raw MegaCam data comes off the telescope with each of the chips stored in extensions of a MEF file. 4118 4119 # How to identify this type 4120 RULE METADATA 4121 TELESCOP STR CFHT 3.6m 4122 DETECTOR STR MegaCam 4123 EXTEND BOOL T 4124 NEXTEND S32 72 1033 SIMPLE BOOL TRUE 1034 NAXIS S32 2 1035 TELESCOP STR ISP-1 1036 INSTRUME STR ISP-Apogee 1037 DETECTOR STR ISP-Apogee-01 1038 ISPCAMER STR Apogee U42 4125 1039 END 4126 1040 4127 1041 # How to read this data 4128 PHU STR FPA # The FITS file represents an entire FPA 4129 EXTENSIONS STR CELL # The extensions represent cells 1042 FILE METADATA 1043 PHU STR FPA # The FITS file represents an entire FPA 1044 EXTENSIONS STR NONE # There are no extensions 1045 FPA.NAME STR SEQID # A PHU keyword for unique identifier within the hierarchy level 1046 END 4130 1047 4131 1048 # What's in the FITS file? 4132 CONTENTS METADATA 4133 # Extension name, chip name:type 4134 amp00 STR ccd00:left 4135 amp01 STR ccd00:right 4136 amp02 STR ccd01:left 4137 amp03 STR ccd01:right 4138 amp04 STR ccd02:left 4139 amp05 STR ccd02:right 4140 amp06 STR ccd03:left 4141 amp07 STR ccd03:right 4142 amp08 STR ccd04:left 4143 amp09 STR ccd04:right 4144 amp10 STR ccd05:left 4145 amp11 STR ccd05:right 4146 amp12 STR ccd06:left 4147 amp13 STR ccd06:right 4148 amp14 STR ccd07:left 4149 amp15 STR ccd07:right 4150 amp16 STR ccd08:left 4151 amp17 STR ccd08:right 4152 amp18 STR ccd09:left 4153 amp19 STR ccd09:right 4154 amp20 STR ccd10:left 4155 amp21 STR ccd10:right 4156 amp22 STR ccd11:left 4157 amp23 STR ccd11:right 4158 amp24 STR ccd12:left 4159 amp25 STR ccd12:right 4160 amp26 STR ccd13:left 4161 amp27 STR ccd13:right 4162 amp28 STR ccd14:left 4163 amp29 STR ccd14:right 4164 amp30 STR ccd15:left 4165 amp31 STR ccd15:right 4166 amp32 STR ccd16:left 4167 amp33 STR ccd16:right 4168 amp34 STR ccd17:left 4169 amp35 STR ccd17:right 4170 amp36 STR ccd18:left 4171 amp37 STR ccd18:right 4172 amp38 STR ccd19:left 4173 amp39 STR ccd19:right 4174 amp40 STR ccd20:left 4175 amp41 STR ccd20:right 4176 amp42 STR ccd21:left 4177 amp43 STR ccd21:right 4178 amp44 STR ccd22:left 4179 amp45 STR ccd22:right 4180 amp46 STR ccd23:left 4181 amp47 STR ccd23:right 4182 amp48 STR ccd24:left 4183 amp49 STR ccd24:right 4184 amp50 STR ccd25:left 4185 amp51 STR ccd25:right 4186 amp52 STR ccd26:left 4187 amp53 STR ccd26:right 4188 amp54 STR ccd27:left 4189 amp55 STR ccd27:right 4190 amp56 STR ccd28:left 4191 amp57 STR ccd28:right 4192 amp58 STR ccd29:left 4193 amp59 STR ccd29:right 4194 amp60 STR ccd30:left 4195 amp61 STR ccd30:right 4196 amp62 STR ccd31:left 4197 amp63 STR ccd31:right 4198 amp64 STR ccd32:left 4199 amp65 STR ccd32:right 4200 amp66 STR ccd33:left 4201 amp67 STR ccd33:right 4202 amp68 STR ccd34:left 4203 amp69 STR ccd34:right 4204 amp70 STR ccd35:left 4205 amp71 STR ccd35:right 4206 END 1049 CONTENTS STR Chip:Cell:amplifier 4207 1050 4208 1051 # Specify the cell data 4209 1052 CELLS METADATA 4210 left METADATA # Left amplifier 4211 CELL.BIASSEC STR HEADER:BIASSEC 4212 CELL.TRIMSEC STR HEADER:DATASEC 4213 CELL.XPARITY S32 1 # We could have specified this as a DEFAULT, but this works 4214 END 4215 right METADATA # Right amplifier 4216 CELL.BIASSEC STR HEADER:BIASSEC 4217 CELL.TRIMSEC STR HEADER:DATASEC 4218 CELL.XPARITY S32 -1 # This cell is read out in the opposite direction 1053 amplifier METADATA 1054 CELL.TRIMSEC.SOURCE STR HEADER 1055 CELL.BIASSEC.SOURCE STR HEADER 1056 CELL.TRIMSEC STR TRIMSEC 1057 CELL.BIASSEC STR BIASSEC 4219 1058 END 4220 1059 END … … 4222 1061 # How to translate PS concepts into FITS headers 4223 1062 TRANSLATION METADATA 4224 FPA.NAME STR EXPNUM 4225 FPA.AIRMASS STR AIRMASS 4226 FPA.FILTER STR FILTER 4227 FPA.POSANGLE STR ROTANGLE 1063 FPA.OBSTYPE STR OBSTYPE 1064 FPA.OBJECT STR OBSTYPE 1065 FPA.FILTER STR FILTNAME 4228 1066 FPA.RA STR RA 4229 1067 FPA.DEC STR DEC 4230 1068 FPA.RADECSYS STR RADECSYS 4231 FPA.MJD STR MJD-OBS 1069 FPA.ALT STR ALT 1070 FPA.AZ STR AZ 1071 FPA.POSANGLE STR ROTANGLE 1072 FPA.AIRMASS STR AIRMASS 1073 FPA.TIME STR MJD-OBS 1074 CHIP.TEMP STR CCDTEMP 4232 1075 CELL.EXPOSURE STR EXPTIME 4233 1076 CELL.DARKTIME STR DARKTIME 4234 CELL.XBIN STR CCDBIN1 4235 CELL.YBIN STR CCDBIN2 1077 CELL.TIME STR MJD-OBS 4236 1078 CELL.GAIN STR GAIN 4237 1079 CELL.READNOISE STR RDNOISE 4238 CELL.SATURATION STR SATURATE 1080 CELL.XBIN STR XBIN 1081 CELL.YBIN STR YBIN 1082 # CELL.SATURATION STR SATURATE ### Currently set to 0 ??? 1083 CELL.BAD STR BADLEVEL 4239 1084 END 4240 1085 4241 1086 # Default PS concepts that may be specified by value 4242 1087 DEFAULTS METADATA 4243 CELL.BAD S32 0 4244 CELL.YPARITY_DEPEND STR CHIP.NAME 4245 CELL.YPARITY METADATA 4246 ccd00 S32 -1 4247 ccd01 S32 -1 4248 ccd02 S32 -1 4249 ccd03 S32 -1 4250 ccd04 S32 -1 4251 ccd05 S32 -1 4252 ccd06 S32 -1 4253 ccd07 S32 -1 4254 ccd08 S32 -1 4255 ccd09 S32 -1 4256 ccd10 S32 -1 4257 ccd11 S32 -1 4258 ccd12 S32 -1 4259 ccd13 S32 -1 4260 ccd14 S32 -1 4261 ccd15 S32 -1 4262 ccd16 S32 -1 4263 ccd17 S32 -1 4264 ccd18 S32 1 4265 ccd19 S32 1 4266 ccd20 S32 1 4267 ccd21 S32 1 4268 ccd22 S32 1 4269 ccd23 S32 1 4270 ccd24 S32 1 4271 ccd25 S32 1 4272 ccd26 S32 1 4273 ccd27 S32 1 4274 ccd28 S32 1 4275 ccd29 S32 1 4276 ccd30 S32 1 4277 ccd31 S32 1 4278 ccd32 S32 1 4279 ccd33 S32 1 4280 ccd34 S32 1 4281 ccd35 S32 1 4282 END 4283 END 4284 4285 # How to translate PS concepts into database lookups 4286 DATABASE METADATA 4287 TYPE dbEntry TABLE COLUMN GIVENDBCOL GIVENPS 4288 # CELL.GAIN dbEntry Camera gain chipId,cellId CHIP.NAME,CELL.NAME 4289 # CELL.READNOISE dbEntry Camera readNoise chipId,cellId CHIP.NAME,CELL.NAME 4290 4291 # A database entry refers to a particular column (COLUMN) in a 4292 # particular table (TABLE), given certain PS concepts (GIVENPS) that 4293 # match certain database columns (GIVENDBCOL). 4294 4295 END 4296 \end{verbatim} 4297 4298 \subsection{MegaCam Splice} 4299 4300 \begin{verbatim} 4301 # The spliced MecaCam data is stored in single extensions for each chip 4302 4303 # How to recognise this type 4304 RULE METADATA 4305 TELESCOP STR CFHT 3.6m 4306 DETECTOR STR MegaCam 4307 EXTEND BOOL T 4308 NEXTEND S32 36 4309 END 4310 4311 # How to read this data 4312 PHU STR FPA # The FITS file represents an entire FPA 4313 EXTENSIONS STR CHIP # The extensions represent chips 4314 4315 # What's in the FITS file? 4316 CONTENTS METADATA 4317 # Extension name, components 4318 ccd00 STR left right 4319 ccd01 STR left right 4320 ccd02 STR left right 4321 ccd03 STR left right 4322 ccd04 STR left right 4323 ccd05 STR left right 4324 ccd06 STR left right 4325 ccd07 STR left right 4326 ccd08 STR left right 4327 ccd09 STR left right 4328 ccd10 STR left right 4329 ccd11 STR left right 4330 ccd12 STR left right 4331 ccd13 STR left right 4332 ccd14 STR left right 4333 ccd15 STR left right 4334 ccd16 STR left right 4335 ccd17 STR left right 4336 ccd18 STR left right 4337 ccd19 STR left right 4338 ccd20 STR left right 4339 ccd21 STR left right 4340 ccd22 STR left right 4341 ccd23 STR left right 4342 ccd24 STR left right 4343 ccd25 STR left right 4344 ccd26 STR left right 4345 ccd27 STR left right 4346 ccd28 STR left right 4347 ccd29 STR left right 4348 ccd30 STR left right 4349 ccd31 STR left right 4350 ccd32 STR left right 4351 ccd33 STR left right 4352 ccd34 STR left right 4353 ccd35 STR left right 4354 END 4355 4356 # Specify the cells 4357 CELLS METADATA 4358 left METADATA 4359 CELL.BIASSEC STR HEADER:BSECA 4360 CELL.TRIMSEC STR HEADER:TSECA 4361 END 4362 4363 right METADATA 4364 CELL.BIASSEC STR HEADER:BSECB 4365 CELL.TRIMSEC STR HEADER:TSECB 4366 END 4367 END 4368 4369 # How to translate PS concepts into FITS headers 4370 TRANSLATION METADATA 4371 FPA.NAME STR EXPNUM 4372 FPA.AIRMASS STR AIRMASS 4373 FPA.FILTER STR FILTER 4374 FPA.POSANGLE STR ROTANGLE 4375 FPA.RA STR RA 4376 FPA.DEC STR DEC 4377 FPA.RADECSYS STR RADECSYS 4378 FPA.MJD STR MJD-OBS 4379 CELL.EXPOSURE STR EXPTIME 4380 CELL.DARKTIME STR DARKTIME 4381 CELL.XBIN STR CCDBIN1 4382 CELL.YBIN STR CCDBIN2 4383 CELL.GAIN STR GAIN 4384 CELL.READNOISE STR RDNOISE 4385 CELL.SATURATION STR SATURATE 4386 END 4387 4388 # Default PS concepts that may be specified by value 4389 DEFAULTS METADATA 4390 CELL.BAD S32 0 4391 CELL.XPARITY S32 1 4392 CELL.YPARITY S32 1 4393 END 4394 4395 4396 # How to translate PS concepts into database lookups 4397 DATABASE METADATA 4398 TYPE dbEntry TABLE COLUMN GIVENDBCOL GIVENPS 4399 # CELL.GAIN dbEntry Camera gain chipId,cellId CHIP.NAME,CELL.NAME 4400 # CELL.READNOISE dbEntry Camera readNoise chipId,cellId CHIP.NAME,CELL.NAME 4401 4402 # A database entry refers to a particular column (COLUMN) in a 4403 # particular table (TABLE), given certain PS concepts (GIVENPS) that 4404 # match certain database columns (GIVENDBCOL). 4405 4406 END 4407 \end{verbatim} 4408 4409 \subsection{LRIS Blue} 4410 4411 \begin{verbatim} 4412 # The Low Resolution Imager and Spectrograph (LRIS) blue side 4413 4414 # We have no choice but to hard-code the various regions, because Keck 4415 # only stores them as: 4416 # WINDOW = '1,0,0,2048,4096' 4417 # PREPIX = 51 4418 # POSTPIX = 80 4419 # BINNING = '1,1 ' 4420 # AMPPSIZE= '[1:1024,1:4096]' 4421 4422 # I don't know how we would get the IPP to react to changes in the 4423 # windowing on the fly --- we have no mechanism for setting the region 4424 # sizes on the basis of the above keywords. Therefore, we hard-code 4425 # the regions and assert on our assumptions in the RULE. 4426 4427 4428 # How to identify this type 4429 RULE METADATA 4430 TELESCOP STR Keck I 4431 INSTRUME STR LRISBLUE 4432 AMPLIST STR 1,4,0,0 4433 WINDOW STR 1,0,0,2048,4096 4434 PREPIX S32 51 4435 POSTPIX S32 80 4436 BINNING STR 1,1 4437 AMPPSIZE STR [1:1024,1:4096] 4438 NAXIS1 S32 4620 4439 NAXIS2 S32 4096 4440 END 4441 4442 # How to read this data 4443 PHU STR FPA # The FITS file represents an entire FPA 4444 EXTENSIONS STR NONE # There are no extensions 4445 4446 # What's in the FITS file? 4447 CONTENTS METADATA 4448 LeftChip STR amp1 amp2 4449 RightChip STR amp3 amp4 4450 END 4451 4452 # Specify the cell data 4453 CELLS METADATA 4454 amp1 METADATA 4455 CELL.BIASSEC STR VALUE:[1:51,1:4096];[4301:4380,1:4096] 4456 CELL.TRIMSEC STR VALUE:[205:1228,1:4096] 4457 CELL.GAIN STR VALUE:1.2 4458 CELL.READNOISE STR VALUE:5.6 4459 END 4460 4461 amp2 METADATA 4462 CELL.BIASSEC STR VALUE:[52:102,1:4096];[4381:4460,1:4096] 4463 CELL.TRIMSEC STR VALUE:[1229:2252,1:4096] 4464 CELL.GAIN STR VALUE:1.3 4465 CELL.READNOISE STR VALUE:6.7 4466 END 4467 4468 amp3 METADATA 4469 CELL.BIASSEC STR VALUE:[103:153,1:4096];[4461:4540,1:4096] 4470 CELL.TRIMSEC STR VALUE:[2253:3276,1:4096] 4471 CELL.GAIN STR VALUE:1.4 4472 CELL.READNOISE STR VALUE:7.8 4473 END 4474 4475 amp4 METADATA 4476 CELL.BIASSEC STR VALUE:[154:204,1:4096];[4541:4620,1:4096] 4477 CELL.TRIMSEC STR VALUE:[3277:4300,1:4096] 4478 CELL.GAIN STR VALUE:1.5 4479 CELL.READNOISE STR VALUE:8.9 4480 END 4481 END 4482 4483 # How to translate PS concepts into FITS headers 4484 TRANSLATION METADATA 4485 FPA.AIRMASS STR AIRMASS 4486 FPA.FILTER STR BLUFILT 4487 FPA.POSANGLE STR ROTPOSN 4488 FPA.RA STR RA 4489 FPA.DEC STR DEC 4490 CELL.EXPOSURE STR EXPOSURE 4491 CELL.DARKTIME STR EXPOSURE // No special darktime header; use exposure time 4492 CELL.DATE STR DATE // NOTE: There are TWO keywords called "DATE" (creation, exp)! 4493 CELL.TIME STR UT 4494 END 4495 4496 # Default PS concepts that may be specified by value 4497 DEFAULTS METADATA 4498 FPA.RADECSYS STR ICRS 4499 END 4500 \end{verbatim} 4501 4502 \subsection{LRIS Red} 4503 4504 \begin{verbatim} 4505 # The Low Resolution Imager and Spectrograph (LRIS) red side 4506 4507 # We have no choice but to hard-code the various regions, because Keck 4508 # only stores them as: 4509 # WINDOW = '0,0,0,2048,2048' 4510 # PREPIX = 20 4511 # POSTPIX = 80 4512 # BINNING = '1,1 ' 4513 # AMPPSIZE= '[1:1024,1:4096]' 4514 4515 # I don't know how we would get the IPP to react to changes in the 4516 # windowing on the fly --- we have no mechanism for setting the region 4517 # sizes on the basis of the above keywords. Therefore, we hard-code 4518 # the regions and assert on our assumptions in the RULE. 4519 4520 4521 # How to identify this type 4522 RULE METADATA 4523 TELESCOP STR Keck I 4524 INSTRUME STR LRIS 4525 AMPLIST STR 2,1,0,0 4526 WINDOW STR 0,0,0,2048,2048 4527 PREPIX S32 20 4528 POSTPIX S32 80 4529 BINNING STR 1, 1 4530 CCDPSIZE STR [1:2048,1:2048] 4531 NAXIS1 S32 2248 4532 NAXIS2 S32 2048 4533 IMTYPE STR TWOAMPTOP 4534 END 4535 4536 # How to read this data 4537 PHU STR CHIP # The FITS file represents a single chip 4538 EXTENSIONS STR NONE # There are no extensions 4539 4540 # What's in the FITS file? 4541 CONTENTS STR LeftSide RightSide 4542 4543 # Specify the cell data 4544 CELLS METADATA 4545 LeftSide METADATA 4546 CELL.BIASSEC STR VALUE:[1:20,1:2048];[2089:2168,1:2048] 4547 CELL.TRIMSEC STR VALUE:[41:1064,1:2048] 4548 CELL.GAIN STR VALUE:1.2 4549 CELL.READNOISE STR VALUE:5.6 4550 END 4551 4552 RightSide METADATA 4553 CELL.BIASSEC STR VALUE:[21:40,1:2048];[2169:2248,1:2048] 4554 CELL.TRIMSEC STR VALUE:[1065:2088,1:2048] 4555 CELL.GAIN STR VALUE:1.3 4556 CELL.READNOISE STR VALUE:6.5 4557 END 4558 END 4559 4560 # How to translate PS concepts into FITS headers 4561 TRANSLATION METADATA 4562 FPA.AIRMASS STR AIRMASS 4563 FPA.FILTER STR FILTER 4564 FPA.POSANGLE STR POSANG 4565 FPA.RA STR OBJ-RA 4566 FPA.DEC STR OBJ-DEC 4567 CELL.EXPOSURE STR EXPTIME 4568 CELL.DARKTIME STR DARKTIME 4569 CELL.DATE STR DATE-OBS 4570 CELL.TIME STR TIME-OBS 4571 END 4572 4573 # Default PS concepts that may be specified by value 4574 DEFAULTS METADATA 4575 FPA.RADECSYS STR ICRS 4576 END 4577 \end{verbatim} 4578 4579 \subsection{GPC OTA} 4580 4581 \begin{verbatim} 4582 # The raw GPC data comes off the telescope with each of the chips stored in separate files 4583 4584 # How to identify this type 4585 RULE METADATA 4586 # TELESCOP STR PS1 4587 # DETECTOR STR GPC1 4588 EXTEND BOOL T 4589 NEXTEND S32 64 4590 NAMPS S32 64 4591 END 4592 4593 # How to read this data 4594 PHU STR CHIP # The FITS file represents a single chip 4595 EXTENSIONS STR CELL # The extensions represent cells 4596 4597 # What's in the FITS file? 4598 CONTENTS METADATA 4599 # Extension name, type 4600 xy00 STR pitch10u 4601 xy01 STR pitch10u 4602 xy02 STR pitch10u 4603 xy03 STR pitch10u 4604 xy04 STR pitch10u 4605 xy05 STR pitch10u 4606 xy06 STR pitch10u 4607 xy07 STR pitch10u 4608 xy10 STR pitch10u 4609 xy11 STR pitch10u 4610 xy12 STR pitch10u 4611 xy13 STR pitch10u 4612 xy14 STR pitch10u 4613 xy15 STR pitch10u 4614 xy16 STR pitch10u 4615 xy17 STR pitch10u 4616 xy20 STR pitch10u 4617 xy21 STR pitch10u 4618 xy22 STR pitch10u 4619 xy23 STR pitch10u 4620 xy24 STR pitch10u 4621 xy25 STR pitch10u 4622 xy26 STR pitch10u 4623 xy27 STR pitch10u 4624 xy30 STR pitch10u 4625 xy31 STR pitch10u 4626 xy32 STR pitch10u 4627 xy33 STR pitch10u 4628 xy34 STR pitch10u 4629 xy35 STR pitch10u 4630 xy36 STR pitch10u 4631 xy37 STR pitch10u 4632 xy40 STR pitch10u 4633 xy41 STR pitch10u 4634 xy42 STR pitch10u 4635 xy43 STR pitch10u 4636 xy44 STR pitch10u 4637 xy45 STR pitch10u 4638 xy46 STR pitch10u 4639 xy47 STR pitch10u 4640 xy50 STR pitch10u 4641 xy51 STR pitch10u 4642 xy52 STR pitch10u 4643 xy53 STR pitch10u 4644 xy54 STR pitch10u 4645 xy55 STR pitch10u 4646 xy56 STR pitch10u 4647 xy57 STR pitch10u 4648 xy60 STR pitch10u 4649 xy61 STR pitch10u 4650 xy62 STR pitch10u 4651 xy63 STR pitch10u 4652 xy64 STR pitch10u 4653 xy65 STR pitch10u 4654 xy66 STR pitch10u 4655 xy67 STR pitch10u 4656 xy70 STR pitch10u 4657 xy71 STR pitch10u 4658 xy72 STR pitch10u 4659 xy73 STR pitch10u 4660 xy74 STR pitch10u 4661 xy75 STR pitch10u 4662 xy76 STR pitch10u 4663 xy77 STR pitch10u 4664 END 4665 4666 # Specify the cell data 4667 CELLS METADATA 4668 pitch10u METADATA 4669 CELL.BIASSEC STR VALUE:[575:606,1:594] 4670 CELL.TRIMSEC STR VALUE:[1:574,1:594] 4671 # CELL.BIASSEC STR HEADER:BIASSEC 4672 # CELL.TRIMSEC STR HEADER:DATASEC 4673 END 4674 4675 # This is just in here for fun 4676 pitch12u METADATA 4677 CELL.BIASSEC STR VALUE:[1:10,1:512];[523:574,1:512] 4678 CELL.TRIMSEC STR VALUE:[11:522,1:512] 4679 # CELL.BIASSEC STR HEADER:BIASSEC 4680 # CELL.TRIMSEC STR HEADER:TRIMSEC 4681 END 4682 END 4683 4684 4685 # How to translate PS concepts into FITS headers 4686 TRANSLATION METADATA 4687 CELL.BIN STR CCDSUM 4688 CELL.SATURATION STR SATURATE 4689 END 4690 4691 # Default PS concepts that may be specified by value 4692 DEFAULTS METADATA 4693 FPA.AIRMASS F32 0.0 4694 FPA.FILTER STR NONE 4695 FPA.POSANGLE F32 0.0 4696 FPA.RA STR 0:0:0 4697 FPA.DEC STR 0:0:0 4698 FPA.RADECSYS STR ICRS 4699 FPA.NAME S32 0 4700 FPA.MJD F32 12345.6789 4701 CELL.EXPOSURE F32 0.0 4702 CELL.DARKTIME F32 0.0 4703 CELL.GAIN F32 1.0 4704 CELL.READNOISE F32 0.0 4705 CELL.BAD S32 0 4706 CELL.BIN S32 1 1088 FPA.TIMESYS STR UTC 1089 CELL.SATURATION F32 65535 1090 CELL.READDIR S32 1 1091 CELL.TIMESYS STR UTC 1092 CHIP.XPARITY S32 1 1093 CHIP.YPARITY S32 1 1094 CHIP.X0 S32 0 1095 CHIP.Y0 S32 0 4707 1096 CELL.XPARITY S32 1 4708 1097 CELL.YPARITY S32 1 4709 END 4710 4711 # How to translate PS concepts into database lookups 1098 CELL.X0 S32 0 1099 CELL.Y0 S32 0 1100 END 1101 1102 FORMATS METADATA 1103 FPA.RA STR HOURS 1104 FPA.DEC STR DEGREES 1105 FPA.TIME STR MJD 1106 CELL.TIME STR MJD 1107 END 1108 1109 # PS Concepts to get from the database 4712 1110 DATABASE METADATA 4713 TYPE dbEntry TABLE COLUMN GIVENDBCOL GIVENPS 4714 CELL.GAIN dbEntry Camera gain chipId,cellId CHIP,CELL 4715 CELL.READNOISE dbEntry Camera readNoise chipId,cellId CHIP,CELL 4716 4717 # A database entry refers to a particular column (COLUMN) in a 4718 # particular table (TABLE), given certain PS concepts (GIVENPS) that 4719 # match certain database columns (GIVENDBCOL). 4720 1111 # None. 4721 1112 END 4722 1113 \end{verbatim} 4723 1114 1115 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 1116 1117 \section{Recipes} 1118 1119 \subsection{Locations} 1120 1121 Recipes may be specified in a number of locations. Firstly, they may 1122 be specified on the command line with the \code{-recipe} option, 1123 giving a symbolic name and a filename or another symbolic name to link 1124 to. In addition, they may be specified in the site configuration and 1125 the camera configuration under the \code{RECIPES} metadata. Note that 1126 the \code{PATH(STR)} in the site configuration defines the search 1127 paths for these files. 1128 1129 \subsection{Contents} 1130 1131 The contents of the recipe files depends on the particular recipe. 1132 1133 \subsubsection{PPIMAGE} 1134 1135 The \code{PPIMAGE} recipe contains options for \code{ppImage}: 1136 \begin{itemize} 1137 \item \code{MASK(BOOL)} indicates if bad pixels are to be masked. 1138 \item \code{MASK.VALUE(U8)} specifies a bitmask (matching the bad 1139 pixel mask) for pixels to mask in the input image. 1140 \item \code{NONLIN(BOOL)} indicates if the non-linearity correction is 1141 to be performed. 1142 \item \code{OVERSCAN(BOOL)} indicates if the overscan correction is to be performed. 1143 \item \code{BIAS(BOOL)} indicates if the bias correction is to be performed. 1144 \item \code{DARK(BOOL)} indicates if the dark correction is to be performed. 1145 \item \code{SHUTTER(BOOL)} indicates if the shutter correction is to be performed. 1146 \item \code{FLAT(BOOL)} indicates if the flat-field correction is to be performed. 1147 \item \code{FRINGE(BOOL)} indicates if the fringe correction is to be performed. 1148 \item \code{PHOTOM(BOOL)} indicates if the photometry is to be performed. 1149 \item \code{ASTROM.CHIP(BOOL)} indicates if the astrometry is to be performed on a chip level. 1150 \item \code{ASTROM.MOSAIC(BOOL)} indicates if the astrometry is to be performed on a mosaic (FPA) level. 1151 \item \code{BASE.FITS(BOOL)} indicates if the base detrended image is to be saved. 1152 \item \code{CHIP.FITS(BOOL)} indicates if the chip mosaicked image is to be saved. 1153 \item \code{FPA1.FITS(BOOL)} indicates if the FPA mosaicked image with first level binning is to be saved. 1154 \item \code{FPA2.FITS(BOOL)} indicates if the FPA mosaicked image with second level binning is to be saved. 1155 \item \code{BIN1.FITS(BOOL)} indicates if the chip mosaicked image with first level binning is to be saved. 1156 \item \code{BIN2.FITS(BOOL)} indicates if the chip mosaicked image with second level binning is to be saved. 1157 \item \code{BIN1.JPEG(BOOL)} indicates if the JPEG image with first level binning is to be saved. 1158 \item \code{BIN2.JPEG(BOOL)} indicates if the JPEG image with second level binning is to be saved. 1159 \item \code{NONLIN.DATA} may be: 1160 \begin{itemize} 1161 \item A vector of type \code{F32}, in which case it provides the 1162 (ordinary) polynomial coefficients for the non-linear correction. 1163 \item Of type \code{STR}, in which case it provides a filename with 1164 the lookup table (consisting of two columns of values, the first 1165 the input flux and the second the corresponding corrected flux). 1166 \item Of type \code{METADATA}, in which case it is a menu, with menu 1167 items with types and values according to one of the other two 1168 options. The menu key is provided by \code{NONLIN.SOURCE(STR}), 1169 which gives a concept name to look up (\code{CHIP.NAME} would be a 1170 good choice). 1171 \end{itemize} 1172 \item \code{OVERSCAN.SINGLE(BOOL)} indicates if the entire overscan is 1173 to be reduced to a single value. 1174 \item \code{OVERSCAN.FIT(STR)} indicates the type of fit that is to be 1175 performed to the overscan (if \code{OVERSCAN.SINGLE} is 1176 \code{FALSE}): \code{NONE}, \code{POLYNOMIAL} or \code{SPLINE}. 1177 \item \code{OVERSCAN.ORDER(S32)} gives the order of the polynomial fit 1178 (or number of spline pieces). 1179 \item \code{OVERSCAN.STAT(STR)} gives the statistic to apply to the 1180 overscan: \code{MEAN} or \code{MEDIAN}. \tbd{Would like to change 1181 this to allow the full range of statistics.} 1182 \item \code{BIN1.XBIN(S32)} gives the level 1 binning in x 1183 \item \code{BIN2.YBIN(S32)} gives the level 1 binning in y 1184 \item \code{BIN2.XBIN(S32)} gives the level 2 binning in x 1185 \item \code{BIN2.YBIN(S32)} gives the level 2 binning in y: 1186 \item \code{PHOTCODE.RULE(STR)} gives a rule for producing a 1187 photometry code, with values in curly brackets interpolated. 1188 \end{itemize} 1189 1190 \subsubsubsection{Example} 1191 1192 \begin{verbatim} 1193 ### ppImage recipe configuration file 1194 1195 # List of tasks to perform 1196 MASK BOOL FALSE # Mask bad pixels 1197 MASK.VALUE U8 0xff # Only mask pixels matching this bitmask 1198 NONLIN BOOL FALSE # Non-linearity correction 1199 OVERSCAN BOOL TRUE # Overscan subtraction 1200 BIAS BOOL TRUE # Bias subtraction 1201 DARK BOOL TRUE # Dark subtraction 1202 FLAT BOOL TRUE # Flat-field normalisation 1203 FRINGE BOOL FALSE # Fringe subtraction 1204 PHOTOM BOOL FALSE # Source identification and photometry 1205 ASTROM.CHIP BOOL FALSE # Astrometry on chip 1206 ASTROM.MOSAIC BOOL FALSE # Astrometry on mosaic 1207 1208 BASE.FITS BOOL TRUE # Save base detrended image? 1209 CHIP.FITS BOOL TRUE # Save chip-mosaic-ed image? 1210 FPA1.FITS BOOL TRUE # Save 1st binned fpa image? 1211 FPA2.FITS BOOL TRUE # Save 2nd binned fpa image? 1212 BIN1.FITS BOOL TRUE # Save 1st binned chip image? 1213 BIN2.FITS BOOL TRUE # Save 2nd binned chip image? 1214 BIN1.JPEG BOOL TRUE # Save 1st binned jpeg? 1215 BIN2.JPEG BOOL FALSE # Save 2nd binned jpeg? 1216 1217 # Non-linearity correction 1218 NONLIN.SOURCE STR CHIP.NAME # How to determine the source 1219 #@NONLIN.DATA F32 0.0 1.001 0.001 # A polynomial 1220 #NONLIN.DATA STR nonlin.dat # Filename for lookup table 1221 NONLIN.DATA METADATA # Source of non-linearity data 1222 ccd00 STR nonlin00.dat # A lookup table 1223 @ccd01 F32 0.0 1.001 0.001 # A polynomial 1224 @ccd02 F32 1.2345 # A polynomial 1225 END 1226 1227 # Overscan subtraction 1228 OVERSCAN.SINGLE BOOL FALSE # Reduce overscan to a single value? 1229 #OVERSCAN.FIT STR SPLINE # NONE | POLYNOMIAL | SPLINE 1230 OVERSCAN.FIT STR POLYNOMIAL # NONE | POLYNOMIAL | SPLINE 1231 OVERSCAN.ORDER S32 5 # Order of polynomial fit 1232 OVERSCAN.STAT STR MEAN # MEAN | MEDIAN 1233 1234 # binned output image options 1235 BIN1.XBIN S32 8 1236 BIN1.YBIN S32 8 1237 BIN2.XBIN S32 64 1238 BIN2.YBIN S32 64 1239 1240 PHOTCODE.RULE STR {CAMERA}.{FILTER.ID}.{CHIP.N} 1241 \end{verbatim} 1242 1243 \subsubsection{PPMERGE} 1244 1245 The \code{PPMERGE} recipe contains options for \code{ppMerge}: 1246 \begin{itemize} 1247 \item \code{ROWS(S32)} gives the number of rows to be read at once (a 1248 number larger than the physical size will read all rows). 1249 \item \code{ELECTRONS(F32)} gives the minimum number of electrons for 1250 useful signal. \tbd{Don't think this is implemented yet.} 1251 \item \code{SAMPLE(S32)} specifies a sampling frequency for 1252 determining the background level. 1253 \item \code{REJ(F32)} specifies a rejection threshold, in standard 1254 deviations. 1255 \item \code{ITER(S32)} specifies the number of rejection iterations. 1256 \item \code{FRACHIGH(F32)} gives the fraction of high pixels to reject immediately. 1257 \item \code{FRACLOW(F32)} gives the fraction of low pixels to reject immediately. 1258 \item \code{NKEEP(S32)} gives the minimum number of pixels in the stack to keep. 1259 \item \code{MASKVAL(S32)} gives the mask value for input data. 1260 \item \code{COMBINE(STR)} gives the statistic to use for combination. 1261 \item \code{BACKGROUND(STR)} gives the statistic to use to measure the background. 1262 \end{itemize} 1263 1264 Statistics specified by a string (for \code{COMBINE} and 1265 \code{BACKGROUND}) may be one of \code{MEAN}, \code{MEDIAN}, 1266 \code{ROBUST}, \code{FITTED} or \code{CLIPPED}. 1267 1268 \subsubsubsection{Example} 1269 1270 \begin{verbatim} 1271 # Recipe configuration for ppMerge 1272 1273 ROWS S32 128 # Number of rows to read at once 1274 ELECTRONS F32 100.0 # Minimum number of electrons for useful signal 1275 SAMPLE S32 100 # Sampling factor for measuring the background 1276 REJ F32 3.0 # Rejection threshold (sigma) 1277 ITER S32 1 # Number of rejection iterations 1278 FRACHIGH F32 0.3 # Fraction of high pixels to reject immediately 1279 FRACLOW F32 0.1 # Fraction of low pixels to reject immediately 1280 NKEEP S32 5 # Minimum number of pixels in stack to keep 1281 MASKVAL S32 0xff # Mask value for input data 1282 ### Statistics options: MEAN | MEDIAN | ROBUST | FITTED | CLIPPED 1283 COMBINE STR MEAN # Statistic to use for combination: 1284 BACKGROUND STR MEDIAN # Statistic to use to measure the background 1285 \end{verbatim} 1286 1287 1288 \subsubsection{PPSTATS} 1289 1290 The \code{PPSTATS} recipe contains options for \code{ppStats} or its 1291 library used within another program: 1292 \begin{itemize} 1293 \item \code{SAMPLE(F32)} specifies the fraction of the cell to sample 1294 (for statistical measurements). 1295 \item \code{MASKVAL(U8)} specifies a mask value to use for the 1296 statistics. 1297 \item \code{HEADER(STR)} specifies headers (may be listed, separated 1298 by whitespace) to print. Multiple \code{HEADER} entries may exist, 1299 if it is declared \code{MULTI}. 1300 \item \code{CONCEPT(STR)} specifies concepts (may be listed, separated 1301 by whitespace) to print. Multiple \code{CONCEPT} entries may exist, 1302 if it is declared \code{MULTI}. 1303 \item \code{STAT(STR)} specifies statistics (may be listed, separated 1304 by whitespace) to print. Multiple \code{STAT} entries may exist, if 1305 it is declared \code{MULTI}. Acceptable statistics names are those 1306 parsed by \code{psStatsOptionFromString}. 1307 \end{itemize} 1308 1309 \subsubsubsection{Example} 1310 1311 \begin{verbatim} 1312 ### ppStats recipe for Phase 0 with MegaCam 1313 1314 # Options governing statistics 1315 SAMPLE F32 0.1 # Fraction of cell to sample 1316 MASKVAL U8 0xff # Mask value to use for statistics 1317 1318 # Define the outputs as MULTI 1319 HEADER MULTI 1320 CONCEPT MULTI 1321 STAT MULTI 1322 1323 # Values to return 1324 HEADER STR OBSERVER # Observer name 1325 CONCEPT STR FPA.OBJECT # Object name 1326 CONCEPT STR FPA.OBSTYPE # Observation type 1327 CONCEPT STR FPA.FILTER # Filter 1328 CONCEPT STR FPA.RA FPA.DEC # Telescope pointing 1329 CONCEPT STR FPA.AIRMASS # Airmass 1330 CONCEPT STR FPA.ALT FPA.AZ # Telescopy alt/az 1331 CONCEPT STR FPA.POSANGLE # Rotator angle 1332 CONCEPT STR CHIP.TEMP # Detector temperature 1333 CONCEPT STR CELL.EXPOSURE # Exposure time 1334 CONCEPT STR CELL.TIME # Time of exposure 1335 STAT STR ROBUST_MEDIAN # Background estimator 1336 STAT STR ROBUST_STDEV # Background standard deviation estimator 1337 \end{verbatim} 1338 1339 \subsubsection{PSPHOT} 1340 1341 \tbd{EAM to fill this in.} 1342 1343 \subsubsection{PSASTRO} 1344 1345 \tbd{EAM to fill this in.} 1346 1347 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 1348 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 1349 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 1350 1351 \section{Revision Change Log} 1352 %\input{ChangeLog.tex} 1353 1354 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 1355 1356 %\bibliographystyle{plain} 1357 %\bibliography{panstarrs} 1358 1359 \end{document} 1360
Note:
See TracChangeset
for help on using the changeset viewer.
