Index: /trunk/doc/ipptools/p4tool.txt =================================================================== --- /trunk/doc/ipptools/p4tool.txt (revision 11739) +++ /trunk/doc/ipptools/p4tool.txt (revision 11740) @@ -1,59 +1,204 @@ --------------------------------------------------------------------------------- - -The "p4tool -definerun" command creates a new p4Run for a single cell. A -unique identifier for the run, or a p4Run ID, is returned. - - p4tool -definerun -survey_mode -filter -skycell_id -tess_id -ra -decl -input_ss -output_ss -imfiles - - p4tool -addinputimfile -p4_id -exp_tag -p3_version -class_id - --------------------------------------------------------------------------------- - - p4tool -towarpedimfile [-p4_id ] - -Blocks until the appropriate # of inputs have been registered in a given -p4run. Returns a list of imfiles along with their skycell_id & tessalation id -that have not yet been warped. - - p4tool -addwarpedimfile -p4_id -exp_tag -p3_version --class_id -uri [-bg ...] - -Registers a warped imfile with the database. - - p4tool -warpedimfile [-p4_id ] - -Lists warped imfiles. - --------------------------------------------------------------------------------- - - p4tool -tostackedimfile -p4_id - -List a complete set of class_ids for a given p4run that are ready to be -stacked. - - p4tool -addstackedimfile -p4_id -exp_tag -p3_version --class_id -uri [-bg ...] - -Registered a stacked imfiles with the database - - p4tool -stackedimfile [-p4_id ] - -Lists stacked imfiles. - --------------------------------------------------------------------------------- - - p4tool -todiffimfile [-p4_id ] - -Lists stacked imfiles along with the uri for an imfile to use for -differencing. - - p4tool -adddiffimfile -p4_id -exp_tag -p3_version --class_id -uri [-bg ...] - -Registered a difference imfiles with the database. - - p4tool -diffimfile [-p4_id ] - -Lists difference imfiles. - --------------------------------------------------------------------------------- +Give me a list of all pending exposures which could have phase 4 run: + p4tool -query -survey_mode -filter -ra -decl (plus other search options) + +The "p4tool -definerun" command creates a new p4Run for an exposure. A +unique identifier for the run, or a p4Run ID, is returned: + + p4tool -definerun -exp_tag -p3_ver + +Give me all p4runs to have overlaps calculated: + p4tool -tooverlap + +Calculate overlaps for each exposure: + phase4_overlap.pl --p4_id +{ + Get list of component imfiles for exposure: + p4tool -imfile -p4_id + + (Calculates, in some as-yet unknown way the overlaps between imfiles +and skycells) + + Set the relationship between imfiles and skycells: + p4tool -overlap -p4_id -info FILE + where FILE contains: + imfile.0 STR skycell.123 skycell.456 + imfile.1 STR skycell.789 + OR: + imfile.0 MULTI + imfile.0 STR skycell.123 + imfile.0 STR skycell.456 + imfile.1 STR skycell.789 +} + +Give me all skycells that need to have data warped to them: + p4tool -towarped + +Do the warp for each skycell: + phase4_warp.pl --p4_id --skycell +{ + Get list of imfiles for skycell: + p4tool -imfile -p4_id -skycell + + Do the warping: + pswarp + + Register the skycell as done warping: + p4tool -warped -p4_id -skycell -uri +} + +================================================================================ + +Give me a list of all skycells which could have subtractions performed +(more than two warps): + p5tool -query (some search parameters, including RA, Dec, seeing) + +Register an exposure for subtraction processing. We need to specify which warp +is the input and which is the template. + p5tool -definerun -skycell -input_p4_id -template_p4_id + +Give me all skycells that need to be subtracted: + p5tool -pending + +Do the subtraction for each skycell: + phase5_subtract.pl --p5_id --in_uri --template_uri +{ + Do the subtraction: + pois in_uri.fits template_uri sub.fits + + Register the subtraction as done: + p5tool -done -p5_id -uri +} + +================================================================================ + +[Here I make some assumptions about how magic works] + +Get a list of exposures for which magic processing is possible + magictool -query (some search options) + +There may be multiple subtractions generated out of each exposure. +Given an exposure, what subtractions do I have available? + + magictool -options -exp_tag + +Register an exposure for magic processing, specifying a subtraction set: + magictool -definerun -exp_tag -subs FILE + + where FILE contains something like: + skycell.123 STR p5id.1 + skycell.456 STR p5id.2 + skycell.789 STR p5id.3 + +Give me all skycells that need to have magic performed: + magictool -pendingskycell + +Do magic for each skycell: + magic_skycell.pl --magic_id --skycell --uri +{ + Do streak finding on skycell + + Register sky cell as done: + magictool -doneskycell -magic_id -skycell -uri +} + +Give me all imfiles that need to have magic performed: + magictool -pendingimfile + +Do magic for each imfile: + magic_imfile.pl --magic_id --class_id +{ + Get list of all magic1 output relevant to this imfile + magictool -skycell -magic_id -class_id + + Do streak finding for imfile + + Register imfile as done: + magictool -doneimfile -magic_id -class_id -uri +} + +[Might be a mid-level process or processes, halfway between imfiles and +full exposures, but for now, assume there's not, or that it can all be +done quickly at the exposure level.] + +Give me all exposures that need to have magic performed: + magictool -pendingexp + +Do magic for each exposure: + magic_exp.pl --magic_id +{ + Get list of all magic2 output relevant to this exposure + magictool -imfile -magic_id + + Do streak finding on exposure + + Register exposure as done: + magictool -doneexp -magic_id -uri +} + + +[Need to generate masks, but we need to decide at what level: imfile, or +skycell? If imfile, we need to do the warp over again. If skycell, we +can never give out the flattened CCD images unless they've been warped. +How about doing both --- it's not too hard.] + + +Give me all imfiles that need a magic mask: + magictool -pendingmaskimfile + +Generate mask for each imfile: + magic_mask_imfile.pl --magic_id --class_id --uri +{ + Make mask + + Register mask: + magictool -donemaskimfile -magic_id -class_id -uri +} + +Give me all skycells that need a magic mask: + magictool -pendingmaskskycell + +Generate mask for each skycell: + magic_mask_skycell.pl --magic_id --skycell --uri +{ + Make mask + + Register mask: + magictool -donemaskskycell -magic_id -skycell -uri +} + +Give me a list of all exposures which are clean (so we can release): + magictool -cleanexp (some search criteria) + +Give me a list of all imfiles which are clean (so we can release): + magictool -cleanimfile (some search criteria) + +Give me a list of all skycells which are clean (so we can release): + magictool -cleanskycell (some search criteria) + +================================================================================ + +Give me a list of all skycells for which combination processing is possible: + p6tool -query -min (some other search parameters, including ra, dec, time range) + +Register skycells for combination processing: + p6tool -definerun -skycell -p4_id -p4_id -p4_id + +Give me all pending operations + p6tool -pending + +Run phase 6: + phase6.pl -p6_id +{ + Give me the list of all inputs for this run: + p6tool -inputs -p6_id + + Combine: + ppStac output.fits input1.fits input2.fits ... + + Register the completion: + p6tool -done -p6_id -uri +} + +Give me a list of all stacks (for release): + p6tool -list +