Index: branches/tap_branches/psphot/doc/footprints.txt =================================================================== --- branches/tap_branches/psphot/doc/footprints.txt (revision 25900) +++ branches/tap_branches/psphot/doc/footprints.txt (revision 27838) @@ -1,2 +1,20 @@ + +2010.01.18 + + * we are having some problems with footprints being freed to early in the second pass + + * footprint memory history: + + * entering psphotFindFootprints, on the first pass, we have an + array of detected peak in detections->peaks + + * in pmFootprintsFind, footprints array owns allocated footprints + footprints each own their spans + + * from pmFootprintAssignPeaks: + * the footprints have an array of peaks with valid memory references + * the peaks have just the pointer to the corresponding footprint (not owned) + * all peaks get assigned to a footprint + * at end of pmFootprintAssignPeaks, only a single copy of each peak is/should be available from the collection of footprints pmFootprintCullPeaks is very expensive (15 - 30 msec per object) and Index: branches/tap_branches/psphot/doc/notes.20091026.txt =================================================================== --- branches/tap_branches/psphot/doc/notes.20091026.txt (revision 27838) +++ branches/tap_branches/psphot/doc/notes.20091026.txt (revision 27838) @@ -0,0 +1,44 @@ + +2009.10.26 + + I'm looking into psphot as applied to diff images. For now, I am + running a comparison of a simtest output warp with the convolved + version of the same image as generated by ppSub. + + * we are losing the cores of bright stars, probably to the choices + in the FITS output 16-bit conversion (pixel-value -> NAN) + + * errors for the convolved image seem to be smaller for the same + instrumental mags than for the warp (check chip). + + * the psf model seems to be quite poor -- a lot of structure in the + psf core. + + * (note: not a very good match with the warp image, though better) + + * psfsub images seem to be slightly displaced in X-dir. perhaps + because the convolution kernel has a bit of X-dir tilt in the + peak. Try with PPSUB ISIS orders set to 0.0 + +2009.10.27 + + I need some psphot options / variants: + + * forced photometry: + + I should be able to do: + + psphotForced -file input.fits -psf psf.fits -src src.list output + + and have it only perform the forced photometry on the sources listed + in src.list. + + * psf model creation + + I should be able to do: + + psphotMakePSF -file input.fits -src src.list output + + and have it generate a PSF model using the sources listed in + src.list + Index: branches/tap_branches/psphot/doc/notes.20100131.txt =================================================================== --- branches/tap_branches/psphot/doc/notes.20100131.txt (revision 27838) +++ branches/tap_branches/psphot/doc/notes.20100131.txt (revision 27838) @@ -0,0 +1,66 @@ + +2010.03.24 + + I have finished another iteration on source size analysis. I feel + fairly confident of the extended source and CR identifications. They + are not perfect, but they are in pretty good shape. The CR IDs use + the moments to find things smaller in one dimension than a PSF, while + the EXT is set for things larger than a PSF in one dimention. + + There still seems to be some trouble with SATSTARS : these are being + set for sources which are fainter than the SAT limit -- I'm not sure + if this is due to the quality of the PSF fit or if they really are + SAT, but have integrated fluxes lower than expected (underfitted in + PSF). + + Additional Parameters needed from psphot for distinguishing dipoles: + + -- note regarding CfA parameters -- + + Here are our settings. + + Note that Fpos is the flux which is 3-sigma above background, and + Fneg is the flux which is 3-sigma below background. + + Npos is the # of pixels with flux which is 3-sigma above background. + + etc. + + # for positive objects: + # Ngood = Npos, Nbad = Nneg, Fgood = Fpos, Fbad = Fneg + + # for negative objects: + # Ngood = Nneg, Nbad = Npos, Fgood = Fneg, Fbad = Fpos + # FRATIO=Fgood/(Fgood+Fbad) + # NRATIO_BAD=Ngood/(Ngood+Nbad) + # NRATIO_MASK=Ngood/(Ngood+Nmask) + # NRATIO_ALL=Ngood/(Ngood+Nbad+Nmask) + DC_FRATIO_MIN 0.75 + DC_NRATIO_BAD_MIN 0.70 + DC_NRATIO_MASK_MIN 0.60 + DC_NRATIO_ALL_MIN 0.50 + DC_NGOOD_MIN 3 + + *** double-check on the use of fluxScale in pmSourcePhotometry.c:106 + +20100131 + + We have several stand-alone psphot versions, and a few different + psphotReadout versions called by other programs. Here is a complete + list: + + Stand-alone psphot versions | associated psphotReadout version + + o psphot | psphotReadout + o psphotForced | psphotForcedReadout + o psphotMakePSF | psphotMakePSFReadout + x psphotDetect | psphotDetectReadout + x psphotStack | psphotStackReadout + + External programs | associated psphotReadout version + + o ppImage | psphotReadout + o pswarp | psphotReadoutFindPSF + o ppSub | psphotReadoutFindPSF, psphotReadoutMinimal + o ppStack | psphotReadoutKnownSources + Index: branches/tap_branches/psphot/doc/stack.txt =================================================================== --- branches/tap_branches/psphot/doc/stack.txt (revision 25900) +++ branches/tap_branches/psphot/doc/stack.txt (revision 27838) @@ -1,2 +1,84 @@ + +20100126: + + * watch out for psphotSetMomentsWindow & MOMENTS_SX_MAX,etc + * watch out for psphotSignificanceImage.c:,psphotEfficiency using the FWHM_MAJ from psphotChoosePSF + * ppSimDetections.c : SIGMA_SMOOTH +ppSmooth/src/ppSmoothReadout.c: psMetadataAddF32(recipe, PS_LIST_TAIL, "EFFECTIVE_AREA", PS_META_REPLACE, "Effective Area", effArea); +ppSmooth/src/ppSmoothReadout.c: psMetadataAddF32(recipe, PS_LIST_TAIL, "SIGNIFICANCE_SCALE_FACTOR", PS_META_REPLACE, "Signicance scale factor", factor); + + +20100120 : more stack processing mods: + + there are a number of data collections generated during psphotReadout: + + * detections (currently a structure containing multiple arrays) + * sources (a psArray) + * psf (the psf model structure) + + * new sources vs old sources: + there is a sequence on the second pass in which we need to distinguish the sources + from the first pass from those on the second pass: + + - add noise (old sources only) + - find detections + - subtract noise (old sources only) + - generate sources (new detections only) + - source classifications (new sources only) + - guess models (new sources only) + - replace sources (old sources only) + - merge sources (new + old -> sources) + + currently we are distiguishing the old vs new based on different arrays. + can we use the processing flags to distinguish the these cases and carry around + only a single source list? + + * detections->peaks holds the most recently detected set of peaks + detections->oldPeaks holds the previous collection + + detections->footprints holds the full set of merged footprints, + including assigned peaks from the old and new set. + +20100107 : updates for stack processing + + I am building a new version of psphot that operates on a set of + images (eg, multi-filter or multi-epoch). The assumptions are: + + * each input image represents the same sky pixels : they are warped to a common frame. + * each image has been previously processed, with the background + subtracted (and the psf model determined?) + * all planes are / can be loaded into memory at once (otherwise I + need to add a lot of I/O layers) + + psphot currently expects the pmFPAfile of interest to be available in + config->files with the name PSPHOT.INPUT. To update the code for + stacks, I am extending this concept with the pmFPAfileSelectSingle + API: config->files may contain multiple PSPHOT.INPUT entries, and + functions which access this file / these files need to specify + *which* image they want. The number of PSPHOT.INPUT entries is + saved in config->arguments as PSPHOT.INPUT.NUM (this can be + generated from the filerule PSPHOT.INPUT for extension). + + As a result, we need a number of wrapper functions which loop over + all PSPHOT.INPUT.NUM entries and perform a particular operation on + one of the entries. Here are the functions which I have modified + in this way (function -> child) + + * psphotAddPhotcode -> psphotAddPhotcodeReadout + * note that the photcode is now saved on readout->analysis + * psphotSetMaskAndVariance -> psphotSetMaskAndVarianceReadout + * psphotModelBackground -> psphotModelBackgroundReadoutFileIndex + * psphotSubtractBackground -> psphotSubtractBackgroundReadout + + side notes on + + * psphotModelBackground vs psphotBackgroundModel : I've renamed + psphotBackgroundModel (used only by ppStack for now) to + psphotModelBackgroundReadoutNoFile. I've also named the + single-readout version used by psphotModelBackground to + psphotModelBackgroundReadoutFileIndex. These two functions do the + same thing, but psphotModelBackgroundReadoutNoFile does not save + a pmFPAfile on config->files + 20090606 : design notes on the multi-image photometry analysis Index: branches/tap_branches/psphot/src/Makefile.am =================================================================== --- branches/tap_branches/psphot/src/Makefile.am (revision 25900) +++ branches/tap_branches/psphot/src/Makefile.am (revision 27838) @@ -25,7 +25,6 @@ libpsphot_la_LDFLAGS = $(PSPHOT_LIBS) $(PSMODULE_LIBS) $(PSLIB_LIBS) -bin_PROGRAMS = psphot psphotTest psphotMomentsStudy -# bin_PROGRAMS = psphotPetrosianStudy -# bin_PROGRAMS = psphot +bin_PROGRAMS = psphot psphotForced psphotMakePSF psphotStack +# bin_PROGRAMS = psphotPetrosianStudy psphotTest psphotMomentsStudy psphot_CFLAGS = $(PSPHOT_CFLAGS) $(PSMODULE_CFLAGS) $(PSLIB_CFLAGS) @@ -33,11 +32,19 @@ psphot_LDADD = libpsphot.la -psphotTest_CFLAGS = $(PSPHOT_CFLAGS) $(PSMODULE_CFLAGS) $(PSLIB_CFLAGS) -psphotTest_LDFLAGS = $(PSPHOT_LIBS) $(PSMODULE_LIBS) $(PSLIB_LIBS) -psphotTest_LDADD = libpsphot.la - -psphotMomentsStudy_CFLAGS = $(PSPHOT_CFLAGS) $(PSMODULE_CFLAGS) $(PSLIB_CFLAGS) -psphotMomentsStudy_LDFLAGS = $(PSPHOT_LIBS) $(PSMODULE_LIBS) $(PSLIB_LIBS) -psphotMomentsStudy_LDADD = libpsphot.la +psphotForced_CFLAGS = $(PSPHOT_CFLAGS) $(PSMODULE_CFLAGS) $(PSLIB_CFLAGS) +psphotForced_LDFLAGS = $(PSPHOT_LIBS) $(PSMODULE_LIBS) $(PSLIB_LIBS) +psphotForced_LDADD = libpsphot.la + +psphotMakePSF_CFLAGS = $(PSPHOT_CFLAGS) $(PSMODULE_CFLAGS) $(PSLIB_CFLAGS) +psphotMakePSF_LDFLAGS = $(PSPHOT_LIBS) $(PSMODULE_LIBS) $(PSLIB_LIBS) +psphotMakePSF_LDADD = libpsphot.la + +psphotStack_CFLAGS = $(PSPHOT_CFLAGS) $(PSMODULE_CFLAGS) $(PSLIB_CFLAGS) +psphotStack_LDFLAGS = $(PSPHOT_LIBS) $(PSMODULE_LIBS) $(PSLIB_LIBS) +psphotStack_LDADD = libpsphot.la + +# psphotMomentsStudy_CFLAGS = $(PSPHOT_CFLAGS) $(PSMODULE_CFLAGS) $(PSLIB_CFLAGS) +# psphotMomentsStudy_LDFLAGS = $(PSPHOT_LIBS) $(PSMODULE_LIBS) $(PSLIB_LIBS) +# psphotMomentsStudy_LDADD = libpsphot.la # psphotPetrosianStudy_CFLAGS = $(PSPHOT_CFLAGS) $(PSMODULE_CFLAGS) $(PSLIB_CFLAGS) @@ -45,27 +52,67 @@ # psphotPetrosianStudy_LDADD = libpsphot.la +# psphotTest_CFLAGS = $(PSPHOT_CFLAGS) $(PSMODULE_CFLAGS) $(PSLIB_CFLAGS) +# psphotTest_LDFLAGS = $(PSPHOT_LIBS) $(PSMODULE_LIBS) $(PSLIB_LIBS) +# psphotTest_LDADD = libpsphot.la + +# standard psphot for generic photometry psphot_SOURCES = \ psphot.c \ psphotArguments.c \ - psphotCleanup.c \ + psphotParseCamera.c \ psphotImageLoop.c \ psphotMosaicChip.c \ - psphotParseCamera.c - -psphotDetect_SOURCES = \ - psphotDetect.c \ - psphotDetectArguments.c \ - psphotDetectParseCamera.c \ - psphotDetectImageLoop.c \ - psphotDetectReadout.c \ - psphotMosaicChip.c \ - psphotCleanup.c - -psphotTest_SOURCES = \ - psphotTest.c - -psphotMomentsStudy_SOURCES = \ - psphotMomentsStudy.c - + psphotCleanup.c + +# forced photometry of specified positions given a specified psf +psphotForced_SOURCES = \ + psphotForced.c \ + psphotForcedArguments.c \ + psphotForcedImageLoop.c \ + psphotForcedReadout.c \ + psphotParseCamera.c \ + psphotMosaicChip.c \ + psphotCleanup.c + +# a psphot-variant that simply generates the PSF model +psphotMakePSF_SOURCES = \ + psphotMakePSF.c \ + psphotMakePSFArguments.c \ + psphotMakePSFImageLoop.c \ + psphotMakePSFReadout.c \ + psphotParseCamera.c \ + psphotMosaicChip.c \ + psphotCleanup.c + +# a psphot-variant for stack photometry +psphotStack_SOURCES = \ + psphotStack.c \ + psphotStackArguments.c \ + psphotStackParseCamera.c \ + psphotStackImageLoop.c \ + psphotStackReadout.c \ + psphotStackChisqImage.c \ + psphotFitSourcesLinearStack.c \ + psphotSourceMatch.c \ + psphotCleanup.c + + + +# # psphot analysis of the detectability of specified positions +# psphotDetect_SOURCES = \ +# psphotDetect.c \ +# psphotDetectArguments.c \ +# psphotDetectParseCamera.c \ +# psphotDetectImageLoop.c \ +# psphotDetectReadout.c \ +# psphotMosaicChip.c \ +# psphotCleanup.c + +# psphotTest_SOURCES = \ +# psphotTest.c +# +# psphotMomentsStudy_SOURCES = \ +# psphotMomentsStudy.c +# # psphotPetrosianStudy_SOURCES = \ # psphotPetrosianStudy.c @@ -115,5 +162,4 @@ psphotKernelFromPSF.c \ psphotPSFConvModel.c \ - psphotModelTest.c \ psphotFitSet.c \ psphotSourceFreePixels.c \ @@ -121,4 +167,5 @@ psphotMergeSources.c \ psphotLoadPSF.c \ + psphotLoadSRCTEXT.c \ psphotReadoutCleanup.c \ psphotSourcePlots.c \ @@ -138,8 +185,13 @@ psphotEllipticalContour.c \ psphotEllipticalProfile.c \ + psphotRadialBins.c \ psphotPetrosian.c \ psphotPetrosianRadialBins.c \ psphotPetrosianStats.c \ + psphotPetrosianVisual.c \ psphotEfficiency.c + +# XXX need to fix this for the new apis +# psphotModelTest.c # re-instate these @@ -147,5 +199,4 @@ # psphotAnnuli.c \ # psphotKron.c \ -# psphotPetrosianVisual.c \ # Index: branches/tap_branches/psphot/src/psphot.h =================================================================== --- branches/tap_branches/psphot/src/psphot.h (revision 25900) +++ branches/tap_branches/psphot/src/psphot.h (revision 27838) @@ -12,4 +12,12 @@ #define PSPHOT_RECIPE_PSF_FAKE_ALLOW "PSF.FAKE.ALLOW" // Name for recipe component permitting fake PSFs + +// pmPCMData : PSF Convolved Model data storage structure +typedef struct { + psImage *model; + psArray *dmodels; + psImage *modelConv; + psArray *dmodelsConv; +} pmPCMData; // top-level psphot functions @@ -29,5 +37,7 @@ bool psphotReadoutMinimal(pmConfig *config, const pmFPAview *view); -bool psphotReadoutCleanup (pmConfig *config, pmReadout *readout, psMetadata *recipe, pmDetections *detections, pmPSF *psf, psArray *sources); +bool psphotReadoutCleanup (pmConfig *config, const pmFPAview *view); +bool psphotReadoutCleanupReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe); + bool psphotDefineFiles (pmConfig *config, pmFPAfile *input); void psphotFilesActivate(pmConfig *config, bool state); @@ -38,42 +48,122 @@ // XXX test functions psArray *psphotFakeSources (void); +bool psphotMaskCosmicRayFootprintCheck (psArray *sources); // psphotReadout functions -bool psphotModelBackground (pmConfig *config, const pmFPAview *view, const char *filename); -bool psphotSubtractBackground (pmConfig *config, const pmFPAview *view, const char *filename) ; -pmDetections *psphotFindDetections (pmDetections *detections, pmReadout *readout, psMetadata *recipe); - -bool psphotRoughClass (pmReadout *readout, psArray *sources, psMetadata *recipe, const bool findPsfClump); -bool psphotBasicDeblend (psArray *sources, psMetadata *recipe); -pmPSF *psphotChoosePSF (pmReadout *readout, psArray *sources, psMetadata *recipe); -bool psphotPSFstats (pmReadout *readout, psMetadata *recipe, pmPSF *psf); -bool psphotMomentsStats (pmReadout *readout, psMetadata *recipe, psArray *sources); -bool psphotFitSourcesLinear (pmReadout *readout, psArray *sources, const psMetadata *recipe, const pmPSF *psf, bool final); -bool psphotReplaceUnfitSources (psArray *sources, psImageMaskType maskVal); - -bool psphotReplaceAllSources (psArray *sources, psMetadata *recipe); -bool psphotRemoveAllSources (const psArray *sources, const psMetadata *recipe); - -bool psphotBlendFit (pmConfig *config, pmReadout *readout, psArray *sources, pmPSF *psf); +bool psphotAddPhotcode (pmConfig *config, const pmFPAview *view); +bool psphotAddPhotcodeReadout (pmConfig *config, const pmFPAview *view, const char *filerule, int index); + +bool psphotSetMaskAndVariance (pmConfig *config, const pmFPAview *view); +bool psphotSetMaskAndVarianceReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe); + +bool psphotModelBackground (pmConfig *config, const pmFPAview *view); +bool psphotModelBackgroundReadoutFileIndex (pmConfig *config, const pmFPAview *view, const char *filename, int index); + +bool psphotSubtractBackground (pmConfig *config, const pmFPAview *view); +bool psphotSubtractBackgroundReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe); + +bool psphotFindDetections (pmConfig *config, const pmFPAview *view, bool firstPass); +bool psphotFindDetectionsReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe, bool firstPass); + +bool psphotSourceStats (pmConfig *config, const pmFPAview *view, bool setWindow); +bool psphotSourceStatsReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe, bool setWindow); + +bool psphotDeblendSatstars (pmConfig *config, const pmFPAview *view); +bool psphotDeblendSatstarsReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index); + +bool psphotBasicDeblend (pmConfig *config, const pmFPAview *view); +bool psphotBasicDeblendReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index); + +bool psphotRoughClass (pmConfig *config, const pmFPAview *view); +bool psphotRoughClassReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe); +bool psphotRoughClassRegion (int nRegion, psRegion *region, psArray *sources, psMetadata *analysis, psMetadata *recipe, const bool havePSF); + +bool psphotImageQuality (pmConfig *config, const pmFPAview *view); +bool psphotImageQualityReadout(pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe); + +bool psphotChoosePSF (pmConfig *config, const pmFPAview *view); +bool psphotChoosePSFReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe); + +bool psphotGuessModels (pmConfig *config, const pmFPAview *view); +bool psphotGuessModelsReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index); + +bool psphotMergeSources (pmConfig *config, const pmFPAview *view); +bool psphotMergeSourcesReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index); + +bool psphotFitSourcesLinear (pmConfig *config, const pmFPAview *view, bool final); +bool psphotFitSourcesLinearReadout (psMetadata *recipe, pmReadout *readout, psArray *sources, pmPSF *psf, bool final); + +bool psphotSourceSize (pmConfig *config, const pmFPAview *view, bool getPSFsize); +bool psphotSourceSizeReadout(pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe, bool getPSFsize); + +bool psphotBlendFit (pmConfig *config, const pmFPAview *view); +bool psphotBlendFitReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe); bool psphotBlendFit_Threaded (psThreadJob *job); -psArray *psphotSourceStats (pmConfig *config, pmReadout *readout, pmDetections *detections, bool setWindow); -bool psphotSourceStats_Threaded (psThreadJob *job); - -bool psphotGuessModels (pmConfig *config, pmReadout *readout, psArray *sources, pmPSF *psf); -bool psphotGuessModel_Threaded (psThreadJob *job); - -bool psphotMagnitudes (pmConfig *config, pmReadout *readout, const pmFPAview *view, psArray *sources, const pmPSF *psf); +bool psphotReplaceAllSources (pmConfig *config, const pmFPAview *view); +bool psphotReplaceAllSourcesReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe); + +bool psphotAddNoise (pmConfig *config, const pmFPAview *view); +bool psphotSubNoise (pmConfig *config, const pmFPAview *view); +bool psphotAddOrSubNoise (pmConfig *config, const pmFPAview *view, bool add); +bool psphotAddOrSubNoiseReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe, bool add); + +bool psphotExtendedSourceAnalysis (pmConfig *config, const pmFPAview *view); +bool psphotExtendedSourceAnalysisReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe); + +bool psphotExtendedSourceFits (pmConfig *config, const pmFPAview *view); +bool psphotExtendedSourceFitsReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe); + +bool psphotApResid (pmConfig *config, const pmFPAview *view); +bool psphotApResidReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe); + +bool psphotMagnitudes (pmConfig *config, const pmFPAview *view); +bool psphotMagnitudesReadout(pmConfig *config, psMetadata *recipe, const pmFPAview *view, pmReadout *readout, psArray *sources, pmPSF *psf); bool psphotMagnitudes_Threaded (psThreadJob *job); + +bool psphotEfficiency (pmConfig *config, const pmFPAview *view); +bool psphotEfficiencyReadout(pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe); bool psphotPSFWeights(pmConfig *config, pmReadout *readout, const pmFPAview *view, psArray *sources); bool psphotPSFWeights_Threaded (psThreadJob *job); -bool psphotApResid (pmConfig *config, pmReadout *readout, psArray *sources, pmPSF *psf); - bool psphotSkyReplace (pmConfig *config, const pmFPAview *view); -bool psphotExtendedSourceAnalysis (pmReadout *readout, psArray *sources, psMetadata *recipe); -bool psphotExtendedSourceFits (pmReadout *readout, psArray *sources, psMetadata *recipe); -bool psphotEfficiency(pmConfig *config, pmReadout *readout, const pmFPAview *view, const pmPSF *psf, psMetadata *recipe, const psArray *realSources); +bool psphotSkyReplaceReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index); + +bool psphotSourceFreePixels (pmConfig *config, const pmFPAview *view); +bool psphotSourceFreePixelsReadout(pmConfig *config, const pmFPAview *view, const char *filename, int index); + +// in psphotSourceStats.c: +bool psphotSourceStats_Threaded (psThreadJob *job); +bool psphotSourceStatsUpdate (psArray *sources, pmConfig *config, pmReadout *readout); +bool psphotSetMomentsWindow (psMetadata *recipe, psMetadata *analysis, psArray *sources); + +// in psphotChoosePSF.c: +bool psphotPSFstats (pmReadout *readout, pmPSF *psf); +bool psphotMomentsStats (pmReadout *readout, psArray *sources); + +// in psphotGuessModel.c +bool psphotGuessModel_Threaded (psThreadJob *job); + +// in psphotMergeSources.c: +bool psphotLoadExtSources (pmConfig *config, const pmFPAview *view); +psArray *psphotLoadPSFSources (pmConfig *config, const pmFPAview *view); +bool psphotRepairLoadedSources (pmConfig *config, const pmFPAview *view); +bool psphotCheckExtSources (pmConfig *config, const pmFPAview *view); + +// generate the detection structure for the supplied array of sources +bool psphotDetectionsFromSources (pmConfig *config, const pmFPAview *view, psArray *sources); + +// generate the detection structure for the supplied array of sources +bool psphotSetSourceParams (pmConfig *config, psArray *sources, pmPSF *psf); + +// in psphotModelBackground.c: +// Create a background model for a readout, without saving the result as a pmFPAfile on config->files. Otherwise identical to psphotModelBackgroundFileIndex. +psImage *psphotModelBackgroundReadoutNoFile(pmReadout *readout, const pmConfig *config); +psImageBinning *psphotBackgroundBinning(const psImage *image, const pmConfig *config); + +// in psphotReplaceUnfit.c: +bool psphotRemoveAllSources (const psArray *sources, const psMetadata *recipe); +bool psphotReplaceUnfitSources (psArray *sources, psImageMaskType maskVal); // thread-related: @@ -87,10 +177,7 @@ psArray *psphotFindPeaks (psImage *significance, pmReadout *readout, psMetadata *recipe, const float threshold, const int nMax); bool psphotFindFootprints (pmDetections *detections, psImage *significance, pmReadout *readout, psMetadata *recipe, const float threshold, const int pass, psImageMaskType maskVal); -psErrorCode psphotCullPeaks(const psImage *img, const psImage *weight, const psMetadata *recipe, psArray *footprints); - -// generate the detection structure for the supplied array of sources -pmDetections *psphotDetectionsFromSources (pmConfig *config, psArray *sources); - -// used by ApResid +psErrorCode psphotCullPeaks(const pmReadout *readout, const psMetadata *recipe, psArray *footprints); + +// in psphotApResid.c: pmTrend2D *psphotApResidTrend (float *apResidSysErr, pmReadout *readout, int Nx, int Ny, psVector *xPos, psVector *yPos, psVector *apResid, psVector *dMag); bool psphotApResidMags_Threaded (psThreadJob *job); @@ -99,9 +186,8 @@ void psphotModelClassInit (void); bool psphotGrowthCurve (pmReadout *readout, pmPSF *psf, bool ignore, psImageMaskType maskVal); -bool psphotSetMaskAndVariance (pmConfig *config, pmReadout *readout, psMetadata *recipe); -void psphotSourceFreePixels (psArray *sources); // functions to set the correct source pixels -bool psphotInitRadiusPSF (const psMetadata *recipe, const pmModelType type); +bool psphotInitRadiusPSF(const psMetadata *recipe, const psMetadata *analysis, const pmModelType type); + bool psphotCheckRadiusPSF (pmReadout *readout, pmSource *source, pmModel *model, psImageMaskType markVal); bool psphotCheckRadiusPSFBlend (pmReadout *readout, pmSource *source, pmModel *model, psImageMaskType markVal, float dR); @@ -110,6 +196,4 @@ float psphotSetRadiusEXT (pmReadout *readout, pmSource *source, psImageMaskType markVal); -// output functions -bool psphotAddPhotcode (psMetadata *recipe, pmConfig *config, const pmFPAview *view, const char *filerule); bool psphotDumpMoments (psMetadata *recipe, psArray *sources); psMetadata *psphotDefineHeader (psMetadata *recipe); @@ -142,23 +226,12 @@ bool psphotFitSummary (void); -bool psphotMergeSources (psArray *oldSources, psArray *newSources); -bool psphotLoadExtSources (pmConfig *config, const pmFPAview *view, psArray *sources); -psArray *psphotLoadPSFSources (pmConfig *config, const pmFPAview *view); - -pmPSF *psphotLoadPSF (pmConfig *config, const pmFPAview *view, psMetadata *recipe); +bool psphotLoadPSF (pmConfig *config, const pmFPAview *view); +bool psphotLoadPSFReadout (pmConfig *config, const pmFPAview *view, const char *outFilename, const char *inFilename, int index); + bool psphotSetHeaderNstars (psMetadata *recipe, psArray *sources); -bool psphotAddNoise (pmReadout *readout, const psArray *sources, const psMetadata *recipe); -bool psphotSubNoise (pmReadout *readout, const psArray *sources, const psMetadata *recipe); -bool psphotAddOrSubNoise (pmReadout *readout, const psArray *sources, const psMetadata *recipe, bool add); bool psphotRadialPlot (int *kapa, const char *filename, pmSource *source); bool psphotSourcePlots (pmReadout *readout, psArray *sources, psMetadata *recipe); bool psphotMosaicSubimage (psImage *outImage, pmSource *source, int Xo, int Yo, int DX, int DY, bool normalize); -bool psphotAddWithTest (pmSource *source, bool useState, psImageMaskType maskVal); -bool psphotSubWithTest (pmSource *source, bool useState, psImageMaskType maskVal); -bool psphotSetState (pmSource *source, bool curState, psImageMaskType maskVal); -bool psphotDeblendSatstars (pmReadout *readout, psArray *sources, psMetadata *recipe); -bool psphotSourceSize (pmConfig *config, pmReadout *readout, psArray *sources, psMetadata *recipe, pmPSF *psf, long first); - bool psphotMakeResiduals (psArray *sources, psMetadata *recipe, pmPSF *psf, psImageMaskType maskVal); @@ -168,34 +241,33 @@ // functions related to extended source analysis -bool psphotRadialProfile (pmSource *source, psMetadata *recipe, float skynoise, psImageMaskType maskVal); -bool psphotRadialProfilesByAngles (pmSource *source, int Nsec, float Rmax); -float psphotRadiusFromProfile (pmSource *source, psVector *radius, psVector *flux, float fluxMin, float fluxMax); -bool psphotRadiiFromProfiles (pmSource *source, float fluxMin, float fluxMax); -bool psphotEllipticalProfile (pmSource *source); -bool psphotEllipticalContour (pmSource *source); +bool psphotRadialProfile (pmSource *source, psMetadata *recipe, float skynoise, psImageMaskType maskVal); +bool psphotRadialProfilesByAngles (pmSource *source, int Nsec, float Rmax); +float psphotRadiusFromProfile (pmSource *source, psVector *radius, psVector *flux, float fluxMin, float fluxMax); +bool psphotRadiiFromProfiles (pmSource *source, float fluxMin, float fluxMax); +bool psphotEllipticalProfile (pmSource *source, bool RAW_RADIUS); +bool psphotEllipticalContour (pmSource *source); // psphotVisual functions -bool psphotVisualShowImage (pmReadout *readout); -bool psphotVisualShowBackground (pmConfig *config, const pmFPAview *view, pmReadout *readout); -bool psphotVisualShowSignificance (psImage *image); -bool psphotVisualShowPeaks (pmDetections *detections); -bool psphotVisualShowFootprints (pmDetections *detections); -bool psphotVisualShowMoments (psArray *sources); -bool psphotVisualPlotMoments (psMetadata *recipe, psMetadata *analysis, psArray *sources); -bool psphotVisualShowRoughClass (psArray *sources); -bool psphotVisualShowPSFStars (psMetadata *recipe, pmPSF *psf, psArray *sources); -bool psphotVisualShowSatStars (psMetadata *recipe, pmPSF *psf, psArray *sources); -bool psphotVisualShowPSFModel (pmReadout *readout, pmPSF *psf); -bool psphotVisualPlotRadialProfile (int myKapa, pmSource *source, psImageMaskType maskVal); -bool psphotVisualPlotRadialProfiles (psMetadata *recipe, psArray *sources); -bool psphotVisualShowFlags (psArray *sources); -bool psphotVisualShowSourceSize (pmReadout *readout, psArray *sources); -bool psphotVisualShowResidualImage (pmReadout *readout); -bool psphotVisualPlotApResid (psArray *sources, float mean, float error); -bool psphotVisualPlotSourceSize (psMetadata *recipe, psMetadata *analysis, psArray *sources); -bool psphotVisualShowPetrosians (psArray *sources); - -// bool psphotPetrosianAnalysis (pmReadout *readout, psArray *sources, psMetadata *recipe); -// bool psphotPetrosianProfile (pmReadout *readout, pmSource *source, float skynoise); +bool psphotVisualShowImage (pmReadout *readout); +bool psphotVisualShowBackground (pmConfig *config, const pmFPAview *view, pmReadout *readout); +bool psphotVisualShowSignificance (psImage *image, float min, float max); +bool psphotVisualShowPeaks (pmDetections *detections); +bool psphotVisualShowFootprints (pmDetections *detections); +bool psphotVisualShowMoments (psArray *sources); +bool psphotVisualPlotMoments (psMetadata *recipe, psMetadata *analysis, psArray *sources); +bool psphotVisualShowRoughClass (psArray *sources); +bool psphotVisualShowPSFStars (psMetadata *recipe, pmPSF *psf, psArray *sources); +bool psphotVisualShowSatStars (psMetadata *recipe, pmPSF *psf, psArray *sources); +bool psphotVisualShowPSFModel (pmReadout *readout, pmPSF *psf); +bool psphotVisualPlotRadialProfile (int myKapa, pmSource *source, psImageMaskType maskVal); +bool psphotVisualPlotRadialProfiles (psMetadata *recipe, psArray *sources); +bool psphotVisualShowFlags (psArray *sources); +bool psphotVisualShowSourceSize (pmReadout *readout, psArray *sources); +bool psphotVisualShowResidualImage (pmReadout *readout); +bool psphotVisualPlotApResid (psArray *sources, float mean, float error); +bool psphotVisualPlotChisq (psArray *sources); +bool psphotVisualPlotSourceSize (psMetadata *recipe, psMetadata *analysis, psArray *sources); +bool psphotVisualShowPetrosians (psArray *sources); +bool psphotVisualEraseOverlays (int channel, char *overlay); bool psphotPetrosian (pmSource *source, psMetadata *recipe, float skynoise, psImageMaskType maskVal); @@ -203,5 +275,5 @@ bool psphotPetrosianStats (pmSource *source); -// XXX old versions, currently disabled +// currently disabled: // bool psphotIsophotal (pmSource *source, psMetadata *recipe, psImageMaskType maskVal); // bool psphotAnnuli (pmSource *source, psMetadata *recipe, psImageMaskType maskVal); @@ -209,25 +281,16 @@ // XXX visualization functions related to radial profiles (disabled) -// bool psphotPetrosianVisualProfileByAngle (psVector *radius, psVector *flux); -// bool psphotPetrosianVisualProfileRadii (psVector *radius, psVector *flux, psVector *radiusBin, psVector *fluxBin, float peakFlux, float RadiusRef); -// bool psphotPetrosianVisualEllipticalContour (pmPetrosian *petrosian); -// bool psphotPetrosianVisualStats (psVector *radBin, psVector *fluxBin, -// psVector *refRadius, psVector *meanSB, -// psVector *petRatio, psVector *petRatioErr, psVector *fluxSum, -// float petRadius, float ratioForRadius, -// float petFlux, float radiusForFlux); - -bool psphotImageQuality (psMetadata *recipe, psArray *sources); +bool psphotPetrosianVisualProfileByAngle (psVector *radius, psVector *flux); +bool psphotPetrosianVisualProfileRadii (psVector *radius, psVector *flux, psVector *radiusBin, psVector *fluxBin, float peakFlux, float RadiusRef); +bool psphotPetrosianVisualEllipticalContour (pmSourceRadialFlux *radFlux, pmSourceExtendedPars *extpars); +bool psphotPetrosianVisualStats (psVector *radBin, psVector *fluxBin, + psVector *refRadius, psVector *meanSB, + psVector *petRatio, psVector *petRatioErr, psVector *fluxSum, + float petRadius, float ratioForRadius, + float petFlux, float radiusForFlux); + +bool psphotRadialBins (psMetadata *recipe, pmSource *source, float radiusMax, float skynoise); // structures & functions to support psf-convolved model fitting - -// pmPCMData : PSF Convolved Model data storage structure -typedef struct { - psImage *model; - psArray *dmodels; - psImage *modelConv; - psArray *dmodelsConv; -} pmPCMData; - // psf-convolved model fitting @@ -267,14 +330,40 @@ bool psphotDumpPSFStars (pmReadout *readout, pmPSFtry *try, float radius, psImageMaskType maskVal, psImageMaskType markVal); +bool psphotLoadSRCTEXT (pmFPA *fpa, pmConfig *config); + bool psphotCheckStarDistribution (psArray *sources, psArray *stars, pmPSFOptions *options); int psphotSupplementStars (psArray *stars, psArray *sources, psImageBinning *binning, int ix, int iy); - -// Create a background model for a readout -// Essentially identical to psphotModelBackground, but with a more convenient API for outsiders. -psImage *psphotBackgroundModel(pmReadout *ro, // Readout for which to generate background model - const pmConfig *config // Configuration - ); - +pmConfig *psphotForcedArguments(int argc, char **argv); +bool psphotForcedImageLoop (pmConfig *config); +bool psphotForcedReadout(pmConfig *config, const pmFPAview *view); + +pmConfig *psphotMakePSFArguments(int argc, char **argv); +bool psphotMakePSFImageLoop (pmConfig *config); +bool psphotMakePSFReadout(pmConfig *config, const pmFPAview *view); + +/**** psphotStack prototypes ****/ + +pmConfig *psphotStackArguments(int argc, char **argv); +bool psphotStackParseCamera (pmConfig *config); +bool psphotStackImageLoop (pmConfig *config); +bool psphotStackReadout (pmConfig *config, const pmFPAview *view); +bool psphotStackChisqImage (pmConfig *config, const pmFPAview *view); +bool psphotStackChisqImageAddReadout(const pmConfig *config, // Configuration + const pmFPAview *view, + pmReadout **chiReadout, + char *filename, + int index); + +bool psphotStackRemoveChisqFromInputs (pmConfig *config); +bool pmFPAfileRemoveSingle(psMetadata *files, const char *name, int num); + +psArray *psphotMatchSources (pmConfig *config, const pmFPAview *view); +bool psphotMatchSourcesReadout (psArray *objects, pmConfig *config, const pmFPAview *view, char *filename, int index); +bool psphotMatchSourcesToObjects (psArray *objects, psArray *sources, float RADIUS); + +bool psphotFitSourcesLinearStack (pmConfig *config, psArray *objects, bool final); +int pmPhotObjSortBySN (const void **a, const void **b); +int pmPhotObjSortByX (const void **a, const void **b); #endif Index: branches/tap_branches/psphot/src/psphotAddNoise.c =================================================================== --- branches/tap_branches/psphot/src/psphotAddNoise.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotAddNoise.c (revision 27838) @@ -1,13 +1,35 @@ # include "psphotInternal.h" -bool psphotAddNoise (pmReadout *readout, const psArray *sources, const psMetadata *recipe) { - return psphotAddOrSubNoise (readout, sources, recipe, true); +bool psphotAddNoise (pmConfig *config, const pmFPAview *view) { + return psphotAddOrSubNoise (config, view, true); } -bool psphotSubNoise (pmReadout *readout, const psArray *sources, const psMetadata *recipe) { - return psphotAddOrSubNoise (readout, sources, recipe, false); +bool psphotSubNoise (pmConfig *config, const pmFPAview *view) { + return psphotAddOrSubNoise (config, view, false); } -bool psphotAddOrSubNoise (pmReadout *readout, const psArray *sources, const psMetadata *recipe, bool add) { +// for now, let's store the detections on the readout->analysis for each readout +bool psphotAddOrSubNoise (pmConfig *config, const pmFPAview *view, bool add) +{ + bool status = true; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (!psphotAddOrSubNoiseReadout (config, view, "PSPHOT.INPUT", i, recipe, add)) { + psError (PSPHOT_ERR_CONFIG, false, "failed on to modify noise for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + +bool psphotAddOrSubNoiseReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe, bool add) { bool status = false; @@ -16,7 +38,18 @@ psEllipseAxes axes; - PS_ASSERT (readout, false); - PS_ASSERT (readout->parent, false); - PS_ASSERT (readout->parent->concepts, false); + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + psAssert (readout->parent, "missing cell?"); + psAssert (readout->parent->concepts, "missing concepts?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *sources = detections->allSources; + psAssert (sources, "missing sources?"); psTimerStart ("psphot.noise"); @@ -24,5 +57,5 @@ // user-defined masks to test for good/bad pixels (build from recipe list if not yet set) psImageMaskType maskVal = psMetadataLookupImageMask(&status, recipe, "MASK.PSPHOT"); // Mask value for bad pixels - assert (maskVal); + psAssert (maskVal, "missing mask value?"); // increase variance by factor*(object noise): Index: branches/tap_branches/psphot/src/psphotApResid.c =================================================================== --- branches/tap_branches/psphot/src/psphotApResid.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotApResid.c (revision 27838) @@ -4,5 +4,32 @@ // measure the aperture residual statistics and 2D variations -bool psphotApResid (pmConfig *config, pmReadout *readout, psArray *sources, pmPSF *psf) +// for now, let's store the detections on the readout->analysis for each readout +bool psphotApResid (pmConfig *config, const pmFPAview *view) +{ + bool status = true; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // skip the chisq image (optionally?) + int chisqNum = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.CHISQ.NUM"); + if (!status) chisqNum = -1; + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (i == chisqNum) continue; // skip chisq image + if (!psphotApResidReadout (config, view, "PSPHOT.INPUT", i, recipe)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to measure aperture residual for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + +bool psphotApResidReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe) { int Nfail = 0; @@ -13,14 +40,26 @@ pmSource *source; - PS_ASSERT_PTR_NON_NULL(config, false); - PS_ASSERT_PTR_NON_NULL(readout, false); - PS_ASSERT_PTR_NON_NULL(sources, false); - PS_ASSERT_PTR_NON_NULL(psf, false); - psTimerStart ("psphot.apresid"); - // select the appropriate recipe information - psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); - assert (recipe); + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *sources = detections->allSources; + psAssert (sources, "missing sources?"); + + if (!sources->n) { + psLogMsg ("psphot", PS_LOG_INFO, "no sources, skipping ap resid"); + return true; + } + + pmPSF *psf = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.PSF"); + psAssert (psf, "missing psf?"); // determine the number of allowed threads @@ -33,8 +72,8 @@ if (!measureAptrend) { // save nan values since these were not calculated - psMetadataAdd (recipe, PS_LIST_TAIL, "APMIFIT", PS_DATA_F32 | PS_META_REPLACE, "aperture residual", NAN); - psMetadataAdd (recipe, PS_LIST_TAIL, "DAPMIFIT", PS_DATA_F32 | PS_META_REPLACE, "ap residual scatter", NAN); - psMetadataAdd (recipe, PS_LIST_TAIL, "NAPMIFIT", PS_DATA_S32 | PS_META_REPLACE, "number of apresid stars", 0); - psMetadataAdd (recipe, PS_LIST_TAIL, "APLOSS", PS_DATA_F32 | PS_META_REPLACE, "aperture loss (mag)", NAN); + psMetadataAdd (readout->analysis, PS_LIST_TAIL, "APMIFIT", PS_DATA_F32 | PS_META_REPLACE, "aperture residual", NAN); + psMetadataAdd (readout->analysis, PS_LIST_TAIL, "DAPMIFIT", PS_DATA_F32 | PS_META_REPLACE, "ap residual scatter", NAN); + psMetadataAdd (readout->analysis, PS_LIST_TAIL, "NAPMIFIT", PS_DATA_S32 | PS_META_REPLACE, "number of apresid stars", 0); + psMetadataAdd (readout->analysis, PS_LIST_TAIL, "APLOSS", PS_DATA_F32 | PS_META_REPLACE, "aperture loss (mag)", NAN); return true; } @@ -217,8 +256,21 @@ } - // XXX set the min number of needed source more carefully - if ((Npsf < 15) && (APTREND_ORDER_MAX >= 4)) APTREND_ORDER_MAX = 3; - if ((Npsf < 11) && (APTREND_ORDER_MAX >= 3)) APTREND_ORDER_MAX = 2; - if ((Npsf < 8) && (APTREND_ORDER_MAX >= 2)) APTREND_ORDER_MAX = 1; + // this is a bit tricky, because we have two cases (MAP vs POLY), and they have a different + // definition for 'order' (order_MAP = order_POLY + 1). in addition, we have a + // user-specified MAX order, which we should respect, regardless of the mode + + // set the max order (0 = constant) which the number of psf stars can support: + // rule of thumb: require 3 stars per 'cell' (order+1)^2 + int MaxOrderForStars = 0; + if (Npsf >= 12) MaxOrderForStars = 1; // 4 cells + if (Npsf >= 27) MaxOrderForStars = 2; // 9 cells + if (Npsf >= 48) MaxOrderForStars = 3; // 16 cells + if (Npsf > 75) MaxOrderForStars = 4; // 25 cells + + pmTrend2DMode mode = PM_TREND_MAP; + if (mode == PM_TREND_MAP) { + MaxOrderForStars ++; + } + APTREND_ORDER_MAX = PS_MIN (APTREND_ORDER_MAX, MaxOrderForStars); psFree (psf->ApTrend); @@ -275,8 +327,8 @@ // save results for later output - psMetadataAdd (recipe, PS_LIST_TAIL, "APMIFIT", PS_DATA_F32 | PS_META_REPLACE, "aperture residual", psf->ApResid); - psMetadataAdd (recipe, PS_LIST_TAIL, "DAPMIFIT", PS_DATA_F32 | PS_META_REPLACE, "ap residual scatter", psf->dApResid); - psMetadataAdd (recipe, PS_LIST_TAIL, "NAPMIFIT", PS_DATA_S32 | PS_META_REPLACE, "number of apresid stars", psf->nApResid); - psMetadataAdd (recipe, PS_LIST_TAIL, "APLOSS", PS_DATA_F32 | PS_META_REPLACE, "aperture loss (mag)", psf->growth->apLoss); + psMetadataAdd (readout->analysis, PS_LIST_TAIL, "APMIFIT", PS_DATA_F32 | PS_META_REPLACE, "aperture residual", psf->ApResid); + psMetadataAdd (readout->analysis, PS_LIST_TAIL, "DAPMIFIT", PS_DATA_F32 | PS_META_REPLACE, "ap residual scatter", psf->dApResid); + psMetadataAdd (readout->analysis, PS_LIST_TAIL, "NAPMIFIT", PS_DATA_S32 | PS_META_REPLACE, "number of apresid stars", psf->nApResid); + psMetadataAdd (readout->analysis, PS_LIST_TAIL, "APLOSS", PS_DATA_F32 | PS_META_REPLACE, "aperture loss (mag)", psf->growth->apLoss); psLogMsg ("psphot.apresid", PS_LOG_DETAIL, "aperture residual: %f +/- %f\n", psf->ApResid, psf->dApResid); @@ -295,8 +347,8 @@ escape: // save nan values since these were not calculated - psMetadataAdd (recipe, PS_LIST_TAIL, "APMIFIT", PS_DATA_F32 | PS_META_REPLACE, "aperture residual", NAN); - psMetadataAdd (recipe, PS_LIST_TAIL, "DAPMIFIT", PS_DATA_F32 | PS_META_REPLACE, "ap residual scatter", NAN); - psMetadataAdd (recipe, PS_LIST_TAIL, "NAPMIFIT", PS_DATA_S32 | PS_META_REPLACE, "number of apresid stars", 0); - psMetadataAdd (recipe, PS_LIST_TAIL, "APLOSS", PS_DATA_F32 | PS_META_REPLACE, "aperture loss (mag)", NAN); + psMetadataAdd (readout->analysis, PS_LIST_TAIL, "APMIFIT", PS_DATA_F32 | PS_META_REPLACE, "aperture residual", NAN); + psMetadataAdd (readout->analysis, PS_LIST_TAIL, "DAPMIFIT", PS_DATA_F32 | PS_META_REPLACE, "ap residual scatter", NAN); + psMetadataAdd (readout->analysis, PS_LIST_TAIL, "NAPMIFIT", PS_DATA_S32 | PS_META_REPLACE, "number of apresid stars", 0); + psMetadataAdd (readout->analysis, PS_LIST_TAIL, "APLOSS", PS_DATA_F32 | PS_META_REPLACE, "aperture loss (mag)", NAN); psFree (xPos); @@ -305,5 +357,6 @@ psFree (mag); psFree (dMag); - return false; + return true; + // this is a quality error, not a programming error } Index: branches/tap_branches/psphot/src/psphotArguments.c =================================================================== --- branches/tap_branches/psphot/src/psphotArguments.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotArguments.c (revision 27838) @@ -218,8 +218,8 @@ // override any configuration-specified source for these files // - pmConfigFileSetsMD (config->arguments, &argc, argv, "MASK", "-mask", "-masklist"); - pmConfigFileSetsMD (config->arguments, &argc, argv, "VARIANCE", "-variance", "-variancelist"); - pmConfigFileSetsMD (config->arguments, &argc, argv, "PSPHOT.PSF", "-psf", "-psflist"); - pmConfigFileSetsMD (config->arguments, &argc, argv, "SRC", "-src", "-srclist"); + pmConfigFileSetsMD (config->arguments, &argc, argv, "MASK", "-mask", "-masklist"); + pmConfigFileSetsMD (config->arguments, &argc, argv, "VARIANCE", "-variance", "-variancelist"); + pmConfigFileSetsMD (config->arguments, &argc, argv, "PSPHOT.PSF", "-psf", "-psflist"); + pmConfigFileSetsMD (config->arguments, &argc, argv, "SRC", "-src", "-srclist"); if (argc == 1) { @@ -228,6 +228,6 @@ } - if (psArgumentGet(argc, argv, "-help") - || psArgumentGet(argc, argv, "-h")) + if (psArgumentGet(argc, argv, "-help") || + psArgumentGet(argc, argv, "-h")) writeHelpInfo(argv[0], config, stdout); Index: branches/tap_branches/psphot/src/psphotBasicDeblend.c =================================================================== --- branches/tap_branches/psphot/src/psphotBasicDeblend.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotBasicDeblend.c (revision 27838) @@ -1,5 +1,23 @@ # include "psphotInternal.h" -bool psphotBasicDeblend (psArray *sources, psMetadata *recipe) { +// for now, let's store the detections on the readout->analysis for each readout +bool psphotBasicDeblend (pmConfig *config, const pmFPAview *view) +{ + bool status = true; + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (!psphotBasicDeblendReadout (config, view, "PSPHOT.INPUT", i)) { + psError (PSPHOT_ERR_CONFIG, false, "failed on basic deblend analysis for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + +bool psphotBasicDeblendReadout (pmConfig *config, const pmFPAview *view, const char *filename, int fileIndex) { int N; @@ -8,7 +26,29 @@ pmSource *source, *testSource; + psTimerStart ("psphot.deblend.basic"); + int Nblend = 0; - psTimerStart ("psphot.deblend.basic"); + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, fileIndex); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *sources = detections->newSources; + psAssert (sources, "missing sources?"); + + if (!sources->n) { + psLogMsg ("psphot", PS_LOG_INFO, "no sources, skipping basic deblend"); + return true; + } + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); float FRACTION = psMetadataLookupF32 (&status, recipe, "DEBLEND_PEAK_FRACTION"); Index: branches/tap_branches/psphot/src/psphotBlendFit.c =================================================================== --- branches/tap_branches/psphot/src/psphotBlendFit.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotBlendFit.c (revision 27838) @@ -1,6 +1,28 @@ # include "psphotInternal.h" +// for now, let's store the detections on the readout->analysis for each readout +bool psphotBlendFit (pmConfig *config, const pmFPAview *view) +{ + bool status = true; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (!psphotBlendFitReadout (config, view, "PSPHOT.INPUT", i, recipe)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to fit sources (non-linear) for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + // XXX I don't like this name -bool psphotBlendFit (pmConfig *config, pmReadout *readout, psArray *sources, pmPSF *psf) { +bool psphotBlendFitReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe) { int Nfit = 0; @@ -12,7 +34,24 @@ psTimerStart ("psphot.fit.nonlinear"); - // select the appropriate recipe information - psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); - assert (recipe); + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *sources = detections->allSources; + psAssert (sources, "missing sources?"); + + if (!sources->n) { + psLogMsg ("psphot", PS_LOG_INFO, "no sources, skipping blend fit"); + return true; + } + + pmPSF *psf = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.PSF"); + psAssert (psf, "missing psf?"); // determine the number of allowed threads @@ -40,5 +79,5 @@ psphotInitLimitsPSF (recipe, readout); psphotInitLimitsEXT (recipe); - psphotInitRadiusPSF (recipe, psf->type); + psphotInitRadiusPSF (recipe, readout->analysis, psf->type); // starts the timer, sets up the array of fitSets @@ -47,4 +86,8 @@ // source analysis is done in S/N order (brightest first) sources = psArraySort (sources, pmSourceSortBySN); + if (!sources->n) { + psLogMsg ("psphot", PS_LOG_INFO, "no sources, skipping blend"); + return true; + } // choose Cx, Cy (see psphotThreadTools.c for overview of the concepts) @@ -272,109 +315,2 @@ } -# if (0) -bool psphotBlendFit_Unthreaded (int *nfit, int *npsf, int *next, int *nfail, pmReadout *readout, psMetadata *recipe, psArray *sources, pmPSF *psf, psArray *newSources) { - - bool status = false; - int Nfit = 0; - int Npsf = 0; - int Next = 0; - int Nfail = 0; - - // bit-masks to test for good/bad pixels - psImageMaskType maskVal = psMetadataLookupImageMask(&status, recipe, "MASK.PSPHOT"); - assert (maskVal); - - // bit-mask to mark pixels not used in analysis - psImageMaskType markVal = psMetadataLookupImageMask(&status, recipe, "MARK.PSPHOT"); - assert (markVal); - - // maskVal is used to test for rejected pixels, and must include markVal - maskVal |= markVal; - - // S/N limit to perform full non-linear fits - float FIT_SN_LIM = psMetadataLookupF32 (&status, recipe, "FULL_FIT_SN_LIM"); - - // option to limit analysis to a specific region - char *region = psMetadataLookupStr (&status, recipe, "ANALYSIS_REGION"); - psRegion AnalysisRegion = psRegionForImage (readout->image, psRegionFromString (region)); - if (psRegionIsNaN (AnalysisRegion)) psAbort("analysis region mis-defined"); - - for (int i = 0; i < sources->n; i++) { - pmSource *source = sources->data[i]; - - // skip non-astronomical objects (very likely defects) - if (source->mode & PM_SOURCE_MODE_BLEND) continue; - if (source->mode & PM_SOURCE_MODE_CR_LIMIT) continue; - if (source->type == PM_SOURCE_TYPE_DEFECT) continue; - if (source->type == PM_SOURCE_TYPE_SATURATED) continue; - - // skip DBL second sources (ie, added by psphotFitBlob) - if (source->mode & PM_SOURCE_MODE_PAIR) continue; - - // limit selection to some SN limit - if (source->peak->SN < FIT_SN_LIM) continue; - - // exclude sources outside optional analysis region - if (source->peak->xf < AnalysisRegion.x0) continue; - if (source->peak->yf < AnalysisRegion.y0) continue; - if (source->peak->xf > AnalysisRegion.x1) continue; - if (source->peak->yf > AnalysisRegion.y1) continue; - - // if model is NULL, we don't have a starting guess - if (source->modelPSF == NULL) continue; - - // skip sources which are insignificant flux? - // XXX this is somewhat ad-hoc - if (source->modelPSF->params->data.F32[1] < 0.1) { - psTrace ("psphot", 5, "skipping near-zero source: %f, %f : %f\n", - source->modelPSF->params->data.F32[1], - source->modelPSF->params->data.F32[2], - source->modelPSF->params->data.F32[3]); - continue; - } - - // replace object in image - if (source->tmpFlags & PM_SOURCE_TMPF_SUBTRACTED) { - pmSourceAdd (source, PM_MODEL_OP_FULL, maskVal); - } - Nfit ++; - - // try fitting PSFs or extended sources depending on source->mode - // these functions subtract the resulting fitted source - if (source->mode & PM_SOURCE_MODE_EXT_LIMIT) { - if (psphotFitBlob (readout, source, newSources, psf, maskVal, markVal)) { - source->type = PM_SOURCE_TYPE_EXTENDED; - psTrace ("psphot", 5, "source at %7.1f, %7.1f is ext", source->peak->xf, source->peak->yf); - Next ++; - continue; - } - } else { - if (psphotFitBlend (readout, source, psf, maskVal, markVal)) { - source->type = PM_SOURCE_TYPE_STAR; - psTrace ("psphot", 5, "source at %7.1f, %7.1f is psf", source->peak->xf, source->peak->yf); - Npsf ++; - continue; - } - } - - psTrace ("psphot", 5, "source at %7.1f, %7.1f failed", source->peak->xf, source->peak->yf); - Nfail ++; - - // re-subtract the object, leave local sky - pmSourceCacheModel (source, maskVal); - pmSourceSub (source, PM_MODEL_OP_FULL, maskVal); - } - - // change the value of a scalar on the array (wrap this and put it in psArray.h) - *nfit = Nfit; - *npsf = Npsf; - *next = Next; - *nfail = Nfail; - - // moments are modified by the fit; re-display - psphotVisualPlotMoments (recipe, sources); - psphotVisualShowResidualImage (readout); - - return true; -} -# endif Index: branches/tap_branches/psphot/src/psphotChoosePSF.c =================================================================== --- branches/tap_branches/psphot/src/psphotChoosePSF.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotChoosePSF.c (revision 27838) @@ -1,18 +1,69 @@ # include "psphotInternal.h" +// generate a PSF model for inputs without PSF models already loaded +bool psphotChoosePSF (pmConfig *config, const pmFPAview *view) +{ + bool status = true; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // skip the chisq image (optionally?) + int chisqNum = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.CHISQ.NUM"); + if (!status) chisqNum = -1; + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (i == chisqNum) continue; // skip chisq image + if (!psphotChoosePSFReadout (config, view, "PSPHOT.INPUT", i, recipe)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to choose a psf model for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + // try PSF models and select best option -pmPSF *psphotChoosePSF (pmReadout *readout, psArray *sources, psMetadata *recipe) { +bool psphotChoosePSFReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe) { bool status; psTimerStart ("psphot.choose.psf"); + + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + // do not generate a PSF if we already were supplied one + if (psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.PSF")) { + psLogMsg ("psphot", PS_LOG_DETAIL, "psf model supplied for input file %d", index); + return true; + } + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *sources = detections->newSources; + psAssert (sources, "missing sources?"); + + if (!sources->n) { + psLogMsg ("psphot", PS_LOG_INFO, "no sources, skipping PSF model"); + return false; + } // bit-masks to test for good/bad pixels psImageMaskType maskVal = psMetadataLookupImageMask(&status, recipe, "MASK.PSPHOT"); - assert (maskVal); + psAssert (maskVal, "missing mask value?"); // bit-mask to mark pixels not used in analysis psImageMaskType markVal = psMetadataLookupImageMask(&status, recipe, "MARK.PSPHOT"); - assert (markVal); + psAssert (markVal, "missing mark value?"); // maskVal is used to test for rejected pixels, and must include markVal @@ -61,7 +112,10 @@ // assert (status); - // We have calculated a Gaussian window function, use that for both the PSF fit radius and - // the aperture radius (scaling SIGMA) - float gaussSigma = psMetadataLookupF32(&status, recipe, "MOMENTS_GAUSS_SIGMA"); + // values on readout->analysis if we have calculated a Gaussian window function, use that + // for both the PSF fit radius and the aperture radius (scaling SIGMA), otherwise base the value on the recipe value for MOMENTS_GAUSS_SIGMA + float gaussSigma = psMetadataLookupF32 (&status, readout->analysis, "MOMENTS_GAUSS_SIGMA"); + if (!status) { + gaussSigma = psMetadataLookupF32 (&status, recipe, "MOMENTS_GAUSS_SIGMA"); + } float fitScale = psMetadataLookupF32(&status, recipe, "PSF_FIT_RADIUS_SCALE"); float apScale = psMetadataLookupF32(&status, recipe, "PSF_APERTURE_SCALE"); @@ -70,10 +124,6 @@ // XXX use the same radii for standard analysis as for the PSF creation - psMetadataAddF32(recipe, PS_LIST_TAIL, "PSF_FIT_RADIUS", PS_META_REPLACE, "fit radius", options->fitRadius); - psMetadataAddF32(recipe, PS_LIST_TAIL, "PSF_APERTURE", PS_META_REPLACE, "psf aperture", options->apRadius); - - // XXX ROBUST seems to be too agressive given the small numbers. - // options->stats = psStatsAlloc (PS_STAT_ROBUST_MEDIAN | PS_STAT_ROBUST_STDEV); - options->stats = psStatsAlloc (PS_STAT_SAMPLE_MEDIAN | PS_STAT_SAMPLE_STDEV); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "PSF_FIT_RADIUS", PS_META_REPLACE, "fit radius", options->fitRadius); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "PSF_APERTURE", PS_META_REPLACE, "psf aperture", options->apRadius); // dimensions of the field for which the PSF is defined @@ -116,4 +166,11 @@ psLogMsg ("psphot.pspsf", PS_LOG_DETAIL, "selected candidate %ld PSF objects\n", stars->n); + if (stars->n < 50) { + // ROBUST is too agressive if we only have a small number of PSF stars + options->stats = psStatsAlloc (PS_STAT_SAMPLE_MEDIAN | PS_STAT_SAMPLE_STDEV); + } else { + options->stats = psStatsAlloc (PS_STAT_ROBUST_MEDIAN | PS_STAT_ROBUST_STDEV); + } + // get the list pointers for the PSF_MODEL entries psArray *modelNames = NULL; @@ -155,5 +212,5 @@ bool status = true; status &= psphotMakeFluxScale (readout->image, recipe, psf); - status &= psphotPSFstats (readout, recipe, psf); + status &= psphotPSFstats (readout, psf); if (!status) { psError(PSPHOT_ERR_PSF, false, "Failed to fit any models when choosing PSF"); @@ -230,5 +287,5 @@ bool status = true; status &= psphotMakeFluxScale (readout->image, recipe, psf); - status &= psphotPSFstats (readout, recipe, psf); + status &= psphotPSFstats (readout, psf); if (!status) { psError(PSPHOT_ERR_PSF, false, "Failed to fit any models when choosing PSF"); @@ -287,5 +344,5 @@ psFree (models); - if (!psphotPSFstats (readout, recipe, psf)) { + if (!psphotPSFstats (readout, psf)) { psError(PSPHOT_ERR_PSF, false, "cannot measure PSF shape terms"); psFree(options); @@ -299,9 +356,20 @@ psFree (options); - return (psf); + + // save PSF on readout->analysis + if (!psMetadataAddPtr (readout->analysis, PS_LIST_TAIL, "PSPHOT.PSF", PS_META_REPLACE | PS_DATA_UNKNOWN, "psphot psf model", psf)) { + psError (PSPHOT_ERR_UNKNOWN, false, "problem saving sources on readout"); + return false; + } + psFree (psf); // XXX double-check + + // move into psphotChoosePSF + psphotVisualShowPSFModel (readout, psf); + + return true; } // measure average parameters of the PSF model -bool psphotPSFstats (pmReadout *readout, psMetadata *recipe, pmPSF *psf) { +bool psphotPSFstats (pmReadout *readout, pmPSF *psf) { psEllipseShape shape; @@ -309,5 +377,4 @@ PS_ASSERT_PTR_NON_NULL(readout, false); - PS_ASSERT_PTR_NON_NULL(recipe, false); PS_ASSERT_PTR_NON_NULL(psf, false); @@ -329,7 +396,7 @@ pmModel *modelPSF = pmModelFromPSFforXY (psf, xc, yc, 1.0); if (!modelPSF) { - fprintf (stderr, "?"); - continue; - } + fprintf (stderr, "?"); + continue; + } // get the model full-width at half-max @@ -345,7 +412,7 @@ float FWHM_MINOR = FWHM_MAJOR * (axes.minor / axes.major); if (!isfinite(FWHM_MAJOR) || !isfinite(FWHM_MINOR)) { - fprintf (stderr, "!"); - continue; - } + fprintf (stderr, "!"); + continue; + } psVectorAppend (fwhmMajor, FWHM_MAJOR); psVectorAppend (fwhmMinor, FWHM_MINOR); @@ -359,8 +426,10 @@ } - psMetadataAddF32 (recipe, PS_LIST_TAIL, "FWHM_MAJ", PS_META_REPLACE, "PSF FWHM Major axis (mean)", stats->sampleMean); - psMetadataAddF32 (recipe, PS_LIST_TAIL, "FW_MJ_SG", PS_META_REPLACE, "PSF FWHM Major axis (sigma)", stats->sampleStdev); - psMetadataAddF32 (recipe, PS_LIST_TAIL, "FW_MJ_LQ", PS_META_REPLACE, "PSF FWHM Major axis (lower quartile)", stats->sampleLQ); - psMetadataAddF32 (recipe, PS_LIST_TAIL, "FW_MJ_UQ", PS_META_REPLACE, "PSF FWHM Major axis (upper quartile)", stats->sampleUQ); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "FWHM_MAJ", PS_META_REPLACE, "PSF FWHM Major axis (mean)", stats->sampleMean); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "FW_MJ_SG", PS_META_REPLACE, "PSF FWHM Major axis (sigma)", stats->sampleStdev); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "FW_MJ_LQ", PS_META_REPLACE, "PSF FWHM Major axis (lower quartile)", stats->sampleLQ); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "FW_MJ_UQ", PS_META_REPLACE, "PSF FWHM Major axis (upper quartile)", stats->sampleUQ); + + float fwhmMaj = stats->sampleMean; // FWHM on major axis if (!psVectorStats (stats, fwhmMinor, NULL, NULL, 0)) { @@ -368,12 +437,20 @@ return false; } - psMetadataAddF32 (recipe, PS_LIST_TAIL, "FWHM_MIN", PS_META_REPLACE, "PSF FWHM Minor axis (mean)", stats->sampleMean); - psMetadataAddF32 (recipe, PS_LIST_TAIL, "FW_MN_SG", PS_META_REPLACE, "PSF FWHM Minor axis (sigma)", stats->sampleStdev); - psMetadataAddF32 (recipe, PS_LIST_TAIL, "FW_MN_LQ", PS_META_REPLACE, "PSF FWHM Minor axis (lower quartile)", stats->sampleLQ); - psMetadataAddF32 (recipe, PS_LIST_TAIL, "FW_MN_UQ", PS_META_REPLACE, "PSF FWHM Minor axis (upper quartile)", stats->sampleUQ); - - psMetadataAddF32 (recipe, PS_LIST_TAIL, "ANGLE", PS_META_REPLACE, "PSF angle", axes.theta); - psMetadataAddS32 (recipe, PS_LIST_TAIL, "NPSFSTAR", PS_META_REPLACE, "Number of stars used to make PSF", psf->nPSFstars); - psMetadataAddBool(recipe, PS_LIST_TAIL, "PSFMODEL", PS_META_REPLACE, "Valid PSF Model?", true); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "FWHM_MIN", PS_META_REPLACE, "PSF FWHM Minor axis (mean)", stats->sampleMean); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "FW_MN_SG", PS_META_REPLACE, "PSF FWHM Minor axis (sigma)", stats->sampleStdev); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "FW_MN_LQ", PS_META_REPLACE, "PSF FWHM Minor axis (lower quartile)", stats->sampleLQ); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "FW_MN_UQ", PS_META_REPLACE, "PSF FWHM Minor axis (upper quartile)", stats->sampleUQ); + + float fwhmMin = stats->sampleMean; // FWHM on minor axis + if (readout->parent) { + pmChip *chip = readout->parent->parent; // Parent chip + psAssert(chip, "Cell should be attached to a chip."); + psMetadataItem *item = psMetadataLookup(chip->concepts, "CHIP.SEEING"); // Item with chip + item->data.F32 = 0.5 * (fwhmMaj + fwhmMin); + } + + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "ANGLE", PS_META_REPLACE, "PSF angle", axes.theta); + psMetadataAddS32 (readout->analysis, PS_LIST_TAIL, "NPSFSTAR", PS_META_REPLACE, "Number of stars used to make PSF", psf->nPSFstars); + psMetadataAddBool(readout->analysis, PS_LIST_TAIL, "PSFMODEL", PS_META_REPLACE, "Valid PSF Model?", true); psFree (fwhmMajor); @@ -384,5 +461,5 @@ // determine approximate PSF shape parameters based on the moments -bool psphotMomentsStats (pmReadout *readout, psMetadata *recipe, psArray *sources) { +bool psphotMomentsStats (pmReadout *readout, psArray *sources) { // without the PSF model, we can only rely on the source->moments @@ -398,5 +475,4 @@ PS_ASSERT_PTR_NON_NULL(readout, false); - PS_ASSERT_PTR_NON_NULL(recipe, false); PS_ASSERT_PTR_NON_NULL(sources, false); @@ -426,17 +502,22 @@ FWHM_T /= FWHM_N; - psMetadataAddF32 (recipe, PS_LIST_TAIL, "FWHM_MAJ", PS_META_REPLACE, "PSF FWHM Major axis (mean)", FWHM_X); - psMetadataAddF32 (recipe, PS_LIST_TAIL, "FW_MJ_SG", PS_META_REPLACE, "PSF FWHM Major axis (sigma)", 0); - psMetadataAddF32 (recipe, PS_LIST_TAIL, "FW_MJ_LQ", PS_META_REPLACE, "PSF FWHM Major axis (lower quartile)", 0); - psMetadataAddF32 (recipe, PS_LIST_TAIL, "FW_MJ_UQ", PS_META_REPLACE, "PSF FWHM Major axis (upper quartile)", 0); - - psMetadataAddF32 (recipe, PS_LIST_TAIL, "FWHM_MIN", PS_META_REPLACE, "PSF FWHM Minor axis (mean)", FWHM_Y); - psMetadataAddF32 (recipe, PS_LIST_TAIL, "FW_MN_SG", PS_META_REPLACE, "PSF FWHM Minor axis (sigma)", 0); - psMetadataAddF32 (recipe, PS_LIST_TAIL, "FW_MN_LQ", PS_META_REPLACE, "PSF FWHM Minor axis (lower quartile)", 0); - psMetadataAddF32 (recipe, PS_LIST_TAIL, "FW_MN_UQ", PS_META_REPLACE, "PSF FWHM Minor axis (upper quartile)", 0); - - psMetadataAddF32 (recipe, PS_LIST_TAIL, "ANGLE", PS_META_REPLACE, "PSF angle", FWHM_T); - psMetadataAddS32 (recipe, PS_LIST_TAIL, "NPSFSTAR", PS_META_REPLACE, "Number of stars used to make PSF", 0); - psMetadataAddBool(recipe, PS_LIST_TAIL, "PSFMODEL", PS_META_REPLACE, "Valid PSF Model?", false); + if (readout->parent) { + pmChip *chip = readout->parent->parent; // Parent chip + psAssert(chip, "Cell should be attached to a chip."); + psMetadataItem *item = psMetadataLookup(chip->concepts, "CHIP.SEEING"); // Item with chip + item->data.F32 = 0.5 * (FWHM_X + FWHM_Y); + } + + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "FWHM_MAJ", PS_META_REPLACE, "PSF FWHM Major axis (mean)", FWHM_X); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "FW_MJ_SG", PS_META_REPLACE, "PSF FWHM Major axis (sigma)", 0); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "FW_MJ_LQ", PS_META_REPLACE, "PSF FWHM Major axis (lower quartile)", 0); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "FW_MJ_UQ", PS_META_REPLACE, "PSF FWHM Major axis (upper quartile)", 0); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "FWHM_MIN", PS_META_REPLACE, "PSF FWHM Minor axis (mean)", FWHM_Y); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "FW_MN_SG", PS_META_REPLACE, "PSF FWHM Minor axis (sigma)", 0); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "FW_MN_LQ", PS_META_REPLACE, "PSF FWHM Minor axis (lower quartile)", 0); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "FW_MN_UQ", PS_META_REPLACE, "PSF FWHM Minor axis (upper quartile)", 0); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "ANGLE", PS_META_REPLACE, "PSF angle", FWHM_T); + psMetadataAddS32 (readout->analysis, PS_LIST_TAIL, "NPSFSTAR", PS_META_REPLACE, "Number of stars used to make PSF", 0); + psMetadataAddBool(readout->analysis, PS_LIST_TAIL, "PSFMODEL", PS_META_REPLACE, "Valid PSF Model?", false); return true; Index: branches/tap_branches/psphot/src/psphotCleanup.c =================================================================== --- branches/tap_branches/psphot/src/psphotCleanup.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotCleanup.c (revision 27838) @@ -14,9 +14,6 @@ psFree (config); - psThreadPoolFinalize (); - psTimerStop (); psMemCheckCorruption (stderr, true); pmModelClassCleanup (); - psTimeFinalize (); pmSourceFitSetDone (); pmConceptsDone (); Index: branches/tap_branches/psphot/src/psphotCullPeaks.c =================================================================== --- branches/tap_branches/psphot/src/psphotCullPeaks.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotCullPeaks.c (revision 27838) @@ -5,9 +5,10 @@ */ psErrorCode -psphotCullPeaks(const psImage *image, // the image wherein lives the footprint - const psImage *weight, // corresponding variance image +psphotCullPeaks(const pmReadout *readout, // the image wherein lives the footprint const psMetadata *recipe, psArray *footprints) { // array of pmFootprints + bool status = false; + float nsigma_delta = psMetadataLookupF32(&status, recipe, "FOOTPRINT_CULL_NSIGMA_DELTA"); if (!status) { @@ -22,14 +23,26 @@ fPadding = 0; } - const float skyStdev = psMetadataLookupF32(NULL, recipe, "SKY_STDEV"); + const float skyStdev = psMetadataLookupF32(&status, readout->analysis, "SKY_STDEV"); + psAssert (status, "cannot find SKY_STDEV in readout->analysis"); + const float min_threshold = nsigma_min*skyStdev; for (int i = 0; i < footprints->n; i++) { + // if (i % 50 == 0) fprintf (stderr, "cull %d\n", i); pmFootprint *fp = footprints->data[i]; - if (pmFootprintCullPeaks(image, weight, fp, nsigma_delta, fPadding, min_threshold) != PS_ERR_NONE) { + if (fp->npix > 30000) { + fprintf (stderr, "big footprint: %f %f to %f %f (%d pix)\n", fp->bbox.x0, fp->bbox.y0, fp->bbox.x1, fp->bbox.y1, fp->npix); + } + psTimerStart ("psphot.cull.footprints"); + if (pmFootprintCullPeaks(readout->image, readout->variance, fp, nsigma_delta, fPadding, min_threshold) != PS_ERR_NONE) { return psError(PS_ERR_UNKNOWN, false, "Culling pmFootprint %d", fp->id); } + float dtime = psTimerMark ("psphot.cull.footprints"); + if (dtime > 1.0) { + fprintf (stderr, "slow cull for %d (%f sec)\n", i, dtime); + } } - return PS_ERR_NONE; } + +// psLogMsg ("psphot", PS_LOG_INFO, "%ld peaks, %ld total footprints: %f sec\n", detections->peaks->n, detections->footprints->n, Index: branches/tap_branches/psphot/src/psphotDeblendSatstars.c =================================================================== --- branches/tap_branches/psphot/src/psphotDeblendSatstars.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotDeblendSatstars.c (revision 27838) @@ -1,8 +1,27 @@ # include "psphotInternal.h" -bool psphotDeblendSatstars (pmReadout *readout, psArray *sources, psMetadata *recipe) { +// for now, let's store the detections on the readout->analysis for each readout +bool psphotDeblendSatstars (pmConfig *config, const pmFPAview *view) +{ + bool status = true; + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (!psphotDeblendSatstarsReadout (config, view, "PSPHOT.INPUT", i)) { + psError (PSPHOT_ERR_CONFIG, false, "failed on saturated star deblend analysis for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + +bool psphotDeblendSatstarsReadout (pmConfig *config, const pmFPAview *view, const char *filename, int fileIndex) { int N; pmSource *source; + bool status; psTimerStart ("psphot.deblend.sat"); @@ -11,5 +30,26 @@ float SAT_MIN_RADIUS = 5.0; - bool status; + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, fileIndex); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *sources = detections->newSources; + psAssert (sources, "missing sources?"); + + if (!sources->n) { + psLogMsg ("psphot", PS_LOG_INFO, "no sources, skipping satstar blend"); + return true; + } + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + pmCell *cell = readout->parent; float SATURATION = 0.75*psMetadataLookupF32 (&status, cell->concepts, "CELL.SATURATION"); Index: branches/tap_branches/psphot/src/psphotDefineFiles.c =================================================================== --- branches/tap_branches/psphot/src/psphotDefineFiles.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotDefineFiles.c (revision 27838) @@ -126,4 +126,13 @@ } + if (psMetadataLookupPtr(NULL, config->arguments, "SRCTEXT")) { + // XXX cannot use pmFPAfileDefineFromArgs: this is explicitly a FITS-based I/O function + // supply the attach the + if (!psphotLoadSRCTEXT(input->fpa, config)) { + psError(PSPHOT_ERR_CONFIG, false, "Failed to load PSPHOT.INPUT.TEXT"); + return status; + } + } + if (psMetadataLookupPtr(NULL, config->arguments, "PSPHOT.PSF")) { pmFPAfileBindFromArgs(&status, input, config, "PSPHOT.PSF.LOAD", "PSPHOT.PSF"); Index: branches/tap_branches/psphot/src/psphotEfficiency.c =================================================================== --- branches/tap_branches/psphot/src/psphotEfficiency.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotEfficiency.c (revision 27838) @@ -4,6 +4,5 @@ PM_MODEL_STATUS_BADARGS | PM_MODEL_STATUS_LIMITS) // Mask to apply to models -//#define TESTING - +# define TESTING 0 // Calculate the limiting magnitude for an image @@ -17,5 +16,5 @@ float *covarFactor,// Covariance factor const pmReadout *ro, // Readout of interest - const pmPSF *psf, // Point-spread function + pmPSF *psf, // Point-spread function float thresh, // Threshold for source identification float smoothSigma, // Gaussian smoothing sigma @@ -33,8 +32,6 @@ psKernel *kernel = psImageSmoothKernel(smoothSigma, smoothNsigma); // Kernel used for smoothing - psKernel *newCovar = psImageCovarianceCalculate(kernel, ro->covariance); // Covariance matrix + *covarFactor = psImageCovarianceCalculateFactor(kernel, ro->covariance); psFree(kernel); - *covarFactor = psImageCovarianceFactor(newCovar); // Variance factor - psFree(newCovar); psStats *stats = psStatsAlloc(PS_STAT_ROBUST_MEDIAN); // Statistics for variance @@ -128,4 +125,6 @@ psFree(rng); + bool oldThreads = pmReadoutFakeThreads(true); // Old threading status + pmReadout *fakeRO = pmReadoutAlloc(NULL); // Fake readout if (!pmReadoutFakeFromVectors(fakeRO, numCols, numRows, xAll, yAll, magAll, @@ -142,4 +141,6 @@ psFree(yAll); + pmReadoutFakeThreads(oldThreads); + psBinaryOp(ro->image, ro->image, "+", fakeRO->image); psFree(fakeRO); @@ -148,48 +149,82 @@ } +bool psphotEfficiency (pmConfig *config, const pmFPAview *view) +{ + bool status = true; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // skip the chisq image (optionally?) + int chisqNum = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.CHISQ.NUM"); + if (!status) chisqNum = -1; + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (i == chisqNum) continue; // skip chisq image + if (!psphotEfficiencyReadout (config, view, "PSPHOT.INPUT", i, recipe)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to measure detection efficiency for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} // Determine detection efficiency -bool psphotEfficiency(pmConfig *config, pmReadout *readout, const pmFPAview *view, const pmPSF *psf, - psMetadata *recipe, const psArray *realSources) +bool psphotEfficiencyReadout(pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe) { + bool status = true; + + psTimerStart("psphot.fake"); + + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *realSources = detections->allSources; + psAssert (realSources, "missing sources?"); + + // XXX do we need to skip this step if we do not have a psf? + pmPSF *psf = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.PSF"); + psAssert (psf, "missing psf?"); + PM_ASSERT_READOUT_NON_NULL(readout, false); PM_ASSERT_READOUT_IMAGE(readout, false); PS_ASSERT_PTR_NON_NULL(psf, false); PS_ASSERT_METADATA_NON_NULL(recipe, false); - PS_ASSERT_ARRAY_NON_NULL(realSources, false); - - psTimerStart("psphot.fake"); // Collect recipe information - float fwhmMajor = psMetadataLookupF32(NULL, recipe, "FWHM_MAJ"); // PSF size in x - float fwhmMinor = psMetadataLookupF32(NULL, recipe, "FWHM_MIN"); // PSF size in y + float smoothNsigma = psMetadataLookupF32(&status, recipe, "PEAKS_SMOOTH_NSIGMA"); // Smoothing limit + psAssert (status && isfinite(smoothNsigma), "Unable to find PEAKS_SMOOTH_NSIGMA in recipe (or invalid value)"); + + float thresh = psMetadataLookupF32(&status, recipe, "PEAKS_NSIGMA_LIMIT_2"); + psAssert (status && isfinite(thresh), "Unable to find PEAKS_NSIGMA_LIMIT_2 in recipe (or invalid value)"); + + psVector *magOffsets = psMetadataLookupVector(&status, recipe, "EFF.MAG"); // Magnitude offsets + psAssert (status, "Unable to find EFF.MAG F32 vector in recipe"); + psAssert (magOffsets->type.type == PS_TYPE_F32, "Unable to find EFF.MAG F32 vector in recipe"); + + int numSources = psMetadataLookupS32(&status, recipe, "EFF.NUM"); // Number of sources for each bin + psAssert (status && (numSources > 0), "Unable to find EFF.NUM in recipe (or invalid value)"); + + float minGauss = psMetadataLookupF32(&status, recipe, "PEAKS_MIN_GAUSS"); // Minimum valid fraction of kernel + psAssert (status && isfinite(minGauss), "PEAKS_MIN_GAUSS is not set in recipe (or invalid)"); + + // find the PSF size information (why is this not part of the psf structure?) + float fwhmMajor = psMetadataLookupF32(NULL, readout->analysis, "FWHM_MAJ"); // PSF size in x + float fwhmMinor = psMetadataLookupF32(NULL, readout->analysis, "FWHM_MIN"); // PSF size in y if (!isfinite(fwhmMajor) || !isfinite(fwhmMinor) || fwhmMajor == 0.0 || fwhmMinor == 0.0) { - psError(PSPHOT_ERR_CONFIG, false, "Unable to find FWHM_MAJ and FWHM_MIN in recipe"); - return false; - } - float smoothNsigma = psMetadataLookupF32(NULL, recipe, "PEAKS_SMOOTH_NSIGMA"); // Smoothing limit - if (!isfinite(smoothNsigma)) { - psError(PSPHOT_ERR_CONFIG, false, "Unable to find PEAKS_SMOOTH_NSIGMA in recipe"); - return false; - } - float thresh = psMetadataLookupF32(NULL, recipe, "PEAKS_NSIGMA_LIMIT_2"); - if (!isfinite(thresh)) { - psError(PSPHOT_ERR_CONFIG, false, "Unable to find PEAKS_NSIGMA_LIMIT_2 in recipe"); - return false; - } - psVector *magOffsets = psMetadataLookupVector(NULL, recipe, "EFF.MAG"); // Magnitude offsets - if (!magOffsets || magOffsets->type.type != PS_TYPE_F32) { - psError(PSPHOT_ERR_CONFIG, false, "Unable to find EFF.MAG F32 vector in recipe"); - return NULL; - } - int numSources = psMetadataLookupS32(NULL, recipe, "EFF.NUM"); // Number of sources for each bin - if (numSources == 0) { - psError(PSPHOT_ERR_CONFIG, false, "Unable to find EFF.NUM in recipe"); - return NULL; - } - float minGauss = psMetadataLookupF32(NULL, recipe, "PEAKS_MIN_GAUSS"); // Minimum valid fraction of kernel - if (!isfinite(minGauss)) { - psWarning("PEAKS_MIN_GAUSS is not set in recipe; using default value"); - minGauss = 0.5; + psError(PSPHOT_ERR_CONFIG, false, "Unable to find FWHM_MAJ and FWHM_MIN in readout->analysis"); + return false; } @@ -202,8 +237,6 @@ // remove all sources, adding noise for subtracted sources psphotRemoveAllSources(realSources, recipe); - // psphotAddNoise(readout, realSources, recipe); - - -#if defined(TESTING) && 0 + +#if TESTING { psRandom *rng = psRandomAlloc(PS_RANDOM_TAUS); @@ -234,5 +267,5 @@ } -#ifdef TESTING +#if TESTING psphotSaveImage(NULL, readout->image, "orig_image.fits"); psphotSaveImage(NULL, readout->variance, "orig_variance.fits"); @@ -249,5 +282,5 @@ } -#ifdef TESTING +#if TESTING psphotSaveImage(NULL, readout->image, "fake_image.fits"); psphotSaveImage(NULL, readout->variance, "fake_variance.fits"); @@ -364,5 +397,5 @@ psFree(significance); - if (!psphotFitSourcesLinear(readout, fakeSourcesAll, recipe, psf, true)) { + if (!psphotFitSourcesLinearReadout(recipe, readout, fakeSourcesAll, psf, true)) { psError(PS_ERR_UNKNOWN, false, "Unable to perform linear fit on fake sources."); psFree(fakeSources); @@ -371,19 +404,19 @@ } - // Disable aperture corrections; casting away const! + // Disable aperture corrections (save current value) pmTrend2D *apTrend = psf->ApTrend; // Aperture trend - ((pmPSF*)psf)->ApTrend = NULL; - - if (!psphotMagnitudes(config, readout, view, fakeSourcesAll, psf)) { + psf->ApTrend = NULL; + + if (!psphotMagnitudesReadout(config, recipe, view, readout, fakeSourcesAll, psf)) { psError(PS_ERR_UNKNOWN, false, "Unable to measure magnitudes of fake sources."); psFree(fakeSources); psFree(count); - ((pmPSF*)psf)->ApTrend = apTrend; // Casting away const! + psf->ApTrend = apTrend; // Casting away const! return false; } psFree(fakeSourcesAll); - // Re-enable aperture corrections; casting away const! - ((pmPSF*)psf)->ApTrend = apTrend; + // Replace aperture corrections + psf->ApTrend = apTrend; psVector *magDiffMean = psVectorAlloc(numBins, PS_TYPE_F32); // Mean difference in magnitude for each bin @@ -399,5 +432,5 @@ psVectorInit(magMask, 0); -#ifdef TESTING +#if TESTING psString name = NULL; psStringAppend(&name, "fake_%d.dat", i); @@ -415,5 +448,5 @@ } -#ifdef TESTING +#if TESTING fprintf(file, "%f %f %f %f %f %f %f\n", source->peak->xf, source->peak->yf, source->modelPSF->params->data.F32[PM_PAR_XPOS], @@ -451,5 +484,5 @@ } -#ifdef TESTING +#if TESTING fclose(file); #endif @@ -476,5 +509,4 @@ psLogMsg("psphot", PS_LOG_INFO, "Detection efficiency: %lf sec\n", psTimerClear("psphot.fake")); - return true; } Index: branches/tap_branches/psphot/src/psphotEllipticalContour.c =================================================================== --- branches/tap_branches/psphot/src/psphotEllipticalContour.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotEllipticalContour.c (revision 27838) @@ -7,5 +7,10 @@ bool psphotEllipticalContour (pmSource *source) { - pmSourceRadialProfile *profile = source->extpars->profile; + psAssert (source, "missing source"); + psAssert (source->extpars, "missing extpars"); + psAssert (source->extpars->radFlux, "missing radFlux"); + + pmSourceRadialFlux *profile = source->extpars->radFlux; + pmSourceExtendedPars *extpars = source->extpars; // use LMM to fit theta vs radius to an ellipse @@ -85,20 +90,20 @@ /// XXX rationalize? if epsilon > 1, flip major and minor axes (rotate by 90 degrees) if (params->data.F32[PAR_EPSILON] < 1.0) { - profile->axes.major = params->data.F32[PAR_RMIN] / params->data.F32[PAR_EPSILON]; - profile->axes.minor = params->data.F32[PAR_RMIN]; - profile->axes.theta = params->data.F32[PAR_PHI]; + extpars->axes.major = params->data.F32[PAR_RMIN] / params->data.F32[PAR_EPSILON]; + extpars->axes.minor = params->data.F32[PAR_RMIN]; + extpars->axes.theta = params->data.F32[PAR_PHI]; } else { - profile->axes.major = params->data.F32[PAR_RMIN]; - profile->axes.minor = params->data.F32[PAR_RMIN] / params->data.F32[PAR_EPSILON]; - profile->axes.theta = params->data.F32[PAR_PHI] + 0.5*M_PI; + extpars->axes.major = params->data.F32[PAR_RMIN]; + extpars->axes.minor = params->data.F32[PAR_RMIN] / params->data.F32[PAR_EPSILON]; + extpars->axes.theta = params->data.F32[PAR_PHI] + 0.5*M_PI; } psTrace ("psphot", 4, "# fitted values:\n"); - psTrace ("psphot", 4, "Phi: %f\n", profile->axes.theta*PS_DEG_RAD); - psTrace ("psphot", 4, "Rmaj: %f\n", profile->axes.major); - psTrace ("psphot", 4, "Rmin: %f\n", profile->axes.minor); + psTrace ("psphot", 4, "Phi: %f\n", extpars->axes.theta*PS_DEG_RAD); + psTrace ("psphot", 4, "Rmaj: %f\n", extpars->axes.major); + psTrace ("psphot", 4, "Rmin: %f\n", extpars->axes.minor); // show the results - // psphotPetrosianVisualEllipticalContour (petrosian); + // psphotPetrosianVisualEllipticalContour (profile, extpars); psFree (x); Index: branches/tap_branches/psphot/src/psphotEllipticalProfile.c =================================================================== --- branches/tap_branches/psphot/src/psphotEllipticalProfile.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotEllipticalProfile.c (revision 27838) @@ -1,7 +1,16 @@ # include "psphotInternal.h" -bool psphotEllipticalProfile (pmSource *source) { +bool psphotEllipticalProfile (pmSource *source, bool RAW_RADIUS) { - pmSourceRadialProfile *profile = source->extpars->profile; + psAssert (source, "missing source"); + psAssert (source->extpars, "missing extpars"); + psAssert (source->pixels, "missing pixels"); + + pmSourceExtendedPars *extpars = source->extpars; + + if (!source->extpars->ellipticalFlux) { + source->extpars->ellipticalFlux = pmSourceEllipticalFluxAlloc(); + } + pmSourceEllipticalFlux *profile = source->extpars->ellipticalFlux; profile->radiusElliptical = psVectorAllocEmpty(100, PS_TYPE_F32); @@ -21,11 +30,17 @@ psEllipseAxes axes; - axes.major = M_SQRT1_2; - axes.minor = M_SQRT1_2 * (profile->axes.minor / profile->axes.major); + if (RAW_RADIUS) { + // force circular profile + axes.major = M_SQRT1_2; + axes.minor = M_SQRT1_2; + } else { + axes.major = M_SQRT1_2; + axes.minor = M_SQRT1_2 * (extpars->axes.minor / extpars->axes.major); + } // axes.major = 1.0; - // axes.minor = profile->axes.minor / profile->axes.major; + // axes.minor = extpars->axes.minor / extpars->axes.major; - axes.theta = profile->axes.theta; + axes.theta = extpars->axes.theta; psEllipseShape shape = psEllipseAxesToShape (axes); @@ -46,9 +61,9 @@ float r2 = 0.5*PS_SQR(x/Sxx) + 0.5*PS_SQR(y/Syy) + x*y*Sxy; + float Rraw = hypot(x, y); psVectorAppend(radius, sqrt(r2)); psVectorAppend(flux, source->pixels->data.F32[iy][ix]); - float Rraw = hypot(x, y); psVectorAppend(radiusRaw, Rraw); psVectorAppend(fluxRaw, source->pixels->data.F32[iy][ix]); @@ -67,5 +82,5 @@ // } - // psphotPetrosianVisualProfileRadii (radius, flux, radiusRaw, fluxRaw, 0.0); + psphotPetrosianVisualProfileRadii (radius, flux, radiusRaw, fluxRaw, source->peak->flux, 0.0); // psphotPetrosianVisualProfileByAngle (radius, flux); Index: branches/tap_branches/psphot/src/psphotErrorCodes.dat =================================================================== --- branches/tap_branches/psphot/src/psphotErrorCodes.dat (revision 25900) +++ branches/tap_branches/psphot/src/psphotErrorCodes.dat (revision 27838) @@ -2,4 +2,5 @@ # This file is used to generate psphotErrorClasses.h # +BASE = 3000 First value we use UNKNOWN Unknown PM error code NOT_IMPLEMENTED Desired feature is not yet implemented Index: branches/tap_branches/psphot/src/psphotExtendedSourceAnalysis.c =================================================================== --- branches/tap_branches/psphot/src/psphotExtendedSourceAnalysis.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotExtendedSourceAnalysis.c (revision 27838) @@ -1,6 +1,34 @@ # include "psphotInternal.h" +// for now, let's store the detections on the readout->analysis for each readout +bool psphotExtendedSourceAnalysis (pmConfig *config, const pmFPAview *view) +{ + bool status = true; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + // perform full non-linear fits / extended source analysis? + if (!psMetadataLookupBool (&status, recipe, "EXTENDED_SOURCE_ANALYSIS")) { + psLogMsg ("psphot", PS_LOG_INFO, "skipping extended source measurements\n"); + return true; + } + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (!psphotExtendedSourceAnalysisReadout (config, view, "PSPHOT.INPUT", i, recipe)) { + psError (PSPHOT_ERR_CONFIG, false, "failed on measure extended source aperture-like parameters for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + // aperture-like measurements for extended sources -bool psphotExtendedSourceAnalysis (pmReadout *readout, psArray *sources, psMetadata *recipe) { +bool psphotExtendedSourceAnalysisReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe) { bool status; @@ -11,13 +39,27 @@ int Nkron = 0; + psTimerStart ("psphot.extended"); + + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *sources = detections->allSources; + psAssert (sources, "missing sources?"); + + if (!sources->n) { + psLogMsg ("psphot", PS_LOG_INFO, "no sources, skipping source size"); + return true; + } + // user-defined masks to test for good/bad pixels (build from recipe list if not yet set) psImageMaskType maskVal = psMetadataLookupImageMask(&status, recipe, "MASK.PSPHOT"); // Mask value for bad pixels assert (maskVal); - - // perform full non-linear fits / extended source analysis? - if (!psMetadataLookupBool (&status, recipe, "EXTENDED_SOURCE_ANALYSIS")) { - psLogMsg ("psphot", PS_LOG_INFO, "skipping extended source measurements\n"); - return true; - } // XXX require petrosian analysis for non-linear fits? @@ -25,11 +67,4 @@ // XXX temporary user-supplied systematic sky noise measurement (derive from background model) float skynoise = psMetadataLookupF32 (&status, recipe, "SKY.NOISE"); - -# if (0) - // if backModel or backStdev are missing, the values of sky and/or skyErr will be set to NAN - // XXX use this to set skynoise - pmReadout *backModel = psphotSelectBackground (config, view); - pmReadout *backStdev = psphotSelectBackgroundStdev (config, view); -# endif // S/N limit to perform full non-linear fits @@ -41,4 +76,11 @@ bool doAnnuli = psMetadataLookupBool (&status, recipe, "EXTENDED_SOURCE_ANNULI"); bool doKron = psMetadataLookupBool (&status, recipe, "EXTENDED_SOURCE_KRON"); + +# if (0) + // if backModel or backStdev are missing, the values of sky and/or skyErr will be set to NAN + // XXX use this to set skynoise + pmReadout *backModel = psphotSelectBackground (config, view); + pmReadout *backStdev = psphotSelectBackgroundStdev (config, view); +# endif // source analysis is done in S/N order (brightest first) @@ -79,4 +121,11 @@ Next ++; + // force source image to be a bit larger... + float radius = source->peak->xf - source->pixels->col0; + radius = PS_MAX (radius, source->peak->yf - source->pixels->row0); + radius = PS_MAX (radius, source->pixels->numRows - source->peak->yf + source->pixels->row0); + radius = PS_MAX (radius, source->pixels->numCols - source->peak->xf + source->pixels->col0); + pmSourceRedefinePixels (source, readout, source->peak->xf, source->peak->yf, 1.5*radius); + // if we request any of these measurements, we require the radial profile if (doPetrosian || doIsophotal || doAnnuli || doKron) { @@ -94,5 +143,5 @@ if (doPetrosian) { if (!psphotPetrosian (source, recipe, skynoise, maskVal)) { - psTrace ("psphot", 5, "measured petrosian flux & radius for source at %7.1f, %7.1f", source->moments->Mx, source->moments->My); + psTrace ("psphot", 5, "FAILED petrosian flux & radius for source at %7.1f, %7.1f", source->moments->Mx, source->moments->My); } else { psTrace ("psphot", 5, "measured petrosian flux & radius for source at %7.1f, %7.1f", source->moments->Mx, source->moments->My); @@ -129,9 +178,10 @@ if (source->extpars) { - pmSourceRadialProfileFreeVectors(source->extpars->profile); + psFree(source->extpars->radFlux); + psFree(source->extpars->ellipticalFlux); } } - psLogMsg ("psphot", PS_LOG_INFO, "extended source analysis: %f sec for %d objects\n", psTimerMark ("psphot"), Next); + psLogMsg ("psphot", PS_LOG_INFO, "extended source analysis: %f sec for %d objects\n", psTimerMark ("psphot.extended"), Next); psLogMsg ("psphot", PS_LOG_INFO, " %d petrosian\n", Npetro); psLogMsg ("psphot", PS_LOG_INFO, " %d isophotal\n", Nisophot); Index: branches/tap_branches/psphot/src/psphotExtendedSourceFits.c =================================================================== --- branches/tap_branches/psphot/src/psphotExtendedSourceFits.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotExtendedSourceFits.c (revision 27838) @@ -1,6 +1,34 @@ # include "psphotInternal.h" +// for now, let's store the detections on the readout->analysis for each readout +bool psphotExtendedSourceFits (pmConfig *config, const pmFPAview *view) +{ + bool status = true; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + // perform full extended source non-linear fits? + if (!psMetadataLookupBool (&status, recipe, "EXTENDED_SOURCE_FITS")) { + psLogMsg ("psphot", PS_LOG_INFO, "skipping extended source measurements\n"); + return true; + } + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (!psphotExtendedSourceFitsReadout (config, view, "PSPHOT.INPUT", i, recipe)) { + psError (PSPHOT_ERR_CONFIG, false, "failed on to fit extended sources for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + // non-linear model fitting for extended sources -bool psphotExtendedSourceFits (pmReadout *readout, psArray *sources, psMetadata *recipe) { +bool psphotExtendedSourceFitsReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe) { bool status; @@ -12,4 +40,22 @@ bool savePics = false; + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *sources = detections->allSources; + psAssert (sources, "missing sources?"); + + if (!sources->n) { + psLogMsg ("psphot", PS_LOG_INFO, "no sources, skipping source size"); + return true; + } + // user-defined masks to test for good/bad pixels (build from recipe list if not yet set) psImageMaskType maskVal = psMetadataLookupImageMask(&status, recipe, "MASK.PSPHOT"); // Mask value for bad pixels @@ -21,10 +67,4 @@ // maskVal is used to test for rejected pixels, and must include markVal maskVal |= markVal; - - // perform full extended source non-linear fits? - if (!psMetadataLookupBool (&status, recipe, "EXTENDED_SOURCE_FITS")) { - psLogMsg ("psphot", PS_LOG_INFO, "skipping extended source measurements\n"); - return true; - } // select the collection of desired models Index: branches/tap_branches/psphot/src/psphotFake.c =================================================================== --- branches/tap_branches/psphot/src/psphotFake.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotFake.c (revision 27838) @@ -28,8 +28,6 @@ float smoothSigma = 0.5*(FWHM_X + FWHM_Y) / (2.0*sqrtf(2.0*log(2.0))); psKernel *kernel = psImageSmoothKernel(smoothSigma, smoothNsigma); // Kernel used for smoothing - psKernel *newCovar = psImageCovarianceCalculate(kernel, readout->covariance); // Covariance matrix + float factor = psImageCovarianceCalculateFactor(kernel, readout->covariance); // Covariance matrix psFree(kernel); - float factor = psImageCovarianceFactor(newCovar); // Variance factor - psFree(newCovar); psStats *stats = psStatsAlloc(PS_STAT_ROBUST_MEDIAN); // Statistics for variance @@ -213,10 +211,7 @@ // Putting results on recipe because that appears to be the psphot standard, but it's not a good idea - psMetadataAddVector(recipe, PS_LIST_TAIL, "FAKE.EFF", PS_META_REPLACE, - "Efficiency fractions", frac); - psMetadataAddVector(recipe, PS_LIST_TAIL, "FAKE.MAG", PS_META_REPLACE, - "Efficiency magnitudes", magOffsets); - psMetadataAddF32(recipe, PS_LIST_TAIL, "FAKE.REF", PS_META_REPLACE, - "Efficiency reference magnitude", magLim); + psMetadataAddVector(readout->analysis, PS_LIST_TAIL, "FAKE.EFF", PS_META_REPLACE, "Efficiency fractions", frac); + psMetadataAddVector(readout->analysis, PS_LIST_TAIL, "FAKE.MAG", PS_META_REPLACE, "Efficiency magnitudes", magOffsets); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "FAKE.REF", PS_META_REPLACE, "Efficiency reference magnitude", magLim); psFree(frac); Index: branches/tap_branches/psphot/src/psphotFindDetections.c =================================================================== --- branches/tap_branches/psphot/src/psphotFindDetections.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotFindDetections.c (revision 27838) @@ -1,6 +1,30 @@ # include "psphotInternal.h" +// we store the detections on the readout->analysis for each readout this function finds new +// peaks and new footprints. any old peaks are saved on oldPeaks. the resulting footprint set +// contains all footprints (old and new) +bool psphotFindDetections (pmConfig *config, const pmFPAview *view, bool firstPass) +{ + bool status = true; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (!psphotFindDetectionsReadout (config, view, "PSPHOT.INPUT", i, recipe, firstPass)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to find initial detections for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + // smooth the image, search for peaks, optionally define footprints based on the peaks -pmDetections *psphotFindDetections (pmDetections *detections, pmReadout *readout, psMetadata *recipe) { +bool psphotFindDetectionsReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe, bool firstPass) { bool status; @@ -9,14 +33,34 @@ int NMAX = 0; + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + // user-defined masks to test for good/bad pixels (build from recipe list if not yet set) psImageMaskType maskVal = psMetadataLookupImageMask(&status, recipe, "MASK.PSPHOT"); // Mask value for bad pixels - assert (maskVal); + psAssert (maskVal, "missing mask value?"); // Use the new pmFootprints approach? const bool useFootprints = psMetadataLookupBool(NULL, recipe, "USE_FOOTPRINTS"); - // on first pass, detections have not yet been allocated + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + // on the initial pass, detections have not yet been allocated or saved on readout->analysis if (!detections) { + // create the container detections = pmDetectionsAlloc(); + + // save detections on the readout->analysis + if (!psMetadataAddPtr (readout->analysis, PS_LIST_TAIL, "PSPHOT.DETECTIONS", PS_META_REPLACE | PS_DATA_UNKNOWN, "psphot detections", detections)) { + psError (PSPHOT_ERR_CONFIG, false, "problem saving detections on readout"); + return false; + } + } else { + psMemIncrRefCounter(detections); // so we can free the detections below + } + + if (firstPass) { pass = 1; NSIGMA_PEAK = psMetadataLookupF32 (&status, recipe, "PEAKS_NSIGMA_LIMIT"); PS_ASSERT (status, NULL); @@ -30,6 +74,10 @@ float threshold = PS_SQR(NSIGMA_PEAK); - // move the old peaks array (if it exists) to oldPeaks - // XXX generically, we should be able to call this function an arbitrary number of times + // move the old peaks array (if it exists) to oldPeaks XXX generically, we should be able + // to call this function an arbitrary number of times the old peaks are saved so they can + // be freed later -- the have to be freed after psphotFindFootprints is called below, since + // they are also owned by the oldFootprints, which are in turn merged into the new + // footprints. (what about the source->peak entry?) + assert (detections->oldPeaks == NULL); detections->oldPeaks = detections->peaks; @@ -39,18 +87,28 @@ psImage *significance = psphotSignificanceImage (readout, recipe, pass, maskVal); - // display the backsub and backgnd images - psphotVisualShowSignificance (significance); + // display the significance image + psphotVisualShowSignificance (significance, -1.0, PS_SQR(3.0*NSIGMA_PEAK)); + + // XXX getting some strange results from significance image + if (0) { + psImage *lsig = (psImage *) psUnaryOp (NULL, significance, "log"); + psphotVisualShowSignificance (lsig, 0.0, 4.0); + psFree (lsig); + } // detect the peaks in the significance image detections->peaks = psphotFindPeaks (significance, readout, recipe, threshold, NMAX); - psMetadataAddF32 (recipe, PS_LIST_TAIL, "PEAK_THRESHOLD", PS_META_REPLACE, "Peak Detection Threshold", threshold); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "PEAK_THRESHOLD", PS_META_REPLACE, "Peak Detection Threshold", threshold); if (!detections->peaks) { - // No peaks found - return NULL; + // we only get a NULL peaks array due to a programming or config error. + // this will result in a failure. + psFree (detections); + psError (PSPHOT_ERR_CONFIG, false, "failed on peak search"); + return false; } // optionally merge peaks into footprints if (useFootprints) { - psphotFindFootprints (detections, significance, readout, recipe, threshold, pass, maskVal); + psphotFindFootprints (detections, significance, readout, recipe, threshold, pass, maskVal); } @@ -61,5 +119,7 @@ psphotVisualShowFootprints (detections); - return detections; + psFree (detections); + + return true; } Index: branches/tap_branches/psphot/src/psphotFindFootprints.c =================================================================== --- branches/tap_branches/psphot/src/psphotFindFootprints.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotFindFootprints.c (revision 27838) @@ -41,7 +41,5 @@ if (pass == 2) { // merge in old peaks; - const int includePeaks = 0x0 | 0x2; // i.e. just from newFootprints - - // XXX EAM : still not sure I understand this: are we double-couning or undercounting peaks + const int includePeaks = 0x1 | 0x2; // i.e. both new and old footprints psArray *mergedFootprints = pmFootprintArraysMerge(detections->footprints, footprints, includePeaks); @@ -50,4 +48,6 @@ psFree(footprints); psFree(detections->footprints); + + // replace the merged footprints on the detection structure detections->footprints = mergedFootprints; } else { @@ -55,5 +55,5 @@ } - psphotCullPeaks(readout->image, readout->variance, recipe, detections->footprints); + psphotCullPeaks(readout, recipe, detections->footprints); detections->peaks = pmFootprintArrayToPeaks(detections->footprints); psLogMsg ("psphot", PS_LOG_INFO, "%ld peaks, %ld total footprints: %f sec\n", detections->peaks->n, detections->footprints->n, psTimerMark ("psphot.footprints")); @@ -61,2 +61,17 @@ return detections; } + +bool psphotCheckFootprints (pmDetections *detections) { + + // check for messed up footprints in the old peaks + for (int i = 0; i < detections->oldPeaks->n; i++) { + pmPeak *peak = detections->oldPeaks->data[i]; + pmFootprint *footprint = peak->footprint; + if (!footprint) continue; + for (int j = 0; j < footprint->spans->n; j++) { + pmSpan *sp = footprint->spans->data[j]; + psAssert (sp, "missing span"); + } + } + return true; +} Index: branches/tap_branches/psphot/src/psphotFindPeaks.c =================================================================== --- branches/tap_branches/psphot/src/psphotFindPeaks.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotFindPeaks.c (revision 27838) @@ -13,10 +13,15 @@ psArray *peaks = pmPeaksInImage (significance, threshold); if (peaks == NULL) { - // XXX should we be sending back an empty array instead of NULL? - // XXX this may also be due to a programming or config error + // we only get a NULL peaks array due to a programming or config error. + // this will result in a failure. + psError(PSPHOT_ERR_DATA, false, "no peaks found in this image"); + return NULL; + } + + if (peaks->n == 0) { // XXX do we need to set something in the readout->analysis to indicate that // we tried and failed to find peaks (something in the header data) - psError(PSPHOT_ERR_DATA, false, "no peaks found in this image"); - return false; + psLogMsg ("psphot", PS_LOG_INFO, "no peaks found in this image"); + return peaks; } Index: branches/tap_branches/psphot/src/psphotFitSourcesLinear.c =================================================================== --- branches/tap_branches/psphot/src/psphotFitSourcesLinear.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotFitSourcesLinear.c (revision 27838) @@ -12,5 +12,44 @@ static bool SetBorderMatrixElements (psSparseBorder *border, pmReadout *readout, psArray *sources, bool constant_weights, int SKY_FIT_ORDER, psImageMaskType markVal); -bool psphotFitSourcesLinear (pmReadout *readout, psArray *sources, const psMetadata *recipe, const pmPSF *psf, bool final) { +// for now, let's store the detections on the readout->analysis for each readout +bool psphotFitSourcesLinear (pmConfig *config, const pmFPAview *view, bool final) +{ + bool status = true; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + assert (recipe); + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, "PSPHOT.INPUT", i); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *sources = detections->allSources; + psAssert (sources, "missing sources?"); + + pmPSF *psf = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.PSF"); + psAssert (psf, "missing psf?"); + + if (!psphotFitSourcesLinearReadout (recipe, readout, sources, psf, final)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to fit sources (linear) for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + +bool psphotFitSourcesLinearReadout (psMetadata *recipe, pmReadout *readout, psArray *sources, pmPSF *psf, bool final) { bool status; @@ -18,5 +57,9 @@ float y; float f; - // float r; + + if (!sources->n) { + psLogMsg ("psphot", PS_LOG_INFO, "no sources, skipping linear fit"); + return true; + } psTimerStart ("psphot.linear"); @@ -59,4 +102,11 @@ } + // XXX test: choose a larger-than expected radius: + float covarFactor = psImageCovarianceFactorForAperture(readout->covariance, 10.0); // Covariance matrix + psLogMsg ("psphot.ensemble", PS_LOG_MINUTIA, "covariance factor: %f\n", covarFactor); + + // XXX do not apply covarFactor for the moment... + // covarFactor = 1.0; + // select the sources which will be used for the fitting analysis for (int i = 0; i < sources->n; i++) { @@ -120,10 +170,10 @@ // diagonal elements of the sparse matrix (auto-cross-product) - f = pmSourceModelDotModel (SRCi, SRCi, CONSTANT_PHOTOMETRIC_WEIGHTS); + f = pmSourceModelDotModel (SRCi, SRCi, CONSTANT_PHOTOMETRIC_WEIGHTS, covarFactor); psSparseMatrixElement (sparse, i, i, f); // the formal error depends on the weighting scheme if (CONSTANT_PHOTOMETRIC_WEIGHTS) { - float var = pmSourceModelDotModel (SRCi, SRCi, false); + float var = pmSourceModelDotModel (SRCi, SRCi, false, covarFactor); errors->data.F32[i] = 1.0 / sqrt(var); } else { @@ -133,5 +183,5 @@ // find the image x model value - f = pmSourceDataDotModel (SRCi, SRCi, CONSTANT_PHOTOMETRIC_WEIGHTS); + f = pmSourceDataDotModel (SRCi, SRCi, CONSTANT_PHOTOMETRIC_WEIGHTS, covarFactor); psSparseVectorElement (sparse, i, f); @@ -139,11 +189,11 @@ switch (SKY_FIT_ORDER) { case 1: - f = pmSourceModelWeight (SRCi, 1, CONSTANT_PHOTOMETRIC_WEIGHTS); + f = pmSourceModelWeight (SRCi, 1, CONSTANT_PHOTOMETRIC_WEIGHTS, covarFactor); psSparseBorderElementB (border, i, 1, f); - f = pmSourceModelWeight (SRCi, 2, CONSTANT_PHOTOMETRIC_WEIGHTS); + f = pmSourceModelWeight (SRCi, 2, CONSTANT_PHOTOMETRIC_WEIGHTS, covarFactor); psSparseBorderElementB (border, i, 2, f); case 0: - f = pmSourceModelWeight (SRCi, 0, CONSTANT_PHOTOMETRIC_WEIGHTS); + f = pmSourceModelWeight (SRCi, 0, CONSTANT_PHOTOMETRIC_WEIGHTS, covarFactor); psSparseBorderElementB (border, i, 0, f); break; @@ -165,5 +215,5 @@ // got an overlap; calculate cross-product and add to output array - f = pmSourceModelDotModel (SRCi, SRCj, CONSTANT_PHOTOMETRIC_WEIGHTS); + f = pmSourceModelDotModel (SRCi, SRCj, CONSTANT_PHOTOMETRIC_WEIGHTS, covarFactor); psSparseMatrixElement (sparse, j, i, f); } @@ -219,9 +269,10 @@ // measure chisq for each source - for (int i = 0; final && (i < fitSources->n); i++) { + // for (int i = 0; final && (i < fitSources->n); i++) { + for (int i = 0; i < fitSources->n; i++) { pmSource *source = fitSources->data[i]; if (source->mode & PM_SOURCE_MODE_NONLINEAR_FIT) continue; pmModel *model = pmSourceGetModel (NULL, source); - pmSourceChisq (model, source->pixels, source->maskObj, source->variance, maskVal); + pmSourceChisq (model, source->pixels, source->maskObj, source->variance, maskVal, covarFactor); } psLogMsg ("psphot.ensemble", PS_LOG_MINUTIA, "get chisqs: %f sec (%d elements)\n", psTimerMark ("psphot.linear"), sparse->Nelem); @@ -238,5 +289,10 @@ psphotVisualShowResidualImage (readout); + psphotVisualPlotChisq (sources); // psphotVisualShowFlags (sources); + + // We have to place this visualization here because the models are not realized until + // psphotGuessModels or fitted until psphotFitSourcesLinear. + psphotVisualShowPSFStars (recipe, psf, sources); return true; @@ -244,4 +300,5 @@ // XXX do we need this? +// XXX disallow the simultaneous sky fit and remove this code... // Calculate the weight terms for the sky fit component of the matrix. This function operates Index: branches/tap_branches/psphot/src/psphotFitSourcesLinearStack.c =================================================================== --- branches/tap_branches/psphot/src/psphotFitSourcesLinearStack.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotFitSourcesLinearStack.c (revision 27838) @@ -1,22 +1,16 @@ # include "psphotInternal.h" +// XXX need array of covar factors for each image +// XXX define the 'good' / 'bad' flags? -// fit flux (and optionally sky model) to all reasonable sources -// with the linear fitting process. sources must have an associated -// model with selected pixels, and the fit radius must be defined +# define COVAR_FACTOR 1.0 -// given the set of sources, each of which points to the pixels in the -// science image, we construct a set of simulated sources with their own pixels. -// these are used to determine the simultaneous linear fit of fluxes. -// the analysis is performed wrt the simulated pixel values - -static bool SetBorderMatrixElements (psSparseBorder *border, pmReadout *readout, psArray *sources, bool constant_weights, psImageMaskType markVal); - -bool psphotFitSourcesLinear (pmReadout *readout, psArray *sources, psMetadata *recipe, pmPSF *psf, bool final) { +bool psphotFitSourcesLinearStack (pmConfig *config, psArray *objects, bool final) { bool status; - float x; - float y; float f; - // float r; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + assert (recipe); psTimerStart ("psphot.linear"); @@ -33,45 +27,44 @@ maskVal |= markVal; - // source analysis is done in spatial order - sources = psArraySort (sources, pmSourceSortByY); + // analysis is done in spatial order (to speed up overlap search) + // sort by first element in each source list + objects = psArraySort (objects, pmPhotObjSortByX); // storage array for fitSources - psArray *fitSources = psArrayAllocEmpty (sources->n); + psArray *fitSources = psArrayAllocEmpty (objects->n); - bool CONSTANT_PHOTOMETRIC_WEIGHTS = - psMetadataLookupBool(&status, recipe, "CONSTANT_PHOTOMETRIC_WEIGHTS"); - if (!status) { - psAbort("You must provide a value for the BOOL recipe CONSTANT_PHOTOMETRIC_WEIGHTS"); - } + bool CONSTANT_PHOTOMETRIC_WEIGHTS = psMetadataLookupBool(&status, recipe, "CONSTANT_PHOTOMETRIC_WEIGHTS"); + psAssert (status, "You must provide a value for the BOOL recipe CONSTANT_PHOTOMETRIC_WEIGHTS"); + + // XXX store a local static array of covar factors for each of the images (by image ID) + // float covarFactor = psImageCovarianceFactorForAperture(readout->covariance, 10.0); // Covariance matrix + // psLogMsg ("psphot.ensemble", PS_LOG_MINUTIA, "covariance factor: %f\n", covarFactor); // select the sources which will be used for the fitting analysis - for (int i = 0; i < sources->n; i++) { - pmSource *source = sources->data[i]; + for (int i = 0; i < objects->n; i++) { + pmPhotObj *object = objects->data[i]; + if (!object) continue; + if (!object->sources) continue; - // turn this bit off and turn it on again if we pass this test - source->mode &= ~PM_SOURCE_MODE_LINEAR_FIT; + // XXX check an element of the group to see if we should use it + // if (!object->flags & PM_PHOT_OBJ_BAD) continue; - // skip non-astronomical objects (very likely defects) - if (source->type == PM_SOURCE_TYPE_DEFECT) continue; - if (source->type == PM_SOURCE_TYPE_SATURATED) continue; + for (int j = 0; j < object->sources->n; j++) { + pmSource *source = object->sources->data[j]; + if (!source) continue; - // do not include CRs in the full ensemble fit - if (source->mode & PM_SOURCE_MODE_CR_LIMIT) continue; + // turn this bit off and turn it on again if we keep this source + source->mode &= ~PM_SOURCE_MODE_LINEAR_FIT; - if (final) { - if (source->tmpFlags & PM_SOURCE_TMPF_SUBTRACTED) continue; - } else { - if (source->mode & PM_SOURCE_MODE_BLEND) continue; - } + // generate model for sources without, or skip if we can't + if (!source->modelFlux) { + if (!pmSourceCacheModel (source, maskVal)) continue; + } - // generate model for sources without, or skip if we can't - if (!source->modelFlux) { - if (!pmSourceCacheModel (source, maskVal)) continue; - } - - source->mode |= PM_SOURCE_MODE_LINEAR_FIT; - psArrayAdd (fitSources, 100, source); + source->mode |= PM_SOURCE_MODE_LINEAR_FIT; + psArrayAdd (fitSources, 100, source); + } } - psLogMsg ("psphot.ensemble", PS_LOG_MINUTIA, "built fitSources: %f sec (%ld objects)\n", psTimerMark ("psphot.linear"), sources->n); + psLogMsg ("psphot.ensemble", PS_LOG_MINUTIA, "built fitSources: %f sec (%ld objects)\n", psTimerMark ("psphot.linear"), objects->n); if (fitSources->n == 0) { @@ -83,8 +76,6 @@ psVector *errors = psVectorAlloc (fitSources->n, PS_TYPE_F32); - // create the border matrix (includes the sparse matrix) - // for just sky: 1 row; for x,y terms: 3 rows + // create the sparse matrix psSparse *sparse = psSparseAlloc (fitSources->n, 100); - psSparseBorder *border = psSparseBorderAlloc (sparse, 1); // fill out the sparse matrix elements and border elements (B) @@ -95,10 +86,10 @@ // diagonal elements of the sparse matrix (auto-cross-product) - f = pmSourceModelDotModel (SRCi, SRCi, CONSTANT_PHOTOMETRIC_WEIGHTS); + f = pmSourceModelDotModel (SRCi, SRCi, CONSTANT_PHOTOMETRIC_WEIGHTS, COVAR_FACTOR); psSparseMatrixElement (sparse, i, i, f); // the formal error depends on the weighting scheme if (CONSTANT_PHOTOMETRIC_WEIGHTS) { - float var = pmSourceModelDotModel (SRCi, SRCi, false); + float var = pmSourceModelDotModel (SRCi, SRCi, false, COVAR_FACTOR); errors->data.F32[i] = 1.0 / sqrt(var); } else { @@ -106,11 +97,7 @@ } - // find the image x model value - f = pmSourceDataDotModel (SRCi, SRCi, CONSTANT_PHOTOMETRIC_WEIGHTS); + f = pmSourceDataDotModel (SRCi, SRCi, CONSTANT_PHOTOMETRIC_WEIGHTS, COVAR_FACTOR); psSparseVectorElement (sparse, i, f); - - f = pmSourceModelWeight (SRCi, 0, CONSTANT_PHOTOMETRIC_WEIGHTS); - psSparseBorderElementB (border, i, 0, f); // loop over all other stars following this one @@ -118,12 +105,15 @@ pmSource *SRCj = fitSources->data[j]; + // we only need to generate dot terms for source on the same image + if (SRCj->imageID != SRCi->imageID) { continue; } + // skip over disjoint source images, break after last possible overlap - if (SRCi->pixels->row0 + SRCi->pixels->numRows < SRCj->pixels->row0) break; - if (SRCj->pixels->row0 + SRCj->pixels->numRows < SRCi->pixels->row0) continue; - if (SRCi->pixels->col0 + SRCi->pixels->numCols < SRCj->pixels->col0) continue; - if (SRCj->pixels->col0 + SRCj->pixels->numCols < SRCi->pixels->col0) continue; + if (SRCj->pixels->row0 + SRCj->pixels->numRows < SRCi->pixels->row0) continue; // source(i) is above source(j) + if (SRCi->pixels->row0 + SRCi->pixels->numRows < SRCj->pixels->row0) continue; // source(i) is below source(j) + if (SRCj->pixels->col0 + SRCj->pixels->numCols < SRCi->pixels->col0) continue; // source(i) is right of source(j) + if (SRCi->pixels->col0 + SRCi->pixels->numCols < SRCj->pixels->col0) break; // source(i) is left of source(j) [no other source(j) can overlap source(i)] // got an overlap; calculate cross-product and add to output array - f = pmSourceModelDotModel (SRCi, SRCj, CONSTANT_PHOTOMETRIC_WEIGHTS); + f = pmSourceModelDotModel (SRCi, SRCj, CONSTANT_PHOTOMETRIC_WEIGHTS, COVAR_FACTOR); psSparseMatrixElement (sparse, j, i, f); } @@ -133,7 +123,4 @@ psLogMsg ("psphot.ensemble", PS_LOG_MINUTIA, "built matrix: %f sec (%d elements)\n", psTimerMark ("psphot.linear"), sparse->Nelem); - // set the sky, sky_x, sky_y components of border matrix - SetBorderMatrixElements (border, readout, fitSources, CONSTANT_PHOTOMETRIC_WEIGHTS, SKY_FIT_ORDER, markVal); - psSparseConstraint constraint; constraint.paramMin = 0.0; @@ -142,13 +129,7 @@ // solve for normalization terms (need include local sky?) - psVector *norm = psSparseSolve (NULL, constraint, sparse, 5); - + psVector *norm = NULL; + norm = psSparseSolve (NULL, constraint, sparse, 5); psLogMsg ("psphot.ensemble", PS_LOG_MINUTIA, "solve matrix: %f sec (%d elements)\n", psTimerMark ("psphot.linear"), sparse->Nelem); - - // XXXX **** philosophical question: - // we measure bright objects in three passes: 1) linear fit; 2) non-linear fit; 3) linear fit: - // should retain the chisq and errors from the intermediate non-linear fit? - // the non-linear fit provides better values for the position errors, and for - // extended sources, the shape errors // adjust I0 for fitSources and subtract @@ -164,5 +145,4 @@ model->params->data.F32[PM_PAR_I0] = norm->data.F32[i]; model->dparams->data.F32[PM_PAR_I0] = errors->data.F32[i]; - // XXX is the value of 'errors' modified by the sky fit? // subtract object @@ -172,9 +152,10 @@ // measure chisq for each source - for (int i = 0; final && (i < fitSources->n); i++) { + // for (int i = 0; final && (i < fitSources->n); i++) { + for (int i = 0; i < fitSources->n; i++) { pmSource *source = fitSources->data[i]; if (source->mode & PM_SOURCE_MODE_NONLINEAR_FIT) continue; pmModel *model = pmSourceGetModel (NULL, source); - pmSourceChisq (model, source->pixels, source->maskObj, source->variance, maskVal); + pmSourceChisq (model, source->pixels, source->maskObj, source->variance, maskVal, COVAR_FACTOR); } psLogMsg ("psphot.ensemble", PS_LOG_MINUTIA, "get chisqs: %f sec (%d elements)\n", psTimerMark ("psphot.linear"), sparse->Nelem); @@ -185,67 +166,22 @@ psFree (norm); psFree (errors); - psFree (border); psLogMsg ("psphot.ensemble", PS_LOG_INFO, "measure ensemble of PSFs: %f sec\n", psTimerMark ("psphot.linear")); - - psphotVisualShowResidualImage (readout); - psphotVisualShowFlags (sources); return true; } -// Calculate the weight terms for the sky fit component of the matrix. This function operates -// on the pixels which correspond to all of the sources of interest. These elements fill in -// the border matrix components in the sparse matrix equation. -static bool SetBorderMatrixElements (psSparseBorder *border, pmReadout *readout, psArray *sources, bool constant_weights, psImageMaskType markVal) { +// sort by X (ascending) +int pmPhotObjSortByX (const void **a, const void **b) +{ + pmPhotObj *objA = *(pmPhotObj **)a; + pmPhotObj *objB = *(pmPhotObj **)b; - // generate the image-wide weight terms - // turn on MARK for all image pixels - psRegion fullArray = psRegionSet (0, 0, 0, 0); - fullArray = psRegionForImage (readout->mask, fullArray); - psImageMaskRegion (readout->mask, fullArray, "OR", markVal); + psF32 fA = objA->x; + psF32 fB = objB->x; - // turn off MARK for all object pixels - for (int i = 0; i < sources->n; i++) { - pmSource *source = sources->data[i]; - pmModel *model = pmSourceGetModel (NULL, source); - if (model == NULL) continue; - float x = model->params->data.F32[PM_PAR_XPOS]; - float y = model->params->data.F32[PM_PAR_YPOS]; - psImageMaskCircle (source->maskView, x, y, model->radiusFit, "AND", PS_NOT_IMAGE_MASK(markVal)); - } - - // accumulate the image statistics from the masked regions - psF32 **image = readout->image->data.F32; - psF32 **variance = readout->variance->data.F32; - psImageMaskType **mask = readout->mask->data.PS_TYPE_IMAGE_MASK_DATA; - - double w, x, y, x2, xy, y2, xc, yc, wt, f, fo, fx, fy; - w = x = y = x2 = xy = y2 = fo = fx = fy = 0; - - int col0 = readout->image->col0; - int row0 = readout->image->row0; - - for (int j = 0; j < readout->image->numRows; j++) { - for (int i = 0; i < readout->image->numCols; i++) { - if (mask[j][i]) continue; - if (constant_weights) { - wt = 1.0; - } else { - wt = variance[j][i]; - } - f = image[j][i]; - w += 1/wt; - fo += f/wt; - } - } - - // turn off MARK for all image pixels - psImageMaskRegion (readout->mask, fullArray, "AND", PS_NOT_IMAGE_MASK(markVal)); - - // set the Border T elements - psSparseBorderElementG (border, 0, fo); - psSparseBorderElementT (border, 0, 0, w); - - return true; + psF32 diff = fA - fB; + if (diff > FLT_EPSILON) return (+1); + if (diff < FLT_EPSILON) return (-1); + return (0); } Index: branches/tap_branches/psphot/src/psphotForced.c =================================================================== --- branches/tap_branches/psphot/src/psphotForced.c (revision 27838) +++ branches/tap_branches/psphot/src/psphotForced.c (revision 27838) @@ -0,0 +1,35 @@ +# include "psphotStandAlone.h" + +int main (int argc, char **argv) { + + psTimerStart ("complete"); + pmErrorRegister(); // register psModule's error codes/messages + psphotInit(); + + // load command-line arguments, options, and system config data + pmConfig *config = psphotForcedArguments (argc, argv); + assert(config); + + psphotVersionPrint(); + + // load input data (config and images (signal, noise, mask) + if (!psphotParseCamera (config)) { + psErrorStackPrint(stderr, "Error setting up the camera\n"); + exit (psphotGetExitStatus()); + } + + // call psphot for each readout + if (!psphotForcedImageLoop (config)) { + psErrorStackPrint(stderr, "Error in the psphot image loop\n"); + exit (psphotGetExitStatus()); + } + + psLogMsg ("psphot", 3, "complete psphot run: %f sec\n", psTimerMark ("complete")); + + psErrorCode exit_status = psphotGetExitStatus(); + psphotCleanup (config); + exit (exit_status); +} + +// all functions which return to this level must raise one of the top-level error codes if they +// exit with an error. these error codes are used to specify the program exit status Index: branches/tap_branches/psphot/src/psphotForcedArguments.c =================================================================== --- branches/tap_branches/psphot/src/psphotForcedArguments.c (revision 27838) +++ branches/tap_branches/psphot/src/psphotForcedArguments.c (revision 27838) @@ -0,0 +1,199 @@ +# include "psphotStandAlone.h" + +static void writeHelpInfo(const char* program, pmConfig* config, FILE* ofile) +{ + fprintf(ofile, + "Usage: one of the following\n" + "%s -file fname1[,fname2,...] -mask maskfile1[,maskfile2,...]\n" + " -variance varfile1[,varfile2,...] OutFileBaseName\n" + "\n" + "%s -list FileNameList [-masklist MaskFileNameList] \n" + " -variancelist VarFileNameList OutFileBaseName\n" + "\n" + "%s -help\n" + "\n" + "%s -version\n" + "\n" + "where:\n" + " FileNameList is a text file containing filenames, one per line\n" + " MaskFileNameList is a text file of mask filenames, one per line\n" + " VarFileNameList is a text file of variance filenames, one per line\n" + " OutFileBaseName is the 'root name' for output files\n" + "\n" + "additional options:\n" + " -psf PsfFile1[,PsfFile2,...] or -psflist PsfFileNameList\n" + " specify PSF rather than letting %s estimate it\n" + " -src SrcFile1[,SrcFile2,...] or -srclist SrcFileNameList\n" + " specify additional sources for PSF generation\n" + " -chip nn[,nn,...]\n" + " select detector chips to process; default is all.\n" + " Indices correspond to zero-based offset in the FPA metadata table.\n" + " -photcode PhotoCodeName\n" + " specify photocode\n" + " -region RegionString\n" + " specify analysis region. String is of form '[x0:x1,y0:y1]'\n" + " To use this option you must define a default in psphot.config\n" + " -visual\n" + " turns on interactive display mode\n" + " -dumpconfig CfgFileName\n" + " causes config info to be dumped to the named file.\n" + " -break NOTHING|BACKMDL|PEAKS|MOMENTS|PSFMODEL|ENSEMBLE|PASS1\n" + " choose a point at which to exit processing early\n" + " -nthreads n\n" + " set number of parallel threads of execution\n" + " -F OldFileRule ReplacementFileRule\n" + " change file naming rule; e.g. '-F PSPHOT.OUTPUT PSPHOT.OUT.CMF.MEF'\n" + " -D name stringval\n" + " set a string-valued config parameter\n" + " -Di name intval\n" + " set an integer-valued config parameter\n" + " -Df name fval\n" + " set a float-valued config parameter\n" + " -Db name boolval\n" + " set a boolean-valued config parameter\n" + " -v, -vv, -vvv\n" + " set increasing levels of verbosity\n" + " -logfmt FormatString\n" + " set format string used for log messages\n" + " -trace Fac Lvl\n" + " set tracing for facility Fac to integer Lvl, e.g. '-trace err 10'\n" + " -trace-levels\n" + " print current trace levels\n", + program,program,program,program,program); + psFree(config); + pmConfigDone(); + psLibFinalize(); + exit(PS_EXIT_SUCCESS); +} + +static void usage(const char *program, // Name of the program + psMetadata *arguments, // Command-line arguments + pmConfig *config, // Configuration + int exitCode + ) +{ + fprintf(stderr, + "Usage: one of the following\n" + "%s -file fname1[,fname2,...] -mask maskfile1[,maskfile2,...]\n" + " -variance varfile1[,varfile2,...] OutFileBaseName\n" + "\n" + "%s -list FileNameList [-masklist MaskFileNameList] \n" + " -variancelist VarFileNameList OutFileBaseName\n" + "\n" + "Try '%s -help' for more options and explanation\n", + program,program,program); + if (exitCode != PS_EXIT_SUCCESS) + psErrorStackPrint(stderr, "Error reading arguments\n"); + psFree(config); + pmConfigDone(); + psLibFinalize(); + exit(exitCode); +} + +pmConfig *psphotForcedArguments(int argc, char **argv) { + + int N; + bool status, status1, status2; + + // load config data from default locations + pmConfig *config = pmConfigRead(&argc, argv, PSPHOT_RECIPE); + if (config == NULL) { + psErrorStackPrint(stderr, "Can't read site configuration"); + exit(PS_EXIT_CONFIG_ERROR); + } + + PSARGUMENTS_INSTANTIATE_GENERICS( psphot, config, argc, argv ); + + // save the following additional recipe values based on command-line options + // these options override the PSPHOT recipe values loaded from recipe files + psMetadata *options = pmConfigRecipeOptions (config, PSPHOT_RECIPE); + + // Number of threads is handled + PSARGUMENTS_INSTANTIATE_THREADSARG( psphot, config, argc, argv ) + + // visual : interactive display mode + if ((N = psArgumentGet (argc, argv, "-visual"))) { + psArgumentRemove (N, &argc, argv); + pmVisualSetVisual(true); + } + + // break : used from recipe throughout psphotReadout + if ((N = psArgumentGet (argc, argv, "-break"))) { + if (argc<=N+1) { + psErrorStackPrint(stderr, "Expected to see 1 more argument; saw %d", argc - 1); + exit(PS_EXIT_CONFIG_ERROR); + } + psArgumentRemove (N, &argc, argv); + psMetadataAddStr (options, PS_LIST_TAIL, "BREAK_POINT", PS_META_REPLACE, "", argv[N]); + psArgumentRemove (N, &argc, argv); + } + + // analysis region : overrides recipe value, used in psphotReadout/psphotEnsemblePSF + if ((N = psArgumentGet (argc, argv, "-region"))) { + if (argc<=N+1) { + psErrorStackPrint(stderr, "Expected to see 1 more argument; saw %d", argc - 1); + exit(PS_EXIT_CONFIG_ERROR); + } + psArgumentRemove (N, &argc, argv); + psMetadataAddStr (options, PS_LIST_TAIL, "ANALYSIS_REGION", 0, "", argv[N]); + psArgumentRemove (N, &argc, argv); + } + + // chip selection is used to limit chips to be processed + if ((N = psArgumentGet (argc, argv, "-chip"))) { + if (argc<=N+1) { + psErrorStackPrint(stderr, "Expected to see 1 more argument; saw %d", argc - 1); + exit(PS_EXIT_CONFIG_ERROR); + } + psArgumentRemove (N, &argc, argv); + psMetadataAddStr (config->arguments, PS_LIST_TAIL, "CHIP_SELECTIONS", PS_DATA_STRING, "", argv[N]); + psArgumentRemove (N, &argc, argv); + } + + // if these command-line options are supplied, load the file name lists into config->arguments + // override any configuration-specified source for these files + // + pmConfigFileSetsMD (config->arguments, &argc, argv, "MASK", "-mask", "-masklist"); + pmConfigFileSetsMD (config->arguments, &argc, argv, "VARIANCE", "-variance", "-variancelist"); + + status = pmConfigFileSetsMD (config->arguments, &argc, argv, "PSPHOT.PSF", "-psf", "-psflist"); + if (!status) { + psError(PSPHOT_ERR_ARGUMENTS, true, "No psf model is supplied (use -psf)"); + usage(argv[0], config->arguments, config, PS_EXIT_CONFIG_ERROR); + } + + status1 = pmConfigFileSetsMD (config->arguments, &argc, argv, "SRC", "-src", "-srclist"); + status2 = pmConfigFileSetsMD (config->arguments, &argc, argv, "SRCTEXT", "-srctext", "-srctextlist"); + + if (!status1 && !status2) { + psError(PSPHOT_ERR_ARGUMENTS, true, "No source list is supplied (use one of -src, -srctext, -srclist, or -srctextlist)"); + usage(argv[0], config->arguments, config, PS_EXIT_CONFIG_ERROR); + } + + if (argc == 1) { + psError(PSPHOT_ERR_ARGUMENTS, true, "Too few arguments: %d", argc); + usage(argv[0], config->arguments, config, PS_EXIT_CONFIG_ERROR); + } + + if (psArgumentGet(argc, argv, "-help") || + psArgumentGet(argc, argv, "-h")) + writeHelpInfo(argv[0], config, stdout); + + // the input file is a required argument; if not found, we will exit + status = pmConfigFileSetsMD (config->arguments, &argc, argv, "INPUT", "-file", "-list"); + if (!status) { + psError(PSPHOT_ERR_ARGUMENTS, false, "pmConfigFileSetsMD failed to parse arguments"); + usage(argv[0], config->arguments, config, PS_EXIT_CONFIG_ERROR); + } + + if (argc != 2) { + psError(PSPHOT_ERR_ARGUMENTS, true, "Expected to see one more argument; saw %d", argc - 1); + usage(argv[0], config->arguments, config, PS_EXIT_CONFIG_ERROR); + } + + // output position is fixed + psMetadataAddStr (config->arguments, PS_LIST_TAIL, "OUTPUT", 0, "", argv[1]); + + psTrace("psphot", 1, "Done with psphotForcedArguments...\n"); + return (config); +} Index: branches/tap_branches/psphot/src/psphotForcedImageLoop.c =================================================================== --- branches/tap_branches/psphot/src/psphotForcedImageLoop.c (revision 27838) +++ branches/tap_branches/psphot/src/psphotForcedImageLoop.c (revision 27838) @@ -0,0 +1,129 @@ +# include "psphotStandAlone.h" + +# define ESCAPE(MESSAGE) { \ + psError(PSPHOT_ERR_DATA, false, MESSAGE); \ + psFree (view); \ + return false; \ +} + +bool psphotForcedImageLoop (pmConfig *config) { + + bool status; + pmChip *chip; + pmCell *cell; + pmReadout *readout; + + pmFPAfile *load = psMetadataLookupPtr (&status, config->files, "PSPHOT.LOAD"); + if (!status) { + psError(PSPHOT_ERR_PROG, false, "Can't find input data!"); + return false; + } + pmFPAfile *input = psMetadataLookupPtr (&status, config->files, "PSPHOT.INPUT"); + if (!status) { + psError(PSPHOT_ERR_PROG, false, "Can't find input data!"); + return false; + } + + pmFPAview *view = pmFPAviewAlloc (0); + pmHDU *lastHDU = NULL; // Last HDU updated + + // files associated with the science image + if (!pmFPAfileIOChecks (config, view, PM_FPA_BEFORE)) ESCAPE ("failed input for fpa in psphot."); + + // for psphot, we force data to be read at the chip level + while ((chip = pmFPAviewNextChip (view, load->fpa, 1)) != NULL) { + psLogMsg ("psphot", 4, "Chip %d: %x %x\n", view->chip, chip->file_exists, chip->process); + if (! chip->process || ! chip->file_exists) { continue; } + + // load just the input image data (image, mask, weight) + pmFPAfileActivate (config->files, false, NULL); + pmFPAfileActivate (config->files, true, "PSPHOT.LOAD"); + pmFPAfileActivate (config->files, true, "PSPHOT.MASK"); + pmFPAfileActivate (config->files, true, "PSPHOT.VARIANCE"); + if (!pmFPAfileIOChecks (config, view, PM_FPA_BEFORE)) ESCAPE ("failed input for Chip in psphot."); + + // mosaic the cells of a chip into a single contiguous (trimmed) chip + if (!psphotMosaicChip(config, view, "PSPHOT.INPUT", "PSPHOT.LOAD")) ESCAPE ("Unable to mosaic chip."); + + // try to load other supporting data (PSF, SRC, etc). + // do not re-load the following three files + pmFPAfileActivate (config->files, true, NULL); + pmFPAfileActivate (config->files, false, "PSPHOT.LOAD"); + pmFPAfileActivate (config->files, false, "PSPHOT.MASK"); + pmFPAfileActivate (config->files, false, "PSPHOT.VARIANCE"); + if (!pmFPAfileIOChecks (config, view, PM_FPA_BEFORE)) ESCAPE ("failed input for Chip in psphot."); + + // re-activate files so they will be closed and freed below + pmFPAfileActivate (config->files, true, NULL); + + // there is now only a single chip (multiple readouts?). loop over it and process + while ((cell = pmFPAviewNextCell (view, input->fpa, 1)) != NULL) { + psLogMsg ("psphot", 5, "Cell %d: %x %x\n", view->cell, cell->file_exists, cell->process); + + // process each of the readouts + while ((readout = pmFPAviewNextReadout (view, input->fpa, 1)) != NULL) { + psLogMsg ("psphot", 6, "Cell %d: %x %x\n", view->cell, cell->file_exists, cell->process); + if (! readout->data_exists) { continue; } + + // Update the header + pmHDU *hdu = pmHDUGetHighest(input->fpa, chip, cell); + if (hdu && hdu != lastHDU) { + psphotVersionHeaderFull(hdu->header); + lastHDU = hdu; + } + + // if an external mask is supplied, ensure that NAN pixels are also masked + if (readout->mask) { + psImageMaskType maskSat = pmConfigMaskGet("SAT", config); // Mask value for saturated pixels + if (!pmReadoutMaskNonfinite(readout, maskSat)) { + psError(psErrorCodeLast(), false, "Unable to mask non-finite pixels."); + psFree(view); + return false; + } + } + + // run the actual photometry analysis on this chip/cell/readout + if (!psphotForcedReadout (config, view)) { + psError(psErrorCodeLast(), false, "failure in psphotReadout for chip %d, cell %d, readout %d\n", view->chip, view->cell, view->readout); + psFree (view); + return false; + } + } + + status = true; + status &= pmFPAfileDropInternal (config->files, "PSPHOT.BACKMDL"); + status &= pmFPAfileDropInternal (config->files, "PSPHOT.BACKMDL.STDEV"); + status &= pmFPAfileDropInternal (config->files, "PSPHOT.BACKGND"); + if (!status) { + psError(PSPHOT_ERR_PROG, false, "trouble dropping internal files"); + psFree (view); + return false; + } + } + + // save output which is saved at the chip level + if (!pmFPAfileIOChecks (config, view, PM_FPA_AFTER)) ESCAPE ("failed output for Chip in psphot."); + } + // save output which is saved at the fpa level + if (!pmFPAfileIOChecks (config, view, PM_FPA_AFTER)) ESCAPE ("failed ouput for FPA in psphot."); + + // fail if we failed to handle an error + if (psErrorCodeLast() != PS_ERR_NONE) psAbort ("failed to handle an error!"); + + psFree (view); + return true; +} + +// I/O files related to psphot: +// PSPHOT.INPUT : input image file(s) +// PSPHOT.RESID : residual image +// PSPHOT.OUTPUT : output object tables (object) + +// PSPHOT.BACKSUB : background subtracted image +// PSPHOT.BACKGND : background model (full-scale image?) +// PSPHOT.BACKMDL : background model (binned image?) +// PSPHOT.PSF : sample PSF images + +// PSPHOT.MASK +// PSPHOT.VARIANCE +// Index: branches/tap_branches/psphot/src/psphotForcedReadout.c =================================================================== --- branches/tap_branches/psphot/src/psphotForcedReadout.c (revision 27838) +++ branches/tap_branches/psphot/src/psphotForcedReadout.c (revision 27838) @@ -0,0 +1,89 @@ +# include "psphotInternal.h" + +bool psphotForcedReadout(pmConfig *config, const pmFPAview *view) { + + // measure the total elapsed time in psphotReadout. XXX the current threading plan + // for psphot envisions threading within psphotReadout, not multiple threads calling + // the same psphotReadout. In the current plan, this dtime is the elapsed time used + // jointly by the multiple threads, not the total time used by all threads. + psTimerStart ("psphotReadout"); + + // allow objects to be fit with ugly models (central holes, extreme asymmetry, etc) + pmModelClassSetLimits(PM_MODEL_LIMITS_LAX); + + // select the current recipe + psMetadata *recipe = psMetadataLookupPtr (NULL, config->recipes, PSPHOT_RECIPE); + if (!recipe) { + psError(PSPHOT_ERR_CONFIG, false, "missing recipe %s", PSPHOT_RECIPE); + return false; + } + + // set the photcode for this image + if (!psphotAddPhotcode (config, view)) { + psError (PSPHOT_ERR_CONFIG, false, "trouble defining the photcode"); + return false; + } + + // optional break-point for processing + char *breakPt = psMetadataLookupStr (NULL, recipe, "BREAK_POINT"); + PS_ASSERT_PTR_NON_NULL (breakPt, false); + + // Generate the mask and weight images, including the user-defined analysis region of interest + psphotSetMaskAndVariance (config, view); + if (!strcasecmp (breakPt, "NOTHING")) { + return psphotReadoutCleanup(config, view); + } + + // generate a background model (median, smoothed image) + if (!psphotModelBackground (config, view)) { + return psphotReadoutCleanup (config, view); + } + if (!psphotSubtractBackground (config, view)) { + return psphotReadoutCleanup (config, view); + } + if (!strcasecmp (breakPt, "BACKMDL")) { + return psphotReadoutCleanup (config, view); + } + + if (!psphotLoadPSF (config, view)) { + // this only happens if we had a programming error in psphotLoadPSF + psError (PSPHOT_ERR_UNKNOWN, false, "error loading psf model"); + return psphotReadoutCleanup (config, view); + } + + // include externally-supplied sources + psphotLoadExtSources (config, view); + + // construct an initial model for each object, set the radius to fitRadius, set circular fit mask + psphotGuessModels (config, view); + + // merge the newly selected sources into the existing list + // NOTE: merge OLD and NEW + psphotMergeSources (config, view); + + // linear PSF fit to source peaks, subtract the models from the image (in PSF mask) + psphotFitSourcesLinear (config, view, false); + + // identify CRs and extended sources + // XXX do I want to do this step? + // XXX do I want to / need to calculate the moments? + // psphotSourceSize (config, readout, sources, recipe, psf, 0); + + // calculate source magnitudes + psphotMagnitudes(config, view); + + // XXX do I want to do this? + // if (!psphotEfficiency(config, readout, view, psf, recipe, sources)) { + // psErrorStackPrint(stderr, "Unable to determine detection efficiencies from fake sources"); + // psErrorClear(); + // } + + // replace background in residual image + psphotSkyReplace (config, view); + + // drop the references to the image pixels held by each source + psphotSourceFreePixels (config, view); + + // create the exported-metadata and free local data + return psphotReadoutCleanup(config, view); +} Index: branches/tap_branches/psphot/src/psphotGuessModels.c =================================================================== --- branches/tap_branches/psphot/src/psphotGuessModels.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotGuessModels.c (revision 27838) @@ -3,14 +3,58 @@ // XXX : the threading here is not great. this may be due to blocks between elements, but // the selection of the objects in a cell is not optimal. To fix: -// 1) define the boundaries of the cells up front +// 1) define the boundaries of the cells up front // 2) loop over the sources once and associate them with their cell // 3) define the threaded function to work with sources for a given cell -// construct an initial PSF model for each object -bool psphotGuessModels (pmConfig *config, pmReadout *readout, psArray *sources, pmPSF *psf) { +// for now, let's store the detections on the readout->analysis for each readout +bool psphotGuessModels (pmConfig *config, const pmFPAview *view) +{ + bool status = true; + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // skip the chisq image (optionally?) + int chisqNum = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.CHISQ.NUM"); + if (!status) chisqNum = -1; + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (i == chisqNum) continue; // skip chisq image + if (!psphotGuessModelsReadout (config, view, "PSPHOT.INPUT", i)) { + psError (PSPHOT_ERR_CONFIG, false, "failed on to guess models for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + +// construct an initial PSF model for each object (new sources only) +bool psphotGuessModelsReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index) { bool status; psTimerStart ("psphot.models"); + + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *sources = detections->newSources; + psAssert (sources, "missing sources?"); + + if (!sources->n) { + psLogMsg ("psphot", PS_LOG_INFO, "no sources, skipping model guess"); + return true; + } + + pmPSF *psf = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.PSF"); + psAssert (psf, "missing psf?"); // select the appropriate recipe information @@ -21,5 +65,5 @@ int nThreads = psMetadataLookupS32(&status, config->arguments, "NTHREADS"); // Number of threads if (!status) { - nThreads = 0; + nThreads = 0; } @@ -36,5 +80,5 @@ // setup the PSF fit radius details - psphotInitRadiusPSF (recipe, psf->type); + psphotInitRadiusPSF (recipe, readout->analysis, psf->type); // choose Cx, Cy (see psphotThreadTools.c for overview of the concepts) @@ -46,43 +90,43 @@ for (int i = 0; i < cellGroups->n; i++) { - psArray *cells = cellGroups->data[i]; - - for (int j = 0; j < cells->n; j++) { - - // allocate a job -- if threads are not defined, this just runs the job - psThreadJob *job = psThreadJobAlloc ("PSPHOT_GUESS_MODEL"); - psArrayAdd(job->args, 1, readout); - psArrayAdd(job->args, 1, cells->data[j]); // sources - psArrayAdd(job->args, 1, psf); - - // XXX change these to use abstract mask type info - PS_ARRAY_ADD_SCALAR(job->args, maskVal, PS_TYPE_IMAGE_MASK); - PS_ARRAY_ADD_SCALAR(job->args, markVal, PS_TYPE_IMAGE_MASK); - - if (!psThreadJobAddPending(job)) { - psError(PS_ERR_UNKNOWN, false, "Unable to guess model."); - psFree (job); - return false; - } - psFree(job); - } - - // wait for the threads to finish and manage results - // wait here for the threaded jobs to finish - // fprintf (stderr, "wait for threads (%d, %d)\n", jx, jy); - if (!psThreadPoolWait (false)) { - psError(PS_ERR_UNKNOWN, false, "Unable to guess model."); - return false; - } - - // we have only supplied one type of job, so we can assume the types here - psThreadJob *job = NULL; - while ((job = psThreadJobGetDone()) != NULL) { - // we have no returned data from this operation - if (job->args->n < 1) { - fprintf (stderr, "error with job\n"); - } - psFree(job); - } + psArray *cells = cellGroups->data[i]; + + for (int j = 0; j < cells->n; j++) { + + // allocate a job -- if threads are not defined, this just runs the job + psThreadJob *job = psThreadJobAlloc ("PSPHOT_GUESS_MODEL"); + psArrayAdd(job->args, 1, readout); + psArrayAdd(job->args, 1, cells->data[j]); // sources + psArrayAdd(job->args, 1, psf); + + // XXX change these to use abstract mask type info + PS_ARRAY_ADD_SCALAR(job->args, maskVal, PS_TYPE_IMAGE_MASK); + PS_ARRAY_ADD_SCALAR(job->args, markVal, PS_TYPE_IMAGE_MASK); + + if (!psThreadJobAddPending(job)) { + psError(PS_ERR_UNKNOWN, false, "Unable to guess model."); + psFree (job); + return false; + } + psFree(job); + } + + // wait for the threads to finish and manage results + // wait here for the threaded jobs to finish + // fprintf (stderr, "wait for threads (%d, %d)\n", jx, jy); + if (!psThreadPoolWait (false)) { + psError(PS_ERR_UNKNOWN, false, "Unable to guess model."); + return false; + } + + // we have only supplied one type of job, so we can assume the types here + psThreadJob *job = NULL; + while ((job = psThreadJobGetDone()) != NULL) { + // we have no returned data from this operation + if (job->args->n < 1) { + fprintf (stderr, "error with job\n"); + } + psFree(job); + } } @@ -90,9 +134,9 @@ int nMiss = 0; for (int i = 0; i < sources->n; i++) { - pmSource *source = sources->data[i]; - if (source->tmpFlags & PM_SOURCE_TMPF_MODEL_GUESS) { - continue; - } - nMiss ++; + pmSource *source = sources->data[i]; + if (source->tmpFlags & PM_SOURCE_TMPF_MODEL_GUESS) { + continue; + } + nMiss ++; } psAssert (nMiss == 0, "failed to attempt to build models for %d objects\n", nMiss); @@ -110,5 +154,5 @@ psArray *sources = job->args->data[1]; pmPSF *psf = job->args->data[2]; - + psImageMaskType maskVal = PS_SCALAR_VALUE(job->args->data[3],PS_TYPE_IMAGE_MASK_DATA); psImageMaskType markVal = PS_SCALAR_VALUE(job->args->data[4],PS_TYPE_IMAGE_MASK_DATA); @@ -117,69 +161,69 @@ for (int i = 0; i < sources->n; i++) { - pmSource *source = sources->data[i]; - - // this is used to mark sources for which the model is measured. We check later that - // all are used. - source->tmpFlags |= PM_SOURCE_TMPF_MODEL_GUESS; - - // skip non-astronomical objects (very likely defects) - if (source->type == PM_SOURCE_TYPE_DEFECT) continue; - if (source->type == PM_SOURCE_TYPE_SATURATED) continue; - if (!source->peak) continue; - - nSrc ++; - - // the guess central intensity comes from the peak: - float Io = source->peak->flux; - - // We have two options to get a guess for the object position: the position from the - // peak and the position from the moments. Use the peak position if (a) there are no - // moments and (b) the sources is not saturated - - bool useMoments = false; - useMoments = (source->mode & PM_SOURCE_MODE_SATSTAR); // we only want to try if SATSTAR is set, but.. - useMoments = (useMoments && source->moments); // can't if there are no moments - useMoments = (useMoments && source->moments->nPixels); // can't if the moments were not measured - useMoments = (useMoments && !(source->mode && PM_SOURCE_MODE_MOMENTS_FAILURE)); // can't if the moments failed... - - float Xo, Yo; - if (useMoments) { - Xo = source->moments->Mx; - Yo = source->moments->My; - } else { - Xo = source->peak->xf; - Yo = source->peak->yf; - } - - // set PSF parameters for this model (apply 2D shape model to coordinates Xo, Yo) - pmModel *modelPSF = pmModelFromPSFforXY(psf, Xo, Yo, Io); - - if (modelPSF == NULL) { - psWarning ("Failed to determine PSF model at (%f,%f); trying image center", Xo, Yo); - - float Xc = 0.5*readout->image->numCols; - float Yc = 0.5*readout->image->numRows; - pmModel *modelPSF = pmModelFromPSFforXY(psf, Xc, Yc, Io); - if (modelPSF == NULL) { - psError(PSPHOT_ERR_PSF, false, "Failed to determine PSF model at center of image"); - return false; - } - - // Now set the object position at the expected location: - modelPSF->params->data.F32[PM_PAR_XPOS] = Xo; - modelPSF->params->data.F32[PM_PAR_YPOS] = Yo; - source->mode |= PM_SOURCE_MODE_BADPSF; - } - - // set the fit radius based on the object flux limit and the model - // this function affects the mask pixels - psphotCheckRadiusPSF (readout, source, modelPSF, markVal); - - // set the source PSF model - psAssert (source->modelPSF == NULL, "failed to free one of the models?"); - source->modelPSF = modelPSF; - source->modelPSF->residuals = psf->residuals; - - pmSourceCacheModel (source, maskVal); // ALLOC x14 (!) + pmSource *source = sources->data[i]; + + // this is used to mark sources for which the model is measured. We check later that + // all are used. + source->tmpFlags |= PM_SOURCE_TMPF_MODEL_GUESS; + + // skip non-astronomical objects (very likely defects) + if (source->type == PM_SOURCE_TYPE_DEFECT) continue; + if (source->type == PM_SOURCE_TYPE_SATURATED) continue; + if (!source->peak) continue; + + nSrc ++; + + // the guess central intensity comes from the peak: + float Io = source->peak->flux; + + // We have two options to get a guess for the object position: the position from the + // peak and the position from the moments. Use the peak position if (a) there are no + // moments and (b) the sources is not saturated + + bool useMoments = false; + useMoments = (source->mode & PM_SOURCE_MODE_SATSTAR); // we only want to try if SATSTAR is set, but.. + useMoments = (useMoments && source->moments); // can't if there are no moments + useMoments = (useMoments && source->moments->nPixels); // can't if the moments were not measured + useMoments = (useMoments && !(source->mode && PM_SOURCE_MODE_MOMENTS_FAILURE)); // can't if the moments failed... + + float Xo, Yo; + if (useMoments) { + Xo = source->moments->Mx; + Yo = source->moments->My; + } else { + Xo = source->peak->xf; + Yo = source->peak->yf; + } + + // set PSF parameters for this model (apply 2D shape model to coordinates Xo, Yo) + pmModel *modelPSF = pmModelFromPSFforXY(psf, Xo, Yo, Io); + + if (modelPSF == NULL) { + psWarning ("Failed to determine PSF model at (%f,%f); trying image center", Xo, Yo); + + float Xc = 0.5*readout->image->numCols; + float Yc = 0.5*readout->image->numRows; + pmModel *modelPSF = pmModelFromPSFforXY(psf, Xc, Yc, Io); + if (modelPSF == NULL) { + psError(PSPHOT_ERR_PSF, false, "Failed to determine PSF model at center of image"); + return false; + } + + // Now set the object position at the expected location: + modelPSF->params->data.F32[PM_PAR_XPOS] = Xo; + modelPSF->params->data.F32[PM_PAR_YPOS] = Yo; + source->mode |= PM_SOURCE_MODE_BADPSF; + } + + // set the fit radius based on the object flux limit and the model + // this function affects the mask pixels + psphotCheckRadiusPSF (readout, source, modelPSF, markVal); + + // set the source PSF model + psAssert (source->modelPSF == NULL, "failed to free one of the models?"); + source->modelPSF = modelPSF; + source->modelPSF->residuals = psf->residuals; + + pmSourceCacheModel (source, maskVal); // ALLOC x14 (!) } Index: branches/tap_branches/psphot/src/psphotImageLoop.c =================================================================== --- branches/tap_branches/psphot/src/psphotImageLoop.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotImageLoop.c (revision 27838) @@ -91,15 +91,15 @@ } - status = true; - status &= pmFPAfileDropInternal (config->files, "PSPHOT.BACKMDL"); - status &= pmFPAfileDropInternal (config->files, "PSPHOT.BACKMDL.STDEV"); - status &= pmFPAfileDropInternal (config->files, "PSPHOT.BACKGND"); - if (!status) { - psError(PSPHOT_ERR_PROG, false, "trouble dropping internal files"); - psFree (view); - return false; - } + // drop all versions of the internal files + status = true; + status &= pmFPAfileDropInternal (config->files, "PSPHOT.BACKMDL"); + status &= pmFPAfileDropInternal (config->files, "PSPHOT.BACKMDL.STDEV"); + status &= pmFPAfileDropInternal (config->files, "PSPHOT.BACKGND"); + if (!status) { + psError(PSPHOT_ERR_PROG, false, "trouble dropping internal files"); + psFree (view); + return false; + } } - // save output which is saved at the chip level if (!pmFPAfileIOChecks (config, view, PM_FPA_AFTER)) ESCAPE ("failed output for Chip in psphot."); Index: branches/tap_branches/psphot/src/psphotImageQuality.c =================================================================== --- branches/tap_branches/psphot/src/psphotImageQuality.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotImageQuality.c (revision 27838) @@ -4,7 +4,59 @@ // XXX Lots of code duplication here +// for now, let's store the detections on the readout->analysis for each readout +bool psphotImageQuality (pmConfig *config, const pmFPAview *view) +{ + bool status = true; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // skip the chisq image (optionally?) + int chisqNum = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.CHISQ.NUM"); + if (!status) chisqNum = -1; + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (i == chisqNum) continue; // skip chisq image + if (!psphotImageQualityReadout (config, view, "PSPHOT.INPUT", i, recipe)) { + psError (PSPHOT_ERR_CONFIG, false, "failed on to measure image quality for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + // selecting the 'good' stars (likely to be psf stars), measure the M_cn, M_sn terms for n = 2,3,4 -bool psphotImageQuality(psMetadata *recipe, psArray *sources) +bool psphotImageQualityReadout(pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe) { + bool status = true; + + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + // if we have a PSF, skip this analysis + if (psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.PSF")) { + return true; + } + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *sources = detections->newSources; + psAssert (sources, "missing sources?"); + + if (!sources->n) { + psLogMsg ("psphot", PS_LOG_INFO, "no sources, skipping image quality"); + return true; + } + psVector *FWHM_MAJOR = psVectorAllocEmpty(100, PS_TYPE_F32); psVector *FWHM_MINOR = psVectorAllocEmpty(100, PS_TYPE_F32); @@ -81,10 +133,9 @@ if (num == 0) { - psWarning("Unable to find sources from which to measure image quality"); - return false; - } - - psMetadataAddS32(recipe, PS_LIST_TAIL, "IQ_NSTAR", PS_META_REPLACE, - "Number of stars used for IQ measurements", M2->n); + psLogMsg ("psphot", PS_LOG_INFO, "no valid sources for image quality, skipping"); + return true; + } + + psMetadataAddS32(readout->analysis, PS_LIST_TAIL, "IQ_NSTAR", PS_META_REPLACE, "Number of stars used for IQ measurements", M2->n); // XXX make this a recipe option @@ -98,8 +149,6 @@ float fwhm_major = stats->sampleMean; - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_FW1", PS_META_REPLACE, - "FWHM of Major Axis from moments", stats->sampleMean); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_FW1_E", PS_META_REPLACE, - "FWHM scatter (Major) from moments", stats->sampleStdev); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_FW1", PS_META_REPLACE, "FWHM of Major Axis from moments", stats->sampleMean); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_FW1_E", PS_META_REPLACE, "FWHM scatter (Major) from moments", stats->sampleStdev); if (!psVectorStats(stats, FWHM_MINOR, NULL, NULL, 0)) { @@ -108,8 +157,6 @@ } float fwhm_minor = stats->sampleMean; - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_FW2", PS_META_REPLACE, - "FWHM of Minor Axis from moments", stats->sampleMean); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_FW2_E", PS_META_REPLACE, - "FWHM scatter (Minor) from moments", stats->sampleStdev); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_FW2", PS_META_REPLACE, "FWHM of Minor Axis from moments", stats->sampleMean); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_FW2_E", PS_META_REPLACE, "FWHM scatter (Minor) from moments", stats->sampleStdev); if (!psVectorStats(stats, M2, NULL, NULL, 0)) { @@ -119,12 +166,8 @@ float vM2 = stats->sampleMean; float dM2 = stats->sampleStdev; - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2", PS_META_REPLACE, - "M_2 = sqrt (M_c2^2 + M_s2^2)", vM2); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2_ER", PS_META_REPLACE, - "Stdev of M_2", dM2); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2_LQ", PS_META_REPLACE, - "Lower Quartile of M_2", stats->sampleLQ); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2_UQ", PS_META_REPLACE, - "Upper Quartile of M_2", stats->sampleUQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2", PS_META_REPLACE, "M_2 = sqrt (M_c2^2 + M_s2^2)", vM2); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2_ER", PS_META_REPLACE, "Stdev of M_2", dM2); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2_LQ", PS_META_REPLACE, "Lower Quartile of M_2", stats->sampleLQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2_UQ", PS_META_REPLACE, "Upper Quartile of M_2", stats->sampleUQ); if (!psVectorStats(stats, M2c, NULL, NULL, 0)) { @@ -132,12 +175,8 @@ goto FAIL; } - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2C", PS_META_REPLACE, - "M_2c = sum f r^2 cos(2phi) / sum f", stats->sampleMean); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2C_E", PS_META_REPLACE, - "Stdev of M_2c", stats->sampleStdev); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2C_L", PS_META_REPLACE, - "Lower Quartile of M_2c", stats->sampleLQ); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2C_U", PS_META_REPLACE, - "Upper Quartile of M_2c", stats->sampleUQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2C", PS_META_REPLACE, "M_2c = sum f r^2 cos(2phi) / sum f", stats->sampleMean); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2C_E", PS_META_REPLACE, "Stdev of M_2c", stats->sampleStdev); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2C_L", PS_META_REPLACE, "Lower Quartile of M_2c", stats->sampleLQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2C_U", PS_META_REPLACE, "Upper Quartile of M_2c", stats->sampleUQ); if (!psVectorStats(stats, M2s, NULL, NULL, 0)) { @@ -145,12 +184,8 @@ goto FAIL; } - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2S", PS_META_REPLACE, - "M_2s = sum f r^2 cos(2phi) / sum f", stats->sampleMean); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2S_E", PS_META_REPLACE, - "Stdev of M_2s", stats->sampleStdev); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2S_L", PS_META_REPLACE, - "Lower Quartile of M_2s", stats->sampleLQ); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2S_U", PS_META_REPLACE, - "Upper Quartile of M_2s", stats->sampleUQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2S", PS_META_REPLACE, "M_2s = sum f r^2 cos(2phi) / sum f", stats->sampleMean); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2S_E", PS_META_REPLACE, "Stdev of M_2s", stats->sampleStdev); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2S_L", PS_META_REPLACE, "Lower Quartile of M_2s", stats->sampleLQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2S_U", PS_META_REPLACE, "Upper Quartile of M_2s", stats->sampleUQ); if (!psVectorStats(stats, M3, NULL, NULL, 0)) { @@ -160,12 +195,8 @@ float vM3 = stats->sampleMean; float dM3 = stats->sampleStdev; - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M3", PS_META_REPLACE, - "M_3 = sqrt (M_c3^2 + M_s3^2)", vM3); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M3_ER", PS_META_REPLACE, - "Stdev of M_3", dM3); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M3_LQ", PS_META_REPLACE, - "Lower Quartile of M_3", stats->sampleLQ); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M3_UQ", PS_META_REPLACE, - "Upper Quartile of M_3", stats->sampleUQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M3", PS_META_REPLACE, "M_3 = sqrt (M_c3^2 + M_s3^2)", vM3); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M3_ER", PS_META_REPLACE, "Stdev of M_3", dM3); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M3_LQ", PS_META_REPLACE, "Lower Quartile of M_3", stats->sampleLQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M3_UQ", PS_META_REPLACE, "Upper Quartile of M_3", stats->sampleUQ); if (!psVectorStats(stats, M4, NULL, NULL, 0)) { @@ -175,12 +206,8 @@ float vM4 = stats->sampleMean; float dM4 = stats->sampleStdev; - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M4", PS_META_REPLACE, - "M_4 = sqrt (M_c4^2 + M_s4^2)", vM4); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M4_ER", PS_META_REPLACE, - "Stdev of M_4", dM4); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M4_LQ", PS_META_REPLACE, - "Lower Quartile of M_4", stats->sampleLQ); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M4_UQ", PS_META_REPLACE, - "Upper Quartile of M_4", stats->sampleUQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M4", PS_META_REPLACE, "M_4 = sqrt (M_c4^2 + M_s4^2)", vM4); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M4_ER", PS_META_REPLACE, "Stdev of M_4", dM4); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M4_LQ", PS_META_REPLACE, "Lower Quartile of M_4", stats->sampleLQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M4_UQ", PS_META_REPLACE, "Upper Quartile of M_4", stats->sampleUQ); #else @@ -192,8 +219,6 @@ } float fwhm_major = stats->robustMedian; - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_FW1", PS_META_REPLACE, - "FWHM of Major Axis from moments", stats->robustMedian); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_FW1_E", PS_META_REPLACE, - "FWHM scatter (Major) from moments", stats->robustStdev); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_FW1", PS_META_REPLACE, "FWHM of Major Axis from moments", stats->robustMedian); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_FW1_E", PS_META_REPLACE, "FWHM scatter (Major) from moments", stats->robustStdev); if (!psVectorStats(stats, FWHM_MINOR, NULL, NULL, 0)) { @@ -202,8 +227,6 @@ } float fwhm_minor = stats->robustMedian; - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_FW2", PS_META_REPLACE, - "FWHM of Minor Axis from moments", stats->robustMedian); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_FW2_E", PS_META_REPLACE, - "FWHM scatter (Minor) from moments", stats->robustStdev); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_FW2", PS_META_REPLACE, "FWHM of Minor Axis from moments", stats->robustMedian); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_FW2_E", PS_META_REPLACE, "FWHM scatter (Minor) from moments", stats->robustStdev); if (!psVectorStats(stats, M2, NULL, NULL, 0)) { @@ -213,12 +236,8 @@ float vM2 = stats->robustMedian; float dM2 = stats->robustStdev; - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2", PS_META_REPLACE, - "M_2 = sqrt (M_c2^2 + M_s2^2)", vM2); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2_ER", PS_META_REPLACE, - "Stdev of M_2", dM2); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2_LQ", PS_META_REPLACE, - "Lower Quartile of M_2", stats->robustLQ); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2_UQ", PS_META_REPLACE, - "Upper Quartile of M_2", stats->robustUQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2", PS_META_REPLACE, "M_2 = sqrt (M_c2^2 + M_s2^2)", vM2); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2_ER", PS_META_REPLACE, "Stdev of M_2", dM2); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2_LQ", PS_META_REPLACE, "Lower Quartile of M_2", stats->robustLQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2_UQ", PS_META_REPLACE, "Upper Quartile of M_2", stats->robustUQ); if (!psVectorStats(stats, M2c, NULL, NULL, 0)) { @@ -226,12 +245,8 @@ goto FAIL; } - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2C", PS_META_REPLACE, - "M_2c = sum f r^2 cos(2phi) / sum f", stats->robustMedian); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2C_E", PS_META_REPLACE, - "Stdev of M_2c", stats->robustStdev); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2C_L", PS_META_REPLACE, - "Lower Quartile of M_2c", stats->robustLQ); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2C_U", PS_META_REPLACE, - "Upper Quartile of M_2c", stats->robustUQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2C", PS_META_REPLACE, "M_2c = sum f r^2 cos(2phi) / sum f", stats->robustMedian); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2C_E", PS_META_REPLACE, "Stdev of M_2c", stats->robustStdev); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2C_L", PS_META_REPLACE, "Lower Quartile of M_2c", stats->robustLQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2C_U", PS_META_REPLACE, "Upper Quartile of M_2c", stats->robustUQ); if (!psVectorStats(stats, M2s, NULL, NULL, 0)) { @@ -239,12 +254,8 @@ goto FAIL; } - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2S", PS_META_REPLACE, - "M_2s = sum f r^2 cos(2phi) / sum f", stats->robustMedian); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2S_E", PS_META_REPLACE, - "Stdev of M_2s", stats->robustStdev); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2S_L", PS_META_REPLACE, - "Lower Quartile of M_2s", stats->robustLQ); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M2S_U", PS_META_REPLACE, - "Upper Quartile of M_2s", stats->robustUQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2S", PS_META_REPLACE, "M_2s = sum f r^2 cos(2phi) / sum f", stats->robustMedian); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2S_E", PS_META_REPLACE, "Stdev of M_2s", stats->robustStdev); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2S_L", PS_META_REPLACE, "Lower Quartile of M_2s", stats->robustLQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M2S_U", PS_META_REPLACE, "Upper Quartile of M_2s", stats->robustUQ); if (!psVectorStats(stats, M3, NULL, NULL, 0)) { @@ -254,12 +265,8 @@ float vM3 = stats->robustMedian; float dM3 = stats->robustStdev; - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M3", PS_META_REPLACE, - "M_3 = sqrt (M_c3^2 + M_s3^2)", vM3); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M3_ER", PS_META_REPLACE, - "Stdev of M_3", dM3); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M3_LQ", PS_META_REPLACE, - "Lower Quartile of M_3", stats->robustLQ); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M3_UQ", PS_META_REPLACE, - "Upper Quartile of M_3", stats->robustUQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M3", PS_META_REPLACE, "M_3 = sqrt (M_c3^2 + M_s3^2)", vM3); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M3_ER", PS_META_REPLACE, "Stdev of M_3", dM3); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M3_LQ", PS_META_REPLACE, "Lower Quartile of M_3", stats->robustLQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M3_UQ", PS_META_REPLACE, "Upper Quartile of M_3", stats->robustUQ); if (!psVectorStats(stats, M4, NULL, NULL, 0)) { @@ -269,12 +276,8 @@ float vM4 = stats->robustMedian; float dM4 = stats->robustStdev; - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M4", PS_META_REPLACE, - "M_4 = sqrt (M_c4^2 + M_s4^2)", vM4); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M4_ER", PS_META_REPLACE, - "Stdev of M_4", dM4); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M4_LQ", PS_META_REPLACE, - "Lower Quartile of M_4", stats->robustLQ); - psMetadataAddF32(recipe, PS_LIST_TAIL, "IQ_M4_UQ", PS_META_REPLACE, - "Upper Quartile of M_4", stats->robustUQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M4", PS_META_REPLACE, "M_4 = sqrt (M_c4^2 + M_s4^2)", vM4); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M4_ER", PS_META_REPLACE, "Stdev of M_4", dM4); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M4_LQ", PS_META_REPLACE, "Lower Quartile of M_4", stats->robustLQ); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "IQ_M4_UQ", PS_META_REPLACE, "Upper Quartile of M_4", stats->robustUQ); #endif Index: branches/tap_branches/psphot/src/psphotKernelFromPSF.c =================================================================== --- branches/tap_branches/psphot/src/psphotKernelFromPSF.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotKernelFromPSF.c (revision 27838) @@ -4,8 +4,8 @@ assert (source); - assert (source->psfFlux); // XXX build if needed? + assert (source->psfImage); // XXX build if needed? - int x0 = source->peak->xf - source->psfFlux->col0; - int y0 = source->peak->yf - source->psfFlux->row0; + int x0 = source->peak->xf - source->psfImage->col0; + int y0 = source->peak->yf - source->psfImage->row0; // need to decide on the size: dynamically? statically? @@ -17,12 +17,12 @@ // if the realized PSF for this object does not cover the full kernel, give up for now if (x0 + psf->xMin < 0) goto escape; - if (x0 + psf->xMax >= source->psfFlux->numCols) goto escape; + if (x0 + psf->xMax >= source->psfImage->numCols) goto escape; if (y0 + psf->yMin < 0) goto escape; - if (y0 + psf->yMax >= source->psfFlux->numRows) goto escape; + if (y0 + psf->yMax >= source->psfImage->numRows) goto escape; double sum = 0.0; for (int j = psf->yMin; j <= psf->yMax; j++) { for (int i = psf->xMin; i <= psf->xMax; i++) { - double value = source->psfFlux->data.F32[y0 + j][x0 + i]; + double value = source->psfImage->data.F32[y0 + j][x0 + i]; psf->kernel[j][i] = value; sum += value; Index: branches/tap_branches/psphot/src/psphotLoadPSF.c =================================================================== --- branches/tap_branches/psphot/src/psphotLoadPSF.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotLoadPSF.c (revision 27838) @@ -1,28 +1,79 @@ # include "psphotInternal.h" +// NOTE : pmPSF_IO.c functions must load the psf model onto the chip->analysis metadata because +// the I/O operation likely occurs before the readout exists. This implementation assumes that +// a single psf model is valid for the entire set of readouts (not valid for a time series of readouts) + +// XXX for now (2010.01.27), the supporting programs do not define multiple PSPHOT.PSF.LOAD +// files to go with multiple PSPHOT.INPUT files. as a result, the implementation below is +// currently going to work for the case of a single input file, but will fail if we try with a +// stack of images. + // load an externally supplied psf model -pmPSF *psphotLoadPSF (pmConfig *config, const pmFPAview *view, psMetadata *recipe) { +bool psphotLoadPSFReadout (pmConfig *config, const pmFPAview *view, const char *outFilename, const char *inFilename, int index) { + + bool status; + + // find the currently selected readout + pmFPAfile *inFile = pmFPAfileSelectSingle(config->files, inFilename, index); // File of interest + if (inFile == NULL) { + psLogMsg ("psphot", 3, "no psf supplied for this chip"); + return true; + } + + // find the currently selected readout + pmFPAfile *outFile = pmFPAfileSelectSingle(config->files, outFilename, index); // File of interest + if (outFile == NULL) { + psLogMsg ("psphot", 3, "no psf supplied for this chip"); + return true; + } // find the currently selected chip - pmChip *chip = pmFPAfileThisChip (config->files, view, "PSPHOT.PSF.LOAD"); - if (!chip) return NULL; + pmChip *chip = pmFPAviewThisChip (view, inFile->fpa); + if (!chip) return false; // find the currently selected readout - pmReadout *readout = pmFPAfileThisReadout (config->files, view, "PSPHOT.PSF.LOAD"); - if (!readout) return NULL; + pmReadout *readout = pmFPAviewThisReadout (view, outFile->fpa); + if (!readout) return false; // check if a PSF model is supplied by the user - pmPSF *psf = psMetadataLookupPtr (NULL, chip->analysis, "PSPHOT.PSF"); + pmPSF *psf = psMetadataLookupPtr (&status, chip->analysis, "PSPHOT.PSF"); if (psf == NULL) { psLogMsg ("psphot", 3, "no psf supplied for this chip"); - return NULL; + return true; } - if (!psphotPSFstats (readout, recipe, psf)) psAbort("cannot measure PSF shape terms"); + if (!psphotPSFstats (readout, psf)) { + psAbort("cannot measure PSF shape terms"); + } psLogMsg ("psphot", 3, "using externally supplied PSF model for this readout"); - // we return a psf which can be free (and which may be removed from the metadata) - psMemIncrRefCounter (psf); - return psf; + // save PSF on readout->analysis + if (!psMetadataAddPtr (readout->analysis, PS_LIST_TAIL, "PSPHOT.PSF", PS_META_REPLACE | PS_DATA_UNKNOWN, "psphot psf model", psf)) { + psError (PSPHOT_ERR_UNKNOWN, false, "problem saving sources on readout"); + return false; + } + + return true; } + +bool psphotLoadPSF (pmConfig *config, const pmFPAview *view) { + + bool status = false; + + // XXX PSPHOT.PSF.LOAD vs PSPHOT.INPUT -- see note at top + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + + // Generate the mask and weight images, including the user-defined analysis region of interest + if (!psphotLoadPSFReadout (config, view, "PSPHOT.INPUT", "PSPHOT.PSF.LOAD", i)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to load PSF model for PSPHOT.PSF.LOAD entry %d", i); + return false; + } + } + return true; +} Index: branches/tap_branches/psphot/src/psphotLoadSRCTEXT.c =================================================================== --- branches/tap_branches/psphot/src/psphotLoadSRCTEXT.c (revision 27838) +++ branches/tap_branches/psphot/src/psphotLoadSRCTEXT.c (revision 27838) @@ -0,0 +1,135 @@ +# include "psphotInternal.h" + +bool psphotLoadSRCTEXT (pmFPA *fpa, pmConfig *config) { + + pmChip *chip; + + // we have a psArray of filenames, but there is no information to tie the filenames to the + // corresponding chip/cell/readout. Fortunately: (a) psphot only really operates on chips + // and (b) we can make some basic assumptions: + + // 1) if there is only a single entry in the array, read the file and attach the results to + // each of the chip->analysis metadata entries (just an extra ref counter). + + // 2) if there are multiple entries, the number of filenames must match the number of + // chips. + + psArray *files = psMetadataLookupPtr(NULL, config->arguments, "SRCTEXT"); + + // XXX is this allowed? how do we get an empty entry in arguments? + if (!files->n) return true; + + // we use a default model type of GAUSS to define the sources + pmModelType modelType = pmModelClassGetType("PS_MODEL_GAUSS"); + + psArray *sourceArrays = psArrayAllocEmpty(files->n); + for (int i = 0; i < files->n; i++) { + + char *filename = files->data[i]; + + // each file contains a list of X Y coordinates and nothing else? + FILE *f = fopen (filename, "r"); + if (f == NULL) { + psError(PSPHOT_ERR_CONFIG, true, "Failed to read source text list %s", filename); + return false; + } + + double X, Y; + + psArray *sources = psArrayAllocEmpty (100); + while (fscanf (f, "%lf %lf", &X, &Y) != EOF) { + + psEllipseAxes axes; + + pmSource *source = pmSourceAlloc (); + pmModel *model = pmModelAlloc (modelType); + source->modelPSF = model; + source->type = PM_SOURCE_TYPE_STAR; + + // NOTE: A SEGV here because "model" is NULL is probably caused by not initialising the models. + psF32 *PAR = model->params->data.F32; + psF32 *dPAR = model->dparams->data.F32; + + // NOTE: most of these values are irrelevant for loaded source positions + source->seq = 0; + PAR[PM_PAR_XPOS] = X; + PAR[PM_PAR_YPOS] = Y; + dPAR[PM_PAR_XPOS] = 0.0; + dPAR[PM_PAR_YPOS] = 0.0; + axes.major = 1.0; + axes.minor = 1.0; + axes.theta = 0.0; + + PAR[PM_PAR_SKY] = 0.0; + dPAR[PM_PAR_SKY] = 0.0; + source->sky = PAR[PM_PAR_SKY]; + source->skyErr = dPAR[PM_PAR_SKY]; + + source->psfMag = 0.0; + source->errMag = 0.0; + source->apMag = 0.0; + + PAR[PM_PAR_I0] = 1.0; + dPAR[PM_PAR_I0] = 0.0; + + pmPSF_AxesToModel (PAR, axes); + + float peakFlux = 1.0; + + source->peak = pmPeakAlloc(PAR[PM_PAR_XPOS], PAR[PM_PAR_YPOS], peakFlux, PM_PEAK_LONE); + source->peak->flux = peakFlux; + source->peak->dx = dPAR[PM_PAR_XPOS]; + source->peak->dy = dPAR[PM_PAR_YPOS]; + source->peak->xf = PAR[PM_PAR_XPOS]; // the alloc function takes the pixel index, + source->peak->yf = PAR[PM_PAR_YPOS]; // but we know the pixel coordinate + + source->pixWeight = 1.0; + source->crNsigma = 0.0; + source->extNsigma = 0.0; + source->apRadius = 0.0; + + model->chisq = 0.0; + model->nDOF = 0; + model->nPix = 0; + + source->moments = pmMomentsAlloc (); + source->moments->Mxx = 0.0; + source->moments->Mxy = 0.0; + source->moments->Myy = 0.0; + + source->mode = 0; + + psArrayAdd (sources, 100, source); + psFree(source); + } + psArrayAdd (sourceArrays, 100, sources); + psFree (sources); + } + + if (files->n == 1) { + psArray *sources = sourceArrays->data[0]; + pmFPAview *view = pmFPAviewAlloc (0); + while ((chip = pmFPAviewNextChip (view, fpa, 1)) != NULL) { + if (!chip->process) { continue; } + psMetadataAddPtr (chip->analysis, PS_LIST_TAIL, "PSPHOT.SOURCES.TEXT", PS_DATA_ARRAY | PS_META_REPLACE, "loaded source from textfile", sources); + } + psFree (view); + } else { + int nSrc = 0; + pmFPAview *view = pmFPAviewAlloc (0); + while ((chip = pmFPAviewNextChip (view, fpa, 1)) != NULL) { + if (!chip->process) { continue; } + + if (nSrc >= sourceArrays->n) { + psError(PSPHOT_ERR_CONFIG, true, "mismatch between number of source lists and number of chips (readouts)"); + return false; + } + psArray *sources = sourceArrays->data[nSrc]; + psMetadataAddPtr (chip->analysis, PS_LIST_TAIL, "PSPHOT.SOURCES.TEXT", PS_DATA_ARRAY | PS_META_REPLACE, "loaded source from textfile", sources); + nSrc ++; + } + psFree (view); + } + psFree (sourceArrays); + return true; +} Index: branches/tap_branches/psphot/src/psphotMagnitudes.c =================================================================== --- branches/tap_branches/psphot/src/psphotMagnitudes.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotMagnitudes.c (revision 27838) @@ -1,14 +1,58 @@ # include "psphotInternal.h" -bool psphotMagnitudes(pmConfig *config, pmReadout *readout, const pmFPAview *view, psArray *sources, const pmPSF *psf) { +bool psphotMagnitudes (pmConfig *config, const pmFPAview *view) +{ + bool status = true; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // skip the chisq image (optionally?) + int chisqNum = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.CHISQ.NUM"); + if (!status) chisqNum = -1; + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (i == chisqNum) continue; // skip chisq image + + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, "PSPHOT.INPUT", i); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *sources = detections->allSources; + psAssert (sources, "missing sources?"); + + pmPSF *psf = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.PSF"); + psAssert (psf, "missing psf?"); + + if (!psphotMagnitudesReadout (config, recipe, view, readout, sources, psf)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to measure magnitudes for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + +bool psphotMagnitudesReadout(pmConfig *config, psMetadata *recipe, const pmFPAview *view, pmReadout *readout, psArray *sources, pmPSF *psf) { bool status = false; int Nap = 0; + if (!sources->n) { + psLogMsg ("psphot", PS_LOG_INFO, "no sources, skipping source magnitudes"); + return true; + } + psTimerStart ("psphot.mags"); - - // select the appropriate recipe information - psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); - assert (recipe); // determine the number of allowed threads @@ -41,10 +85,12 @@ } - bool IGNORE_GROWTH = psMetadataLookupBool (&status, recipe, "IGNORE_GROWTH"); + bool IGNORE_GROWTH = psMetadataLookupBool (&status, recipe, "IGNORE_GROWTH"); bool INTERPOLATE_AP = psMetadataLookupBool (&status, recipe, "INTERPOLATE_AP"); + bool DIFF_STATS = psMetadataLookupBool (&status, recipe, "INTERPOLATE_AP"); pmSourcePhotometryMode photMode = PM_SOURCE_PHOT_APCORR | PM_SOURCE_PHOT_WEIGHT; if (!IGNORE_GROWTH) photMode |= PM_SOURCE_PHOT_GROWTH; if (INTERPOLATE_AP) photMode |= PM_SOURCE_PHOT_INTERP; + if (DIFF_STATS) photMode |= PM_SOURCE_PHOT_DIFFSTATS; // choose Cx, Cy (see psphotThreadTools.c for overview of the concepts) @@ -64,5 +110,5 @@ psArrayAdd(job->args, 1, cells->data[j]); // sources - psArrayAdd(job->args, 1, (pmPSF*)psf); // Casting away const + psArrayAdd(job->args, 1, psf); psArrayAdd(job->args, 1, binning); psArrayAdd(job->args, 1, backModel); @@ -179,43 +225,4 @@ } -# if (0) -bool psphotMagnitudes_Unthreaded (int *nap, psArray *sources, pmPSF *psf, psImageBinning *binning, pmReadout *backModel, pmReadout *backStdev, pmSourcePhotometryMode photMode, psImageMaskType maskVal) { - - bool status; - int Nap = 0; - - for (int i = 0; i < sources->n; i++) { - pmSource *source = (pmSource *) sources->data[i]; - status = pmSourceMagnitudes (source, psf, photMode, maskVal); - if (status && isfinite(source->apMag)) Nap ++; - - if (backModel) { - psAssert (binning, "if backModel is defined, so should binning be"); - source->sky = psImageUnbinPixel(source->peak->x, source->peak->y, backModel->image, binning); - if (isnan(source->sky) && false) { - psLogMsg ("psphot.magnitudes", PS_LOG_DETAIL, "error setting pmSource.sky"); - } - } else { - source->sky = NAN; - } - - if (backStdev) { - psAssert (binning, "if backStdev is defined, so should binning be"); - source->skyErr = psImageUnbinPixel(source->peak->x, source->peak->y, backStdev->image, binning); - if (isnan(source->skyErr) && false) { - psLogMsg ("psphot.magnitudes", PS_LOG_DETAIL, "error setting pmSource.skyErr"); - } - } else { - source->skyErr = NAN; - } - } - - // change the value of a scalar on the array (wrap this and put it in psArray.h) - *nap = Nap; - - return true; -} -# endif - bool psphotPSFWeights(pmConfig *config, pmReadout *readout, const pmFPAview *view, psArray *sources) { Index: branches/tap_branches/psphot/src/psphotMakePSF.c =================================================================== --- branches/tap_branches/psphot/src/psphotMakePSF.c (revision 27838) +++ branches/tap_branches/psphot/src/psphotMakePSF.c (revision 27838) @@ -0,0 +1,35 @@ +# include "psphotStandAlone.h" + +int main (int argc, char **argv) { + + psTimerStart ("complete"); + pmErrorRegister(); // register psModule's error codes/messages + psphotInit(); + + // load command-line arguments, options, and system config data + pmConfig *config = psphotMakePSFArguments (argc, argv); + assert(config); + + psphotVersionPrint(); + + // load input data (config and images (signal, noise, mask) + if (!psphotParseCamera (config)) { + psErrorStackPrint(stderr, "Error setting up the camera\n"); + exit (psphotGetExitStatus()); + } + + // call psphot for each readout + if (!psphotMakePSFImageLoop (config)) { + psErrorStackPrint(stderr, "Error in the psphot image loop\n"); + exit (psphotGetExitStatus()); + } + + psLogMsg ("psphot", 3, "complete psphot run: %f sec\n", psTimerMark ("complete")); + + psErrorCode exit_status = psphotGetExitStatus(); + psphotCleanup (config); + exit (exit_status); +} + +// all functions which return to this level must raise one of the top-level error codes if they +// exit with an error. these error codes are used to specify the program exit status Index: branches/tap_branches/psphot/src/psphotMakePSFArguments.c =================================================================== --- branches/tap_branches/psphot/src/psphotMakePSFArguments.c (revision 27838) +++ branches/tap_branches/psphot/src/psphotMakePSFArguments.c (revision 27838) @@ -0,0 +1,189 @@ +# include "psphotStandAlone.h" + +static void writeHelpInfo(const char* program, pmConfig* config, FILE* ofile) +{ + fprintf(ofile, + "Usage: one of the following\n" + "%s -file fname1[,fname2,...] -mask maskfile1[,maskfile2,...]\n" + " -variance varfile1[,varfile2,...] OutFileBaseName\n" + "\n" + "%s -list FileNameList [-masklist MaskFileNameList] \n" + " -variancelist VarFileNameList OutFileBaseName\n" + "\n" + "%s -help\n" + "\n" + "%s -version\n" + "\n" + "where:\n" + " FileNameList is a text file containing filenames, one per line\n" + " MaskFileNameList is a text file of mask filenames, one per line\n" + " VarFileNameList is a text file of variance filenames, one per line\n" + " OutFileBaseName is the 'root name' for output files\n" + "\n" + "additional options:\n" + " -psf PsfFile1[,PsfFile2,...] or -psflist PsfFileNameList\n" + " specify PSF rather than letting %s estimate it\n" + " -src SrcFile1[,SrcFile2,...] or -srclist SrcFileNameList\n" + " specify additional sources for PSF generation\n" + " -chip nn[,nn,...]\n" + " select detector chips to process; default is all.\n" + " Indices correspond to zero-based offset in the FPA metadata table.\n" + " -photcode PhotoCodeName\n" + " specify photocode\n" + " -region RegionString\n" + " specify analysis region. String is of form '[x0:x1,y0:y1]'\n" + " To use this option you must define a default in psphot.config\n" + " -visual\n" + " turns on interactive display mode\n" + " -dumpconfig CfgFileName\n" + " causes config info to be dumped to the named file.\n" + " -break NOTHING|BACKMDL|PEAKS|MOMENTS|PSFMODEL|ENSEMBLE|PASS1\n" + " choose a point at which to exit processing early\n" + " -nthreads n\n" + " set number of parallel threads of execution\n" + " -F OldFileRule ReplacementFileRule\n" + " change file naming rule; e.g. '-F PSPHOT.OUTPUT PSPHOT.OUT.CMF.MEF'\n" + " -D name stringval\n" + " set a string-valued config parameter\n" + " -Di name intval\n" + " set an integer-valued config parameter\n" + " -Df name fval\n" + " set a float-valued config parameter\n" + " -Db name boolval\n" + " set a boolean-valued config parameter\n" + " -v, -vv, -vvv\n" + " set increasing levels of verbosity\n" + " -logfmt FormatString\n" + " set format string used for log messages\n" + " -trace Fac Lvl\n" + " set tracing for facility Fac to integer Lvl, e.g. '-trace err 10'\n" + " -trace-levels\n" + " print current trace levels\n", + program,program,program,program,program); + psFree(config); + pmConfigDone(); + psLibFinalize(); + exit(PS_EXIT_SUCCESS); +} + +static void usage(const char *program, // Name of the program + psMetadata *arguments, // Command-line arguments + pmConfig *config, // Configuration + int exitCode + ) +{ + fprintf(stderr, + "Usage: one of the following\n" + "%s -file fname1[,fname2,...] -mask maskfile1[,maskfile2,...]\n" + " -variance varfile1[,varfile2,...] OutFileBaseName\n" + "\n" + "%s -list FileNameList [-masklist MaskFileNameList] \n" + " -variancelist VarFileNameList OutFileBaseName\n" + "\n" + "Try '%s -help' for more options and explanation\n", + program,program,program); + if (exitCode != PS_EXIT_SUCCESS) + psErrorStackPrint(stderr, "Error reading arguments\n"); + psFree(config); + pmConfigDone(); + psLibFinalize(); + exit(exitCode); +} + +pmConfig *psphotMakePSFArguments(int argc, char **argv) { + + int N; + bool status; + + // load config data from default locations + pmConfig *config = pmConfigRead(&argc, argv, PSPHOT_RECIPE); + if (config == NULL) { + psErrorStackPrint(stderr, "Can't read site configuration"); + exit(PS_EXIT_CONFIG_ERROR); + } + + PSARGUMENTS_INSTANTIATE_GENERICS( psphot, config, argc, argv ); + + // save the following additional recipe values based on command-line options + // these options override the PSPHOT recipe values loaded from recipe files + psMetadata *options = pmConfigRecipeOptions (config, PSPHOT_RECIPE); + + // Number of threads is handled + PSARGUMENTS_INSTANTIATE_THREADSARG( psphot, config, argc, argv ) + + // visual : interactive display mode + if ((N = psArgumentGet (argc, argv, "-visual"))) { + psArgumentRemove (N, &argc, argv); + pmVisualSetVisual(true); + } + + // break : used from recipe throughout psphotReadout + if ((N = psArgumentGet (argc, argv, "-break"))) { + if (argc<=N+1) { + psErrorStackPrint(stderr, "Expected to see 1 more argument; saw %d", argc - 1); + exit(PS_EXIT_CONFIG_ERROR); + } + psArgumentRemove (N, &argc, argv); + psMetadataAddStr (options, PS_LIST_TAIL, "BREAK_POINT", PS_META_REPLACE, "", argv[N]); + psArgumentRemove (N, &argc, argv); + } + + // analysis region : overrides recipe value, used in psphotReadout/psphotEnsemblePSF + if ((N = psArgumentGet (argc, argv, "-region"))) { + if (argc<=N+1) { + psErrorStackPrint(stderr, "Expected to see 1 more argument; saw %d", argc - 1); + exit(PS_EXIT_CONFIG_ERROR); + } + psArgumentRemove (N, &argc, argv); + psMetadataAddStr (options, PS_LIST_TAIL, "ANALYSIS_REGION", 0, "", argv[N]); + psArgumentRemove (N, &argc, argv); + } + + // chip selection is used to limit chips to be processed + if ((N = psArgumentGet (argc, argv, "-chip"))) { + if (argc<=N+1) { + psErrorStackPrint(stderr, "Expected to see 1 more argument; saw %d", argc - 1); + exit(PS_EXIT_CONFIG_ERROR); + } + psArgumentRemove (N, &argc, argv); + psMetadataAddStr (config->arguments, PS_LIST_TAIL, "CHIP_SELECTIONS", PS_DATA_STRING, "", argv[N]); + psArgumentRemove (N, &argc, argv); + } + + // if these command-line options are supplied, load the file name lists into config->arguments + // override any configuration-specified source for these files + // + pmConfigFileSetsMD (config->arguments, &argc, argv, "MASK", "-mask", "-masklist"); + pmConfigFileSetsMD (config->arguments, &argc, argv, "VARIANCE", "-variance", "-variancelist"); + + // an input list of sources to use is allowed, but not required + pmConfigFileSetsMD (config->arguments, &argc, argv, "SRC", "-src", "-srclist"); + pmConfigFileSetsMD (config->arguments, &argc, argv, "SRCTEXT", "-srctext", "-srctextlist"); + + if (argc == 1) { + psError(PSPHOT_ERR_ARGUMENTS, true, "Too few arguments: %d", argc); + usage(argv[0], config->arguments, config, PS_EXIT_CONFIG_ERROR); + } + + if (psArgumentGet(argc, argv, "-help") || + psArgumentGet(argc, argv, "-h")) + writeHelpInfo(argv[0], config, stdout); + + // the input file is a required argument; if not found, we will exit + status = pmConfigFileSetsMD (config->arguments, &argc, argv, "INPUT", "-file", "-list"); + if (!status) { + psError(PSPHOT_ERR_ARGUMENTS, false, "pmConfigFileSetsMD failed to parse arguments"); + usage(argv[0], config->arguments, config, PS_EXIT_CONFIG_ERROR); + } + + if (argc != 2) { + psError(PSPHOT_ERR_ARGUMENTS, true, "Expected to see one more argument; saw %d", argc - 1); + usage(argv[0], config->arguments, config, PS_EXIT_CONFIG_ERROR); + } + + // output position is fixed + psMetadataAddStr (config->arguments, PS_LIST_TAIL, "OUTPUT", 0, "", argv[1]); + + psTrace("psphot", 1, "Done with psphotMakePSFArguments...\n"); + return (config); +} Index: branches/tap_branches/psphot/src/psphotMakePSFImageLoop.c =================================================================== --- branches/tap_branches/psphot/src/psphotMakePSFImageLoop.c (revision 27838) +++ branches/tap_branches/psphot/src/psphotMakePSFImageLoop.c (revision 27838) @@ -0,0 +1,129 @@ +# include "psphotStandAlone.h" + +# define ESCAPE(MESSAGE) { \ + psError(PSPHOT_ERR_DATA, false, MESSAGE); \ + psFree (view); \ + return false; \ +} + +bool psphotMakePSFImageLoop (pmConfig *config) { + + bool status; + pmChip *chip; + pmCell *cell; + pmReadout *readout; + + pmFPAfile *load = psMetadataLookupPtr (&status, config->files, "PSPHOT.LOAD"); + if (!status) { + psError(PSPHOT_ERR_PROG, false, "Can't find input data!"); + return false; + } + pmFPAfile *input = psMetadataLookupPtr (&status, config->files, "PSPHOT.INPUT"); + if (!status) { + psError(PSPHOT_ERR_PROG, false, "Can't find input data!"); + return false; + } + + pmFPAview *view = pmFPAviewAlloc (0); + pmHDU *lastHDU = NULL; // Last HDU updated + + // files associated with the science image + if (!pmFPAfileIOChecks (config, view, PM_FPA_BEFORE)) ESCAPE ("failed input for fpa in psphot."); + + // for psphot, we force data to be read at the chip level + while ((chip = pmFPAviewNextChip (view, load->fpa, 1)) != NULL) { + psLogMsg ("psphot", 4, "Chip %d: %x %x\n", view->chip, chip->file_exists, chip->process); + if (! chip->process || ! chip->file_exists) { continue; } + + // load just the input image data (image, mask, weight) + pmFPAfileActivate (config->files, false, NULL); + pmFPAfileActivate (config->files, true, "PSPHOT.LOAD"); + pmFPAfileActivate (config->files, true, "PSPHOT.MASK"); + pmFPAfileActivate (config->files, true, "PSPHOT.VARIANCE"); + if (!pmFPAfileIOChecks (config, view, PM_FPA_BEFORE)) ESCAPE ("failed input for Chip in psphot."); + + // mosaic the cells of a chip into a single contiguous (trimmed) chip + if (!psphotMosaicChip(config, view, "PSPHOT.INPUT", "PSPHOT.LOAD")) ESCAPE ("Unable to mosaic chip."); + + // try to load other supporting data (PSF, SRC, etc). + // do not re-load the following three files + pmFPAfileActivate (config->files, true, NULL); + pmFPAfileActivate (config->files, false, "PSPHOT.LOAD"); + pmFPAfileActivate (config->files, false, "PSPHOT.MASK"); + pmFPAfileActivate (config->files, false, "PSPHOT.VARIANCE"); + if (!pmFPAfileIOChecks (config, view, PM_FPA_BEFORE)) ESCAPE ("failed input for Chip in psphot."); + + // re-activate files so they will be closed and freed below + pmFPAfileActivate (config->files, true, NULL); + + // there is now only a single chip (multiple readouts?). loop over it and process + while ((cell = pmFPAviewNextCell (view, input->fpa, 1)) != NULL) { + psLogMsg ("psphot", 5, "Cell %d: %x %x\n", view->cell, cell->file_exists, cell->process); + + // process each of the readouts + while ((readout = pmFPAviewNextReadout (view, input->fpa, 1)) != NULL) { + psLogMsg ("psphot", 6, "Cell %d: %x %x\n", view->cell, cell->file_exists, cell->process); + if (! readout->data_exists) { continue; } + + // Update the header + pmHDU *hdu = pmHDUGetHighest(input->fpa, chip, cell); + if (hdu && hdu != lastHDU) { + psphotVersionHeaderFull(hdu->header); + lastHDU = hdu; + } + + // if an external mask is supplied, ensure that NAN pixels are also masked + if (readout->mask) { + psImageMaskType maskSat = pmConfigMaskGet("SAT", config); // Mask value for saturated pixels + if (!pmReadoutMaskNonfinite(readout, maskSat)) { + psError(psErrorCodeLast(), false, "Unable to mask non-finite pixels."); + psFree(view); + return false; + } + } + + // run the actual photometry analysis on this chip/cell/readout + if (!psphotMakePSFReadout (config, view)) { + psError(psErrorCodeLast(), false, "failure in psphotReadout for chip %d, cell %d, readout %d\n", view->chip, view->cell, view->readout); + psFree (view); + return false; + } + } + + status = true; + status &= pmFPAfileDropInternal (config->files, "PSPHOT.BACKMDL"); + status &= pmFPAfileDropInternal (config->files, "PSPHOT.BACKMDL.STDEV"); + status &= pmFPAfileDropInternal (config->files, "PSPHOT.BACKGND"); + if (!status) { + psError(PSPHOT_ERR_PROG, false, "trouble dropping internal files"); + psFree (view); + return false; + } + } + + // save output which is saved at the chip level + if (!pmFPAfileIOChecks (config, view, PM_FPA_AFTER)) ESCAPE ("failed output for Chip in psphot."); + } + // save output which is saved at the fpa level + if (!pmFPAfileIOChecks (config, view, PM_FPA_AFTER)) ESCAPE ("failed ouput for FPA in psphot."); + + // fail if we failed to handle an error + if (psErrorCodeLast() != PS_ERR_NONE) psAbort ("failed to handle an error!"); + + psFree (view); + return true; +} + +// I/O files related to psphot: +// PSPHOT.INPUT : input image file(s) +// PSPHOT.RESID : residual image +// PSPHOT.OUTPUT : output object tables (object) + +// PSPHOT.BACKSUB : background subtracted image +// PSPHOT.BACKGND : background model (full-scale image?) +// PSPHOT.BACKMDL : background model (binned image?) +// PSPHOT.PSF : sample PSF images + +// PSPHOT.MASK +// PSPHOT.VARIANCE +// Index: branches/tap_branches/psphot/src/psphotMakePSFReadout.c =================================================================== --- branches/tap_branches/psphot/src/psphotMakePSFReadout.c (revision 27838) +++ branches/tap_branches/psphot/src/psphotMakePSFReadout.c (revision 27838) @@ -0,0 +1,76 @@ +# include "psphotInternal.h" + +bool psphotMakePSFReadout(pmConfig *config, const pmFPAview *view) { + + // measure the total elapsed time in psphotReadout. XXX the current threading plan + // for psphot envisions threading within psphotReadout, not multiple threads calling + // the same psphotReadout. In the current plan, this dtime is the elapsed time used + // jointly by the multiple threads, not the total time used by all threads. + psTimerStart ("psphotReadout"); + + pmModelClassSetLimits(PM_MODEL_LIMITS_LAX); + + // select the current recipe + psMetadata *recipe = psMetadataLookupPtr (NULL, config->recipes, PSPHOT_RECIPE); + if (!recipe) { + psError(PSPHOT_ERR_CONFIG, false, "missing recipe %s", PSPHOT_RECIPE); + return false; + } + + // set the photcode for this image + if (!psphotAddPhotcode (config, view)) { + psError (PSPHOT_ERR_CONFIG, false, "trouble defining the photcode"); + return false; + } + + // optional break-point for processing + char *breakPt = psMetadataLookupStr (NULL, recipe, "BREAK_POINT"); + PS_ASSERT_PTR_NON_NULL (breakPt, false); + + // Generate the mask and weight images, including the user-defined analysis region of interest + psphotSetMaskAndVariance (config, view); + if (!strcasecmp (breakPt, "NOTHING")) { + return psphotReadoutCleanup(config, view); + } + + // generate a background model (median, smoothed image) + if (!psphotModelBackground (config, view)) { + return psphotReadoutCleanup (config, view); + } + if (!psphotSubtractBackground (config, view)) { + return psphotReadoutCleanup (config, view); + } + if (!strcasecmp (breakPt, "BACKMDL")) { + return psphotReadoutCleanup (config, view); + } + + psphotLoadExtSources (config, view); + + // If sources have been supplied, then these should be used to measure the PSF include + // externally-supplied sources; if not, we need to generate a set of possible PSF sources. + // This function updates the SN entries for the loaded sources or generates a set of + // detections from the image, if no external ones have been supplied. Sources loaded from + // a text file have no valid SN values, but psphotChoosePSF needs to select the top + // PSF_MAX_NSTARS to generate the PSF. + if (!psphotCheckExtSources (config, view)) { + psLogMsg ("psphot", 3, "failure to select possible PSF sources (external or internal)"); + return psphotReadoutCleanup (config, view); + } + + // Use bright stellar objects to measure PSF. If we do not have enough stars to generate + // the PSF, build one from the SEEING guess and model class + if (!psphotChoosePSF (config, view)) { + psLogMsg ("psphot", 3, "failure to construct a psf model"); + return psphotReadoutCleanup (config, view); + } + + // measure aperture photometry corrections +# if 0 + if (!psphotApResid (config, view)) { + psLogMsg ("psphot", 3, "failed on psphotApResid"); + return psphotReadoutCleanup (config, view); + } +# endif + + return psphotReadoutCleanup (config, view); +} Index: branches/tap_branches/psphot/src/psphotMakeResiduals.c =================================================================== --- branches/tap_branches/psphot/src/psphotMakeResiduals.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotMakeResiduals.c (revision 27838) @@ -305,4 +305,7 @@ if (fabs(resid->Ro->data.F32[oy][ox]) < pixelSN*dRo/sqrt(nKeep)) { resid->mask->data.PM_TYPE_RESID_MASK_DATA[oy][ox] = 1; + resid->Ro->data.F32[oy][ox] = 0.0; + resid->Rx->data.F32[oy][ox] = 0.0; + resid->Ry->data.F32[oy][ox] = 0.0; } } Index: branches/tap_branches/psphot/src/psphotMaskReadout.c =================================================================== --- branches/tap_branches/psphot/src/psphotMaskReadout.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotMaskReadout.c (revision 27838) @@ -1,12 +1,40 @@ # include "psphotInternal.h" +bool psphotSetMaskAndVariance (pmConfig *config, const pmFPAview *view) { + + bool status = false; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + + // Generate the mask and weight images, including the user-defined analysis region of interest + if (!psphotSetMaskAndVarianceReadout (config, view, "PSPHOT.INPUT", i, recipe)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to generate mask for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + // generate mask and variance if not defined, additional mask for restricted subregion -bool psphotSetMaskAndVariance (pmConfig *config, pmReadout *readout, psMetadata *recipe) { +bool psphotSetMaskAndVarianceReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe) { bool status; - // ** Interpret the mask values: - // XXX drop the write to recipe and move config into psphotRoughClass? - // XXX alternatively, define a function to set the psphot recipe masks + pmFPAfile *file = pmFPAfileSelectSingle(config->files, "PSPHOT.INPUT", index); // File of interest + psAssert (file, "missing file?"); + + // find the currently selected readout + pmReadout *readout = pmFPAviewThisReadout (view, file->fpa); + psAssert (readout, "missing readout?"); + + // save maskSat and maskBad on the psphot recipe (mostly for psphotRoughClass) psImageMaskType maskSat = pmConfigMaskGet("SAT", config); // Mask value for saturated pixels psMetadataAddImageMask (recipe, PS_LIST_TAIL, "MASK.SAT", PS_META_REPLACE, "user-defined mask", maskSat); @@ -14,5 +42,5 @@ psImageMaskType maskBad = pmConfigMaskGet("LOW", config); // Mask value for low pixels if (!maskBad) { - // XXX: for backward compatability look up old name + // for backward compatability look up old name maskBad = pmConfigMaskGet("BAD", config); } @@ -73,4 +101,7 @@ } + // display the image, weight, mask (ch 1,2,3) + psphotVisualShowImage (readout); + return true; } Index: branches/tap_branches/psphot/src/psphotMergeSources.c =================================================================== --- branches/tap_branches/psphot/src/psphotMergeSources.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotMergeSources.c (revision 27838) @@ -5,57 +5,170 @@ PM_SOURCE_MODE_CR_LIMIT | PM_SOURCE_MODE_EXT_LIMIT) // Mask to apply for PSF sources +// for now, let's store the detections on the readout->analysis for each readout +bool psphotMergeSources (pmConfig *config, const pmFPAview *view) +{ + bool status = true; + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (!psphotMergeSourcesReadout (config, view, "PSPHOT.INPUT", i)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to merge sources for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + // add newly selected sources to the existing list of sources -bool psphotMergeSources (psArray *oldSources, psArray *newSources) { +bool psphotMergeSourcesReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index) { + + bool status; + + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *newSources = detections->newSources; + psAssert (newSources, "missing sources?"); + + // XXX TEST: + if (detections->allSources) { + psphotMaskCosmicRayFootprintCheck(detections->allSources); + } + if (detections->newSources) { + psphotMaskCosmicRayFootprintCheck(detections->newSources); + } + + if (!detections->allSources) { + detections->allSources = psArrayAllocEmpty(newSources->n); + } + psArray *allSources = detections->allSources; for (int i = 0; i < newSources->n; i++) { pmSource *source = newSources->data[i]; - psArrayAdd (oldSources, 100, source); - } - return true; -} - -// merge the externally supplied sources with the existing sources. mark them as having -// mode PM_SOURCE_MODE_EXTERNAL -bool psphotLoadExtSources (pmConfig *config, const pmFPAview *view, psArray *sources) { - - // find the currently selected readout - pmReadout *readout = pmFPAfileThisReadout (config->files, view, "PSPHOT.INPUT.CMF"); - if (!readout) { - psLogMsg ("psphot", 3, "no external sources supplied"); - return true; - } - - psArray *extSources = psMetadataLookupPtr (NULL, readout->analysis, "PSPHOT.SOURCES"); - if (!extSources) { + psArrayAdd (allSources, 100, source); + } + + psFree (detections->newSources); + detections->newSources = NULL; + + return true; +} + +// Merge the externally supplied sources with the existing sources. Mark them as having mode +// PM_SOURCE_MODE_EXTERNAL. + +// XXX This function needs to be updated to loop over set of input files. At the moment, we +// only expect a single entry for PSPHOT.INPUT.CMF and PSPHOT.SOURCES.TEXT, so we can only +// associate input sources with a single entry for PSPHOT.INPUT +bool psphotLoadExtSources (pmConfig *config, const pmFPAview *view) { + + bool status; + pmDetections *extCMF = NULL; + psArray *extSourcesTXT = NULL; + int index = 0; + + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, "PSPHOT.INPUT", index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + if (!detections) { + detections = pmDetectionsAlloc(); + detections->newSources = psArrayAllocEmpty (100); + // save detections on the readout->analysis + if (!psMetadataAddPtr (readout->analysis, PS_LIST_TAIL, "PSPHOT.DETECTIONS", PS_META_REPLACE | PS_DATA_UNKNOWN, "psphot detections", detections)) { + psError (PSPHOT_ERR_CONFIG, false, "problem saving detections on readout"); + return false; + } + } else { + psMemIncrRefCounter(detections); // so we can free the detections below + } + + psArray *sources = detections->newSources; + psAssert (sources, "missing sources?"); + + // load data from input CMF file: + { + pmReadout *readoutCMF = pmFPAfileThisReadout (config->files, view, "PSPHOT.INPUT.CMF"); + if (!readoutCMF) goto loadTXT; + + extCMF = psMetadataLookupPtr (NULL, readoutCMF->analysis, "PSPHOT.DETECTIONS"); + if (extCMF) { + for (int i = 0; i < extCMF->allSources->n; i++) { + pmSource *source = extCMF->allSources->data[i]; + source->mode |= PM_SOURCE_MODE_EXTERNAL; + + // the supplied peak flux needs to be re-normalized + source->peak->flux = 1.0; + source->peak->value = 1.0; + + // drop the loaded source modelPSF + psFree (source->modelPSF); + source->modelPSF = NULL; + source->imageID = index; + + psArrayAdd (detections->newSources, 100, source); + } + } + } + +loadTXT: + + // load data from input TXT file: + { + pmChip *chipTXT = pmFPAfileThisChip (config->files, view, "PSPHOT.INPUT"); + if (!chipTXT) goto finish; + + extSourcesTXT = psMetadataLookupPtr (NULL, chipTXT->analysis, "PSPHOT.SOURCES.TEXT"); + if (extSourcesTXT) { + for (int i = 0; i < extSourcesTXT->n; i++) { + pmSource *source = extSourcesTXT->data[i]; + source->mode |= PM_SOURCE_MODE_EXTERNAL; + + // drop the loaded source modelPSF + psFree (source->modelPSF); + source->modelPSF = NULL; + source->imageID = index; + + psArrayAdd (detections->newSources, 100, source); + } + } + } + +finish: + + psFree (detections); + + if (!extCMF && !extSourcesTXT) { psLogMsg ("psphot", 3, "no external sources for this readout"); return true; } - for (int i = 0; i < extSources->n; i++) { - pmSource *source = extSources->data[i]; - source->mode |= PM_SOURCE_MODE_EXTERNAL; - pmModel *model = source->modelPSF; - - float xpos = model->params->data.F32[PM_PAR_XPOS]; - float ypos = model->params->data.F32[PM_PAR_YPOS]; - - source->peak = pmPeakAlloc(xpos, ypos, 1.0, PM_PEAK_LONE); - source->peak->xf = xpos; - source->peak->yf = ypos; - source->peak->flux = 1.0; - - // drop the loaded source modelPSF - psFree (source->modelPSF); - source->modelPSF = NULL; - } - - psphotMergeSources (sources, extSources); - psLogMsg ("psphot", 3, "%ld external sources merged to yield %ld total sources", extSources->n, sources->n); - + int nCMF = extCMF ? extCMF->allSources->n : 0; + int nTXT = extSourcesTXT ? extSourcesTXT->n : 0; + + psLogMsg ("psphot", 3, "%d external sources (%d cmf, %d text) merged to yield %ld total sources", + nCMF + nTXT, nCMF, nTXT, sources->n); return true; } // extract the input sources corresponding to this readout +// XXX this function needs to be updated to work with the new context of pshot inputs psArray *psphotLoadPSFSources (pmConfig *config, const pmFPAview *view) { + + bool status; // find the currently selected readout @@ -66,7 +179,14 @@ } - psArray *sources = psMetadataLookupPtr (NULL, readout->analysis, "PSPHOT.SOURCES"); + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + if (!detections) { + psLogMsg ("psphot", 3, "no psf sources for this readout"); + return NULL; + } + + psArray *sources = detections->allSources; if (!sources) { psLogMsg ("psphot", 3, "no psf sources for this readout"); + return NULL; } @@ -74,6 +194,45 @@ } +// this function is used to fix sources which were loaded externally, but have passed from +// psphotDetectionsFromSources to psphotSourceStats and are now stored on +// detections->newSources. +bool psphotRepairLoadedSources (pmConfig *config, const pmFPAview *view) { + + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, "PSPHOT.INPUT", 0); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (NULL, readout->analysis, "PSPHOT.DETECTIONS"); + if (!detections) { + psError(PSPHOT_ERR_CONFIG, false, "missing detections"); + return false; + } + + psArray *sources = detections->newSources; + psAssert (sources, "missing sources?"); + + // peak flux is wrong : set based on previous image + // use the peak measured in the moments analysis: + for (int i = 0; i < sources->n; i++) { + pmSource *source = sources->data[i]; + source->peak->flux = source->moments->Peak; + } + + return true; +} + // generate the detection structure for the supplied array of sources -pmDetections *psphotDetectionsFromSources (pmConfig *config, psArray *sources) { +// XXX this currently assumes there is a single input file +bool psphotDetectionsFromSources (pmConfig *config, const pmFPAview *view, psArray *sources) { + + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, "PSPHOT.INPUT", 0); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); pmDetections *detections = pmDetectionsAlloc(); @@ -88,5 +247,5 @@ float snMin = psMetadataLookupF32(NULL, recipe, "MOMENTS_SN_MIN"); if (!isfinite(snMin)) { - return NULL; + return false; } @@ -96,4 +255,11 @@ if (source->mode & PSF_SOURCE_MASK || !isfinite(source->psfMag)) { + continue; + } + + // use the existing peak information, otherwise generate a new peak + if (source->peak) { + source->peak->assigned = false; // So the moments will be measured + psArrayAdd (detections->peaks, 100, source->peak); continue; } @@ -108,9 +274,9 @@ peak->flux = flux; // this are being set wrong, but does it matter? - if (isfinite (source->errMag) && (source->errMag > 0.0)) { - peak->SN = 1.0 / source->errMag; - } else { - peak->SN = 0.0; - } + if (isfinite (source->errMag) && (source->errMag > 0.0)) { + peak->SN = 1.0 / source->errMag; + } else { + peak->SN = 0.0; + } psArrayAdd (detections->peaks, 100, peak); @@ -120,5 +286,11 @@ psLogMsg ("psphot", 3, "%ld PSF sources loaded", detections->peaks->n); - return detections; + // save detections on the readout->analysis + if (!psMetadataAddPtr (readout->analysis, PS_LIST_TAIL, "PSPHOT.DETECTIONS", PS_META_REPLACE | PS_DATA_UNKNOWN, "psphot detectinos", detections)) { + psError (PSPHOT_ERR_CONFIG, false, "problem saving detections on readout"); + return false; + } + psFree (detections); + return true; } @@ -149,9 +321,9 @@ peak->flux = flux; // this are being set wrong, but does it matter? - if (isfinite (source->errMag) && (source->errMag > 0.0)) { - peak->SN = 1.0 / source->errMag; - } else { - peak->SN = 0.0; - } + if (isfinite (source->errMag) && (source->errMag > 0.0)) { + peak->SN = 1.0 / source->errMag; + } else { + peak->SN = 0.0; + } source->peak = peak; @@ -162,2 +334,67 @@ return true; } + +bool psphotCheckExtSources (pmConfig *config, const pmFPAview *view) { + + bool status; + + psMetadata *recipe = psMetadataLookupPtr (NULL, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe"); + + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, "PSPHOT.INPUT", 0); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + // XXX allSources of newSources? + psArray *sources = detections->allSources; + psAssert (sources, "missing sources?"); + + if (sources->n) { + // the user wants to make the psf from these stars; define them as psf stars: + for (int i = 0; i < sources->n; i++) { + pmSource *source = sources->data[i]; + source->mode |= PM_SOURCE_MODE_PSFSTAR; + } + // force psphotChoosePSF to use all loaded sources + psMetadataAddS32 (recipe, PS_LIST_TAIL, "PSF_MAX_NSTARS", PS_META_REPLACE, "max number of sources for PSF model", sources->n); + + // measure stats of externally specified sources + if (!psphotSourceStatsUpdate (sources, config, readout)) { + psError(PSPHOT_ERR_CONFIG, false, "failure to measure stats of existing sources"); + return false; + } + } else { + + // find the detections (by peak and/or footprint) in the image. + if (!psphotFindDetections (config, view, true)) { + psError(PSPHOT_ERR_CONFIG, false, "unable to find detections in this image"); + return psphotReadoutCleanup (config, view); + } + + // construct sources and measure basic stats + psphotSourceStats (config, view, true); + + // find blended neighbors of very saturated stars + psphotDeblendSatstars (config, view); + + // mark blended peaks PS_SOURCE_BLEND + if (!psphotBasicDeblend (config, view)) { + psLogMsg ("psphot", 3, "failed on deblend analysis"); + return psphotReadoutCleanup (config, view); + } + + // classify sources based on moments, brightness + if (!psphotRoughClass (config, view)) { + psLogMsg ("psphot", 3, "failed to find a valid PSF clump for image"); + return psphotReadoutCleanup (config, view); + } + } + + return true; +} Index: branches/tap_branches/psphot/src/psphotModelBackground.c =================================================================== --- branches/tap_branches/psphot/src/psphotModelBackground.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotModelBackground.c (revision 27838) @@ -5,5 +5,5 @@ // Determine the binning characteristics for the background model // Sets the ruff image size and skip based on recipe values for the binning -static psImageBinning *backgroundBinning(const psImage *image, // Image for which to generate a bg model +psImageBinning *psphotBackgroundBinning(const psImage *image, // Image for which to generate a bg model const pmConfig *config // Configuration ) @@ -28,12 +28,20 @@ } + // Generate the background model -static bool backgroundModel(psImage *model, // Model image - psImage *modelStdev, // Model stdev image - psMetadata *analysis, // Analysis metadata for outputs - pmReadout *readout, // Readout for which to generate a background model - psImageBinning *binning, // Binning parameters - const pmConfig *config // Configuration - ) +// generate the median in NxN boxes, clipping heavily +// linear interpolation to generate full-scale model +// +// NOTE that the 'analysis' metedata pass in here is used to store the binning information. +// This may be the analysis for this readout, but it may be the analysis for the pmFPAfile +// corresponding to the model. Other information about the background model is saved on the +// readout->analysis +static bool psphotModelBackgroundReadout(psImage *model, // Model image + psImage *modelStdev, // Model stdev image + psMetadata *analysis, // Analysis metadata for outputs + pmReadout *readout, // Readout for which to generate a background model + psImageBinning *binning, // Binning parameters + const pmConfig *config // Configuration + ) { psTimerStart ("psphot.background"); @@ -137,6 +145,5 @@ // we save the binning structure for use in psphotMagnitudes - psMetadataAddPtr(analysis, PS_LIST_TAIL, "PSPHOT.BACKGROUND.BINNING", - PS_DATA_UNKNOWN | PS_META_REPLACE, "Background binning", binning); + psMetadataAddPtr(analysis, PS_LIST_TAIL, "PSPHOT.BACKGROUND.BINNING", PS_DATA_UNKNOWN | PS_META_REPLACE, "Background binning", binning); psF32 **modelData = model->data.F32; @@ -144,4 +151,7 @@ // XXXX we can thread this here by running blocks in parallel + + int nFailures = 0; + psImageBackgroundInit(); // measure clipped median for subimages @@ -211,7 +221,10 @@ stats->options = PS_STAT_ROBUST_MEDIAN | PS_STAT_ROBUST_STDEV; if (!psImageBackground(stats, &sample, subset, submask, maskVal, rng)) { - psLogMsg ("psphot", PS_LOG_WARN, "Failed to estimate background using ROBUST_MEDIAN for " - "(%dx%d, (row0,col0) = (%d,%d)", - subset->numRows, subset->numCols, subset->row0, subset->col0); + if ((nFailures < 3) || (nFailures % 100 == 0)) { + psLogMsg ("psphot", PS_LOG_WARN, "Failed to estimate background using ROBUST_MEDIAN for " + "(%dx%d, (row0,col0) = (%d,%d)", + subset->numRows, subset->numCols, subset->row0, subset->col0); + } + nFailures ++; modelData[iy][ix] = modelStdevData[iy][ix] = NAN; } else { @@ -232,4 +245,8 @@ psFree (submask); } + } + + if (nFailures) { + psLogMsg ("psphot", PS_LOG_WARN, "Failed to estimate background for %d of %d subimages", nFailures, (model->numRows*model->numCols)); } @@ -293,6 +310,6 @@ psLogMsg ("psphot", PS_LOG_INFO, "built background image: %f sec\n", psTimerMark ("psphot.background")); - psMetadataAddF32(recipe, PS_LIST_TAIL, "SKY_MEAN", PS_META_REPLACE, "sky mean", Value); - psMetadataAddF32(recipe, PS_LIST_TAIL, "SKY_STDEV", PS_META_REPLACE, "sky stdev", ValueStdev); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "SKY_MEAN", PS_META_REPLACE, "sky mean", Value); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "SKY_STDEV", PS_META_REPLACE, "sky stdev", ValueStdev); psLogMsg ("psphot", PS_LOG_INFO, "image sky : mean %f stdev %f", Value, ValueStdev); @@ -303,16 +320,10 @@ PS_STAT_MAX); psImageStats (statsBck, model, NULL, 0); - psMetadataAddF32 (recipe, PS_LIST_TAIL, "MSKY_MN", - PS_META_REPLACE, "sky model mean", statsBck->sampleMean); - psMetadataAddF32 (recipe, PS_LIST_TAIL, "MSKY_SIG", - PS_META_REPLACE, "sky model stdev", statsBck->sampleStdev); - psMetadataAddF32 (recipe, PS_LIST_TAIL, "MSKY_MAX", - PS_META_REPLACE, "sky model maximum value", statsBck->max); - psMetadataAddF32 (recipe, PS_LIST_TAIL, "MSKY_MIN", - PS_META_REPLACE, "sky model minimum value", statsBck->min); - psMetadataAddS32 (recipe, PS_LIST_TAIL, "MSKY_NX", - PS_META_REPLACE, "sky model size (x)", model->numCols); - psMetadataAddS32 (recipe, PS_LIST_TAIL, "MSKY_NY", - PS_META_REPLACE, "sky model size (y)", model->numRows); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "MSKY_MN", PS_META_REPLACE, "sky model mean", statsBck->sampleMean); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "MSKY_SIG", PS_META_REPLACE, "sky model stdev", statsBck->sampleStdev); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "MSKY_MAX", PS_META_REPLACE, "sky model maximum value", statsBck->max); + psMetadataAddF32 (readout->analysis, PS_LIST_TAIL, "MSKY_MIN", PS_META_REPLACE, "sky model minimum value", statsBck->min); + psMetadataAddS32 (readout->analysis, PS_LIST_TAIL, "MSKY_NX", PS_META_REPLACE, "sky model size (x)", model->numCols); + psMetadataAddS32 (readout->analysis, PS_LIST_TAIL, "MSKY_NY", PS_META_REPLACE, "sky model size (y)", model->numRows); psLogMsg ("psphot", PS_LOG_INFO, "background sky : min %f mean %f max %f stdev %f", statsBck->min, statsBck->sampleMean, statsBck->max, statsBck->sampleStdev); @@ -327,17 +338,16 @@ } - - -psImage *psphotBackgroundModel(pmReadout *ro, const pmConfig *config) +// generate a background model for a single readout. do not save an associated pmFPAfile for possible output +psImage *psphotModelBackgroundReadoutNoFile(pmReadout *readout, const pmConfig *config) { - PM_ASSERT_READOUT_NON_NULL(ro, NULL); - PM_ASSERT_READOUT_IMAGE(ro, NULL); + PM_ASSERT_READOUT_NON_NULL(readout, NULL); + PM_ASSERT_READOUT_IMAGE(readout, NULL); PS_ASSERT_PTR_NON_NULL(config, NULL); - psImageBinning *binning = backgroundBinning(ro->image, config); // Image binning parameters + psImageBinning *binning = psphotBackgroundBinning(readout->image, config); // Image binning parameters psImage *model = psImageAlloc(binning->nXruff, binning->nYruff, PS_TYPE_F32); // Background model psImage *modelStdev = psImageAlloc(binning->nXruff, binning->nYruff, PS_TYPE_F32); // Standard deviation - if (!backgroundModel(model, modelStdev, ro->analysis, ro, binning, config)) { + if (!psphotModelBackgroundReadout(model, modelStdev, readout->analysis, readout, binning, config)) { psFree(model); psFree(modelStdev); @@ -345,27 +355,24 @@ return NULL; } - psFree(modelStdev); - return model; } - -// generate the median in NxN boxes, clipping heavily -// linear interpolation to generate full-scale model -bool psphotModelBackground (pmConfig *config, const pmFPAview *view, const char *filename) +// generate a background model for readout number index; save an associated pmFPAfile for possible output +bool psphotModelBackgroundReadoutFileIndex (pmConfig *config, const pmFPAview *view, const char *filename, int index) { - bool status = true; - // find the currently selected readout - pmFPAfile *file = psMetadataLookupPtr (&status, config->files, filename); + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + pmFPA *inFPA = file->fpa; pmReadout *readout = pmFPAviewThisReadout(view, inFPA); - - psImageBinning *binning = backgroundBinning(readout->image, config); // Image binning parameters - pmReadout *model = pmFPAGenerateReadout(config, view, "PSPHOT.BACKMDL", inFPA, binning); - pmReadout *modelStdev = pmFPAGenerateReadout(config, view, "PSPHOT.BACKMDL.STDEV", inFPA, binning); - - if (!backgroundModel(model->image, modelStdev->image, model->analysis, readout, binning, config)) { + psAssert (readout, "missing readout?"); + + psImageBinning *binning = psphotBackgroundBinning(readout->image, config); // Image binning parameters + pmReadout *model = pmFPAGenerateReadout(config, view, "PSPHOT.BACKMDL", inFPA, binning, index); + pmReadout *modelStdev = pmFPAGenerateReadout(config, view, "PSPHOT.BACKMDL.STDEV", inFPA, binning, index); + + if (!psphotModelBackgroundReadout(model->image, modelStdev->image, model->analysis, readout, binning, config)) { psError(PS_ERR_UNKNOWN, false, "Unable to generate background model"); return false; @@ -375,2 +382,20 @@ return true; } + +// XXX supply filename or keep PSPHOT.INPUT fixed? +bool psphotModelBackground (pmConfig *config, const pmFPAview *view) +{ + bool status = false; + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (!psphotModelBackgroundReadoutFileIndex(config, view, "PSPHOT.INPUT", i)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to model background for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} Index: branches/tap_branches/psphot/src/psphotModelTest.c =================================================================== --- branches/tap_branches/psphot/src/psphotModelTest.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotModelTest.c (revision 27838) @@ -139,5 +139,5 @@ // get the source moments - status = pmSourceMoments (source, mRADIUS, 0.0, 1.0); + status = pmSourceMoments (source, mRADIUS, 0.0, 1.0, maskVal); if (!status) psAbort("psSourceMoments error"); source->peak->value = source->moments->Peak; Index: branches/tap_branches/psphot/src/psphotMomentsStudy.c =================================================================== --- branches/tap_branches/psphot/src/psphotMomentsStudy.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotMomentsStudy.c (revision 27838) @@ -148,5 +148,5 @@ pmSourceAdd(source, PM_MODEL_OP_FUNC, 0); - pmSourceMoments (source, 32.0, sigWindow, 0.0); + pmSourceMoments (source, 32.0, sigWindow, 0.0, 0xffff); // fprintf (stderr, "sigOut : %f\n", sqrt(source->moments->Mxx)); Index: branches/tap_branches/psphot/src/psphotOutput.c =================================================================== --- branches/tap_branches/psphot/src/psphotOutput.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotOutput.c (revision 27838) @@ -126,16 +126,33 @@ } -bool psphotAddPhotcode (psMetadata *recipe, pmConfig *config, const pmFPAview *view, const char *filerule) { - - bool status; - - pmFPAfile *input = psMetadataLookupPtr (&status, config->files, filerule); - PS_ASSERT (status, false); - - // determine PHOTCODE from fpa & view, overwrite in recipe - char *photcode = pmConceptsPhotcodeForView (input, view); +bool psphotAddPhotcode (pmConfig *config, const pmFPAview *view) { + + bool status = false; + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (!psphotAddPhotcodeReadout (config, view, "PSPHOT.INPUT", i)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to add photcode to PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + +bool psphotAddPhotcodeReadout (pmConfig *config, const pmFPAview *view, const char *filerule, int index) { + + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filerule, index); // File of interest + PS_ASSERT (file, false); + + pmReadout *readout = pmFPAviewThisReadout (view, file->fpa); + + // determine PHOTCODE from fpa & view, overwrite in readout->analysis + char *photcode = pmConceptsPhotcodeForView (file, view); PS_ASSERT (photcode, false); - psMetadataAddStr (recipe, PS_LIST_TAIL, "PHOTCODE", PS_META_REPLACE, "photcode from FPA concepts", photcode); + psMetadataAddStr (readout->analysis, PS_LIST_TAIL, "PHOTCODE", PS_META_REPLACE, "photcode from FPA concepts", photcode); psLogMsg ("psphot", 3, "PHOTCODE is %s", photcode); @@ -144,5 +161,5 @@ } -bool psphotSetHeaderNstars (psMetadata *recipe, psArray *sources) { +bool psphotSetHeaderNstars (psMetadata *header, psArray *sources) { int nSrc = 0; @@ -173,93 +190,93 @@ } - psMetadataAddS32 (recipe, PS_LIST_TAIL, "NSTARS", PS_META_REPLACE, "Number of sources", nSrc); - psMetadataAddS32 (recipe, PS_LIST_TAIL, "NDET_EXT", PS_META_REPLACE, "Number of extended sources", nEXT); - psMetadataAddS32 (recipe, PS_LIST_TAIL, "NDET_CR", PS_META_REPLACE, "Number of cosmic rays", nCR); - return true; -} - -// these values are saved in an output header stub - they are added to either the -// PHU header (CMP) or the MEF table header (CMF) -psMetadata *psphotDefineHeader (psMetadata *recipe) { + psMetadataAddS32 (header, PS_LIST_TAIL, "NSTARS", PS_META_REPLACE, "Number of sources", nSrc); + psMetadataAddS32 (header, PS_LIST_TAIL, "NDET_EXT", PS_META_REPLACE, "Number of extended sources", nEXT); + psMetadataAddS32 (header, PS_LIST_TAIL, "NDET_CR", PS_META_REPLACE, "Number of cosmic rays", nCR); + return true; +} + +// these values are saved in an output header stub - they are added to either the PHU header +// (CMP) or the MEF table header (CMF). before the header is created, each readout has these +// values stored on readout->analysis +psMetadata *psphotDefineHeader (psMetadata *analysis) { + + bool status = true; psMetadata *header = psMetadataAlloc (); // write necessary information to output header - psMetadataItemSupplement (header, recipe, "ZERO_PT"); - psMetadataItemSupplement (header, recipe, "PHOTCODE"); - - psMetadataItemSupplement (header, recipe, "APMIFIT"); - psMetadataItemSupplement (header, recipe, "DAPMIFIT"); - psMetadataItemSupplement (header, recipe, "NAPMIFIT"); + psMetadataItemSupplement (&status, header, analysis, "ZERO_PT"); + psMetadataItemSupplement (&status, header, analysis, "PHOTCODE"); + + psMetadataItemSupplement (&status, header, analysis, "APMIFIT"); + psMetadataItemSupplement (&status, header, analysis, "DAPMIFIT"); + psMetadataItemSupplement (&status, header, analysis, "NAPMIFIT"); // PSF model parameters (shape values for image center) - psMetadataItemSupplement (header, recipe, "NPSFSTAR"); - psMetadataItemSupplement (header, recipe, "APLOSS"); - - psMetadataItemSupplement (header, recipe, "FWHM_MAJ"); - psMetadataItemSupplement (header, recipe, "FW_MJ_SG"); - psMetadataItemSupplement (header, recipe, "FW_MJ_LQ"); - psMetadataItemSupplement (header, recipe, "FW_MJ_UQ"); - - psMetadataItemSupplement (header, recipe, "FWHM_MIN"); - psMetadataItemSupplement (header, recipe, "FW_MN_SG"); - psMetadataItemSupplement (header, recipe, "FW_MN_LQ"); - psMetadataItemSupplement (header, recipe, "FW_MN_UQ"); - - psMetadataItemSupplement (header, recipe, "ANGLE"); + psMetadataItemSupplement (&status, header, analysis, "NPSFSTAR"); + psMetadataItemSupplement (&status, header, analysis, "APLOSS"); + + psMetadataItemSupplement (&status, header, analysis, "FWHM_MAJ"); + psMetadataItemSupplement (&status, header, analysis, "FW_MJ_SG"); + psMetadataItemSupplement (&status, header, analysis, "FW_MJ_LQ"); + psMetadataItemSupplement (&status, header, analysis, "FW_MJ_UQ"); + + psMetadataItemSupplement (&status, header, analysis, "FWHM_MIN"); + psMetadataItemSupplement (&status, header, analysis, "FW_MN_SG"); + psMetadataItemSupplement (&status, header, analysis, "FW_MN_LQ"); + psMetadataItemSupplement (&status, header, analysis, "FW_MN_UQ"); + + psMetadataItemSupplement (&status, header, analysis, "ANGLE"); // Image Quality measurements - psMetadataItemSupplement (header, recipe, "IQ_NSTAR"); - - psMetadataItemSupplement (header, recipe, "IQ_FW1"); - psMetadataItemSupplement (header, recipe, "IQ_FW1_E"); - psMetadataItemSupplement (header, recipe, "IQ_FW2"); - psMetadataItemSupplement (header, recipe, "IQ_FW2_E"); - - psMetadataItemSupplement (header, recipe, "IQ_M2"); - psMetadataItemSupplement (header, recipe, "IQ_M2_ER"); - psMetadataItemSupplement (header, recipe, "IQ_M2_LQ"); - psMetadataItemSupplement (header, recipe, "IQ_M2_UQ"); - - psMetadataItemSupplement (header, recipe, "IQ_M2C"); - psMetadataItemSupplement (header, recipe, "IQ_M2C_E"); - psMetadataItemSupplement (header, recipe, "IQ_M2C_L"); - psMetadataItemSupplement (header, recipe, "IQ_M2C_U"); - - psMetadataItemSupplement (header, recipe, "IQ_M2S"); - psMetadataItemSupplement (header, recipe, "IQ_M2S_E"); - psMetadataItemSupplement (header, recipe, "IQ_M2S_L"); - psMetadataItemSupplement (header, recipe, "IQ_M2S_U"); - - psMetadataItemSupplement (header, recipe, "IQ_M3"); - psMetadataItemSupplement (header, recipe, "IQ_M3_ER"); - psMetadataItemSupplement (header, recipe, "IQ_M3_LQ"); - psMetadataItemSupplement (header, recipe, "IQ_M3_UQ"); - - psMetadataItemSupplement (header, recipe, "IQ_M4"); - psMetadataItemSupplement (header, recipe, "IQ_M4_ER"); - psMetadataItemSupplement (header, recipe, "IQ_M4_LQ"); - psMetadataItemSupplement (header, recipe, "IQ_M4_UQ"); + psMetadataItemSupplement (&status, header, analysis, "IQ_NSTAR"); + + psMetadataItemSupplement (&status, header, analysis, "IQ_FW1"); + psMetadataItemSupplement (&status, header, analysis, "IQ_FW1_E"); + psMetadataItemSupplement (&status, header, analysis, "IQ_FW2"); + psMetadataItemSupplement (&status, header, analysis, "IQ_FW2_E"); + + psMetadataItemSupplement (&status, header, analysis, "IQ_M2"); + psMetadataItemSupplement (&status, header, analysis, "IQ_M2_ER"); + psMetadataItemSupplement (&status, header, analysis, "IQ_M2_LQ"); + psMetadataItemSupplement (&status, header, analysis, "IQ_M2_UQ"); + + psMetadataItemSupplement (&status, header, analysis, "IQ_M2C"); + psMetadataItemSupplement (&status, header, analysis, "IQ_M2C_E"); + psMetadataItemSupplement (&status, header, analysis, "IQ_M2C_L"); + psMetadataItemSupplement (&status, header, analysis, "IQ_M2C_U"); + + psMetadataItemSupplement (&status, header, analysis, "IQ_M2S"); + psMetadataItemSupplement (&status, header, analysis, "IQ_M2S_E"); + psMetadataItemSupplement (&status, header, analysis, "IQ_M2S_L"); + psMetadataItemSupplement (&status, header, analysis, "IQ_M2S_U"); + + psMetadataItemSupplement (&status, header, analysis, "IQ_M3"); + psMetadataItemSupplement (&status, header, analysis, "IQ_M3_ER"); + psMetadataItemSupplement (&status, header, analysis, "IQ_M3_LQ"); + psMetadataItemSupplement (&status, header, analysis, "IQ_M3_UQ"); + + psMetadataItemSupplement (&status, header, analysis, "IQ_M4"); + psMetadataItemSupplement (&status, header, analysis, "IQ_M4_ER"); + psMetadataItemSupplement (&status, header, analysis, "IQ_M4_LQ"); + psMetadataItemSupplement (&status, header, analysis, "IQ_M4_UQ"); // XXX these need to be defined from elsewhere psMetadataAdd (header, PS_LIST_TAIL, "FSATUR", PS_DATA_F32 | PS_META_REPLACE, "SATURATION MAG", 0.0); psMetadataAdd (header, PS_LIST_TAIL, "FLIMIT", PS_DATA_F32 | PS_META_REPLACE, "COMPLETENESS MAG", 0.0); - psMetadataItemSupplement (header, recipe, "NSTARS"); - - psMetadataItemSupplement (header, recipe, "NDET_EXT"); - psMetadataItemSupplement (header, recipe, "NDET_CR"); + psMetadataItemSupplement (&status, header, analysis, "NSTARS"); + + psMetadataItemSupplement (&status, header, analysis, "NDET_EXT"); + psMetadataItemSupplement (&status, header, analysis, "NDET_CR"); // sky background model statistics - psMetadataItemSupplement (header, recipe, "MSKY_MN"); - psMetadataItemSupplement (header, recipe, "MSKY_SIG"); - psMetadataItemSupplement (header, recipe, "MSKY_MIN"); - psMetadataItemSupplement (header, recipe, "MSKY_MAX"); - psMetadataItemSupplement (header, recipe, "MSKY_NX"); - psMetadataItemSupplement (header, recipe, "MSKY_NY"); + psMetadataItemSupplement (&status, header, analysis, "MSKY_MN"); + psMetadataItemSupplement (&status, header, analysis, "MSKY_SIG"); + psMetadataItemSupplement (&status, header, analysis, "MSKY_MIN"); + psMetadataItemSupplement (&status, header, analysis, "MSKY_MAX"); + psMetadataItemSupplement (&status, header, analysis, "MSKY_NX"); + psMetadataItemSupplement (&status, header, analysis, "MSKY_NY"); psMetadataAddF32 (header, PS_LIST_TAIL, "DT_PHOT", PS_META_REPLACE, "elapsed psphot time", psTimerMark ("psphotReadout")); - - // XXX : don't require any of these about values to exist - psErrorClear (); return header; Index: branches/tap_branches/psphot/src/psphotPSFConvModel.c =================================================================== --- branches/tap_branches/psphot/src/psphotPSFConvModel.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotPSFConvModel.c (revision 27838) @@ -39,5 +39,5 @@ // adjust the pixels based on the footprint float radius = psphotSetRadiusEXT (readout, source, markVal); - if (!pmSourceMoments (source, radius, 0.0, 0.0)) return false; + if (!pmSourceMoments (source, radius, 0.0, 0.0, maskVal)) return false; // XXX test : modify the Io, SXX, SYY terms based on the psf SXX, SYY terms: Index: branches/tap_branches/psphot/src/psphotParseCamera.c =================================================================== --- branches/tap_branches/psphot/src/psphotParseCamera.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotParseCamera.c (revision 27838) @@ -39,4 +39,6 @@ return NULL; } + // specify the number of psphot input images + psMetadataAddS32 (config->arguments, PS_LIST_TAIL, "PSPHOT.INPUT.NUM", PS_META_REPLACE, "number of inputs", 1); // define the additional input/output files associated with psphot Index: branches/tap_branches/psphot/src/psphotPetrosian.c =================================================================== --- branches/tap_branches/psphot/src/psphotPetrosian.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotPetrosian.c (revision 27838) @@ -7,5 +7,5 @@ psAssert (source->extpars, "need to run psphotRadialProfile first"); - psAssert (source->extpars->profile, "need to run psphotRadialProfile first"); + psAssert (source->extpars->ellipticalFlux, "need to run psphotRadialProfile first"); // integrate the radial profile for radial bins defined for the petrosian measurement: @@ -24,8 +24,8 @@ psTrace ("psphot", 3, "source at %f,%f: petrosian radius: %f, flux: %f, axis ratio: %f, angle: %f", source->peak->xf, source->peak->yf, - source->extpars->petrosian_80->radius, - source->extpars->petrosian_80->flux, - source->extpars->profile->axes.minor/source->extpars->profile->axes.major, - source->extpars->profile->axes.theta*PS_DEG_RAD); + source->extpars->petrosianRadius, + source->extpars->petrosianFlux, + source->extpars->axes.minor/source->extpars->axes.major, + source->extpars->axes.theta*PS_DEG_RAD); return true; Index: branches/tap_branches/psphot/src/psphotPetrosianRadialBins.c =================================================================== --- branches/tap_branches/psphot/src/psphotPetrosianRadialBins.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotPetrosianRadialBins.c (revision 27838) @@ -1,3 +1,4 @@ # include "psphotInternal.h" +float InterpolateValues (float X0, float Y0, float X1, float Y1, float X); // convert the flux vs elliptical radius to annular bins @@ -5,5 +6,5 @@ // we are guaranteed to be limited by either the seeing (1 - few pixels) or by the pixels // themselves. this function does not attempt to measure the radial profiles accurately -// for radii that are smaller than a minimum (currently 1.0 pixels). +// for radii that are smaller than ~2 pixels // for small radii, we are measuring the mean surface brightness in non-overlapping radial @@ -13,16 +14,27 @@ // track the non-overlapping radius values. +// Photo interpolates the image of interest to place the peak on the center of the central +// pixel, and then uses the exact fractions of the pixels in each of the first few annuli. +// Seems like a reasonable thing, but is there any significance to the difference? + // XXX move the resulting elements from profile to extpars->petrosian? bool psphotPetrosianRadialBins (pmSource *source, float radiusMax, float skynoise) { - pmSourceRadialProfile *profile = source->extpars->profile; - - float skyModelErrorSQ = PS_SQR(skynoise); - - psVector *radius = profile->radiusElliptical; - psVector *flux = profile->fluxElliptical; + psAssert (source, "missing source"); + psAssert (source->extpars, "missing extpars"); + psAssert (source->extpars->ellipticalFlux, "missing ellipticalFlux"); + + psVector *radius = source->extpars->ellipticalFlux->radiusElliptical; + psVector *flux = source->extpars->ellipticalFlux->fluxElliptical; // sort incoming vectors by radius pmSourceRadialProfileSortPair (radius, flux); + + if (!source->extpars->petProfile) { + source->extpars->petProfile = pmSourceRadialProfileAlloc(); + } + pmSourceRadialProfile *profile = source->extpars->petProfile; + + float skyModelErrorSQ = PS_SQR(skynoise); int nMax = radiusMax; @@ -107,8 +119,4 @@ psVector *values = psVectorAllocEmpty (flux->n, PS_TYPE_F32); psStats *stats = psStatsAlloc(PS_STAT_ROBUST_MEDIAN | PS_STAT_ROBUST_STDEV); - // psStats *stats = psStatsAlloc(PS_STAT_FITTED_MEAN_V4 | PS_STAT_FITTED_STDEV_V4); - - // integrate flux, radius for each of these bins. since flux is sorted by radius, - // we can do this fairly quickly bool done = false; @@ -133,9 +141,7 @@ dvalue = NAN; } - // binSB->data.F32[nOut] = stats->sampleMedian; + binSB->data.F32[nOut] = value; binSBstdev->data.F32[nOut] = sqrt(PS_SQR(dvalue) / values->n + skyModelErrorSQ); - // binSB->data.F32[nOut] = stats->fittedMean; - // binSBstdev->data.F32[nOut] = sqrt(PS_SQR(stats->fittedStdev) / values->n + skyModelErrorSQ); // error in the SB is the stdev per bin / sqrt (number of pixels) @@ -143,6 +149,5 @@ // residual flux, but the sky from the sky model) - psTrace ("psphot", 5, "%3d %5.1f %5.1f : %5.1f %5.2f\n", - nOut, radAlp->data.F32[nOut], radBet->data.F32[nOut], binSB->data.F32[nOut], binSBstdev->data.F32[nOut]); + psTrace ("psphot", 5, "%3d %5.1f %5.1f : %5.1f %5.2f\n", nOut, radAlp->data.F32[nOut], radBet->data.F32[nOut], binSB->data.F32[nOut], binSBstdev->data.F32[nOut]); nOut ++; @@ -163,4 +168,39 @@ // XXX I think this misses the last radial bin -- do we care? + // interpolate any bins that were empty (extrapolate to center if needed) + if (!isfinite(binSB->data.F32[0]) && !isfinite(binSB->data.F32[1])) { + psWarning ("center 2 bins of source at %f, %f are NAN, skipping this source", source->peak->xf, source->peak->yf); + // XXX raise a flag + psFree(binSB); + psFree(binSBstdev); + psFree(binRad); + psFree(binArea); + psFree(radMin); + psFree(radMax); + psFree(radAlp); + psFree(radBet); + psFree(values); + psFree(stats); + return false; + } + + // if center bin is empty assume same SB as next radius (probably true due to PSF) + if (!isfinite(binSB->data.F32[0])) { + binSB->data.F32[0] = binSB->data.F32[1]; + binSBstdev->data.F32[0] = binSBstdev->data.F32[1]; + } + + // interpolate any bins that were empty (if center if needed) + for (int i = 1; i < binSB->n - 1; i++) { + if (isfinite(binSB->data.F32[i])) continue; + binSB->data.F32[i] = InterpolateValues (binRad->data.F32[i-1], binSB->data.F32[i-1], binRad->data.F32[i+1], binSB->data.F32[i+1], binRad->data.F32[i]); + binSBstdev->data.F32[i] = InterpolateValues (binRad->data.F32[i-1], binSBstdev->data.F32[i-1], binRad->data.F32[i+1], binSBstdev->data.F32[i+1], binRad->data.F32[i]); + } + + psFree(profile->binSB); + psFree(profile->binSBstdev); + psFree(profile->radialBins); + psFree(profile->area); + // save the vectors profile->radialBins = binRad; Index: branches/tap_branches/psphot/src/psphotPetrosianStats.c =================================================================== --- branches/tap_branches/psphot/src/psphotPetrosianStats.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotPetrosianStats.c (revision 27838) @@ -10,8 +10,9 @@ bool psphotPetrosianStats (pmSource *source) { - pmSourceRadialProfile *profile = source->extpars->profile; - - float petRadius = NAN; - float petFlux = NAN; + psAssert (source, "missing source"); + psAssert (source->extpars, "missing extpars"); + psAssert (source->extpars->petProfile, "missing petProfile"); + + pmSourceRadialProfile *profile = source->extpars->petProfile; psVector *binSB = profile->binSB; @@ -28,4 +29,7 @@ psVector *areaSum = psVectorAllocEmpty(binSB->n, PS_TYPE_F32); + float petRadius = NAN; + float petFlux = NAN; + bool anyPetro = false; bool manyPetro = false; @@ -38,4 +42,6 @@ int lowestSignificantRadius = 0; float lowestSignificantRatio = 1.0; + + // find the Petrosian Radius and Petrosian Flux int nOut = 0; @@ -142,14 +148,48 @@ } - if (!source->extpars->petrosian_80) { - source->extpars->petrosian_80 = pmSourceExtendedFluxAlloc (); - } - pmSourceExtendedFlux *petrosian = source->extpars->petrosian_80; + // now measure the radii R90 and R50 where flux = 0.9 (or 0.5) * petFlux; + float flux90 = 0.9 * petFlux; + float flux50 = 0.5 * petFlux; + float R50 = NAN; + float R90 = NAN; + bool found50 = false; + bool found90 = false; + // XXX use bisection to do this faster: + for (int i = 0; !(found50 && found90) && i < refRadius->n; i++) { + if (!found50 && (fluxSum->data.F32[i] > flux50)) { + if (i == 0) { + psWarning ("does this case make any sense? (fluxSum[0] > flux50)"); + continue; + } else { + R50 = InterpolateValues (fluxSum->data.F32[i-1], refRadius->data.F32[i-1], fluxSum->data.F32[i], refRadius->data.F32[i], flux50); + found50 = true; + } + } + if (!found90 && (fluxSum->data.F32[i] > flux90)) { + if (i == 0) { + psWarning ("does this case make any sense? (fluxSum[0] > flux90)"); + continue; + } else { + R90 = InterpolateValues (fluxSum->data.F32[i-1], refRadius->data.F32[i-1], fluxSum->data.F32[i], refRadius->data.F32[i], flux90); + found90 = true; + } + } + } + // XXX save flags (anyPetro, manyPetro) - petrosian->radius = petRadius; - petrosian->flux = petFlux; - - // psphotPetrosianVisualStats (binRad, binSB, refRadius, meanSB, petRatio, petRatioErr, fluxSum, petRadius, PETROSIAN_RATIO, petFlux, apRadius); + source->extpars->petrosianRadius = petRadius; + source->extpars->petrosianFlux = petFlux; + source->extpars->petrosianR50 = R50; + source->extpars->petrosianR90 = R90; + + // XXX add the errors + source->extpars->petrosianRadiusErr = NAN; + source->extpars->petrosianFluxErr = NAN; + source->extpars->petrosianR50Err = NAN; + source->extpars->petrosianR90Err = NAN; + + fprintf (stderr, "source @ %f,%f\n", source->peak->xf, source->peak->yf); + psphotPetrosianVisualStats (binRad, binSB, refRadius, meanSB, petRatio, petRatioErr, fluxSum, petRadius, PETROSIAN_RATIO, petFlux, apRadius); psFree(fluxSum); Index: branches/tap_branches/psphot/src/psphotPetrosianVisual.c =================================================================== --- branches/tap_branches/psphot/src/psphotPetrosianVisual.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotPetrosianVisual.c (revision 27838) @@ -1,3 +1,4 @@ # include "psphotInternal.h" +# define FORCE_VISUAL 0 // this function displays representative images as the psphot analysis progresses: @@ -54,5 +55,5 @@ // return true; - if (!pmVisualIsVisual()) return true; + if (!FORCE_VISUAL && !pmVisualIsVisual()) return true; if (kapa2 == -1) { @@ -101,5 +102,5 @@ // return true; - if (!pmVisualIsVisual()) return true; + if (!FORCE_VISUAL && !pmVisualIsVisual()) return true; if (kapa == -1) { @@ -172,5 +173,143 @@ KapaSection section; - if (!pmVisualIsVisual()) return true; + if (!FORCE_VISUAL && !pmVisualIsVisual()) return true; + + if (kapa2 == -1) { + kapa2 = KapaOpenNamedSocket ("kapa", "psphot:stats"); + if (kapa2 == -1) { + fprintf (stderr, "failure to open kapa; visual mode disabled\n"); + pmVisualSetVisual(false); + return false; + } + } + + KapaClearPlots (kapa2); + KapaInitGraph (&graphdata); + KapaSetFont (kapa2, "courier", 14); + + // radius vs flux + // radius vs mean SB + // radius vs petRatio + + // *** section 1: radius vs mean SB + section.dx = 1.00; + section.dy = 0.33; + section.x = 0.00; + section.y = 0.00; + section.name = psStringCopy ("meanSB"); + KapaSetSection (kapa2, §ion); + psFree (section.name); + + graphdata.color = KapaColorByName ("black"); + pmVisualLimitsFromVectors (&graphdata, radBin, fluxBin); + KapaSetLimits (kapa2, &graphdata); + + KapaBox (kapa2, &graphdata); + KapaSendLabel (kapa2, "radius", KAPA_LABEL_XM); + KapaSendLabel (kapa2, "mean SB", KAPA_LABEL_YM); + + graphdata.color = KapaColorByName ("black"); + graphdata.style = 2; + graphdata.ptype = 0; + graphdata.size = 1.0; + KapaPrepPlot (kapa2, radBin->n, &graphdata); + KapaPlotVector (kapa2, radBin->n, radBin->data.F32, "x"); + KapaPlotVector (kapa2, radBin->n, fluxBin->data.F32, "y"); + + graphdata.color = KapaColorByName ("red"); + graphdata.style = 2; + graphdata.ptype = 1; + graphdata.size = 2.0; + KapaPrepPlot (kapa2, refRadius->n, &graphdata); + KapaPlotVector (kapa2, refRadius->n, refRadius->data.F32, "x"); + KapaPlotVector (kapa2, refRadius->n, meanSB->data.F32, "y"); + + // *** section 2: radius vs petrosian ratio + section.dx = 1.00; + section.dy = 0.33; + section.x = 0.00; + section.y = 0.33; + section.name = psStringCopy ("ratio"); + KapaSetSection (kapa2, §ion); + psFree (section.name); + + graphdata.color = KapaColorByName ("black"); + graphdata.ymax = +1.05; + graphdata.ymin = -0.05; + pmVisualLimitsFromVectors (&graphdata, radBin, NULL); + KapaSetLimits (kapa2, &graphdata); + + KapaBox (kapa2, &graphdata); + KapaSendLabel (kapa2, "radius", KAPA_LABEL_XM); + KapaSendLabel (kapa2, "ratio", KAPA_LABEL_YM); + + graphdata.color = KapaColorByName ("black"); + graphdata.style = 2; + graphdata.ptype = 0; + graphdata.size = 1.0; + graphdata.etype = 0x01; + KapaPrepPlot (kapa2, refRadius->n, &graphdata); + KapaPlotVector (kapa2, refRadius->n, refRadius->data.F32, "x"); + KapaPlotVector (kapa2, refRadius->n, petRatio->data.F32, "y"); + KapaPlotVector (kapa2, refRadius->n, petRatioErr->data.F32, "dym"); + KapaPlotVector (kapa2, refRadius->n, petRatioErr->data.F32, "dyp"); + graphdata.etype = 0; + + graphdata.color = KapaColorByName ("red"); + graphdata.style = 2; + graphdata.ptype = 2; + graphdata.size = 2.0; + KapaPrepPlot (kapa2, 1, &graphdata); + KapaPlotVector (kapa2, 1, &petRadius, "x"); + KapaPlotVector (kapa2, 1, &ratioForRadius, "y"); + + // *** section 3: radius vs integrated flux + section.dx = 1.00; + section.dy = 0.33; + section.x = 0.00; + section.y = 0.66; + section.name = psStringCopy ("flux"); + KapaSetSection (kapa2, §ion); + psFree (section.name); + + graphdata.color = KapaColorByName ("black"); + pmVisualLimitsFromVectors (&graphdata, radBin, fluxSum); + KapaSetLimits (kapa2, &graphdata); + + KapaBox (kapa2, &graphdata); + KapaSendLabel (kapa2, "radius", KAPA_LABEL_XM); + KapaSendLabel (kapa2, "integrated flux", KAPA_LABEL_YM); + + graphdata.color = KapaColorByName ("black"); + graphdata.style = 2; + graphdata.ptype = 0; + graphdata.size = 1.0; + KapaPrepPlot (kapa2, refRadius->n, &graphdata); + KapaPlotVector (kapa2, refRadius->n, refRadius->data.F32, "x"); + KapaPlotVector (kapa2, refRadius->n, fluxSum->data.F32, "y"); + + graphdata.color = KapaColorByName ("red"); + graphdata.ptype = 2; + graphdata.style = 2; + graphdata.size = 2.0; + KapaPrepPlot (kapa2, 1, &graphdata); + KapaPlotVector (kapa2, 1, &radiusForFlux, "x"); + KapaPlotVector (kapa2, 1, &petFlux, "y"); + + // pause and wait for user input: + // continue, save (provide name), ?? + char key[10]; + fprintf (stdout, "[c]ontinue? "); + if (!fgets(key, 8, stdin)) { + psWarning("Unable to read option"); + } + return true; +} + +bool psphotPetrosianVisualEllipticalContour (pmSourceRadialFlux *radFlux, pmSourceExtendedPars *extpars) { + + Graphdata graphdata; + + if (!FORCE_VISUAL && !pmVisualIsVisual()) return true; if (kapa == -1) { @@ -187,112 +326,71 @@ KapaSetFont (kapa, "courier", 14); - // radius vs flux - // radius vs mean SB - // radius vs petRatio - - // *** section 1: radius vs mean SB - section.dx = 1.00; - section.dy = 0.33; - section.x = 0.00; - section.y = 0.00; - section.name = psStringCopy ("meanSB"); - KapaSetSection (kapa, §ion); - psFree (section.name); - - graphdata.color = KapaColorByName ("black"); - pmVisualLimitsFromVectors (&graphdata, radBin, fluxBin); + psVector *theta = radFlux->theta; + psVector *radius = radFlux->isophotalRadii; + + // find Rmin and Rmax for the initial guess + float Rmin = radius->data.F32[0]; + float Rmax = radius->data.F32[0]; + + psVector *Rx = psVectorAlloc(radius->n, PS_TYPE_F32); + psVector *Ry = psVectorAlloc(radius->n, PS_TYPE_F32); + + for (int i = 0; i < theta->n; i++) { + Rx->data.F32[i] = radius->data.F32[i]*cos(theta->data.F32[i]); + Ry->data.F32[i] = radius->data.F32[i]*sin(theta->data.F32[i]); + + // check the radius range + Rmin = MIN (Rmin, radius->data.F32[i]); + Rmax = MAX (Rmax, radius->data.F32[i]); + } + + psVector *rx = psVectorAlloc(361, PS_TYPE_F32); + psVector *ry = psVectorAlloc(361, PS_TYPE_F32); + + float epsilon = extpars->axes.minor / extpars->axes.major; + + for (int i = 0; i < 361; i++) { + + float alpha = PS_RAD_DEG * i; + + float cs_alpha = cos(alpha); + float sn_alpha = sin(alpha); + + float cs_phi = cos(alpha - extpars->axes.theta); + float sn_phi = sin(alpha - extpars->axes.theta); + + float r = 1.0 / sqrt(SQ(sn_phi) + SQ(epsilon*cs_phi)); + + // generate the model fit here + rx->data.F32[i] = extpars->axes.minor * cs_alpha * r; + ry->data.F32[i] = extpars->axes.minor * sn_alpha * r; + } + + graphdata.color = KapaColorByName ("black"); + graphdata.xmin = -1.1*Rmax; + graphdata.ymin = -1.1*Rmax; + graphdata.xmax = +1.1*Rmax; + graphdata.ymax = +1.1*Rmax; KapaSetLimits (kapa, &graphdata); KapaBox (kapa, &graphdata); - KapaSendLabel (kapa, "radius", KAPA_LABEL_XM); - KapaSendLabel (kapa, "mean SB", KAPA_LABEL_YM); - - graphdata.color = KapaColorByName ("black"); - graphdata.style = 2; - graphdata.ptype = 0; - graphdata.size = 1.0; - KapaPrepPlot (kapa, radBin->n, &graphdata); - KapaPlotVector (kapa, radBin->n, radBin->data.F32, "x"); - KapaPlotVector (kapa, radBin->n, fluxBin->data.F32, "y"); + KapaSendLabel (kapa, "R_x", KAPA_LABEL_XM); + KapaSendLabel (kapa, "R_y", KAPA_LABEL_YM); graphdata.color = KapaColorByName ("red"); graphdata.style = 2; - graphdata.ptype = 1; - graphdata.size = 2.0; - KapaPrepPlot (kapa, refRadius->n, &graphdata); - KapaPlotVector (kapa, refRadius->n, refRadius->data.F32, "x"); - KapaPlotVector (kapa, refRadius->n, meanSB->data.F32, "y"); - - // *** section 2: radius vs petrosian ratio - section.dx = 1.00; - section.dy = 0.33; - section.x = 0.00; - section.y = 0.33; - section.name = psStringCopy ("ratio"); - KapaSetSection (kapa, §ion); - psFree (section.name); - - graphdata.color = KapaColorByName ("black"); - graphdata.ymax = +1.05; - graphdata.ymin = -0.05; - pmVisualLimitsFromVectors (&graphdata, radBin, NULL); - KapaSetLimits (kapa, &graphdata); - - KapaBox (kapa, &graphdata); - KapaSendLabel (kapa, "radius", KAPA_LABEL_XM); - KapaSendLabel (kapa, "ratio", KAPA_LABEL_YM); - - graphdata.color = KapaColorByName ("black"); - graphdata.style = 2; - graphdata.ptype = 0; - graphdata.size = 1.0; - graphdata.etype = 0x01; - KapaPrepPlot (kapa, refRadius->n, &graphdata); - KapaPlotVector (kapa, refRadius->n, refRadius->data.F32, "x"); - KapaPlotVector (kapa, refRadius->n, petRatio->data.F32, "y"); - KapaPlotVector (kapa, refRadius->n, petRatioErr->data.F32, "dym"); - KapaPlotVector (kapa, refRadius->n, petRatioErr->data.F32, "dyp"); - graphdata.etype = 0; - - graphdata.color = KapaColorByName ("red"); - graphdata.style = 2; graphdata.ptype = 2; - graphdata.size = 2.0; - KapaPrepPlot (kapa, 1, &graphdata); - KapaPlotVector (kapa, 1, &petRadius, "x"); - KapaPlotVector (kapa, 1, &ratioForRadius, "y"); - - // *** section 3: radius vs integrated flux - section.dx = 1.00; - section.dy = 0.33; - section.x = 0.00; - section.y = 0.66; - section.name = psStringCopy ("flux"); - KapaSetSection (kapa, §ion); - psFree (section.name); - - graphdata.color = KapaColorByName ("black"); - pmVisualLimitsFromVectors (&graphdata, radBin, fluxSum); - KapaSetLimits (kapa, &graphdata); - - KapaBox (kapa, &graphdata); - KapaSendLabel (kapa, "radius", KAPA_LABEL_XM); - KapaSendLabel (kapa, "integrated flux", KAPA_LABEL_YM); - - graphdata.color = KapaColorByName ("black"); - graphdata.style = 2; - graphdata.ptype = 0; - graphdata.size = 1.0; - KapaPrepPlot (kapa, refRadius->n, &graphdata); - KapaPlotVector (kapa, refRadius->n, refRadius->data.F32, "x"); - KapaPlotVector (kapa, refRadius->n, fluxSum->data.F32, "y"); - - graphdata.color = KapaColorByName ("red"); - graphdata.ptype = 2; - graphdata.style = 2; - graphdata.size = 2.0; - KapaPrepPlot (kapa, 1, &graphdata); - KapaPlotVector (kapa, 1, &radiusForFlux, "x"); - KapaPlotVector (kapa, 1, &petFlux, "y"); + graphdata.size = 1.0; + KapaPrepPlot (kapa, Rx->n, &graphdata); + KapaPlotVector (kapa, Rx->n, Rx->data.F32, "x"); + KapaPlotVector (kapa, Rx->n, Ry->data.F32, "y"); + + graphdata.color = KapaColorByName ("black"); + graphdata.style = 0; + graphdata.ptype = 0; + graphdata.size = 1.0; + KapaPrepPlot (kapa, rx->n, &graphdata); + KapaPlotVector (kapa, rx->n, rx->data.F32, "x"); + KapaPlotVector (kapa, rx->n, ry->data.F32, "y"); // pause and wait for user input: @@ -306,100 +404,3 @@ } -bool psphotPetrosianVisualEllipticalContour (pmPetrosian *petrosian) { - - Graphdata graphdata; - - if (!pmVisualIsVisual()) return true; - - if (kapa == -1) { - kapa = KapaOpenNamedSocket ("kapa", "psphot:plots"); - if (kapa == -1) { - fprintf (stderr, "failure to open kapa; visual mode disabled\n"); - pmVisualSetVisual(false); - return false; - } - } - - KapaClearPlots (kapa); - KapaInitGraph (&graphdata); - KapaSetFont (kapa, "courier", 14); - - psVector *theta = petrosian->theta; - psVector *radius = petrosian->isophotalRadii; - - // find Rmin and Rmax for the initial guess - float Rmin = radius->data.F32[0]; - float Rmax = radius->data.F32[0]; - - psVector *Rx = psVectorAlloc(radius->n, PS_TYPE_F32); - psVector *Ry = psVectorAlloc(radius->n, PS_TYPE_F32); - - for (int i = 0; i < theta->n; i++) { - Rx->data.F32[i] = radius->data.F32[i]*cos(theta->data.F32[i]); - Ry->data.F32[i] = radius->data.F32[i]*sin(theta->data.F32[i]); - - // check the radius range - Rmin = MIN (Rmin, radius->data.F32[i]); - Rmax = MAX (Rmax, radius->data.F32[i]); - } - - psVector *rx = psVectorAlloc(361, PS_TYPE_F32); - psVector *ry = psVectorAlloc(361, PS_TYPE_F32); - - float epsilon = petrosian->axes.minor / petrosian->axes.major; - - for (int i = 0; i < 361; i++) { - - float alpha = PS_RAD_DEG * i; - - float cs_alpha = cos(alpha); - float sn_alpha = sin(alpha); - - float cs_phi = cos(alpha - petrosian->axes.theta); - float sn_phi = sin(alpha - petrosian->axes.theta); - - float r = 1.0 / sqrt(SQ(sn_phi) + SQ(epsilon*cs_phi)); - - // generate the model fit here - rx->data.F32[i] = petrosian->axes.minor * cs_alpha * r; - ry->data.F32[i] = petrosian->axes.minor * sn_alpha * r; - } - - graphdata.color = KapaColorByName ("black"); - graphdata.xmin = -1.1*Rmax; - graphdata.ymin = -1.1*Rmax; - graphdata.xmax = +1.1*Rmax; - graphdata.ymax = +1.1*Rmax; - KapaSetLimits (kapa, &graphdata); - - KapaBox (kapa, &graphdata); - KapaSendLabel (kapa, "R_x", KAPA_LABEL_XM); - KapaSendLabel (kapa, "R_y", KAPA_LABEL_YM); - - graphdata.color = KapaColorByName ("red"); - graphdata.style = 2; - graphdata.ptype = 2; - graphdata.size = 1.0; - KapaPrepPlot (kapa, Rx->n, &graphdata); - KapaPlotVector (kapa, Rx->n, Rx->data.F32, "x"); - KapaPlotVector (kapa, Rx->n, Ry->data.F32, "y"); - - graphdata.color = KapaColorByName ("black"); - graphdata.style = 0; - graphdata.ptype = 0; - graphdata.size = 1.0; - KapaPrepPlot (kapa, rx->n, &graphdata); - KapaPlotVector (kapa, rx->n, rx->data.F32, "x"); - KapaPlotVector (kapa, rx->n, ry->data.F32, "y"); - - // pause and wait for user input: - // continue, save (provide name), ?? - char key[10]; - fprintf (stdout, "[c]ontinue? "); - if (!fgets(key, 8, stdin)) { - psWarning("Unable to read option"); - } - return true; -} - # endif Index: branches/tap_branches/psphot/src/psphotRadialBins.c =================================================================== --- branches/tap_branches/psphot/src/psphotRadialBins.c (revision 27838) +++ branches/tap_branches/psphot/src/psphotRadialBins.c (revision 27838) @@ -0,0 +1,204 @@ +# include "psphotInternal.h" +float InterpolateValues (float X0, float Y0, float X1, float Y1, float X); + +// convert the flux vs elliptical radius to annular bins + +// we are guaranteed to be limited by either the seeing (1 - few pixels) or by the pixels +// themselves. this function does not attempt to measure the radial profiles accurately +// for radii that are smaller than ~2 pixels + +// for small radii, we are measuring the mean surface brightness in non-overlapping radial +// bins. for large radii (r > 2 pixels), we are measuring the surface brightness for a +// radius range \alpha r_i < i < \beta r_i, but performing this measurement for radii more +// finely spaced than r_{i+1} = r_i * \beta / \alpha. for the integration, we need to +// track the non-overlapping radius values. + +// Photo interpolates the image of interest to place the peak on the center of the central +// pixel, and then uses the exact fractions of the pixels in each of the first few annuli. +// Seems like a reasonable thing, but is there any significance to the difference? + +// XXX move the resulting elements from profile to extpars->petrosian? +bool psphotRadialBins (psMetadata *recipe, pmSource *source, float radiusMax, float skynoise) { + + psAssert (source, "missing source"); + psAssert (source->extpars, "missing extpars"); + psAssert (source->extpars->ellipticalFlux, "missing ellipticalFlux"); + + psVector *radius = source->extpars->ellipticalFlux->radiusElliptical; + psVector *flux = source->extpars->ellipticalFlux->fluxElliptical; + + // sort incoming vectors by radius + pmSourceRadialProfileSortPair (radius, flux); + + if (!source->extpars->radProfile) { + source->extpars->radProfile = pmSourceRadialProfileAlloc(); + } + pmSourceRadialProfile *profile = source->extpars->radProfile; + + float skyModelErrorSQ = PS_SQR(skynoise); + psEllipseAxes axes = source->extpars->axes; + float AxialRatio = axes.minor / axes.major; + + // radMin, radMax store the bounds of the annuli + bool status = false; + psVector *radMin = psMetadataLookupPtr (&status, recipe, "RADIAL.ANNULAR.BINS.LOWER"); + psVector *radMax = psMetadataLookupPtr (&status, recipe, "RADIAL.ANNULAR.BINS.UPPER"); + if (!radMin || !radMin->n) return false; + if (!radMax || !radMax->n) return false; + + psVector *binSB = psVectorAllocEmpty(radMin->n, PS_TYPE_F32); // surface brightness of radial bin + psVector *binSBstdev = psVectorAllocEmpty(radMin->n, PS_TYPE_F32); // surface brightness error of radial bin + psVector *binSum = psVectorAllocEmpty(radMin->n, PS_TYPE_F32); // surface brightness of radial bin + psVector *binFill = psVectorAllocEmpty(radMin->n, PS_TYPE_F32); // surface brightness of radial bin + psVector *binRad = psVectorAllocEmpty(radMin->n, PS_TYPE_F32); // mean radius of radial bin + psVector *binArea = psVectorAllocEmpty(radMin->n, PS_TYPE_F32); // area of radial bin (contiguous, non-overlapping) + + psVectorInit (binSB, 0.0); + psVectorInit (binSBstdev, 0.0); + psVectorInit (binSum, 0.0); + psVectorInit (binFill, 0.0); + + psVectorInit (binRad, 0.0); + + // generate radial area-weighted mean radius & non-overlapping areas + for (int i = 0; i < radMin->n; i++) { + float rMin2 = PS_SQR(radMin->data.F32[i]); + float rMax2 = PS_SQR(radMax->data.F32[i]); + + float rMin3 = rMin2*radMin->data.F32[i]; + float rMax3 = rMax2*radMax->data.F32[i]; + + float rBin = 2.0 * (rMax3 - rMin3) / (rMax2 - rMin2) / 3.0; + + // XXX calculate area-weighted radius rather than asserting? + binRad->data.F32[i] = rBin; + binArea->data.F32[i] = M_PI * (rMax2 - rMin2); + } + + // storage vector for stats + psVector *values = psVectorAllocEmpty (flux->n, PS_TYPE_F32); + psStats *stats = psStatsAlloc(PS_STAT_ROBUST_MEDIAN | PS_STAT_ROBUST_STDEV); + + float fluxSum = 0.0; + int nPixSum = 0; + + bool done = false; + int nOut = 0; + float Rmin = radMin->data.F32[nOut]; + float Rmax = radMax->data.F32[nOut]; + float Rnxt = radMin->data.F32[nOut+1]; // minimum radius for next range + int iNext = 0; + int iLast = -1; + for (int i = 0; !done && (i < radius->n); i++) { + if (radius->data.F32[i] < Rnxt) { + iNext = i; // on the next pass, we will start back here to ensure we hit all pixels in the next bin + } + if (radius->data.F32[i] > Rmax) { + // calculate the surface brightness for bin 'nOut' + float value, dvalue; + if (values->n > 0) { + psVectorStats (stats, values, NULL, NULL, 0); + value = stats->robustMedian; + dvalue = stats->robustStdev; + } else { + value = NAN; + dvalue = NAN; + } + + binSB->data.F32[nOut] = value; + binSBstdev->data.F32[nOut] = sqrt(PS_SQR(dvalue) / values->n + skyModelErrorSQ); + + // calculate the total flux for bin 'nOut' + float Area = M_PI*AxialRatio*PS_SQR(Rmax); + binSum->data.F32[nOut] = fluxSum; + binFill->data.F32[nOut] = nPixSum / Area; + + psTrace ("psphot", 5, "radial bins: %3d %5.1f %5.1f : %7.1f %6.2f : %8.1f %4.2f %6.1f\n", + nOut, radMin->data.F32[nOut], radMax->data.F32[nOut], + binSB->data.F32[nOut], binSBstdev->data.F32[nOut], + binSum->data.F32[nOut], binFill->data.F32[nOut], Area); + + nOut ++; + if (nOut >= radMin->n) break; + Rmin = radMin->data.F32[nOut]; + Rmax = radMax->data.F32[nOut]; + Rnxt = (nOut < radMin->n - 1) ? radMin->data.F32[nOut+1] : Rmax; // minimum radius for next range + values->n = 0; + psStatsInit(stats); + iLast = i; + i = iNext; + } + if (radius->data.F32[i] < Rmin) { + continue; + } + psVectorAppend (values, flux->data.F32[i]); + + if (i > iLast) { + fluxSum += flux->data.F32[i]; + nPixSum ++; + } + } + binSB->n = binSBstdev->n = binRad->n = binArea->n = nOut; + + // interpolate any bins that were empty (extrapolate to center if needed) + if (!isfinite(binSB->data.F32[0]) && !isfinite(binSB->data.F32[1])) { + psWarning ("center 2 bins of source at %f, %f are NAN, skipping this source", source->peak->xf, source->peak->yf); + // XXX raise a flag + psFree(binSB); + psFree(binSBstdev); + + psFree(binSum); + psFree(binFill); + + psFree(binRad); + psFree(binArea); + psFree(values); + psFree(stats); + return false; + } + + // if center bin is empty assume same SB as next radius (probably true due to PSF) + if (!isfinite(binSB->data.F32[0])) { + binSB->data.F32[0] = binSB->data.F32[1]; + binSBstdev->data.F32[0] = binSBstdev->data.F32[1]; + } + + // interpolate any bins that were empty (if center if needed) + for (int i = 1; i < binSB->n - 1; i++) { + if (isfinite(binSB->data.F32[i])) continue; + binSB->data.F32[i] = InterpolateValues (binRad->data.F32[i-1], binSB->data.F32[i-1], binRad->data.F32[i+1], binSB->data.F32[i+1], binRad->data.F32[i]); + binSBstdev->data.F32[i] = InterpolateValues (binRad->data.F32[i-1], binSBstdev->data.F32[i-1], binRad->data.F32[i+1], binSBstdev->data.F32[i+1], binRad->data.F32[i]); + } + + psFree(profile->binSB); + psFree(profile->binSBstdev); + + psFree(profile->binSum); + psFree(profile->binFill); + + psFree(profile->radialBins); + psFree(profile->area); + + // save the vectors + profile->binSB = binSB; + profile->binSBstdev = binSBstdev; + + profile->binSum = binSum; + profile->binFill = binFill; + + profile->radialBins = binRad; + profile->area = binArea; + + psFree(values); + psFree(stats); + + return true; +} + +// the area-weighted mean radius is given by: + +// integral r * 2 pi r dr / integral 2 pi r dr + +// = 2/3 pi (r_max^3 - r_min^3) / pi (r_max^2 - r_min^2) +// = 2/3 (r_max^3 - r_min^3) / (r_max^2 - r_min^2) + Index: branches/tap_branches/psphot/src/psphotRadialProfile.c =================================================================== --- branches/tap_branches/psphot/src/psphotRadialProfile.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotRadialProfile.c (revision 27838) @@ -3,11 +3,9 @@ bool psphotRadialProfile (pmSource *source, psMetadata *recipe, float skynoise, psImageMaskType maskVal) { + bool status; + // allocate pmSourceExtendedParameters, if not already defined if (!source->extpars) { source->extpars = pmSourceExtendedParsAlloc (); - } - - if (!source->extpars->profile) { - source->extpars->profile = pmSourceRadialProfileAlloc (); } @@ -17,4 +15,6 @@ float fluxMin = 0.0; float fluxMax = source->peak->flux; + + bool RAW_RADIUS = psMetadataLookupBool (&status, recipe, "EXTENDED_SOURCE_RAW_RADIUS"); // generate a series of radial profiles at Nsec evenly spaced angles. the profile flux @@ -28,5 +28,5 @@ // use the radial profiles to determine the radius of a given isophote. this isophote // is used to determine the elliptical shape of the object, so it has a relatively high - // value (nominally 50% of the peak) + // value (nominally 25% of the peak) if (!psphotRadiiFromProfiles (source, fluxMin, fluxMax)) { psError (PS_ERR_UNKNOWN, false, "failed to measure isophotal radii from profiles"); @@ -42,6 +42,12 @@ // generate a single, normalized radial profile following the elliptical contours. // the radius is normalized by the axis ratio so that on the major axis, 1 pixel = 1 pixel - if (!psphotEllipticalProfile (source)) { + if (!psphotEllipticalProfile (source, RAW_RADIUS)) { psError (PS_ERR_UNKNOWN, false, "failed to generate elliptical profile"); + return false; + } + + // generated profile in averaged bins + if (!psphotRadialBins (recipe, source, Rmax, skynoise)) { + psError (PS_ERR_UNKNOWN, false, "failed to generate radial bins"); return false; } Index: branches/tap_branches/psphot/src/psphotRadialProfileByAngles.c =================================================================== --- branches/tap_branches/psphot/src/psphotRadialProfileByAngles.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotRadialProfileByAngles.c (revision 27838) @@ -12,9 +12,14 @@ bool psphotRadialProfilesByAngles (pmSource *source, int Nsec, float Rmax) { + psAssert (source->extpars, "define extpars"); + // we want to have an even number of sectors so we can do 180 deg symmetrizing Nsec = (Nsec % 2) ? Nsec + 1 : Nsec; float dtheta = 2.0*M_PI / Nsec; - pmSourceRadialProfile *profile = source->extpars->profile; + if (!source->extpars->radFlux) { + source->extpars->radFlux = pmSourceRadialFluxAlloc(); + } + pmSourceRadialFlux *profile = source->extpars->radFlux; psFree(profile->radii); psFree(profile->fluxes); @@ -24,5 +29,4 @@ profile->fluxes = psArrayAllocEmpty(Nsec); profile->theta = psVectorAllocEmpty(Nsec, PS_TYPE_F32); - for (int i = 0; i < Nsec; i++) { Index: branches/tap_branches/psphot/src/psphotRadiiFromProfiles.c =================================================================== --- branches/tap_branches/psphot/src/psphotRadiiFromProfiles.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotRadiiFromProfiles.c (revision 27838) @@ -5,5 +5,9 @@ bool psphotRadiiFromProfiles (pmSource *source, float fluxMin, float fluxMax) { - pmSourceRadialProfile *profile = source->extpars->profile; + psAssert (source, "missing source"); + psAssert (source->extpars, "missing extpars"); + psAssert (source->extpars->radFlux, "missing radFlux"); + + pmSourceRadialFlux *profile = source->extpars->radFlux; psFree(profile->isophotalRadii); @@ -146,5 +150,5 @@ // show the results - // psphotPetrosianVisualProfileRadii (radius, flux, radiusBinned, fluxBinned, Ro); + // psphotPetrosianVisualProfileRadii (radius, flux, radiusBinned, fluxBinned, fluxMax, Ro); psFree(fluxBinned); @@ -152,2 +156,5 @@ return Ro; } + + + Index: branches/tap_branches/psphot/src/psphotRadiusChecks.c =================================================================== --- branches/tap_branches/psphot/src/psphotRadiusChecks.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotRadiusChecks.c (revision 27838) @@ -4,18 +4,24 @@ static float PSF_FIT_NSIGMA; static float PSF_FIT_PADDING; +static float PSF_APERTURE = 0; // radius to use in PSF aperture mags static float PSF_FIT_RADIUS = 0; // radius to use in fitting (ignored if <= 0, // and a per-object radius is calculated) -static float PSF_APERTURE = 0; // radius to use in PSF aperture mags - - -bool psphotInitRadiusPSF(const psMetadata *recipe, const pmModelType type) { +bool psphotInitRadiusPSF(const psMetadata *recipe, const psMetadata *analysis, const pmModelType type) { bool status = true; - PSF_FIT_NSIGMA = psMetadataLookupF32(&status, recipe, "PSF_FIT_NSIGMA"); + PSF_FIT_NSIGMA = psMetadataLookupF32(&status, recipe, "PSF_FIT_NSIGMA"); PSF_FIT_PADDING = psMetadataLookupF32(&status, recipe, "PSF_FIT_PADDING"); - PSF_FIT_RADIUS = psMetadataLookupF32(&status, recipe, "PSF_FIT_RADIUS"); - PSF_APERTURE = psMetadataLookupF32(&status, recipe, "PSF_APERTURE"); + + PSF_FIT_RADIUS = psMetadataLookupF32(&status, analysis, "PSF_FIT_RADIUS"); + if (!status) { + PSF_FIT_RADIUS = psMetadataLookupF32(&status, recipe, "PSF_FIT_RADIUS"); + } + + PSF_APERTURE = psMetadataLookupF32(&status, analysis, "PSF_APERTURE"); + if (!status) { + PSF_APERTURE = psMetadataLookupF32(&status, recipe, "PSF_APERTURE"); + } return true; Index: branches/tap_branches/psphot/src/psphotReadout.c =================================================================== --- branches/tap_branches/psphot/src/psphotReadout.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotReadout.c (revision 27838) @@ -3,5 +3,4 @@ // this should be called by every program that links against libpsphot bool psphotInit (void) { - psphotErrorRegister(); // register our error codes/messages psphotModelClassInit (); // load implementation-specific models @@ -12,8 +11,6 @@ bool psphotReadout(pmConfig *config, const pmFPAview *view) { - // measure the total elapsed time in psphotReadout. XXX the current threading plan - // for psphot envisions threading within psphotReadout, not multiple threads calling - // the same psphotReadout. In the current plan, this dtime is the elapsed time used - // jointly by the multiple threads, not the total time used by all threads. + // measure the total elapsed time in psphotReadout. dtime is the elapsed time used jointly + // by the multiple threads, not the total time used by all threads. psTimerStart ("psphotReadout"); @@ -26,181 +23,176 @@ return false; } + // optional break-point for processing + char *breakPt = psMetadataLookupStr (NULL, recipe, "BREAK_POINT"); + psAssert (breakPt, "configuration error: set BREAK_POINT"); // set the photcode for this image - if (!psphotAddPhotcode (recipe, config, view, "PSPHOT.INPUT")) { + if (!psphotAddPhotcode (config, view)) { psError (PSPHOT_ERR_CONFIG, false, "trouble defining the photcode"); return false; } - // find the currently selected readout - pmReadout *readout = pmFPAfileThisReadout (config->files, view, "PSPHOT.INPUT"); - PS_ASSERT_PTR_NON_NULL (readout, false); - - // optional break-point for processing - char *breakPt = psMetadataLookupStr (NULL, recipe, "BREAK_POINT"); - PS_ASSERT_PTR_NON_NULL (breakPt, false); - // Generate the mask and weight images, including the user-defined analysis region of interest - psphotSetMaskAndVariance (config, readout, recipe); + if (!psphotSetMaskAndVariance (config, view)) { + return psphotReadoutCleanup(config, view); + } if (!strcasecmp (breakPt, "NOTHING")) { - return psphotReadoutCleanup(config, readout, recipe, NULL, NULL, NULL); - } - - // display the image, weight, mask (ch 1,2,3) - psphotVisualShowImage (readout); + return psphotReadoutCleanup(config, view); + } // generate a background model (median, smoothed image) - if (!psphotModelBackground (config, view, "PSPHOT.INPUT")) { - return psphotReadoutCleanup (config, readout, recipe, NULL, NULL, NULL); - } - if (!psphotSubtractBackground (config, view, "PSPHOT.INPUT")) { - return psphotReadoutCleanup (config, readout, recipe, NULL, NULL, NULL); + if (!psphotModelBackground (config, view)) { + return psphotReadoutCleanup (config, view); + } + if (!psphotSubtractBackground (config, view)) { + return psphotReadoutCleanup (config, view); } if (!strcasecmp (breakPt, "BACKMDL")) { - return psphotReadoutCleanup (config, readout, recipe, NULL, NULL, NULL); - } - - // display the backsub and backgnd images - psphotVisualShowBackground (config, view, readout); - - // run a single-model test if desired (exits from here if test is run) - psphotModelTest (config, view, recipe); - - // load the psf model, if suppled. FWHM_X,FWHM_Y,etc are saved in the recipe - pmPSF *psf = psphotLoadPSF (config, view, recipe); - - // several functions below behave differently if we have a PSF model already - bool havePSF = (psf != NULL); + return psphotReadoutCleanup (config, view); + } + + // load the psf model, if suppled. FWHM_X,FWHM_Y,etc are determined and saved on + // readout->analysis XXX this function currently only works with a single PSPHOT.INPUT + if (!psphotLoadPSF (config, view)) { + psError (PSPHOT_ERR_UNKNOWN, false, "error loading psf model"); + return psphotReadoutCleanup (config, view); + } // find the detections (by peak and/or footprint) in the image. - pmDetections *detections = psphotFindDetections (NULL, readout, recipe); - if (!detections) { - psLogMsg ("psphot", 3, "unable to find detections in this image"); - return psphotReadoutCleanup (config, readout, recipe, detections, psf, NULL); - } - - // construct sources and measure basic stats - psArray *sources = psphotSourceStats (config, readout, detections, true); - if (!sources) return false; + if (!psphotFindDetections (config, view, true)) { // pass 1 + // this only happens if we had an error in psphotFindDetections + psError (PSPHOT_ERR_UNKNOWN, false, "failure in peak analysis"); + return psphotReadoutCleanup (config, view); + } + + // construct sources and measure basic stats (saved on detections->newSources) + if (!psphotSourceStats (config, view, true)) { // pass 1 + psError(PSPHOT_ERR_UNKNOWN, false, "failure to generate sources"); + return psphotReadoutCleanup (config, view); + } if (!strcasecmp (breakPt, "PEAKS")) { - return psphotReadoutCleanup(config, readout, recipe, detections, psf, sources); - } - - // find blended neighbors of very saturated stars - // XXX merge this with Basic Deblend? - psphotDeblendSatstars (readout, sources, recipe); - - // mark blended peaks PS_SOURCE_BLEND - if (!psphotBasicDeblend (sources, recipe)) { - psLogMsg ("psphot", 3, "failed on deblend analysis"); - return psphotReadoutCleanup (config, readout, recipe, detections, psf, sources); - } - - // classify sources based on moments, brightness - if (!psphotRoughClass (readout, sources, recipe, havePSF)) { - psLogMsg ("psphot", 3, "failed to find a valid PSF clump for image"); - return psphotReadoutCleanup (config, readout, recipe, detections, psf, sources); + return psphotReadoutCleanup(config, view); + } + + // find blended neighbors of very saturated stars (detections->newSources) + if (!psphotDeblendSatstars (config, view)) { + psError (PSPHOT_ERR_UNKNOWN, false, "failed on satstar deblend analysis"); + return psphotReadoutCleanup (config, view); + } + + // mark blended peaks PS_SOURCE_BLEND (detections->newSources) + if (!psphotBasicDeblend (config, view)) { + psError (PSPHOT_ERR_UNKNOWN, false, "failed on deblend analysis"); + return psphotReadoutCleanup (config, view); + } + + // classify sources based on moments, brightness. if a PSF model has been loaded, the PSF + // clump defined for it is used not measured (detections->newSources) + if (!psphotRoughClass (config, view)) { // pass 1 + psError (PSPHOT_ERR_UNKNOWN, false, "failed to determine rough classifications"); + return psphotReadoutCleanup (config, view); + } + // if we were not supplied a PSF model, determine the IQ stats here (detections->newSources) + if (!psphotImageQuality (config, view)) { // pass 1 + psError (PSPHOT_ERR_UNKNOWN, false, "failed to measure image quality"); + return psphotReadoutCleanup(config, view); } if (!strcasecmp (breakPt, "MOMENTS")) { - return psphotReadoutCleanup(config, readout, recipe, detections, psf, sources); - } - - if (!havePSF && !psphotImageQuality (recipe, sources)) { - psLogMsg("psphot", 3, "failed to measure image quality"); - return psphotReadoutCleanup(config, readout, recipe, detections, psf, sources); - } - - // if we were not supplied a PSF, choose one here - if (psf == NULL) { - // use bright stellar objects to measure PSF - // XXX if we do not have enough stars to generate the PSF, build one - // from the SEEING guess and model class - psf = psphotChoosePSF (readout, sources, recipe); - if (psf == NULL) { - psLogMsg ("psphot", 3, "failure to construct a psf model"); - return psphotReadoutCleanup (config, readout, recipe, detections, psf, sources); - } - havePSF = true; + return psphotReadoutCleanup(config, view); + } + + // use bright stellar objects to measure PSF if we were supplied a PSF for any input file, + // this step is skipped + if (!psphotChoosePSF (config, view)) { // pass 1 + psLogMsg ("psphot", 3, "failure to construct a psf model"); + return psphotReadoutCleanup (config, view); } if (!strcasecmp (breakPt, "PSFMODEL")) { - return psphotReadoutCleanup (config, readout, recipe, detections, psf, sources); - } - psphotVisualShowPSFModel (readout, psf); + return psphotReadoutCleanup (config, view); + } // include externally-supplied sources - psphotLoadExtSources (config, view, sources); - - // construct an initial model for each object, set the radius to fitRadius, set circular fit mask - psphotGuessModels (config, readout, sources, psf); + // XXX fix this in the new multi-input context + // psphotLoadExtSources (config, view); // pass 1 + + // construct an initial model for each object, set the radius to fitRadius, set circular + // fit mask (detections->newSources) + psphotGuessModels (config, view); // pass 1 + + // merge the newly selected sources into the existing list + // NOTE: merge OLD and NEW + psphotMergeSources (config, view); // linear PSF fit to source peaks, subtract the models from the image (in PSF mask) - psphotFitSourcesLinear (readout, sources, recipe, psf, FALSE); - - // We have to place these visualizations here because the models are not realized until - // psphotGuessModels or fitted until psphotFitSourcesLinear. - psphotVisualShowPSFStars (recipe, psf, sources); - - // identify CRs and extended sources - psphotSourceSize (config, readout, sources, recipe, psf, 0); + psphotFitSourcesLinear (config, view, false); // pass 1 (detections->allSources) + + // identify CRs and extended sources (only unmeasured sources are measured) + psphotSourceSize (config, view, true); // pass 1 (detections->allSources) if (!strcasecmp (breakPt, "ENSEMBLE")) { goto finish; } - psphotVisualShowSatStars (recipe, psf, sources); // non-linear PSF and EXT fit to brighter sources // replace model flux, adjust mask as needed, fit, subtract the models (full stamp) - psphotBlendFit (config, readout, sources, psf); + psphotBlendFit (config, view); // pass 1 (detections->allSources) // replace all sources - psphotReplaceAllSources (sources, recipe); + psphotReplaceAllSources (config, view); // pass 1 (detections->allSources) // linear fit to include all sources (subtract again) - psphotFitSourcesLinear (readout, sources, recipe, psf, TRUE); + // NOTE : apply to ALL sources (extended + psf) + psphotFitSourcesLinear (config, view, true); // pass 2 (detections->allSources) // if we only do one pass, skip to extended source analysis - if (!strcasecmp (breakPt, "PASS1")) { - goto pass1finish; - } - // NOTE: possibly re-measure background model here with objects subtracted + if (!strcasecmp (breakPt, "PASS1")) goto pass1finish; + + // NOTE: possibly re-measure background model here with objects subtracted / or masked // add noise for subtracted objects - psphotAddNoise (readout, sources, recipe); - - // find fainter sources (pass 2) - detections = psphotFindDetections (detections, readout, recipe); + psphotAddNoise (config, view); // pass 1 (detections->allSources) + + // find fainter sources + // NOTE: finds new peaks and new footprints, OLD and FULL set are saved on detections + psphotFindDetections (config, view, false); // pass 2 (detections->peaks, detections->footprints) // remove noise for subtracted objects (ie, return to normal noise level) - psphotSubNoise (readout, sources, recipe); + // NOTE: this needs to operate only on the OLD sources + psphotSubNoise (config, view); // pass 1 (detections->allSources) // define new sources based on only the new peaks - psArray *newSources = psphotSourceStats (config, readout, detections, false); + // NOTE: new sources are saved on detections->newSources + psphotSourceStats (config, view, false); // pass 2 (detections->newSources) // set source type - if (!psphotRoughClass (readout, newSources, recipe, havePSF)) { + // NOTE: apply only to detections->newSources + if (!psphotRoughClass (config, view)) { // pass 2 (detections->newSources) psLogMsg ("psphot", 3, "failed to find a valid PSF clump for image"); - return psphotReadoutCleanup (config, readout, recipe, detections, psf, sources); + return psphotReadoutCleanup (config, view); } // create full input models, set the radius to fitRadius, set circular fit mask - psphotGuessModels (config, readout, newSources, psf); + // NOTE: apply only to detections->newSources + psphotGuessModels (config, view); // pass 2 (detections->newSources) // replace all sources so fit below applies to all at once - psphotReplaceAllSources (sources, recipe); + // NOTE: apply only to OLD sources (which have been subtracted) + psphotReplaceAllSources (config, view); // pass 2 // merge the newly selected sources into the existing list - psphotMergeSources (sources, newSources); - psFree (newSources); - - // linear fit to all sources - psphotFitSourcesLinear (readout, sources, recipe, psf, TRUE); + // NOTE: merge OLD and NEW + // XXX check on free of sources... + psphotMergeSources (config, view); // (detections->newSources + detections->allSources -> detections->allSources) + + // NOTE: apply to ALL sources + psphotFitSourcesLinear (config, view, true); // pass 3 (detections->allSources) pass1finish: // measure source size for the remaining sources - psphotSourceSize (config, readout, sources, recipe, psf, 0); - - psphotExtendedSourceAnalysis (readout, sources, recipe); - - psphotExtendedSourceFits (readout, sources, recipe); + // NOTE: applies only to NEW (unmeasured) sources + psphotSourceSize (config, view, false); // pass 2 (detections->allSources) + + psphotExtendedSourceAnalysis (config, view); // pass 1 (detections->allSources) + psphotExtendedSourceFits (config, view); // pass 1 (detections->allSources) finish: @@ -210,13 +202,13 @@ // measure aperture photometry corrections - if (!psphotApResid (config, readout, sources, psf)) { + if (!psphotApResid (config, view)) { // pass 1 (detections->allSources) psLogMsg ("psphot", 3, "failed on psphotApResid"); - return psphotReadoutCleanup (config, readout, recipe, detections, psf, sources); + return psphotReadoutCleanup (config, view); } // calculate source magnitudes - psphotMagnitudes(config, readout, view, sources, psf); - - if (!psphotEfficiency(config, readout, view, psf, recipe, sources)) { + psphotMagnitudes(config, view); // pass 1 (detections->allSources) + + if (!psphotEfficiency(config, view)) { // pass 1 psErrorStackPrint(stderr, "Unable to determine detection efficiencies from fake sources"); psErrorClear(); @@ -227,11 +219,10 @@ // replace background in residual image - psphotSkyReplace (config, view); + psphotSkyReplace (config, view); // pass 1 // drop the references to the image pixels held by each source - psphotSourceFreePixels (sources); + psphotSourceFreePixels (config, view); // pass 1 // create the exported-metadata and free local data - return psphotReadoutCleanup(config, readout, recipe, detections, psf, sources); + return psphotReadoutCleanup(config, view); } - Index: branches/tap_branches/psphot/src/psphotReadoutCleanup.c =================================================================== --- branches/tap_branches/psphot/src/psphotReadoutCleanup.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotReadoutCleanup.c (revision 27838) @@ -1,8 +1,8 @@ # include "psphotInternal.h" -// psphotReadoutCleanup is called on exit from psphotReadout. If the last raised error is -// not a DATA error, then there was a serious problem. Only in this case, or if the fail -// on the stats measurement, do we return false -bool psphotReadoutCleanup (pmConfig *config, pmReadout *readout, psMetadata *recipe, pmDetections *detections, pmPSF *psf, psArray *sources) { +// for now, let's store the detections on the readout->analysis for each readout +bool psphotReadoutCleanup (pmConfig *config, const pmFPAview *view) +{ + bool status = true; // remove internal pmFPAfiles, if created @@ -12,17 +12,52 @@ } if (psErrorCodeLast() != PS_ERR_NONE) { - psFree (psf); - psFree (sources); - psFree (detections); return false; } + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (!psphotReadoutCleanupReadout (config, view, "PSPHOT.INPUT", i, recipe)) { + psError (PSPHOT_ERR_CONFIG, false, "failed on psphotReadoutCleanup for PSPHOT.INPUT entry %d", i); + return false; + } + } + + // XXX move this to top of loop? + pmKapaClose (); + + return true; +} + +// psphotReadoutCleanup is called on exit from psphotReadout. If the last raised error is +// not a DATA error, then there was a serious problem. Only in this case, or if the fail +// on the stats measurement, do we return false +bool psphotReadoutCleanupReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe) { + + bool status = true; + + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + // when psphotReadoutCleanup is called, these are not necessarily defined + pmPSF *psf = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.PSF"); + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psArray *sources = detections ? detections->allSources : NULL; + // XXX where do we free these, in here (psMetadataRemove?) + // use the psf-model to measure FWHM stats if (psf) { - if (!psphotPSFstats (readout, recipe, psf)) { + if (!psphotPSFstats (readout, psf)) { psError(PSPHOT_ERR_PROG, false, "Failed to measure PSF shape parameters"); - psFree (psf); - psFree (sources); - psFree (detections); return false; } @@ -30,9 +65,6 @@ // otherwise, use the source moments to measure FWHM stats if (!psf && sources) { - if (!psphotMomentsStats (readout, recipe, sources)) { + if (!psphotMomentsStats (readout, sources)) { psError(PSPHOT_ERR_PROG, false, "Failed to measure Moment shape parameters"); - psFree (psf); - psFree (sources); - psFree (detections); return false; } @@ -40,29 +72,27 @@ // Check to see if the image quality was measured - if (!psf) { + // XXX not sure we want / need this test + if (0 && !psf) { bool mdok; // Status of MD lookup int nIQ = psMetadataLookupS32(&mdok, recipe, "IQ_NSTAR"); // Number of stars for IQ measurement if (!mdok || nIQ <= 0) { psError(PSPHOT_ERR_DATA, false, "Unable to measure image quality"); - psFree (psf); - psFree (sources); - psFree (detections); return false; } } + // create an output header with stats results currently saved on readout->analysis + psMetadata *header = psphotDefineHeader (readout->analysis); // write NSTARS to the image header - psphotSetHeaderNstars (recipe, sources); - - // create an output header with stats results - psMetadata *header = psphotDefineHeader (recipe); + psphotSetHeaderNstars (header, sources); // save the results of the analysis + // this should happen way up stream (when needed?) psMetadataAdd (readout->analysis, PS_LIST_TAIL, "PSPHOT.HEADER", PS_DATA_METADATA, "header stats", header); - if (sources) { - psMetadataAdd (readout->analysis, PS_LIST_TAIL, "PSPHOT.SOURCES", PS_DATA_ARRAY, "psphot sources", sources); - } + if (psf) { + // XXX this seems a little silly : we saved the psf on readout->analysis above, but now + // we are moving it to chip->analysis. // save the psf for possible output. if there was already an entry, it was loaded from external sources // the new one may have been updated or modified, so replace the existing entry. We @@ -79,12 +109,5 @@ } - // XXX move this to top of loop? - pmKapaClose (); - - psFree (detections); - psFree (psf); psFree (header); - psFree (sources); - return true; } Index: branches/tap_branches/psphot/src/psphotReadoutFindPSF.c =================================================================== --- branches/tap_branches/psphot/src/psphotReadoutFindPSF.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotReadoutFindPSF.c (revision 27838) @@ -7,26 +7,12 @@ psTimerStart ("psphotReadout"); - // select the current recipe - psMetadata *recipe = psMetadataLookupPtr (NULL, config->recipes, PSPHOT_RECIPE); - if (!recipe) { - psError(PSPHOT_ERR_CONFIG, false, "missing recipe %s", PSPHOT_RECIPE); - return false; - } - - // set the photcode for this image - if (!psphotAddPhotcode(recipe, config, view, "PSPHOT.INPUT")) { + // set the photcode for the PSPHOT.INPUT + if (!psphotAddPhotcode(config, view)) { psError(PSPHOT_ERR_CONFIG, false, "trouble defining the photcode"); return false; } - // find the currently selected readout - pmReadout *readout = pmFPAfileThisReadout (config->files, view, "PSPHOT.INPUT"); - PS_ASSERT_PTR_NON_NULL (readout, false); - // Generate the mask and variance images, including the user-defined analysis region of interest - psphotSetMaskAndVariance (config, readout, recipe); - - // display the image, weight, mask (ch 1,2,3) - psphotVisualShowImage (readout); + psphotSetMaskAndVariance (config, view); // Note that in this implementation, we do NOT model the background and we do not @@ -34,40 +20,62 @@ // include externally-supplied sources (supplied as PSPHOT.INPUT.CMF) - pmDetections *detections = psphotDetectionsFromSources (config, inSources); - if (!detections || !detections->peaks) { + // XXX we assume a single set of input sources is supplied + if (!psphotDetectionsFromSources (config, view, inSources)) { psError(PSPHOT_ERR_ARGUMENTS, true, "Can't find PSF stars"); - return psphotReadoutCleanup(config, readout, recipe, detections, NULL, NULL); + return psphotReadoutCleanup(config, view); } - // construct sources and measure basic stats (moments, local sky) - psArray *sources = psphotSourceStats(config, readout, detections, true); - if (!sources) return false; + // construct detections->newSources and measure basic stats (moments, local sky) + if (!psphotSourceStats(config, view, true)) { + psError(PSPHOT_ERR_UNKNOWN, false, "failure to generate sources"); + return false; + } - // peak flux is wrong : set based on previous image - // use the peak measured in the moments analysis: - for (int i = 0; i < sources->n; i++) { - pmSource *source = sources->data[i]; - source->peak->flux = source->moments->Peak; + // peak flux is wrong : use the peak measured in the moments analysis: + if (!psphotRepairLoadedSources(config, view)) { + psError(PSPHOT_ERR_UNKNOWN, false, "failure to repair sources"); + return false; } // classify sources based on moments, brightness (psf is not known) - if (!psphotRoughClass (readout, sources, recipe, false)) { - psLogMsg ("psphot", 3, "failed to find a valid PSF clump for image"); - return psphotReadoutCleanup (config, readout, recipe, detections, NULL, sources); + if (!psphotRoughClass (config, view)) { + psError (PSPHOT_ERR_UNKNOWN, false, "failed to determine rough source class"); + return psphotReadoutCleanup (config, view); } - pmPSF *psf = psphotChoosePSF(readout, sources, recipe); - if (!psf) { + if (!psphotImageQuality (config, view)) { + psError (PSPHOT_ERR_UNKNOWN, false, "failed to measure image quality"); + return psphotReadoutCleanup(config, view); + } + + if (!psphotChoosePSF(config, view)) { psError(PSPHOT_ERR_PSF, false, "Failed to construct a psf model"); - psFree(sources); - return psphotReadoutCleanup(config, readout, recipe, detections, NULL, NULL); + return psphotReadoutCleanup(config, view); } - psphotVisualShowPSFModel(readout, psf); + +# if 0 + // XXX if we want to determine the aperture residual correction here, we either + // need to carry it out of the PSF determination analysis above, or save the model + // fits from that analysis, or run the linear PSF fit for all objects currently in hand + // construct an initial model for each object, set the radius to fitRadius, set circular fit mask + psphotGuessModels (config, view); +# endif + + // merge the newly selected sources into the existing list + // NOTE: merge OLD and NEW + psphotMergeSources (config, view); + +# if 0 + // measure aperture photometry corrections + if (!psphotApResid (config, view)) { + psLogMsg ("psphot", 3, "failed on psphotApResid"); + return psphotReadoutCleanup (config, view); + } +# endif // drop the references to the image pixels held by each source - psphotSourceFreePixels(sources); - psFree(sources); + psphotSourceFreePixels(config, view); // create the exported-metadata and free local data - return psphotReadoutCleanup(config, readout, recipe, detections, psf, NULL); + return psphotReadoutCleanup(config, view); } Index: branches/tap_branches/psphot/src/psphotReadoutKnownSources.c =================================================================== --- branches/tap_branches/psphot/src/psphotReadoutKnownSources.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotReadoutKnownSources.c (revision 27838) @@ -1,32 +1,18 @@ # include "psphotInternal.h" -// in this psphotReadout-variant, we are only measuring the photometry for known sources, -// using a PSF generated from this observation those sources +// in this psphotReadout-variant, we are only measuring the photometry for known sources, using +// a PSF generated for this observation from those sources bool psphotReadoutKnownSources(pmConfig *config, const pmFPAview *view, psArray *inSources) { psTimerStart ("psphotReadout"); - // select the current recipe - psMetadata *recipe = psMetadataLookupPtr (NULL, config->recipes, PSPHOT_RECIPE); - if (!recipe) { - psError(PSPHOT_ERR_CONFIG, false, "missing recipe %s", PSPHOT_RECIPE); - return false; - } - // set the photcode for this image - if (!psphotAddPhotcode(recipe, config, view, "PSPHOT.INPUT")) { + if (!psphotAddPhotcode(config, view)) { psError(PSPHOT_ERR_CONFIG, false, "trouble defining the photcode"); return false; } - // find the currently selected readout - pmReadout *readout = pmFPAfileThisReadout (config->files, view, "PSPHOT.INPUT"); - PS_ASSERT_PTR_NON_NULL (readout, false); - // Generate the mask and weight images, including the user-defined analysis region of interest - psphotSetMaskAndVariance (config, readout, recipe); - - // display the image, weight, mask (ch 1,2,3) - psphotVisualShowImage (readout); + psphotSetMaskAndVariance (config, view); // Note that in this implementation, we do NOT model the background and we do not @@ -34,58 +20,55 @@ // include externally-supplied sources (supplied as PSPHOT.INPUT.CMF) - pmDetections *detections = psphotDetectionsFromSources (config, inSources); - if (!detections || !detections->peaks) { + if (!psphotDetectionsFromSources (config, view, inSources)) { psError(PSPHOT_ERR_ARGUMENTS, true, "Can't find PSF stars"); - return psphotReadoutCleanup(config, readout, recipe, detections, NULL, NULL); + return psphotReadoutCleanup(config, view); } // construct sources and measure basic stats - psArray *sources = psphotSourceStats (config, readout, detections, true); - if (!sources) return false; + if (!psphotSourceStats (config, view, true)) { + psError(PSPHOT_ERR_UNKNOWN, false, "failure to generate sources"); + return false; + } - // peak flux is wrong : set based on previous image - // use the peak measured in the moments analysis: - for (int i = 0; i < sources->n; i++) { - pmSource *source = sources->data[i]; - source->peak->flux = source->moments->Peak; + // peak flux is wrong : use the peak measured in the moments analysis: + if (!psphotRepairLoadedSources(config, view)) { + psError(PSPHOT_ERR_UNKNOWN, false, "failure to repair sources"); + return false; } // classify sources based on moments, brightness (psf is not known) - if (!psphotRoughClass (readout, sources, recipe, false)) { - psLogMsg ("psphot", 3, "failed to find a valid PSF clump for image"); - return psphotReadoutCleanup (config, readout, recipe, detections, NULL, sources); + if (!psphotRoughClass (config, view)) { + psError (PSPHOT_ERR_UNKNOWN, false, "failed to determine rough source class"); + return psphotReadoutCleanup(config, view); } - pmPSF *psf = psphotChoosePSF (readout, sources, recipe); - if (!psf) { - psLogMsg ("psphot", 3, "failure to construct a psf model"); - return psphotReadoutCleanup (config, readout, recipe, detections, psf, sources); + if (!psphotChoosePSF (config, view)) { + psError(PSPHOT_ERR_PSF, false, "Failed to construct a psf model"); + return psphotReadoutCleanup(config, view); } - psphotVisualShowPSFModel (readout, psf); // construct an initial model for each object - psphotGuessModels (config, readout, sources, psf); + psphotGuessModels (config, view); + + // merge the newly selected sources into the existing list + // NOTE: merge OLD and NEW + psphotMergeSources (config, view); // linear PSF fit to source peaks - psphotFitSourcesLinear (readout, sources, recipe, psf, FALSE); - - // We have to place these visualizations here because the models are not realized until - // psphotGuessModels or fitted until psphotFitSourcesLinear. - psphotVisualShowPSFStars (recipe, psf, sources); - psphotVisualShowSatStars (recipe, psf, sources); + psphotFitSourcesLinear (config, view, false); // measure aperture photometry corrections - if (!psphotApResid (config, readout, sources, psf)) { + if (!psphotApResid (config, view)) { psLogMsg ("psphot", 3, "failed on psphotApResid"); - return psphotReadoutCleanup (config, readout, recipe, detections, psf, sources); + return psphotReadoutCleanup(config, view); } // calculate source magnitudes - psphotMagnitudes(config, readout, view, sources, psf); + psphotMagnitudes(config, view); // drop the references to the image pixels held by each source - psphotSourceFreePixels (sources); + psphotSourceFreePixels (config, view); // create the exported-metadata and free local data - return psphotReadoutCleanup(config, readout, recipe, detections, psf, sources); + return psphotReadoutCleanup(config, view); } Index: branches/tap_branches/psphot/src/psphotReadoutMinimal.c =================================================================== --- branches/tap_branches/psphot/src/psphotReadoutMinimal.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotReadoutMinimal.c (revision 27838) @@ -1,7 +1,7 @@ # include "psphotInternal.h" -// this badly-named function performs photometry assuming (a) a supplied PSF, (b) -// background subtraction, (c) linear psf-model fits only. it is currently only being -// used by ppSub. +// this badly-named function performs photometry assuming (a) a supplied PSF, (b) background +// subtraction, (c) linear psf-model fits only, (d) a prior analysis has supplied the moments +// window parameters. It is currently only being used by ppSub. // NOTE: ppSub needs to perform extended source analysis for comets and trails. @@ -17,87 +17,70 @@ pmModelClassSetLimits(PM_MODEL_LIMITS_LAX); - // select the current recipe - psMetadata *recipe = psMetadataLookupPtr (NULL, config->recipes, PSPHOT_RECIPE); - if (!recipe) { - psError(PSPHOT_ERR_CONFIG, false, "missing recipe %s", PSPHOT_RECIPE); - return false; - } - // set the photcode for this image - if (!psphotAddPhotcode(recipe, config, view, "PSPHOT.INPUT")) { + if (!psphotAddPhotcode(config, view)) { psError(PSPHOT_ERR_CONFIG, false, "trouble defining the photcode"); return false; } - // find the currently selected readout - pmReadout *readout = pmFPAfileThisReadout (config->files, view, "PSPHOT.INPUT"); - PS_ASSERT_PTR_NON_NULL (readout, false); + // Generate the mask and weight images, including the user-defined analysis region of interest + psphotSetMaskAndVariance (config, view); - // Generate the mask and weight images, including the user-defined analysis region of interest - psphotSetMaskAndVariance (config, readout, recipe); - - // display the image, weight, mask (ch 1,2,3) - psphotVisualShowImage (readout); - - // load the psf model, if suppled. FWHM_X,FWHM_Y,etc are saved in the recipe - pmPSF *psf = psphotLoadPSF (config, view, recipe); - if (!psf) { + // load the psf model, if suppled. FWHM_X,FWHM_Y,etc are saved on readout->analysis + if (!psphotLoadPSF (config, view)) { psError (PSPHOT_ERR_CONFIG, false, "missing psf model"); - return psphotReadoutCleanup (config, readout, recipe, NULL, NULL, NULL); + return psphotReadoutCleanup (config, view); } - // find the detections (by peak and/or footprint) in the image. - pmDetections *detections = pmDetectionsAlloc(); // New detections; allocated to ensure pass=2 - detections = psphotFindDetections(detections, readout, recipe); - if (!detections) { - psLogMsg ("psphot", 3, "unable to find detections in this image"); - return psphotReadoutCleanup (config, readout, recipe, detections, psf, NULL); + // find the detections (by peak and/or footprint) in the image. (final pass) + if (!psphotFindDetections(config, view, false)) { + psError (PSPHOT_ERR_UNKNOWN, false, "failure in peak analysis"); + return psphotReadoutCleanup (config, view); } - // construct sources and measure basic stats - psArray *sources = psphotSourceStats (config, readout, detections, true); - if (!sources) return false; + // construct sources and measure basic stats (saved on detections->newSources) + if (!psphotSourceStats (config, view, false)) { // pass 1 + psError(PSPHOT_ERR_UNKNOWN, false, "failure to generate sources"); + return psphotReadoutCleanup (config, view); + } // find blended neighbors of very saturated stars - // XXX merge this with Basic Deblend? - psphotDeblendSatstars (readout, sources, recipe); + psphotDeblendSatstars (config, view); // mark blended peaks PS_SOURCE_BLEND - if (!psphotBasicDeblend (sources, recipe)) { + if (!psphotBasicDeblend (config, view)) { psLogMsg ("psphot", 3, "failed on deblend analysis"); - return psphotReadoutCleanup (config, readout, recipe, detections, psf, sources); + return psphotReadoutCleanup (config, view); } // classify sources based on moments, brightness (use supplied psf shape parameters) - if (!psphotRoughClass (readout, sources, recipe, true)) { + if (!psphotRoughClass (config, view)) { psLogMsg ("psphot", 3, "failed to find a valid PSF clump for image"); - return psphotReadoutCleanup (config, readout, recipe, detections, psf, sources); + return psphotReadoutCleanup (config, view); } // construct an initial model for each object - psphotGuessModels (config, readout, sources, psf); + psphotGuessModels (config, view); + + // merge the newly selected sources into the existing list + psphotMergeSources (config, view); // linear PSF fit to source peaks - psphotFitSourcesLinear (readout, sources, recipe, psf, FALSE); - - // We have to place these visualizations here because the models are not realized until - // psphotGuessModels or fitted until psphotFitSourcesLinear. - psphotVisualShowPSFStars (recipe, psf, sources); - psphotVisualShowSatStars (recipe, psf, sources); + psphotFitSourcesLinear (config, view, false); // XXX eventually, add the extended source fits here # if (0) // measure source size for the remaining sources - psphotSourceSize (config, readout, sources, recipe, 0); + psphotSourceSize (config, view); - psphotExtendedSourceAnalysis (readout, sources, recipe); + psphotExtendedSourceAnalysis (config, view); - psphotExtendedSourceFits (readout, sources, recipe); + psphotExtendedSourceFits (config, view); # endif // calculate source magnitudes - psphotMagnitudes(config, readout, view, sources, psf); + psphotMagnitudes(config, view); - if (!psphotEfficiency(config, readout, view, psf, recipe, sources)) { + // XXX ensure this is measured if the analysis succeeds (even if quality is low) + if (!psphotEfficiency(config, view)) { psErrorStackPrint(stderr, "Unable to determine detection efficiencies from fake sources"); psErrorClear(); @@ -105,7 +88,7 @@ // drop the references to the image pixels held by each source - psphotSourceFreePixels (sources); + psphotSourceFreePixels (config, view); // create the exported-metadata and free local data - return psphotReadoutCleanup(config, readout, recipe, detections, psf, sources); + return psphotReadoutCleanup(config, view); } Index: branches/tap_branches/psphot/src/psphotReplaceUnfit.c =================================================================== --- branches/tap_branches/psphot/src/psphotReplaceUnfit.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotReplaceUnfit.c (revision 27838) @@ -22,5 +22,27 @@ } -bool psphotReplaceAllSources (psArray *sources, psMetadata *recipe) { +// for now, let's store the detections on the readout->analysis for each readout +bool psphotReplaceAllSources (pmConfig *config, const pmFPAview *view) +{ + bool status = true; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (!psphotReplaceAllSourcesReadout (config, view, "PSPHOT.INPUT", i, recipe)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to replace all sources for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + +bool psphotReplaceAllSourcesReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe) { bool status; @@ -29,7 +51,20 @@ psTimerStart ("psphot.replace"); + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *sources = detections->allSources; + psAssert (sources, "missing sources?"); + // user-defined masks to test for good/bad pixels (build from recipe list if not yet set) psImageMaskType maskVal = psMetadataLookupImageMask(&status, recipe, "MASK.PSPHOT"); // Mask value for bad pixels - assert (maskVal); + psAssert (maskVal, "missing mask value?"); for (int i = 0; i < sources->n; i++) { @@ -67,42 +102,2 @@ return true; } - -# if (0) -// add source, if the source has been subtracted; do not modify state -bool psphotAddWithTest (pmSource *source, bool useState, psImageMaskType maskVal) { - - // what is current state? (true : add; false : sub) - bool state = !(source->tmpFlags & PM_SOURCE_TMPF_SUBTRACTED); - if (state && useState) return true; - - pmSourceAdd (source, PM_MODEL_OP_FULL, maskVal); - return true; -} - -// sub source, if the source has been added; do not modify state -bool psphotSubWithTest (pmSource *source, bool useState, psImageMaskType maskVal) { - - // what is current state? (true : sub; false : add) - bool state = (source->tmpFlags & PM_SOURCE_TMPF_SUBTRACTED); - if (state && useState) return true; - - pmSourceSub (source, PM_MODEL_OP_FULL, maskVal); - return true; -} - -// add or sub source to match recorded state: supply current state as true (add) or false (sub) -bool psphotSetState (pmSource *source, bool curState, psImageMaskType maskVal) { - - // what is desired state? (true : add; false : sub) - bool newState = !(source->tmpFlags & PM_SOURCE_TMPF_SUBTRACTED); - if (curState == newState) return true; - - if (curState && !newState) { - pmSourceSub (source, PM_MODEL_OP_FULL, maskVal); - } - if (newState && !curState) { - pmSourceAdd (source, PM_MODEL_OP_FULL, maskVal); - } - return true; -} -# endif Index: branches/tap_branches/psphot/src/psphotRoughClass.c =================================================================== --- branches/tap_branches/psphot/src/psphotRoughClass.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotRoughClass.c (revision 27838) @@ -7,12 +7,60 @@ } } -bool psphotRoughClassRegion (int nRegion, psRegion *region, psArray *sources, psMetadata *target, psMetadata *recipe, const bool havePSF); +// for now, let's store the detections on the readout->analysis for each readout +bool psphotRoughClass (pmConfig *config, const pmFPAview *view) +{ + bool status = true; -// 2006.02.02 : no leaks -bool psphotRoughClass (pmReadout *readout, psArray *sources, psMetadata *recipe, const bool havePSF) { + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // skip the chisq image (optionally?) + int chisqNum = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.CHISQ.NUM"); + if (!status) chisqNum = -1; + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (i == chisqNum) continue; // skip chisq image + if (!psphotRoughClassReadout (config, view, "PSPHOT.INPUT", i, recipe)) { + psError (PSPHOT_ERR_CONFIG, false, "failed on rough classification for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + +bool psphotRoughClassReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe) { bool status; psTimerStart ("psphot.rough"); + + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + // if we have a PSF, use the existing PSF clump region below + bool havePSF = false; + if (psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.PSF")) { + havePSF = true; + } + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *sources = detections->newSources; + psAssert (sources, "missing sources?"); + + if (!sources->n) { + psLogMsg ("psphot", PS_LOG_INFO, "no sources, skipping rough classification"); + return true; + } // we make this measurement on a NxM grid of regions across the readout @@ -37,5 +85,5 @@ } } - psMetadataAddS32 (recipe, PS_LIST_TAIL, "PSF.CLUMP.NREGIONS", PS_META_REPLACE, "psf clump regions", nRegion); + psMetadataAddS32 (readout->analysis, PS_LIST_TAIL, "PSF.CLUMP.NREGIONS", PS_META_REPLACE, "psf clump regions", nRegion); // optional printout of source moments only @@ -51,5 +99,5 @@ } -bool psphotRoughClassRegion (int nRegion, psRegion *region, psArray *sources, psMetadata *target, psMetadata *recipe, const bool havePSF) { +bool psphotRoughClassRegion (int nRegion, psRegion *region, psArray *sources, psMetadata *analysis, psMetadata *recipe, const bool havePSF) { bool status; @@ -61,10 +109,11 @@ assert (maskSat); + // the regions are saved on the readout->analysis metadata folder (passed to us as analysis) snprintf (regionName, 64, "PSF.CLUMP.REGION.%03d", nRegion); - psMetadata *regionMD = psMetadataLookupPtr (&status, target, regionName); + psMetadata *regionMD = psMetadataLookupPtr (&status, analysis, regionName); if (!regionMD) { // allocate the region metadata folder and add this region to it. regionMD = psMetadataAlloc(); - psMetadataAddMetadata (target, PS_LIST_TAIL, regionName, PS_META_REPLACE, "psf clump region", regionMD); + psMetadataAddMetadata (analysis, PS_LIST_TAIL, regionName, PS_META_REPLACE, "psf clump region", regionMD); psFree (regionMD); } @@ -74,5 +123,26 @@ // determine the PSF parameters from the source moment values // XXX why not save the psfClump as a PTR? - psfClump = pmSourcePSFClump (region, sources, target); + + float PSF_SN_LIM = psMetadataLookupF32(&status, recipe, "PSF_SN_LIM"); psAssert (status, "missing PSF_SN_LIM"); + float MOMENTS_AR_MAX = psMetadataLookupF32(&status, recipe, "MOMENTS_AR_MAX"); psAssert (status, "missing MOMENTS_AR_MAX"); + + float PSF_CLUMP_GRID_SCALE = psMetadataLookupF32(&status, analysis, "PSF_CLUMP_GRID_SCALE"); + if (!status) { + PSF_CLUMP_GRID_SCALE = psMetadataLookupF32(&status, recipe, "PSF_CLUMP_GRID_SCALE"); + psAssert (status, "missing PSF_CLUMP_GRID_SCALE"); + } + float MOMENTS_SX_MAX = psMetadataLookupF32(&status, analysis, "MOMENTS_SX_MAX"); + if (!status) { + MOMENTS_SX_MAX = psMetadataLookupF32(&status, recipe, "MOMENTS_SX_MAX"); + psAssert (status, "missing MOMENTS_SX_MAX"); + } + float MOMENTS_SY_MAX = psMetadataLookupF32(&status, analysis, "MOMENTS_SY_MAX"); + if (!status) { + MOMENTS_SY_MAX = psMetadataLookupF32(&status, recipe, "MOMENTS_SY_MAX"); + psAssert (status, "missing MOMENTS_SY_MAX"); + } + + psfClump = pmSourcePSFClump (NULL, region, sources, PSF_SN_LIM, PSF_CLUMP_GRID_SCALE, MOMENTS_SX_MAX, MOMENTS_SY_MAX, MOMENTS_AR_MAX); + psMetadataAddF32 (regionMD, PS_LIST_TAIL, "PSF.CLUMP.X", PS_META_REPLACE, "psf clump center", psfClump.X); psMetadataAddF32 (regionMD, PS_LIST_TAIL, "PSF.CLUMP.Y", PS_META_REPLACE, "psf clump center", psfClump.Y); @@ -102,6 +172,10 @@ psLogMsg ("psphot", 3, "psf clump DX, DY: %f, %f\n", psfClump.dX, psfClump.dY); + // get basic parameters, or set defaults + float PSF_SN_LIM = psMetadataLookupF32 (&status, recipe, "PSF_SN_LIM"); psAssert (status, "missing PSF_SN_LIM"); + float PSF_CLUMP_NSIGMA = psMetadataLookupF32 (&status, recipe, "PSF_CLUMP_NSIGMA"); psAssert (status, "missing PSF_CLUMP_NSIGMA"); + // group into STAR, COSMIC, EXTENDED, SATURATED, etc. - if (!pmSourceRoughClass (region, sources, recipe, psfClump, maskSat)) { + if (!pmSourceRoughClass (region, sources, PSF_SN_LIM, PSF_CLUMP_NSIGMA, psfClump, maskSat)) { psError(PSPHOT_ERR_PROG, false, "programming error calling pmSourceRoughClass"); return false; Index: branches/tap_branches/psphot/src/psphotSetThreads.c =================================================================== --- branches/tap_branches/psphot/src/psphotSetThreads.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotSetThreads.c (revision 27838) @@ -25,5 +25,5 @@ psFree(task); - task = psThreadTaskAlloc("PSPHOT_SOURCE_STATS", 5); + task = psThreadTaskAlloc("PSPHOT_SOURCE_STATS", 10); task->function = &psphotSourceStats_Threaded; psThreadTaskAdd(task); Index: branches/tap_branches/psphot/src/psphotSignificanceImage.c =================================================================== --- branches/tap_branches/psphot/src/psphotSignificanceImage.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotSignificanceImage.c (revision 27838) @@ -22,7 +22,8 @@ } + // if we have already determined the PSF model, then we have a better idea how to smooth this image bool statusMajor, statusMinor; - float fwhmMajor = psMetadataLookupF32(&statusMajor, recipe, "FWHM_MAJ"); - float fwhmMinor = psMetadataLookupF32(&statusMinor, recipe, "FWHM_MIN"); + float fwhmMajor = psMetadataLookupF32(&statusMajor, readout->analysis, "FWHM_MAJ"); + float fwhmMinor = psMetadataLookupF32(&statusMinor, readout->analysis, "FWHM_MIN"); if (statusMajor && statusMinor) { // if we know the FHWM, use that to set the smoothing kernel (XXX allow an optional override?) @@ -43,6 +44,5 @@ } // record the actual smoothing sigma - psMetadataAddF32(recipe, PS_LIST_TAIL, "SIGMA_SMOOTH", PS_META_REPLACE, "Smoothing sigma for detections", - SIGMA_SMTH); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "SIGMA_SMOOTH", PS_META_REPLACE, "Smoothing sigma for detections", SIGMA_SMTH); // smooth the image, applying the mask as we go @@ -59,6 +59,5 @@ // effective per-pixel variance is maintained. psImage *smooth_wt = psImageCopy(NULL, readout->variance, PS_TYPE_F32); - psImageSmoothMask_Threaded(smooth_wt, smooth_wt, readout->mask, maskVal, SIGMA_SMTH * M_SQRT1_2, - NSIGMA_SMTH, minGauss); + psImageSmoothMask_Threaded(smooth_wt, smooth_wt, readout->mask, maskVal, SIGMA_SMTH * M_SQRT1_2, NSIGMA_SMTH, minGauss); psLogMsg("psphot", PS_LOG_MINUTIA, "smooth variance: %f sec\n", psTimerMark("psphot.smooth")); @@ -77,14 +76,11 @@ // Calculate correction factor for the covariance produced by the (potentially multiple) smoothing psKernel *kernel = psImageSmoothKernel(SIGMA_SMTH, NSIGMA_SMTH); // Kernel used for smoothing - psKernel *covar = psImageCovarianceCalculate(kernel, readout->covariance); // Covariance matrix + float factor = 1.0 / psImageCovarianceCalculateFactor(kernel, readout->covariance); psFree(kernel); - float factor = 1.0 / psImageCovarianceFactor(covar); - psFree(covar); // record the effective area and significance scaling factor float effArea = 8.0 * M_PI * PS_SQR(SIGMA_SMTH); - psMetadataAddF32(recipe, PS_LIST_TAIL, "EFFECTIVE_AREA", PS_META_REPLACE, "Effective Area", effArea); - psMetadataAddF32(recipe, PS_LIST_TAIL, "SIGNIFICANCE_SCALE_FACTOR", PS_META_REPLACE, - "Signicance scale factor", factor); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "EFFECTIVE_AREA", PS_META_REPLACE, "Effective Area", effArea); + psMetadataAddF32(readout->analysis, PS_LIST_TAIL, "SIGNIFICANCE_SCALE_FACTOR", PS_META_REPLACE, "Signicance scale factor", factor); // XXX multithread this if needed Index: branches/tap_branches/psphot/src/psphotSkyReplace.c =================================================================== --- branches/tap_branches/psphot/src/psphotSkyReplace.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotSkyReplace.c (revision 27838) @@ -1,13 +1,38 @@ # include "psphotInternal.h" + +bool psphotSkyReplace (pmConfig *config, const pmFPAview *view) +{ + bool status = true; + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // skip the chisq image (optionally?) + int chisqNum = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.CHISQ.NUM"); + if (!status) chisqNum = -1; + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (i == chisqNum) continue; // skip chisq image + if (!psphotSkyReplaceReadout (config, view, "PSPHOT.INPUT", i)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to replace sky for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} // XXX make this an option? // in order to successfully replace the sky, we must define a corresponding file... -bool psphotSkyReplace (pmConfig *config, const pmFPAview *view) { +bool psphotSkyReplaceReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index) { psTimerStart ("psphot.skyreplace"); // find the currently selected readout - pmReadout *readout = pmFPAfileThisReadout (config->files, view, "PSPHOT.INPUT"); - if (readout == NULL) psAbort("input not defined"); + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); // select background pixels, from output background file, or create Index: branches/tap_branches/psphot/src/psphotSourceFits.c =================================================================== --- branches/tap_branches/psphot/src/psphotSourceFits.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotSourceFits.c (revision 27838) @@ -219,4 +219,7 @@ double chiTrend; pmModel *ONE = NULL; + pmSource *tmpSrc = NULL; + pmModel *EXT = NULL; + psArray *DBL = NULL; // skip the source if we don't think it is extended @@ -232,5 +235,5 @@ // at this stage, skip Gaussian windowing, and do not clip pixels by S/N // this uses the footprint to judge both radius and aperture? - if (!pmSourceMoments (source, radius, 0.0, 0.0)) return false; + if (!pmSourceMoments (source, radius, 0.0, 0.0, maskVal)) return false; psTrace ("psphot", 5, "trying blob...\n"); @@ -240,9 +243,10 @@ // this temporary source is used as a place-holder by the psphotEval functions below - pmSource *tmpSrc = pmSourceAlloc (); + tmpSrc = pmSourceAlloc (); // XXX need to handle failures better here - pmModel *EXT = psphotFitEXT (readout, source, modelTypeEXT, maskVal, markVal); - if (!isfinite(EXT->params->data.F32[PM_PAR_I0])) psAbort("nan in fit"); + EXT = psphotFitEXT (readout, source, modelTypeEXT, maskVal, markVal); + if (!EXT) goto escape; + if (!isfinite(EXT->params->data.F32[PM_PAR_I0])) goto escape; okEXT = psphotEvalEXT (tmpSrc, EXT); @@ -250,5 +254,8 @@ // DBL will always be defined, but DBL->data[n] might not - psArray *DBL = psphotFitDBL (readout, source, maskVal, markVal); + DBL = psphotFitDBL (readout, source, maskVal, markVal); + if (!DBL) goto escape; + if (!DBL->n) goto escape; + okDBL = psphotEvalDBL (tmpSrc, DBL->data[0]); okDBL &= psphotEvalDBL (tmpSrc, DBL->data[1]); @@ -362,4 +369,11 @@ psFree (DBL); return true; + +escape: + psImageMaskPixels (source->maskObj, "AND", PS_NOT_IMAGE_MASK(markVal)); + psFree (tmpSrc); + psFree (EXT); + psFree (DBL); + return false; } @@ -425,5 +439,8 @@ // use the source moments, etc to guess basic model parameters pmModel *EXT = pmSourceModelGuess (source, modelType); - PS_ASSERT (EXT, NULL); + if (!EXT) { + psTrace ("psphot", 5, "failed to generate a model for source: moments: %f %f\n", source->moments->Mxx, source->moments->Myy); + return NULL; + } if ((source->moments->Mxx < 1e-3) || (source->moments->Myy < 1e-3)) { Index: branches/tap_branches/psphot/src/psphotSourceFreePixels.c =================================================================== --- branches/tap_branches/psphot/src/psphotSourceFreePixels.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotSourceFreePixels.c (revision 27838) @@ -1,7 +1,37 @@ # include "psphotInternal.h" -void psphotSourceFreePixels (psArray *sources) { +bool psphotSourceFreePixels (pmConfig *config, const pmFPAview *view) +{ + bool status = true; - if (!sources) return; + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (!psphotSourceFreePixelsReadout (config, view, "PSPHOT.INPUT", i)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to free source pixels for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + +bool psphotSourceFreePixelsReadout(pmConfig *config, const pmFPAview *view, const char *filename, int index) { + + bool status; + + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *sources = detections->allSources; + psAssert (sources, "missing sources?"); for (int i = 0; i < sources->n; i++) { @@ -9,4 +39,4 @@ pmSourceFreePixels (source); } - return; + return true; } Index: branches/tap_branches/psphot/src/psphotSourceMatch.c =================================================================== --- branches/tap_branches/psphot/src/psphotSourceMatch.c (revision 27838) +++ branches/tap_branches/psphot/src/psphotSourceMatch.c (revision 27838) @@ -0,0 +1,256 @@ +# include "psphotInternal.h" + +bool psphotMatchSourcesGenerate (pmConfig *config, const pmFPAview *view, psArray *objects); + +psArray *psphotMatchSources (pmConfig *config, const pmFPAview *view) +{ + bool status = true; + + psArray *objects = psArrayAllocEmpty(100); + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (!psphotMatchSourcesReadout (objects, config, view, "PSPHOT.INPUT", i)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to merge sources for PSPHOT.INPUT entry %d", i); + psFree (objects); + return NULL; + } + } + + psphotMatchSourcesGenerate (config, view, objects); + + return objects; +} + +bool psphotMatchSourcesReadout (psArray *objects, pmConfig *config, const pmFPAview *view, char *filename, int index) { + + bool status = false; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + int RADIUS = psMetadataLookupF32 (&status, recipe, "PSPHOT.STACK.MATCH.RADIUS"); + psAssert (status, "programming error: must define PSPHOT.STACK.MATCH.RADIUS"); + + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + psAssert (detections->newSources, "new sources not defined?"); + psAssert (!detections->allSources, "all sources already defined?"); + + // XXX TEST: + if (detections->newSources) { + psphotMatchSourcesToObjects(objects, detections->newSources, RADIUS); + } + + return true; +} + +# define NEXT1 { i++; continue; } +# define NEXT2 { j++; continue; } +bool psphotMatchSourcesToObjects (psArray *objects, psArray *sources, float RADIUS) { + + float dx, dy; + + float RADIUS2 = RADIUS*RADIUS; + + // sort the source list by X + sources = psArraySort (sources, pmSourceSortByX); + objects = psArraySort (objects, pmPhotObjSortByX); + + psVector *found = psVectorAlloc(sources->n, PS_TYPE_U8); + psVectorInit (found, 0); + + // match sources to existing objects + + psLogMsg ("psphot", PS_LOG_DETAIL, "attempt to match sources (%ld vs %ld)", sources->n, objects->n); + + int i, j; + for (i = j = 0; (i < sources->n) && (j < objects->n); ) { + + pmSource *src = sources->data[i]; + pmPhotObj *obj = objects->data[j]; + + if (!src) NEXT1; + if (!src->peak) NEXT1; + if (!finite(src->peak->xf)) NEXT1; + if (!finite(src->peak->yf)) NEXT1; + + if (!obj) NEXT2; + if (!finite(obj->x)) NEXT2; + if (!finite(obj->y)) NEXT2; + + dx = src->peak->xf - obj->x; + if (dx < -1.02*RADIUS) NEXT1; + if (dx > +1.02*RADIUS) NEXT2; + + // we are within match range, look for matches: + int Jmin = -1; + float Rmin = RADIUS2; + for (int J = j; (dx > -1.02*RADIUS) && (J < objects->n); J++) { + + obj = objects->data[J]; + + dx = src->peak->xf - obj->x; + dy = src->peak->yf - obj->y; + + float dr = dx*dx + dy*dy; + if (dr > RADIUS2) continue; + if (dr < Rmin) { + Rmin = dr; + Jmin = J; + } + } + + // no match, try next source + if (Jmin == -1) { + i++; + continue; + } + obj = objects->data[Jmin]; + + // add to object + pmPhotObjAddSource (obj, src); + found->data.U8[i] = 1; + i++; + } + + // add missed sources to new objects + + for (i = 0; i < sources->n; i++) { + + if (found->data.U8[i]) continue; + + pmSource *src = sources->data[i]; + + pmPhotObj *obj = pmPhotObjAlloc(); + pmPhotObjAddSource(obj, src); + psArrayAdd (objects, 100, obj); + psFree (obj); + } + psLogMsg ("psphot", PS_LOG_DETAIL, "matched sources (%ld vs %ld)", sources->n, objects->n); + + psFree (found); + return true; +} + +bool psphotMatchSourcesGenerate (pmConfig *config, const pmFPAview *view, psArray *objects) { + + bool status = false; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + // determine properties (sky, moments) of initial sources + float OUTER = psMetadataLookupF32 (&status, recipe, "SKY_OUTER_RADIUS"); + psAssert (status, "missing SKY_OUTER_RADIUS in recipe?"); + + int nImages = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // generate look-up arrays for detections and readouts + psArray *detArrays = psArrayAlloc(nImages); + psArray *readouts = psArrayAlloc(nImages); + + for (int i = 0; i < nImages; i++) { + + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, "PSPHOT.INPUT", i); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + detArrays->data[i] = psMemIncrRefCounter(detections); + readouts->data[i] = psMemIncrRefCounter(readout); + } + + // vector to track if source for an image is found + psVector *found = psVectorAlloc(nImages, PS_TYPE_U8); + + for (int i = 0; i < objects->n; i++) { + pmPhotObj *obj = objects->data[i]; + + // mark the images for which sources have been found + psVectorInit (found, 0); + for (int j = 0; j < obj->sources->n; j++) { + + pmSource *src = obj->sources->data[j]; + int index = src->imageID; + psAssert (index >= 0, "invalid index"); + psAssert (index < found->n, "invalid index"); + + found->data.U8[index] = 1; + } + + // generate new sources for the image that are missing + for (int index = 0; index < found->n; index++) { + if (found->data.U8[index]) continue; + + pmDetections *detections = detArrays->data[index]; + pmReadout *readout = readouts->data[index]; + int row0 = readout->image->row0; + int col0 = readout->image->col0; + + // XXX the peak type is not really used in psphot + // PM_PEAK_LONE is certainly not true, but irrelevant + float peakFlux = readout->image->data.F32[(int)(obj->y-row0-0.5)][(int)(obj->x-col0-0.5)]; + pmPeak *peak = pmPeakAlloc(obj->x, obj->y, peakFlux, PM_PEAK_LONE); + peak->flux = peakFlux; + peak->SN = 1.0; + peak->xf = obj->x; + peak->yf = obj->y; + peak->dx = NAN; + peak->dy = NAN; + + // XXX assign to a footprint? + + // create a new source + pmSource *source = pmSourceAlloc(); + source->imageID = index; + + // add the peak + source->peak = psMemIncrRefCounter(peak); + + // allocate space for moments + source->moments = pmMomentsAlloc(); + + // allocate image, weight, mask arrays for each peak (square of radius OUTER) + pmSourceDefinePixels (source, readout, source->peak->x, source->peak->y, OUTER); + + peak->assigned = true; + pmPhotObjAddSource(obj, source); + psArrayAdd (detections->newSources, 100, source); + psFree (source); + psFree (peak); + } + } + + // how many sources do we have now? + int nSources = 0; + for (int i = 0; i < objects->n; i++) { + pmPhotObj *obj = objects->data[i]; + nSources += obj->sources->n; + } + psLogMsg ("psphot", PS_LOG_DETAIL, "total of %d sources for %d images", nSources, nImages); + + + psFree (found); + psFree (detArrays); + psFree (readouts); + return true; +} Index: branches/tap_branches/psphot/src/psphotSourceSize.c =================================================================== --- branches/tap_branches/psphot/src/psphotSourceSize.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotSourceSize.c (revision 27838) @@ -13,12 +13,16 @@ float soft; int grow; + int xtest, ytest; + bool apply; // apply CR mask? } psphotSourceSizeOptions; +// local functions: bool psphotSourceSizePSF (psphotSourceSizeOptions *options, psArray *sources, pmPSF *psf); bool psphotSourceClass (pmReadout *readout, psArray *sources, psMetadata *recipe, pmPSF *psf, psphotSourceSizeOptions *options); bool psphotSourceClassRegion (psRegion *region, pmPSFClump *psfClump, psArray *sources, psMetadata *recipe, pmPSF *psf, psphotSourceSizeOptions *options); bool psphotSourceSizeCR (pmReadout *readout, psArray *sources, psphotSourceSizeOptions *options); -bool psphotMaskCosmicRay (psImage *mask, pmSource *source, psImageMaskType maskVal, psImageMaskType crMask); -bool psphotMaskCosmicRayIsophot (pmSource *source, psImageMaskType maskVal, psImageMaskType crMask); +bool psphotMaskCosmicRay (pmReadout *readout, pmSource *source, psImageMaskType maskVal); +bool psphotMaskCosmicRayFootprintCheck (psArray *sources); +int psphotMaskCosmicRayConnected (int xPeak, int yPeak, psImage *mymask, psImage *myvar, psImage *edges, int binning, float sigma_thresh); // we need to call this function after sources have been fitted to the PSF model and @@ -28,6 +32,33 @@ // deviation from the psf model at the r = FWHM/2 position -// XXX use an internal flag to mark sources which have already been measured -bool psphotSourceSize(pmConfig *config, pmReadout *readout, psArray *sources, psMetadata *recipe, pmPSF *psf, long first) +// for now, let's store the detections on the readout->analysis for each readout +bool psphotSourceSize (pmConfig *config, const pmFPAview *view, bool getPSFsize) +{ + bool status = true; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // skip the chisq image (optionally?) + int chisqNum = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.CHISQ.NUM"); + if (!status) chisqNum = -1; + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (i == chisqNum) continue; // skip chisq image + if (!psphotSourceSizeReadout (config, view, "PSPHOT.INPUT", i, recipe, getPSFsize)) { + psError (PSPHOT_ERR_CONFIG, false, "failed on source size analysis for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + +// this function use an internal flag to mark sources which have already been measured +bool psphotSourceSizeReadout(pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe, bool getPSFsize) { bool status; @@ -36,4 +67,25 @@ psTimerStart ("psphot.size"); + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + psArray *sources = detections->allSources; + psAssert (sources, "missing sources?"); + + if (!sources->n) { + psLogMsg ("psphot", PS_LOG_INFO, "no sources, skipping source size"); + return true; + } + + pmPSF *psf = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.PSF"); + psAssert (psf, "missing psf?"); + // user-defined masks to test for good/bad pixels (build from recipe list if not yet set) options.maskVal = psMetadataLookupImageMask(&status, recipe, "MASK.PSPHOT"); // Mask value for bad pixels @@ -55,4 +107,9 @@ // XXX recipe name is not great options.nSigmaMoments = psMetadataLookupF32 (&status, recipe, "PSPHOT.EXT.NSIGMA.MOMENTS"); + assert (status); + + // XXX recipe name is not great + options.xtest = psMetadataLookupS32 (&status, recipe, "PSPHOT.CRMASK.XTEST"); + options.ytest = psMetadataLookupS32 (&status, recipe, "PSPHOT.CRMASK.YTEST"); assert (status); @@ -69,26 +126,614 @@ } + options.apply = psMetadataLookupBool(&status, recipe, "PSPHOT.CRMASK.APPLY"); // Growth size for CRs + if (!status) { + psError(PS_ERR_BAD_PARAMETER_VALUE, true, "PSPHOT.CRMASK.APPLY is not defined."); + return false; + } + // We are using the value psfMag - 2.5*log10(moment.Sum) as a measure of the extendedness - // of and object. We need to model this distribution for the PSF stars before we can test + // of an object. We need to model this distribution for the PSF stars before we can test // the significance for a specific object // XXX move this to the code that generates the PSF? // XXX store the results on pmPSF? - psphotSourceSizePSF (&options, sources, psf); + + // XXX this should only be done on the first pass (ie, if we have newSources or allSources?) + if (getPSFsize) { + psphotSourceSizePSF (&options, sources, psf); + } // classify the sources based on ApResid and Moments (extended sources) + // NOTE: only sources not already measured !(source->tmpFlags & PM_SOURCE_TMPF_SIZE_MEASURED) psphotSourceClass(readout, sources, recipe, psf, &options); + // NOTE: only sources not already measured !(source->tmpFlags & PM_SOURCE_TMPF_SIZE_MEASURED) psphotSourceSizeCR (readout, sources, &options); - psLogMsg ("psphot.size", PS_LOG_INFO, "measure source sizes for %ld sources: %f sec\n", sources->n - first, psTimerMark ("psphot.size")); + // XXX fix this (was source->n - first) + psLogMsg ("psphot.size", PS_LOG_INFO, "measure source sizes for %ld sources: %f sec\n", sources->n, psTimerMark ("psphot.size")); psphotVisualPlotSourceSize (recipe, readout->analysis, sources); psphotVisualShowSourceSize (readout, sources); psphotVisualPlotApResid (sources, options.ApResid, options.ApSysErr); + psphotVisualShowSatStars (recipe, psf, sources); return true; } -bool psphotMaskCosmicRay (psImage *mask, pmSource *source, psImageMaskType maskVal, psImageMaskType crMask) { +// model the apmifit distribution for the psf stars: +bool psphotSourceSizePSF (psphotSourceSizeOptions *options, psArray *sources, pmPSF *psf) { + + // select stats from the psf stars + psVector *Ap = psVectorAllocEmpty (100, PS_TYPE_F32); + psVector *ApErr = psVectorAllocEmpty (100, PS_TYPE_F32); + + psImageMaskType maskVal = options->maskVal | options->markVal; + + // XXX why PHOT_WEIGHT?? + pmSourcePhotometryMode photMode = PM_SOURCE_PHOT_WEIGHT; + + int num = 0; // Number of sources measured + for (int i = 0; i < sources->n; i++) { + pmSource *source = sources->data[i]; + if (!(source->mode & PM_SOURCE_MODE_PSFSTAR)) continue; + num++; + + // replace object in image + if (source->tmpFlags & PM_SOURCE_TMPF_SUBTRACTED) { + pmSourceAdd (source, PM_MODEL_OP_FULL, options->maskVal); + } + + // clear the mask bit and set the circular mask pixels + psImageMaskPixels (source->maskObj, "AND", PS_NOT_IMAGE_MASK(options->markVal)); + psImageKeepCircle (source->maskObj, source->peak->x, source->peak->y, source->apRadius, "OR", options->markVal); + + // XXX can we test if psfMag is set and calculate only if needed? + pmSourceMagnitudes (source, psf, photMode, maskVal); // maskVal includes markVal + + // clear the mask bit + psImageMaskPixels (source->maskObj, "AND", PS_NOT_IMAGE_MASK(options->markVal)); + + // re-subtract the object, leave local sky + pmSourceSub (source, PM_MODEL_OP_FULL, options->maskVal); + + float apMag = -2.5*log10(source->moments->Sum); + float dMag = source->psfMag - apMag; + + psVectorAppend (Ap, dMag); + psVectorAppend (ApErr, source->errMag); + } + if (num == 0) { + // Not raising an error, because errors aren't being checked elsewhere in this function + psFree(Ap); + psFree(ApErr); + return false; + } + + // model the distribution as a mean or median value and a systematic error from that value: + psStats *stats = psStatsAlloc(PS_STAT_ROBUST_MEDIAN); + psVectorStats (stats, Ap, NULL, NULL, 0); + + psVector *dAp = psVectorAlloc (Ap->n, PS_TYPE_F32); + for (int i = 0; i < Ap->n; i++) { + dAp->data.F32[i] = Ap->data.F32[i] - stats->robustMedian; + } + + options->ApResid = stats->robustMedian; + options->ApSysErr = psVectorSystematicError(dAp, ApErr, 0.05); + // XXX this is quite arbitrary... + if (!isfinite(options->ApSysErr)) options->ApSysErr = 0.01; + psLogMsg ("psphot", PS_LOG_DETAIL, "psf - Sum: %f +/- %f\n", options->ApResid, options->ApSysErr); + + psFree (Ap); + psFree (ApErr); + psFree (stats); + psFree (dAp); + + return true; +} + +// classify sources based on the combination of psf-mag, Mxx, Myy +bool psphotSourceClass (pmReadout *readout, psArray *sources, psMetadata *recipe, pmPSF *psf, psphotSourceSizeOptions *options) { + + bool status; + pmPSFClump psfClump; + char regionName[64]; + + psLogMsg("psModules.objects", PS_LOG_INFO, "Source Size classifications: %4s %4s %4s %4s %4s %4s", "Npsf", "Next", "Nsat", "Ncr", "Nmiss", "Nskip"); + + int nRegions = psMetadataLookupS32 (&status, readout->analysis, "PSF.CLUMP.NREGIONS"); + for (int i = 0; i < nRegions; i ++) { + snprintf (regionName, 64, "PSF.CLUMP.REGION.%03d", i); + psMetadata *regionMD = psMetadataLookupPtr (&status, readout->analysis, regionName); + psAssert (regionMD, "regions must be defined by earlier call to psphotRoughClassRegion"); + + psRegion *region = psMetadataLookupPtr (&status, regionMD, "REGION"); + psAssert (region, "regions must be defined by earlier call to psphotRoughClassRegion"); + + // pull FWHM_X,Y from the recipe, use to define psfClump.X,Y + psfClump.X = psMetadataLookupF32 (&status, regionMD, "PSF.CLUMP.X"); psAssert (status, "missing PSF.CLUMP.X"); + psfClump.Y = psMetadataLookupF32 (&status, regionMD, "PSF.CLUMP.Y"); psAssert (status, "missing PSF.CLUMP.Y"); + psfClump.dX = psMetadataLookupF32 (&status, regionMD, "PSF.CLUMP.DX"); psAssert (status, "missing PSF.CLUMP.DX"); + psfClump.dY = psMetadataLookupF32 (&status, regionMD, "PSF.CLUMP.DY"); psAssert (status, "missing PSF.CLUMP.DY"); + + if ((psfClump.X < 0) || (psfClump.Y < 0) || !psfClump.X || !psfClump.Y || isnan(psfClump.X) || isnan(psfClump.Y)) { + psLogMsg ("psphot", 4, "Failed to find a valid PSF clump for region %f,%f - %f,%f\n", region->x0, region->y0, region->x1, region->y1); + continue; + } + + if (!psphotSourceClassRegion (region, &psfClump, sources, recipe, psf, options)) { + psLogMsg ("psphot", 4, "Failed to determine source classification for region %f,%f - %f,%f\n", region->x0, region->y0, region->x1, region->y1); + continue; + } + // psphotVisualPlotSourceSize (recipe, readout->analysis, sources); + } + + return true; +} + +# define SIZE_SN_LIM 10 +bool psphotSourceClassRegion (psRegion *region, pmPSFClump *psfClump, psArray *sources, psMetadata *recipe, pmPSF *psf, psphotSourceSizeOptions *options) { + + PS_ASSERT_PTR_NON_NULL(sources, false); + PS_ASSERT_PTR_NON_NULL(recipe, false); + + int Nsat = 0; + int Next = 0; + int Npsf = 0; + int Ncr = 0; + int Nmiss = 0; + int Nskip = 0; + + pmSourceMode noMoments = PM_SOURCE_MODE_MOMENTS_FAILURE | PM_SOURCE_MODE_SKYVAR_FAILURE | PM_SOURCE_MODE_SKY_FAILURE | PM_SOURCE_MODE_BELOW_MOMENTS_SN; + pmSourcePhotometryMode photMode = PM_SOURCE_PHOT_WEIGHT; + + psImageMaskType maskVal = options->maskVal | options->markVal; + + for (psS32 i = 0 ; i < sources->n ; i++) { + + pmSource *source = (pmSource *) sources->data[i]; + + // psfClumps are found for image subregions: + // skip sources not in this region + if (source->peak->x < region->x0) continue; + if (source->peak->x >= region->x1) continue; + if (source->peak->y < region->y0) continue; + if (source->peak->y >= region->y1) continue; + + // skip source if it was already measured + if (source->tmpFlags & PM_SOURCE_TMPF_SIZE_MEASURED) { + psTrace("psphot", 7, "Not calculating source size since it has already been measured\n"); + continue; + } + + // source must have been subtracted + if (!(source->tmpFlags & PM_SOURCE_TMPF_SUBTRACTED)) { + source->mode |= PM_SOURCE_MODE_SIZE_SKIPPED; + psTrace("psphot", 7, "Not calculating source size since source is not subtracted\n"); + continue; + } + + // we are basically classifying by moments + psAssert (source->moments, "why is this source missing moments?"); + if (source->mode & noMoments) { + Nskip ++; + continue; + } + + // convert to Mmaj, Mmin: + psF32 Mxx = source->moments->Mxx; + psF32 Myy = source->moments->Myy; + + // replace object in image + if (source->tmpFlags & PM_SOURCE_TMPF_SUBTRACTED) { + pmSourceAdd (source, PM_MODEL_OP_FULL, options->maskVal); + } + + // clear the mask bit and set the circular mask pixels + psImageMaskPixels (source->maskObj, "AND", PS_NOT_IMAGE_MASK(options->markVal)); + psImageKeepCircle (source->maskObj, source->peak->x, source->peak->y, source->apRadius, "OR", options->markVal); + + // XXX can we test if psfMag is set and calculate only if needed? + pmSourceMagnitudes (source, psf, photMode, maskVal); // maskVal includes markVal + + // clear the mask bit + psImageMaskPixels (source->maskObj, "AND", PS_NOT_IMAGE_MASK(options->markVal)); + + // re-subtract the object, leave local sky + pmSourceSub (source, PM_MODEL_OP_FULL, options->maskVal); + + float apMag = -2.5*log10(source->moments->Sum); + float dMag = source->psfMag - apMag; + + // set nSigma to include both systematic and poisson error terms + // XXX the 'poisson error' contribution for size is probably wrong... + float nSigmaMAG = (dMag - options->ApResid) / hypot(source->errMag, options->ApSysErr); + float nSigmaMXX = (Mxx - psfClump->X) / hypot(psfClump->dX, psfClump->X*psfClump->X*source->errMag); + float nSigmaMYY = (Myy - psfClump->Y) / hypot(psfClump->dY, psfClump->Y*psfClump->Y*source->errMag); + + // partially-masked sources are more likely to be mis-measured PSFs + float sizeBias = 1.0; + if (source->pixWeight < 0.9) { + sizeBias = 3.0; + } + + float minMxx = psfClump->X - sizeBias*options->nSigmaMoments*psfClump->dX; + float minMyy = psfClump->Y - sizeBias*options->nSigmaMoments*psfClump->dY; + + // include MAG, MXX, and MYY? + source->extNsigma = nSigmaMAG; + + // notes to clarify the source size classification rules: + // * a defect should be functionally equivalent to a cosmic ray + // * CR & defect should have a faintess limit (min S/N) + // * SAT stars should not be faint, but defects may? + + // Anything within this region is a probably PSF-like object. Saturated stars may land + // in this region, but are detected elsewhere on the basis of their peak value. + bool isPSF = (fabs(nSigmaMAG) < options->nSigmaApResid) && (fabs(nSigmaMXX) < sizeBias*options->nSigmaMoments) && (fabs(nSigmaMYY) < sizeBias*options->nSigmaMoments); + if (isPSF) { + psTrace("psphotSourceClassRegion.PSF",4,"CLASS: %g %g\t%g %g %g %g %g %g\t%g %g\t%g PSF\t%g %g\n", + source->peak->xf,source->peak->yf,Mxx,Myy,psfClump->X,psfClump->Y,psfClump->dX,psfClump->dY,apMag,dMag,nSigmaMAG, + options->nSigmaApResid,sizeBias*options->nSigmaMoments); + source->tmpFlags |= PM_SOURCE_TMPF_SIZE_MEASURED; + Npsf ++; + continue; + } + + // Defects may not always match CRs from peak curvature analysis + // Defects may also be marked as SATSTAR -- XXX deactivate this flag? + // XXX this rule is not great + // XXX only accept brightish detections as CRs + // (nSigmaMAG < -options->nSigmaApResid) || + bool isCR = isCR = (source->errMag < 1.0 / SIZE_SN_LIM) && ((Mxx < minMxx) || (Myy < minMyy)); + if (isCR) { + psTrace("psphotSourceClassRegion.CR",4,"CLASS: %g %g %f\t%g %g %g %g %g %g\t%g %g\t%g CR\t%g %g\n", + source->peak->xf,source->peak->yf,source->pixWeight,Mxx,Myy,psfClump->X,psfClump->Y,psfClump->dX,psfClump->dY,apMag,dMag,nSigmaMAG, + options->nSigmaApResid,sizeBias*options->nSigmaMoments); + source->mode |= PM_SOURCE_MODE_DEFECT; + source->tmpFlags |= PM_SOURCE_TMPF_SIZE_CR_CANDIDATE; + Ncr ++; + continue; + } + + // saturated star (determined in PSF fit). These may also be saturated galaxies, or + // just large saturated regions. + if (source->mode & PM_SOURCE_MODE_SATSTAR) { + psTrace("psphotSourceClassRegion.SAT",4,"CLASS: %g %g\t%g %g %g %g %g %g\t%g %g\t%g SAT\t%g %g\n", + source->peak->xf,source->peak->yf,Mxx,Myy,psfClump->X,psfClump->Y,psfClump->dX,psfClump->dY,apMag,dMag,nSigmaMAG, + options->nSigmaApResid,sizeBias*options->nSigmaMoments); + source->tmpFlags |= PM_SOURCE_TMPF_SIZE_MEASURED; + Nsat ++; + continue; + } + + // XXX allow the Mxx, Myy to be less than psfClump->X,Y (by some nSigma)? + bool isEXT = (nSigmaMAG > options->nSigmaApResid) || (Mxx > minMxx) || (Myy > minMyy); + if (isEXT) { + psTrace("psphotSourceClassRegion.EXT",4,"CLASS: %g %g\t%g %g %g %g %g %g\t%g %g\t%g Ext\t%g %g\n", + source->peak->xf,source->peak->yf,Mxx,Myy,psfClump->X,psfClump->Y,psfClump->dX,psfClump->dY,apMag,dMag,nSigmaMAG, + options->nSigmaApResid,sizeBias*options->nSigmaMoments); + + source->mode |= PM_SOURCE_MODE_EXT_LIMIT; + source->tmpFlags |= PM_SOURCE_TMPF_SIZE_MEASURED; + Next ++; + continue; + } + psTrace("psphotSourceClassRegion.MISS",4,"CLASS: %g %g\t%g %g %g %g %g %g\t%g %g\t%g Unk\t%g %g\n", + source->peak->xf,source->peak->yf,Mxx,Myy,psfClump->X,psfClump->Y,psfClump->dX,psfClump->dY,apMag,dMag,nSigmaMAG, + options->nSigmaApResid,sizeBias*options->nSigmaMoments); + + // sources that reach here are probably too faint for a reasonable source size measurement + // psWarning ("sourse size was missed for %f,%f : %f %f -- %f\n", source->peak->xf, source->peak->yf, Mxx, Myy, nSigmaMAG); + source->tmpFlags |= PM_SOURCE_TMPF_SIZE_MEASURED; + Nmiss ++; + } + + psLogMsg("psModules.objects", PS_LOG_INFO, "Source Size classifications: %4d %4d %4d %4d %4d %4d", Npsf, Next, Nsat, Ncr, Nmiss, Nskip); + + return true; +} + +// given an object suspected to be a defect, generate a pixel mask using the Lapacian transform +// if enough of the object is detected as 'sharp', consider the object a cosmic ray +bool psphotSourceSizeCR (pmReadout *readout, psArray *sources, psphotSourceSizeOptions *options) { + + psTimerStart ("psphot.cr"); + + int nMasked = 0; + for (int i = 0; i < sources->n; i++) { + pmSource *source = sources->data[i]; + + // skip source if it was already measured + if (source->tmpFlags & PM_SOURCE_TMPF_SIZE_MEASURED) { + psTrace("psphot", 7, "Not calculating source size since it has already been measured\n"); + continue; + } + + // only check candidates marked above + if (!(source->tmpFlags & PM_SOURCE_TMPF_SIZE_CR_CANDIDATE)) { + psTrace("psphot", 7, "Not calculating source size since it has already been measured\n"); + continue; + } + + // skip unless this source is thought to be a cosmic ray. flag the detection and mask the pixels + // XXX this may be degenerate with the above test + if (!(source->mode & PM_SOURCE_MODE_DEFECT)) continue; + + // Integer position of peak + int xPeak = source->peak->xf - source->pixels->col0 + 0.5; + int yPeak = source->peak->yf - source->pixels->row0 + 0.5; + + // Skip sources which are too close to a boundary. These are mostly caught as DEFECT + if (xPeak < 1 || xPeak > source->pixels->numCols - 2 || + yPeak < 1 || yPeak > source->pixels->numRows - 2) { + psTrace("psphot", 7, "Not calculating crNsigma due to edge\n"); + continue; + } + + // XXX for testing, only CRMASK a single source: + if (options->xtest && (fabs(source->peak->xf - options->xtest) > 5)) continue; + if (options->ytest && (fabs(source->peak->yf - options->ytest) > 5)) continue; + + // replace object in image + if (source->tmpFlags & PM_SOURCE_TMPF_SUBTRACTED) { + pmSourceAdd (source, PM_MODEL_OP_FULL, options->maskVal); + } + + // XXX this is running slowly and is too agressive, but it more-or-less works + psTrace("psphot", 6, "mask cosmic ray at %f, %f\n", source->peak->xf, source->peak->yf); + if (options->apply) { + psphotMaskCosmicRay(readout, source, options->crMask); + } else { + source->mode |= PM_SOURCE_MODE_CR_LIMIT; + } + nMasked ++; + + // re-subtract the object, leave local sky + pmSourceSub (source, PM_MODEL_OP_FULL, options->maskVal); + } + + // now that we have masked pixels associated with CRs, we can grow the mask + if (options->grow > 0) { + bool oldThreads = psImageConvolveSetThreads(true); // Old value of threading for psImageConvolveMask + psImage *newMask = psImageConvolveMask(NULL, readout->mask, options->crMask, options->crMask, -options->grow, options->grow, -options->grow, options->grow); + psImageConvolveSetThreads(oldThreads); + if (!newMask) { + psError(PS_ERR_UNKNOWN, false, "Unable to grow CR mask"); + return false; + } + psFree(readout->mask); + readout->mask = newMask; + } + + psLogMsg ("psphot.cr", PS_LOG_INFO, "mask CR: %d masked in %f sec\n", nMasked, psTimerMark ("psphot.cr")); + + // XXX test : save the mask image + if (0) { + psphotSaveImage (NULL, readout->mask, "mask.fits"); + } + + return true; +} + +# define DUMPPICS 0 +# define LIMIT_XRANGE(X, IMAGE) { X = PS_MIN(PS_MAX(0, X), IMAGE->numCols); } +# define LIMIT_YRANGE(Y, IMAGE) { Y = PS_MIN(PS_MAX(0, Y), IMAGE->numRows); } + +// Comments by CZW 20091209 : Mechanics of how to identify CR pixels taken from "Cosmic-Ray +// Rejection by Laplacian Edge Detection" by Pieter van Dokkum, arXiv:astro-ph/0108003. This +// does no repair or recovery of the CR pixels, it only masks them out. My test code can be +// found at /data/ipp031.0/watersc1/psphot.20091209/algo_check.c +bool psphotMaskCosmicRay (pmReadout *readout, pmSource *source, psImageMaskType maskVal) { + + // Get the actual images and information about the peak. + psImage *mask = readout->mask; + pmPeak *peak = source->peak; + pmFootprint *footprint = peak->footprint; + + // Bounding boxes are inclusive of final pixel + int xs = footprint->bbox.x0; + int xe = footprint->bbox.x1 + 1; + int ys = footprint->bbox.y0; + int ye = footprint->bbox.y1 + 1; + + LIMIT_XRANGE(xs, mask); + LIMIT_XRANGE(xe, mask); + LIMIT_YRANGE(ys, mask); + LIMIT_YRANGE(ye, mask); + + int dx = xe - xs; + int dy = ye - ys; + + psImage *image= readout->image; + psImage *variance = readout->variance; + + int binning = 2; + float sigma_thresh = 3.0; + int max_iter = 1; // XXX with isophot masking, we only want to do a single pass + + // Temporary images. + psImage *mypix = psImageAlloc(dx,dy,image->type.type); + psImage *myfix = psImageAlloc(dx,dy,image->type.type); + psImage *myvar = psImageAlloc(dx,dy,image->type.type); + psImage *binned = psImageAlloc(dx * binning,dy * binning,image->type.type); + psImage *conved = psImageAlloc(dx * binning,dy * binning,image->type.type); + psImage *edges = psImageAlloc(dx,dy,image->type.type); + psImage *mymask = psImageAlloc(dx,dy,PS_TYPE_IMAGE_MASK); + + // Load my copy of things. + for (int y = 0; y < dy; y++) { + for (int x = 0; x < dx; x++) { + mypix->data.F32[y][x] = image->data.F32[y+ys][x+xs]; + myvar->data.F32[y][x] = variance->data.F32[y+ys][x+xs]; + mymask->data.PS_TYPE_IMAGE_MASK_DATA[y][x] = 0x00; + } + } + // Mask so I can see on the output image where the footprint is. + for (int i = 0; i < footprint->spans->n; i++) { + pmSpan *sp = footprint->spans->data[i]; + for (int j = sp->x0; j <= sp->x1; j++) { + int y = sp->y - ys; + int x = j - xs; + mymask->data.PS_TYPE_IMAGE_MASK_DATA[y][x] |= 0x01; + } + } + + int nCRpix = 1; // force at least one pass... + for (int iteration = 0; (iteration < max_iter) && (nCRpix > 0); iteration++) { + nCRpix = 0; + psImageInit (binned, 0.0); + psImageInit (conved, 0.0); + psImageInit (edges, 0.0); + + // Make subsampled image. Maybe this should be called "unbinned" or something + for (int y = 0; y < binning * dy; y++) { + int yraw = y / binning; + for (int x = 0; x < binning * dx; x++) { + int xraw = x / binning; + binned->data.F32[y][x] = mypix->data.F32[yraw][xraw]; + } + } + + // Apply Laplace transform (kernel = [[0 -0.25 0][-0.25 1 -0.25][0 -0.25 0]]), clipping at zero + for (int y = 1; y < binning * dy - 1; y++) { + for (int x = 1; x < binning * dx - 1; x++) { + float value = binned->data.F32[y][x] - 0.25 * + (binned->data.F32[y+0][x-1] + binned->data.F32[y+0][x+1] + + binned->data.F32[y-1][x+0] + binned->data.F32[y+1][x+0]); + value = PS_MAX(0.0, value); + + conved->data.F32[y][x] = value; + } + } + + // Create an edge map by rebinning + for (int y = 0; y < binning * dy; y++) { + int yraw = y / binning; + for (int x = 0; x < binning * dx; x++) { + int xraw = x / binning; + edges->data.F32[yraw][xraw] += conved->data.F32[y][x]; + } + } + + // coordinate of peak in subimage pixels: + int xPeak = peak->x - xs; + int yPeak = peak->y - ys; + + // Modify my mask if we're above the significance threshold, but only for connected pixels + nCRpix = psphotMaskCosmicRayConnected (xPeak, yPeak, mymask, myvar, edges, binning, sigma_thresh); + +# if DUMPPICS + psphotSaveImage (NULL, mypix, "crmask.pix.fits"); +# endif + +// XXX do not repair the pixels in isophot version +# if 0 + // "Repair" Masked pixels for the next round. + for (int y = 1; y < dy - 1; y++) { + for (int x = 1; x < dx - 1; x++) { + if (!(mymask->data.PS_TYPE_IMAGE_MASK_DATA[y][x] & 0x40)) { + myfix->data.F32[y][x] = mypix->data.F32[y][x]; + continue; + } + myfix->data.F32[y][x] = 0.25 * + (mypix->data.F32[y+0][x-1] + mypix->data.F32[y+0][x+1] + + mypix->data.F32[y-1][x+0] + mypix->data.F32[y+1][x+0]); + } + } + + // "Repair" Masked pixels for the next round. + for (int y = 1; y < dy - 1; y++) { + for (int x = 1; x < dx - 1; x++) { + mypix->data.F32[y][x] = myfix->data.F32[y][x]; + } + } +# endif + +# if DUMPPICS + fprintf (stderr, "CRMASK %d %d %d %d %d\n", xs, ys, dx, dy, iteration); + psphotSaveImage (NULL, mypix, "crmask.fix.fits"); + psphotSaveImage (NULL, myvar, "crmask.var.fits"); + psphotSaveImage (NULL, binned, "crmask.binn.fits"); + psphotSaveImage (NULL, conved, "crmask.conv.fits"); + psphotSaveImage (NULL, edges, "crmask.edge.fits"); + psphotSaveImage (NULL, mymask, "crmask.mask.fits"); +# endif + psTrace("psphot.czw",2,"Iter: %d Count: %d",iteration, nCRpix); + } + +# if 0 + // A solitary masked pixel is likely a lie. Remove those + // XXX can't we use nCRpix == 1 to test for these? + for (int x = 0; x < dx; x++) { + for (int y = 0; y < dy; y++) { + if (!(mymask->data.PS_TYPE_IMAGE_MASK_DATA[y][x] & 0x40)) continue; + if ((x-1 >= 0) && (mymask->data.PS_TYPE_IMAGE_MASK_DATA[y][x-1] & 0x40)) { + continue; + } + if ((y-1 >= 0) && (mymask->data.PS_TYPE_IMAGE_MASK_DATA[y-1][x] & 0x40)) { + continue; + } + if ((x+1 < dx) && (mymask->data.PS_TYPE_IMAGE_MASK_DATA[y][x+1] & 0x40)) { + continue; + } + if ((y+1 < dy) && (mymask->data.PS_TYPE_IMAGE_MASK_DATA[y+1][x] & 0x40)) { + continue; + } + mymask->data.PS_TYPE_IMAGE_MASK_DATA[y][x] ^= 0x40; + } + } +# endif + + // transfer temporary mask to real mask & count masked pixels + nCRpix = 0; + for (int x = 0; x < dx; x++) { + for (int y = 0; y < dy; y++) { + if (mymask->data.PS_TYPE_IMAGE_MASK_DATA[y][x] & 0x40) { + mask->data.PS_TYPE_IMAGE_MASK_DATA[y+ys+mask->row0][x+xs+mask->col0] |= maskVal; + nCRpix ++; + } + } + } + + // XXX if we decide this REALLY is a cosmic ray, set the CR_LIMIT bit + if (nCRpix > 1) { + source->mode |= PM_SOURCE_MODE_CR_LIMIT; + source->tmpFlags |= PM_SOURCE_TMPF_SIZE_MEASURED; + } + // fprintf (stderr, "CRMASK %d %d %d %d %d\n", peak->x, peak->y, dx, dy, nCRpix); + + psFree(mypix); + psFree(myfix); + psFree(myvar); + psFree(binned); + psFree(conved); + psFree(edges); + psFree(mymask); + + return true; +} + +bool psphotMaskCosmicRayFootprintCheck (psArray *sources) { + + for (int i = 0; i < sources->n; i++) { + pmSource *source = sources->data[i]; + pmPeak *peak = source->peak; + pmFootprint *footprint = peak->footprint; + if (!footprint) continue; + for (int j = 0; j < footprint->spans->n; j++) { + pmSpan *sp = footprint->spans->data[j]; + psAssert (sp, "missing span"); + } + } + return true; +} + +/**** ------ old versions of cosmic ray masking ----- ****/ + +bool psphotMaskCosmicRayIsophot (pmSource *source, psImageMaskType maskVal, psImageMaskType crMask); + +// This attempt to mask the cosmic rays used the isophotal boundary +bool psphotMaskCosmicRay_V1 (psImage *mask, pmSource *source, psImageMaskType maskVal, psImageMaskType crMask) { // replace the source flux @@ -103,4 +748,6 @@ pmFootprint *footprint = peak->footprint; if (!footprint) { + psTrace("psphot.czw",2,"Using isophot CR mask code."); + // if we have not footprint, use the old code to mask by isophot psphotMaskCosmicRayIsophot (source, maskVal, crMask); @@ -109,9 +756,11 @@ if (!footprint->spans) { + psTrace("psphot.czw",2,"Using isophot CR mask code."); + // if we have no footprint, use the old code to mask by isophot psphotMaskCosmicRayIsophot (source, maskVal, crMask); return true; } - + psphotMaskCosmicRayIsophot (source, maskVal, crMask); // mask all of the pixels covered by the spans of the footprint for (int j = 1; j < footprint->spans->n; j++) { @@ -127,4 +776,147 @@ } return true; +} + +# define VERBOSE 0 +int psphotMaskCosmicRayConnected (int xPeak, int yPeak, psImage *mymask, psImage *myvar, psImage *edges, int binning, float sigma_thresh) { + + int xLo, xRo; + int nCRpix = 0; + + float noise_factor = 5.0 / 4.0; // Intrinsic to the Laplacian making noise spikes spikier. + + // mark the pixels in this row to the left, then the right. stay within footprint + int xL = xPeak; // find the range of valid pixels in this row + int xR = xPeak; + for (int ix = xPeak; (ix >= 0) && (mymask->data.PS_TYPE_IMAGE_MASK_DATA[yPeak][ix] & 0x01); ix--) { + float noise = binning * sqrt(noise_factor * myvar->data.F32[yPeak][ix]); + float value = edges->data.F32[yPeak][ix] / noise; + if (value < sigma_thresh ) break; + mymask->data.PS_TYPE_IMAGE_MASK_DATA[yPeak][ix] |= 0x40; + xL = ix; + nCRpix ++; + if (VERBOSE) fprintf (stderr, "mark %d,%d (%d) : %d - %d\n", ix, yPeak, nCRpix, xL, xR); + } + for (int ix = xPeak; (ix < mymask->numCols) && (mymask->data.PS_TYPE_IMAGE_MASK_DATA[yPeak][ix] & 0x01); ix++) { + float noise = binning * sqrt(noise_factor * myvar->data.F32[yPeak][ix]); + float value = edges->data.F32[yPeak][ix] / noise; + if (value < sigma_thresh ) break; + mymask->data.PS_TYPE_IMAGE_MASK_DATA[yPeak][ix] |= 0x40; + xR = ix; + nCRpix ++; + if (VERBOSE) fprintf (stderr, "mark %d,%d (%d) : %d - %d\n", ix, yPeak, nCRpix, xL, xR); + } + // xL and xR mark the first and last valid pixel in the row + + // for each of the neighboring rows, mark the high pixels if they touch the range xL to xR + xLo = PS_MAX(xL - 1, 0); + xRo = PS_MIN(xR + 1, mymask->numCols); + + // first go down: + for (int iy = yPeak - 1; iy >= 0; iy--) { + + int xLn = -1; + int xRn = -1; + int newPix = 0; + + // mark the pixels in the good range + for (int ix = xLo; ix < xRo; ix++) { + if (!(mymask->data.PS_TYPE_IMAGE_MASK_DATA[iy][ix] & 0x01)) continue; // only use pixels in the footprint + float noise = binning * sqrt(noise_factor * myvar->data.F32[iy][ix]); + float value = edges->data.F32[iy][ix] / noise; + if (value < sigma_thresh ) continue; + mymask->data.PS_TYPE_IMAGE_MASK_DATA[iy][ix] |= 0x40; + if (xLn == -1) xLn = ix; // first valid pixel in this row + xRn = ix; // last valid pixel in this row + nCRpix ++; + newPix ++; + if (VERBOSE) fprintf (stderr, "mark C %d,%d (%d) : %d - %d | %d - %d | %d - %d \n", ix, iy, nCRpix, xL, xR, xLo, xRo, xLn, xRn); + } + + // mark the pixels to the left of the good range + for (int ix = xLo; (ix >= 0) && (mymask->data.PS_TYPE_IMAGE_MASK_DATA[iy][ix] & 0x01); ix--) { + float noise = binning * sqrt(noise_factor * myvar->data.F32[iy][ix]); + float value = edges->data.F32[iy][ix] / noise; + if (value < sigma_thresh ) break; + mymask->data.PS_TYPE_IMAGE_MASK_DATA[iy][ix] |= 0x40; + if (xRn == -1) xRn = ix; // last valid pixel in this row + xLn = ix; + nCRpix ++; + newPix ++; + if (VERBOSE) fprintf (stderr, "mark L %d,%d (%d) : %d - %d | %d - %d | %d - %d \n", ix, iy, nCRpix, xL, xR, xLo, xRo, xLn, xRn); + } + + // mark the pixels to the right of the good range + for (int ix = xRo; (ix < mymask->numCols) && (mymask->data.PS_TYPE_IMAGE_MASK_DATA[iy][ix] & 0x01); ix++) { + float noise = binning * sqrt(noise_factor * myvar->data.F32[iy][ix]); + float value = edges->data.F32[iy][ix] / noise; + if (value < sigma_thresh ) break; + mymask->data.PS_TYPE_IMAGE_MASK_DATA[iy][ix] |= 0x40; + if (xLn == -1) xLn = ix; // first valid pixel in this row + xRn = ix; + nCRpix ++; + newPix ++; + if (VERBOSE) fprintf (stderr, "mark R %d,%d (%d) : %d - %d | %d - %d | %d - %d \n", ix, iy, nCRpix, xL, xR, xLo, xRo, xLn, xRn); + } + if (newPix == 0) break; + xLo = PS_MAX(xLn - 1, 0); + xRo = PS_MIN(xRn + 1, mymask->numCols); + } + + xLo = PS_MAX(xL - 1, 0); + xRo = PS_MIN(xR + 1, mymask->numCols); + + // next go up: + for (int iy = yPeak + 1; iy < mymask->numRows; iy++) { + + int xLn = -1; + int xRn = -1; + int newPix = 0; + + // mark the pixels in the good range + for (int ix = xLo; ix < xRo; ix++) { + if (!(mymask->data.PS_TYPE_IMAGE_MASK_DATA[iy][ix] & 0x01)) continue; // only use pixels in the footprint + float noise = binning * sqrt(noise_factor * myvar->data.F32[iy][ix]); + float value = edges->data.F32[iy][ix] / noise; + if (value < sigma_thresh ) continue; + mymask->data.PS_TYPE_IMAGE_MASK_DATA[iy][ix] |= 0x40; + if (xLn == -1) xLn = ix; // first valid pixel in this row + xRn = ix; // last valid pixel in this row + nCRpix ++; + newPix ++; + if (VERBOSE) fprintf (stderr, "mark C %d,%d (%d) : %d - %d | %d - %d | %d - %d \n", ix, iy, nCRpix, xL, xR, xLo, xRo, xLn, xRn); + } + + // mark the pixels to the left of the good range + for (int ix = xLo; (ix >= 0) && (mymask->data.PS_TYPE_IMAGE_MASK_DATA[iy][ix] & 0x01); ix--) { + float noise = binning * sqrt(noise_factor * myvar->data.F32[iy][ix]); + float value = edges->data.F32[iy][ix] / noise; + if (value < sigma_thresh ) break; + mymask->data.PS_TYPE_IMAGE_MASK_DATA[iy][ix] |= 0x40; + if (xRn == -1) xRn = ix; // last valid pixel in this row + xLn = ix; + nCRpix ++; + newPix ++; + if (VERBOSE) fprintf (stderr, "mark L %d,%d (%d) : %d - %d | %d - %d | %d - %d \n", ix, iy, nCRpix, xL, xR, xLo, xRo, xLn, xRn); + } + + // mark the pixels to the right of the good range + for (int ix = xRo; (ix < mymask->numCols) && (mymask->data.PS_TYPE_IMAGE_MASK_DATA[iy][ix] & 0x01); ix++) { + float noise = binning * sqrt(noise_factor * myvar->data.F32[iy][ix]); + float value = edges->data.F32[iy][ix] / noise; + if (value < sigma_thresh ) break; + mymask->data.PS_TYPE_IMAGE_MASK_DATA[iy][ix] |= 0x40; + if (xLn == -1) xLn = ix; // first valid pixel in this row + xRn = ix; + nCRpix ++; + newPix ++; + if (VERBOSE) fprintf (stderr, "mark R %d,%d (%d) : %d - %d | %d - %d | %d - %d \n", ix, iy, nCRpix, xL, xR, xLo, xRo, xLn, xRn); + } + if (newPix == 0) break; + xLo = PS_MAX(xLn - 1, 0); + xRo = PS_MIN(xRn + 1, mymask->numCols); + } + + return nCRpix; } @@ -192,221 +984,4 @@ } } - return true; -} - -// model the apmifit distribution for the psf stars: -bool psphotSourceSizePSF (psphotSourceSizeOptions *options, psArray *sources, pmPSF *psf) { - - // select stats from the psf stars - psVector *Ap = psVectorAllocEmpty (100, PS_TYPE_F32); - psVector *ApErr = psVectorAllocEmpty (100, PS_TYPE_F32); - - psImageMaskType maskVal = options->maskVal | options->markVal; - - // XXX why PHOT_WEIGHT?? - pmSourcePhotometryMode photMode = PM_SOURCE_PHOT_WEIGHT; - - for (int i = 0; i < sources->n; i++) { - pmSource *source = sources->data[i]; - if (!(source->mode & PM_SOURCE_MODE_PSFSTAR)) continue; - - // replace object in image - if (source->tmpFlags & PM_SOURCE_TMPF_SUBTRACTED) { - pmSourceAdd (source, PM_MODEL_OP_FULL, options->maskVal); - } - - // clear the mask bit and set the circular mask pixels - psImageMaskPixels (source->maskObj, "AND", PS_NOT_IMAGE_MASK(options->markVal)); - psImageKeepCircle (source->maskObj, source->peak->x, source->peak->y, source->apRadius, "OR", options->markVal); - - // XXX can we test if psfMag is set and calculate only if needed? - pmSourceMagnitudes (source, psf, photMode, maskVal); // maskVal includes markVal - - // clear the mask bit - psImageMaskPixels (source->maskObj, "AND", PS_NOT_IMAGE_MASK(options->markVal)); - - // re-subtract the object, leave local sky - pmSourceSub (source, PM_MODEL_OP_FULL, options->maskVal); - - float apMag = -2.5*log10(source->moments->Sum); - float dMag = source->psfMag - apMag; - - psVectorAppend (Ap, 100, dMag); - psVectorAppend (ApErr, 100, source->errMag); - } - - // model the distribution as a mean or median value and a systematic error from that value: - psStats *stats = psStatsAlloc(PS_STAT_ROBUST_MEDIAN); - psVectorStats (stats, Ap, NULL, NULL, 0); - - psVector *dAp = psVectorAlloc (Ap->n, PS_TYPE_F32); - for (int i = 0; i < Ap->n; i++) { - dAp->data.F32[i] = Ap->data.F32[i] - stats->robustMedian; - } - - options->ApResid = stats->robustMedian; - options->ApSysErr = psVectorSystematicError(dAp, ApErr, 0.05); - psLogMsg ("psphot", PS_LOG_DETAIL, "psf - Sum: %f +/- %f\n", options->ApResid, options->ApSysErr); - - psFree (Ap); - psFree (ApErr); - psFree (stats); - psFree (dAp); - - return true; -} - -// classify sources based on the combination of psf-mag, Mxx, Myy -bool psphotSourceClass (pmReadout *readout, psArray *sources, psMetadata *recipe, pmPSF *psf, psphotSourceSizeOptions *options) { - - bool status; - pmPSFClump psfClump; - char regionName[64]; - - psLogMsg("psModules.objects", PS_LOG_INFO, "Source Size classifications: %4s %4s %4s %4s %4s %4s", "Npsf", "Next", "Nsat", "Ncr", "Nmiss", "Nskip"); - - int nRegions = psMetadataLookupS32 (&status, readout->analysis, "PSF.CLUMP.NREGIONS"); - for (int i = 0; i < nRegions; i ++) { - snprintf (regionName, 64, "PSF.CLUMP.REGION.%03d", i); - psMetadata *regionMD = psMetadataLookupPtr (&status, readout->analysis, regionName); - psAssert (regionMD, "regions must be defined by earlier call to psphotRoughClassRegion"); - - psRegion *region = psMetadataLookupPtr (&status, regionMD, "REGION"); - psAssert (region, "regions must be defined by earlier call to psphotRoughClassRegion"); - - // pull FWHM_X,Y from the recipe, use to define psfClump.X,Y - psfClump.X = psMetadataLookupF32 (&status, regionMD, "PSF.CLUMP.X"); psAssert (status, "missing PSF.CLUMP.X"); - psfClump.Y = psMetadataLookupF32 (&status, regionMD, "PSF.CLUMP.Y"); psAssert (status, "missing PSF.CLUMP.Y"); - psfClump.dX = psMetadataLookupF32 (&status, regionMD, "PSF.CLUMP.DX"); psAssert (status, "missing PSF.CLUMP.DX"); - psfClump.dY = psMetadataLookupF32 (&status, regionMD, "PSF.CLUMP.DY"); psAssert (status, "missing PSF.CLUMP.DY"); - - if ((psfClump.X < 0) || (psfClump.Y < 0) || !psfClump.X || !psfClump.Y || isnan(psfClump.X) || isnan(psfClump.Y)) { - psLogMsg ("psphot", 4, "Failed to find a valid PSF clump for region %f,%f - %f,%f\n", region->x0, region->y0, region->x1, region->y1); - continue; - } - - if (!psphotSourceClassRegion (region, &psfClump, sources, recipe, psf, options)) { - psLogMsg ("psphot", 4, "Failed to determine source classification for region %f,%f - %f,%f\n", region->x0, region->y0, region->x1, region->y1); - continue; - } - } - - return true; -} - -bool psphotSourceClassRegion (psRegion *region, pmPSFClump *psfClump, psArray *sources, psMetadata *recipe, pmPSF *psf, psphotSourceSizeOptions *options) { - - PS_ASSERT_PTR_NON_NULL(sources, false); - PS_ASSERT_PTR_NON_NULL(recipe, false); - - int Nsat = 0; - int Next = 0; - int Npsf = 0; - int Ncr = 0; - int Nmiss = 0; - int Nskip = 0; - - pmSourceMode noMoments = PM_SOURCE_MODE_MOMENTS_FAILURE | PM_SOURCE_MODE_SKYVAR_FAILURE | PM_SOURCE_MODE_SKY_FAILURE | PM_SOURCE_MODE_BELOW_MOMENTS_SN; - pmSourcePhotometryMode photMode = PM_SOURCE_PHOT_WEIGHT; - - psImageMaskType maskVal = options->maskVal | options->markVal; - - for (psS32 i = 0 ; i < sources->n ; i++) { - - pmSource *source = (pmSource *) sources->data[i]; - - // psfClumps are found for image subregions: - // skip sources not in this region - if (source->peak->x < region->x0) continue; - if (source->peak->x >= region->x1) continue; - if (source->peak->y < region->y0) continue; - if (source->peak->y >= region->y1) continue; - - // skip source if it was already measured - if (source->tmpFlags & PM_SOURCE_TMPF_SIZE_MEASURED) { - psTrace("psphot", 7, "Not calculating source size since it has already been measured\n"); - continue; - } - - // source must have been subtracted - if (!(source->tmpFlags & PM_SOURCE_TMPF_SUBTRACTED)) { - source->mode |= PM_SOURCE_MODE_SIZE_SKIPPED; - psTrace("psphot", 7, "Not calculating source size since source is not subtracted\n"); - continue; - } - - // we are basically classifying by moments; use the default if not found - psAssert (source->moments, "why is this source missing moments?"); - if (source->mode & noMoments) { - Nskip ++; - continue; - } - - psF32 Mxx = source->moments->Mxx; - psF32 Myy = source->moments->Myy; - - // replace object in image - if (source->tmpFlags & PM_SOURCE_TMPF_SUBTRACTED) { - pmSourceAdd (source, PM_MODEL_OP_FULL, options->maskVal); - } - - // clear the mask bit and set the circular mask pixels - psImageMaskPixels (source->maskObj, "AND", PS_NOT_IMAGE_MASK(options->markVal)); - psImageKeepCircle (source->maskObj, source->peak->x, source->peak->y, source->apRadius, "OR", options->markVal); - - // XXX can we test if psfMag is set and calculate only if needed? - pmSourceMagnitudes (source, psf, photMode, maskVal); // maskVal includes markVal - - // clear the mask bit - psImageMaskPixels (source->maskObj, "AND", PS_NOT_IMAGE_MASK(options->markVal)); - - // re-subtract the object, leave local sky - pmSourceSub (source, PM_MODEL_OP_FULL, options->maskVal); - - float apMag = -2.5*log10(source->moments->Sum); - float dMag = source->psfMag - apMag; - float nSigma = (dMag - options->ApResid) / hypot(source->errMag, options->ApSysErr); - - source->extNsigma = nSigma; - source->tmpFlags |= PM_SOURCE_TMPF_SIZE_MEASURED; - - // Anything within this region is a probably PSF-like object. Saturated stars may land - // in this region, but are detected elsewhere on the basis of their peak value. - bool isPSF = (fabs(nSigma) < options->nSigmaApResid) && (fabs(Mxx - psfClump->X) < options->nSigmaMoments*psfClump->dX) && (fabs(Myy - psfClump->Y) < options->nSigmaMoments*psfClump->dY); - if (isPSF) { - Npsf ++; - continue; - } - - // Defects may not always match CRs from peak curvature analysis - // Defects may also be marked as SATSTAR -- XXX deactivate this flag? - // XXX this rule is not great - if ((Mxx < psfClump->X) || (Myy < psfClump->Y)) { - source->mode |= PM_SOURCE_MODE_DEFECT; - Ncr ++; - continue; - } - - // saturated star (determined in PSF fit). These may also be saturated galaxies, or - // just large saturated regions. - if (source->mode & PM_SOURCE_MODE_SATSTAR) { - Nsat ++; - continue; - } - - // XXX allow the Mxx, Myy to be less than psfClump->X,Y (by some nSigma)? - bool isEXT = (nSigma > options->nSigmaApResid) || ((Mxx > psfClump->X) && (Myy > psfClump->Y)); - if (isEXT) { - source->mode |= PM_SOURCE_MODE_EXT_LIMIT; - Next ++; - continue; - } - - psWarning ("sourse size was missed for %f,%f : %f %f -- %f\n", source->peak->xf, source->peak->yf, Mxx, Myy, nSigma); - Nmiss ++; - } - - psLogMsg("psModules.objects", PS_LOG_INFO, "Source Size classifications: %4d %4d %4d %4d %4d %4d", Npsf, Next, Nsat, Ncr, Nmiss, Nskip); - return true; } @@ -486,5 +1061,6 @@ } -bool psphotSourceSizeCR (pmReadout *readout, psArray *sources, psphotSourceSizeOptions *options) { +// this was an old attempt to identify cosmic rays based on the peak curvature +bool psphotSourcePeakCurvature (pmReadout *readout, psArray *sources, psphotSourceSizeOptions *options) { // classify the sources based on the CR test (place this in a function?) @@ -618,2 +1194,3 @@ return true; } + Index: branches/tap_branches/psphot/src/psphotSourceStats.c =================================================================== --- branches/tap_branches/psphot/src/psphotSourceStats.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotSourceStats.c (revision 27838) @@ -1,7 +1,30 @@ # include "psphotInternal.h" -bool psphotSetMomentsWindow (psMetadata *recipe, psMetadata *analysis, psArray *sources); - -psArray *psphotSourceStats (pmConfig *config, pmReadout *readout, pmDetections *detections, bool setWindow) { +// convert detections to sources and measure their basic properties (moments, local sky, sky +// variance) Note: this function only generates sources for the new peaks (peak->assigned). +// The new sources are added to any existing sources on detections->newSources. The sources +// on detections->allSources are ignored. +bool psphotSourceStats (pmConfig *config, const pmFPAview *view, bool setWindow) +{ + bool status = true; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (!psphotSourceStatsReadout (config, view, "PSPHOT.INPUT", i, recipe, setWindow)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to find initial detections for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} + +bool psphotSourceStatsReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe, bool setWindow) { bool status = false; @@ -10,7 +33,21 @@ psTimerStart ("psphot.stats"); - // select the appropriate recipe information - psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); - assert (recipe); + // find the currently selected readout + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (file, "missing file?"); + + pmReadout *readout = pmFPAviewThisReadout(view, file->fpa); + psAssert (readout, "missing readout?"); + + pmDetections *detections = psMetadataLookupPtr (&status, readout->analysis, "PSPHOT.DETECTIONS"); + psAssert (detections, "missing detections?"); + + // XXX TEST: + if (detections->allSources) { + psphotMaskCosmicRayFootprintCheck(detections->allSources); + } + if (detections->newSources) { + psphotMaskCosmicRayFootprintCheck(detections->newSources); + } // determine the number of allowed threads @@ -21,20 +58,40 @@ // determine properties (sky, moments) of initial sources - float OUTER = psMetadataLookupF32 (&status, recipe, "SKY_OUTER_RADIUS"); - if (!status) return NULL; - + float OUTER = psMetadataLookupF32 (&status, recipe, "SKY_OUTER_RADIUS"); + psAssert (status, "missing SKY_OUTER_RADIUS in recipe?"); + + // XXX this seems like an arbitrary number... OUTER = PS_MAX(OUTER, 20.0); // XXX Guarantee that we can encompass the max moments radius char *breakPt = psMetadataLookupStr (&status, recipe, "BREAK_POINT"); - if (!status) return NULL; + psAssert (status, "missing BREAK_POINT?"); + + float INNER = psMetadataLookupF32 (&status, recipe, "SKY_INNER_RADIUS"); psAssert (status, "missing SKY_INNER_RADIUS"); + float MIN_SN = psMetadataLookupF32 (&status, recipe, "MOMENTS_SN_MIN"); psAssert (status, "missing MOMENTS_SN_MIN"); + + // bit-masks to test for good/bad pixels + psImageMaskType maskVal = psMetadataLookupImageMask(&status, recipe, "MASK.PSPHOT"); + psAssert (maskVal, "missing MASK.PSPHOT"); + + // bit-mask to mark pixels not used in analysis + psImageMaskType markVal = psMetadataLookupImageMask(&status, recipe, "MARK.PSPHOT"); + psAssert (markVal, "missing MARK.PSPHOT"); psArray *peaks = detections->peaks; if (!peaks) { psError(PS_ERR_UNEXPECTED_NULL, false, "No peaks found!"); - return NULL; + return false; } // generate the array of sources, define the associated pixel - sources = psArrayAllocEmpty (peaks->n); + if (!detections->newSources) { + detections->newSources = psArrayAllocEmpty (peaks->n); + } + sources = detections->newSources; + + // if there are no peaks, we save the empty source array and return + if (!peaks->n) { + return true; + } for (int i = 0; i < peaks->n; i++) { @@ -45,4 +102,5 @@ // create a new source pmSource *source = pmSourceAlloc(); + source->imageID = index; // add the peak @@ -58,4 +116,154 @@ psArrayAdd (sources, 100, source); psFree (source); + } + + if (!strcasecmp (breakPt, "PEAKS")) { + psLogMsg ("psphot", PS_LOG_INFO, "%ld sources : %f sec\n", sources->n, psTimerMark ("psphot.stats")); + psLogMsg ("psphot", PS_LOG_INFO, "break point PEAKS, skipping MOMENTS\n"); + psphotVisualShowMoments (sources); + return true; + } + + if (setWindow) { + if (!psphotSetMomentsWindow(recipe, readout->analysis, sources)) { + psError(PS_ERR_UNEXPECTED_NULL, false, "Failed to determine Moments Window!"); + psFree(detections->newSources); + return false; + } + } + + // if we have measured the window, we will be saving the modified version of these recipe values on readout->analysis + float SIGMA = psMetadataLookupF32 (&status, readout->analysis, "MOMENTS_GAUSS_SIGMA"); + if (!status) { + SIGMA = psMetadataLookupF32 (&status, recipe, "MOMENTS_GAUSS_SIGMA"); + } + float RADIUS = psMetadataLookupF32 (&status, readout->analysis, "PSF_MOMENTS_RADIUS"); + if (!status) { + RADIUS = psMetadataLookupF32 (&status, recipe, "PSF_MOMENTS_RADIUS"); + } + + // threaded measurement of the source magnitudes + int Nfail = 0; + int Nmoments = 0; + int Nfaint = 0; + + // choose Cx, Cy (see psphotThreadTools.c for overview of the concepts) + int Cx = 1, Cy = 1; + psphotChooseCellSizes (&Cx, &Cy, readout, nThreads); + + psArray *cellGroups = psphotAssignSources (Cx, Cy, sources); + + for (int i = 0; i < cellGroups->n; i++) { + + psArray *cells = cellGroups->data[i]; + + for (int j = 0; j < cells->n; j++) { + + // allocate a job -- if threads are not defined, this just runs the job + psThreadJob *job = psThreadJobAlloc ("PSPHOT_SOURCE_STATS"); + + psArrayAdd(job->args, 1, cells->data[j]); // sources + + PS_ARRAY_ADD_SCALAR(job->args, INNER, PS_TYPE_F32); + PS_ARRAY_ADD_SCALAR(job->args, MIN_SN, PS_TYPE_F32); + PS_ARRAY_ADD_SCALAR(job->args, RADIUS, PS_TYPE_F32); + PS_ARRAY_ADD_SCALAR(job->args, SIGMA, PS_TYPE_F32); + + PS_ARRAY_ADD_SCALAR(job->args, maskVal, PS_TYPE_IMAGE_MASK); + PS_ARRAY_ADD_SCALAR(job->args, markVal, PS_TYPE_IMAGE_MASK); + + PS_ARRAY_ADD_SCALAR(job->args, 0, PS_TYPE_S32); // this is used as a return value for Nmoments + PS_ARRAY_ADD_SCALAR(job->args, 0, PS_TYPE_S32); // this is used as a return value for Nfail + PS_ARRAY_ADD_SCALAR(job->args, 0, PS_TYPE_S32); // this is used as a return value for Nfaint + + if (!psThreadJobAddPending(job)) { + psError(PS_ERR_UNKNOWN, false, "Unable to launch thread job PSPHOT_SOURCE_STATS"); + psFree (job); + psFree(detections->newSources); + return false; + } + psFree(job); + } + + // wait for the threads to finish and manage results + if (!psThreadPoolWait (false)) { + psError(PS_ERR_UNKNOWN, false, "Failure in thread job PSPHOT_SOURCE_STATS"); + psFree(detections->newSources); + return false; + } + + // we have only supplied one type of job, so we can assume the types here + psThreadJob *job = NULL; + while ((job = psThreadJobGetDone()) != NULL) { + if (job->args->n < 1) { + fprintf (stderr, "error with job\n"); + } else { + psScalar *scalar = NULL; + scalar = job->args->data[7]; + Nmoments += scalar->data.S32; + scalar = job->args->data[8]; + Nfail += scalar->data.S32; + scalar = job->args->data[9]; + Nfaint += scalar->data.S32; + } + psFree(job); + } + } + + psFree (cellGroups); + + psLogMsg ("psphot", PS_LOG_INFO, "%ld sources, %d moments, %d faint, %d failed: %f sec\n", sources->n, Nmoments, Nfaint, Nfail, psTimerMark ("psphot.stats")); + + psphotVisualShowMoments (sources); + + if (detections->allSources) { + psphotMaskCosmicRayFootprintCheck(detections->allSources); + } + if (detections->newSources) { + psphotMaskCosmicRayFootprintCheck(detections->newSources); + } + + return true; +} + +// this function is currently only called by psphotCheckExtSources +bool psphotSourceStatsUpdate (psArray *sources, pmConfig *config, pmReadout *readout) { + + bool status = false; + + psTimerStart ("psphot.stats"); + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + assert (recipe); + + // determine the number of allowed threads + int nThreads = psMetadataLookupS32(&status, config->arguments, "NTHREADS"); // Number of threads + if (!status) { + nThreads = 0; + } + + // determine properties (sky, moments) of initial sources + float OUTER = psMetadataLookupF32 (&status, recipe, "SKY_OUTER_RADIUS"); + if (!status) return false; + + OUTER = PS_MAX(OUTER, 20.0); // XXX Guarantee that we can encompass the max moments radius + + char *breakPt = psMetadataLookupStr (&status, recipe, "BREAK_POINT"); + if (!status) return false; + + for (int i = 0; i < sources->n; i++) { + + pmSource *source = sources->data[i]; + if (!source->peak) continue; // XXX how can we have a peak-less source? + + // allocate space for moments + if (!source->moments) { + source->moments = pmMomentsAlloc(); + } + + // allocate image, weight, mask arrays for each peak (square of radius OUTER) + pmSourceDefinePixels (source, readout, source->peak->x, source->peak->y, OUTER); + source->peak->assigned = true; } @@ -67,9 +275,8 @@ } - if (setWindow) { - if (!psphotSetMomentsWindow(recipe, readout->analysis, sources)) { - psError(PS_ERR_UNEXPECTED_NULL, false, "Failed to determine Moments Window!"); - return NULL; - } + // XXX how else could we get the window size in? + if (!psphotSetMomentsWindow(recipe, readout->analysis, sources)) { + psError(PS_ERR_UNEXPECTED_NULL, false, "Failed to determine Moments Window!"); + return NULL; } @@ -94,4 +301,5 @@ psThreadJob *job = psThreadJobAlloc ("PSPHOT_SOURCE_STATS"); + // XXX: this must match the above psArrayAdd(job->args, 1, cells->data[j]); // sources psArrayAdd(job->args, 1, recipe); @@ -148,22 +356,12 @@ psArray *sources = job->args->data[0]; - psMetadata *recipe = job->args->data[1]; - - float INNER = psMetadataLookupF32 (&status, recipe, "SKY_INNER_RADIUS"); - if (!status) return false; - float MIN_SN = psMetadataLookupF32 (&status, recipe, "MOMENTS_SN_MIN"); - if (!status) return false; - float RADIUS = psMetadataLookupF32 (&status, recipe, "PSF_MOMENTS_RADIUS"); - if (!status) return false; - float SIGMA = psMetadataLookupF32 (&status, recipe, "MOMENTS_GAUSS_SIGMA"); - if (!status) return false; - - // bit-masks to test for good/bad pixels - psImageMaskType maskVal = psMetadataLookupImageMask(&status, recipe, "MASK.PSPHOT"); - assert (maskVal); - - // bit-mask to mark pixels not used in analysis - psImageMaskType markVal = psMetadataLookupImageMask(&status, recipe, "MARK.PSPHOT"); - assert (markVal); + + float INNER = PS_SCALAR_VALUE(job->args->data[1],F32); + float MIN_SN = PS_SCALAR_VALUE(job->args->data[2],F32); + float RADIUS = PS_SCALAR_VALUE(job->args->data[3],F32); + float SIGMA = PS_SCALAR_VALUE(job->args->data[4],F32); + + psImageMaskType maskVal = PS_SCALAR_VALUE(job->args->data[5],PS_TYPE_IMAGE_MASK_DATA); + psImageMaskType markVal = PS_SCALAR_VALUE(job->args->data[6],PS_TYPE_IMAGE_MASK_DATA); // maskVal is used to test for rejected pixels, and must include markVal @@ -176,4 +374,7 @@ for (int i = 0; i < sources->n; i++) { pmSource *source = sources->data[i]; + + if (source->tmpFlags & PM_SOURCE_TMPF_MOMENTS_MEASURED) continue; + source->tmpFlags |= PM_SOURCE_TMPF_MOMENTS_MEASURED; // skip faint sources for moments measurement @@ -205,5 +406,5 @@ // measure basic source moments (no S/N clipping on input pixels) - status = pmSourceMoments (source, RADIUS, SIGMA, 0.0); + status = pmSourceMoments (source, RADIUS, SIGMA, 0.0, maskVal); if (status) { Nmoments ++; @@ -215,5 +416,5 @@ BIG_RADIUS = PS_MIN (INNER, 3*RADIUS); psTrace ("psphot", 4, "retrying moments for %d, %d\n", source->peak->x, source->peak->y); - status = pmSourceMoments (source, BIG_RADIUS, 3.0*SIGMA, 0.0); + status = pmSourceMoments (source, BIG_RADIUS, 3.0*SIGMA, 0.0, maskVal); if (status) { source->mode |= PM_SOURCE_MODE_BIG_RADIUS; @@ -229,11 +430,11 @@ // change the value of a scalar on the array (wrap this and put it in psArray.h) - scalar = job->args->data[2]; + scalar = job->args->data[7]; scalar->data.S32 = Nmoments; - scalar = job->args->data[3]; + scalar = job->args->data[8]; scalar->data.S32 = Nfail; - scalar = job->args->data[4]; + scalar = job->args->data[9]; scalar->data.S32 = Nfaint; @@ -242,14 +443,17 @@ // this function attempts to iteratively determine the best value for sigma of the moments weighting Gaussian +// this function modifies the recipe values MOMENTS_SX_MAX, MOMENTS_SY_MAX, and PSF_CLUMP_GRID_SCALE, used by pmSourcePSFClump bool psphotSetMomentsWindow (psMetadata *recipe, psMetadata *analysis, psArray *sources) { bool status; - float MIN_SN = psMetadataLookupF32 (&status, recipe, "MOMENTS_SN_MIN"); - if (!status) return false; + float MIN_SN = psMetadataLookupF32 (&status, recipe, "MOMENTS_SN_MIN"); psAssert (status, "missing MOMENTS_SN_MIN"); + float PSF_SN_LIM = psMetadataLookupF32(&status, recipe, "PSF_SN_LIM"); psAssert (status, "missing PSF_SN_LIM"); + psF32 MOMENTS_AR_MAX = psMetadataLookupF32(&status, recipe, "MOMENTS_AR_MAX"); psAssert (status, "missing MOMENTS_AR_MAX"); // XXX move this to a config file? - float sigma[4] = {1.0, 2.0, 3.0, 4.5}; - float Sout[4]; + # define NSIGMA 8 + float sigma[NSIGMA] = {1.0, 2.0, 3.0, 4.5, 6.0, 9.0, 12.0, 18.0}; + float Sout[NSIGMA]; // this sorts by peak->SN @@ -257,5 +461,5 @@ // loop over radii: - for (int i = 0; i < 4; i++) { + for (int i = 0; i < NSIGMA; i++) { // XXX move max source number to config @@ -272,14 +476,14 @@ // measure basic source moments (no S/N clipping on input pixels) - status = pmSourceMoments (source, 4*sigma[i], sigma[i], 0.0); + status = pmSourceMoments (source, 4*sigma[i], sigma[i], 0.0, 0xffff); } // choose a grid scale that is a fixed fraction of the psf sigma^2 - psMetadataAddF32(recipe, PS_LIST_TAIL, "PSF_CLUMP_GRID_SCALE", PS_META_REPLACE, "clump grid", 0.1*PS_SQR(sigma[i])); - psMetadataAddF32(recipe, PS_LIST_TAIL, "MOMENTS_SX_MAX", PS_META_REPLACE, "moments limit", 2.0*PS_SQR(sigma[i])); - psMetadataAddF32(recipe, PS_LIST_TAIL, "MOMENTS_SY_MAX", PS_META_REPLACE, "moments limit", 2.0*PS_SQR(sigma[i])); + float PSF_CLUMP_GRID_SCALE = 0.1*PS_SQR(sigma[i]); + float MOMENTS_SX_MAX = 2.0*PS_SQR(sigma[i]); + float MOMENTS_SY_MAX = 2.0*PS_SQR(sigma[i]); // determine the PSF parameters from the source moment values - pmPSFClump psfClump = pmSourcePSFClump (NULL, sources, recipe); + pmPSFClump psfClump = pmSourcePSFClump (NULL, NULL, sources, PSF_SN_LIM, PSF_CLUMP_GRID_SCALE, MOMENTS_SX_MAX, MOMENTS_SY_MAX, MOMENTS_AR_MAX); psLogMsg ("psphot", 3, "radius %.1f, nStars: %d, nSigma: %5.2f, X, Y: %f, %f (%f, %f)\n", sigma[i], psfClump.nStars, psfClump.nSigma, psfClump.X, psfClump.Y, sqrt(psfClump.X) / sigma[i], sqrt(psfClump.Y) / sigma[i]); @@ -296,5 +500,5 @@ psMetadataAddF32 (regionMD, PS_LIST_TAIL, "PSF.CLUMP.DY", PS_META_REPLACE, "psf clump center", psfClump.dY); - // psphotVisualPlotMoments (recipe, sources); + psphotVisualPlotMoments (recipe, analysis, sources); Sout[i] = sqrt(0.5*(psfClump.X + psfClump.Y)) / sigma[i]; @@ -302,16 +506,16 @@ // we are looking for sigma for which Sout = 0.65 (or some other value) - float Sigma = NAN; float minS = Sout[0]; float maxS = Sout[0]; - for (int i = 0; i < 4; i++) { + for (int i = 0; i < NSIGMA; i++) { minS = PS_MIN(Sout[i], minS); maxS = PS_MAX(Sout[i], maxS); } - if (minS > 0.65) Sigma = sigma[3]; + if (minS > 0.65) Sigma = sigma[NSIGMA-1]; if (maxS < 0.65) Sigma = sigma[0]; - for (int i = 0; i < 3; i++) { + for (int i = 0; i < NSIGMA - 1 && isnan(Sigma); i++) { + if (!isfinite(Sout[i]) || !isfinite(Sout[i+1])) continue; if ((Sout[i] > 0.65) && (Sout[i+1] > 0.65)) continue; if ((Sout[i] < 0.65) && (Sout[i+1] < 0.65)) continue; @@ -321,11 +525,15 @@ // choose a grid scale that is a fixed fraction of the psf sigma^2 - psMetadataAddF32(recipe, PS_LIST_TAIL, "PSF_CLUMP_GRID_SCALE", PS_META_REPLACE, "clump grid", 0.1*PS_SQR(Sigma)); - psMetadataAddF32(recipe, PS_LIST_TAIL, "MOMENTS_SX_MAX", PS_META_REPLACE, "moments limit", 2.0*PS_SQR(Sigma)); - psMetadataAddF32(recipe, PS_LIST_TAIL, "MOMENTS_SY_MAX", PS_META_REPLACE, "moments limit", 2.0*PS_SQR(Sigma)); - psMetadataAddF32(recipe, PS_LIST_TAIL, "MOMENTS_GAUSS_SIGMA", PS_META_REPLACE, "moments limit", Sigma); - psMetadataAddF32(recipe, PS_LIST_TAIL, "PSF_MOMENTS_RADIUS", PS_META_REPLACE, "moments limit", 4.0*Sigma); + psMetadataAddF32(analysis, PS_LIST_TAIL, "PSF_CLUMP_GRID_SCALE", PS_META_REPLACE, "clump grid", 0.1*PS_SQR(Sigma)); + psMetadataAddF32(analysis, PS_LIST_TAIL, "MOMENTS_SX_MAX", PS_META_REPLACE, "moments limit", 2.0*PS_SQR(Sigma)); + psMetadataAddF32(analysis, PS_LIST_TAIL, "MOMENTS_SY_MAX", PS_META_REPLACE, "moments limit", 2.0*PS_SQR(Sigma)); + psMetadataAddF32(analysis, PS_LIST_TAIL, "MOMENTS_GAUSS_SIGMA", PS_META_REPLACE, "moments limit", Sigma); + psMetadataAddF32(analysis, PS_LIST_TAIL, "PSF_MOMENTS_RADIUS", PS_META_REPLACE, "moments limit", 4.0*Sigma); psLogMsg ("psphot", 3, "using window function with sigma = %f\n", Sigma); return true; } + +// if we use the footprints, the output peaks list contains both old and new peaks, +// otherwise it only contains the new peaks. + Index: branches/tap_branches/psphot/src/psphotStack.c =================================================================== --- branches/tap_branches/psphot/src/psphotStack.c (revision 27838) +++ branches/tap_branches/psphot/src/psphotStack.c (revision 27838) @@ -0,0 +1,35 @@ +# include "psphotStandAlone.h" + +int main (int argc, char **argv) { + + psTimerStart ("complete"); + pmErrorRegister(); // register psModule's error codes/messages + psphotInit(); + + // load command-line arguments, options, and system config data + pmConfig *config = psphotStackArguments (argc, argv); + assert(config); + + psphotVersionPrint(); + + // load input data (config and images (signal, noise, mask) + if (!psphotStackParseCamera (config)) { + psErrorStackPrint(stderr, "Error setting up the camera\n"); + exit (psphotGetExitStatus()); + } + + // call psphot for each readout + if (!psphotStackImageLoop (config)) { + psErrorStackPrint(stderr, "Error in the psphot image loop\n"); + exit (psphotGetExitStatus()); + } + + psLogMsg ("psphot", 3, "complete psphot run: %f sec\n", psTimerMark ("complete")); + + psErrorCode exit_status = psphotGetExitStatus(); + psphotCleanup (config); + exit (exit_status); +} + +// all functions which return to this level must raise one of the top-level error codes if they +// exit with an error. these error codes are used to specify the program exit status Index: branches/tap_branches/psphot/src/psphotStackArguments.c =================================================================== --- branches/tap_branches/psphot/src/psphotStackArguments.c (revision 27838) +++ branches/tap_branches/psphot/src/psphotStackArguments.c (revision 27838) @@ -0,0 +1,145 @@ +# include "psphotStandAlone.h" + +static void usage(pmConfig *config, int exitCode); +static void writeHelpInfo(FILE* ofile); + +pmConfig *psphotStackArguments(int argc, char **argv) { + + int N; + + // print help info + if (psArgumentGet(argc, argv, "-help")) writeHelpInfo(stdout); + if (psArgumentGet(argc, argv, "-h")) writeHelpInfo(stdout); + + // load config data from default locations + pmConfig *config = pmConfigRead(&argc, argv, PSPHOT_RECIPE); + if (config == NULL) { + psErrorStackPrint(stderr, "Can't read site configuration"); + exit(PS_EXIT_CONFIG_ERROR); + } + + // -version and -dumpconfig arguments + PSARGUMENTS_INSTANTIATE_GENERICS( psphot, config, argc, argv ); + + // save the following additional recipe values based on command-line options + // these options override the PSPHOT recipe values loaded from recipe files + psMetadata *options = pmConfigRecipeOptions (config, PSPHOT_RECIPE); + + // Number of threads is handled + PSARGUMENTS_INSTANTIATE_THREADSARG( psphot, config, argc, argv ) + + // visual : interactive display mode + if ((N = psArgumentGet (argc, argv, "-visual"))) { + psArgumentRemove (N, &argc, argv); + pmVisualSetVisual(true); + } + + // break : used from recipe throughout psphotReadout to stop processing early + if ((N = psArgumentGet (argc, argv, "-break"))) { + if (argc <= N+1) { + psErrorStackPrint(stderr, "Expected to see 1 more argument; saw %d", argc - 1); + exit(PS_EXIT_CONFIG_ERROR); + } + psArgumentRemove (N, &argc, argv); + psMetadataAddStr (options, PS_LIST_TAIL, "BREAK_POINT", PS_META_REPLACE, "", argv[N]); + psArgumentRemove (N, &argc, argv); + } + + // find the input data file (an mdc file) + if ((N = psArgumentGet(argc, argv, "-input"))) { + if (argc <= N+1) { + psErrorStackPrint(stderr, "Expected to see 1 more argument; saw %d", argc - 1); + exit(PS_EXIT_CONFIG_ERROR); + } + psArgumentRemove(N, &argc, argv); + + unsigned int numBad = 0; // Number of bad lines + psMetadata *inputs = psMetadataConfigRead(NULL, &numBad, argv[N], false); // Input file info + if (!inputs || numBad > 0) { + psErrorStackPrint(stderr, "Unable to cleanly read MDC file with inputs."); + exit(PS_EXIT_CONFIG_ERROR); + } + psMetadataAddMetadata(config->arguments, PS_LIST_TAIL, "INPUTS", 0, "Metadata with input details", inputs); + psFree(inputs); + + psArgumentRemove(N, &argc, argv); + } + + if (argc != 2) { + psError(PSPHOT_ERR_ARGUMENTS, true, "Expected to see one more argument; saw %d", argc - 1); + usage(config, PS_EXIT_CONFIG_ERROR); + } + + // output position is fixed + psMetadataAddStr (config->arguments, PS_LIST_TAIL, "OUTPUT", 0, "", argv[1]); + + psTrace("psphot", 1, "Done with psphotArguments...\n"); + return (config); +} + +static void writeHelpInfo(FILE* ofile) +{ + fprintf(ofile, + "Usage: psphotStack -input (INPUTS.mdc) (OUTROOT)\n" + "\n" + "where INPUTS.mdc contains various METADATAs, each with:\n" + "\tIMAGE(STR): Image filename\n" + "\tMASK(STR): Mask filename\n" + "\tVARIANCE(STR): Variance map filename\n" + "OUTROOT is the 'root name' for output files\n" + "\n" + "additional options:\n" + " -chip nn[,nn,...]\n" + " select detector chips to process; default is all.\n" + " Indices correspond to zero-based offset in the FPA metadata table.\n" + " -photcode PhotoCodeName\n" + " specify photocode\n" + " -region RegionString\n" + " specify analysis region. String is of form '[x0:x1,y0:y1]'\n" + " To use this option you must define a default in psphot.config\n" + " -visual\n" + " turns on interactive display mode\n" + " -dumpconfig CfgFileName\n" + " causes config info to be dumped to the named file.\n" + " -break NOTHING|BACKMDL|PEAKS|MOMENTS|PSFMODEL|ENSEMBLE|PASS1\n" + " choose a point at which to exit processing early\n" + " -nthreads n\n" + " set number of parallel threads of execution\n" + " -F OldFileRule ReplacementFileRule\n" + " change file naming rule; e.g. '-F PSPHOT.OUTPUT PSPHOT.OUT.CMF.MEF'\n" + " -D name stringval\n" + " set a string-valued config parameter\n" + " -Di name intval\n" + " set an integer-valued config parameter\n" + " -Df name fval\n" + " set a float-valued config parameter\n" + " -Db name boolval\n" + " set a boolean-valued config parameter\n" + " -v, -vv, -vvv\n" + " set increasing levels of verbosity\n" + " -logfmt FormatString\n" + " set format string used for log messages\n" + " -trace Fac Lvl\n" + " set tracing for facility Fac to integer Lvl, e.g. '-trace err 10'\n" + " -trace-levels\n" + " print current trace levels\n"); + psLibFinalize(); + exit(PS_EXIT_SUCCESS); +} + +static void usage(pmConfig *config, // Configuration + int exitCode + ) +{ + fprintf(stderr, + "Usage: psphotStack -input input.mdc outroot\n" + "Try 'psphotStack -help' for more options and explanation\n"); + + if (exitCode != PS_EXIT_SUCCESS) psErrorStackPrint(stderr, "Error reading arguments\n"); + + psFree(config); + pmConfigDone(); + psLibFinalize(); + exit(exitCode); +} + Index: branches/tap_branches/psphot/src/psphotStackChisqImage.c =================================================================== --- branches/tap_branches/psphot/src/psphotStackChisqImage.c (revision 27838) +++ branches/tap_branches/psphot/src/psphotStackChisqImage.c (revision 27838) @@ -0,0 +1,181 @@ +# include "psphotInternal.h" + +// Chi = sqrt(\sum_i im_i^2/var_i) + +static int npass = 0; + +// XXX supply filename or keep PSPHOT.INPUT fixed? +bool psphotStackChisqImage (pmConfig *config, const pmFPAview *view) +{ + bool status = false; + + psTimerStart ("psphot.chisq.image"); + + pmFPAfile *chisqFile = pmFPAfileSelectSingle(config->files, "PSPHOT.CHISQ.IMAGE", 0); + psAssert (chisqFile, "missing chisq image FPA?"); + + pmReadout *chiReadout = NULL; + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (!psphotStackChisqImageAddReadout(config, view, &chiReadout, "PSPHOT.INPUT", i)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to model background for PSPHOT.INPUT entry %d", i); + return false; + } + } + + num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + psMetadataAddS32(config->arguments, PS_LIST_TAIL, "PSPHOT.CHISQ.NUM", PS_META_REPLACE, "", num); + num++; + psMetadataAddS32(config->arguments, PS_LIST_TAIL, "PSPHOT.INPUT.NUM", PS_META_REPLACE, "", num); + + // need to save the resulting image somewhere (PSPHOT.INPUT?) + if (!psMetadataAddPtr(config->files, PS_LIST_TAIL, "PSPHOT.INPUT", PS_DATA_UNKNOWN | PS_META_DUPLICATE_OK, "", chisqFile)) { + psError(PM_ERR_CONFIG, false, "could not add chisqFPA to config files"); + return false; + } + + psLogMsg ("psphot", PS_LOG_INFO, "built chisq image: %f sec\n", psTimerMark ("psphot.chisq.image")); + + return true; +} + +bool psphotStackChisqImageAddReadout(const pmConfig *config, // Configuration + const pmFPAview *view, + pmReadout **chiReadout, + char *filename, + int index) +{ + bool status = true; + + // find the currently selected readout + pmFPAfile *input = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + psAssert (input, "missing file?"); + + pmReadout *inReadout = pmFPAviewThisReadout(view, input->fpa); + psAssert (inReadout, "missing readout?"); + + psImage *inImage = inReadout->image; + psAssert (inImage, "missing image?"); + + psImage *inVariance = inReadout->variance; + psAssert (inVariance, "missing variance?"); + + psImage *inMask = inReadout->mask; + psAssert (inMask, "missing mask?"); + + if (*chiReadout == NULL) { + *chiReadout = pmFPAGenerateReadout(config, view, "PSPHOT.CHISQ.IMAGE", input->fpa, NULL, 0); + } + + psImage *chiImage = (*chiReadout)->image; + psAssert (chiImage, "missing chi image"); + + psImage *chiVariance = (*chiReadout)->variance; + psAssert (chiVariance, "missing chi variance"); + + psImage *chiMask = (*chiReadout)->mask; + psAssert (chiMask, "missing chi mask"); + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + assert (recipe); + + // user-defined masks to test for good/bad pixels (build from recipe list if not yet set) + psImageMaskType maskVal = psMetadataLookupImageMask(&status, recipe, "MASK.PSPHOT"); // Mask value for bad pixels + assert (maskVal); + + // generate the chisq image: + for (int iy = 0; iy < inImage->numRows; iy++) { + for (int ix = 0; ix < inImage->numCols; ix++) { + if (inMask->data.PS_TYPE_IMAGE_MASK_DATA[iy][ix] & maskVal) continue; + chiImage->data.F32[iy][ix] += PS_SQR(inImage->data.F32[iy][ix]) / inVariance->data.F32[iy][ix]; + chiVariance->data.F32[iy][ix] = 1.0; // ?? what is the right value? just init to this? + chiMask->data.PS_TYPE_IMAGE_MASK_DATA[iy][ix] = 0x00; // we have valid data so unmask this pixel + } + } + + if (psTraceGetLevel("psphot") > 5) { + char name[256]; + sprintf (name, "chiImage.%02d.fits", npass); + psphotSaveImage (NULL, chiImage, name); + } + npass ++; + + return true; +} + +bool psphotStackRemoveChisqFromInputs (pmConfig *config) { + + bool status = false; + + int chisqNum = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.CHISQ.NUM"); + psAssert (status, "programming error: must define PSPHOT.CHISQ.NUM"); + + int inputNum = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + pmFPAfileRemoveSingle (config->files, "PSPHOT.INPUT", chisqNum); + + inputNum --; + psMetadataAddS32(config->arguments, PS_LIST_TAIL, "PSPHOT.INPUT.NUM", PS_META_REPLACE, "", inputNum); + + return true; +} + +bool pmFPAfileRemoveSingle(psMetadata *files, const char *name, int num) +{ + PS_ASSERT_PTR_NON_NULL(files, NULL); + PS_ASSERT_INT_NONNEGATIVE(num, NULL); + + psList* mdList = files->list; + psHash* mdHash = files->hash; + + // Generate a REGEX to select only items that match 'name' + psString regex = NULL; // Regular expression + if (name) { + if (!psMetadataLookup(files, name)) { + // No files match the requested name + return false; + } + psStringAppend(®ex, "^%s$", name); + } + + psMetadataIterator *iter = psMetadataIteratorAlloc(files, PS_LIST_HEAD, regex); // Iterator + psFree(regex); + psMetadataItem *item; // Item from iteration + + bool found = false; + int i = 0; // Counter + for (i = 0; !found && (item = psMetadataGetAndIncrement(iter)); i++) { + if (i == num) found = true; + } + psFree(iter); + if (!found) { + return false; + } + + char *key = item->name; + + // look up the name via hash to see if we have a multi or not + psMetadataItem* hashItem = psHashLookup(mdHash, name); + if (hashItem == NULL) { + psError(PS_ERR_UNKNOWN, false, _("Failed to remove metadata item, %s, from metadata table."), key); + return false; + } + + if (hashItem->type == PS_DATA_METADATA_MULTI) { + // multiple entries with same key, remove just the specified one + psListRemoveData(hashItem->data.list, item); + } else { + psHashRemove(mdHash, key); + } + psListRemoveData(mdList, item); + + return true; +} Index: branches/tap_branches/psphot/src/psphotStackImageLoop.c =================================================================== --- branches/tap_branches/psphot/src/psphotStackImageLoop.c (revision 27838) +++ branches/tap_branches/psphot/src/psphotStackImageLoop.c (revision 27838) @@ -0,0 +1,79 @@ +# include "psphotStandAlone.h" + +# define ESCAPE(MESSAGE) { \ + psError(PSPHOT_ERR_DATA, false, MESSAGE); \ + psFree (view); \ + return false; \ +} + +bool psphotStackImageLoop (pmConfig *config) { + + bool status; + pmChip *chip; + pmCell *cell; + pmReadout *readout; + + pmFPAfile *input = psMetadataLookupPtr (&status, config->files, "PSPHOT.INPUT"); + if (!status) { + psError(PSPHOT_ERR_PROG, false, "Can't find input data!"); + return false; + } + + pmFPAview *view = pmFPAviewAlloc (0); + + // XXX for now, just load the full set of images up front + if (!pmFPAfileIOChecks (config, view, PM_FPA_BEFORE)) ESCAPE ("failed input for fpa in psphot."); + + // for psphot, we force data to be read at the chip level + while ((chip = pmFPAviewNextChip (view, input->fpa, 1)) != NULL) { + psLogMsg ("psphot", 4, "Chip %d: %x %x\n", view->chip, chip->file_exists, chip->process); + if (! chip->process || ! chip->file_exists) { continue; } + if (!pmFPAfileIOChecks (config, view, PM_FPA_BEFORE)) ESCAPE ("failed input for Chip in psphotStack."); + + // there is now only a single chip (multiple readouts?). loop over it and process + while ((cell = pmFPAviewNextCell (view, input->fpa, 1)) != NULL) { + psLogMsg ("psphot", 5, "Cell %d: %x %x\n", view->cell, cell->file_exists, cell->process); + if (!pmFPAfileIOChecks (config, view, PM_FPA_BEFORE)) ESCAPE ("failed input for Cell in psphotStack."); + + // process each of the readouts + while ((readout = pmFPAviewNextReadout (view, input->fpa, 1)) != NULL) { + psLogMsg ("psphot", 6, "Readout %d: %x %x\n", view->readout, cell->file_exists, cell->process); + if (! readout->data_exists) { continue; } + + // XXX for now, we assume there is only a single chip in the PHU: + if (!psphotStackReadout (config, view)) { + psError(psErrorCodeLast(), false, "failure in psphotReadout for chip %d, cell %d, readout %d\n", view->chip, view->cell, view->readout); + psFree (view); + return false; + } + + } + // drop all versions of the internal files + status = true; + status &= pmFPAfileDropInternal (config->files, "PSPHOT.BACKMDL"); + status &= pmFPAfileDropInternal (config->files, "PSPHOT.BACKMDL.STDEV"); + status &= pmFPAfileDropInternal (config->files, "PSPHOT.BACKGND"); + if (!status) { + psError(PSPHOT_ERR_PROG, false, "trouble dropping internal files"); + psFree (view); + return false; + } + } + // save output which is saved at the chip level + if (!pmFPAfileIOChecks (config, view, PM_FPA_AFTER)) ESCAPE ("failed output for Chip in psphot."); + } + // save output which is saved at the fpa level + if (!pmFPAfileIOChecks (config, view, PM_FPA_AFTER)) ESCAPE ("failed ouput for FPA in psphot."); + + // fail if we failed to handle an error + if (psErrorCodeLast() != PS_ERR_NONE) psAbort ("failed to handle an error!"); + + psFree (view); + return true; +} + +/* + the easiest way to implement this is to assume we can pre-load the full set of images up front. + with 5 filters and 6000^2 (image, mask, var = 10 byte per pixel), we need 1.8GB, which is not too bad. +*/ + Index: branches/tap_branches/psphot/src/psphotStackParseCamera.c =================================================================== --- branches/tap_branches/psphot/src/psphotStackParseCamera.c (revision 27838) +++ branches/tap_branches/psphot/src/psphotStackParseCamera.c (revision 27838) @@ -0,0 +1,146 @@ +# include "psphotStandAlone.h" + +static pmFPAfile *defineFile(pmConfig *config, pmFPAfile *bind, const char *name, const char *filename, pmFPAfileType type); + +// define the needed / desired I/O files +bool psphotStackParseCamera (pmConfig *config) { + + bool status = false; + + // the input images are defined as a set of metadatas in the INPUTS metadata folder + psMetadata *inputs = psMetadataLookupMetadata(&status, config->arguments, "INPUTS"); // The inputs info + if (!inputs) { + psError(PS_ERR_UNEXPECTED_NULL, false, "Unable to find inputs."); + return false; + } + + int nInputs = inputs->list->n; + for (int i = 0; i < nInputs; i++) { + psMetadataItem *item = psMetadataGet(inputs, i); + if (item->type != PS_DATA_METADATA) { + psError(PS_ERR_BAD_PARAMETER_TYPE, true, "Component %s of the input metadata is not of type METADATA", item->name); + return false; + } + + psMetadata *input = item->data.md; // The input metadata of interest + + // look for 'IMAGE', 'MASK', 'VARIANCE' in folder (only 'IMAGE' is required) + + psString image = psMetadataLookupStr(NULL, input, "IMAGE"); // Name of image + if (!image || strlen(image) == 0) { + psError(PS_ERR_BAD_PARAMETER_VALUE, true, "Component %s lacks IMAGE of type STR", item->name); + return false; + } + pmFPAfile *imageFile = defineFile(config, NULL, "PSPHOT.INPUT", image, PM_FPA_FILE_IMAGE); // File for image + if (!imageFile) { + psError(PS_ERR_UNKNOWN, false, "Unable to define file from image %d (%s)", i, image); + return false; + } + + psString mask = psMetadataLookupStr(&status, input, "MASK"); // Name of mask + if (mask && strlen(mask) > 0) { + if (!defineFile(config, imageFile, "PSPHOT.MASK", mask, PM_FPA_FILE_MASK)) { + psError(PS_ERR_UNKNOWN, false, "Unable to define file from mask %d (%s)", i, mask); + return false; + } + } + + psString variance = psMetadataLookupStr(&status, input, "VARIANCE"); // Name of variance map + if (variance && strlen(variance) > 0) { + if (!defineFile(config, imageFile, "PSPHOT.VARIANCE", variance, PM_FPA_FILE_VARIANCE)) { + psError(PS_ERR_UNKNOWN, false, "Unable to define file from variance %d (%s)", i, variance); + return false; + } + } + // the output sources are carried on the input->fpa structures + pmFPAfile *outsources = pmFPAfileDefineOutputFromFile (config, imageFile, "PSPHOT.STACK.OUTPUT"); + if (!outsources) { + psError(PSPHOT_ERR_CONFIG, false, "Cannot find a rule for PSPHOT.STACK.OUTPUT"); + return false; + } + outsources->save = true; + outsources->fileID = i; // this is used to generate output names + } + psMetadataRemoveKey(config->arguments, "FILENAMES"); + psMetadataAddS32 (config->arguments, PS_LIST_TAIL, "PSPHOT.INPUT.NUM", PS_META_REPLACE, "number of inputs", nInputs); + + if (!psphotSetMaskBits (config)) { + psError (PS_ERR_UNKNOWN, false, "failed to set mask bit values"); + return NULL; + } + + // generate an pmFPAimage for the chisqImage + pmFPAfile *chisqImage = pmFPAfileDefineOutput(config, NULL, "PSPHOT.CHISQ.IMAGE"); + if (!chisqImage) { + psError(PSPHOT_ERR_CONFIG, false, "Trouble defining PSPHOT.CHISQ.IMAGE"); + return false; + } + chisqImage->save = true; + + pmFPAfile *chisqMask = pmFPAfileDefineOutput(config, chisqImage->fpa, "PSPHOT.CHISQ.MASK"); + if (!chisqMask) { + psError(PS_ERR_IO, false, _("Unable to generate output file from PSPHOT.CHISQ.MASK")); + return NULL; + } + if (chisqMask->type != PM_FPA_FILE_MASK) { + psError(PS_ERR_IO, true, "PSPHOT.CHISQ.MASK is not of type MASK"); + return NULL; + } + chisqMask->save = true; + + pmFPAfile *chisqVariance = pmFPAfileDefineOutput(config, chisqImage->fpa, "PSPHOT.CHISQ.VARIANCE"); + if (!chisqVariance) { + psError(PS_ERR_IO, false, _("Unable to generate output file from PSPHOT.CHISQ.VARIANCE")); + return NULL; + } + if (chisqVariance->type != PM_FPA_FILE_VARIANCE) { + psError(PS_ERR_IO, true, "PSPHOT.CHISQ.VARIANCE is not of type VARIANCE"); + return NULL; + } + chisqVariance->save = true; + +# if (0) + // define the additional input/output files associated with psphot + // XXX figure out which files are needed by psphotStack + if (false && !psphotDefineFiles (config, input)) { + psError(PSPHOT_ERR_CONFIG, false, "Trouble defining the additional input/output files"); + return false; + } +# endif + + psTrace("psphot", 1, "Done with psphotStackParseCamera...\n"); + + psErrorClear(); // some metadata lookup may have failed + return true; +} + +// Define a file +static pmFPAfile *defineFile(pmConfig *config, // Configuration + pmFPAfile *bind, // File to which to bind + const char *name, // Name of file rule + const char *filename, // Name of file + pmFPAfileType type // Type of file + ) +{ + + psArray *dummy = psArrayAlloc(1); // Dummy array of filenames for this FPA + dummy->data[0] = psStringCopy(filename); + psMetadataAddArray(config->arguments, PS_LIST_TAIL, "FILENAMES", PS_META_REPLACE, + "Filenames for file rule definition", dummy); + psFree(dummy); + + bool found = false; // Found the file? + pmFPAfile *file = bind ? pmFPAfileBindFromArgs(&found, bind, config, name, "FILENAMES") : + pmFPAfileDefineFromArgs(&found, config, name, "FILENAMES"); + if (!file || !found) { + psError(PS_ERR_UNKNOWN, false, "Unable to define file %s from %s", name, filename); + return NULL; + } + if (file->type != type) { + psError(PS_ERR_IO, true, "%s is not of type %s", name, pmFPAfileStringFromType(type)); + return NULL; + } + + return file; +} + Index: branches/tap_branches/psphot/src/psphotStackReadout.c =================================================================== --- branches/tap_branches/psphot/src/psphotStackReadout.c (revision 27838) +++ branches/tap_branches/psphot/src/psphotStackReadout.c (revision 27838) @@ -0,0 +1,169 @@ +# include "psphotInternal.h" + +bool psphotStackReadout (pmConfig *config, const pmFPAview *view) { + + psTimerStart ("psphotReadout"); + + pmModelClassSetLimits(PM_MODEL_LIMITS_LAX); // allow models to have ugly fits (eg, central cusp) + + // select the current recipe + psMetadata *recipe = psMetadataLookupPtr (NULL, config->recipes, PSPHOT_RECIPE); + if (!recipe) { + psError(PSPHOT_ERR_CONFIG, false, "missing recipe %s", PSPHOT_RECIPE); + return false; + } + // optional break-point for processing + char *breakPt = psMetadataLookupStr (NULL, recipe, "BREAK_POINT"); + PS_ASSERT_PTR_NON_NULL (breakPt, false); + + // set the photcode for each image + if (!psphotAddPhotcode (config, view)) { + psError (PSPHOT_ERR_CONFIG, false, "trouble defining the photcode"); + return false; + } + + // Generate the mask and weight images + if (!psphotSetMaskAndVariance (config, view)) { + return psphotReadoutCleanup (config, view); + } + if (!strcasecmp (breakPt, "NOTHING")) { + return psphotReadoutCleanup (config, view); + } + + // generate a background model (median, smoothed image) + // XXX I think this is not defined correctly for an array of images. + if (!psphotModelBackground (config, view)) { + return psphotReadoutCleanup (config, view); + } + if (!psphotSubtractBackground (config, view)) { + return psphotReadoutCleanup (config, view); + } + if (!strcasecmp (breakPt, "BACKMDL")) { + return psphotReadoutCleanup (config, view); + } + + // load the psf model, if suppled. FWHM_X,FWHM_Y,etc are determined and saved on + // readout->analysis XXX this function currently only works with a single PSPHOT.INPUT + if (!psphotLoadPSF (config, view)) { + psError (PSPHOT_ERR_UNKNOWN, false, "error loading psf model"); + return psphotReadoutCleanup (config, view); + } + + if (!psphotStackChisqImage(config, view)) { + psError (PSPHOT_ERR_UNKNOWN, false, "failure to generate chisq image"); + return psphotReadoutCleanup (config, view); + } + if (!strcasecmp (breakPt, "CHISQ")) { + return psphotReadoutCleanup (config, view); + } + + // find the detections (by peak and/or footprint) in the image. + // This finds the detections on Chisq image as well as the individuals + if (!psphotFindDetections (config, view, true)) { // pass 1 + // this only happens if we had an error in psphotFindDetections + psError (PSPHOT_ERR_UNKNOWN, false, "failure in peak analysis"); + return psphotReadoutCleanup (config, view); + } + + // construct sources and measure basic stats (saved on detections->newSources) + // only run this on detections from the input images, not chisq image + if (!psphotSourceStats (config, view, true)) { // pass 1 + psError(PSPHOT_ERR_UNKNOWN, false, "failure to generate sources"); + return psphotReadoutCleanup (config, view); + } + + // *** generate the objects (which unify the sources from the different images) + psArray *objects = psphotMatchSources (config, view); + + // construct sources for the newly-generated sources (from other images) + if (!psphotSourceStats (config, view, false)) { // pass 1 + psError(PSPHOT_ERR_UNKNOWN, false, "failure to generate sources"); + return psphotReadoutCleanup (config, view); + } + + // find blended neighbors of very saturated stars (detections->newSources) + // if (!psphotDeblendSatstars (config, view)) { + // psError (PSPHOT_ERR_UNKNOWN, false, "failed on satstar deblend analysis"); + // return psphotReadoutCleanup (config, view); + // } + + // mark blended peaks PS_SOURCE_BLEND (detections->newSources) + // if (!psphotBasicDeblend (config, view)) { + // psError (PSPHOT_ERR_UNKNOWN, false, "failed on deblend analysis"); + // return psphotReadoutCleanup (config, view); + // } + + // classify sources based on moments, brightness + // only run this on detections from the input images, not chisq image + if (!psphotRoughClass (config, view)) { + psError (PSPHOT_ERR_UNKNOWN, false, "failed to determine rough classifications"); + return psphotReadoutCleanup (config, view); + } + // if we were not supplied a PSF model, determine the IQ stats here (detections->newSources) + // only run this on detections from the input images, not chisq image + if (!psphotImageQuality (config, view)) { // pass 1 + psError (PSPHOT_ERR_UNKNOWN, false, "failed to measure image quality"); + return psphotReadoutCleanup(config, view); + } + if (!strcasecmp (breakPt, "MOMENTS")) { + return psphotReadoutCleanup (config, view); + } + + // use bright stellar objects to measure PSF if we were supplied a PSF for any input file, + // this step is skipped + if (!psphotChoosePSF (config, view)) { // pass 1 + psLogMsg ("psphot", 3, "failure to construct a psf model"); + return psphotReadoutCleanup (config, view); + } + if (!strcasecmp (breakPt, "PSFMODEL")) { + return psphotReadoutCleanup (config, view); + } + + // include externally-supplied sources + // XXX fix this in the new multi-input context + // psphotLoadExtSources (config, view); // pass 1 + + // construct an initial model for each object, set the radius to fitRadius, set circular fit mask + psphotGuessModels (config, view); + + // merge the newly selected sources into the existing list + // NOTE: merge OLD and NEW + psphotMergeSources (config, view); + + // linear PSF fit to source peaks, subtract the models from the image (in PSF mask) + psphotFitSourcesLinearStack (config, objects, FALSE); + psFree (objects); + + // identify CRs and extended sources + psphotSourceSize (config, view, TRUE); + + // measure aperture photometry corrections + if (!psphotApResid (config, view)) { + psLogMsg ("psphot", 3, "failed on psphotApResid"); + return psphotReadoutCleanup (config, view); + } + + // calculate source magnitudes + psphotMagnitudes(config, view); + + if (!psphotEfficiency(config, view)) { + psErrorStackPrint(stderr, "Unable to determine detection efficiencies from fake sources"); + psErrorClear(); + } + + // replace failed sources? + // psphotReplaceUnfitSources (sources); + + // replace background in residual image + psphotSkyReplace (config, view); + + // drop the references to the image pixels held by each source + psphotSourceFreePixels (config, view); + + // remove chisq image from config->file:PSPHOT.INPUT (why?) + psphotStackRemoveChisqFromInputs(config); + + // create the exported-metadata and free local data + return psphotReadoutCleanup (config, view); +} + Index: branches/tap_branches/psphot/src/psphotSubtractBackground.c =================================================================== --- branches/tap_branches/psphot/src/psphotSubtractBackground.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotSubtractBackground.c (revision 27838) @@ -4,5 +4,5 @@ // generate the median in NxN boxes, clipping heavily // linear interpolation to generate full-scale model -bool psphotSubtractBackground (pmConfig *config, const pmFPAview *view, const char *filename) +bool psphotSubtractBackgroundReadout (pmConfig *config, const pmFPAview *view, const char *filename, int index, psMetadata *recipe) { bool status = true; @@ -13,5 +13,6 @@ // find the currently selected readout - pmFPAfile *file = psMetadataLookupPtr (&status, config->files, filename); + pmFPAfile *file = pmFPAfileSelectSingle(config->files, filename, index); // File of interest + pmFPA *inFPA = file->fpa; pmReadout *readout = pmFPAviewThisReadout (view, inFPA); @@ -19,11 +20,15 @@ psImage *mask = readout->mask; - // find the currently selected readout - pmReadout *model = pmFPAfileThisReadout (config->files, view, "PSPHOT.BACKMDL"); + // find the currently selected readout (XXX note that the model is saved on PSPHOT.BACKMDL regardless of 'filename' + pmFPAfile *modelFile = pmFPAfileSelectSingle(config->files, "PSPHOT.BACKMDL", index); // File of interest + assert (modelFile); + + pmReadout *model = NULL; + if (modelFile->mode == PM_FPA_MODE_INTERNAL) { + model = modelFile->readout; + } else { + model = pmFPAviewThisReadout (view, modelFile->fpa); + } assert (model); - - // select the appropriate recipe information - psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); - assert (recipe); // user-defined masks to test for good/bad pixels (build from recipe list if not yet set) @@ -35,4 +40,5 @@ // select background pixels, from output background file, or create + // XXX for now, we will only allow a single background image to be generated file = psMetadataLookupPtr (&status, config->files, "PSPHOT.BACKGND"); if (file) { @@ -66,4 +72,5 @@ // back-sub image pixels, from output background file (don't create if not requested) + // XXX for now, we will only allow a single background-subtracted image to be generated file = psMetadataLookupPtr (&status, config->files, "PSPHOT.BACKSUB"); if (file) { @@ -109,5 +116,30 @@ // the pmReadout selected in this function are all view on entries in config->files + // display the backsub and backgnd images + // move this inthe the subtract background loop + psphotVisualShowBackground (config, view, readout); + npass ++; return true; } + +bool psphotSubtractBackground (pmConfig *config, const pmFPAview *view) +{ + bool status = false; + + // select the appropriate recipe information + psMetadata *recipe = psMetadataLookupPtr (&status, config->recipes, PSPHOT_RECIPE); + psAssert (recipe, "missing recipe?"); + + int num = psMetadataLookupS32 (&status, config->arguments, "PSPHOT.INPUT.NUM"); + psAssert (status, "programming error: must define PSPHOT.INPUT.NUM"); + + // loop over the available readouts + for (int i = 0; i < num; i++) { + if (!psphotSubtractBackgroundReadout (config, view, "PSPHOT.INPUT", i, recipe)) { + psError (PSPHOT_ERR_CONFIG, false, "failed to subtract background for PSPHOT.INPUT entry %d", i); + return false; + } + } + return true; +} Index: branches/tap_branches/psphot/src/psphotVisual.c =================================================================== --- branches/tap_branches/psphot/src/psphotVisual.c (revision 25900) +++ branches/tap_branches/psphot/src/psphotVisual.c (revision 27838) @@ -60,4 +60,18 @@ } +bool psphotVisualEraseOverlays (int channel, char *overlay) { + + int myKapa = psphotKapaChannel (channel); + if (!(strcasecmp (overlay, "all"))) { + KiiEraseOverlay (myKapa, "red"); + KiiEraseOverlay (myKapa, "green"); + KiiEraseOverlay (myKapa, "blue"); + KiiEraseOverlay (myKapa, "yellow"); + return true; + } + KiiEraseOverlay (myKapa, overlay); + return true; +} + bool psphotVisualShowMask (int kapaFD, psImage *inImage, const char *name, int channel) { @@ -106,5 +120,5 @@ } -bool psphotVisualScaleImage (int kapaFD, psImage *inImage, const char *name, int channel) { +bool psphotVisualScaleImage (int kapaFD, psImage *inImage, psImage *inMask, const char *name, int channel) { KiiImage image; @@ -116,5 +130,5 @@ psStats *stats = psStatsAlloc (PS_STAT_ROBUST_MEDIAN | PS_STAT_ROBUST_STDEV); psRandom *rng = psRandomAlloc(PS_RANDOM_TAUS); - if (!psImageBackground(stats, NULL, inImage, NULL, 0, rng)) { + if (!psImageBackground(stats, NULL, inImage, inMask, 0xffff, rng)) { fprintf (stderr, "failed to get background values\n"); return false; @@ -171,7 +185,7 @@ if (kapa == -1) return false; - // psphotVisualShowMask (kapa, readout->mask, "mask", 2); - psphotVisualScaleImage (kapa, readout->variance, "variance", 1); - psphotVisualScaleImage (kapa, readout->image, "image", 0); + psphotVisualShowMask (kapa, readout->mask, "mask", 2); + psphotVisualScaleImage (kapa, readout->variance, readout->mask, "variance", 1); + psphotVisualScaleImage (kapa, readout->image, readout->mask, "image", 0); pmVisualAskUser(NULL); @@ -197,6 +211,6 @@ } - psphotVisualScaleImage (kapa, backgnd->image, "backgnd", 2); - psphotVisualScaleImage (kapa, readout->image, "backsub", 0); + psphotVisualScaleImage (kapa, backgnd->image, readout->mask, "backgnd", 2); + psphotVisualScaleImage (kapa, readout->image, readout->mask, "backsub", 0); pmVisualAskUser(NULL); @@ -204,5 +218,5 @@ } -bool psphotVisualShowSignificance (psImage *image) { +bool psphotVisualShowSignificance (psImage *image, float min, float max) { if (!pmVisualIsVisual()) return true; @@ -211,5 +225,5 @@ if (kapa == -1) return false; - psphotVisualRangeImage (kapa, image, "signif", 2, -1.0, 25.0*25.0); + psphotVisualRangeImage (kapa, image, "signif", 2, min, max); pmVisualAskUser(NULL); @@ -247,36 +261,5 @@ overlay[Noverlay].text = NULL; Noverlay ++; - -# if (0) - overlay[Noverlay].type = KII_OVERLAY_BOX; - overlay[Noverlay].x = peak->x; - overlay[Noverlay].y = peak->y; - overlay[Noverlay].dx = 1.0; - overlay[Noverlay].dy = 1.0; - overlay[Noverlay].angle = 0.0; - overlay[Noverlay].text = NULL; - Noverlay ++; - - overlay[Noverlay].type = KII_OVERLAY_CIRCLE; - overlay[Noverlay].x = peak->xf; - overlay[Noverlay].y = peak->yf; - overlay[Noverlay].dx = 2.0; - overlay[Noverlay].dy = 2.0; - overlay[Noverlay].angle = 0.0; - overlay[Noverlay].text = NULL; - Noverlay ++; -# endif - } - -# if (1) - overlay[Noverlay].type = KII_OVERLAY_BOX; - overlay[Noverlay].x = 10.0; - overlay[Noverlay].y = 10.0; - overlay[Noverlay].dx = 1.0; - overlay[Noverlay].dy = 1.0; - overlay[Noverlay].angle = 0.0; - overlay[Noverlay].text = NULL; - Noverlay ++; -# endif + } KiiLoadOverlay (kapa, overlay, Noverlay, "red"); @@ -463,4 +446,6 @@ KapaSetFont (myKapa, "courier", 14); + section.bg = KapaColorByName ("none"); // XXX probably should be 'none' + float SN_LIM = psMetadataLookupF32(&status, recipe, "PSF_SN_LIM"); @@ -494,4 +479,8 @@ Xmin = PS_MAX(Xmin, -0.1); Ymin = PS_MAX(Ymin, -0.1); + + // XXX test: hardwire plot limits + // Xmin = -0.1; Ymin = -0.1; + // Xmax = 20.1; Ymax = 20.1; // storage vectors for data to be plotted @@ -551,5 +540,10 @@ KapaSetLimits (myKapa, &graphdata); + graphdata.padXm = NAN; + graphdata.padYm = NAN; + graphdata.padXp = 0.5; + graphdata.padYp = 0.5; KapaBox (myKapa, &graphdata); + KapaSendLabel (myKapa, "M_xx| (pixels)", KAPA_LABEL_XM); KapaSendLabel (myKapa, "M_yy| (pixels)", KAPA_LABEL_YM); @@ -560,4 +554,5 @@ graphdata.style = 2; KapaPrepPlot (myKapa, nF, &graphdata); + KapaPlotVector (myKapa, nF, xFaint->data.F32, "x"); KapaPlotVector (myKapa, nF, yFaint->data.F32, "y"); @@ -575,5 +570,5 @@ section.dy = 0.25; section.x = 0.00; - section.y = 0.80; + section.y = 0.75; section.name = psStringCopy ("MagMyy"); KapaSetSection (myKapa, §ion); @@ -587,4 +582,8 @@ KapaSetLimits (myKapa, &graphdata); + graphdata.padXm = 0.5; + graphdata.padYm = NAN; + graphdata.padXp = NAN; + graphdata.padYp = 0.5; strcpy (graphdata.labels, "0210"); KapaBox (myKapa, &graphdata); @@ -611,5 +610,5 @@ section.dx = 0.25; section.dy = 0.75; - section.x = 0.80; + section.x = 0.75; section.y = 0.00; section.name = psStringCopy ("MagMxx"); @@ -624,4 +623,8 @@ KapaSetLimits (myKapa, &graphdata); + graphdata.padXm = NAN; + graphdata.padYm = 0.5; + graphdata.padXp = 0.5; + graphdata.padYp = NAN; strcpy (graphdata.labels, "2001"); KapaBox (myKapa, &graphdata); @@ -654,5 +657,5 @@ int nRegions = psMetadataLookupS32 (&status, analysis, "PSF.CLUMP.NREGIONS"); - float PSF_CLUMP_NSIGMA = psMetadataLookupF32 (&status, analysis, "PSF_CLUMP_NSIGMA"); + float PSF_CLUMP_NSIGMA = psMetadataLookupF32 (&status, recipe, "PSF_CLUMP_NSIGMA"); graphdata.color = KapaColorByName ("blue"); @@ -1107,9 +1110,21 @@ } - psphotVisualScaleImage (myKapa, outsat, "satstar", 2); + psphotVisualScaleImage (myKapa, outsat, NULL, "satstar", 2); pmVisualAskUser(NULL); psFree (outsat); return true; +} + +static void plotline (int myKapa, Graphdata *graphdata, float x0, float y0, float x1, float y1) +{ + float x[2], y[2]; + x[0] = x0; + x[1] = x1; + y[0] = y0; + y[1] = y1; + KapaPrepPlot (myKapa, 2, graphdata); + KapaPlotVector (myKapa, 2, x, "x"); + KapaPlotVector (myKapa, 2, y, "y"); } @@ -1151,4 +1166,64 @@ } + // generate model profiles (major and minor axis): + // create a model with theta = 0.0 so major and minor axes are equiv to x and y: + psEllipseShape rawShape, rotShape; + + rawShape.sx = source->modelPSF->params->data.F32[PM_PAR_SXX] / M_SQRT2; + rawShape.sy = source->modelPSF->params->data.F32[PM_PAR_SYY] / M_SQRT2; + rawShape.sxy = source->modelPSF->params->data.F32[PM_PAR_SXY]; + + psEllipseAxes axes = psEllipseShapeToAxes (rawShape, 20.0); + + axes.theta = 0.0; + + rotShape = psEllipseAxesToShape (axes); + + psVector *params = psVectorAlloc(source->modelPSF->params->n, PS_TYPE_F32); + for (int i = 0; i < source->modelPSF->params->n; i++) { + params->data.F32[i] = source->modelPSF->params->data.F32[i]; + } + params->data.F32[PM_PAR_SXX] = rotShape.sx * M_SQRT2; + params->data.F32[PM_PAR_SYY] = rotShape.sy * M_SQRT2; + params->data.F32[PM_PAR_SXY] = rotShape.sxy; + params->data.F32[PM_PAR_XPOS] = 0.0; + params->data.F32[PM_PAR_YPOS] = 0.0; + + psVector *rmod = psVectorAlloc(300, PS_TYPE_F32); + psVector *fmaj = psVectorAlloc(300, PS_TYPE_F32); + psVector *fmin = psVectorAlloc(300, PS_TYPE_F32); + + psVector *coord = psVectorAlloc(2, PS_TYPE_F32); + + float r = 0.0; + for (int i = 0; i < rmod->n; i++) { + r = i*0.1; + rmod->data.F32[i] = r; + + coord->data.F32[1] = r; + coord->data.F32[0] = 0.0; + fmaj->data.F32[i] = log10(source->modelPSF->modelFunc (NULL, params, coord)); + + coord->data.F32[0] = r; + coord->data.F32[1] = 0.0; + fmin->data.F32[i] = log10(source->modelPSF->modelFunc (NULL, params, coord)); + } + psFree (coord); + psFree (params); + + float FWHM_MAJOR = 2.0*source->modelPSF->modelRadius (source->modelPSF->params, 0.5*source->modelPSF->params->data.F32[PM_PAR_I0]); + float FWHM_MINOR = FWHM_MAJOR * (axes.minor / axes.major); + if (FWHM_MAJOR < FWHM_MINOR) PS_SWAP (FWHM_MAJOR, FWHM_MINOR); + + psEllipseMoments emoments; + emoments.x2 = source->moments->Mxx; + emoments.xy = source->moments->Mxy; + emoments.y2 = source->moments->Myy; + axes = psEllipseMomentsToAxes (emoments, 20.0); + float MOMENTS_MAJOR = 2.355*axes.major; + float MOMENTS_MINOR = 2.355*axes.minor; + + float logHM = log10(0.5*source->modelPSF->params->data.F32[PM_PAR_I0]); + // reset source Add/Sub state to recorded if (subtracted) pmSourceSub (source, PM_MODEL_OP_FULL, maskVal); @@ -1187,4 +1262,33 @@ KapaPlotVector (myKapa, nb, fb->data.F32, "y"); + graphdata.color = KapaColorByName ("blue"); + graphdata.ptype = 0; + graphdata.size = 0.0; + graphdata.style = 0; + KapaPrepPlot (myKapa, rmod->n, &graphdata); + KapaPlotVector (myKapa, rmod->n, rmod->data.F32, "x"); + KapaPlotVector (myKapa, rmod->n, fmin->data.F32, "y"); + plotline (myKapa, &graphdata, 0.0, logHM, 30.0, logHM); + plotline (myKapa, &graphdata, 0.5*FWHM_MINOR, 0.0, 0.5*FWHM_MINOR, 5.0); + graphdata.ltype = 1; + plotline (myKapa, &graphdata, 0.5*MOMENTS_MINOR, 0.0, 0.5*MOMENTS_MINOR, 5.0); + graphdata.ltype = 0; + + graphdata.color = KapaColorByName ("green"); + graphdata.ptype = 0; + graphdata.size = 0.0; + graphdata.style = 0; + KapaPrepPlot (myKapa, rmod->n, &graphdata); + KapaPlotVector (myKapa, rmod->n, rmod->data.F32, "x"); + KapaPlotVector (myKapa, rmod->n, fmaj->data.F32, "y"); + plotline (myKapa, &graphdata, 0.5*FWHM_MAJOR, 0.0, 0.5*FWHM_MAJOR, 5.0); + graphdata.ltype = 1; + plotline (myKapa, &graphdata, 0.5*MOMENTS_MAJOR, 0.0, 0.5*MOMENTS_MAJOR, 5.0); + graphdata.ltype = 0; + + for (int i = 0; i < rmod->n; i++) { + rmod->data.F32[i] = log10(rmod->data.F32[i]); + } + // ** loglog ** KapaSelectSection (myKapa, "loglog"); @@ -1195,4 +1299,5 @@ graphdata.ymin = -0.05; graphdata.ymax = +5.05; + graphdata.color = KapaColorByName ("black"); KapaSetLimits (myKapa, &graphdata); @@ -1217,4 +1322,24 @@ KapaPlotVector (myKapa, nb, Rb->data.F32, "x"); KapaPlotVector (myKapa, nb, fb->data.F32, "y"); + + graphdata.color = KapaColorByName ("blue"); + graphdata.ptype = 0; + graphdata.size = 0.0; + graphdata.style = 0; + KapaPrepPlot (myKapa, rmod->n, &graphdata); + KapaPlotVector (myKapa, rmod->n, rmod->data.F32, "x"); + KapaPlotVector (myKapa, rmod->n, fmin->data.F32, "y"); + + graphdata.color = KapaColorByName ("green"); + graphdata.ptype = 0; + graphdata.size = 0.0; + graphdata.style = 0; + KapaPrepPlot (myKapa, rmod->n, &graphdata); + KapaPlotVector (myKapa, rmod->n, rmod->data.F32, "x"); + KapaPlotVector (myKapa, rmod->n, fmaj->data.F32, "y"); + + psFree (rmod); + psFree (fmin); + psFree (fmaj); psFree (rg); @@ -1240,4 +1365,6 @@ psImageMaskType maskVal = psMetadataLookupImageMask(&status, recipe, "MASK.PSPHOT"); // Mask value for bad pixels assert (maskVal); + + section.bg = KapaColorByName ("none"); // XXX probably should be 'none' KapaClearPlots (myKapa); @@ -1405,5 +1532,4 @@ if (source == NULL) continue; - // if (source->type != type) continue; if (mode) { if (keep) { @@ -1478,4 +1604,6 @@ KapaSetFont (myKapa, "courier", 14); + section.bg = KapaColorByName ("none"); // XXX probably should be 'none' + // select the max psfX,Y values for the plot limits float Xmin = 1000.0, Xmax = 0.0; @@ -1529,4 +1657,9 @@ psVector *sDEF = psVectorAllocEmpty (sources->n, PS_TYPE_F32); + psVector *xLOW = psVectorAllocEmpty (sources->n, PS_TYPE_F32); + psVector *yLOW = psVectorAllocEmpty (sources->n, PS_TYPE_F32); + psVector *mLOW = psVectorAllocEmpty (sources->n, PS_TYPE_F32); + psVector *sLOW = psVectorAllocEmpty (sources->n, PS_TYPE_F32); + psVector *xCR = psVectorAllocEmpty (sources->n, PS_TYPE_F32); psVector *yCR = psVectorAllocEmpty (sources->n, PS_TYPE_F32); @@ -1539,8 +1672,12 @@ int nPSF = 0; int nDEF = 0; + int nLOW = 0; int nCR = 0; for (int i = 0; i < sources->n; i++) { pmSource *source = sources->data[i]; if (source->moments == NULL) continue; + + // only plot the measured sources... + if (!(source->tmpFlags & PM_SOURCE_TMPF_SIZE_MEASURED)) continue; if (source->mode & PM_SOURCE_MODE_CR_LIMIT) { @@ -1574,5 +1711,10 @@ continue; } - if ((source->mode & PM_SOURCE_MODE_CR_LIMIT) || (source->mode & PM_SOURCE_MODE_SATSTAR)) { + if (source->errMag > 0.1) { + xLOW->data.F32[nLOW] = source->moments->Mxx; + yLOW->data.F32[nLOW] = source->moments->Myy; + mLOW->data.F32[nLOW] = -2.5*log10(source->moments->Sum); + sLOW->data.F32[nLOW] = source->extNsigma; + nLOW++; continue; } @@ -1583,4 +1725,5 @@ nPSF++; } + xSAT->n = nSAT; ySAT->n = nSAT; @@ -1607,4 +1750,9 @@ mDEF->n = nDEF; sDEF->n = nDEF; + + xLOW->n = nLOW; + yLOW->n = nLOW; + mLOW->n = nLOW; + sLOW->n = nLOW; // four sections: MxxMyy, MagMxx, MagMyy, MagSigma @@ -1626,4 +1774,8 @@ KapaSetLimits (myKapa, &graphdata); + graphdata.padXm = NAN; + graphdata.padYm = NAN; + graphdata.padXp = 0.5; + graphdata.padYp = 0.5; KapaBox (myKapa, &graphdata); KapaSendLabel (myKapa, "M_xx| (pixels)", KAPA_LABEL_XM); @@ -1669,4 +1821,12 @@ KapaPlotVector (myKapa, nSAT, xSAT->data.F32, "x"); KapaPlotVector (myKapa, nSAT, ySAT->data.F32, "y"); + + graphdata.color = KapaColorByName ("black"); + graphdata.ptype = 7; + graphdata.size = 1.0; + graphdata.style = 2; + KapaPrepPlot (myKapa, nLOW, &graphdata); + KapaPlotVector (myKapa, nLOW, xLOW->data.F32, "x"); + KapaPlotVector (myKapa, nLOW, yLOW->data.F32, "y"); // second section: MagMyy @@ -1681,9 +1841,13 @@ graphdata.color = KapaColorByName ("black"); graphdata.xmin = -17.1; - graphdata.xmax = -7.9; + graphdata.xmax = -6.9; graphdata.ymin = Ymin; graphdata.ymax = Ymax; KapaSetLimits (myKapa, &graphdata); + graphdata.padXm = 0.5; + graphdata.padYm = NAN; + graphdata.padXp = NAN; + graphdata.padYp = 0.5; strcpy (graphdata.labels, "0210"); KapaBox (myKapa, &graphdata); @@ -1734,5 +1898,5 @@ section.dx = 0.25; section.dy = 0.60; - section.x = 0.80; + section.x = 0.75; section.y = 0.00; section.name = psStringCopy ("MagMxx"); @@ -1743,8 +1907,12 @@ graphdata.xmin = Xmin; graphdata.xmax = Xmax; - graphdata.ymin = -7.9; + graphdata.ymin = -6.9; graphdata.ymax = -17.1; KapaSetLimits (myKapa, &graphdata); + graphdata.padXm = NAN; + graphdata.padYm = 0.5; + graphdata.padXp = 0.5; + graphdata.padYp = NAN; strcpy (graphdata.labels, "2001"); KapaBox (myKapa, &graphdata); @@ -1794,7 +1962,7 @@ // fourth section: MagSigma section.dx = 0.75; - section.dy = 0.15; + section.dy = 0.20; section.x = 0.00; - section.y = 0.65; + section.y = 0.60; section.name = psStringCopy ("MagSigma"); KapaSetSection (myKapa, §ion); @@ -1802,5 +1970,5 @@ graphdata.color = KapaColorByName ("black"); - graphdata.xmax = -7.9; + graphdata.xmax = -6.9; graphdata.xmin = -17.1; graphdata.ymin = -20.1; @@ -1808,4 +1976,8 @@ KapaSetLimits (myKapa, &graphdata); + graphdata.padXm = 0.5; + graphdata.padYm = NAN; + graphdata.padXp = 0.5; + graphdata.padYp = 0.5; strcpy (graphdata.labels, "0100"); KapaBox (myKapa, &graphdata); @@ -1862,5 +2034,5 @@ int nRegions = psMetadataLookupS32 (&status, analysis, "PSF.CLUMP.NREGIONS"); - float PSF_CLUMP_NSIGMA = psMetadataLookupF32 (&status, analysis, "PSF_CLUMP_NSIGMA"); + float PSF_CLUMP_NSIGMA = psMetadataLookupF32 (&status, recipe, "PSF_CLUMP_NSIGMA"); graphdata.color = KapaColorByName ("blue"); @@ -1901,20 +2073,30 @@ psFree (ySAT); psFree (mSAT); + psFree (sSAT); psFree (xEXT); psFree (yEXT); psFree (mEXT); + psFree (sEXT); psFree (xPSF); psFree (yPSF); psFree (mPSF); + psFree (sPSF); psFree (xDEF); psFree (yDEF); psFree (mDEF); + psFree (sDEF); + + psFree (xLOW); + psFree (yLOW); + psFree (mLOW); + psFree (sLOW); psFree (xCR); psFree (yCR); psFree (mCR); + psFree (sCR); pmVisualAskUser(NULL); @@ -1929,5 +2111,5 @@ if (myKapa == -1) return false; - psphotVisualScaleImage (myKapa, readout->image, "resid", 1); + psphotVisualScaleImage (myKapa, readout->image, readout->mask, "resid", 1); pmVisualAskUser(NULL); @@ -2054,4 +2236,87 @@ } +bool psphotVisualPlotChisq (psArray *sources) { + + Graphdata graphdata; + + if (!pmVisualIsVisual()) return true; + + int myKapa = psphotKapaChannel (2); + if (myKapa == -1) return false; + + KapaClearPlots (myKapa); + KapaInitGraph (&graphdata); + + psVector *x = psVectorAllocEmpty (sources->n, PS_TYPE_F32); + psVector *y = psVectorAllocEmpty (sources->n, PS_TYPE_F32); + + graphdata.xmin = +32.0; + graphdata.xmax = -32.0; + graphdata.ymin = +32.0; + graphdata.ymax = -32.0; + + FILE *f = fopen ("chisq.dat", "w"); + + // construct the plot vectors + int n = 0; + for (int i = 0; i < sources->n; i++) { + pmSource *source = sources->data[i]; + if (!source) continue; + if (source->type != PM_SOURCE_TYPE_STAR) continue; + if (!source->moments) continue; + if (!isfinite(source->moments->Sum)) continue; + if (!source->modelPSF) continue; + if (!isfinite(source->modelPSF->chisq)) continue; + + x->data.F32[n] = -2.5*log10(source->moments->Sum); + y->data.F32[n] = source->modelPSF->chisq / source->modelPSF->nDOF; + graphdata.xmin = PS_MIN(graphdata.xmin, x->data.F32[n]); + graphdata.xmax = PS_MAX(graphdata.xmax, x->data.F32[n]); + graphdata.ymin = PS_MIN(graphdata.ymin, y->data.F32[n]); + graphdata.ymax = PS_MAX(graphdata.ymax, y->data.F32[n]); + + fprintf (f, "%d %d %f %f\n", i, n, x->data.F32[n], y->data.F32[n]); + + n++; + } + x->n = y->n = n; + fclose (f); + + float range; + range = graphdata.xmax - graphdata.xmin; + graphdata.xmax += 0.05*range; + graphdata.xmin -= 0.05*range; + range = graphdata.ymax - graphdata.ymin; + graphdata.ymax += 0.05*range; + graphdata.ymin -= 0.05*range; + + // XXX test + graphdata.xmin = -17.0; + graphdata.xmax = -3.0; + graphdata.ymin = -0.1; + graphdata.ymax = +10.1; + + KapaSetLimits (myKapa, &graphdata); + + KapaSetFont (myKapa, "helvetica", 14); + KapaBox (myKapa, &graphdata); + KapaSendLabel (myKapa, "PSF Mag", KAPA_LABEL_XM); + KapaSendLabel (myKapa, "ChiSq", KAPA_LABEL_YM); + + graphdata.color = KapaColorByName ("black"); + graphdata.ptype = 2; + graphdata.size = 0.5; + graphdata.style = 2; + KapaPrepPlot (myKapa, n, &graphdata); + KapaPlotVector (myKapa, n, x->data.F32, "x"); + KapaPlotVector (myKapa, n, y->data.F32, "y"); + + psFree (x); + psFree (y); + + pmVisualAskUser(NULL); + return true; +} + bool psphotVisualShowPetrosians (psArray *sources) { @@ -2073,29 +2338,28 @@ if (!source) continue; if (!source->extpars) continue; - if (!source->extpars->profile) continue; - if (!source->extpars->petrosian_80) continue; - - pmSourceRadialProfile *profile = source->extpars->profile; - pmSourceExtendedFlux *petrosian = source->extpars->petrosian_80; + if (!source->extpars->petProfile) continue; + + float petrosianRadius = source->extpars->petrosianRadius; + psEllipseAxes *axes = &source->extpars->axes; overlay[Noverlay].type = KII_OVERLAY_CIRCLE; overlay[Noverlay].x = source->peak->xf; overlay[Noverlay].y = source->peak->yf; - overlay[Noverlay].dx = 2.0*petrosian->radius; - overlay[Noverlay].dy = 2.0*petrosian->radius*profile->axes.minor/profile->axes.major; - overlay[Noverlay].angle = profile->axes.theta * PS_DEG_RAD; + overlay[Noverlay].dx = 1.0*petrosianRadius; + overlay[Noverlay].dy = 1.0*petrosianRadius*axes->minor/axes->major; + overlay[Noverlay].angle = axes->theta * PS_DEG_RAD; overlay[Noverlay].text = NULL; Noverlay ++; CHECK_REALLOCATE (overlay, KiiOverlay, NOVERLAY, Noverlay, 100); - overlay[Noverlay].type = KII_OVERLAY_CIRCLE; - overlay[Noverlay].x = source->peak->xf; - overlay[Noverlay].y = source->peak->yf; - overlay[Noverlay].dx = 4.0*petrosian->radius; - overlay[Noverlay].dy = 4.0*petrosian->radius*profile->axes.minor/profile->axes.major; - overlay[Noverlay].angle = profile->axes.theta * PS_DEG_RAD; - overlay[Noverlay].text = NULL; - Noverlay ++; - CHECK_REALLOCATE (overlay, KiiOverlay, NOVERLAY, Noverlay, 100); + // overlay[Noverlay].type = KII_OVERLAY_CIRCLE; + // overlay[Noverlay].x = source->peak->xf; + // overlay[Noverlay].y = source->peak->yf; + // overlay[Noverlay].dx = 2.0*petrosianRadius; + // overlay[Noverlay].dy = 2.0*petrosianRadius*axes->minor/axes->major; + // overlay[Noverlay].angle = axes->theta * PS_DEG_RAD; + // overlay[Noverlay].text = NULL; + // Noverlay ++; + // CHECK_REALLOCATE (overlay, KiiOverlay, NOVERLAY, Noverlay, 100); }