Bill looked at a sample of the IPP to PSPS batches posted on the data store. Below are some notes on some things that I noticed. Comments on the various batch samples examined. I looked for -999 valued columns, zero valued columns that look suspicous and things like that. Note: In the comments below a leading x in column name means that comment applies to columns for all filters in grizy. For example xQFPerfect. PSPS_SAS39_OB "Object Batch" ----------------------------- Looks like these batches contain the data for the table MeanObject. xQFPerfect is always -999 Do these columns make sense for an average object? Seems like this column is a detection level thing. Maybe an average would be useful? PSPS_SAS39_P2 "P2 Batch" ------------------------------ This contains the data for an individual exposure. Extensions FrameMeta, 60 * (ImageMeta, Detection) FrameMeta: 1 row -999 columns: p1Recip, p2Recip, p3Recip ImageMeta: 1 row -999 columns: psfModelID In IPP speak this whould be "3" which is the value for PS_MODEL_PS1_V1 which is listed in the cmf header. Here I suppose it should be in psps space. There is no corresponding value in the FitModel table. Is the intent that it should be PS_MODEL_PSF (1) but. That doesn't allow for the possibility for using a different PSF model type in future releases. detrend4-8 I think that there are a couple of other detrend types that are used in the processing could be saved here if those would be interesting. 0 valued columns should DataRelease be 0? Definitely not in PV3 I think. calibModNum. Maybe we should start with 1 instead of 0 for first calibration? Detection: many rows of course -999 columns: None 0 valued columns: dataRelease PSPS_SAS39_DF "Diff Batch" ------------------------------ Extensions: Primary Array, NSkycells * (DiffMeta, DiffToImage, DiffDetection) These 2 items are more of a comment on the schema. Or perhaps I am missing something important. Where is the ipp diff_id ? Looks like the diffs for SAS 39 were type 2: warp-stack DiffMeta It looks like the diffMeta has ImageIDs for the positive and negative images. But PSPS ImageIDs are for chips. Diff inputs are for skycells so there is a many to one problem. What are posImageID and negImageID ? NEEDS REVIEW: Ok it looks like posImageID is the warp_id for the diffRun and negImageID looks like it's trying to be the stack_id of the template. However the skycell_ids aren't SAS. They seem to pv2 stacks (which were recently set to be cleaned). Or maybe I'm lost. DiffToImage Perhaps this table resolves the issue. It looks like all of these DiffToImage tables are the same. It gives a list of the input ImageMetas. Detection -999 columns. There are several. Perhaps they are things that aren't measured for diffs. Needs more study. PSPS_SAS39_DO "Diff Object Batch" --------------------------------- Extensions: Primary Array, DiffDetObject I don't know what a diff object is? Diff detections matched to each other from different diffRuns? Looks like it. Must be created in DVO. No attempted match to PSPS objects? Sounds compute intensive I guess. -999 columns: htmID zoneID cx,cy,cz lamda, beta, l, b, xQfPerfect PSPS_SAS39_ST "Stack Batch" one per skycell ------------------------------------------- Extensions ---------- 5 StackMeta (5 filters) StackObjectThin StackObjectAttributes And "the various other stack tables: model fits, aperture fluxes, etc" StackToImage: list of images that went into the stack. (Bill, recall ImageID == exp_id * 100 + (int) class_id) StackMeta Description of each stack (stackMetaID == stackRun.stack_id) -999 columns: magSat, competMag, astroScat, photoScat, nAstro, nPhoRef, calibModNum StackObjectThin Magnitudes only. No fluxes -999 columns: stackDetectRowID: which of 5 possible stack detections this row refers to. Up to 3 secondary, 1 best. I think I heard that there were problems setting the primary flag in SAS 39. 0 valued columns: primary bestDetection xInfoFlag2 !!!!!!!!!!! Stack detections very hard to use without this flag! StackObjectAttributes Fluxes only (in janskys) no magnitudes. Has the detection attributes, but not the flags. StackModelFitExtra galaxy parameters StackModelFitExp, DeV, Ser model fit data StackPetrosian petrosian extension stuff StackApFlux results for aperture flux measurements for radius 5, 6, and 7 for the 3 images Looks like a choice of the "best" stack for each object. StackApFlxExGalUnc,Con6, Con8 This has all of the app flux radii measurements. Other radii. Non best stacks. (what did I mean here?) -------------------------------------------------------------------------------------------------- PSPS_SAS_39_FO "Forced object" ForcedMeanObject -999 columns: xGamma Forced MeanLensing xpsfE1 xpsfE2 are -999 There is no columns gpsfE1 or gpsfE2 Is this intentional? -------------------------------------------------------------------------------------------------- PSPS_SAS_39_FW "Forced object" Batches are by skycell/filter and thus by fullForceRun Extensions: ForcedWarpMeta -------------- skycellID has format 1405.07. For consistency with the rest of the schema it should probably be 2 columns projectionCell and skycellID -999 columns photoCalID analVer astroScat photoScat nAstro nPoRef 0 valued columns magSat completMag ForcedWarpMeasurement --------------------- -999 columns psfLikelihood ForcedWarpExtended --------------------- No comments ForcedWarpLensing --------------------- No comments ForcedWarpToImage ----------------- what is forcedWarpID ? It doesn't seem to be an ff_id imageID is a pointer to the input's ImageMetas (IPP's chipProcessedImfile) My comments above on the diff's tables may apply although here I did not see anything that specifies the warp. Here psModelID is a string PS_MODEL_PS1_V1