| | 36 | |
| | 37 | |
| | 38 | |
| | 39 | == schema changes == |
| | 40 | |
| | 41 | * MeanObject:xqfPerfect is always -999 : check in dvo (HAF/EAM), it is set from PSF_QF_PERF_MAX in cps table in ipptopsps |
| | 42 | * FrameMeta: remove p1Recipe, p2Recipe, p3Recipe (HAF) |
| | 43 | * FrameMeta: add md5sum (from smf)- make it a varchar of reasonable length (10?) (HAF) |
| | 44 | * ImageMeta:psfModelID - should be "3" for PS_MODEL_PS1_V1, have it map to FitModel (HAF/EAM) |
| | 45 | * FitModel: fill in correct models, SMF headers has PSFMODEL keyword (HAF/EAM) |
| | 46 | * ImageMeta:detrend4-8 fill with NOISEMAP, NONLIN, VIDEODARK (7,8 left blank) (HAF) |
| | 47 | * (alltables):dataRelease changed to processingVersion and to use 3 for PV3 (HAF) |
| | 48 | * (alltables):calibModNum - drop. It's in ImageMeta, might be in others. (HAF) |
| | 49 | * DiffMeta:posImageID, negImageID need more documentation (HAF/EAM) |
| | 50 | * WarpToImage: need to add this table (HAF) |
| | 51 | * DiffDetection: find -999 columns and list them (WS) |
| | 52 | * DiffDetObject:html,zoneID,cz,cy,cz,lambda,beta,l,b to be filled by PSPS (CH/TC) |
| | 53 | * DiffDetObject:description of table is vague and needs to be cleaned up (HAF/EAM) |
| | 54 | * DiffDetObject:xqfPerfect is always -999 : check in dvo (HAF/EAM), it is set from PSF_QF_PERF_MAX in cps table in ipptopsps |
| | 55 | * StackMeta:magsat - drop it, we do not calculate (in detection/diff/ff as well) (HAF) |
| | 56 | * StackMeta:completMag - detectionThreshold --> DETEFF.MAGREF (instrumental mag, needs zp / exp correction for stack and detection and diff) |
| | 57 | * StackMeta: astroScat - AST_CDX, AST_CDY calc= sqrt(a2+b2) - this is not the astrometric error - a measurement of the calibration but not definitive |
| | 58 | * StackMeta:photoScat - ZPT_ERR - describe it better: the scatter of the calibration stars |
| | 59 | * StackMeta:nAstro - NASTRO : change to nRefStars |
| | 60 | * StackMeta:nPhoRef - (delete) |
| | 61 | * StackMeta: add set of similar names (same information, from relexp/relstack) recalAstroScat, recalPhotoScat, recalNAstroStars, recalNPhotoStars |
| | 62 | * StackMeta:calibModNum - HAF to remove from schema |
| | 63 | * StackObjectThin:stackDetectRowID (WS/HAF/EAM) is it set correctly |
| | 64 | * StackObjectThin:primary not set (we understand, from dvo addstar -list bug) (HAF/EAM) |
| | 65 | * StackObjectThin:bestDetection not set (???) EAM??? |
| | 66 | * StackObjectThin:xInfoFlag2 not set, looks like bug in ipptopsps (HAF) |
| | 67 | * (alltables):(allfluxes) convert to jansky - need conversion from EAM |
| | 68 | * ForcedMeanObject:xGamma drop |
| | 69 | * ForcedMeanLensing:xpsfE1 xpsfE2 are -999 (HAF/EAM) to investigate - is it set in dvo? is it a typo in ipptopsps |
| | 70 | * ForcedWarpMeta:skycellID to be split into projectionCell and skycellID as done for other tables (HAF) |
| | 71 | * ForcedWarpMeta:photocalID (???) EAM??? |
| | 72 | * ForcedWarpMeta:analVer should be set to svn number (HAF) |
| | 73 | * ForcedWarpMeta:astroScat - drop (HAF) |
| | 74 | * ForcedWarpMeta:photoScat - same as in stack (HAF) |
| | 75 | * ForcedWarpMeta:nAstro - drop (HAF) |
| | 76 | * ForcedWarpMeta:nPhoRef - same as in stack (HAF) |
| | 77 | * ForcedWarpMeta:magSat -drop (HAF) |
| | 78 | * ForcedWarpMeta:completMag -- as done for stack/p2 |
| | 79 | * ForceWarpMeta:add 2 columns recalPhotoScat, recalPhoRef (and grab from relexp (fullforce) info for photoscat/ nphoref) |
| | 80 | |
| | 81 | === ipptopsps issues from pspswg page === |
| | 82 | |
| | 83 | * missing zp in stackobjecthin/stackobjectattributes mags/ fluxes |
| | 84 | * Nigel: primaryflag not set - Gene knows the cause of the problem. |
| | 85 | * Nigel: due to the above, forced mean photometry includes primary and secondary measurements into the averages. |
| | 86 | * Nigel: in ForcedMeanObject FluxStd is in ADU/sec but Flx and FlxErr are in Jansky. |
| | 87 | * Nigel: quick note on naming conventions. Detection and ForcedWarpMeasurement have the same names for columns. Is that OK? (HAF to rename) |
| | 88 | * Nigel: there is a population with halflightrad < 0.1. This is not a star/galaxy issue (using SDSS types, everything with rad<~2 is a star). Colours show density of points. |
| | 89 | * HAF/KCC: as listed in Schema 21 the halflight radius is not treated consistently. There is momenRH (labeld as half light radius, sometimes with DROP), in Detection, diffDetection, ForcedWarp, and Stack. This comes from momentRH in the cmf files. We also have a true halflightrad in stack, which is filled from the cmf (HALF_LIGHT_RADIUS. We need to check that this is present in all types of cmf files and use HALF_LIGHT_RADIUS where it was measured and drop momentRH everywhere. |
| | 90 | * Nigel: psfMajor/MinorFWHM are in pixels but the schema says arcsec. |
| | 91 | * HAF: ipptopsps grabs it directly from dvo (no conversions, i see min/majorFWHM is in stack, detection, diff, forcedwarp) - is it wrong for everything or only certain tables? If only certain tables, EAM needs to apply something in dvo. If it's wrong for all, HAF needs to apply correction in ipptopsps. (see other examples, in ipptopsps) |
| | 92 | * Roberto: In ForcedMeanObject all g,r,i,r,y-Gamma values are still -999. |
| | 93 | * apflux and apfluxerr still follow multiple, noise-free sequences in all filters. The values of apfluxstd are strange, why can they have the value of -999 when apfluxerr is 'normal'? This happens when nTotal>1: for example, obID=ForcedMeanObject in the gband has gnTotal=4, gApFlux=8.19722E-06, gApFluxErr=3.59973E-07 and gApFluxStd=-999. |
| | 94 | * HAF: these come directly from the cmf, no other translations done - EAM? EAM will look at |
| | 95 | * Roberto: in StackModelFitDeV the DeVCf are all -999 in all filters. |
| | 96 | * Roberto: in StackModelFitExp ExpCf are all -999. There are still 'sensible' fits (example: objID=101703338181088339, gband) where RAexp and DECexp are -999 |
| | 97 | * Roberto: in StackModelFitSer ExpCf are all -999. nuerr=0.1 always |
| | 98 | * HAF CF is not loaded for Dev/Exp/Ser in ipptopsps, need to investigte if in cmf. EAM : not measured, drop it |
| | 99 | * Roberto: in StackModelFitExtra logC and logA are -999 |
| | 100 | * HAF logC I couldn't find a column for in the CMF, logA is loaded by ipptopsps so must be null in the smf file. EAM: not measured, drop them |
| | 101 | * Roberto: in StackPetrosian many petRadiusErr are -999, although all the other errors are not (example: objID=101663337788271850, gband) |
| | 102 | * HAF PetRadErr is loaded in ipptopsps, might be null in cmf (EAM?) EAM to investigate |
| | 103 | * Roberto: in ForcedWarpMeasurement psfLikelihood is always -999 |
| | 104 | * HAF: loaded in ipptopsps: might not be set in CMF (EAM?) EAM: do we have a chi sq? calculated in ipptopsps from a chi sq (HAF to investigate) |
| | 105 | * Roberto: in Detection the apflux and apfluxerr still follow multiple, noise-free sequences, errors on kron fluxes don't |
| | 106 | * HAF: EAM investigate? |
| | 107 | * Roberto: in DiffDetObject cx, cy,cz, l, b, lambda, beta are -999, raErr and DecErr are 0 |
| | 108 | * HAF: cx/cz/cy/l/b/lambda/beta should be calculated by PSPS. |
| | 109 | * HAF raErr/decErr sounds like dvo calibration problem (bug EAM, ipptopsps tries to load) HAF: need to remove ra/dec err does not make sense for diff |
| | 110 | * Roberto: in DiffDetection psfMajorFWHM, psfMinorFWHM, momentM4C, momentM4S, momentR1, momentM3C, momentM3S are -999 |
| | 111 | * HAF these are commented out in ipptopsps: I think they may not actually be set, I need to investigate the CMF. |
| | 112 | |
| | 113 | === issues on confluence page === |
| | 114 | |
| | 115 | * lambda,beta,l,b,gQfPerfect,rQfPerfect,iQfPerfect,zQfPerfect,yQfPerfect are set to -999 in DiffDetObj |
| | 116 | * psfMajorFWHM, psfMinorFWHM, psfCore are set to -999 in DiffDetections |
| | 117 | * psfQfPerfect = 0 for all g band detections in DiffDetections, but ~0.99 for the other filters |
| | 118 | * MomentM* are set to -999 in DiffDetections. MomentM* might be ill defined since it is a diffim with negative weights. |
| | 119 | * Why is obsTime in DiffDetections slightly different than the obsTime in ForcedWarpMeasurement? It looks like obstime in ForcedWarpMeasurement is the same as in the Detections table. obsTime in DiffDetections is ~0.0006 days = 50 seconds earlier than in ForcedWarpMeasurement and Detections. |
| | 120 | * in diffMeta, there is a diffMetaID. In DiffDetection, there is a diffDetMetaID. It seems like that these refer to the same attribute, so shouldn't it be named the same? |
| | 121 | zp=25 in DiffDetections. Is this correct, i.e. are all diffims normalized to zp=25? |
| | 122 | * Do we really need moment* and kron* for difference image detections? |
| | 123 | * Let's pick a random warp. This warp has detections in the warp, forced photometry measurements, and also difference image detections for the diffim using this warp as positive image. It's not clear how to associate these detections. |
| | 124 | |
| | 125 | === additional schema changes from HAF/KCC IAU booth === |
| | 126 | * to be added later, on HAF's laptop |
| | 127 | |
| | 128 | |
| | 129 | |
| | 130 | |
