DR2 Wiki

DVO/cmf/smf level

IppToPsps Notes

Batch Progress

* last updated 9/27/2017 *

batch type	dvo/cmf/smf status	schema status	ipptopsps ready?	batches expected	batches made	comments
IN
FW	duplicates in poles, needs gene	done	yes	997k (need exact number)	85% (need exact number)
P2	done	done	yes	374446	374272	needs cleanup - had missing smf files
OB	need new dvo cal	changes	no (changes)
ST	need new dvo cal
FO	??
FG	need to build master dvodb
DF	not for dr2
DO	not for dr2

PSPS database checks

specific for Conrad

The min/max objid are incorrect for a majority of the FW batches. This is because the FW batches are a pile of forced warps at a given skycell, and the min/max objid are set from the last warp skycell that was placed in the batch. depending on the coverage of the warp skycell (due to various reasonable conditions), chunks can be missing (example, if it's the edge of an exposure, there will be a chunk missing). The missing chunks will cause the min/max objid to not be the proper range, preventing PSPS loading to see that the batch crosses slice boundaries. Heather's fix is to read the log file from batch creation, get the proper min/max fw batch id range, and feed back to Conrad. Current FW objid updates are

fixes are for roughly up to batch id 3025592

specific for Conrad/Bernie

• batch row count checks for stacks

  select (uniquePspsSTid/1000000000) , count(*) from StackObjectThin group by (uniquePspsSTid/1000000000)
  
  select (uniquePspsSTid/1000000000) , count(*) from StackObjectThinAttributes group by (uniquePspsSTid/1000000000)
  
  select (uniquePspsSTid/1000000000) , count(*) from StackApFlx group by (uniquePspsSTid/1000000000)
  
  select (uniquePspsSTid/1000000000) , count(*) from StackApFlxExGalUnc group by (uniquePspsSTid/1000000000)
  
  select (uniquePspsSTid/1000000000) , count(*) from StackApFlxExGalCon6 group by (uniquePspsSTid/1000000000)
  
  select (uniquePspsSTid/1000000000) , count(*) from StackApFlxExGalCon8 group by (uniquePspsSTid/1000000000)
  

• batch row count checks for forced warps

• ForcedWarpMeta -- need to think about this, it's a small table, should be easy to verify.
• ForcedWarpToImage -- need to to think about this, it's a small table, should be easy to verify.

These queries will effectively make a list of 'batchid', and rows in each batch, for each of these tables. HAF needs to make a master list of what we expect. It's easy to join in topcat, and correlate batchid with ra/dec area of sky, it will be possible to make plots to verify coverage.

select (uniquePspsFWid/1000000000) , count(*) from ForcedWarpMeasurement group by (uniquePspsFWid/1000000000)

select (uniquePspsFWid/1000000000) , count(*) from ForcedWarpMasked group by (uniquePspsFWid/1000000000)

select (uniquePspsFWid/1000000000) , count(*) from ForcedWarpExtended group by (uniquePspsFWid/1000000000)

select (uniquePspsFWid/1000000000) , count(*) from ForcedWarpLensing group by (uniquePspsFWid/1000000000)