This is the current status of MDs on PSPS on 12-02-2014

Plots of coverage

I did a selection of random 100,000 ras and decs from detectionFull, stackDetectionFull, and Object for each field. The results are plotted below, you can click on the image to make it bigger.

the queries I used for each MD (on production cluster):

select top 100000 ra, dec from detectionFull order by newid() 
select top 100000 ra, dec, primaryF from stackDetectionFull order by newid() 
select top 100000 ra, dec, batchid from Object order by newid() 
select batchid, count(*)  from StackDetectionFull join StackMeta on StackDetectionFull.StackMetaID = StackMeta.StackMetaID group by batchid 
select batchid, count(objid)  from Detection join ImageMeta on Detection.imageID = ImageMeta.imageID join FrameMeta on FrameMeta.frameID = imageMeta.frameID group by batchid
select batchId, count(*) from Object group by batchID;

The results (see images below):

md field	det	stack	obj
md01	missing	good	good
md02	good	good	good
md03	good	good	good
md04	good	good	good
md05	good	good	missing
md06	missing	good	good
md07	good	good	good
md08	good	good	good
md09	missing	good	good
md10	good	good	good

The images (for stacks):

md field	stack (random 100000)	stack (primaryF = 0)	stack (primaryF = 1)
md01
md02
md03
md04
md05
md06
md07
md08
md09
md10

Stacks look good.

The images (for detections and objects):

md field	detection (random 100000)	object (good)	object (bad)
md01
md02
md03
md04
md05
md06
md07
md08
md09
md10

Detections look good. Objects look good (except for MD05).

I'm doing counts now to verify.

These are missing detection batches :

datastore       batchID    expected  actual
PSPS_MD06_PV2   1452046 is missing (not loaded) on psps
PSPS_MD06_PV2   1452071 is missing (not loaded) on psps
PSPS_MD06_PV2   1452047 is missing (not loaded) on psps
PSPS_MD06_PV2   1452077 is missing (not loaded) on psps
PSPS_MD06_PV2   1452049 is missing (not loaded) on psps
PSPS_MD06_PV2   1452072 is missing (not loaded) on psps
PSPS_MD06_PV2   1452078 is missing (not loaded) on psps
PSPS_MD06_PV2   1452076 is missing (not loaded) on psps
PSPS_MD06_PV2   1452050 is missing (not loaded) on psps
PSPS_MD06_PV2   1452079 is missing (not loaded) on psps
PSPS_MD06_PV2   1452045 is missing (not loaded) on psps
PSPS_MD06_PV2   1452042 is missing (not loaded) on psps
PSPS_MD06_PV2   1452075 is missing (not loaded) on psps
PSPS_MD06_PV2   1452048 is missing (not loaded) on psps
PSPS_MD06_PV2   1452044 is missing (not loaded) on psps
PSPS_MD06_PV2   1452074 is missing (not loaded) on psps
PSPS_MD01_PV2   1482881 is missing (not loaded) on psps
PSPS_MD09_PV2   1491049 is missing (not loaded) on psps
PSPS_MD06_PV2   1452080 is missing (not loaded) on psps
PSPS_MD06_PV2   1452043 is missing (not loaded) on psps
PSPS_MD06_PV2   1452073 is missing (not loaded) on psps

These are short OB batches (missing objects):

datastore       batchID    expected  actual percent missing
PSPS_MD05_PV2   1456104    31089     29008  0.0669368586960018
PSPS_MD05_PV2   1456135    795015     732077  0.0791658019031087
PSPS_MD05_PV2   1456341    1911208     1824427  0.0454063607938016
PSPS_MD05_PV2   1456462    1947224     1856578  0.0465513982982954
PSPS_MD05_PV2   1456711    1957854     1869703  0.0450242970109109
PSPS_MD05_PV2   1457094    302     286  0.0529801324503311
PSPS_MD05_PV2   1457121    1578878     1493558  0.0540383740859015