This is just a quick summary of MD PV2 into PSPS

• all MDs have been created in dvo, calibrated, and loaded onto the datastore.
• as of 9-26-2014, 2,3,4,5,6 are ready to be queried
• as of Oct 2014, all are ready to be queried.

Plots of coverage

I did a selection of random 100,000 ras and decs from detectionFull, stackDetectionFull, and Object for each field. The results are plotted below, you can click on the image to make it bigger.

the queries I used for each MD (on production cluster):

select top 100000 ra, dec from detectionFull order by newid() 
select top 100000 ra, dec from stackDetectionFull order by newid() 
select top 100000 ra, dec from Object order by newid() 

The results (see images below):

MD01 DetectionFull, MD02 DetectionFull have stripe-like problems MD07 StackDetectionFull has a stripe-like problems

md field	detectionFull	stackDetectionFull	object
md01
md02
md03
md04
md05
md06
md07
md08
md09
md10

Plots with batchid or mag

There is a problem where a number of object batches failed to be made. The black/blue colors show the missing holes for the Object Tables

select top 100000 ra, dec, objid from object order by newid()

MD field	plot of batch id	comments
MD01
MD02		no holes
MD03		no holes
MD04		no holes
MD05
MD06
MD07
MD08
MD09
MD10

Comparisions of ndetections

Another way to check the integrity is to do comparisons of the number of detections for each batch. in the ipptopsps.batch table, there is a column (ndetections) that is the number of detections published (per batch). The following queries make counts of the number of detections for each of stack/detection and grouped by batchid:

select batchid, count(objid) INTO mydb.[md06detcounts2] from Detection join ImageMeta on Detection.imageID = ImageMeta.imageID join FrameMeta on FrameMeta.frameID = imageMeta.frameID group by batchid

select batchid, count(*) INTO mydb.[md01stackdetcounts] from StackDetectionFull join StackMeta on StackDetectionFull.StackMetaID = StackMeta.StackMetaID group by batchid

I wrote a perl script to compare the numbers between these 2 databases. I find the following

MD field	Detections	Stacks	Objects
MD01	1565 out of 3797 batches are missing detections	all good	not done yet
MD02	all good	all good	not done yet
MD03	all good	all good	not done yet
MD04	all good	all good	not done yet
MD05	not done yet	all good	not done yet
MD06	not done yet	all good	not done yet
MD07	not done yet	164 out of 321 batches have extra detections	not done yet
MD08	not done yet	all good	not done yet
MD09	not done yet	all good	not done yet
MD10	not done yet	all good	not done yet

So far, MD02/MD03/MD04 are consistent (for the Stack/Detection tables)

MD01 has problems with Detections

MD07 has problems with Stack Detections