This is ipp's wiki of 3pi pv2 ob in psps

First step is to do some sanity check queries to determine what is good / bad, second is to flag things to fix for PV3.

Detection Counts

Total counts:

• select count_big(*) from ObjectThin

• 4,460,992,385

I then did counts to find out how many have x detections or x stackdetections (up to 20 in both). For easy comparisons, I show both tables side by side (they are not related):

stacks	stacks		single exp	single exp
nstackdetections	count		ndetections	count	comment
0	3,271,281,682		0	462,983,177	3.2 billion things have no stack detections. .46 billion have 0 detections (aka, stack only)
1	420,775,092		1	3,158,911,897	3.1 billion things have only 1 detection, and .42 billion have only 1 stack
2	156,692,049		2	254,952,659	n detections decreases (not surprising)
3	110,625,647		3	52,419,438
4	139,184,223		4	30,750,181
5	254,028,340		5	21,236,521	not surprising, a peak at 5 nstack detections (1 per filter) , and a peak at 10, and a peak at 15, and t 20...
6	19,227,731		6	17,913,219
7	3,859,474		7	15,038,140
8	23,245,260		8	13,648,608
9	3,314,548		9	12,232,408
10	52,135,360		10	11,502,291
11	343,750		11	10,612,853
12	1,179,799		12	10,129,705
13	165,876		13	9,571,724
14	158,566		14	9,257,316
15	1,310,614		15	8,869,369
16	886,362		16	8,656,862
17	85,019		17	8,368,348
18	82,064		18	8,235,649
19	102,699		19	8,019,806
20	2,259,605		20	7,964,360

2d detection/stackdetection counts

• Another sanity check, do we have any with ndetections = 0 and nstackdetections = 0 ?
  • yes we do: this happened on a full batch, 12884 - all detections were basically null, except for the ramean/decmean. This was at ra = 0 dec = -33.
  • Is it PSPS or IPPTOPSPS's fault? I downloaded 12884's batch and investigated, it looks like we shipped them 0's.
  • Is it IPPTOPSPS or DVO's fault? Where are the 0's? This is stage_id 121470, which translates to region s3000/6992.13. This is located here: /data/stsci00.0/eugene/lap.pv2.20141215/catdir.01/s3000/6992.13* . I opened the files in topcat, and there are 0's, so ipptopsps did the correct thing, but the dvo was incorrect.

Here is a plot of counts of nstackdetection vs ndetection:

You can see the hole at 0,0 -- that should be 0 but instead is something like a million or less entries have that.

• can I categorize this better, can I find where there are more of these funky batches?
  • yes : can I do a query of select batchid, count_big(*) from ObjectThin where nstackdetections = 0 and ndetections = 0 group by batchid --> into query called 'funkybatches'