Serge populated a database consisting of all the columns in the false detection fits files that were sent to us.

I made some cuts and found some things:

There are 397521 false detections. Of these, there are only 45 different 'flags' that are used:

mysql> select count(*), flags, flags2 from DetectionFeatures group by flags; +----------+-----------+--------+ | count(*) | flags | flags2 | +----------+-----------+--------+ | 128 | 2107393 | 128 | | 791 | 35653633 | 128 | | 1 | 35653645 | 160 | | 1 | 35653889 | 224 | | 454 | 35661825 | 128 | | 24 | 102762501 | 192 | | 108851 | 169869313 | 208 | | 191 | 169869325 | 208 | | 11 | 169869333 | 192 | | 1 | 169869389 | 128 | | 26004 | 169869441 | 192 | | 51 | 169869453 | 192 | | 61 | 169869569 | 192 | | 53 | 169869577 | 192 | | 1 | 169869641 | 128 | | 69 | 169869697 | 192 | | 23 | 169869705 | 192 | | 224897 | 169885697 | 240 | | 315 | 169885701 | 224 | | 1 | 169885761 | 192 | | 444 | 169885953 | 192 | | 56 | 169885961 | 192 | | 2 | 169885969 | 192 | | 8 | 170917889 | 192 | | 5 | 170917901 | 240 | | 8 | 170918017 | 208 | | 1 | 170918029 | 224 | | 6 | 203423749 | 128 | | 4 | 203423877 | 128 | | 2865 | 236978181 | 128 | | 315 | 236978245 | 128 | | 119 | 236978309 | 128 | | 3780 | 236994567 | 192 | | 294 | 236994597 | 128 | | 4 | 236994631 | 192 | | 26 | 238026757 | 144 | | 19 | 238026885 | 224 | | 2 | 238043143 | 240 | | 1 | 241172485 | 192 | | 618 | 268437504 | 128 | | 3 | 268437760 | 128 | | 1 | 268437768 | 128 | | 21738 | 402685952 | 128 | | 5273 | 402686080 | 128 | | 1 | 403734656 | 128 | +----------+-----------+--------+ 45 rows in set (1.40 sec)

Most lump into only a handful of flags.

I did investigations of flag2 first, because there are fewer of those. I did cuts to remove 64,32,16,8, since those all had to do with mask, cosmicray, burntool, starspikes (ON_SPIKE, ON_STARCORE, ON_BURNTOOL, ON_CONVPOOR), which are not necessary objects we want to count.

I then cut on DEFECT, SATURATED, CR_LIMIT, EXT_LIMIT, and SIZE_SKIPPED on flag. These seemed like reasonable candidates for flags that maybe should be used to filter out bad detections.

I also did cuts to remove NULL mags (is there any mag which should ALWAYS be calculated? I can cut on that instead of the mag I cut)

Here's what we have left: mysql> select count(*), flags, flags2 from DetectionFeatures where (!(flags2 & 64) and !(flags2 & 32) and !(flags2 & 16) and !(flags2&8) and !(flags & 2048) and !(flags & 4096) and !(flags&8192) and !(flags&16384) and !(flags&268435456)) and !(cal_psf_mag is null) group by flags,flags2; +----------+-----------+--------+ | count(*) | flags | flags2 | +----------+-----------+--------+ | 55035 | 169869313 | 128 | | 12 | 169869325 | 128 | | 1 | 169869389 | 128 | | 191 | 169869441 | 128 | | 2 | 169869569 | 128 | | 8 | 169869577 | 128 | | 1 | 169869641 | 128 | | 3 | 169869697 | 128 | | 6 | 203423749 | 128 | | 4 | 203423877 | 128 | | 959 | 236978181 | 128 | | 252 | 236978245 | 128 | | 75 | 236978309 | 128 | | 8 | 238026757 | 128 | | 1 | 238026885 | 128 | +----------+-----------+--------+ 15 rows in set (0.24 sec)

for a total of 56558 false detections.

I investigated flag 169869313, which contains most of the false detections. Unfortunately, the flag is not interesting (it did a fit, it calculated a mag, etc), there are no further things to cut out. So, I created region files for the remaining false detections (see /data/ipp009.0/heather/heather), and looked at a few of the ones that had the largest number of detections. I see nothing that stands out:

The numbers represent the flag setting, and the circles are the remaining false detections.