Go to: ​http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/Zoology

Introduction

This page details the investigations into the high number of false detections reported by the Durham group. Durham reported around 400,000 false-detections across all exposures in the SAS footprint survey.

Procedure

We decided to populate a MySQL database consisting of all the columns in the false detection FITS files sent to us by the Durham group, enabling us to easily query the data and experiment with various cuts on flags, flux errors etc. Serge populated the database on his desktop machine.

Heather was able to overlay the locations of the false detections on top of the actual images, allowing to visualize the data during the various cuts.

Cutting using flags

I made some cuts and found some things:

There are 397521 false detections. Of these, there are only 45 different 'flags' that are used:

select count(*), flags, flags2 from FalsePositives group by flags;

Gives:

+----------+-----------+--------+
| count(*) | flags     | flags2 |
+----------+-----------+--------+
|      128 |   2107393 |    128 | 
|      791 |  35653633 |    128 | 
|        1 |  35653645 |    160 | 
|        1 |  35653889 |    224 | 
|      454 |  35661825 |    128 | 
|       24 | 102762501 |    192 | 
|   108851 | 169869313 |    208 | 
|      191 | 169869325 |    208 | 
|       11 | 169869333 |    192 | 
|        1 | 169869389 |    128 | 
|    26004 | 169869441 |    192 | 
|       51 | 169869453 |    192 | 
|       61 | 169869569 |    192 | 
|       53 | 169869577 |    192 | 
|        1 | 169869641 |    128 | 
|       69 | 169869697 |    192 | 
|       23 | 169869705 |    192 | 
|   224897 | 169885697 |    240 | 
|      315 | 169885701 |    224 | 
|        1 | 169885761 |    192 | 
|      444 | 169885953 |    192 | 
|       56 | 169885961 |    192 | 
|        2 | 169885969 |    192 | 
|        8 | 170917889 |    192 | 
|        5 | 170917901 |    240 | 
|        8 | 170918017 |    208 | 
|        1 | 170918029 |    224 | 
|        6 | 203423749 |    128 | 
|        4 | 203423877 |    128 | 
|     2865 | 236978181 |    128 | 
|      315 | 236978245 |    128 | 
|      119 | 236978309 |    128 | 
|     3780 | 236994567 |    192 | 
|      294 | 236994597 |    128 | 
|        4 | 236994631 |    192 | 
|       26 | 238026757 |    144 | 
|       19 | 238026885 |    224 | 
|        2 | 238043143 |    240 | 
|        1 | 241172485 |    192 | 
|      618 | 268437504 |    128 | 
|        3 | 268437760 |    128 | 
|        1 | 268437768 |    128 | 
|    21738 | 402685952 |    128 | 
|     5273 | 402686080 |    128 | 
|        1 | 403734656 |    128 | 
+----------+-----------+--------+
45 rows in set (1.40 sec)

Most lump into only a handful of flags.

I did investigations of flag2 first, because there are fewer of those. I did cuts to remove 64,32,16,8, since those all had to do with mask, cosmicray, burntool, starspikes (ON_SPIKE, ON_STARCORE, ON_BURNTOOL, ON_CONVPOOR), which are not necessary objects we want to count.

I then cut the following using the flag field

• DEFECT
• SATURATED
• CR_LIMIT
• EXT_LIMIT
• SIZE_SKIPPED

These seemed like reasonable candidates for flags that maybe should be used to filter out bad detections.

I also did cuts to remove NULL mags (is there any mag which should ALWAYS be calculated? I can cut on that instead of the mag I cut)

select count(*), flags, flags2 
from FalsePositives 
where (!(flags2 & 64) 
and !(flags2 & 32) 
and !(flags2 & 16) 
and !(flags2&8) 
and !(flags & 2048) 
and !(flags & 4096) 
and !(flags&8192) 
and !(flags&16384) 
and !(flags&268435456)) 
and !(cal_psf_mag is null) 
group by flags,flags2;

Here's what we have left:

+----------+-----------+--------+
| count(*) | flags     | flags2 |
+----------+-----------+--------+
|    55035 | 169869313 |    128 | 
|       12 | 169869325 |    128 | 
|        1 | 169869389 |    128 | 
|      191 | 169869441 |    128 | 
|        2 | 169869569 |    128 | 
|        8 | 169869577 |    128 | 
|        1 | 169869641 |    128 | 
|        3 | 169869697 |    128 | 
|        6 | 203423749 |    128 | 
|        4 | 203423877 |    128 | 
|      959 | 236978181 |    128 | 
|      252 | 236978245 |    128 | 
|       75 | 236978309 |    128 | 
|        8 | 238026757 |    128 | 
|        1 | 238026885 |    128 | 
+----------+-----------+--------+
15 rows in set (0.24 sec)

for a total of 56558 false detections.

I investigated flag 169869313, which contains most of the false detections. Unfortunately, the flag is not interesting (it did a fit, it calculated a mag, etc), there are no further things to cut out. So, I created region files for the remaining false detections (see /data/ipp009.0/heather/heather), and looked at a few of the ones that had the largest number of detections. I see nothing that stands out:

The numbers represent the flag setting, and the circles are the remaining false detections.

This is AFTER I did the flag cuts. Prior to that there were 8x as many detections, often landing on burntools and edges.

Eye-balling the images confirmed that this method removed the majority of 'obvious' false detections, such as those on the edges of chips or in areas of saturation from a bright star.

Cutting using error in flux

We looked at the error in the flux, specifically, FLUX_SIG compared to FLUX. By eliminating all detections with a an error greater than 5% of the total flux, we were left with just 2000 detections, a reduction (from 56,000) of about 96%.

We generated histograms of flux_sig/flux for all of the non-detections and for the non-detections after the flag cuts.

Here are the histograms using the "True Detections" (matched to Sloan). Doing the flag cuts removes 2/3 of the valid detections (why?)

Filter Dependence

Are the false positives filter dependent? A quick query of the Zoology database shows that they are:

mysql> select flags,count(*), filter from AnalysisStage010 where psf_inst_flux_sig/psf_inst_flux > 0.05 group by flags, filter;

+-----------+----------+--------+
| flags     | count(*) | filter |
+-----------+----------+--------+
| 169869313 |    30823 | g      | 
| 169869313 |     7498 | i      | 
| 169869313 |     7033 | r      | 
| 169869313 |     3949 | y      | 
| 169869313 |     4856 | z      | 
| 169869441 |       54 | g      | 
| 169869441 |       39 | i      | 
| 169869441 |       43 | r      | 
| 169869441 |        3 | y      | 
| 169869441 |       25 | z      | 
| 169869569 |        1 | i      | 
| 169869577 |        2 | y      | 
| 169869697 |        1 | i      | 
| 203423749 |        2 | i      | 
| 203423749 |        1 | r      | 
| 203423749 |        2 | y      | 
| 203423749 |        1 | z      | 
| 203423877 |        1 | y      | 
| 236978181 |        4 | g      | 
| 236978181 |        8 | i      | 
| 236978181 |        3 | r      | 
| 236978181 |        2 | y      | 
| 236978181 |        6 | z      | 
+-----------+----------+--------+

There are more false detections in g than in the other bands. These are the histograms for inst_flux_sig/inst_flux for different filters. The filter g has a different shaped histogram than the other 4 filters.

Applying same cuts to 'true' detections

Subset	Total	Total after flag cuts	Total after flux cut
g False positives		31190()	309()
r False positives		7609()	529()
i False positives		8133()	584()
z False positives		5412()	524()
y False positives		4214()	255()
Total False positives	397,521	56,558 (14%)	2201 (0.5%)
g true positives		36602()	12854()
r true positives		73317()	27224()
i true positives		93568()	35298()
z true positives		74214()	30457()
y true positives		46434()	21100()
Total True positives	917,726	324,135 (35%)	126,933 (13.8)

Answers to A-014

Counting the detection with "exclusion" flags

count(*)	1315247	917726	397521
From flags
Flag	DetectionFeatures	TruePositives	FalsePositives	MySQL Statement
DEFECT	3204	1183	2021	WHERE (flags & 0x800)
SATURATED	0	0	0	WHERE (flags & 0x1000)
MOMENTS_FAILURE	38398	11386	27012	WHERE (flags & 0x8000)
BIG_RADIUS	0	0	0	WHERE (flags & 0x80000)
AP_MAGS	806	736	70	WHERE (flags & 0x100000)
SIZE_SKIPPED	1312043	916543	395500	WHERE (flags & 0x8000000)	?'''
ON_SPIKE	39125	11491	27634	WHERE (flags & 0x10000000)
ON_GHOST	0	0	0	WHERE (flags & 0x20000000)
OFF_CHIP	0	0	0	WHERE (flags & 0x40000000)
From flags2
ON_SPIKE	699	130	569	WHERE (flags2 & 0x8)
ON_BURNTOOL	2293	812	1481	WHERE (flags2 & 0x10)

Detections where FITFAIL is set and FITTED is not

Are there any detection where FITFAIL (0x8) is set and FITTED (0x4) is not?

SELECT count(*) FROM <Table> WHERE (flags & 0x8) AND NOT (flags & 0x4);

DetectionFeatures	TruePositives	FalsePositives
1755	1621	134

Flags which should always be 0

EXTERNALPOS, CR_LIMIT, SKY_FAILURE, SKYVAR_FAILURE, MOMENTS_SN should always be 0

Flag	DetectionFeatures	TruePositives	FalsePositives	MySQL Statement
EXTERNALPOS	0	0	0	WHERE (flags & 0x200)
CR_LIMIT	750	168	582	WHERE (flags & 0x2000)
SKY_FAILURE	0	0	0	WHERE (flags & 0x10000)
SKYVAR_FAILURE	0	0	0	WHERE (flags & 0x20000)
MOMENTS_SN	0	0	0	WHERE (flags & 0x40000)

Conclusions

• By rejecting detections with flagged errors, the original total of 400,000 is reduced to 56,000 - a reduction of 86%
• By rejecting detections with an error in flux greater than 5%, the remaining total of 56,000 comes down to 2000 - a reduction of 96%
• This is a total reduction of around 99%