Updated plots with unconvolved processing (2012-11-27)

Density plots

The original diff_sn diff_r plots were difficult to read due to the overlays. I've split these plots up into direct density plots, which was what the original plots were intended to show. As before, these plot diff_r values as a function of the ratio of the diff_sn to the detection S/N (abbreviated DSNR for this sections comments). The first table shows the R and S/N for the Positive measurements. The second shows the same figures for the Minus measurements.

These plots show the following set of features:

  1. Real asteroids form a "ridge" of approximately constant DSNR for a small range of R values in the Positive measurements. This ridge overlays a population of "stellar like" objects that have a detection in the warp image. In the Minus measurements, this ridge maintains the DSNR value but shifts to a larger R value, indicating the the source on the inverse image is further away.
  2. In the unconvolved difference, the ridge in DSNR of likely asteroids seems to separate from the group containing largely "stellar like" objects, with the asteroids having DSNR ~ 1.0, and the stars DSNR > 3.
  3. The MOPS cut in the Positive measurements does work reasonably for the convolved difference, but is overly permissive in the unconvolved case. This is not the case for the (warp warp) Minus measurements, as the true asteroids tend to fall at large R values with the same lower DSNR.
  4. The unconvolved Warp-Stack difference does not provide much discrimination between stars and asteroids in DSNR/R space. This is especially true in the Minus measurements, where true asteroids are associated largely at random DSNR values.
  5. The unconvolved WW diffs go deeper than the convolved diffs, and have a corresponding higher signal to noise at a given instrumental magnitude.

Warp-warp diffs

Feature (P) Original Updated Unconvolved
All detections
Stars
Asteroids
MOPS OK
IPP OK
Feature (M) Original Updated Unconvolved
All detections
Stars
Asteroids
MOPS OK
IPP OK

Warp-warp diffs (inverse cmf)

Feature (P) Original Updated Unconvolved
All detections
Stars
Asteroids
MOPS OK
IPP OK
Feature (M) Original Updated Unconvolved
All detections
Stars
Asteroids
MOPS OK
IPP OK

Warp-stack diffs

Feature (P) Original Updated Unconvolved
All detections
Stars
Asteroids
MOPS OK
IPP OK
Feature (M) Original Updated Unconvolved
All detections
Stars
Asteroids
MOPS OK
IPP OK

Updated Depth plots

WW
WWI
WS

Updated S/N plots

WW
WWI
WS

"Complete" comparison study (2012-11-09)

I've reprocessed the OSS data from MJD 56164 (2012-08-25) using the new convolution recipe changes, and have compared these results against those obtained in the previous analysis that has already been delivered to MOPS. Because of this previous analysis, I have a list of real asteroid detections in these data, and have used those to guide the discussion of false positive rejection.

Changes in depth

With the correct convolution direction being selected, we expect that there should be an increase in the detection limit. The following figures show the cumulative distribution functions for detections the warp-warp, inverse warp-warp, and warp-stack cmf files. For the warp-warp difference images, the median depth is approximately 0.25 magnitudes deeper in the updated processing than in the original processing.

The warp-stack median depth seems worse in this reduction. The depth in the original processing was clearly incorrect, with the depth largely representative of an erroneously small variance value. The depth in the updated reduction does not appear to be sufficiently deep, as it is brighter than that found for the warp-warp differences.

WW
WWI
WS

Changes in signal-to-noise

The improvement in the convolution direction increased the signal-to-noise of detections in the warp-warp diffs. I do not understand the bifurcation in the updated WW inverse data. Objects that match stars also show this split. As suggested by the change in depth above, the signal-to-noise of the warp-stack diffs has decreased, again to below that of the warp-warp data. Further investigation of the photometry process is needed to determine if this lower signal to noise is in fact correct.

WW
WWI
WS

Accuracy of SINGLE1 convolution

Mode = 1 corresponds to the SINGLE1 convolution, mode = 2 to SINGLE2, and mode = 5 to DUAL. This figure shows that the original analysis was only doing the warp-warp convolution correctly 41% of the time.

diff_sn/r p/m

This series of plots is somewhat complicated due to the number of values plotted. The goal is to represent the regions in diff_rp/diff_sn_p space occupied by different objects. The red points show all detections in the diff CMF. The green points are objects that have a "star" match, based on a comparison of diff and warp CMF files. Blue points show the region currently accepted by MOPS (if (r < 9) then diff_sn_p > 5 * object_SN). The purple points have flags set that exclude bad and poor measurements, as well as those that have low PSF_QF values. Overplotted in yellow are diff detections that match the locations of known asteroids, as supplied by Larry.

The first thing to note is that the MOPS cut does a reasonable job of isolating the large "tail" of asteroid detections. In this projection, the majority of asteroids have a positive source close (often less than 1 pixel, with some up to ~3 pixels away), and the S/N of that source is less than 10x the S/N of the difference detection. Therefore, relaxing the MOPS cut to allow these lower significance diffs to be accepted seems reasonable. This tail also has a large number of "star" associations. This is reasonable, as the positive object is likely found in the warp image as well. However, the bulk of the star population falls at larger S/N ratios (as the positive source is far more significant than the source after subtraction) and at larger radii (as the residual is not necessarily going to be symmetric for a star, resulting in a shift between the center between the positive source and the subtracted source).

The bifurcation of the S/N values shown above is visible in this plot as well, suggesting that the split is due to some issue in the difference process.

Positive Original Updated
WW
WWI
WS

These plots are the same, but showing the relations between diff_rm/diff_sn_m for the different diff images (the plot labels are incorrect; these are the M versions). These measurements are somewhat less obvious than for the P versions. Although the signal to noise ratios (diff_sn_p / mops_sn and diff_sn_m / mops_sn) are similar for the warp-warp diff detections of asteroids, the diff_rm is not as small as for diff_rp. This is reasonable, as the minus source is assumed to have moved between exposures. The stellar sources occupy the same SNR/R space as in the P version, as the same residual issues create the matches. The MOPS cut on diff_rm may exclude objects with large shifts, and should probably be relaxed. If a source has a small diff_rp, and diff_sn_p / mops_sn ~= diff_sn_m / mops_sn, then requiring that diff_rm be less than 9 can exclude faster moving objects.

Warp-stack diffs are far less likely to have a correct minus source associated, as any objects moving through the warp field is unlikely to have been there during the frames used for the stack. As a result, the S/N ratio for known asteroid objects has a large amount of scatter, indicating that a random minus source is being associated with these objects.

Minus Original Updated
WW
WWI
WS

Convolution issues (2012-11-02)

Examining the difference images for a WS diff led to the conclusion that the convolution direction was being chosen incorrectly. This appears to be due to the default difference recipe convolving the input file (the warp). The following image shows a comparison of the warp, stack, original difference, and a manual difference that convolves the stack instead.

The manual difference has fewer and smaller residual features, as would be expected from an improved convolution.

Signal to Noise changes

As the original difference was convolving the warp, the output diff variance was smaller than it should have been (as the convolution reduces the variance). With the convolution being done to the stack instead, the output variance is now dominated by the variance of the input warp (as would be expected). However, this results in the S/N of detections in the manual difference being reduced relative to the values in the original difference.

DIFF_R_P, DIFF_SN_P changes

One way of mitigating the influence of poor subtraction in the difference images is to use the DIFF_R_P (radius to the nearest positive source) and DIFF_SN_P (signal to noise of this source). If a detection is very near a positive source, then it may in fact be simply a poor subtraction of a star. Plotting these values for the original and manual difference shows that the manual difference image has a very different distribution of points, with a significant reduction in the points that represent bright wings of stars being detected at increasing distance for increasingly bright sources. This again confirms the improved subtraction, and suggests that the false positive rate in the difference images can be reduced.

Attachments