Summary

MOPS has reported some peculiar magnitudes in y-band MD data offsets and larger scatter compared to predictions of asteroid y-band magnitudes while the same comparison to 3PI photometry looks reasonable (both cases are photometry from the respective diffims). Also, have found photometry from Warp-Stack (WS) difference images for two sequential MD exposures unrealistically different by >0.5 mag.

Plot of average value for (observed - predicted) magnitudes per night for 3pi and MD detections in y-band.An offset in the MD data making objects ~0.4 mag brighter is seen.

mean y-band (O-C)

Plot of average value for (observed - predicted) magnitudes per night for 3pi and MD detections in z-band. No offset is seen.

Mean z-band (O-C)

Issue of PSF photometry scatter

The MD10 sequential exposure pair, asteroid (12891) example was used to trace through the code to identify problems. The camera SMF exposure photometry was taken as the "true" target photometry to achieve from the diffim photometry. Offset of exposure photometry to some predicted/expected magnitude is a different problem altogether.

Key problem -- coordinate list for determining PSF was a matched+merged list of both the input and reference image rather than just a matched list (poor PSF resulting in 0.3+ irregular mag). When the list is sorted by inst_flux and the top hardcoded 500 sources are used, ones that are not in the input warp from masking etc reduce the available "good" sources. Normally that is only a small fraction, but when a deeper refstack is used the missing sources have a much higher inst_flux and begin to dominate the set for determining the PSF making for a poor solution. Newer MD refstacks have been much deeper, some fields before (and typically in y-band) had only order 10-20 inputs, and now have 60-100.

  • however, there was a specific routine (pmSourceMatchMerge) added for ppSub to make the merged list. Really the specific input list should be used and not a mix (match/merge) of both, even for the stamp selection, but the input source lists are combined into a master detection list.

Various other minor(?) issues identified:

  • background binning for photometry on diffims has not been the same as other photometry (~0.1-0.2 offset mag) -- using 128 for now, may need additional/proper tuning
  • psphot trace level >5 modifies the photometry output -- ticket 1510 -- to be fixed
  • refConv outputs aren't needed since magic not used, turn off? not an option in gpc1/psphot.config etc -- hard coded in diff_skycell.pl and would need option to tie to if general use on/off or with magic..
    • normally only made in original nightly processing, then cleaned up and not remade on update
    • This output highlighted another possible problem in the COMP_WT output where the range is insufficient and cores of moderately bright sources are clipped to NAN -- unknown extent or if critial and how to fix, ticket 1514
  • ppSub missing -threads option and many others in help -- see psphot help for the more generic options.. should be unified..
  • sometimes a check is made for a single input list of sources (input or ref) and allowed with the one provided being the master source list, other places if either doesn't exist then ppSub exits so that sources for both must be provided -- unclear if one source list is acceptable for all parts, still to be addressed
  • REVERSE is wanted as a command line argument and not defined for choosing if the input or the reference is used to determine the PSF for photometry on the diffim -- this ever necessary?
  • is convolved refstack really necessary here or would the unconvolved stack be better (basically the refstack is just sky for moving objects but for other transient science may want unconvolved) as the unconvolved stack is typically of better FWHM and requires less deconvolution to the warp seeing -- needs to be investigated further
  • psphot run on diffim compared to ppSub photometry does not necessarily agree -- unclear still if config/setup or actual problem/bug

For results using only a matched list and a background binning, see MD_WSdiffim_photcheck_test20121004#Largesamplediffimphotometryfory-band

Issue of photometry offset

Exposure (SMF) photometry versus diffim photometry: unless the target is contaminated with background (self or host), photometry at the exposure level will be the better choice for photometry. However, will look at the MD09 example since it covers past/present, most warps are still available (MD09 is the current test field) and does not appear intrinsically variable (MD_WSdiffim_photcheck_test20121004#Largesamplediffimphotometryfory-band).

  • appears to be an offset of ~0.08 mag in the PSF photometry on the diffims (PSF mag brighter) and from the ppSub photometry
  • photometric error on the diffims is ~30% larger on the WARP-STACK(convolved -- default for MOPS) versus the WARP-STACK(unconvolved -- default for static transients)

Predicted/expected photometry versus exposure photometry: beyond the scope of diffim photometry investigation.

Example: MD10 sequential exposure pair, asteroid (12891) (Peter Vereš)

o6157g0218o
o6157g0219o

ra: 353.142622 
ra: 353.142215

dec:-0.086658
dec: -0.086867

Chip/camera CAL_PSF_MAG photometry from the SMF catalogs below agree to what expected? yes, but can't be more exact because asteroid (12891) previously unstudied.

  • o6157g0218o: 17.06
  • o6157g0219o: 17.048

camera SMFs (from tag ipp-20120802):

IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2 rsep(")

2384872506.gpc1:MD10.nt:2012:08:18:o6157g0218o.512581:o6157g0218o.512581.cm.524588.smf  XY41
 00300   17.064  0.116  -12.201  0.011  -12.243   6.285   0.999  0x06200005 0x00000080   0.03 

2384878611.gpc1:MD10.nt:2012:08:18:o6157g0219o.512582:o6157g0219o.512582.cm.524591.smf  XY31
 00353   17.048  0.120  -12.215  0.011  -12.221   6.292   0.999  0x06200005 0x00000080   0.02 

WSdiff CMFs (from tag ipp-20120802):

IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  DIFF_NRATIO_BAD  rsep (")
---- o6157g0218o
2385003229.gpc1:MD10.nightlyscience:2012:08:18:MD10.V3:skycell.052:MD10.V3.skycell.052.WS.dif.272923.cmf
 00180   17.159  0.000  -12.106  0.055  -12.059   6.915   0.999  0x02200001 0x00000881   0.59   0.00
---- o6157g0219o
2385006586.gpc1:MD10.nightlyscience:2012:08:18:MD10.V3:skycell.052:MD10.V3.skycell.052.WS.dif.272924.cmf
 00143   16.543  0.000  -12.720  0.051  -12.072  10.512   0.999  0x02200001 0x00000081   0.61   0.00

  • AP_MAG close between the two exposures (~0.013, ok) but largely offset from SMF ~0.2 (fainter, not ok). Also seems like the target conv may be poor and a large difference in psf_major (ref is conv so target source not in ref since mover and unclear what is driving the PSF poorly)

manually running psphot on input warps

  • appears to be an issue with psphot and debug options triggered @level 6, re-run to use normal ops build w/o trace or with values <6
  • original warp CMF, transformed photometry in warps from SMF -- should be same as SMF and it is
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  AP_MAG_RAW  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  DIFF_NRATIO_BAD  rsep(")
    ---- o6157g0218o
     00486   17.064  0.000  -12.201  0.011  -12.243     nan     nan   0.999  0x06200005 0x00000000   0.03 
    ---- o6157g0218o
     00473   17.048  0.000  -12.215  0.011  -12.221     nan     nan   0.999  0x06200005 0x00000000   0.02
    
  • basic psphot run with current ipp-20121026 tag, random seed -- similar except exposure 0219 is different by ~0.035 mag (0218 offset by ~0.014). seems larger than should be..
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  AP_MAG_RAW  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2    rsep
    ---- o6157g0218o
     00379   17.050  0.000  -12.215  0.011  -12.233 -12.237   6.610   0.999  0x06200005 0x00000080   0.03 
    ---- o6157g0219o
     00380   17.081  0.000  -12.181  0.011  -12.186 -12.191   6.376   0.999  0x06200005 0x00000080   0.02 
    
    
  • psphot run with old ipp-20120802 tag, random seed (-seed 8568951837626357297) -- similar to previous tag with exposure 0218 w/in ~0.01 but now exposure 0219 has even smaller offset
    ---- o6157g0218o
     00379   17.057  0.000  -12.208  0.011  -12.245 -12.250   6.434   0.999  0x06200005 0x00000080   0.03 
    ---- o6157g0219o
     00381   17.080  0.000  -12.183  0.011  -12.192 -12.196   6.445   0.999  0x06200005 0x00000080   0.02 
    
    
  • psphot run with current ipp-20121026 tag using the same seed as the run with ipp-20120802 tag -- with same seed, mags mostly match between tags (hope so).
    ---- o6157g0218o
     00379   17.057  0.000  -12.208  0.011  -12.245 -12.250   6.434   0.999  0x06200005 0x00000080   0.03 
    ---- o6157g0219o
     00381   17.079  0.000  -12.184  0.011  -12.192 -12.196   6.445   0.999  0x06200005 0x00000080   0.02
    
  • psphot run with last year ipp-20110622 tag using the same seed as the prior psphot run with ipp-20120802 tag: again similar AP_MAG but PSF_INST_MAG offset by ~0.01-0.02, flags for exposure 0218 also different, PSF_MAJOR also different.
    ---- o6157g0218o
     00393   17.070  0.000  -12.195  0.011  -12.246 -12.250   4.303   0.999  0x06200045 0x00000080   0.03 
    ---- o6157g0219o
     00406   17.057  0.000  -12.206  0.011  -12.192 -12.196   4.431   0.999  0x06200005 0x00000080   0.02 
    

manually re-running WSdiffims

  • psphot trace level >5 until bug not a problem in ppSub
  • conv direction correct? (default is conv refstack) Yes according the inputs from ppSub logs (conv refstack only, unconv better by ~1 pixel), and Kron R measure appears similar on resampled warp to SMF PSF size.
    ---- o6157g0218o
          Input FWHM: 9.713536
          Reference FWHM: 4.809747
            Kron Radii vs FWHMs 1: fwhm: 9.713536, kron 7.794752
            Kron Radii vs FWHMs 2: fwhm: 4.809747, kron 4.897604
    ---- o6157g0219o
          Input FWHM: 9.825516
          Reference FWHM: 4.809747
            Kron Radii vs FWHMs 1: fwhm: 9.825516, kron 7.951603
            Kron Radii vs FWHMs 2: fwhm: 4.809747, kron 4.905538
    
    
  • with similar ipp-20120802 tag and new seed (conv + unconv refstack): conv/unconv are close and the difference between exposures is smaller ~0.4 vs 0.6 but maybe not in the proper direction, the PSF_ offset from SMF is still ~0.5 (brighter). AP_MAG similar to orig diffim and offset from SMF ~0.2 mag (fainter).
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  DIFF_NRATIO_BAD  rsep
    --- o6157g0218o (conv and unconv)
     00146   16.892  0.000  -12.373  0.055  -12.050   8.657   0.999  0x02200001 0x00000881   0.59   0.02 
     00197   16.774  0.000  -12.491  0.032  -12.050   8.802   0.999  0x02200001 0x00000081   0.59   0.01 
    --- o6157g0219o (conv and unconv)
     00139   16.493  0.000  -12.770  0.052  -12.070  11.301   0.999  0x02200001 0x00000081   0.62   0.01 
     00176   16.465  0.000  -12.798  0.025  -12.020  11.069   0.999  0x02200001 0x00000081   0.60   0.01 
    
    
  • with more current ipp-20121026 tag, new and same seed as original WSdiff (default conv refstack always unless noted):
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  AP_MAG_RAW  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  DIFF_NRATIO_BAD  rsep
    --- o6157g0218o (new, new  and original seed)
     00100   16.698  0.000  -12.567  0.053  -12.058 -12.099   9.068   0.999  0x02200001 0x00000081   0.59   0.01
     00157   16.991  0.000  -12.274  0.054  -12.058 -12.096   7.047   0.999  0x02200001 0x00000881   0.59   0.01 
    
     00180   17.159  0.000  -12.106  0.055  -12.059 -12.095   6.915   0.999  0x02200001 0x00000881   0.59   0.00 
    --- o6157g0219o (new, new and original seed)
     00143   16.518  0.000  -12.745  0.051  -12.056 -12.093  10.469   0.999  0x02200001 0x00000081   0.61   0.00
     00146   16.515  0.000  -12.748  0.050  -12.070 -12.106  10.707   0.999  0x02200001 0x00000081   0.61   0.00 
    
     00143   16.543  0.000  -12.720  0.051  -12.072 -12.106  10.512   0.999  0x02200001 0x00000081   0.61   0.00 
    
  • with older ipp-20110622 tag and new seed that was active when photometry "looked better" last year (see example mag plot for TNO145452 below): clearly something changed since with the PSF measure, the difference is smaller ~0.1 mag but the offset from the SMF is larger 0.3 (fainter).
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  DIFF_NRATIO_BAD  rsep
    --- o6157g0218o --
     00170   16.837  0.000  -12.428  0.053     -inf   5.545   0.999  0x02200001 0x00000081   0.59   0.11 
    --- o6157g0219o -- 
     00164   16.727  0.000  -12.536  0.051     -inf   5.761   0.999  0x02200001 0x00000081   0.61   0.09 
    
    
  • with more current ipp-20121026 tag and changing the PPSUB:PSPHOT PSF_MODEL from PS_MODEL_PS1_V1 to PS_MODEL_GAUSS -- model seems to also change the AP_MAG, and closer to what is should be (~0.1 mag offset now)
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  AP_MAG_RAW  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  DIFF_NRATIO_BAD  rsep
    --- o6157g0218o (new seed, new seed, same seed as original WSdiff)
     00081   17.142  0.000  -12.123  0.055  -12.100 -12.100  10.658   1.000  0x02200001 0x00000881   0.60   0.00 
     00157   16.991  0.000  -12.274  0.054  -12.058 -12.096   7.047   0.999  0x02200001 0x00000881   0.59   0.01 
    
     00081   17.185  0.000  -12.081  0.053  -12.095 -12.095   9.102   1.000  0x02200001 0x00000881   0.59   0.00 
    
    --- o6157g0219o (new seed, new seed, same seed as above)
     00085   17.188  0.000  -12.075  0.050  -12.105 -12.107  11.527   1.000  0x02200001 0x00000881   0.62   0.02 
     00067   17.054  0.000  -12.209  0.050  -12.107 -12.109   9.943   1.000  0x02200001 0x00000881   0.62   0.02 
    
     00085   17.160  0.000  -12.103  0.050  -12.105 -12.106   9.236   1.000  0x02200001 0x00000881   0.62   0.02 
    
    
  • o6157g0218o change BACKGROUND.XBIN,YBIN from 32 to -- 128 much closer SMF/warp AP_MAG but offset continues on at 256 up to ~400. psf_major all over
    -- 128: same ppSub seed, other ppSub seed 
    00175   17.110  0.000  -12.156  0.054  -12.253 -12.290   6.915   0.999  0x02200001 0x00000881   0.63   0.00 
    
    00119   16.602  0.000  -12.663  0.056  -12.217 -12.255  12.361   0.999  0x02200001 0x00000081   0.63   0.01 
    00116   16.585  0.000  -12.680  0.056  -12.237 -12.274  12.260   0.999  0x02200001 0x00000081   0.64   0.01 
    00192   17.009  0.000  -12.256  0.054  -12.232 -12.274   7.333   0.999  0x02200001 0x00000881   0.63   0.01 
    00169   17.112  0.000  -12.153  0.056  -12.240 -12.276   6.915   0.999  0x02200001 0x00000881   0.63   0.00 
    00127   16.535  0.000  -12.731  0.052  -12.211 -12.259  10.990   0.999  0x02200001 0x00000081   0.62   0.01 
    
    -- 256: same ppSub seed, other ppSub seed 
    00178   17.097  0.000  -12.168  0.053  -12.327 -12.364   6.915   0.999  0x02200001 0x00000881   0.65   0.00 
    
    00122   16.591  0.000  -12.674  0.055  -12.262 -12.300  12.307   0.999  0x02200001 0x00000081   0.64   0.01 
    00117   16.980  0.000  -12.285  0.055  -12.278 -12.325   7.746   0.999  0x02200001 0x00000881   0.64   0.01 
    00108   17.006  0.000  -12.259  0.054  -12.278 -12.322   8.638   0.999  0x02200001 0x00000881   0.64   0.01 
    
    -- 400: same ppSub seed, other ppSub seed 
    00177   17.103  0.000  -12.162  0.054  -12.292 -12.329   6.915   0.999  0x02200001 0x00000881   0.64   0.00 
    
    00153   17.188  0.000  -12.077  0.054  -12.301 -12.351   9.547   0.999  0x02200001 0x00000881   0.65   0.01 
    00142   17.174  0.000  -12.091  0.055  -12.313 -12.359   6.878   0.999  0x02200001 0x00000881   0.65   0.01 
    00149   17.010  0.000  -12.255  0.052  -12.332 -12.370   6.956   0.999  0x02200001 0x00000881   0.65   0.01 
    00149   16.567  0.000  -12.698  0.051  -12.290 -12.332   9.998   0.999  0x02200001 0x00000081   0.64   0.01 
    
    
    

manually running psphot on WSdiffim

exp_name=o6157g0218o, diff_id=272923, using ppSub output PSF -- found PSF was poorly determined so PSF_MAG not right here and focus is on AP_MAG

  • with PSPHOT DIFF recipe option (key difference is setting BACKGROUND.XBIN/YBIN 32, not 400) for AP_MAG
    --- same ppSub seed
     00218   18.355  0.000  -10.910  0.056  -11.881 -11.948   6.915   0.999  0x02200001 0x00000080   0.04 
    
    --- set XBIN/YBIN 128 -- same ppSub seed, random, random, random
     00184   18.290  0.000  -10.975  0.055  -12.035 -12.102   6.915   0.999  0x02200001 0x00000080   0.04  
     00186   18.295  0.000  -10.970  0.056  -12.003 -12.069   6.915   0.999  0x02200001 0x00000080   0.04 
     00185   18.286  0.000  -10.979  0.055  -12.056 -12.123   6.915   0.999  0x02200001 0x00000080   0.04 
     00185   18.291  0.000  -10.974  0.056  -12.033 -12.099   6.915   0.999  0x02200001 0x00000080   0.04 
    --- set to 6400 (full skycell so constant), same seed
     00181   18.294  0.000  -10.971  0.056  -12.002 -12.069   6.915   0.999  0x02200001 0x00000080   0.04 
    
  • w/o PSPHOT DIFF recipe option
    --- same ppSub seed, random, random, random
     00190   18.323  0.000  -10.943  0.055  -12.010 -12.076   6.915   0.999  0x02200001 0x00000080   0.04 
     00190   18.292  0.000  -10.973  0.056  -11.996 -12.063   6.915   0.999  0x02200001 0x00000080   0.04 
     00191   18.284  0.000  -10.981  0.055  -12.067 -12.133   6.915   0.999  0x02200001 0x00000080   0.04 
     00189   18.286  0.000  -10.979  0.055  -12.066 -12.133   6.915   0.999  0x02200001 0x00000080   0.04 
    
    

manually re-running WSdiffims with fix-A

Defining fix-A to be

  • BACKGROUND.XBIN,YBIN changed from 32->128
  • using a match list for PSF determination and not a match+merge list
  • with more current tag/trunk with fix (201302xx), new and same seed as original WSdiff (default conv refstack always unless noted) -- while AP_MAG similar with background bin change, PSF_MAG still offset ~0.1 mag
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  AP_MAG_RAW  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  DIFF_NRATIO_BAD  rsep
    --- o6157g0218o (original seed, random 3x)
    00164   16.928  0.000  -12.337  0.051  -12.230 -12.232   6.175   0.999  0x06a04007 0x00000881   0.68   0.01 
    
    00160   16.928  0.000  -12.337  0.052  -12.235 -12.238   6.166   0.999  0x06a04007 0x00000881   0.69   0.01 
    00162   16.932  0.000  -12.333  0.053  -12.222 -12.224   6.190   0.999  0x06a04007 0x00000881   0.68   0.01 
    00162   16.890  0.000  -12.375  0.052  -12.229 -12.231   6.413   0.999  0x06a04007 0x00000881   0.68   0.01 
    
    --- o6157g0219o (original seed, random 3x)
    00164   16.893  0.000  -12.369  0.048  -12.247 -12.251   6.331   0.999  0x06a04007 0x00000881   0.68   0.00 
    
    00167   16.883  0.000  -12.380  0.046  -12.241 -12.245   6.433   0.999  0x06a04007 0x00000881   0.68   0.00 
    00165   16.906  0.000  -12.356  0.049  -12.227 -12.229   6.289   0.999  0x06a04007 0x00000881   0.69   0.00 
    00163   16.882  0.000  -12.381  0.049  -12.235 -12.239   6.488   0.999  0x06a04007 0x00000881   0.68   0.00 
    
    
    
  • manually run psphot on WSdiffim with ppSub PSF and seed using fix-A
    --- simple run, default options - 
    00176   17.852  0.000  -11.413  0.052  -12.206 -12.214   6.175   0.999  0x02200001 0x00000080   0.03 
    
    --- adding recipe PSPHOT DIFF option -- similar to ppSub result now, ppSub PSF properly determined now
    00165   16.932  0.000  -12.333  0.052  -12.221 -12.229   6.175   0.999  0x06a04007 0x00000080   0.04 
    
  • aside: running psphot on warp using the recipe PSPHOT DIFF option -- oddly PSF_MAG similar to ppSub
    --- normally determines own PSF since warp (seed, random) -- 
    00381   16.918  0.000  -12.347  0.011  -12.179 -12.183   6.335   0.999  0x06a04007 0x00000080   0.02 
    
    00381   16.903  0.000  -12.362  0.011  -12.234 -12.238   6.360   0.999  0x06a04007 0x00000080   0.02 
    
    --- using ppSub PSF  (seed)
    00376   16.942  0.000  -12.323  0.011  -12.166 -12.174   6.175   0.999  0x06a04007 0x00000080   0.04
    
    

problems tracing out

  • psphot debug trace level >5 photometry different
    --- tiggered in psphotDumpPSFStars with 
    pmModelSub (source->pixels, source->maskObj, source->modelPSF, PM_MODEL_OP_FULL, maskVal);
    
  • diffphot AP_MAG offset ~0.2 brighter than SMF/warp (-12.05 diff, -12.25 smf/warp)
    --- influenced by gpc1/psphot.config DIFF option BACKGROUND.XBIN,YBIN at 32 when default psphot is 400
    
  • diffphot PSF_INST mag offset and unstable (as is psf_major) -- +/- 0.3+ mag (6.9--12.3 pixels)
    • traced to input list for finding PSF in ppSubMakePSF, seems to be using a merged list with refstk and many missing rather than the inConv list...
    • hack to add more into list helps, but moments analysis still restricted somewhere else and solution still generally poor
    • SSdiffs suspect mostly okay since both stacks but wouldn't be surprised if isn't okay and just generally appears so (seems to be a general trend with IPP...)
    • WWdiffs on the 3PI may have slight issue as well, WSdiffs on 3PI MOPS is just starting to look at an suspect will have similar problem then
    • traces back to pmSourceMatchMerge set for keeping a full merge list and not just matched -- how do other parts use this sources data?
  • psphot on diffim conflicts with ppSub phot. problem or setup wrong?
  • COMP_WT type for refConv.wt doesn't scale right and NANs the cores of brighter stars
  • -threads isn't listed as an option in the ppSub arguements
  • REVERSE in ppSubMakePSF.c never defined, should be INVERSE for -inverse/INVERSE?
  • later should also evaluate whether conv refstack is the better one to use here
  • NOCONV case not working well? recheck normalization
  • WS diffims use TARGET2 only or choice?

Example: MD09, TNO145452 (Alan Fitzsimmons)

TNO 145452

  • a bit faint but its' the best one I've found so far, as it stays in the field for a long long time.
  • in MD09 where we can predict the y magnitude should be within 0.1 mag of ~19.3 for the below observations
  • a bunch of y-band exposures in 2011 and 2012.
  • columns are Exp ID, MJD, RA, Dec.
    2011:
    Exposure ID  MJD  RA  Dec
    
    o5756g0570o  55756.592040       333.326857   +0.597387    
    o5756g0571o  55756.595015       333.326815   +0.597377    
    o5756g0572o  55756.597969       333.326774   +0.597368    
    o5756g0573o  55756.600918       333.326733   +0.597359    
    o5756g0574o  55756.603875       333.326692   +0.597350    
    o5756g0576o  55756.609792       333.326609   +0.597332    
    o5756g0577o  55756.612748       333.326568   +0.597323    
    o5757g0356o  55757.496424       333.314491   +0.594541    
    o5757g0357o  55757.499384       333.314449   +0.594532    
    o5757g0358o  55757.502357       333.314407   +0.594522    
    o5757g0359o  55757.505335       333.314365   +0.594513    
    o5757g0360o  55757.508292       333.314323   +0.594503    
    o5757g0361o  55757.511248       333.314281   +0.594494    
    o5757g0362o  55757.514208       333.314239   +0.594484    
    o5757g0363o  55757.517163       333.314197   +0.594475    
    o5766g0299o  55766.477073       333.180973   +0.559502    
    o5784g0173o  55784.507883       332.868350   +0.454736    
    o5784g0174o  55784.510828       332.868295   +0.454715    
    o5784g0175o  55784.513776       332.868239   +0.454695    
    o5784g0176o  55784.516717       332.868184   +0.454674    
    o5784g0177o  55784.519667       332.868128   +0.454654    
    o5784g0178o  55784.522608       332.868073   +0.454633    
    o5784g0180o  55784.528498       332.867962   +0.454592    
    (ending at 08-11-2011)
    
    2012: (starting 06-29-2012)
    Exposure ID  MJD  RA  Dec
    
    o6107g0064o  56107.597172       334.910525   +0.746688    
    o6107g0065o  56107.600110       334.910498   +0.746686    
    o6107g0066o  56107.603062       334.910470   +0.746684    
    o6107g0067o  56107.606006       334.910442   +0.746681    
    o6107g0068o  56107.608993       334.910414   +0.746679    
    o6201g0056o  56201.242691       333.383584   +0.125386    
    o6201g0057o  56201.245657       333.383540   +0.125358    
    o6201g0058o  56201.248616       333.383497   +0.125330    
    
    

This is the best example in the database because of the density of measurements and the known small amplitude of any lightcurve. In warp-warp diffs the slow motion will mean that 3PI and OSS measurements would be incorrect due to self-subtraction of the flux. But this should not only occur in MD fields as MOPS is being fed warp-stack measurements, and it would make the object fainter not brighter as seen. The variation in z-band and in other filters in 2012 is also puzzling.

Plot of observed magnitudes converted to V (top), and residual of magnitude (observed - predicted), in 2011 and 2012: (Key: g=green, r=red,i=orange,z=purple,y=cyan) lightcurve of 145452 in MD09

camera SMFs: both the original nightly processing and MD.GR0 reprocessing

Filename  IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG  PSF_INST_MAG_SIG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  rsep  chipXY

>>>>>>>> original V2 processing, synthcat
allsmf_y/1091859485.gpc1:MD09.nt:2011:07:14:o5756g0570o.361984:o5756g0570o.361984.cm.233853.smf 01006   19.542  0.011   -9.877  0.053   -9.764   1.989   1.000  0x02200001 0x00000080   0.29   XY13
allsmf_y/1091858911.gpc1:MD09.nt:2011:07:14:o5756g0571o.361983:o5756g0571o.361983.cm.233852.smf 01542   19.308  0.008  -10.115  0.037  -10.443   1.995   1.000  0x06200005 0x00000080   0.31   XY14
allsmf_y/1091937189.gpc1:MD09.nt:2011:07:14:o5756g0572o.361986:o5756g0572o.361986.cm.233864.smf 02746   19.161  0.000  -10.269  0.039  -11.142   2.814   0.999  0x02204001 0x00000080   0.37  XY14
allsmf_y/1091890096.gpc1:MD09.nt:2011:07:14:o5756g0573o.361985:o5756g0573o.361985.cm.233854.smf 00831   19.368  0.000  -10.049  0.041  -10.092   1.774   0.998  0x02200001 0x00000080   0.26  XY15
allsmf_y/1091937005.gpc1:MD09.nt:2011:07:14:o5756g0574o.361988:o5756g0574o.361988.cm.233861.smf 00890   19.271  0.000  -10.136  0.033  -10.241   1.580   0.997  0x06200045 0x00000080   0.29  XY25
allsmf_y/1092057932.gpc1:MD09.nt:2011:07:14:o5756g0576o.361989:o5756g0576o.361989.cm.233862.smf 00741   19.232  0.000  -10.172  0.031  -10.289   1.493   0.998  0x06204007 0x00000080   0.32  XY36
allsmf_y/1091915330.gpc1:MD09.nt:2011:07:14:o5756g0577o.361991:o5756g0577o.361991.cm.233856.smf 00714   19.226  0.011  -10.188  0.035  -10.240   1.703   0.997  0x06200005 0x00000080   0.25  XY36

allsmf_y/1094681457.gpc1:MD09.nt:2011:07:15:o5757g0356o.362440:o5757g0356o.362440.cm.234103.smf 00709   19.299  0.014  -10.026  0.036  -10.095   1.699   0.997  0x06200005 0x00000080   0.25  XY26
allsmf_y/1094682661.gpc1:MD09.nt:2011:07:15:o5757g0357o.362439:o5757g0357o.362439.cm.234105.smf 00825   19.279  0.000  -10.032  0.033  -10.068   1.609   0.998  0x06200005 0x00000080   0.25  XY36
allsmf_y/1094757324.gpc1:MD09.nt:2011:07:15:o5757g0358o.362441:o5757g0358o.362441.cm.234104.smf 00783   19.371  0.000   -9.926  0.033   -9.896   1.503   0.997  0x06200045 0x00000080   0.29  XY36
allsmf_y/1096254860.gpc1:MD09.nt:2011:07:15:o5757g0359o.362442:o5757g0359o.362442.cm.234355.smf 00755   19.239  0.014  -10.060  0.026  -10.177   1.250   0.998  0x06204007 0x00000080   0.27  XY46
allsmf_y/1094702845.gpc1:MD09.nt:2011:07:15:o5757g0360o.362443:o5757g0360o.362443.cm.234106.smf 00760   19.292  0.011  -10.005  0.027  -10.044   1.323   0.998  0x06200045 0x00000080   0.16  XY56
allsmf_y/1094717069.gpc1:MD09.nt:2011:07:15:o5757g0361o.362444:o5757g0361o.362444.cm.234108.smf 00763   19.260  0.011  -10.041  0.025  -10.046   1.151   0.612  0x06200005 0x000000a0   0.23  XY55
allsmf_y/1094852307.gpc1:MD09.nt:2011:07:15:o5757g0362o.362445:o5757g0362o.362445.cm.234107.smf 00778   19.324  0.000   -9.986  0.027  -10.039   1.295   0.998  0x06200045 0x00000080   0.24  XY65
allsmf_y/1094715184.gpc1:MD09.nt:2011:07:15:o5757g0363o.362447:o5757g0363o.362447.cm.234110.smf 01119   19.273  0.000  -10.042  0.029   -9.927   1.429   0.943  0x02204001 0x00000080   0.32  XY65

allsmf_y/1128179911.gpc1:MD09.nt:2011:07:24:o5766g0299o.367502:o5766g0299o.367502.cm.239536.smf 00638   19.196  0.014   -9.318  0.057   -9.741   1.237   0.999  0x02204001 0x00000080   0.31  XY66

allsmf_y/1177140372.gpc1:MD09.nt:2011:08:11:o5784g0173o.374858:o5784g0173o.374858.cm.246059.smf 00709   19.202  0.011  -10.199  0.036  -10.238   2.102   0.998  0x06200005 0x00000080   0.29  XY02
allsmf_y/1177146799.gpc1:MD09.nt:2011:08:11:o5784g0174o.374859:o5784g0174o.374859.cm.246060.smf 00735   19.111  0.000  -10.271  0.031  -10.319   1.926   0.999  0x06200005 0x00000080   0.31  XY03
allsmf_y/1177195723.gpc1:MD09.nt:2011:08:11:o5784g0175o.374860:o5784g0175o.374860.cm.246063.smf 00727   19.223  0.008  -10.156  0.031  -10.161   1.718   0.998  0x06200005 0x00000080   0.35  XY04
allsmf_y/1177158950.gpc1:MD09.nt:2011:08:11:o5784g0176o.374861:o5784g0176o.374861.cm.246064.smf 00549   19.235  0.000  -10.129  0.029  -10.127   1.590   0.998  0x06200005 0x00000080   0.32  XY05
allsmf_y/1177171394.gpc1:MD09.nt:2011:08:11:o5784g0177o.374862:o5784g0177o.374862.cm.246065.smf 00597   19.251  0.000  -10.115  0.030  -10.088   1.569   0.998  0x06200045 0x00000080   0.32  XY05
allsmf_y/1177178772.gpc1:MD09.nt:2011:08:11:o5784g0178o.374863:o5784g0178o.374863.cm.246066.smf 00673   19.186  0.000  -10.180  0.029  -10.259   1.625   0.998  0x06200045 0x00000080   0.34  XY16
allsmf_y/1177199195.gpc1:MD09.nt:2011:08:11:o5784g0180o.374865:o5784g0180o.374865.cm.246068.smf 00698   19.323  0.016  -10.056  0.033   -9.877   1.721   0.906  0x06200045 0x00000080   0.29  XY27

>>>>>>>> MD.GR0 reprocessed with ubercal
allsmf_y/2119523678.gpc1:MD09.GR0.20120601:o5756g0570o.361984:o5756g0570o.361984.cm.444636.smf 00927   19.403  0.035   -9.869  0.054   -9.556   3.071   0.998  0x02200001 0x00000000   0.17  XY13
allsmf_y/2119525130.gpc1:MD09.GR0.20120601:o5756g0571o.361983:o5756g0571o.361983.cm.444635.smf 01430   19.151  0.038  -10.121  0.042   -9.949   2.991   0.984  0x02200001 0x00000080   0.28  XY14
allsmf_y/2119527511.gpc1:MD09.GR0.20120601:o5756g0572o.361986:o5756g0572o.361986.cm.444638.smf 02385   19.554  0.038   -9.723  0.055   -9.786   4.134   0.997  0x02200001 0x00000000   0.28  XY14
allsmf_y/2119526275.gpc1:MD09.GR0.20120601:o5756g0573o.361985:o5756g0573o.361985.cm.444637.smf 00774   19.171  0.035  -10.097  0.039  -10.215   2.668   0.998  0x02200001 0x00000080   0.29  XY15
allsmf_y/2119528877.gpc1:MD09.GR0.20120601:o5756g0574o.361988:o5756g0574o.361988.cm.444640.smf 00838   19.121  0.035  -10.135  0.033  -10.212   2.317   0.997  0x06200005 0x00000080   0.31  XY25
allsmf_y/2119536441.gpc1:MD09.GR0.20120601:o5756g0576o.361989:o5756g0576o.361989.cm.444641.smf 00706   19.046  0.035  -10.207  0.031  -10.353   2.188   0.997  0x02204001 0x00000080   0.30  XY36
allsmf_y/2119539963.gpc1:MD09.GR0.20120601:o5756g0577o.361991:o5756g0577o.361991.cm.444642.smf 00680   19.073  0.034  -10.192  0.035  -10.233   2.500   0.997  0x06200005 0x00000080   0.25  XY36

allsmf_y/2119543361.gpc1:MD09.GR0.20120601:o5757g0356o.362440:o5757g0356o.362440.cm.444644.smf 00684   19.136  0.036  -10.037  0.036  -10.074   2.599   0.997  0x06200005 0x00000080   0.25  XY26
allsmf_y/2119541070.gpc1:MD09.GR0.20120601:o5757g0357o.362439:o5757g0357o.362439.cm.444643.smf 00771   19.105  0.037  -10.055  0.033  -10.072   2.382   0.998  0x06200005 0x00000080   0.23  XY36
allsmf_y/2119550643.gpc1:MD09.GR0.20120601:o5757g0358o.362441:o5757g0358o.362441.cm.444645.smf 00729   19.180  0.042   -9.962  0.032   -9.961   2.250   0.997  0x06200005 0x00000080   0.26  XY36
allsmf_y/2119551751.gpc1:MD09.GR0.20120601:o5757g0359o.362442:o5757g0359o.362442.cm.444646.smf 00763   19.093  0.045  -10.045  0.026  -10.117   1.838   0.998  0x06200005 0x00000080   0.26  XY46
allsmf_y/2119564253.gpc1:MD09.GR0.20120601:o5757g0360o.362443:o5757g0360o.362443.cm.444651.smf 00736   19.142  0.043   -9.994  0.027  -10.027   2.126   0.998  0x06200005 0x00000080   0.21  XY56
allsmf_y/2119554727.gpc1:MD09.GR0.20120601:o5757g0361o.362444:o5757g0361o.362444.cm.444648.smf 00721   19.105  0.041  -10.036  0.025  -10.037   1.673   0.609  0x06200005 0x000000a0   0.26  XY55
allsmf_y/2119560202.gpc1:MD09.GR0.20120601:o5757g0362o.362445:o5757g0362o.362445.cm.444649.smf 00734   19.136  0.040  -10.015  0.027  -10.064   1.949   0.998  0x06200045 0x00000080   0.23  XY65
allsmf_y/2119558442.gpc1:MD09.GR0.20120601:o5757g0363o.362447:o5757g0363o.362447.cm.444650.smf 00972   19.113  0.039  -10.046  0.029   -9.933   2.143   0.940  0x06200005 0x00000080   0.29  XY65

allsmf_y/2119575990.gpc1:MD09.GR0.20120601:o5766g0299o.367502:o5766g0299o.367502.cm.444658.smf 00609   19.022  0.059   -9.326  0.057   -9.720   1.860   0.999  0x02204001 0x00000000   0.25  XY66

allsmf_y/2119581769.gpc1:MD09.GR0.20120601:o5784g0173o.374858:o5784g0173o.374858.cm.444664.smf 00624   18.992  0.035  -10.256  0.036  -10.200   3.258   0.998  0x06200005 0x00000080   0.23  XY02
allsmf_y/2119581016.gpc1:MD09.GR0.20120601:o5784g0174o.374859:o5784g0174o.374859.cm.444665.smf 00684   18.953  0.033  -10.275  0.031  -10.298   2.889   0.999  0x06200005 0x00000080   0.25  XY03
allsmf_y/2119581238.gpc1:MD09.GR0.20120601:o5784g0175o.374860:o5784g0175o.374860.cm.444666.smf 00696   19.078  0.036  -10.146  0.031  -10.121   2.508   0.997  0x06200005 0x00000080   0.31  XY04
allsmf_y/2119587747.gpc1:MD09.GR0.20120601:o5784g0176o.374861:o5784g0176o.374861.cm.444667.smf 00518   19.059  0.035  -10.152  0.029  -10.159   2.365   0.997  0x06200005 0x00000080   0.25  XY05
allsmf_y/2119589883.gpc1:MD09.GR0.20120601:o5784g0177o.374862:o5784g0177o.374862.cm.444668.smf 00557   19.104  0.036  -10.106  0.029  -10.121   2.495   0.996  0x06200005 0x00000080   0.27  XY05
allsmf_y/2119590278.gpc1:MD09.GR0.20120601:o5784g0178o.374863:o5784g0178o.374863.cm.444669.smf 00624   19.065  0.034  -10.149  0.030  -10.197   2.473   0.998  0x06200005 0x00000080   0.27  XY16
allsmf_y/2119589969.gpc1:MD09.GR0.20120601:o5784g0180o.374865:o5784g0180o.374865.cm.444670.smf 00641   19.130  0.034  -10.096  0.032   -9.945   2.512   0.901  0x06200005 0x00000080   0.27  XY27

>>>>>>>> 2012 w/ ubercal
allsmf_y/2152846158.gpc1:MD09.nt:2012:06:29:o6107g0064o.500972:o6107g0064o.500972.cm.452770.smf 00621   19.137  0.038  -10.103  0.037  -10.119   3.645   0.998  0x06200005 0x00000080   0.36  XY64
allsmf_y/2152850545.gpc1:MD09.nt:2012:06:29:o6107g0065o.500973:o6107g0065o.500973.cm.452771.smf 00751   19.263  0.040   -9.977  0.042   -9.584   3.641   0.998  0x02200001 0x00000080   0.30  XY64
allsmf_y/2152853156.gpc1:MD09.nt:2012:06:29:o6107g0066o.500974:o6107g0066o.500974.cm.452772.smf 00687   19.085  0.034  -10.147  0.033  -10.300   3.273   0.999  0x02204001 0x00000080   0.35  XY63
allsmf_y/2152855541.gpc1:MD09.nt:2012:06:29:o6107g0067o.500975:o6107g0067o.500975.cm.452773.smf 00802   19.239  0.035   -9.995  0.039   -9.818   3.353   0.999  0x06200045 0x00000080   0.33  XY63
allsmf_y/2152858750.gpc1:MD09.nt:2012:06:29:o6107g0068o.500976:o6107g0068o.500976.cm.452774.smf 00683   19.285  0.037   -9.948  0.043   -9.940   3.404   0.998  0x02200001 0x00000080   0.32  XY62

allsmf_y/2619124827.gpc1:MD09.nt:2012:10:01:o6201g0056o.524345:o6201g0056o.524345.cm.593200.smf 00705   19.116  0.043   -9.862  0.055   -9.606   2.500   0.997  0x02200001 0x00000000   0.24  XY55
allsmf_y/2619134453.gpc1:MD09.nt:2012:10:01:o6201g0057o.524346:o6201g0057o.524346.cm.593201.smf 00785   19.083  0.039  -10.065  0.038  -10.225   2.450   0.998  0x06200005 0x00000080   0.30  XY55
allsmf_y/2619131479.gpc1:MD09.nt:2012:10:01:o6201g0058o.524347:o6201g0058o.524347.cm.593203.smf 00808   19.272  0.037   -9.863  0.046   -9.778   2.578   0.997  0x02200001 0x00000080   0.32  XY55


Test diffims with o5766g0299o: MD.GR0 with 2 skycell overlap (056,057):

  • skycell FWHM -- looks like will be some deconvolution, more so (>1.5 pixel) with the convolved refstack
    --- skycell.056
          Input FWHM: 2.826513
          Reference FWHM: 4.538850
            Iso-kernel region: [21:6378,21:6378] out of 6400,6400
          Found 276 stamps
            Kron Radii vs FWHMs 1: fwhm: 2.826513, kron 4.354785
            Kron Radii vs FWHMs 2: fwhm: 4.538850, kron 4.764421
    
    -- skycell.057
          Input FWHM: 3.011812
          Reference FWHM: 4.999094
            Iso-kernel region: [21:6378,21:6378] out of 6400,6400
          Found 263 stamps
            Kron Radii vs FWHMs 1: fwhm: 3.011812, kron 4.395961
            Kron Radii vs FWHMs 2: fwhm: 4.999094, kron 4.996646
    
    
    --- skycell.056 unconvolved refstack
          Input FWHM: 2.826513
          Reference FWHM: 3.358581
            Iso-kernel region: [21:6378,21:6378] out of 6400,6400
          Found 276 stamps
            Kron Radii vs FWHMs 1: fwhm: 2.826513, kron 4.328077
            Kron Radii vs FWHMs 2: fwhm: 3.358581, kron 4.211567
    
    --- skycell.057 unconvolved refstack
          Input FWHM: 3.011812
          Reference FWHM: 3.642784
            Iso-kernel region: [21:6378,21:6378] out of 6400,6400
          Found 263 stamps
            Kron Radii vs FWHMs 1: fwhm: 3.011812, kron 4.754183
            Kron Radii vs FWHMs 2: fwhm: 3.642784, kron 4.327400
    
  • psphot run on warp with ipp-20121218 tag using basically random seeds --
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2   rsep
    --- skycell.056  
    00836   19.023  0.000   -9.325  0.056   -9.677   1.924   0.999  0x02204001 0x00000000    0.22 
    00834   19.026  0.000   -9.321  0.056   -9.674   1.922   0.999  0x02204001 0x00000000    0.22 
    
    --- skycell.057
    00766   18.999  0.000   -9.348  0.055   -9.697   1.920   0.999  0x02204001 0x00000000    0.23 
    00768   19.004  0.000   -9.343  0.056   -9.666   1.922   0.999  0x02204001 0x00000000    0.23 
    
    
  • original WSdiff, V2, synthcat (rerun with old tag give similar values) -- agrees with relative SMF w/in uncertainties
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  DIFF_NRATIO_BAD  rsep
    00430   19.262  0.000   -9.252  0.225   -9.487   1.596   0.997  0x02200001 0x00000081   0.70   0.29 
    
  • rerun WSdiff with recent tag ipp-20121026 but original V2 pixels with synthcat -- agrees with relative SMF w/in uncertainties and to original tag version.
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  DIFF_NRATIO_BAD  rsep
    00430   19.233  0.000   -9.282  0.220   -9.481   2.364   0.998  0x02200001 0x00000881   0.69   0.31 
    
  • WSdiff with MD.GR0 V3 and ubercal -- skycell.056 agrees with relative SMF w/in uncertainties but skycell.057 has a problem.
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  DIFF_NRATIO_BAD  rsep
    --- skycell.056
    00269   19.102  0.000   -9.245  0.115   -9.531   1.956   0.999  0x02200001 0x00000882   0.70   0.21 
    --- skycell.057
    00234   18.799  0.000   -9.548  0.120   -9.658   2.679   0.998  0x02200001 0x00000882   0.71   0.21 
    
  • WSdiff for MD are typically with convolved refstack, run with unconvolved from MD.GR0 -- similar to the normal WSdiff convolved case with skycell.057 having a problem, but skycell.056 also >1-sig different from the SMF. Extra flags mean ??
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  DIFF_NRATIO_BAD  rsep
    --- skycell.056
    00129   19.240  0.000   -9.108  0.090   -9.515   1.897   0.999  0x06a04007 0x00000882   0.69   0.21 
    --- skycell.057
    00115   18.881  0.000   -9.466  0.090   -9.635   2.420   0.999  0x02200001 0x00000882   0.71   0.21 
    
  • since slow moving TNO (<1 pixel in 4 exposures covered in night stack), is in SSdiff for comparison (unconv SSdiff is what SN/transients use and haven't noticed a problem) -- all agree with relative SMF w/in uncertainty and relative to each similar run but are systematically fainter by ~0.03--0.1.
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  DIFF_NRATIO_BAD  rsep
    --- V2 original nightly stack SSdiff (night 20110724)
    00186   19.294  0.000  -13.914  0.059  -14.083   1.791   0.998  0x02200001 0x00000081   0.70   0.26 
    --- V2 rerun with recent tag version 
    00169   19.251  0.000  -13.957  0.059  -14.092   2.742   0.998  0x02200001 0x00000881   0.69   0.28 
    
          Input FWHM: 3.328463
          Reference FWHM: 3.358581
            Iso-kernel region: [21:6378,21:6378] out of 6400,6400
          Found 475 stamps
            Kron Radii vs FWHMs 1: fwhm: 3.328463, kron 3.856692
            Kron Radii vs FWHMs 2: fwhm: 3.358581, kron 4.226551
    
          Input FWHM: 3.840636
          Reference FWHM: 3.642784
            Iso-kernel region: [21:6378,21:6378] out of 6400,6400
          Found 455 stamps
            Kron Radii vs FWHMs 1: fwhm: 3.840636, kron 3.999478
            Kron Radii vs FWHMs 2: fwhm: 3.642784, kron 4.368307
    
    
    ---  MD.GR0 V3 skycell.056 old and current tag version -- with ubercal catalog --
    00289   19.053  0.000  -14.155  0.071  -14.311   2.112   0.999  0x02200001 0x00000882   0.72   0.22 
    00241   19.069  0.000  -14.139  0.069  -14.307   2.130   0.999  0x02200001 0x00000882   0.69   0.21 
    ---  MD.GR0 V3 skycell.057 old and current tag version
    00292   19.101  0.000  -14.107  0.071  -14.309   1.380   0.999  0x02200001 0x00000082   0.72   0.21 
    00240   19.099  0.000  -14.109  0.069  -14.303   1.418   0.999  0x02200001 0x00000082   0.69   0.19 
    
  • rerun WS diffim on MD.GR0 warps only with recent tag/trunk fix-A 201302xx -- mostly agree to SMF now and across skycell
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  DIFF_NRATIO_BAD  rsep
    --- skycell.056 (same seed, random 2x)
    00326   19.004  0.000   -9.343  0.108   -9.654   1.912   0.999  0x06a04007 0x00000882   0.76   0.21 
    
    00297   19.002  0.000   -9.345  0.106   -9.658   1.916   0.999  0x06a04007 0x00000882   0.77   0.21 
    00325   19.008  0.000   -9.339  0.108   -9.645   1.915   0.999  0x06a04007 0x00000882   0.76   0.21
     
    --- skycell.057
    00237   18.991  0.000   -9.357  0.114   -9.658   1.896   0.999  0x06a04007 0x00000882   0.75   0.21 
    
    00260   18.985  0.000   -9.362  0.113   -9.664   1.900   0.999  0x06a04007 0x00000882   0.75   0.21 
    00252   18.986  0.000   -9.361  0.113   -9.669   1.902   0.999  0x06a04007 0x00000882   0.75   0.21
    
  • rerun SSdiff on MD.GR0 stacks only with fix-A also -- will want to test other SSdiffs but don't expect it to do much other than improve PSF determination in rare cases maybe on edges. PSF appears to be improved, closer to SMF and between each other, but AP between exposures has a larger difference. However, the psf_major before was clearly too small.
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  DIFF_NRATIO_BAD  rsep
    --- skycell.056 
    00287   19.010  0.000  -14.198  0.068  -14.432   2.111   0.999  0x06a04007 0x00000882   0.76   0.22 
    
    00223   19.000  0.000  -14.208  0.067  -14.448   2.123   0.999  0x06a04007 0x00000882   0.76   0.22 
    
    --- skycell.057
    00294   19.022  0.000  -14.186  0.067  -14.426   2.090   0.999  0x06a04007 0x00000882   0.74   0.21 
    
    00235   19.028  0.000  -14.181  0.068  -14.423   2.133   0.999  0x06a04007 0x00000882   0.74   0.21 
    

Large sample diffim photometry for y-band

With fix-A applied --

  • large scatter removed (see scatter in original plot at beginning of section)
  • appears to be a clear mean offset of ~0.07 mag (diffim PSF_MAG brighter) Is this similar to other convolution mag offsets? Also not all unlikely that a few could be as high as ~0.15 mag offset as seen in the MD10 example above. It is also important to note also that the photerr on the diffim photometry is typically of order 0.1-0.15 for this sample as well when using the default convolved refstack.
  • also appears AP_MAG may have two groupings, splitting earlier and later data moves contribution mostly randomly (time split can involve data quality and skycell position since mover)

  • a larger background bin of 300 (rather than 128 in fix-A, but with same seed) seems to reduce the second group in the AP_MAG

With fix-A removing large PSF scatter, where is the ~0.07 mag offset coming from?

  • the SMF and psphot (ipp-20121218 tag) run on warps

  • the same psphot on warps and ppSub with fix-A and background bin 128 -- so from ppSub photometry

Example: MD01, MBA 131130 (Alan Fitzsimmons)

This is a nice example of the offset plus large scatter in y-band for a Main Belt Asteroid seen this year.

Exposure ID   RA  Dec
-- start 20120830
o6169g0184o 36.353429    -4.350323    
o6169g0185o 36.353441    -4.350616    
o6170g0586o 36.360948    -4.446235    
o6170g0588o 36.360938    -4.446831    
o6170g0590o 36.360927    -4.447427    
o6170g0591o 36.360922    -4.447725    
o6170g0593o 36.360911    -4.448320    
o6171g0613o 36.362699    -4.549196    
o6171g0614o 36.362676    -4.549499    
o6171g0616o 36.362631    -4.550103    
o6171g0617o 36.362608    -4.550406    
o6171g0619o 36.362562    -4.551012    
o6172g0677o 36.358383    -4.654112    
o6172g0682o 36.358181    -4.655647    
o6172g0683o 36.358141    -4.655956    
o6172g0684o 36.358101    -4.656264    
Plot of observed magnitudes converted to V (top), and residual of magnitude (observed - predicted) in 2012. The large gap in the data was caused by telescope shutdown due to rotator problems. (Key: g=green, r=red,i=orange,z=purple,y=cyan)
131130 lightcurve in MD01

camera SMFs: all with ubercal catalog -- clearly some variation upwards ~0.5 mag variation night-to-night, ~0.1 w/in a night

Filename  IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG  PSF_INST_MAG_SIG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  rsep  chipXY

allsmf_y/2458688059.gpc1:MD01.nt:2012:08:30:o6169g0184o.516307:o6169g0184o.516307.cm.547162.smf 00807   19.535  0.066   -9.568  0.050  -10.349   2.406   0.996  0x02204001 0x00000080   0.29 XY42
allsmf_y/2458693061.gpc1:MD01.nt:2012:08:30:o6169g0185o.516308:o6169g0185o.516308.cm.547163.smf 00652   19.576  0.066   -9.521  0.051  -10.036   2.179   0.996  0x02204001 0x00000000   0.15 XY32

allsmf_y/2467608609.gpc1:MD01.nt:2012:08:31:o6170g0586o.516969:o6170g0586o.516969.cm.549810.smf 00726   20.144  0.067   -9.003  0.077   -8.749   1.870   0.998  0x02200001 0x00000000   0.20 XY53
allsmf_y/2467613539.gpc1:MD01.nt:2012:08:31:o6170g0588o.516971:o6170g0588o.516971.cm.549819.smf 00880   19.936  0.064   -9.219  0.072   -9.085   2.046   0.996  0x02200001 0x00000000   0.25 XY42
allsmf_y/2467623316.gpc1:MD01.nt:2012:08:31:o6170g0590o.516973:o6170g0590o.516973.cm.549824.smf 00946   19.830  0.062   -9.333  0.068   -9.464   2.078   0.996  0x02200001 0x00000000   0.27 XY42
allsmf_y/2467633036.gpc1:MD01.nt:2012:08:31:o6170g0591o.516974:o6170g0591o.516974.cm.549826.smf 00720   19.933  0.066   -9.223  0.071   -9.228   1.950   0.997  0x02200001 0x00000000   0.13 XY32
allsmf_y/2467676288.gpc1:MD01.nt:2012:08:31:o6170g0593o.516976:o6170g0593o.516976.cm.549832.smf 01233   19.826  0.055   -9.339  0.067   -9.389   2.183   0.997  0x02200001 0x00000000   0.15 XY32

allsmf_y/2476572150.gpc1:MD01.nt:2012:09:01:o6171g0613o.517689:o6171g0613o.517689.cm.552533.smf 00632   19.784  0.057   -9.443  0.060   -9.659   2.326   0.999  0x02200001 0x00000000   0.26 XY52
allsmf_y/2476588989.gpc1:MD01.nt:2012:09:01:o6171g0616o.517692:o6171g0616o.517692.cm.552537.smf 00950   19.883  0.063   -9.343  0.066   -9.675   2.505   0.998  0x02200001 0x00000000   0.15 XY42
allsmf_y/2476595216.gpc1:MD01.nt:2012:09:01:o6171g0617o.517693:o6171g0617o.517693.cm.552538.smf 01019   19.954  0.056   -9.259  0.069   -9.436   2.307   0.998  0x02204001 0x00000000   0.25 XY32
allsmf_y/2476598472.gpc1:MD01.nt:2012:09:01:o6171g0614o.517690:o6171g0614o.517690.cm.552534.smf 00594   19.762  0.065   -9.464  0.057   -9.811   2.268   0.997  0x02200001 0x00000000   0.25 XY42
allsmf_y/2476607853.gpc1:MD01.nt:2012:09:01:o6171g0619o.517695:o6171g0619o.517695.cm.552543.smf 00758   19.976  0.075   -9.248  0.077   -9.087   2.363   0.997  0x02200001 0x00000000   0.25 XY32

allsmf_y/2484999709.gpc1:MD01.nt:2012:09:02:o6172g0677o.518454:o6172g0677o.518454.cm.555033.smf 00625   19.900  0.066   -9.287  0.068   -9.649   2.198   0.997  0x02204001 0x00000000   0.31 XY53
allsmf_y/2485059002.gpc1:MD01.nt:2012:09:02:o6172g0682o.518459:o6172g0682o.518459.cm.555046.smf 00657   19.951  0.070   -9.176  0.070   -9.610   1.906   0.999  0x02204001 0x00000000   0.14 XY32
allsmf_y/2485059159.gpc1:MD01.nt:2012:09:02:o6172g0683o.518460:o6172g0683o.518460.cm.555048.smf 00999   20.006  0.058   -9.172  0.063   -9.019   1.894   0.998  0x02200001 0x00000000   0.22 XY32
allsmf_y/2485060268.gpc1:MD01.nt:2012:09:02:o6172g0684o.518461:o6172g0684o.518461.cm.555050.smf 00660   19.934  0.053   -9.257  0.057   -9.620   1.998   0.999  0x02204001 0x00000000   0.26 XY32

Test diffims: with o6170g0586o, o6170g0588o -- MD01.V3 skycell.043, xy=4970.30, 4633.17 and ubercal catalog being normally processed after April 2012 --

  • psphot run on warp with ipp-20121218 tag using basically random seeds --
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2   rsep
    --- o6170g0586o 
    00766   20.128  0.000   -9.019  0.074   -8.851   1.884   0.999  0x02200001 0x00000000     0.21 
    00762   20.116  0.000   -9.031  0.073   -8.881   1.880   0.999  0x02200001 0x00000000     0.21 
    
    --- o6170g0588o 
    00745   19.919  0.000   -9.236  0.071   -9.155   2.222   0.998  0x02200001 0x00000000     0.24 
    00746   19.919  0.000   -9.236  0.072   -8.962   2.252   0.998  0x02200001 0x00000000     0.22 
    
  • original WSdiff using tag ipp-20120802 -- again well off from the SMF photometry by ~0.6 mag
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  DIFF_NRATIO_BAD  rsep
    --- o6170g0586o 
    00096   19.528  0.000   -9.619  0.154   -9.187   3.370   0.992  0x02200001 0x00000881   0.59   0.20 
    
    --- o6170g0588o 
    00103   19.454  0.000   -9.701  0.166   -9.210   3.294   0.992  0x02200001 0x00000881   0.56   0.25 
    
  • rerun WSdiff with recent tag ipp-20121218 -- no real reason to do this...
  • rerun with recent tag/trunk fix-A 201302xx and default conv refstack -- now photometry matching SMF photometry w/in photerr
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  DIFF_NRATIO_BAD  rsep
    --- o6170g0586o (same original ppSub seed, random seed)
    00172   20.112  0.000   -9.035  0.162   -8.866   1.849   0.998  0x02204801 0x00000881   0.59   0.20 
    
    00177   20.117  0.000   -9.030  0.160   -8.889   1.884   0.998  0x02204801 0x00000881   0.59   0.20 
    
    --- o6170g0588o (same original ppSub seed, random seed)
    00151   19.889  0.000   -9.266  0.175   -9.168   2.151   0.998  0x06a04007 0x00000881   0.61   0.23 
    
    00152   19.890  0.000   -9.264  0.176   -9.158   2.146   0.998  0x06a04007 0x00000881   0.61   0.23 
    
  • rerun again with recent tag/trunk fix-A 201302xx and raise background bin size 128->300 -- AP_MAG closer
    --- o6170g0586o (same original ppSub seed, random seed)
    00176   20.137  0.000   -9.010  0.164   -8.754   1.849   0.998  0x02200801 0x00000881   0.57   0.20 
    
    00141   20.128  0.000   -9.019  0.162   -8.771   1.860   0.998  0x02200801 0x00000881   0.58   0.20 
    
    --- o6170g0588o (same original ppSub seed)
    00169   19.901  0.000   -9.254  0.149   -9.121   2.151   0.998  0x06a04007 0x00000881   0.60   0.23
    
  • rerun with recent tag/trunk fix-A 201302xx and unconv refstack -- photerr reduced using the unconv refstack likely from the larger deconv needed on the conv refstacks
    IPP_IDET  CAL_PSF_MAG  CAL_PSF_MAG_SIG  PSF_INST_MAG  PSF_INST_MAG_SIG  AP_MAG  PSF_MAJOR  PSF_QF_PERFECT  FLAGS  FLAGS2  DIFF_NRATIO_BAD  rsep
    --- o6170g0586o (same original ppSub seed, random seed)
    00158   20.109  0.000   -9.038  0.127   -8.874   1.849   0.998  0x02204801 0x00000881   0.60   0.20 
    
    --- o6170g0588o (same original ppSub seed, random seed)
    00173   19.892  0.000   -9.263  0.146   -9.157   2.151   0.998  0x06a04007 0x00000881   0.61   0.23 
    
    

Example: SSdiffim photometry for transients

Do any of the modifications impact the photometry from the SSdiffims critical for the static transient use? An example is difficult with a brighter, host-less or offset source.

Try PS1-12sn in i-band:

  • from QUB table -- Diff ID is QUB internal, need diff_id from CMF File
    Diff ID	 MJD	 RA	 DEC	 Instrument Mag	 Aperture Mag	 Calibrated Mag	 Filter	 CMF File	 Image
    14210333	56015.330	10:57:31.24	+57:17:51.6	-14.312	-14.262	18.896	i	MD05.V3.skycell.031.SS.dif.229930.cmf	None
    14155799	56007.442	10:57:31.27	+57:17:51.7	-13.879	-13.450	19.329	i	MD05.V3.skycell.031.SS.dif.229024.cmf	None
    14531719	56044.311	10:57:31.24	+57:17:51.7	-13.795	-13.771	19.413	i	MD05.V3.skycell.031.SS.dif.235265.cmf	None
    14407496	56033.299	10:57:31.24	+57:17:51.7	-13.581	-13.528	19.627	i	MD05.V3.skycell.031.SS.dif.233195.cmf	None
    14560332	56048.393	10:57:31.25	+57:17:51.7	-13.743	-13.663	19.465	i	MD05.V3.skycell.031.SS.dif.235748.cmf	None
    14440193	56038.309	10:57:31.24	+57:17:51.7	-13.717	-13.713	19.492	i	MD05.V3.skycell.031.SS.dif.233748.cmf	None
    14267638	56019.336	10:57:31.24	+57:17:51.7	-14.266	-14.212	18.943	i	MD05.V3.skycell.031.SS.dif.230876.cmf	None
    14308445	56027.416	10:57:31.25	+57:17:51.7	-13.710	-13.697	19.498	i	MD05.V3.skycell.031.SS.dif.231546.cmf	None
    14972938	56066.273	10:57:31.23	+57:17:51.7	-12.982	-12.874	20.226	i	MD05.V3.skycell.031.SS.dif.242606.cmf	None
    14481823	56041.307	10:57:31.24	+57:17:51.7	-13.798	-13.781	19.410	i	MD05.V3.skycell.031.SS.dif.234442.cmf	None
    
  • id and photometry -- w/in 0.01-0.02 mag
    select * from diffInputSkyfile where skycell_id like "%31" and diff_id=229930;
    | diff_id | skycell_id  | warp1 | stack1 | warp2 | stack2 | tess_id | diff_skyfile_id | -- rerun ori cal here / past ori cal above/ fixv0
    +---------+-------------+-------+--------+-------+--------+---------+-----------------+
    |  229930 | skycell.031 |  NULL | 843378 |  NULL | 462996 | MD05.V3 |        14210333 | --  pre-ubercal, so cant compare
    |  229024 | skycell.031 |  NULL | 841463 |  NULL | 462996 | MD05.V3 |        14155799 | -- pre-ubercal, so cant compare
    |  235265 | skycell.031 |  NULL | 885629 |  NULL | 462996 | MD05.V3 |        14531719 | -- 19.424 /19.413 / 19.424
    |  233195 | skycell.031 |  NULL | 860793 |  NULL | 462996 | MD05.V3 |        14407496 | -- 19.621 / 19.627 / 19.621
    |  235748 | skycell.031 |  NULL | 887369 |  NULL | 462996 | MD05.V3 |        14560332 | -- 19.448 / 19.465 / 19.448
    |  233748 | skycell.031 |  NULL | 873779 |  NULL | 462996 | MD05.V3 |        14440193 | -- 19.469 / 19.465 / 19.469
    |  230876 | skycell.031 |  NULL | 845233 |  NULL | 462996 | MD05.V3 |        14267638 | --  pre-ubercal, so cant compare
    |  231546 | skycell.031 |  NULL | 855758 |  NULL | 462996 | MD05.V3 |        14308445 | -- 19.500 / 19.498 / 19.501
    |  242606 | skycell.031 |  NULL | 941152 |  NULL | 462996 | MD05.V3 |        14972938 | -- 20.212 / 20.226 / 20.215
    |  234442 | skycell.031 |  NULL | 875829 |  NULL | 462996 | MD05.V3 |        14481823 | -- 19.397 / 19.410 / 19.397
    
  • original photometry from the SSdiffim, rerun using the ipp-20121218 tag

  • using fix-A, very small change relatively

  • larger background bin of 300 (rather than 128 in fix-A, but with same seed), little change relatively in the AP_MAG

Attachments