Summary of ppSub_testing_201201 ---IN PROGRESS PLEASE DO NOT EDIT---

Primary purpose is to wrap up and activate the convolution direction choice in ppSub. (pseudo-primary, purpose really to reduce number of false diffim detections)

Secondary purpose is to monitor if working as expected and address any additional issues (convolution troubles) and baseline to verify against any future changes in code (test set).

Background

Samples of previous information/work/comparisons

• ​http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/ppSub_vs_Hotpants
• ​http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/Basic_GPC1_Processing_IPP_Photpipe_Comparison

• ​http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/ppSubModifications_201009
  • ​http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/ppSub_Modifications_20101021.single
  • ​http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/ppSub_Modifications_20100918.single

• ​http://ps1sc.ifa.hawaii.edu/PS1wiki/index.php/TransientPhotfestSept2010
  • ​http://ps1sc.ifa.hawaii.edu/PS1wiki/index.php/Performance_Tests_July2010#A.X_Diffim_photometry_reprocessing_data_sample
  • ​http://ps1sc.ifa.hawaii.edu/PS1wiki/index.php/Diffimtest_IPP_photpipe_sne
  • ​http://ps1sc.ifa.hawaii.edu/PS1wiki/index.php/Diffimtest_ut100510_md07s056

Convolution Issues

Convolution direction choice

We want to avoid large deconvolution in PSF-matching image subtraction. As the image quality of the images continues to improve, always convolving the reference stack will not produce the best difference image and convolving the input image is desired. The quality of the difference image is evaluated in both directions and the better is chosen for the output (add specific details).

Current tag for testing base diffims is ipp-20111220 (since local tests should stay at some r)

Current branch for testing the choice of convolution direction and is updated to r33098 (from 1/11/2012)

• ​http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/browser/branches/meh_branches/ppsub_test

Selected Test Samples

MD04 primary sample

Initial test for if choosing the proper direction in stack-stack diffims now using MD04. Was initially done using MD10, but we want to continue to build up test samples over other IPP codes using MD04.

Start with just 2 skycells having example events from last year for photometry comparisons as well -

055 --> 077.V2 -- center -

• ​http://telescopes.rc.fas.harvard.edu/ps1/alerts2011/2010-C-020101/2010-C-020101.html - AGN@0.345 (xy?)
• ​http://telescopes.rc.fas.harvard.edu/ps1/alerts2011/2010-D-030288/2010-D-030288.html - poor sub/var?
• ​http://telescopes.rc.fas.harvard.edu/ps1/alerts2011/2011-A-120103/2011-A-120103.html - alert
• ​http://telescopes.rc.fas.harvard.edu/ps1/alerts2011/2011-A-120706/2011-A-120706.html - alert and spectrum
• new events
  • ​http://telescopes.rc.fas.harvard.edu/ps1/alerts2012/2012-A-350113/2012-A-350113.html - residual peak?
  • ​http://telescopes.rc.fas.harvard.edu/ps1/alerts2012/2012-A-350492/2012-A-350492.html - bright star
  • ​http://telescopes.rc.fas.harvard.edu/ps1/alerts2012/2012-A-350211/2012-A-350211.html - alert

058 --> 073.V2 -- edge -

• ​http://telescopes.rc.fas.harvard.edu/ps1/alerts2011/2010-A-000541/2010-A-000541.html - QSO@2.051
• ​http://telescopes.rc.fas.harvard.edu/ps1/alerts2011/2011-B-130093/2011-B-130093.html - A star
• ​http://telescopes.rc.fas.harvard.edu/ps1/alerts2011/2011-C-140196/2011-C-140196.html - sdss target QSO
• new events

QUB cross-match page: ​http://star.pst.qub.ac.uk/ps1/psdb/crossmatchipptocfa/ (add when time.. difficult to search the new crossmatch page..)

Sample nightly stacks for both skycells with same obsdate to look at variation of PSF center and edge (0.25 "/pixel), limited to r-band for now and a sample size that can be manually rerun with some ease.

obsdate	stack_id	FWHM_MAJ (pixel)	IQ_FW1 (pixel)
	skycell.055
refstack.20111012	450933	5.206454 (+-0.6-0.8) not the 4.5 on the MD.GR0 page..	4.08811 (+-0.3-0.4)
20100110	444020	4.878624	3.891729
20100121	444079	5.843588	4.574009
20110129	444251	4.424473	3.378561
20110201	644876	5.746622	4.748873
20110312	446658	4.958342	3.74775
20110315	645114	4.270276	3.232871
20110423	446715	4.603964	3.510957
20110429	645474	7.178107	6.027955
20120115	622752	5.805826	4.622056
20120121	646801	6.346761	5.359003
(swap 20120126 for 20120115?)	676389	4.443175	3.475093
	skycell.058
refstack.20111012	450936	4.616929	3.926292
20100110	444023	4.777016	3.687181
20100121	444082	5.567342	4.243489
20110129	444254	3.976701	3.100284
20110201	644879	5.023516	4.421264
20110312	446661	5.114515	3.701454
20110315	645117	4.58284	3.376984
20110423	446718	4.358292	3.138227
20110429	645477	6.088856	5.447231
20120115	622755	5.300241	4.337034
20120121	646804	5.627441	5.058287
(swap 20120126 for 20120115?)

Difficult to use past examples of wrong convolution direction since refstacks remade, so select sample to just test current choices.

QUB examples of wrong convolution choice:

• ​http://star.pst.qub.ac.uk/ps1/psdb/userlist/10/
• ​http://star.pst.qub.ac.uk/ps1/psdb/userlist/5/?sort=-followup_flag_date

• ​http://star.pst.qub.ac.uk/ps1/psdb/candidate/1095456431022404400/

Not all seem to be really due to convolving in the wrong direction and need to identify and separate the cases/causes if can. Running samples forced in both direction useful to see.

CfA photpipe/HOTPANTs direction: using single template/reference stack so locally get keyword CONVOL00 from header except slight issue with amps. All are convolving the TEMPLATE only so also backs up the issue seems not a convolution direction issue.

Sample diffims with direction forced and detections marked (note both stack-refstack and stack-stack cases). The stack-stack case is the really the only way to test the convolution direction choice now unless go back to the MD04.V2 sample. (easiest to use event stamps from pages, but may want image photpipe diffim..)

event (auto direction)	TARGET2 (convolved ref)	TARGET1 (convolved stk)	photpipe automatic (direction)	comment
X (TARGET2)			(TEMPLATE)	conv refstk better,

Photometry/detection comparisons:

Others Samples

Need tests to verify changes will not break diffims for other surveys (MOPS etc) but do not need to worry about magic now. Not for testing if they will choose the right direction (only if using AUTO), but changes involved part of the normalization and will need to verify doesn't change flux/variance values:

• simulated image set WW,WS,SS, SSref -- since SSref for MD04 seem to prefer convolving the deeper refstack still, will need to generate
• warp-warp diffim: MD and 3PI TTI
• warp-stack diffim: MD and LAP stack

Poor PSF-matching?

Not sure yet what to classify this as other than the PSF not being matched well. Suspect similar to issue with convolved stacks in ppStack.

No-convolution?

Reports from other surveys that diffims are poorer with images having similar PSFs, convolution isn't well matched? Is this for dual only?

Monitoring

Nightly tests of convolutions

Is there any conflict with the expected desired direction? Compare to CfA photpipe, detection counts.

• having IQ/FWHM keywords in stack images or at least the gpc1 DB would be helpful here since are calculated for the PSF matching in the diffim process, doing manually here takes a good amount of time.

Is the convolution not optimal?

HOTPANTs comparisons

Difficult to automate and may want to force both directions for comparison as well as shallower reference stacks to use. Are the diffims continuing to produce similar results.

• for select night samples should setup and run locally. still on the todo list.