This page tracks events in the static sky processing for Large Area Processing for label LAP.ThreePi.20120706

Bill Sweeney

Current Status

The following 5 plots show the progress in LAP static sky and sky calibration processing as of 2013-01-26 18:28 UT. The system is currently working on the area from 22-24 hours RA. Some regions near the galactic plane have been deferred for now.

Holes in the completed areas are regions where a stack in the given filter was not constructed, or in a small number of cases the staticsky or skycalibration processing failed due to software or data quality errors. Note processing for areas with less than five filter coverage has not been performed for the areas between 0 and 4 hours RA and 22 - 24 hours RA. Many of the holes there will close up in some of the bands






Subsequent plots are for earlier dates and combine the results for all 5 filters

static sky status 2013-01-24 04:40 UTC

LAP static sky progress as of 2013-01-21

The following plot shows the skycells that had completed static sky processing as of 2013-01-17 21:30 UTC

LAP static sky progress as of January 17, 2013


static sky progress January 14, 2013

The following plot shows the skycells that have completed static sky processing as of 2013-01-03 21:59 UTC

Static Sky Progress 2013-01-03

The following plot shows the skycells that have completed static sky processing as of 2012-12-19 22:30 UTC

Static sky progress 2012-12-19

The following plot shows the skycells that have been queued for staticsky but have had processing deferred by setting the state to 'wait' as of 2012-12-19. Note as of 2013-01-24 all but the region from 22 - 24 hours have been either run or moved to the '.gp' label as they are near the galactic plane

The plot below shows the number of sources detected per skycell in the r band.

Similar plot for the Kepler region log scale (RA increases to the left in this plot)

False Detections from y band stacks

We have noticed that some skycells are getting large numbers of matched (forced photometry) detections. These are caused by false detections in the y band. Recall that forced photometry is performed for all y band sources that are not matched with a detection in another band while detections for the other filters must match 2 other bands before forced photometry is done.

This plot shows the number of matched detetctions in i band verses the number of detected sources in y band.

If we plot the ratio of the number of matched detections to the number of detected sources for each band we can see these outliers. The following plots shows the ratio for the r band versus that in the ratio in i band for each stack. The plot on the left shows the full range including the outliers wile the plot on the right zooms in on the values near zero.

The plots for g and z show the same shape while the y band plot does not show the tail.

We propose re-running psphotStack for these skycells changing the rule that matched detections are created for all unmatched y band detections if some criteria is met. We can get rid of the worst of the outliers with the following cut.

             Nmatched_i / NDetected_i > 0.4 && Nmatched_r /  NDetected_r > 0.6

This effect is seen in 3% of the skycells north of 30 degrees declination.

After testing a more conservative cut was chosen. It is based on the ratio of the number of sources in y to the number of sources detected in r, i, and z.

Automatic forced photometry for sources detected in y band but not any other is disabled for skycells where

   (NDetected_y / NDetected_r) > 1.7 && (NDetected_y / NDetected_i > 1.6) &&  && (NDetectected_z > 1.65)

About 1 percent of skycells is affected by this feature.

Archival Notes

The following sections are notes on the early days of static sky processing for LAP 20120706

Beginnings 2012-11-21

It was decided to begin the stack photometry with the region containing the Kepler field (18h30m,+33d45') -> (20h20m,+52d30'). These skycells were queued for staticsky processing on November 21.

After those runs completed a list of projection cells that had completed LAP stacking were queued for processing. These were queued in order of projection cell number. This turns out to be a band at low declinations. These proceeded over the Thanksgiving holiday weekend.

Change of direction 1

After the holiday a change of direction was made. Rather than go in band of declination we went to stripes in RA. The outstanding runs were set to state ='wait' and skycells were queued in the band 9 < RA < 10

Change of direction 2

Due to some mis-communication we decided that it would be better for PSPS to load in decliation bands, so the existing runs were set back to state ='new' and new runs were queued in this region.

Change of direction 3

As a result of further discussions it was determined to go back to processing by RA stripes. The RA band including the Kepler field was made the highest priority. The skycells with centers above declination 10 were completed on 2012-12-04. The plot below shows the completion at that time.

Sky Calibration

The staticsky calibration processing was begun on 2012-12-03. This proceeds at a higher rate then the psphot operations.

For the region of proposed first ingest (270 < RA < 300) there are 55733 skycells to process. 14865 were completed on the first day.

Going Forward

The current plan is to postpone processing of the region near the galactic center (270 < RA < 300, DEC < 10) and proceed with ra stripes at 120 degrees. So far I have left the existing runs in wait state, and change them to run as needed. For simplicity we could drop them and change the label, but so far I've been managing the states. Here are the locations of the "waiting" skycells.

Change in direction 4

It was decided that rather than starting at ra 120 degrees, to instead process southern most part of the sky between 0 and 15 hours and declination < 10 degrees. The goal is to provide potential quasar targets for a KP10 group at MPIA that has observing time at La Silla net month.