IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links
wiki:PS1_Operations/broken_files

Version 19 (modified by bills, 15 years ago) ( diff )

--

This Page is used to record faults due to corrupt files.

Note: As of 2011-01-26 enter new faults at the top of this page

2011-02-10

  • corrupted warp file
perl ~bills/ipp/tools/runwarpskycell.pl --dbname gpc1 --redirect-output --warp_id 160444 --skycell_id skycell.076

2011-02-07

  • bad burntool table instances
(ippc17:~ipp/stdscience) bills% nls -l -a neb://ipp006.0/gpc1/20100917/o5456g0167o/o5456g0167o.ota03.burn.tbl
-rw-rw-r-- 1 ipp nebulous 0 Sep 27 06:54 /data/ipp012.0/nebulous/3e/1c/469256569.gpc1:20100917:o5456g0167o:o5456g0167o.ota03.burn.tbl
-rw-rw-r-- 1 apache nebulous 441410 Sep 27 07:18 /data/ipp042.0/nebulous/3e/1c/469281321.gpc1:20100917:o5456g0167o:o5456g0167o.ota03.burn.tbl
(ippc17:~ipp/stdscience) bills% cp /data/ipp042.0/nebulous/3e/1c/469281321.gpc1:20100917:o5456g0167o:o5456g0167o.ota03.burn.tbl /data/ipp012.0/nebulous/3e/1c/469256569.gpc1:20100917:o5456g0167o:o5456g0167o.ota03.burn.tbl
(ippc17:~ipp/stdscience) bills% nls -l -a neb://ipp009.0/gpc1/20100917/o5456g0124o/o5456g0124o.ota12.burn.tbl
-rw-rw-r-- 1 ipp nebulous 0 Sep 27 06:54 /data/ipp012.0/nebulous/62/db/469256183.gpc1:20100917:o5456g0124o:o5456g0124o.ota12.burn.tbl
-rw-rw-r-- 1 apache nebulous 345997 Sep 27 07:18 /data/ipp028.0/nebulous/62/db/469281316.gpc1:20100917:o5456g0124o:o5456g0124o.ota12.burn.tbl
(ippc17:~ipp/stdscience) bills% cp /data/ipp028.0/nebulous/62/db/469281316.gpc1:20100917:o5456g0124o:o5456g0124o.ota12.burn.tbl /data/ipp012.0/nebulous/62/db/469256183.gpc1:20100917:o5456g0124o:o5456g0124o.ota12.burn.tbl
(ippc17:~ipp/stdscience) bills% nls -l -a neb://ipp032.0/gpc1/20100917/o5456g0124o/o5456g0124o.ota47.burn.tbl
-rw-rw-r-- 1 ipp nebulous 0 Sep 27 06:55 /data/ipp012.0/nebulous/24/59/469255422.gpc1:20100917:o5456g0124o:o5456g0124o.ota47.burn.tbl
-rw-rw-r-- 1 apache nebulous 431964 Sep 27 07:17 /data/ipp042.0/nebulous/24/59/469280580.gpc1:20100917:o5456g0124o:o5456g0124o.ota47.burn.tbl
(ippc17:~ipp/stdscience) bills% cp /data/ipp042.0/nebulous/24/59/469280580.gpc1:20100917:o5456g0124o:o5456g0124o.ota47.burn.tbl /data/ipp012.0/nebulous/24/59/469255422.gpc1:20100917:o5456g0124o:o5456g0124o.ota47.burn.tbl
(ippc17:~ipp/stdscience) bills% nls -l -a neb://ipp009.0/gpc1/20100917/o5456g0127o/o5456g0127o.ota12.burn.tbl
-rw-rw-r-- 1 ipp nebulous 0 Sep 27 06:55 /data/ipp012.0/nebulous/0d/da/469257299.gpc1:20100917:o5456g0127o:o5456g0127o.ota12.burn.tbl
-rw-rw-r-- 1 apache nebulous 355784 Sep 27 07:18 /data/ipp044.0/nebulous/0d/da/469281330.gpc1:20100917:o5456g0127o:o5456g0127o.ota12.burn.tbl
(ippc17:~ipp/stdscience) bills% cp /data/ipp044.0/nebulous/0d/da/469281330.gpc1:20100917:o5456g0127o:o5456g0127o.ota12.burn.tbl /data/ipp012.0/nebulous/0d/da/469257299.gpc1:20100917:o5456g0127o:o5456g0127o.ota12.burn.tbl
(ippc17:~ipp/stdscience) bills% nls -l -a neb://ipp040.0/gpc1/20100917/o5456g0162o/o5456g0162o.ota15.burn.tbl 
-rw-rw-r-- 1 ipp nebulous 0 Sep 27 06:53 /data/ipp012.0/nebulous/48/22/469255379.gpc1:20100917:o5456g0162o:o5456g0162o.ota15.burn.tbl
-rw-rw-r-- 1 apache nebulous 298715 Sep 27 07:18 /data/ipp011.0/nebulous/48/22/469281301.gpc1:20100917:o5456g0162o:o5456g0162o.ota15.burn.tbl
(ippc17:~ipp/stdscience) bills% cp /data/ipp011.0/nebulous/48/22/469281301.gpc1:20100917:o5456g0162o:o5456g0162o.ota15.burn.tbl /data/ipp012.0/nebulous/48/22/469255379.gpc1:20100917:o5456g0162o:o5456g0162o.ota15.burn.tbl
(ippc17:~ipp/stdscience) bills% nls -l -a neb://ipp040.0/gpc1/20100917/o5456g0172o/o5456g0172o.ota15.burn.tbl
-rw-rw-r-- 1 ipp nebulous 312007 Feb  7 19:22 /data/ipp012.0/nebulous/2c/ab/469256190.gpc1:20100917:o5456g0172o:o5456g0172o.ota15.burn.tbl
-rw-rw-r-- 1 apache nebulous 312007 Sep 27 07:18 /data/ipp053.0/nebulous/2c/ab/469281313.gpc1:20100917:o5456g0172o:o5456g0172o.ota15.burn.tbl
(ippc17:~ipp/stdscience) bills% nls -l -a  neb://ipp024.0/gpc1/20100917/o5456g0192o/o5456g0192o.ota13.fits
-rw-rw-r-- 1 apache nebulous 22423680 Sep 16 22:17 /data/ipp024.0/nebulous/e1/67/451765379.gpc1:20100917:o5456g0192o:o5456g0192o.ota13.fits
-rw-rw-r-- 1 apache nebulous 22423680 Sep 16 22:17 /data/ipp033.0/nebulous/e1/67/451765381.gpc1:20100917:o5456g0192o:o5456g0192o.ota13.fits
(ippc17:~ipp/stdscience) bills% chiptool -revertprocessedimfile -label STS.201009


2011-01-26

  • Detect streaks for magic_id 115722 fails due to corrupt input file. Fixed with
perl ~bills/ipp/tools/rundiffskycell.pl --redirect-output --dbname gpc1 --diff_id 104225 --skycell_id skycell.0931.126

2010-11-23

  • Destreak magic_ds_id 332055 diff stage diff_id 94387 skycell.1147.020
    From Log file
    
    failed to read /data/ipp036.0/nebulous/dd/d0/554710910.gpc1:ThreePi.nightlyscience:2010:11:22:RINGS.V0:skycell.1147.020:RINGS.V0.skycell.1147.020.dif.94387.inv.fits (PPSUB.INPUT)
    
    Confirm with funpack
    
    
    (ipp053:~ipp/distribution) bills% checkfits  /data/ipp036.0/nebulous/dd/d0/554710910.gpc1:ThreePi.nightlyscience:2010:11:22:RINGS.V0:skycell.1147.020:RINGS.V0.skycell.1147.020.dif.94387.inv.fits
    /data/ipp036.0/nebulous/dd/d0/554710910.gpc1:ThreePi.nightlyscience:2010:11:22:RINGS.V0:skycell.1147.020:RINGS.V0.skycell.1147.020.dif.94387.inv.fits is corrupt.
    
    
    Repaired with:
    
    ipp@ipp036:/home/panstarrs/ipp>perl ~bills/ipp/tools/rundiffskycell.pl --redirect-output --diff_id 94387 --skycell_id skycell.1147.020
    
    

  • Warp run failed due to corrupt camera output.
    From log file
     -> pmFPAfileRead (pmFPAfileIO.c:241): unknown psLib error
         failed to read /data/ipp046.0/nebulous/bd/c0/556055971.gpc1:ThreePi.nt:2010:11:23:o5523g0071o.260118:o5523g0071o.260118.cm.146194.XY71.mk.fits (PSWARP.MASK)
    
    Repaired with 
    ipp@ipp046:/home/panstarrs/ipp>perl ~bills/ipp/tools/runcameraexp.pl --redirect-output --cam_id 146194
    command to process this camRun
    camera_exp.pl --exp_tag o5523g0071o.260118 --cam_id 146194 --camera GPC1 --outroot neb://any/gpc1/ThreePi.nt/2010/11/23/o5523g0071o.260118/o5523g0071o.260118.cm.146194 --run-state new  --redirect-output --no-update --verbose --dbname gpc1
    
    

2010-11-27

  • Chip run for older MD02 image fails due to bad burntool table
    From log file
    Running [/home/panstarrs/ipp/psconfig/ipp-20101029.lin64/bin/funpack -S /data/ipp032.0/nebulous/84/50/117904636.gpc1:20091227:o5192g0138o:o5192g0138o.ota73.fits > /tmp/chip.114762.XY73.deburned.YPlB.fits]...
    Couldn't find burntool table: neb://ipp015.0/gpc1/20091227/o5192g0138o/o5192g0138o.ota73.burn.tbl at /home/panstarrs/ipp/psconfig//ipp-20101029.lin64/bin/chip_imfile.pl line 476
    
    Check files
    (ippc18:~) bills% nls -a -l neb://ipp015.0/gpc1/20091227/o5192g0138o/o5192g0138o.ota73.burn.tbl
    -rw-rw-r-- 1 apache nebulous 0 Mar  2  2010 /data/ipp017.0/nebulous/b2/49/118675808.gpc1:20091227:o5192g0138o:o5192g0138o.ota73.burn.tbl
    -rw-rw-r-- 1 apache nebulous 23098 Dec 31  2009 /data/ipp028.0/nebulous/b2/49/120108035.gpc1:20091227:o5192g0138o:o5192g0138o.ota73.burn.tbl
    
    There are two instances. The replicant is empty.
    
    Fixed by copying good one over bad
    
    (ippc18:~) bills% cp /data/ipp028.0/nebulous/b2/49/120108035.gpc1:20091227:o5192g0138o:o5192g0138o.ota73.burn.tbl /data/ipp017.0/nebulous/b2/49118675808.gpc1:20091227:o5192g0138o:o5192g0138o.ota73.burn.tbl
    (ippc18:~) bills% chiptool -revertprocessedimfile -chip_id 165205
    
    
    

2010-12-08

  • exp_name o5255g0538o class_id XY53 both instances of burntool table are empty.
(ippc17:panstarrs/ipp/update) bills% nls neb://ipp033.0/gpc1/20100228/o5255g0538o/o5255g0538o.ota53.burn.tbl -l -a
-rw-rw-r-- 1 apache nebulous 0 Feb 28  2010 /data/ipp033.0/nebulous/bb/53/178126786.gpc1:20100228:o5255g0538o:o5255g0538o.ota53.burn.tbl
-rw-rw-r-- 1 apache nebulous 0 Mar  3  2010 /data/ipp039.0/nebulous/bb/53/183062726.gpc1:20100228:o5255g0538o:o5255g0538o.ota53.burn.tbl


2010-12-15

  • warp_id 136656 skycell_id skycell.079 corrupt mask file (won't decompress) found during stacking. Fixed with
perl ~bills/ipp/tools/runwarpskycell.pl --warp_id 136656 --skycell_id skycell.079 --redirect-output --dbname gpc1

Happy New Year 2011-01-05

  • warp_id 139171 has several skycells that fail due to corrupt camera mask files from cam_id 152676 Fixed with
perl ~bills/ipp/tools/runcameraexp.pl --dbname gpc1 --redirect-output --cam_id 152676

  • diff_id 99149 skycell_id skycell.1162.062 fails repeatedly due to corrupted input file. Fixed with
perl ~bills/ipp/tools/runwarpskycell.pl --dbname gpc1 --redirect-output --warp_id 142806 --skycell_id skycell.1162.062

  • magic failed due to corrupted variance file from diff_id 102590 skycell.1771.069 fixed with rundiffskyfile

  • 3 chips failed from MD05.refstack.20110122. The root cause was empty instances of burntool tables. Luckily the other instances looked ok. Hopefully fixed with the following sequence of commands
Note nls is a script of mine that I find useful uses neb-locate to examine the status of files. Once resolving the 
files it dereferences them using ls

(ipp004:~ipp/stdscience) bills% nls -a -l neb://ipp050.0/gpc1/20100118/o5214g0180o/o5214g0180o.ota60.burn.tbl -a
-rw-rw-r-- 1 apache nebulous 0 Feb  9  2010 /data/ipp031.0/nebulous/a0/9f/129847054.gpc1:20100118:o5214g0180o:o5214g0180o.ota60.burn.tbl
-rw-rw-r-- 1 apache nebulous 179589 Jan 18  2010 /data/ipp010.0/nebulous/a0/9f/129894067.gpc1:20100118:o5214g0180o:o5214g0180o.ota60.burn.tbl
(ipp004:~ipp/stdscience) bills% cp /data/ipp010.0/nebulous/a0/9f/129894067.gpc1:20100118:o5214g0180o:o5214g0180o.ota60.burn.tbl /data/ipp031.0/nebulous/a0/9f/129847054.gpc1:20100118:o5214g0180o:o5214g0180o.ota60.burn.tbl

(ipp004:~ipp/stdscience) bills% !nls
nls -l neb://ipp050.0/gpc1/20100118/o5214g0180o/o5214g0180o.ota60.burn.tbl -a
-rw-rw-r-- 1 apache nebulous 179589 Jan 22 15:03 /data/ipp031.0/nebulous/a0/9f/129847054.gpc1:20100118:o5214g0180o:o5214g0180o.ota60.burn.tbl
-rw-rw-r-- 1 apache nebulous 179589 Jan 18  2010 /data/ipp010.0/nebulous/a0/9f/129894067.gpc1:20100118:o5214g0180o:o5214g0180o.ota60.burn.tbl


(ipp004:~ipp/stdscience) bills% nls -l -a neb://ipp026.0/gpc1/20100118/o5214g0181o/o5214g0181o.ota16.burn.tbl
-rw-rw-r-- 1 apache nebulous 0 Feb  9  2010 /data/ipp036.0/nebulous/30/ed/129840926.gpc1:20100118:o5214g0181o:o5214g0181o.ota16.burn.tbl
-rw-rw-r-- 1 apache nebulous 230306 Jan 18  2010 /data/ipp044.0/nebulous/30/ed/129868504.gpc1:20100118:o5214g0181o:o5214g0181o.ota16.burn.tbl
(ipp004:~ipp/stdscience) bills% cp /data/ipp044.0/nebulous/30/ed/129868504.gpc1:20100118:o5214g0181o:o5214g0181o.ota16.burn.tbl /data/ipp036.0/nebulous/30/ed/129840926.gpc1:20100118:o5214g0181o:o5214g0181o.ota16.burn.tbl

(ipp004:~ipp/stdscience) bills% nls -l -a neb://ipp050.0/gpc1/20100118/o5214g0192o/o5214g0192o.ota60.burn.tbl
-rw-rw-r-- 1 apache nebulous 0 Feb  9  2010 /data/ipp031.0/nebulous/17/3f/129847347.gpc1:20100118:o5214g0192o:o5214g0192o.ota60.burn.tbl
-rw-rw-r-- 1 apache nebulous 188268 Jan 18  2010 /data/ipp043.0/nebulous/17/3f/129894862.gpc1:20100118:o5214g0192o:o5214g0192o.ota60.burn.tbl
(ipp004:~ipp/stdscience) bills% cp /data/ipp043.0/nebulous/17/3f/129894862.gpc1:20100118:o5214g0192o:o5214g0192o.ota60.burn.tbl /data/ipp031.0/nebulous/17/3f/129847347.gpc1:20100118:o5214g0192o:o5214g0192o.ota60.burn.tbl


Note: As of 2011-01-26 enter new faults at the top of this page

Note: See TracWiki for help on using the wiki.