Here are the instructions for Bill/Gene/anyone else to build the dvo, fix ipptopsp and run ipptopsps: There are far more instructions on ippToPsps, these are just the quick version with specifics for SA10:

IPPTOPSPS INSTRUCTIONS

There are a number of columns to be fixed, which I haven't had a chance to get to yet (and likely won't). Here is a table of remaining changes

to be checked/verified for ipptopsps. These live here:

~ipp/src/ippTopsps/jython/
stackbatch.py
objectbatch.py
detectionbatch.py

• these need to be checked in stack:
  • StackDetection
    • psfFlux/psfFluxErr (formerly instFlux/instFluxErr) - was unclear if this was just a name change or do we grab something else instead?
    • peakFlux -> covert to adu/s
    • sky/skyErr -> convert to adu/s/pixel
    • sgSep -> needs a formula
    • psfLikelihood -> fhe formula was modded, (see wiki)
    • apFlux/apFluxErr -> these were once mags, need to be converted to flux, also needs exptime correction
  • StackApFlx
    • isophotMag/Err -> were mags, need to be converted to flux
    • pefFlux/Err -> were mags, need to be converted to flux
  • StackModelFit
    • devFlux/Err -> were mags, need to be converted to flux
    • expFlux/Err -> were mags, need to be converted to flux
    • serFlux/Err -> were mags, need to be converted to flux
  • !stackDetectionCalib
    • need ra/dec/raErr/decErr to be populated
    • need zp/zpErr to be populated
    • need expTime to be populated
    • need airMass to be populated
• these need to be checked in detection:
  • Detection
    • psfFlux/psfFluxErr (formerly instFlux/instFluxErr) - was unclear if this was just a name change or do we grab something else instead?
    • peakFlux -> covert to adu/s
    • sky/skyErr -> convert to adu/s/pixel
    • sgSep -> needs a formula
    • psfLikelihood -> fhe formula was modded, (see wiki)
    • apFlux/apFluxErr -> these were once mags, need to be converted to flux
  • DetectionCalib
    • need ra/dec/raErr/decErr to be populated
    • need zp/zpErr to be populated
    • need expTime to be populated
    • need airMass to be populated
  • Object:
    • [filter]stackDetectID needs to be populated - I think the only other change was a flux to a mag and I did that already

I advise to make some tests of these to make sure that the batches contain what you want and such. There is already a config for this, SAS.2120905.sa10.test1 (yes, typo in the date). The config is located in the ipptopsps database, (ippdb01, same user/pass as gpc1), and you can see it here:

mysql> select * from config where name = 'SAS.2120905.sa10.test1';
+---------------------+------------------------+-------------------+----------------+-------------------+--------------
+------------------------------------------+--------+--------+---------+---------+----------+---------------------------
+--------------+--------------+------------------+----------------+---------------------+--------+-------------+----------
+----------+----------+--------+
| timestamp           | name                   | datastore_product | datastore_type | datastore_publish | dvo_label    
| dvo_location                             | min_ra | max_ra | min_dec | max_dec | box_size | base_path                 
| data_release | delete_local | delete_datastore | delete_dxlayer | epoch               | survey | psps_survey | queue_P2 
| queue_ST | queue_OB | active |
+---------------------+------------------------+-------------------+----------------+-------------------+--------------
+------------------------------------------+--------+--------+---------+---------+----------+---------------------------
+--------------+--------------+------------------+----------------+---------------------+--------+-------------+----------
+----------+----------+--------+
| 2012-10-26 11:51:12 | SAS.2120905.sa10.test1 | PSPS_test         | IPP_PSPS       |                 1 | SAS.20120905 
| /data/ipp005.0/gpc1/catdirs/SAS/20120901 |    327 |    341 |      -7 |       7 |        2 | /data/ipp005.0/ipptopsps/ 
|           10 |            1 |                1 |              1 | 2012-10-26 11:53:00 | 3PI    | SA10        |        0 
|        0 |        1 |      1 | 
+---------------------+------------------------+-------------------+----------------+-------------------+--------------
+------------------------------------------+--------+--------+---------+---------+----------+---------------------------
+--------------+--------------+------------------+----------------+---------------------+--------+-------------+----------
+----------+----------+--------+
1 row in set (0.00 sec)

testing ipptopsps changes

you can turn off/on queue_P2/queue_ST/queue_OB as needed, and to test do the following as ipp in ~/src/ippToPsps/jython on ipp005 or ipp006

psconfig ipptrunk-20120913
./run.sh queue.py SAS.2120905.SA10.test1
./run.sh loader.py SAS.2120905.SA10.test1

The first command changes the psconfig to grab the proper libraries for dvograbber

The second command makes a list of what needs to be made into batches (so if you turn off/on ST/P2/OB it will only pick up the ones that are on)

The third command starts loading

The batches get placed on ​http://datastore.ipp.ifa.hawaii.edu/PSPS_test/, you can pick them up from there and play with them.

do psconfig ipptrunk-20120913 prior to any of the ./run.py commands - this picks up the correct libraries for dvo.

recompiling for the new SAS

there will be a dvo schema change - there are 2 ways to work with this:

• Step 1:
  • update the tag and recompile the tag
  • check out a copy of the truck and access it by something like ipptrunk-2012date
• Step 2: once you are linked to the proper dvo library, recompile dvograbber (in ~/src/ippTopsps/src) I honestly don't remember how I did this, wheter from ~src/ippToPsps or from the directory that contains dvograbber, and what if any things I did to make sure the make files picked up the correct libraries. I did stuff multiple times... as ipp on ipp005 you can test it like this: ../src/dvograbber ../config/settings.xml ipptopsps_scratch3 /data/ipp005.0/gpc1/catdirs/SAS/20120901 to see if it works (change to new SAS directory)
• Step 3: once you verify dvograbber works it should be ok... I hope...

queueing the new SAS

MAKE sure you have used the proper psconfig for this otherwise it will not work (dvograbber will fail!!)

• do psconfig ipptrunk-20120913 for testing
• do psconfig ipptrunk-(whatever) for the actual loading for PSPS (see above section)

Step 1: make new config

./run.sh queue.py edit

answer questions, basically give it the same answers as for SAS.2120905.sa10.test1 with a different name/ dvo_label / dvo_location, and put it on datastore_product = PSPS_JHU, and tell it to queue everything (P2/ST/OB). for epoch give it whatever the current time is - I don't understand why but that's the rules from Roy.

if you make errors correct it by going into mysql and editing that entry in config table

Step 2: make an init batch (only do this once)

./run.sh loader.py (configname) init

this should just work - i've not had problems in the past.

Step 3: start loading.

you can do this step multiple times to load on multiple machines. There are certain machines that can do loading, but I recommend doing it on ipp005 and ipp006 (up to 3 on ipp005, and I think 2-3 times on ipp006)

./run.sh loader.py (configname)

trouble shooting

reverting errors

First is how do you tell? There is funny logic here. Preferably you will know how many OB/ST/P2 batches there are. Do queries in addRun to find out how many cam/stacks there are (cam should be 620, stacks are a little variable due to those few that don't staticsky) objects = number of cpt files in SAS dvodb. Do a query in the ipptopsps database to get counts of stuff (example from SAS.v9):

mysql> select batch_type, count(*) from batch where processed = 1 and loaded_to_datastore =1 and dvo_db = 'SAS.20120905' and datastore_product = 'PSPS_JHU' group by batch_type;
+------------+----------+
| batch_type | count(*) |
+------------+----------+
| IN         |        1 | 
| OB         |      368 | 
| P2         |      620 | 
| ST         |     2967 | 
+------------+----------+
4 rows in set (0.12 sec)

those counts should add up to what you expect (it does for sas_v9). Note that you can't just count up the ones that have not processed, because once a batch fails it is forgotten about (we don't recycle batch numbers, we just make a new batch with the same contents):

mysql> select batch_type, count(*) from batch where dvo_db = 'SAS.20120905' and datastore_product = 'PSPS_JHU' group by batch_type, processed;+------------+----------+
| batch_type | count(*) |
+------------+----------+
| IN         |        1 | 
| OB         |      123 | 
| OB         |      368 | 
| P2         |      620 | 
| ST         |     2967 | 
+------------+----------+
5 rows in set (0.13 sec)

Those 123 OBs failed during some craziness the last time (always due to nfs/system/neb/mysql hiccups) and were requeued... Always only trust the final count of what's been processed and loaded_to_datastore...

To try again, requeue and reload:

./run.sh queue.py SAS.(name)
./run.sh loader.py SAS.(name)

it might take a couple of iterations, depending on the nature of the error. check database to verify it has all.

logs

These live in /data/ipp005.0/ipptopsps/logs

databases

• the main ipptopsps database has information about the config and the batches.
• each loader has a database as well on teh machines they are running on: mysql -h localhost -u ipp ipptopsps_scratch (or ipptopsps_scratch2 or ipptopsps_scratch3) useful to diagnose stuff when working on ipptopsps

IPPDVO recompile of ipp trunk instructions

ippdvo runs off of the trunk. To differentiate I call them by the date that I cecked out the trunk... So once the changes are in and it is ready to do, make adirectory called src/ipptrunk-201210day check out and compile the trunk, and edit .tcshrc to use that psconfig instead. I keep the old ones around just in case...

ADDSTAR INSTRUCTIONS

This is to be run on addstarlap (ipp005) since that is where the other SASs live

I was pondering about just reusing the old cam stage from previously, but I don't necessarily trust that since I think Gene changed some stuff in the dvo, also, it's substantially easier to explain without queueing older minidvodbs, and also, building 620 cam stages isn't going to take too long.

I generally build the cam stage first and then the staticsky stage. For simplicity you can do both simultaneously (I don't think it matters):

Bill edits: We are using the skycal stage not staticsky. I have updated the following to reflect that

to queue the minidvodb stuff:

  survey.add.addstar SAS.cam SAS_v8 SAS.20121029 SAS.20121029 cam
  survey.add.addstar SAS.skycal SAS_v10 SAS.20121029 SAS.20121029 skycal
  add.minidvodb  SAS.20121029 SAS.20121029.MINI SAS.20121029 1 GPC1
  add.label SAS.20121029

once you are satisfied that skycal addstars are working properly and doing the right stuff, you can turn on creation/merge of the merge dvodb:

  add.mergedvodb SAS.20121029
  survey.add.mergedvodb SAS.20121029 SAS.20121029 SAS.20121029

Note that once the skycals have completed, there will still be that last minidvodb that is in 'active', since it kicks to be merged after a day. If all the skycals have been addstarred, you can set the state (with mysql) of the current active minidvodb for this to the state 'to_be_merged' which will cause it to go get merged (wait for it...)

Once the merge dvodb is built, it needs to be 'spiffed':

These are the commands to use. (Edited to use the corrected tree file and to use the v10 catdir name

relphot g,r,i,z,y -nloop 0  -update -boundary-tree ~ipp/RINGS.V3.tree.v1.fits -D CAMERA gpc1 -D CATDIR /data/ipp005.0/gpc1/catdirs/SAS/20121029
relastro -update-objects -reset -update -D CAMERA gpc1 -D CATDIR /data/ipp005.0/gpc1/catdirs/SAS/20121029

Please verify with Gene that these are the commands.