Changes between Initial Version and Version 1 of ippToPspsSa10Vacation

Timestamp:

Oct 27, 2012, 12:10:39 PM (14 years ago)

Author:

heather

Comment:

--

ippToPspsSa10Vacation

@@ +1 @@
+Here are the instructions for Bill/Gene/anyone else to build the dvo, fix ipptopsp and run ipptopsps: There are far more instructions on [wiki:ippToPsps], these are just the quick version with specifics for SA10:
+
+== IPPTOPSPS INSTRUCTIONS ==
+
+There are a number of columns to be fixed, which I haven't had a chance to get to yet (and likely won't). Here is a table of remaining changes
+
+to be checked/verified for ipptopsps. These live here:
+{{{
+~ipp/src/ippTopsps/jython/
+stackbatch.py
+objectbatch.py
+detectionbatch.py
+}}}
+
+ * these need to be checked in stack:
+   * !StackDetection
+     * psfFlux/psfFluxErr (formerly instFlux/instFluxErr) - was unclear if this was just a name change or do we grab something else instead?
+     * peakFlux -> covert to adu/s
+     * sky/skyErr -> convert to adu/s/pixel
+     * sgSep -> needs a formula
+     * psfLikelihood -> fhe formula was modded, (see wiki)
+     * apFlux/apFluxErr -> these were once mags, need to be converted to flux, also needs exptime correction
+   * !StackApFlx
+     * isophotMag/Err -> were mags, need to be converted to flux
+     * pefFlux/Err -> were mags, need to be converted to flux
+   * !StackModelFit
+     * devFlux/Err -> were mags, need to be converted to flux
+     * expFlux/Err -> were mags, need to be converted to flux
+     * serFlux/Err -> were mags, need to be converted to flux
+   * !stackDetectionCalib
+     * need ra/dec/raErr/decErr to be populated
+     * need zp/zpErr to be populated
+     * need expTime to be populated
+     * need airMass to be populated
+ * these need to be checked in detection:
+   * Detection
+     * psfFlux/psfFluxErr (formerly instFlux/instFluxErr) - was unclear if this was just a name change or do we grab something else instead?
+     * peakFlux -> covert to adu/s
+     * sky/skyErr -> convert to adu/s/pixel
+     * sgSep -> needs a formula
+     * psfLikelihood -> fhe formula was modded, (see wiki)
+     * apFlux/apFluxErr -> these were once mags, need to be converted to flux
+   * !DetectionCalib
+     * need ra/dec/raErr/decErr to be populated
+     * need zp/zpErr to be populated
+     * need expTime to be populated
+     * need airMass to be populated
+   * Object:
+     * [filter]stackDetectID needs to be populated - I think the only other change was a flux to a mag and I did that already
+
+
+
+I advise to make some tests of these to make sure that the batches contain what you want and such.  There is already a config for this, SAS.2120905.sa10.test1 (yes, typo in the date). The config is located in the ipptopsps database, (ippdb01, same user/pass as gpc1), and you can see it here:
+
+{{{
+mysql> select * from config where name = 'SAS.2120905.sa10.test1';
++---------------------+------------------------+-------------------+----------------+-------------------+--------------
++------------------------------------------+--------+--------+---------+---------+----------+---------------------------
++--------------+--------------+------------------+----------------+---------------------+--------+-------------+----------
++----------+----------+--------+
+| timestamp           | name                   | datastore_product | datastore_type | datastore_publish | dvo_label    
+| dvo_location                             | min_ra | max_ra | min_dec | max_dec | box_size | base_path                 
+| data_release | delete_local | delete_datastore | delete_dxlayer | epoch               | survey | psps_survey | queue_P2 
+| queue_ST | queue_OB | active |
++---------------------+------------------------+-------------------+----------------+-------------------+--------------
++------------------------------------------+--------+--------+---------+---------+----------+---------------------------
++--------------+--------------+------------------+----------------+---------------------+--------+-------------+----------
++----------+----------+--------+
+| 2012-10-26 11:51:12 | SAS.2120905.sa10.test1 | PSPS_test         | IPP_PSPS       |                 1 | SAS.20120905 
+| /data/ipp005.0/gpc1/catdirs/SAS/20120901 |    327 |    341 |      -7 |       7 |        2 | /data/ipp005.0/ipptopsps/ 
+|           10 |            1 |                1 |              1 | 2012-10-26 11:53:00 | 3PI    | SA10        |        0 
+|        0 |        1 |      1 | 
++---------------------+------------------------+-------------------+----------------+-------------------+--------------
++------------------------------------------+--------+--------+---------+---------+----------+---------------------------
++--------------+--------------+------------------+----------------+---------------------+--------+-------------+----------
++----------+----------+--------+
+1 row in set (0.00 sec)
+}}}
+
+==  testing ipptopsps changes ==
+
+you can turn off/on queue_P2/queue_ST/queue_OB as needed, and to test do the following as ipp in ~/src/ippToPsps/jython on ipp005 or ipp006
+
+{{{
+psconfig ipptrunk-20120913
+./run.sh queue.py SAS.2120905.SA10.test1
+./run.sh loader.py SAS.2120905.SA10.test1
+}}}
+
+The first command changes the psconfig to grab the proper libraries for dvograbber
+
+The second command makes a list of what needs to be made into batches (so if you turn off/on ST/P2/OB it will only pick up the ones that are on) 
+
+The third command starts loading
+
+The batches get placed on http://datastore.ipp.ifa.hawaii.edu/PSPS_test/, you can pick them up from there and play with them.
+
+
+=== do psconfig ipptrunk-20120913 prior to any of the ./run.py commands - this picks up the correct libraries for dvo. ===
+
+== recompiling for the new SAS ==
+ there will be a dvo schema change - there are 2 ways to work with this:
+  * Step 1:
+   * update the tag and recompile the tag
+   * check out a copy of the truck and access it by something like ipptrunk-2012date
+  * Step 2: once you are linked to the proper dvo library, recompile dvograbber (in ~/src/ippTopsps/src) I honestly don't remember how I did this, wheter from ~src/ippToPsps or from the directory that contains dvograbber, and what if any things I did to make sure the make files picked up the correct libraries. I did stuff multiple times... as ipp on ipp005 you can test it like this: ../src/dvograbber ../config/settings.xml ipptopsps_scratch3 /data/ipp005.0/gpc1/catdirs/SAS/20120901 to see if it works (change to new SAS directory)
+  * Step 3: once you verify dvograbber works it should be ok... I hope...
+
+== queueing the new SAS ==
+
+=== MAKE sure you have used the proper psconfig for this otherwise it will not work (dvograbber will fail!!) ===
+  * do psconfig ipptrunk-20120913 for testing
+  * do psconfig ipptrunk-(whatever) for the actual loading for PSPS (see above section)
+
+
+=== Step 1: make new config ===
+
+{{{
+./run.sh queue.py edit
+}}}
+answer questions, basically give it the same answers as for SAS.2120905.sa10.test1 with a different name/ dvo_label / dvo_location, and put it on datastore_product = PSPS_JHU, and tell it to queue everything (P2/ST/OB). for epoch give it whatever the current time is - I don't understand why but that's the rules from Roy. 
+
+if you make errors correct it by going into mysql and editing that entry in config table 
+
+=== Step 2: make an init batch (only do this once) ===
+{{{
+./run.sh loader.py (configname) init
+}}}
+this should just work - i've not had problems in the past.
+
+=== Step 3: start loading. ===
+ you can do this step multiple times to load on multiple machines. There are certain machines that can do loading, but I recommend doing it on ipp005 and ipp006 (up to 3 on ipp005, and I think 2-3 times on ipp006)
+{{{
+./run.sh loader.py (configname)
+}}}
+
+== trouble shooting ==
+
+=== reverting errors ===
+
+First is how do you tell? There is funny logic here. Preferably you will know how many OB/ST/P2 batches there are. Do queries in addRun to find out how many cam/stacks there are (cam should be 620, stacks are a little variable due to those few that don't staticsky) objects = number of cpt files in SAS dvodb.  Do a query in the ipptopsps database to get counts of stuff (example from SAS.v9):
+{{{
+mysql> select batch_type, count(*) from batch where processed = 1 and loaded_to_datastore =1 and dvo_db = 'SAS.20120905' and datastore_product = 'PSPS_JHU' group by batch_type;
++------------+----------+
+| batch_type | count(*) |
++------------+----------+
+| IN         |        1 | 
+| OB         |      368 | 
+| P2         |      620 | 
+| ST         |     2967 | 
++------------+----------+
+4 rows in set (0.12 sec)
+}}}
+those counts should add up to what you expect (it does for sas_v9). Note that you can't just count up the ones that have not processed, because once a batch fails it is forgotten about (we don't recycle batch numbers, we just make a new batch with the same contents):
+
+{{{
+mysql> select batch_type, count(*) from batch where dvo_db = 'SAS.20120905' and datastore_product = 'PSPS_JHU' group by batch_type, processed;+------------+----------+
+| batch_type | count(*) |
++------------+----------+
+| IN         |        1 | 
+| OB         |      123 | 
+| OB         |      368 | 
+| P2         |      620 | 
+| ST         |     2967 | 
++------------+----------+
+5 rows in set (0.13 sec)
+}}}
+Those 123 OBs failed during some craziness the last time (always due to nfs/system/neb/mysql hiccups) and were requeued... Always only trust the final count of what's been processed and loaded_to_datastore...
+
+To try again, requeue and reload:
+
+{{{
+./run.sh queue.py SAS.(name)
+./run.sh loader.py SAS.(name)
+}}}
+
+it might take a couple of iterations, depending on the nature of the error. check database to verify it has all.
+ 
+=== logs ===
+  These live in /data/ipp005.0/ipptopsps/logs
+
+=== databases ===
+  * the main ipptopsps database has information about the config and the batches.
+  * each loader has a database as well on teh machines they are running on:
+    mysql -h localhost -u ipp ipptopsps_scratch (or ipptopsps_scratch2 or ipptopsps_scratch3)
+    useful to diagnose stuff when working on ipptopsps
+
+
+== IPPDVO recompile of ipp trunk instructions ==
+
+ippdvo runs off of the trunk.  To differentiate I call them by the date that I cecked out the trunk...  So once the changes are in and it is ready to do, make adirectory called src/ipptrunk-201210day check out and compile the trunk, and edit .tcshrc to use that psconfig instead. I keep the old ones around just in case...
+
+
+== ADDSTAR INSTRUCTIONS ==
+
+This is to be run on addstarlap (ipp005) since that is where the other SASs live
+
+I was pondering about just reusing the old cam stage from previously, but I don't necessarily trust that since I think Gene changed some stuff in the dvo, also, it's substantially easier to explain without queueing older minidvodbs, and also, building 620 cam stages isn't going to take too long. 
+
+I generally build the cam stage first and then the staticsky stage. For simplicity you can do both simultaneously (I don't think it matters):
+
+to queue the minidvodb stuff:
+{{{
+  survey.add.addstar SAS.cam SAS_v10 SAS.20121027 SAS.20121027 cam
+  survey.add.addstar SAS.staticsky SAS_v10 SAS.20121027 SAS.20121027 staticsky
+  add.minidvodb  SAS.20121027 SAS.20121027.MINI SAS.20121027 1 GPC1
+  add.label SAS.20121027
+}}}
+once you are satisfied that staticsky is working properly and doing the right stuff, you can turn on creation/merge of the merge dvodb:
+{{{
+  add.mergedvodb SAS.20121027
+  survey.add.mergedvodb SAS.20121027 SAS.20121027 SAS.20121027
+}}}
+Note that once the staticskys have completed, there will still be that last minidvodb that is in 'active', since it kicks to be merged after a day. If all the staticskys have been addstarred, you can set the state (with mysql) of the current active minidvodb for this to the state 'to_be_merged'  which will cause it to go get merged (wait for it...)
+
+Once the merge dvodb is built, it needs to be 'spiffed':
+
+These are what I used last time (change SAS as needed)
+{{{
+relphot -averages -reset -update -boundary-tree ~ipp/RINGS.V3.tree.fits -D CAMERA gpc1 -D CATDIR /data/ipp005.0/gpc1/catdirs/SAS/20120901
+relastro -update-objects -reset -update -D CAMERA gpc1 -D CATDIR /data/ipp005.0/gpc1/catdirs/SAS/20120901
+}}}
+
+Please verify with Gene that these are the commands.
+
+