Changeset 34047

Timestamp:

Jun 20, 2012, 12:21:48 PM (14 years ago)

Author:

eugene

Message:

merge changes from trunk

Location:

branches/eam_branches/ipp-20120601

Files:

• dvodist/scripts (copied) (copied from trunk/dvodist/scripts )
• dvodist/scripts/mkdist.20120509.sh (copied) (copied from trunk/dvodist/scripts/mkdist.20120509.sh )
• dvodist/scripts/mkdist.20120525.sh (copied) (copied from trunk/dvodist/scripts/mkdist.20120525.sh )
• dvodist/scripts/mkdist.testcat.sh (copied) (copied from trunk/dvodist/scripts/mkdist.testcat.sh )
• dvodist/scripts/mkindex.sh (copied) (copied from trunk/dvodist/scripts/mkindex.sh )
• dvodist/www-root (copied) (copied from trunk/dvodist/www-root )
• dvodist/www-root/PScolorlogo2.jpg (copied) (copied from trunk/dvodist/www-root/PScolorlogo2.jpg )
• dvodist/www-root/docs.txt (copied) (copied from trunk/dvodist/www-root/docs.txt )
• dvodist/www-root/help.txt (copied) (copied from trunk/dvodist/www-root/help.txt )
• dvodist/www-root/index.php (copied) (copied from trunk/dvodist/www-root/index.php )
• dvodist/www-root/index.txt (copied) (copied from trunk/dvodist/www-root/index.txt )
• dvodist/www-root/ipp.css (copied) (copied from trunk/dvodist/www-root/ipp.css )
• dvodist/www-util (copied) (copied from trunk/dvodist/www-util )
• dvodist/www-util/blank.html (copied) (copied from trunk/dvodist/www-util/blank.html )
• dvodist/www-util/ipp.menu.dat (copied) (copied from trunk/dvodist/www-util/ipp.menu.dat )
• dvodist/www-util/ipp.php (copied) (copied from trunk/dvodist/www-util/ipp.php )
• dvodist/www-util/phptest.php (copied) (copied from trunk/dvodist/www-util/phptest.php )
• ippData (copied) (copied from trunk/ippData )
• ippData/htdocs (copied) (copied from trunk/ippData/htdocs )
• ippData/htdocs/docs.txt (copied) (copied from trunk/ippData/htdocs/docs.txt )
• ippData/htdocs/help.txt (copied) (copied from trunk/ippData/htdocs/help.txt )
• ippData/htdocs/index.info.txt (copied) (copied from trunk/ippData/htdocs/index.info.txt )
• ippData/htdocs/index.php (copied) (copied from trunk/ippData/htdocs/index.php )
• ippData/htdocs/index.txt (copied) (copied from trunk/ippData/htdocs/index.txt )
• ippData/htdocs/ipp.css (copied) (copied from trunk/ippData/htdocs/ipp.css )
• ippData/scripts (copied) (copied from trunk/ippData/scripts )
• ippData/scripts/mkindex.sh (copied) (copied from trunk/ippData/scripts/mkindex.sh )
• ippData/utils (copied) (copied from trunk/ippData/utils )
• ippData/utils/blank.html (copied) (copied from trunk/ippData/utils/blank.html )
• ippData/utils/ipp.menu.dat (copied) (copied from trunk/ippData/utils/ipp.menu.dat )
• ippData/utils/ipp.php (copied) (copied from trunk/ippData/utils/ipp.php )
• ippData/utils/phptest.php (copied) (copied from trunk/ippData/utils/phptest.php )
• ippMonitor (modified) (1 prop)
• ippMonitor/INSTALL (modified) (1 diff)
• ippMonitor/czartool/czartool/czarconfig.xml (modified) (3 diffs)
• ippMonitor/def/autocode.php (modified) (2 diffs)
• ippMonitor/def/camProcessedExp_Images.d (modified) (2 diffs)
• ippMonitor/def/diffSummary_Images.d (modified) (1 diff)
• ippMonitor/def/newImfile.d (modified) (1 diff)
• ippMonitor/def/pzDoneImfile.d (modified) (1 diff)
• ippMonitor/def/pzPendingImfile.d (modified) (1 diff)
• ippMonitor/def/rawBurntoolState.d (modified) (1 diff)
• ippMonitor/def/rawImfile.d (modified) (1 diff)
• ippMonitor/def/summitImfile.d (modified) (1 diff)
• ippMonitor/raw/czartool_labels.php (modified) (2 diffs)
• ippMonitor/raw/site.php.in (modified) (2 diffs)
• ippMonitor/scripts/generate (modified) (12 diffs)
• ippMonitor/supervision (copied) (copied from trunk/ippMonitor/supervision )
• ippMonitor/supervision/check_if_mysql_slave_is_running_fine.py (copied) (copied from trunk/ippMonitor/supervision/check_if_mysql_slave_is_running_fine.py )
• ippScripts/scripts/addstar_run.pl (modified) (4 diffs)
• ippScripts/scripts/automate_stacks.pl (modified) (1 diff)
• ippScripts/scripts/bundle_detrends.pl (modified) (1 diff)
• ippScripts/scripts/destreak_restore_camera.pl (modified) (1 prop)
• ippScripts/scripts/ipp_apply_burntool_single.pl (modified) (1 prop)
• ippScripts/scripts/magic_destreak.pl (modified) (1 prop)
• ippScripts/scripts/make_burntool_pcontrol.pl (modified) (1 diff)
• ippScripts/scripts/nightly_science.pl (modified) (2 diffs)
• ippScripts/scripts/staticsky.pl (modified) (9 diffs)
• ippTools/share/addtool_find_sky_id_multi.sql (modified) (1 diff)
• ippTools/share/addtool_find_sky_id_multi_dvo.sql (modified) (1 diff)
• ippTools/share/camtool_find_pendingimfile.sql (modified) (1 prop)
• ippTools/share/chiptool_setimfiletoupdate.sql (modified) (1 prop)
• ippTools/share/pxadmin_create_tables.sql (modified) (1 prop)
• ippTools/share/warptool_towarped.sql (modified) (1 prop)
• ippTools/src (modified) (1 prop)
• ippTools/src/addtool.c (modified) (5 diffs)
• ippTools/src/magictool.c (modified) (1 prop)
• ippconfig (modified) (1 prop)
• ippconfig/recipes/filerules-mef.mdc (modified) (1 diff)
• ippconfig/recipes/filerules-simple.mdc (modified) (1 diff)
• ippconfig/recipes/filerules-split.mdc (modified) (1 diff)
• ippconfig/recipes/reductionClasses.mdc (modified) (1 prop)

branches/eam_branches/ipp-20120601/ippMonitor/INSTALL

@@ -15 +15 @@
 # note that we have a special version for ippmonitor without $PSCONFIG/$PSCONFDIR values:
 ipp@ippdb03 / ippdb03 example:
- psconfigure --dbhost ippdb01 --dbuser ipp --dbpass ipp --htdocs /var/www/localhost/htdocs --site ~/ippconfig/ippmonitor.config
+# psconfigure --dbhost ippdb01 --dbuser ipp --dbpass ipp --htdocs /var/www/localhost/htdocs --site ~/ippconfig/ippmonitor.config
 
+#ipp@ippdb03 / ippdb01 example:
+ psconfigure --dbhost ippdb03 --dbuser ipp --dbpass ipp --htdocs /var/www/localhost/htdocs --site ~/ippconfig/ippmonitor.config
+
+#
+# Regular installation on ippMonitor.ifa.hawaii.edu
+# Note: we use a 'science databases (replication) slave' and the ippMonitor read-only account to connect to that database
+psconfigure --dbhost scidbs1 --dbuser ippMonitor --dbpass ippMonitor --htdocs /var/www/localhost/htdocs --site ~/ippconfig/ippmonitor.config

branches/eam_branches/ipp-20120601/ippMonitor/czartool/czartool/czarconfig.xml

@@ -21 +21 @@
   <gpc1database>
     <name>gpc1</name>
-    <host>ippdb01</host>
+    <host>scidbs1</host>
     <user>ippMonitor</user>
     <password>ippMonitor</password>
@@ -29 +29 @@
   <czardatabase>
     <name>czardb</name>
-    <host>ippdb01</host>
+    <host>scidbm</host>
     <user>ipp</user>
     <password>ipp</password>
@@ -46 +46 @@
 
 </czarconfig>
+

branches/eam_branches/ipp-20120601/ippMonitor/def/autocode.php

@@ -28 +28 @@
 // ** TABLE RESTRICTIONS **
 
+// Note: the restriction is now moved AFTER the count query
+// unless the RESTRICT_COUNT keyword is used
 if ($restricted == 0) {
-  if ("$UNRESTRICTED" != "none") {
-    $WHERE = "$WHERE $UNRESTRICTED";
+  if ("$RESTRICT_COUNT" != "none") {
+    $WHERE = "$WHERE $RESTRICT_COUNT";
   }
 }
@@ -56 +58 @@
   menu_end();
 }
+
+// Restrict the query if it is necessary
+if ($restricted == 0) {
+  if ( ("$UNRESTRICTED" != "none") && ("$RESTRICT_COUNT" == "none") ) {
+    $WHERE = "$WHERE $UNRESTRICTED";
+  }
+}
+
 // set up the row counter variables
 if ($ID['from']) {

branches/eam_branches/ipp-20120601/ippMonitor/def/camProcessedExp_Images.d

@@ -3 +3 @@
 FILE  camProcessedExp_Images.php
 MENU  ipp.science.dat
+
+UNRESTRICTED AND 0>1
 
 # the following WHERE clauses are added to all queries joined by AND
@@ -10 +12 @@
 WHERE camProcessedExp.cam_id  = camRun.cam_id
 WHERE camProcessedExp.fault = 0
-
 # define image names to be used below
 # IMAGE VAR basename filerule camera class_id

branches/eam_branches/ipp-20120601/ippMonitor/def/diffSummary_Images.d

@@ -3 +3 @@
 FILE  diffSummary_Images.php
 MENU  ipp.stack.dat
+
+UNRESTRICTED AND 0>1
 
 WHERE diffRun.diff_id             = diffSkyfile.diff_id

branches/eam_branches/ipp-20120601/ippMonitor/def/newImfile.d

@@ -4 +4 @@
 MENU  ipp.load.dat
 
+UNRESTRICTED WHERE 0>1
+
 #        field       size  format  name           show   link to         extras
 FIELD    exp_id,        5, %d,     exposure ID

branches/eam_branches/ipp-20120601/ippMonitor/def/pzDoneImfile.d

@@ -3 +3 @@
 FILE  pzDoneImfile.php
 MENU  ipp.load.dat
+
+UNRESTRICTED AND 0>1
 
 WHERE pzDownloadImfile.fault     = 0

branches/eam_branches/ipp-20120601/ippMonitor/def/pzPendingImfile.d

@@ -3 +3 @@
 FILE  pzPendingImfile.php
 MENU  ipp.load.dat
+
+UNRESTRICTED AND 0>1
 
 WHERE summitImfile.exp_name  = pzDownloadExp.exp_name 

branches/eam_branches/ipp-20120601/ippMonitor/def/rawBurntoolState.d

@@ -5 +5 @@
 
 # if no query restrictions are supplied, we want to limit the query or it is extremely long running:
+RESTRICT_COUNT WHERE 0>1
 UNRESTRICTED WHERE exp_id = 0
 

branches/eam_branches/ipp-20120601/ippMonitor/def/rawImfile.d

@@ -5 +5 @@
 
 # HEAD PHP insert_backref ("rawExp.php", "exp_id", $ID['link']);
+
+UNRESTRICTED WHERE 0>1
 
 OP   OP1  $ra * 57.295783

branches/eam_branches/ipp-20120601/ippMonitor/def/summitImfile.d

@@ -3 +3 @@
 FILE  summitImfile.php
 MENU  ipp.load.dat
+
+UNRESTRICTED WHERE 0>1
 
 #        field     size  format  name             show   link to         extras

branches/eam_branches/ipp-20120601/ippMonitor/raw/czartool_labels.php

@@ -171 +171 @@
 echo "<tr><td>Who's using the cluster?</td><td><a href=\"http://ippmonitor.ipp.ifa.hawaii.edu/ippMonitor/clusterMonitor/top.html\"><font color=\"blue\">here</font></a></td></tr>";
 echo "<tr><td>IPP metrics</td><td><a href=\"http://ipp0022.ifa.hawaii.edu/ps1sc/ippMetrics\"><font color=\"blue\">here</font></a></td></tr>";
-echo "<tr><td>Czar log pages</td><td><a href=\"http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/PS1_IPP_CzarLogs\"><font color=\"blue\">here</font></a></td></tr>";
+echo "<tr><td>Czar log pages</td><td><a href=\"http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/PS1_IPP_CzarLogs\"><font color=\"blue\">All</font></a>&nbsp;&nbsp;&nbsp;";
+echo "<a href=\"http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/PS1_IPP_CzarLog_".(date("Ymd")-date("w")+"1")."\">This week</a>";
+echo "</td></tr>";
 echo "<tr><td>Exposure summary</td><td><a href=\"czartool_exposures.php?pass=$pass&proj=$proj\"><font color=\"blue\">here</font></a></td></tr>";
 
@@ -215 +217 @@
       showReplicationsStatus($REPL_HOST_PSTAMP, $REPL_USER_PSTAMP, $REPL_PASSWORD_PSTAMP, $REPL_DBNAME_PSTAMP);
       showReplicationsStatus($REPL_HOST_IPPADMIN, $REPL_USER_IPPADMIN, $REPL_PASSWORD_IPPADMIN, $REPL_DBNAME_IPPADMIN);
+      showReplicationsStatus($REPL_HOST_ISP, $REPL_USER_ISP, $REPL_PASSWORD_ISP, $REPL_DBNAME_ISP);
       echo "</table>";
 

branches/eam_branches/ipp-20120601/ippMonitor/raw/site.php.in

@@ -25 +25 @@
 $REPL_USER_IPPADMIN = "ippMonitor";
 $REPL_PASSWORD_IPPADMIN = "ippMonitor";
-$REPL_DBNAME_IPPADMIN = "ipp001 (gpc1, isp, ippadmin, czardb)";
+$REPL_DBNAME_IPPADMIN = "ipp001 (gpc1, isp, ippadmin, czardb, ipptopsps, megacam, ssp, uip)" ;
 
 $REPL_HOST_NEBULOUS = "ippdb02.IfA.Hawaii.Edu";
 $REPL_USER_NEBULOUS = "ippMonitor";
 $REPL_PASSWORD_NEBULOUS = "ippMonitor";
-$REPL_DBNAME_NEBULOUS = "ippdb02 (Nebulous)";
+$REPL_DBNAME_NEBULOUS = "ippdb02 (nebulous)";
 
 $REPL_HOST_PSTAMP = "ippc19.IfA.Hawaii.Edu";
@@ -40 +40 @@
 $REPL_USER_GPC1 = "ippMonitor";
 $REPL_PASSWORD_GPC1 = "ippMonitor";
-$REPL_DBNAME_GPC1 = "ippdb03 (gpc1, isp, ippadmin, czardb)";
+$REPL_DBNAME_GPC1 = "ippdb03 (czardb, gpc1, ippadmin, ipptopsps, megacam, skycells, ssp, uip)";
+
+$REPL_HOST_ISP = "ippc17.IfA.Hawaii.Edu";
+$REPL_USER_ISP = "ippMonitor";
+$REPL_PASSWORD_ISP = "ippMonitor";
+$REPL_DBNAME_ISP = "ippc17 (isp)";
+
+
 
 ?>

branches/eam_branches/ipp-20120601/ippMonitor/scripts/generate

@@ -29 +29 @@
     &init_key ("MODE");
     &init_key ("UNRESTRICTED");
+    &init_key ("RESTRICT_COUNT");
     &init_key ("TABLE");
     &init_key ("TITLE");
@@ -46 +47 @@
         if ($key eq "TABLE") { ($value) = $value =~ m|^\s*(.+)\s*$|; }
         if ($key eq "TYPE")  { ($value) = $value =~ m|^\s*(.+)\s*$|; }
-        if ($key eq "TOPLOT") { ($value) = $value =~ m|^\s*(.+)\s*$|; }
-        if ($key eq "PLOTTER") { ($value) = $value =~ m|^\s*(.+)\s*$|; }
-        if ($key eq "PLOTTITLE") { 
-            ($value) = $value =~ m|^\s*(.+)\s*$|;
-            $value =~ s/\s/_/g;
-        }
+        if ($key eq "TOPLOT") { ($value) = $value =~ m|^\s*(.+)\s*$|; }
+        if ($key eq "PLOTTER") { ($value) = $value =~ m|^\s*(.+)\s*$|; }
+        if ($key eq "PLOTTITLE") {
+            ($value) = $value =~ m|^\s*(.+)\s*$|;
+            $value =~ s/\s/_/g;
+        }
 
         &set_keypair ($key, $value);
@@ -61 +62 @@
             if ($field =~ m|\S+\s+as\s+\S+|) {
                 ($fieldreal) = $field =~ m|\S+\s+as\s+(\S+)|;
-                #print STDERR "Case 1\n";
+                #print STDERR "Case 1\n";
             } elsif ($field =~ m|\S+\s+AS\s+\S+|) {
-                #print STDERR "field=$field\n";
-                $fieldreal = $field;
-                $fieldreal =~ s/\s+AS.*$//;
+                #print STDERR "field=$field\n";
+                $fieldreal = $field;
+                $fieldreal =~ s/\s+AS.*$//;
                 #($fieldreal) = $field =~ m|\S+\s+as\s+(\S+)|;
-                #print STDERR "Case 2\n";
+                #print STDERR "Case 2\n";
             } else {
                 $fieldreal = $field;
-                #print STDERR "Case 3\n";
+                #print STDERR "Case 3\n";
             }
-            #print STDERR "field=[$field]\tfieldreal=[$fieldreal]\n";
+            #print STDERR "field=[$field]\tfieldreal=[$fieldreal]\n";
 
             if ($VERBOSE) { printf "%-20s %-20s %-20s %-20s %-20s %-20s\n", $field, $name, $format, $show, $link, $extras; }
@@ -141 +142 @@
     &check_key ("MODE", "basic");
     &check_key ("UNRESTRICTED", "none");
+    &check_key ("RESTRICT_COUNT", "none");
     &check_key ("TABLE", "");
     &check_key ("TITLE", "");
@@ -324 +326 @@
 # generate a WHERE test for each field (field != *)
 sub write_table_restrict {
-
     if ($WHERE) {
         print FILE "\$WHERE = \"WHERE $WHERE\";\n";
@@ -348 +349 @@
 }
 
-# Generate GROUP BY statement if any 
+# Generate GROUP BY statement if any
 sub write_group_by {
     print FILE "\$WHERE = check_ordering ('$GROUPS', \$WHERE);\n";
@@ -536 +537 @@
     @extfields = split (/,/, $extras);
     for ($i = 0; $i < @extfields; $i++) {
-        # print STDERR $extfields[$i],"\n";
+        # print STDERR $extfields[$i],"\n";
         ($label, $value, $outline) = &parse_label ($extfields[$i]);
-        # print STDERR "[",$label,"]","[",$value,"]","[",$outline,"\n";
+        # print STDERR "[",$label,"]","[",$value,"]","[",$outline,"\n";
         print FILE "  \$link = \$link . \"&$outline\";\n";
     }
@@ -553 +554 @@
     # print STDERR "#############################\n";
     # foreach my $key ( keys %linkarg ) {
-    #   print STDERR "key: $key, value: $linkarg{$key}\n";
+    #   print STDERR "key: $key, value: $linkarg{$key}\n";
     # }
 
@@ -560 +561 @@
 
     if ($linkarg{$word}) {
-        # print STDERR "Case 1\n";
+        # print STDERR "Case 1\n";
         $linkarg{$word} = "$linkarg{$word},$value";
     } else {
-        # print STDERR "Case 2\n";
+        # print STDERR "Case 2\n";
         $linkarg{$word} = $value;
     }
@@ -585 +586 @@
     # print STDERR "### parse_label ###\n";
     # for ($i = 0; $i < @field; $i++) {
-    #   print STDERR "field[$i]=$field[$i]\n";
+    #   print STDERR "field[$i]=$field[$i]\n";
     # }
     # for ($i = 0; $i < @fieldreal; $i++) {
-    #   print STDERR "fieldreal[$i]=$fieldreal[$i]\n";
+    #   print STDERR "fieldreal[$i]=$fieldreal[$i]\n";
     # }
     # for ($i = 0; $i < @count; $i++) {
-    #   print STDERR "count[$i]=$count[$i]\n";
+    #   print STDERR "count[$i]=$count[$i]\n";
     # }
 
@@ -598 +599 @@
     if ($string =~ m|\S+=\S+|) {
         ($label, $value) = $string =~ m|(\S+)=(\S+)|;
-        # print STDERR "value=$value\n";
+        # print STDERR "value=$value\n";
         if ($value =~ m|^\$|) {
           MATCH_LABEL:
@@ -669 +670 @@
     $name   = &parse_fieldname ($name);
     if (exists $ops{$name}) {
-        #if OPx was defined, replace it by its value
-        $name = $ops{$name};
-        $name =~ s/\s+//g;
+        #if OPx was defined, replace it by its value
+        $name = $ops{$name};
+        $name =~ s/\s+//g;
     }
     $camera = &parse_fieldname ($camera);

branches/eam_branches/ipp-20120601/ippScripts/scripts/addstar_run.pl

@@ -38 +38 @@
 }
 my $minidvodb_path;
-my $stage_extra1 ;
-my ( $add_id, $camera, $stage, $stage_id, $multi_num, $outroot, $stageroot, $dbname, $reduction, $dvodb, $minidvodb, $minidvodb_name, $minidvodb_group, $image_only, $verbose, $no_update,
+
+my ( $add_id, $camera, $stage, $stage_id, $stage_extra1, $outroot, $stageroot, $dbname, $reduction, $dvodb, $minidvodb, $minidvodb_name, $minidvodb_group, $image_only, $verbose, $no_update,
      $no_op, $redirect, $save_temps);
 GetOptions(
@@ -46 +46 @@
     'stage|s=s'        => \$stage, # Camera
     'stage_id|w=s'   => \$stage_id,
-    'multi_num|w=s'    => \$multi_num, # the number for a staticskymulti (for finding cmf)
+    'stage_extra1|w=s'    => \$stage_extra1, # the number for a staticskymulti (for finding cmf), or stack_id
     'dbname|d=s'        => \$dbname, # Database name
     'outroot|w=s'       => \$outroot, # output file base name
@@ -162 +162 @@
     }
 }
-if (($stage =~/staticsky/) || ($stage =~/stack/)) {
+if (($stage =~/stack/)) {
     $fpaObjects =~ s/smf$/cmf/;
     $fpaObjectsAlt =~ s/smf$/cmf/;
-}
+
+}
+my $fpaObjects1;
+my $fpaObjects2;
 my $checkalt = 0;
 if ($stage =~/staticsky/) {
-    $fpaObjects =~ s/smf$/cmf/; #this should do nothing because it was already modded above
-    &my_die( "can't find the filter_num for staticsky_multi, giving up.", $add_id, $PS_EXIT_SYS_ERROR) unless (defined $multi_num);
-    my $nice_num = sprintf ("%03d", $multi_num);
-    $fpaObjects =~ s/cmf$/$nice_num.cmf/;  #this make it look for .001.cmf, etc
-    if ($multi_num == 0) {
-        $checkalt =1 ; #if it's staticsky and multi_num = 0 there's an alternate file to try if this one doesn't work
-    }
+    $checkalt = 1;
+    my $sources   = $ipprc->filename("PSPHOT.OUT.CMF.MEF", $stageroot); #this is mostly rigtht except the .cmf needs either
+    # .000.cmf or .stk.xxxxx.cmf
+    print "$sources\n\n\n";
+    &my_die( "can't find the filter_num for staticsky_multi, giving up.", $add_id, $PS_EXIT_SYS_ERROR) unless (defined $stage_extra1);
+
+    
+    $fpaObjects1 = $sources;
+    $fpaObjects2 = $sources;
+    $fpaObjects = $sources;
+
+    my $nice_num = sprintf ("%03d", $stage_extra1);
+
+    $fpaObjects1 =~ s/cmf$/stk.$stage_extra1.cmf/;
+    $fpaObjects2 =~ s/cmf$/$nice_num.cmf/;  #this make it look for .001.cmf, etc
+ # we have 3 of them to try
+    my $realFile = $ipprc->file_resolve($fpaObjects);
+    my $realFile1 = $ipprc->file_resolve($fpaObjects1);
+    my $realFile2 = $ipprc->file_resolve($fpaObjects2);
+    if (!defined($realFile1)) {
+        if (!defined($realFile2)) {
+            print "using $fpaObjects\n";
+            
+        } else {
+            print "using $fpaObjects2\n";
+            $fpaObjects = $fpaObjects2;
+        }
+        
+    } else {
+        print "using $fpaObjects1\n";
+        $fpaObjects = $fpaObjects1;
+    }
+     
+
 }
 
@@ -230 +260 @@
             my $camdir = $ipprc->dvo_cameradir(); # Camera directory for addstar
            
-            my $realFile = $ipprc->file_resolve($fpaObjects);
-            if (!defined($realFile)) {
-                print "can't find $fpaObjects\n";
-                if ($checkalt)  {
-                    $realFile = $ipprc->file_resolve($fpaObjectsAlt) or &my_die("Unable to resolve $fpaObjectsAlt", $add_id, $PS_EXIT_SYS_ERROR);
-                    $fpaObjects = $fpaObjectsAlt; # want to make sure the correct value is used.
-                } else {
-                    # try the 'good one' again and fail i
-                    #$realFile = $ipprc->file_resolve($fpaObjects) or 
-                    &my_die("Unable to resolve $fpaObjects", $add_id, $PS_EXIT_SYS_ERROR);
-                }
-                
-            }
+            my $realFile = $ipprc->file_resolve($fpaObjects) or  &my_die("Unable to resolve $fpaObjects", $add_id, $PS_EXIT_SYS_ERROR);
+            
+            
+           
 
             # require a defined output dvo database to run addstar (ie, refuse to use the .ptolemyrc default)

branches/eam_branches/ipp-20120601/ippScripts/scripts/automate_stacks.pl

@@ -1588 +1588 @@
 
 sub init_gpc_db {
+    ## change to use the site.config setting now, however may want to use replicated scidbs instead
+    my $ipprc =  PS::IPP::Config->new(); # IPP Configuration
+    my $siteConfig = $ipprc->{_siteConfig};
     use constant DB_SOCKET => '/var/run/mysqld/mysqld.sock';
-    my $dbserver = 'ippdb01';
+    # my $dbserver = 'ippdb01';
     my $dbuser = 'ippuser';
     my $dbpass = 'ippuser';
+    my $dbserver = metadataLookupStr($siteConfig, 'DBSERVER');
+    die "database configuration set up" unless defined($dbserver);
     $db = DBI->connect("DBI:mysql:database=${dbname};host=${dbserver};" .
                        "mysql_socket=" . DB_SOCKET(),

branches/eam_branches/ipp-20120601/ippScripts/scripts/bundle_detrends.pl

@@ -128 +128 @@
 
 sub init_gpc_db {
+    ## change to use the site.config setting now, however may want to use replicated scidbs instead
+    my $siteConfig = $ipprc->{_siteConfig};
     use constant DB_SOCKET => '/var/run/mysqld/mysqld.sock';
-    my $dbserver = 'ippdb01';
+    # my $dbserver = 'ippdb01';
     my $dbuser = 'ippuser';
     my $dbpass = 'ippuser';
+    my $dbserver = metadataLookupStr($siteConfig, 'DBSERVER');
+    die "database configuration set up" unless defined($dbserver);
     $db = DBI->connect("DBI:mysql:database=${dbname};host=${dbserver};" .
                        "mysql_socket=" . DB_SOCKET(),

branches/eam_branches/ipp-20120601/ippScripts/scripts/make_burntool_pcontrol.pl

@@ -29 +29 @@
 
 # Load database
+## change to use the site.config setting now
 $dbname = 'gpc1';
-$dbserver = 'ippdb01';
+#$dbserver = 'ippdb01';
 $dbuser = 'ipp';
 $dbpass = 'ipp';
+my $ipprc =  PS::IPP::Config->new(); # IPP Configuration
+my $siteConfig = $ipprc->{_siteConfig};
+my $dbserver = metadataLookupStr($siteConfig, 'DBSERVER');
+die "database configuration not set up" unless defined($dbserver);
 $db = DBI->connect("DBI:mysql:database=${dbname};host=${dbserver};" .
                    "mysql_socket=" . DB_SOCKET(),

branches/eam_branches/ipp-20120601/ippScripts/scripts/nightly_science.pl

@@ -32 +32 @@
     exit($PS_EXIT_CONFIG_ERROR);
 }
+
+## for DB connection, want to use siteConfig DB settings
+my $ipprc =  PS::IPP::Config->new(); # IPP Configuration
+my $siteConfig = $ipprc->{_siteConfig};
 
 my $db;
@@ -2052 +2056 @@
 
 sub init_gpc_db {
+    ## change to use the siteConfig setting, while readonly probably do not want to use a readonly replicated DB incase it gets behind
+    my $dbserver = metadataLookupStr($siteConfig, 'DBSERVER');
+    my $dbuser = metadataLookupStr($ipprc->{_siteConfig}, "RO_DBUSER");
+    my $dbpass = metadataLookupStr($ipprc->{_siteConfig}, "RO_DBPASSWORD");
+    die "database configuration not set up" unless defined($dbserver);
+    die "database configuration not set up" unless defined($dbuser);
+    die "database configuration not set up" unless defined($dbpass);
+    #my $dbserver = 'ippdb01';
+    #my $dbuser = 'ippuser';
+    #my $dbpass = 'ippuser';
     use constant DB_SOCKET => '/var/run/mysqld/mysqld.sock';
-    my $dbserver = 'ippdb01';
-    my $dbuser = 'ippuser';
-    my $dbpass = 'ippuser';
     $db = DBI->connect("DBI:mysql:database=${dbname};host=${dbserver};" .
                        "mysql_socket=" . DB_SOCKET(),

branches/eam_branches/ipp-20120601/ippScripts/scripts/staticsky.pl

@@ -81 +81 @@
     -msg => "Required options: --sky_id --outroot --camera",
     -exitval => 3,
-          ) unless 
+          ) unless
     defined $sky_id and
     defined $outroot and
@@ -120 +120 @@
 
 
-# generate the input 
+# generate the input
 print $listFile "INPUT   MULTI\n";
 my $nInputs = @$files;
@@ -194 +194 @@
         # Perform stack photometry analysis
         {
-            my $command = "$psphotStack $outroot.stk";
+            my $command = "$psphotStack $outroot";
             $command .= " -input $listName";
             $command .= " -threads $threads" if defined $threads;
@@ -235 +235 @@
                     # we have one set of output files per input file set
                     for (my $i = 0; $i < @$files; $i++) {
-                        my $outputName     = $ipprc->filename("PSPHOT.STACK.OUTPUT.IMAGE", $outroot, $i);
-                        my $outputMask     = $ipprc->filename("PSPHOT.STACK.OUTPUT.MASK", $outroot, $i);
-                        my $outputVariance = $ipprc->filename("PSPHOT.STACK.OUTPUT.VARIANCE", $outroot, $i);
-                        my $outputSources  = $ipprc->filename("PSPHOT.STACK.OUTPUT", $outroot, $i);
+                        my $stack_id = @$files[$i]->{stack_id};
+                        my $outputName     = $ipprc->filename("PSPHOT.STACK.OUTPUT.IMAGE", $outroot, $stack_id);
+                        my $outputMask     = $ipprc->filename("PSPHOT.STACK.OUTPUT.MASK", $outroot, $stack_id);
+                        my $outputVariance = $ipprc->filename("PSPHOT.STACK.OUTPUT.VARIANCE", $outroot, $stack_id);
+                        my $outputSources  = $ipprc->filename("PSPHOT.STACK.OUTPUT", $outroot, $stack_id);
 
                         # XXX these are optional and not generated by default
@@ -256 +257 @@
 
                     # XXX check the recipe -- should we expect these to exist?
-                    # &my_die("Couldn't find expected output file: $chisqName",           $sky_id, $PS_EXIT_SYS_ERROR) unless $ipprc->file_exists($chisqName);
-                    # &my_die("Couldn't find expected output file: $chisqMask",           $sky_id, $PS_EXIT_SYS_ERROR) unless $ipprc->file_exists($chisqMask);
+                    # &my_die("Couldn't find expected output file: $chisqName",           $sky_id, $PS_EXIT_SYS_ERROR) unless $ipprc->file_exists($chisqName);
+                    # &my_die("Couldn't find expected output file: $chisqMask",           $sky_id, $PS_EXIT_SYS_ERROR) unless $ipprc->file_exists($chisqMask);
                     # &my_die("Couldn't find expected output file: $chisqVariance", $sky_id, $PS_EXIT_SYS_ERROR) unless $ipprc->file_exists($chisqVariance);
                 }
@@ -346 +347 @@
                 }
 
-                &my_die("Couldn't find expected output file: $configuration", $sky_id, $PS_EXIT_SYS_ERROR) 
+                &my_die("Couldn't find expected output file: $configuration", $sky_id, $PS_EXIT_SYS_ERROR)
                     unless $ipprc->file_exists($configuration);
             } else {
@@ -359 +360 @@
     {
         my $command = "$staticskytool -sky_id $sky_id";
-        $command .= " -addresult -path_base $outroot";
-        $command .= " -num_inputs $nInputs";
-        # $command .= " $cmdflags";
-        $command .= (" -dtime_script " . ((DateTime->now->mjd - $mjd_start) * 86400));
-        $command .= " -hostname $host" if defined $host;
+        $command .= " -addresult -path_base $outroot";
+        $command .= " -num_inputs $nInputs";
+        # $command .= " $cmdflags";
+        $command .= (" -dtime_script " . ((DateTime->now->mjd - $mjd_start) * 86400));
+        $command .= " -hostname $host" if defined $host;
         $command .= " -dbname $dbname" if defined $dbname;
 
@@ -371 +372 @@
             $error_code = (($error_code >> 8) or $PS_EXIT_PROG_ERROR);
             my $err_message = "Unable to perform staticskytool -addresult";
-            warn($err_message);
-            exit $error_code;
+            warn($err_message);
+            exit $error_code;
         }
     }
@@ -389 +390 @@
     if (defined $sky_id and not $no_update) {
         my $command = "$staticskytool -sky_id $sky_id -fault $exit_code";
-        $command .= " -addresult";
-        $command .= (" -dtime_script " . ((DateTime->now->mjd - $mjd_start) * 86400));
-        $command .= " -hostname $host" if defined $host;
-        $command .= " -path_base $outroot" if defined $outroot;
-        $command .= " -dbname $dbname" if defined $dbname;
+        $command .= " -addresult";
+        $command .= (" -dtime_script " . ((DateTime->now->mjd - $mjd_start) * 86400));
+        $command .= " -hostname $host" if defined $host;
+        $command .= " -path_base $outroot" if defined $outroot;
+        $command .= " -dbname $dbname" if defined $dbname;
         run(command => $command, verbose => $verbose);
     }

branches/eam_branches/ipp-20120601/ippTools/share/addtool_find_sky_id_multi.sql

@@ -2 +2 @@
      distinct(sky_id) as stage_id,
      staticskyRun.*,
-    staticskyResult.num_inputs
+    staticskyResult.num_inputs, stack_id
 FROM 
      staticskyRun

branches/eam_branches/ipp-20120601/ippTools/share/addtool_find_sky_id_multi_dvo.sql

@@ -1 @@
-SELECT distinct(sky_id) as stage_id, staticskyRun.*, staticskyResult.num_inputs FROM staticskyResult
+SELECT distinct(sky_id) as stage_id, staticskyRun.*, staticskyResult.num_inputs, stack_id FROM staticskyResult
 JOIN staticskyRun USING(sky_id)
 join staticskyInput using(sky_id)

branches/eam_branches/ipp-20120601/ippTools/src/addtool.c

@@ -292 +292 @@
       // }
     if (strcmp(stage,"staticsky") == 0) {
-      psStringAppend(&query, " GROUP BY sky_id");  //some reason it needs this
+      psStringAppend(&query, " GROUP BY sky_id, stack_id");  //some reason it needs this
       }
 
@@ -525 +525 @@
         psS64 stage_id =0; 
         bool status = false;
-        psS32 num_inputs = psMetadataLookupS32(&status, md, "num_inputs");
+        //psS32 num_inputs = psMetadataLookupS32(&status, md, "num_inputs");
+        psS32 stack_id = psMetadataLookupS32(&status, md, "stack_id");
+
         if (!status) {
-          psError(PS_ERR_UNKNOWN, true, "failed to lookup value for item num_inputs");
+          psError(PS_ERR_UNKNOWN, true, "failed to lookup value for item stack_id");
           return false;
         }
@@ -540 +542 @@
 
         // queue the exp
-        for (int filter_id = 0; filter_id < num_inputs; filter_id++) {  
-        if (!pxaddQueueByCamID(config,
+        //for (int filter_id = 0; filter_id < num_inputs; filter_id++) {  
+          if (!pxaddQueueByCamID(config,
                                stage,
-                               stage_id,
-                               filter_id,
+                               stage_id,
+                               stack_id,
                                workdir     ? workdir   : row->workdir,
                                reduction   ? reduction : row->reduction,
@@ -555 +557 @@
                                minidvodb_group,
                                minidvodb_name
-        )) {
+                                 )) {
             if (!psDBRollback(config->dbh)) {
-                psError(PS_ERR_UNKNOWN, false, "database error sfg");
+              psError(PS_ERR_UNKNOWN, false, "database error sfg");
             }
             psError(PS_ERR_UNKNOWN, false,
@@ -564 +566 @@
             psFree(output);
             return false;
-        }
-        }
+          }
+          //}
         psFree(row);
     }

branches/eam_branches/ipp-20120601/ippconfig/recipes/filerules-mef.mdc

@@ -239 +239 @@
 PSPHOT.CHISQ.MASK       OUTPUT {OUTPUT}.{CHIP.NAME}.chisq.mk.fits MASK     NONE       FPA        TRUE      SIMPLE
 PSPHOT.CHISQ.VARIANCE   OUTPUT {OUTPUT}.{CHIP.NAME}.chisq.wt.fits VARIANCE NONE       FPA        TRUE      SIMPLE
-PSPHOT.STACK.OUTPUT     OUTPUT {OUTPUT}.{CHIP.NAME}.{FILE.ID}.cmf CMF      NONE       FPA        TRUE      NONE
+PSPHOT.STACK.OUTPUT     OUTPUT {OUTPUT}.{CHIP.NAME}.sky.{FILE.ID}.cmf CMF      NONE       FPA        TRUE      NONE
                                                      
 SOURCE.PLOT.MOMENTS     OUTPUT {OUTPUT}.mnt.png                  KAPA      NONE       FPA        TRUE      NONE

branches/eam_branches/ipp-20120601/ippconfig/recipes/filerules-simple.mdc

@@ -216 +216 @@
 PSPHOT.CHISQ.MASK            OUTPUT {OUTPUT}.chisq.mk.fits        MASK            NONE       FPA        TRUE      NONE
 PSPHOT.CHISQ.VARIANCE        OUTPUT {OUTPUT}.chisq.wt.fits        VARIANCE        NONE       FPA        TRUE      NONE
-PSPHOT.STACK.OUTPUT.IMAGE    OUTPUT {OUTPUT}.{FILE.ID}.im.fits    IMAGE           NONE       FPA        TRUE      NONE
-PSPHOT.STACK.OUTPUT.MASK     OUTPUT {OUTPUT}.{FILE.ID}.mk.fits    MASK            NONE       FPA        TRUE      NONE
-PSPHOT.STACK.OUTPUT.VARIANCE OUTPUT {OUTPUT}.{FILE.ID}.wt.fits    VARIANCE        NONE       FPA        TRUE      NONE
-PSPHOT.STACK.OUTPUT          OUTPUT {OUTPUT}.{FILE.ID}.cmf        CMF             NONE       FPA        TRUE      NONE
+PSPHOT.STACK.OUTPUT.IMAGE    OUTPUT {OUTPUT}.stk.{FILE.ID}.im.fits    IMAGE           NONE       FPA        TRUE      NONE
+PSPHOT.STACK.OUTPUT.MASK     OUTPUT {OUTPUT}.stk.{FILE.ID}.mk.fits    MASK            NONE       FPA        TRUE      NONE
+PSPHOT.STACK.OUTPUT.VARIANCE OUTPUT {OUTPUT}.stk.{FILE.ID}.wt.fits    VARIANCE        NONE       FPA        TRUE      NONE
+PSPHOT.STACK.OUTPUT          OUTPUT {OUTPUT}.stk.{FILE.ID}.cmf        CMF             NONE       FPA        TRUE      NONE
                                                      
 SOURCE.PLOT.MOMENTS          OUTPUT {OUTPUT}.mnt.png              KAPA            NONE       FPA        TRUE      NONE

branches/eam_branches/ipp-20120601/ippconfig/recipes/filerules-split.mdc

@@ -235 +235 @@
 PSPHOT.CHISQ.MASK            OUTPUT {OUTPUT}.chisq.mk.fits            MASK            NONE       FPA        TRUE      NONE
 PSPHOT.CHISQ.VARIANCE        OUTPUT {OUTPUT}.chisq.wt.fits            VARIANCE        NONE       FPA        TRUE      NONE
-PSPHOT.STACK.OUTPUT.IMAGE    OUTPUT {OUTPUT}.{FILE.ID}.im.fits        IMAGE           NONE       FPA        TRUE      NONE
-PSPHOT.STACK.OUTPUT.MASK     OUTPUT {OUTPUT}.{FILE.ID}.mk.fits        MASK            NONE       FPA        TRUE      NONE
-PSPHOT.STACK.OUTPUT.VARIANCE OUTPUT {OUTPUT}.{FILE.ID}.wt.fits        VARIANCE        NONE       FPA        TRUE      NONE
-PSPHOT.STACK.OUTPUT          OUTPUT {OUTPUT}.{FILE.ID}.cmf            CMF             NONE       FPA        TRUE      NONE
+PSPHOT.STACK.OUTPUT.IMAGE    OUTPUT {OUTPUT}.stk.{FILE.ID}.im.fits        IMAGE           NONE       FPA        TRUE      NONE
+PSPHOT.STACK.OUTPUT.MASK     OUTPUT {OUTPUT}.stk.{FILE.ID}.mk.fits        MASK            NONE       FPA        TRUE      NONE
+PSPHOT.STACK.OUTPUT.VARIANCE OUTPUT {OUTPUT}.stk.{FILE.ID}.wt.fits        VARIANCE        NONE       FPA        TRUE      NONE
+PSPHOT.STACK.OUTPUT          OUTPUT {OUTPUT}.stk.{FILE.ID}.cmf            CMF             NONE       FPA        TRUE      NONE
 PSPHOT.STACK.CONFIG          OUTPUT {OUTPUT}.psphotStack.mdc          TEXT            NONE       FPA        TRUE      NONE