Changes between Version 3 and Version 4 of shepherding_the_pstamp_server

Timestamp:

May 27, 2010, 3:16:30 PM (16 years ago)

Author:

bills

Comment:

--

shepherding_the_pstamp_server

@@ -12 +12 @@
 
 Each file is represented in the database by a row in the table pstampRequest. The request starts
-life in state 'new'.
-
-Request files are parsed into jobs by the task pstamp.job.run
+life in state 'new'. Once it is done parsing the state is set to 'run'. Request files are parsed into jobs by the task pstamp.job.run. The working directory for a request is /data/ippdb02.0/pstamp/work/$datestr/$req_id.
+
+For example here is the entry for a request while it is being parsed 
+
+{{{
+
+
+mysql> select * from pstampRequest where req_id = 5977;
++--------+-------+-------+------------------+---------+--------+---------------+---------------------------------------------------------+-------------------------------------------+-------+
+| req_id | ds_id | state | name             | reqType | label  | outProduct    | uri                                                     | outdir                                    | fault |
++--------+-------+-------+------------------+---------+--------+---------------+---------------------------------------------------------+-------------------------------------------+-------+
+|   5977 |     0 | new   | Niall.1274580688 | pstamp  | WEB.UP | pstampresults | /data/ippdb02.0/pstamp/work/20100527/5977/web_1695.fits | /data/ippdb02.0/pstamp/work/20100527/5977 |     0 | 
++--------+-------+-------+------------------+---------+--------+---------------+---------------------------------------------------------+-------------------------------------------+-------+
+1 row in set (0.00 sec)
+
+
+}}}
 
 Each row in the request table contains a request specification. Each row causes one or more pstampJobs to
-be entered into the database. Jobs can be entered with state = 'stop' and fault > 10. These are due
+be entered into the database. Jobs are often entered with state = 'stop' and fault > 10. These are due
 to requests wanting images that haven't been destreaked or have been purged, bycoord requests that
-have no images or errors in the request.
+have no images or errors in the request. The set of errors is listed at the bottom of this page.
 
 Once the file has completed parsing successfully it's state is set to 'run'. Once the request is
@@ -29 +43 @@
 The task pstamp.dependent.run runs the script pstamp_checkdependent.pl. This  script queries the
 gpc1 database for the state of the dependent component. This script's job is to queue and monitor
-update processing for the dependent component. CAREFUL THERE BE SPIDERS
+update processing for the dependent component. CAREFUL. THERE BE SPIDERS!
+
+The log file for the dependent is stored in the file named "$pstampDependent->{outdir}/checkdep.$dep_id.log". 
+The script runs periodically, the log file is appended to each time. The outdir for the dependent is
+the outdir of the first request that needs the component. 
+
+Dependents can start being processed before parsing completes. The -pendingdependent query is
+
+
+{{{
+SELECT DISTINCT
+    pstampDependent.*,
+    IFNULL(Label.priority, 10000) AS priority
+FROM pstampDependent
+JOIN pstampJob USING(dep_id)
+JOIN pstampRequest USING(req_id)
+LEFT JOIN Label ON pstampRequest.label = Label.label
+WHERE pstampDependent.state = 'new'
+    AND pstampDependent.fault = 0
+    AND pstampJob.state = 'run'
+    AND (pstampRequest.state = 'run' OR pstampRequest.state = 'new')
+    AND (Label.active OR Label.active IS NULL)
+ ORDER BY priority DESC, req_id  LIMIT 1
+
+}}}
+
 
 
@@ -85 +124 @@
 
 Jobs can fault due to typical nfs errors. There is a revert task for jobs. It only resets jobs with
-fault > 0 and < 10. Faults >= 10 are faults for the users. Jobs which are faulted in this way have
-shoul have had their state set to stop thus they are finished.
+fault > 0 and < 10. Faults >= 10 are faults for the users. Jobs which are faulted in this way should
+have had their state set to stop thus they are finished.
 
 The vast majority of the problems these days are related to the update processing. There are a large
-number of error conditions that can gum up the works. When a dependent run faults (that is when a
-update of a chipProcessedImfile, warpSkyfile, etc faults) the dependent is faulted as well.
-
+number of error conditions that can gum up the works. W
 Here is an example
 
@@ -119 +156 @@
 is destreaked chipProcessedImfile.magicked > 0 it's done and the pstampDependent is set to 'full'
 
-The update processing is managed by the 'update' pantasks
-
-
-
-
-
-
+The update processing is managed by the 'update' pantasks.
+
+When a dependent run faults (that is when a
+update of a chipProcessedImfile, warpSkyfile, etc faults) the dependent is faulted as well. Thus to revert
+the dependent one needs to revert the underlying failure first and then revert the dependent. This is not
+yet done automatically.
+
+For example say the dependent listed above fails due to an nfs error. The chipProcessedImfile.fault should be
+set to a PS_EXIT_SYS_ERROR (2). pstamp_checkdependent.pl will notice this and fault the dependent. It does this
+to avoid faulted dependents from filling up the pantasks queue thus preventing other dependents from running.
+
+Once the chip fault has been cleared
+
+
+
+
+== Postage Stamp Error Codes ==
+
+
+{{{
+
+typedef enum {
+        PSTAMP_SUCCESS          = 0,
+        PSTAMP_FIRST_ERROR_CODE = 10,
+        PSTAMP_SYSTEM_ERROR     = 10,
+        PSTAMP_NOT_IMPLEMENTED  = 11,
+        PSTAMP_UNKNOWN_ERROR    = 12,
+        PSTAMP_DUP_REQUEST      = 20,
+        PSTAMP_INVALID_REQUEST  = 21,
+        PSTAMP_UNKNOWN_PRODUCT  = 22,
+        PSTAMP_NO_IMAGE_MATCH   = 23,
+        PSTAMP_NOT_DESTREAKED   = 24,
+        PSTAMP_NOT_AVAILABLE    = 25,
+        PSTAMP_GONE             = 26,
+        PSTAMP_NO_JOBS_QUEUED   = 27,
+        PSTAMP_NO_OVERLAP       = 28
+} pstampJobErrors;
+}}}
+
+
+
+
+
+
+
+