# # ChangeLog for trunk/ippToPsps # # Generated by Trac 1.6 # Jul 20, 2026, 11:07:22 AM Sat, 26 Nov 2011 01:22:41 GMT rhenders [32826] * trunk/ippToPsps/jython/configs/newsas.xml (modified) production config for new SAS Sat, 26 Nov 2011 00:55:01 GMT rhenders [32825] * trunk/ippToPsps/xmlschema/BatchManifest.xsd (modified) Added all the new PSPS surveys to the XML schema for batch manifest files Sat, 26 Nov 2011 00:50:04 GMT rhenders [32824] * trunk/ippToPsps/jython/batch.py (modified) improved some log messages Sat, 26 Nov 2011 00:49:24 GMT rhenders [32823] * trunk/ippToPsps/jython/configs/newsas.xml (modified) added comment Fri, 25 Nov 2011 22:02:31 GMT rhenders [32821] * trunk/ippToPsps/jython/configs/newsas.xml (added) New config for the new SAS database Fri, 25 Nov 2011 21:17:01 GMT rhenders [32819] * trunk/ippToPsps/jython/batch.py (modified) improved logic so that batches are only marked as 'successfully' ... Thu, 24 Nov 2011 02:16:33 GMT rhenders [32802] * trunk/ippToPsps/jython/ipptopspsdb.py (modified) add extra constraints when gathering batches that have not loaded to ... Thu, 24 Nov 2011 02:15:13 GMT rhenders [32801] * trunk/ippToPsps/jython/odm.py (modified) savinf ODM message for display; setting default message for 'details' ... Thu, 24 Nov 2011 02:13:09 GMT rhenders [32800] * trunk/ippToPsps/jython/datastoreDupeChecker.sh (added) quick and dirty shell script to check for dupes on the datastore Wed, 23 Nov 2011 22:00:23 GMT rhenders [32781] * trunk/ippToPsps/jython/ipptopspsdb.py (modified) increased 'processing now' limit from 2 to 4 hours, as when network ... Wed, 23 Nov 2011 21:56:52 GMT rhenders [32780] * trunk/ippToPsps/jython/ipptopspsdb.py (modified) removed 'loaded_to_datastore != 0' constraint when checking if a ... Wed, 23 Nov 2011 19:56:05 GMT rhenders [32778] * trunk/ippToPsps/jython/gpc1db.py (modified) removed redundant method used to testing Wed, 23 Nov 2011 18:49:00 GMT rhenders [32777] * trunk/ippToPsps/jython/metrics.py (modified) added a timer so program will run in a loop Tue, 22 Nov 2011 02:14:45 GMT rhenders [32737] * trunk/ippToPsps/jython/configs/lap20110809.xml (modified) added czar Db info to config Tue, 22 Nov 2011 01:54:18 GMT rhenders [32736] * trunk/ippToPsps/jython/configs/oldthreepi.xml (modified) added czar db section to config Mon, 21 Nov 2011 22:12:30 GMT rhenders [32730] * trunk/ippToPsps/jython/configs/oldthreepi.xml (modified) Using master DVO database on ipp005; now processing up to 18 hrs RA Fri, 11 Nov 2011 02:00:22 GMT rhenders [32655] * trunk/ippToPsps/jython/pollOdm.py (modified) Neatened this up so that the 'all' option will sequentially check ... Fri, 11 Nov 2011 01:59:12 GMT rhenders [32654] * trunk/ippToPsps/jython/ipptopspsdb.py (modified) Removed redundant getAllUnfinishedBatchIDs() method. Not needed by ... Sat, 29 Oct 2011 08:24:40 GMT rhenders [32596] * trunk/ippToPsps/jython/batch.py (modified) some re-arranging so that, if the FITS file is not found in Nebulous, ... Fri, 28 Oct 2011 21:44:46 GMT rhenders [32592] * trunk/ippToPsps/jython/detectionbatch.py (modified) Added high-level comment Fri, 28 Oct 2011 21:43:38 GMT rhenders [32591] * trunk/ippToPsps/jython/setupScratchDb.py (modified) Calling createPsfLikelihoodFunction() explicitly Fri, 28 Oct 2011 21:41:25 GMT rhenders [32590] * trunk/ippToPsps/jython/scratchdb.py (modified) Fixed method to install psfLikelihood() stored procedure Fri, 28 Oct 2011 21:40:06 GMT rhenders [32589] * trunk/ippToPsps/jython/batch.py (modified) now exiting cleanly if no FITS file is found for non-IN batches; ... Fri, 28 Oct 2011 21:39:14 GMT rhenders [32588] * trunk/ippToPsps/jython/stackbatch.py (modified) Added high-level comment Fri, 28 Oct 2011 21:38:48 GMT rhenders [32587] * trunk/ippToPsps/jython/metrics.py (modified) Now makes an estimate of remaining time required to complete ... Fri, 28 Oct 2011 21:38:12 GMT rhenders [32586] * trunk/ippToPsps/jython/setupScratchDb.py (added) New program to setup a new scratch database on the local machine Fri, 28 Oct 2011 01:06:50 GMT rhenders [32582] * trunk/ippToPsps/jython/configs/oldthreepi.xml (modified) Just processing RA 9-10 hrs Thu, 27 Oct 2011 23:41:47 GMT rhenders [32581] * trunk/ippToPsps/jython/configs/init.xml (added) config for setting up new scratch Dbs with init tables Thu, 27 Oct 2011 23:41:11 GMT rhenders [32580] * trunk/ippToPsps/jython/ipptopspsdb.py (modified) changed maximum fails count from 3 to 4 Thu, 27 Oct 2011 23:40:40 GMT rhenders [32579] * trunk/ippToPsps/jython/load.py (modified) Now reading ra/dec coords from config when queueing-up Thu, 27 Oct 2011 23:40:11 GMT rhenders [32578] * trunk/ippToPsps/jython/configs/oldthreepi.xml (modified) Ready for loading.... Thu, 13 Oct 2011 19:58:45 GMT rhenders [32543] * trunk/ippToPsps/jython/cleanup.py (modified) added time argument so it will cleanup every n hours Thu, 13 Oct 2011 19:57:27 GMT rhenders [32542] * trunk/ippToPsps/jython/pollOdm.py (modified) main polling stuff now in a function; added time argument so it will ... Thu, 13 Oct 2011 19:50:28 GMT rhenders [32541] * trunk/ippToPsps/jython/ipptopspsdb.py (modified) no need for 'DISTINCT batch_id as that column is an index, so they ... Thu, 13 Oct 2011 19:48:23 GMT rhenders [32540] * trunk/ippToPsps/jython/odm.py (modified) now assuming that 'OnHold' batches have failed Fri, 07 Oct 2011 20:49:49 GMT rhenders [32519] * trunk/ippToPsps/jython/cleanup.py (modified) now cleaning up DXLayer as well Fri, 07 Oct 2011 20:49:23 GMT rhenders [32518] * trunk/ippToPsps/jython/ipptopspsdb.py (modified) changes order of stages now that we are deleting DXLayer batches Fri, 07 Oct 2011 20:48:53 GMT rhenders [32517] * trunk/ippToPsps/jython/dxlayer.py (added) first commit of dxlayer class, so far only providing methods to ... Fri, 07 Oct 2011 20:47:42 GMT rhenders [32516] * trunk/ippToPsps/jython/stackbatch.py (modified) put back old code to deal with skycell/projectioncell as changes ... Fri, 07 Oct 2011 20:02:58 GMT rhenders [32515] * trunk/ippToPsps/config/IN/tables.vot (modified) removing duplicated 'z' filter Fri, 07 Oct 2011 01:26:19 GMT rhenders [32512] * trunk/ippToPsps/jython/stackbatch.py (modified) adding projectionCellID to to the SkinnyObject table Fri, 07 Oct 2011 01:18:12 GMT rhenders [32508] * trunk/ippToPsps/jython/stackbatch.py (modified) now calculating psfLikelihood in StackDetection table Fri, 07 Oct 2011 00:06:50 GMT rhenders [32501] * trunk/ippToPsps/jython/stackbatch.py (modified) fixed error with skycell and added projectioncellID. Thu, 06 Oct 2011 23:41:11 GMT rhenders [32500] * trunk/ippToPsps/jython/stackbatch.py (modified) added psfFwhm_mean and psfFwhm_max Thu, 06 Oct 2011 22:50:22 GMT rhenders [32499] * trunk/ippToPsps/jython/configs/oldthreepi.xml (added) config for loading old 3pi data Thu, 06 Oct 2011 03:05:14 GMT rhenders [32497] * trunk/ippToPsps/jython/republishbatch.py (added) first commit of program that republishes a batch that has already ... Thu, 06 Oct 2011 03:04:07 GMT rhenders [32496] * trunk/ippToPsps/jython/ipptopspsdb.py (modified) new method that returns the batch type for the provided batch ID; ... Thu, 06 Oct 2011 03:01:52 GMT rhenders [32495] * trunk/ippToPsps/jython/metrics.py (modified) now reading eq-coords from config, if there are any Thu, 06 Oct 2011 03:00:42 GMT rhenders [32494] * trunk/ippToPsps/jython/configs/sas12.xml (modified) * trunk/ippToPsps/jython/configs/sas123.xml (modified) * trunk/ippToPsps/jython/configs/sasFootprint.xml (modified) deleted local batches for everything merged Thu, 06 Oct 2011 02:19:56 GMT rhenders [32493] * trunk/ippToPsps/jython/gpc1db.py (modified) added min/max RA/Dec to constraints on DVO query Thu, 06 Oct 2011 02:18:10 GMT rhenders [32492] * trunk/ippToPsps/jython/pollOdm.py (modified) now reporting counts of new merge_worthy and new merged batches Thu, 06 Oct 2011 02:16:57 GMT rhenders [32491] * trunk/ippToPsps/jython/datastore.py (modified) fixed comment Thu, 06 Oct 2011 02:16:09 GMT rhenders [32490] * trunk/ippToPsps/jython/odm.py (modified) closing stdout Thu, 06 Oct 2011 02:15:35 GMT rhenders [32489] * trunk/ippToPsps/jython/cleanup.py (modified) using new ipptopspsdb method, and changed some log messages Fri, 30 Sep 2011 01:33:14 GMT rhenders [32468] * trunk/ippToPsps/jython/load.py (modified) big changes to support multi-client loading; now locks batch table ... Fri, 30 Sep 2011 01:31:08 GMT rhenders [32467] * trunk/ippToPsps/jython/batch.py (modified) now does not create a new batchID itself, but has one passed-in to ... Fri, 30 Sep 2011 01:29:40 GMT rhenders [32466] * trunk/ippToPsps/jython/datastore.py (modified) now accepting an IppToPspsDb object as a constructor argument and ... Fri, 30 Sep 2011 01:25:31 GMT rhenders [32465] * trunk/ippToPsps/jython/cleanup.py (modified) updating of ipptopsps database now done within Datastore object, so ... Fri, 30 Sep 2011 01:24:21 GMT rhenders [32464] * trunk/ippToPsps/jython/pslogger.py (modified) now accepts host and PID arguments into the constructor for use when ... Fri, 30 Sep 2011 01:23:44 GMT rhenders [32463] * trunk/ippToPsps/jython/detectionbatch.py (modified) constructor now accepting batchID and passing it up to the base-class ... Fri, 30 Sep 2011 01:23:28 GMT rhenders [32462] * trunk/ippToPsps/jython/stackbatch.py (modified) constructor now accepting batchID and passing it up to the base-class ... Fri, 30 Sep 2011 01:23:06 GMT rhenders [32461] * trunk/ippToPsps/jython/ipptopspsdb.py (modified) added a method to lock the batch table; reworked methid to get a new ... Fri, 30 Sep 2011 01:20:12 GMT rhenders [32460] * trunk/ippToPsps/jython/pollOdm.py (modified) now taking a batch type argument at command line; using different ... Fri, 30 Sep 2011 01:19:10 GMT rhenders [32459] * trunk/ippToPsps/jython/scratchdb.py (modified) just whitespace Fri, 30 Sep 2011 01:18:43 GMT rhenders [32458] * trunk/ippToPsps/jython/initbatch.py (modified) constructor now accepting batchID and passing it up to the base-class ... Fri, 30 Sep 2011 01:18:01 GMT rhenders [32457] * trunk/ippToPsps/jython/mysql.py (modified) added a method to lock a table and a method to unlock all tables Fri, 30 Sep 2011 01:17:22 GMT rhenders [32456] * trunk/ippToPsps/jython/removeFromDatastore.py (modified) now creating a IppToPspsDb object and passing it to the Datastore ... Fri, 30 Sep 2011 01:16:31 GMT rhenders [32455] * trunk/ippToPsps/jython/fits.py (modified) temporarily not copying FITS files to local filesystem as there is ... Tue, 27 Sep 2011 20:42:19 GMT rhenders [32454] * trunk/ippToPsps/jython/initbatch.py (modified) implemented alterPspsTables() method to satisfy base-class Mon, 26 Sep 2011 23:22:50 GMT rhenders [32453] * trunk/ippToPsps/jython/metrics.py (modified) code to print some high-level stats about each batch type Mon, 26 Sep 2011 23:22:07 GMT rhenders [32452] * trunk/ippToPsps/jython/ipptopspsdb.py (modified) added seconds to method get time of last publish. stacks are fast Sat, 24 Sep 2011 01:42:18 GMT rhenders [32443] * trunk/ippToPsps/jython/ipptopspsdb.py (modified) including loaded_to_datastore = 1 when counting detections published; ... Sat, 24 Sep 2011 01:41:16 GMT rhenders [32442] * trunk/ippToPsps/jython/stackbatch.py (modified) certain methods were not returning True upon completion Fri, 23 Sep 2011 08:55:19 GMT rhenders [32436] * trunk/ippToPsps/jython/ipptopspsdb.py (modified) new method to count total detections published for this label, epoch etc Wed, 07 Sep 2011 02:29:05 GMT rhenders [32362] * trunk/ippToPsps/jython/ipptopspsdb.py (modified) changed order of processing stages; added some comments Tue, 06 Sep 2011 23:37:06 GMT rhenders [32354] * trunk/ippToPsps/jython/metrics.py (modified) now reporting batches-per-hour and last P2 and St batch published; ... Tue, 06 Sep 2011 23:35:45 GMT rhenders [32353] * trunk/ippToPsps/jython/ipptopspsdb.py (modified) removed some redundant metrics queries; new method to get batches- ... Tue, 06 Sep 2011 19:46:27 GMT rhenders [32332] * trunk/ippToPsps/jython/batch.py (modified) removing 'size' and 'md5' fields from BatchManifest file. Time ... Tue, 06 Sep 2011 19:35:17 GMT rhenders [32331] * trunk/ippToPsps/jython/datastore.py (modified) Just whitespace Mon, 05 Sep 2011 19:51:54 GMT rhenders [32327] * trunk/ippToPsps/jython/detectionbatch.py (modified) shifted ImageMeta and FrameMeta population into different method so ... Mon, 05 Sep 2011 19:51:02 GMT rhenders [32326] * trunk/ippToPsps/jython/batch.py (modified) checking for error, and aborting if necessary, after call to ... Mon, 05 Sep 2011 02:34:24 GMT rhenders [32306] * trunk/ippToPsps/jython/configs/lap20110809.xml (modified) publishing all LAP to PSP_JHU datastore Mon, 05 Sep 2011 02:33:36 GMT rhenders [32305] * trunk/ippToPsps/jython/detectionbatch.py (modified) now specifying columns to import from IPP smf file Mon, 05 Sep 2011 02:32:45 GMT rhenders [32304] * trunk/ippToPsps/jython/batch.py (modified) now takes extra argument of columns to method that imports IPP tables Sun, 04 Sep 2011 08:59:33 GMT rhenders [32303] * trunk/ippToPsps/jython/scratchdb.py (modified) Added to comments Fri, 02 Sep 2011 21:43:24 GMT rhenders [32295] * trunk/ippToPsps/jython/scratchdb.py (modified) added stored procedure for calculating psfLikelihoods; not being ... Fri, 02 Sep 2011 21:42:19 GMT rhenders [32294] * trunk/ippToPsps/jython/metrics.py (modified) fixed typo in call to ippToPspsDb.getMergeWorthyIDs() method Fri, 02 Sep 2011 21:41:41 GMT rhenders [32293] * trunk/ippToPsps/jython/mysql.py (modified) trivial improvements to log messages and SQL Fri, 02 Sep 2011 21:40:48 GMT rhenders [32292] * trunk/ippToPsps/jython/ipptopspsdb.py (modified) tightened loaded_to_datastore constraint in alreadyProcessed() ... Fri, 02 Sep 2011 10:05:34 GMT rhenders [32286] * trunk/ippToPsps/jython/detectionbatch.py (modified) now using a stored procedure to calculate likelihoods. much, much ... Fri, 02 Sep 2011 09:15:10 GMT rhenders [32285] * trunk/ippToPsps/jython/detectionbatch.py (modified) deleting detections with flux < 0.0000001. a temporary hack Thu, 01 Sep 2011 02:09:19 GMT rhenders [32262] * trunk/ippToPsps/jython/batch.py (modified) writing -1 to processed column in db when a batch fails Thu, 01 Sep 2011 02:08:34 GMT rhenders [32261] * trunk/ippToPsps/jython/ipptopspsdb.py (modified) added method to return all batch IDs not yet loaded to ODM Thu, 01 Sep 2011 02:07:45 GMT rhenders [32260] * trunk/ippToPsps/jython/odm.py (modified) now grabbing 'Details' part of SOAP response when a load has failed Thu, 01 Sep 2011 02:07:12 GMT rhenders [32259] * trunk/ippToPsps/jython/metrics.py (modified) removed some redundant imports; now writing to file only thos batches ... Thu, 01 Sep 2011 02:06:27 GMT rhenders [32258] * trunk/ippToPsps/jython/configs/lap20110621.xml (modified) added deletion policy Thu, 01 Sep 2011 02:06:06 GMT rhenders [32257] * trunk/ippToPsps/jython/configs/sas12.xml (modified) * trunk/ippToPsps/jython/configs/sas123.xml (modified) * trunk/ippToPsps/jython/configs/sasFootprint.xml (modified) added deletion policy Thu, 01 Sep 2011 02:05:33 GMT rhenders [32256] * trunk/ippToPsps/jython/configs/lap20110809.xml (modified) added deletion policy Thu, 01 Sep 2011 02:05:06 GMT rhenders [32255] * trunk/ippToPsps/jython/cleanup.py (modified) now deletes from datastore and local disk if set in the config Thu, 01 Sep 2011 02:04:18 GMT rhenders [32254] * trunk/ippToPsps/jython/pollOdm.py (modified) added option to poll only for those batches not yet loaded to the ODM