Other DVO Tools

dvomerge

dvomerge is used to merge dvo databases. there are two main modes of operation:

* merge two databases into a third, previously absent, database: dvomerge (input1) and (intput2) to (output) * merge one database into an existing database : dvomerge (input) into (output)

Note that all dvomerge options may include the -region (RAmin) (RAmax) (DECmin) (DECmax) options

to restrict the merge to a portion of the sky.

The second mode of operation has two minor variants:

* dvomerge (input) into (output) continue

the 'continue' option allows the dvomerge process to operate on a fraction of the sky, in the assumption that an initial merge was performed (with the 'into' mode). The 'continue' is needed to tell dvomerge that the images of the two databases have already been merged; in this case, dvomerge will examine the image tables and determine the mapping of image IDs from the state of the output database.

* dvomerge (input) into (output) from (list)

the 'from' variant is equivalent to the 'continue' variant, but in this case the catalog tables to be merged are specified in the list file. The file contains one catalog file per line, with the name giving the directory and filename of the cpt files, without the catdir component (eg, n0000/0001.01.cpt).

Threads : threading support is available for dvomerge .. into, except for the 'from' mode. Add the option '-threads N' to specify the number of threads to be used. Each thread handles a separate catalog table (read from disk, calculate the merge, write to disk). With only a few threads, it is easy to saturate the disk I/O system, at which point the threading does not improve the speed of the merge process.

Some other words of warning:

* dvomerge does not attempt to check for previous merges (successful

or not). A re-run of dvomerge will simply duplicate the data previously merged. You must track your own results: which cpt files were merged in a given run.

* the region option selects the cpt files which touch the region. to

select exactly the cpt files you want, reduce the region by a small amount to avoid overlapping neighboring regions. For example, if you want to include only a specific cpt file with RA,DEC bounds of 0.0 - 1.0, 2.5 - 3.5, you might use -region 0.01 0.99 2.51 3.49

* the cpt headers define their boundaries (RA0, RA1, DEC0, DEC1).

* the merged dvo database is not sorted (indexed). To use it with the

dvo shell or programs like relphot or relastro, it is necessary to re-index with addstar -resort -D CATDIR (catdir). (Note that addstar -resort is also now able to use multiple threads: add the -threads N option).

dvorepair

dvorepair can be used to fix damaged dvo databases. Repair is possible because the measurement tables (cpm) and the image table together contain all of the information needed to regenerate the average (cpt) and secfilt (cps) tables.

dvorepair has several modes of operation:

* dvorepair -images-vs-measures (catdir) (Ntol)

this mode scans the database and compares the number of detections tied to a specific image (in the measure tables) with the number of detections claimed by that image (in the average table). Images which have too few detections are reported to standard out. The value of Ntol specifies the number of detections which can be dropped -- a few missing detections for a given image is not catastrophic.

* dvorepair -delete-image-list (catdir) (list)

this mode accepts the list of bad images as reported by the -images-vs-measures mode. It then scans the database and deletes all detections associated with the deleted images. It also repairs the average (cpt) and secfilt (cps) tables so they are consistent with the repaired measurement tables. The repaired output tables do not overwrite the existing tables; instead they are written with the suffix ".fixed". It is up to the user to find and replace the original files with the .fixed version.

Note that the user may edit the output list from -images-vs-measures and choose not to delete images as desired. But be careful: the format of the lines of the input file must match the original output format.

Note that a repaired database is not indexed: you must run addstar -resort to re-index the database.

* dvorepair -fix-images (catdir) (list)

This mode accepts the same for list file as -delete-image-list, and in this case, deletes the images from the image table (as opposed to the detections).

* dvorepair -fix-cpt (images) (list)

This mode accepts a list of corrupted cpt files and rebuilds the average and secfilt tables from the measurements. Instead of a (catdir), the inputs are the full path or relative path to the image table and a list of the full or relative paths to the cpt files which are broken, ** with the .cpt extension removed**.

Again, repaired cpt, cps, cpm files are saved with the extension .fixed.

dvoverify

dvoverify will scan a dvo database or subset (by region) and verify the database integrity. At the moment, the only check it makes is to verify that the tables are FITS compliant. (ie, no check is made for internal consistency of the number of average, measure, or secfilt entries).