This page contains scripts made by Mike Liu, which aren't really a cookbook but show examples for many dvo commands. ​Mike Liu's three dvo challenge slides from the 2008 Heidelberg meeting also contain useful examples.

If you have questions, the most sensible forum is the ​ps-ipp-users@ifa.hawaii.edu mailing list (subscription page), and the ​ps-ipp-users@ifa.hawaii.edu list archive (accessible to subscribers) for looking for previous answers. You need a username/pw to be able to see the web page, the usual combination for the science consortium pages works; and then you need your email address and mail list password to be able to see the archive.

TOC

FAQ / Suggestions for Changes

Please post both new questions about dvo, as well as answers to old ones if you have them!

Questions

• what is the comment character for DVO?
• errors in 2MASS phot?
• line continuation character (for extra long lines)
• is it possible to extract the detection limit for a given ra,dec or image,x,y?

Change requests/suggestions

• command to clear all variables from memory
• change command name from 'vstat' to 'vstats' (or create an alias)
• 'czplot' should be able to work without specifying a min or max value
• need columnar printing tool for printing vectors
• major nuisance: how to know which variables are available for current catalog? Need "help catdir" or something like that
• when writing to file with 'write', should generate column headings (or able to add comments)
• consistency needed between input files and typing on command line, e.g. "#" and "!" usage

Examples

These examples assume that you have a local IPP installation.

DVO tutorial

============================================================ DVO tutorial with Gene Magnier 03/10/08 ============================================================

​http://kiawe.ifa.hawaii.edu/Elixir/Ohana/DVO/

ra values are in DEGREES

ssh host_with_dvo_installed

Make sure you source the psconfig script that's appropriate for your shell, and set up your environment for using ipp, e.g.:

psconfig ipp-2.5

Mucking around with dvo

script dvo.logfile make recording of session dvo

# Set up variables with catdir names $catdir_2mass = /local/path/to/catdir.2mass 2MASS catalog $catdir_merge = /local/path/to/catdir.2mass/catdir.merge 2MASS+USNOB catalog

$a = 0 set scalar echo $a $a = 5*6 do math

echo {5*6} things in {} mean math operation

create x 0 100 0.1 make an array full of values echo x[] echo x[0] echo x[999]

set y = x^{2 do an array operation set b = $a}2 need to handle scalars with "$", but not arrays

create x 0 100 0.1 set x = y limits x y set limits for plotting box box set y = x^{2 plot x y png -name plot.png ps -name plot.ps}

mcreate foo 100 100 make array of 100x100 ("m" = matrix) set xr = xramp(foo) 'xramp' is a function, makes 2d image as a slope in x-axis set yr = xramp(foo) 'xramp' is a function, makes 2d image as a slope in y-axis set zr = xr^{2 + yr}2 stats xr give image statistics

tv xr 0 100 display the results with a range (must give a range) clear -image clear -graph

buffers list all arrays vectors list all images ?? list all scalars

Messing around with databases

catdir $catdir_2mass 2MASS catalog catdir $catdir_merge 2MASS+USNOB catalog

region 0 0 0.5 set center of region to (RA,Dec)=(0,0) with 0.5 deg half-box size region 0 0 0.5 sin sine projection region 0 0 0.5 tan tan projection region 0 0 90 ait ; cgrid Aitoff projection of the whole sky & show celestial grid

region 0 0 0.5 tan tan projection pmeasure -all -m 10 15 -pt 7 -c blue plot all mag=[10,15] detections (J, H, and K)

using blue point symbol

pmeasure -all -m 10 15 -pt 7 -c blue -photcode 2MASS_J plot only J-band (bigger symbol = brighter) pmeasure -all -m 10 15 -pt 7 -c red -photcode 2MASS_K

mextract -region ra, dec, photcode, mag extract info from define plotting region region show current value of region vectors show results cplot ra dec plot results on celestial sky

pcat -c red overlay box sizes for sub-regions

skyregion 0 1 0 1 set search region (alternative to using 'region' command) mextract RA, DEC, MAG now don't have to use "-region" flag

use capitals to define different input arrays (but catalog fields are case-insensitive)

czplot ra dec mag 10 15 plot {ra,dec} results with mag value encoded in symbol size

must be given the min & max value

mextract -region RA, DEC, MAG where (MAG < 12.0) restricted extraction mextract -region RA, DEC, MAG where (MAG < 12.0) && (MAG > 2.0) && (RA < 0.25) cplot RA DEC -c red

delete ra dec photcode mag RA DEC MAG avextract -region ra dec 2MASS_J 2MASS_K avextract -region ra dec 2MASS_J 2MASS_K 2MASS_J:err vectors (note that resulting array names can't start with a number, so "v_" appended) clear czplot ra dec v_2MASS_J 15 10 -c blue -pt 7 pmeasure -all -m 10 15 -pt 7 -c red -photcode 2MASS_J

avextract ra, dec, 2MASS_J, glon, glat where (abs(glat) > 10) extract based on galactic latitude

limits -n 1 -0.5 3.5 17.0 8.0; clear; box plot {v_2MASS_J - v_2MASS_K} v_2MASS_K

NOTE: now that window 1 is active, this will be the default range for "-region"

dev -n 0 change back to device 0 avextract -region ra, dec, 2MASS_J, 2MASS_K where (2MASS_J-2MASS_K > 0) && (2MASS_J-2MASS_K < 0.5) plot -n 1 -c red {v_2MASS_J - v_2MASS_K} v_2MASS_K

cursor poke plotting window, click on mouse to return coordinates

"q" to quit

write -f "%10.6f %10.6f %7.3f" test.dat ra dec dump to file !more test.dat "!" is shell escape exec more test.dat also shell escape

list data -x "head test.dat" input files

subset r = ra if (dec < 0.2) subset d = dec if (dec < 0.2) subset m = mag if (dec < 0.2)

set bool = (dec < 0.2) create boolean variable for selection setset r = ra if bool

macro foo create a macro

echo $1 for i 0 10 echo $i end end

foo 5 foo sdf sdfsd sdfdsf

input filename import file of commands

in the file, can use "#" as comment character (though this doesn't work on the command line directly) cannot use "!" in the file as shell escape, must use "exec"

vstat get statistics for a vector (no "s" on the command name)

results are stored in variables $MEAN, $MIN, etc.

stats get statistics for a scalar

sort ra dec mag will re-sort the variables using first variable as the sorting field

----------------------------------------------------------------------

the 'date' command is what you want here:

dvo: date -h USAGE: date [-var variable] [-seconds] [-reftime seconds]

• by itself, just echoes the date & time
• to set a variable with the date, add -var name, where 'name' is the name of your variable. by default, this is a human readable date, eg Wed Mar 12 08:30:11 2008
• to calculate time differences, add the -seconds flag, and the value will be in seconds since 1970 Jan 01 00:00:00. NOTE: There is a bug in that the variable is represented as a float, and the value is now > 1.2e9 seconds. The difference is thus not very accurate. To get around this, use the -reftime feature to report the seconds since a more recent date. A value of something like 1205340000 works at the moment.

examples:

dvo: date Wed Mar 12 08:30:11 2008

dvo: date -var foo -seconds dvo: echo $foo 1205346664

dvo: date -var foo -seconds -reftime 1205340000 dvo: echo $foo 6700 dvo: date -var bar -seconds -reftime 1205340000 dvo: echo $bar 6708 dvo: echo {$bar - $foo} 8

Sample dvo script

You can save this as nstars.dvo and then execute all the commands from the dvo shell with

input nstars.dvo

#---------------------------------------------------------------------------# # Re-create 2MASS part of extraction used by Reid et al (2007, Paper XI) # used to find mid-M dwarfs # 02/11/08 M. Liu #---------------------------------------------------------------------------#

# define sky region and output file # (remember that RA is in *degrees*) $OUTFILE = junk #$OUTFILE = nstars.2mass $RA_MIN = 0*15 $RA_MAX = 1*15 $DEC_MIN = -0 $DEC_MAX = 20

#$OUTFILE = big.2mass #$RA_MIN = 0*15 #$RA_MAX = 4*15 #$DEC_MIN = -0 #$DEC_MAX = 20

# cut by galactic latitude $GLAT_MIN = 10

# define selection criteria $SELECT = (2MASS_J < 12) && (2MASS_H - 2MASS_K > 0.23) && (abs(glat) > $GLAT_MIN)

#---------------------------------------------------------------------- # choose 2MASS catdir $catdir_2mass

# define region skyregion $RA_MIN $RA_MAX $DEC_MIN $DEC_MAX clear limits $RA_MIN $RA_MAX $DEC_MIN $DEC_MAX box grid

# do extraction and keep track of time echo "running AVEXTRACT" echo "extraction area (sq.degs) = " {($RA_MAX- $RA_MIN) * ($DEC_MAX - $DEC_MIN)} date date -var d1 -seconds avextract ra dec 2MASS_J 2MASS_H 2MASS_K glon glat where $SELECT echo " found" ra[] "objects" date -var d2 -seconds echo " run time =" {$d2-$d1} "seconds"

# plot results vstat v_2MASS_J zplot ra dec v_2MASS_J $MAX $MIN zplot ra dec v_2MASS_J $MIN $MAX -c red zplot ra dec v_2MASS_J $MAX $MIN -c blue -pt 7

# print results to file and sort by RA write -f "%10.6f%+010.6f %7.3f %7.3f %7.3f %12.6f %10.6f" tmp.dat ra dec v_2MASS_J v_2MASS_H v_2MASS_K glon glat exec sort -nk 1,1 tmp.dat > $OUTFILE

echo " " echo " FINISHED " echo "wrote results to outfile = " $OUTFILE echo " "

DVO tutorial (old)

Since this tutorial is from 2007, some information here may be outdated by now.

================================================== DVO tutorial with Gene Magnier 02/13/07 ==================================================

ssh local_host_with_dvo_installed

copy over test materials (works only on IfA cluster, if at all)

cp -r ~ipp/715624o .

DVO overview

sky is stored in tables, can adjust density of RA/DEC divisions default divisions are based on those used for HST Guide Star Catalog

photometry systems stored in ~ipp/psconfig/share/ippconfig/dvo.photcodes

1st five lines give *average* magnitudes systems (not for any specific night) "ref" lines are externally provided photometry "dep" lines are average measurements for a given instrument (e.g. one Megacam chip)

data from specific nights are strapped onto this average these can then be set onto the master photometric system

info for DVO commands

​http://kiawe.ifa.hawaii.edu/Elixir/Ohana/DVO/

goofing around DVO

module navigate tool for graphically exploring database

catdir $catdir_merge point to catalog of interest

each subdirectoy is a band in DEC, containing FITS photometry tables "Images.dat" = contains list of images in directory (none in this case) this test directory is Tycho, 2MASS, and USNO-B merged

region 0 30 90 sin set viewing region

0,0 = center of the field in degrees 90 = radius of view sin = sin projection (ait = Aitoff projection, also "tan", "par", "gls")

cgrid show celetial sphere

region 0 0 0.5 tan; cgrid zoom into a region

pmeasure -all -m 10 15 plot a bunch of data

help style show all plotting options

style -pt 7 set for circles clear; pmeasure -all -m 10 15 redraw

region 0 0 0.05 tan; cgrid choose smaller region clear; pmeasure -all -m 10 15 redraw $MAG = 10 set plotting region info $dMAG = 5 navigate poke around

go into plotting window

m show data for a given object

shows "final" grizy magnitude then shows individual measurements from diff catalogs

x quit navigate h show list of commands S redraw the stars

pcat show boundaries of catalogs that contribute to region

mextract extract vector of measurements from database (e.g. RA, DEC) avextract extract vector of measurements averaged for the merged catalog imextract extract statistics about images (e.g. rms)

mextract help show list of all possible columns mextract ra get list of RAs that overlap the defined region mextract mag:inst mextract mag:cat avextract RA get lists of RAs for merged catalog ("RA" is new name) vectors show list of defined vectors

echo ra[] show number of elements echo ra[5000] show value for an individual element

for i 0 10 echo ra[$i] print first 10 elements

ra[7] = 10 change value of vector

mextract dec lim -n 1 ra dec ; clear ; box ; plot ra dec -x 2 -pt 7 plot data

write show list of options write <file> <list of vectors> -f "format (C-style)" write to file

Look around the 715624o/ stuff

*.sx are Sextractor output *.smp has header info, data in ASCII (reorganization of Sextractor output), astrometry *.xmp has astrometry info

and let's put some data into DVO

cd 715642o/ addstar -D CAMERA megacam addstar -D CAMERA megacam -help addstar -D CAMERA megacam 715624o00.smp

produces "default/" directory which contains the database

addstar -D CAMERA megacam "715624o??.smp" enter them all into the database

go into dvo

imlist show list of images entered in default/

(works b/c have set "default/" as nominal setting in "catdir" command")

gives field center {RA,DEC}, date, photcode info, airmass, relative phot info

region 280.4 0.15 1 set plotting region images show image regions pmeasure -all -m 10 20 plot all data cursor poke around images and mark locations

can press any key (e.g. "1", "g", whatever) and location is saved "q" to quit this

echo $R1 show results from 'cursor' results echo $Rg

region $R1 $D1 0.1 pmeasure -all -m 10 20 ctrl-R search old command history

make some CMDs from 2MASS

catdir $catdir_merge region 0 0 2 cmd 2MASS_J - 2MASS_H : 2MASS_H generate vectors for plotting CMD ("xv" and "yv") vectors lim -n 1 xv yv set box limits based on vectors (for window 1) plot xv yv -x 2 -pt 2 plot CMD

lim -1 4 18 5 set plotting limits clear; box; plot xv yv cursor

eyeball two regions, type "1" and "2"

subset hs = yv if (xv > $X1) && (xv < $X2) && (yv < $Y1) && (yv > $Y2) select subset of data subset jhs = xv if (xv > $X1) && (xv < $X2) && (yv < $Y1) && (yv > $Y2) select subset of data plot -c red jhs hs

do extraction along a line

set yl = 11 - xv vector that defines a line subset jhs = xv if (yv< yl) subset hhs = xv if (yv< yl) plot -c blue jhs hs -n 1

set a boolean expression to do selection

set flag = (xv > $1) subset jhs = xv if flag subset hs = xv if flag plot -c green jhs hs -n 1

can also do data extraction in original 'mextract' commands

can input file to dvo

input <filename>

to add data from an external source

make a copy of the "dvo.photcodes" file for yourself

add new data are "ref" data, assign a unique number for 1st column choose a unique name, e.g. "othercamera"

make a file with columns of {RA DEC magnitude error}

addstar -D RADIUS 2.0 -help addstar -D RADIUS 2.0 -ref <filename> -photcode <othercamera> -time 2007/02/13,00:300:00