| Version 2 (modified by , 17 years ago) ( diff ) |
|---|
This page contains scripts made by Mike Liu, which aren't really a cookbook but show examples for many dvo commands. Mike Liu's three dvo challenge slides from the 2008 Heidelberg meeting also contain useful examples.
If you have questions, the most sensible forum is the ps-ipp-users@ifa.hawaii.edu mailing list (subscription page), and the ps-ipp-users@ifa.hawaii.edu list archive (accessible to subscribers) for looking for previous answers. You need a username/pw to be able to see the web page, the usual combination for the science consortium pages works; and then you need your email address and mail list password to be able to see the archive.
FAQ / Suggestions for Changes
Please post both new questions about dvo, as well as answers to old ones if you have them!
Questions
- what is the comment character for DVO?
- errors in 2MASS phot?
- line continuation character (for extra long lines)
- is it possible to extract the detection limit for a given ra,dec or image,x,y?
Change requests/suggestions
- command to clear all variables from memory
- change command name from 'vstat' to 'vstats' (or create an alias)
- 'czplot' should be able to work without specifying a min or max value
- need columnar printing tool for printing vectors
- major nuisance: how to know which variables are available for current catalog? Need "help catdir" or something like that
- when writing to file with 'write', should generate column headings (or able to add comments)
- consistency needed between input files and typing on command line, e.g. "#" and "!" usage
Examples
These examples assume that you have a local IPP installation.
DVO tutorial
============================================================
DVO tutorial with Gene Magnier
03/10/08
============================================================
http://kiawe.ifa.hawaii.edu/Elixir/Ohana/DVO/
**ra values are in DEGREES**
ssh host_with_dvo_installed
Make sure you source the psconfig script that's appropriate for your shell,
and set up your environment for using ipp, e.g.:
psconfig ipp-2.5
Mucking around with dvo
> script dvo.logfile make recording of session
> dvo
# Set up variables with catdir names
$catdir_2mass = /local/path/to/catdir.2mass 2MASS catalog
$catdir_merge = /local/path/to/catdir.2mass/catdir.merge 2MASS+USNOB catalog
$a = 0 set scalar
echo $a
$a = 5*6 do math
echo {5*6} things in {} mean math operation
create x 0 100 0.1 make an array full of values
echo x[]
echo x[0]
echo x[999]
set y = x^2 do an array operation
set b = $a^2 need to handle scalars with "$", but not arrays
create x 0 100 0.1
set x = y
limits x y set limits for plotting box
box
set y = x^2
plot x y
png -name plot.png
ps -name plot.ps
mcreate foo 100 100 make array of 100x100 ("m" = matrix)
set xr = xramp(foo) 'xramp' is a function, makes 2d image as a slope in x-axis
set yr = xramp(foo) 'xramp' is a function, makes 2d image as a slope in y-axis
set zr = xr^2 + yr^2
stats xr give image statistics
tv xr 0 100 display the results with a range (must give a range)
clear -image
clear -graph
buffers list all arrays
vectors list all images
?? list all scalars
Messing around with databases
catdir $catdir_2mass 2MASS catalog
catdir $catdir_merge 2MASS+USNOB catalog
region 0 0 0.5 set center of region to (RA,Dec)=(0,0) with 0.5 deg half-box size
region 0 0 0.5 sin sine projection
region 0 0 0.5 tan tan projection
region 0 0 90 ait ; cgrid Aitoff projection of the whole sky & show celestial grid
region 0 0 0.5 tan tan projection
pmeasure -all -m 10 15 -pt 7 -c blue plot all mag=[10,15] detections (J, H, and K)
using blue point symbol
pmeasure -all -m 10 15 -pt 7 -c blue -photcode 2MASS_J plot only J-band (bigger symbol = brighter)
pmeasure -all -m 10 15 -pt 7 -c red -photcode 2MASS_K
mextract -region ra, dec, photcode, mag extract info from define plotting region
region show current value of region
vectors show results
cplot ra dec plot results on celestial sky
pcat -c red overlay box sizes for sub-regions
skyregion 0 1 0 1 set search region (alternative to using 'region' command)
mextract RA, DEC, MAG now don't have to use "-region" flag
use capitals to define different input arrays
(but catalog fields are case-insensitive)
czplot ra dec mag 10 15 plot {ra,dec} results with mag value encoded in symbol size
must be given the min & max value
mextract -region RA, DEC, MAG where (MAG < 12.0) restricted extraction
mextract -region RA, DEC, MAG where (MAG < 12.0) && (MAG > 2.0) && (RA < 0.25)
cplot RA DEC -c red
delete ra dec photcode mag RA DEC MAG
avextract -region ra dec 2MASS_J 2MASS_K
avextract -region ra dec 2MASS_J 2MASS_K 2MASS_J:err
vectors (note that resulting array names can't start with a number, so "v_" appended)
clear
czplot ra dec v_2MASS_J 15 10 -c blue -pt 7
pmeasure -all -m 10 15 -pt 7 -c red -photcode 2MASS_J
avextract ra, dec, 2MASS_J, glon, glat where (abs(glat) > 10) extract based on galactic latitude
limits -n 1 -0.5 3.5 17.0 8.0; clear; box
plot {v_2MASS_J - v_2MASS_K} v_2MASS_K
NOTE: now that window 1 is active, this will be the default range for "-region"
dev -n 0 change back to device 0
avextract -region ra, dec, 2MASS_J, 2MASS_K where (2MASS_J-2MASS_K > 0) && (2MASS_J-2MASS_K < 0.5)
plot -n 1 -c red {v_2MASS_J - v_2MASS_K} v_2MASS_K
cursor poke plotting window, click on mouse to return coordinates
"q" to quit
write -f "%10.6f %10.6f %7.3f" test.dat ra dec dump to file
!more test.dat "!" is shell escape
exec more test.dat also shell escape
list data -x "head test.dat" input files
subset r = ra if (dec < 0.2)
subset d = dec if (dec < 0.2)
subset m = mag if (dec < 0.2)
set bool = (dec < 0.2) create boolean variable for selection
setset r = ra if bool
macro foo create a macro
echo $1
for i 0 10
echo $i
end
end
foo 5
foo sdf sdfsd sdfdsf
input filename import file of commands
in the file, can use "#" as comment character (though this doesn't work on the command line directly)
cannot use "!" in the file as shell escape, must use "exec"
vstat get statistics for a vector (no "s" on the command name)
results are stored in variables $MEAN, $MIN, etc.
stats get statistics for a scalar
sort ra dec mag will re-sort the variables using first variable as the sorting field
----------------------------------------------------------------------
the 'date' command is what you want here:
dvo: date -h
USAGE: date [-var variable] [-seconds] [-reftime seconds]
* by itself, just echoes the date & time
* to set a variable with the date, add -var name, where 'name' is the
name of your variable. by default, this is a human readable date, eg
Wed Mar 12 08:30:11 2008
* to calculate time differences, add the -seconds flag, and the value
will be in seconds since 1970 Jan 01 00:00:00. NOTE: There is a bug
in that the variable is represented as a float, and the value is now >
1.2e9 seconds. The difference is thus not very accurate. To get
around this, use the -reftime feature to report the seconds since a
more recent date. A value of something like 1205340000 works at the
moment.
examples:
dvo: date
Wed Mar 12 08:30:11 2008
dvo: date -var foo -seconds
dvo: echo $foo
1205346664
dvo: date -var foo -seconds -reftime 1205340000
dvo: echo $foo
6700
dvo: date -var bar -seconds -reftime 1205340000
dvo: echo $bar
6708
dvo: echo {$bar - $foo}
8
Sample dvo script
You can save this as nstars.dvo and then execute all the commands from the dvo shell with
input nstars.dvo
#---------------------------------------------------------------------------#
# Re-create 2MASS part of extraction used by Reid et al (2007, Paper XI)
# used to find mid-M dwarfs
# 02/11/08 M. Liu
#---------------------------------------------------------------------------#
# define sky region and output file
# (remember that RA is in *degrees*)
$OUTFILE = junk
#$OUTFILE = nstars.2mass
$RA_MIN = 0*15
$RA_MAX = 1*15
$DEC_MIN = -0
$DEC_MAX = 20
#$OUTFILE = big.2mass
#$RA_MIN = 0*15
#$RA_MAX = 4*15
#$DEC_MIN = -0
#$DEC_MAX = 20
# cut by galactic latitude
$GLAT_MIN = 10
# define selection criteria
$SELECT = (2MASS_J < 12) && (2MASS_H - 2MASS_K > 0.23) && (abs(glat) > $GLAT_MIN)
#----------------------------------------------------------------------
# choose 2MASS
catdir $catdir_2mass
# define region
skyregion $RA_MIN $RA_MAX $DEC_MIN $DEC_MAX
clear
limits $RA_MIN $RA_MAX $DEC_MIN $DEC_MAX
box
grid
# do extraction and keep track of time
echo "running AVEXTRACT"
echo "extraction area (sq.degs) = " {($RA_MAX- $RA_MIN) * ($DEC_MAX - $DEC_MIN)}
date
date -var d1 -seconds
avextract ra dec 2MASS_J 2MASS_H 2MASS_K glon glat where $SELECT
echo " found" ra[] "objects"
date -var d2 -seconds
echo " run time =" {$d2-$d1} "seconds"
# plot results
vstat v_2MASS_J
zplot ra dec v_2MASS_J $MAX $MIN
zplot ra dec v_2MASS_J $MIN $MAX -c red
zplot ra dec v_2MASS_J $MAX $MIN -c blue -pt 7
# print results to file and sort by RA
write -f "%10.6f%+010.6f %7.3f %7.3f %7.3f %12.6f %10.6f" tmp.dat ra dec v_2MASS_J v_2MASS_H v_2MASS_K glon glat
exec sort -nk 1,1 tmp.dat > $OUTFILE
echo " "
echo "** FINISHED **"
echo "wrote results to outfile = " $OUTFILE
echo " "
DVO tutorial (old)
Since this tutorial is from 2007, some information here may be outdated by now.
==================================================
DVO tutorial with Gene Magnier
02/13/07
==================================================
ssh local_host_with_dvo_installed
copy over test materials (works only on IfA cluster, if at all)
> cp -r ~ipp/715624o .
DVO overview
sky is stored in tables, can adjust density of RA/DEC divisions
default divisions are based on those used for HST Guide Star Catalog
photometry systems stored in ~ipp/psconfig/share/ippconfig/dvo.photcodes
1st five lines give *average* magnitudes systems (not for any specific night)
"ref" lines are externally provided photometry
"dep" lines are average measurements for a given instrument (e.g. one Megacam chip)
data from specific nights are strapped onto this average
these can then be set onto the master photometric system
info for DVO commands
http://kiawe.ifa.hawaii.edu/Elixir/Ohana/DVO/
goofing around DVO
module navigate tool for graphically exploring database
catdir $catdir_merge point to catalog of interest
each subdirectoy is a band in DEC, containing FITS photometry tables
"Images.dat" = contains list of images in directory (none in this case)
this test directory is Tycho, 2MASS, and USNO-B merged
region 0 30 90 sin set viewing region
0,0 = center of the field in degrees
90 = radius of view
sin = sin projection (ait = Aitoff projection, also "tan", "par", "gls")
cgrid show celetial sphere
region 0 0 0.5 tan; cgrid zoom into a region
pmeasure -all -m 10 15 plot a bunch of data
help style show all plotting options
style -pt 7 set for circles
clear; pmeasure -all -m 10 15 redraw
region 0 0 0.05 tan; cgrid choose smaller region
clear; pmeasure -all -m 10 15 redraw
$MAG = 10 set plotting region info
$dMAG = 5
navigate poke around
go into plotting window
m show data for a given object
shows "final" grizy magnitude
then shows individual measurements from diff catalogs
x quit navigate
h show list of commands
S redraw the stars
pcat show boundaries of catalogs that contribute to region
mextract extract vector of measurements from database (e.g. RA, DEC)
avextract extract vector of measurements averaged for the merged catalog
imextract extract statistics about images (e.g. rms)
mextract help show list of all possible columns
mextract ra get list of RAs that overlap the defined region
mextract mag:inst
mextract mag:cat
avextract RA get lists of RAs for merged catalog ("RA" is new name)
vectors show list of defined vectors
echo ra[] show number of elements
echo ra[5000] show value for an individual element
for i 0 10
echo ra[$i] print first 10 elements
ra[7] = 10 change value of vector
mextract dec
lim -n 1 ra dec ; clear ; box ; plot ra dec -x 2 -pt 7 plot data
write show list of options
write <file> <list of vectors> -f "format (C-style)" write to file
Look around the 715624o/ stuff
*.sx are Sextractor output
*.smp has header info, data in ASCII (reorganization of Sextractor output), astrometry
*.xmp has astrometry info
and let's put some data into DVO
> cd 715642o/
> addstar -D CAMERA megacam
> addstar -D CAMERA megacam -help
> addstar -D CAMERA megacam 715624o00.smp
produces "default/" directory which contains the database
> addstar -D CAMERA megacam "715624o??.smp" enter them all into the database
go into dvo
imlist show list of images entered in default/
(works b/c have set "default/" as nominal setting in "catdir" command")
gives field center {RA,DEC}, date, photcode info, airmass, relative phot info
region 280.4 0.15 1 set plotting region
images show image regions
pmeasure -all -m 10 20 plot all data
cursor poke around images and mark locations
can press any key (e.g. "1", "g", whatever) and location is saved
"q" to quit this
echo $R1 show results from 'cursor' results
echo $Rg
region $R1 $D1 0.1
pmeasure -all -m 10 20
ctrl-R search old command history
make some CMDs from 2MASS
catdir $catdir_merge
region 0 0 2
cmd 2MASS_J - 2MASS_H : 2MASS_H generate vectors for plotting CMD ("xv" and "yv")
vectors
lim -n 1 xv yv set box limits based on vectors (for window 1)
plot xv yv -x 2 -pt 2 plot CMD
lim -1 4 18 5 set plotting limits
clear; box; plot xv yv
cursor
eyeball two regions, type "1" and "2"
subset hs = yv if (xv > $X1) && (xv < $X2) && (yv < $Y1) && (yv > $Y2) select subset of data
subset jhs = xv if (xv > $X1) && (xv < $X2) && (yv < $Y1) && (yv > $Y2) select subset of data
plot -c red jhs hs
do extraction along a line
set yl = 11 - xv vector that defines a line
subset jhs = xv if (yv< yl)
subset hhs = xv if (yv< yl)
plot -c blue jhs hs -n 1
set a boolean expression to do selection
set flag = (xv > $1)
subset jhs = xv if flag
subset hs = xv if flag
plot -c green jhs hs -n 1
can also do data extraction in original 'mextract' commands
can input file to dvo
input <filename>
to add data from an external source
make a copy of the "dvo.photcodes" file for yourself
add new data are "ref" data, assign a unique number for 1st column
choose a unique name, e.g. "othercamera"
make a file with columns of {RA DEC magnitude error}
> addstar -D RADIUS 2.0 -help
> addstar -D RADIUS 2.0 -ref <filename> -photcode <othercamera> -time 2007/02/13,00:300:00
