Heather's notes on MD addstar work

big table of tracking

MD field

cam label

cam dvo

cam queued *

cam done

skycal label

skycal dvo

skycal queued *

skycal done

merged dvo

merged

calibrated

created psps datastore

loaded to psps datastore

in psps

MD01

MD01.PV2.20140601.Cam

MD01.PV2.20140601.Skycal

MD01.PV2.20140601

PSPS_MD01_PV2

MD02

MD02.PV2.20140602.Cam

MD02.PV2.20140602.Skycal

MD02.PV2.20140602

PSPS_MD02_PV2

MD03

MD03.PV2.20140603.Cam

MD03.PV2.20140603.Skycal

MD03.PV2.20140603

PSPS_MD03_PV2

MD04

MD04.pv2.20130903

MD04.PV2.20140604.Cam

3394

yes

MD04.pv2.deepstack.20140506.Nx20140511.cal20140511

MD04.PV2.20140604.Skycal

295

yes

MD04.PV2.20140604

yes

yes?

PSPS_MD04_PV2

MD05

MD05.PV2.20140605.Cam

MD05.PV2.20140605.Skycal

MD05.PV2.20140605

PSPS_MD05_PV2

MD06

MD06.PV2.20140606.Cam

MD06.PV2.20140606.Skycal

MD06.PV2.20140606

PSPS_MD06_PV2

MD07

MD07.PV2.20140607.Cam

MD07.PV2.20140607.Skycal

MD07.PV2.20140607

PSPS_MD07_PV2

MD08

MD08.pv2.20131028

MD08.PV2.20140608.Cam

4159

yes

MD08.pv2.deepstack.20140529.Nx20140605.cal20140605

MD08.PV2.20140608.Skycal

298

yes

MD08.PV2.20140608

yes

yes?

PSPS_MD08_PV2

MD09

MD09.pv2.20131030

MD09.PV2.20140609.Cam

4176

yes

MD09.pv2.deepstack.20140512.Nx20140521.cal20140521

MD09.PV2.20140609.Skycal

297

yes

MD09.PV2.20140609

yes

PSPS_MD09_PV2

MD10

MD10.pv2.20131103

MD10.PV2.20140610.Cam

3972

yes

MD10.pv2.deepstack.20140520.Nx20140529.cal20140529

MD10.PV2.20140610.Skycal

299

yes

MD10.PV2.20140610

yes

PSPS_MD10_PV2

(*) refers to how many were queued by addstar - have to have good quality and state of full.

Here is a quick reference of what's done and not done bold is done

md field	camera label	camera mini	skycal label	skycal mini	mergedvodb	notes
MD01
MD02
MD03
MD04	MD04.pv2.20130903	MD04.PV2.20140604.Cam	MD04.pv2.deepstack.20140506.Nx20140511.cal20140511	MD04.PV2.20140604.Skycal	MD04.PV2.20140604*calib?
MD04	MD04.pv2.20130903	MD04.PV2.20140604.Cam	MD04.pv2.deepstack.20140506.Nx20140616.cal20140616	MD04.PV2.20140604.V2.Skycal	MD04.PV2.20140604.V2	*not yet made
MD05	MD05.pv2.20131022		* wait for mark
MD06
MD07	MD07.pv2.20131027		MD07.pv2.deepstack.20140605.Nx20140612.cal20140612	MD07.PV2.20140607.Skycal
MD07	MD07.pv2.20131027		MD07.pv2.deepstack.20140605.Nx20140620.cal20140620	MD07.PV2.20140607.V2.Skycal
MD08	MD08.pv2.20131028	MD08.PV2.20140608.Cam	MD08.pv2.deepstack.20140529.Nx20140605.cal20140605	MD08.PV2.20140608.Skycal	MD08.PV2.20140608 *calib?
MD08	MD08.pv2.20131028	MD08.PV2.20140608.Cam	MD08.pv2.deepstack.20140529.Nx20140620.cal20140620	MD08.PV2.20140608.V2.Skycal
MD09	MD09.pv2.20131030	MD09.PV2.20140609.Cam	MD09.pv2.deepstack.20140512.Nx20140521.cal20140521	MD09.PV2.20140609.Skycal	MD09.PV2.20140609 *need calib
MD09	MD09.pv2.20131030	MD09.PV2.20140609.Cam	MD09.pv2.deepstack.20140512.Nx20140618.cal20140618	MD09.PV2.20140609.V2.Skycal
MD10	MD10.pv2.20131103	MD10.PV2.20140610.Cam	MD10.pv2.deepstack.20140520.Nx20140529.cal20140529	MD10.PV2.20140610.Skycal	MD10.PV2.20140610 *need calib
MD10	MD10.pv2.20131103	MD10.PV2.20140610.Cam	MD10.pv2.deepstack.20140520.Nx20140619.cal20140619	MD10.PV2.20140610.V2.Skycal

Where do things live

field	host
MD01	ipp007
MD02	ipp008
MD03	ipp009
MD04	ipp007
MD05	ipp008
MD06	ipp010
MD07	ipp007
MD08	ipp008
MD09	ipp009
MD10	ipp010

how to addstar

• MD04

addstar.off
survey.add.addstar MD04.PV2.20140604.Cam MD04.pv2.20130903 MD04.PV2.20140604 MD04.PV2.20140604.Cam cam
survey.add.addstar MD04.PV2.20140604.Skycal MD04.pv2.deepstack.20140506.Nx20140511.cal20140511  MD04.PV2.20140604 MD04.PV2.20140604.Skycal skycal
add.minidvodb MD04.PV2.20140604.Cam MD04.PV2.20140604.Cam MD04.PV2.20140604 5 GPC1
add.minidvodb MD04.PV2.20140604.Skycal MD04.PV2.20140604.Skycal MD04.PV2.20140604 5 GPC1
add.label MD04.PV2.20140604.Cam
add.label MD04.PV2.20140604.Skycal
add.label MD04.PV2.20140604

• MD08

addstar.off
survey.add.addstar MD08.PV2.20140608.Cam MD08.pv2.20131028 MD08.PV2.20140608 MD08.PV2.20140608.Cam cam
add.minidvodb MD08.PV2.20140608.Cam MD08.PV2.20140608.Cam MD08.PV2.20140608 5 GPC1
add.minidvodb MD08.PV2.20140608.Skycal MD08.PV2.20140608.Skycal MD08.PV2.20140608 5 GPC1
add.label MD08.PV2.20140608.Cam
add.label MD08.PV2.20140608.Skycal
add.label MD08.PV2.20140608

add skycal later

• MD09

addstar.off
survey.add.addstar MD09.PV2.20140609.Cam MD09.pv2.20131030 MD09.PV2.20140609 MD09.PV2.20140609.Cam cam
survey.add.addstar MD09.PV2.20140609.Skycal  MD09.pv2.deepstack.20140512.Nx20140521.cal20140521   MD09.PV2.20140609 MD09.PV2.20140609.Skycal skycal
add.minidvodb MD09.PV2.20140609.Cam MD09.PV2.20140609.Cam MD09.PV2.20140609 5 GPC1
add.minidvodb MD09.PV2.20140609.Skycal MD09.PV2.20140609.Skycal MD09.PV2.20140609 5 GPC1
add.label MD09.PV2.20140609.Cam
add.label MD09.PV2.20140609.Skycal
add.label MD09.PV2.20140609

• MD10

addstar.off
survey.add.addstar MD10.PV2.20140610.Cam MD10.pv2.20131103 MD10.PV2.20140610 MD10.PV2.20140610.Cam cam
survey.add.addstar MD10.PV2.20140610.Skycal  MD10.pv2.deepstack.20140520.Nx20140529.cal20140529  MD10.PV2.20140610 MD10.PV2.20140610.Skycal skycal
add.minidvodb MD10.PV2.20140610.Cam MD10.PV2.20140610.Cam MD10.PV2.20140610 5 GPC1
add.minidvodb MD10.PV2.20140610.Skycal MD10.PV2.20140610.Skycal MD10.PV2.20140610 5 GPC1
add.label MD10.PV2.20140610.Cam
add.label MD10.PV2.20140610.Skycal
add.label MD10.PV2.20140610

useful queries for monitoring

• check information about addstar Runs

select minidvodb_group, stage, state, fault,  count(*) from addRun left join addProcessedExp using (add_id)  where minidvodb_group like 'MD__.PV2.201406%' group by minidvodb_group, stage, state, fault;

• check information about minidvodbs

select minidvodb_name, state, fault from minidvodbRun left join minidvodbProcessed using (minidvodb_id)  where minidvodb_name like 'MD__.PV2.201406%';

• slower query, to be used to find out if all the minis are done and with no faults:

 select minidvodbRun.minidvodb_name, minidvodbRun.state, minidvodbProcessed.fault,count(*) from minidvodbRun left join minidvodbProcessed using (minidvodb_id) left join addRun using (minidvodb_name)  where minidvodbRun.minidvodb_name like 'MD__.PV2.201406%' group by minidvodb_name;

how to addstar part 2 (merge)

Merging is special for MD:

1. queue and merge the skycal (should be 1 dvodb)

mergedvodb.merge.on
survey.add.mergedvodb sky MD04.PV2.20140604  MD04.PV2.20140604.Skycal
add.mergedvodb MD04.PV2.20140604
mergedvodb.merge.off

2. copy hosttable, edit it for MDxx

3. dvodist -out the thing

4. queue and merge the camruns

calibration steps

create datastores

done for each MD

dsprodtool --add PSPS_MD01_PV2 --type IPP-PSPS --description 'PSPS MD PV2 datastore'

queue for psps

wait for new minidvodb and then go