IPP Czar Daily Checks

These are modeled after the summit wiki's obs log. Basically, there is a template of checks to be done each day, to make sure PS1/PS2 are ready for the night, and to track the observations. For us, we have maintenance tasks to do that make processing go much more smoothly. The definitive list of procedures (if having a problem) is listed on Processing ​http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/Processing

Morning Checks

[ ] Check that summitcopy /registration are complete, help out the stuck exposures

[ ] Check ippMonitor for problems for nightly science / mops processing (restart czarpoll if not running)

[ ] Check ganglia for cluster load / down machines (notify if ganglia not working)

[ ] Check disk space on machines, notify if running out

[ ] LANL checks (?)

[ ] PV3 checks (?)

[ ] nightly science is complete for previous night

[ ] stdlocal tasks (?)

Afternoon Checks

[ ] LANL checks (?)

[ ] PV3 checks (?)

[ ] stdlocal tasks (?)

[ ] check space / load on db machines

[ ] Check / restart apache for nebulous, if needed

[ ] Restart of stdscience , summitcopy, registration, cleanup, others in check_system.sh

Nightly Checks

[ ] Verify that summitcopy / registration are downloading and registering images

[ ] Verify that stdscience and friends are doing nightly processing

[ ] Check machine up/down statuses, note problems, set to repair those which are down

Other Notes

Use this space to add other czar related information.