__group__	ticket	summary	component	version	type	owner	status	created	_changetime	_description	_reporter
Milestone 	1288	relphot needs to be robust against NaN and should probably do flag-checking	DVO	current	defect	eugene	new	2009-08-03T05:25:01-10:00	2009-08-03T05:25:01-10:00	"When relphot is used to compute average magnitudes etc., i currently does not trap NaNs. That means an NaN in any individual measurement or error turns the entire average magnitude or error into an NaN, which then messes up colour computations.

Measurements with ""fatal"" flags set (e.g. saturation, CR, etc. - to be defined in detail) should probably not be included in the average, either. There is an int STAR_BAD used in some places of source:trunk/Ohana/src/relphot/src/relphot.c but it doesn't seem to be used in source:trunk/Ohana/src/relphot/src/liststats.c as far as I can tell.

See changeset [24979] from my Ohana branch for pseudocode with a suggested fix for the NaN part.

(BTW: which liststats.c is the ""proper"" one, the one in photdbc or relphot?)"	Sebastian Jester
Milestone 	1121	Should be able to define per-camera reduction classes without parent in global recipes/*.config file	ippconfig	2.5	enhancement	eugene	assigned	2008-08-01T10:44:22-10:00	2008-08-04T10:15:59-10:00	"As promised...: 

At present, a per-camera config file, e.g. sdssmosaic/ppImage.config, can only define a symbolic name for a collection of parameters if that same symbolic name is included in the global recipes/ppImage.config. Since the entry in recipes/ppImage.config only has to be present, but can be empty, that entry carries no information. As a result, we need to change two files consistently whenever a reduction class is added or renamed.

So to remove a source of error and confusion (and repeated emails ;-) the config file system should allow per-camera config files to add new symbolic names that do *not* already have a parent in the recipes/*.config equivalent."	jester@…
Milestone 	817	all ps* software should have a license attached to it	misc	unspecified	defect	eugene	assigned	2006-09-08T17:12:11-10:00	2010-05-15T11:51:23-10:00	"most of packages don't even contain a license file.  Ideally, all *.[ch] *.pl
*.pm, etc. files would have a copyright/license block at the top."	jhoblitt
Milestone 	818	test gcc4 auto vectorization optimization	misc	unspecified	defect	Paul Price	assigned	2006-09-09T10:07:00-10:00	2008-08-05T16:27:13-10:00	gcc 4.1 should be tested with -02/-03 & -ftree-vectorize	jhoblitt
Milestone 	834	psLib make install installs despite no changes	psconfig	0.11.0	defect	eugene	assigned	2006-10-13T17:56:52-10:00	2010-05-02T15:02:51-10:00	"if I run make && make install in psLib, then run it again, psLib will install a
bunch of header files again.  this means if I want to run a sequence of make
steps (eg, psLib, psModules, psphot, psastro, ppImage), they will all always
have to recompile everything because the psLib header files changed.  is it
possible for autoconf/etc to set the makefile targets not to install unless the
dependencies have actually changed?"	eugene
Milestone 	1009	pcontrol crashes when node crashes	PanTasks	unspecified	defect	eugene	assigned	2007-11-13T17:00:18-10:00	2008-08-05T16:38:19-10:00	"ipp003 went down (presumably the usual ethernet problem), which triggered a pcontrol crash (see below).  Node crashes have been correlated with pcontrol crashes in my mind for some time, but wanted to get this down for reference and tracking.
Note that when pcontrol goes down, it appears to come back up, but the state (in particular, the connections to the pclients) is not preserved, so that processing does not continue.


<found out that ipp003 had gone down; the below happened soon after>
pantasks: controller status
job stack PENDING:  14 objects
0  anyhost  11  warp_skycell.pl --warp_id 21 --skycell_id skycell.0199092 --tess_id MOPS --camera MEGACAM --workdir path://MOPS/run1/ 0.0.0.2ea
[...]
job stack BUSY:  6 objects
0  anyhost  11  warp_skycell.pl --warp_id 21 --skycell_id skycell.0199087 --tess_id MOPS --camera MEGACAM --workdir path://MOPS/run1/ 0.0.0.2e9
1  anyhost  11  warp_skycell.pl --warp_id 21 --skycell_id skycell.0199085 --tess_id MOPS --camera MEGACAM --workdir path://MOPS/run1/ 0.0.0.2e7
2  anyhost  11  warp_skycell.pl --warp_id 20 --skycell_id skycell.0198673 --tess_id MOPS --camera MEGACAM --workdir path://MOPS/run1/ 0.0.0.2c9
3  anyhost  11  warp_skycell.pl --warp_id 21 --skycell_id skycell.0199086 --tess_id MOPS --camera MEGACAM --workdir path://MOPS/run1/ 0.0.0.2e8
4  anyhost  11  warp_skycell.pl --warp_id 21 --skycell_id skycell.0199083 --tess_id MOPS --camera MEGACAM --workdir path://MOPS/run1/ 0.0.0.2e5
5  anyhost  11  warp_skycell.pl --warp_id 20 --skycell_id skycell.0198671 --tess_id MOPS --camera MEGACAM --workdir path://MOPS/run1/ 0.0.0.2c7
job stack DONE:  0 objects
job stack KILL:  0 objects
job stack EXIT:  0 objects
job stack CRASH:  0 objects
host stack OFF:  0 objects
host stack DOWN:  0 objects
host stack IDLE:  0 objects
host stack BUSY:  6 objects
0  bucket00  0.0.0.4
1  ipp002  0.0.0.0
2  ipp003  0.0.0.1
3  bucket00  0.0.0.5
4  bucket00  0.0.0.3
5  ipp003  0.0.0.2
host stack DONE:  0 objects
pantasks: got signal : 13
controller is down
starting controller connection
Connected
 
pantasks: status
 
 Scheduler is running
 Controller is running
 
[...]

 Jobs in Pantasks Queue
 23: warp.skycell.run   720      warp_skycell.pl (790990)
 23: warp.skycell.run   740      warp_skycell.pl (7ed900)
 23: warp.skycell.run    28      warp_skycell.pl (80bed0)
[...]
 jobs currently running remotely:
Njobs: 0
pantasks: controller status
job stack PENDING:  1 objects
0  anyhost  11  warp_skycell.pl --warp_id 22 --skycell_id skycell.0199092 --tess_id MOPS --camera MEGACAM --workdir path://MOPS/run1/ 0.0.0.0
job stack BUSY:  0 objects
job stack DONE:  0 objects
job stack KILL:  0 objects
job stack EXIT:  0 objects
job stack CRASH:  0 objects
host stack OFF:  0 objects
host stack DOWN:  0 objects
host stack IDLE:  0 objects
host stack BUSY:  0 objects
host stack DONE:  0 objects"	Paul Price
Milestone 	1041	2.4-cr-0: psbuild -bootstrap $DIR fails unless $DIR==$PSCONFDIR	psconfig	unspecified	defect	eugene	assigned	2008-02-18T23:57:53-10:00	2008-08-05T16:54:51-10:00	"psbuild -bootstrap works like a charm and is an instant success... however, only if the directory given matches the PSCONFDIR set in ~/.psconfigrc, if that file is present. If there's a mismatch, psconfig runs half-way through (until it reaches the point where PSCONFDIR is read) and then messes up the PATH etc. and can't find the .csh scripts any more in the place where psbuild -bootstrap copied them because it's looking in $PSCONFDIR instead.

Perhaps psbuild -bootstrap could check whether PSCONFDIR is set in ~/.psconfigrc and then offer the choice to either use that directory instead of what's given as argument to -bootstrap, or to replace what's in PSCONFDIR with the -bootstrap argument.

Also, the installation instructions don't know about -bootstrap yet."	jester@…
Milestone 	1071	ppStats on compressed images yield different results than uncompressed	ppStats	2.5	defect	Paul Price	assigned	2008-04-02T10:05:13-10:00	2008-04-02T12:40:35-10:00	"The following are some of the statistical measures for the residuals
for two different flat detrend runs with the same input exposures.

For run 20 the ppMerge output images were compressed, for run 20
they were not.

Note that the bg and bg_stdev values are somewhat different, and the
bin_stdev values are signficantly different.

Another experiement where the same ppMerge output was used and
only the residual calculations (which include the binning process)
were done on compresed images had similar results.

Perhaps this has something to do with how nan pixels are handled in
the compression process.

Note that the statistics are different. In the case of bin_stdev the values 
are very  different.
In the one case I looked at in detail, the cause of the high bin_stdev was a
huge outlier value from one chip.

mysql> select det_id, exp_id, bg, bg_stdev, bin_stdev from detResidImfile where det_id = 21 or det_id = 20 order by exp_id;
+--------+--------+------------------+------------------+------------------+
| det_id | exp_id | bg               | bg_stdev         | bin_stdev        |
+--------+--------+------------------+------------------+------------------+
|     20 |  50714 | 852.253815277778 | 20.2682251571526 | 253.213919128524 |
|     21 |  50714 | 852.315426388889 |  20.305931572922 | 2.27909648890663 |
|     21 |  50715 | 11407.4040277778 | 93.6562971802432 | 26.2769122627497 |
|     20 |  50715 | 11407.1608333333 | 93.7857139494351 |  3016.2631486739 |
|     21 |  50716 | 12253.7090277778 |   99.28685315589 | 29.1512105290881 |
|     20 |  50716 | 12253.4465277778 | 99.1299406999631 |  3045.1881543219 |
|     20 |  50717 |      12878.60625 | 103.696981163564 | 2525.55742331584 |
|     21 |  50717 | 12878.8972222222 | 103.714281568562 | 33.5483923541618 |
|     20 |  50718 |         11540.21 | 97.4627620960095 | 3940.88768852193 |
|     21 |  50718 | 11540.5791666667 | 97.3087913068767 | 24.9529588441886 |
|     20 |  50719 | 9865.94091666666 | 89.7526273109547 | 5647.43080833958 |
|     21 |  50719 | 9866.03281944444 | 89.3763152948967 | 46.0943350782944 |
|     20 |  50720 | 7346.96540277778 | 73.1578091868154 | 6054.02308449873 |
|     21 |  50720 | 7347.31634722222 | 72.5178396846033 | 31.5039281117779 |
+--------+--------+------------------+------------------+------------------+
14 rows in set (0.00 sec)"	bills
Milestone 	1086	Inconsistent allowed variable names and parsing: CTIO4m.I.MOSAIC2	DVO	2.5	defect	eugene	new	2008-04-27T08:26:16-10:00	2008-04-27T08:26:16-10:00	"DVO appears to be inconsistent with its allowed vector names and command-line parsing in the context of variable names with periods in them, which arise quite naturally in specifying photcodes:

dvo: mextract ra, dec, nmeas, CTIO4m.I.MOSAIC2 -region where nmeas >= 6
dvo: echo ra[$i]
352.493805
dvo: vectors
    N       name      size
    0         ra       2792
    1        dec       2792
    2      nmeas       2792
    3 CTIO4m.I.MOSAIC2       2792
dvo: echo $i
100
dvo: echo ra[$i]
352.493805
dvo: echo CTIO4m.I.MOSAIC2[$i]
invalid vector MOSAIC2
CTIO4m.I.MOSAIC2[100]
dvo: set a = CTIO4m.I.MOSAIC2
dvo: echo a[$i]
25.0087509

Am I misunderstanding something about how this is supposed to work?"	Michael Wood-Vasey
Milestone 	1090	psphot should use weight map in calculating background	psphot	2.5	defect	eugene	accepted	2008-05-01T05:02:00-10:00	2009-03-20T10:31:10-10:00	psphot currently appears to not use the weight map in calculating the background.  I was using it on some warped images (warped using SWarp as part of the standard ESSENCE reduction pipeline) where the edges were unfilled and thus represented in the variance map as 1e30.  But the background map appears to have fit including those areas.  Is this the correct behavior?  Did I misinterpret how the background is being fit?	Michael Wood-Vasey
Milestone 	1108	pantasks gives a Bus Error on my Mac OS 10.5.4 PowerPC	PanTasks	2.5	defect	eugene	reopened	2008-07-17T09:47:04-10:00	2008-07-17T11:06:18-10:00	"[wwoodvas@serenity simtest] gdb `which pantasks`
GNU gdb 6.3.50-20050815 (Apple version gdb-768) (Tue Oct  2 04:11:19 UTC 2007)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type ""show copying"" to see the conditions.
There is absolutely no warranty for GDB.  Type ""show warranty"" for details.This GDB was configured as ""powerpc-apple-darwin""...Reading symbols for shared libr
aries .............. done

(gdb) run
Starting program: /Volumes/data/PS1/code/ipp-2.6.beta/default.darwin/bin/pantasks Reading symbols for shared libraries +++++++++++++....warning: Could not find object file ""/sw/src/fink.build/gcc43-4.3.0-1001/darwin_objdir/powerpc-apple-darwin9/libgcc/_muldi3_s.o"" - no debug information available for ""../../../gcc-4.3.0/libgcc/..
/gcc/libgcc2.c"".

[...]warning: Could not find object file ""/sw/src/fink.build/gcc43-4.3.0-1001/darwin_objdir/powerpc-apple-darwin9/libgcc/emutls_s.o"" - no debug information available for ""
../../../gcc-4.3.0/libgcc/../gcc/emutls.c"".

. done

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x73000000
[Switching to process 19848 thread 0x117]
0x96371cc4 in strcmp ()
(gdb) bt
#0  0x96371cc4 in strcmp ()#1  0x00051248 in get_variable_ptr (name=0x77c88 ""PANTASKS_SERVER_STDERR"") at /Volu
mes/data/PS1/src/ipp-2.6.beta/Ohana/src/opihi/lib.shell/VariableOps.c:165#2  0x00051318 in get_variable (name=0x77c88 ""PANTASKS_SERVER_STDERR"") at /Volumes/
data/PS1/src/ipp-2.6.beta/Ohana/src/opihi/lib.shell/VariableOps.c:179#3  0x0004dac8 in VarConfig (keyword=0x77c88 ""PANTASKS_SERVER_STDERR"", mode=0x77c84 ""%s"", ptr=0xf0102dfc) at /Volumes/data/PS1/src/ipp-2.6.beta/Ohana/src/opihi/lib.sh
ell/ConfigInit.c:34#4  0x00002888 in CheckJobsThread (data=0x0) at /Volumes/data/PS1/src/ipp-2.6.beta/
Ohana/src/opihi/pantasks/job_threads.c:25
#5  0x963ab65c in _pthread_start ()
(gdb) up
#1  0x00051248 in get_variable_ptr (name=0x77c88 ""PANTASKS_SERVER_STDERR"") at /Volumes/data/PS1/src/ipp-2.6.beta/Ohana/src/opihi/lib.shell/VariableOps.c:165
165	    if (!strcmp(name, variables[i].name)) {
(gdb) p name
$1 = 0x77c88 ""PANTASKS_SERVER_STDERR""
(gdb) p variables
$2 = (Variable *) 0x500b30
(gdb) p i
$3 = 7
(gdb) p variables[i]
$4 = {
  name = 0x500970 ""KAPA"", 
  value = 0x500980 ""kapa""
}
(gdb) p variables[i].name
$5 = 0x500970 ""KAPA""
(gdb) p (int) strcmp(name, variables[i].name)
$6 = 5
(gdb) p ! (int) strcmp(name, variables[i].name)
$7 = 0

-----------------------------------

But yet it gave a bus error from within 'strcmp'?  I don't understand."	Michael Wood-Vasey
Milestone 	1120	psastro return status must be accurate	psastro	2.6	defect	eugene	new	2008-08-01T09:59:24-10:00	2008-08-01T09:59:24-10:00	"The return status from psastro must reflect the result of the astrometry.

When to return a failed exit status can be a complicated question: we don't want to die just because a single chip failed because we can use a model to tie down a few bad chips using the good astrometry from the rest.  However, I think we must insist that psastro fails when the final astrometry is bad.  To do otherwise is to deceive the pipeline, and to push problems downstream that properly belong in psastro.

psastro should fail under the following circumstances:
* The camera consists of a single chip, which fails to find a solution (I think I've already implemented this case).
* The camera consists of multiple chips which all fail to find a solution.
* The camera consists of multiple chips, a small subset fail to find a solution but there's no model available to apply.
* The camera consists of multiple chips, a large subset fail to find a solution.
* The camera consists of multiple chips, some subset of chips have a solution, but the solutions are discordant.

There should be a note made in the database if the astrometric solution for an exposure required the use of a model to get a solution for some chips."	Paul Price
Milestone 	1127	pschecklibs -build selects tagsets/ipp-2.6.libs even if a 2.6.1 tagset is present	psconfig	2.6	defect	eugene	new	2008-08-13T23:51:26-10:00	2008-08-13T23:51:26-10:00	"psbuild selects the ""latest"" available tagset thus:

if ( @ARGV == 0) {
    @list = <$tagsets/*.libs>;
    $file = $list[-1];
} else {
    $file = ""$tagsets/$ARGV[0].libs"";
}

But if 2.6.1 and 2.6 are both present, 2.6 appears later in the expansion of *.libs than 2.6.1. because ""1"" appears before ""l"" in a lexicographic ordering.

This seems to be no problem in 2.6 but might cause confusion later.

The only quick fix I can think of is calling the 2.6 release 2.6.0 from the beginning."	jester@…
Milestone 	1129	output headers for pswarp (cmf and image) are poorly constructed	pswarp	unspecified	defect	Paul Price	assigned	2008-08-16T12:53:02-10:00	2008-08-18T16:03:47-10:00	"Created an attachment (id=120)
example output pswarp fits image header

There are a variety of problems with the output cmf and image headers from pswarp, at least for the simulation suite.  Attached are an example image header and example cmf header from a simtest warp run.  

The output headers for the cmf files from pswarp are missing various metadata present in the output headers of the image file.  A few problems with the output cmf header:

* missing NAXIS1,NAXIS2 (these are not present even before the call to psFitsWriteBlank).

* missing astrometry WCS keywords (present in image output header)

* output TRIMSEC is nan:nan,nan:nan

In addition, both cmf and image headers suffer from the following problems:

* basic header keywords are kept in the HIERARCH format, and are thus not compatible with the source camera format.

* output BIASSEC is not set"	eugene
Milestone 	1134	smf files are PS1_DEV_0 not PS1_DEV_1 like cmf and as expected	psphot	2.6	defect	eugene	assigned	2008-09-09T11:10:11-10:00	2008-12-04T17:16:33-10:00	As discussed by phone: My sdssmosaic/psphot.config specified PS1_DEV_1 as output format, but the .smf files created from the SDSS pixels have PS1_DEV_0 as output format. .cmf files are PS1_DEV_1 as expected.	jester@…
Milestone 	1136	psconfig needs to make sure all dependent libraries are found with exactly the required versions, even if multiple versions are floating around on the system	psconfig	2.6	defect	eugene	new	2008-09-12T02:53:45-10:00	2009-05-04T23:28:01-10:00	"Gautham Narayan reported this bug on ps-ipp users before at http://pan-starrs.ifa.hawaii.edu/mailman/private/ps-ipp-users/msg00495.html but the fix he got from Paul and forwarded me isn't working for me.

The real underlying bug is that psconfig doesn't seem to have a mechanism for making sure that only and exactly the required version of a required library is used for compilation and at runtime.

The problem is this: I am trying to compile with psbuild -dev -extbuild on a tree that I 'cvs export'ed from cvs head.

This has built cfitsio from the tarball I unpacked into psconfig/../../extlibs

It generates the following configure command for psLib:

./configure --prefix=/IPP/bleed.lin64 --bindir=/IPP/bleed.lin64/bin --libdir=/IPP/bleed.lin64/lib --mandir=/IPP/bleed.lin64/man --includedir=/IPP/bleed.lin64/include --sysconfdir=/IPP/bleed.lin64/etc --datadir=/IPP/bleed.lin64/share

However, configure for psLib doesn't find cfitsio, even though libcfitsio.a is exactly where psconfig put it, and contains the symbol that configure complains it can't find:

checking fitsio.h usability... no
checking fitsio.h presence... no
checking for fitsio.h... no
configure: error: CFITSIO headers not found.  Obtain CFITSIO at  http://heasarc.gsfc.nasa.gov/docs/software/fitsio or use --with-cfitsio to specify location.

# Suggested fix was to set LDFLAGS and point to the library location in there, 
# which I had done:

jester@aida77 psLib$ echo $LDFLAGS 
-L/IPP/bleed.lin64/lib/

# The library is there:
jester@aida77 psLib$ ls /IPP/bleed.lin64/lib/*fits*
/IPP/bleed.lin64/lib/libcfitsio.a

# it contains ffopen:
jester@aida77 psLib$ nm `!!` |grep ffopen
nm `ls /IPP/bleed.lin64/lib/*fits*` |grep ffopen
0000000000008780 T ffopen
000000000000a4d0 T ffopentest
                 U ffopen
                 U ffopen
0000000000002e60 T Cffopen
                 U ffopen

# Moreover, the include directory from the configure command does contain
# fitsio.h but configure hadn't found that, eiher:
jester@aida77 psLib$ ls /IPP/bleed.lin64/include/*fits*
/IPP/bleed.lin64/include/fitsio2.h  /IPP/bleed.lin64/include/fitsio.h  /IPP/bleed.lin64/include/gfitsio.h

Looking at the config.log from psLib, it turns out that the problem wasn't anything to do with ffopen itself, but there were many other undefined symbols pointing to an earlier version of cfitsio compiled for an earlier version of IPP in a different subdirectory of psconfig/../../ -- somehow that confused the linker and/or psconfig. Having deleted that other cfitsio directory, psLib now compiles.

Long story short: far too many people have been having far too many problems like this one with library dependencies, so I don't think the current way psconfig does it is good enough. 

Everyone at IfA and elsewhere could be more productive if these library version dependencies were done by a system that is set up to be guaranteed to get it right.

In addition, over the next ~4 years of Linux evolution during ORR preparation and the scheduled PS1SC mission, this is bound to cause some problem at some point also at IfA, not just in the zoo of Linuxes out there in the collaboration.

So I strongly believe that we need to change the config system to something that is guaranteed to compile IPP correctly whatever else is going on on the target system. In my opinion, this can wait until after ORR, but not much longer."	jester@…
Milestone 	1144	psbuild seems to ignore errors in pscheckperl -build process	psconfig	2.6	defect	eugene	new	2008-10-04T07:40:38-10:00	2008-10-04T07:40:38-10:00	"I was trying to install a fresh cvs export as of 2008-10-04 with
psbuild -dev -extbuild

Installation of the CFITSIO perl module failed (again, because it didn't find the cfitsio pschecklibs had previously defined and I hadn't set the CFITSIO env var by hand), and I didn't have the tarball for Filesys-Df-0.92 on disk. Nevertheless, psbuild continued with building IPP, even though some dependencies hadn't been built correctly. I think it should stop compilation, or somehow issue a reminder at the end that required perl packages are missing."	jester@…
Milestone 	1149	addstar doesn't load dM, dRA, dDec from GPC1 pt 2 .cmf files	DVO	2.6	defect	eugene	new	2008-10-21T10:34:01-10:00	2009-01-12T02:36:35-10:00	"I ran addstar on the 2008/10/06 GPC1 pt 2 release .cmf files and successfully put all of the 8 FPAs and all their chips into a DVO database.  But the database did not have dM, dRA, or dDec loaded in the measures table so relphot and relastro both fail.  I'm trying to trace through how dM gets loaded.  Things still seem to be fine as of the LoadST

---
[wwoodvas@serenity gpc1] ls
gpc1
o4732g0168o.34857/
o4732g0169o.34858/
o4732g0170o.34859/
o4732g0171o.34860/
o4732g0172o.34861/
o4732g0179o.34868/
o4732g0190o.34879/
o4732g0212o.34901/
[wwoodvas@serenity gpc1] # Clear the directory and start again
[wwoodvas@serenity gpc1] rm -rf gpc1
[wwoodvas@serenity gpc1] foreach cmf ( */*.cmf )
foreach? addstar -accept-astrom -D CATDIR gpc1 -D CAMERA gpc1 $cmf
foreach? end
no data in db gpc1/Photcodes.dat
directory gpc1 does not exist, creating...
SUCCESS: elapsed time    0.7603 sec for   320 stars (    0 matches),    320 average,     320 measure
SUCCESS: elapsed time    0.3961 sec for   448 stars (    0 matches),    768 average,     768 measure
SUCCESS: elapsed time    0.3962 sec for   379 stars (    0 matches),   1147 average,    1147 measure
SUCCESS: elapsed time    0.3992 sec for   359 stars (    0 matches),   1506 average,    1506 measure
[...]
SUCCESS: elapsed time    0.6230 sec for   160 stars (   22 matches),  16016 average,   16452 measure
SUCCESS: elapsed time    0.6211 sec for    69 stars (    2 matches),  16083 average,   16521 measure
----

Then we can look at things in DVO:

----
[wwoodvas@serenity gpc1] dvo -D CATDIR gpc1
no file /Users/wwoodvas/.dvorc

Welcome to DVO - desktop virtual observatory

dvo: region 360.5 0.25 1.5
dvo: images
current style (0): -x 2 -c black -pt 2 -lt 0 -lw 0.500000 -sz 1.000000
reading images (mode DVO_MODE_MEF)
plotting 477 images
dvo: cgrid
current style (0): -x 2 -c black -pt 2 -lt 0 -lw 0.500000 -sz 1.000000
dvo: mextract -region ra, dec, i, i:err
dvo: czplot ra dec i 17 12 -c red -pt 7
----

And here's where we see the first problem.  'i:err' is NaN for every value:

----
dvo: limits i i:err
dvo: limits
limits: 6.623180 29.526000 nan nan [-a] [-n device]
----

Here we've asked the ""limits"" function to automatically figure out and set the limits for ""i"" and ""i:err"" (for example because we were going to plot them).  It finds that the min and max values for ""i:err"" are NaN, i.e., all of the values of ""i:err"" are NaN.  This holds true for all of the filters (griy).  A bit of further investigation reveals that all of the ra:err and dec:err values are 0, but I don't think the inputs to these values from the .cmf value are reasonable in the first place since they tend to be in the 1000s or 0 (are the units on X_PSF_SIG thousandths of a pixel or something?).


I don't know what's going on here?  ""PSF_INST_MAG"" and ""PSF_INST_MAG_SIG"" are both defined and reasonable in the .cmf files.  I can only assume that they're not getting set right by 'addstar' in the load into the DVO database.

Even more details follow as the gdb debug of the relastro command which ideally shouldn't seg fault even if things aren't defined in the DVO database:

[wwoodvas@serenity GPC1] gdb `which relastro`
GNU gdb 6.3.50-20050815 (Apple version gdb-768) (Tue Oct  2 04:11:19 UTC 2007)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type ""show copying"" to see the conditions.
There is absolutely no warranty for GDB.  Type ""show warranty"" for details.
This GDB was configured as ""powerpc-apple-darwin""...Warning: /Volumes/data/PS1/data/
GPC1/src: No such file or directory.
 Reading symbols for shared libraries .............. done

(gdb) run  -D CATDIR gpc1 -D CAMERA gpc1 -region 359.5 361.5 -1.25 1.75 -update-simp
le -v
Starting program: /Volumes/data/PS1/code/ipp/default.darwin/bin/relastro -D CATDIR g
pc1 -D CAMERA gpc1 -region 359.5 361.5 -1.25 1.75 -update-simple -v
Reading symbols for shared libraries +++++++++++++.....warning: Could not find objec
t file ""/sw/src/fink.build/gcc43-4.3.1-1000/darwin_objdir/powerpc-apple-darwin9.4.0/
libgcc/_muldi3_s.o"" - no debug information available for ""../../../gcc-4.3.1/libgcc/
../gcc/libgcc2.c"".
[...]
image 472 has 22 measures (2008/09/23,00:00:00, GPC1.y.XY72)
image 473 has 31 measures (2008/09/23,00:00:00, GPC1.y.XY73)image 474 has 6 measures (2008/09/23,00:00:00, GPC1.y.XY74)
image 475 has 22 measures (2008/09/23,00:00:00, GPC1.y.XY75)image 476 has 2 measures (2008/09/23,00:00:00, GPC1.y.XY76)

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0xc075bdb7
0x0001141c in CoordsGetCenter (fit=0x1d05ca0, tol=0.001, xo=0xbfffa330, yo=0xbfffa338) at /Volumes/data/PS1/src/ipp/Ohana/src/relastro/src/mkpolyterm.c:31
31    R  = (xfit[1][0]*yfit[0][1] - xfit[0][1]*yfit[1][0]);
(gdb) p xfit
$1 = (double **) 0x1d7b250
(gdb) p yfit
$2 = (double **) 0x1d6f6d0
(gdb) p yfia[0]t
No symbol ""yfia"" in current context.
(gdb) p yfit[0]
$3 = (double *) 0x1d6f6e0
(gdb) p xfit[0]
$4 = (double *) 0x1d7b260
(gdb) p xfit[1][0]
Cannot access memory at address 0xc075bdb7
(gdb) p yfit[0][1]
$5 = 6.2747415538852984e-311
(gdb) p fit[0].xfit
$6 = (double **) 0x1d7b250
(gdb) p fit[0].xfia[0]t
A syntax error in expression, near `t'.
(gdb) p fit[0].xfit[0]
$7 = (double *) 0x1d7b260
(gdb) p fit[0].xfit[0][1]
$8 = 1.674254679072588e-311
(gdb) p fit[0].xfit[1][0]
Cannot access memory at address 0xc075bdb7
(gdb) p fit[0].xfit[1]
$9 = (double *) 0xc075bdb7
(gdb) p *fit[0].xfit[1]
Cannot access memory at address 0xc075bdb7
(gdb) p *fit[0].xfit[0]
$10 = 1.2704902607295261
(gdb) bt
#0  0x0001141c in CoordsGetCenter (fit=0x1d05ca0, tol=0.001, xo=0xbfffa330, yo=0xbfffa338) at /Volumes/data/PS1/src/ipp/Ohana/src/relastro/src/mkpolyterm.c:31
#1  0x0000f574 in fit_apply_coords (fit=0x1d05ca0, coords=0x7ab000) at /Volumes/data/PS1/src/ipp/Ohana/src/relastro/src/fitpoly.c:283
#2  0x00004808 in FitSimple (raw=0x825a00, ref=0x82e800, Nmatch=15, coords=0x7ab000) at /Volumes/data/PS1/src/ipp/Ohana/src/relastro/src/FitSimple.c:28
#3  0x0000b840 in UpdateSimple (catalog=0x800000, Ncatalog=4) at /Volumes/data/PS1/src/ipp/Ohana/src/relastro/src/UpdateSimple.c:24
#4  0x00014148 in main (argc=14, argv=0xbfffa658) at /Volumes/data/PS1/src/ipp/Ohana/src/relastro/src/relastro.c:47"	Michael Wood-Vasey
Milestone 	1152	PSF_INST_MAG_SIG all NaN in pswarp .cmf files	pswarp	2.6	defect	Paul Price	assigned	2008-11-04T14:59:44-10:00	2008-11-04T17:10:21-10:00	"pswarp doesn't seem to translate PSF_INST_MAG_SIG from the input CMF file to the output CMF file.

Joel Welling and I have been working on warping MOPS Megacam MEF chip images to a given skycell and then doing subtractions.  In this process we've realized that the PSF_INST_MAG_SIG is not being translated properly between the proc/MEF files and the warped CMF files.  PSF_INST_MAG and all of the other columns in the binary FITS table of the original CMF are being translated correctly.  This was true in each of our individual warps as well as Paul's original warps done at the IfA.

[wwoodvas@serenity work] fdump 'proc/906622.cmf[44]' STDOUT PSF_INST_MAG,PSF_INST_MAG_SIG 100-110 prhead=no
 
 
        PSF_INST_MAG    PSF_INST_MAG_SIG
 
    100  -1.1651719E+01   4.8224498E-03
    101  -1.1625567E+01   4.9144300E-03
    102  -1.1624426E+01   4.9266489E-03
    103  -1.1586342E+01   5.0057173E-03
    104  -1.1604173E+01   4.9859094E-03
    105  -1.1597309E+01   5.0032125E-03
    106  -1.1582771E+01   5.0470363E-03
    107  -1.1575814E+01   5.0668400E-03
    108  -1.1558278E+01   5.0968477E-03
    109  -1.1556294E+01   5.1301736E-03
    110  -1.1541912E+01   5.1773461E-03
[wwoodvas@serenity work] fdump skycell.0198755.new/906622.cmf STDOUT PSF_INST_MAG,PSF_INST_MAG_SIG 100-110 prhead=no
 
 
        PSF_INST_MAG    PSF_INST_MAG_SIG
 
    100  -7.3885350E+00 INDEF
    101  -7.3988523E+00 INDEF
    102  -7.6355329E+00 INDEF
    103  -7.3786430E+00 INDEF
    104  -7.5089717E+00 INDEF
    105  -7.5279927E+00 INDEF
    106  -7.4888477E+00 INDEF
    107  -7.3086233E+00 INDEF
    108  -7.5065479E+00 INDEF
    109  -7.4235153E+00 INDEF
    110  -7.4822884E+00 INDEF
---

This is true for all of the detections.  Obviously these aren't the same detections, but the objects that do match do indeed have all of the same columns except for PSF_INST_MAG_SIG."	Michael Wood-Vasey
Milestone 	1161	flat-field creation does not set valid FILTER in header	ppImage	unspecified	defect	eugene	assigned	2008-11-13T13:54:30-10:00	2010-05-02T14:54:01-10:00	the flats created for GPC1 in september do not have FILTERID correctly set.  header values are generally messed up.	eugene
Milestone 	1167	Kapa + region causes long delay over remote X connections.	DVO	unspecified	defect	eugene	assigned	2008-12-05T13:11:19-10:00	2008-12-07T11:09:27-10:00	"Why does 'region' over a remote X connection take minutes with Kapa enabled and seconds with '$KAPA = -noX'?  The larger the DVO database, the longer it takes.  It's not drawing anything to the screen, why does it take so long?

I'll take a few pointers of files to look at if I can be helpful in tracing this down.  Reading through Ohana/src/opihi/cmd.astro/region.c didn't seemed fine, but I didn't trace down to the next level of all of the commands called from that.

This is becoming more important as larger and larger DVO database are assembled and so kept on remote machines instead of just shipped to the machine I'm working on."	Michael Wood-Vasey
Milestone 	1352	psphotForced not working in trunk or eam branch	psphot	current	defect	eugene	new	2010-02-08T10:25:09-10:00	2010-02-23T18:38:51-10:00	"When I run psphotForced in latest eam-20091201 on a subtraction I ran with eam-20091201, I get the following error:

----
[serenity skycell.042] cat test.xy
4116 3503.33
[serenity skycell.042] base=MD04.skycell.042.stk.52179
[serenity skycell.042] psphotForced -file $base.fits -mask $base.mask.fits -variance $base.wt.fits -psf $base.target.psf testforced_eam -srctext test.xy
[serenity skycell.042] psphotForced -file $base.fits -mask $base.mask.fits -variance $base.wt.fits -psf $base.target.psf testforced_eam -srctext test.xy
2010-02-08 20:19:35Z|serenity.phyast.pitt.edu|I|psphot
    psphot at 2010-02-08T20:20:09.370351
2010-02-08 20:19:35Z|serenity.phyast.pitt.edu|I|psphot
    psLib branches/eam_branches/20091201/psLib@26809M from 60eb6cdc-a59c-4636-a4e0-dba66a9721fd, built Feb  7 2010, 15:59:15 with cfitsio-3.100 gsl-1.11 fftw-3.0.1 mysql-5.0.51a unoptimised with trace
2010-02-08 20:19:35Z|serenity.phyast.pitt.edu|I|psphot
    psModules branches/eam_branches/20091201/psModules@26809 from 60eb6cdc-a59c-4636-a4e0-dba66a9721fd, built Feb  7 2010, 14:53:49 unoptimised with nebclient
2010-02-08 20:19:35Z|serenity.phyast.pitt.edu|I|psphot
    psphot branches/eam_branches/20091201/psphot@26809 from 60eb6cdc-a59c-4636-a4e0-dba66a9721fd, built Feb  7 2010, 15:59:51 unoptimised with libkapa
2010-02-08 20:19:35Z|serenity.phyast.pitt.edu|I|psModules.config.format
    Camera _GPC1-SKYCELL, format SKYCELL matches header.
2010-02-08 20:19:35Z|serenity.phyast.pitt.edu|W|formatFromHeader (pmConfig.c:1126)
    Camera SIMPLE, format SIMPLE also matches header --- ignored.
2010-02-08 20:19:35Z|serenity.phyast.pitt.edu|I|psModules.config.format
    Camera _GPC1-SKYCELL, format SKYCELL matches header.
2010-02-08 20:19:35Z|serenity.phyast.pitt.edu|I|psModules.config.format
    Camera _GPC1-SKYCELL, format SKYCELL matches header.
2010-02-08 20:19:35Z|serenity.phyast.pitt.edu|I|psModules.config.format
    Camera _GPC1-SKYCELL, format SKYCELL matches header.
2010-02-08 20:19:35Z|serenity.phyast.pitt.edu|4|psphot
    Chip 0: 1 1
2010-02-08 20:19:37Z|serenity.phyast.pitt.edu|W|addSource (pmFPAConstruct.c:1123)
    FPA.OBS for new source (Stack) doesn't match FPA.OBS for current fpa (UNKNOWN).
2010-02-08 20:19:38Z|serenity.phyast.pitt.edu|5|psphot
    Cell 0: 1 1
2010-02-08 20:19:38Z|serenity.phyast.pitt.edu|6|psphot
    Cell 0: 1 1
2010-02-08 20:19:39Z|serenity.phyast.pitt.edu|I|psphot
    PHOTCODE is GPC1.g.SkyChip
2010-02-08 20:19:41Z|serenity.phyast.pitt.edu|I|psphot
    built background image: 1.709924 sec
2010-02-08 20:19:41Z|serenity.phyast.pitt.edu|I|psphot
    image sky : mean -0.864739 stdev 6.318784
2010-02-08 20:19:41Z|serenity.phyast.pitt.edu|I|psphot
    background sky : min -7.037591 mean -0.864739 max 6.599946 stdev 1.625985
2010-02-08 20:19:42Z|serenity.phyast.pitt.edu|5|psphot
    build resampled image: 0.345939 sec
2010-02-08 20:19:42Z|serenity.phyast.pitt.edu|I|psphot
    subtracted background model: 0.680112 sec
2010-02-08 20:19:42Z|serenity.phyast.pitt.edu|I|psphot
    using externally supplied PSF model for this readout
Assertion failed in function psphotLoadExtSources at psphotMergeSources.c:87. Error stack:
2010-02-08 20:19:42Z|serenity.phyast.pitt.edu|A|psLib.sys
    missing detections?
Abort trap

----"	Michael Wood-Vasey
Milestone 	1372	user requests import of UKIDSS catalogs to DVO	DVO		defect	bills	new	2010-05-05T14:05:48-10:00	2010-05-05T14:05:48-10:00	"Bertrand Goldman of MPIA has been requesting this
"	bills
Milestone 	1375	nebulous replication left an empty file behind	Nebulous		defect	watersc1	accepted	2010-05-07T14:59:11-10:00	2010-05-07T15:05:29-10:00	"When updating a chip imfile today I ran into a case where it failed because the burntool table had zero size.

This is strange since the file existed at one time since the chip processing had completed successfully once. 

Investigating further we found that there were two instances. One of which had zero size.

This seems to indicate that the replication process failed and left a file behind.

For reference the file was

neb://ipp042.0/gpc1/20100118/o5214g0235o/o5214g0235o.ota22.burn.tbl

I have fixed the zero sized file by copying the good one on top of it."	bills
Milestone 	1383	science cleanup needs to relocate log files	ippTasks		defect	watersc1	assigned	2010-05-17T11:14:05-10:00	2010-05-17T11:24:40-10:00	"Currently chip.cleanup.run sets stderr and stdout to  $LOGDIR/chip.cleanup.log 
Thus output from all jobs goes to the same file and the fate of the pantasks_logs for a given job is totally undefined.

The output should go to a log file specific to the chipRun.

This makes it difficult to debug problems when they occur.
Now where should it go since there is no run level path_base ...

Cleanup for the other stages has the same problem.
"	bills
Milestone 	1388	pstamptool -getdependent should return whole object not just id	pstamp		defect	bills	new	2010-05-27T10:57:41-10:00	2010-05-27T10:57:41-10:00	"Currently pstamptool -getdependent ouputs only the dep_id of the
newly created or already existing pstampDependent.

It should print out the entire object as an mdc doc. This would allow the parser to check for errors and fault the job if
necessary.

For example if pstampDependent.fault == PSTAMP_GONE that error should be passed along to the client. As it is now we have to
queue the job and have pstamp_checkdepedent fault it."	bills
Milestone 	1483	PSF_INST_FLUX_SIG is negative for negative values of PSF_INST_FLUX	psphot	Development Stage	defect	magnier	new	2011-04-27T08:40:49-10:00	2011-04-27T08:40:49-10:00	"PSF_INST_FLUX_SIG is negative for negative values of PSF_INST_FLUX.  This is incorrect behavior.

This bug is in ipp-branches/eam-20110404, rev31385.

It is _not_ present in rev31166."	Michael Wood-Vasey
Milestone 	1500	czarpoll crashes when it's restarted	ippMonitor		defect	rhenders	new	2011-11-01T21:04:46-10:00	2011-11-01T21:11:17-10:00	"Illegal division by zero at czartool/CzarDb.pm line 684.
"	Serge CHASTEL
Milestone 	917	Add mechanism for IPP to know that a given file has already been detrended (from header or filename)	ippScripts	unspecified	enhancement	Paul Price	reopened	2007-08-14T11:52:12-10:00	2008-08-05T16:32:53-10:00	"Sometimes (for processing of SDSS images, all the time), we want to inject images into IPP that are already flatfielded etc. I can make the PROCESSED reduction class the default in the camera.config, and/or I can update chipRun.reduction by hand after registering images in ippMonitor. But it would be useful for IPP to have a mechanism of recognizing automatically that a given image has already been detrended. This could be by specifying certain header fields, or in the SDSS case, the filename fpC*.fits tells you that the image is a reduced (""corrected"", hence C) image produced by the frames pipeline (hence fp)."	jester@…
Milestone 	1019	variable interval for .load functions	ippTasks	2.2	enhancement	eugene	assigned	2007-12-12T13:02:13-10:00	2009-04-06T11:16:45-10:00	if the number of in-queue entries is large, we should run this step less frequently.	eugene
Milestone 	1025	M31 Bulge sources ignored with a hack	psastro	unspecified	enhancement	eugene	assigned	2008-01-07T04:48:36-10:00	2008-08-05T17:09:36-10:00	"Astrometry analysis in the field of M31 was crashing for GPC1 because of the large number of sources in the bulge, potentially in conjunction with the large number of dark-current sources.  the significance of matches was washed out by the background and the noise sources matched the bulge to easily.  I hacked a solution to ignore the bulge region when loading the ref sources, but a better solution is needed.  some options:

* detect and ignore clumps in the input lists (raw and ref).  we could use autocorrelation to find clumps and drop them from both.

* have a list of bad regions specified in the recipe to ignore.

* enforce a uniform distribution of sources in the input lists somehow?"	eugene
Milestone 	1033	add uninstall option to Ohana build	psconfig	unspecified	enhancement	eugene	assigned	2008-01-10T14:08:12-10:00	2008-08-05T16:54:25-10:00	it would be nice to implement 'make uninstall' for the Ohana Makefiles	eugene
Milestone 	1047	Should be able to specify chip rotations in format.config	ippconfig	unspecified	enhancement	Paul Price	assigned	2008-02-27T09:28:30-10:00	2008-08-05T17:18:53-10:00	"Creating a config file for the LBC camera on the LBT, I ran into the problem that LBC has 4 chips of which 1 is rotated 90 degrees wrt the rest, but I can't specify that in format.config, which I should be able to.

Since Paul told me that the chip positions only matter for creation of a jpeg thumbnail, it's not strictly required to be able to specify rotations, but it sure would be nice. Hence this enhancement request."	jester@…
Milestone 	1087	Request to have default 'site.config' file created by default install.	ippconfig	2.5	enhancement	Paul Price	new	2008-04-30T02:59:46-10:00	2008-04-30T02:59:46-10:00	"While I understand that users should spend some time actually looking at the configuration files and setting up their own site.config file, it would be helpful if the default install produced a 'site.config' file (as is referenced by the default 'ipprc.config') so that at least running IPP commands doesn't immediately fail with messages like:

[wwoodvas@master src]$ psphot -file ~/workspace/sm050926/2/waa1.050926_0010.069_2.sw.fits test.phot
Error reading arguments

 -> pmConfigFileRead (pmConfig.c:180): parameter is out-of-range
     Error: String name is NULL or empty.
 -> psphotArguments (psphotArguments.c:24): Problem in configure files
     Can't read site configuration

While this seems obvious enough once you understand it, it took me about 30 minutes to figure this out.  

This processes of customizing the site.config file and other files should also be explicitly described with sufficient examples to get something basically working in the INSTALL file in the source tree psconfig directory."	Michael Wood-Vasey
Milestone 	1100	addstar fails before parsing options unless .ptolemyrc points to a valid dvo.config, even if a valid dvo.config is pointed to on the command line	Ohana	2.5	enhancement	eugene	new	2008-05-25T00:33:34-10:00	2008-09-16T03:28:23-10:00	"I had a typo in my ~/.ptolemyrc (misspelled a config directory). As far as I could tell with the help of a debugger, addstar tries to read the configuration pointed to by ~/.ptolemyrc before parsing its options. The misspelled directory in .ptolemyrc made addstar fail (because it requires an ADDSTAR_RADIUS or RADIUS) before it even parses the command-line options. I.e. without a valid, working .ptolemyrc, you can't even find out what the command-line arguments to addstar are, nor can you specify a different, valid configuration on the commandline.

The suggested fix would be to read the default configuration *after* option parsing."	jester@…
Milestone 	1112	background model should not use a small fraction of superpixel at edges	psphot	unspecified	enhancement	eugene	new	2008-07-20T15:19:40-10:00	2008-07-20T15:19:40-10:00	the background model currently attempts to measure the background from the superpixels, placing the remaining pixels at the edge in a single superpixel.  These usually fails since there are too few pixels.  It would be better for the edges to use more pixels than a normal superpixel rather than fewer.	eugene
Milestone 	1113	do not build nebclient stuff in external releases	psconfig	unspecified	enhancement	eugene	new	2008-07-23T10:29:15-10:00	2008-07-23T10:29:15-10:00	these are not being widely used yet, so we can skip them for now.	eugene
Milestone 	1114	issues with extended source fits	psphot	unspecified	enhancement	eugene	new	2008-07-24T13:52:11-10:00	2008-07-24T14:38:52-10:00	"reported extended source fits don't seem consistent with results.  eg, poshih test image:

    fit models: 0.016192 sec for 6 objects (6 psf, 0 ext, 0 failed, 1 skipped)

but one source was fitted...

too many sources were fitted by extended source analysis: only one extended source above S/N ~ 1, but 10-25 were tested...

(most seem to be faint outliers of the single galaxy)."	eugene
Milestone 	1117	Suggestion: shape drawing onto the Kapa window	DVO	unspecified	enhancement	eugene	new	2008-07-25T15:48:36-10:00	2008-07-25T15:48:36-10:00	It would be useful to be able to draw simple shapes like rectangles (outlined or filled) onto the Kapa window using the common style options.	pitts
Milestone 	1128	detrend creation validates detrend even if all are input are excluded	ippScripts	unspecified	enhancement	Paul Price	assigned	2008-08-15T12:05:56-10:00	2008-08-18T10:26:22-10:00	Not sure where this is happening, but probably in detrend_reject_exp.pl.  I have an example of a DARK run in which a bug in ppMerge result in nonsense fits, and the residuals all rejected.  But, the detrend master was accepted anyway.	eugene
Milestone 	1133	ADDSTAR does not run properly on older files	Ohana	2.6	enhancement	eugene	new	2008-09-08T10:03:44-10:00	2008-09-08T10:03:44-10:00	ADDSTAR crashes when running on the files described in http://kiawe.ifa.hawaii.edu/Elixir/Ohana/DVO/example.html. These are older files whose format may now be incompatible with ADDSTAR.	beaumont
Milestone 	1137	pantasks is unable to handle multiple concurrent IPP versions	PanTasks	2.6	enhancement	eugene	new	2008-09-12T04:17:14-10:00	2008-09-12T04:17:14-10:00	"pantasks relies on the user's .bashrc or .cshrc (or equivalent) to call psconfig. That means that a user's pantasks can only ever use a single IPP version at any given time. Thus, even though one can launch pantasks from different IPP versions in parallel, one can't use it to run *commands* from different IPP versions in parallel.

Couldn't the command issued by ssh do the setup to the same IPP version as the pantasks it was launched by?"	jester@…
Milestone 	1140	Request for ppSim to write out noiseless versions of detrend and science image	ppSim	2.6	enhancement	Paul Price	new	2008-09-16T22:21:26-10:00	2010-05-02T15:27:36-10:00	"For improved testing of all stages of simtest, it'd be helpful if ppSim wrote out the noiseless science image, the actual flatfield that was multiplied into the science image, all the noiseless detrend frames (if noiseless ones exist), etc.

Since I couldn't find a ppSim category in bugzilla, I'm calling this ""software"" for now and hope I got the right assignee right!"	jester@…
Milestone 	1146	pantasks should store command lines of commands it issues in a log file	PanTasks	2.6	enhancement	eugene	new	2008-10-08T23:15:10-10:00	2008-10-08T23:15:10-10:00	"Maybe I missed it, but none of the log files produced by pantasks reductions seem to contain the full command issued by pantasks. E.g. I am looking at a log file for the chip stage after pantasks tells me things like

system failure for: chip_imfile.pl --threads @MAX_THREADS@ --exp_id 32 --chip_id 32 --class_id CHIP2 --uri path://S82IN/i/1755-0341/fpC-001755-i2-0341.fits --camera SDSSmosaic --run-state new --outroot path://S82AP/run_ipp//1755-0341.32/1755-0341.32.ch.32 --redirect-output --reduction SDSSphotast --dbname jester_s82_ap --verbose

but that chip_imfile.pl command line is not actually stored in the log file, nor in the pantasks_logs/chip.imfile.log, where it just says that chip_imfile.pl was started. If I catch the command as it's failing, all's well, but for something that ran over night, I have no way of retrieving the command line quickly and reproducing the error becomes difficult.

It would be most useful if the log file into which the chip_imfile.pl output goes started with that command line (above) that otherwise only appears to appear on stderr on failures."	jester@…
Milestone 	1158	"Adding error bars to the ""dot"" command"	DVO	2.6	enhancement	eugene	new	2008-11-11T12:01:49-10:00	2008-11-11T12:01:49-10:00	"It would be useful to add the plot options of ""-dx"" and ""-dy"" to the ""dot"" command. That way individual symbols could be plotted with error bars without needing to create 1-point vectors."	pitts
Milestone 	1168	Provide `basename` for PPSUB.REFERENCE and PPSUB.INPUT	ppSub	unspecified	enhancement	Paul Price	new	2008-12-17T11:45:59-10:00	2008-12-17T11:45:59-10:00	Provide `basename` for PPSUB.REFERENCE and PPSUB.INPUT since they are truncated to 80 characters total line width and so the most important details are lost.  The basename should be descriptive enough and will be far more useful than the directory information.  Alternatively, as long was we're using HIERARCH keywords, do all HIERARCH-aware libraries understand line continuations as well?	Michael Wood-Vasey
Milestone 	1172	inconsistent output headers in smf files	ippconfig	unspecified	enhancement	Paul Price	new	2009-01-03T10:27:09-10:00	2009-01-03T10:27:09-10:00	"I've found some problems with the output header fields for the smf files.  One serious problem is the value of MJD-OBS in the PHU vs the extension headers.  These seem to be rather different, possibly because they are being generated from different sources.  As an example, see:

ipp004:~eugene/data/bdsearch.20081225/smf.files/o4726g0105o.33162.cm.5860.smf

ipp004: fhead smf.files/o4765g0072o.43723.cm.5880.smf | grep MJD
MJD-OBS =     54765.4509398148 / Time of observation                           
ipp004: fhead -x 0 smf.files/o4765g0072o.43723.cm.5880.smf | grep MJD
MJD-OBS =          54765.45016 / Modified Julian date at start of observation  

Other examples exist, though the error may not be present for all exposures.  (This may be a lingering effect of the earlier MJD-OBS vs SHUTOUTC issue)."	eugene
Milestone 	1173	dvo should ask before creating non-existing CATDIR	DVO	unspecified	enhancement	eugene	new	2009-01-12T02:40:20-10:00	2009-01-12T02:40:20-10:00	"We just spent some time figuring out why a CATDIR that worked for everyone didn't work for one particular user. 

The cause: a typo in the directory name (two digits in a date that had swapped places). 

The reason: dvo silently (or perhaps non-silently, but unnnoticed by the newbie user) created the new CATDIR, which then of course didn't have any data in it.

Judging from the occasional nonsensically-named empty CATDIR that I come across on our cluster, inadvertent creation of a new CATDIR happens to every dvo user at least once.

My suggestion would be for dvo to ask before it creates an empty catdir."	jester@…
Milestone 	1278	Request for addstar-compatible .cmf files as output of ppSub+psphot	DVO	current	enhancement	eugene	new	2009-07-17T07:44:33-10:00	2009-07-17T08:03:38-10:00	"The Transient Classification Server has been planning on using DVO to associate stationary transient events.  In order to do this, we need for the .cmf files from ppSub+psphot to result in files that can be loaded into a DVO database using addstar.  Currently, at least in the July 7 data release, .cmf files produced in the diff stage have an EXTTYPE=IMAGE and so are passed over by addstar.  In addition the EXTNAME and EXTHEAD aren't set right.

But perhaps more substantially, there is no astrometric information in the header of the diff.cmf files.  The RA and Dec are recorded individually for the detections, but there's no general WCS given.

What is the currently expected behavior and future planned behavior for ppSub+psphot?

Lack of diff.cmf files that can be loaded into addstar is currently limiting the Transient Classification Server analysis of the MD images from June."	Michael Wood-Vasey
Milestone 	1410	detectability server too cumbersome for forced photometry	ppstamp		enhancement	bills	new	2010-07-19T22:59:53-10:00	2010-09-07T11:23:02-10:00	"
Several projects require forced photometry in bulk. Using the detectability server for this is very annoying because of the one exposure per request rule. We have written scripts to deal with the thousands of tiny fits files the detectability server produces, but it takes roughly 2 seconds per exposure to find each fits file and download it. This becomes unusable for even modest searches.

We need to be able to submit tables with:
expid ra dec
expid ra dec
...

And get the analogous table back. "	morganson
Milestone 	822	IERS table support needs to be rewritten	PSLib	0.12.99	defect	eugene	assigned	2006-09-22T15:28:52-10:00	2010-05-02T12:37:05-10:00	"All of the IERS tablular data has been modified into a pslib specific fortran
like format and checked into CVS.  No utilities are provided for re-parsing the
raw data into this format.  Further, the pslib.config file is full of all sort
of junk values.  Eg., ""psLib.time.predict.mjd F64 53257.0"" which is used by
psTime as the mjd input value when the IERS tables are exceeded."	jhoblitt
Milestone 	1187	unify uri and path_base to just path_base	ippTools		defect	watersc1@…	assigned	2009-03-03T17:21:13-10:00	2010-04-28T16:15:07-10:00	"In a number of places we have both URI and path_base.  this is ambiguous and is not really needed.  Originally, we wanted to be sure we knew where The File (ie, the important file) went.  But, we now have many output files that we require, so we cannot really fall-back on URI.  In contrast, in some tables (eg, detStackedImfile) we have only URI, which makes it difficult (~= impossible) to find the log files or others that get generated.

There may be only a handful of places where we have only a single file, in which case we can keep only URI and not path_base in those locations.
"	eugene
Milestone 	1193	redo version info for remaining packages	misc		defect	eugene	assigned	2009-03-05T16:42:23-10:00	2010-04-28T16:19:12-10:00	"I've defined a psbuild-based version of the svn info handling, and only added it to psLib, psModules, psphot, psastro, ppImage.  it needs to go into pswarp, ppSub, ppStack, ppMerge, ppArith, dvoTools, and pstamp
"	eugene
Milestone 	1219	Segfault in psphot when used on chip images with -F flags	psModules	Development Stage	defect	Paul Price	assigned	2009-04-10T17:30:14-10:00	2010-05-02T13:05:36-10:00	"Here's the command which produces the segfault- it's being run on odyssey.fas.harvard.edu with the top-of-tree SVN trunk code as of April 9.  If the -F flags are omitted the segfault does not occur.

psphot
-file /n/data1/panstarrs/GPC1/MD01_20081002_1/36475/chip/o4741g0234o.36475.ch.11948.XY01.ch.fits -mask /n/data1/panstarrs/GPC1/MD01_20081002_1/36475/chip/o4741g0234o.36475.ch.11948.XY01.ch.mk.fits -variance /n/data1/panstarrs/GPC1/MD01_20081002_1/36475/chip/o4741g0234o.36475.ch.11948.XY01.ch.wt.fits -F PSPHOT.OUTPUT PSPHOT.OUT.CMF.MEF 
-F PSPHOT.PSF.SAVEPSPHOT.PSF.SKY.SAVE 
/n/data1/panstarrs/GPC1/MD01_20081002_1/workspace/jsw_pswarp/MD01_20081002_1/36475/chip/o4741g0234o.36475.ch.11948.XY01

The segfault is in psMetadataLookupStr; the full stack trace and a bit
of additional debug info is below.  Note that the value of '*chip' is
0xdeadbeef .  Looking up the stack, fpa is 0xdeadbeef in
pmFPAAddSourceFromView .



findChipType (chip=0xdeadbeef0bbf5000, contents=0xac53040) at
pmFPAConstruct.c:474
474         const char *chipName = psMetadataLookupStr(NULL, chip->concepts,
""CHIP.NAME""); // Name of chip
(gdb) where
#0  findChipType (chip=0xdeadbeef0bbf5000, contents=0xac53040) at
pmFPAConstruct.c:474
#1  0x00002b1d883438f4 in addSource_CHIP_NONE (view=0xfa1b360,
fpa=0xfcf6f60, chip=0xdeadbeef0bbf5000, 
    format=0xac4f060, phdu=0xfa1b2b0, install=true) at
pmFPAConstruct.c:865
#2  0x00002b1d8834487d in addSource (fpa=0xfcf6f60, fpaObs=<value
optimized out>, phuView=0xf8c4af0, 
    header=0x0, format=0xac4f060, install=true) at pmFPAConstruct.c:1261
#3  0x00002b1d883456cc in pmFPAAddSourceFromView
(fpa=0xdeadbeef0bbf5000, fpaObs=0xac53040 ""�0�\n"", 
    phuView=0x2b1d88f29551, format=0x2b1d8903dee7) at
pmFPAConstruct.c:1401
#4  0x00002b1d88366d6c in pmFPAfileSuitableFPA (file=0xf661c30,
view=0x1036b240, config=0x97148e0, 
    pixels=false) at pmFPAfileFitsIO.c:85
#5  0x00002b1d883d50e0 in pmSource_CMF_WritePHU (view=0xac53040,
file=0xf661c30, config=0x2b1d88f29551)
    at pmSourceIO.c:568
#6  0x00002b1d88364037 in pmFPAfileWritePHU (file=0xf661c30,
view=0x1036b240, config=0x97148e0)
    at pmFPAfileIO.c:963
#7  0x00002b1d88365920 in pmFPAfileWrite (file=0xf661c30,
view=0xf86ee30, config=0x97148e0)
    at pmFPAfileIO.c:406
#8  0x00002b1d883660ae in pmFPAfileIOChecks (config=0x97148e0,
view=0xf86ee30, place=PM_FPA_AFTER)
    at pmFPAfileIO.c:78
#9  0x00000000004029c2 in psphotImageLoop (config=0x97148e0) at
psphotImageLoop.c:100
#10 0x0000000000401c52 in main (argc=<value optimized out>,
argv=0x7fff229d8268) at psphot.c:30
(gdb) p *chip
Cannot access memory at address 0xdeadbeef0bbf5000
(gdb) p *contents
$1 = {list = 0xac530f0, hash = 0xac533f0, lock = {__data = {__lock = 0,
__count = 0, __owner = 0, 
      __nusers = 0, __kind = 0, __spins = 0, __list = {__prev = 0x0,
__next = 0x0}}, 
    __size = '\0' <repeats 39 times>, __align = 0}}
"	welling
Milestone 	1285	addstar sometimes generates SkyTable.fits for wrong table depth	DVO		defect	eugene	new	2009-07-28T22:48:32-10:00	2009-07-28T22:48:32-10:00	"we noticed that the yz dvo database had sky table depth of 2 (z...) but the files were at depth of 3.  
"	dravg.ncu
Milestone 	1286	DVO measure needs dMcal and dMaper	DVO		defect	eugene	new	2009-08-02T13:25:58-10:00	2009-08-02T13:25:58-10:00	"currently, measure only has dM and dMcal, where dMcal is identified as the calibration error.  We need to track 3 types of errors:

dM = poisson error -- fundamental limit
dMaper = aperture - fit scatter -- limit of photometry analysis
dMcal = scatter of zero point -- limit of calibration analysis
"	eugene
Milestone 	1287	zero points in psastro and addstar	DVO		defect	eugene	new	2009-08-02T14:08:11-10:00	2009-08-02T14:08:11-10:00	"psastro measures a zero point as <M_ref - M_raw - 2.5log(exptime)>

addstar uses this value to set Mcal.

Both currently ignore airmass and color terms

psastro uses photcode values from the recipe file; addstar (and all of DVO) uses photcodes from the photcode table.

 "	eugene
Milestone 	1291	skycells needs to allow for overlapping edges.	Ohana		defect	eugene	assigned	2009-08-07T03:35:32-10:00	2010-06-06T16:56:31-10:00	"A buffer of 15 pixels on all edges of the sky cells from the stacks are masked. I found this first in MD08, a quick check by Mark Huber and I of MD09 and MD07 showed the same effect. The anomaly is illustrated on the PS1 wiki at

http://ps1sc.ifa.hawaii.edu/PS1wiki/index.php/Stack_Anomalies#Edges_of_sky_cells_are_masked

The effect was noticed when examining a galaxy at the edge of a sky cell. Examination of the RA,DEC values of the pixels indicates that the sky cells meet seemlessly - little or no gap nor overlap in the vertical edges of the sky cells. However, with this 15 pixels masking of each sky cell edge we lose a strip 6 arcsec wide (2 * 15 * 0.2 arcsec/pixel) of un-imaged sky. It would be good to turn off this unnecessary masking.

In a reply to my original post, Paul-Price noted that some masking is needed for convolution purposes (presumably to remove convolution edge effects).  That may be, but it would then seem that a better way to proceed is to make the sky cells bigger and then trim unnecessary masked pixels before making the final product.

Note - I am not familiar with IPP so my pick for ""component"" is a guess, and I have not picked keywords.

-- Gerhardt Meurer"	kpat.jhu
Milestone 	1297	x_psf_sig and y_psf_sig	psphot		defect	eugene	new	2009-08-13T12:05:49-10:00	2009-08-13T12:05:49-10:00	As of the July 2009 MD data release, x_psf_sig and y_psf_sig look badly calculated for faint objects.  A histogram of x_psf_sig vs psf_inst_mag, for instance, reveals a discontinuity in x_psf_sig, where suddenly errors get a hundred times worse.  I mentioned this a while ago and think a fix had already happened or was in the works.  However, I got bitten by it today and figured I'd make a ticket...	eschlafly@…
Milestone 	1298	suspect bad memory	hardware		defect	cindy	assigned	2009-08-18T10:11:50-10:00	2010-05-15T11:33:17-10:00	"These nodes have been suspect at one time or another of stability issues due to bad memory:

ipp005
ipp008
ipp018
ipp027
ipp025

With ipp018 being the worst offender as MCE errors are being logged.

They should be taken offline and have a memory check run on them.  In the past, we've had dubious results running memtest86+ but we've never gone into the BIOS and disabled ECC checking.  I think it's worth trying this sort of test again with ECC/scrubbing disabled.  The way to track down a defective memory module is via a binary search.  Eg. Run memtest86+ until an error is hit, remove have the memory sticks, run mt86+ again.  If no error is hit, swap the removed memory back in and the good memory out, repeat.  I'd let memtest86+ run for 5-7 days without an error to consider is a negative result."	jhoblitt
Milestone 	1299	Verify that MCE is enabled in the BIOS of all wave #2 nodes	hardware		defect	cindy	assigned	2009-08-18T10:13:12-10:00	2010-05-15T11:33:37-10:00	"We've verified that MCE is on for these wave #2 nodes:

15,17,26,27,28,33,34

The state of the rest is unknown."	jhoblitt
Milestone 	1300	configure the firewall services module in IPP core	hardware		defect	cindy	assigned	2009-08-18T10:18:02-10:00	2010-05-15T11:35:11-10:00	"We need to establish two vlans between the sup module and the fw module.  One vlan is the ""outside"" and the other being the ""inside"" vlan.  Routing needs to be configured being the sup the fw mod via the ""outside"" vlan.  NAT needs to be configured on the ""inside"" vlan for the PSPS.  When it become time for IPP wave #4, another seperate ""inside"" vlan needs to be created so all IPP nodes can be moved over to NAT.  I've had trouble in the past getting this configuration to work and at one point Cisco suggested the fwsvc mode might be bad and need to be replaced.  I never did this as I've never had time/resources to setup a proper test."	jhoblitt
Milestone 	1301	figure out why alala can't rebuild it's raid set	hardware		defect	cindy	assigned	2009-08-18T10:20:11-10:00	2010-05-15T11:35:54-10:00	disk bay #4 on one of the RAID controllers in alala is preventing the array from being rebuilt.  alala needs to be taken down and all of the raid cabling blown out/reconnected.  If that doesn't work I would try to me a direct `dd` copy of this disk to another drive and try that.  If that doesn't work I would suspect a bad RAID controller port.	jhoblitt
Milestone 	1302	test bonding on master.ipp	hardware		defect	cindy	assigned	2009-08-18T10:21:11-10:00	2010-05-15T11:37:02-10:00	Configure ethernet bonding between master.ipp and the IPP switch in the machine shop.  First roll this conf out to all ipp dev machines then the production cluster at MHPCC.	jhoblitt
Milestone 	1304	ipp000 has a dead CPU socket	hardware		defect	cindy	assigned	2009-08-18T10:26:11-10:00	2010-05-15T11:38:26-10:00	Just a reminder that one CPU socket is dead in this system (and that means half the memory is inaccessible too).  This node would be a good candidate for replacement if and/or when space becomes an issue in the machine shop.	jhoblitt
Milestone 	1306	2nd PDU in cab3 needs to be installed	hardware		defect	cindy	assigned	2009-08-18T10:29:06-10:00	2010-05-15T11:38:45-10:00		jhoblitt
Milestone 	1311	masks for non-existent stars in g band MD06 stacks	misc	unspecified	defect	eugene	assigned	2009-08-19T12:47:22-10:00	2010-05-15T11:16:41-10:00	"There are masked areas in the MD06 g band images that do not correspond to anything at other wavelengths. From the cross-hair shape of the masks it seems that IPP decided that a star was there. Since the source of the star masks is 2MASS, perhaps these correspond to sources that have some odd SED shape in the IR that IPP decided would only be bright in g. However, since the sources are not their at other wavelengths, maybe this is just some bug that hits only the g band. I have not seen similar non existent star masks popping up in the other bands.

example RGB=irg color images made from the stacks showing this effect can be found at  http://ps1sc.ifa.hawaii.edu/PS1wiki/index.php/Stack_Anomalies#non-existent_stars_only_masked_in_the_g_band

Some example coordinates are as follows 
#skycell xpix ypix RA Dec
100 3051.0 1355.0  184.259247  47.856604
081  721.0 3501.0  186.533728  47.301351
079 5727.0 4823.0  185.962259  47.380999
"	kpat.jhu
Milestone 	1313	linear artifacts in image stacks	ppStack		defect	Paul Price	assigned	2009-08-20T09:16:27-10:00	2010-05-02T12:50:18-10:00	"Most linear artifacts should not survive to the stack stage: satellite trails should be removed by MAGIC while star glints are masked earlier. Yet some of these streak like artifacts do survive, and have similar positions in the different bands. They tend to be faint. They may be more prevalent in skycells close to the field edge, but they are also seen in skycells away from the edge. Example of these artifacts can be found at http://ps1sc.ifa.hawaii.edu/PS1wiki/index.php/Stack_Anomalies#Streaks 

Here are positions of some of these in the MD06 skycells

#scell bands   x    y     RA         Dec      comment
  081  irg  5583 4175  186.933111  47.332772  edge cell
  079  zi   5879  687  185.970516  47.151152  not edge
  075  z    3847 2035  183.842383  47.224309  not edge
  057  i    5183  879  186.870371  46.484028  edge cell
  057  i    5207  671  186.871904  46.472448  edge cell
  053  g    3259 3719  184.778268  46.656823  not edge
  "	kpat.jhu
Milestone 	1321	Dynamic shared library libcfitsio not referred to most portably in Mac OS X	psconfig		defect	eugene	assigned	2009-08-30T11:41:38-10:00	2010-05-02T15:06:36-10:00	"On Aug 30, 2009, at 16:10 , Michael Wood-Vasey wrote:
My compilation of today's SVN head (r25223) on my Mac Pro (Intel) failed with

---
[serenity ippdb.src] psautogen

You must have  installed to compile ippdb.
Download the appropriate package for your distribution,
or get the source tarball at http://ftp.gnu.org/gnu/libtool/
---

Is this a libtool issue?

I installed the distribution version of libtool, and compilation went fine, but now I get missing dylib library errors:

---
[serenity psconfig] ppSub
dyld: Library not loaded: libcfitsio.dylib
  Referenced from: /Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/bin/ppSub
  Reason: image not found
Trace/BPT trap
[serenity psconfig] echo $CFITSIO
/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86
[serenity psconfig] ls /Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/*cfits*
/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/libcfitsio.a
/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/libcfitsio.dylib*
[serenity psconfig] echo $LD_LIBRARY_PATH
/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib:/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/mysql:/usr/local/lib:/usr/lib:/usr/X11R6/lib:/sw/lib:
[serenity psconfig] otool -D /Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/libcfitsio.dylib*
/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/libcfitsio.dylib:
libcfitsio.dylib
[serenity psconfig] otool -L /Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/libcfitsio.dylib
/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/libcfitsio.dylib:
	libcfitsio.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.4)
[serenity psconfig] otool -L /Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/bin/ppSub 
/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/bin/ppSub:
	/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/libppStats.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/libpsphot.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/libpsmodules.1.dylib (compatibility version 2.0.0, current version 2.1.0)
	/Volumes/data/PS1/src/ipp-2.9/Ohana/src/libkapa/lib/libkapa.darwin_x86.dylib (compatibility version 0.0.0, current version 0.0.0)
	/Volumes/data/PS1/src/ipp-2.9/Ohana/src/libdvo/lib/libdvo.darwin_x86.dylib (compatibility version 0.0.0, current version 0.0.0)
	/Volumes/data/PS1/src/ipp-2.9/Ohana/src/libfits/lib/libFITS.darwin_x86.dylib (compatibility version 0.0.0, current version 0.0.0)
	/Volumes/data/PS1/src/ipp-2.9/Ohana/src/libohana/lib/libohana.darwin_x86.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/X11/lib/libX11.6.dylib (compatibility version 9.0.0, current version 9.0.0)
	/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/libpng12.0.dylib (compatibility version 16.0.0, current version 16.0.0)
	/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/libnebclient-0.17.dylib (compatibility version 0.0.0, current version 0.0.0)
	/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/libpslib.1.dylib (compatibility version 2.0.0, current version 2.1.0)
	/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/mysql/libmysqlclient.15.dylib (compatibility version 16.0.0, current version 16.0.0)
	/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.3)
	libcfitsio.dylib (compatibility version 0.0.0, current version 0.0.0)
	/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/libfftw3f.3.dylib (compatibility version 4.0.0, current version 4.1.0)
	/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/libgsl.0.dylib (compatibility version 13.0.0, current version 13.0.0)
	/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib/libgslcblas.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.4)
	/usr/lib/libmx.A.dylib (compatibility version 1.0.0, current version 292.4.0)
	/usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
[serenity psconfig] otool -L /Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/bin/ppSub | grep cfitsio
	libcfitsio.dylib (compatibility version 0.0.0, current version 0.0.0)
[serenity lib] pwd
/Volumes/data/PS1/code/ipp-2.9/default.darwin_x86/lib
[serenity lib] ppSub
2009-08-30 21:41:13Z|serenity.phyast.pitt.edu|I|ppSub
    ppSub at 2009-08-30T21:41:47.677001
2009-08-30 21:41:13Z|serenity.phyast.pitt.edu|I|ppSub
    psLib trunk/psLib@25223M from 60eb6cdc-a59c-4636-a4e0-dba66a9721fd, built Aug 30 2009, 16:14:57 with cfitsio-3.100 gsl-1.11 fftw-3.0.1 mysql-5.0.51a optimised without trace
2009-08-30 21:41:13Z|serenity.phyast.pitt.edu|I|ppSub
    psModules trunk/psModules@25223 from 60eb6cdc-a59c-4636-a4e0-dba66a9721fd, built Aug 30 2009, 16:16:22 optimised with nebclient
2009-08-30 21:41:13Z|serenity.phyast.pitt.edu|I|ppSub
    psphot trunk/psLib@25223M from 60eb6cdc-a59c-4636-a4e0-dba66a9721fd, built Aug 30 2009, 16:17:12 optimised with libkapa
2009-08-30 21:41:13Z|serenity.phyast.pitt.edu|I|ppSub
    ppStats trunk/ppStats@25223 from 60eb6cdc-a59c-4636-a4e0-dba66a9721fd, built Aug 30 2009, 16:16:44 optimised
2009-08-30 21:41:13Z|serenity.phyast.pitt.edu|I|ppSub
    ppSub trunk/ppSub@25223 from 60eb6cdc-a59c-4636-a4e0-dba66a9721fd, built Aug 30 2009, 16:19:28 optimised

Pan-STARRS PSF-matched image subtraction

Usage: ppSub OUTPUT_ROOT 
	[-psf REFERENCE.psf.fits]

This subtracts the convolved REFERENCE from the INPUT, by default.

Optional arguments, with default values:
  RECIPES         OPTIONS         -inimage      ()       Input image
  -inmask       ()       Input mask image
  -invariance   ()       Input variance image
  -insources    ()       Input source list
  -refimage     ()       Reference image
  -refmask      ()       Reference mask image
  -refvariance  ()       Reference variance image
  -refsources   ()       Reference source list
  -kernel       ()       Precalculated kernel to apply
  -stats        ()       Statistics file
  -stamps       ()       Stamps filename; x,y on each line
  -threads      (0)      Number of threads
  -dumpconfig   ()       file to dump configuration to
  -convolve     (0)      Image to convolve [1 or 2]
  -photometry   (FALSE)  Perform photometry?
  -inverse      (FALSE)  Generate inverse subtractions?
  -visual       (FALSE)  Show diagnostic plots
---

So I think my concern is that 'libcfitsio.dylib' is not fully specified in the ppSub executable and it's not in the default dynamic library paths.  See the helpful post at 

http://lists.apple.com/archives/xcode-users/2009/Aug/msg00165.html

for some details on how this works.  You can muck around with environment variables to try to help find dylib paths, but that actually tends to cause problems when loading just normal programs.  It's my understanding that the ""correct"" thing to do is install the dynamic library with the path of:

@executable_path/../lib/libcfitsio.dylib

In the ideal world this makes the directory completely portable.  You can just move the directory around and all the paths are still find.  This is how Mac OS X handles application bundles.
"	Michael Wood-Vasey
Milestone 	1324	latest Xcode for mac 10.5 breaks building of IPP	Ohana	current	defect	heather	assigned	2009-08-31T10:34:51-10:00	2011-02-02T20:11:00-10:00	"building the ipp (svn 25223) with the latest version of Xcode (version 3.1.3 )

causes this error:

In file included from /usr/X11/include/X11/Xlib.h:64,
                 from /Users/hswan/bin/default.darwin_x86/include/kapa.h:10,
                 from pmKapaPlots.h:23,
                 from pmKapaPlots.c:20:
/usr/X11/include/X11/Xosdefs.h:145:1: error: ""_DARWIN_C_SOURCE"" redefined
<command line>:1:1: error: this is the location of the previous definition
make[3]: *** [libpsmodulesextras_la-pmKapaPlots.lo] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all] Error 2
make: *** [all-recursive] Error 1

_DARWIN_C_SOURCE is now defined in /usr/X11/include/X11/Xosdefs.h, previously it was not.


"	heather
Milestone 	1325	DVO nmeas vs ncode vs nused	DVO		defect	eugene	new	2009-09-01T10:21:51-10:00	2009-09-01T10:21:51-10:00	I became confused today as to why the sum of the ncodes in the cps files did not equal nmeas in the cpt; eventually tracked it to the psfqual being < .85 for those measurements, and so filtered by in StarOps.c line 210.  This might be the right behavior but it is non-obvious; just pinging about it.	eschlafly@…
Milestone 	1326	replace restricted links to project pages if possible	docs		defect	eugene	new	2009-09-08T12:29:20-10:00	2009-09-08T12:29:20-10:00	"I've copied many of the documents under the project pages (with restricted access) to [wiki:IPP_Manuals[ and [wiki:IPP_Engineering].  

Some other pages remain, though, which should be made publicly visible:

* http://pan-starrs.ifa.hawaii.edu/project/IPP/software
* http://pan-starrs.ifa.hawaii.edu/mailman/private/ps-ipp-users/msg00720.html 
 (and other mailman related entries)
"	eugene
Milestone 	1329	DVO SkyTable corruption for mis-matched groups	DVO		defect	eugene	new	2009-09-21T11:01:17-10:00	2009-09-21T11:01:17-10:00	"Roger Lin has found a problem working with a DVO database where different group IDs apparently interacted with the DVO updates to the SkyTable.fits to re-write the SkyTable with an incorrect set of values.

I think the issue here is that the code is checking for permission to lock the skytable, but due to the group IDs, there is an inconsistency with the request for lock and the permission check.  somehow, DVO decides that it needs to update the SkyTable as a result."	eugene
Milestone 	1335	psphot -modeltest cannot do forced photometry of empty sky	psphot	current	defect	eugene	new	2009-10-23T11:18:47-10:00	2009-10-23T11:18:47-10:00	"  I'm looking into problems Michael has reported (see below) with doing forced photometry with psphot via the -modeltest option, and the issue seems to be a feature rather than a bug.  If one tries forced photometry at a point which isn't a statistically significant peak, the tests for significant pixels at pmSourceMoments.c:136 and 137 exclude so many pixels that the test at line 167 returns false and causes photometry to fail.  
  I expected that the routine would calculate moments and variance
estimates for the moments, but that doesn't actually seem to be the
goal- pmSourceMoments() seems to be designed only to generate moments for a source which definitely exists.  We're looking for a way to do forced photometry for locations before the SN or asteroid appears, so we need to look at empty points in space.  Would it be reasonable to elaborate pmSource with some variance information and change pmSourceMoments to cease to exclude based on the tests at lines 136 and 137?

------original message------

Hi Gene,

I'm having trouble with the force-position psphot photometry

""""""
[serenity o5083g0235o.95580] psphot -file o5083g0235o.95580.wrp. 
11968.skycell.064.fits -mask o5083g0235o.95580.wrp.11968.skycell. 
064.mask.fits -variance o5083g0235o.95580.wrp.11968.skycell. 
064.wt.fits -modeltest 817.7 213.0  -model PS_MODEL_PS1_V1    test
[...]
2009-10-13 21:33:18Z|serenity.phyast.pitt.edu|I|psphot
     background sky : min -24.786289 mean 5.599239 max 14.937018 stdev  
2.697644
2009-10-13 21:33:18Z|serenity.phyast.pitt.edu|5|psphot
     build resampled image: 0.359058 sec
2009-10-13 21:33:19Z|serenity.phyast.pitt.edu|I|psphot
     subtracted background model: 0.696939 sec
Aborting in function psphotModelTest at psphotModelTest.c:142. Error  
stack:
2009-10-13 21:33:19Z|serenity.phyast.pitt.edu|A|psLib.sys
     psSourceMoments error
Abort trap
""""""

This worked back in May, 2009 for

""""""
psphot -file o4741g0236o.36477.ch.11950.XY12.ch.fits -threads 4
-modeltest 2.63e+3 3.41e+3 test_out2
""""""

but I don't have that image locally anymore.  So I tried fitset, but I  can't figure out what the third column is supposed to be and then it  didn't seem to work for other reasons.

""""""
[serenity o5083g0235o.95580] psphot -file o5083g0235o.95580.wrp. 
11968.skycell.064.fits -mask o5083g0235o.95580.wrp.11968.skycell. 
064.mask.fits -variance o5083g0235o.95580.wrp.11968.skycell. 
064.wt.fits -fitset forcexy.txt     test
  psphotArguments (psphotArguments.c:242) : Expected to see one more  argument; saw 3
Usage: one of the following
psphot -file fname1[,fname2,...] -mask maskfile1[,maskfile2,...]
      -variance varfile1[,varfile2,...] OutFileBaseName

psphot -list FileNameList [-masklist MaskFileNameList]
      -variancelist VarFileNameList OutFileBaseName

Try 'psphot -help' for more options and explanation
Error reading arguments

  -> psphotArguments (psphotArguments.c:242): Incorrect arguments
      Expected to see one more argument; saw 3
[serenity o5083g0235o.95580] cat forcexy.txt
817.7 213.0 1
""""""

I'm happy to file this as a bug report, but I'm can't figure out what  the desired usage is and so am not sure whether if it's a psphot bug  or just a user bug.

Thanks,
Michael"	welling
Milestone 	1336	ppSim does not generate a valid .cmf file for addstar	DVO		defect	eugene	new	2009-11-02T14:05:58-10:00	2009-11-02T14:05:58-10:00	"Running the ppSim command:

ppSim -camera SIMTEST -recipe PPSIM STACKTEST.MAKE -type OBJECT -filter r -exptime 10.0 -skymags 20.86 -ra 270.70 -dec -23.70 -pa 0.0 -seeing 1.0 refimage -Df STARS.DENSITY 100 -Df STARS.MAG 10

creates a .cmf file that is missing the PHOTCODE, ZERO_PT, and NSTARS header keywords required to be ingested by addstar:

addstar -D CAMERA simtest -D CATDIR catdir.ref -accept-astrom -photcode SYNTH.r -D PHOTCODE_FILE ${SYNTHDIR}/dvo.photcodes refimage.cmf"	watersc1
Milestone 	1340	neb-rm -f is not functional	Nebulous		defect	watersc1	assigned	2009-12-02T13:44:42-10:00	2010-05-02T14:58:44-10:00	"I used neb-rm -f on an instance on ipp014 (which has been rebuilt, so isn't in nebulous, and has no instances on it).  it fails:

 neb-rm -f gpc1/detrend/GPC1.MASK.20090219/GPC1.MASK.20090219.XY01.fits
2009/12/02 13:13:44 | ipp004 | FATAL | Nebulous::Client::find_instances - unhandled fault - database error: no instances available for key: neb:///gpc1/detrend/GPC1.MASK.20090219/GPC1.MASK.20090219.XY01.fits at /usr/lib64/perl5/site_perl/5.8.8/Nebulous/Server.pm line 1673, <DATA> line 12.
unhandled fault - database error: no instances available for key: neb:///gpc1/detrend/GPC1.MASK.20090219/GPC1.MASK.20090219.XY01.fits at /usr/lib64/perl5/site_perl/5.8.8/Nebulous/Server.pm line 1673, <DATA> line 12.

I get an identical error when I used neb-rm. For both cases, it didn't remove the instance from the database.

I was able to get around this by using neb-rm -m to 'move' the key to a different name. "	heather
Milestone 	1342	psMatrix.c ignores gsl return errors	PSLib		defect	eugene	new	2009-12-06T09:24:34-10:00	2009-12-06T09:24:34-10:00	"we need to handle errors reported by the gsl code (most are related to matrix dimensionality which we handle before passing to gsl code, but some are due to the character of the data and should not be ignored).

"	eugene
Milestone 	1344	Optionally link recipe values to dates and/or camera formats	ippconfig		defect	Paul Price	new	2010-01-08T11:11:43-10:00	2010-01-08T11:11:43-10:00	"There are some recipe options which change for a given camera as a function of when the data was take.  For example, before Dec 9, 2009, GPC1 data needed to be corrected with the ""PATTERN"" option (row-by-row pattern subtraction).  After Dec 9, 2009, this is no longer needed.  

Our basic recipe management system can only handle this by having different reduction classes for these two periods.  However, this is not something whic gets automatically set.

We need to be able to specific that certain recipes, recipe values, or reduction classes should be valid in certain time periods."	eugene
Milestone 	1345	psastro fails to find zeropoint in simtest	psastro		defect	eugene	new	2010-01-11T13:40:02-10:00	2010-01-11T13:40:02-10:00	"psastroZeroPointReadoutAccum finds matches == NULL so it
returns a null vector then psastroZeroPointAnalysis prints a message
    zero point NaN +/- NaN

(originally it SEGV'd but i fixed that)"	bills
Milestone 	1346	stacktool bugs	ippTools		defect	watersc1	assigned	2010-01-14T12:07:46-10:00	2010-05-15T11:50:58-10:00	"in sql definebyquery, stackRun.label should be stackRun.data_group, to do the exclusion correctly.

stacktool also needs to be able to correctly handle NULL values of ""num_stack"", and convert these into 0."	watersc1
Milestone 	1348	IPP034 bad raid card	hardware		defect	cindy	assigned	2010-01-25T12:06:06-10:00	2010-02-18T12:39:34-10:00	unble to boot computer or enter raid bios, card returned to Rackmount Jan 25, 2010 attn Patrick Yam	cindy
Milestone 	1350	pantasks number of label limit	PanTasks		defect	eugene	new	2010-02-02T12:09:26-10:00	2010-02-02T12:09:26-10:00	pantasks segfaults if you add.label more than 100 labels.	watersc1
Milestone 	1353	ppSub from eam-20091201 aborts unable to match images that subtracted fine before	ppSub	Development Stage	defect	Paul Price	new	2010-02-08T12:50:10-10:00	2010-02-08T12:50:10-10:00	"ppSub from eam-20091201 aborts with

 subtractionMatchCheck (pmSubtractionMatch.c:170) : Error: !(normFrac > 0.0) (0.000000 0.000000).
 ppSubMatchPSFs (ppSubMatchPSFs.c:385) : Unable to match images.


for images that subtract quite well with SVN TRUNK ppSub.  Tarball to replicate at:

http://serenity.phyast.pitt.edu/~wwoodvas/PS1/MD04_skycell.042_eam_subfail.tgz

Run sub.sh



"	Michael Wood-Vasey
Milestone 	1357	diffraction spike width	psastro		defect	watersc1	new	2010-02-18T11:52:44-10:00	2010-02-18T11:52:44-10:00	The width of the diffraction spike masks is too narrow for the brightest stars.	watersc1
Milestone 	1359	CAL_PSF_MAG wrong, CAL_PSF_MAG_SIG undefined in SSdiff cmf files	psphot		defect	eugene	new	2010-03-03T10:25:28-10:00	2010-03-03T10:25:28-10:00	"The CAL_PSF_MAG values are several magnitudes too faint and CAL_PSF_MAG_SIG is undefined in the recent SSdiff cmf files.  The CAL_PSF_MAG values range from 24.5-29 instead of from say 18-23. For example,

MD06.nightlyscience/2010/02/26/MD06/skycell.105/MD06.skycell.105.SS.dif.40577.cmf

A SNR of 5 is reported as CAL_PSF_MAG 28.4.  I would expect this to be close to 23.5.

This smacks of something like a missing factor of 2.5*log10(EXPTIME) in the zeropoint conversion.  But this isn't quite right.  This exptime in the FITS header says 1920 seconds and 2.5*log10(1920) is 8.2, and we're looking for something more like 5-6, but maybe my sense of what the values should be is slightly off and it is just 2.5*log10(EXPTIME).

Clearly CAL_PSF_MAG_SIG should probably just be PSF_INST_MAG_SIG, and we can use that for now; but this should be fixed as well."	Michael Wood-Vasey
Milestone 	1360	CAL_PSF_MAG_SIG 0 in recent stack file	ppStack		defect	Paul Price	new	2010-03-03T10:27:06-10:00	2010-03-03T10:40:48-10:00	"CAL_PSF_MAG_SIG is set to 0. in recent stack files.  For example,

MD06.nt/2010/02/27/MD06/skycell.105/MD06.skycell.105.stk.65955.cmf

"	Michael Wood-Vasey
Milestone 	1362	some null fields on the DVO  image table	DVO		defect	eugene	new	2010-03-17T01:30:26-10:00	2010-03-17T01:30:26-10:00	"those entries of the image table are null (in the SAS database, around RA=138°, DEC=34°):
FWHM
ncal
apresid
Mcal
while
sky==32768
sourceID == 33 (except sourceID[0] = 0)"	kpat.mpg
Milestone 	1368	Exposures waiting on fixed reference catalogue	DVO		defect	eugene	new	2010-04-14T13:00:36-10:00	2010-04-14T13:00:36-10:00	"The following exposures are waiting on a fix to the reference catalogue (corruption at the pole).

mysql> select cam_id, exp_name, ra, decl from camProcessedExp join camRun using(cam_id) join chipRun using(chip_id) join rawExp using(exp_id) where camProcessedExp.fault != 0 and decl > 1.5;
+--------+-------------+----------+----------+
| cam_id | exp_name    | ra       | decl     |
+--------+-------------+----------+----------+
|  11803 | o4920g0028o | 1.334197 | 1.558217 | 
|  11810 | o4928g0092o |  0.66249 | 1.557952 | 
|  48386 | o5246g0063o | 1.571178 | 1.552927 | 
|  48401 | o5246g0080o | 1.570996 | 1.552928 | 
|  11816 | o4950g0019o | 4.448953 | 1.528762 | 
+--------+-------------+----------+----------+

I've set them to 'wait':

mysql> update camProcessedExp join camRun using(cam_id) join chipRun using(chip_id) join rawExp using(exp_id) set camRun.state = 'wait' where camProcessedExp.fault != 0 and decl > 1.5;
Query OK, 5 rows affected (0.19 sec)
Rows matched: 5  Changed: 5  Warnings: 0
"	Paul Price
Milestone 	1377	pscheckperl doesn't install Params::Validate, Log::Dispatch::Email::MailSend, Test::Pod, Test::Distribution	misc	current	defect	eugene	new	2010-05-11T10:02:20-10:00	2010-05-11T10:02:20-10:00	"pscheckperl doesn't install Params::Validate, Log::Dispatch::Email::MailSend, Test::Pod, Test::Distribution in a location that Perl later finds.

You can run 'pscheckperl -build' and it will install the modules, but apparently not in the locations that pscheckperl later looks for them.

This happens for me on both Mac OS X, and SE Linux 5.4 machines.

Is this something about the MakeMaker vs. Build.pl stuff?"	Michael Wood-Vasey
Milestone 	1380	ppSim does not apply recipe values from camera config	ppSim		defect	eugene	new	2010-05-12T10:14:52-10:00	2010-05-12T10:14:52-10:00	"recently simtest has been having memory problems creating the simulated images. It was attempting to add too many stars.
Changing the value for STARS.SIGMA.LIM in simtest/ppSim.config had no effect.

It turns out that the values in the camera specific recipe file are not overriding the values the default recipe file recipes/ppSim.config

The value in the default recipe file was 0.1 
I've bumped that to 1 which fixes simtest.

The application of the camera specific recipe should be fixed as well."	bills
Milestone 	1384	outstanding pantasks startup issues	misc		defect	eugene	new	2010-05-17T12:34:01-10:00	2010-05-17T12:34:01-10:00	"some outstanding startup issues which should be addressed

* nebulous.site.pro should be in a default location
* ctimes is not defined for pantasks_server (InitAstro not called)
"	eugene
Milestone 	1389	neb-df	Nebulous		defect	watersc1	new	2010-06-01T12:47:14-10:00	2010-06-01T12:47:14-10:00	Nebulous provides answers inconsistent with a regular df.  This is likely due to the numbers of reserved blocks on some hosts.	watersc1
Milestone 	1392	pstamp jobs need to deal with missing mask files	pstamp		defect	bills	new	2010-06-05T07:57:57-10:00	2010-06-05T07:57:57-10:00	"There are a number of camera runs that have missing camera mask files. They were able to get through the system because several months ago there was a bug in warp and destreak processing that fell back to the chip mask in this situation. Now that these bugs are fixed the affected exposures get queued for postage stamp processing but the jobs fail due to the missing mask files.

This error needs to be detected and the jobs faulted. It will be tricking to distinguish between these types of errors and normal nfs faults. 

Perhaps we need a mode of neb-locate that distinguishes between this file is on a not-available node"" and ""really, really there are no instances of such a file"""	bills
Milestone 	1393	burntool needs better failure recovery	misc		defect	watersc1	assigned	2010-06-05T16:31:41-10:00	2010-06-05T16:32:01-10:00	"if a single image is broken, burntool repeatedly fails, and as a result blocks all other processing until it is done.  better ippMonitor notification of these failures would help as well.
"	eugene
Milestone 	1395	CAL_PSF_MAG_SIG 0 in recent stack and stack-stack diff cmf files	psphot		defect	eugene	new	2010-06-08T12:17:02-10:00	2010-06-08T12:17:02-10:00	"CAL_PSF_MAG_SIG is 0 in stacks and stack-stack diffs processed within the last week.  E.g., see stack ID 115417 for MD06.skycell.073. and stack-stack diff ID 53591.

"	Michael Wood-Vasey
Milestone 	1396	postage stamp dependents get stuck if chip destreaking fails	pstamp		defect	bills	new	2010-06-14T15:07:16-10:00	2010-06-14T15:07:16-10:00	"There was a period where camera runs were silently failing to output all of the camera stage mask files. Warp and destreak processing fell back to the chip stage mask in this case. This allowed exposures to go through the system in this state.

Since we no longer allow the fallback updates for runs in this state fail.

These errors are difficult to recognize automatically because they have the same symptom as network outages 'failed to open file blah'

A similar problem is that destreaking fails because the previous run wasn't completely cleaned up. [At some time there was a bug in destreak cleanup that left the chip mask files lying around.]

We need to figure out how to detect these problems and correct them iff possible.

If the problem can't be corrected we need to stop the job and return the proper error code to the client PSTAMP_GONE

"	bills
Milestone 	1397	postage stamp lookup bycoord chip may be using too small a radius	pstamp		defect	bills	new	2010-06-15T11:38:16-10:00	2010-06-15T11:38:16-10:00	"postage stamp requests with REQ_TYPE bycoord and (stage = chip or stage = raw) select exposures by querying the database for exposures whose reported bore sight is within 1.6 degrees of the requested coordinates. 

This value may be too small."	bills
Milestone 	1398	"nebulous ""slow query"" timeout is too short"	Nebulous		defect	watersc1	new	2010-06-16T13:53:29-10:00	2010-06-16T13:53:29-10:00	The mysqld.slow.log is filling up /var on ippdb00 with a large number of queries that have just missed the 1s cut off.  Once processing slows down (preferably a day with no processing due to bad weather at the summit), increase this cut off and restart mysqld on ippdb00.	watersc1
Milestone 	1399	replication fails if neb-admin pid remains	ippTasks		defect	watersc1	new	2010-06-18T16:43:06-10:00	2010-06-18T16:43:06-10:00	"The replication server uses neb-admin to identify files in need of replication. neb-admin creates a /var/tmp/neb-admin pid file to block multiple instances from running.  However, in the event of a nebulous crash, this pid file can stick around after neb-admin finishes.  This blocks future instances, preventing replication from working.

replicate.pro should know about this, and test for the pid file and clean it up if it exists."	watersc1
Milestone 	1400	neb-migrate is painfully slow	Nebulous		defect	watersc1	new	2010-06-29T14:25:14-10:00	2010-06-29T14:25:14-10:00	"This probably should be fixed in some fashion so that it runs in a reasonable timescale. Test case:


mysql> select count(ins_id),NOW() from instance where vol_id = 2 ;
+---------------+---------------------+
| count(ins_id) | NOW()               |
+---------------+---------------------+
|       1531146 | 2010-06-29 13:13:30 | 
+---------------+---------------------+

mysql> select count(ins_id),NOW() from instance where vol_id = 2 ;
+---------------+---------------------+
| count(ins_id) | NOW()               |
+---------------+---------------------+
|       1530712 | 2010-06-29 13:28:18 | 
+---------------+---------------------+


works out to 2.1843s per instance or a total of 928.76h for the full operation.
"	watersc1
Milestone 	1401	RELASTRO_SRC_MEAS_TOOFEW not in current trunk	DVO	current	defect	magnier	new	2010-07-02T05:23:29-10:00	2010-07-02T05:23:29-10:00	"RELASTRO_SRC_MEAS_TOOFEW is not defined in the latest SVN head on the main trunk.

I'm trying following directions on 

http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/PS1_MD04_RefStack

and get the following

[serenity ~] catdir=/Volumes/PS1/data/DVO/MD04.v1
[serenity ~] region=""-region 147.0 152.0 0.0 4.2""
[serenity ~] relastro $region -v -D GRID_TOOFEW 10 -D MOSAICNAME GPC1 -D CATDIR $catdir -update-objects -update
error in config, cannot find RELASTRO_SRC_MEAS_TOOFEW
"	Michael Wood-Vasey
Milestone 	1402	Continued type/integer size inconsistencies	psModules		defect	eugene	new	2010-07-02T11:17:22-10:00	2010-07-02T13:54:17-10:00	"In compiling psModules with SVN HEAD as of today, Jul 2:

----
cc1: warnings being treated as errors
pmDark.c: In function ‘pmDarkCombine’:
pmDark.c:354: warning: cast from pointer to integer of different size
{standard input}:unknown:Undefined local symbol LC6
{standard input}:unknown:Undefined local symbol LC7
{standard input}:unknown:Undefined local symbol LC20
{standard input}:unknown:Undefined local symbol LC21
{standard input}:unknown:Undefined local symbol LC0
---

Similar errors encountered with psLib"	Michael Wood-Vasey
Milestone 	1406	detectability returns wrong error for undestreaked data	ppstamp		defect	watersc1	new	2010-07-09T14:07:20-10:00	2010-07-09T14:07:20-10:00	"When data is not destreaked, detectability returns error 23 (MISSING IMAGE) and not error 24 (NOT DESTREAKED).

Not really a show stopping bug, but it might be more useful to the users."	watersc1
Milestone 	1407	MD04 refstack/altstack skycells don't have pixel data in the overlap regions	misc		defect	eugene	new	2010-07-12T04:52:56-10:00	2010-07-12T04:52:56-10:00	"The MD04 reference stacks have blank/NAN regions on the edges.  Is this the disagreement between the tessellations chosen for the warps and for the stacks?  The skycells have 15-pixel boundaries on each side.  If you ask ds9 to line them up according to WCS, you get 30-pixel NaN boundaries between the skycells.

This is true in both the MD04 refstack (0.258""/pix) and the altstack (0.2""/pix).  This is true for both the unconv and conv images.  All have a 15 pixel buffer.  The 0.258""/pix version is this missing more sky.

Is an error in the tessellation specification or in the stacking?

See, e.g.,

skycell.054/MD04.skycell.054.stk.120969.unconv.fits
skycell.055/MD04.skycell.055.stk.120970.unconv.fits

Image attached.  This image looks the same in the refstack or altstack."	Michael Wood-Vasey
Milestone 	1408	diff updates fail for older diffRuns	ppSub		defect	eugene	new	2010-07-14T10:44:15-10:00	2010-07-14T10:44:15-10:00	"We got some postage stamp requests for diffs from 2009 November.
Updates fail for these because the ppSub.mdc file does not
have a filerule definition for PPSUB.OUTPUT.RESID.JPEG 

The diff_ids for these are <= ~ 23000 

We need to either fix this problem in ppSub or set these runs to
purged so that it is clear that they are unavailable for update."	bills
Milestone 	1409	imextract returns nonsensical results for several fields	DVO		defect	eugene	new	2010-07-19T22:49:39-10:00	2010-07-19T22:49:39-10:00	"imextract returns

0 0 0 0 32768 
for
MCAL dMCAL detlimit FWHM sky"	morganson
Milestone 	1411	ppStack occasionally runs into memory corruption in cfitsio	psModules	current	defect	bills	new	2010-07-20T14:17:45-10:00	2010-11-07T08:14:31-10:00	"I have some news about the bug that I've been trying to catch where ppStack encounters memory corruption in cfitsio
while writing out the output files. (See stack trace below)

What I think I know as of now

1. It is repeatable
2. It happens with a debug build off of the trunk as well as the current tag (so it isn't fixed).
3. It happens with multiple stack runs ( MD08: 127071 129306 129336 MD04: 117659 117660) The oldest run with this error is from mid-June.
4. In the one I've been re-running (127071) it dies during the second call to pmFPAWriteVariance when
compressing the output file for the file rule PPSTACK.OUTPUT.EXPTWT


I currently have it running single threaded with a breakpoint in pmFPAWriteVariance and a non-optimized cfitsio. Hopefully
I'll be able to step through and find something.

I'm not familiar with ppStack or cfitsio compression or how we use it so this is challenging.

These files seem to compress by a large amount so simply turning off compression as a workaround isn't a good idea.
(I decompressed one of these from another run and it grew by a factor of 30)


In summary, the bad news is the bug is more common than I had thought and it takes hours to get to the breakpoint and one fat
fingered error requires restarting.

But the good news is that it appears to be somewhat deterministic.


Sample stack trace. Sometimes it dies with 'double free or corruption' other times it says  'invalid next size' But the stack
trace and addres of the call to cfree is the same.



   Stage 6: Photometry Analysis: 24.772171 sec
*** glibc detected *** /home/panstarrs/ipp/psconfig/ipp-20100701.lin64/bin/ppStack: double free or corruption (!prev): 0x000000000855c0c0 ***
======= Backtrace: =========
/lib/libc.so.6[0x7f2eefe16b88]
/lib/libc.so.6(cfree+0x76)[0x7f2eefe18746]
/home/panstarrs/ipp/psconfig//ipp-20100701.lin64/lib/libcfitsio.so(imcomp_compress_tile+0x4dc)[0x7f2ef1238dfc]
/home/panstarrs/ipp/psconfig//ipp-20100701.lin64/lib/libcfitsio.so(fits_write_compressed_img+0xcd3)[0x7f2ef123aa43]
/home/panstarrs/ipp/psconfig//ipp-20100701.lin64/lib/libcfitsio.so(fits_write_compressed_img_plane+0x257)[0x7f2ef123b167]
/home/panstarrs/ipp/psconfig//ipp-20100701.lin64/lib/libcfitsio.so(fits_write_compressed_pixels+0x35c)[0x7f2ef123b5cc]
/home/panstarrs/ipp/psconfig//ipp-20100701.lin64/lib/libcfitsio.so(ffppri+0xb7)[0x7f2ef1214e07]
/home/panstarrs/ipp/psconfig//ipp-20100701.lin64/lib/libcfitsio.so(ffppr+0x12a)[0x7f2ef120b5aa]
/home/panstarrs/ipp/psconfig//ipp-20100701.lin64/lib/libpslib.so.1(psFitsInsertImageWithMask+0xaae)[0x7f2ef276015e]
/home/panstarrs/ipp/psconfig//ipp-20100701.lin64/lib/libpslib.so.1(psFitsWriteImageWithMask+0x96)[0x7f2ef2760236]
/home/panstarrs/ipp/psconfig//ipp-20100701.lin64/lib/libpslib.so.1(psFitsWriteImageCubeWithMask+0x3df)[0x7f2ef27606ff]
/home/panstarrs/ipp/psconfig//ipp-20100701.lin64/lib/libpsmodules.so.1[0x7f2ef4187b1f]
/home/panstarrs/ipp/psconfig//ipp-20100701.lin64/lib/libpsmodules.so.1[0x7f2ef4186946]
/home/panstarrs/ipp/psconfig//ipp-20100701.lin64/lib/libpsmodules.so.1(pmFPAWriteVariance+0x2c)[0x7f2ef4186a9c]
/home/panstarrs/ipp/psconfig//ipp-20100701.lin64/lib/libpsmodules.so.1[0x7f2ef4197445]
/home/panstarrs/ipp/psconfig//ipp-20100701.lin64/lib/libpsmodules.so.1(pmFPAfileWrite+0x406)[0x7f2ef41954f6]
/home/panstarrs/ipp/psconfig//ipp-20100701.lin64/lib/libpsmodules.so.1(pmFPAfileIOChecks+0x12e)[0x7f2ef4195dfe]
/home/panstarrs/ipp/psconfig/ipp-20100701.lin64/bin/ppStack[0x408c08]
/home/panstarrs/ipp/psconfig/ipp-20100701.lin64/bin/ppStack[0x4140cf]
/home/panstarrs/ipp/psconfig/ipp-20100701.lin64/bin/ppStack[0x409648]
/home/panstarrs/ipp/psconfig/ipp-20100701.lin64/bin/ppStack[0x405579]
/lib/libc.so.6(__libc_start_main+0xe6)[0x7f2eefdc0486] 

-------

Gene suggested that we might be able to write out the affected image uncompressed and use fpack or ppImage to attempt reproduce the problem without waiting for hours and hours.

"	bills
Milestone 	1414	postage stamp server does not handle dynamic masks properly	pstamp		defect	bills	new	2010-08-10T12:42:52-10:00	2010-08-10T12:42:52-10:00	"The postage stamp server assumes that the current PSASTRO
recipe is the same as that used to processe the input data to determine whether a dynamic mask is made at the camera stage
or not.

This causes problems if say a dynamic mask was not made but the standard recipe says that it was. "	bills
Milestone 	1415	protect dvo_catalog_save against HUP signal	DVO		defect	eugene	new	2010-08-10T16:36:01-10:00	2010-08-10T16:36:01-10:00	"we need to avoid generating inconsistent catalog files if a program is killed while writing a dvo catalog.  Add SetProtect entries around dvo_catalog_save entries.  see dvomerge/src/dvomergeUpdate.c
"	eugene
Milestone 	1416	Columns from dvo imextract/mextract not set	misc		defect	eugene	new	2010-08-16T21:29:51-10:00	2010-08-16T21:29:51-10:00	"MCAL dMCAL detlimit FWHM (from imextract) are all 0.

also ra:err and Dec:err (from mextract) seem to be 0.

"	morganson
Milestone 	1417	-set_state goto_cleaned should only be allowed for runs in states 'full' or 'update'	ippTools		defect	eugene	new	2010-08-17T09:57:59-10:00	2010-08-17T09:57:59-10:00	"The cleanup update process will not work properly for runs that have not reached full state. Components that have not finished will not be able to update. 

We should change the various tools to forbid this state transition. goto_purged should be used to cleanup runs in this state."	bills
Milestone 	1418	cleanup should be delayed until after components have completed destreaking	ippScripts		defect	eugene	new	2010-08-17T09:59:16-10:00	2010-08-17T09:59:16-10:00		bills
Milestone 	1420	detectability row drops	pstamp		defect	watersc1	new	2010-08-30T00:38:00-10:00	2010-08-30T00:38:00-10:00	"The detectability server seems to drop some rows that have valid images and components found by the pstamp code.  These rows do not have the results added to their data hash, resulting in nulls that prevent the request from finishing.

- determine why these rows are being dropped and fix (test query file: /data/ippdb02.0/pstamp/work/20100826/10367/morganson.20100826165800.fits ROWNUM 529"	watersc1
Milestone 	1421	ppSub repeatedly gets fault for for diff_id 74827 skycell.039	ppSub	current	defect	eugene	new	2010-09-02T13:56:48-10:00	2010-09-04T09:12:21-10:00	I set fault = 0 and quality = 4 to allow the run to complete.	bills
Milestone 	1422	ppSub assertion failure diff_id 74856 skycell.2150.108	ppSub	current	defect	eugene	new	2010-09-02T14:14:21-10:00	2010-09-07T09:47:18-10:00	": 411, Imin: 3516, obj x,y: 6195.000000, 5642.000000  src x,y: 6197.311035, 5647.168945, SN: 772.544983, ID: 6582
????????????????????????????????????Assertion failed in function psThreadLauncher at psThread.c:244. Error stack:
Unable to perform ppSub: 4 at /home/panstarrs/ipp/psconfig//ipp-20100823.lin64/bin/diff_skycell.pl line 400.
Running [/home/panstarrs/ipp/psconfig/ipp-20100823.lin64/bin/difftool -diff_id 74856 -skycell_id skycell.2150.108 -fault 4 -adddiffskyfile -dtime_script 81.0000032186508 -hostname ippc14 -path_base neb://ipp052.0/gpc1/ThreePi.nightlyscience/2010/08/30/RINGS.V0/skycell.2150.108/RINGS.V0.skycell.2150.108.dif.74856 -dbname gpc1]...
ARRAY(0x1cf9ee0) within metadata is not of type STR.
 at /home/panstarrs/ipp/psconfig//ipp-20100823.lin64/bin/diff_skycell.pl line 76
Running [/home/panstarrs/ipp/psconfig/ipp-20100823.lin64/bin/difftool -inputskyfile -diff_id 74856 -skycell_id skycell.2150.108 -dbname gpc1]...
diffInputSkyfile MULTI

"	bills
Milestone 	1423	ppSub repeatedly gets fault for for diff_id 74716 skycell.2218.058	ppSub	current	defect	eugene	new	2010-09-02T14:16:17-10:00	2010-09-02T14:16:17-10:00	"???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????Unable to determine detection efficiencies from fake sources
 -> psphotGuessModelsReadout (psphotGuessModels.c:116): unknown psLib error
     Unable to guess model.
 -> psphotGuessModels (psphotGuessModels.c:25): Problem in configure files
     failed on to guess models for PSPHOT.INPUT entry 0
 -> psphotMagnitudesReadout (psphotMagnitudes.c:130): unknown psLib error
     Unable to guess model.
 -> psphotMagnitudes (psphotMagnitudes.c:39): Problem in configure files
     failed to measure magnitudes for PSPHOT.INPUT entry 0
 -> effLimit (psphotEfficiency.c:54): unexpected NULL found
     Unable to generate PSF model.
 -> psphotEfficiencyReadout (psphotEfficiency.c:272): unknown psLib error
     Unable to determine limits for image
 -> psphotEfficiency (psphotEfficiency.c:177): Problem in configure files
     failed to measure detection efficiency for PSPHOT.INPUT entry 0
Unable to perform ppSub: 4 at /home/panstarrs/ipp/psconfig//ipp-20100823.lin64/bin/diff_skycell.pl line 400.
"	bills
Milestone 	1424	Example DQtest.txt with detect_query_create doesn't work	misc		defect	eugene	new	2010-09-07T11:47:51-10:00	2010-09-07T11:47:51-10:00	"The example DQtest.txt file doesn't work with detect_query_create SVN r 29072

http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/sample_dquery_request

====
detect_query_create --input DQtest.txt --output DQtest.r
Number of header columns in input does not equal expected number of header words at /Volumes/data/PS1/code/ipp-2.9/mwv.darwin_x86/bin/detect_query_create line 322, <$in> line 2.
====

This seems to have something to do with the way keywords are processed.  I tried to play around a little with blank lines vs. commented lines, but made no real progress.

"	Michael Wood-Vasey
Milestone 	1425	difftool -listrun does not list stack stack diffs	ippTools		defect	eugene	new	2010-09-09T11:54:21-10:00	2010-09-09T11:54:21-10:00	It JOINs down to the rawExp table using warp1 and warp2, but ignores the possiblility of stacks.  This will prevent searches for stack-stack diffs from returning any entries.	watersc1
Milestone 	1427	ppStack fails if No fake sources are suitable for PSF fitting	ppStack		defect	eugene	assigned	2010-09-17T11:19:57-10:00	2011-01-11T09:59:03-10:00	"{{{
StackRun 155123 fails repeatedly with

    PSF 0 is completely bad --- not including in envelope calculation.
    PSF 1 is completely bad --- not including in envelope calculation.
    PSF 2 is completely bad --- not including in envelope calculation.
    PSF 3 is completely bad --- not including in envelope calculation.
    Data error code: 32cf
Unable to perform stack.
 -> pmPSFEnvelope (pmPSFEnvelope.c:327): unknown psLib error
     No fake sources are suitable for PSF fitting.
 -> ppStackPSF (ppStackPSF.c:45): Problem determining PSF
     Unable to determine output PSF.
 -> ppStackPrepare (ppStackPrepare.c:274): Problem determining PSF
     Unable to determine output PSF.
 -> ppStackLoop (ppStackLoop.c:72): Problem determining PSF
     Unable to prepare for stacking.


When pmPSFEnvelope returns to ppStack with a failure ppStack faults.
It would be straightforward to change this to flag the run as bad quality instead if this is the right thing to do.

There are 4 stack inputs were warped chips in the corner of the focal plane otas 01 10 20 and 11

We have tried to stack this skycell MD02 skycell.020 11 times since MD02 observations resumed August 31.

It succeeded 3 times with good quality.


mysql> select stack_id, data_group,quality, fault from stackRun join stackSumSkyfile using(stack_id) where tess_id = 'MD02' and skycell_id ='skycell.020' and stack_id > 101000;
+----------+---------------+---------+-------+
| stack_id | data_group    | quality | fault |
+----------+---------------+---------+-------+
|   146491 | MD02.20100831 |    3006 |     0 |
|   147637 | MD02.20100902 |   13008 |     0 |
|   148177 | MD02.20100903 |    3006 |     0 |
|   149326 | MD02.20100905 |    3006 |     0 |
|   150481 | MD02.20100907 |       0 |     0 |
|   151255 | MD02.20100908 |    3006 |     0 |
|   152256 | MD02.20100910 |   13008 |     0 |
|   152335 | MD02.20100910 |       0 |     0 |
|   153896 | MD02.20100913 |   13008 |     0 |
|   154667 | MD02.20100914 |       0 |     0 |
|   155123 | MD02.20100915 |       0 |     5 |
+----------+---------------+---------+-------+
11 rows in set (0.00 sec)


The quality values are
13008 PPSTACK_ERR_REJECTED
 3006 PSPHOT_ERR_PSF

The Data error code in the message 32cf below is 13007 which is PPSTACK_ERR_PSF

Here are the stats from the warps for two stacks. 154667 succeeded, 155123 is our failure case.
The good_frac values are quite low.
The camera stage fwhm values are 4.25 - 4.67 for 155123


mysql> select stack_id, warp_id, warpSkyfile.good_frac, warpSkyfile.maskfrac_static, warpSkyfile.maskfrac_dynamic,warpSkyfile.maskfrac_advisory from stackRun join stackInputSkyfile using(stack_id) join warpSkyfile using(warp_id, skycell_id,tess_id) where tess_id = 'MD02' and skycell_id = 'skycell.020' and stack_id = 155123 or stack_id = 154667;
+----------+---------+-----------+-----------------+------------------+-------------------+
| stack_id | warp_id | good_frac | maskfrac_static | maskfrac_dynamic | maskfrac_advisory |
+----------+---------+-----------+-----------------+------------------+-------------------+
|   154667 |  104602 |  0.106212 |         0.19026 |          1.6e-05 |          0.001143 |
|   154667 |  104603 |  0.072509 |        0.122273 |            3e-06 |          0.000314 |
|   154667 |  104605 |  0.122376 |        0.362446 |            6e-06 |          0.002877 |
|   154667 |  104607 |   0.07516 |        0.129025 |            6e-06 |          0.000944 |
|   155123 |  105207 |  0.122781 |        0.330894 |          1.1e-05 |          0.003137 |
|   155123 |  105350 |  0.099842 |         0.31159 |          1.5e-05 |          0.001292 |
|   155123 |  105351 |  0.058158 |         0.08223 |          1.2e-05 |          0.001321 |
|   155123 |  105352 |  0.094191 |        0.209738 |            9e-06 |          0.000883 |
+----------+---------+-----------+-----------------+------------------+-------------------+
8 rows in set (0.00 sec) 
}}}
"	bills
Milestone 	1429	perl mdc parser will not accept hexadecimal value for inteter	ippScripts		defect	eugene	new	2010-10-15T11:39:29-10:00	2010-10-15T11:39:29-10:00	"I tried to add the following to gpc1/camera.config

MASK.NO.CENSOR      U32      0x5280

ipp_apply_burntool.pl failed with 

     ERROR (line 165):  of metadata file : could not parse beyond this point

Using a decimal value works fine.

This bug is in the paser not ippScripts but I didn't see a component that matches that.
"	bills
Milestone 	1431	cell saturation concept averaging needs a better sanity check	psModules		defect	eugene	new	2010-10-29T10:55:13-10:00	2010-10-29T10:55:13-10:00	"cell saturation concept averaging now ignores reported values below 10k.  this is completely instrument specific.  also, the averaging should somehow take into account the fraction of unmasked pixels.
 "	eugene
Milestone 	1434	invaild magicRun queued	ippScripts		defect	eugene	new	2010-11-01T11:22:56-10:00	2010-11-01T11:24:22-10:00	"Two magicRuns from ThreePi.20101030 failed to complete.
The problem is that the magicRuns were for the template exposure of a warp warp diff, yet magicRun.inverse == 0.

This causes magictool -exposure to return no data.

The only other cases in the database where a warp - warp diff run had 2 associated magicRuns both with inverse = 0 were testing runs or sweetspot data from 2010-09-15 and 16 which is around the time we first tried doing OSS runs.

I set the inverse bit by hand and the runs completed."	bills
Milestone 	1435	ppMerge memory footprint larger than expected	ppMerge		defect	eugene	new	2010-11-04T10:48:59-10:00	2010-11-04T10:48:59-10:00	"ppMerge seems to be taking too much memory.  For example, running a set of 100 exposures, GPC1, with 'ROWS' set to 128, it is taking ~10GB, which implies something like 1302 bytes per input pixel. 
"	eugene
Milestone 	1436	Ohana Shutdown segfaults	Ohana		defect	eugene	new	2010-11-10T17:35:31-10:00	2010-11-10T17:35:31-10:00	"Shutdown causes segfaults when there are no additional arguments called to Shutdown. This is fixed in relphot by doing the following:

Shutdown (""%s"",""halted by signal (trapped)"");

The other guys in Ohana need a similar fix.
"	heather
Milestone 	1437	publish run in fault-revert-fault endless loop	ippScripts		defect	eugene	new	2010-11-22T11:18:21-10:00	2010-11-22T11:18:21-10:00	"publishRun pub_id = 179098 got into a bad state where it couldn't complete so it got into a fault/revert/fault endless loop.

The problem was that the first time the script ran it successfully
registered the results to the data store, but the script
ran into a problem updating the database.

I set the state to 'full' to stop the endless loop.
Perhaps the file set should be deleted in this situation.
It's tricky though because once the fileset is on the data store
clients will see it and deleting it out from under them is
bad form."	bills
Milestone 	1440	stack runs fail 'Unable to calculate transparency differences'	ppStack		defect	eugene	new	2010-11-26T11:16:12-10:00	2010-11-26T11:16:12-10:00	"stack_id 189072 and 190958 yielded fault 5.
The message in the log was

   Unable to calculate transparency differences
 ->  ppStackLoop (ppStackLoop.c:72): Problem in data values
     Unable to prepare for stacking.

These runs have been set to 'drop'
"	bills
Milestone 	1441	science processing should warn or hinder processing of lossy-compressed data	ippTools		defect	eugene	new	2010-11-28T09:10:18-10:00	2010-11-28T09:10:18-10:00	"we should not allow processing of lossy-compressed data without special overrides.  this could be done by having chiptool require state of full unless otherwise specified.
"	eugene
Milestone 	1442	detrend processing should not use lossy-compressed data unless explicitly chosen	ippTools		defect	eugene	new	2010-11-28T09:11:18-10:00	2010-11-28T09:11:18-10:00	this could be accomplished by requiring data_state of full unless the user overrides.	eugene
Milestone 	1443	publication should store files in nebulous not in datastore/dsroot	misc		defect	eugene	new	2010-11-29T10:55:39-10:00	2010-11-29T10:55:39-10:00	"Currently the publications system copies files into the data store directory. It should instead store the files in nebulous and use the dreg option to link back to files in nebulous.
(When it does this I suggest being clever about the file names so you don't end up with ugly filenames)"	bills
Milestone 	1444	PS::IPP::Config.pm gives an error for non-neb files (line 1109)	ippScripts		defect	eugene	new	2010-12-28T12:53:15-10:00	2010-12-28T12:53:15-10:00	not sure of the cause, but the error appears when running chip_imfile.pl with an output that is not neb:  the error complains about the logic on that line around the 'ne'.	eugene
Milestone 	1445	"Assertion failure: ""fix this code: z1 should not be nan for 0.016674"" from ppStack"	psphot		defect	bills	assigned	2011-01-11T10:00:52-10:00	2011-08-05T10:34:28-10:00	"stack_id 215251 fails with
    Filtered out 81 of 964 sources
    fix this code: z1 should not be nan for 0.016674
    Backtrace depth: 7
    Backtrace 0: p_psAssert
    Backtrace 1: pmModelRadius_PS1_V1
    Backtrace 2: (unknown)
    Backtrace 3: (unknown)
    Backtrace 4: psThreadLauncher
    Backtrace 5: (unknown)
    Backtrace 6: clone
Unable to perform ppStack: 4 at /home/panstarrs/ipp/psconfig//ipp-20101215.lin64/bin/stack_skycell.pl line 354
"	bills
Milestone 	1446	destreak revert fails for camera runs	magic		defect	bills	new	2011-01-12T11:01:41-10:00	2011-01-12T11:13:39-10:00	"Since censorObjects doesn't set the destreaked keyword the script
is unable to determine which file to delete.

Workaround is to neb-rm the files with SR_ in the filename."	bills
Milestone 	1447	PSDESTRK keyword is not set for warp images generated from destreaked chips	magic		defect	bills	new	2011-01-13T13:46:47-10:00	2011-01-13T13:46:47-10:00	"streaksremove adds a card to the header of destreaked imgaes
PSDESTRK=                    T / Have streaks been removed from image?         

When warps are made from destreaked chips, this keyword is not copied to the header of the output warp.

This makes it difficult to determine if such an image has been destreaked or not directly from the contents of the file. This is primarily an issue for postage stamp images.

diff images created from destreaked images have the same problem."	bills
Milestone 	1448	assertion failure from ppSub	ppSub		defect	eugene	new	2011-01-19T08:52:29-10:00	2011-04-14T06:21:50-10:00	"diff_id 102558 skycell_id skycel.1260.089 fails repeatedly with what looks like an assertion failure. The stack trace is garbled.

    Backtrace of waiter:
    Backtrace depth: 11
    Backtrace 0: psThreadPoolWait
    Backtrace 1: psImageSmoothMask_Threaded
    Backtrace 2: psphotSignificanceImage
    Backtrace 3: psphotFindDetectionsReadout
    Backtrace 4: psphotFindDetections
    Backtrace 5: psphotReadoutMinimal
    Backtrace 6: (unknown)
    Backtrace 7: (unknown)
    Backtrace 8: (unknown)
    Backtrace 9: __libc_start_main
    Backtrace 10: (unknown)
    Couldn't find thread task PSLIB_IMAGE_SMOOTHMASK_SCANROWS
    Backtrace of waiter:
    Backtrace depth: 4
    Backtrace 0: p_psAssert
    Backtrace 1: psThreadLauncher
    Backtrace 2: (unknown)
    Backtrace depth: 11
    Backtrace 3: clone
    Backtrace 0: psThreadPoolWait
    Backtrace 1: psImageSmoothMask_Threaded
    Backtrace 2: psphotSignificanceImage
    Backtrace 3: psphotFindDetectionsReadout
    Backtrace 4: psphotFindDetections
    Backtrace 5: psphotReadoutMinimal
    Backtrace 6: (unknown)
    Backtrace 7: (unknown)
    Backtrace 8: (unknown)
    Backtrace 9: __libc_start_mainUnable to perform ppSub: 4 at /home/panstarrs/ipp/psconfig//ipp-20101215.lin64/bin/diff_skycell.pl line 400.
"	bills
Milestone 	1449	magictool -definebyquery options are inconsistent with other tools	ippTools		defect	bills	new	2011-01-20T14:26:18-10:00	2011-01-20T14:26:18-10:00	"magictool -definebyquery has some quirks that should be addressed

Change -label option to -set_label (Acutally change all of the
options that are setting values to be -set_

And selection arguments should be required.


"	bills
Milestone 	1450	psFitsRead / psFitsDumpErrors inconsistency	PSLib		defect	eugene	new	2011-01-22T11:05:53-10:00	2011-01-22T11:05:53-10:00	"Does psFitsDumpErrors work?  I hit a case were I had a fits read error (due to a misunderstanding of the default region), but calling psFitsDumpErrors told me nothing at all.

I wrote a program to read a simple image, with the region set to -1,-1,-1,-1 (instead of the correct default 0,0,0,0).

a call to psFitsReadImage returned NULL, but a subsequent call to psFitsDumpErrors gave no information at all.

"	eugene
Milestone 	1451	"ppStack finds very large ""Target seeing"""	ppStack		defect	bills	new	2011-01-27T15:31:06-10:00	2011-01-27T15:31:06-10:00	"stackRun 215533 fails to match any images to the target PSF.
The log file says

    Target seeing FWHM: 30.364965

ppStack measures the range of input FWHM as 4.6 - 7.8 

This is interesting because the range of fwhm for the associated camera runs is 4.0 -  5.2

"	bills
Milestone 	1452	ns.stacks.run attempts to re-queue stacks	ippScripts	current	defect	waterssc1	new	2011-01-28T16:39:08-10:00	2011-01-28T16:55:56-10:00	"We restarted stdscience this afternoon. 2011-01-28 after 2pm HST.
The input file runs ns.add.date for the past 3 days.

In this state ns.stacks.run attempted to queue stacks for last night that had already been run. 

stacktool fails on this as it should, but it core dumped.
(That's a bug for another ticket)

I worked around the problem by executing 

book setword nsStacks 2011-01-28 nsStackState FINISHED_STACKS
"	bills
Milestone 	1453	postage stamp parser does not detect DB connection problems	pstamp		defect	bills	assigned	2011-02-02T16:40:50-10:00	2011-02-02T16:40:59-10:00	"If one of the ippTools commands issued by the parser cannot
connect to the database, the error is un detected by the parser.

The result is PSTAMP_NO_IMAGE_MATCH errors for the affected rows."	bills
Milestone 	1454	distribution only works for GPC1 camera	misc		defect	bills	new	2011-02-03T15:41:24-10:00	2011-02-03T15:41:24-10:00	"There is a hard coded

$CAMERA = GPC1

in the task dist.advance.run. disttool -toadvance doesn't join back to rawExp so the camera is not available at this time."	bills
Milestone 	1455	magictool -reverttree only uses last label	ippTools		defect	bills	new	2011-02-05T11:06:54-10:00	2011-02-05T11:06:54-10:00	fix it	bills
Milestone 	1458	stack summary failures	ippTools		defect	watersc1@…	new	2011-02-22T15:08:01-10:00	2011-02-22T15:08:01-10:00	"the stack summary analysis is failing to make the correct associations.  for an example, see the stacks with labels of the form MD04.pspstest.2011021. This set of stacks (all with the same label, but 5 different filters) resulted in 4 stack summary entries.  Of those, 3 generated jpegs (b1 & b2 versions) of a single skycell.  The 4th generated a jpeg of the full set of skycells, but at least some of the skycells are from the wrong filter (some of the very ugly ones are from y-band, while the stack claims to be z-band).  Eg, skycell.041 is missing in z-band (at the moment), but is included in the image from the y-band version (you can tell by the bad glint).

Finally, the ippMonitor image links are broken.
"	eugene
Milestone 	1459	psphot unhappy with chip_id 196207 XY47	psphot		defect	eugene	new	2011-02-23T11:18:40-10:00	2011-02-23T11:18:40-10:00	"psphot went a little mental for this chip. It looks like a bright star in the previous image raised havoc. 
It looks like psphot was trying to analyze a large number of extended sources and eventually went used so much memory the process went into swap heck 
After 105 minutes of processing we decided to put it out of it's misery with the following operations.

kill 19851

update chipProcessedImfile set fault = 0, quality = 42 where chip_id = 196207 and class_id = 'xy47';


chiptool -updaterun -set_note 'chip 
xy47 never completed photometry' -chip_id 196207
"	bills
Milestone 	1460	funpack doesn't unpack fpacked data	misc		defect	watersc1	new	2011-02-23T16:10:48-10:00	2011-02-23T16:10:48-10:00	"fpack -h -s 8
produces a file that
funpack 
can't handle.  This causes problems for historical burntool processing"	watersc1
Milestone 	1461	FPA.EXPOSURE is derived from EXPREQ, not EXPTIME	ippconfig		defect	eugene	new	2011-03-03T11:16:06-10:00	2011-03-03T11:16:06-10:00	"FPA.EXPOSURE is derived from EXPREQ, not EXPTIME

is this a mistake?  we probably want the actual time regardless of the requested time...
"	eugene
Milestone 	1462	summit copy issues / pzgetimfiles should fault on error	ippTools		defect	eugene	new	2011-03-06T11:35:08-10:00	2011-03-06T11:35:08-10:00	"short story:  pzgetimfiles does not always set a fault in the db when it fails.

longer story: 

I've discovered the proximate cause for the recent crashes of the
summit copy pantasks.  It seems that there is a single summit exposure
(o5602g0593d) for which the file index is buggered up.  Instead of a
clean listing of files, we have a large number of camera communication
messages at the top of the index:


# fileID          |bytes   |md5sum                          |type|chipname|
= Observing Log =

== Observers ==
Tommy and Bill

== Starting Conditions ==
==Starting Conditions==
*Ugly: humidity 82%, wind speed 9.6 m/s, ground fog blowing up out of the crater
*Otisgui restarted; telescope unpinned; Estop cleared
*Tried to start Az. drive ex. script at 5:45 anyway but got error
message: ""Failed to get Telescope status""
*Software updated; exercise script started at 6:08
*Conditions clearing; restarted az exercise script with 15 sec pause
between moves
*6:45 opened vents; az drive 30 deg

== Tonight's Observing Plan ==
(et cetera)

Further down in the file, there is an actual list corresponding to a
fileset.  But it, too, is buggered up:

c4741g0016d56.fits|49423680|7e22d2cf9d1c722721a7868ca88c41da|chip|ota56   |
o5410g0021o35.fits|49432320|48a67d5d416b6dcfdb1552c3417f583f|chip|ota35   |
o5602g0593d01.fits|49429440|e4ca8b9284bba1d83b79b28a3a2cb765|chip|ota01   |
o5602g0593d02.fits|49429440|dd1ce4d2780d8ab50bdf5c0f0dc88f0d|chip|ota02   |
o5602g0593d03.fits|49429440|ea85652366d88fe5c1a1a900164596e8|chip|ota03   |
o5602g0593d04.fits|49432320|f5ad936554987638b6e00a6c7df0bd27|chip|ota04   |

In addition to the correct (and complete) set of files, the listing
includes 2 files from earlier exposures (much earlier -- like 6 months
and 2.4 years old!).

When pzgetimfiles is run on this entry, it spews back a bunch of
complaints to stderr about the lines that it does not recognize.  It
then attempts to parse the lines it does recognize but, as a result of
the extra files, it raises an error and exits.  But, in the particular
error case, it does NOT set a fault on the summit exposure.  Thus, it
keeps trying to download this same index every so often -- 9242 times
since this problem file appeared  on Feb 10.  We know that
pantasks_server or pcontrol will eventually get unhappy and crash if
there is too much stderr and/or stdout traffic which it must parse, so
this eventually kills it (which ever one it is that is crashing).

So, a summary of the bugs that are revealed by this situation:

1) broken summit datastore index file :
http://conductor.ifa.hawaii.edu/ds/gpc1/o5602g0593d/index.txt
2) pzgetimfiles does not raise an error on any / many parsing errors
in an index [note 1]
3) pzgetimfiles does not validate the filename against the exposure
name [note 2]
4) pzgetimfiles does not always fault the summitExp table on failures
5) pantasks_server and/or pcontrol should not crash under excess
stdout / stderr communications

note 1 : it is not clear that pzgetimfiles *should* raise an error if
it fails on any lines.  that level of rigidity could be good (would
detect problems) or it could be bad (would cause things to fail in
potentially recoverable cases).
note 2 : it is not clear that pzgetimfiles should attempt this file
name validation : this forces pzgetimfiles to have knowledge of the
relationship between file names which would restrict its range of
usability

I have manually faulted the bad exposure.  it is a dark, so it is
acceptable that the download fail forever.

aloha
gene

"	eugene
Milestone 	1463	RA and Dec empty in psphotForced (PS1_DV2)	psphot	current	defect	eugene	accepted	2011-03-09T16:57:31-10:00	2011-03-14T08:14:41-10:00	"The RA and Dec columns are empty in psphotForced-generated PS1_DV2 files.  I've run psphot on an image and RA and Dec are populated in the PS1_V3 file, but they are not populated in the psphotForced PS1_DV2 file.

This has been reported by Ken Smith over email.

Specifically I'm running in Gene's branch

psphot version: branches/eam_branches/ipp-20101205/psphot@30492         
psphot source: 60eb6cdc-a59c-4636-a4e0-dba66a9721fd                     

but Ken's report would have been from the production branch."	Michael Wood-Vasey
Milestone 	1468	exposure got stuck in registration	ippScripts		defect	eugene	new	2011-03-16T10:07:11-10:00	2011-03-16T10:57:49-10:00	"exp_id 310765 never finished registration but no entry in rawExp got inserted into the database.

We looked into this and apparently the registration faulted. See below.

However newExp.state got set to 'stop' so the exposure was stuck.
Changing the state to 'run' allowed the exposure to complete.

Starting script /home/panstarrs/ipp/psconfig//ipp-20110218.lin64/bin/register_exp.pl on ipp030 at Tue Mar 15 15:27:20 HST 2011



exp_id           S64       310765
tmp_exp_name     STR       o5635g0509o
tmp_camera       STR       gpc1
tmp_telescope    STR       ps1
state            STR       run
workdir          STR       neb://@HOST@.0/gpc1/20110315
workdir_state    STR       dirty
reduction        STR       NULL
dvodb            STR       NULL
tess_id          STR       NULL
end_stage        STR       reg
label            STR       NULL
camera           STR       GPC1
filter           STR       i.00000
obs_mode         STR       3PI
obs_group        STR       NULL
epoch            TAI       2011-03-15T07:31:23.000000
dateobs          TAI       2011-03-15T12:40:17.000000
*** stderr ***
Unable to stat Nebulous handle neb://any/gpc1/20110315/o5635g0509o.310765/o5635g0509o.310765.reg.log at /home/panstarrs/ipp/psconfig//ipp-20110218.lin64/bin/register_exp.pl line 56
cannot resolve neb://any/gpc1/20110315/o5635g0509o.310765/o5635g0509o.310765.reg.log at /home/panstarrs/ipp/psconfig//ipp-20110218.lin64/bin/register_exp.pl line 56
Unable to redirect output at /home/panstarrs/ipp/psconfig//ipp-20110218.lin64/bin/register_exp.pl line 347
        main::my_die('Unable to redirect output', 310765, 2) called at /home/panstarrs/ipp/psconfig//ipp-20110218.lin64/bin/register_exp.pl line 56

system failure for: register_exp.pl --exp_id 310765 --exp_tag o5635g0509o.310765 --logfile neb://any/gpc1/20110315/o5635g0509o.310765/o5635g0509o.310765.reg.log     --dbname gpc1 --verbose
job exit status: 2
job host: ipp030
job dtime: 180.85355
job exit date: Tue Mar 15 15:30:22 2011
*** stdout ***
*** stderr ***
"	bills
Milestone 	1474	What is CAL_PSF_MAG_SIG supposed to mean?	psphot		defect	magnier	new	2011-04-05T16:20:37-10:00	2011-04-05T16:20:37-10:00	"I've been assuming that CAL_PSF_MAG_SIG is supposed to be the uncertainty in the total measurement of PSF_MAG, i.e. the equivalent of PSF_INST_MAG_SIG (most likely should be the same as PSF_INST_MAG_SIG).  But I finally looked at the code and I see that it's defined as the uncertainty in the _zeropoint_calibration_, not the uncertainty in the measurement.

        psMetadataAdd (row, PS_LIST_TAIL, ""CAL_PSF_MAG_SIG"",  PS_DATA_F32, ""measured scatter of zero point calibration"", zeroptErr);

I would bet that I am not alone in my confusion.  If CAL_PSF_MAG_SIG is supposed to be the zeropoint calibration uncertainty, can we rename it to something else, e.g., CAL_PSF_ZPT_SIG, to be less confusing?

If it is supposed to just be the PSF_INST_MAG_SIG then it could easily just be set to that, but I appreciate that one would like to avoid redundancy in files."	Michael Wood-Vasey
Milestone 	1475	ippmonitor prints use_begin, but detselect uses time_begin	ippMonitor		defect	eugene	new	2011-04-06T15:39:11-10:00	2011-04-06T15:39:11-10:00	detrends have a use_begin/end date listed on ippmonitor, but this is not time_begin/end, which is the field that detselect actually uses.  use_begin/end should be removed from ippmonitor and replaced with time_begin/end.	watersc1
Milestone 	1476	perl metadata paser fails on vectors unless the line ends with a comment	ippScripts		defect	eugene	new	2011-04-08T15:41:39-10:00	2011-04-08T15:42:15-10:00	"The perl metadata config file parser generates errors on vectors. That is lines of the form

@RADIAL.ANNULAR.BINS.LOWER           F32      0.0 1.2 3.4 5.2  8.8 15.0 23.2 37.2 57.1  91.0 141.0 

Unless the line ends with a comment # this is a comment
"	bills
Milestone 	1477	registration has trouble if dateobs and registered disagree	ippTools		defect	watersc1	new	2011-04-09T02:08:39-10:00	2011-04-09T02:08:39-10:00	"registration became backed up on 2011-04-09 because dateobs fell before the registration cutoff and ""registered"" fell after.  This meant that half the system was waiting for the exposure to be burntooled, and the other half considered it unnecessary to burntool.  This is apparently a very rare condition, but still needs to be fixed.  Given that we now have the --continue option, perhaps it's worthwhile to let everything from UTC midnight onward be burntooled."	watersc1
Milestone 	1478	warp updates get stuck if dependent chip has fault 26	pstamp		defect	bills	new	2011-04-09T07:48:54-10:00	2011-04-09T07:48:54-10:00	"pstampcheckdependent.pl does not handle the case where a warp stage dependent has an input chip with fault == 26 (PSTAMP_GONE)

chiptool attempts to set the chip to be updated, but silently does not change the state if it is faulted with value 26.

pstampcheckdependent should set warpSkyfile.fault = 26"	bills
Milestone 	1479	assertion failure from ppImage	psphot		defect	eugene	new	2011-04-16T08:15:22-10:00	2011-04-16T08:15:22-10:00	"chip_id 214041 XY17 repeatedly faulted in psBlendFitReadout.

The chip was set to quality = 42, fault 0 in order to allow the
processing of the exposure to succeed
"	bills
Milestone 	1480	All sources from diffRun 122547 have negative fluxes	ppSub		defect	eugene	new	2011-04-19T14:39:22-10:00	2011-04-19T14:39:22-10:00	"Both publishRuns for the diffRun 122547 fail with fault 2.
All of the sources in the inputs have negative PSF_INST_FLUX

The two pub_ids are 207018 and 207049.

The root cause appears to be that one of the exposures in the pairwise diff had good image quality fwhm_major 3.87 and the
other had poor image quality 11.0182.

ppSub should probably do a better job handling this.

Note: ppMops is a bit quiet about errors. The log simply says:
Unable to translate at /home/panstarrs/ipp/psconfig//ipp-20110406.lin64/bin/publish_file.pl line 374

I was able to get a clue as to the problem by using a debug build and adding

-trace ppMops 10

Which yields output like:

          Discarding row 342 from input 25 because of non-finite values: 1694.780884 1478.750122 4.622112 -0.020662 nan inf inf inf 0.200000 -45.000000

"	bills
Milestone 	1481	assertion failures from ppSub	ppSub	current	defect	eugene	new	2011-04-23T07:56:25-10:00	2011-04-23T07:56:25-10:00	"On 2011-04-23 we got two repeating faults from ppSub. The two stack traces and the ids are given below. The runs were completed by setting quality to 42 and fault to zero. 

{{{
(warning raised 72300 times)
       Kron Radii vs FWHMs 1: fwhm: 4.801629, kron 99.767906
       Kron Radii vs FWHMs 2: fwhm: 18.929829, kron nan
       Windows from Kron Radii: 274.361755 for 1, nan for 2
   Data error code: 36b6
Error in subtraction:
-> pmSubtractionStampsFind (pmSubtractionStamps.c:462): parameter is out-of-range
    Error: size is 0 or less.
-> pmSubtractionStampsSelect (pmSubtractionStamps.c:402): parameter is out-of-range
    Unable to find stamps.
-> ppSubMatchPSFs (ppSubMatchPSFs.c:446): Problem in data values
    Unable to match images.
-> ppSubLoop (ppSubLoop.c:110): Problem in data values
    Unable to match PSFs.
found 0 leaks at ppSub
Unable to perform ppSub: 5 at /home/panstarrs/ipp/psconfig//ipp-20110406.lin64/bin/diff_skycell.pl line 463.
Running [/home/panstarrs/ipp/psconfig/ipp-20110406.lin64/bin/difftool -diff_id 123766 -skycell_id skycell.077 -fault 5 -adddiffskyfile -dtime_script 32.9999819397926 -hostname ipp049 -path_base neb://ipp006.0/gpc1/MD06.nightlyscience/2011/04/23/MD06/skycell.077/MD06.skycell.077.WS.dif.123766 -dbname gpc1]...





  No valid data in input vector.
   (warning raised 32700 times)
       Kron Radii vs FWHMs 1: fwhm: 2.924699, kron nan
       Kron Radii vs FWHMs 2: fwhm: 13.610703, kron 49.518742
       Windows from Kron Radii: nan for 1, 136.176544 for 2
     Found 2 stamps
     Stamp 334 (5735,4137) is within the region border.
     Stamp 418 (5730,4782) is within the region border.
failed to generate stats
Backtrace depth: 8
Backtrace 0: p_psAbort
Backtrace 1: pmSubtractionStampsGetWindow
Backtrace 2: pmSubtractionMatch
Backtrace 3: (unknown)
Backtrace 4: (unknown)
Backtrace 5: (unknown)
Backtrace 6: __libc_start_main
Backtrace 7: (unknown)
ts.
Unable to perform ppSub: 4 at /home/panstarrs/ipp/psconfig//ipp-20110406.lin64/bin/diff_skycell.pl line 463.
Running [/home/panstarrs/ipp/psconfig/ipp-20110406.lin64/bin/difftool -diff_id 123629 -skycell_id skycell.1.08 -fault 4 -adddiffskyfile -dtime_script 50.0000104308128 -hostname ippc26 -path_base neb://ipp013.0/gpc1/STS.2010.b/2011/04/22/STS.V3/skycell.1.08/STS.V3.skycell.1.08.WS.dif.123629 -dbname gpc1]...

}}}
"	bills
Milestone 	1485	Average Magnitude Error Wrong for Dual PSF Fit	misc		defect	eugene	new	2011-05-12T04:23:27-10:00	2011-05-12T04:23:27-10:00	"
This error was observed in the most recent dvo database release.

We have a source at 
215.151440773 -16.0417386404

It's z magnitude measurements are:

# z z:err flag0x extern_id 
19.223207 0.053754 2204000 6446648
19.159416 0.043235 2200000 6447248
19.238922 0.035382 6200005 10769549
19.347965 0.048594 2200000 10770809
21.407980 0.230573 2004024 11342113
20.807688 0.133663 6004024 11342113
19.291172 0.035008 6200045 11340493

But the average z z:err:
19.2955150604 0.984078884125

That error is suddenly huge and wrong. In one exposure the source was fit as a pair with a double psf. But the flux does not add up to the flux in all other exposures. I assume that the z error is a standard deviation derived quantity rather than the error in the weighted mean (which should always be smaller than the individual errors). But the magnitudes from the exposure in which the source was (incorrectly) identified as a pair should not be included in the average. 

If this happens often, it will be a big problem anytime someone makes mag:err cuts.
"	morganson
Milestone 	1488	assertion failure in psImageInterpolationKernel	pswarp		defect	eugene	new	2011-05-18T09:04:00-10:00	2011-05-18T09:04:00-10:00	"The following skycell faults repeatedly with the assertion failure listed below

-warp_id 199035 -skycell_id skycell.1885.062

    Unable to find header DATE-OBS --- assuming value is NULL
    Unable to find header UTC-OBS --- assuming value is NULL
    Unable to parse concept FPA.TIME, but concept not marked as required.
          warning: both CDi_j and PC00i00j defined in headers, using PC00i00j terms
      maximum error using this grid sampling: 0.005102
Pixel fraction is 1.000000
Backtrace depth: 6
Backtrace 0: p_psAssert
Backtrace 1: psImageInterpolationKernel
Backtrace 2: (unknown)
Backtrace 3: psThreadLauncher
Backtrace 4: (unknown)
Backtrace 5: clone"	bills
Milestone 	1489	ppStack can't find psf for stack_id 305438	ppStack		defect	eugene	new	2011-06-13T13:23:04-10:00	2011-06-13T13:23:04-10:00	"      all zero points are finite; using reported zero points listed above
    Data error code: 32ce
    No analysis metadata for PSF, clump parameters cannot be saved.
    Concept CHIP.XPARITY is specified by the DEFAULTS in the camera format, but the values don't match.
    Concept CHIP.YPARITY is specified by the DEFAULTS in the camera format, but the values don't match.
      Complete ppStack run: 17.743550 sec
arency differences


Inputs have been preserved with label Stack.305438.inputs"	bills
Milestone 	1491	neb-cull should preferentially remove zero byte files	Nebulous		defect	watersc1	new	2011-07-13T13:33:11-10:00	2011-07-13T13:33:11-10:00	Given that it seems the shuffle code occasionally creates a zero byte instance while leaving the original two copies alone, we should ensure that culling removes this empty instance before removing either of other copies.	watersc1
Milestone 	1492	Rare and interesting image features	ppImage		defect	eugene	new	2011-07-15T11:43:50-10:00	2011-07-15T11:43:50-10:00	"Exposure o5757g0359o (exp_id: 362442, chip_id: 256988) is repeatedly failing for class_id XY17

See log file:
http://ipp004.ifa.hawaii.edu/ippMonitor/chipProcessedImfile_failure.php?menu=ipp.science.dat&proj=gpc1&chipRun.chip_id=256988&chipProcessedImfile.class_id=XY17&camera=GPC1&basename=neb://ipp012.0/gpc1/MD09.nt/2011/07/15//o5757g0359o.362442/o5757g0359o.362442.ch.256988&class=XY17

/data/ipp012.0/nebulous/aa/10/1094656579.gpc1:MD09.nt:2011:07:15:o5757g0359o.362442:o5757g0359o.362442.ch.256988.XY17.log
"	Serge CHASTEL
Milestone 	1493	repeated assertion failure in ppImage	psphot		defect	eugene	new	2011-07-28T13:59:49-10:00	2011-07-28T14:00:04-10:00	"chip XY67 for chip_id 259539 and 259540

Backtrace depth: 12
Backtrace 0: p_psAssert
Backtrace 1: pmFootprintCullPeaks
Backtrace 2: psphotCullPeaks
Backtrace 3: psphotFindFootprints
Backtrace 4: psphotFindDetectionsReadout
Backtrace 5: psphotFindDetections
Backtrace 6: psphotReadout
Backtrace 7: (unknown)
Backtrace 8: (unknown)
Backtrace 9: (unknown)
Backtrace 10: __libc_start_main
Backtrace 11: (unknown)
"	bills
Milestone 	1494	assertion failure in ppSub	ppSub		defect	bills	new	2011-07-29T09:05:02-10:00	2011-07-29T09:05:02-10:00	"Unfortunately the message is useless

Assertion failed in function psThreadLauncher at psThread.c:244. Error stack:

-diff_id 148886 -skycell_id skycell.7.25"	bills
Milestone 	1497	Assertion failure from ppSub	ppSub		defect	eugene	new	2011-08-30T17:04:36-10:00	2011-08-30T17:04:36-10:00	"diff_id 158696 skycell: skycell.199.062


ppSub: psEllipse.c:171: psEllipseAxesToShape: Assertion `(sizeof (shape.sx) == sizeof (float) ? __finitef (shape.sx) : sizeof (shape.sx) == sizeof (double) ? __finite (shape.sx) : __finitel (shape.sx))' failed.


skycell quality was set to 42 to allow run to complete."	bills
Milestone 	1498	magicdstool can queue multiple destreak runs for the same object	ippTools		defect	eugene	new	2011-09-01T12:29:14-10:00	2011-09-01T12:29:14-10:00	"If multiple magicRuns get queued for the same exposure and label multiple destreak runs will be queued for each stage. For example
one magicDSRun will get queued for each magic_id and chip_id.

Since magicDSRuns overwrite files in their parent run this leads
to chaos."	bills
Milestone 	1499	warp with completely blank NAN input faults should give bad quality	pswarp		defect	eugene	new	2011-10-05T09:43:52-10:00	2011-10-05T09:43:52-10:00	"warp_id 272000 skycell.009 has only one input. That image appears to be all NAN pixels.

Instead of completing with bad quality warp_skycell.pl instead faults because no psf file is generated

Set quality to non-zero to allow the warpRun to complete."	bills
Milestone 	1501	ppSub fails for  config dump files for diff_id < 88268	ppSub		defect	eugene	new	2011-11-09T11:15:43-10:00	2011-11-09T11:15:43-10:00	There is an incompatibility between the config dump files for diffRuns before 88268 that prevents them from being used for updates. A workaround has been added to diff_skycell.pl	bills
Milestone 	1502	asssertion failure in pmSetFootprintArrayIDS	psphot		defect	eugene	new	2012-01-16T12:37:43-10:00	2012-01-16T12:37:43-10:00	"chip_id 383027 XY60 and XY76 pops an assertion. It turns out that
the exposure was taken with the dome closed so there are no stars.

      --- psphot Fit Source (Linear) ---
          covariance factor: 1.000000
          built fitSources: 0.000121 sec (3 objects)
    add noise for 3 objects: 0.000010 sec
      --- psphot Find Detections ---
          smooth image: 0.394455 sec
          smooth variance: 0.909428 sec
      built smoothed signficance image: 1.063004 sec
    64 peaks: 0.260327 sec
          found 64 footprints: 0.045811 sec
Aborting in function p_psImageAlloc at psImage.c:90. Error stack:
Specified number of rows (0) or columns (0) is invalid.
Backtrace depth: 13
Backtrace 0: p_psAbort
Backtrace 1: (unknown)
Backtrace 2: pmSetFootprintArrayIDs
Backtrace 3: pmFootprintArrayGrow
Backtrace 4: psphotFindFootprints
Backtrace 5: psphotFindDetectionsReadout
Backtrace 6: psphotFindDetections
Backtrace 7: psphotReadout
Backtrace 8: (unknown)
Backtrace 9: (unknown)
Backtrace 10: (unknown)
Backtrace 11: __libc_start_main
Backtrace 12: (unknown)

"	bills
Milestone 	1503	pztool update hardcodes information about exp_name in the SQL	ippTools		defect	watersc1	new	2012-02-24T12:07:47-10:00	2012-02-24T12:07:47-10:00	"I've made a change to pztool_pendingimfile.sql to attempt to prioritize the science exposures over the stare observations.  This looks like it will work in the future and preclude the need for someone to make any changes manually to get the science exposures processed.

However, it forces the decision to be hardcoded into the SQL which is less than optimal."	watersc1
Milestone 	1504	problems registering files into database when you don't summitcopy...	ippTools		defect	eugene	new	2012-04-26T11:46:14-10:00	2012-04-26T11:46:14-10:00	"When you use a database that does not have entries in summit%, for example ssp or megacam which are currently injecting files manually, register.pro fails to load exposures to register because of lack of entries in summitExp. 

Here's an example of how I encountered this:

I inject the files:

ipp_serial_inject.pl --telescope simple --camera simple --workdir neb://@HOST@.0/ssp/20110702 --end_stage reg --dbname ssp *fits 

When I then do a pantasks:
    module pantasks.pro
    module register.pro
    add.database ssp
    controller host add ipp006
    run 

nothing is loaded until I make a fake entry in summitExp:
    insert into summitExp (summit_id, dateobs) values (0,'2012-01-31 20:30:26');
    update summitExp set summit_id = 0; 

This is a hack!  All the manually injected files have summit_id = 0... Why are we joining on summitExp? Not all data downloads through summitcopy....


"	heather
Milestone 	1506	negative window found  in psphotStack  psf matching	psphot		defect	bills	new	2012-08-23T14:35:54-10:00	2012-08-23T14:35:54-10:00	"We are getting a number of staicsky failures where the psf matching fails due to a negative kron radius calculation. See log excerpt below.

The images don't look particularly bad.
Recent examples are sky_id 193594 and 193468

      Filtered out 58 of 538 sources
    built background image: 2.496715 sec
      image sky : mean -5.720879, stdev 296.211211, dQ 0.561366
      background sky : min -31.189825 mean -5.720878 max 16.422895 stdev 8.532555
        Iso-kernel region: [21:6285,21:6267] out of 6307,6289
      Found 29 stamps
        Kron Radii vs FWHMs 1: fwhm: 6.000000, kron 3.414591
        Kron Radii vs FWHMs 2: fwhm: 7.308946, kron -3.452876
        Windows from Kron Radii: 9.390125 for 1, -9.495409 for 2
Error in the psphot image loop

 -> pmSubtractionStampsGetWindow (pmSubtractionStamps.c:945): Unable to select stamps for PSF-matching
     Unable to determine normalisation window size (2).
 -> pmSubtractionMatch (pmSubtractionMatch.c:710): Unable to select stamps for PSF-matching
     Unable to get stamp window.
 -> matchKernel (psphotStackMatchPSFsUtils.c:303): Unable to select stamps for PSF-matching
"	bills
Milestone 	1507	staticsky run faults if one of the inputs has two few pixels	psphot		defect	bills	new	2012-08-23T14:37:42-10:00	2012-08-23T14:37:42-10:00	"sky_id 193468 has a couple of inputs that are mostly empty.
The staticsky run fails (by recipe) on this.

It should probably just exclude the input(s) from the radial apertures measurement."	bills
Milestone 	1508	ppStack memory use explodes for stack_id 1461976	ppStack		defect	eugene	new	2012-10-04T11:45:48-10:00	2012-10-04T11:45:48-10:00	"stack_id 1461976 process size grows to > 50GB

We killed it after it passed

      Stage 1: Load Sources and Generate Target PSF: 281.033317 sec

previous attempts blew up earlier"	bills
Milestone 	1509	pstamp parser should check data_group in bycoord requests	ppstamp		defect	bills	new	2012-10-05T10:50:16-10:00	2012-10-05T10:50:16-10:00	"Currently if the user specifies coordinates in say MD02 but
supplies an MD09 data group, the postage stamp parser does
a whole lot of work to find out that there is no overlap.

It should be smarter in this regard.

"	bills
Milestone 	1510	psphot, psphot.psfstars trace level >5 output different psf photometry	psphot	unspecified	defect	Mark Huber	assigned	2012-11-18T22:25:45-10:00	2012-11-18T22:26:08-10:00	Appears at least the current tag ipp-20121026 outputs a different PSF_INST_MAG when a trace level for psphot or pshot.psfstars is >5. The difference looks to be on the order of the aperture correction.	Mark Huber
Milestone 	1511	psphotStack assertion failure no psf	psphot	current	defect	eugene	new	2012-12-18T12:37:08-10:00	2012-12-18T12:37:08-10:00	"staticskyRun 241706 and others pop an assertion failure

missing psf?
Backtrace depth: 8
Backtrace 0: p_psAssert
Backtrace 1: psphotResetModelsReadout
Backtrace 2: psphotResetModels
Backtrace 3: (unknown)
Backtrace 4: (unknown)
Backtrace 5: (unknown)
Backtrace 6: __libc_start_main
Backtrace 7: (unknown)
 elements.
 -> pmGrowthCurveGenerate (pmGrowthCurveGenerate.c:121): unknown psLib error
     failure to measure stats
 -> psphotMakeGrowthCurve (psphotMakeGrowthCurve.c:50): Problem with aperture photometry
     Fitting aperture corrections
 -> psphotChoosePSFReadout (psphotChoosePSF.c:361): Problem in PSF
     Unable to construct flux scale for PSF
 -> psphotChoosePSF (psphotChoosePSF.c:25): Problem in configure files
     failed to choose a psf model for PSPHOT.STACK.OUTPUT.IMAGE entry 1



The root cause of the assert is that psphotStackReadout does not insure that psphotChoosePSF succeeds before proceeding in the radial apertures loop.

psphotResetModels expects the psf to exist on readout->analysis and pops the assertion.

The reason that psphotChoosePSF is failing is that none of the apMag values in pmGrowthCurveGenerate measured are finite.

I'll update the code to check before calling psphotVectorStats and print a better error message.

The error happens when working on the  g band input convolved stack from stack 1068530 after convolving to 6pixel fwhm. The input image looks pretty bad the further convolved image looks worse.

There is a lot of ringing from deconvolution.

This error is seen in 8 of LAP staticsky runs out of the 60000 run so far"	bills
Milestone 	1512	ippToPsps ImageMeta.photoScat is set incorrectly	misc		defect	heather	new	2013-01-08T14:16:17-10:00	2013-01-08T14:16:17-10:00	"ippToPsps is setting ImageMeta.photoScat incorrectly

It is set to smf.header(""ZPT_OBS"") it should be setting it to the ZPT_ERR value.

Interestingly FrameMeta.photoScat has the correct value which is
the same for all images (otas) in the frame.

"	bills
Milestone 	1513	pswarp exit status is not the same according to the number of threads	misc		defect	eugene	new	2013-02-05T16:09:45-10:00	2013-02-05T16:09:45-10:00	"Test case:
skycell.1955.035, exposure = o6308g0480o/567502

When the command line is:
pswarp -threads 4 [...]
the exit code is 0

When it is:
pswarp -threads 1 [...]
the exit code is 4


"	Serge CHASTEL
Milestone 	1514	brighter but not near saturated stars with core pixels NAN due to scaling using COMP_WT	ppSub		defect	eugene	new	2013-03-11T13:01:52-10:00	2013-03-11T13:01:52-10:00	"Originally found in ppSub save-revconv output *.refConv.wt: 
diff_id=272923, skycell.052 for MD10.20120818
that is a  fairly deep and down-scaled refstack to warp in a WARPSTACK diffim.

Core pixels of brighter sources are outside the allowed data value in the compressed FITS and set to NAN. As the refConv.wt FITS are basically used only for magic, may not be critical priority if magic not in use. Unknown if issue in other compressed files."	Mark Huber
Milestone 	894	"HAVE_KAPA=""true"" should be relaxed so that ./configure --disable-kapa works"	psconfig	unspecified	enhancement	eugene	new	2007-07-31T07:11:56-10:00	2010-05-02T15:04:01-10:00	"In light of various difficulties compiling Kapa, it would be nice if the 

HAVE_KAPA=""true""

line in 'configure' in the psModules directory could be removed so that './configure --disable-kapa' and thus 'psconfigure --disable-kapa' work.  This may require figuring out why that line was necessary in the first place instead of specifying defaults in whatever the more traditional way might be that would allow overriding."	Michael Wood-Vasey
Milestone 	1104	Thresholding when PSF is supplied	psphot	unspecified	enhancement	eugene	assigned	2008-06-24T09:40:01-10:00	2008-06-24T10:20:39-10:00	"When a PSF is supplied, the first (bright) round of thresholding (using PEAKS_NSIGMA_LIMIT) should be unnecessary (since we don't need to choose PSF stars), and the program should immediately proceed to the second (faint) round of thresholding (with PEAKS_NSIGMA_LIMIT_2).  Currently, the program will apply the bright threshold and die if there are no sources above this limit without proceeding to the faint threshold.  The result is that even if PEAKS_NSIGMA_LIMIT_2 is set absurdly low and a PSF is supplied, the program may fail with an error that it couldn't find any (bright) sources on the image.

I also wonder if finding no sources should be an error.  In the general scheme of things, it usually is (e.g., I can't do astrometry when there's no stars), but psphot should simply report no stars, rather than failing with an error since there are times when no sources is exactly right (e.g., subtracted or blank images)."	Paul Price
Milestone 	1171	Nebulous issues with missing files	Nebulous	unspecified	enhancement	watersc1@…	assigned	2009-01-02T09:57:57-10:00	2010-05-02T14:57:51-10:00	"I had some trouble dealing with files that were on ipp027, but were lost when the raid was rebuilt.  Most files had been replicated, but a handful of the detrend images (from detRegisteredImfile) were missed.  I tried to re-build these by removing their detRegisteredImfile entries and resetting the detRun state to 'run'.  This re-triggered the analysis used to build those files, but caught a problem with the interaction with nebulous.  As far as I can tell, the following was happening:

* there was already an output file with the corresponding ext_id in nebulous (eg, gpc1/flatcorr.20081107/GPC1.FLAT.86/GPC1.FLAT.86.XY23.co.fits).  

* in addition, the nebulous object had a single associated instance on ipp027.  

* if ipp027 was marked available (but not 'allocate'), nebulous attempted to supply that instance to the calling program. it did not exist, so this operation failed. 

* when I disabled 'available' for ipp027, nebulous now reported 'no instance available'.

This seems like the right behavior IF I had been requesting a file for 'read', but since I was trying to create a new file, this does not seem right.  In the end, I had to delete (in sql, since I was a little nervous about neb-rm) the nebulous entry for the file."	eugene
Milestone 	1225	psbuild: restrict extlibs and extperl dependencies depending on what's being built	psconfig	current	enhancement	eugene	assigned	2009-04-22T02:53:24-10:00	2010-05-02T15:05:41-10:00	"The extlibs and extperl mechanism by now is quite good at making sure that IPP in its entirety compiles. However, some problems linger, including some we ran into when compiling the Ohana-only tarball at http://kiawe.ifa.hawaii.edu/eugene/downloads/Ohana.20081024.tgz

Many PS1SC people will want to install Ohana, but those don't need to compile all the dependencies of all of IPP; both to circumvent installation problems of some of these dependencies, and just to save time during compilation. However, currently psbuild -extbuild builds all the dependencies of all of IPP.

I.e. psbuild should allow the dependencies actually to depend on what's being built, so that 
`psbuild -extbuild -only Ohana`
doesn't build things that aren't required by Ohana.

Attached to the ticket is an email from Gene to Mark Huber where this came up, and where Gene suggested building only the things required by Ohana for the Ohana-only distribution (""png, jpeg, readline"")."	Sebastian Jester
Milestone 	1254	Ability to step through DVO database in subsections that cover entire catalog	DVO		enhancement	magnier	new	2009-05-19T11:46:21-10:00	2010-04-28T16:56:47-10:00	"Dave Monet and I would be interested in being able to grab subsections of a DVO database in a systematic and comprehensive way.  For example, we would like to grab 20 square degree sections at a time in such a way that we are guaranteed to eventually cover the entire sky.  Being able to choose to either have or not have duplications would be a useful option, but having no duplicates would probably be the preferred default.

As discussed in Tuesday's IPP workshop, one way Gene suggested to do this was to use 'gcat' to get a list of all catalog files and then be able to pass a catalog file to mextract (and potentially related utilities such as imextract).  E.g.

gcat 0 0 180

to get a list of all catalog files in 'catlist' (how do you store the results of this command to arrays?).  And then

for cat in catlist:
  mextract -catfile cat  ra, dec, [etc.]
  write output.txt ra, dec, [etc.]
endfor

I suppose one could manually do this from the outside by manually extract the files and creating a DVO database for each, but that seems a bit ugly and not really a useful thing to include into scripts."	Michael Wood-Vasey
Milestone 	1264	Add ability to build without the -Werror flag	psconfig		enhancement	eugene	assigned	2009-05-26T06:09:53-10:00	2010-05-02T15:06:00-10:00	I  would like to be able to build the pipeline without the -Werror flag. The Intel and other compilers give warnings not found by the gnu compiler. The -Werror flag prevents a build by these compilers.	scamicha@…
Milestone 	1269	How to retrieve DVO list with unique measures	DVO	current	enhancement	magnier	new	2009-06-24T08:05:42-10:00	2010-04-28T17:03:54-10:00	"How do you retrieve a DVO list of source and guarantee that the list consists solely of unique measures.  Dave Monet and I have been finding that using addstar to put together .smf files for a medium-deep field from the 2009/06/03 release and then retrieving the measures results in a small fraction of multiple identical measures being returned.  

Is there an option to ensure a unique list or is this a fundamental feature of DVO?"	Michael Wood-Vasey
Milestone 	1318	Print double-precision values in DVO	DVO		enhancement	magnier	new	2009-08-29T11:04:44-10:00	2009-08-29T11:04:44-10:00	Request (by D. Finkbeiner) to control the precision of printing of values.  E.g., if the variable is a double, printing the variable should reveal the full double precision.	Michael Wood-Vasey
Milestone 	1323	DVO image table parent_id	DVO		enhancement	eugene	new	2009-08-31T04:39:44-10:00	2009-08-31T04:39:44-10:00	It's not clear how to associate chip images with mosaic images in the DVO images table.  It would be nice to get a parent_id or some other field; presently the name tag is the best way to do this that I have found.	eschlafly@…
Milestone 	1327	Images.dat enhancement	DVO		enhancement	eugene	new	2009-09-10T06:30:45-10:00	2009-09-10T06:30:45-10:00	"It would be really useful if images.dat had a few extra / filled in fields:

CCDNUM could be set to be the chip number
PARENT could be the image_id of the PHU corresponding to a chip image; some special value for PHUs
Are we guaranteed that IMAGE_ID-1 always corresponds to the position of the image within the file?  If not, it would be useful to have a image_id -> image position map somewhere."	eschlafly@…
Milestone 	1332	DVO aperture mags	DVO		enhancement	eugene	new	2009-10-15T13:51:39-10:00	2009-10-15T13:51:39-10:00	"I now have a version of DVO that has aperture mags but I do not believe enough information is saved to compute uncertainties.  We need either sky_sigma + aperture size + gain, or we need to save the final result.

Thanks,

Eddie Schlafly"	eschlafly@…
Milestone 	1341	Nebulous delete handling of unavailable instances	Nebulous		enhancement	watersc1	new	2009-12-03T15:32:52-10:00	2010-05-02T14:59:42-10:00	Nebulous delete should save unavailable instances for later deletion.  This requires an added table (orphans) as well as a tool to poll this table and check for orphans that can be removed (possibly a function of nebdiskd?).	watersc1
Milestone 	1351	Make EXTDATA values all upper-case	ippconfig	current	enhancement	eugene	assigned	2010-02-08T10:15:01-10:00	2010-05-02T13:50:33-10:00	"CFITSIO funpack rewrites EXTDATA as all caps.  I'm not sure that this follows any particular standard, but CFITSIO has some implicit power.  As a practical issue, it would be a great convenience is the EXTDATA values could be written, or, more importantly, recognized, by IPP software in all-caps.

This is a ticket file subsequent to discussions with Joel Welling and Gene Magnier. "	Michael Wood-Vasey
Milestone 	1354	Would like to use DVO on stack cmf files	DVO	current	enhancement	eugene	new	2010-02-09T15:31:16-10:00	2010-02-09T15:31:16-10:00	"I would like to use addstar and DVO on cmf files for stacks and diffs derived from those stacks.  But it looks like addstar wants a chip number when loading:

----
[serenity MD04] addstar -v -D CATDIR md04 -accept-astrom skycell.091/MD04.skycell.091.stk.34591.cmf
file skycell.091/MD04.skycell.091.stk.34591.cmf has 2 headers, including 1 images
reading header for SkyChip.hdr (SkyChip.psf)
invalid ccdnum SkyChip.hdr
skipping SkyChip.hdr
no valid image data in any of these files, giving up
----

Is this a fundamental issue with registering images, or a simple mod to allow for a format as followed by the stacks."	Michael Wood-Vasey
Milestone 	1369	update ppSim nebulous changes to be recipe-based	ppSim		enhancement	eugene	new	2010-05-03T12:04:17-10:00	2010-05-03T12:04:17-10:00	"the ppSimSequence mods created by Bill G. for nebulous support are too hardwired (require special code for specific cameras).  move the options to a recipe.
"	eugene
Milestone 	1403	Request to preserve SAT mask in stacks	ppStack		enhancement	eugene	assigned	2010-07-07T12:14:45-10:00	2010-07-07T12:27:45-10:00	"When stacks are made from input warps, it would be useful to preserve the reason pixels are masked from the warps.  While this might not always be well-defined, or might conflicts, the particular case of recording saturate stars should lend itself well to this.  In general stars that are saturated will be saturated in all of the input warps and this information could be carried through the to the stacks.

This came up particularly because my students we're hoping to blend in the center of the saturated stars and were trying to look for MASK=16 in the ref stacks, but found that instead the cores of bright stars were given as MASK=8 (blank), which is unfortunately degenerate with masked pixels in general missing data regions which one wants to be blank in the images."	Michael Wood-Vasey
Milestone 	1405	Why do I need -dev to install the header files?	misc		enhancement	eugene	new	2010-07-09T13:07:48-10:00	2010-07-09T13:07:48-10:00	"I think I don't understand the meaning of -dev.  I thought it was supposed to be only needed for developers, but it appears to me that it's needed to get the header files installed.  Is that intentional?  What is the ideal command that a new install should use?

I nowadays run

psbuild -dev -rebuild -clean -optimize

in the context of an update.  Is this just way I should do it, or is there something I don't understand?

One of the reasons I ask is that -dev installs things like Nebulous and a few other things that I'm not interested in (and sometimes don't compile)."	Michael Wood-Vasey
Milestone 	1413	List tesselation and skycell in FITS header	misc	current	enhancement	eugene	new	2010-08-04T04:09:59-10:00	2010-08-04T04:09:59-10:00	Please list tesselation used and skycell in the FITS header of warped, stacked, and diff files (and associated weight, mask, cmf, etc files).	Michael Wood-Vasey
Milestone 	1432	correct the source guess position based on the mean moments of the PSF model?	psphot		enhancement	eugene	new	2010-10-29T10:58:01-10:00	2010-10-29T10:58:01-10:00	"currently, the guess position uses the moments. but, the psf model could have a non-zero first moment.  should we be offsetting the guess by that amount (or what?)
"	eugene
Milestone 	1433	allow ppBackground to generate full scale background model	misc		enhancement	eugene	new	2010-10-29T16:16:35-10:00	2010-10-29T16:16:35-10:00	Although it can be generated by simply subtracting the background subtracted image from the restored image, it would be useful if ppBackground could optionally generate these images.	watersc1
Milestone 	1456	consider time stamping pantasks error messages	PanTasks		enhancement	eugene	new	2011-02-08T21:58:33-10:00	2011-02-08T21:58:33-10:00	"pantasks for summit copy  died tonight. The stderr log had the output below. The information would be more useful if it were time
stamped so we could correlate it with the time of the crash

bound to port: 2000
controller is not responding (0 tries)
controller is not responding (1 tries)
controller is not responding (2 tries)
controller is not responding (3 tries)
controller is not responding (4 tries)
controller is not responding (0 tries)
controller is not responding (0 tries)
controller is not responding (0 tries)
controller is not responding (0 tries)
controller is not responding (1 tries)
controller is not responding (0 tries)
controller is not responding (0 tries)
controller is not responding (0 tries)
controller is not responding (0 tries)
ERROR: missing metadata output name on first line
ERROR: missing metadata output name on first line
controller is not responding (0 tries)
controller is not responding (1 tries)
controller is not responding (0 tries)
ERROR: missing metadata output name on first line
controller is not responding (0 tries)
controller is not responding (0 tries)
ERROR: missing metadata output name on first line
ReadtoIOBuffer read error: Connection reset by peer
ReadtoIOBuffer read error: Connection reset by peer
garbage in pcontrol reponse
missing PID in pcontrol message : programming error
ControllerCommand returns: 1
ControllerCommand response: Njobs: 0"	bills
Milestone 	1495	czartool kittens	ippMonitor		enhancement	rhenders	new	2011-08-09T14:11:53-10:00	2011-08-09T14:11:53-10:00	"we need a tab on czartool (currently we have standard and update) that shows the top 20 kittens from google images. This distraction is necessary to calm the czar when there is too much blood on czartool.

"	heather
Milestone 	1496	missing the 'icon' in the browser for ippMonitor	ippMonitor		enhancement	eugene	new	2011-08-30T16:20:16-10:00	2011-08-30T16:20:16-10:00	There used to be an icon for ippMonitor in the browser that looked like a galaxy. It disappeared after ippMonitor moved. Can we get it back? It's disorienting and plain without it...	heather
Milestone 	702	C Code Standards astyle appendix is deficient	docs	unspecified	defect	eugene	assigned	2006-02-08T11:24:01-10:00	2010-05-02T13:02:00-10:00	"There are a couple of issues with the .astylerc options as specified in appendix
D and at least astyle 1.15.3.  If astyle is incapable of handling some of these
issues it would be nice to see an appendix for how to configure indent.

if-then-else statements are ""un-cuddled"".

    if (!*dest)
    {
        *dest = psStringCopy("""");
        oldLength = 0;
    }
    else
    {
        // size of existing string
        oldLength = strlen(*dest);
    }


There is no space between variable declarations and there types.

    va_list args;
    size_t length;             // complete string length (sans \0)
    size_t oldLength;          // original string length (sans \0

Switch statement blocks are ""un-cuddled"" as well

    switch (pType)
    {
        case PS_TYPE_S8:
            PS_NAN_ALLOC(myNaN, psS8, PS_MAX_S8);
            break;
     .
     .

Trailing comments are not alligned in some situations.

    psListIterator *cursor;            // row iterator
    psMetadataItem *item;              // field in row
    mysqlType *mType;             // type tmp variable
    static bool isNull = true;      // used in a MYSQL_BIND to indicate NULL,
    // this will be used outside of this func
    psU32 i;                  // field index"	jhoblitt
Milestone 	786	psMetadataItemCompare needs clarification	PSLib	unspecified	defect	Paul Price	assigned	2006-08-03T16:23:17-10:00	2009-03-20T09:19:50-10:00	"There are a couple of issues with psMetadataItemCompare.

First of all, there is no type checking in the compare.  That means that an F64
can be compared with, for example, and S32 or even a Bool.  I'm not sure if this
was the intention or not.  However, I've ran some tests and because of this, if
an item is an S32 with value 1 and compared with a bool item with value 'true'
then the function will return 'true'.

On the flip side, there are cases where similar or identical types with matching
values come back false (non-matching).  For instance, I ran a test comparing an
F32 item with an F64 item, both with values 1.01.  Despite the fact that the
name, comment, and data values were identical, the function reported that they
did not match.  This is due to the way in which the values are compared,
ie,
psMetadataItemCompare.c:32  return (template->data.TEMPLATENAME == value) ? true
: false; \

If you want to compare floats or doubles with each other or themselves, it's
best to do an abs(float - double) < DBL_EPSILON
Also, if you do want the ability to compare a double with an integer, you need
to do something like this and decide what your tolerance should be.  For another
example, I tried comparing an S32 of value 1 with an F64 of value 1.00001.  This
returns false.  I imagine that if you want the ability to compare such values,
you'd like that case to pass.  Consequently, just let me know what way to go
with this if you'd like it fixed.
thanks."	David.Robbins@…
Milestone 	794	psTrace*() is not thread safe	PSLib	unspecified	defect	eugene	assigned	2006-08-08T01:20:08-10:00	2010-05-02T13:11:21-10:00	"'cRoot' should either be protected by a mutex or per thread. I'd don't think it
makes sense to configure tracing per thread so we should probably just use a
mutex.  Another possiblity should be to unify psTrace with psLog.  Then we'd
only have to worry about psLog's thread safety."	jhoblitt
Milestone 	1008	Recipes may be read multiple times	psModules	unspecified	defect	Paul Price	assigned	2007-11-12T18:15:41-10:00	2010-05-02T13:35:59-10:00	"I've suspected some problems in the code that reads the recipes during configuration (psModules/config/pmConfig.c) for some time, but hadn't really noticed anything substantive until just now.

Performing ppArith on a newly created image, an error is generated:

price@mithrandir:/home/mithrandir/price/data2/simtest>ppArith -file1 path://SIMTEST/detrend/SIMTEST.DARKMASK.3/SIMTEST.DARKMASK.norm.3.0.fits -op '|' -file2 path://SIMTEST/detrend/SIMTEST.DARKMASK.4/SIMTEST.DARKMASK.norm.4.0.fits path://SIMTEST/detrend/SIMTEST.20.0 -stats path://SIMTEST/detrend/SIMTEST.20.0.stats -recipe PPSTATS CHIPSTATS -mask
2007-11-13 04:12:41Z|mithrandir          |I|psModules.config
    Camera SIMTEST, format SIMTEST matches header.
2007-11-13 04:12:41Z|mithrandir          |W|formatFromHeader (pmConfig.c:985)
    Camera SIMPLE, format SIMPLE also matches header --- ignored.
2007-11-13 04:12:41Z|mithrandir          |I|psModules.config
    Camera SIMTEST, format SIMTEST matches header.
 psMetadataAddItem (psMetadata.c:684) : Duplicate metadata item name: OVER_VAL is not allowed.  Use a psMetadataFlags option to allow such action.

This appears to be because the PPSTATS recipe is processed multiple times, and since HEADER is a MULTI, we end up with multiple versions of each of the requested headers.

For now, the problems aren't too bad, but eventually, some careful thought needs to go in to the details of reading the recipes."	Paul Price
Milestone 	1180	DVO C-C interrupt does not break out to top level	Ohana		defect	eugene	assigned	2009-02-27T14:18:32-10:00	2010-04-28T14:23:36-10:00	control C seems to only abort the inner most loop.	eugene
Milestone 	1184	Nebulous calls die()	Nebulous	current	defect	watersc1@…	assigned	2009-03-03T11:48:00-10:00	2010-05-02T15:00:41-10:00	"Though calling die() is the Perl-ish thing to do, it's not what most of us are used to and is not the way we operate.

Need to remove all instances of die() in Nebulous, in favour of returning undef (or similar)."	Paul Price
Milestone 	1204	pxinject should not allow end_stage of warp and tess_id is NULL	ippTools		defect	eugene	assigned	2009-03-20T10:05:16-10:00	2010-05-15T11:47:28-10:00	"it is invalid to perform the warp analysis without a tessellation.  this should be enforced by pxinject.

"	eugene
Milestone 	1205	dvo segfaults on unknown db version	DVO		defect	eugene	new	2009-03-20T11:10:33-10:00	2010-04-28T14:38:29-10:00	says it all	eugene
Milestone 	1208	dettool -revertresidexp fails on foreign key constraint	ippTools		defect	eugene	assigned	2009-03-24T10:28:54-10:00	2010-05-15T11:15:10-10:00	"this revert fails because other steps (detreject) have already moved ahead.  this suggests a problem with the constraints

ipp004: dettool -dbname gpc1 -revertresidexp -det_id 100
 p_psDBRunQuery (psDB.c:812) : Failed to execute SQL query.  Error: Cannot delete or update a parent row: a foreign key constraint fails (`gpc1/detRunSummary`, CONSTRAINT `detRunSummary_ibfk_2` FOREIGN KEY (`det_id`, `iteration`) REFERENCES `detResidExp` (`det_id`, `iteration`))
 revertresidexpMode (dettool_residexp.c:291) : database error


 -> p_psDBRunQuery (psDB.c:812): Database error generated by the server
     Failed to execute SQL query.  Error: Cannot delete or update a parent row: a foreign key constraint fails (`gpc1/detRunSummary`, CONSTRAINT `detRunSummary_ibfk_2` FOREIGN KEY (`det_id`, `iteration`) REFERENCES `detResidExp` (`det_id`, `iteration`))
 -> revertresidexpMode (dettool_residexp.c:291): unknown psLib error
     database error
"	eugene
Milestone 	1248	pschecklibs -force build does not work fresh after svn export	psconfig		defect	eugene	new	2009-05-13T03:22:31-10:00	2010-04-28T14:41:11-10:00	"Some Mac oddities require `pschecklibs -force build` to avoid problems during compilation, but one can't run this command fresh after an svn export because the psconfig.csh in export_dir/psconfig doesn't exist yet. However, in a way, forcing compilation of build tools only makes sense if it's the first thing that happens.

Some form of calling `psbuild -dev` is required before `pschecklibs -force` works. Do we need `pschecklibs -dev`?

This problem occurs on all platforms; our Linux version has an older autoconf than required by IPP, so I ran into it also on Linux."	Sebastian Jester
Milestone 	1265	pmReadoutSetVariance does not correctly handle low-signal high-gain	psModules		defect	eugene	assigned	2009-06-11T11:06:53-10:00	2010-05-02T13:03:46-10:00	"in pmReadoutSetVariance, there is a bit of code to keep the variance within sensible ranges: it forces the variance to be positive (taking the absolute value) and it clips the lower end to 1.  the first is an acceptable hack, though it would be worth understanding how the code can ever result in a negative variance (add an assert?).  the second is a bigger problem since the calculation is in DN: the clipping needs to be to the appropriate amount in DN based on the gain equivalent to a 1 e-.
"	eugene
Milestone 	1293	Needed psbuild -dev to build psLib	psconfig	current	defect	eugene	assigned	2009-08-11T07:49:20-10:00	2010-05-15T11:14:15-10:00	I needed to use -dev to build psLib.  Can someone double check that the SVN version has the appropriate configuration files built from the autoconf?	Michael Wood-Vasey
Milestone 	1320	Unspecified missing package for ippdb.src	psconfig		defect	eugene	assigned	2009-08-30T10:11:56-10:00	2010-05-15T11:15:45-10:00	"My compilation of today's SVN head (r25223) on my Mac Pro (Intel) failed with

[serenity ippdb.src] psautogen 

You must have  installed to compile ippdb.
Download the appropriate package for your distribution,
or get the source tarball at http://ftp.gnu.org/gnu/libtool/

Is this a libtool issue?"	Michael Wood-Vasey
Milestone 	1356	ippTools sql files should use @WORD@ replacement instead of %s hooks	ippTools		defect	eugene	assigned	2010-02-15T14:37:19-10:00	2010-02-15T14:37:39-10:00	"in various places, the sql includes commented-out segments with a string format statement (eg, -- WHERE HOOK %s) which is replaced with some optional code block by the c program (eg, a where clause).  This mechanism is fragile, esp since the sql wildcard is %.  The new replacement is to use a special word in the sql statement (@WORD@) that is then explicitly replaced in the c code with a statement of the form psStringSubstitute(&query,whereClause,""@WORD@"").

see example of the latter code in magictool_restore_diff.sql / magictool.c
"	eugene
Milestone 	1358	"Remove ""#include malloc.h"" from Ohana/src/opihi/lib.data/svdcmp_bond_raw.c"	Ohana		defect	eugene	assigned	2010-02-22T11:42:27-10:00	2010-03-04T10:59:52-10:00	"Remove 

#include <malloc.h> 

from Ohana/src/opihi/lib.data/svdcmp_bond_raw.c

It triggers a compilation error on Mac OS X 10.5, gcc 4.2.  It's not necessary."	Michael Wood-Vasey
Milestone 	1376	SDSS Photcodes Poorly Labeled	DVO	current	defect	eugene	new	2010-05-10T11:53:07-10:00	2010-05-10T11:53:07-10:00	"In Photcodes.dat files, we have both x_SDSS and X_SDSS where x = ugriz and X = UGRIZ. Only the X_SDSS are filled in by addstar. This doesn't make a lot of sense since SDSS always refers to their photometric filters with a lower case letter. 

Small issue, but I wasted half an hour looking for other problems before I figured it out. "	morganson@…
Milestone 	1378	Type confusion and strict aliasing in gtfringetable	Ohana		defect	eugene	reopened	2010-05-11T10:34:13-10:00	2010-08-03T08:42:35-10:00	"Paul says
""""""
off_t is probably defined as ""long int"" instead of ""long long int"", in which case the cast is inappropriate.

The quickest workaround would be to turn of -Werr in the Makefile.
""""""

but this looks general so should be fixed.

Joel reported to this to the ps-ipp-dev list last week.

""""""
/home/giga/temp/PS1/src/ipp/Ohana/src/misc/src/gtfringetable.c: In function ‘main’:
/home/giga/temp/PS1/src/ipp/Ohana/src/misc/src/gtfringetable.c:38: warning: dereferencing type-punned pointer will break strict-aliasing rules
/home/giga/temp/PS1/src/ipp/Ohana/src/misc/src/gtfringetable.c:39: warning: dereferencing type-punned pointer will break strict-aliasing rules
/home/giga/temp/PS1/src/ipp/Ohana/src/misc/src/gtfringetable.c:40: warning: dereferencing type-punned pointer will break strict-aliasing rules
/home/giga/temp/PS1/src/ipp/Ohana/src/misc/src/gtfringetable.c:41: warning: dereferencing type-punned pointer will break strict-aliasing rules
"""""""	Michael Wood-Vasey
Milestone 	1385	wrong mask file may be used for chip stage stamps	pstamp		defect	bills	new	2010-05-18T09:45:06-10:00	2010-05-18T09:45:06-10:00	"The postage stamp parser has a number of optimizations that attempt to optimize parsing. One of these is that the camProcessedExp data for each exposure is cached. This avoids multiple invocations of camtool.

Since the postage stamp spec says that the 'best available astrometry' is to be used for sky to chip coordinate transformations the latest camRun for any particular exposure is used for the astrometry. These data are cached by the parser and are reused if a subsequent request row needs astrometry from the same exposure.

Recently I switched the resolution of the mask file to use the camera stage masks. This avoids some nasty destreaking management when chip files are cleaned and updated. The camera masks aren't cleaned and thus once destreaked are always available.

However, since the camRun data is cached, for a byid chip request for a chipRun other than the latest for a given exposure the newer mask file will be used for stamps and NAN masking.

Since usually the latest chipRun is the one of interest to users (IPP processing is always getting better) and the latest is the one chosen as the result of bycoord requests this probably isn't a big deal."	bills
Milestone 	1391	psFits improvement to error message for non-existent file	misc		defect	eugene	new	2010-06-04T07:48:55-10:00	2010-06-04T07:48:55-10:00	"Would it be possible to modify psFits.c so that a missing file triggers a ""file not found"" error instead of ""file doesn't obey FITS standard"".  E.g., the current error message for if you pass a missing file as an argument to psphotForced is:

 -> psFitsOpen (psFits.c:247): file doesn't obey FITS standard
     Could not open file,'/Users/wwoodvas/Pitt/Research/PS1/2657_PTF10hmv/48_1_MD06.skycell.073.stk.81054.unconv.mask.fits'
"	Michael Wood-Vasey
Milestone 	1394	difftool -listrun returns multiple rows for stack-stack diffs	ippTools		defect	bills	new	2010-06-07T11:42:28-10:00	2010-06-07T11:42:28-10:00	"difftool -listrun -diff_id 37719 returns 42 runs. This is because
it is a stack stack diff and each skycell included has a different
stack_id so the select distinct doesn't filter out the different
skycells.

This causes the postage stamp server to get the same stamp
multiple times. 

The postage stamp server can work around this by adding a -limit 1
to the difftool command line"	bills
Milestone 	1430	DVO imlist command printing wrong imageID numbers	DVO	unspecified	defect	rhenders	new	2010-10-15T16:45:22-10:00	2010-10-15T16:45:22-10:00	"When running the 'imlist' command in DVO, for example:

imlist -region -name o5401g0318o.196158.cm.98861.smf

I get a list like this:

...
1706353 o5401g0318o.196158.cm.98861.smf[XY57.hdr]   1.1869  30.7986
1279976492.000000   917 10357 1.08 0.038 0.000
1706354 o5401g0318o.196158.cm.98861.smf[XY60.hdr]   0.0918  33.2137
1279976492.000000   653 10360 1.08 0.038 0.000
1706355 o5401g0318o.196158.cm.98861.smf[XY61.hdr]   0.1913  32.8584
1279976492.000000  1184 10361 1.08 0.038 0.000
...

but the imageIDs (the first column) appear to be wrong. That column
seems to be out of sync with the other columns by 1, such that, for
example, 1706355 is actually XY60 (not XY61). This can be shown by:

dvo: mextract -region RA, DEC, MAG where (imageID == 1706355)
dvo: vec
   N       name      size
   0         RA        653 (FLT)
   1        DEC        653 (FLT)
   2        MAG        653 (FLT)

where the 653 detections corresponds to XY60 (in the list above)."	rhenders
Milestone 	1464	raw destreak fails if chip has bad quality	magic		defect	bills	new	2011-03-15T09:57:13-10:00	2011-03-15T09:57:13-10:00	"If one tries to destreak a rawImage whose associated camRun didn't get an astrometric solution streaksremove just faults. We need
to handle this case

Examples: 
magic_ds_id  class_id
417368         XY05
417369         XY36"	bills
Milestone 	1471	IPP handling of camera commanded c* exposures is woefully PS1/GPC1 dependent.	misc		defect	watersc1	new	2011-04-04T10:56:09-10:00	2011-04-04T10:56:09-10:00	"We currently check to see if an exposure is camera commanded by checking if exp_name starts with 'c'.  This is used to determine if we need to burntool it.  If the exposure is camera commanded, and also has an exp_type of ""DOMEFLAT"", ""DARK"", or ""BIAS"", then we ignore it for the purposes of burntool.  Although this works, it is probably not the most robust way to handle this distinction.  "	watersc1
Milestone 	1472	stacktool.c sass_id may be used unitialized.	ippTools		defect	eugene	new	2011-04-05T15:53:18-10:00	2011-04-05T15:53:18-10:00	"stacktool.c: In function ‘main’:
stacktool.c:489: warning: ‘sass_id’ may be used uninitialized in this function
stacktool.c:489: note: ‘sass_id’ was declared here
---
Relevant code snippet is:

----
        //CZW Add an association entry here.
        // Define the requested association, and insert it if it doesn't already exist
        stackAssociationRow *association = pxStackAssociationDefine(config,stack_id);
        psS64 sass_id;
        if (!association->sass_id) {
----

I rewrote the above line as

psS64 sass_id = -1;

but I don't know if (a) this is the right initalization. (b) if later the return type for sass_id might change to unsigned long long (psU64) and this will have to be changed to 0.  I fear, however, that 0 is a valid value for a sass_id so -1 seemed the better choice for now.
"	Michael Wood-Vasey
Milestone 	1473	pzgetimfiles.c and pztool.c need %lld for summit_id	ippTools		defect	eugene	new	2011-04-05T16:02:50-10:00	2011-04-05T16:02:50-10:00	"Compiling pzgetimfiles.c and pztool.c on my Mac OS X (Intel, x86), gcc 4.2, I get warnings for several ""%ld"", summit_id references in pzgetimfiles.c (lines 127, 143) and pztool.c (lines 640, 915).

----
pzgetimfiles.c: In function ‘go’:
pzgetimfiles.c:127: warning: format ‘%ld’ expects type ‘long int’, but argument 4 has type ‘psS64’
pzgetimfiles.c:143: warning: format ‘%ld’ expects type ‘long int’, but argument 4 has type ‘psS64’
----

I tried OFF_T_FMT as a generalization, but that didn't work, so I set them to ""%lld"" to match psS64 of summit_id.

I've checked this in to Gene's ipp-20110404 dev branch.

There are some queries built using ""%ld"" for summit_id (pzgetimfiles.c 168, 183, 267, 295, 298, 327) that perhaps should be changed as well.  Whether summit_id is long or long long doesn't appear to be an arch-dependent thing; it's a feature of the database.

What's the right fix?"	Michael Wood-Vasey
Milestone 	1482	DVO mextract null vectors	DVO	unspecified	defect	eugene	new	2011-04-25T15:20:21-10:00	2011-04-25T15:20:21-10:00	"mextract seems to return a one entry vector instead of a null vector when no
matches are found. The vector's entry is the first entry of a previous
vector of the same name. See commands below,

dvo: catdir  /data/ipp022.0/ndeacon/hispeedzy-feb11
dvo: skyregion 190 191 -2 -1
dvo: avextract ra dec
dvo: echo ra[0]
190.028796778
dvo: echo ra[]
108
dvo: skyregion 190 191 -62 -61
dvo: mextract ra dec where (y>0.0)
dvo: echo ra[0]
190.028796778
dvo: echo ra[]
1

The bug does not appear to be present in avextract,

dvo: catdir  /data/ipp022.0/ndeacon/hispeedzy-feb11
dvo: skyregion 190 191 -2 -1
dvo: avextract ra dec
dvo: echo ra[0]
190.028796778
dvo: echo ra[]
108
dvo: skyregion 190 191 -62 -61
dvo: avextract ra dec where (y>0.0)
dvo: echo ra[]
0
"	ndeacon
Milestone 	1486	"""status"" not initialized in Ohana/src/libdvo/src/dvo_catalog_split.c"	Ohana		defect	eugene	new	2011-05-17T13:56:34-10:00	2011-05-17T13:56:34-10:00	"""status"" isn't explicitly initialized in 

Ohana/src/libdvo/src/dvo_catalog_split.c

and triggers compiler warnings.  What should it be initialized to: NULL, 0, -1, some particular flag?"	Michael Wood-Vasey
Milestone 	691	"Add an ""arrayType"" member to psArray"	PSLib	unspecified	enhancement	Paul Price	assigned	2006-02-07T04:53:37-10:00	2009-03-10T11:33:01-10:00	"We make very extensive use of psArrays in the PS processing, but there's no way
for a routine to check that it's getting the right type of array.  By using psArrays
we've given up on the option of compile-time checking, but we can include
a type in the psArray struct that at least allows a routine to assert that it's being
passed the right type.  This has two virtues:
   1/ It checks the types
   2/ It allows the reader of the code to know what type's expected.  You could
use a comment for this, but comments tend to get out of date.

I propose that we add a field ""arrayType"" to the psArray struct.  This would be a
(void *) pointer, that could point to a (persistent) member of the type that should
be stored on the array.  We should probably initialise this to NULL, but we could
add a new argument to psArrayAlloc.

Why would this help?  Because (with the additions to the object APIs discussed
in PRs 689/690) the psArrayAdd code could check that the added element was of
the desired type:
       if (!psTypeEqual(newElement, array->arrayType) { ... }
and a consumer of psArrays could say:
      assert (pmIsSource(array->arrayType));

The new element isn't strictly neccessary for this last use, but the test would have
to special-case empty lists, and it would be unable to check their types."	rhl@…
Milestone 	789	psLog & psTrace should have the same prototype	PSLib	0.12.99	enhancement	eugene	assigned	2006-08-07T15:14:17-10:00	2010-05-02T13:11:46-10:00	"The general idea is to unify psLog & psTrace's backend code.  Allowing products
& log levels to be set on everything.  With the difference being that psTrace
calls can be removed at compile time so they can be used heavily for debugging."	jhoblitt
Milestone 	849	merge psImageBackground and psImageStats	PSLib	0.12.99	enhancement	eugene	new	2006-12-06T15:26:28-10:00	2010-05-02T13:33:16-10:00	"these are nearly the same; psImageStats is not well-defined (should respect
nSubsample).  merge them."	eugene
Milestone 	993	psLib fails to compile - could nto read symbold from libfftw3f.a: Bad value	psconfig	2.3	enhancement	eugene	assigned	2007-10-26T05:23:43-10:00	2010-05-02T14:55:46-10:00	"psLib fails to compile with the following error (what's the bugzilla component for psLib?):

/usr/lib64/gcc/x86_64-suse-linux/4.1.2/../../../../x86_64-suse-linux/bin/ld: /IPP/ipp-2.3.lin64//lib/libfftw3f.a(plan-dft-r2c-1d.o): relocation R_X
86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
/IPP/ipp-2.3.lin64//lib/libfftw3f.a: could not read symbols: Bad value

Here's the context:

make[3]: Entering directory `/disk1/jester/IPP/ipp-2.3/psLib/src'
/bin/sh ../libtool --tag=CC   --mode=link gcc -std=gnu99  -pipe -O0 -g -Wall -Werror -D_XOPEN_SOURCE=600 -D_POSIX_C_SOURCE=200112L -version-info 1:
1:0  -o libpslib.la -rpath /IPP//ipp-2.3.lin64/lib  sys/libpslibsys.la astro/libpslibastro.la db/libpslibdb.la fft/libpslibfft.la fits/libpslibfits
.la imageops/libpslibimageops.la jpeg/libpslibjpeg.la math/libpslibmath.la mathtypes/libpslibmathtypes.la types/libpslibtypes.la -pthread  -L/usr/l
ib64/mysql -lmysqlclient -lz -lcrypt -lnsl -lm  -lcfitsio -lm  -lfftw3f -L/IPP//ipp-2.3.lin64/lib -lgsl -lgslcblas -lm  -ljpeg
mkdir .libs
gcc -std=gnu99 -shared  -Wl,--whole-archive sys/.libs/libpslibsys.a astro/.libs/libpslibastro.a db/.libs/libpslibdb.a fft/.libs/libpslibfft.a fits/
.libs/libpslibfits.a imageops/.libs/libpslibimageops.a jpeg/.libs/libpslibjpeg.a math/.libs/libpslibmath.a mathtypes/.libs/libpslibmathtypes.a type
s/.libs/libpslibtypes.a -Wl,--no-whole-archive  -Wl,--rpath -Wl,/usr/lib64/mysql -Wl,--rpath -Wl,/IPP//ipp-2.3.lin64/lib -Wl,--rpath -Wl,/usr/lib64
/mysql -Wl,--rpath -Wl,/IPP//ipp-2.3.lin64/lib -L/usr/lib64/mysql /usr/lib64/mysql/libmysqlclient.so -lz -lcrypt -lnsl -lcfitsio /IPP/ipp-2.3.lin64
//lib/libfftw3f.a -L/IPP//ipp-2.3.lin64/lib /IPP//ipp-2.3.lin64/lib/libgsl.so /IPP//ipp-2.3.lin64/lib/libgslcblas.so -lm -ljpeg  -pthread -Wl,-sona
me -Wl,libpslib.so.1 -o .libs/libpslib.so.1.0.1
/usr/lib64/gcc/x86_64-suse-linux/4.1.2/../../../../x86_64-suse-linux/bin/ld: /IPP/ipp-2.3.lin64//lib/libfftw3f.a(plan-dft-r2c-1d.o): relocation R_X
86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
/IPP/ipp-2.3.lin64//lib/libfftw3f.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
make[3]: *** [libpslib.la] Error 1
make[3]: Leaving directory `/disk1/jester/IPP/ipp-2.3/psLib/src'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/disk1/jester/IPP/ipp-2.3/psLib/src'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/disk1/jester/IPP/ipp-2.3/psLib/src'
make: *** [all-recursive] Error 1
problem building psLib : failure in make"	jester@…
Milestone 	1006	psMetadataItemAlloc() allocates an empty string for comments	PSLib	unspecified	enhancement	eugene	assigned	2007-11-09T12:29:17-10:00	2010-05-02T13:10:44-10:00	"If the comment field is not used, the allocator assigns an ""empty string"".  This wastes the sizeof(psMemBlock) * n psMetadataItem's without a commment (virtually all of them).  psMemBlock is ~100 byte structure so this is wasting a fair ammount of memory.  The code & SDRS should be changed so that ""no comment"" equates to psMetadataItem.comment = NULL."	jhoblitt
Milestone 	1068	time_begin/end params in ipp_detrend_combine.pl	ippScripts	2.4	enhancement	Paul Price	assigned	2008-03-26T11:08:55-10:00	2008-08-05T17:19:13-10:00	"With the essence data it is desirable to create one bad pixel mask for year 2003 and another separate one for 2004-2005 (the broken chip in 2003 was fixed for 2004+).  I can set the time_begin and time_end parameters when running dettool to create a FLATMASK and DARKMASK.  Then when I combine multiple masks together using ipp_detrend_combine.pl, a new detRun is created with  det_type MASK, but it has NULL values for the time parameters.

It would be nice to have a command-line argument for -time_begin and -time_end.

It would be even nicer to have (in addition) some inheritance of the time parameters from the input masks."	rodney
Milestone 	1083	Request for option to show version to generate help for all command line tools	misc	unspecified	enhancement	eugene	assigned	2008-04-17T02:03:38-10:00	2010-05-02T15:04:40-10:00	"It would be useful for all IPP and PsLib (and other software under ""our"" control) to report the version information and help information.  E.g.,

addstar -v
addstar --version

prints the version number (and possibly other relevant version/compilation-related information) to STDOUT and exits.  

addstar -h
addstar --help

For example:

[wwoodvas@calm] gcc --version
powerpc-apple-darwin9-gcc-4.0.1 (GCC) 4.0.1 (Apple Inc. build 5465)
Copyright (C) 2005 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

The choice of the exact option specification is up to system standardization.  E.g., perhaps double - options such as --version are not supported by choice.

P.S.  I had to choose some component to submit this bug report, but really this applies to all of PSLib and all of IPP software."	Michael Wood-Vasey
Milestone 	1131	psastro needs more Q/A plots	psastro	unspecified	enhancement	eugene	assigned	2008-08-29T13:14:19-10:00	2010-05-01T10:57:44-10:00	we need Q/A plots in psastro, initially to the screen, but later to png files.	eugene
Milestone 	1200	chiptool needs method to delete uninitiated runs	ippTools		enhancement	eugene	assigned	2009-03-12T08:45:19-10:00	2010-05-15T11:13:11-10:00	"we need to have a method to clean up mistakes in the db.  runs in the state of 'new' with no output chipProcessedImfiles can simply be deleted, if desired, by deleting the corresponding chipImfile entries and then deleting the chipRun entry.  Better to have a chiptool method to do this rather than force users to do it in the db (more dangerous).
"	eugene
Milestone 	1209	Request for psbuild option to build w/o mysql; -basic option to only build basic utilities	psconfig		enhancement	eugene	assigned	2009-03-25T04:18:18-10:00	2010-05-15T11:14:47-10:00	"Many small-time users who just want to use things like psphot and dvo will never need MySQL.  Because this dependency is one that has given trouble in the past and also because it takes a long time to build from source, having an option to build w/o this would be useful.

Perhaps this should be part of a '-basic' option to psbuild.  This option would only build end-user packages that someone would use just to analyze a few images.  Taking from the Mar 2009 ipp-2.8.dist, this basic package would include:

Ohana
ohana.base
libohana
libfits
libautocode
libdvo
libkapa
libtap.ohana
addstar
delstar
getstar
ohana.tools
kapa2
relphot
relastro
uniphot
opihi.base
mana
dvo
psModules
ppStats
psphot
psastro
ppConfigDump
ppImage
ppNorm
ppMerge
pedestal
dvoTools
pswarp
ppArith
ppStack
ppSub
ppSim
ippScripts
ippTasks
ippconfig
psconfig

Really 'ippScripts' and 'ippTasks' aren't needed either, but they're a trivial compile and contain useful examples that the user may find interesting.  I have intentionally omitted ippTools, since a lot of those have to do with interacting with the database to set up different processing steps and runs.
"	Michael Wood-Vasey
Milestone 	1250	Misleading svn output from `make psVersionDefinition.h`	psconfig	current	enhancement	Paul Price	assigned	2009-05-13T23:41:28-10:00	2010-05-02T15:16:43-10:00	"While trying to diagnose some compile problems in psLib, we found this output just before the end of the compile log that looked like there was a problem with svn:
{{{
make psVersionDefinitions.h
rm -f psVersionDefinitions.h
/usr/bin/sed -e ""s|@PSLIB_VERSION@|\""`/usr/bin/svnversion ../..`\""|"" -e ""s|@PSLIB_BRANCH@|\""`/usr/bin/svn info ../.. | /usr/bin/sed -n -e '/URL:/ h' -e '/Repository Root:/ { x; H; x; s|Repository Root: \(.*\)\nURL: \1\(.*\)|\2| ; s|^/|| ; s|/[a-zA-Z]*/src.*|| ; p }'`\""|"" -e ""s|@PSLIB_SOURCE@|\""`/usr/bin/svn info | /usr/bin/sed -n -e 's/Repository UUID: // p'`\""|"" psVersionDefinitions.h.in > psVersionDefinitions.h
svn: Path '../..' ends in '..', which is unsupported for this operation
svn: '.' is not a working copy
}}}
However, the svn messages are in fact a feature of that version mechanism when you compile and exported copy of IPP. Having them in the compile output mislead us into thinking that they were the source of the problem, when in fact the problem was elsewhere. You're free to blame us, of course, but that output from svn could be suppressed, couldn't it?"	Sebastian Jester
Milestone 	1281	optimize bundling of fakeRuns	ippScripts		enhancement	bills	new	2009-07-21T09:26:08-10:00	2010-05-18T10:29:08-10:00	"We currently go through each chip individually when bundling
fakeRuns. This is a waste of energy."	bills
Milestone 	1319	Add photcode to output of lightcurve	DVO		enhancement	wood-vasey@…	new	2009-08-29T11:57:36-10:00	2009-08-29T11:57:36-10:00	"Add photcode to output of lightcurve

Stretch goal: allow to restrict by filter."	Michael Wood-Vasey
Milestone 	1333	add user-options to modify pcontrol client restart time	PanTasks		enhancement	eugene	new	2009-10-18T10:28:28-10:00	2009-10-18T10:28:28-10:00	pcontrol currently has a hard-wired value of 10 hours for the restart time for the remote clients.  add  method for the user to modify this.	eugene
Milestone 	1370	pantasks_client can only connect to a server at start-up	PanTasks	unspecified	enhancement	eugene	new	2010-05-04T10:29:51-10:00	2010-05-04T10:29:51-10:00	"Currently, pantasks_client can only connect to a server at start-up, by reading PANTASKS_SERVER from the local ptolemy.rc file, or from the one pointed to with the '-c' option. It would be useful if a command existed within pantasks_client to read this information. For example, it would enable us to write pantasks_client interpreted scripts.
"	rhenders
Milestone 	1374	optimize postage stamp creation when multiple stamps requested from same image	pstamp		enhancement	bills	new	2010-05-05T14:32:59-10:00	2010-05-18T10:28:29-10:00	"The postage stamp server currently creates a job for each stamp to be made. The parser collects request specifications that reference the same images and queues them in order.

We could reorganize ppstamp to support creating multiple stamps in the same process. We'd need to change the work flow a bit as well"	bills
Milestone 	1465	Using cfitsio3270?	misc	current	enhancement	magnier	new	2011-03-16T05:42:56-10:00	2011-03-16T05:42:56-10:00	"Is it time to update to cfitsio3270?

I installed in and compiled on my MacBook Air and it seemed to work fine.  In particular, it compiled fpack and funpack fine (which previously didn't work in the stock distribution (see Ticket #1289).

What were some of the other customizations needed from the cfitsio3100 distribution?"	Michael Wood-Vasey
Milestone 	1470	Infinity floating-point values management	PSLib		enhancement	Serge CHASTEL	assigned	2011-03-21T11:59:39-10:00	2011-03-21T12:04:15-10:00	"(Child of NaN management)
Since MySQL does not support infinity floating-point values, the psDBIsPTypeNaN() function should test fp with isinf().

"	Serge CHASTEL
Milestone 	660	psEOC_CEOtoGCRS has largish difference	PSLib	unspecified	defect	eugene	assigned	2006-01-11T11:13:52-10:00	2010-05-02T12:40:35-10:00	"Second components of the rotation quaternion differ: 6.985e-9, compared to
1.219e-10 expected.  The great circle angular separation = 2.107e-8 = 4 mas,
which seems large.  We're not sure what's wrong, but we're pushing this onto the
stack; this bug serves as a reminder."	Paul Price
Milestone 	727	Region for psImageCut	PSLib	unspecified	defect	eugene	assigned	2006-03-20T15:41:37-10:00	2010-05-02T12:47:51-10:00	"It's unclear from the SDRS whether or not psImageCut should function similar to
other related functions in it's interpretation of region coordinates. 
Specifically, indexing from the tail is currently unsupported and seems like it
should be.  Currently, we seem to be discussing/deciding on whether or not the
lower bounds (x0, y0) should allow indexing from the end.  However, in most
functions, at least the upper bounds (x1, y1) are allowed to use negative numbers.  
In ImageCut it also seems to be the case that the current implementation allows
for the upper bounds to be smaller than the lower bounds, ie, x0 > x1, etc.  I
am assuming that this type of behavior should NOT be allowed and it is simply
working this way by mistake?"	David.Robbins@…
Milestone 	757	psSphereRot_ITRStoTEO in EOC has sign difference	PSLib	unspecified	defect	eugene	assigned	2006-05-25T15:17:01-10:00	2009-03-10T17:58:01-10:00	"the output from this function:
  Output sphere rotation   = 1.05676183925e-06, -3.218022392158e-07,
3.410787326425e-13, -0.9999955956179
  Expected sphere rotation = -1.056757184866e-06, 3.218036562932e-07,
-3.35801958072e-12, -0.9999999999994

Shows a sign difference between the expected values and the resulting ones. 
(Note that if the outputted q3=+0.999... then all q's would be opposite the
expected and thus equivalent.  A negative q3 here should mean the inverse).  I
am moving the error statement into the ""VERBOSE"" setting I added recently so
that the test passes for now.  Also, I'm getting the opposite (but same) output
on the server:
  Output sphere rotation   = -1.056757184868e-06, 3.218036565618e-07,
-3.410955728293e-13, 0.9999999999994
  Expected sphere rotation = -1.056757184866e-06, 3.218036562932e-07,
-3.35801958072e-12, -0.9999999999994"	David.Robbins@…
Milestone 	795	all psError() messages strings should be wrapped in _()	PSLib	0.12.99	defect	eugene	assigned	2006-08-08T14:02:38-10:00	2010-05-02T13:11:05-10:00	"All of the error messages that weren't inlined from psErrorText.h need to be
wrapped with _()."	jhoblitt
Milestone 	986	postage stamp coordinates are off for megacam spliced images	ppstamp	unspecified	defect	bills	assigned	2007-10-23T13:47:40-10:00	2010-05-05T13:58:47-10:00	"The command

ppstamp out -file 870505p.fits -skycenter 1:16:32.948 8:23:25.65 -pixrange 200 200

should generate a postage stamp with an object that looks like a galaxy
at the center.

Instead the object is 32 pixels left of center.

The image is extracted from a mosaic image created with pmChipMosaic

       // readout is from the mosaic
        psImage *subsetImage = psImageSubset(readout->image, options->roi);
        
        // make a copy since we're going to change the image's internals
        outReadout->image =psImageCopy(NULL, subsetImage, subsetImage->type.type);
        P_PSIMAGE_SET_COL0(outReadout->image, 0);
        P_PSIMAGE_SET_ROW0(outReadout->image, 0);

where options->roi is a region centered on the chip coordinates for the object.
Pixel (1943, 3251)

The problem is that the coordinate system changes in the process of mosaicking
the chip.

If the image is extracted using the readout for the original input image
instead of the mosaic, the correct pixels are extracted.

Looking at the mosaic image we find that it is has col0=row0=0 and
so is effectively shifted to the left as well. 

32 pixels is size of the overscan region."	bills
Milestone 	1072	NULL database values cause DVO seg faults	DVO	2.5	defect	eugene	assigned	2008-04-02T12:46:14-10:00	2008-08-05T17:32:55-10:00	"I've done some chip processing and I want to investigate the results with DVO.  Some of the chipRuns have failed, raising the FAULT flag and leaving some fields NULL in the chipProcessedImfile table (e.g. sigma_ra and sigma_dec).

I'd like to use DVO to plot some values for those runs that succeeded (where FAULT=0):

1. clean columns

  dvo: dbselect n_astrom from chipProcessedImfile where fault=0;

This works fine, because the n_astrom column has no NULL values, even where fault!=0

2. columns with NULL values

  dvo: dbselect sigma_ra from chipProcessedImfile

This fails with a seg fault that ends the DVO session, b/c some of the sigma_ra values are NULL

3. columns with excluded NULL values

 dvo: dbselect sigma_ra from chipProcessedImfile where fault=0;

This search should return only non-NULL values of sigma_ra, but it still fails with a seg fault."	rodney
Milestone 	1243	Problem with 'sed' again in parsing SVN version info	psconfig	current	defect	Paul Price	accepted	2009-05-09T03:22:23-10:00	2010-05-02T15:17:10-10:00	"/usr/bin/sed -e ""s|@PSLIB_VERSION@|\""`/usr/bin/svnversion ../..`\""|"" -e ""s|@PSLIB_BRANCH@|\""`/usr/bin/svn info ../.. | /usr/bin/sed -n -e '/URL:/ h' -e '/Repository Root:/ { x; H; x; s|Repository Root: \(.*\)\nURL: \1\(.*\)|\2| ; s|^/|| ; s|/[a-zA-Z]*/src.*|| ; p }'`\""|"" -e ""s|@PSLIB_SOURCE@|\""`/usr/bin/svn info | /usr/bin/sed -n -e 's/Repository UUID: // p'`\""|"" psVersionDefinitions.h.in > psVersionDefinitions.h

sed: 1: ""/Repository Root:/ { x; ..."": extra characters at the end of p command
svn: Write error: Broken pipe

on Mac OS X 10.5.6, Intel
"	Michael Wood-Vasey
Milestone 	1290	pschecklibs, pscheckperl, psbuild use 'csh psconfig.csh' instead of 'csh -f psconfig.csh'	psconfig	current	defect	eugene	assigned	2009-08-05T07:11:14-10:00	2010-05-02T15:18:12-10:00	"pschecklibs and pscheckperl both use calls out to 'csh psconfig.csh' to determine libdir, bindir, etc.  This means it uses your .cshrc configuration files and overrides current settings.  It's also somewhat slower to completely sources the .cshrc* files everytime.

Would it be OK to replace these with 'csh -f psconfig.csh' so that it used your current environment configuration?  This is particularly relevant for making these work right from bash.  Or is the loading behavior of 'csh psconfig.csh' desired?

I got caught up by this when my cshrc and bashrc files referred to different versions of the IPP pipeline.  I didn't think that would have been a problem and in a theoretically ideal world I don't think it should be.  But this isn't high priority."	Michael Wood-Vasey
Milestone 	1294	extsrc/gpcsw requires ginstall	misc		defect	heather	accepted	2009-08-11T07:54:09-10:00	2011-02-09T14:25:40-10:00	The new gpcsw package requires ginstall.  Is it using anything in particular that's only in ginstall or is standard install fine?	Michael Wood-Vasey
Milestone 	1314	no error with photcode-table  -import with a non-existing file	DVO	unspecified	defect	kpat.mpg	assigned	2009-08-21T05:26:48-10:00	2009-08-21T05:27:10-10:00	"When running:
photcode-table -D CATDIR CatDir -import file
with a non-existing ""file"", no errors are reported (exit status: 0)."	kpat.mpg
Milestone 	1487	Ohana/src/uniphot/lib in library list but doesn't exist	Ohana		defect	eugene	new	2011-05-17T14:04:45-10:00	2011-05-17T14:04:45-10:00	"Really trivial, but I get the following message after compiling Ohana:

ld: warning: directory '/Users/wmwv/PS1/src/ipp-eam-20110505/Ohana/src/uniphot/lib' following -L not found

This is just because $(LIB) is in the BASE_LDFLAGS in the Ohana/Makefile.Common, but there is not actual uniphot/lib created as part of compiling uniphot."	Michael Wood-Vasey
Milestone 	721	New functions: psFitsIsImage and psFitsIsTable	PSLib	unspecified	enhancement	Paul Price	assigned	2006-03-03T11:47:44-10:00	2010-05-02T14:48:02-10:00	"We need functions that tell us what type of extension we're dealing with, based
on the header:

bool psFitsIsImage(psMetadata *header);
bool psFitsIsTable(psMetadata *header);

psFitsIsImage shall return true if the FITS header corresponds to that of a FITS
image.  psFitsIsTable shall return true if the FITS header corresponds to that
of a FITS table.


I've added this function to the SDRS, and wanted to let you know."	Paul Price
Milestone 	725	Hash algorithm	PSLib	unspecified	enhancement	Paul Price	assigned	2006-03-06T15:32:48-10:00	2010-05-02T12:44:18-10:00	"The current psMetadata implementation sets the size of the hash to 10 buckets. 
This means that if a psMetadata contains significantly more than one entry,
there are hash collisions which trigger a linear search of the linked list. 
This seems to be counter to the idea of using a hash in the first place.  While
we can simply increase the size of the hash for psMetadata (what would be a good
choice?), I understand there are alternatives, such as dynamically scaling the
size of the hash.  This might be something to consider some time in the future."	Paul Price
Milestone 	1466	Have pschecklibs check for right version of fftw	misc	current	enhancement	eugene	new	2011-03-16T07:13:05-10:00	2011-03-16T07:13:05-10:00	Would it be possible for pschecklibs to check that fftw with --enable-float exists, as opposed to just some fftw library (e.g., the one that Fink installs by default)?  Is this easy or hard?	Michael Wood-Vasey
	1178	opihi should do something more reasonable if the assignment is a syntax error	Ohana		defect	eugene	new	2009-02-23T18:12:56-10:00	2009-02-23T18:12:56-10:00	"the following code in opihi gives a confusing result:

$value = something

$value = something else with a $variable

if $variable is not defined, opihi will complain about a syntax error, but $value will be left at the old value (something, in this case).  "	eugene
	1214	psphot output does not correct sigmas to fwhms for psf parameters.	psphot		defect	eugene	new	2009-04-02T09:40:44-10:00	2009-04-02T09:40:44-10:00	in pmSourcesWrite_CMF_PS1_V1 and equivalent, the function pmPSF_ModelToAxes is used to convert the internal representations (Sxx, Sxy, Syy) to major, minor, theta.  these fail to make a correction for sigma -> fwhm.  (the correction should be different from 2.35 depending on the model profile)	eugene
	1230	Satellite streaks and cells edges can be masked as CRs	psphot	current	defect	eugene	assigned	2009-04-24T15:19:37-10:00	2009-05-07T11:43:59-10:00	"A bright satellite streak may be identified as a CR by psphot, making them more difficult to detect.

There are also pixels on the edge of cells which are masked as CRs."	Paul Price
	1232	psVectorStats raises errors in too many places	PSLib		defect	eugene	new	2009-04-26T16:31:06-10:00	2009-04-26T16:31:06-10:00	"psVectorStats raises errors for various situations that really represent a limitation of the data.  For example, robustMedian will raise an error if there are no valid elements.  these situations should be handled by setting the result to nan or appropriate saturation values, and maybe raising a warning.

I've modified psStats.c to avoid the error in the appropriate cases in my copy of the code.  This needs to be made consistent with the uses in the rest of the IPP: places that relied only on a false return value should also check for NAN results (or we should always return false if the desired value is NAN, even if that does not result in an error being raised).
"	eugene
	1233	detselect logic is wrong in some cases	ippTools		defect	eugene	accepted	2009-04-29T10:54:08-10:00	2009-04-30T14:01:00-10:00	"detselect generates SQL that does not really make sense for the following cases:

* no time constraint is provided : in this case, it is clear that all detRuns should potentially match.  in fact, the sql generates where conditions of the form (time_begin IS NULL OR time_begin IS NULL).  it should just drop the time_begin and time_end conditions

* exptime supplied, but detRun exp_time_max or min are NULL : if one of these values is null, that should represent no constraint.  however, detselect generates sql forcing a range check (exp_time_min < ...).  this sql should include conditions of the form 'OR exp_time_min IS NULL'
"	eugene
	1183	dettool -defineby... should require more things by default	ippTools		defect	eugene	new	2009-03-03T10:17:15-10:00	2009-03-03T10:17:30-10:00	"dettool -defineby.. functions require only a minimal amount of fields to be set.  Most of the restrictions (airmass, solangle, etc) are not normally going to be set, but a few should normally be set in some cases:

filter : any light-type of detrun (flat, fringe, etc?) should require a filter unless explicitly overridden (maybe with filter = any?).

time_begin : all detrend types should require a time_begin.  even if a large range is desired, it is better to force the user to think about the lower bound.

"	eugene
	1191	Failed Raw Imfiles page doesn't link to log	ippMonitor		defect	eugene	new	2009-03-04T10:21:02-10:00	2009-03-04T11:08:47-10:00	"The natural question to ask when looking at the Failed Raw Imfiles page is ""why did it fail"".  For this, we need a link to the log file."	Paul Price
	1195	diff_skycell.pl needs to call ppSub with -refsources now?	ippScripts		defect	Paul Price	new	2009-03-10T18:10:45-10:00	2009-03-10T18:10:45-10:00	"While running simtest on revision 23243 I got an error when diff_skycell.pl would call ppSub

     Unable to perform ppSub: 3 at /home/ipp/IPP/ipp-svnhead.linrh64/bin/diff_skycell.pl line 278.

Looks like it called ppSub before with -sources for the template sources and now needs to be -refsources (line 213?)"	Mark Huber
	1196	pantasks_client command to show server commands (server help)	PanTasks		defect	eugene	new	2009-03-11T14:55:31-10:00	2009-03-11T14:55:31-10:00	"currently, it is difficult to know what macros have been defined on the server side in pantasks: tab-completion does not work for server commands.  adding server help function would be, well, helpful...
"	eugene
	1199	chiptool -updaterun need 'pretend' option	ippTools		defect	eugene	assigned	2009-03-12T08:43:18-10:00	2009-03-12T08:45:30-10:00	"we need to be able to see the impact of, eg, changing labels, before actually doing it.
"	eugene
	1213	chiptool -definebyquery -dateobs_begin fails to get HH:MM:SS	ippTools		defect	eugene	new	2009-03-31T22:10:19-10:00	2009-03-31T22:10:19-10:00	"it needs to be more flexible about the format as well.

YYYY/MM/DD,HH:MM 
YYYY/MM/DD,HH:MM:SS
""YYYY/MM/DD HH:MM""
etc
"	eugene
	1239	"""Ohana-only"" tarball needs to include ippconfig"	DVO	current	defect	eugene	new	2009-05-05T02:24:58-10:00	2009-05-05T02:24:58-10:00	"After building the experimental Ohana-only tarball http://kiawe.ifa.hawaii.edu/eugene/downloads/Ohana.20081024.tgz you have a compiled version of dvo, but one that won't run without the config files it needs, like ~/.ptolemyrc and the camera-specific configuration directory. Those config files and directories live in ippconfig, so ippconfig needs to be distributed and built for ""Ohana-only"", too."	Sebastian Jester
	1253	DVO seg faults with mextract on 2.3 million+ measures from ThreePi.Run2.v0 catdir	DVO	current	defect	magnier	new	2009-05-19T09:49:30-10:00	2009-05-19T09:49:30-10:00	"dvo compiled as of May 6 from SVN head that (or previous) day seg faults when reading lots of records.  I can only suspect that there's an int that's not long long enough.

[wwoodvas@heroint2 data]$ gdb `which dvo`
GNU gdb Red Hat Linux (6.5-25.el5rh)
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type ""show copying"" to see the conditions.
There is absolutely no warranty for GDB.  Type ""show warranty"" for details.
This GDB was configured as ""x86_64-redhat-linux-gnu""...Using host libthread_db library ""/lib64/libthread_db.so.1"".

(gdb) set args -D CATDIR ThreePi.Run2.v0/ -D CAMERA gpc1
(gdb) run
Starting program: /odyssey/apps/ipp/ipp-dev-welling/psconfig/ipp-dev-welling.linrh64/bin/dvo -D CATDIR ThreePi.Run2.v0/ -D CAMERA gpc1
[Thread debugging using libthread_db enabled]
[New Thread 47658106601360 (LWP 32164)]
no file /n/home/wwoodvas/.dvorc

Welcome to DVO - desktop virtual observatory


dvo: $KAPA = kapa -noX
dvo: region 0 0 180
dvo: mextract -region xccd, yccd, mag, mag:err, ra, dec, time, ra:ave, dec:ave,objID, catID, photcode, fwhm, mag:cat, phot_flags
reading images (mode DVO_MODE_MEF)

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 47658106601360 (LWP 32164)]
0x0000000000433a5c in mextract (argc=15, argv=0xd36f4b0)
    at /odyssey/apps/src/ipp/ipp-dev-welling/ipp/Ohana/src/opihi/dvo/mextract.c:149
149		    vec[n][0].elements.Flt[Npts] = values[n].Flt;
(gdb) bt
#0  0x0000000000433a5c in mextract (argc=15, argv=0xd36f4b0)
    at /odyssey/apps/src/ipp/ipp-dev-welling/ipp/Ohana/src/opihi/dvo/mextract.c:149
#1  0x00002b5843851bcb in command (
    line=0xd36f380 ""mextract -region xccd, yccd, mag, mag:err, ra, dec, time, ra:ave, dec:ave,objID, catID, photcode, fwhm, mag:cat, phot_flags"", 
    outline=0x7fff678d6ae0, VERBOSE=1)
    at /odyssey/apps/src/ipp/ipp-dev-welling/ipp/Ohana/src/opihi/lib.shell/command.c:46
#2  0x00002b584385978c in multicommand (
    line=0xd36f260 ""mextract -region xccd, yccd, mag, mag:err, ra, dec, time, ra:ave, dec:ave,objID, catID, photcode, fwhm, mag:cat, phot_flags"")
    at /odyssey/apps/src/ipp/ipp-dev-welling/ipp/Ohana/src/opihi/lib.shell/multicommand.c:79
#3  0x00002b584387ca1c in opihi (argc=1, argv=0x7fff678d6c78)
    at /odyssey/apps/src/ipp/ipp-dev-welling/ipp/Ohana/src/opihi/lib.shell/opihi.c:42
#4  0x000000000041e347 in main (argc=7, argv=0x7fff678d6c78)
    at /odyssey/apps/src/ipp/ipp-dev-welling/ipp/Ohana/src/opihi/dvo/dvo.c:71
(gdb) p n
$1 = 0
(gdb) p Npts
$2 = 268435000
(gdb) p values
$3 = (dbValue *) 0xdb08fd0
(gdb) p vec[n][0].elements.Flt
$4 = (double *) 0x2b613272a010
(gdb) p vec[n][0].elements.Flt[268435000]
Cannot access memory at address 0x2b61b27291d0
(gdb) p vec[n][0].elements.Flt[268434999]
Cannot access memory at address 0x2b61b27291c8
"	Michael Wood-Vasey
	1268	CAL_PSF_MAG_SIG all same value; should be equal to PSF_INST_MAG_SIG values	psphot		defect	magnier	new	2009-06-23T11:13:54-10:00	2009-06-23T11:13:54-10:00	The 'CAL_PSF_MAG_SIG' entries in 'o4985g0270o.73464.cm.12795.smf' are almost all the same with a value of 0.19833371.  They should be instead equal to PSF_INST_MAG_SIG.  I check this on all .smf files from the June 3 Medium deep fields for chip XY35 and XY26.	Michael Wood-Vasey
	1273	$Var ++ processing is only parsed correctly at the start of a line	PanTasks	current	defect	eugene	new	2009-06-29T14:34:42-10:00	2009-06-29T14:34:42-10:00	"The following works
    if ($magicDSRevertStage == 0)
            $magicDSToRevert_DB ++
    end

while the following does not

if ($magicDSRevertStage == 0) $magicDSToRevert_DB ++

This was confusing to me.

"	bills
	1216	Request for -version option for all command-line executables	misc		enhancement	welling	assigned	2009-04-03T09:28:15-10:00	2009-04-13T11:14:22-10:00	"I would like to request for a

--version  

option be added to all command-line executables in IPP.  This could print out what is already printed out in the first few lines of running a lot of the executables.  E.g.

pswarp --version

    psLib trunk/psconfig@23595M from 60eb6cdc-a59c-4636-a4e0-dba66a9721fd, built Mar 30 2009, 14:01:48 with cfitsio-3.100 gsl-1.11 fftw-3.0.1 mysql-5.0.22 optimised without trace
    psModules trunk/psconfig@23595M from 60eb6cdc-a59c-4636-a4e0-dba66a9721fd, b
uilt Mar 30 2009, 14:02:51 optimised without nebclient
    psphot trunk/psconfig@23595M from 60eb6cdc-a59c-4636-a4e0-dba66a9721fd, built Mar 30 2009, 14:06:19 optimised with libkapa
    ppStats trunk/psconfig@23595M from 60eb6cdc-a59c-4636-a4e0-dba66a9721fd, built Mar 30 2009, 14:06:03 optimised
    pswarp trunk@23595 from 60eb6cdc-a59c-4636-a4e0-dba66a9721fd, built Mar 30 2009, 14:10:52 optimised

as is currently spit out.  Although I think I would request that the order be reversed.  Spit out version information for calling executable first and then trace down.

The priority would be the ones we use a lot like pswarp, psphot, ppSub, ppStack, etc., but also things like ppConfigDump and other lower-level routines."	Michael Wood-Vasey
	1218	Debugging ppSub: option to save stamp position used to determine the kernel	ppSub	unspecified	enhancement	Paul Price	accepted	2009-04-06T08:30:38-10:00	2009-04-13T16:27:12-10:00	"One of the key elements of difference imaging is the choice of stamps used to determine the kernel. For debugging and testing, it is important to know:

#1 which stamps were rejected
#2 which stamps were used
#3 all possible stamps (used, rejected, and unused)

This information (in particular the x,y position, but also other info liek the reason why a stamp was rejected) could be saved in files, either straight ASCII tables or alternatively ds9 region files."	arminrest
	1224	pswarp should generate jpegs	ippTasks		enhancement	Paul Price	new	2009-04-21T11:19:19-10:00	2009-04-21T11:19:57-10:00	"we need some visualization at the warp stage.  either pswarp should generate jpegs for each skyfile, or another task should generate jpegs for projection cells in a warp run.  

"	eugene
	1247	psphot -modeltest should write the appropriate .cmf file	psphot		enhancement	eugene	new	2009-05-11T12:25:46-10:00	2009-05-11T12:25:46-10:00	psphot supports the -modeltest X Y ... option, which allows a specified model to be tested against an input image at a specific location.  Summary information for the fit is printed, but the 'usual' .cmf table is not created.  I request that a 1-row .cmf file be produced containing the usual information for the tested model.	welling
	1257	relphot solution applied to diff mags	DVO		enhancement	eugene	new	2009-05-21T12:55:34-10:00	2009-05-21T12:55:34-10:00	When relphot is run by DVO on the static image catalog to normalize the zero points between exposures, the solution should be applied to the corresponding difference image catalog.	suvi@…
	1240	Calling psconfig with non-existing option (e.g. -build) shouldn't unset PATH	psconfig		defect	eugene	new	2009-05-05T10:55:03-10:00	2009-05-05T10:55:03-10:00	"Sometimes I mistype or misguess an option to psconfig, e.g.
{{{
psconfig -build
}}}
After issuing that command, PATH is empty, and the shell I'm in has become useless. While that's probably mostly only a nuisance, maybe worse things could happen by accident, and it always breaks the flow of whatever one is doing at the time. Ideally, psconfig would complain about invalid options being passed."	Sebastian Jester
	1242	Ohana: Multiple groups not respected	Ohana	current	defect	eugene	new	2009-05-08T17:00:43-10:00	2009-05-08T17:00:43-10:00	"libohana/src/findexec.c checks the group using:

{{{
gid = getgid();
if ((gid == filestat.st_gid) && (filestat.st_mode & S_IRGRP) && (filestat.st_mode & S_IWGRP)) {
   // Can read
}
}}}

However, this neglects the possibility that the user is a member of a group besides his main group that does have access to the file.  All groups the user is a member of can be retrieved with getgroups(), and these should all be tested individually.
"	Paul Price
	1252	dvo par, dpar set to RA, Dec instead of NULL, 0, or	DVO		defect	magnier	new	2009-05-19T09:47:29-10:00	2009-05-19T09:47:29-10:00	"After using 'addstar' to put together a catdir from redone psphot calculations of MD01 2008/10/02 data and running relphot (see script below), I find that the 'par' and 'dpar' values are just set to the RA, Dec values.  It seems like they should be set to 0, Null until relastro is run.  What is the proscribed value before astrometric solutions are calculated?

---
#/bin/sh

obs=36475
exp=o4741g0234o

for obs in 36475 36476 36477 36478 36479 36480 36481 36625 ; do
    for cmf in `ls $obs/chip/*.cmf` ; do
        addstar -D CATDIR md01_update -D CAMERA gpc1 -accept-astrom $cmf 
    done
done

addstar -resort -D CATDIR md01_update -D CAMERA gpc1
relphot -averages -update -D CATDIR md01_update -D CAMERA gpc1 

---
[wwoodvas@serenity astrometry] dvo -D CATDIR md01_20081002
no file /Users/wwoodvas/.dvorc

Welcome to DVO - desktop virtual observatory


dvo: region 35.90 -4.25 2
dvo: mextract -region xccd, yccd, mag, mag:err, ra, dec, time, ra:ave, dec:ave, ra:err, dec:err, objID, catID, photcode, fwhm, mag:cat, phot_flags, ura, udec, par, dpar, nmeas, nmiss
reading images (mode DVO_MODE_MEF)
dvo: $i = 100
dvo: echo ra:ave[$i] ra[$i] par[$i]
34.6246127337 34.6246127337 34.6246127337

"	Michael Wood-Vasey
	1206	in pantasks, non-zero exit status should not always be a failure	PanTasks		enhancement	eugene	new	2009-03-20T12:32:23-10:00	2009-03-20T12:32:23-10:00	"if we want to use the exit status as a signaling mechanism (see neb-admin), then we should be able to specify certain non-zero exit status values as success.  this could be accomplished by adding an argument to the exit handler; if it is not specified as a failure, it would be a success (or maybe vice versa).

(alternatively, we would need to specify an out-of-band communication method)."	eugene
	1212	dvo.site should be generic	DVO		defect	eugene	new	2009-03-31T11:49:01-10:00	2009-05-14T01:22:28-10:00	"Since the dvo.site is supposed to be copied into the home directory as .ptolemyrc, it needs to be able to select different builds.  With the .ipprc file, this is done using environment variables (e.g., $PSCONFDIR)

This is not a big deal at the moment, since the builds have been consistent for the DVO files."	Paul Price
	1251	psphotTest doesn't compiled under -Werror w/ optimized	psphot	current	defect	magnier	reopened	2009-05-14T15:20:51-10:00	2009-05-14T16:05:58-10:00	"Compiling r25188 with

psbuild -rebuild -dev -clean -optimize

the compilation runs through fine until it gets to:

cc -std=gnu99 -DHAVE_CONFIG_H -I.    -I/Volumes/data/PS1/code/ipp-2.8/ipp-2.8.darwin/include -D_DARWIN_C_SOURCE   -D_DARWIN_C_SOURCE -D_THREAD_SAFE -DHAVE_PSDB -I/Volumes/data/PS1/code/ipp-2.8/ipp-2.8.darwin/include/psmodules -I/Volumes/data/PS1/code/ipp-2.8/ipp-2.8.darwin/include -I/usr/X11/include -I/Volumes/data/PS1/code/ipp-2.8/ipp-2.8.darwin/include/pslib -I/Volumes/data/PS1/code/ipp-2.8/ipp-2.8.darwin/include/mysql -I/sw/include   -D_THREAD_SAFE -DHAVE_PSDB -I/Volumes/data/PS1/code/ipp-2.8/ipp-2.8.darwin/include/pslib -I/Volumes/data/PS1/code/ipp-2.8/ipp-2.8.darwin/include/mysql -I/Volumes/data/PS1/code/ipp-2.8/ipp-2.8.darwin/include -I/sw/include   -pipe -O2 -g -DPS_NO_TRACE -Wall -Werror -MT psphotTest-psphotTest.o -MD -MP -MF .deps/psphotTest-psphotTest.Tpo -c -o psphotTest-psphotTest.o `test -f 'psphotTest.c' || echo './'`psphotTest.c
cc1: warnings being treated as errors
psphotTest.c: In function 'main':
psphotTest.c:52: error: statement with no effect
make[2]: *** [psphotTest-psphotTest.o] Error 1
make[1]: *** [all] Error 2
make: *** [all-recursive] Error 1

"	Michael Wood-Vasey
	1192	Why is psphot dependent on libmysql?	psphot	current	enhancement	eugene	accepted	2009-03-04T12:00:04-10:00	2009-03-25T04:11:25-10:00	"I just had a case where there was a problem with the mysql installation on one of the nodes on a cluster and psphot refused to run.  Conceptually this seems wrong.  psphot shouldn't know anything about libmysql.  Why is it required?

I specifically ask because libmsysql is one of the things people have the most trouble compiling and if we'd like the tools in IPP to be as widely used as possible it would be nice to strip mysql and other related functionality that make sense in the standard IPP installation but not just for someone who wants to do photometry."	Michael Wood-Vasey
	1238	Would be nice to be able to ctrl-z out of pantasks	DVO	current	enhancement	eugene	new	2009-05-05T02:02:26-10:00	2009-05-05T02:02:26-10:00	"Often I find myself needing to do something in the shell in the middle of a pantasks session. Pressing ctrl-z (by instinct) just yields ""got signal: 20"". I can of course do things like !xterm & but diagnosing errors is often easier in the same shell where pantasks is running. Is there a strong reason for trapping ctrl-z in pantasks? If not, it would be nice to be able to suspend it."	Sebastian Jester