
2005.10.06
	split / nosort / update
	I have added a few new concepts to addstar recently: split
	catalog files, nosort for the measurement table, and
	update-only.

	split mode

	The split mode is quite straightforeward.  In this mode, each
	catalog is represented by a set of four files: *.cpt, *.cpm,
	*.cpn, *.cps.  Each file contains only one FITS table of the
	data, along with basic header and empty matrix.  Having
	individual tables for each component of the database lets me
	add entries without re-writing the entire table.  This should
	save on I/O operations in the long run.  

	The first file contains the table of averages, and is the file
	normally identified in the table lookup functions.  The header
	of this file contains the names of the other table files
	(paths relative to the directory containing the cpt file).
	The names and extensions are specified in 'mkcatalog.c'; all
	other functions use the defined filename references, rather
	than expecting a naming convention.

	The additional files contain the measures (cpm), missings
	(cpn), and secfilt (cps) elements of the catalog tables.  

	To facilitate the handling of the additional filenames, file
	pointers, and headers, Catalog was extended to include
	pointers to the measure, missing, and secfilt files as
	additional catalogs.  When the data are loaded into memory,
	these catalogs are locked (as usual), and file information is
	stored in the individual Catalog entries; the data segements
	are all loaded into the main catalog pointers (eg, measures
	are loaded into catalog[0].measure, rather than
	catalog[0].measure_catalog[0].measure).
	
	The function 'load_catalog' auto-recognizes the SPLIT format
	by looking for the header keyword MEASURE, identifying the
	file containing the measures.  The identification of the RAW
	format and the SPLIT format are not cross-checked: if the
	NAXIS keyword is set to 2, the file is assumed to be RAW, even
	if the MEASURE keyword is present.  Careful with this (though
	there is no reason the main matrix should be used in a basic
	database table).

	nosort 

	the nosort option by itself provides a minor processing
	speed-up by deferring the re-sorting of the measurement table
	until after multiple addstar processes are run.  addstar
	should not require the measurements to be sorted, so this step
	can be safetly deferred if only addstars are being performed.
	the other DVO operations require the sorted table, so the sort
	must be performed before they are run (either as part of the
	catalog load, not implemented yet, or with a call to addstar
	without the -nosort option set.  the real goal of the nosort
	option is to enable the -update concept in addstar, in which
	only the new rows are written out; this will only work if
	addstar can handle unsorted measures. 

	the nosort option required the addition of a 'sorted' element
	in the Catalog structure to track if the data are sorted or
	not.  On load, this flag is set based on the value of the
	header keyword SORTED; if the data is sorted during addstar,
	the flag is appropriately set, otherwise it is set FALSE be
	default.  

	The nosort option requires a function which can generate the
	'next_meas' link sequence based on the measure table.  there
	is now a function called 'build_meas_link' which generates a
	correct link list; there is also the pair of functions
	'init_meas_links' and'init_miss_links' to generate the links
	in the event that the table is sorted (should be must
	quicker).

	The 'missing' table is problematic: the LONEOS and ELIXIR
	formats do not carry an averef entry, thus they do not have
	enough information to define the links based only on the
	missing table.  This means we are forced to write out a sorted
	missing table; the nosort option is invalid for the missing
	table.  One future upgrade path is to add the averef entry to
	the PANSTARRS format and then only require the missing table
	to be sorted if the format is old and does not support
	-nosort.  (Note also that, for the moment, the missing table
	has only a single valid format).

	In the process of defining the nosort option, I also cleaned
	up a bit the find_matches functions to use clearer functions
	for the links.  
	
	update

	The 'update' process in principle allows addstar to
	substantially reduce the amount of I/O it needs to perform by
	only requiring addstar to write out new measures and new
	average/secfilt entries.

	The 'missing' table is problematic: since the format does not
	support the 'nosort' option, it is not possible to use update
	with the missing table.  This means we are forced to write out
	a complete, sorted missing table.  This is currently
	implemented in update_catalog_split by simply writing out the
	complete missing table.  In fact, this choice is still flawed
	because the average table, since it is not written out in full
	each time, is inconsistent with the missing table: the Nn
	entries for each average, which identifies the number of
	missing entries, are not updated.  In practice, this means
	that the -update option forces the use of the -missed option,
	though at the moment, this is not forced or checked in any
	way.

	Note that the 'missed' table contains duplicate information
	and can, in principle, be completely regenerated at any time.
	This should be an addstar option: to re-construct the missing
	table, potentially with constraints on the images which are
	searched for matches.  

2005.10.04
	- moved measure/missing list manipulation to separate functions
	- added concept of sorted / unsorted measure catalog
	- defined build_meas_links and reorder_measure,missing
	- some cleanup of both find_matches.c and find_matches_closest.c

2005.10.03:
	- dropping GSCRegion *region entry from find_matches (unused!)
	- adding function find_matches_closest (alternate matches)

2005.08.19:
	changed load_photcode to handle CATMODE and CATFORMAT variations
	- addstar.h: added CATMODE and CATFORMAT globals
	- ConfigInit: read CATMODE and CATFORMAT from config
	- gcatalog: set catalog.catmode from CATMODE
	- mkcatalog: set CATFORMAT and CATMODE for new catalog
	- wcatalog: set CATFORMAT for new catalog
	
	using full photometry conversions in find_matches
	added SetZeroPoint to gstars to enable phot conversions
	
2005.08.15:
	cleanup of the minor Wall,Werror messages

2005.07.06 : current release is addstar-1.1

	I have made a variety of fairly substantial changes since
        v1.0.  I have added autocode for the basic DVO data types.  I
        have merged the functionality of addspphot, addrefs, and
        addusno into the single addstar function.  There are now
        options to add an ascii text table or stars from known
        external reference catalogs (HST GSC, USNO, 2MASS).  Also, the
        functions added for skyprobe (in-flight Mcal calculation,
        color terms) are also incorporated.  I also substantially
        re-organized the basic code to make the different modes more
        sensible.  There are still a few changes needed for addstar
        v2.0 (which will be able to handle the pan-starrs bandwidth).
        specifically, addstar needs to handle the catalog files using
        the TBD sky.db interfaces, it needs to interact with a DVO
        server in a client/server way (to divide up the sky), it needs
        to handle multiple versions of the table formats, and it might
        need to know about the mysql backend interface.  

2005.02.24 : I am adding the mosaic two-level astrometry information to the Image table.

