There is a desire to organize the datastores for PV3 into chunks that are of a similar size of data, organized by dxlayer. This describes the tessellation.

Step 1

Subdivide into the 32 layers. Each of the 32 layers has an equal amount of data in each layer (so, the thickness of the layers will vary, in order to increase/decrease the amount of data.

Step 2

There are 8 DXlayer machines, each dxlayer machine gets 4 of the layers.

Step 3

For each set of 4 layers, called dxlayer[num], subdivide those to have segments of equal amount of data. This example here only has 8 segments per dxlayer[num]. Also, note that everything is divided in integer units of ra/dec (no fractional degrees)

Step 4

Each of these segments corresponds to a datastore. Each datastore is of the same size, and roughly maps to 1 dxlayer machine.

Final Construction

For OB/FO/DO/DF, because these are small batch sizes, we will subdivide each dxlayer[num] into 6 segments, giving a total of 48 datastores. ST/FG will have more than Object/Diff types of batches, so will divide each dxlayer[num] into 12 segments, giving a total of 96 datastores. P2/FW are the heaviest, and will be subdivided into 24 segments per dxlayer, for a total of 192 datastores.

The goal is to have PSPS load 8 datastores per merge (or possibly a multiple of that).

The names of the datastores will follow this format: PSPS_PV3_[batch]_dx[dxlayer]_seg[num], an example would be PSPS_PV3_OB_dx1_01

Expectations for the different batch types

type	dxlayers	segments per dxlayer	total datastores	load order
OB	8	6	48	1st
ST	8	12	96	2nd
P2	8	24	192	3rd
FW	8	24	192	3rd
FO	8	6	48	1st
FG	8	12	96	2nd
DO	8	6	48	last
DF	8	6	48	last