hdf5

Type Members

1. trait Hdf5Common extends AnyRef

   

   Common functionality used by Hdf5 object savers and restorers

2. class Hdf5ObjectRestorer extends ObjectRestorer with Hdf5Common

   

   Cog ComputeGraph restorer based on HDF5

   Cog ComputeGraph restorer based on HDF5

   See Hdf5ObjectSaver for more on the approach used to save/restore ComputeGraphs.

3. class Hdf5ObjectSaver extends ObjectSaver with Hdf5Common

   

   Cog ComputeGraph saver based on HDF5

   Cog ComputeGraph saver based on HDF5

   First realize that HDF5 provides considerable flexibility in how ones "web of objects" gets mapped to the HDF5 concepts of groups, datasets, attributes, etc. We have taken the approach of not using attributes and mapping the objects to groups and datasets as follows:

   - Primitive values (Ints, Floats, Strings, etc.) are stored each in their own dataset, which is named by the writeXXX() call that wrote the primitive. - Arrays of primitive values are similarly stored each in their own named dataset. Only 1D Arrays are currently supported and the length of the array is stored and can be retrieved from the dataspace of the dataset. - Single Objects are created as a group (which is like a directory in the filesystem analogy) with the name given in the writeObject() call. The primitive data members of the object are stored in datasets that are within the object's group. - Arrays of Objects are created with a two-level group hierarchy. First a group is created that has the name provided by the writeObjectArray() call (e.g. "fields"). Then a special dataSet called "fieldsSize" is created within this group to hold the integer length of the array. Finally, each object of the array is stored in the group under a name that includes the array index (e.g. "fields[3]"). Note that the storing of the individual array objects will add a second level to the group hierarchy.

   Note that this scheme permits a nesting of Objects and Arrays[Objects].

   By example, assume you had the following class:

   class Datapoint { val time: Int, val data: Float }

   and instance:

   val points = new Array[Datapoint](3)

   Assume further that the Datpoint class is set up as a "Saveable" that wrote out its fields with names "time" and "data".

   Then the call to writeObjectArray("points", points) would generate the following hdf5 datasets:

   points/pointsSize points/points[0]/time points/points[0]/data points/points[1]/time points/points[1]/data points/points[2]/time points/points[2]/data

   Note that under the hdf5 root group "/", the following groups were created:

   points points/points[0] points/points[1] points/points[2]