Advanced Scientific Computing Research (ASCR) under the U.S. Department of Energy Office of Science (Office of Science)

Project Team Members:

Northwestern University

The HDF Group

Argonne National Laboratory

North Carolina State University

Northwestern University - EECS Dept.

Damsel Programming Model

A Typical Damsel programming flow for creating a file consists of the followings.
  1. Initialize the Damsel environment
  2. Create a Damsel model
  3. Create a Damsel container
  4. Add tags to the handles in a containers
  5. Create a colelction of containers
  6. Map the model to a file
  7. Commit I/O to the file
  8. Close the model
  9. Exit the Damsel environment
Figure below illustrates the objects used by Damsel and their relationships.

Initialize the Damsel environment

The Damsel environment is initialized by calling the following function.

damsel_err_t DMSLlib_init(damsel_library *lib);

return to top

Create a Damsel model

A Damsel model is an in-memory store that describes the data model from the application perspective. One Damsel file contains one model. It is common practice to attach file name and file access hints right after a model is created. The following API is used to create a model. (The only supported 'app_handle_type' at this time is DAMSEL_HANDLE_TYPE_HANDLE64.)

damsel_err_t DMSLmodel_create(damsel_handle_type app_handle_type, damsel_model *model);

The APIs below are used to set the file name and file create mode.

damsel_err_t DMSLtrait_create(damsel_trait_bundle *tb);

damsel_err_t DMSLtrait_put(damsel_trait_bundle     tb,
                              char                *key,
                              char                *value);

damsel_err_t DMSLmodel_attach(damsel_model         model,
                              const char           *filename,
                              MPI_Comm             comm,
                              damsel_trait_bundle  trait);

damsel_err_t DMSLtrait_release(damsel_trait_bundle tb);

Example usage:

    damsel_trait_bundle tb;
    DMSLtrait_put(tb, "mode", "WRITE");
    DMSLmodel_attach(model, filename, MPI_COMM_WORLD, tb);

return to top

Create Damsel containers

A Damsel container is a holder of a set of Damsel handles. A Damsel handle is an integral number that refers to an entity or a collection of entities. Entities are application objects, such as vertices and edges of a grid. The set of Damsel handles in a container can be of a list of contiguous integral values or noncontiguous, independent values.

Containers aren't really part of the Damsel data model, i.e., they are not stored in the files. Instead, they are used to group handles, so we can pass into other Damsel APIs. Damsel library collects containers from all processes and integrate them into a single view data model in file. Damsel containers are like MPI fileviews that are defined independently by each MPI process. They are used to tell the I/O library (Damsel and MPI-IO) how the local defined data should be mapped to the files. The per-process information (containers/fileviews) will not appear in the file contents.

The two container constructors below are used to create a container with handles of a sequence of contiguous integral values and a vector of independent values, respectively.

    DMSLcontainer_create_sequence(damsel_model      model,
                                  damsel_handle     base,
                                  unsigned int      quantity,
                                  size_t            step,
                                  damsel_container  *container);

damsel_err_t DMSLcontainer_create_vector(damsel_model        model,
                                         damsel_handle    *values,
                                         /* [num] */
                                         unsigned int      num,

return to top

Add tags to the handles in a container

One can tag the handle set in a container with a name, I/O buffer, and data type of data in the buffer. A tag can be considered as a solution variable, for example, associated to all vertices of a grid. A tag is represented by a name that must be uniquely defined in a model, a data type, and an I/O buffer pointing to the memory space where the data is stored. Below are the primitive data types predefined in Damsel.


We give each buffer element in memory an identifier, or "handle". For example, if the buffer were an array of 5 integers one would assign each of those array elements a handle. In this case, a handle can be a "memory address". Or in mesh land it'd be some mesh-land entity ID (I am vertex 5 of five billion).

The APIs below are used to define a tag and tag it to the top-level handles in a container.

damsel_err_t DMSLtag_define(damsel_model      model,
                            damsel_handle    *tag_handle,
                            damsel_data_type  type,
                            const void       *buffer);
damsel_err_t DMSLmodel_map_tag(void          *buffer,
                            damsel_container  container,
                            damsel_handle    *tag_handle);

return to top

Create a collection of containers

A container can also contain other containers. This is achieved by first "converting" a Damsel container to a Damsel collection. A Damsel collection is of type Damsel handle, so a set of collection handles can be used to create a container. Thus containers can form a hierarchical structure. However, a tag can only be created for the top-level handles of a container. The collection constructor API is:

damsel_collection DMSLcoll_create(damsel_model model,
                                  damsel_handle   *coll_handle,
                                  damsel_container container,

return to top

Map the model to the file

Once a model is defined, the corresponding file-side store must be mapped and created. This is achieved by making the calls to the two APIs below for all containers and tag handles created in the model.

    DMSLmodel_map_handles_inventing_file_handles(damsel_container container);

    DMSLmodel_map_single_handle_inventing_file_handle(damsel_model model, 
                                           damsel_handle tag_handle);

return to top

Commit the I/O to the file

In Damsel, data transfer from memory to a file is done in a either blocking or nonblocking fashion. For nonblocking, once the transfer begins, I/O buffers registered in the model are expected not to be changed, until the transfer is completed. The three APIs below are used for blocking transfer, starting the I/O for nonblocking, and waiting for completion, respectively.

damsel_err_t DMSLmodel_transfer_sync(damsel_model model,

damsel_err_t DMSLmodel_transfer_async(damsel_model model,

damsel_err_t DMSLmodel_wait(damsel_request_t  transfer_request,
                            damsel_status_t  *request_status);

return to top

Close the model

To free up the memory used in model construction, the following APIs are used to close models and containers. A program may release containers as soon as they are passed to Damsel and need not maintain a reference until after I/O completes.

damsel_err_t DMSLmodel_close(damsel_model model);

damsel_err_t DMSLcontainer_close(damsel_container container);

return to top

Exit the Damsel environment

All resource allocated by Damsel internally will be freed once exiting the environment.

damsel_err_t DMSLlib_finalize(damsel_library lib);

return to top

Northwestern University EECS Home | McCormick Home | Northwestern Home | Calendar: Plan-It Purple
© 2011 Robert R. McCormick School of Engineering and Applied Science, Northwestern University
"Tech": 2145 Sheridan Rd, Tech L359, Evanston IL 60208-3118  |  Phone: (847) 491-5410  |  Fax: (847) 491-4455
"Ford": 2133 Sheridan Rd, Ford Building, Rm 3-320, Evanston, IL 60208  |  Fax: (847) 491-5258
Email Director

Last Updated: $LastChangedDate: 2014-09-17 14:51:09 -0500 (Wed, 17 Sep 2014) $