Next: , Previous: , Up: Files   [Index]

2.11 Leave Define Mode with Performance Tuning: ncmpi__enddef

The function ncmpi__enddef takes an opened netCDF file out of define mode. The changes made to the netCDF file while it was in define mode are checked and committed to disk if no problems occurred. Non-record variables may be initialized to a "fill value" as well. See ncmpi_set_fill. The netCDF file is then placed in data mode, so variable data can be read or written.

This call may involve moving (shifting) data under some circumstances. For instance, when opening an existing file and entering redefine more to add more metadata which causes file header to expands, all fixed and record variables will be moved to higher file offsets to make room for the new file header. Such operations can be very expensive. Users are encouraged to set h_minfree to reserve a sufficient large space for file header if it is expected to expand.

The current netcdf file format has three sections, the "header" section, the data section for fixed-size variables, and the data section for variables which have an unlimited dimension (record variables).

The header begins at the beginning of the file. The index (offset) of the beginning of the other two sections is contained in the header. Typically, there is no space between the sections. This causes copying (moving) overhead to accrue if one wishes to change the size of the sections, as may happen when changing names of things, text attribute values, adding attributes or adding variables.

The minfree parameters allow one to control costs of future calls to ncmpi_redef, ncmpi_enddef by requesting that minfree bytes be available at the end of the section.

The align parameters allow one to set the alignment of the beginning of the corresponding sections. The beginning of the section is rounded up to an index which is a multiple of the align parameter.

The file format requires a 4-byte alignment, so the align parameters are silently rounded up to multiples of 4.

The usual call of 

ncmpi_enddef(ncid);

is equivalent to 

ncmpi__enddef(ncid, 0, 0, 0, 0);

The file format does not contain a "record size" value, this is calculated from the sizes of the record variables. This unfortunate fact prevents us from providing minfree and alignment control of the "records" in a netcdf file. If you add a variable which has an unlimited dimension, the third section will always be copied with the new variable added.

Note the alignment hints set in the environment variable "PNETCDF_HINTS" have the highest precedence, which overwrite the alignment hints set in the MPI_Info object used in the call of ncmpi_create() and ncmpi_open(), which overwrite the alignment arguments, i.e. v_align and r_align, used in the call to ncmpi__enddef().

Operational Mode

This API is a collective routine: all processes must provide the same values for all arguments.

This API must be called while the file is in define mode.

Usage

int ncmpi__enddef(int        ncid,
                  MPI_Offset h_minfree,
                  MPI_Offset v_align,
                  MPI_Offset v_minfree,
                  MPI_Offset r_align);
ncid

NetCDF ID, from a previous call to ncmpi_open or ncmpi_create.

h_minfree

Sets the pad at the end of the "header" section.

v_align

Controls the alignment of the beginning of the data section for fixed-size variables. 1 means no alignment. 0 means to let PnetCDF chose the best value. This value is ignored if the PnetCDF hint nc_header_align_size is set.

v_minfree

Sets the pad at the end of the data section for fixed-size variables.

r_align

Controls the alignment of the beginning of the data section for variables which have an unlimited dimension (record variables). 1 means no alignment. 0 means to let PnetCDF chose the best value. This value can be set at the run time using PnetCDF hint environment variable nc_record_align_size.

Return Error Codes

ncmpi__enddef returns the value NC_NOERR if no errors occurred. Otherwise, the returned status indicates an error. Possible causes of errors include:

Example

Here is an example using ncmpi__enddef to finish the definitions of a new netCDF file named foo.nc and put it into data mode: 

#include <pnetcdf.h>
   ... 
int err, ncid, cmode = NC_NOCLOBBER | NC_64BIT_DATA;
MPI_Offset header_size, header_extent;
MPI_Offset h_minfree, v_align, v_minfree, r_align;
   ... 
err = ncmpi_create(MPI_COMM_WORLD, "foo.nc", cmode, MPI_INFO_NULL, &ncid);
if (err != NC_NOERR) printf("Error: %s\n",ncmpi_strerror(err));

   ...       /* create dimensions, variables, attributes */

h_minfree = 4194304;  /* 4 MB */
v_align   = 1048576;  /* 1 MB */
v_minfree = 4194304;  /* 4 MB */
r_align   = 1048576;  /* 1 MB */
err = ncmpi__enddef(ncid, h_minfree, v_align, v_minfree, r_align);  /*leave define mode*/
if (err != NC_NOERR) printf("Error: %s\n",ncmpi_strerror(err));

err = ncmpi_inq_header_size(ncid, &header_size);
if (err != NC_NOERR) printf("Error: %s\n",ncmpi_strerror(err));

err = ncmpi_inq_header_extent(ncid, &header_extent);
if (err != NC_NOERR) printf("Error: %s\n",ncmpi_strerror(err));

/* check if minimum free space for file header is honored */
assert(header_extent - header_size >= h_minfree);

Next: , Previous: , Up: Files   [Index]