c++ - Creating extendable datasets in parallel HDF5 -

i trying write data hdf5 file in parallel. each node has own dataset, unique (although same size). trying write them separate datasets in hdf5 file in parallel. catch later may want overwrite them different size datasets (different size compared original set of datasets --- dataset on each processor same size). know how this?

(code relies on boost , eigen)

i have code first open file:

boost::mpi::environment env(argc, argv);  // set info hdf5 , mpi mpi_comm comm = mpi_comm_self; mpi_info info = mpi_info_null;  // set file access property list parallel i/o access hid_t plist_id = h5pcreate(h5p_file_access); h5pset_fapl_mpio(plist_id, comm, info);  // declare file id std::string filename = "test.h5";  // create file hid_t fileid = h5fcreate(filename.c_str(), h5f_acc_trunc, h5p_default, plist_id);  // close property list h5pclose(plist_id); 

then create , write datasets:

// mpi communicator unique_ptr<mpi::communicator> worldcomm(new mpi::communicator);  const eigen::vectorxd dataset = worldcomm->rank() * eigen::vectorxd::ones(3); const std::string name = "/vector";  // sleep bit processors doing different sleep(worldcomm->rank() * 2.0);  // sizes of data set const hsize_t dimsf[2] = {(hsize_t)dataset.rows(), (hsize_t)dataset.cols()};  // set maximum size of data set unlimited const hsize_t maxdim[2] = {h5s_unlimited, h5s_unlimited};  // size of each chuck --- there better way choose these numbers!?!?!?! const hsize_t chunkdims[2] = {2, 5};  // create dataspace dataset. const hid_t filespace = h5screate_simple(2, dimsf, maxdim);  assert(filespace>0);  // modify data set creation properties --- enable chunking const hid_t prop = h5pcreate(h5p_dataset_create); const hid_t status = h5pset_chunk(prop, 2, chunkdims);  // create dataset default properties each process std::vector<hid_t> dsetvec(worldcomm->size()); for( int i=0; i<worldcomm->size(); ++i ) {   const std::string datasetname = name+"_rank_"+std::to_string(i);    dsetvec[i] = h5dcreate2(fileid, datasetname.c_str(), h5t_native_double, filespace, h5p_default, prop, h5p_default); }  // create property list dataset write. const hid_t plistid = h5pcreate(h5p_dataset_xfer);  // write data file h5dwrite(dsetvec[worldcomm->rank()], h5t_native_double, h5s_all, h5s_all, plistid, dataset.data());  // close filespace  h5sclose(filespace);  // close datasets for( int i=0; i<worldcomm->size(); ++i ) {   h5dclose(dsetvec[i]); }  // close file h5fclose(fileid); 

what expect 4 datasets named "/vector_rank_i" (i=0,1,2,3) each of size 3 , valued [0, 0, 0], [1, 1, 1], [2, 2, 2], , [3, 3, 3], respectively. however, being produced 4 datasets named "/vector_rank_i" (i=0,1,2,3) each of size 3 values [0, 0, 0], [0, 0, 0], [0, 0, 0], , [3, 3, 3].

this exact code works if don't use chunking. however, since i'll need able extend datasets later less ideal. know work around?

before answering specific code, i'd know more why "one dataset per process" how chose decompose problem. seems mess if you're ever going scale beyond handful of processes.

you doing parallel i/o dataset, , have enabled mpi-io not collective access. that's unlikely yield terribly performance @ scale.

your chunk dims seem small me. i'd make them larger, "how large" depends on lot of factors. well, see how performance these values. maybe won't bad if turn on collective i/o?

those initial impressions aside, perhaps wish experiment hdf5. don't know why turning chunking on make datasets empty... unless writing nfs. if writing nfs, well, luck, buddy it's hopeless.