Skip to content

Allow concurrent read-only access with HDF5 backend #67

Description

@rnhmjoj

There's no reason why two processes shouldn't be allowed to read the same database. Instead:

Process a:

>>> import imas
>>> a = imas.DBEntry("imas:hdf5?path=./inputs/pulse_schedule", "r")
>>> 

Process b:

>>> import imas
>>> b = imas.DBEntry("imas:hdf5?path=./inputs/pulse_schedule", "r")
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    b = imas.DBEntry("imas:hdf5?path=./inputs/pulse_schedule", "r")
  File "/lib/python3.13/site-packages/imas/db_entry.py", line 189, in __init__
    self._dbe_impl = cls.from_uri(self.uri, mode, self._ids_factory)
                     ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lib/python3.13/site-packages/imas/backends/imas_core/db_entry_al.py", line 84, in from_uri
    return cls(uri, _OPEN_MODES[mode], factory)
  File "/lib/python3.13/site-packages/imas/backends/imas_core/db_entry_al.py", line 62, in __init__
    status, ctx = ll_interface.begin_dataentry_action(uri, mode)
                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "imas_core/_al_lowlevel.pyx", line 318, in imas_core._al_lowlevel.al_begin_dataentry_action
imas_core.exception.ImasCoreBackendException: b'al_begin_dataentry_action: [ALBackendException = Unable to open HDF5 master file: ./inputs/pulse_schedule/master.h5]'
Error status=-3

It seems this is due to the fact IMAS-Core always tries to open the master file RW first, even if mode="r" is asked:

*file_id = H5Fopen(filePath.c_str(), H5F_ACC_RDWR, H5P_DEFAULT);

If the H5F_ACC_RDWR flag is set, HDF5 puts an exclusive lock on the file, preventing other processing from opening the DB, unless the HDF5 file locking mechanism is disabled.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions