Python bindings development

There are two ways to build OGS with Python bindings:

• A regular build with OGS_USE_PIP=ON (preferred method)
  This creates a virtual environment inside your build-directory and installs test dependencies into it.
• A wheel build

Regular build

• Configure your build with -DOGS_USE_PIP=ON
• Build

To run the Python-based tests:

• Source build-dir/.envrc (i.e., run source build-dir/.envrc in your terminal. This activates the virtual environment and sets OGS_USE_PATH environment variable).
• Run e.g. pytest from inside the source directory. On Windows run Invoke-Expression build-dir/.envrc.ps1 in your PowerShell.
• Run e.g. pytest from inside the source directory or pytest ../path/to/source/dir.
• Alternatively you can also run the Python-based with ctest, e.g. ctest -R pytest.

If you make modifications on the C++ side you need to run make or ninja in the build directory again. If you modify Python code you need to run cmake . again in the build directory. Modifications on the Python tests are immediately available to pytest.

To get the output of a specific test:

pytest --capture=tee-sys ../../ogs/Tests/Python/test_wrapped_cli_tools.py

# Create a virtual environment inside your source directory
python3 -m venv .venv
# Activate the environment
source .venv/bin/activate
# Install (build) the local Python project
pip install -v --group dev-all
...
Successfully installed ogs-6.4.2.dev1207

# To build with additional CMake arguments, e.g.:
pip install -v --group dev-all --config-settings=cmake.define.OGS_BUILD_PROCESSES=SteadyStateDiffusion"

Testing with pytest

# Run python tests
pytest ../../ogs  # path to your ogs source code
============================================== test session starts ===============================================
platform darwin -- Python 3.10.6, pytest-7.1.3, pluggy-1.0.0
rootdir: ~/code/ogs/ogs, configfile: pyproject.toml, testpaths: Tests
collected 2 items

../../ogs/Tests/Python/test_cli.py .                                                                       [ 50%]

...

# Start the python interpreter
python3
>>> import ogs.mesh as mesh
>>> # The following works in a wheel build only:
>>> import ogs.simulator as sim
>>> sim.initialize(["", "--help"])

If you make modifications on the C++ side you need to run make (regular build) or pip install --group dev-all (wheel build) again. Modifications on the Python tests are immediately available to pytest.

Module structure

Python modules added in CMake via pybind11_add_module() should only contain the binding code and other (helper code) which is used by that module only! If you need helper code which is used by several modules (e.g. OGSMesh-class used in mesh- and simulator-module) it needs to be defined in a regular library and linked to the modules.

If you don’t do this you will get unresolved externals between the modules and when you try to link them together you will get runtime errors or it doesn’t link at all (at least on Windows, because of the .pyd library format).

CI

For generating the various wheels for different Python versions and platforms cibuildwheel is used.

You can test it locally with, e.g. only building for Python 3.10:

CIBW_BUILD="cp310*" uvx cibuildwheel

Please note that on Linux cibuildwheel runs the builds inside manylinux Docker containers. On other platforms the build happens with native tools. See the cibuildwheel docs for more information.

Wheels are generated in the wheelhouse/-folder.

cibuildwheel is configured in pyproject.toml:

[tool.cibuildwheel]
...