Building snakemake 9

User groups EasyBuild
1

The poetry version shipped with EasyBuild 2024a toolchain, is now quite old.

Typical errors that appear when compiling with an outdated version of Poetry:

Preparing metadata (pyproject.toml) did not run successfully.
   exit code: 1
> [26 lines of output]
      Traceback (most recent call last):
        File "/work/projects/dlsm-soft/develop/software/poetry/1.8.3-GCCcore-13.3.0/lib/python3.12/site-packages/poetry/core/pyproject/toml.py", line 59, i
n poetry_config
          config = tool["poetry"]
                   ~~~~^^^^^^^^^^
      KeyError: 'poetry'

We will try to build a relatively new version of snakemake that is supported by the Poetry released with 2024a. We should be able to build new versions as soon as we install a newer version of Poetry, and possibly uv_build.

@aginolhac Was the Apptainer issue resolved in v9.0.0?@aginolhac Was the Apptainer issue resolved in v9.0.0?

1 Like
2

Thanks Geogios for this, really much appreciated.
I replied to the ticket serviceNow that Apptainer 1.4.1 fails independently of snakemake . Could we have version 1.4.0 in release2024a please?

3

@gkaf thanks for fixing the issue with Apptainer 1.4.1, it works. snakemake v9.x works with it but it seems that notebook is missing in the install of the module. Could you try adding it please?

4

The compilation of noteboot 7.1.3 fails with the error:

== 2025-10-21 09:28:00,264 run.py:627 INFO Output of '/opt/apps/easybuild/systems/aion/rhel810-20251006/2024a/epyc/software/Python/3.12.3-GCCcore-13.3.0/bin/python ...' shell command (stdout + stderr):
Processing /tmp/easybuild/aion/2024a/epyc/build/snakemake/9.3.5/foss-2024a/notebook/notebook-7.1.3
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [18 lines of output]
      Traceback (most recent call last):
        File "/opt/apps/easybuild/systems/aion/rhel810-20251006/2024a/epyc/software/Python/3.12.3-GCCcore-13.3.0/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/opt/apps/easybuild/systems/aion/rhel810-20251006/2024a/epyc/software/Python/3.12.3-GCCcore-13.3.0/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/apps/easybuild/systems/aion/rhel810-20251006/2024a/epyc/software/Python/3.12.3-GCCcore-13.3.0/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 152, in prepare_metadata_for_build_wheel
          whl_basename = backend.build_wheel(metadata_directory, config_settings)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/apps/easybuild/systems/aion/rhel810-20251006/2024a/epyc/software/hatchling/1.24.2-GCCcore-13.3.0/lib/python3.12/site-packages/hatchling/build.py", line 58, in build_wheel
          return os.path.basename(next(builder.build(directory=wheel_directory, versions=['standard'])))
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/apps/easybuild/systems/aion/rhel810-20251006/2024a/epyc/software/hatchling/1.24.2-GCCcore-13.3.0/lib/python3.12/site-packages/hatchling/builders/plugin/interface.py", line 116, in build
          configured_build_hooks = self.get_build_hooks(directory)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/apps/easybuild/systems/aion/rhel810-20251006/2024a/epyc/software/hatchling/1.24.2-GCCcore-13.3.0/lib/python3.12/site-packages/hatchling/builders/plugin/interface.py", line 386, in get_build_hooks
          raise UnknownPluginError(message)
      hatchling.plugin.exceptions.UnknownPluginError: Unknown build hook: jupyter-builder
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

It seems that we need to add one more dependency, jupyter-builder. Trying now.

5

@gkaf I solved it on my laptop with Apptainer 1.4.3 and snakemake 9.13.3. The root cause is to align the snakemake version that is installed on the machine AND in the docker image.
Upgrading the docker images it works now. I tried on HPC with

module load env/development
module load tools/Apptainer
module load tools/snakemake

But I have the old 8.9.0 so I am running into a new Attribute error (a new one)

AttributeError: Can’t get attribute ‘Namedlist._used_attribute’ on <module 'snakemake.io

Can you restore the snakemake you had last Friday? Version 9.3.5 if I am not mistaken using 2024a?

6

So, no need for the notebook package then. I am reverting to the original script. We can debug latter in the pull request in EB why the package fails to build, if it’s required.

7

thanks a lot @gkaf ! I can confirm it works with snakemake v9.3.5 and a apptainer container that contains snakemake 9.13.3, I can get my test template running and the final report too! Thanks a lot, much appreciated.

8

For Apptainer, one must be careful to be in the path without symbolic link if he/she wants the automatic bind of the current folder to the working folder in the container.

9

@aginolhac We created a PR on the EasyBuild repository. Do you have any connection in the development team of Snakemake? Could they review our PR?

1 Like
10

@gkaf the module as it is now works perfectly. I will need with some students again on the 13th of November. I don’t know people in the snakemake team but we can open an issue on their repo, we might have someone interested

11

Understood, we won’t edit the module before the 17th of November then! This means the we will miss the 5.2 release of EasyBuild, but it’s probably for the best since it allows us to contact the developer. I will open an issue on their repo and add a link here.

1 Like