In a nutshell
Singularity automatically binds the current folder to its internal user folder. Depending on what is in the current folder, the behaviour of the programs inside the container could be altered, e.g. if your current folder contains Python libraries with different versions than the one setup in the container.
To prevent such issue to occur, it is recommended to explicitely bind what you need in the container and prevent the automatic binding.
Binding folders and automatically bound folders
Containers are a great tool to ensure reproducibility of your experiments because their content is read-only. However, it is very likely that you need to folders in which data can be modified or created, e.g. input and output folders. To do so, Singularity allows you to specify a list of external folders (not originally in the container) which will be accessible from inside the container. Specifying which external folder will be accessible in the container at a specific location is calling binding.
For example, if you need the container to access a folder from your home directory, e.g., /home/users/jschleich/my-folder
you could do it as follows:
singularity run -B /home/users/jschleich/my-folder:/inside-folder container.sif
In the above example, the content of folder /home/users/jschleich/my-folder
will be available in the container at the following location /inside-folder
.
For more bind example, you can have a look at the Singularity documentation.
By default, Singularity will automatically bind some folders, notably the current directory which will be bound to the container user directory. As highlighted in the next section, this feature can lead to difficult to notice behaviour alteration or even errors.
Example of problem with the automatic binding
For this example, I am using a TensorFlow container and I want it to execute a Python script which will use the TensorFlow package.
However, when I try to run it, I encounter the following error:
singularity exec tensorflow2.sif python tensorflow2-test.py
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
The problem here is that Singularity will automatically bind the folder I currently in, e.g. my HPC home folder, inside the container home user. Because I happen to have some Python libraries in my home folder, they are now also accessible in the container which in turn will use them instead of the one it was built with.
In order to prevent such error to occur, we have to explicitely tell Singularity not to bind the current folder via the --no-home
option. The consequence is however that we do not have access to the tensorflow2-test.py
file inside the container anymore. To allow access again, we explicitely bind the folder container the Python script to an arbitrary internal folder, in the following example, we bind to to the /inside-folder
folder:
singularity exec --no-home \
-B folder-container-my-script:/inside-folder \
tensorflow2.sif python /inside-folder/tensorflow2-test.py
Alternatively, you can bind a specific file only instead of the folder:
singularity exec --no-home \
-B tensorflow2-test.py:/tensorflow2-test.py \
tensorflow2.sif python /tensorflow2-test.py