No methods available - ESRF Jupyter-slurm

[jadball]

Hello,

I'm running into issues trying to use the method_registry module of pyFAI on the ESRF Jupyter-slurm machines:

import pyFAI
from pyFAI.method_registry import Method, IntegrationMethod
print("pyFAI version:", pyFAI.version)
print("Number of ways of performing integration:", len(pyFAI.method_registry.IntegrationMethod.list_available()))

This yields the following:

pyFAI version: 2025.3.0
Number of ways of performing integration: 0

This is on the default Jupyter env at the ESRF these days:

/cvmfs/hpc.esrf.fr/software/packages/linux/x86_64/jupyter-slurm/2025.04.5/envs/jupyter-slurm/bin/python
which is Python 3.12.9

[EdgarGF93]

Hi, James. Maybe I can address this one, the methods are 'manufactured' at the moment of importing the correct module, so if you import first directly the method_registry, nothing is created yet. If you import first (for example), from pyFAI.integrator.azimuthal import AzimuthalIntegrator, you get all the available methods. At the end, this is the module you want to load:

from pyFAI.integrator import load_engines

[kif]

The answer of Edgar is perfectly correct.

Just to comment of the reason for the choice of lazy-loading of all integrator engines:

There are dozens of sub-modules in pyFAI containing "rebinning engines", some of them are extensions (binary modules), some of them are implemented in OpenCL and when they get loaded, pyFAI checks all devices and registers the one which are useable for processing. This takes several seconds.

Previously, it was done during the call to the integrate method, but this is inefficient in a performance critical loop.
Then this check was performed at the import of pyFAI, but I got complains that import pyFAI was taking too long, especially when performed on hundreds of times on a cluster, and those user knew precisely what they wanted to perform.

The Lazy loading of today is simply a compromise between those two extremes.

[kif]

I will switch this issue to a discussion so that it remains available for the record. Closing it makes little sense.