Skip to content

refactor: drop the need to use importlib#61

Open
12rambau wants to merge 1 commit into
r-earthengine:masterfrom
12rambau:files
Open

refactor: drop the need to use importlib#61
12rambau wants to merge 1 commit into
r-earthengine:masterfrom
12rambau:files

Conversation

@12rambau

Copy link
Copy Markdown
Contributor

To reach the file system you are using importlib.resources which is basically reading the file tree that exists within the wheel. I was trying to restore functionalities of ee_extra in geetools and I got many failures related to this mechanism (https://github.com/gee-community/geetools/actions/runs/16901688812/job/47882812551?pr=467)

geetools/ee_image.py:1041: in preprocess
    return ee_extra.QA.pipelines.preprocess(self._obj, **kwargs)
.nox/ci-test/lib/python3.9/site-packages/ee_extra/QA/pipelines.py:37: in preprocess
    x = maskClouds(x, **kwargs)
.nox/ci-test/lib/python3.9/site-packages/ee_extra/QA/clouds.py:341: in maskClouds
    platformDict = _get_platform_STAC(x)
.nox/ci-test/lib/python3.9/site-packages/ee_extra/STAC/utils.py:21: in _get_platform_STAC
    eeDict = _load_JSON()
.nox/ci-test/lib/python3.9/site-packages/ee_extra/utils.py:19: in _load_JSON
    data_file = files("ee_extra.data") / x
/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/importlib/resources.py:147: in files
    return _common.from_package(_get_package(package))
/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/importlib/_common.py:14: in from_package
    return fallback_resources(package.__spec__)
/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/importlib/_common.py:18: in fallback_resources
    package_directory = pathlib.Path(spec.origin).parent
/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/pathlib.py:1000: in __new__
    self = cls._from_parts(args, init=False)
/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/pathlib.py:625: in _from_parts
    drv, root, parts = self._parse_args(args)

TypeError: expected str, bytes or os.PathLike object, not NoneType

I don't really know how to debug this and I personnaly think relying on importlib is an overkill. In this PR I propose a method that only rely on pathlib to find all the files we need to perform the computation. It will work just as pckg_resources as long as you are not editing the pakg file names when building the wheel (whic is not the case now)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant