Allow non-strict BIDS naming scheme

[lmichalke]

We tried to load the NARPS dataset with the following code. Pybids returns a list of 108 files, but with ancpbids the list is empty.

from ancpbids import BIDSLayout
#from bids.layout import BIDSLayout


layout = BIDSLayout('/data2/openneuro-ds/ds001734/derivatives/fmriprep/', validate=False)

test = layout.get(datatype="func", suffix="preproc", run=1, extension=[".nii.gz"],
                  space='MNI152NLin2009cAsym', return_type="file")

[erdalkaraca]

@lmichalke can you try run="01" (integer vs. string value)? This issue should have been fixed in master but not in latest pypi package.

[lmichalke]

Just tried with run="01" and with run="1", ancpbids still returns an empty list.

[erdalkaraca]

some more hints:

• datatype is no parameter in ancpbids' get() function and also no such BIDS entity exist -> omit it for now
• the dataset does not comply with BIDS naming scheme, for example: sub-001_task-MGT_run-01**bold**space-MNI152NLin2009cAsym_brainmask.nii.gz -> what entity value does "bold" belong to?
  -- I would have used a pattern like this: desc-brainmask_bold.nii.gz

Did you check if pybids indeed returns files that you expect?

[lmichalke]

I see that the derivatives folder does not comply with the BIDS naming scheme. Pybids seems to ignore the misplaced bold substring and returns the result we expect. Would you want to replicate pybids' behaviour of not enforcing the naming scheme strictly, or do we have to rename all the files?

[erdalkaraca]

makes sense to introduce a new parameter to enforce non-strict parsing of file names to better comply with pybids

[erdalkaraca]

@effigies would like your comment on this issue: PyBIDS seems to ignore ill-formed file names and extracts just what can be extracted according to the BIDS naming scheme.
For example:
sub-001_task-MGT_run-01_bold_space-MNI152NLin2009cAsym_brainmask.nii.gz

the term "bold" (following run entity) is dangling and will be ignored when doing a search for file paths.
ancpBIDS will not offer that file as its name obviously does not conform to the BIDS naming scheme.

[effigies]

PyBIDS' BIDSLayout has a validate={True,False} parameter. The basic process is to use regular expressions to extract path components. If validate=False, we're done, which allows non-conforming filenames to make it through. The main use case for this is files that have new suffixes or extensions without having to write a custom configuration file. That it happens to also work with very old fMRIPrep derivatives (that predate BIDS Derivatives) is a kind of bonus for people, but not anything I would expect most BIDS tools to handle.

I think it's reasonable for ancpBIDS to reject these files without making an extreme effort to match accidental features of PyBIDS.

It could still be useful to have some way of referencing files that are non-conforming, even if the normal query infrastructure can't pick them up.