Skip to content

Make solr field/schema resolvers respect the tokenized field of attributes#7003

Open
clockard wants to merge 2 commits into
masterfrom
fix-solr-field-reslover
Open

Make solr field/schema resolvers respect the tokenized field of attributes#7003
clockard wants to merge 2 commits into
masterfrom
fix-solr-field-reslover

Conversation

@clockard
Copy link
Copy Markdown
Contributor

@clockard clockard commented Jun 2, 2026

What does this PR do?

Updates the solr resolvers to respect the tokenized field of attribute definitions instead of assuming all text fields are tokenized.
Also makes metacard-tags tokenized.

One question I still have is why wildcard id queries don't work in the testing framework. By default the solr schema does have id defined as tokenized but that's not how the attribute is defined. It should work either way but it wasn't in the tests which is why I had to switch them to anyText filters.

Who is reviewing it?

@jaymcnallie
@jrnorth

Any background context you want to provide?

The default ddf solr schema uses primarily dynamic field definitions. These definitions determine how solr stores/handles the data and completely disregards the MetacardType definitions of attributes when it comes to fields like indexed and tokenized. If a downstream project wants to define their fields explicitly for better data handling the resolvers assumption of all string fields being tokenized breaks down and causes issues.

Notes on Review Process

Please see Notes on Review Process for further guidance on requirements for merging and abbreviated reviews.

Review Comment Legend:

  • ✏️ (Pencil) This comment is a nitpick or style suggestion, no action required for approval. This comment should provide a suggestion either as an in line code snippet or a gist.
  • ❓ (Question Mark) This comment is to gain a clearer understanding of design or code choices, clarification is required but action may not be necessary for approval.
  • ❗ (Exclamation Mark) This comment is critical and requires clarification or action before approval.

@clockard clockard requested a review from jaymcnallie June 2, 2026 19:46
@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


chris.lockard seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copy link
Copy Markdown
Member

@jaymcnallie jaymcnallie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to figure out why the unit tests lit up red on the PR build. I think you introduced a query side regression by selectively writing the tokenized copy of the attribute data.

Queries also need to respect the tokenized/not-tokenized field behavior when they run. As it stands, fuzzy searches will look in tokenized fields for tokenized=false attributes and never get results because those fields are now empty.

@clockard clockard force-pushed the fix-solr-field-reslover branch from d2c89f0 to 2981850 Compare June 4, 2026 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants