Skip to content

SOLR-18237: Do not copy data before noggit parsing #4415

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
psalagnac wants to merge 1 commit into
base: main
Choose a base branch
from
+11 −13
Open

SOLR-18237: Do not copy data before noggit parsing #4415

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions changelog/unreleased/SOLR-18237-no-buffer.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
title: Do not copy full data for UTF8 to Java string conversion for deserializing a collection state
type: fixed
Copy link
Copy Markdown
Contributor

@dsmiley dsmiley May 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

users will not perceive this as a fix of a bug. It's "changed". But I'm skeptical it warrants a changelog at all.

authors:
- name: Pierre Salagnac
links:
- name: SOLR-18237
url: https://issues.apache.org/jira/browse/SOLR-18237
17 changes: 4 additions & 13 deletions solr/solrj/src/java/org/apache/solr/common/util/Utils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,12 +83,9 @@
import org.noggit.JSONWriter;
import org.noggit.ObjectBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class Utils {

private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());

public static final Random RANDOM;

static {
Expand Down Expand Up @@ -297,16 +294,10 @@ public static Object fromJSON(
if (utf8 == null || utf8.length == 0 || length == 0) {
return Map.of();
}
// convert directly from bytes to chars
// and parse directly from that instead of going through
// intermediate strings or readers
CharArr chars = new CharArr();
ByteUtils.UTF8toUTF16(utf8, offset, length, chars);
JSONParser parser = new JSONParser(chars.getArray(), chars.getStart(), chars.length());
parser.setFlags(
parser.getFlags()
| JSONParser.ALLOW_MISSING_COLON_COMMA_BEFORE_OBJECT
| JSONParser.OPTIONAL_OUTER_BRACES);
// convert from bytes to chars on-the-fly and parse directly
// from that instead of going through intermediate buffers
Reader reader = new InputStreamReader(new ByteArrayInputStream(utf8, offset, length), UTF_8);
Comment on lines +297 to +299
Copy link
Copy Markdown
Contributor

@dsmiley dsmiley May 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is definitely the most natural/clear way to do things instead of anything else. Maybe it's faster; I trust your judgement.

JSONParser parser = getJSONParser(reader);
try {
return fun.apply(parser).getValStrict();
} catch (IOException e) {
Expand Down
Loading