Bug (CSVHelper) - empty multivalue separator corrupts field processing

[arthurianresolve]

Issue

CSVHelper accepts an empty multi-value separator.

Problem

CSVHelper.setup(Configuration) reads <datatype>.data.multivalued.separator without checking whether it is empty:

datawave/warehouse/ingest-core/src/main/java/datawave/ingest/data/config/CSVHelper.java

Line 232 in 02d6867

this.multiValueSeparator = config.get(this.getType().typeName() + MULTI_VALUED_SEPARATOR, ";");

this.multiValueSeparator = config.get(this.getType().typeName() + MULTI_VALUED_SEPARATOR, ";");

The separator is later used to build the split pattern and remove escape characters:

datawave/warehouse/ingest-core/src/main/java/datawave/ingest/data/config/CSVHelper.java

Lines 311 to 317 in 02d6867

	/**
	* @return a pattern based on the multivalueseparator value that will not match that value preceeded by a '\\' (backslash) character. Useful as an argument
	* to the String.split(..) function or similar methods
	*/
	public String getEscapeSafeMultiValueSeparatorPattern() {
	return BACKSLASH_ESCAPE_LOOKBEHIND_PATTERN + getMultiValueSeparator();
	}

datawave/warehouse/ingest-core/src/main/java/datawave/ingest/data/config/CSVHelper.java

Lines 367 to 379 in 02d6867

	/**
	* Remove the escape characters from escaped multi value separators in field value
	*
	* @param fieldValue
	* the field value to clean
	* @return the cleaned field value
	*/
	public String cleanEscapedMultivalueSeparators(String fieldValue) {
	// remove escaped multvalue separators.
	if (fieldValue.contains("\\" + getMultiValueSeparator())) {
	fieldValue = fieldValue.replaceAll("\\\\" + getMultiValueSeparator(), getMultiValueSeparator());
	}
	return fieldValue;

With an empty separator, the split pattern matches between characters. A value such as abc can be split into a, b, and c. Escape cleanup can also remove backslashes from field values.

Expected behavior

CSVHelper.setup(Configuration) should reject an empty multi-value separator.

The default ; should still be used when the setting is absent.

Actual behavior

The empty value is accepted and can split fields at every character or remove backslashes.

Affected helpers

Ingest helpers that use CSVHelper multi-value processing may also be affected, including:

• CSVIngestHelper
• ExtendedCSVIngestHelper

Suggested fix

Check the separator during CSVHelper.setup(Configuration) and throw IllegalArgumentException when it is empty.

The error should name the invalid configuration key.

Test

The current upstream tests do not cover an empty <datatype>.data.multivalued.separator value.

Add a setup test that confirms an empty separator is rejected.