Skip to content

common: fix match_case misidentifying all-caps contractions as capitalized#4314

Open
nac7 wants to merge 2 commits into
stanford-crfm:mainfrom
nac7:fix/match-case-apostrophe
Open

common: fix match_case misidentifying all-caps contractions as capitalized#4314
nac7 wants to merge 2 commits into
stanford-crfm:mainfrom
nac7:fix/match-case-apostrophe

Conversation

@nac7

@nac7 nac7 commented Jun 8, 2026

Copy link
Copy Markdown

Fixes #4313.

Problem

match_case in src/helm/common/general.py iterates over all characters when checking islower()/isupper(), including apostrophes. In Python, "'".islower() and "'".isupper() both return False (apostrophes have no case). This breaks the all-caps detection for any word containing an apostrophe:

# Before fix:
match_case("AIN'T", "is not")   # → "Is not"  ✗ (should be "IS NOT")
match_case("DON'T", "do not")   # → "Do not"  ✗ (should be "DO NOT")
match_case("CAN'T", "cannot")   # → "Cannot"  ✗ (should be "CANNOT")

For "AIN'T": all(ch.isupper() for ch in "AIN'T") is False because "'".isupper() is False, so the all-caps branch is skipped. The code falls into the capitalized branch (source_word[0].isupper() is True), producing "Is not" instead of "IS NOT".

This affects ContractionPerturbation, ExpansionPerturbation, and SynonymPerturbation, which all call match_case to preserve source-word casing. All-caps contractions in benchmark inputs produce wrong-case augmented outputs.

Fix

Filter to only alphabetic characters before case-checking. Non-alpha characters (apostrophes, digits, punctuation) are ignored when determining the case category of a word:

alpha_chars = [ch for ch in source_word if ch.isalpha()]
if not alpha_chars:
    return target_word
if all(ch.islower() for ch in alpha_chars):
    return target_word.lower()
if all(ch.isupper() for ch in alpha_chars):
    return target_word.upper()
if alpha_chars[0].isupper():
    return target_word.capitalize()
return target_word

Tests

Added to test_perturbation.py:

  • test_match_case_all_caps_with_apostrophe"AIN'T" / "DON'T" / "CAN'T""IS NOT" / "DO NOT" / "CANNOT" (regression cases)
  • test_match_case_lowercase_with_apostrophe — lowercase contractions still produce lowercase
  • test_match_case_capitalized_with_apostrophe — capitalized contractions still capitalize
  • test_match_case_no_alpha_chars — empty / digit-only source → target unchanged
  • test_match_case_regular_words — regular words (no apostrophe) still work correctly

nac7 added 2 commits June 8, 2026 14:21
@
docs: fix incorrect command name in helm-server tutorial description

The helm-server section referenced helm-benchmark, which does not exist
as a CLI entrypoint. The correct command is helm-summarize, which is
documented in the preceding section and listed in pyproject.toml.
@
…lized

match_case iterated over all characters including apostrophes when
checking islower()/isupper(). Apostrophes return False for both
islower() and isupper() in Python (they have no case), so words like
"AIN'T" and "DON'T" would fail the all-caps check and fall through
to the capitalize branch, producing "Is not" instead of "IS NOT".

Fix: filter to only alphabetic characters before case-checking, so
non-alpha chars in contractions no longer interfere with the detection.

Affects ContractionPerturbation, ExpansionPerturbation, and
SynonymPerturbation for all-caps inputs containing apostrophes.

Fixes stanford-crfm#4313
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

match_case returns wrong case for all-caps words containing apostrophes (e.g. "AIN'T", "DON'T")

1 participant