Skip to content

Replace print() with structured logging #94

Description

@maxnutz

Problem

Warnings/info messages throughout the package are emitted via bare print() calls instead of the standard library logging module. This means:

  • Output can't be filtered by severity (WARNING vs INFO all look the same, distinguished only by an ad-hoc string prefix like "WARNING: ...").
  • Callers embedding this package (e.g. as part of a larger workflow) can't redirect, silence, or capture these messages via standard logging handlers.
  • Tests assert on print output via capsys (tests/test_format_timestamps.py), which is brittle compared to caplog.
  • No way to control verbosity (e.g. suppress INFO in production runs, or elevate to DEBUG for troubleshooting).

Current print() usage

File Line(s) Level (by prefix)
pypsa_validation_processing/class_definitions.py 75, 89 WARNING (format_timestamps)
pypsa_validation_processing/class_definitions.py 320 WARNING (_execute_function)
pypsa_validation_processing/class_definitions.py 669 WARNING (_get_target_unit or similar)
pypsa_validation_processing/class_definitions.py 675, 684, 686 INFO/WARNING (multi-unit parsing)
pypsa_validation_processing/class_definitions.py 723, 731, 735 INFO/WARNING (_get_network_config)
pypsa_validation_processing/statistics_functions.py 829 WARNING (BEV charger efficiency)
pypsa_validation_processing/workflow.py 30 WARNING (resolve_config_path)

This is consistent with the blueprint script sister_packages/pypsa-de/scripts/pypsa-de/export_ariadne_variables.py and sister_packages/pypsa-de/scripts/_helpers.py, which both already use logger = logging.getLogger(__name__) + logger.info/.warning/.error(...), configured centrally via logging.basicConfig(**kwargs) in _helpers.py. We should follow the same convention for consistency across the PyPSA-AT ecosystem.

Proposed implementation plan

  1. Central logging setup helper — add setup_logging(level: int | str = logging.WARNING) -> None to pypsa_validation_processing/utils.py. Wraps logging.basicConfig(level=level, format=...). Called once, at the entrypoint only (workflow.py::main) — never inside statistics_functions.py, per the "statistics functions are independent, only call utils.py helpers" rule in CLAUDE.md.
  2. Module-level loggers — add logger = logging.getLogger(__name__) near the top of:
    • pypsa_validation_processing/class_definitions.py
    • pypsa_validation_processing/statistics_functions.py
    • pypsa_validation_processing/workflow.py
  3. Replace print() calls with the matching logger.warning(...) / logger.info(...) call, dropping the manual "WARNING: " / "INFO: " string prefixes (the log level now carries that information; formatters can add it back for console output if desired).
  4. Wire up setup_logging() in workflow.py::main() before Network_Processor is constructed, defaulting to WARNING. Do not add a new log_level config key to configs/*.yaml as part of this change (config files are off-limits per CLAUDE.md unless explicitly requested) — default level is fine for a first pass; a follow-up issue can add config-driven verbosity if desired.
  5. Update teststests/test_format_timestamps.py::test_format_timestamps_sets_nat_on_localization_failure currently uses capsys; switch it to caplog and assert on the log record's level/message instead of stdout text. Add/extend unit tests for the other converted call sites where they're already covered.
  6. Docs — add a short "Logging" note to docs/contributing.md (or README.md) describing the logger = logging.getLogger(__name__) convention for new statistics functions/helpers, so future contributions don't reintroduce print().

Out of scope

  • Anything under sister_packages/ (read-only, per CLAUDE.md).
  • Adding a log_level config option to configs/*.yaml (separate follow-up if wanted).
  • Structured/JSON logging, log file rotation, or third-party logging libraries (e.g. loguru) — plain stdlib logging matches the PyPSA-DE blueprint and needs no new dependency.

Acceptance criteria

  • No print( calls remain in pypsa_validation_processing/*.py (excluding sister_packages/).
  • Each affected module has its own logger = logging.getLogger(__name__).
  • workflow.py::main() configures logging once via utils.setup_logging().
  • pixi run test passes, including the updated caplog-based test.
  • pixi run workflow_test runs successfully with log output visible at WARNING level by default.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions