Feature: Readiness Report#123
Open
biqar wants to merge 26 commits into
Open
Conversation
Introduce a non-interactive Readiness Report in the inspector that runs without user feature or target selection. Add GET /readiness-report to aggregate dataset overview (per-feature profiles, histograms, categorical charts), data quality scorecards, impact-on-AI signals from vectorized all-pairs correlation (leakage, redundancy, isolated features), and automated fairness & bias checks with auto-selected columns. Wire the new sidebar panel and lazy-loaded UI renderers, add chart helpers for histograms and categorical distributions, and include info-tooltips that explain each readiness metric.
Add per-section GET /readiness-report/<section> endpoints and refactor the full-report route to reuse shared section builders. Update the readiness UI to render dataset overview first, then fetch and display Data Quality, Impact on AI, Fairness, and Governance in parallel while other sections keep their loading spinners.
Align Data Quality, Impact on AI, Fairness, and Governance sections on a consistent structure: auto-selection criteria (or analysis scope), overall grade with KPI tiles, then needs attention. Add Impact section grading and column-pruning metadata on the backend, and move supplementary charts and tables into collapsible details.
Enrich backend needs_attention payloads with target, sensitive, and quasi-identifier metadata (including worst k-anonymity groups) so each flag names the columns and groups that triggered it. Refactor Fairness, Governance, and Impact renderers to shared NA helpers for consistent, context-rich display.
…sion Use shared formatters so floats default to two decimal places and automatically gain precision when values would otherwise show as 0.00 (e.g. minority class shares). Apply consistent decimals within each metric group across all report sections; return raw numerical summary values from the backend for client-side formatting.
Expose build_time_seconds from section API responses and render a footer (e.g. "Prepared in 55.01 seconds") after each scorecard loads, matching the timing already logged on the server.
Stop escaping the secondary line in _readinessNaRow so intentional markup (e.g. monospace column names) renders correctly instead of appearing as literal span tags.
… scorecards. Add include_visualization flags to shared metric functions (default True so inspector tabs are unchanged), skip chart generation in readiness section builders, expose GET /readiness-report/<section>/visualizations, and load charts from inspector.js when users expand detail panels.
Add generate_*_MM_risk_scores_groupby implementations that match legacy outputs, wire them only into the readiness report governance builders, and add parity tests against the original privacy-tab functions.
Expose a download button after all five sections load, then build the PDF from the rendered scorecard HTML so metrics are not recomputed. Omit detail charts and normalize dark-mode/animation styles so html2canvas captures crisp output.
Collaborator
Author
|
Todo items that will go in this PR
|
When datasets exceed 500 columns, return poor → warning → good rows first, include full status counts in metadata, and show a truncation note in the UI/PDF.
Cap overview detail payloads (numerical summary, categorical distributions, excluded-column lists) and mirror those limits in the UI so collapsed panels do not build huge DOM trees. Export PDFs from scorecard sections only instead of cloning the full panel, and use html2canvas scale 1 to cut rasterization cost. Readiness scores are unchanged.
Generate histograms and categorical pie charts only for the features shown in the profile table (up to 500, poor → warning → good), including on-demand chart loads. Update the details panel to reflect this scope.
Append a metadata-driven FAIR card at the bottom of the report that reuses POST /fair-assessment and shared render helpers. Automated sections still gate PDF export; FAIR is included only after a successful evaluation.
… reload Store each readiness section and its on-demand visualizations in TEMP_RESULTS_CACHE under user/file-scoped keys with a 30-minute TTL. Reuse existing upload invalidation via clear_all_user_cache; expose cached status in section, visualization, and full-report API responses. Add unit tests for key format, store/retrieve, error skip, expiry, and upload-style clearing.
Expose GET /cached-result/readiness_report to return all cached sections in one response when every section is available. On first panel open, try that endpoint before progressive section fetches so reloads skip spinners and multiple round trips. Add unit tests for aggregated cache lookup.
…oad. Store optional FAIR assessment results under user/file-scoped keys when submitted from the readiness report, keyed by metadata fingerprint so re-uploading the same JSON is instant. Expose GET /cached-result/readiness_report_fair and include fair_compliance in the aggregated readiness cache response. Restore cached FAIR on panel open without requiring metadata re-upload after page reload.
Replace client html2pdf with a WeasyPrint pipeline that builds the scorecard from cached section JSON via GET /readiness-report/pdf. Includes print templates, glossary appendix, per-section footnotes, Needs attention layout/pagination rules, FAIR principle detail tables, and unit/integration tests.
…ables. Extend the server-side PDF pipeline with mode=full so users can download a scorecard-plus-details report matching each section's Show details panel. Add pdf_details view-model builders, detail/chart print templates and CSS, separate Scorecard PDF and Full report PDF actions in the readiness panel, and unit tests for detail blocks and full-report filenames.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issues / Pull Requests
N/A
Description
This PR completes the automated AI Data Readiness Report feature in the inspector: a zero-configuration scorecard that aggregates dataset overview, data quality, impact on AI, fairness & bias, and data governance into one view.
Key themes:
What changes are proposed in this pull request?
Backend (
web/routes/metrics.py)GET /readiness-report/<section>for progressive section loading (dataset-overview,data-quality,impact-on-ai,fairness-bias,data-governance)GET /readiness-report/<section>/visualizationsfor on-demand chart imagesinclude_visualizationparameter (defaultTrue) to metric helpers; readiness builders passFalsefor faster JSON responsesMetric layer (
aidrin/structured_data_metrics/)generate_single_attribute_MM_risk_scores_groupbyandgenerate_multiple_attribute_MM_risk_scores_groupbyfor readiness-report MM linkage risk (parity with legacy implementations)include_visualizationsupport in completeness, outliers, correlation, statistical rate, and add_noise where used by the reportFrontend (
web/static/js/inspector.js, templates, CSS)<details>panels are expandedTests
TestMMRiskGroupbyParityintests/unit/test_privacy.py— descriptive stats and dataset risk score parity vs legacy MM functionsChecklist:
Test plan
adult.csv) and open Readiness Report from the sidebar.uv run pytest tests/unit/test_privacy.py::TestMMRiskGroupbyParity -q.