Skip to content

Track TUI button telemetry#6346

Merged
lorenzejay merged 4 commits into
mainfrom
codex/track-tui-telemetry-buttons
Jun 26, 2026
Merged

Track TUI button telemetry#6346
lorenzejay merged 4 commits into
mainfrom
codex/track-tui-telemetry-buttons

Conversation

@joaomdmoura

@joaomdmoura joaomdmoura commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add a sanitized TUI Button Clicked telemetry span in crewai-core
  • emit view_traces and deploy click events from the completed-run TUI buttons
  • add focused tests for the TUI click path and core span shape

Verification

  • uv run pytest lib/cli/tests/test_crew_run_tui.py lib/crewai-core/tests/test_smoke.py
  • uv run ruff check lib/cli/src/crewai_cli/crew_run_tui.py lib/cli/tests/test_crew_run_tui.py lib/crewai-core/src/crewai_core/telemetry.py lib/crewai-core/tests/test_smoke.py
  • uv run ruff format --check lib/cli/src/crewai_cli/crew_run_tui.py lib/cli/tests/test_crew_run_tui.py lib/crewai-core/src/crewai_core/telemetry.py lib/crewai-core/tests/test_smoke.py

Note

Low Risk
Opt-in style usage telemetry only; errors are caught and telemetry respects existing disable env vars, with no change to auth or deploy logic beyond recording clicks.

Overview
Adds anonymous feature-usage telemetry for post-run TUI sidebar actions.

crewai-core: New Telemetry.feature_usage_span(feature) emits an OpenTelemetry Feature Usage span with crewai_version and a feature attribute (e.g. cli_usage:view_traces), using the same safe/disabled-env behavior as other core spans.

CLI TUI: CrewRunApp lazily initializes Telemetry and calls _record_tui_button_click when View Traces (btn-traces / btn-traces-done) or Deploy is pressed, before the existing trace-open and deploy-exit behavior. Failures are swallowed so the UI keeps working.

Tests: Unit tests assert the TUI passes the expected cli_usage:* feature strings and that the core span name, feature attribute, and end() are correct.

Reviewed by Cursor Bugbot for commit 5107624. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • New Features
    • Added telemetry tracking for key CLI TUI actions, including “view traces” and “deploy”.
    • Introduced a feature-usage telemetry span that records the current version and the selected feature name.
  • Tests
    • Added unit tests covering telemetry spans and expected outcomes for “view traces” and “deploy”.
    • Added a smoke test to verify telemetry span creation, attributes, and proper span termination.

@corridor-security corridor-security Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary: This PR adds telemetry for fixed TUI button click events and does not introduce new authentication, authorization, data access, or externally controlled execution paths.

Risk: Low risk. No exploitable security vulnerabilities were identified because the added telemetry values are constant internal event names and failures are contained without exposing a security boundary.

@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d93664c8-6523-4723-945b-78cba0e1ac27

📥 Commits

Reviewing files that changed from the base of the PR and between 278663a and 5107624.

📒 Files selected for processing (1)
  • lib/crewai-core/tests/test_smoke.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • lib/crewai-core/tests/test_smoke.py

📝 Walkthrough

Walkthrough

The PR adds telemetry spans for crew run TUI button clicks. It adds a core feature-usage span helper, wires the TUI to record trace and deploy clicks, and adds unit and smoke tests for the telemetry behavior.

Changes

TUI telemetry click tracking

Layer / File(s) Summary
Telemetry span method
lib/crewai-core/src/crewai_core/telemetry.py, lib/crewai-core/tests/test_smoke.py
Telemetry.feature_usage_span creates a "Feature Usage" span with crewai_version and feature attributes, and the smoke test verifies tracer interaction and span closure.
CrewRunApp button telemetry
lib/cli/src/crewai_cli/crew_run_tui.py, lib/cli/tests/test_crew_run_tui.py
CrewRunApp lazily stores telemetry, records trace and deploy button clicks before continuing the existing actions, and the CLI tests cover trace URL opening and deploy exit handling.

Sequence Diagram(s)

sequenceDiagram
  participant CrewRunApp
  participant Telemetry
  participant WebBrowser
  CrewRunApp->>CrewRunApp: _record_tui_button_click(button_name)
  CrewRunApp->>Telemetry: feature_usage_span("cli_usage:" + button_name)
  Telemetry->>Telemetry: create "Feature Usage" span
  Telemetry->>Telemetry: set_attribute("feature", feature)
  Telemetry->>Telemetry: end()
  alt btn-traces or btn-traces-done
    CrewRunApp->>WebBrowser: open(trace_url)
  else btn-deploy
    CrewRunApp->>CrewRunApp: _want_deploy = True
    CrewRunApp->>CrewRunApp: exit(_crew_result)
  end
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: adding telemetry tracking for TUI button clicks.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/track-tui-telemetry-buttons

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Comment thread lib/cli/src/crewai_cli/crew_run_tui.py
Comment thread lib/crewai-core/src/crewai_core/telemetry.py
@joaomdmoura

Copy link
Copy Markdown
Collaborator Author

Addressed the review feedback in 278663af7:

  • replaced the custom tui_button_clicked_span path with generic feature_usage_span
  • TUI now emits cli_usage:view_traces and cli_usage:deploy
  • resolved both review threads

Verification:

  • uv run pytest lib/cli/tests/test_crew_run_tui.py lib/crewai-core/tests/test_smoke.py
  • uv run ruff check lib/cli/src/crewai_cli/crew_run_tui.py lib/cli/tests/test_crew_run_tui.py lib/crewai-core/src/crewai_core/telemetry.py lib/crewai-core/tests/test_smoke.py
  • uv run ruff format --check lib/cli/src/crewai_cli/crew_run_tui.py lib/cli/tests/test_crew_run_tui.py lib/crewai-core/src/crewai_core/telemetry.py lib/crewai-core/tests/test_smoke.py

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
lib/crewai-core/tests/test_smoke.py (1)

147-156: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Assert the crewai_version attribute too.

This test only locks down half of the new span shape. Since feature_usage_span() now emits both crewai_version and feature, it’s worth asserting the version attribute here as well so that contract regressions fail in this smoke test.

Suggested change
     monkeypatch.setattr(
         "crewai_core.telemetry.trace.get_tracer",
         lambda _name: tracer,
     )
+    monkeypatch.setattr(
+        "crewai_core.version.get_crewai_version",
+        lambda: "1.2.3",
+    )

     telemetry = Telemetry()
     telemetry.feature_usage_span("cli_usage:view_traces")

     tracer.start_span.assert_called_once_with("Feature Usage")
+    span.set_attribute.assert_any_call("crewai_version", "1.2.3")
     span.set_attribute.assert_any_call("feature", "cli_usage:view_traces")
     span.end.assert_called_once()
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai-core/tests/test_smoke.py` around lines 147 - 156, The smoke test
for Telemetry.feature_usage_span only verifies the feature attribute, but the
span now also includes crewai_version. Update the assertions in test_smoke.py to
check that the span created by
Telemetry.feature_usage_span("cli_usage:view_traces") sets both feature and
crewai_version, using the existing tracer/span mocks so this contract is covered
by the test.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@lib/crewai-core/tests/test_smoke.py`:
- Around line 147-156: The smoke test for Telemetry.feature_usage_span only
verifies the feature attribute, but the span now also includes crewai_version.
Update the assertions in test_smoke.py to check that the span created by
Telemetry.feature_usage_span("cli_usage:view_traces") sets both feature and
crewai_version, using the existing tracer/span mocks so this contract is covered
by the test.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c2f9171d-f917-4b64-8c34-421fb266afa1

📥 Commits

Reviewing files that changed from the base of the PR and between 1dfaf3d and 278663a.

📒 Files selected for processing (4)
  • lib/cli/src/crewai_cli/crew_run_tui.py
  • lib/cli/tests/test_crew_run_tui.py
  • lib/crewai-core/src/crewai_core/telemetry.py
  • lib/crewai-core/tests/test_smoke.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • lib/cli/tests/test_crew_run_tui.py

@lorenzejay lorenzejay merged commit 8eaae40 into main Jun 26, 2026
56 checks passed
@lorenzejay lorenzejay deleted the codex/track-tui-telemetry-buttons branch June 26, 2026 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants