Add runtime policy, revocation, and audit correlation for generated tools#2547
Open
vaddisrinivas wants to merge 2 commits into
Open
Add runtime policy, revocation, and audit correlation for generated tools#2547vaddisrinivas wants to merge 2 commits into
vaddisrinivas wants to merge 2 commits into
Conversation
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthroughAdds generated-tool runtime context, RequireApproval decision, a GeneratedToolRuntimePolicy, a Tool hook to provide context, executor threading and blocking_reason() extraction, and audit correlation for generated-tool events. ChangesGenerated Tool Policy and Audit
Sequence DiagramsequenceDiagram
participant Agent as Agent::execute_tool_call
participant Policy as GeneratedToolRuntimePolicy
participant Decision as ToolPolicyDecision
participant Audit as AuditLogger
Agent->>Policy: check(request + generated_tool context)
Policy->>Decision: Allow / RequireApproval / Deny
Decision->>Agent: decision (optional blocking reason)
alt blocked or requires approval
Agent->>Audit: log_generated_tool_event(provider, capability, policy_decision, approval_id)
else allowed
Agent->>Agent: proceed with execution
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
coderabbitai
Bot
added
feature
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
src/openhuman/agent/tool_policy.rs (1)
236-310: ⚡ Quick winAdd branch-level debug tracing for runtime policy decisions.
At Lines 236-310, generated-tool policy evaluation has multiple enforcement branches (revoked provider/capability, provider/capability/risk mappings, default allow) but emits no local debug/trace diagnostics, which makes runtime policy outcomes hard to audit during incidents.
♻️ Suggested patch
impl GeneratedToolRuntimePolicy { @@ fn action_for( &self, tool_name: &str, context: &GeneratedToolRuntimeContext, ) -> (RuntimeToolPolicyAction, String) { @@ - if let Some(action) = self.config.risk_actions.get(&context.risk) { + if let Some(action) = self.config.risk_actions.get(&context.risk) { return ( *action, format!("risk `{:?}` matched runtime policy", context.risk), ); } ( RuntimeToolPolicyAction::Allow, format!("tool `{tool_name}` allowed"), ) } } @@ async fn check(&self, request: &ToolPolicyRequest) -> ToolPolicyDecision { if !self.config.enabled { + tracing::trace!( + policy = self.name(), + tool = request.tool_name.as_str(), + "[rpc] generated tool runtime policy disabled; allowing" + ); return ToolPolicyDecision::Allow; } let Some(context) = request.generated_tool.as_ref() else { + tracing::trace!( + policy = self.name(), + tool = request.tool_name.as_str(), + "[rpc] generated tool context missing; allowing" + ); return ToolPolicyDecision::Allow; }; let (action, reason) = self.action_for(&request.tool_name, context); + tracing::debug!( + policy = self.name(), + tool = request.tool_name.as_str(), + provider_id = context.provider_id.as_str(), + capability_id = context.capability_id.as_str(), + risk = ?context.risk, + action = ?action, + reason = reason.as_str(), + "[rpc] generated tool runtime policy decision" + ); match action { RuntimeToolPolicyAction::Allow => ToolPolicyDecision::Allow, RuntimeToolPolicyAction::RequireApproval => { ToolPolicyDecision::require_approval(reason) } RuntimeToolPolicyAction::Deny => ToolPolicyDecision::deny(reason), } } }As per coding guidelines: "Use
log/tracingatdebugortracelevel on ... state transitions, and any branch that is hard to infer from tests alone."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/openhuman/agent/tool_policy.rs` around lines 236 - 310, The action_for/check flow in GeneratedToolRuntimePolicy lacks branch-level debug tracing; add debug or trace logs inside GeneratedToolRuntimePolicy::action_for for each branch (revoked provider, revoked capability, capability_actions hit, provider_actions hit, risk_actions hit, and default allow) emitting the chosen RuntimeToolPolicyAction, the formatted reason, tool_name, context.provider_id, context.capability_id, and context.risk as appropriate, and also log the final decision in GeneratedToolRuntimePolicy::check right before returning the ToolPolicyDecision (use ToolPolicyDecision::require_approval / deny / Allow paths); reference the action_for function, the RuntimeToolPolicyAction enum, and the check method to locate insertion points.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/openhuman/agent/harness/session/turn.rs`:
- Around line 1202-1205: The policy check is using ToolPolicyRequest::new which
leaves generated_tool as None so generated-tool runtime policy rules never run;
before calling self.tool_policy.check(&policy_request).await, populate the
request's generated_tool context (e.g., via the request builder/constructor that
accepts generated tool metadata or by setting the generated_tool field on the
ToolPolicyRequest you created) so the generated-tool runtime policy (revoked
provider/capability/risk action mappings) is available to the check; update the
code path around ToolPolicyRequest::new and the call site of
self.tool_policy.check to pass the generated tool context into the request.
---
Nitpick comments:
In `@src/openhuman/agent/tool_policy.rs`:
- Around line 236-310: The action_for/check flow in GeneratedToolRuntimePolicy
lacks branch-level debug tracing; add debug or trace logs inside
GeneratedToolRuntimePolicy::action_for for each branch (revoked provider,
revoked capability, capability_actions hit, provider_actions hit, risk_actions
hit, and default allow) emitting the chosen RuntimeToolPolicyAction, the
formatted reason, tool_name, context.provider_id, context.capability_id, and
context.risk as appropriate, and also log the final decision in
GeneratedToolRuntimePolicy::check right before returning the ToolPolicyDecision
(use ToolPolicyDecision::require_approval / deny / Allow paths); reference the
action_for function, the RuntimeToolPolicyAction enum, and the check method to
locate insertion points.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: e9224089-8318-4a32-ba09-0fbac4e981b7
📒 Files selected for processing (3)
src/openhuman/agent/harness/session/turn.rssrc/openhuman/agent/tool_policy.rssrc/openhuman/security/audit.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Problem
Solution
ToolPolicyRequestwith optionalGeneratedToolRuntimeContext.ToolPolicyDecision::RequireApprovaland a sharedblocking_reasonhelper so the existing session runner blocks non-allow decisions.GeneratedToolRuntimePolicyand structured generated-tool audit helpers while preserving the default allow-all policy.Submission Checklist
## Related: N/A, no coverage-matrix feature ID applies.docs/RELEASE-MANUAL-SMOKE.md): N/A, no release-cut UI/runtime smoke surface changed.Closes #NNNin the## Relatedsection.Impact
Related
AI Authored PR Metadata (required for Codex/Linear PRs)
Linear Issue
Commit & Branch
codex/oh-2543-generated-tool-runtime-policyf810dfa716ad3fb86fa5b70755b3784726ef673eValidation Run
pnpm --filter openhuman-app format:check: N/A, Rust-only backend policy/audit change.pnpm typecheck: N/A, Rust-only backend policy/audit change.cargo test --manifest-path Cargo.toml generated_runtime_policy --lib;cargo test --manifest-path Cargo.toml audit_log_generated_tool --lib;cargo test --manifest-path Cargo.toml threads_generated_tool_context --libcargo fmt --manifest-path Cargo.toml --all --check;git diff --checknode scripts/check-pr-checklist.mjs /tmp/openhuman-pr-body-2543.mdValidation Blocked
command:N/Aerror:N/Aimpact:N/ABehavior Changes
Parity Contract
Duplicate / Superseded PR Handling
vaddisrinivas:codex/oh-2543-generated-tool-runtime-policy.Summary by CodeRabbit
New Features
Tests