Skip to content

feat(aws): add EC2, ECS and EKS resource detectors - #721

Draft
JeremieRodon wants to merge 1 commit into
open-telemetry:mainfrom
RustyServerless:feat/aws-resource-detectors
Draft

feat(aws): add EC2, ECS and EKS resource detectors#721
JeremieRodon wants to merge 1 commit into
open-telemetry:mainfrom
RustyServerless:feat/aws-resource-detectors

Conversation

@JeremieRodon

Copy link
Copy Markdown
Contributor

Fixes #705
Fixes #706
Part of #707
Part of #94

Note

Marked as a draft on purpose. The code works (see "Testing" below), but several design points listed under "Discussion / open questions" need maintainer input before this is ready for final review.
I'll be AFK for the next 2 weeks and will address any and all remarks upon returning.

Changes

Adds three AWS resource detectors to the opentelemetry-aws crate, each behind its own feature flag:

  • Ec2ResourceDetector (detector-aws-ec2) — queries IMDSv2 for cloud.* and host.* attributes.
  • EcsResourceDetector (detector-aws-ecs) — reads the ECS container metadata endpoint v4 (ECS_CONTAINER_METADATA_URI_V4) and IMDSv2 on EC2 workers for cloud.*, aws.ecs.*,
    container.* and aws.log.* attributes.
  • EksResourceDetector (detector-aws-eks) — detects a pod via the
    service-account namespace file and ties it to AWS via IMDSv2, reporting
    cloud.*, k8s.*, aws.eks.cluster.arn, container.id and node host.*.

Supporting changes:

  • New internal feature groups _detector / _detector-http to share deps.
  • New optional dependency ureq (blocking HTTP client — ResourceDetector::detect
    is synchronous). Empty attribute values are filtered out; errors are best-effort
    logged (gated by internal-logs) and the attribute is skipped.
  • The existing LambdaResourceDetector is intentionally untouched.

Usage

use opentelemetry_aws::detector::Ec2ResourceDetector;
use opentelemetry_sdk::Resource;

let resource = Resource::builder()
    .with_detector(Box::new(Ec2ResourceDetector))
    .build();

Discussion / open questions

I'd like maintainer input on the following before finalizing:

  1. Warning on wrong platform. A detector run on the wrong platform emits a
    single warning log line (disableable by dropping the internal-logs
    feature). Pro: surfaces likely misconfiguration cheaply. Con: an app that
    legitimately runs on multiple platforms (e.g. EC2 and ECS) and enables
    several detectors will always warn for the ones that don't match. Keep,
    downgrade to debug, or drop?
  2. IMDS use in ECS/EKS vs. Fargate. Both ECS and EKS detectors contact
    IMDSv2 to enrich with node info, which does not work on Fargate. Should these
    detectors instead focus only on ECS/EKS-specific sources and leave node
    detection to the EC2 detector (user opts in by also enabling it)? If so, how
    should cloud.platform be resolved when several detectors are chained —
    relying on detector order is poor API design; is there a better contract?
  3. IMDSv2 only. I only implement IMDSv2 (no v1 fallback). Is v1 support
    wanted? Relatedly, endpoints use /latest/; should a pinned, dated IMDS
    version be used instead?
  4. Empty-value filtering. Every detector filters out empty/blank values to
    avoid populating keys with empty strings. Is the added code worth it, or is
    it acceptable to emit empty values?
  5. EKS approach differs from [Feature]: AWS EKS resource detector #707 which proposes the k8s-API route
    (read aws-auth / cluster-info configmaps, container.id from cgroup).
    This PR instead uses the service-account namespace file + IMDSv2
    aws:eks:cluster-name instance tag + downward-API env vars, which avoids a
    Kubernetes client dependency but requires instance-tags-in-metadata to be
    enabled (or AWS_CLUSTER_NAME to be set) for k8s.cluster.name. Is this
    trade-off acceptable, or should it follow [Feature]: AWS EKS resource detector #707's configmap approach?
  6. Refactor the Lambda detector? The existing lambda.rs populates
    attributes with unwrap_or_default(), producing empty values on error —
    inconsistent with the empty-filtering the new detectors do. Should a small
    refactor be included in this PR, or kept separate?

Testing

Unit tests cover the pure logic (arch mapping, value filtering, ARN parsing, etc.).

End-to-end behavior was verified by running a probe program on real AWS (EC2, ECS, EKS). See: https://github.com/RustyServerless/opentelemetry-rust-contrib/tree/wip/aws-resource-detectors/otel-aws-probe-deploy

Merge requirement checklist

@codecov

codecov Bot commented Jul 31, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 55.89041% with 322 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.4%. Comparing base (d2eeff7) to head (59b1218).

Files with missing lines Patch % Lines
opentelemetry-aws/src/detector/ecs.rs 58.5% 143 Missing ⚠️
opentelemetry-aws/src/detector/eks.rs 54.2% 92 Missing ⚠️
opentelemetry-aws/src/detector/imds.rs 54.6% 34 Missing ⚠️
opentelemetry-aws/src/detector/utils.rs 65.1% 30 Missing ⚠️
opentelemetry-aws/src/detector/ec2.rs 0.0% 23 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (d2eeff7) and HEAD (59b1218). Click for more details.

HEAD has 2 uploads less than BASE
Flag BASE (d2eeff7) HEAD (59b1218)
user-events-integration 1 0
etw-integration 1 0
Additional details and impacted files
@@           Coverage Diff           @@
##            main    #721     +/-   ##
=======================================
- Coverage   79.7%   71.4%   -8.3%     
=======================================
  Files        133     136      +3     
  Lines      28369   28621    +252     
=======================================
- Hits       22623   20452   -2171     
- Misses      5746    8169   +2423     
Flag Coverage Δ
etw-integration ?
user-events-integration ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@chinmaychahar

Copy link
Copy Markdown
Contributor

It's well-built code at this scale. Thanks for that! Thorough docs, unit tests and live testing

@chinmaychahar

chinmaychahar commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Warning on wrong platform. A detector run on the wrong platform emits a
single warning log line (disableable by dropping the internal-logs
feature). Pro: surfaces likely misconfiguration cheaply. Con: an app that
legitimately runs on multiple platforms (e.g. EC2 and ECS) and enables
several detectors will always warn for the ones that don't match. Keep,
downgrade to debug, or drop?

My leaning is to downgrade to debug, what do you think? But if we are on the platform and a call that should've worked fails, that still feels worth a warn

IMDS use in ECS/EKS vs. Fargate. Both ECS and EKS detectors contact
IMDSv2 to enrich with node info, which does not work on Fargate. Should these
detectors instead focus only on ECS/EKS-specific sources and leave node
detection to the EC2 detector (user opts in by also enabling it)? If so, how
should cloud.platform be resolved when several detectors are chained —
relying on detector order is poor API design; is there a better contract?

We can keep IMDS in the ECS/EKS detectors, seems safe without having to add another detector, but I'll need to analyze this more

IMDSv2 only. I only implement IMDSv2 (no v1 fallback). Is v1 support
wanted? Relatedly, endpoints use /latest/; should a pinned, dated IMDS
version be used instead?

v2-only should be fine. AWS is pushing v2 as the default. @cijothomas could answer more if there are any OTel standards. Also, I'd keep /latest/. Is there a specific field you're worried about?

@chinmaychahar

Copy link
Copy Markdown
Contributor

Empty-value filtering. Every detector filters out empty/blank values to
avoid populating keys with empty strings. Is the added code worth it, or is
it acceptable to emit empty values?

I think an empty data can look like real data and break things downstream, unlike null so since these are utils and a few small re-used helpers (with tests) - we should be good to keep

EKS approach differs from #707 which proposes the k8s-API route
(read aws-auth / cluster-info configmaps, container.id from cgroup).
This PR instead uses the service-account namespace file + IMDSv2
aws:eks:cluster-name instance tag + downward-API env vars, which avoids a
Kubernetes client dependency but requires instance-tags-in-metadata to be
enabled (or AWS_CLUSTER_NAME to be set) for k8s.cluster.name. Is this
trade-off acceptable, or should it follow #707 configmap approach?

I looked at how the other SDKs do it and Go uses the full client-go library, but Python and JS both just hit the k8s API directly over plain HTTPS with the service-account toke. So, curious to know how quantitative are the benefits of avoiding the kubernetes client dependency? I'm good with your lighter-weight approach, just wanna understand more.

Also, right now the platform tie is AWS_REGION (eks.rs:170, eks.rs:185) and AWS_REGION is set by pretty much any app which will use a AWS SDK, so what if a GKE/AKS pod reads from S3 and gets labelled as cloud.platform=aws_eks? Let's think about both happy cases and edge cases

Refactor the Lambda detector? The existing lambda.rs populates
attributes with unwrap_or_default(), producing empty values on error —
inconsistent with the empty-filtering the new detectors do. Should a small
refactor be included in this PR, or kept separate?

Let's keep it separate, what do you think? This PR is already big

@chinmaychahar

Copy link
Copy Markdown
Contributor

As you work on the re-factor, the only red check is codecov currently - can we add more mockable scenarios?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: AWS ECS resource detector [Feature]: AWS EC2 resource detector

2 participants