Problem
Consul uses this library for Cloud Auto-join. On AWS EKS, if we follow AWS best practices and "Restrict access to the instance profile assigned to the worker node" by configuring our nodes with http_put_response_hop_limit=1, consul won't be able to get credentials from AWS EKS IRSA.
In a nutshell, this is how I think it works:
- When
http_put_response_hop_limit=2 if we do http://169.254.169.254/latest/meta-data/
- Inside a pod that has a K8s ServiceAccount WITH the correct eks.amazonaws.com/role-arn annotation -> The workflows should get credentials from the variables
AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN. We want this 👍
- Inside a pod with a K8s ServiceAccount WITHOUT an annotation -> We see the metadata from the ⚠️EC2 instance⚠️ . Ideally, we don't want this, but this is what is supported by Consul.
- When
http_put_response_hop_limit=1 if we do http://169.254.169.254/latest/meta-data/
- Inside a pod that has a K8s ServiceAccount WITH the correct eks.amazonaws.com/role-arn annotation -> The workflows should get credentials from the variables
AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN. We want this 👍
- Inside a pod that has a K8s ServiceAccount WITHOUT an annotation -> Not available 🤷
Related issue: hashicorp/consul#8532
Problem
Consul uses this library for Cloud Auto-join. On AWS EKS, if we follow AWS best practices and "Restrict access to the instance profile assigned to the worker node" by configuring our nodes with
http_put_response_hop_limit=1, consul won't be able to get credentials from AWS EKS IRSA.In a nutshell, this is how I think it works:
http_put_response_hop_limit=2if we dohttp://169.254.169.254/latest/meta-data/AWS_WEB_IDENTITY_TOKEN_FILEandAWS_ROLE_ARN. We want this 👍http_put_response_hop_limit=1if we dohttp://169.254.169.254/latest/meta-data/AWS_WEB_IDENTITY_TOKEN_FILEandAWS_ROLE_ARN. We want this 👍Related issue: hashicorp/consul#8532