Provider-defined functions fail inside dynamic block for_each filters (works elsewhere)

[BillyWeans]

Community note

Tip

👋 Hi there, OpenTofu community! The OpenTofu team prioritizes issues based on upvotes. Please make sure to upvote this issue and describe how it affects you in detail in the comments to show your support.

OpenTofu Version

OpenTofu v1.10.5
on darwin_amd64
+ provider registry.opentofu.org/hashicorp/google v6.17.0

OpenTofu Configuration Files

non-working.tf

# This does NOT work and yields a 'no functions for provider' error. See BUG comment below
terraform {
  required_providers {
    google = { source = "hashicorp/google", version = "6.17.0" }
  }
}

provider "google" {
  project = "MY_PROJECT_ID"
}

resource "google_compute_firewall" "repro" {
  name    = "repro"
  network = "default"

  source_ranges = ["0.0.0.0/0"]

  dynamic "allow" {
    # BUG: provider functions apparently 'disappear' when in-lined in for_each
    for_each = {
      for z in ["us-central1-a"] :
      z => z if provider::google::region_from_zone(z) == "us-central1"
    }
    content {
      protocol = "tcp"
      ports    = ["80"]
    }
  }
}

working.tf

# This works when moving the provider function out of the dynamic for_each
terraform {
  required_providers {
    google = { source = "hashicorp/google", version = "6.17.0" }
  }
}

provider "google" {
  project = "MY_PROJECT_ID"
}

locals {
  # Precompute with the provider function OUTSIDE the dynamic block
  filtered_zones = {
    for z in ["us-central1-a"] :
    z => z if provider::google::region_from_zone(z) == "us-central1"
  }
}

resource "google_compute_firewall" "repro" {
  name    = "repro"
  network = "default"

  # satisfy schema: at least one of source_* must be set
  source_ranges = ["0.0.0.0/0"]

  dynamic "allow" {
    for_each = local.filtered_zones
    content {
      protocol = "tcp"
      ports    = ["80"]
    }
  }
}

Debug Output

https://gist.github.com/BillyWeans/d5465e8f30b7461e7f3eea299ae39864

Expected Behavior

Calling a provider function inside of a dynamic for_each should plan properly as it does with a terraform plan.

Actual Behavior

│ on main.tf line 22, in resource "google_compute_firewall" "repro":
│ 22: z => z if provider::google::region_from_zone(z) == "us-central1"
│
│ There are no functions in namespace "provider::google::".

Steps to Reproduce

1. tofu init
2. tofu plan

Additional Context

No response

References

No response

[apparentlymart]

Thanks for reporting this, @BillyWeans! The core maintainers will discuss it soon.

dynamic blocks are unfortunately quite tricky because their implementation belongs to HCL rather than to OpenTofu, but OpenTofu needs to provide a suitable evaluation context (where the functions come from) when asking HCL to "expand" the functions.

Therefore probably what's going on here is that we're doing that dynamic block expansion with the wrong eval context, where only core functions are available and the provider-defined functions have not yet been added.

Working backwards from calls to dynblock.Expand would be one way to understand how this is happening, but unfortunately we indirectly call that from lots of different callers (tofu.EvalContext.ExpandBlock calls it and that is called all over) so probably worth trying to narrow down which of the callers is involved in this particular error, perhaps by temporarily adding a panic just before that error message gets returned so we can see what else is in the stack trace there.

[apparentlymart]

(Continued thinking aloud...)

Another tricky thing about the dynblock extension is that the for_each expression in a dynamic block can create additional dependencies on objects elsewhere in the module.

OpenTofu handles provider-defined functions as a weird new kind of dependency via hashicorp/hcl#676 (which we use from our local fork of HCL), and that PR does include additions to dynblock to find the functions used in a dynamic block's for_each, so perhaps we can attack this a different way by looking for calls to dynblock.FunctionsHCLDec or dynblock.ExpandFunctionsHCLDec to verify that we're always incorporating the results from one of those two functions into the set of functions that we report that a function depends on, so that the rest of the system knows to include those functions in the scope.

[apparentlymart]

What I described over in #3243 is likely the same root cause for this issue too, since both of these are relying on OpenTofu's ability to detect references in the resource configuration. In this case it's a reference to a function rather than to a symbol, but the same caveats apply to both.

Edit: on further inspection I'm actually not so sure, because if the allow "block type" here were actually using attributes-as-blocks mode then the reference to local.filtered_zones would also fail for the same reason as #3243. So there seems to be some additional subtlety here, but possibly still ultimately caused by the weird code in the blocktoattr package.

[yottta]

After diving deep into this, the issue was that the context that is used to expand the dynamic block did not contain the provider functions referenced in the for_each section.

During the evaluation, there is a place where we extract the provider functions to pass them in the context for the evaluation step, but at that point, the dynamic block is already "wrapped" in the structure responsible with the expansion so we get no references out of it, because the for_each statement is already expanded.