diff --git a/docs/content/reference/resource.md b/docs/content/reference/resource.md index 921af666156a..26aeb6ed79d4 100644 --- a/docs/content/reference/resource.md +++ b/docs/content/reference/resource.md @@ -486,69 +486,8 @@ properties: type: String ``` -## Examples +## Samples -### `examples` +`Samples` are configurations used to generate documentation and tests. Each sample contains one or more `steps` representing consecutive update test footprints. -A list of configurations that are used to generate documentation and tests. Each example supports the following common -attributes – for a full reference, see -[examples.go ↗](https://github.com/GoogleCloudPlatform/magic-modules/blob/main/mmv1/api/resource/examples.go): - -- `name`: snake_case name of the example. This corresponds to the configuration file in - [mmv1/templates/terraform/examples](https://github.com/GoogleCloudPlatform/magic-modules/tree/main/mmv1/templates/terraform/examples) (excluding the `.go.tmpl` suffix) and is used to generate the test name and the documentation header. -- `primary_resource_id`: The id of the resource under test. This is used by tests to automatically run additional checks. - Configuration files should reference this to avoid getting out of sync. For example: - `resource "google_compute_address" ""{{$.PrimaryResourceId}}" {` -- `bootstrap_iam`: specify member/role pairs that should always exist. `{project_number}` will be replaced with the - default project's project number, and `{organization_id}` will be replaced with the "target" test organization's ID. This avoids race conditions when modifying the global IAM permissions. - Permissions attached to resources created _in_ a test should instead be provisioned with standard terraform resources. -- `vars`: Key/value pairs of variables to inject into the configuration file. These can be referenced in the configuration file - with `{{index $.Vars "key"}}`. All resource IDs (even for resources not under test) should be declared with variables that - contain a `-` or `_`; this will ensure that, in tests, the resources are created with a `tf-test` prefix to allow automatic cleanup of dangling resources and a random suffix to avoid name collisions. -- `test_env_vars`: Key/value pairs of variable names and special values indicating variables that should be pulled from the - environment during tests. These will receive a neutral default value in documentation. Common special values include: - `PROJECT_NAME`, `REGION`, `ORG_ID`, `ORG_TARGET` (a separate test org for testing certain org-level resources such as IAM), `BILLING_ACCT`, `SERVICE_ACCT` (the test runner service account). -- `test_vars_overrides`: Key/value pairs of literal overrides for variables used in tests. This can be used to call functions to - generate or determine a variable's value. -- `min_version`: Set this to `beta` if the resource is in the `google` provider but the example will only work with the - `google-beta` provider (for example, because it includes a beta-only field.) -- `ignore_read_extra`: Properties to not check on import. This should be used in cases where a property will not be set on import, - for example write-only fields. -- `exclude_test`: If set to `true`, no test will be generated based on this example. -- `exclude_docs`: If set to `true`, no documentation will be generated based on this example. -- `exclude_import_test`: If set to `true`, no import test will be generated for this example. -- `skip_vcr`: See [Skip tests in VCR replaying mode]({{< ref "/test/test#skip-vcr" >}}) for more information about this flag. -- `skip_test`: If not empty, the test generated based on this example will always be skipped. In most cases, the value should be a - link to a ticket explaining the issue that needs to be resolved before the test can be unskipped. -- `external_providers`: A list of external providers that are needed for the testcase. This does add some latency to the testcase, - so only use if necessary. Common external providers: `random`, `time`. - -Example: - -```yaml -examples: - - name: service_resource_basic - primary_resource_id: example - bootstrap_iam: - - member: "serviceAccount:service-{project_number}@gcp-sa-healthcare.iam.gserviceaccount.com" - role: "roles/bigquery.dataEditor" - - member: "serviceAccount:service-org-{organization_id}@gcp-sa-osconfig.iam.gserviceaccount.com" - role: "roles/osconfig.serviceAgent" - vars: - dataset_id: "my-dataset" - network_name: "my-network" - test_env_vars: - org_id: "ORG_ID" - test_vars_overrides: - network_name: 'acctest.BootstrapSharedServiceNetworkingConnection(t, "service-resource-network-config")' - min_version: "beta" - ignore_read_extra: - - 'foo' - exclude_test: true - exclude_docs: true - exclude_import_test: true - skip_vcr: true - skip_test: "https://github.com/hashicorp/terraform-provider-google/issues/20574" - external_providers: - - "time" -``` \ No newline at end of file +See the dedicated [MMv1 sample reference ↗]({{< ref "/reference/sample" >}}) page for a comprehensive list of all top-level sample and nested step attributes. \ No newline at end of file diff --git a/docs/content/reference/sample.md b/docs/content/reference/sample.md new file mode 100644 index 000000000000..7a36195b9812 --- /dev/null +++ b/docs/content/reference/sample.md @@ -0,0 +1,87 @@ +--- +title: "MMv1 sample reference" +weight: 25 +--- + +# MMv1 sample reference + +A sample is a collection of one or more `steps`, where each step represents a distinct Terraform configuration and test step (for example, create, update). + +Each sample supports the following attributes at the top level, with more granular control inside each step. + +--- + +## Sample Attributes (Top-Level) + +These attributes are defined once for an entire sample. + +* `name`: `snake_case` name for the overall sample. This is used for generating test names. +* `primary_resource_id`: The ID of the main resource under test for the entire sample. Tests use this to run additional checks automatically. +* `primary_resource_type`: Optional resource type override of the primary resource. Used for import assertion validations. +* `bootstrap_iam`: Specify member/role pairs that should exist before the test runs. This avoids race conditions on global IAM permissions. `{project_number}` and `{organization_id}` are replaced automatically. +* `min_version`: Sets a minimum provider version for the entire sample (for example, `beta`). This can be overridden by the `min_version` attribute within a specific step. +* `exclude_test`: If `true`, no tests are generated for this entire sample. +* `exclude_basic_doc`: If `true`, excludes the first step of this sample from the generated documentation. By default, the first step is automatically included as a use case in the documentation. Use this if you want to skip it. +* `skip_vcr`: If `true`, skips VCR testing for the entire sample. +* `skip_test`: If not empty, the entire sample is skipped during tests. The value should be a link to a ticket explaining why. +* `skip_func`: A custom function call to run to determine if tests should be skipped. +* `region_override`: Overrides location/region identifiers specifically inside IAM assertion checks. +* `external_providers`: A list of external providers (such as `random`, `time`) needed for the sample. +* `tgc_skip_test`: Skips generated conversion tests specifically running inside the TGC (Terraform Google Conversion) suite (value should be a ticket link reason). + +--- + +## Step Attributes + +A sample contains a list of one or more `steps`. Each step has its own configuration and test-specific attributes. + +* `name`: `snake_case` name of the individual step. This is used for generating test configuration function names and documentation headers. +* `config_path`: The path to the step's configuration file. If omitted, it defaults to `templates/terraform/samples/services/{{product}}/{{step_name}}.tf.tmpl`. +* `resource_id_vars`: Key/value pairs to inject into the configuration file. Reference them with `{{index $.ResourceIdVars "key"}}`. Values here automatically receive a `tf-test` prefix and random suffix, unless they contain an underscore `_`, in which case they receive a `tf_test` prefix and random suffix. **Note:** If a resource name doesn't support hyphens `-` or underscores `_`, use `test_vars_overrides` instead. For non-identifier variables, use `vars`. +* `vars`: Key/value pairs that are copied directly to tests without a prefix. Reference with `{{index $.Vars "key"}}`. **Note:** This should ONLY be used for fields that vary between steps (for example, to test update functionality). Constant values should be hardcoded directly in the `.tf.tmpl` file. +* `test_env_vars`: Key/value pairs that map variable names to environment variables for tests (for example, `PROJECT_NAME`, `REGION`, `ORG_ID`). +* `test_vars_overrides`: Key/value pairs to override variables with literal values or function calls specifically for tests. +* `oics_vars_overrides`: Key/value pairs to override variables with literal values specifically for Open in Cloud Shell (OiCS) tutorial generation. +* `min_version`: Overrides the sample-level `min_version` for this specific step. +* `ignore_read_extra`: A list of properties to ignore during the import test for this step, typically for write-only fields. +* `exclude_import_test`: If `true`, no import test is generated for this specific step. +* `include_step_doc`: If `true`, forces this specific step to be included in the generated documentation. By default, only the first step of a sample is included in the documentation as a use case. Use this on later steps to showcase update scenarios or complex configurations. This will override a top-level `exclude_basic_doc` setting if applied to the first step. + +--- + +## Example + +```yaml +samples: + - name: service_resource_update + primary_resource_id: example + bootstrap_iam: + - member: "serviceAccount:service-{project_number}@gcp-sa-healthcare.iam.gserviceaccount.com" + role: "roles/bigquery.dataEditor" + - member: "serviceAccount:service-org-{organization_id}@gcp-sa-osconfig.iam.gserviceaccount.com" + role: "roles/osconfig.serviceAgent" + min_version: "beta" + skip_vcr: true + external_providers: + - "time" + steps: + - name: service_resource_minimal # Matches templates/terraform/samples/services/{{product}}/service_resource_minimal.tf.tmpl + vars: # Varies between steps to test update functionality + description: "A minimal description" + resource_id_vars: # Used for resource id in the configuration file + dataset_id: "my-dataset" + network_name: "my-network" + test_env_vars: + org_id: "ORG_ID" + test_vars_overrides: + network_name: 'acctest.BootstrapSharedServiceNetworkingConnection(t, "service-resource-network-config")' + ignore_read_extra: + - 'foo' + exclude_import_test: true + - name: service_resource_update # Matches templates/terraform/samples/services/{{product}}/service_resource_update.tf.tmpl + vars: + description: "An updated description" # This value updates the description field + resource_id_vars: + dataset_id: "my-dataset" + network_name: "my-network" +``` diff --git a/docs/content/reference/update-test-changes.md b/docs/content/reference/update-test-changes.md new file mode 100644 index 000000000000..596a7cec6b36 --- /dev/null +++ b/docs/content/reference/update-test-changes.md @@ -0,0 +1,168 @@ +--- +title: "Test Template Migration Guide" +weight: 50 +--- +# Test Template Migration Guide + +The test generation framework in Magic Modules has been updated to natively support multi-step tests (create and update) directly from the resource's YAML definition. This guide is for experienced contributors who are familiar with the previous `examples` based workflow and provides a concise overview of what has changed. + +## Transition Timeline + +* **Recommended Path:** The new `samples` configuration block is the recommended path for adding tests. +* **Legacy Support:** The old `examples` block path is still supported until Mid May 2026. +* **Action Required:** Contributors are encouraged to use `samples` for all new tests and migrate existing `examples` to `samples`. + + + +## YAML Changes: From `examples` to `samples` and `steps` + +The most significant change is the replacement of the `examples` block with a new `samples` block. Each `sample` can now contain one or more `steps`, where the first step corresponds to the resource creation and any subsequent steps correspond to updates. + +This new structure eliminates the need for handwritten update tests for MMv1 resources, as they can now be generated automatically. + +### Create Test Comparison + +A simple create test is now defined as a sample with a single step. + +Old `examples` format + +```yaml +examples: + - name: "pubsub_topic_basic" + primary_resource_id: "example" + vars: + topic_name: "example-topic" + field_1: "value-one" + field_2: "value-two" + test_vars_overrides: + field_1: "value-one" + field_2: "value-two" +``` + +New `samples` format + +```yaml +samples: + - name: "pubsub_topic_basic" + primary_resource_id: "example" + steps: + - name: "pubsub_topic_basic" + resource_id_vars: + topic_name: "example-topic" + vars: + field_1: "value-one" + field_2: "value-two" +``` + +> [!NOTE] +> ### Why `test_vars_overrides` is no longer needed +> In the old `examples` block, any key placed under `vars` was automatically appended with a random suffix (for example, `example-topic-12345`). If you wanted to pass a plain value (without random suffixes), you had to use `test_vars_overrides`. +> +> In the new `samples` format, these two use cases are explicitly separated into different fields within a step: +> * **`resource_id_vars`**: **Use this exclusively for resource identifier variables (such as resource names or IDs).** It will automatically prepend a `tf-test` (or `tf_test`) prefix and append a random suffix. If a resource identifier doesn't support hyphens `-` or underscores `_`, use `test_vars_overrides` instead. For non-identifier variables, use `vars`. +> * **`vars`**: Use this for plain literal values that should be passed to the test exactly as written (replaces the need for `test_vars_overrides`). **Note:** This should ONLY be used for fields that vary between steps (for example, to test update functionality). Constant values should be hardcoded directly in the `.tf.tmpl` file. + +### Update Test Comparison + +Previously, update tests had to be handwritten. Now, they can be defined by adding a second step to a sample. + +New samples format (Create and Update) + +```yaml +samples: + - name: "pubsub_topic_update" + primary_resource_id: "example" + steps: + - name: "pubsub_topic_full" + resource_id_vars: + topic_name: "example-topic" + vars: + field_1: "value-one" + field_2: "value-two" + - name: "pubsub_topic_full" + resource_id_vars: + topic_name: "example-topic" + vars: + field_1: "value-one-updated" + field_2: "value-two-updated" +``` + +## YAML Field Migration + +The migration moves fields from the old `Examples` structure to either the new top-level `Sample` or the nested `Step`. The following tables detail where each field has been moved. + +### Fields Mapped to `Sample` (Top-Level) +These fields remain at the top level, moving from the old `examples` object to the new `sample` object. + +| Old Field | New Location | Notes | +| :--- | :--- | :--- | +| `examples.name` | `sample.name` | Used for the overall sample name. A `step.name` is also used for each step. | +| `examples.primary_resource_id` | `sample.primary_resource_id` | Remains at the sample level. | +| `examples.primary_resource_type` | `sample.primary_resource_type` | Remains at the sample level. | +| `examples.bootstrap_iam` | `sample.bootstrap_iam` | Remains at the sample level. | +| `examples.min_version` | `sample.min_version` | Remains at the sample level. | +| `examples.exclude_test` | `sample.exclude_test` | Remains at the sample level. | +| `examples.region_override` | `sample.region_override` | Remains at the sample level. | +| `examples.skip_vcr` | `sample.skip_vcr` | Remains at the sample level. | +| `examples.skip_test` | `sample.skip_test` | Remains at the sample level. | +| `examples.skip_func` | `sample.skip_func` | Remains at the sample level. | +| `examples.external_providers` | `sample.external_providers` | Remains at the sample level. | +| `examples.tgc_skip_test` | `sample.tgc_skip_test` | Remains at the sample level. | + +--- + +### Fields Mapped to `Step` +These fields are now configured within each individual `Step` of a `Sample`. + +| Old Field | New Location | Notes | +| :--- | :--- | :--- | +| `examples.vars` | `step.resource_id_vars` | The entire `vars` map is moved into `resource_id_vars` within each step. | +| `examples.test_env_vars` | `step.test_env_vars` | Moved to the step level. | +| `examples.test_vars_overrides` | `step.test_vars_overrides` | Moved to the step level. | +| `examples.oics_vars_overrides` | `step.oics_vars_overrides` | Moved to the step level. | +| `examples.ignore_read_extra` | `step.ignore_read_extra` | Moved to the step level. | +| `examples.exclude_docs` | `sample.exclude_basic_doc` | Replaced by `sample.exclude_basic_doc` (or `step.include_step_doc` to override). | +| `examples.exclude_import_test` | `step.exclude_import_test` | Moved to the step level. | +| `examples.config_path` | `step.config_path` | Path is updated to the new service-specific directory within the step. | + +--- + +### New Fields +This field is new in the `steps` object and has no direct equivalent in the old `examples` structure. + +| Old Field | New Location | Notes | +| :--- | :--- | :--- | +| *(N/A)* | `step.vars` | Newly added at the step level. Values are copied directly to tests. | +| *(N/A)* | `step.min_version` | Newly added to set a version for a specific step | +| *(N/A)* | `step.include_step_doc` | Explicitly forcing a step's docs generation (overriding Sample block). | + + +## Template `.tf.`tmpl File Changes + +The location for template files has moved from `templates/terraform/examples/` to a service-specific directory under `templates/terraform/samples/services/`. + +Additionally, the variable object passed into the templates has been updated. `$.ResourceIdVars` will append `tf-test` prefixes and random string suffixes, which is used for resource identifiers in most cases. `$.Vars` will apply plain values from the YAML configuration. + +### Example .tf.tmpl variables + +Old template `pubsub_topic_basic.tf.tmpl` (in `templates/terraform/examples/`) + +```tf +resource "google_pubsub_topic" "{{$.PrimaryResourceId}}" { + name = "{{index $.Vars "topic_name"}}" + + field_1 = "{{index $.Vars "field_1"}}" + field_2 = "{{index $.Vars "field_2"}}" +} +``` + +New template `pubsub_topic_basic.tf.tmpl` (in `templates/terraform/samples/services/pubsub/`) + +```tf +resource "google_pubsub_topic" "{{$.PrimaryResourceId}}" { + name = "{{index $.ResourceIdVars "topic_name"}}" + + field_1 = "{{index $.Vars "field_1"}}" + field_2 = "{{index $.Vars "field_2"}}" +} +``` \ No newline at end of file diff --git a/docs/content/test/run-tests.md b/docs/content/test/run-tests.md index 770758a07364..ef61de5e632e 100644 --- a/docs/content/test/run-tests.md +++ b/docs/content/test/run-tests.md @@ -62,7 +62,7 @@ aliases: make testacc TEST=./google/services/container TESTARGS='-run=TestAccContainerNodePool_basic$$' ``` - To run all tests matching, e.g., `TestAccContainerNodePool*`, omit the trailing `$$`: + To run all tests matching, for example, `TestAccContainerNodePool*`, omit the trailing `$$`: ```bash make testacc TEST=./google/services/container TESTARGS='-run=TestAccContainerNodePool' @@ -100,7 +100,7 @@ aliases: make testacc TEST=./google-beta/services/container TESTARGS='-run=TestAccContainerNodePool_basic$$' ``` - To run all tests matching, e.g., `TestAccContainerNodePool*`, omit the trailing `$$`: + To run all tests matching, for example, `TestAccContainerNodePool*`, omit the trailing `$$`: ```bash make testacc TEST=./google-beta/services/container TESTARGS='-run=TestAccContainerNodePool' @@ -171,7 +171,7 @@ Tests require all of the providers they use (except the one actually being teste ``` - GA+beta test: This indicates that one of the `google_*` resources in the test has `provider = google-beta` set. `provider = google-beta` can't be set unless the test is beta-only. - If the error mentions some other provider: The test relies on an external provider, such as `time`, and that is not explicitly declared - - For MMv1 example-based tests, use [`examples.external_providers`](https://googlecloudplatform.github.io/magic-modules/reference/resource/#examples). + - For MMv1 sample-based tests, use [`samples.external_providers`]({{< ref "/reference/sample/#sample-attributes-top-level" >}}). - For Handwritten tests, use TestCase.ExternalProviders: ```go acctest.VcrTest(t, resource.TestCase{ diff --git a/docs/content/test/test.md b/docs/content/test/test.md index c18bacefb8d3..e07d7c601cac 100644 --- a/docs/content/test/test.md +++ b/docs/content/test/test.md @@ -73,89 +73,125 @@ func TestSignatureAlgorithmDiffSuppress(t *testing.T) { } ``` -## Add a create test +## Add an acceptance test -A create test is an **acceptance test** that creates the target resource and immediately destroys it. +An **acceptance test** verifies that a resource can be created, updated, and destroyed successfully. -> **Note:** All resources should have a "basic" create test, which uses the smallest possible number of fields. Additional create tests can be used to ensure all fields on the resource are used in at least one test. +> **Note:** All resources should have a "basic" test covering the minimal required fields. Additional tests should be added to cover all fields, with updatable fields being covered by an update step. Updatable fields are fields that can be updated without recreating the entire resource; that is, they are not marked `immutable` in MMv1 or `ForceNew` in handwritten code -{{% tabs "create" %}} +{{% tabs "add-acceptance-test" %}} {{< tab "MMv1" >}} -1. Add an entry to your `RESOURCE_NAME.yaml` file's `examples`. The fields listed here are the most commonly-used. For a comprehensive reference, see [MMv1 resource reference: `examples` ↗]({{}}). + +### Steps + +1. Add an entry to your `RESOURCE_NAME.yaml` file's `samples` list. Each sample can contain multiple steps. The first step will generate a `create` test, and any subsequent steps will generate `update` tests. For a comprehensive reference, see [MMv1 sample reference ↗]({{< ref "/reference/sample" >}}). + + When defining variables for your steps, follow these guidelines: + - **Use `resource_id_vars` for resource identifiers** (like names or IDs) that need to be unique. Values automatically receive a `tf-test` prefix and random suffix, unless they contain an underscore `_`, in which case they receive a `tf_test` prefix and random suffix. If a resource name doesn't support hyphens `-` or underscores `_`, use `test_vars_overrides` instead. For non-identifier variables, use `vars`. + - **Use `vars` only for fields that vary between steps** (for example, to test the update functionality of specific fields). + - **Hardcode all other values** directly in the `.tf.tmpl` configuration file. Don't use `vars` for values that remain constant across all steps. ```yaml - examples: - # name must correspond to a configuration file that you'll create in the next step. - # The name should include the product name, resource name, and a basic description - # of the test. This will be used to generate the test name and the documentation - # header. - - name: "PRODUCT_RESOURCE_basic" + samples: + # name is used to generate the test name. + - name: "pubsub_topic_update" # primary_resource_id will be used for the Terraform resource id in the configuration file. - primary_resource_id: "example" - # vars contains key/value pairs of variables to inject into the configuration file. - # These can be referenced in the configuration file as a key inside `{{$.Vars}}`. - # All resource IDs (even for resources not under test) should be declared - # with variables that contain a `-` or `_`; this will ensure that, in tests, - # the resources are created with a `tf-test` prefix to allow automatic cleanup - # of dangling resources and a random suffix to avoid name collisions. - vars: - network_name: "example-network" - # test_vars_overrides contains key/value pairs of literal overrides for - # variables used in tests. This can be used to call functions to - # generate or determine a variable's value – for example, bootstrapping - # a shared network for your product to avoid test failures due to limits - # on the default network. - test_vars_overrides: - network_name: 'acctest.BootstrapSharedServiceNetworkingConnection(t, "PRODUCT-RESOURCE-network-config")' - # Set min_version: beta if the resource is not beta-only and any beta-only fields are being tested. + primary_resource_id: "default" + # min_version can be set at the top level if it applies to all steps. min_version: beta + steps: + # The first step defines the initial create configuration. + # Step name: Matches the template file name by default (for example, pubsub_topic_minimal.tf.tmpl). Use config_path to override this. + - name: "pubsub_topic_minimal" + # resource_id_vars contains key/value pairs to inject into the configuration file. + # These can be referenced as a key inside `{{$.ResourceIdVars}}`. + resource_id_vars: + resource_name: "example-resource" + network_name: "example-network" + # test_vars_overrides contains literal overrides for variables in tests. + test_vars_overrides: + network_name: 'acctest.BootstrapSharedServiceNetworkingConnection(t, "pubsub-topic-network-config")' + # Subsequent steps define update configurations. + - name: "pubsub_topic_full" + resource_id_vars: + resource_name: "example-resource" + network_name: "example-network" + test_vars_overrides: + network_name: 'acctest.BootstrapSharedServiceNetworkingConnection(t, "pubsub-topic-network-config")' + # vars should ONLY be used for fields that vary between steps. + # Fields that stay constant across steps should be hardcoded in the .tf.tmpl file. + vars: + display_name: "Display Name" + - name: "pubsub_topic_full" + resource_id_vars: + resource_name: "example-resource" + network_name: "example-network" + test_vars_overrides: + network_name: 'acctest.BootstrapSharedServiceNetworkingConnection(t, "pubsub-topic-network-config")' + vars: + display_name: "Updated Display Name" # The new value for the updatable field ``` -2. Create a `.tf.tmpl` file in [`mmv1/templates/terraform/examples/`](https://github.com/GoogleCloudPlatform/magic-modules/tree/main/mmv1/templates/terraform/examples). The name of the file should match the name of the example created in the previous step. For example, `PRODUCT_RESOURCE_basic.tf.tmpl`. -3. In that file, write the Terraform configuration for your test. This should include all of the required dependencies. For example, `google_compute_subnetwork` has a dependency on `google_compute_network`: +2. Create one or more `.tf.tmpl` files in `mmv1/templates/terraform/samples/services/SERVICE_NAME/`. The file names should match the step name (for example, `pubsub_topic_minimal.tf.tmpl` and `pubsub_topic_full.tf.tmpl`). +3. In those files, write the Terraform configuration for your test steps. This should include all required dependencies. + + `pubsub_topic_minimal.tf.tmpl`: ```tf - resource "google_compute_subnetwork" "{{$.PrimaryResourceId}}" { - name = "{{index $.Vars "subnetwork_name"}}" - ip_cidr_range = "10.1.0.0/16" - region = "us-central1" + resource "google_pubsub_topic" "{{.PrimaryResourceId}}" { + name = "{{index $.ResourceIdVars "resource_name"}}" network = google_compute_network.network.name + + labels = { + env = "test" + } } resource "google_compute_network" "network" { - name = "{{index $.Vars "network_name"}}" + name = "{{index $.ResourceIdVars "network_name"}}" auto_create_subnetworks = false + routing_mode = "REGIONAL" } ``` -4. If the resource or the example is beta-only: - - Add `provider = google-beta` to every resource in the file. -{{< /tab >}} -{{< tab "Handwritten" >}} -This section assumes you've used the [Add a resource]({{< ref "/develop/add-resource" >}}) guide to create your handwritten resource, and you have a working MMv1 config. -> **Note:** If not, you can create one now, or skip this guide and construct the test by hand. Writing tests by hand can sometimes be a better option if there is a similar test you can copy from. + `pubsub_topic_full.tf.tmpl`: (This file added the `display_name` variable). + ```tf + resource "google_pubsub_topic" "{{.PrimaryResourceId}}" { + name = "{{index $.ResourceIdVars "resource_name"}}" + + # This is an example of a field whose value changes between steps + # to test update functionality of display_name + display_name = "{{index $.Vars "display_name"}}" + + # The rest of the fields can be baked in the configuration directly + message_retention_duration = "86600s" + labels = { + env = "test" + } + + network = google_compute_network.network.name + } -1. Add the test in MMv1. Repeat for all the create tests you will need. -2. [Generate the beta provider]({{< ref "/develop/generate-providers" >}}). -3. From the beta provider, copy and paste the generated `*_generated_test.go` file into the appropriate service folder inside [`magic-modules/mmv1/third_party/terraform/services`](https://github.com/GoogleCloudPlatform/magic-modules/tree/main/mmv1/third_party/terraform/services/) as a new file call `*_test.go`. -4. Modify the tests as needed. - - Replace all occurrences of `github.com/hashicorp/terraform-provider-google-beta/google-beta` with `github.com/hashicorp/terraform-provider-google/google` - - Remove the comments at the top of the file. - - Remove the `Example` suffix from all function names. - - If beta-only fields are being tested, do the following: - - Change the file suffix to `.go.tmpl` - - Wrap each beta-only test in a separate version guard: `{{- if ne $.TargetVersionName "ga" -}}...{{- else }}...{{- end }}` - - In each beta-only test, ensure that the TestCase sets `ProtoV5ProviderFactories: acctest.ProtoV5ProviderBetaFactories(t)` - - In each beta-only test, ensure that all Terraform resources in all configs have `provider = google-beta` set -{{< /tab >}} -{{% /tabs %}} + resource "google_compute_network" "network" { + name = "{{index $.ResourceIdVars "network_name"}}" + auto_create_subnetworks = false + routing_mode = "REGIONAL" + } + ``` -## Add an update test +4. For beta-only resources or features: + - If the resource or the whole test is beta-only + - Add `provider = google-beta` to every resource in the file. + - Add `min_version: beta` at the top sample level + - If only a single test step is beta-only + - Add `min_version: beta` at the individual step level + +{{< /tab >}} +{{< tab "Handwritten" >}} -An update test is an **acceptance test** that creates the target resource and then makes updates to fields that are updatable. Updatable fields are fields that can be updated without recreating the entire resource; that is, they are not marked `immutable` in MMv1 or `ForceNew` in handwritten code. +> [!NOTE] +> This workflow is an alternative for when the recommended `samples` generator framework is insufficient (for example, when you require a custom `CheckFunction` or other complex test assertions). -> **Note:** All updatable fields must be covered by at least one update test. In most cases, only a single update test is needed to test all fields at once. +An update test ensures that updatable fields can be changed without recreating the entire resource. All updatable fields must be covered by at least one update test (often a single update step covering all fields is sufficient). -{{% tabs "update" %}} -{{< tab "MMv1" >}} 1. [Generate the beta provider]({{< ref "/develop/generate-providers" >}}). 2. From the beta provider, copy and paste the generated `*_generated_test.go` file into the appropriate service folder inside [`magic-modules/mmv1/third_party/terraform/services`](https://github.com/GoogleCloudPlatform/magic-modules/tree/main/mmv1/third_party/terraform/services) as a new file call `*_test.go`. 3. Using an editor of your choice, delete the `*DestroyProducer` function, and all but one test. The remaining test should be the "full" test, or if there is no "full" test, the "basic" test. This will be the starting point for your new update test. @@ -218,64 +254,67 @@ An update test is an **acceptance test** that creates the target resource and th - Wrap each beta-only test in a separate version guard: `{{- if ne $.TargetVersionName "ga" -}}...{{- else }}...{{- end }}` - In each beta-only test, ensure that the TestCase sets `ProtoV5ProviderFactories: acctest.ProtoV5ProviderBetaFactories(t)` - In each beta-only test, ensure that all Terraform resources in all configs have `provider = google-beta` set + {{< /tab >}} -{{< tab "Handwritten" >}} -1. Using an editor of your choice, open the existing `*_test.go` or `*_test.go.tmpl` file in the appropriate service folder inside [`magic-modules/mmv1/third_party/terraform/services`](https://github.com/GoogleCloudPlatform/magic-modules/tree/main/mmv1/third_party/terraform/services) which contains your create tests. -2. Copy the `TestAcc*` *test function* for the existing "full" test. If there is no "full" test, use the "basic" test. This will be the starting point for your new update test. -3. Modify the test function to support updates. - - Change the suffix of the test function to `_update`. - - Copy the 2 `TestStep` blocks and paste them immediately after, so that there are 4 total test steps. - - Change the suffix of the second `Config` value to `_update`. - - Add `ConfigPlanChecks` to the update step of the test to ensure the resource is updated in-place. - - The resulting test function would look similar to this: - ```go - import "github.com/hashicorp/terraform-plugin-testing/plancheck" +{{% /tabs %}} - func TestAccPubsubTopic_update(t *testing.T) { - ... - acctest.VcrTest(t, resource.TestCase{ - ... - Steps: []resource.TestStep{ - { - Config: testAccPubsubTopic_full(...), - }, - { - ... - }, - { - Config: testAccPubsubTopic_update(...), - ConfigPlanChecks: resource.ConfigPlanChecks{ - PreApply: []plancheck.PlanCheck{ - plancheck.ExpectResourceAction("google_pubsub_topic.foo", plancheck.ResourceActionUpdate), - }, - }, - }, - { - ... - }, - }, - }) - } +## Legacy: Examples-based Framework + +> [!WARNING] +> The `examples` block is legacy and has been replaced by `samples`. This footprint is still supported for approximately 2 months to allow for transition. New tests should use the `samples` framework. + +Tests within the legacy `examples` generator are configured directly on the resource YAML. **Note that this legacy framework only supports generating create tests;** multi-step update tests must be configured via the new `samples` framework or handwritten manual testing procedures. + +### Steps + +1. Add an entry to your `RESOURCE_NAME.yaml` file's `examples`. The fields listed here are the most commonly-used. For a comprehensive reference, see [MMv1 resource reference: `examples` ↗]({{}}). + ```yaml + examples: + # name must correspond to a configuration file that you'll create in the next step. + # The name should include the product name, resource name, and a basic description + # of the test. This will be used to generate the test name and the documentation + # header. + - name: "PRODUCT_RESOURCE_basic" + # primary_resource_id will be used for the Terraform resource id in the configuration file. + primary_resource_id: "example" + # vars contains key/value pairs of variables to inject into the configuration file. + # These can be referenced in the configuration file as a key inside `{{$.Vars}}`. + # All resource IDs (even for resources not under test) should be declared + # with variables that contain a `-` or `_`; this will ensure that, in tests, + # the resources are created with a `tf-test` prefix to allow automatic cleanup + # of dangling resources and a random suffix to avoid name collisions. + vars: + network_name: "example-network" + # test_vars_overrides contains key/value pairs of literal overrides for + # variables used in tests. This can be used to call functions to + # generate or determine a variable's value – for example, bootstrapping + # a shared network for your product to avoid test failures due to limits + # on the default network. + test_vars_overrides: + network_name: 'acctest.BootstrapSharedServiceNetworkingConnection(t, "PRODUCT-RESOURCE-network-config")' + # Set min_version: beta if the resource is not beta-only and any beta-only fields are being tested. + min_version: beta ``` -4. Add a Terraform *template function* to support updates. - - Copy the full (or basic) `testAcc*` template function. - - Change the suffix of the new template function to `_update`. - - The new template function would look similar to this: - ```go - func testAccPubsubTopic_update(...) string { - ... + +2. Create a `.tf.tmpl` file in [`mmv1/templates/terraform/examples/`](https://github.com/GoogleCloudPlatform/magic-modules/tree/main/mmv1/templates/terraform/examples). The name of the file should match the name of the example created in the previous step. For example, `PRODUCT_RESOURCE_basic.tf.tmpl`. +3. In that file, write the Terraform configuration for your test. This should include all of the required dependencies. For example, `google_compute_subnetwork` has a dependency on `google_compute_network`: + ```tf + resource "google_compute_subnetwork" "{{$.PrimaryResourceId}}" { + name = "{{index $.Vars "subnetwork_name"}}" + ip_cidr_range = "10.1.0.0/16" + region = "us-central1" + network = google_compute_network.network.name + } + + resource "google_compute_network" "network" { + name = "{{index $.Vars "network_name"}}" + auto_create_subnetworks = false } ``` -5. Modify the test as needed. - - Modify the new template function so that updatable fields are changed or removed. This may require additions to the `context` map in the test function. - - Remove the comments at the top of the file. - - If beta-only fields are being tested, do the following: - - Change the file suffix to `.go.tmpl` - - Wrap each beta-only test in a separate version guard: `{{- if ne $.TargetVersionName "ga" -}}...{{- else }}...{{- end }}` - - In each beta-only test, ensure that the TestCase sets `ProtoV5ProviderFactories: acctest.ProtoV5ProviderBetaFactories(t)` - - In each beta-only test, ensure that all Terraform resources in all configs have `provider = google-beta` set -{{< /tab >}} -{{% /tabs %}} +4. If the resource or the example is beta-only: + - Add `provider = google-beta` to every resource in the file. + +For instructions on how to migrate existing `examples` over to `samples`, see the [Test Template Migration Guide]({{< ref "/reference/update-test-changes" >}}). ## Bootstrap API resources {#bootstrapping} @@ -300,13 +339,15 @@ Example usage: {{% tabs "bootstrap-cryptokeys" %}} {{< tab "MMv1" >}} ```yaml -examples: +samples: - name: service_resource_basic primary_resource_id: example - vars: - kms_key_name: 'kms-key' - test_vars_overrides: - kms_key_name: 'acctest.BootstrapKMSKey(t).CryptoKey.Name' + steps: + - name: service_resource_basic + resource_id_vars: + kms_key_name: 'kms-key' + test_vars_overrides: + kms_key_name: 'acctest.BootstrapKMSKey(t).CryptoKey.Name' ``` {{< /tab >}} {{< tab "Handwritten" >}} @@ -336,24 +377,29 @@ Example usage: {{< tab "MMv1" >}} ```yaml # Project-level IAM -examples: +samples: - name: service_resource_basic primary_resource_id: example bootstrap_iam: - member: "serviceAccount:service-{project_number}@gcp-sa-healthcare.iam.gserviceaccount.com" role: "roles/bigquery.dataEditor" + steps: + - name: service_resource_basic + config_path: samples/basic.tf.tmpl ``` ```yaml # Org-level IAM -examples: +samples: - name: service_resource_basic primary_resource_id: example bootstrap_iam: - member: "serviceAccount:service-org-{organization_id}@gcp-sa-osconfig.iam.gserviceaccount.com" role: "roles/osconfig.serviceAgent" - test_env_vars: - org_id: ORG_TARGET + steps: + - name: service_resource_basic + test_env_vars: + org_id: ORG_TARGET # Resolves to envvar.GetTestOrgTargetFromEnv in tests ``` {{< /tab >}} {{< tab "Handwritten" >}} @@ -416,15 +462,17 @@ Example usage: {{% tabs "bootstrap-networks" %}} {{< tab "MMv1" >}} ```yaml -examples: +samples: - name: service_resource_basic primary_resource_id: example - vars: - network_name: 'default' - subnetwork_name: 'default' - test_vars_overrides: - network_name: 'acctest.BootstrapSharedTestNetwork(t, "network-identifier")' - subnetwork_name: 'acctest.BootstrapSubnet(t, "subnet-identifier", acctest.BootstrapSharedTestNetwork(t, "network-identifier"))' + steps: + - name: service_resource_basic + resource_id_vars: + network_name: 'default' + subnetwork_name: 'default' + test_vars_overrides: + network_name: 'acctest.BootstrapSharedTestNetwork(t, "network-identifier")' + subnetwork_name: 'acctest.BootstrapSubnet(t, "subnet-identifier", acctest.BootstrapSharedTestNetwork(t, "network-identifier"))' ``` {{< /tab >}} {{< tab "Handwritten" >}} @@ -527,12 +575,14 @@ Acceptance tests are run in VCR replaying mode on PRs (using pre-recorded HTTP r Skipping acceptance tests that are generated from example files can be achieved by adding `skip_vcr: true` in the example's YAML: ```yaml -examples: +samples: - name: 'bigtable_app_profile_anycluster' ... # bigtable instance does not use the shared HTTP client, this test creates an instance skip_vcr: true + steps: + - name: service_resource_basic ``` If you skip a test in VCR mode, include a code comment explaining the reason for skipping (for example, a link to a GitHub issue.) @@ -566,10 +616,12 @@ Please include a comment with context where the skip is defined. Skipping acceptance tests that are generated from example files can be achieved by adding `skip_func: acctest.SkipTestUntil(t, "YYYY-MM-DD")` in the example's YAML: ```yaml -examples: +samples: - name: 'compute_address_basic' ... skip_func: acctest.SkipTestUntil(t, "2026-01-31") # waiting for rollout + steps: + - name: service_resource_basic ``` {{< /tab >}} @@ -591,7 +643,7 @@ func TestAccPubsubTopic_update(t *testing.T) { | Problem | How to fix/Other info | Skip in VCR replaying? | | ------------------------------------------------ | ---------------------- |------------- | -| *Incorrect or insufficient data is present in VCR recordings to replay tests*. Tests will fail with `Requested interaction not found` errors during REPLAYING mode | Make sure that you're not introducing randomness into the test, e.g. by unnecessarily using the random provider to set a resource's name.| If you cannot avoid this issue you should skip the test, but try to ensure that it cannot be fixed first.| +| *Incorrect or insufficient data is present in VCR recordings to replay tests*. Tests will fail with `Requested interaction not found` errors during REPLAYING mode | Make sure that you're not introducing randomness into the test, such as by unnecessarily using the random provider to set a resource's name.| If you cannot avoid this issue you should skip the test, but try to ensure that it cannot be fixed first.| *Bigtable acceptance tests aren't working in VCR mode*. `Requested interaction not found` errors are seen during Bigtable tests run in REPLAYING mode | Currently the provider uses a separate client than the rest of the provider to interact with the Bigtable API. As HTTP traffic to the Bigtable API doesn't go via the shared client it cannot be recorded in RECORDING mode.| Skip the test in VCR for Bigtable. | | *Using multiple provider aliases doesn't work in VCR*. You may have two instances of the google provider in the test config but one of them doesn't seem to be using its provider arguments - for example, using the wrong default project. | See this GitHub issue: https://github.com/hashicorp/terraform-provider-google/issues/20019 . The problem is that, due to how the VCR system works, one provider instance will be configured and the other will be forced to reuse the first instance's configuration, despite them being given different provider arguments. | Skip the test in VCR is using aliases is unavoidable. | | *Using multiple versions of the google/google-beta provider in a single test isn't working in VCR*. Unexpected test failures may occur during tests in REPLAYING mode where `ExternalProviders` is used to pull in past versions of the google/google-beta provider. | When ExternalProviders is used to pulling in other versions of the provider, any HTTP traffic through the external provider will not be recorded. If the HTTP traffic produces an unexpected result or returns an API error then the test will fail in REPLAYING mode. | Skip the test in VCR when testing the current provider behaviour versus previous released versions. | @@ -611,5 +663,5 @@ These tests can still run in VCR replaying mode; however, REPLAYING mode can't b ## References * [Official Terraform documentation on Acceptance Tests](https://developer.hashicorp.com/terraform/plugin/sdkv2/testing/acceptance-tests) -* [MMv1 resource reference: `examples` ↗]({{}}) +* [MMv1 resource reference: `samples` ↗]({{}})