Update Test Generation: new contribution documentation (Relaunch)

docs/content/reference/resource.md

@@ -486,69 +486,8 @@ properties:
     type: String
 ```
 
-## Examples
+## Samples
 
-### `examples`
+`Samples` are configurations used to generate documentation and tests. Each sample contains one or more `steps` representing consecutive update test footprints.
 
-A list of configurations that are used to generate documentation and tests. Each example supports the following common
-attributes – for a full reference, see
-[examples.go ↗](https://github.com/GoogleCloudPlatform/magic-modules/blob/main/mmv1/api/resource/examples.go):
-
-- `name`: snake_case name of the example. This corresponds to the configuration file in
-  [mmv1/templates/terraform/examples](https://github.com/GoogleCloudPlatform/magic-modules/tree/main/mmv1/templates/terraform/examples) (excluding the `.go.tmpl` suffix) and is used to generate the test name and the documentation header.
-- `primary_resource_id`: The id of the resource under test. This is used by tests to automatically run additional checks.
-  Configuration files should reference this to avoid getting out of sync. For example:
-  `resource "google_compute_address" ""{{$.PrimaryResourceId}}" {`
-- `bootstrap_iam`: specify member/role pairs that should always exist. `{project_number}` will be replaced with the
-  default project's project number, and `{organization_id}` will be replaced with the "target" test organization's ID. This avoids race conditions when modifying the global IAM permissions.
-  Permissions attached to resources created _in_ a test should instead be provisioned with standard terraform resources.
-- `vars`: Key/value pairs of variables to inject into the configuration file. These can be referenced in the configuration file
-  with `{{index $.Vars "key"}}`. All resource IDs (even for resources not under test) should be declared with variables that
-  contain a `-` or `_`; this will ensure that, in tests, the resources are created with a `tf-test` prefix to allow automatic cleanup of dangling resources and a random suffix to avoid name collisions.
-- `test_env_vars`: Key/value pairs of variable names and special values indicating variables that should be pulled from the
-  environment during tests. These will receive a neutral default value in documentation. Common special values include:
-  `PROJECT_NAME`, `REGION`, `ORG_ID`, `ORG_TARGET` (a separate test org for testing certain org-level resources such as IAM), `BILLING_ACCT`, `SERVICE_ACCT` (the test runner service account).
-- `test_vars_overrides`: Key/value pairs of literal overrides for variables used in tests. This can be used to call functions to
-  generate or determine a variable's value.
-- `min_version`: Set this to `beta` if the resource is in the `google` provider but the example will only work with the
-  `google-beta` provider (for example, because it includes a beta-only field.)
-- `ignore_read_extra`: Properties to not check on import. This should be used in cases where a property will not be set on import,
-  for example write-only fields.
-- `exclude_test`: If set to `true`, no test will be generated based on this example.
-- `exclude_docs`: If set to `true`, no documentation will be generated based on this example.
-- `exclude_import_test`: If set to `true`, no import test will be generated for this example.
-- `skip_vcr`: See [Skip tests in VCR replaying mode]({{< ref "/test/test#skip-vcr" >}}) for more information about this flag.
-- `skip_test`: If not empty, the test generated based on this example will always be skipped. In most cases, the value should be a
-  link to a ticket explaining the issue that needs to be resolved before the test can be unskipped.
-- `external_providers`: A list of external providers that are needed for the testcase. This does add some latency to the testcase,
-  so only use if necessary. Common external providers: `random`, `time`.
-
-Example:
-
-```yaml
-examples:
-  - name: service_resource_basic
-    primary_resource_id: example
-    bootstrap_iam:
-      - member: "serviceAccount:service-{project_number}@gcp-sa-healthcare.iam.gserviceaccount.com"
-        role: "roles/bigquery.dataEditor"
-      - member: "serviceAccount:service-org-{organization_id}@gcp-sa-osconfig.iam.gserviceaccount.com"
-        role: "roles/osconfig.serviceAgent"
-    vars:
-      dataset_id: "my-dataset"
-      network_name: "my-network"
-    test_env_vars:
-      org_id: "ORG_ID"
-    test_vars_overrides:
-      network_name: 'acctest.BootstrapSharedServiceNetworkingConnection(t, "service-resource-network-config")'
-    min_version: "beta"
-    ignore_read_extra: 
-      - 'foo'
-    exclude_test: true
-    exclude_docs: true
-    exclude_import_test: true
-    skip_vcr: true
-    skip_test: "https://github.com/hashicorp/terraform-provider-google/issues/20574"
-    external_providers:
-      - "time"
-```
+See the dedicated [MMv1 sample reference ↗]({{< ref "/reference/sample" >}}) page for a comprehensive list of all top-level sample and nested step attributes.

docs/content/reference/sample.md

@@ -0,0 +1,87 @@
+---
+title: "MMv1 sample reference"
+weight: 25
+---
+
+# MMv1 sample reference
+
+A sample is a collection of one or more `steps`, where each step represents a distinct Terraform configuration and test step (for example, create, update).
+
+Each sample supports the following attributes at the top level, with more granular control inside each step.
+
+---
+
+## Sample Attributes (Top-Level)
+
+These attributes are defined once for an entire sample.
+
+* `name`: `snake_case` name for the overall sample. This is used for generating test names.
+* `primary_resource_id`: The ID of the main resource under test for the entire sample. Tests use this to run additional checks automatically.
+* `primary_resource_type`: Optional resource type override of the primary resource. Used for import assertion validations.
+* `bootstrap_iam`: Specify member/role pairs that should exist before the test runs. This avoids race conditions on global IAM permissions. `{project_number}` and `{organization_id}` are replaced automatically.
+* `min_version`: Sets a minimum provider version for the entire sample (for example, `beta`). This can be overridden by the `min_version` attribute within a specific step.
+* `exclude_test`: If `true`, no tests are generated for this entire sample.
+* `exclude_basic_doc`: If `true`, excludes the first step of this sample from the generated documentation. By default, the first step is automatically included as a use case in the documentation. Use this if you want to skip it.
+* `skip_vcr`: If `true`, skips VCR testing for the entire sample.
+* `skip_test`: If not empty, the entire sample is skipped during tests. The value should be a link to a ticket explaining why.
+* `skip_func`: A custom function call to run to determine if tests should be skipped.
+* `region_override`: Overrides location/region identifiers specifically inside IAM assertion checks.
+* `external_providers`: A list of external providers (such as `random`, `time`) needed for the sample.
+* `tgc_skip_test`: Skips generated conversion tests specifically running inside the TGC (Terraform Google Conversion) suite (value should be a ticket link reason).
+
+---
+
+## Step Attributes
+
+A sample contains a list of one or more `steps`. Each step has its own configuration and test-specific attributes.
+
+* `name`: `snake_case` name of the individual step. This is used for generating test configuration function names and documentation headers.
+* `config_path`: The path to the step's configuration file. If omitted, it defaults to `templates/terraform/samples/services/{{product}}/{{step_name}}.tf.tmpl`.
+* `resource_id_vars`: Key/value pairs to inject into the configuration file. Reference them with `{{index $.ResourceIdVars "key"}}`. Values here automatically receive a `tf-test` prefix and random suffix, unless they contain an underscore `_`, in which case they receive a `tf_test` prefix and random suffix. **Note:** If a resource name doesn't support hyphens `-` or underscores `_`, use `test_vars_overrides` instead. For non-identifier variables, use `vars`.
+* `vars`: Key/value pairs that are copied directly to tests without a prefix. Reference with `{{index $.Vars "key"}}`. **Note:** This should ONLY be used for fields that vary between steps (for example, to test update functionality). Constant values should be hardcoded directly in the `.tf.tmpl` file.
+* `test_env_vars`: Key/value pairs that map variable names to environment variables for tests (for example, `PROJECT_NAME`, `REGION`, `ORG_ID`).
+* `test_vars_overrides`: Key/value pairs to override variables with literal values or function calls specifically for tests.
+* `oics_vars_overrides`: Key/value pairs to override variables with literal values specifically for Open in Cloud Shell (OiCS) tutorial generation.
+* `min_version`: Overrides the sample-level `min_version` for this specific step.
+* `ignore_read_extra`: A list of properties to ignore during the import test for this step, typically for write-only fields.
+* `exclude_import_test`: If `true`, no import test is generated for this specific step.
+* `include_step_doc`: If `true`, forces this specific step to be included in the generated documentation. By default, only the first step of a sample is included in the documentation as a use case. Use this on later steps to showcase update scenarios or complex configurations. This will override a top-level `exclude_basic_doc` setting if applied to the first step.
+
+---
+
+## Example
+
+```yaml
+samples:
+  - name: service_resource_update
+    primary_resource_id: example
+    bootstrap_iam:
+      - member: "serviceAccount:service-{project_number}@gcp-sa-healthcare.iam.gserviceaccount.com"
+        role: "roles/bigquery.dataEditor"
+      - member: "serviceAccount:service-org-{organization_id}@gcp-sa-osconfig.iam.gserviceaccount.com"
+        role: "roles/osconfig.serviceAgent"
+    min_version: "beta"
+    skip_vcr: true
+    external_providers:
+      - "time"
+    steps:
+      - name: service_resource_minimal # Matches templates/terraform/samples/services/{{product}}/service_resource_minimal.tf.tmpl
+        vars: # Varies between steps to test update functionality
+          description: "A minimal description" 
+        resource_id_vars: # Used for resource id in the configuration file
+          dataset_id: "my-dataset"
+          network_name: "my-network"
+        test_env_vars:
+          org_id: "ORG_ID"
+        test_vars_overrides:
+          network_name: 'acctest.BootstrapSharedServiceNetworkingConnection(t, "service-resource-network-config")'
+        ignore_read_extra:
+          - 'foo'
+        exclude_import_test: true
+      - name: service_resource_update # Matches templates/terraform/samples/services/{{product}}/service_resource_update.tf.tmpl
+        vars:
+          description: "An updated description" # This value updates the description field
+        resource_id_vars:
+          dataset_id: "my-dataset"
+          network_name: "my-network"
+```

docs/content/reference/update-test-changes.md

@@ -0,0 +1,168 @@
+---
+title: "Test Template Migration Guide"
+weight: 50
+---
+# Test Template Migration Guide
+
+The test generation framework in Magic Modules has been updated to natively support multi-step tests (create and update) directly from the resource's YAML definition. This guide is for experienced contributors who are familiar with the previous `examples` based workflow and provides a concise overview of what has changed.
+
+## Transition Timeline
+
+*   **Recommended Path:** The new `samples` configuration block is the recommended path for adding tests.
+*   **Legacy Support:** The old `examples` block path is still supported until Mid May 2026.
+*   **Action Required:** Contributors are encouraged to use `samples` for all new tests and migrate existing `examples` to `samples`.
+
+
+
+## YAML Changes: From `examples` to `samples` and `steps`
+
+The most significant change is the replacement of the `examples` block with a new `samples` block. Each `sample` can now contain one or more `steps`, where the first step corresponds to the resource creation and any subsequent steps correspond to updates.
+
+This new structure eliminates the need for handwritten update tests for MMv1 resources, as they can now be generated automatically.
+
+### Create Test Comparison
+
+A simple create test is now defined as a sample with a single step.
+
+Old `examples` format
+
+```yaml
+examples:
+  - name: "pubsub_topic_basic"
+    primary_resource_id: "example"
+    vars:
+      topic_name: "example-topic"
+      field_1: "value-one"
+      field_2: "value-two"
+    test_vars_overrides:
+      field_1: "value-one"
+      field_2: "value-two"
+```
+
+New `samples` format
+
+```yaml
+samples:
+  - name: "pubsub_topic_basic"
+    primary_resource_id: "example"
+    steps:
+      - name: "pubsub_topic_basic" 
+        resource_id_vars:
+          topic_name: "example-topic"
+        vars:
+          field_1: "value-one"
+          field_2: "value-two"
+```
+
+> [!NOTE]
+> ### Why `test_vars_overrides` is no longer needed
+> In the old `examples` block, any key placed under `vars` was automatically appended with a random suffix (for example, `example-topic-12345`). If you wanted to pass a plain value (without random suffixes), you had to use `test_vars_overrides`.
+>
+> In the new `samples` format, these two use cases are explicitly separated into different fields within a step:
+> * **`resource_id_vars`**: **Use this exclusively for resource identifier variables (such as resource names or IDs).** It will automatically prepend a `tf-test` (or `tf_test`) prefix and append a random suffix. If a resource identifier doesn't support hyphens `-` or underscores `_`, use `test_vars_overrides` instead. For non-identifier variables, use `vars`.
+> * **`vars`**: Use this for plain literal values that should be passed to the test exactly as written (replaces the need for `test_vars_overrides`). **Note:** This should ONLY be used for fields that vary between steps (for example, to test update functionality). Constant values should be hardcoded directly in the `.tf.tmpl` file.
+
+### Update Test Comparison
+
+Previously, update tests had to be handwritten. Now, they can be defined by adding a second step to a sample.
+
+New samples format (Create and Update)
+
+```yaml
+samples:
+  - name: "pubsub_topic_update"
+    primary_resource_id: "example"
+    steps:
+      - name: "pubsub_topic_full"
+        resource_id_vars:
+          topic_name: "example-topic"
+        vars:
+          field_1: "value-one"
+          field_2: "value-two"
+      - name: "pubsub_topic_full"
+        resource_id_vars:
+          topic_name: "example-topic" 
+        vars:
+          field_1: "value-one-updated"
+          field_2: "value-two-updated"
+```
+
+## YAML Field Migration
+
+The migration moves fields from the old `Examples` structure to either the new top-level `Sample` or the nested `Step`. The following tables detail where each field has been moved.
+
+### Fields Mapped to `Sample` (Top-Level)
+These fields remain at the top level, moving from the old `examples` object to the new `sample` object.
+
+| Old Field | New Location | Notes |
+| :--- | :--- | :--- |
+| `examples.name` | `sample.name` | Used for the overall sample name. A `step.name` is also used for each step. |
+| `examples.primary_resource_id` | `sample.primary_resource_id` | Remains at the sample level. |
+| `examples.primary_resource_type` | `sample.primary_resource_type` | Remains at the sample level. |
+| `examples.bootstrap_iam` | `sample.bootstrap_iam` | Remains at the sample level. |
+| `examples.min_version` | `sample.min_version` | Remains at the sample level. |
+| `examples.exclude_test` | `sample.exclude_test` | Remains at the sample level. |
+| `examples.region_override` | `sample.region_override` | Remains at the sample level. |
+| `examples.skip_vcr` | `sample.skip_vcr` | Remains at the sample level. |
+| `examples.skip_test` | `sample.skip_test` | Remains at the sample level. |
+| `examples.skip_func` | `sample.skip_func` | Remains at the sample level. |
+| `examples.external_providers` | `sample.external_providers` | Remains at the sample level. |
+| `examples.tgc_skip_test` | `sample.tgc_skip_test` | Remains at the sample level. |
+
+---
+
+### Fields Mapped to `Step`
+These fields are now configured within each individual `Step` of a `Sample`.
+
+| Old Field | New Location | Notes |
+| :--- | :--- | :--- |
+| `examples.vars` | `step.resource_id_vars` | The entire `vars` map is moved into `resource_id_vars` within each step. |
+| `examples.test_env_vars` | `step.test_env_vars` | Moved to the step level. |
+| `examples.test_vars_overrides` | `step.test_vars_overrides` | Moved to the step level. |
+| `examples.oics_vars_overrides` | `step.oics_vars_overrides` | Moved to the step level. |
+| `examples.ignore_read_extra` | `step.ignore_read_extra` | Moved to the step level. |
+| `examples.exclude_docs` | `sample.exclude_basic_doc` | Replaced by `sample.exclude_basic_doc` (or `step.include_step_doc` to override). |
+| `examples.exclude_import_test` | `step.exclude_import_test` | Moved to the step level. |
+| `examples.config_path` | `step.config_path` | Path is updated to the new service-specific directory within the step. |
+
+---
+
+### New Fields
+This field is new in the `steps` object and has no direct equivalent in the old `examples` structure.
+
+| Old Field | New Location | Notes |
+| :--- | :--- | :--- |
+| *(N/A)* | `step.vars` | Newly added at the step level. Values are copied directly to tests. |
+| *(N/A)* | `step.min_version` | Newly added to set a version for a specific step |
+| *(N/A)* | `step.include_step_doc` | Explicitly forcing a step's docs generation (overriding Sample block). |
+
+
+## Template `.tf.`tmpl File Changes
+
+The location for template files has moved from `templates/terraform/examples/` to a service-specific directory under `templates/terraform/samples/services/`.
+
+Additionally, the variable object passed into the templates has been updated. `$.ResourceIdVars` will append `tf-test` prefixes and random string suffixes, which is used for resource identifiers in most cases. `$.Vars` will apply plain values from the YAML configuration.
+
+### Example .tf.tmpl variables
+
+Old template `pubsub_topic_basic.tf.tmpl` (in `templates/terraform/examples/`)
+
+```tf
+resource "google_pubsub_topic" "{{$.PrimaryResourceId}}" {
+  name = "{{index $.Vars "topic_name"}}"
+
+  field_1 = "{{index $.Vars "field_1"}}"
+  field_2 = "{{index $.Vars "field_2"}}"
+}
+```
+
+New template `pubsub_topic_basic.tf.tmpl` (in `templates/terraform/samples/services/pubsub/`)
+
+```tf
+resource "google_pubsub_topic" "{{$.PrimaryResourceId}}" {
+  name = "{{index $.ResourceIdVars "topic_name"}}"
+
+  field_1 = "{{index $.Vars "field_1"}}"
+  field_2 = "{{index $.Vars "field_2"}}"
+}
+```