Vertex AI Terraform Resource

01 · the problem

Google Cloud partners wanted to deploy generative AI models from Vertex AI Model Garden at scale, and asked for a way to do it with Terraform. Most partner companies already managed the rest of their Google Cloud infrastructure with Terraform, so the existing ways to deploy models didn't work for them. Clicking through the Cloud Console couldn't scale to thousands of models. Deploying models with the gcloud CLI or REST API calls meant writing and maintaining custom scripts. The Vertex AI SDK only supported certain languages, so a development team that builds in another language would have had to learn one just to deploy models. Even in a supported language, a team didn't always get the latest features, since they usually reached the Python SDK first and were ported to Java and TypeScript later on. That gap was blocking a $19.2M ARR opportunity.

02 · what I built

• Built the vertex_ai_endpoint_with_model_garden_deployment resource in Go for the official Terraform provider for Google Cloud.
• The resource is the only built-in way to deploy Model Garden models in parallel: you add a resource block for each model and run terraform apply once to deploy all the models.
• Found and fixed a race condition in the Vertex AI backend, written in Java, that made requests to deploy the same model in the same Google Cloud project and region fail if they arrived within the same second.
• Wrote comprehensive tests for the fix using JUnit and Mockito.

03 · how I built it

1. 01design

   I designed the resource's schema and mapped it onto the Model Garden deployment APIs, which are defined in protobuf.

   Terraform resource→ Model Garden API · DeployRequest

   publisher_model_name→→ publisher_model_name

   hugging_face_model_id→→ hugging_face_model_id

   project, location→→ destination

   model_config→→ model_config

   endpoint_config→→ endpoint_config

   deploy_config→→ deploy_config
2. 02build

   I wrote the resource in Go and YAML in Google's Magic Modules project, which the Terraform provider for Google Cloud is built from.

   example: deploying three models at once

   main.tf

   resource "google_vertex_ai_endpoint_with_model_garden_deployment" "gemma" {
     publisher_model_name = "publishers/google/models/gemma@gemma-1.1-2b-it"
     location             = "us-central1"
     model_config { accept_eula = true }
   }
   
   resource "google_vertex_ai_endpoint_with_model_garden_deployment" "qwen" {
     hugging_face_model_id = "Qwen/Qwen3-0.6B"
     location              = "us-central1"
     model_config { accept_eula = true }
   }
   
   resource "google_vertex_ai_endpoint_with_model_garden_deployment" "llama" {
     publisher_model_name = "publishers/meta/models/llama3-2@llama-3.2-1b"
     location             = "us-central1"
     model_config { accept_eula = true }
   }

3. 03fix

   While testing the resource's ability to deploy several models at once, I found that requests to deploy the same model in the same Google Cloud project and region failed if they arrived within the same second. With guidance from my manager, I traced this to a race condition in the Vertex AI backend, which is written in Java, and wrote the code changes to fix it.

   Google Cloud project: my-project

   region: us-central1

   request	sent at	before the fix	after the fix
   gemma	12:00:01.120	deployed	deployed
   gemma	12:00:01.480	failed	deployed
   qwen	12:00:01.910	deployed	deployed

4. 04test

   I wrote comprehensive tests with the JUnit and Mockito libraries to cover the code changes.

   JavaJUnit + Mockito
5. 05publish

   The resource went through Google's review process and was published to the Terraform Registry, with usage documented in the official Google Cloud docs.

   Terraform Registryregistry.terraform.ioGoogle Cloud Docsdocs.cloud.google.com

04 · key decisions

05 · where it stands

• ✓Published in the Terraform Registry and documented in the Google Cloud docs.
• ✓$19.2M ARR opportunity unblocked for Google Cloud partners.
• ✓Vertex AI now handles requests to deploy the same model in the same Google Cloud project and region that arrive within the same second.

06 · technologies used

GoLanguage

TerraformInfrastructure

JavaLanguage

Vertex AIAI

Protocol BuffersBackend

JUnit + MockitoTesting