A Field Guide to GCP Vertex AI Serverless Endpoints: From Zero to Production

First, ensure the Vertex AI API is enabled in your project.

# Enable the Vertex AI API
gcloud services enable aiplatform.googleapis.com

This operation can take a minute or two to complete.

In a real project, you'd bring your own trained model. For this guide, we'll train a toy sentiment classifier. A common mistake I see is pickling a model and its preprocessor (like a tokenizer or vectorizer) separately. This creates a dependency nightmare during deployment. The correct approach is to combine them into a single sklearn.pipeline.Pipeline object and pickle that.

Create a file named create_model.py:

# create_model.py
import pickle
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline

# Sample data
corpus = [
    "I love this product, it's amazing!",
    "This is terrible, I hate it.",
    "It's okay, nothing special.",
    "Absolutely fantastic experience."
]
labels = [1, 0, 0, 1]  # 1 for positive, 0 for negative/neutral

# Create a scikit-learn pipeline
# This bundles the vectorizer and the model into a single artifact
pipeline = Pipeline([
    ('tfidf', TfidfVectorizer()),
    ('clf', LogisticRegression())
])

# Train the pipeline
pipeline.fit(corpus, labels)

# Save the entire pipeline to a single file
with open('model.pkl', 'wb') as f:
    pickle.dump(pipeline, f)

print("Pipeline saved as model.pkl")

Run the script to generate your model.pkl file.

python3.12 create_model.py
# Expected Output: Pipeline saved as model.pkl

I insist on Infrastructure as Code (IaC) for all cloud resources. It ensures deployments are repeatable, version-controlled, and auditable. We'll use Terraform to define our Vertex AI Endpoint.

Create a main.tf file:

# main.tf
terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = ">= 4.50.0"
    }
  }
}

provider "google" {
  project = var.project_id
  region  = var.region
}

variable "project_id" {
  description = "The GCP project ID."
  type        = string
}

variable "region" {
  description = "The GCP region for Vertex AI resources."
  type        = string
  default     = "europe-west1"
}

resource "google_vertex_ai_endpoint" "sentiment_endpoint" {
  name         = "sentiment-analysis-endpoint"
  display_name = "Sentiment Analysis Serverless Endpoint"
  description  = "Managed serverless endpoint for sentiment analysis."
  project      = var.project_id
  location     = var.region

  labels = {
    environment = "production"
    service     = "sentiment-analysis"
  }
}

output "endpoint_resource_name" {
  description = "The full resource name of the created Vertex AI Endpoint."
  value       = google_vertex_ai_endpoint.sentiment_endpoint.name
}

Next, create a terraform.tfvars file to hold your project-specific ID. Remember to replace the placeholder with your actual GCP project ID.

# terraform.tfvars
project_id = "your-gcp-project-id"

Now, initialize Terraform and apply the configuration:

terraform init
terraform apply --auto-approve

After a moment, Terraform will output the full resource name of your endpoint. Note this down; you'll need it shortly.

With our endpoint provisioned, we need to register our model. We'll upload model.pkl to a Google Cloud Storage (GCS) bucket and then create a Model resource in the Vertex AI Model Registry that points to it.

First, create a GCS bucket (if you don't have one) and upload your model. Bucket names must be globally unique.

# Replace with your unique bucket name
BUCKET_NAME="your-unique-model-bucket-ew1"
PROJECT_ID=$(gcloud config get-value project)

gcloud storage buckets create gs://${BUCKET_NAME} --project=${PROJECT_ID} --location=europe-west1
gcloud storage cp model.pkl gs://${BUCKET_NAME}/sentiment_model/model.pkl

Now, we'll use the Python SDK to register this artifact as a new model version. We'll point Vertex AI to a pre-built scikit-learn serving container, which knows how to load and serve a pickled model.

Create a file named upload_model.py:

# upload_model.py
import os
from google.cloud import aiplatform

# --- Configuration ---
# Please update these with your own values
PROJECT_ID = os.getenv("GCLOUD_PROJECT")
REGION = "europe-west1"
BUCKET_NAME = "your-unique-model-bucket-ew1" # The bucket you just created
MODEL_DISPLAY_NAME = "sentiment-analysis-model"
# --- End Configuration ---

# The GCS path to the directory containing the model artifact
ARTIFACT_URI = f"gs://{BUCKET_NAME}/sentiment_model/"

# Official Google pre-built container for scikit-learn 1.4
SERVING_CONTAINER_IMAGE = "europe-docker.pkg.dev/vertex-ai/prediction/sklearn-cpu.1-4:latest"

# Initialize the Vertex AI SDK
aiplatform.init(project=PROJECT_ID, location=REGION)

# Upload the model to Vertex AI Model Registry
model = aiplatform.Model.upload(

    display_name=MODEL_DISPLAY_NAME,
    artifact_uri=ARTIFACT_URI,
    serving_container_image_uri=SERVING_CONTAINER_IMAGE,
)

print(f"Model uploaded. Resource name: {model.resource_name}")

# To see the model in the console, visit:
# https://console.cloud.google.com/vertex-ai/models?project={PROJECT_ID}

Run the script. Make sure you replace your-unique-model-bucket-ew1 with the bucket name you created.

export GCLOUD_PROJECT=$(gcloud config get-value project)
python3.12 upload_model.py

This will register your model and output its full resource name, which we'll need for the final deployment step.

This is the final step where everything comes together. We'll take the endpoint created by Terraform and the model we just registered, and deploy the model to it. This is the action that triggers Vertex AI to provision the serverless infrastructure.

Create deploy_model.py:

# deploy_model.py
import os
from google.cloud import aiplatform

# --- Configuration ---
PROJECT_ID = os.getenv("GCLOUD_PROJECT")
REGION = "europe-west1"

# Use the endpoint resource name from your terraform output
ENDPOINT_RESOURCE_NAME = f"projects/{PROJECT_ID}/locations/{REGION}/endpoints/your-endpoint-numeric-id"

# Use the model resource name from the upload_model.py output
MODEL_RESOURCE_NAME = f"projects/{PROJECT_ID}/locations/{REGION}/models/your-model-numeric-id"

DEPLOYED_MODEL_DISPLAY_NAME = "sentiment-model-v1-prod"
# --- End Configuration ---


aiplatform.init(project=PROJECT_ID, location=REGION)

# Get a reference to the existing endpoint
endpoint = aiplatform.Endpoint(ENDPOINT_RESOURCE_NAME)

print("Deploying model to endpoint...")
# This can take 10-15 minutes
endpoint.deploy(
    model=MODEL_RESOURCE_NAME,
    deployed_model_display_name=DEPLOYED_MODEL_DISPLAY_NAME,
    # For serverless, you define the 'shape' of the machine, not a specific type
    machine_type="n1-standard-2",
    min_replica_count=1, # Can be 0 for pure serverless, 1 for warm start
    max_replica_count=3,
    traffic_split={"0": 100}, # Send 100% of traffic to this new model
    sync=True, # Wait for the deployment to complete
)

print(f"Model deployed successfully to endpoint: {endpoint.display_name}")

Before running, you must replace your-endpoint-numeric-id and your-model-numeric-id with the actual IDs from the previous steps. You can find them in your terminal output or in the GCP console.

export GCLOUD_PROJECT=$(gcloud config get-value project)
python3.12 deploy_model.py

This step takes several minutes as Vertex AI provisions resources, pulls the container image, and deploys the model. Be patient.

Once deployment is complete, your endpoint is live. Let's send it some sample data.

Create predict_sentiment.py:

# predict_sentiment.py
import os
from google.cloud import aiplatform

# --- Configuration ---
PROJECT_ID = os.getenv("GCLOUD_PROJECT")
REGION = "europe-west1"
ENDPOINT_RESOURCE_NAME = f"projects/{PROJECT_ID}/locations/{REGION}/endpoints/your-endpoint-numeric-id"
# --- End Configuration ---


aiplatform.init(project=PROJECT_ID, location=REGION)

# Get a reference to the endpoint
endpoint = aiplatform.Endpoint(ENDPOINT_RESOURCE_NAME)

instances = [
    "This service is incredible, highly recommend!",
    "I'm very disappointed with the slow response.",
    "The food was average, nothing to write home about."
]

# The predict method takes a list of instances
# The pre-built container returns a simple list of predictions
response = endpoint.predict(instances=instances)

print("Prediction Response:")
print(response)

print("\n--- Results ---")
for i, instance in enumerate(instances):
    prediction = response.predictions[i]
    sentiment = "Positive" if prediction == 1 else "Negative/Neutral"
    print(f"Input: '{instance}'\n  -> Prediction: {sentiment} ({prediction})\n")

Again, update your-endpoint-numeric-id and run the script.

export GCLOUD_PROJECT=$(gcloud config get-value project)
python3.12 predict_sentiment.py

You should see the model's predictions for your sample inputs.

1. Permission Denied: If you see a 403 Permission 'aiplatform.endpoints.deployModel' denied, your user or service account lacks the Vertex AI User (roles/aiplatform.user) IAM role. Grant it, and try again.

2. Deployed Model Failed to Come Up: This is a generic error, usually meaning the container is crash-looping. The cause is almost always in the logs. Common reasons include:

   • Pickle version mismatch: The scikit-learn version used for local training is incompatible with the version inside the pre-built container. Ensure they match.
   • Incorrect artifact path: The artifact_uri in Model.upload must point to the GCS directory containing model.pkl, not the file itself.

3. Cost Overruns: If you set min_replica_count to 1 or more, you will be billed for that compute 24/7, even with no traffic. For development or non-critical endpoints, set min_replica_count=0 to allow the service to scale down completely and avoid idle costs.

• Serverless is the Default: For most real-time inference workloads, start with Vertex AI Serverless Endpoints. They provide the best balance of performance, cost, and operational simplicity.
• IaC is Non-Negotiable: Always provision core resources like endpoints with Terraform. This provides a repeatable, auditable foundation for your MLOps platform.
• Package Models Correctly: Use tools like scikit-learn's Pipeline to bundle preprocessing and modeling steps into a single, portable artifact. This dramatically simplifies deployment.
• Govern Your Endpoints: Use labels to track cost and ownership. Set min_replica_count=0 for non-production workloads to manage spend.

With a deployed endpoint, the next step is to integrate it into a real application. Beyond that, you can explore more advanced features like:

• Traffic Splitting: Deploy a new model version to the same endpoint and gradually shift traffic to perform A/B tests or canary releases.
• Vertex AI Explainability: Integrate model explainability to understand why your model is making certain predictions.
• CI/CD Automation: Build a pipeline in Cloud Build or GitHub Actions to automatically train, register, and deploy new model versions upon code changes.

For a deeper dive into the broader implications of managed AI infrastructure, you might want to check out the Technical Analysis of the Semiconductors Sector on Clear Signals.

• Complete Source Code: You can find the full code and Terraform configurations used in this guide at [link to a public GitHub repository with the complete source code].
• Official GCP Samples: For more advanced use cases, the official GoogleCloudPlatform/vertex-ai-samples repository on GitHub is an excellent resource.