Vertex AI

Vertex AI Cheat Sheet

Vertex AI is a unified machine learning (ML) platform that lets you train and deploy ML models and AI applications. It combines data engineering, data science, and ML engineering workflows, enabling teams to collaborate using a common toolset. The platform supports both generative AI (GenAI) workflows and traditional ML (AI inference) workflows with end-to-end MLOps tools and enterprise-grade controls.

Key Capabilities

Area	Capabilities
Generative AI	Prompt design in Vertex AI Studio; Model Garden with 200+ models (Google foundation models like Gemini, partner models like Claude, open-source like Llama); Model customization (grounding, supervised fine-tuning, PEFT); Gen AI evaluation service; Agent builder (Vertex AI Agent Engine, Agent Development Kit); Grounding, function calling, RAG; Responsible AI safety features
Data preparation	Vertex AI Workbench notebooks for exploratory data analysis; Integration with Cloud Storage and BigQuery; Dataproc Serverless Spark for large-scale processing
Model training	AutoML (code-free) or custom training (full control); Vertex AI Experiments; Vertex AI Training (serverless or training clusters); Ray on Vertex AI; Vertex AI Vizier for hyperparameter tuning
Model evaluation	Evaluation metrics; Integration with Vertex AI Pipelines
Model serving	Online inferences (prebuilt or custom containers); Batch inferences; Optimized TensorFlow runtime; Vertex Explainable AI; Vertex AI Feature Store; Models trained with BigQuery ML
Model monitoring	Vertex AI Model Monitoring for training-serving skew and inference drift

AutoML Beginner’s Guide

Vertex AI Workflow

Gather your data: Determine what you need based on desired outcome.
Prepare your data: Ensure proper formatting and labeling.
Train: Set parameters and build model.
Evaluate: Review model metrics.

Deploy and predict: Make model available for use.

Data Preparation Guidelines

Guideline	Recommendation
Minimum examples per label	100 for classification; target 1000+
Distribution across labels	Smallest label should have at least 10% of examples of largest label
Variation	Capture diversity of problem space
Match to production data	Training examples should resemble real-world inference data

Dataset Splits

Split	Purpose	Default
Training set	Learn model parameters (weights)	80%
Validation set	Tune hyperparameters	10%
Test set	Final evaluation (unseen during training)	10%

Model Evaluation Metrics

Metric	Description
Score threshold	Confidence level required to assign a category
True positives/negatives	Correctly identified/non-identified items
False positives/negatives	Incorrectly identified/missed items
Precision	Of items labeled, how many were correct
Recall	Of items that should be labeled, how many were identified
Average precision	Area under precision-recall curve (closer to 1.0 is better)

Deployment Options

Type	Use Case	Characteristics
Batch inference	Many requests at once	Asynchronous, returns JSON Lines file
Online inference	Real-time, single requests	Synchronous, REST API

MLOps on Vertex AI

Service	Purpose
Vertex AI Pipelines	Automate, monitor, and govern ML workflows
Vertex AI Training	Flexible, fully managed training (serverless or training clusters)
Vertex ML Metadata	Record and query metadata, parameters, and artifacts
Vertex AI Experiments	Track and analyze model architectures, hyperparameters, training environments
Vertex AI TensorBoard	Track, visualize, and compare ML experiments
Vertex AI Model Registry	Central repository to organize, track, and train new model versions
Vertex AI Feature Store	Centralized repository for organizing, storing, and serving ML features
Vertex AI Model Monitoring	Monitor models for training-serving skew and inference drift
Ray on Vertex AI	Scale Python and ML workloads using open-source Ray framework

Interfaces for Vertex AI

Interface	When to Use
Google Cloud Console	Graphical UI to manage resources, datasets, models, endpoints, jobs; access other Google Cloud services
gcloud	Command-line tool for scripting and automation
Terraform	Infrastructure as code to manage Vertex AI resources
Python (Vertex AI SDK)	Programmatic access for data scientists and ML engineers
Client libraries	Access from various programming languages
REST API	Direct HTTP access to Vertex AI services