Hyperparameter Tuning and Fine-Tuning Foundation Models

What it is

Hyperparameters are settings chosen before training, such as learning rate, tree depth or batch size. A hyperparameter tuning job trains many versions (trials) with different values and keeps the best by a metric you choose.

Fine-tuning adapts a pretrained foundation model, such as Gemini or an open model from Model Garden, to your task with your own examples. The exam asks how to set up tuning jobs and, for foundation models, whether tuning is needed at all.

How it works

Hyperparameter tuning jobs

An Agent Platform hyperparameter tuning job wraps a custom training job. You define:

from google.cloud import aiplatform
from google.cloud.aiplatform import hyperparameter_tuning as hpt

PREBUILT_TRAINING_IMAGE = "..."   # a prebuilt training container URI

aiplatform.init(project="my-project", location="us-central1",
                staging_bucket="gs://my-staging")
custom_job = aiplatform.CustomJob.from_local_script(
    display_name="churn-train",
    script_path="train.py",
    container_uri=PREBUILT_TRAINING_IMAGE,
)
hp_job = aiplatform.HyperparameterTuningJob(
    display_name="churn-hpt",
    custom_job=custom_job,
    metric_spec={"auc": "maximize"},
    parameter_spec={
        "learning_rate": hpt.DoubleParameterSpec(min=0.001, max=0.1, scale="log"),
        "max_depth": hpt.IntegerParameterSpec(min=3, max=10, scale="linear"),
    },
    max_trial_count=30,
    parallel_trial_count=3,
)
hp_job.run()

The training script receives the trial's values as command-line arguments and reports the metric back:

import hypertune

hypertune.HyperTune().report_hyperparameter_tuning_metric(
    hyperparameter_metric_tag="auc", metric_value=auc, global_step=epoch)

BigQuery ML also tunes hyperparameters with the num_trials option, and AutoML tunes them for you.

When to fine-tune a foundation model

Tuning needs quality labeled examples (prompt and ideal response), an evaluation set, and budget for training and testing. It does not keep facts current: RAG does that.

How foundation models are tuned

Parameter-efficient tuning (for example LoRA adapters) trains a small number of extra weights instead of the whole model. It needs far less compute and memory than full fine-tuning and usually works well with hundreds to thousands of examples.

A worked example

A legal firm's Gemini prompt extracts clauses from contracts, but outputs vary in structure and miss firm-specific clause types, even with ten examples in the prompt. The engineer collects 1,500 reviewed prompt and response pairs, splits off 150 for validation, and runs supervised fine-tuning on a Flash model.

The tuned model is compared with the prompted base model on the held-out set and wins. It also needs a much shorter prompt, so each request costs less.

How the exam asks about this

• "Hyperparameter tuning takes too long; budget allows more parallel compute." Increase parallel_trial_count.
• "A chatbot must answer about prices that change daily." Use RAG or grounding, not fine-tuning.
• "An open model must learn a domain task on limited GPU memory." Use parameter-efficient tuning such as LoRA.