Parameter Adjustment and Hyperparameters

Parameter Adjustment and Hyperparameters

Epochs, Batch Size, and Learning Rate. Learn what these knobs do and how to set them for a Gemini tuning job.

Parameter Adjustment and Hyperparameters

When you click "Tune", you are presented with a few confusing sliders.

1. Epochs

An Epoch is one full pass through your training dataset.

  • Low (1-3): Model sees data once. Good for large datasets. avoiding overfitting.
  • High (5-10): Model memorizes the data more. Good for small datasets (~50 examples) where you really want to drill the pattern in.
  • Risk: Too high = "Overfitting". The model memorizes your examples but loses the ability to generalize to new ones.

2. Batch Size

How many examples does the model look at before updating its brain?

  • Usually handled automatically by Google, but sometimes configurable. Larger batches = smoother learning, but requires more RAM.

3. Learning Rate

How drastic are the changes to the brain?

  • High: Fast learning, but might overshoot and become unstable.
  • Low: Stable, slow learning.
  • Multilier: Gemini allows a "Learning Rate Multiplier". Start with default. If the model isn't learning (loss is flat), increase it. If the model is spouting gibberish (loss spikes), decrease it.

Summary

  • Small Dataset (<100): Higher Epochs (~5-10).
  • Large Dataset (>1000): Lower Epochs (~1-3).
  • Default: Google's defaults are usually very good. Trust them first.

In the next lesson, we cover Uploading Custom Data.

Subscribe to our newsletter

Get the latest posts delivered right to your inbox.

Subscribe on LinkedIn