Pipeline parallelism, for training models larger than can fit on a single GPU Useful metrics logged to Tensorboard Compute metrics on a held-out eval set, for measuring generalization Training state ...