Glossary

LoRA

LoRA (Low-Rank Adaptation) is a technique for fine-tuning large AI models by training a small set of low-rank matrices instead of the full weights. A 13B LLM fine-tune via LoRA fits comfortably in 40GB of VRAM.