# Memory Efficient Training 

## Optimizers 
| Short Name | Full Name | URL |
|:----:|:----:|:----:|
| CAME | Confidence-guided Adaptive Memory Efficient Optimization (ACL23 outstanding paper award) |  https://github.com/huawei-noah/Pretrained-Language-Model/blob/master/CAME/came.py |
| LOMO | LOw-Memory Optimization | https://github.com/OpenLMLab/LOMO/blob/main/src/lomo.py |

## Low Rank Decomposition
LORA (Low Rank Adaption): https://github.com/microsoft/LoRA or https://github.com/huggingface/peft/blob/main/src/peft/tuners/lora.py

## Adapters 

## Quantization
QLORA: https://github.com/artidoro/qlora/blob/main/qlora.py