Created
March 27, 2026 10:41
-
-
Save danieltorscho/ae4e820b59190a523105c77d83ace268 to your computer and use it in GitHub Desktop.
unsloth-studio-ft-llm-runpod-setup
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| teacher: nemotron-cascade-2 | |
| model to ft: https://huggingface.co/allenai/OLMo-2-0425-1B | |
| dataset: https://huggingface.co/datasets/yahma/alpaca-cleaned | |
| hf token: hf_ | |
| ## Create Pod on runpod.io | |
| Pod GPU: H200 SXM 141GB (alternative RTX PRO 6000 96GB) | |
| Edit Pod Template: | |
| Container disk: 300 GB | |
| Volume disk: 0 GB | |
| Expose ports: 8888,8889,8005,8006 | |
| ## Setup environment | |
| 1. update libs: | |
| apt-get update && apt-get upgrade -y | |
| 2. prepare ollama install dependencies: | |
| apt-get install zstd | |
| 3. install ollama: | |
| curl -fsSL https://ollama.com/install.sh | sh | |
| 4. ollama server with logs: | |
| nohup ollama serve > serve.log 2>&1 & | |
| 5. check ollama: | |
| curl 127.0.0.1:11434 | |
| 6. ollama run teacher: | |
| ollama run nemotron-cascade-2 | |
| 7. install unsloth: | |
| curl -fsSL https://unsloth.ai/install.sh | sh | |
| ## Run Unsloth | |
| - source unsloth_studio/bin/activate | |
| - unsloth studio -H 0.0.0.0 -p 8888 | |
| if errors missing libs: | |
| - /root/.unsloth/studio/.venv/bin/python -m pip install litellm | |
| - /root/.unsloth/studio/.venv/bin/python -m pip install "litellm>=1.73.6,<1.80.12" | |
| - pkill -f "studio.backend" | |
| ## Unsloth recipe | |
| import JSON unsloth_recipe_diagram.json | |
| ### Check and run training | |
| Click on CHECK to check the recipe diagram | |
| Then after SUCCESS click on START TEST RUN |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment