Last active
December 2, 2023 21:44
-
-
Save msaelices/6c3eb284dbbab35aa55455da26bbdcbc to your computer and use it in GitHub Desktop.
Installing Code LLama in Ubuntu
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # We assume you have an Nvidia GPU | |
| # Ubuntu dependencies | |
| sudo apt install nvidia-cuda-toolkit | |
| # From https://docs.anaconda.com/free/anaconda/install/linux/ | |
| apt-get install libgl1-mesa-glx libegl1-mesa libxrandr2 libxrandr2 libxss1 libxcursor1 libxcomposite1 libasound2 libxi6 libxtst6 | |
| # Install conda | |
| # Download the installer from https://www.anaconda.com/download/#linux | |
| # Execute it, entering “yes” to initialize Anaconda Distribution when asked | |
| # Activate conda | |
| source ~/anaconda3/bin/activate # change to the location you have decided to install conda | |
| # Install the textgen conda environment | |
| conda create -n textgen python=3.10.9 | |
| conda activate textgen | |
| # Install pytorch | |
| pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117 | |
| # Download Text generation web UI | |
| git clone https://github.com/oobabooga/text-generation-webui | |
| cd text-generation-webui | |
| # Install dependencies | |
| python -m pip install -r requirements.txt | |
| # Checks you have CUDA enabled | |
| python -m torch.utils.collect_env | |
| # Run the web server | |
| python server.py | |
| # Navigate to http://127.0.0.1:7860/ | |
| # Click in the "model" tab and download this model: https://huggingface.co/TheBloke/WizardCoder-Python-13B-V1.0-GPTQ | |
| # In the "parameters" tab, set the temperature to the minimum and the max tokens to somewhere around 2048 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment