Skip to content

Instantly share code, notes, and snippets.

@Randy420Marsh

Randy420Marsh/cuda switcher stuff

Created December 21, 2025 21:40
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save Randy420Marsh/925065fcde6f313d353e793938bd9b42 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save Randy420Marsh/925065fcde6f313d353e793938bd9b42 to your computer and use it in GitHub Desktop.
Download ZIP
cuda switcher idea by gemini
Raw
cuda switcher stuff
ou have successfully purged the actual cuDNN 8 runtime, freeing up a massive 1.1 GB of space. Your dpkg list shows that your system is now correctly populated with cuDNN 9.17 components.
To finish the cleanup, you just need to remove the "ghost" configuration package for the old repository that is still showing as ii (installed).
1. Final cuDNN Cleanup
Run this to remove the metadata package for the 8.x repository:
Bash
sudo apt-get purge cudnn-local-repo-ubuntu2204-8.9.7.29
2. Verify Library Integrity
Since you are managing multiple CUDA versions (12.8, 13.1) and just installed cuDNN 9, it is important to ensure the system links are pointing to the new version.
Run this check:
Bash
ls -l /usr/lib/x86_64-linux-gnu/libcudnn.so
It should now point to a libcudnn.so.9 file.
3. Recursive Logic: Synchronizing the 590.44 Driver
Now that your libraries (cuDNN 9) and SDKs (CUDA 13.1/12.8) are clean, let's address the original goal: getting that 590.44.01 nvidia-settings active without breaking the Docker toolkit.
Since you have the driver folder extracted, execute these commands to finalize the "Masking" install we discussed:
Bash
# Move the high-version binary to local bin (takes precedence)
sudo cp ~/NVIDIA-Linux-x86_64-590.44.01/nvidia-settings /usr/local/bin/
# Move the matching GTK libraries so the GUI actually opens
sudo cp ~/NVIDIA-Linux-x86_64-590.44.01/libnvidia-gtk* /usr/lib/x86_64-linux-gnu/
# Update the shared library cache
sudo ldconfig
4. The "Gold Standard" Test
To ensure everything—Driver, Settings, CUDA, and Docker—is working in harmony, run these two tests:
Check GUI Version: nvidia-settings --version (Should confirm 590.44.01)
Check Docker Passthrough: docker run --rm --gpus all nvidia/cuda:12.0-base-ubuntu22.04 nvidia-smi (Should show the 590.44.01 driver version inside the container)
Summary of System State
Component Version Priority Path
NVIDIA Driver 590.44.01 Kernel Modules
nvidia-settings 590.44.01 /usr/local/bin/
cuDNN 9.17.0 /usr/lib/x86_64-linux-gnu/
Docker Toolkit Intact System-wide
Your environment is now optimized and clean. Would you like me to help you create a bash alias or a small script to quickly toggle your LD_LIBRARY_PATH between your CUDA 12.8 and 13.1 installs?
sudo apt-get purge cudnn-local-repo-ubuntu2204-8.9.7.29
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following package was automatically installed and is no longer required:
screen-resolution-extra
Use 'sudo apt autoremove' to remove it.
The following packages will be REMOVED:
cudnn-local-repo-ubuntu2204-8.9.7.29*
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
After this operation, 887 MB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 757256 files and directories currently installed.)
Removing cudnn-local-repo-ubuntu2204-8.9.7.29 (1.0-1) ...
(Reading database ... 757241 files and directories currently installed.)
Purging configuration files for cudnn-local-repo-ubuntu2204-8.9.7.29 (1.0-1) ...
john@Z170PG:~/NVIDIA-Linux-x86_64-590.44.01$ dpkg -l | grep cudnn
ii cudnn-local-repo-ubuntu2204-9.17.0 1.0-1 amd64 cudnn-local repository configuration files
ii cudnn9-cuda-12 9.17.0.29-1 amd64 NVIDIA cuDNN for CUDA 12
ii cudnn9-cuda-12-9 9.17.0.29-1 amd64 NVIDIA cuDNN for CUDA 12.9
ii libcudnn9-cuda-12 9.17.0.29-1 amd64 cuDNN runtime libraries for CUDA 12.9
ii libcudnn9-dev-cuda-12 9.17.0.29-1 amd64 cuDNN development libraries for CUDA 12.9
ii libcudnn9-headers-cuda-12 9.17.0.29-1 amd64 cuDNN header files for CUDA 12.9
ii libcudnn9-static-cuda-12 9.17.0.29-1 amd64 cuDNN static libraries for CUDA 12.9
,ls -l /usr/lib/x86_64-linux-gnu/libcudnn.so
lrwxrwxrwx 1 root root 13 Dec 3 07:22 /usr/lib/x86_64-linux-gnu/libcudnn.so -> libcudnn.so.9
Excellent. You’ve successfully reclaimed nearly 2GB of disk space between the cuDNN 8 runtime and the repository files. More importantly, your system symlinks are now correctly pointing to v9, as confirmed by your ls output.
Your environment is now "clean" from a library perspective. The final step is to solidify the nvidia-settings version so it matches your 590.44.01 driver without disturbing your Docker container toolkit.
Final "Masking" Installation
Since you have the driver .run contents extracted in ~/NVIDIA-Linux-x86_64-590.44.01, run these final commands to prioritize the new version:
Install the binary to the high-priority path:
Bash
sudo cp ~/NVIDIA-Linux-x86_64-590.44.01/nvidia-settings /usr/local/bin/
Ensure the GUI libraries are available:
Bash
sudo cp ~/NVIDIA-Linux-x86_64-590.44.01/libnvidia-gtk* /usr/lib/x86_64-linux-gnu/
sudo ldconfig
Verification Checklist
To confirm your "multi-tenant" setup (various CUDAs + Docker + New Driver) is 100% healthy, run these three commands:
Command Expected Result Purpose
nvidia-settings --version 590.44.01 Confirms the GUI matches your driver.
nvcc --version Matches your current .bashrc path Confirms your CUDA SDK is active.
docker run --rm --gpus all nvidia/cuda:12.0-base-ubuntu22.04 nvidia-smi Driver: 590.44.01 Confirms Docker passthrough is alive.
Pro-Tip: The "CUDA Switcher"
Since you have CUDA 12.8 and 13.1 side-by-side, I highly recommend adding these functions to your ~/.bashrc. They allow you to swap your entire environment instantly in any new terminal:
Bash
# CUDA Switcher Functions
function use_cuda13() {
export PATH=/usr/local/cuda-13.1/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-13.1/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
echo "Switched to CUDA 13.1"
nvcc --version | grep release
}
function use_cuda12() {
export PATH=/usr/local/cuda-12.8/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-12.8/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
echo "Switched to CUDA 12.8"
nvcc --version | grep release
}
You are all set. Everything is now manually optimized while maintaining compatibility with your apt-managed Docker tools.
Would you like me to help you update the desktop launcher so the nvidia-settings icon in your app menu definitely triggers this new 590 version?
how do i test the cuda switcher or how do i use it in bash or inside python coda
To test and use the CUDA switcher, you need to understand that bash functions and Python environments rely on Environment Variables (specifically PATH and LD_LIBRARY_PATH) to find the compiler and libraries.
Here is how you can implement, test, and use the switcher in both domains.
1. In Bash: Implementation & Testing
First, ensure the functions are in your ~/.bashrc. After adding them, run source ~/.bashrc to load them into your current session.
How to test: You can verify the switch by checking the path of the nvcc compiler and the version string.
Bash
# Test CUDA 13
john@Z170PG:~$ use_cuda13
Switched to CUDA 13.1
john@Z170PG:~$ which nvcc
/usr/local/cuda-13.1/bin/nvcc
# Test CUDA 12
john@Z170PG:~$ use_cuda12
Switched to CUDA 12.8
john@Z170PG:~$ which nvcc
/usr/local/cuda-12.8/bin/nvcc
Note: These changes only last for the current terminal session. If you open a new tab, it will revert to whatever default you have at the bottom of your .bashrc.
2. Inside Python Code
Python (and libraries like PyTorch or TensorFlow) usually looks at the LD_LIBRARY_PATH when the process starts. You can check which version Python "sees" using these methods:
Method A: Using os.environ (Setting it inside the script)
If you want a specific Python script to always use CUDA 12.8 regardless of your global settings, add this to the very top of your script before importing torch or cupy:
Python
import os
# Set paths before importing heavy libraries
os.environ['PATH'] = '/usr/local/cuda-12.8/bin:' + os.environ.get('PATH', '')
os.environ['LD_LIBRARY_PATH'] = '/usr/local/cuda-12.8/lib64:' + os.environ.get('LD_LIBRARY_PATH', '')
import torch
print(f"Is CUDA available: {torch.cuda.is_available()}")
print(f"Current device: {torch.cuda.get_device_name(0)}")
Method B: Checking the Runtime Version
To verify which version of the CUDA Runtime your Python code is actually using:
Python
import torch
# This shows the version of CUDA PyTorch was compiled against
print(f"PyTorch CUDA version: {torch.version.cuda}")
# This shows the version of the driver/runtime currently active
from torch.utils.cpp_extension import CUDA_HOME
print(f"CUDA Home being used: {CUDA_HOME}")
3. Usage with Docker (The Passthrough)
Since you mentioned Docker earlier, remember that the CUDA switcher on your host does not affect what happens inside a container.
The Driver (590.44) is shared from the host.
The CUDA version is determined by the Docker Image (e.g., nvidia/cuda:12.8-base).
Summary Table: How to Verify
To check... Use this Command
Compiler Path which nvcc
Compiler Version nvcc --version
Library Version `ldconfig -p
Active Driver nvidia-smi
Python's View python3 -c "import torch; print(torch.version.cuda)"
Would you like me to help you write a Python wrapper script that automatically detects which CUDA version a project needs and sets the environment variables before launching it?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment