Skip to content

Instantly share code, notes, and snippets.

@padeoe

padeoe/enhanced-nvidia-smi.md

Last active March 3, 2026 13:05
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save padeoe/771c4972ae185c9a7d3d497fa4e1ecab to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save padeoe/771c4972ae185c9a7d3d497fa4e1ecab to your computer and use it in GitHub Desktop.
Download ZIP
Show Username & full process command with nvidia-smi
Raw
nvidia-smi-process-name.md

This script enhances the functionality of nvidia-smi and provides the following information:

  • Username
  • full process Command
  • GPU ID
  • PID

This is useful on multi-user servers and can be used to quickly identify which user is using the GPU and running what kind of program.

nvidia-smi && (nvidia-smi |tr -s ' '|grep -Eo "| [0123456789]+ N/A N/A [0-9]{3,} .*"|awk -F' ' '{system("s=$(cat /proc/"$4"/cmdline| tr \"\\0\" \" \");u=$(ps -o uname= -p "$4");echo "$1"sep"$4"sep$u sep"$7"sep$s" ) }'|sed 's/sep/\t/g')```
The output is like this

+-----------------------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |=============================================================================| | 0 3296 C python 5825MiB | | 0 21266 C /opt/conda/bin/python 6319MiB | | 1 3296 C python 137MiB | | 1 8063 C /usr/bin/python 273MiB | | 1 21266 C /opt/conda/bin/python 4735MiB | | 1 30439 C /usr/bin/python 199MiB | | 2 3296 C python 137MiB | | 2 21266 C /opt/conda/bin/python 4735MiB | +-----------------------------------------------------------------------------+ 0 3296 user1 5825MiB python playground2.py 0 21266 user2 6319MiB /opt/conda/bin/python -u /cail/model.py 1 3299 root 137MiB python playground2.py 1 8063 root 273MiB /usr/bin/python -u /sentsim/cli_predict.py 1 21266 root 4735MiB /opt/conda/bin/python -u /cail/model.py 1 30439 root 199MiB /usr/bin/python -u /sentsim/train.py 2 3296 root 137MiB python playground2.py 2 21266 root 4735MiB /opt/conda/bin/python -u /cail/model.py


For convenience, you can alias the command.
@savanovich
Copy link

savanovich commented Nov 25, 2024

That's extremely helpful, thanks

@awcator
Copy link

awcator commented Dec 10, 2024

Awesome. thanks

@huangrt01
Copy link

huangrt01 commented Apr 1, 2025

That's extremely helpful, thanks

@Lenzelot
Copy link

Lenzelot commented Jun 16, 2025

Thanks so much for the little script @padeoe! I also prefer simpler tools over nvitools though it is cool.

For zsh users, instead of making an alias (which would be a pain because of all the quotes) you can simply make a function.

function nvidia-smi-pro() {
    nvidia-smi && (nvidia-smi |tr -s ' '|grep -Eo "| [0123456789]+ N/A N/A [0-9]{3,} .*"|awk -F' ' '{system("s=$(cat /proc/"$4"/cmdline| tr \"\\0\" \" \");u=$(ps -o uname= -p "$4");echo "$1"sep"$4"sep$u sep"$7"sep$s" ) }'|sed 's/sep/\t/g')
}

@rleiser1995
Copy link

rleiser1995 commented Mar 3, 2026

If you are running the command on a MIG setup it won't work. For that, the fixed command is

nvidia-smi && echo -e "\nEnriched Process List:" && nvidia-smi | sed -n '/Processes:/,$p' | grep -E '^\| +[0-9]+' | tr -s ' ' | awk -F' ' '{ pid=$(NF-4); gpu=$2; "ps -o user= -p "pid | getline user; "cat /proc/"pid"/cmdline | tr \"\\0\" \" \"" | getline cmd; printf "GPU %s | PID %s | User %s | Mem %s | Cmd %s\n", gpu, pid, user, $(NF-1), cmd; close("ps -o user= -p "pid); close("cat /proc/"pid"/cmdline | tr \"\\0\" \" \"") }'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment