How to install NVIDIA Docker 2 package on Ubuntu and Debian:

If you came to this result (from Google or elsewhere) after realizing that Nvidia-docker's entry on this subject does not result in a working installation, here are the basic steps needed to install this package correctly:

For starters, ensure that you've installed the latest Docker Community edition by following the steps below:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install docker-ce
sudo service docker restart

Then, proceed to adding the required repositories and installing the needed software:

Start with nvidia-docker-runtime:

For Ubuntu distributions (Xenial x86_64):

First, if you have older nvidia-docker installations, purge the installation and all associated GPU containers:

docker volume ls -q -f driver=nvidia-docker | xargs -r -I{} -n1 docker ps -q -a -f volume={} | xargs -r docker rm -f
sudo apt-get purge -y nvidia-docker

Then proceed:

curl -s -L https://nvidia.github.io/nvidia-container-runtime/gpgkey | \
  sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-container-runtime/ubuntu16.04/amd64/nvidia-container-runtime.list | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-runtime.list
sudo apt-get update

For Debian distributions (Stretch x86_64):

curl -s -L https://nvidia.github.io/nvidia-container-runtime/gpgkey | \
  sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-container-runtime/debian9/amd64/nvidia-container-runtime.list | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-runtime.list
sudo apt-get update

Install the nvidia-container-runtime package:

sudo apt-get install nvidia-container-runtime

Followed by nvidia-docker2:

sudo apt-get install nvidia-docker2

Docker Engine setup:

To register the nvidia runtime, use either method below on Ubuntu 16.04LTS+. It is recommended that you use the systemd drop-in file method to prevent docker upgrades from directly overwriting the docker daemon configuration file.

(a).Systemd drop-in file (Recommended approach, but see notes below):

sudo mkdir -p /etc/systemd/system/docker.service.d
sudo tee /etc/systemd/system/docker.service.d/override.conf <<EOF
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd --host=fd:// --add-runtime=nvidia=/usr/bin/nvidia-container-runtime
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker

(b). Daemon configuration file (Use if you're on a distribution such as Devuan, without a systemd dependency):

sudo tee /etc/docker/daemon.json <<EOF
{
    "runtimes": {
        "nvidia": {
            "path": "/usr/bin/nvidia-container-runtime",
            "runtimeArgs": []
        }
    }
}
EOF
sudo pkill -SIGHUP dockerd

Basic usage:

nvidia-docker registers a new container runtime to the Docker daemon. You must select the nvidia runtime when using docker run, as illustrated below (with nvidia-smi):

docker run --runtime=nvidia --rm nvidia/cuda nvidia-smi

Notes:

If the systemd unit method fails, skip straight to the configuration file method. The result is the same.