How to enable NVIDIA Docker support on a GPU Cloud Server?

To enable NVIDIA Docker support on a GPU cloud server, you need to install the NVIDIA GPU drivers on the host machine, install Docker, and then install the NVIDIA Container Toolkit. This toolkit bridges Docker and the NVIDIA GPU drivers, allowing Docker containers to access the GPU hardware for accelerated computing. After installation, running GPU-enabled Docker containers requires the --gpus flag. Cyfuture Cloud supports this setup on their GPU Cloud Servers, enabling seamless containerized GPU workloads.

Introduction

NVIDIA Docker support is essential for using GPU acceleration within containerized applications. Traditional Docker does not natively support GPUs due to specialized hardware dependencies. NVIDIA has developed the NVIDIA Container Toolkit to enable Docker containers to utilize GPUs without requiring drivers inside the containers themselves. This is particularly valuable for AI, machine learning, and HPC workloads running on cloud GPU servers like those provided by Cyfuture Cloud.

Prerequisites for NVIDIA Docker Support

A GPU cloud server with NVIDIA GPUs (preferably supported CUDA-compatible GPUs).

Latest NVIDIA GPU drivers installed on the host (visible via nvidia-smi).

A supported Linux operating system on the server (Ubuntu 18.04, 20.04, 22.04 common).

Docker installed and working on the server.

Access to root or sudo privileges to install and configure software.

Step-by-Step Setup Guide

Install NVIDIA Drivers
Update the server and install NVIDIA drivers compatible with your GPUs. Verify installation with:

bash

nvidia-smi  

This command should show your GPU device details.

Install Docker
Install Docker Engine following official Docker documentation or Cyfuture Cloud’s specific instructions. Post-install steps often include managing user permissions to run Docker without sudo.

Install NVIDIA Container Toolkit
Add the NVIDIA package repositories and install the container toolkit:

bash

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)  

curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg  

curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | sudo tee /etc/apt/sources.list.d/nvidia-docker.list  

sudo apt-get update  

sudo apt-get install -y nvidia-container-toolkit  

sudo systemctl restart docker  

Test NVIDIA Docker Setup
Run a test container to verify GPU access inside the container:

bash

docker run --rm --gpus all nvidia/cuda:latest nvidia-smi  

This command should output GPU details as visible from inside the container.

Running GPU-Enabled Docker Containers

Once setup is complete, you can run any GPU-accelerated workloads with Docker on your Cyfuture Cloud GPU Server by using:

bash

docker run --gpus all   

You can also limit GPU access to specific GPUs using options like --gpus '"device=0,1"'. This flexibility supports workload isolation and efficient GPU resource sharing.

Common Issues and Troubleshooting

NVIDIA drivers not detected: Verify drivers are installed and working outside Docker with nvidia-smi.

Docker not recognizing GPUs: Ensure the NVIDIA Container Toolkit is installed and docker daemon was restarted after installation.

Permission errors: Add your user to the Docker group and re-login.

Container runtime errors: Use Docker’s default runtime or configure the NVIDIA runtime in /etc/docker/daemon.json.

Follow-Up Questions

Q: Can I use multiple GPUs in one Docker container?
A: Yes, specify multiple GPUs using the --gpus flag with device IDs, e.g., --gpus '"device=0,1"'.

Q: Do I need to install CUDA inside the container?
A: No, NVIDIA containers come with CUDA libraries. The container can access the host’s GPU drivers via the toolkit.

Q: Is this supported on all Linux distributions?
A: Most common distros like Ubuntu and CentOS are supported. Check NVIDIA’s docs for compatibility.

Conclusion

Enabling NVIDIA Docker support on a GPU cloud server transforms your infrastructure into a powerful, flexible environment for AI, machine learning, and GPU-accelerated applications. Cyfuture Cloud provides optimized GPU servers preconfigured or configurable for NVIDIA Docker support, making it easy to deploy containerized GPU workloads with efficiency and scalability. Following the installation steps for NVIDIA drivers, Docker, and the Container Toolkit ensures you leverage full GPU power inside Docker containers. This setup accelerates development workflows with portability, consistency, and GPU access across cloud instances.