How To Install Horovod with CUDA and NCCL (on Ubuntu 18.04)
Index
Prerequisite
- NCCL
- CUDA
- NVIDIA Driver
- Using Mulit-GPU
Install OpenMPI
1. Download OpenMPI
wget https://download.open-mpi.org/release/open-mpi/v4.0/openmpi-4.0.0.tar.gz
2. Install OpenMPI
tar -xzvf openmpi-4.0.0.tar.gz
cd openmpi-4.0.0
./configure --prefix=/usr/local --with-cuda
sudo make all install
3. run ldconfig
sudo ldconfig
Install Horovod
HOROVOD_GPU_ALLREDUCE=NCCL sudo -H pip2 install --no-cache-dir horovod
or
HOROVOD_GPU_ALLREDUCE=NCCL sudo -H pip3 install --no-cache-dir horovod