I was previously using the AMI from the fast.ai course for my deep learning needs, but recently decided to try out the AWS Deep Learning AMI and I wanted to document a few tips for a few issues I encountered when I first set it up.
The AWS Deep Learning AMI does not come with the latest version of Keras, so you’ll need to update the keras package using:
sudo pip install keras --upgrade
Or, if you’re using Python 3, you can update it using
sudo pip3 install keras --upgrade
You’ll also need to remove older Keras configurations (if any) using:
I ran into some issues with CUDA initially, which turned out to be a configuration problem. If you encounter the following error (or something similar):
ImportError: libcusolver.so.8.0: cannot open shared object file: No such file or directory Failed to load the native TensorFlow runtime. See https://www.tensorflow.org/install/install_sources#common_installation_problems for some common reasons and solutions. Include the entire stack trace above this error message when asking for help.
Try updating the
LD_LIBRARY_PATH using the following:
sudo ldconfig /usr/local/cuda/lib64
Bonus: Autostarting Jupyter Notebook
Since I usually turn off my instance when I’m not using it, I wanted to autostart Jupyter whenever the instance was re-started to avoid SSHing everytime to start it up manually. Luckily, there’s a way to do that!
Create a new file called
jupyter_start.sh in your home directory, and add the following:
export PATH="$PATH:/home/ubuntu/anaconda2/bin" jupyter notebook --notebook-dir=/home/ubuntu/ --profile=nbserver > /tmp/ipynb.out 2>&1 &
anaconda3 if you’re using Python 3. You can also change
notebook-dir to the location of your Jupyter notebooks.
Next, edit the
/etc/rc.local file and add the following line before
su ubuntu -c 'bash /home/ubuntu/jupyter_start.sh'
Let me know if you encounter any issues, or have any additional useful tips!