Configuration
See the Configuration
section of the Jupyter package.
Usage
What can the deep-learning-tools package be used for?
The purpose of the deep-learning-tools package is to provide a pre-configure environment for performing deep-learning related tasks. Widely used deep-learning libraries such as PyTorch, Tensorflow, Keras, CNTK, mxnet, Theano and caffe2 are pre-installed. This package also provides all the necessary drivers for using these tools with GPUs, as well as common data-science libraries such as pandas, scipy and numpy (and many more).
This package is an extension of the Jupyter package, so it may be worth reading its documentation first, as it contains basic information on how to use the Jupyter notebook.
Using GPUs in Jupyter
In order to use a GPU, select a machine type which provides a GPU when
installing or reconfiguring. The machine types that provide GPU are suffixed
with w/ GPU
.
After installing, navigate to the Jupyter notebook, and run the following code
to test whether a GPU is found. The code will output the model name of the GPU.
import torch
torch.cuda.get_device_name(0)
This example uses PyTorch, but all the other libraries should also be able to find and use GPUs.
The output should be similar to Pascal Titan X
.
Useful introductions to various deep learning libraries
- Deep learning with PyTorch: A 60 minute blitz
- Getting started with Tensorflow + Keras
- MXNet - In-depth guide
Using Jupyter with Apache Spark
The Connecting a Jupyter notebook to Apache Spark
section of the following
link describes how to use Apache Spark with Jupyter
How to add new packages
In case you are missing some packages from the default application image, you can add those packages yourself by creating a custom docker image. See this tutorial for generic instructions on how to add packages.
After having read the tutorial above, you can use the dockerfile below as a starting point when creating the dockerfile that adds new packages.
# See the value of dockerImage in
#
# https://github.com/Uninett/helm-charts/blob/master/repos/stable/deep-learning-tools/values.yaml
#
# to determine the latest base image
FROM quay.io/uninett/deep-learning-tools:<use latest tag here>
# Install system packages
USER root
RUN apt update && apt install -y vim
# Install other packages
USER notebook
RUN pip install fastai