• Do you need to run a different operating system entirely?

$ cat /etc/os-release
NAME="SLES"
VERSION="15"
VERSION_ID="15"
PRETTY_NAME="SUSE Linux Enterprise Server 15" ID="sles"
ID_LIKE="suse"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:15"

$ singularity shell ubuntu.sif
Singularity ubuntu.sif:~> cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.2 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.2 LTS"
VERSION_ID="18.04"
...
...

• Did you already invest in Docker?

The Singularity software can import your Docker images

●    without being a superuser
●    without having Docker installed

• Need to share your code?

Put it in a Singularity container and your collaborator won’t have to go through the pain of installing missing dependencies.

The basic idea is that every group creates its own singularity images and shares them with its own (or other) group participants.

This means that you don’t have to ask your cluster admin to install anything for you - you can put it in a Singularity container and run.

• What does your workflow look like?

Interactive Development                                           

sudo singularity build --sandbox tmpdir/Singularity

sudo singularity build --writable container.img Singularity

Build from Recipe

sudo singularity build container.img Singularity

Build from Singularity

sudo singularity build container.img shub://vsoch/hello-world

Build from Docker

sudo singularity build container.img docker://ubuntu

BUILD ENVIRONMENT == YOUR PC

PRODUCTION ENVIRONMENT == CAPRI 

Domain Adaptation for Semantic Segmentation (DEI)

●    […] We plan to exploit recent deep learning architectures on large images, thus we would highly benefit from most performing GPUs with many CUDA cores…

●    Python 3.6.7 (2.X is still default on most linuxes)
●    TensorFlow-GPU 1.12.0
●    CUDA Toolkit 9.0 
●    OpenCV 3.4.2
●    Matplotlib 3.0.2
●    Six 1.12.0
●    Numpy 1.15.4
●    Pandas 0.24.2
●    Keras 2.2.4
●    Scikit-Learn 0.20.2
●    Scipy 1.2.1

●    We decided to customize an existing Docker providing python 3 and Tensorflow-GPU (based on Ubuntu 18.04)
●    We created the Singularity file:

BootStrap: docker
From: tensorflow/tensorflow:latest-gpu-py3

%help
    This Singularity definition contains a TensorFlow-gpu installation

%post   

pip install scipy==1.2.1 six==1.12.0 numpy==1.15.4 pandas==0.24.2 matplotlib==3.0.2                          apt-get -y install libsm6 libxrender1 libxext6 libx11-6
pip install keras==2.2.4 scikit-learn==0.20.2 opencv-python==3.4.2.17                                   

%environment
    export LC_ALL=C

%runscript

%labels

    Author paoloemilio.mazzon

●      I have used my PC to build the singularity image

$ sudo singularity build tflow_opencv.sif Singularity

●      The image was then uploaded to Runner and was succesfully tested (including tensorflow “-GPU” part)

●    Singularity as a tool has proven to be

●    extremely flexible
●    easy to learn
●    sysadmin friendly

●     Singularity can access docker, singularity or local repositories (managed by sysadmins or CAPRI users): it's a huge amount of software!

●     Software management is made by you

●     By using it you create tools and know-how for you and your collaborators