Welcome to DataScience01

Overview

Welcome to the documentation for ds01.sintef.no.

This machine has 24 physical cores, 512 GB of RAM, and 2x NVIDIA A30.

It is intended to support data science, data processing, and machine learning work.

You can access the machine through ssh or use Jupyter Lab.

You can monitor resource consumption in Grafana.

Now, you will probably want to read

1. Jupyter Lab
2. Shell Access, First Steps
3. Python
4. Job Scheduling, First Steps
5. Job Scheduling, Examples
6. Job Scheduling, Reference
7. Deep Learning (Tensorflow, Pytorch)

If you have questions, get in touch with Volker (volker.hoffmann@sintef.no).

Recent Changes

Date	Change
28 Jan 2025	Added a guide for Ollama and Open-WebUI. See here.
13 Oct 2023	Added an admin guide with some recipes. See here.
11 Aug 2023	Added /yolo partition to specs
06 Feb 2023	Updated link to JupyterLab. See here.
04 Nov 2022	Updated PyTorch instructions. See here.
22 Apr 2022	Now using reverse proxy for JupyterLab.
13 Jan 2022	Updated Deep Learning specific documentation. See here.
13 Jan 2022	Document how to request GPUs for jobs. See here.
08 Dec 2021	Updated documentation on useing Conda environments as Jupyter kernels. See here.
04 Dec 2021	Installed 2x NVIDIA A30. Removed Tesla K40c. Documentation update pending.
27 Mar 2020	Students (UiO, NTNU) can only run four jobs at once.
26 Feb 2020	Instructions for installing and using Tensorflow and Pytorch available.
21 Feb 2020	Tutorial for using (conda) environments from Jupyter Lab available.
02 Dec 2019	Now enforcing job memory limits. Add #SBATCH --mem=xxGB to your scripts.
01 Nov 2019	GPU-accelerated XGBoost available. Disabled shells within JupyterHub.
28 Aug 2019	System usage available in Grafana.

Specifications

Type	Description
CPU	2x Xeon Gold 6126 (12 Cores, 2.4 GHz, HT)
Main Memory	512 GB
GPU	2x NVIDIA Tesla A30
Storage	15 TB HDD RAID10 (/data), 800 GB SSD (/home), 8 TB SSD RAID0(!) (/yolo)