Getting Started

Before the hackathon, you need to prepare your environment. This consists basically in 3 steps:

Getting an account on Grid’5000 and validating your connection settings
Getting an account on IoT-LAB and validating your connection settings
Getting the dedicated lab environment and validate that everything works as expected

Grid’5000 part

Your Account

If you already have an account you can use it.

Otherwise we have automatically created an account for you prior to the school. This account is associated with the email used for the school’s registration and you should have received an email from Grid’5000 some days ago. The email looks like this. Please make sure you’ve filled the form correcty.

Note

You must have an pair of SSH key: for instance ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub.
The public part of the key ~/.ssh/id_rsa.pub must have been added to your account settings (https://api.grid5000.fr/ui/account)
This document proposes a quick way to get started on Grid’5000. Nevertheless the Getting Started tutorial is a must read.

Grid’5000 Crash test

Before proceeding the various tutorials during the school you need to make sure that you can connect properly to the platform. We’ll need two types of connections: first the SSH connection to the various nodes and the HTTPS connection to connect to the REST API (for instance to manage the reservations).

Please make sure you succeed in running all the steps below: ⚠**this is super important**⚠

Connection to the access machine (replace the g5k_login by your login on Grid’5000)

👍 Expected outcome: you can connect to the access machine from your local machine.

👎 Possible issues: your login is wrong, you don’t have set your SSH keys in the web ui …
```
$laptop) ssh <g5k_login>@access.grid5000.fr
```
Note

Consecutive SSH connections shouldn’t ask for any password (even if you set a passphrase to unlock the SSH private key). It’s indeed the SSH agent responsability to store your passphrase. Most likely the agent is activated on your system. If not, it’s a good time to have a look before going on.
Connection to a site frontend from you laptop

👍 Expected outcome: you can connect to a site from a site frontend from your local machine

👎 Possible issues: the transparent SSH jump is not set correctly

Please add the following to your .ssh/config to configure a transparent jump over the access machine:
```
Host !access.grid5000.fr *.grid5000.fr
ProxyCommand ssh -A <g5k_login>@access.grid5000.fr -W "$(basename %h):%p"
User <g5k_login>
ForwardAgent yes
```
Connect to a site frontend (directly from your local machine)
```
$laptop) ssh rennes.grid5000.fr
```
Connection to a node from your laptop

👍 Expected outcome: you can connect to a reserved node on Grid’5000 from you local machine

👎 Possible issues: the transparent SSH jump is not set correctly

On a first terminal reserve a node on Grid’5000:
```
$laptop) ssh rennes.grid5000.fr
$frennes) oarsub -I -t "allow_classic_ssh"
```
On another terminal, connect to the reserved node:
```
$laptop) ssh <nodename>.rennes.grid5000.fr
```
GET request on the REST API

👍 Expected outcome: you get the list of the latest jobs (a big json)

👎 Possible issues: your login/password is wrong
```
curl -u <g5k_login> -X GET https://api.grid5000.fr/stable/sites/rennes/jobs
```

POST request on the REST API

👍 Expected outcome: you’ve created and deleted a job

👎 Possible issues: your login/password is wrong

curl -u <g5k_login> -X POST https://api.grid5000.fr/stable/sites/rennes/jobs -d "command=sleep 100"
# delete it
curl -u <g5k_login> -X DELETE https://api.grid5000.fr/stable/sites/rennes/jobs/<uid>

IoT-LAB part

Your account

At your inscription, you received your username and password to access the IoT-LAB platform. Use them to configure and test your access to the testbed.

Further configuration

You need to add your SSH key in the WEB UI and create a credential files (~/.iotlabrc) to access the REST API.

This can be done:
- Automatically: There’s a tool to configure your SSH access and REST API access:
```
$laptop) python -m pip install --user iotlabcli
```
  You can reuse the key used for Grid’5000
```
# create the ~/.iotlabrc
$laptop) iotlab-auth -u $IOTLAB_USER -p $IOTLAB_PWD

# upload the public part of the key
$laptop) iotlab-auth -u $IOTLAB_USER -p $IOTLAB_PWD -k -i ~/.ssh/id_rsa
```
- or manually:
  - You need to add the public part of your ssh key using the Web Portal
  - Create the credential file:
    $laptop) echo <fit_login>:$(echo -n <fit_password> | base64) > ~/.iotlabrc
You need to create the nodes file (~/.iotlab_nodes) and set the list of IoT-LAB sensors assigned to you.
You can check the nodes available for your login by looking at the file: https://gitlab.inria.fr/rsdschool21/lab/-/blob/master/setup/rsd2021hack.csv
$laptop) cat > ~/.iotlab_nodes << EOF m3-1.grenoble.iot-lab.info m3-2.grenoble.iot-lab.info m3-3.grenoble.iot-lab.info m3-4.grenoble.iot-lab.info EOF

IoT-LAB crash test

SSH connection to a frontend

👍 Expected outcome: you can connect to the access machine from your local machine.

👎 Possible issues: your login is wrong, you don’t have set your SSH keys in the web ui …

$laptop) ssh <fit_login>@grenoble.iot-lab.info

GET request on the REST API

👍 Expected outcome: you get the list of the latest experiments (a big json)

👎 Possible issues: your login/password is wrong
```
curl -u <fit_login> -X GET www.iot-lab.info/api/experiments
```

Lab environment

The lab environment consists in a container containing all the necessary materials. You’ll need to perform the following step:

Configure python-grid5000: python-grid5000 is a python client to the Grid’5000 API.
Clone repository: Clone the repository of the hackathon: https://gitlab.inria.fr/rsdschool21/lab
Get the container: We pack all necessary software in a container image. You should use docker to run your environment (See below).
Run jupyter lab: Validate all the things :).

Configure python-grid5000

You’ll need to create a file ~/.python-grid5000.yaml which contains your credential information to access the Grid’5000 REST API.

echo '
username: MYLOGIN
password: MYPASSWORD
' > ~/.python-grid5000.yaml

# protect it
chmod 600 ~/.python-grid5000.yaml

Clone the lab repository

git clone https://gitlab.inria.fr/rsdschool21/lab rsd21lab
cd rsd21lab

Containerized environment

If you already have docker installed, you can use it directly to run the container. Otherwise, please install it: https://docs.docker.com/get-docker/.

You can directly download the docker image from Inria’s registry by doing:

docker pull registry.gitlab.inria.fr/rsdschool21/lab:latest
# make the image name shorter:
docker tag registry.gitlab.inria.fr/rsdschool21/lab:latest rsd-hackathon:latest

Alternatively, you can build the container yourself using the command below:

# from the root directory of the repo
docker build -t rsd-hackathon:latest docker/

Running Jupyter Lab

Make sure you are inside the rsd21lab directory (where the clone of https://gitlab.inria.fr/rsdschool21/lab has been made)

docker run -ti -v ~/.iotlabrc:/home/jovyan/.iotlabrc \
    -v ~/.ssh:/home/jovyan/.ssh \
    -v ~/.python-grid5000.yaml:/home/jovyan/.python-grid5000.yaml \
    -v ~/.iotlab_nodes:/home/jovyan/.iotlab_nodes \
    -v $(pwd):/home/jovyan/work \
    -v $SSH_AUTH_SOCK:$SSH_AUTH_SOCK \
    -e SSH_AUTH_SOCK=$SSH_AUTH_SOCK \
    -e JUPYTER_ENABLE_LAB=yes \
    -e NB_UID=$UID \
    -u root \
    --network host rsd-hackathon:latest

Magic command hints:

-v src:dest: mount the directory src into the dest directory inside the container
- needed by all the configuration files/directory (SSH, credentials)
- also needed if you are using a SSH Agent (we share the agent socket)
-e VAR=VALUE: set the environment variable VAR to VALUE inside the container
- SSH_AUTH_SOCK is used by the SSH client to find the location of the agent’s socket
- JUPYTER_ENABLE_LAB, NB_UID are captured by the startup script of the container. It allows to start jupyter lab and change the id of the jovyan user to the passed ID. This allows to share files more gracefully between the host and the container. Other options can be found here.

docker run -ti -v ~/.iotlabrc:/home/jovyan/.iotlabrc \
    -v ~/.ssh:/home/jovyan/.ssh \
    -v ~/.python-grid5000.yaml:/home/jovyan/.python-grid5000.yaml \
    -v ~/.iotlab_nodes:/home/jovyan/.iotlab_nodes \
    -v $(pwd):/home/jovyan/work \
    -v $SSH_AUTH_SOCK:$SSH_AUTH_SOCK \
    -e SSH_AUTH_SOCK=$SSH_AUTH_SOCK \
    -p 8888:8888 -P rsd-hackathon:latest jupyter-lab

On Mac OS, docker daemon runs in a VM, so sharing the Host’s network is a little tricky. In this case, we share the jupyter-lab’s port individually and use “-P” to expose all container ports in random host ports.

To verify the mapping used:

docker port <container_id>

Validate all the things

From the Jupyter Lab UI, open a terminal (File->New->Terminal) and

Redo Grid’5000 Crash test and IoT-LAB crash test.

👍 Expected outcome: Everything works as before

👎 Possible issues: The various mounting points are not set correctly.
Validate the use of python-grid5000:

👍 Expected outcome: you can browse the API, create a job and delete a job

👎 Possible issues: the configuration file ~/.python-grid5000.yaml is wrong or not mounted correctly inside the container.

(base) jovyan@mymachine:~$ grid5000
Python 3.9.6 | packaged by conda-forge | (default, Jul 11 2021, 03:39:48)

In [1]: gk.sites.list()
Out[1]:
[<Site uid:grenoble>,
<Site uid:lille>,
<Site uid:luxembourg>,
<Site uid:lyon>,
<Site uid:nancy>,
<Site uid:nantes>,
<Site uid:rennes>,
<Site uid:sophia>]

In [2]: job = gk.sites["rennes"].jobs.create({"name": "pyg5k", "command": "sleep 3600"})

In [3]: job
Out[3]: <Job uid:1817820 site:rennes state:waiting user:msimonin>

In [4]: job.delete()