Run a GL application on the Nef cluster with virtualGL

The nef cluster comes with 6 nodes equipped with Nvidia GPUs.

Typically, they are used with cuda or opencl to run some highly parallel computations. But you may also run OpenGL applications on them.

Preliminary setup

If order to do this, a X server must be running on the nodes, with no display settings. It’s easy to generate a xorg.conf file for this setup using nvidia-xconfig:

nvidia-xconfig -a --virtual=1920x1200 --use-display-device=none

Now you can start the X server manually:

/usr/bin/X :0 -ac -audit 0  -nolisten tcp

note: these steps are done by the administrator. All GPU nodes on nef (OAR) are running an X server.

Setup when running on the INRIA network

Now as a user, reserve a GPU node:

oarsub -I -p "gpu='YES'" -l /nodes=1,walltime=2:0:0 

You now have a shell on nefgpuXX, start a vnc server:

vncserver -geometry 1920x1024

A lightweight window manager is installed on nef (fluxbox), so you may want to use use this .vnc/xstartup file:

#!/bin/sh
vncconfig -iconic &
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
fluxbox

Run a vnc client on your desktop (krdc, vinagre, …). If you are on the production network, you can directly connect to the vnc server at nefgpuXX:5901

Start an xterm on the vnc. Now in order to use the GPU accelerated X, you have to use vglrun to start your application: You can test it with glxgears: vglrun glxgears. virtualgl will use the local X server to render the application (using the GPU), and will display the result on the unaccelerated vnc.

If you have to run a qt5 application, don’t forget to add the +xcb option to vglrun, otherwise it will fail.

How to do it outside INRIA network

If you are not on the production network at sophia, it’s a bit more tricky.

Submit your job with an explicit private key (your local private key of nef with no passphrase)

oarsub -p "gpu='YES'" -k -i ~/.ssh/id_rsa -I -l /nodes=1,walltime=2:0:0

Copy this private on your laptop/workstation in ~/.ssh/id_rsa_nef, and don’t forget to
add a passphrase (ssh-keygen -f id_rsa_nef -p)

Add this in your .ssh/config (on your laptop/workstation)

Host *.nefoar
     ProxyCommand ssh nef-frontal.inria.fr -W "$(basename %h .nefoar):6667"
     User oar
     LocalForward 5901 127.0.0.1:5901
     IdentityFile ~/.ssh/id_rsa_nef
     ForwardAgent no

Now you can connect to your gpu node to establish the ssh tunnel

ssh nefgpuXX.nefoar

You can connect to your vnc server on localhost:5901