Admin Notes

User Database

A shared user database is maintained in /mnt/nfs-main/etc/extrausers. The passwd, group, shadow files from this directory must be copied into /var/lib/extrausers for the changes to take effect. This directory is then read by the nss-extrausers package.

Creating a user works as follows:

  1. cd /mnt/nfs-main/etc/extrausers

  2. cat passwd to find a free UID

  3. Create it locally on some machine: adduser --uid NNN USERNAME

  4. ./relocate-user USERNAME

  5. vi passwd (in extrausers) to change home directory to /exthome/USERNAME

  6. vi group (in extrausers) to add user to relevant gpu groups

  7. mv /home/USERNAME /exthome/USERNAME

  8. Run /mnt/nfs-main/etc/extrausers/update-system on each participating machine. (done automatically at boot) This will perform the copy to /var/lib/extrausers described above.

  9. Don't forget to add SSH keys to /exthome/USERNAME/.ssh/authorized_keys.

The user db should have home directories point to /exthome/<username>, which (for now) symlinks to /mnt/nfs-main/home, which is where home directories physically live.

UIDs and GIDs in the shared user DB should be 2000 or greater to avoid clashes with locally-created users.

Changing someone's password

  1. Pick a nice password for them, pwgen works well for this. It spits out a screenful of halfway-pronounceable passwords.

  2. Pick a salt. Another one of the passwords spit out by pwgen should work.

  3. Stick the output of
    python -c 'import crypt; print crypt.crypt("THEIR_NEW_PASSWORD", "$6$SALT$")'

    in the password field of that user in /mnt/nfs-main/etc/extrausers/shadow. Make sure to replace THEIR_NEW_PASSWORD and SALT with the actual values. Notice that the salt appears verbatim at the beginning of the output.

Access to haamster

Simply drop the new user's SSH key into /home/extuser/.ssh/authorized_keys on haamster.

GPU Access

GPU access is by the group gpu for Nvidia, and amdgpu for the (unnecessarily) more sensitive access to the entire X11 server and thereby the AMD GPU.

How's this implemented? Check /etc/init.d/nvidia-kernel where there's an extra

setfacl -m g:gpu:rw /dev/nvidiactl

and

setfacl -m g:gpu:rw /dev/nvidia$i

Also NVIDIA_CARDS=N must be correct in /etc/default/nvidia-kernel.

For AMD cards, we use a snippet in /etc/gdm3/Init/Default to call a script /etc/enable-amd-compute, which contains:

xauth extract /tmp/x11-auth-file "$DISPLAY"
chmod 660 /tmp/x11-auth-file
chgrp amdgpu /tmp/x11-auth-file

cat > /tmp/enable-amd-compute <<EOF
export COMPUTE=$DISPLAY
unset DISPLAY
export XAUTHORITY=/tmp/x11-auth-file
EOF

OpenCL

The AMD and Nvidia OpenCL implementations (and the GPU drivers) are available as packages from Debian. Also install opencl-headers. The Intel implementation must be installed by hand.

Setting up a new machine

  1. Install libnss-extrausers.

  2. Change /etc/nsswitch.conf to read

    passwd:         compat extrausers
    group:          compat extrausers
    shadow:         compat extrausers
    Just change these three lines, leave the rest alone.
  3. cd /; ln -s /mnt/nfs-main/home exthome

  4. Make sure to add the following line to /etc/rc.local:

    /mnt/nfs-main/etc/extrausers/update-system
  5. Run /mnt/nfs-main/etc/extrausers/update-system

WarburtonCluster/AdminNotes (last edited 2013-08-19 22:35:29 by AndreasKloeckner)