DEMAC CLUSTER > Board Setup Guide

Board Setup Guide

Preparing First Node

Creating SD with Headless Distribution of Parallella OS

Note: Check the version of the board - you will need to know if it is z7010 or z7020 for the .img download.

Download the .img file: parallella - Creating a Bootable SD Card
- The z7010 chip is in the P1600 (Microserver) and P1601 (Desktop/Motherboard) versions
- The z7020 chip is in the P1602 (Embedded) and A101040 (Kickstarter) versions
- The chip version can be seen in the filename
- The DEMAC cluster uses the headless version with Epiphany support
Unzip the .img.gz file with gunzip
cd to the directory containing the OS file
Check the name of the SD Card. <d_name> should be mmcblk0 or sdb. <d_name> refers to the disk name, not the partition names
```
lsblk
```
Unmount the SD Card partitions with sudo umount <partition_name> (if they are mounted, you will see a mountpoint in lsblk output) and write the image to the SD Card
```
sudo dd bs=4M if=<parabuntu_filename>.img of=/dev/<d_name> bs=64k
```
Expand the larger partition so that it uses the rest of the SD Card’s free space (the dd command probably resized the SD Card partition). To do this, we used the Resize command inside of the partitioning utility cfdisk (but you can use whatever you find easiest)
Insert the SD Card into the parallella board before you power it on to continue

Note: There should be two partitions on the SD Card at the end of this process: one 100MB partition and one that is the remainder of the SD Card’s space and contains the .img file. You can check this by mounting the larger partition and examining its contents.

Setting Up Router

If you are doing the headless OS install while you’re on a large network, you will need to use a router and connect it to the internet, the parallella boards, and your personal computer.

Once everything is connected, open your router’s interface to find out what IP Address is being given to your parallella by default. Look for a tab that says Network. Then find a section labeled DHCP Client List. There you should see your parallella board and its IP Address.

With this IP Address, you can SSH into the parallella and set up a static IP address.

Connecting to Parallella Board with SSH

Note: For this section, <default_IP> is the dynamic IP address you found in the DHCP Client List.

Check connection to the board
```
ping <default_IP>
```
1. SSH into the board for the first time (default password is parallella) bash ssh parallella@<default_IP>

Setting Up Network

Manual Configuration

The configuration described below is the same that DEMAC_NC.sh executes.

Change hostname: edit /etc/hostname
Set other boards IP Addresses: edit /etc/hosts

Set static IP Address: use the below text for /etc/network/interfaces.d/eth0

	#The primary network interface
auto eth0
iface eth0 inet static
	address 10.0.0.41
	netmask 255.255.255.0
	gateway 192.168.10.1 #this should be the address of the router
	nameserver 8.8.8.8
	nameserver 8.8.4.4

Restart connection with the command below
```
ifdown eth0; ifup eth0
```
Automatic Configuration Using the Script

With <##> as the two digit identifier/node number (starting from 01), the name of each node is NOPA<##> by default.

git clone https://github.com/Darptolus/DEMAC cd Node_Config/ chmod a+x DEMAC_NC.sh ./DEMAC_NC.sh <##>

Perform a hard reboot of the boards that were changed after running the script by unplugging the power cable and restarting them.

Setting Up Keygen and Password-less Access on Boards

Set up a private public key pair on every node (including the head node). Make a temporary folder, generate a new key and make it an authorized key, and add all NOPAs to the known hosts as shown below.
```
mkdir tmp_ssh
cd tmp_ssh
ssh-keygen -f ./id_rsa
#Press enter twice to set and confirm an empty password
cp id_rsa.pub authorized_keys
for i in `seq 0 24`; do j=$(echo $i | awk '{printf "%02d\n", $0}');
ssh-keyscan NOPA$J >> known_hosts; done
cd ..
```

Manually Copying Keys for Every Node

Using reverse order, the loop below assumes that the configuration will occur from NOPA0. Reverse order avoids overwriting the ssh in the current node first.

for i in `seq 24 0`; do j=$(echo $i | awk '{printf "%02d\n", $0}');
scp tmp_ssh/* NOPA$j:.ssh/; done

Note: While not recommended for other systems, it should be possible to use the sshpass tool to avoid having to type the same password for all nodes. THIS IS A SECURITY RISK, AS IT EXPOSES THE PASSWORD IN PLAIN TEXT.

Running the SSH Config Script

Running this script is an alternative to manually copying the keys as described above.

./DEMAC_ssh_Config.sh

If the node has been added before, it is necessary to remove the key first:

ssh-keygen -f "/home/parallella/.ssh/known_hosts" -R "nopa<##>"
ssh-keygen -R 192.168.10.1<##>

Copy the updated files
```
scp -r ~/.ssh NOPA<##>:.ssh
```
- Test password-less access to the node bash ssh parallella@NOPA<##>
Update ssh keys and hosts file to all other nodes
```
./DEMAC_ssh_update.sh
sudo sed -i -re
's/([a-z]{2}\.)?archive.ubuntu.com|security.ubuntu.com/old-releases.ubuntu.com/g' /etc/apt/sources.list
```
Restoring a Single Node’s SSH Setup

This can be used when a single node has been re-installed. We need to obtain the new node’s host key, add it into the current known hosts to avoid getting the typical ssh response, then give the key to the new node and provide the host’s key information to all nodes. Change the environment variable NEW_NODE accordingly.

```

export NEW_NODE=NOPA24 ssh-keyscan $NEW_NODE >> ~/.ssh/known_hosts scp ~/.ssh/* $NEW_NODE:.ssh/ for i in seq 24 0; do j=$(echo $i | awk ‘{printf “%02d\n”, $0}‘); scp ~/.ssh/known_hosts NOPA$j:.ssh/known_hosts; done


**Note: If the node has been added before, it is necessary to the remove the key first as shown below.**

ssh-keygen -f “/home/parallella/.ssh/known_hosts” -R “nopa<##>” ssh-keygen -R 192.168.10.1<##>


## Configuring sshfs

### Updating & Upgrading

Comment out all existing release links and add to the file `/etc/apt/sources.list` the following:


```bash
sudo apt-get update && sudo apt-get upgrade

Installing sshfs

sudo apt-get install -y sshfs

Checking for / Creating Fuse Group

sshfs guide

Check if the fuse group exists
```
cat /etc/group | grep 'fuse'
```
- If the group exists, execute the following command bash sudo usermod -a -G fuse parallella
If the group doesn’t exist, create it and add the user to it
```
sudo groupadd fuse
sudo usermod -a -G fuse parallella
```
- Uncomment the line user_allow_other in the file fuse.config bash sudo vim /etc/fuse.conf

Configure NFS Folder

Modify the file /etc/fstab
```
sudo vim /etc/fstab
```

Replace the contents with the text shown below

# <file system>       <mount point>         <type>  <options>
sshfs#parallella@NOPA01:/home/parallella/DEMAC_nfs /home/parallella/DEMAC_nfs fuse comment=sshfs,noauto,users,exec,rw,uid=1000,gid=1000,allow_other,reconnect,transform_symlinks,BatchMode=yes,nonempty,_netdev,identityfile=/home/parallella/.ssh/id_rsa,default_permissions 0 0

Installing MPI

Unpack the tar file
```
tar xfz mpich-3.3.1.tar.gz
```
- Choose an intallation directory (default is /usr/local/bin) bash cd ~/DEMAC_config mkdir mpich-install
Choose a build directory
```
mkdir mpich-3.3
```
- Install fortran (optional) bash sudo apt-get install gfortran
Choose any configure options. See section 2.5.1 for a description of the most important options to consider.

Configure MPICH, specifying the installation directory, and running the configure script in the source directory

cd mpich-3.3.1
./configure --prefix=/home/parallella/DEMAC_config/mpich-install 2>&1 | tee c.txt

If fortran is not installed, use the disable-fortran option as shown below:

./configure --prefix=/home/parallella/DEMAC_config/mpich-install --disable-fortran 2>&1 | tee c.txt

Build and install MPICH
```
make 2>&1 | tee m.txt
make install 2>&1 | tee mi.txt
```
Refer to the MPICH installation guide for more information.

Enabling Epiphany (multinode)
- Include the line below in /etc/ssh/ssh_config ```

PermitUserEnvironment yes


- Check eSDK libraries with `cat /opt/adapteva/esdk/setup.sh`. The script `setup.sh` initializes the environment variables

- Add the below lines to `~/.ssh/environment` to modify the ssh enivronment variables

PATH=/opt/adapteva/esdk/tools/host.armv7l/bin:/opt/adapteva/esdk/tools/e-gnu.armv7l/bin:/opt/openmpi/bin:/usr/local/browndeer/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games LD_LIBRARY_PATH=/opt/adapteva/esdk/tools/e-gnu.armv7l/lib:/opt/adapteva/esdk/tools/host.armv7l/lib:/opt/openmpi/lib:/usr/local/browndeer/lib:/usr/local/lib: EPIPHANY_HDF=/opt/adapteva/esdk/bsps/current/platform.hdf MANPATH=/opt/adapteva/esdk/tools/e-gnu/share/man


## Miscellaneous

### Image Backup Example

```bash
df -h
sudo dd if=/dev/mmcblk0 of=~/CAPSL/DEMAC/Backup_SD/P1_171109.img
sudo dd bs=4M if=~/CAPSL/DEMAC/Backup_SD/P1_171109.img of=/dev/mmcblk0

Copying Files Through SSH

scp <source> <destination>

-Copying files from B to A while logged into B

scp /path/to/file username@A:/path/to/destination

-Copying files from B to A while logged into A

scp username@B:/path/to/file /path/to/destination