VMW Research Group Raspberry Pi Cluster

Update! See info on the newer, pi-4 based cluster we built in 2024

Summary

We have created a 24-node Raspberry Pi 2 cluster, with each node instrumented for detailed power measurement. This totals 96 cores with 24GB of RAM.

The cluster draws around 92W when under load, and around 70W when idle.

There is a head node (also a Pi2). It was planned to have a touch-screen interface and run off of the ATX standby power but we never managed to finish that. It has two ethernet adapters and serves DHCP, NFS, and ganglia. We use SLURM for job submission.

It obtains 15.4 GFlops of performance, meaning in June 1993 it would have been #7 on the Top 500 list.

================================================================================
T/V                N    NB     P     Q               Time                 Gflops
--------------------------------------------------------------------------------
WR00R2L1       48000   256     6     4            4780.68              1.542e+01

Publications

Cloutier, M.F.; Paradis, C.; Weaver, V.M. A Raspberry Pi Cluster Instrumented for Fine-Grained Power Measurement. Electronics 2016, 5, 61.
An earlier version of this work was presented at the Co-HPC workshop at SC'14 in New Orleans, 17 November 2014. An extended tech-report of that presentation can be found here: 2014_cohpc_cluster_extended.pdf (14 November 2014)
The raw data used in our plots can be obtained here:
git clone https://github.com/deater/performance_results

Documentation

Full cluster build instructions, code used, etc, will be posted RSN.

Cluster Construction Progress

13 November 2015

Finally have all 24 nodes up and running and ran a full Linpack run.

11 November 2015

Cluster is together enough that students are using it for ECE574 HW#6 to run MPI jobs.

30 October 2015

Finally soldering together the front-end boards for the rest of the cluster:

Putting together the i2c and USB cables was a bit of a job too:

Soldering some 7" USB cables. This is such a pain, I'm impressed Michael managed to make most of the rest of them.

One quad in place, DHCP server synched, ready to run jobs:

13 August 2015

Case modification is complete:

4 August 2015

Got a shipment of Pi2's, time to upgrade the cluster:

30 July 2015

Making progress for modifying the case to hold the cluster:

18 June 2015

Finally put together a PCB for the master nodes. Each sub-node is a group of 4 Raspberry Pi systems connected to a front panel. The first pi in each group acts as a master node. The front-panel has a power switch and provides power to the four nodes. The power is measured with an SPI A/D converter which is connected to the master node. The master node also has a 1-wire connector for hooking up temperature probes. Each node has one 8x8 bi-color LED display connected via i2c. Each node also brings out a UART connector to the front panel.

Mostly populated front-panel board:

The back showing the 0.1 Ohm sense resistors and the 5v to 3.3 level shifter. Note there are mounting holes so either Model B or Model B+ can be used.

Prototype in action, showing all 4 displays going. The individual pis are powered via the USB interface.

Now it's time to consolidate everything into the case.

26 May 2015

Begun laying out the PCB for the power boards.

January 2015

Prototype power management board made, with display. Also tried various i2c and 1-wire ways of trying to get detailed thermal readings.

14 November 2014

Prototype cluster with 16 Raspberry Pi Model Bs and 16 Model B+. The fans are needed when they are running overclocked. There's a touch screen display that gives current status of the cluster.

16 July 2014

Initial prototype of the cluster put together by undergrad summer researcher Michael:

More Pictures, Video, and Construction Details will be Forthcoming.

Back to projects page.