Our embedded lab and the testbeds
Inside the wilh121 lab in Vienna, and how we turn a shelf of development boards into repeatable, remote-controllable testbeds.
The lab at Wilhelminenstraße 121 in Vienna combines an embedded hardware lab, a workshop, a hackerspace and a hacker residency, with its own server room in the basement. It is where boards get designed, flashed, broken and fixed. Over time, the interesting question shifted from how do we build it to how do we test it, every time, without walking to the shelf.
From bench to testbed
A testbed is a small contract: every board can be powered, flashed and observed from software. Once that holds, firmware tests run like any other automated job, and a board that hangs at 3 a.m. can be recovered remotely.
Debugging is twice as hard as writing the code in the first place.
— Brian W. Kernighan, The Elements of Programming Style
That is the reason we want the loop to be automatic.
What each board gets
| Capability | How | Why |
|---|---|---|
| Power cycle | Switchable USB hub or relay | Recover hung boards without touching them |
| Flash | SWD, DFU or UART bootloader | Deterministic firmware under test |
| Console | UART to the host | Read logs, send commands |
| Reset | GPIO or hub port | Start every test from the same state |
Describing a testbed
We keep the hardware description in plain text next to the firmware. This is an example of the idea:
testbed: example-bench
boards:
- name: dut-a
mcu: esp32-s3
console: /dev/ttyACM0
power: hub1/port1
- name: dut-b
mcu: rp2040
console: /dev/ttyUSB0
power: hub1/port2
A small runner then flashes, resets and waits for a marker on the console:
#!/usr/bin/env bash
set -euo pipefail
board="$1"; image="$2"
power-cycle "$board"
flash "$board" "$image"
timeout 30 grep -m1 "BOOT OK" < "$(console-of "$board")"
echo "$board: pass"
To watch a console by hand, use Ctrl+A then K in screen.
What we test
- Secure boot and crypto engines on ESP32-S3 based edge security devices.
- Firmware migrations, for example OpenWrt on devices that never shipped with it.
- Hardened embedded and VoIP devices, including the bare-metal port of the Yealink T19P-E2.
Next steps
- Add current measurement per board to catch power regressions.
- Publish the runner and testbed descriptions as open source.
- Run the same tests against every new board revision automatically.
More on the lab: About RIAT LABS.