Getting started with the JWRD key management hardware kit

Filed under: Gales Linux, Hardware, JWRD, Networks, Paidagogia, Software — Jacob Welsh @ 07:37

It's high time I captured the previously oral, informal or otherwise behind-closed-doors introduction to the hardware package we've been selling slowly but steadily for some years now.


The complete kit presently consists of the following items:

  • 2x ThinkPad X200 notebook computer with US keyboard, power supply and battery (Lenovo; refurbished, tested, Coreboot firmware flashed, and Gales Linux installed on new drive, all by JWRD)
  • 1x random number generator board (No Such lAbs FUCKGOATS)
  • 1x 3-port Gigabit Ethernet router and firewall with power supply (Ubiquiti Networks EdgeRouter Lite; OpenBSD installed by JWRD)
  • Optical Data Diode consisting of:
    • 2x visible red light serial transceiver board (MIKROE Fiber Opt Click)
    • 1x jacketed plastic optical fiber
  • 1x USB to RJ45 serial console adapter cable (not pictured; for router management)
  • 3x USB to TTL serial adapter breakout cable (for RNG and optical boards)

photo of Offline and Online machines, Optical Data Diode, TRNG, and Router-Firewall

The ThinkPads are configured one for Online and the other for Offline use. One difference is that the Online has a Solid-State Drive (SSD) to meet the performance demands of a Bitcoin node, while the Offline retains the older Hard Disk Drive (HDD) technology.(i) The other difference is that the Offline has its front-right daughterboard disconnected, which disables the front-right I/O ports including SD card reader, USB, audio and modem, on the general security principle of minimizing non-essential components. Naturally, drives can be swapped and boards reconnected at the owner's will. For both machines, any WiFi, wireless WAN, or Bluetooth radio adapters that may have been present are fully removed with antenna leads duly insulated.

The main tool required for servicing the ThinkPads is a Phillips head #0 (jeweller's) screwdriver. As laptops go, they are well designed for serviceability with many "Field-Replaceable Unit" (FRU) parts, though availability of new replacement parts is mixed.

Basic ThinkPad operation

The power button is hard to miss; it's at the top of the keyboard, round with a white dot in the center. The screen should light up promptly; there won't be much output from Coreboot as it's designed to initialize the basic hardware, transfer control to its payload and get out of the way as quickly as possible.(ii) The first thing you see should be a SeaBIOS boot prompt, which pauses for about a second; in this interval you can interrupt it by pressing Escape and it will offer a menu of boot devices. Assuming the local drive is booted, next comes a LILO prompt, which again can be interrupted for about a second to choose an alternative kernel or boot options, if such alternatives have been configured.

As the Linux kernel boots, the screen may blank briefly as the graphics driver loads, switching the LCD panel to its native resolution. If you connect an external display to the VGA port, it will also be auto-configured at this time.(iii) Within a few blinks of an eye you'll be looking at the login prompt. Unless requested otherwise, our machines ship standard with the highly secure default logins of "root", password "root" for administrative operations and "user", password "user" as a lower-privileged account for routine work. (Without the quotes, of course.)

To change these passwords you can run the passwd command, as the respective user, and follow the prompts. If you're unaccustomed to Unix systems, note that as a security feature, passwords will not display ("echo") on screen while you're typing them, not even as dots or stars. You just hit Enter when you're done.

To safely shut down the machine, run the poweroff command. This must be done as the root user.(iv) This goes even faster than the power-on sequence. To reboot the machine, run the reboot command. From the text mode console, it also suffices to press Ctrl-Alt-Delete, whether logged in or not.

Suspend to RAM aka sleep mode does work to some degree, but I tend to see flaky hardware behavior some time after resuming, ultimately requiring a reboot to resolve. If you want to try, the command is printf mem > /sys/power/state.

To control backlight brightness or toggle the ThinkLight (keyboard lamp located at top of screen), hold the alternate function (Fn) key and press the Home, End or PageUp keys which include the respective symbols.

Note that there is little indication of battery capacity given by the hardware status lights nor, at present, in our software environment. It can be queried from Linux sysfs: cat /sys/class/power_supply/BAT0/capacity which reports the percent full. But battery life on these machines has never been all that great anyway; so in general it's best to think of the battery as a short-term backup while moving the machine around or during a power failure, and otherwise use the AC adapter as primary power source.

Complete source code for the various system components is included under /var/build and /gales/dist/src (the Gales mirror). Beyond the base system, a number of handy gports are also prebuilt, which can be seen under /gales/pkg.

Serial adapter configuration

To be written, but for the bare essentials: all devices used here (RNG, Click boards and router console) run at the traditional maximum of 115`200 baud. Set this using stty -F /dev/ttyUSBx 115200 where x identifies the specific adapter, counting up from 0. For most uses you'll also need to set the fully raw mode, stty ... raw -echo -echoe -echok.

Router and network setup

As mentioned, the EdgeRouter Lite (ERL) is a three-port router. Why, then, does it have four same-looking ports on the front? Look closely: three of them are in a group labeled eth0, eth1 and eth2, while the fourth is on its own to the left and labeled CONSOLE. They do indeed use the same RJ45 type connector, but are wired to completely different circuitry, so don't confuse them! At best it won't work and at worst it could damage something (though I haven't observed this yet).(v)

The three networks we configure the router to serve are known as External, Trusted and Untrusted, mapping respectively to eth0, eth1 and eth2. (The OpenBSD OS on the router labels these instead cnmac0, cnmac1 and cnmac2.)

External means the upstream network, where (by default) the router will obtain its IP address configuration automatically from the provider using DHCP. The other two are considered the "internal" networks, and address translation (NAT) is performed on the internal-external boundary.

Trusted is intended to serve machines with higher security requirements, such as the online ThinkPad. There is no automatic configuration provided on this network, meaning each machine on it must have a static IP configuration (address, netmask and default route). By the same token, service is more reliable because there's no risk of the IP dropping or changing unintentionally (perhaps due to a flaky link or a rogue peer on the network). The firewall is somewhat strict, blocking outbound connections by default except to specifically allowed port numbers.

Untrusted is a more traditional home-style network which serves DHCP clients and allows a wider spectrum of outbound connections, while still blocking inbound connections and insulating the Trusted machines from traffic originating here.

Thus, you can use the ERL as your primary home router (barring inept configurations from non-service providers), perhaps connecting a wireless access point downstream on the Untrusted network (eth2) for guest use, Ethernet-impoverished devices and the like. If switching from a basic consumer router, you may well notice performance or reliability improvements across the network, especially when dealing with large numbers of connections.

Alternatively, for a less disruptive change, you can connect its External port (eth0) downstream of an existing home router. Downsides of this approach are that it creates a double NAT, has two stateful connection trackers rather than one (more ways for things to fail), and potentially exposes traffic from the Trusted network to more listeners than it otherwise would have been exposed to, especially via WiFi. That said, due to the constraints of not fully owning my living space, I've been running this way for several years now without complaint.

The online ThinkPad and router have been preconfigured to work together on the Trusted network; thus, the router acts as a dedicated interfacing appliance, allowing a stable configuration for the computer inside a possibly shifting exterior network. The ThinkPad should also have an SSH key pre-generated which is authorized to log in to the router for administration, or it can be used as a serial console host for more direct access when the network isn't functional (the initial console login credentials as before are username root, password root). These topics are covered in more depth in our operator training program.

Each router we ship uses a randomized prefix in the range for its internal networks. This reduces the odds of addressing conflicts with existing or future private networks, even when tunneling or otherwise bridging together multiple such networks previously operated separately.

A note about domain name service is in order: neither the ThinkPads nor the ERL ship with any nameservers configured. For the ThinkPad, this is a basic security hygiene measure and it means anything you wish to connect to by domain name needs to be listed in the machine's own local addressing directory, its /etc/hosts file (we include a few of our own sites there for starters). For the ERL, it means that even DHCP clients on the Untrusted network won't receive a nameserver configuration. This is mainly because we view the choice of provider there as a personal decision that we can't really make for you. If you require DNS service, the premium option is to run your own local DNS cache (perhaps on a Linux machine on the trusted network with port 53 opened in the firewall). Second best is perhaps to check the configuration your ISP distributes to see if they're running their own, or pick some specific service provider. As a last resort, there are the public toilets like Google's or Cloudflare's, pick your poison. Once chosen, you add the IPs in the router's /etc/dhcpd.conf file and either reboot it or just restart the DHCP daemon: /etc/rc.d/dhcpd restart.

Ideally, you would also always safely shut down the router before unplugging it; the OpenBSD spelling for this is: shutdown -p now. OpenBSD is worse than Linux at hands-free filesystem recovery following a hard reset, but from experience we've made a number of configuration changes which greatly reduce the chances of there being any filesystem operations in progress subject to corruption or otherwise requiring manual intervention.

tl;dr: eth0 goes to your upstream provider. eth1 goes to your online ThinkPad. eth2 goes to your existing shit, if applicable.

  1. HDDs were once thought to be more amenable to secure data destruction (archived) by both electronic and physical means. Meanwhile they got the shingles and experience shows SSDs might not even be so bad in comparison (archived) so I dunno, maybe it's time to revisit this design and go all SSD for laptops, saving some logical warehouse space at the very least. [^]
  2. Due to the complexity of modern and even not-quite-so-modern PC hardware, even this seemingly simple task still requires extravagant quantities of code. [^]
  3. The way this seems to work is that each display runs at native resolution, with the console shown on all outputs but only covering the greatest common sub-rectangle i.e. so that the full character matrix can be seen on all displays. Not ideal, but short of getting X11 ported it seems to be the best we can do. [^]
  4. Unix was designed as a multi-user system, first in the minicomputer era serving multiple concurrent serial terminal users, then into the age of networks where those concurrent users could be anywhere. Imagine the havoc if some troublemaking troll in the midst of those users could shut down the whole system on demand! Actually, in practice there tend to be more ways that ordinary users can still cause such trouble or worse, though it tends to require a bit more deliberate malfeasance. [^]
  5. Use of this same connector type for both network and console ports is fairly common in enterprise gear; I guess it economizes on connector stock-keeping, plus in a pinch you can get away with using an Ethernet cable as a console cable. [^]

1 Comment »

  1. [...] yet matured to that point. I'll connect it to untrusted networks such as cafes and given the enlightened design of my network, I connect it to my untrusted network at headquarters so there's a low probability a [...]

    Pingback by The root of our dispute is I value substance over form and thus manage to learn and grow, while you cling to form and, thus, dwell in mediocrity. on Dorion Mode - A blog by Robinson Dorion. — 2023-07-28 @ 13:46

RSS feed for comments on this post. TrackBack URL

Leave a comment

Powered by MP-WP. Copyright Jacob Welsh.