Local Signals

I already had two Raspberry Pis doing useful things around the house. One runs WeeWX and records data from my weather station, while another runs BirdNET-PiPy and spends its time listening for birds and trying to work out what is making the noise. Both systems collect rather more data than I need on a website, but I liked the idea of making some of it visible.

That became Local Signals.

The aim was not to move either system onto the web, nor did I particularly want my website talking directly to Raspberry Pis sitting on my home network. I really just wanted a small public window into what they were seeing: what the weather is doing, which birds have been around, and perhaps eventually how both of those things change over time.

The slightly awkward part is that Chips’nCode is a static site. That is generally exactly how I want it, because there is very little to maintain and not much to go wrong, but rebuilding the site every few minutes because the temperature changed or a bird chirped would be a fairly silly way of adding live data.

So Local Signals ended up with a small data layer of its own.

Getting the data out

On the weather side, WeeWX is already doing most of the difficult work. It collects the observations from the weather station and keeps its archive locally in SQLite, so I did not need another service polling the hardware or maintaining a second copy of the station data. Instead, a small exporter reads the latest WeeWX archive record, converts the values I want into sensible metric units, works out things such as the accumulated rainfall for the day and writes the result to weather.json.

BirdNET-PiPy works in much the same way from the website’s point of view, although the underlying data is obviously rather different. It listens through a local microphone, runs BirdNET-based acoustic identification and records its detections in its own SQLite database. My exporter reads that database and turns the useful public bits into birds.json, including the day’s detection and species counts, the birds appearing most often, recent detections and confidence values.

I could have had the website query those systems directly, but I really did not want either Raspberry Pi accepting connections from the public internet just so somebody could find out whether it was raining at my place.

Instead, both Pis push their JSON files to a small VPS using SSH and rsync. nginx serves them over HTTPS from data.chipsncode.com, with the appropriate CORS headers, and the Local Signals dashboard fetches them in the browser when somebody opens the page.

That leaves the website itself completely static. There are no automated repository commits every time the weather changes and no GitHub Actions jobs rebuilding pages throughout the day. The Pis continue doing their normal jobs at home, the VPS holds a tiny public representation of their latest data, and Chips’nCode simply asks for the current version when it needs it.

For what I wanted, I much prefer that separation.

What actually gets published

There is an important distinction between making some of the data public and publishing everything the systems know.

BirdNET-PiPy, for example, has location information associated with its detections. That is useful locally, but there is no good reason for a bird observation on my website to publish the exact coordinates of the microphone that heard it. The exporter therefore removes that precision before anything leaves the local system, rounding the location to one decimal place and identifying it only as an approximate Brisbane southside location.

The same principle applies to the rest of the export. Internal paths, usernames, private network details and other information that belongs to the systems rather than the observations stay at home. The JSON files are deliberately constructed public outputs, rather than sanitised copies of the underlying databases, which also makes it much easier to see exactly what can and cannot reach the website.

At the moment there are only two of those public feeds, weather.json and birds.json. Weather is exported roughly every 15 minutes and the bird data roughly every 30, which is why I describe the dashboard as live-ish rather than live. There is no real benefit in pretending a weather observation that is several minutes old is a live telemetry stream.

The dashboard also pays attention to the age of each feed. If one of the Pis stops publishing, the page can say that its data is stale rather than quietly presenting an old observation as current. More importantly, the two feeds are independent, so a problem with BirdNET does not stop the weather from appearing and a weather-station problem does not take the birds with it. Given that these are Raspberry Pis, home networking and assorted bits of software running for my own amusement, graceful failure seemed rather more useful than assuming everything would remain available forever.

What I ended up with

The result is a fairly small system, but I rather like what it joins together. A physical weather station and a microphone are collecting things happening in the backyard, WeeWX and BirdNET-PiPy are making sense of those observations locally, a couple of small scripts reduce them to something suitable for publication, and the VPS gives the otherwise static website somewhere to retrieve them from.

In practical terms, the flow looks like this:

WeeWX Raspberry Pi
    → local WeeWX SQLite database
    → weather.json
    → rsync over SSH
                         ↘
                           VPS / nginx
                         ↗       ↓
BirdNET-PiPy Raspberry Pi       data.chipsncode.com
    → local SQLite database             ↓
    → birds.json                    Local Signals
    → rsync over SSH

There is plenty more I could do with it. The data being collected locally lends itself to historical weather graphs, rainfall summaries, seasonal bird activity, individual species pages and comparisons between weather and what BirdNET is hearing. I am particularly interested in the longer-term data because that turns Local Signals from a snapshot of what is happening now into a way of looking back at what has been happening around the house over months or years.

For the moment, though, I’m happy with the fairly modest version that exists now. The original problem was simply to get useful observations from two private systems onto a public static website without exposing the systems themselves, and the little JSON layer does that rather neatly.