In the off-grid and emergency preparedness communities that pride themselves on removing their reliance on traditional infrastructure, many individuals have begun using Meshtastic devices. This project allows for messages to be sent peer-to-peer by utilizing LoRa (Long Range) radio technologies for the transmission. The network that is built up between these Meshtastic radios becomes extremely powerful as it has mesh capabilities that allow messages to be automatically forwarded or repeated to other Meshtastic radio users. This functionality is extremely useful when cellular towers are either nonexistent in the area or are overloaded due to the sheer number of users attempting to utilize them. This is something that occurs quite frequently in natural disasters, or even community events where individuals are gathering in large numbers.
By nature of this Meshtastic technology, any user that would like to communicate on this network is required to use their own Meshtastic radio. This can be cumbersome for some users as they may not have the technical expertise or desire to acquire this radio equipment. This need for lowering the barrier to entry prompted the creation of a gateway service, Mesh Relay, to forward on messages from the Meshtastic network to SMS and receive from the other direction. Utilizing Mesh Relay, users are able to send messages to their friends or family as well as receive messages when cellular coverage is limited in their local area.
In order to eventually support many Mesh Relay gateway instances, the system was designed to utilize a cloud-hosted hub server as well as a locally hosted server component running on the edge. Since these two components just communicate over HTTP, the only requirement for the local gateway device is that it has an internet connection. This means that a Mesh Relay instance can be set up very easily in a location that has either cable or fiber internet connectivity, and no special provisions for networking need to be made. While the hub server wouldn’t have been strictly needed for a single relay gateway device, this was designed to be scalable and allow a number of instances to connect.
The local gateway software component consists of a Python script that utilizes the Meshtastic Python library to both send and receive messages over the Meshtastic network. This library allows a developer to control almost any portion of a Meshtastic radio over this interface. The library supports connectivity over serial, Bluetooth, or TCP/IP, but we chose TCP/IP for simplicity for the end user. After the program receives an incoming message from the Meshtastic radio, it then forwards it on to the hub server using SocketIO technology. SocketIO was selected in this situation to allow for multiple gateway servers to easily connect from anywhere on the internet. Since the local gateway server is packaged and deployed as a Docker container, it is extremely portable and can be put on any hardware that supports Docker. This edge device can be anything from a laptop or PC to even a Raspberry Pi microcomputer that has Docker installed. This gives incredible flexibility to users wishing to deploy Mesh Relay as there are no strict hardware requirements.
While the local gateway is relatively simple and is just a single Python program, the hub server is much more complex in both its architecture and functionality. Rather than building this as a monolithic application, a microservice approach was taken that splits each separate function into its own separate little program. Each microservice then communicates to each other as needed, mostly by means of REST APIs. Taking an approach like this is critical for scalability as well as reliability as the application grows in its complexity as well as its userbase. Obviously, the most critical service running accepts the SocketIO connections from the gateway servers running at the edge, but other microservices perform actions such as manipulating the cloud-hosted NoSQL database by MongoDB Atlas and interacting with the Twilio SMS services. Most of the microservices are written in JavaScript with the Node.js runtime environment as both Twilio and MongoDB Atlas provide excellent support for this platform with the libraries they provide. While much is written in JavaScript, the microservice handling the orchestration of relaying messages from the Meshtastic network to SMS is written in Go for lightweight, efficient operations.
In the development process, a Docker image was built for each one of these microservices so they could be deployed in their own, unique containers. This allows the applications to run in their own isolated instances and be entirely separated except for the API interfaces that are defined. By putting each application in a Docker image, it makes for an extremely simple deployment process. While this hub server was originally just running as individual containers on a single cloud-hosted virtual machine running Docker, it has since been migrated to Kubernetes to allow for better scalability and management.
The microservices are currently running on a three-node Kubernetes cluster by a cloud hosting provider, which means that there are three separate servers each running their own copy of the microservices. An example of where this becomes extremely powerful is when a microservice needs to be updated to a newer release, Kubernetes automatically scales up to six instances of that microservice, replaces them all with proper sequencing, and then scales back down to just three, all while ensuring the service never becomes unavailable. Additionally, if a server goes offline in the cluster, it will automatically spin up new instances of the microservice on the other hosts to ensure that the service performance is not reduced. Since Kubernetes automatically deploys these instances and directs web traffic to the microservice that is most capable of handling that request, it makes it extremely easy to administer this cluster and scale up resources as needed.
This project has been extremely helpful in demonstrating how critical it is to properly architect a solution before beginning to implement it. Even though this project is at a much smaller scale compared to commercial software development projects that takes years for a team of individuals to complete, there were still a number of design decisions that were made early on that ended up needing to change slightly due to unexpected issues that arose. One of the primary constraints holding me back was the documentation, or the lack thereof, of the Python library used to interface with the Meshtastic radio. Even though this is fully open-source and available to all, it still required an element of reverse engineering to be able to understand how other developers have used this in the past. Additionally, other technologies often have functionality that evolves over time, and it is critical to stay on top of those changes to ensure that downstream development is not affected.