Mender is an open-source, over-the-air (OTA) update manager for IoT and embedded Linux devices. Its client-server architecture enables the central management of software deployments, including functionality such as;
- Dynamic Grouping
- Phased Deployments
- Delta Updates
Key Features
Mender enables secure and robust over-the-air updates for all device software. Some of the core functionalities include:
- Flexible management server and client architecture for secure OTA software update deployments and fleet management.
- Standalone deployment support, triggered at the device-level (no server needed) for unconnected or USB delivered software updates.
- Automatic rollback upon update failure with an A/B partition design.
- Support for a full root file system, application, files, and containerized updates.
- Dynamic grouping phased rollouts to ensure update success.
- Advanced configuration, monitoring, and troubleshooting for software updates.
- Extensive logging, audits, reporting, and security and regulatory compliance capabilities.
Architecture
There are two components in the Mender architecture.
- Server
- Client (Linux Devices)
Other components can also be a part of hierarchical architecture are;
- Client connected peripheral devices
These peripheral devices aren’t directly updated by mender clients by user define plugins for mender client known as mender update modules
Mender Server
The Mender server stores and controls the deployment of software updates over-the-air to devices. Mender Server provides us with;
- Mender’s web UI or REST APIs to manage devices;
- Visualising Inventory
- Accessing device using SSH
- APIs to upload artifacts to the server
- Managing software releases to the server
- Create deployments to roll out software releases
- Mender server APIs to integrate Mender with CI environment;
- For example to automatically upload build output from your CI system to the Mender server and create deployments.
Dashboard
For testing a free demo server can be used, which is available at: https://hosted.mender.io/ui/signup
Mender Client
The Mender client runs on the device and reports to the server periodically to check for updates; if there is a software update available for that device, the client downloads and installs it.
For operating system updates, we require a dual A/B rootfs partition layout on the device. This ensures that the device can recover even if the deployment is incomplete or corrupted during installation for any reason, e.g. due to power loss during the update process.
The Mender client daemon runs in user space in the currently booted rootfs partition.
Update types supported by Mender
Mender support 2 types of updates;
- Full OS Updates
- Partial Updates
- Application updates
- Updates to client connected peripheral devices
Working
Full OS Updates
During the update process, the Mender client writes the updated image to the rootfs partition that is not running and configures U-Boot flags to boot from the updated rootfs partition. The device is then rebooted. If booting the updated partition fails, the partition that was running is booted instead, ensuring that the device does not get bricked(unable to boot).
If the boot succeeds, Mender sets the updated partition to boot permanently when Mender starts as part of the boot process. As Mender downloads and installs the image, other applications on the device continue to run as normal.
The only time the device has downtime is during the reboot into the updated partition, which typically takes a minute, depending on the device configuration. Persistent data can be stored in the data partition, which is left unchanged during the update process.
For partial updates like application updates, the A/B partition is not necessary. For example, the Mender client can be installed as a .deb package.
Partial System Updates
The Mender client can be extended to support different types of updates, using a framework called Update Modules. Installing the appropriate Update Module will enable support for installing different types of software packages such asl
- Application
- Containers
- Package Managers
- Bootloaders
- Proxy Deployment for peripherals
Update Modules
An Update Module can be tailored to a specific device or environment (e.g. update a proprietary bootloader) or be more general-purpose (e.g. install a set of .deb packages).
When the Mender client downloads new software from the server, it will run the Update Module executable associated with the type of software downloaded. The Update Module is responsible for carrying out the steps needed to install software of this type. An Update Module can also support more advanced functionality such as rolling back a failed update.
Update Modules give full control and customizability over how a software type is installed. An existing Update Module can be used as a starting point and adjusted to fit your exact needs. For example, the Docker Update Module can be adjusted to include the container data itself if you do not want to rely on remote docker registries.
Update Modules are independent executables that can be written in any programming language. Due to their simple nature, Update Modules are typically written in a scripting language like shell or Python.
Deploying new release
After uploading new release, it appears on the server as follows:
Mender Client Inventory
Mender client Inventory shows the client related data on the mender server. Using mender inventory info related to Connected devices can also be shown on the Mender Server dashboard.
Device Inventory keeps Information about the device, on which the mender client is running. Information like;
- Device Type
- Installed Artifact Name
- CPU Model
- Host Name
- IPv6 eth0
This information is then sent to the mender server after an specific interval, “InventoryPollIntervalSeconds”.
Mender Inventory Scripts are located at “/usr/share/mender/inventory/” on the mender client. We can write any custom information that is required on the server, to these files.
For example, we can add version names or numbers:
Mender Artifact
Mender provides mender-artifact utility for signing and verification work by ensuring that updates to devices come only from trusted sources. When an artifact (software update package) is created, it’s signed with a private key, attaching a secure digital signature that verifies the authenticity and integrity of the update.
| $ mender-artifact write module-image -t “DEVICE_NAME” -o “OUTPUT_FILE_*.mender” -T “$ARTIFACT_TYPE” -n “ARTIFACT_NAME” -f “$PAYLOAD_FILE” $ mender-artifact sign artifact-v1.mender -k private.key -o artifact-v1-signed.mender $ mender-artifact validate artifact-v1-signed.mender -k public.key $ mender-artifact validate artifact-v1-signed.mender -k public.pem $ mender-artifact read artifact-v1-signed.mender -k public.pem |
On the device side, the Mender client uses a stored public key to verify this signature before accepting the update. If the signature is valid, the update proceeds, safeguarding against tampering and unauthorized changes. This setup integrates smoothly with CI/CD systems, enabling secure, automated OTA (over-the-air) deployments for IoT and embedded devices.
