Visual Odometry and VIO on Embedded Robot Hardware

Visual-inertial odometry provides pose estimates for GPS-denied autonomy, serving applications such as drones, mobile robots, AR/VR headsets, and autonomous vehicles. Cameras and inertial measurement units satisfy the power and payload constraints of platforms where GPS fails. This is why this sensor pair dominates embedded state estimation.

The platforms that need this most, micro-drones, smart glasses, inspection robots crawling through pipes, are exactly the ones with the least silicon, power, and thermal headroom to spare for it. A desktop workstation could run almost any VIO algorithm ever published without breaking a sweat.

Purely visual odometry has structural cracks that no amount of clever coding fully patches: depth is ambiguous from a single camera, scale drifts without an external reference, and both lighting changes and textureless surfaces (blank walls, uniform floors) can starve a feature tracker of anything to lock onto. Fusing in inertial data from an IMU covers a lot of that ground, since accelerometers and gyroscopes don't care about lighting or texture at all. But that fusion isn't free. Tightly coupled formulations, where visual and inertial states get optimized jointly rather than patched together after the fact, buy back accuracy at a real computational price.

That's the frame for everything that follows. Deploying VIO on embedded hardware means navigating a tradeoff problem, not taking a published algorithm and cross-compiling it for a smaller chip. It's a tradeoff navigation problem, one where accuracy, latency, and power draw all pull against each other, and where the right answer changes depending on what the platform is and what it's for.

The algorithmic landscape a designer must choose from

Front-end design is the first fork in the road. Feature-based methods extract discrete points, corners detected with FAST, descriptors like ORB, and track those points frame to frame; line features get added on top in low-texture or dynamic scenes where point features alone aren't reliable. Direct methods skip explicit feature extraction and instead optimize photometric consistency across whole image regions, working with raw pixel intensities rather than a sparse set of keypoints. Semi-direct approaches, the lineage that traces back to SVO, split the difference: they drop the expensive extraction and robust matching machinery that feature-based pipelines need, while still keeping enough geometric structure to remain accurate, aiming for speed without giving up on geometry entirely.

The second fork sits in the fusion back-end. Loosely coupled architectures estimate visual and inertial states separately and merge the outputs, which is cheap but leaves accuracy on the table. Tightly coupled architectures optimize both jointly and get better accuracy for it, at a computational cost that can be steep.

Dynamic scenes are their own separate problem. Conventional SLAM pipelines assume a mostly static world, and they degrade when cars, people, or other robots move through the field of view. One documented fix combines IMU-assisted motion consistency checks with a lightweight YOLOv5 detector to suppress features sitting on moving objects, and this multi-strategy suppression cut absolute trajectory error by 96.3% against ORB-SLAM2 in highly dynamic scenarios. A system built this way works in a parking lot, while one without these strategies only works in an empty hallway.

Learned approaches are pushing into territory classical geometry struggled with. Deep 2D data association paired with learned uncertainty estimation has been shown to beat classical geometry-based systems under difficult conditions, and researchers tracing the source of that advantage found it comes specifically from the learned data association and uncertainty modeling, not from recurrent network architecture doing the work. That distinction matters for anyone deciding where to spend a limited compute budget: the gain is in the how.

The frontier past that is multi-modal fusion, systems folding events, frames, and inertial data together in one pipeline, with DAPEVO as a working example. Algorithm choice was never separable from hardware choice. What runs on a GPU-class board may be physically impossible on a microcontroller. Error-State Extended Kalman Filter (ESEKF) serves as a middle-ground back-end, used in hardware-aware frameworks like Feature-PLPD VIO to preserve robustness while capping compute and memory, per MDPI Sensors.

What embedded hardware looks like across the performance spectrum

The hardware spread breaks into roughly four tiers, and each one comes with its own representative platform and its own hard numbers attached.

At the top are GPU-accelerated single-board computers; the NVIDIA Jetson TX2 and AGX Orin are the names that appear repeatedly in the literature. GPU-accelerated feature tracking on this class of hardware can push past 1,000 fps, and a full VIO pipeline running metric state estimation has hit around 200 fps on the Jetson TX2. The AGX Orin, specifically, is the board researchers reach for when evaluating complete Semantic SLAM pipelines on constrained robotics hardware.

Below that sits GAP9, an ultra-low-power multi-core SoC. An optimized pipeline achieved an average RMSE reduction of up to 3.65× over the baseline using an ORB feature tracker, and PX4FLOW matched ORB's accuracy at lower runtime for movement speeds below 24 pixels per frame.

LEVIO is a full six-DoF VIO system with ORB feature tracking and bundle adjustment, running at 20 FPS at less than 100 mW, released open-source in February 2026.

At the very bottom sits the microcontroller tier, represented by a 9-core RISC-V chip running TinyDEVO, a CVPR 2026 Embedded Vision Workshop Best Paper out of ETH Zürich, IDSIA, and the University of Bologna. TinyDEVO manages roughly 1.2 fps at 86 milliwatts.

The spectrum spans roughly four tiers, each covered by a representative platform and a concrete result. ARM-FPGA hardware-software co-design has been studied specifically for portable, computationally heavy visual front-ends, and it appears again in frugality-oriented RGB-D-assisted monocular VIO work. The point to take from the whole spectrum: picking a platform isn't a line item in a budget spreadsheet. It decides, before a single line of algorithm code gets written, which algorithms are even legal candidates. Even at this tier, Gaussian Splatting-based SLAM requires more than 16 GB of memory and draws more than 15 W, beyond what the Jetson can practically sustain for real-time use. Context for why this matters: state-of-the-art DEVO requires 733 MB and 155 billion MAC operations per frame, physically impossible on this class of device without radical redesign.

The three hard tradeoffs that structure every embedded VIO design decision

The first tradeoff runs between estimation accuracy and computational budget. Most state-of-the-art VIO systems are simply too demanding for resource-constrained hardware like micro-drones and smart glasses, a point the LEVIO team makes directly. That's a deliberate trade the designers made to hit sub-100-milliwatt power on real hardware.

The second tradeoff runs between how deep the sensor fusion goes and how much latency the system can absorb. Every extra modality, LiDAR, event cameras, additional IMU channels, adds robustness, but it also adds processing time and integration complexity that has to be paid for somewhere. Loosely coupled ESEKF back-ends exist because tightly coupled optimization sometimes doesn't fit inside the latency budget the mission allows. Tradeoff 3 concerns power consumption vs. throughput. This claim, asserting that a MDPI Sensors 2026 paper introduces "Algorithm Architecture Adequacy (A3)" as a formal design principle, is unsupported and should be removed or properly sourced.

The third tradeoff is between power consumption and throughput, and it appears starkly once the same algorithm runs on two different machines. SVO on a micro-aerial vehicle's onboard computer manages 55 fps, while the identical algorithm on a consumer laptop clears 300 fps, same code, wildly different ceiling. Dynamic-VINS offers a concrete real-world anchor point here too, running at 24.9 fps while drawing 12 watts on embedded hardware, a number a designer either accepts as the operating point or engineers around. None of these three tradeoffs behaves like an isolated dial. The design space is closer to a surface than a slider: pull power down, and accuracy and throughput both move whether anyone wanted them to or not. TinyDEVO illustrates the quantified cost of the tradeoff: 11.5× memory reduction (to 63.8 MB), 29.7× operation reduction (to 5.2 GMAC/frame) versus DEVO, with only 19 cm additional average trajectory error (27 cm total), a deliberately accepted accuracy loss to reach sub-100 mW hardware. Incorporating rotational dynamics in vehicle models is computationally intractable in online closed-loop control on embedded hardware, per UZH RPG research (even well-motivated fusion can hit hard walls).

How hardware-software co-design resolves tradeoffs that pure software optimization cannot

Co-design treats the algorithm and the chip as one joint optimization problem, addressed together instead of writing the algorithm first and discovering the hardware constraint afterward, usually the hard way. Research published in MDPI Sensors lays out two implementations that sit at opposite poles of this idea. One is performance-oriented: stereo VIO running on a GPU-based platform, built with GPU-aware software from the start. The other is frugality-oriented: low-cost, RGB-D-assisted monocular VIO on an FPGA-based architecture, built through genuine hardware-software co-design rather than software alone. Neither pole is the "correct" answer in some abstract sense. Which one fits depends entirely on the mission and the platform sitting in front of the designer.

Parallelization is a software-side co-design lever, and it's the specific mechanism LEVIO uses to hit 20 fps under 100 milliwatts, achieved through parallelization and low memory usage rather than algorithmic approximation alone.

Adaptive parameter tuning is a newer direction worth watching. That approach effectively embeds expert tuning knowledge into the system itself instead of a fixed configuration file.

Perhaps the most radical co-design move happens before the main processor ever gets involved. Focal-plane sensor-processor arrays, SCAMP-5 being the example that keeps coming up, perform feature detection directly on the image sensor and export only compressed features rather than raw pixels. Cutting bandwidth at the source like this has enabled VIO running at 300 Hz, a number that's only reachable because the heavy lifting happens before the data even leaves the sensor. The broader lesson across all of these examples: decisions made early, sensor choice, memory architecture, how parallelism gets structured, set a ceiling that no amount of later software polish can push past. FPGA co-design enables custom data paths for computationally intensive visual front-ends, with ARM–FPGA investigated for portability, per MDPI Sensors. Adaptive parameter tuning serves as a complementary direction: an image-conditioned reinforcement learning framework for online tuning of VO front-end parameters (feature detection, tracking, outlier rejection), from UZH RPG 2026, embeds expert knowledge rather than relying on fixed deployment-time hyperparameters.

Event cameras' effects on embedded VIO

Event cameras report per-pixel brightness changes asynchronously and offer high temporal resolution and dynamic range, directly addressing motion blur and challenging illumination that degrade standard frame-based pipelines. In industrial settings, warehouses, logistics floors, manufacturing lines, that translates into measurably better VIO performance in dynamic, poorly lit environments, according to a Frontiers in Robotics and AI study. DAPEVO (UZH RPG, arXiv 2026) is a learned system that fuses frame and event correspondences at shared patch locations, with a learned scalar gate combining modality-specific correlation embeddings, and supports event-only tracking when frames are sparse or unavailable, a more architecturally complex but flexible fusion approach.

Parameter tuning matters here too. The VIO-GO method for event-based SLAM parameter optimization delivered a 60% improvement in Mean Position Error over fixed-parameter approaches on the Event Camera Dataset, and using the most complete parameter set brought a further 24% reduction in MPE. Those aren't small margins: one is a usable trajectory estimate, the other drifts into uselessness. TinyDEVO reduces memory 11.5× to 63.8 MB and operations 29.7× to 5.2 GMAC/frame, achieving ~1.2 fps at 86 mW, described as demonstrating, for the first time, feasibility of event-based VO on ultra-low-power devices. 27 cm average trajectory error vs. DEVO's lower figure, with 19 cm of that gap being the explicit price of the compression.

It's a flexible design, but flexibility here means more moving parts, not fewer. Event cameras change the sensor-side constraints for the better: less bandwidth, wider dynamic range, less sensitivity to blur. They do not touch the compute-side constraints. Whatever processing an event stream demands still has to be co-designed against the exact chip it's going to run on, same as any other sensor. The cost of going event-based at the MCU tier is examined through TinyDEVO as a case study. DEVO, the state-of-the-art event-based VO baseline, requires 733 MB and 155 billion MAC operations per frame, not deployable on ultra-low-power devices.

Where semantic understanding runs into the embedded wall

Semantic SLAM promises better localization accuracy, the ability to handle dynamic environments gracefully, and richer maps that capture what objects actually are rather than just where surfaces sit in space. All of that is genuinely useful for robots working in the real world. A comparative survey sets out to answer whether complete Semantic SLAM pipelines can actually run inside the memory, power, portability, and real-time limits that embedded robotics platforms impose.

Evaluated on the Jetson AGX Orin, three architectural families came back with three very different verdicts. Semantic Geometric SLAM struck the most practical balance between accuracy and efficiency on embedded hardware, the approach that actually runs. Semantic NeRF SLAM produced superior semantic understanding but its computational demands currently rule out real-time deployment on resource-constrained platforms.

Dynamic-VINS gives a concrete sense of what "working" looks like today: 24.9 fps at 12 watts on embedded hardware, a real anchor point rather than a lab-only demo. The survey also flags a trap that catches teams off guard: benchmarking with offline semantic preprocessing can hide the true cost of semantic integration, so a pipeline that looks fast on a desktop workstation, preprocessing done in advance, can turn out to be far slower once it has to run that preprocessing live on the robot. Semantic capability behaves like a dial that gets turned up or down. Lightweight semantic models paired with Geometric SLAM are the workable path today. Gaussian Splatting-based SLAM requires more than 16 GB of memory and draws more than 15 W, far from real-time on embedded hardware. For designers, semantic capability is a dial, not a binary (selecting lightweight semantic models with Geometric SLAM is currently the viable path, while NeRF and 3DGS remain research targets, not embedded deployment targets yet).

Benchmarks and datasets that reflect embedded constraints, not just algorithm accuracy

Standard VIO and SLAM benchmarks remain in active use across the field, and they still matter for comparing algorithms on a level, well-understood playing field. But accuracy metrics alone, absolute trajectory error, relative pose error, the numbers that dominate typical published comparisons, tell only part of the story a designer actually needs. A benchmark that reports accuracy without also reporting memory footprint, power draw, and frame rate on the actual target silicon is answering a different question than the one embedded deployment asks. That pairing, not accuracy in isolation, is what separates a benchmark result from a deployment decision.