I Ran Five Small Multimodal Models on a Jetson. The Fastest One Was Not the Best Baseline.

I have been building WearEdge Pro, a wearable industrial edge AI runtime. Think of a frontline operator wearing a smart-glasses device, capturing a first-person image of a machine, and getting back a structured action card from a local Jetson box.

The key phrase is "structured action card." This is not a chat demo. In a factory setting, an answer needs an audit trail, a mode boundary, a human-confirmation gate, and a way to hand off to maintenance, quality, EHS, or work-instruction workflows.

I recently tested five compact multimodal models on the same Jetson path:

• Gemma 4 E2B
• Qwen2.5-VL-3B
• SmolVLM2-2.2B
• InternVL3-2B
• Qwen2.5-Omni-3B

The goal was not to crown a universal benchmark champion. I wanted to know which model was the best current baseline for an industrial edge Agent runtime.

The Harness

Every model was exposed through a local OpenAI-compatible llama.cpp endpoint on the Jetson. Each model got the same five prompts and images:

• maintenance
• quality inspection
• changeover
• work instruction
• hazard review

The main run used 560 image tokens, which matches the current WearEdge gateway budget. Qwen2.5-VL also got a 1024-image-token pass because grounding can improve with more visual tokens.

The Results

SmolVLM2 was the speed star. But the answers were often too generic for real operator guidance. In changeover and work-instruction tasks, it returned fields that looked more like placeholders than grounded industrial guidance.

Qwen2.5-VL was the most impressive challenger. It nailed a changeover OCR task with LABELER-FL1 and SKU-C500, where Gemma had a machine-label typo. It also produced useful IQC defect scores. If I were building a pure OCR or visual inspection assistant, I would take Qwen very seriously.

InternVL3 reminded me that token speed is not the whole story. At 2048 context it failed three of five tasks with context errors. At 4096 context it finished, but the latency was high and one raw IQC answer had unsafe release-style wording.

Qwen2.5-Omni ran cleanly, but its strongest value is probably a future audio/video workflow rather than this current image+text industrial baseline.

Why Gemma Still Won

Gemma 4 E2B did not win every micro-test. It stayed the baseline because it fit the product runtime:

• local Jetson deployment
• structured multimodal prompts
• long-context workflow design
• function-calling-oriented architecture
• deterministic guards
• human confirmation
• action cards
• audit logs

In an industrial setting, "fast and fluent" is not enough. The model has to behave inside a system that can say: this came from this image, this route, this required field, this action boundary, and this audit record.

That is why Gemma remained the WearEdge baseline, while Qwen2.5-VL became the serious A/B challenger for OCR-heavy branches.

Lesson Learned

Edge AI model selection is not just a leaderboard exercise. The right question is:

Can this model run locally, understand the evidence, obey the workflow boundary, and produce an action that the system can audit?

For WearEdge Pro today, the answer is Gemma 4 E2B as the baseline, Qwen2.5-VL as the next challenger, and a clear path to keep testing without pretending every benchmark cell means the same thing.

Public artifact link: Benchmark results and public discussion: https://www.hackster.io/ryanon2008/wearedge-pro-jetson-edge-ai-agent-50ec35