Prépublication · 2026
RealEmbedBench: Benchmarking LLMs on Real-World Application-Layer Embedded Software Tasks
Yujun Huang, Yuwei Zhang et al. — Monde
Source : arXiv
Résumé
Large language models (LLMs) perform well on general coding tasks, but their capabilities in application-layer embedded software development remain underexplored. Existing embedded benchmarks primarily focus on hardware-in-the-loop validation in simulated environments. We introduce RealEmbedBench, a benchmark of 5,142 samples covering three application-layer tasks: code generation and completion, state machine modeling and implementation, and cross-OS code translation. We evaluate six representative LLMs and find substantial performance differences across tasks and models. Project context, esp…