A robot-planning system that converts equipment manuals into machine-readable instructions rejected 26.3% of the 357 planning runs that entered its planning loop, keeping unresolved candidates from reaching physical action. In the same evaluation, the system reported 43.2% success on the hardest class of tasks and a 2.7% violation rate, the strongest Level-3 success and lowest listed violation rate among the compared methods.
The results come from an arXiv preprint dated 28 August 2026. They suggest that formal checks and manual-derived knowledge can help with complex robot operation under the tested conditions, but they do not establish readiness for unattended industrial use.
Turning manuals into a plan
The system, called MaCoPlanner, is designed for a problem that is easy to underestimate: industrial manuals are written for people, while a robot needs explicit descriptions of devices, actions, states and safety rules. The pipeline compiles manuals from different equipment domains into a typed intermediate representation, a structured form that software can use. It then retrieves evidence relevant to the task and current state, checks a proposed plan symbolically before actuation, and either repairs or rejects the plan before execution.
The evaluation covered four manual domains: a motor-drive controller simulator, a generator control system, a laser controller and an engine controller. It included a 400-task retrieval benchmark and a 480-task planning benchmark, with 80 Level-1 tasks, 200 Level-2 tasks and 200 Level-3 tasks. The counts describe separate evaluation sets, so they are not meant to be added together.
The translation was accurate, with room for error
An audit of 350 gold units reported 97.14% overall completeness and 97.32% overall semantic consistency in the manual-to-structured-knowledge conversion. Completeness for safety clauses was reported at 100%. Those figures are audit results, not estimates with confidence intervals.
Retrieval results were more mixed in emphasis. Typed indexing delivered the larger improvement in coverage-oriented measures, while evidence gating more clearly improved ranking at nDCG@8 and nDCG@12. The supplied analysis does not provide the exact plotted retrieval values or uncertainty estimates.
The formal safety layer was also mostly, but not perfectly, faithful to the source material. The audit counted 75 semantically correct rules out of 81 temporal-logic rules, alongside approximate rules, syntax or atomic-proposition problems, mismatches with the finite-state safety model and corrected rules. For the finite-state model, 45 of 48 transitions or guards were semantically correct, with additional syntax, mismatch and correction categories.
The advantage appeared as tasks became harder
On the shared planning benchmark, MaCoPlanner reported 93.5% success on Level 1, 84.4% on Level 2 and 43.2% on Level 3, with a 2.7% violation rate. Its Level-2 and Level-3 success rates were the highest among the listed methods, and its violation rate was the lowest. It did not have the highest Level-1 success rate: ISR-LLM reported 93.6%, compared with MaCoPlanner's 93.5%.
The comparison included Raw-Manual, Prompt-Safety, ISR-LLM, LLM 3 and SafePlan. Their reported Level-3 success rates were 25.9%, 33.8%, 35.7%, 37.1% and 39.6%, respectively, compared with 43.2% for MaCoPlanner. The study reports these as benchmark outcomes, not as proof that one system caused the differences, because the methods varied in evidence representation, verification, repair and memory.
A component analysis pointed in the same direction. Removing the temporal-logic and finite-state checks, targeted repair or device memory was associated with larger performance losses on Levels 2 and 3 than on Level 1. A slight Level-1 increase when the finite-state check was removed was described as non-monotonic, rather than as evidence that the component was generally harmful.
Physical tests exposed a sharper weakness
The physical demonstrations used a no-load controller-panel simulator rather than an industrial plant. Across five prescribed viewpoints, ordinary button and knob operations had high repeated-operation success, but the panel-mounted emergency stop was the weakest operation, with 80.0% reported success. The authors judged that result insufficient for unattended reliance.
In 25 long-horizon physical trials, 23 were completed and 19 completed on the first attempt. The system averaged 0.36 replans and 0.16 safety triggers per trial. Two trials remained incomplete, and the fixed panel, calibration, viewpoint and illumination limit how far the result can be generalized.
The study's safety guarantees are conditional on several links in the chain working correctly: the manuals must be compiled faithfully, the robot must estimate symbolic state correctly, the model abstraction must be adequate and the planned actions must correspond correctly to control commands. Level-3 planning remained difficult, and unresolved plans were rejected rather than universally repaired.
A useful test, not a deployment verdict
The authors interpret the findings as evidence that compiled manual knowledge and proactive verification are useful for complex panel planning, with targeted repair and device memory becoming more important as tasks become more tightly coupled. A backbone analysis reported the highest point estimates for GPT-5, close performance for GPT-4o and larger deficits for lower-capacity Gemma 3 planners as complexity rose.
The evaluation was limited to the reported retrieval and planning benchmarks and physical interaction on a no-load controller-panel simulator with a separate external safety oracle. The paper does not test real industrial loads or plant-level safety architectures, and it offers no general industrial cycle-time guarantee. The physical sample was small, with no formal power calculation reported.
The authors state that the code and data are publicly available in the MaCoPlanner repository. The work is an arXiv preprint, not a reported peer-reviewed journal publication, and its results are best read as a feasibility evaluation for manual-grounded robot planning rather than evidence of production-ready autonomy.
Paper data and sources
Original title: MaCoPlanner: LLM-Assisted Manual-Compiled Task Planning with Proactive Safety Verification for Robotic Industrial Panel Operation
Authors: Guipeng Xin, Jiahe Xua, Mohammad Deghat et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-28
DOI: Not available
Original paper · Full text