A survey of AI systems built for data science proposes an iterative workflow for handling complex analytical work. Its central structure has five stages: perception, planning, execution, verification and repair. The authors present the framework as a way to organize the capabilities needed when an AI system must work through data, tools and possible failures rather than simply produce a one-off answer.
The review calls these systems Data Agents. It defines a Data Agent as an LLM-driven system that performs multi-step data-science work in a persistent data environment, uses computational tools and produces analytical artifacts that can be checked. In that definition, the surrounding data environment and the evidence behind an answer are part of the task.
A workflow built to revisit its own work
The proposed harness is not a straight line from a question to a final answer. Perception comes first, followed by planning and execution, then verification and repair. Because the structure is iterative, the work can return to earlier stages when the result needs checking or correction.
Breaking the work into technical routes
Across the five stages, the survey groups methods into 15 distinct technical routes. Perception is divided into data structure probing, cross-source data alignment and confidence-calibrated state scoring. Planning is organized around tree search, heuristic search and strategy routing.
Execution covers tool capability boundary learning, compositional tool reasoning and feedback-adaptive tool calling. Verification uses process score estimation, rule-based constraint checking and trace-level explainable attribution. Repair includes data state reconstruction, search-guided intervention-based repair and reusable memory skills.
The survey presents these routes as a classification of technical approaches, not as a ranking. It does not quantify how common the routes are or validate the categorization through an experiment.
Two ways to describe the same analytical work
The review also separates two ways of organizing Data Agent research. One taxonomy groups systems by their analytical objectives, while the workflow taxonomy describes the functions used to carry out the work. The distinction separates the question an agent is trying to answer from the steps it uses to answer it.
For vertical applications, the survey describes larger workflows made up of multiple analytical capabilities rather than a single isolated skill. An application can therefore combine several types of analysis while still being understood through the broader workflow structure.
The reliability problems still left open
The review identifies four open reliability problems: semantic calibration, clarification, experience transfer and a shared verification-repair repository. In practical terms, the concerns are about keeping an agent's interpretation aligned with the intended meaning, knowing when to ask instead of assume, carrying useful repair experience across different settings and building common resources for checking and fixing failures.
These are described as open gaps, not measured rates of failure. The paper does not estimate how often any of the four problems occurs or how much each one affects performance.
The authors' broader position is that reliability has to be handled across the workflow, with uncertainty, ambiguity, experience and failure information considered alongside model and tool capabilities.
Why a final answer is not enough
That concern carries into benchmarking. The review says outcome-based metrics can miss whether the system understood the request, selected adequate evidence or grounded its result correctly. A final answer may look successful while hiding problems in the steps that produced it.
The paper recommends broader evaluation that connects final outcomes with evidence quality, intermediate execution, efficiency and failure attribution. It presents this as a direction for benchmarking, not as a standardized framework already validated in the study.
A map of the field, not a performance result
This is a literature-based conceptual synthesis rather than a head-to-head reliability trial. It organizes the field around workflow stages, analytical objectives and data environments, but it does not report comparative efficacy or quantitative reliability gains for the proposed harness.
Readers should therefore treat the survey as a map for designing and evaluating Data Agents, not evidence that any listed route is optimal for a particular task or data environment. The same caution applies to the four reliability problems, whose prevalence is not established.
The source is an arXiv v1 preprint dated 28 September 2026 and reports a companion repository for the survey.
Paper data and sources
Original title: Towards Reliable AI Data Scientists: Data Agents with Workflow Harnesses
Authors: Huachi Zhou, Yujing Zhang, Jiahe Du et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-09-28
DOI: Not available
Original paper · Full text