The top-scoring version of CommerceVibe in the paper's automatic benchmark was designed to produce e-commerce creatives as executable code and used both rule feedback and feedback from a vision-language model, or VLM. It scored 94.0 out of 100, compared with 87.3 after fine-tuning and 70.1 for the base Qwen3.5-9B policy.
In a single decoding pass, CommerceVibe writes a complete HTML/CSS document, directly references supplied product images and represents text as native HTML elements instead of synthesizing lettering into a raster image. The generated creative is therefore expressed as a web document in code rather than only as a finished bitmap.
The approach was built in two stages. Qwen3.5-9B was first fine-tuned on 28,568 quality-controlled examples. After quality filtering, the SFT corpus contained 3,174 validation examples. The SFT policy then underwent reinforcement-learning optimization under the dual-feedback setup.
The three reinforcement-learning variants shared the same SFT initialization, training data, sampling and optimization settings. Their stated difference was whether optimization used rule feedback, VLM preference feedback or both.
The benchmark and its scoring
The benchmark contained 1,300 cases, including 1,140 Single-image cases and 160 Multi-image cases. It was separate from the SFT and RL data at the product level, and no product was repeated across cases. The split created a product-level boundary between development data and evaluation cases.
Automatic evaluation put the rule score and the preference score on separate 0-to-50 scales and added them for an overall score from 0 to 100. On that scale, the base policy reached 70.1, SFT reached 87.3 and Rule+Preference RL reached 94.0.
Rule+Preference RL also ranked above the external models reported in the comparison. GPT-5.5 was the strongest external model, followed by Claude Opus 4.8 and Gemini 3.5 Flash. The reported margin differed by image count: against the stronger single-feedback variant, Rule+Preference RL gained 4.8 points on Multi-image cases and 1.2 points on Single-image cases.
Human review
Automatic scoring was not the only assessment. Five e-commerce design experts, each with one to five years of professional experience, carried out an independent blind benchmark evaluation. Rule+Preference RL received an expert score of 90.0, the highest reported, with a 95% case-bootstrap confidence interval of 88.5 to 91.3.
The reported inter-rater agreement statistic was 0.811. Automatic and expert scores showed a positive but imperfect relationship: the Spearman correlation, a rank-based measure of how closely two score orderings track each other, was 0.612, with a 95% confidence interval from 0.562 to 0.657. Across methods, the median method-specific correlation was 0.385, with a range of 0.075 to 0.492.
The paper also tested what happened when one family of structural rules was removed. The largest listed within-family change was in Layout, at minus 4.47 points after Layout rules were removed. The corresponding changes were minus 2.15 in Text after removing Text rules and minus 0.50 in Product after removing Product rules.
The limits of the result
These figures describe performance on a model benchmark and an expert panel, not customer or business outcomes. The benchmark contained 1,300 cases, including 160 Multi-image cases, and the system generated its HTML/CSS document in a single decoding pass. The supplied results therefore do not establish increased sales, click-through, conversion or customer satisfaction, nor do they quantify editing or reuse costs in real workflows.
The paper is an arXiv version 1 preprint dated 28 Aug 2026. For now, its clearest conclusion is narrow: on the reported benchmark, the dual-feedback variant ranked highest in automatic scoring and received the highest reported expert score. Whether that benchmark advantage carries into broader product sets, iterative workflows or commercial production remains an open question.
Paper data and sources
Original title: CommerceVibe: Learning to Design E-Commerce Creatives as Executable Visual Code via Dual-Feedback Reinforcement Learning
Authors: Yajiao Xu, Jin Zhang, Jiangbo Ai et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-28
DOI: Not available
Original paper · Full text