Intelligence for serving users in real-world scenarios
Serving users in real-world scenarios requires models to learn from context, identify relevant signals in noisy behavior histories, integrate evidence over time, and adapt to changing preferences.
Vela explores user behavior as supervision: using observed behavior to build tasks that teach LLMs to understand preferences, intent, and likely actions. The benefits extend to general capabilities and subsequent reinforcement learning.
Based on anonymized histories from 100,000 users, the framework constructs 100,000 candidate tasks across 91 templates and retains approximately 75,000 training examples.
A stronger 35B model
Fine-tuning Qwen3.6-35B-A3B on Vela-Instruct produces Vela-35B-A3B. It improves over the base model on all ten evaluated benchmarks.
On VelaBench, the model reaches 63.0, compared with 51.5 for the base model and 62.3 for Qwen3.8-2.4T-A95B. This comparison concerns the user-understanding benchmark.
Full benchmark comparison
From behavior to supervision
Vela organizes six kinds of behavior into evidence-grounded tasks: views, searches, purchases, posts, follows, and dislikes. Predictive questions use observed future behavior as ground truth. Open and Attribution questions use task-specific rubrics.
Ground the task
Anonymize the records and organize timestamped histories into visible evidence and hidden future behavior.
Make supervision explicit
Define what the model sees, what it should answer, and how the answer will be evaluated.
Build the training set
Filter task difficulty and retain the highest-scoring trajectory when it exceeds the base model’s mean response score.
One history. Different learning tasks.
A lightweight backpack search, a daypack comparison, and negative feedback on expedition gear provide evidence about a user’s needs. Select a question type to see how that history becomes a task.
Visible behavior history
Illustrative records before the cutoff
“Lightweight backpack for a day hike”
A comparison of 20 L daypacks
“Planning a short day hike.”
Marks a 65 L expedition-backpack recommendation as “not interested”
See evaluation signal
Each tab illustrates an independent task. An Attribution question and a prediction about the same outcome are not combined.
More behavior data, stronger LLM capabilities
Increasing Vela-Instruct from 4.7K to 75K examples raises VelaBench scores from 59.2 to 63.0 and the independent user-related average from 43.5 to 47.5. General benchmarks remain above the base model at every tested scale.
A better foundation for RL
With the same reinforcement-learning data and settings, first training on Vela-Instruct leads to better final results than applying RL directly to the base model.
After RL, Vela-based models reach 66.2 on VelaBench and 37.6 on ProfileSummary, compared with 58.5 and 29.9. Both gains are 7.7 points.
Explore Vela
The repository includes 74 selected task templates and the paper’s Composer and Scorer system prompts in English and Chinese. Browse the evidence boundaries, question instructions, and scoring criteria for each task.
Questions about the project?
yang-alan.xiao@connect.polyu.hk
Citation
@misc{xiao2026velauserbehaviorsupervision,
title = {Vela: User Behavior as Supervision Sparks LLM Intelligence},
author = {Yang Xiao and Yusong Sun and Haoyi Wu and Wenyang Hui and Weiqiang Hang and Luqing Wang and Zejian Wang and Zirui Tu and Jingsheng Mao and Shaobo Hu and Yang Luo and Jiale Guo and Jin Zhang and Yudan Liu and Zhaokai Luo and Ren Huan and Yao Hu and Chengyue Jiang and Wenjie Li and Chuan Mu},
year = {2026},
eprint = {2610.14709},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
url = {https://arxiv.org/abs/2610.14709}
}