Tencent Hunyuan, in collaboration with Renmin University of China, open-sourced PlanningBench, a framework for evaluating and training large language model (LLM) planning capabilities. This release aims to help LLMs move from generating text to autonomously executing multi-step actions in complex scenarios.
Tencent Hunyuan, in collaboration with the Gaoling School of Artificial Intelligence at Renmin University of China, has open-sourced PlanningBench, a framework with over 30 real-world planning tasks for evaluating and training large language models (LLMs).
- Evaluation features
- Automated verification, adaptive difficulty control, instance-level verification checklists
- Training support
- Reinforcement learning on verified data
- Task domains
- Scheduling & Timetabling (28.42%), Project & Production Operations (21.90%), Routing & Traveling (17.21%), and 3 more
Planning is a fundamental capability for AI agents, enabling them to coordinate goals, constraints, and resources to achieve complex objectives. PlanningBench's evaluations show that current frontier models still struggle to produce complete solutions under coupled constraints, a challenge also observed in benchmarks like ITBench-AA for agentic tasks.
PlanningBench provides a controllable source of planning data for diagnosing and improving generalizable planning abilities in LLMs. The framework is available on GitHub and Hugging Face. This advances the development of more capable AI agents, a focus also seen in platforms like Arena.ai's Agent Mode.
