-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathpapers.json
More file actions
666 lines (666 loc) · 60.8 KB
/
Copy pathpapers.json
File metadata and controls
666 lines (666 loc) · 60.8 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
[
{
"Bibtex": "@inproceedings{yang2026towards,\n title={Towards Self-Evolving Agents: Enabling Autonomy through Interactive Experience Refinement},\n author={Yang, Cheng and Yang, Xuemeng and Wen, Licheng and Fu, Daocheng and Mei, Jianbiao and Wu, Rong and Cai, Pinlong and Shen, Yufan and Deng, Nianchen and Xu, Jia and Shi, Botian and Qiao, Yu and Li, Haifeng},\n booktitle={Findings of the Association for Computational Linguistics: ACL 2026},\n year={2026},\n publisher={Association for Computational Linguistics},\n}",
"Github仓库链接": {
"link": "https://github.com/KnowledgeXLab/MUSE",
"text": "https://github.com/KnowledgeXLab/MUSE"
},
"arXiv主页": {
"link": "https://arxiv.org/abs/2510.08002",
"text": "https://arxiv.org/abs/2510.08002"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Cheng Yang 1,2,*;\nXuemeng Yang 2,* ;\nLicheng Wen 2,4,6,*; \nDaocheng Fu 3,2;\nJianbiao Mei 5,2;\nRong Wu 5,2;\nPinlong Cai 2;\nYufan Shen 2;\nNianchen Deng 2; \nJia Xu 2;\nBotian Shi 2,^ ;\nYu Qiao 2;\nHaifeng Li 1,^",
"刊印链接": {
"link": "https://openreview.net/forum?id=nu2tOJgxPi",
"text": "https://openreview.net/forum?id=nu2tOJgxPi"
},
"单位信息(每个单位一行,分号换行)": "1 Central South University;\n2 Shanghai Artificial Intelligence Laboratory; \n3 Fudan University;\n4 Shanghai Innovation Institute;\n5 Zhejiang University; \n6 Shanghai Jiao Tong University",
"录用类型": [
"Findings"
],
"摘要": "Large Language Models often struggle with complex, multi-step operational tasks because they remain static during inference and cannot learn from past experience. To address this, we propose MUSE, a framework that enables iterative self-improvement through a hierarchical Memory Module. MUSE organizes cross-domain insights to facilitate the orchestration of long-horizon workflows. The core of our approach is an autonomous post-execution critique mechanism: after completing each sub-task, the system analyzes its operational logs and distills raw execution data into structured, reusable knowledge. This allows the agent to evolve dynamically rather than relying on fixed parameters. Evaluated on the rigorous TAC productivity benchmark, MUSE achieves new state-of-the-art results, significantly outperforming previous methods using only the streamlined Gemini-2.5 Flash model. Our analysis demonstrates that MUSE’s performance scales with the accumulation of insights and exhibits strong cross-task transferability, marking a key step toward autonomous systems capable of lifelong learning in professional environments. Demo videos can be found in our supplementary materials.",
"是否为团队主导工作": true,
"期刊/会议": "ACL-2026",
"记录创建日期": 1775750400000,
"论文pdf": [
{
"file_token": "Td5lbmdeMoAbhbxOu4TcJkhInRh",
"name": "2510.08002v1.pdf",
"size": 4475156,
"tmp_url": "https://open.feishu.cn/open-apis/drive/v1/medias/batch_get_tmp_download_url?file_tokens=Td5lbmdeMoAbhbxOu4TcJkhInRh",
"type": "application/pdf",
"url": "https://open.feishu.cn/open-apis/drive/v1/medias/Td5lbmdeMoAbhbxOu4TcJkhInRh/download"
}
],
"论文发表日期": 1772380800000,
"论文标题": "Towards Self-Evolving Agents: Enabling Autonomy through Interactive Experience Refinement",
"论文状态": "已录用",
"责任人": [
{
"email": "yangxuemeng@pjlab.org.cn",
"en_name": "杨雪梦",
"id": "ou_4894b11e0a99edbf65cc82679f993880",
"name": "杨雪梦"
}
],
"_thumbnail": "/assets/images/papers/2510.08002.png"
},
{
"Bibtex": "@inproceedings{fu2026agentfirstday,\n title = {The Agent's First Day: Benchmarking Learning, Exploration, and Scheduling in the Workplace Scenarios},\n author = {Fu*, Daocheng and Mei*, Jianbiao and Wu*, Rong and Yang, Xuemeng and Xu, Jia and Wang, Ding and Cai, Pinlong and Liu, Yong and Wen, Licheng and Shi, Botian},\n journal = {Findings of the Association for Computational Linguistics (ACL)},\n year = {2026},\n}",
"Github仓库链接": {
"link": "https://github.com/KnowledgeXLab/EvoEnv",
"text": "https://github.com/KnowledgeXLab/EvoEnv"
},
"arXiv主页": {
"link": "https://arxiv.org/pdf/2601.08173",
"text": "https://arxiv.org/pdf/2601.08173"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Daocheng Fu1,2,*, Jianbiao Mei3,2,* , Rong Wu3,2,* , Xuemeng Yang2,*, Jia Xu2 , Ding Wang2 , Pinlong Cai 2 , Yong Liu3,^ , Licheng Wen2,4,5,^ , Botian Shi2,^",
"单位信息(每个单位一行,分号换行)": "1 Fudan University, 2 Shanghai AI Laboratory, 3 Zhejiang University, 4 Shanghai Innovation Institute, 5 Shanghai Jiao Tong University",
"录用类型": [
"Findings"
],
"摘要": "The rapid evolution of Multi-modal Large Language Models (MLLMs) has advanced workflow automation; however, existing research mainly targets performance upper bounds in static environments, overlooking robustness for stochastic real-world deployment. We identify three key challenges: dynamic task scheduling, active exploration under uncertainty, and continuous learning from experience. To bridge this gap, we introduce a dynamic evaluation environment that simulates a \"trainee\" agent continuously exploring a novel setting. Unlike traditional benchmarks, it evaluates agents along three dimensions: (1) context-aware scheduling for streaming tasks with varying priorities; (2) prudent information acquisition to reduce hallucination via active exploration; and (3) continuous evolution by distilling generalized strategies from rule-based, dynamically generated tasks. Experiments show that cutting-edge agents have significant deficiencies in dynamic environments, especially in active exploration and continual learning. Our work establishes a framework for assessing agent reliability, shifting evaluation from static tests to realistic, production-oriented scenarios.",
"是否为团队主导工作": true,
"期刊/会议": "ACL-2026",
"记录创建日期": 1776182400000,
"论文发表日期": 1768233600000,
"论文标题": "The Agent's First Day: Benchmarking Learning, Exploration, and Scheduling in the Workplace Scenarios",
"论文状态": "已录用",
"责任人": [
{
"email": "wenlicheng@pjlab.org.cn",
"en_name": "温力成",
"id": "ou_821979fbe056e3db6b89c95261e8c1c0",
"name": "温力成"
}
],
"_thumbnail": "/assets/images/papers/2601.08173.png"
},
{
"Bibtex": "@article{fu2025researcher,\n title = {RE-Searcher: Robust Agentic Search with Goal-oriented Planning and Self-reflection},\n author = {Fu*, Daocheng and Mei*, Jianbiao and Wen*, Licheng and Yang*, Xuemeng and Yang, Cheng and Wu, Rong and Hu, Tao and Li, Siqi and Shen, Yufan and Cai, Xinyu and Cai, Pinlong and Shi, Botian and Liu, Yong and Qiao, Yu},\n journal = {arXiv preprint arXiv:2509.26048},\n year = {2025},\n}",
"arXiv主页": {
"link": "https://arxiv.org/abs/2509.26048",
"text": "https://arxiv.org/abs/2509.26048"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Daocheng Fu*, Jianbiao Mei*, Licheng Wen*, and Xuemeng Yang*, Cheng Yang, Rong Wu, Tao Hu, Siqi Li, Yufan Shen, Xinyu Cai, Pinlong Cai, Botian Shi^, Yong Liu, Yu Qiao",
"单位信息(每个单位一行,分号换行)": "1 Fudan University, 2 Shanghai Artificial Intelligence Laboratory, 3 Zhejiang University\n4 Shanghai Innovation Institute, 5 Shanghai Jiao Tong University, 6 Central South University",
"录用类型": [
"Preprint"
],
"摘要": "Large language models (LLMs) excel at knowledge-intensive question answering and reasoning, yet their real-world deployment remains constrained by knowledge cutoff, hallucination, and limited interaction modalities. Augmenting LLMs with external search tools helps alleviate these issues, but it also exposes agents to a complex search environment in which small, plausible variations in query formulation can steer reasoning into unproductive trajectories and amplify errors. We present a systematic analysis that quantifies how environmental complexity induces fragile search behaviors and, in turn, degrades overall performance. To address this challenge, we propose a simple yet effective approach to instantiate a search agent, RE-Searcher. During search, RE-Searcher explicitly articulates a concrete search goal and subsequently reflects on whether the retrieved evidence satisfies that goal. This combination of goal-oriented planning and self-reflection enables RE-Searcher to resist spurious cues in complex search environments and perform robust search. Extensive experiments show that our method improves search accuracy and achieves state-of-the-art results. Perturbation studies further demonstrate substantial resilience to noisy or misleading external signals, mitigating the fragility of the search process. We believe these findings offer practical guidance for integrating LLM-powered agents into more complex interactive environments and enabling more autonomous decision-making.",
"是否为团队主导工作": true,
"期刊/会议": "Under Submission",
"记录创建日期": 1776182400000,
"论文发表日期": 1759161600000,
"论文标题": "RE-Searcher: Robust Agentic Search with Goal-oriented Planning and Self-reflection",
"论文状态": "已挂arXiv",
"责任人": [
{
"email": "fudaocheng@pjlab.org.cn",
"en_name": "付道成",
"id": "ou_2d7056d82db1ecc541cd2312b6de69ff",
"name": "付道成"
}
],
"_thumbnail": "/assets/images/papers/2509.26048.png"
},
{
"Bibtex": "@misc{mei2025o2searchersearchingbasedagentmodel,\n title={O$^2$-Searcher: A Searching-based Agent Model for Open-Domain Open-Ended Question Answering}, \n author={Jianbiao Mei and Tao Hu and Daocheng Fu and Licheng Wen and Xuemeng Yang and Rong Wu and Pinlong Cai and Xinyu Cai and Xing Gao and Yu Yang and Chengjun Xie and Botian Shi and Yong Liu and Yu Qiao},\n year={2025},\n eprint={2505.16582},\n archivePrefix={arXiv},\n primaryClass={cs.CL},\n url={https://arxiv.org/abs/2505.16582}, \n}",
"Github仓库链接": {
"link": "https://github.com/KnowledgeXLab/O2-Searcher",
"text": "https://github.com/KnowledgeXLab/O2-Searcher"
},
"arXiv主页": {
"link": "https://arxiv.org/abs/2505.16582v2",
"text": "https://arxiv.org/abs/2505.16582v2"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Jianbiao Mei*, Tao Hu*, Daocheng Fu*, Licheng Wen, Xuemeng Yang, Rong Wu, Pinlong Cai, Xinyu Cai, Xing Gao, Yu Yang, Chengjun Xie, Botian Shi^, Yong Liu^, Yu Qiao",
"单位信息(每个单位一行,分号换行)": "1 Zhejiang University, 2 Shanghai Artificial Intelligence Laboratory, 3 University of Science and Technology of China, 4 Fudan University, 5 State Key Laboratory of Industrial Control Technology",
"录用类型": [
"Journal Article(Regular Issue)"
],
"摘要": "Large Language Models (LLMs), despite their advancements, are fundamentally limited by their static parametric knowledge, hindering performance on tasks requiring open-domain up-to-date information. While enabling LLMs to interact with external knowledge environments is a promising solution, current efforts primarily address closed-end problems. Open-ended questions, which characterized by lacking a standard answer or providing non-unique and diverse answers, remain underexplored. To bridge this gap, we present O-Searcher, a novel search agent leveraging reinforcement learning to effectively tackle both open-ended and closed-ended questions in the open domain. O-Searcher leverages an efficient, locally simulated search environment for dynamic knowledge acquisition, effectively decoupling the external world knowledge from model's sophisticated reasoning processes. It employs a unified training mechanism with meticulously designed reward functions, enabling the agent to identify problem types and adapt different answer generation strategies. Furthermore, to evaluate performance on complex open-ended tasks, we construct O-QA, a high-quality benchmark featuring 300 manually curated, multi-domain open-ended questions with associated web page caches. Extensive experiments show that O-Searcher, using only a 3B model, significantly surpasses leading LLM agents on O-QA. It also achieves SOTA results on various closed-ended QA benchmarks against similarly-sized models, while performing on par with much larger ones.",
"是否为团队主导工作": true,
"期刊/会议": "TMLR",
"记录创建日期": 1776268800000,
"论文发表日期": 1747843200000,
"论文标题": "O2-Searcher: A Searching-based Agent Model for Open-Domain Open-Ended Question Answering",
"论文状态": "已录用",
"责任人": [
{
"email": "meijianbiao@pjlab.org.cn",
"en_name": "梅剑标",
"id": "ou_74b580b9940052a733b858083c602b33",
"name": "梅剑标"
}
],
"_thumbnail": "/assets/images/papers/2505.16582.png"
},
{
"Bibtex": "@misc{deng2025internspatial,\n title={InternSpatial: A Comprehensive Dataset for Spatial Reasoning in Vision-Language Models}, \n author={Nianchen Deng and Lixin Gu and Shenglong Ye and Yinan He and Zhe Chen and Songze Li and Haomin Wang and Xingguang Wei and Tianshuo Yang and Min Dou and Tong He and Wenqi Shao and Kaipeng Zhang and Yi Wang and Botian Shi and Yanting Zhang and Jifeng Dai and Yu Qiao and Hongjie Zhang and Wenhai Wang},\n year={2025},\n eprint={2506.18385},\n archivePrefix={arXiv},\n primaryClass={cs.CV},\n url={https://arxiv.org/abs/2506.18385}, \n}",
"Github仓库链接": {
"link": "https://github.com/dengnianchen/intern-spatial",
"text": "https://github.com/dengnianchen/intern-spatial"
},
"arXiv主页": {
"link": "https://arxiv.org/abs/2506.18385",
"text": "https://arxiv.org/abs/2506.18385"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Nianchen Deng 1,*;\nLixin Gu 1,*;\nShenglong Ye 1,*;\nYinan He 1;\nZhe Chen 1,2;\nSongze Li 1,3;\nHaomin Wang 1,3;\nJinhui Yin 1,2;\nQi Wei 1,2;\nTianshuo Yang 1;\nMin Dou 1;\nTong He 1;\nWenqi Shao 1;\nKaipeng Zhang 1;\nYi Wang 1;\nBotian Shi 1;\nYanting Zhang 4;\nJifeng Dai 1,5;\nYu Qiao 1;\nWenhai Wang 1,6,^;\nHongjie Zhang 1,^;",
"单位信息(每个单位一行,分号换行)": "1. Shanghai AI Laboratory;\n2. Nanjing University;\n3. Shanghai Jiao Tong University;\n4. Donghua University;\n5. Tsinghua University;\n6. The Chinese University of Hong Kong;",
"录用类型": [
"Poster"
],
"摘要": "Recent benchmarks and datasets have been proposed to improve spatial reasoning in vision-language models (VLMs), yet existing open resources remain constrained by limited scale, narrow visual diversity, and restricted instruction expressiveness. To address these gaps, we present InternSpatial---the largest open-source dataset for spatial reasoning in VLMs---alongside InternSpatial-Bench, a comprehensive evaluation benchmark designed to assess spatial understanding across diverse instruction formats. InternSpatial contains 12 million question-answer(QA) pairs covering both single-view and multi-view scenarios, sourced from varied visual environments and supporting 19 distinct instruction formats that mirror real-world query patterns. InternSpatial-Bench aims to single-view assessment and also extends multi-view reasoning through a novel rotation estimation task. Experimental validation demonstrates that models trained on \\trainset achieve substantial performance improvement of 12.1% on InternSpatial-Bench and 10.7% on VSI-Bench, while preserving competitive performance on general-purpose benchmarks. We expect these resources can advance the development of spatially-capable VLMs for practical applications in robotics and embodied AI systems. Our codes and datasets are publicly available at https://github.com/dengnianchen/intern-spatial.",
"是否为团队主导工作": false,
"期刊/会议": "ICLR-2026",
"记录创建日期": 1776268800000,
"论文发表日期": 1769356800000,
"论文标题": "InternSpatial: A Comprehensive Dataset for Spatial Reasoning in Vision-Language Models",
"论文状态": "已录用",
"责任人": [
{
"email": "dengnianchen@pjlab.org.cn",
"en_name": "邓念晨",
"id": "ou_f6ffc90f46f0c01a4876cf3dbb8174bd",
"name": "邓念晨"
}
],
"_thumbnail": "/assets/images/papers/2506.18385.png"
},
{
"Bibtex": "@article{chen2025iwr,\n title={IWR-Bench: Can LVLMs reconstruct interactive webpage from a user interaction video?},\n author={Chen, Yang and Liu, Minghao and Shen, Yufan and Li, Yunwen and Huang, Tianyuan and Fang, Xinyu and Zheng, Tianyu and Huang, Wenxuan and Yang, Cheng and Fu, Daocheng and others},\n journal={arXiv preprint arXiv:2509.24709},\n year={2025}\n}",
"Github仓库链接": {
"link": "https://github.com/SIGMME/IWR-Bench",
"text": "https://github.com/SIGMME/IWR-Bench"
},
"arXiv主页": {
"link": "https://arxiv.org/pdf/2509.24709",
"text": "https://arxiv.org/pdf/2509.24709"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Yang Chen, Minghao Liu, Yufan Shen, Yunwen Li, Tianyuan Huang, Xinyu Fang, Tianyu Zheng, Wenxuan Huang, Cheng Yang, Daocheng Fu, Jianbiao Mei, Rong Wu, Yunfei Zhao, Licheng Wen, Xuemeng Yang, Song Mao, Qunshu Lin, Zhi Yu, Yongliang Shen, Yu Qiao, Botian Shi",
"单位信息(每个单位一行,分号换行)": "1. Shanghai AI Laboratory;\n2. Zhejiang Universtiy;\n3. 2077AI;\n4. M-A-P;\n5. Chinese University of Hong Kong(shenzhen);\n6. Chinese University of Hong Kong\n7. Central South University\n8. Fudan University",
"录用类型": [
"Poster"
],
"摘要": "The webpage-to-code task requires models to understand visual representations of\nwebpages and generate corresponding code. However, existing benchmarks primarily focus on static screenshot-to-code tasks, thereby overlooking the dynamic\ninteractions fundamental to real-world web applications. To address this limitation, this paper introduces IWR-Bench, a novel benchmark for evaluating the\ncapabilities of Large Vision-Language Models (LVLMs) in interactive webpage\nreconstruction from video. IWR-Bench comprises 113 meticulously curated tasks\nfrom 100 real-world websites, with 1,001 actions and featuring diverse interaction complexities (e.g., web games), visual styles, and domains. Aligning with\nstandard web development practices, each task includes not only user interaction\nvideos but also all crawled static assets (e.g., images, videos). This benchmark\nevaluates models on two fundamental challenges: comprehensive multi-modal\nreasoning to infer interaction logic from video and assets, and advanced code\ngeneration to translate this logic into functional code. An agent-as-a-judge framework with a comprehensive metric system automatically assesses the functional\ncorrectness and visual fidelity of generated webpages. Extensive experiments on\n28 LVLMs reveal a significant challenge: the best model achieves an overall score\nof only 36.35%, as functional correctness (24.39% IFS) lags significantly behind\nvisual fidelity (64.25% VFS). These results highlight critical limitations in current\nmodels’ ability to reason about temporal dynamics and synthesize event-driven\nlogic, establishing IWR-Bench as a challenging frontier for vision-language research. The benchmark and evaluation code will be made publicly available athttps://github.com/SIGMME/IWR-Bench.",
"是否为团队主导工作": true,
"期刊/会议": "ICLR-2026",
"记录创建日期": 1776268800000,
"论文发表日期": 1769356800000,
"论文标题": "IWR-Bench: Can LVLMs reconstruct interactive webpage from a user interaction video?\n",
"论文状态": "已录用",
"责任人": [
{
"email": "shenyufan@pjlab.org.cn",
"en_name": "沈宇帆",
"id": "ou_7b10a35b8b8655e16e76f5536a5a054a",
"name": "沈宇帆"
}
],
"_thumbnail": "/assets/images/papers/2509.24709.png"
},
{
"arXiv主页": {
"link": "https://arxiv.org/abs/2508.05502",
"text": "https://arxiv.org/abs/2508.05502"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Yufei Gao, Jiaying Fei, Nuo Chen, Ruirui Chen, Guohang Yan, Yunshi Lan, Botian Shi",
"单位信息(每个单位一行,分号换行)": "1 Shanghai AI Laboratory\n2 East China Normal University\n3 The Chinese University of Hong Kong, Shenzhen\n4\nInstitute of High Performance Computing, A*STAR",
"录用类型": [
"Poster"
],
"摘要": "Multimodal Large Language Models (MLLMs) perform strongly in high-resource languages, yet their effectiveness drops sharply in low-resource settings, largely due to the scarcity of aligned and culturally informative multimodal data. Existing multilingual enhancement approaches predominantly rely on text-only resources or translation-based pipelines, which improve surface-level fluency but often fail to capture culturally specific visual knowledge.\nIn this work, we present MELLA, a large-scale multimodal multilingual dataset designed to support both linguistic fluency and culturally grounded visual understanding in low-resource languages. MELLA is constructed using a dual-source data curation strategy that combines (i) native web image-alt-text pairs, which provide in-context, culture-specific visual-textual alignments, and (ii) high-quality image descriptions generated in a high-resource language and translated into target languages to ensure linguistic richness and structural completeness. Rather than expanding multilingual coverage alone, this design explicitly disentangles two complementary learning signals that are conflated in existing multilingual multimodal datasets.\nMELLA covers eight low-resource languages and contains 6.8M image-text pairs spanning diverse domains and visual categories. Through controlled diagnostic fine-tuning experiments on multiple MLLM backbones, we show that training on MELLA mitigates the cultural hallucination gap, often manifested as culturally “thin“ descriptions, by enabling models to recognize and articulate culturally specific entities that are systematically overlooked by translation-centric pipelines. Our findings underscore the central role of data alignment, rather than model modification, in achieving culturally grounded multimodal understanding for low-resource languages.",
"是否为团队主导工作": true,
"期刊/会议": "IJCAI-2026",
"记录创建日期": 1776268800000,
"论文发表日期": 1777996800000,
"论文标题": "MELLA: Bridging Linguistic Capability and Cultural Groundedness for Low-Resource Language MLLMs",
"论文状态": "已录用",
"责任人": [
{
"email": "yanguohang@pjlab.org.cn",
"en_name": "闫国行",
"id": "ou_33cfcf2048b911c367917bf320f12862",
"name": "闫国行"
}
],
"_thumbnail": "/assets/images/papers/2508.05502.png"
},
{
"Bibtex": "@misc{zhang2026evipropseededrelevancediffusion,\n title={EviProp: Seeded Relevance Diffusion on Chunk-Page Graphs for Long Multimodal Document Retrieval}, \n author={Hongwei Zhang and Xiaoman Wang and Zehui Ling and Ruicheng Zhu and Yue Zhang and Pinlong Cai and Fuke Shen and Botian Shi and Tongquan Wei and Guohang Yan},\n year={2026},\n eprint={2606.08979},\n archivePrefix={arXiv},\n primaryClass={cs.IR},\n url={https://arxiv.org/abs/2606.08979}, \n}",
"arXiv主页": {
"link": "https://arxiv.org/abs/2606.08979",
"text": "https://arxiv.org/abs/2606.08979"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Hongwei Zhang, Xiaoman Wang, Zehui Ling, Ruicheng Zhu, Yue Zhang, Pinlong Cai, Fuke Shen, Botian Shi, Tongquan Wei, Guohang Yan",
"单位信息(每个单位一行,分号换行)": "1 East China Normal University\n2 Shanghai AI Laboratory",
"录用类型": [
"Preprint"
],
"摘要": "Retrieving evidence pages from visually rich long documents is a key challenge in document question answering. Existing page-level visual retrievers operate under an independent matching paradigm: each page is scored in isolation based on query-page similarity. This paradigm can under-rank evidence pages whose signals are localized in fine-grained chunks or depend on document-internal associations. We propose EviProp, a retrieval method that recovers such pages via seeded relevance diffusion. EviProp models each document as a multimodal Chunk-Page graph with hierarchical, sequential, and similarity links. Given a query, it combines dense visual page priors with sparse chunk seeds, then runs Personalized PageRank to diffuse relevance over the graph. Experiments on MMLongBench-Doc and LongDocURL show consistent gains in evidence-page retrieval over independent visual retrieval and text-visual fusion baselines. Downstream QA results further show that improved retrieval translates into better answer accuracy, with negligible online retrieval overhead.",
"是否为团队主导工作": true,
"期刊/会议": "Under Submission",
"记录创建日期": 1781193600000,
"论文发表日期": 1780848000000,
"论文标题": "EviProp: Seeded Relevance Diffusion on Chunk-Page Graphs for Long Multimodal Document Retrieval",
"论文状态": "已投稿并挂arXiv",
"责任人": [
{
"email": "yanguohang@pjlab.org.cn",
"en_name": "闫国行",
"id": "ou_33cfcf2048b911c367917bf320f12862",
"name": "闫国行"
}
],
"_thumbnail": "/assets/images/papers/2606.08979.png"
},
{
"Bibtex": "@misc{lei2026memcottesttimescalingmemorydriven,\n title={MemCoT: Test-Time Scaling through Memory-Driven Chain-of-Thought}, \n author={Haodong Lei and Junming Liu and Yirong Chen and Ding Wang and Hongsong Wang},\n year={2026},\n eprint={2604.08216},\n archivePrefix={arXiv},\n primaryClass={cs.MA},\n url={https://arxiv.org/abs/2604.08216}, \n}",
"Github仓库链接": {
"link": "https://github.com/Haodong-Lei-Ray/MemCoT",
"text": "https://github.com/Haodong-Lei-Ray/MemCoT"
},
"arXiv主页": {
"link": "https://arxiv.org/abs/2604.08216",
"text": "https://arxiv.org/abs/2604.08216"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Haodong Lei, Junming Liu, Yirong Chen, Ding Wang, Hongsong Wang",
"单位信息(每个单位一行,分号换行)": "1 Southeast University\n2 Shanghai AI Laboratory",
"录用类型": [
"Preprint"
],
"摘要": "Large Language Models (LLMs) still suffer from severe hallucinations and catastrophic forgetting during causal reasoning over massive, fragmented long contexts. Existing memory mechanisms typically treat retrieval as a static, single-step passive matching process, leading to severe semantic dilution and contextual fragmentation. To overcome these fundamental bottlenecks, we propose MemCoT, a test-time memory scaling framework that redefines the reasoning process by transforming long-context reasoning into an iterative, stateful information search. MemCoT introduces a multi-view long-term memory perception module that enables Zoom-In evidence localization and Zoom-Out contextual expansion, allowing the model to first identify where relevant evidence resides and then reconstruct the surrounding causal structure necessary for reasoning. In addition, MemCoT employs a task-conditioned dual short-term memory system composed of semantic state memory and episodic trajectory memory. This short-term memory records historical search decisions and dynamically guides query decomposition and pruning across iterations. Empirical evaluations demonstrate that MemCoT establishes a state-of-the-art performance. Empowered by MemCoT, several open- and closed-source models achieve SOTA performance on the LoCoMo benchmark and LongMemEval-S benchmark.",
"是否为团队主导工作": true,
"期刊/会议": "Under Submission",
"记录创建日期": 1776268800000,
"论文pdf": [
{
"file_token": "FmJobtze3oAaIexgWmzc55hHnig",
"name": "MemCoT.pdf",
"size": 2331220,
"tmp_url": "https://open.feishu.cn/open-apis/drive/v1/medias/batch_get_tmp_download_url?file_tokens=FmJobtze3oAaIexgWmzc55hHnig",
"type": "application/pdf",
"url": "https://open.feishu.cn/open-apis/drive/v1/medias/FmJobtze3oAaIexgWmzc55hHnig/download"
}
],
"论文发表日期": 1775664000000,
"论文标题": "MemCoT: Test-Time Scaling through Memory-Driven Chain-of-Thought",
"论文状态": "已投稿并挂arXiv",
"责任人": [
{
"email": "leihaodong@pjlab.org.cn",
"en_name": "雷浩东",
"id": "ou_6b062ced20a3e2633d1b3be928b9728f",
"name": "雷浩东"
}
],
"_thumbnail": "/assets/images/papers/2604.08216.png"
},
{
"Bibtex": "@article{wu2025evolver,\n title={Evolver: Self-evolving llm agents through an experience-driven lifecycle},\n author={Wu, Rong and Wang, Xiaoman and Mei, Jianbiao and Cai, Pinlong and Fu, Daocheng and Yang, Cheng and Wen, Licheng and Yang, Xuemeng and Shen, Yufan and Wang, Yuxin and others},\n journal={arXiv preprint arXiv:2510.16079},\n year={2025}\n}\n\n\n\n\n\n",
"Github仓库链接": {
"link": "https://github.com/KnowledgeXLab/EvolveR",
"text": "https://github.com/KnowledgeXLab/EvolveR"
},
"arXiv主页": {
"link": "https://arxiv.org/abs/2510.16079",
"text": "https://arxiv.org/abs/2510.16079"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Rong Wu 1,2∗;\nXiaoman Wang 3∗;\nJianbiao Mei 1,2;\nPinlong Cai 2;\nDaocheng Fu 2,4; \nCheng Yang 2,5;\nLicheng Wen 2,7,8; \nXuemeng Yang 2; \nYufan Shen 2;\nYuxin Wang 9;\nBotian Shi 2 ^",
"单位信息(每个单位一行,分号换行)": "1 Zhejiang University; \n2 Shanghai Artificial Intelligence Laboratory;\n3 East China Normal University;\n4 Fudan University;\n5 Central South University;\n7 Shanghai Innovation Institute;\n8 Shanghai Jiao Tong University;\n9 University of Science and Technology of China",
"录用类型": [
"Poster"
],
"摘要": "Current Large Language Model (LLM) agents show strong performance in tool use, but lack the crucial capability to systematically learn from their own experiences. While existing frameworks mainly focus on mitigating external knowledge gaps, they fail to address a more fundamental limitation: the inability to iteratively refine problem-solving strategies. In this work, we introduce \\textbf{EvolveR}, a framework designed to enable agent to self-improve through a complete, closed-loop experience lifecycle. This lifecycle comprises two key stages: (1) \\textbf{Offline Self-Distillation}, where the agent's interaction trajectories are synthesized into a structured repository of abstract, reusable strategic principles; (2) \\textbf{Online Interaction}, where the agent interacts with tasks and actively retrieves distilled principles to guide its decision-making, accumulating a diverse set of behavioral trajectories. This loop employs a policy reinforcement mechanism to iteratively update the agent based on its performance. We demonstrate the effectiveness of EvolveR on complex multi-hop question-answering benchmarks, where it achieves superior performance over strong agentic baselines. Our work presents a comprehensive blueprint for agents that learn not only from external data but also from the consequences of their own actions, paving the way for more autonomous and continuously improving systems. Code is available at \\textcolor{magenta}{\\url{https://github.com/Edaizi/EvolveR}}.",
"是否为团队主导工作": true,
"是否已宣发": "已宣发",
"期刊/会议": "ICML 2026",
"记录创建日期": 1776355200000,
"论文pdf": [
{
"file_token": "E55LbIE2mot0NtxyO80cYbWOn1g",
"name": "2510.16079v1.pdf",
"size": 2933251,
"tmp_url": "https://open.feishu.cn/open-apis/drive/v1/medias/batch_get_tmp_download_url?file_tokens=E55LbIE2mot0NtxyO80cYbWOn1g",
"type": "application/pdf",
"url": "https://open.feishu.cn/open-apis/drive/v1/medias/E55LbIE2mot0NtxyO80cYbWOn1g/download"
}
],
"论文发表日期": 1760630400000,
"论文标题": "EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle",
"论文状态": "已录用",
"责任人": [
{
"email": "wurong@pjlab.org.cn",
"en_name": "吴荣",
"id": "ou_e2c69b87d4cb138c76221ca2f8d5326f",
"name": "吴荣"
}
],
"_thumbnail": "/assets/images/papers/2510.16079.png"
},
{
"Bibtex": "@article{wu2025kg,\n title={Kg-traces: Enhancing large language models with knowledge graph-constrained trajectory reasoning and attribution supervision},\n author={Wu, Rong and Cai, Pinlong and Mei, Jianbiao and Wen, Licheng and Hu, Tao and Yang, Xuemeng and Fu, Daocheng and Shi, Botian},\n journal={arXiv preprint arXiv:2506.00783},\n year={2025}\n}\n\n\n\n\n\n",
"Github仓库链接": {
"link": "https://github.com/KnowledgeXLab/KG-TRACES",
"text": "https://github.com/KnowledgeXLab/KG-TRACES"
},
"arXiv主页": {
"link": "https://arxiv.org/abs/2506.00783",
"text": "https://arxiv.org/abs/2506.00783"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Rong Wu 1,2;\nPinlong Cai 2;\nJianbiao Mei 1,2;\nLicheng Wen 2;\nTao Hu 2,3;\nXuemeng Yang 2;\nDaocheng Fu 2,4;\nBotian Shi 2^",
"单位信息(每个单位一行,分号换行)": "1 Zhejiang University;\n2 Shanghai Artificial Intelligence Laboratory;\n3 University of Science and Technology of China;\n4 Fudan University",
"录用类型": [
"Preprint"
],
"摘要": "Large language models (LLMs) have made remarkable strides in various natural language processing tasks, but their performance on complex reasoning problems remains hindered by a lack of explainability and trustworthiness. This issue, often manifesting as hallucinations or unattributable reasoning processes, limits their applicability in complex reasoning scenarios. To address this, we propose \\textbf{K}nowledge \\textbf{G}raph-constrained \\textbf{T}rajectory \\textbf{R}easoning \\textbf{A}ttribution and \\textbf{C}hain \\textbf{E}xplanation \\textbf{S}upervision (\\textbf{KG-TRACES}), a novel framework that enhances the reasoning ability of LLMs through explicit supervision over reasoning paths and processes. KG-TRACES jointly supervises the model to: (1) predict symbolic relation paths, (2) predict full triple-level reasoning paths, and (3) generate attribution-aware reasoning processes grounded in the reasoning paths. At inference phase, the model adapts to both KG-available and KG-unavailable scenarios, retrieving reasoning paths from a KG when possible or predicting plausible reasoning paths with only intrinsic knowledge when not. This design enables the model to reason in an explainable and source-attributable pattern. Through extensive experiments on complex reasoning tasks, we demonstrate that KG-TRACES significantly outperforms existing SOTA: it improves Hits@1 by 1.6\\% and F1 by 4.7\\% on WebQSP, and achieves improvements of 4.8\\% in Hits@1 and 2.1\\% in F1 on CWQ. Moreover, we show its transferability to specialized domains such as medicine. By visualizing the intermediate steps of reasoning processes, we further show that the explicit supervision introduced by KG-TRACES leads to more stable and goal-directed reasoning processes, aligning closely with correct answers. Code is available at \\textcolor{magenta}{\\url{https://github.com/Edaizi/KG-TRACES}}.",
"是否为团队主导工作": true,
"期刊/会议": "Under Submission",
"记录创建日期": 1776355200000,
"论文pdf": [
{
"file_token": "I88hbWsm3od3MjxmdFQcJKkHnae",
"name": "2506.00783v2.pdf",
"size": 11472894,
"tmp_url": "https://open.feishu.cn/open-apis/drive/v1/medias/batch_get_tmp_download_url?file_tokens=I88hbWsm3od3MjxmdFQcJKkHnae",
"type": "application/pdf",
"url": "https://open.feishu.cn/open-apis/drive/v1/medias/I88hbWsm3od3MjxmdFQcJKkHnae/download"
}
],
"论文发表日期": 1748707200000,
"论文标题": "Kg-traces: Enhancing large language models with knowledge graph-constrained trajectory reasoning and attribution supervision",
"论文状态": "已投稿并挂arXiv",
"责任人": [
{
"email": "wurong@pjlab.org.cn",
"en_name": "吴荣",
"id": "ou_e2c69b87d4cb138c76221ca2f8d5326f",
"name": "吴荣"
}
],
"_thumbnail": "/assets/images/papers/2506.00783.png"
},
{
"arXiv主页": {
"link": "https://arxiv.org/abs/2510.24168",
"text": "https://arxiv.org/abs/2510.24168"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Weihua Cheng;\nJunming Liu;\nYifei Sun;\nBotian Shi;\nYirong Chen^;\nDing Wang^",
"单位信息(每个单位一行,分号换行)": "1.Shanghaitech University;\n2 Shanghai AI Laboratory",
"录用类型": [
"Preprint"
],
"期刊/会议": "Under Submission",
"记录创建日期": 1776355200000,
"论文发表日期": 1761580800000,
"论文标题": "MGA: Memory-Driven GUI Agent for Observation-Centric Interaction",
"论文状态": "已投稿并挂arXiv",
"责任人": [
{
"email": "wangding@pjlab.org.cn",
"en_name": "王锭",
"id": "ou_03ab7a597be21b722a09d791e2eecba2",
"name": "王锭"
}
],
"_thumbnail": "/assets/images/papers/2510.24168.png"
},
{
"arXiv主页": {
"link": "https://arxiv.org/pdf/2507.20766",
"text": "https://arxiv.org/pdf/2507.20766"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Yang Chen, Yufan Shen, Wenxuan Huang, Sheng Zhou, Qunshu Lin, Xinyu Cai, Zhi Yu, Jiajun Bu, Botian Shi, Yu Qiao",
"单位信息(每个单位一行,分号换行)": "1 Zhejiang University,2 Shanghai Artificial Intelligence Laboratory,3 East China Normal University, 4 Shanghai Innovation Institute, 5 Abaka AI",
"期刊/会议": "Under Submission",
"记录创建日期": 1776355200000,
"论文标题": "Learning only with images: Visual reinforcement learning with reasoning, rendering, and visual feedback",
"论文状态": "已投稿并挂arXiv",
"责任人": [
{
"email": "chenyang3@pjlab.org.cn",
"en_name": "陈杨",
"id": "ou_dbf786559c799b862b7591861f546e1d",
"name": "陈杨"
}
],
"_thumbnail": "/assets/images/papers/2507.20766.png"
},
{
"Bibtex": "@inproceedings{zhang2026leanrag,\n title={Leanrag: Knowledge-graph-based generation with semantic aggregation and hierarchical retrieval},\n author={Zhang, Yaoze and Wu, Rong and Cai, Pinlong and Wang, Xiaoman and Yan, Guohang and Mao, Song and Wang, Ding and Shi, Botian},\n booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},\n volume={40},\n number={41},\n pages={34862--34869},\n year={2026}\n}\n\n\n\n\n\n",
"Github仓库链接": {
"link": "https://github.com/KnowledgeXLab/LeanRAG",
"text": "https://github.com/KnowledgeXLab/LeanRAG"
},
"arXiv主页": {
"link": "https://arxiv.org/abs/2508.10391",
"text": "https://arxiv.org/abs/2508.10391"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Yaoze Zhang 1,2*;\nRong Wu 1,3*;\nPinlong Cai 1^;\nXiaoman Wang 4;\nGuohang Yan 1;\nSong Mao 1;\nDing Wang 1;\nBotian Shi 1",
"刊印链接": {
"link": "https://ojs.aaai.org/index.php/AAAI/article/download/40789/44750",
"text": "https://ojs.aaai.org/index.php/AAAI/article/download/40789/44750"
},
"单位信息(每个单位一行,分号换行)": "1 Shanghai Artificial Intelligence Laboratory;\n2 University of Shanghai for Science and Technology;\n3 Zhejiang University;\n4 East China Normal University",
"录用类型": [
"Poster"
],
"摘要": "Retrieval-Augmented Generation (RAG) plays a crucial role in grounding Large Language Models by leveraging external knowledge, whereas the effectiveness is often compromised by the retrieval of contextually flawed or incomplete information. To address this, knowledge graph-based RAG methods have evolved towards hierarchical structures, organizing knowledge into multi-level summaries. However, these approaches still suffer from two critical, unaddressed challenges: high-level conceptual summaries exist as disconnected``semantic islands'', lacking the explicit relations needed for cross-community reasoning; and the retrieval process itself remains structurally unaware, often degenerating into an inefficient flat search that fails to exploit the graph's rich topology. To overcome these limitations, we introduce LeanRAG, a framework that features a deeply collaborative design combining knowledge aggregation and retrieval strategies. LeanRAG first employs a novel semantic aggregation algorithm that forms entity clusters and constructs new explicit relations among aggregation-level summaries, creating a fully navigable semantic network. Then, a bottom-up, structure-guided retrieval strategy anchors queries to the most relevant fine-grained entities and then systematically traverses the graph's semantic pathways to gather concise yet contextually comprehensive evidence sets. The LeanRAG can mitigate the substantial overhead associated with path retrieval on graphs and minimize redundant information retrieval. Extensive experiments on four challenging QA benchmarks with different domains demonstrate that LeanRAG significantly outperforms existing methods in response quality while reducing 46% retrieval redundancy.",
"是否为团队主导工作": true,
"是否已宣发": "已宣发",
"期刊/会议": "AAAI 2026",
"记录创建日期": 1776355200000,
"论文pdf": [
{
"file_token": "OwCTbmt58olMp2xHV4fcLM4Kn0c",
"name": "40789-Article Text-44880-1-2-20260314.pdf",
"size": 840145,
"tmp_url": "https://open.feishu.cn/open-apis/drive/v1/medias/batch_get_tmp_download_url?file_tokens=OwCTbmt58olMp2xHV4fcLM4Kn0c",
"type": "application/pdf",
"url": "https://open.feishu.cn/open-apis/drive/v1/medias/OwCTbmt58olMp2xHV4fcLM4Kn0c/download"
}
],
"论文发表日期": 1768838400000,
"论文标题": "Leanrag: Knowledge-graph-based generation with semantic aggregation and hierarchical retrieval",
"论文状态": "已录用",
"责任人": [
{
"email": "wurong@pjlab.org.cn",
"en_name": "吴荣",
"id": "ou_e2c69b87d4cb138c76221ca2f8d5326f",
"name": "吴荣"
}
],
"_thumbnail": "/assets/images/papers/2508.10391.png"
},
{
"arXiv主页": {
"link": "https://arxiv.org/pdf/2604.01670",
"text": "https://arxiv.org/pdf/2604.01670"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Junming Liu, Yifei Sun, Weihua Cheng, Haodong Lei, Yuqi Li, Yirong Chen, Ding Wang",
"单位信息(每个单位一行,分号换行)": "1 Shanghai AI Laboratory\n2 The City University of New York",
"录用类型": [
"Preprint"
],
"期刊/会议": "Under Submission",
"记录创建日期": 1776700800000,
"论文标题": "Hierarchical Memory Orchestration for Personalized Persistent Agents",
"论文状态": "已投稿并挂arXiv",
"_thumbnail": "/assets/images/papers/2604.01670.png"
},
{
"Bibtex": "@inproceedings{\n wang2026investigating,\n title={Investigating Redundancy in Multimodal Large Language Models with Multiple Vision Encoders},\n author={Yizhou Wang and Song Mao and Yang Chen and Yufan Shen and Pinlong Cai and Ding Wang and Guohang Yan and Zhi Yu and Yinqiao Yan and Xuming Hu and Botian Shi},\n booktitle={The Fourteenth International Conference on Learning Representations},\n year={2026},\n url={https://openreview.net/forum?id=cAopJVLKvi}\n}",
"Github仓库链接": {
"link": "https://maosong.website/Encoder-Redundancy/",
"text": "https://maosong.website/Encoder-Redundancy/"
},
"arXiv主页": {
"link": "https://arxiv.org/abs/2507.03262",
"text": "https://arxiv.org/abs/2507.03262"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Yizhou Wang*,†, Song Mao*, Yang Chen*,†, Yufan Shen, Yinqiao Yan, Pinlong Cai, Ding Wang, Guohang Yan, Zhi Yu, Xuming Hu, Botian Shi",
"刊印链接": {
"link": "https://openreview.net/forum?id=cAopJVLKvi",
"text": "https://openreview.net/forum?id=cAopJVLKvi"
},
"单位信息(每个单位一行,分号换行)": "1. Shanghai AI Lab\n2. HKUST(GZ)\n3. Zhejiang University\n4. Beijing University of Technology",
"录用类型": [
"Poster"
],
"摘要": "Recent multimodal large language models (MLLMs) increasingly integrate multiple vision encoders to improve performance on various benchmarks, assuming that diverse pretraining objectives yield complementary visual signals. However, we show this assumption often fails in practice. Through systematic encoder masking across representative multi-encoder MLLMs, we find that performance typically degrades gracefully—and sometimes even improves—when selected encoders are masked, revealing pervasive encoder redundancy. To quantify this effect, we introduce two principled metrics: the Conditional Utilization Rate (CUR), which measures an encoder’s marginal contribution in the presence of others, and the Information Gap (IG), which captures heterogeneity in encoder utility within a model. Using these tools, we observe: (i) strong specialization on tasks like OCR & Chart, where a single encoder can dominate with a CUR >90%, (ii) high redundancy on general VQA and knowledge-based tasks, where encoders are largely interchangeable, (iii) instances of detrimental encoders with negative CUR. Notably, masking specific encoders can yield up to 16% higher accuracy on a specific task category and 3.6% overall performance boost compared to the full model. Furthermore, single- and dual- encoder variants recover over 90% of baseline on most non-OCR tasks. Our analysis challenges the “more encoders are better” heuristic in MLLMs and provides actionable diagnostics for developing more efficient and effective multimodal architectures.\n",
"是否为团队主导工作": true,
"是否已宣发": "已宣发",
"期刊/会议": "ICLR-2026",
"记录创建日期": 1776700800000,
"论文pdf": [
{
"file_token": "SW16b7CTsozzyjxYk8JcwUOvnze",
"name": "Encoder_Redundancy_ICLR2026_camera_ready.pdf",
"size": 1416170,
"tmp_url": "https://open.feishu.cn/open-apis/drive/v1/medias/batch_get_tmp_download_url?file_tokens=SW16b7CTsozzyjxYk8JcwUOvnze",
"type": "application/pdf",
"url": "https://open.feishu.cn/open-apis/drive/v1/medias/SW16b7CTsozzyjxYk8JcwUOvnze/download"
}
],
"论文发表日期": 1769356800000,
"论文标题": "Investigating Redundancy in Multimodal Large Language Models with Multiple Vision Encoders\n",
"论文状态": "已录用",
"责任人": [
{
"email": "maosong@pjlab.org.cn",
"en_name": "毛松",
"id": "ou_80e5c8509a8f5d3a8ff9e786499f32f2",
"name": "毛松"
}
],
"_thumbnail": "/assets/images/papers/2507.03262.png"
},
{
"Bibtex": "@article{yang2026spiral,\n title = {SPIRAL: Self-Evolving Action-Conditioned Video Generation via Reflective Planning Agents},\n author = {Yang, Yu and Liao, Yue and Mei, Jianbiao and Wang, Baisen and Yang, Xuemeng and Wen, Licheng and Zhang, Jiangning and Li, Xiangtai and Lv, Liang and Chen, Hanlin and Shi, Botian and Liu, Yong and Yan, Shuicheng and Lee, Gim Hee},\n journal = {arXiv preprint arXiv:2603.08403},\n year = {2026}\n}",
"Github仓库链接": {
"link": "https://yuyang-cloud.github.io/spiral",
"text": "https://yuyang-cloud.github.io/spiral"
},
"arXiv主页": {
"link": "https://arxiv.org/pdf/2603.08403",
"text": "https://arxiv.org/pdf/2603.08403"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Yu Yang*, Yue Liao*, Jianbiao Mei*, Baisen Wang*, Xuemeng Yang, Licheng Wen, Jiangning Zhang, Xiangtai Li, Liang Lv, Hanlin Chen, Botian Shi, Yong Liu†, Shuicheng Yan, Gim Hee Lee",
"单位信息(每个单位一行,分号换行)": "1. Zhejiang University\n2. Shanghai AI Laboratory\n3. National University of Singapore\n4. Chinese Academy of Sciences\n5. Tencent Youtu Lab\n6. Nanyang Technological University\n7. Wuhan University",
"摘要": "Long-horizon action-conditioned video generation aims to synthesize temporally coherent videos that follow complex action instructions over extended horizons, requiring procedural ordering, persistent action execution, and scene consistency beyond conventional TI2V's short-term fidelity. Existing single-shot video generation models typically operate in an open-loop manner, leading to incomplete action execution, hallucinated motions, and temporal drift. To address this, we propose SPIRAL, a closed-loop framework that performs sequential planning and iterative reflection for action-conditioned long-horizon video generation. Specifically, SPIRAL instantiates a think-act-reflect process: a PlanAgent decomposes high-level goals into sub-actions, which condition a VideoGenerator to synthesize each segment alongside a memory context, while a CriticAgent evaluates intermediate video segments to provide corrective feedback for iterative refinement. This closed-loop design further supports self-evolution by utilizing PlanAgent-proposed actions and CriticAgent-derived rewards for GRPO-based post-training to enhance the video generator's long-horizon consistency. Moreover, we introduce ActVideoGen-Dataset for task-specific training, and establish ActVideoGen-Bench as a dedicated evaluation suite for measuring action quality and temporal coherence. Experiments across multiple TI2V backbones alongside the self-evolving strategy show consistent gains on ActVideoGen-Bench and VBench, demonstrating the effectiveness of SPIRAL.",
"期刊/会议": "Under Submission",
"记录创建日期": 1779379200000,
"论文pdf": [
{
"file_token": "Z2CDbEmx2oIACUxOew8cw8zUnoh",
"name": "2603.08403v3.pdf",
"size": 14618830,
"tmp_url": "https://open.feishu.cn/open-apis/drive/v1/medias/batch_get_tmp_download_url?file_tokens=Z2CDbEmx2oIACUxOew8cw8zUnoh",
"type": "application/pdf",
"url": "https://open.feishu.cn/open-apis/drive/v1/medias/Z2CDbEmx2oIACUxOew8cw8zUnoh/download"
}
],
"论文标题": "SPIRAL: Self-Evolving Action-Conditioned Video Generation via Reflective Planning Agents",
"论文状态": "已投稿并挂arXiv",
"责任人": [
{
"email": "yangyu1@pjlab.org.cn",
"en_name": "杨煜",
"id": "ou_b3984e2ddd4f99a0a77ac8b59fec5f23",
"name": "杨煜"
}
],
"_thumbnail": "/assets/images/papers/2603.08403.png"
},
{
"Bibtex": "@article{wang2026ia,\n title={{IA-RAG}: Interval-Algebra-Driven Temporal Reasoning for Dynamic Knowledge Retrieval},\n author={Wang, Xiaoman and Zhang, Yaoze and Fan, Wenzhuo and Zhang, Hongwei and Wang, Ding and Yan, Guohang and Mao, Song and Shi, Botian and Lan, Yunshi and Cai, Pinlong},\n journal={arXiv preprint arXiv:2606.06044},\n year={2026}\n}",
"arXiv主页": {
"link": "https://arxiv.org/pdf/2606.06044",
"text": "https://arxiv.org/pdf/2606.06044"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Xiaoman Wang*, Yaoze Zhang*, Wenzhuo Fan, Hongwei Zhang, Ding Wang, Guohang Yan, Song Mao, Botian Shi, Yunshi Lan#, Pinlong Cai#",
"单位信息(每个单位一行,分号换行)": "1. East China Normal University\n2. Shanghai Artificial Intelligence Laboratory\n3. University of Shanghai for Science and Technology\n4. Harbin Engineering University",
"录用类型": [
"Preprint"
],
"摘要": "Retrieval-Augmented Generation (RAG) has shown strong effectiveness in grounding Large Language Models (LLMs) with external knowledge. However, existing RAG and Graph RAG frameworks largely treat knowledge as static or associate time with coarse-grained timestamps or metadata, failing to capture rich temporal structures such as duration, overlap, and containment. We propose IA-RAG, a hierarchical temporal RAG framework that models knowledge as time intervals and performs retrieval under formal temporal constraints. IA-RAG represents facts as Interval Event Units (IEUs) and organizes them into a hierarchical Thematic Forest, where temporal dependencies are governed by Allen's Interval Algebra. To handle incomplete or uncertain temporal boundaries, IA-RAG further introduces a Sub-graph Time Tightening mechanism that refines fuzzy intervals through logical constraints within connected event subgraphs. In addition, IA-RAG supports implicit temporal semantic retrieval through interval-algebra-guided traversal. Experiments on multiple temporal question answering benchmarks, including TimeQA, TempReason, and ComplexTR, demonstrate that IA-RAG achieves strong temporal retrieval and reasoning performance, particularly on complex compositional temporal reasoning tasks.",
"是否为团队主导工作": true,
"是否已宣发": "已宣发",
"期刊/会议": "Under Submission",
"记录创建日期": 1781193600000,
"论文标题": "IA-RAG: Interval-Algebra-Driven Temporal Reasoning for Dynamic Knowledge Retrieval",
"论文状态": "已投稿并挂arXiv",
"责任人": [
{
"email": "caipinlong@pjlab.org.cn",
"en_name": "蔡品隆",
"id": "ou_138e540bde4fdeed0e45f85615a113cc",
"name": "蔡品隆"
}
],
"_thumbnail": "/assets/images/papers/2606.06044.png"
},
{
"Bibtex": "@misc{dong2026assemcadproductionreadycadassembly,\n title={ASSEMCAD: Production-Ready CAD Assembly Generation from Natural Language}, \n author={Yurui Dong and Shu Zou and Siqi Li and Nianchen Deng and Hongbin Zhou and Xuemeng Yang and Pinlong Cai and Licheng Wen and Xinyu Cai and Botian Shi},\n year={2026},\n eprint={2607.05123},\n archivePrefix={arXiv},\n primaryClass={cs.AI},\n url={https://arxiv.org/abs/2607.05123}, \n}",
"arXiv主页": {
"link": "https://arxiv.org/pdf/2607.05123",
"text": "https://arxiv.org/pdf/2607.05123"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Yurui Dong*, Shu Zou*, Siqi Li, Nianchen Deng, Hongbin Zhou, Xuemeng Yang, Pinlong Cai, Licheng Wen, Xinyu Cai^, Botian Shi",
"单位信息(每个单位一行,分号换行)": "1 Shanghai Artificial Intelligence Laboratory, 2 Fudan University, 3 Shanghai Innovation Institute 4 The Australian National University, 5 Zhejiang University",
"录用类型": [
"Preprint"
],
"摘要": "Recent advances in large language models and programmatic CAD have significantly improved Text-to-CAD generation for individual parts. However, production-ready mechanical assembly generation remains largely unsolved. Unlike single-part modeling, assemblies require coordinated reasoning over multiple components, functional interfaces, assembly relations, engineering principles, and physical consistency. Consequently, directly generating executable CAD code is insufficient for constructing mechanically valid and reusable assemblies. We present AssemCAD, an axiom-grounded framework for production-ready CAD assembly generation from natural language. Instead of representing an assembly as monolithic CAD code, AssemCAD first constructs an axiomatic Assembly Specification consisting of typed parts, geometry-backed ports, executable mates, and engineering axioms. Each assembly relation is explicitly grounded in one or more engineering principles, making the resulting specification interpretable, reusable, and verifiable. To realize this specification, AssemCAD introduces a port- and mate-based CAD assembly library that executes symbolic assembly relations through deterministic mate transformations and validates declared interfaces using concrete B-Rep geometric evidence. Built on this representation and library, AssemCAD further supports on-demand synthesis of reusable parametric component factories for both standard and open-world geometries. Experiments on AssemBench show that AssemCAD substantially improves assembly preservation and physical validity over code-centric CAD generation baselines, while generalizing across different foundation-model backbones. By combining axiom-grounded assembly reasoning with deterministic geometric execution, AssemCAD extends Text-to-CAD from isolated part generation toward production-ready mechanical assembly design.",
"是否为团队主导工作": true,
"记录创建日期": 1783353600000,
"论文标题": "ASSEMCAD: Production-Ready CAD Assembly Generation from Natural Language",
"论文状态": "已挂arXiv",
"责任人": [
{
"email": "caixinyu@pjlab.org.cn",
"en_name": "蔡新雨",
"id": "ou_61b448e4ccdd233da29d28e88a7069d5",
"name": "蔡新雨"
}
],
"_thumbnail": "/assets/images/papers/2607.05123.png"
},
{
"Github仓库链接": {
"link": "https://github.com/KnowledgeXLab/SemFlowRAG",
"text": "https://github.com/KnowledgeXLab/SemFlowRAG"
},
"arXiv主页": {
"link": "https://arxiv.org/abs/2606.28447",
"text": "https://arxiv.org/abs/2606.28447"
},
"作者信息(每人一行,分号换行,数字表示单位信息,*表示Equal Contribution, ^表示通讯作者)": "Houyuan Qin, Rong Wu, Qinyuan Qin, Botian Shi, Jingjing Qu, Yang Sun, Pinlong Cai",
"单位信息(每个单位一行,分号换行)": "1. Shanghai Artificial Intelligence Laboratory \n2. Zhejiang University \n3. Fudan University",
"录用类型": [
"Preprint"
],
"摘要": "Retrieval-Augmented Generation (RAG) enhanced by Knowledge Graphs has shown promise in complex multi-hop reasoning tasks. However, existing graph-based retrieval methods typically rely on flat, undirected topologies. During the retrieval process, the probability flow often gets trapped in high-degree abstract concept nodes which we define as ``probability black holes'', leading to semantic drift and noise accumulation. To address this, we propose SemFlowRAG, a framework that reconstructs the flat retrieval space into a corpus-adaptive semantic gradient graph. This data-driven self-organization enables a hierarchical structure to emerge naturally from the data distribution, capturing the intrinsic semantic granularity of the corpus to suppress structural noise. By quantifying the semantic abstractness of entities through the embedding variance of their associated passages, we transform static undirected edges into directed semantic constraints. Furthermore, we design an abstractness-guided directed PageRank algorithm that forces the retrieval trajectory to follow a ``high-to-low semantic abstractness'' gradient. This mechanism ensures layer-by-layer evidence convergence, smoothly guiding the retrieval process from abstract concepts to specific document evidence. Extensive experiments on complex QA datasets demonstrate that SemFlowRAG effectively mitigates the ``probability black holes'' issue, outperforming existing baselines in both retrieval and downstream reasoning performance.",
"是否为团队主导工作": true,
"期刊/会议": "Under Submission",
"记录创建日期": 1783440000000,
"论文标题": "SemFlowRAG: Directed Semantic Flow from Abstraction to Evidence for Complex Reasoning",
"论文状态": "已挂arXiv",
"责任人": [
{
"email": "caipinlong@pjlab.org.cn",
"en_name": "蔡品隆",
"id": "ou_138e540bde4fdeed0e45f85615a113cc",
"name": "蔡品隆"
}
],
"_thumbnail": "/assets/images/papers/2606.28447.png"
},
{
"记录创建日期": 1783440000000
}
]