Deprecated: The each() function is deprecated. This message will be suppressed on further calls in /home/zhenxiangba/zhenxiangba.com/public_html/phproxy-improved-master/index.php on line 456 Paper page - TimeChat-Captioner: Scripting Multi-Scene Videos with Time-Aware and Structural Audio-Visual Captions
Please give a thumbs up to this comment if you found it helpful!
\n
If you want recommendations for any Paper on Hugging Face checkout this Space
\n
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
\n","updatedAt":"2026-02-13T01:42:29.379Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":318,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6662951707839966},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2602.08711","authors":[{"_id":"698ab9841b2dc6b37d61b092","name":"Linli Yao","hidden":false},{"_id":"698ab9841b2dc6b37d61b093","name":"Yuancheng Wei","hidden":false},{"_id":"698ab9841b2dc6b37d61b094","name":"Yaojie Zhang","hidden":false},{"_id":"698ab9841b2dc6b37d61b095","name":"Lei Li","hidden":false},{"_id":"698ab9841b2dc6b37d61b096","name":"Xinlong Chen","hidden":false},{"_id":"698ab9841b2dc6b37d61b097","name":"Feifan Song","hidden":false},{"_id":"698ab9841b2dc6b37d61b098","name":"Ziyue Wang","hidden":false},{"_id":"698ab9841b2dc6b37d61b099","name":"Kun Ouyang","hidden":false},{"_id":"698ab9841b2dc6b37d61b09a","name":"Yuanxin Liu","hidden":false},{"_id":"698ab9841b2dc6b37d61b09b","name":"Lingpeng Kong","hidden":false},{"_id":"698ab9841b2dc6b37d61b09c","name":"Qi Liu","hidden":false},{"_id":"698ab9841b2dc6b37d61b09d","name":"Pengfei Wan","hidden":false},{"_id":"698ab9841b2dc6b37d61b09e","name":"Kun Gai","hidden":false},{"_id":"698ab9841b2dc6b37d61b09f","name":"Yuanxing Zhang","hidden":false},{"_id":"698ab9841b2dc6b37d61b0a0","name":"Xu Sun","hidden":false}],"publishedAt":"2026-02-09T14:21:58.000Z","submittedOnDailyAt":"2026-02-12T01:40:42.627Z","title":"TimeChat-Captioner: Scripting Multi-Scene Videos with Time-Aware and Structural Audio-Visual Captions","submittedOnDailyBy":{"_id":"655ca347f426a304c6b393a1","avatarUrl":"/avatars/67f0310d59c5912d38c2ad8e6448614d.svg","isPro":false,"fullname":"Linli Yao","user":"yaolily","type":"user"},"summary":"This paper proposes Omni Dense Captioning, a novel task designed to generate continuous, fine-grained, and structured audio-visual narratives with explicit timestamps. To ensure dense semantic coverage, we introduce a six-dimensional structural schema to create \"script-like\" captions, enabling readers to vividly imagine the video content scene by scene, akin to a cinematographic screenplay. To facilitate research, we construct OmniDCBench, a high-quality, human-annotated benchmark, and propose SodaM, a unified metric that evaluates time-aware detailed descriptions while mitigating scene boundary ambiguity. Furthermore, we construct a training dataset, TimeChatCap-42K, and present TimeChat-Captioner-7B, a strong baseline trained via SFT and GRPO with task-specific rewards. Extensive experiments demonstrate that TimeChat-Captioner-7B achieves state-of-the-art performance, surpassing Gemini-2.5-Pro, while its generated dense descriptions significantly boost downstream capabilities in audio-visual reasoning (DailyOmni and WorldSense) and temporal grounding (Charades-STA). All datasets, models, and code will be made publicly available at https://github.com/yaolinli/TimeChat-Captioner.","upvotes":27,"discussionId":"698ab9851b2dc6b37d61b0a1","githubRepo":"https://github.com/yaolinli/TimeChat-Captioner","githubRepoAddedBy":"user","ai_summary":"Omni Dense Captioning introduces a six-dimensional structural schema for generating time-aware audio-visual narratives with explicit timestamps, along with a unified evaluation metric and strong baseline model.","ai_keywords":["Omni Dense Captioning","structured schema","script-like captions","TimeAware","SFT","GRPO","task-specific rewards","dense descriptions","audio-visual reasoning","temporal grounding"],"githubStars":17,"organization":{"_id":"61dcd8e344f59573371b5cb6","name":"PekingUniversity","fullname":"Peking University","avatar":"https://cdn-uploads.huggingface.co/production/uploads/noauth/vavgrBsnkSejriUF4lXDE.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"655ca347f426a304c6b393a1","avatarUrl":"/avatars/67f0310d59c5912d38c2ad8e6448614d.svg","isPro":false,"fullname":"Linli Yao","user":"yaolily","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"674e77fa59a127e4eacf5dba","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/674e77fa59a127e4eacf5dba/W7qr94Buvvaio8zhKrEha.jpeg","isPro":false,"fullname":"Yifan Dai","user":"Moonwines","type":"user"},{"_id":"66650d38b52f0890724f3b07","avatarUrl":"/avatars/c25a365bff4985ebb71c96dd097b804f.svg","isPro":false,"fullname":"Xinlong Chen","user":"XinlongChen","type":"user"},{"_id":"65768087e58db15694d1edbe","avatarUrl":"/avatars/94aa8526d1fa9686788512be697d9a08.svg","isPro":false,"fullname":"Yifan Xu","user":"geekifan","type":"user"},{"_id":"660781a450d2b7a71091240d","avatarUrl":"/avatars/da9439b8920605d8427893d0ebc32dfa.svg","isPro":false,"fullname":"Bohan Zeng","user":"zbh0217","type":"user"},{"_id":"61540338e5b9ae6774201e58","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/61540338e5b9ae6774201e58/h_159VrXOlIgu0N0pNgXj.png","isPro":false,"fullname":"jingyun","user":"hjy","type":"user"},{"_id":"673c7319d11b1c2e246ead9c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/673c7319d11b1c2e246ead9c/IjFIO--N7Hm_BOEafhEQv.jpeg","isPro":false,"fullname":"Yang Shi","user":"DogNeverSleep","type":"user"},{"_id":"6507fbecffc738079ca592bf","avatarUrl":"/avatars/1cb0f39ac6dc2dba2292846a8d7746da.svg","isPro":false,"fullname":"Ming Chen","user":"ChenMing-thu14","type":"user"},{"_id":"659e345aa9bc1f6018944842","avatarUrl":"/avatars/5a64051b62c6423e824ed75af648f0ee.svg","isPro":false,"fullname":"x","user":"DoBetter","type":"user"},{"_id":"6751c2fd472d4a1f5270bbdb","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6751c2fd472d4a1f5270bbdb/tNW2jEJkNl9g_oIxV82KD.jpeg","isPro":false,"fullname":"Yuzhuo Chen","user":"Suchenl","type":"user"},{"_id":"672f09ccb51e2cd60e87d9c4","avatarUrl":"/avatars/a31bae93c4f2e55501ccd4c8bab8a220.svg","isPro":false,"fullname":"Kevin Ryan","user":"chocStar","type":"user"}],"acceptLanguages":["*"],"dailyPaperRank":0,"organization":{"_id":"61dcd8e344f59573371b5cb6","name":"PekingUniversity","fullname":"Peking University","avatar":"https://cdn-uploads.huggingface.co/production/uploads/noauth/vavgrBsnkSejriUF4lXDE.png"}}">
Omni Dense Captioning introduces a six-dimensional structural schema for generating time-aware audio-visual narratives with explicit timestamps, along with a unified evaluation metric and strong baseline model.
AI-generated summary
This paper proposes Omni Dense Captioning, a novel task designed to generate continuous, fine-grained, and structured audio-visual narratives with explicit timestamps. To ensure dense semantic coverage, we introduce a six-dimensional structural schema to create "script-like" captions, enabling readers to vividly imagine the video content scene by scene, akin to a cinematographic screenplay. To facilitate research, we construct OmniDCBench, a high-quality, human-annotated benchmark, and propose SodaM, a unified metric that evaluates time-aware detailed descriptions while mitigating scene boundary ambiguity. Furthermore, we construct a training dataset, TimeChatCap-42K, and present TimeChat-Captioner-7B, a strong baseline trained via SFT and GRPO with task-specific rewards. Extensive experiments demonstrate that TimeChat-Captioner-7B achieves state-of-the-art performance, surpassing Gemini-2.5-Pro, while its generated dense descriptions significantly boost downstream capabilities in audio-visual reasoning (DailyOmni and WorldSense) and temporal grounding (Charades-STA). All datasets, models, and code will be made publicly available at https://github.com/yaolinli/TimeChat-Captioner.
TimeChat-Captioner is a multimodal model designed to generate detailed, time-aware, and structurally coherent captions for multi-scene videos. It effectively coordinates visual and audio information to provide comprehensive video descriptions.