MUSE is an open-source, run-centric platform for multimodal safety evaluation and red-teaming of LLMs. It integrates automatic cross-modal payload generation, three multi-turn attack algorithms (Crescendo, PAIR, Violent Durian), provider-agnostic routing, and an LLM judge with a five-level safety taxonomy into a single browser-based system. A dual-metric framework distinguishes hard from soft Attack Success Rate to capture partial information leakage that binary metrics miss. Experiments across six multimodal LLMs show that multi-turn strategies can achieve up to 90–100% ASR against models with near-perfect single-turn refusal, and Inter-Turn Modality Switching accelerates convergence by destabilizing early-turn defenses.

HippoMM tackles long, noisy audiovisual streams with a memory system inspired by the hippocampus. It combines pattern separation and completion, short-to-long consolidation, and cross-modal associative retrieval so audio and vision episodes can be stored and queried efficiently. On the HippoVlog benchmark, HippoMM jumps to 78.2% accuracy versus 64.2% for the prior best while cutting response time from 112.5s to 20.4s, showing that biologically grounded memory can be both sharper and faster.

CoFi-Agent bridges the perception gap in edge audio systems through tool-augmented coarse-to-fine processing. It runs fast local perception on a 7B Audio-LLM, then conditionally triggers forensic refinement only when uncertainty is detected, using lightweight plans for on-device tools like temporal re-listening and local ASR. On the MMAR benchmark, CoFi-Agent improves accuracy from 27.20% to 53.60% while achieving better accuracy-efficiency trade-offs than always-on investigation pipelines, enabling practical edge-cloud collaboration under real system constraints.

VERA is a voice-native benchmark that finally tests reasoning rather than speech perception under real-time conversational constraints. It packs 2,931 spoken episodes across Math, Web, Science, Long-Context, and Factual tracks, each adapted from text tasks so voice and text models can be compared apples-to-apples. The evaluations expose a systematic voice-to-text reasoning gap and a clear latency-versus-accuracy frontier: fast voice systems cluster around low accuracy, while text-level performance demands sacrificing real-time response. Diagnostics reveal distinct failure signatures across streaming, end-to-end, and cascade pipelines, giving a principled way to measure progress toward assistants that can both talk and think.

AsyncVoice wraps any slow reasoning or planning backend with a live voice layer that explains what is happening while it waits. It streams updates about the planner’s hypotheses, tool calls, and confidence as soon as they surface, so users can stay engaged with a long-running system instead of waiting in silence. Because the voice presenter is decoupled from the underlying model, it can be dropped in front of existing agents to make their “thinking time” conversational without retraining or redesigning the core planner.

KVTP trims long-form video for VLMs without losing the timeline by adapting the pruning rate per frame based on query relevance. It softly elevates keyframes while keeping just enough context from the rest. Across SparseKV-QA and other long-video QA tasks, KVTP cuts up to 80% of tokens and about 64% of FLOPs while preserving accuracy, unifying keyframe selection and token pruning in a drop-in module.

We introduce SpeechPrune alongside SPIRAL, a 1,012-example benchmark that probes long-form queries where a single spoken snippet matters. The method prunes tokens using speech-text similarity and approximated early-layer attention, all without retraining. On SPIRAL it raises accuracy by about 29% on average (up to 47%) at 20% pruning and stays competitive even after removing 80% of tokens, delivering efficient long-context understanding for speech LLMs.

SD-EVG bridges face, voice, and text with a shared diffusion backbone. It implements three pipelines: voice to face for data enrichment, face to voice to map facial traits to vocal style, and prompt to face for style staging, then feeds the learned style into TTS. The recipe delivers expressive voice generation when the available modalities do not align, making multimodal voice design practical.

ACE-Opencpop and ACE-KiSing scale singing voice data with a high-quality synthesizer and careful manual tuning. The corpora span 30 and 33 singers respectively at 48 kHz, ship with standardized splits, and plug directly into ESPnet and Muskits recipes. Across downstream SVS systems they consistently lift quality, giving practitioners a ready-made path to large-scale singing voice training.

SD-NAE synthesizes natural adversarial examples by steering Stable Diffusion with classifier feedback. A controlled optimization loop nudges latents while preserving semantics, yielding photorealistic yet adversarial images that meaningfully stress-test vision models. The pipeline delivers a practical generator for robustness studies with fully released code.

OpenOOD v1.5 upgrades OOD evaluation to ImageNet scale with foundation models like CLIP and DINOv2 plus a spectrum of semantic and covariate shifts. It ships an easy evaluator, standardized splits, and a public leaderboard so researchers can compare methods apples-to-apples, enabling reproducible, trustworthy OOD testing for the modern era.

BiSinger trains a single pop-singing model that handles Chinese, English, and natural code-switching. By mapping phonetics through CMUdict and carefully fusing data, the system learns a language-independent representation, so multilingual singing no longer requires juggling separate models. The demos show strong English and mixed-language vocals while maintaining Chinese quality.