2026-08-11 AI 리서치 브리핑

Beyond Simply Environment Scaling: Designing Effective Environment Distributions for Multimodal Agent Learning에 관한 최근 업데이트입니다. 자세한 내용은 원문 링크에서 확인할 수 있습니다.

VLM(Vision-language model)의 발전 속도에 비해 벤치마크 개발이 뒤처져 약점 파악이 어렵습니다. 이 논문은 VLM 스트레스 테스트를 위한 확장 가능하고 자동화된 파이프라인인 SABRE를 제안합니다. SABRE는 테스트 프라이머를 구조화된 사양, 이미지, 질문-답변 쌍으로 변환하며, SABRE-Prior를 통해 VLM이 시각적 증거 대신 세계 사전 지식에 의존하는 경향이 있음을 보여줍니다.

English While Vision-language models (VLMs) are rapidly advancing, benchmark development lags, making it difficult to identify their weaknesses. This paper introduces SABRE, a scalable and automated pipeline for creating VLM stress tests. SABRE converts task designs into structured specifications, images, and Q&A pairs, and its SABRE-Prior instantiation reveals that VLMs often rely on world priors rather than visual evidence, demonstrating low accuracy.

Recent advances in video diffusion models (VDMs) have enabled high-fidelity video synthesis. However, generating mirror reflections remains challenging because the content within a mirror must remain consistent with the surrounding scene. Existing VDMs are not specifically designed to model scene-to-mirror relationships, which can lead to reflections with incorrect content or inconsistent spatial arrangements. We observe that mirror reflection generation involves two complementary challenges: determining what scene content should be reflected and how the reflected content should be spatially arranged within the mirror region. Motivated by this observation, we propose MirrorWorld, a reflection-aware video inpainting framework that models scene-to-mirror relationships during generation. Specifically, we introduce Semantic Relation Distillation (SRD), which transfers relational information from a frozen visual foundation model to encourage semantic associations between visible scene content and mirror regions. We further propose Geometric Transformation Alignment (GTA), which learns a transformation that guides the spatial arrangement of reflected content. The two components play complementary roles, with SRD modeling what should be reflected and GTA modeling how it should be arranged. To facilitate research on this problem, we construct a benchmark for video mirror reflection generation by repurposing four existing video mirror datasets into a unified reflection reconstruction task. Experimental results show that MirrorWorld achieves improved reflection reconstruction quality over representative image-based reflection generation methods and strong video inpainting baselines.

Event cameras excel at detecting small, fast drones, but today's detectors give away their key advantage: they convert the sparse event stream into dense grids and pay dense-processing cost on inputs that are almost entirely empty. We present SparseVoxelDet, to our knowledge the first coordinate-sparse 3D event voxel bounding-box detector: backbone, feature pyramid, temporal reduction, and detection head all operate directly on coordinate-indexed features, with no dense spatial grid at any stage. Building it exposed a hidden failure mode we name support inflation: an input filling a median 0.0652% of the voxel lattice inflates stage by stage until standard pyramid fusion leaves the detection head locally near-dense. We answer with two ideas. Expansion-free inverse-convolution fusion provably creates no active sites beyond the stored backbone supports, cutting head occupancy from a median 78.88% to 10.53%; quality-aligned supervision then recovers more accuracy than preserving sparsity costs. The payoff is measured, not assumed: executing the same trained network densely costs a median 27.5x the work and 4.65x the latency across 5,000 paired frames, with no frame cheaper dense, while the 6.22M-parameter detector reaches 87.01 AP50 on the FRED drone benchmark, ahead of matched dense controls, and holds its lead on the held-out test partition evaluated once. Sparsity, preserved by construction and supervised well, delivers the efficiency and the accuracy together.

SFT Conflicts, RL Coexists: A Theoretical and Empirical Analysis of Multi-Task Learning for LLMs에 관한 최근 업데이트입니다. 자세한 내용은 원문 링크에서 확인할 수 있습니다.

SimWAM: A Simple World Action Model for End-to-End Autonomous Driving에 관한 최근 업데이트입니다. 자세한 내용은 원문 링크에서 확인할 수 있습니다.

YOLO-PEFT: Parameter-Efficient Fine-Tuning on YOLO Family에 관한 최근 업데이트입니다. 자세한 내용은 원문 링크에서 확인할 수 있습니다.

StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding에 관한 최근 업데이트입니다. 자세한 내용은 원문 링크에서 확인할 수 있습니다.

World-Action Models (WAMs) improve end-to-end autonomous driving by transferring video dynamics priors to action prediction, but existing methods require costly future generation at inference. We present SimWAM, a simple yet effective WAM that uses video generation purely as a training signal. It co-trains a pretrained video expert and a lightweight action expert with joint flow matching. An isolated attention mask keeps action prediction independent of future frames, allowing the video branch to be discarded after training and leaving a self-contained planner that directly predicts trajectories. Since the two experts share no parameters and interact only through a unified attention interface, the video backbone could be replaced and the action expert scaled independently without modifying the learning objective or inference pipeline. We further apply reinforcement learning to optimize a compositional driving reward beyond trajectory imitation. Our SimWAM achieves $91.5$ PDMS on NAVSIM, surpasses state-of-the-art WAM-based planners with substantially lower latency, and transfers zero-shot to nuScenes. These results position SimWAM as a simple yet solid baseline that could readily benefit from advances in video generation for efficient autonomous driving. The code and model weights are available at https://github.com/H-EmbodVis/SimWAM/

연속 비디오의 이벤트 경계는 모호하지만, 기존 비디오 시간 접지 모델은 신뢰도 없이 단일 구간을 반환합니다. 이 논문은 어떤 접지 모델이든 참 순간을 특정 확률로 포함하는 시간 영역을 출력하도록 변환하는 사후 모델 불가지론적 래퍼인 COVER를 제안합니다. COVER는 보정된 비적합성 점수를 통해 예측을 확장하여 재훈련이나 화이트박스 접근 없이 유한 표본 및 분포 무관 보장을 제공합니다.

English Event boundaries in continuous video are ambiguous, yet existing video temporal grounders output single intervals without reliability statements. This paper proposes COVER, a post-hoc, model-agnostic wrapper that transforms any grounder to output a temporal region guaranteed to contain the true moment with a specified probability. COVER achieves this by calibrating a nonconformity score and widening predictions, offering finite-sample and distribution-free guarantees without requiring retraining or white-box access.