← 전체 목록

2026-11-16 AI 리서치 브리핑

최신 VLM, sLLM, on-device AI 논문과 연구 블로그를 한눈에 정리합니다. 중복 기사 방지를 위해 URL 기준으로 추적합니다.

총 18건 요약 자동 생성

VLM 업데이트

멀티모달 비전-언어 모델의 최신 논문과 리더보드 변화

Unbundling Multimodal Vision Encoding

Paper arXiv cs.CV (recent)

How multimodal language models should encode visual input remains up for debate. Most models use a pretrained vision encoder, while some recent recipes train the encoder from scratch with the language model or remove it entirely. Vision encoders bundle together three design choices: a visual prior from upstream pretraining, dedicated vision capacity, and pre-fusion placement of that capacity (i.e., before the multimodal backbone). Because published models also differ in data, architecture, and compute, comparing them cannot isolate the effect of each choice. We disentangle these choices via a controlled framework where we pretrain all models from scratch on common data and compare them on visual question answering and language benchmarks at matched lifetime compute. We find that the visual prior is not necessary; an encoder trained jointly from scratch is on par with a pretrained one, without a separate training stage. What matters instead is dedicated vision capacity: an encoder-free model trails on visual understanding even when its backbone is enlarged by the encoder's parameter count. Yet that capacity need not precede multimodal fusion, as vision-only modules added in-fusion to early backbone layers perform comparably to a pre-fusion encoder and can lower time to first token. These results also hold across backbone sizes, when encoder pretraining is treated as free, and when starting from a pretrained language model. Overall, our findings favor dedicated vision capacity trained jointly with the backbone and placed early within it.

원문 보기

ProtoVis-Nav: Prototypical Visual Imagination for Visual-Spatial Aligned UAV Navigation

Paper arXiv cs.CV (recent)

Vision language navigation (VLN) is very challenging for Unmanned Aerial Vehicles (UAVs) since small errors in 3D motion prediction can lead to navigation errors and collisions. Unable to fathom execution outcomes in 3D environments, existing methods often struggle to predict accurate navigation waypoints from language instructions. To solve this, we introduce the Prototypical Visual Imagination framework, shortened as ProtoVis-Nav, which forces the model to imagine an outcome-associated local visual representation for the intended next UAV movement. Specifically, we predict feature prototypes and use their linear combination to estimate the visual feature of the target region. In this manner, the VLN model first imagines the local visual representation of the next intended location, and then uses predicted prototypes as context for generating the final action, promoting alignment between visual context and waypoint outputs. Extensive experiments demonstrate the effectiveness of the ProtoVis-Nav framework and show substantial improvements over existing baselines.

원문 보기

On-Device AI

디바이스 내 추론 및 엣지 최적화 동향

TTB: Test-time Baking for Efficient Rendering of Decoder-only View Synthesis Models

Paper arXiv cs.CV (recent)

Decoder-only Large View Synthesis Models (LVSMs) with KV-cache have recently achieved state-of-the-art quality by regressing novel views from neural networks without reconstructing 3D geometry, while Test-Time Training (TTT) layers save computation but sacrifice quality. We propose Test-Time Baking (TTB), a strategy that learns to bake global scene information from cross-view attention into a lightweight MLP, enabling fast and high-fidelity novel-view decoding. The first key is to repurpose the fast-weight MLP of TTT to learn the cross-view attention mapping explicitly from its input-output pairs, rather than replacing attention entirely, achieving the same rendering efficiency as TTT while delivering substantially higher quality. The second key addresses a modality misalignment between context prefilling and novel-view rendering: we disentangle context tokens into camera-only context rendering tokens to serve as TTB input, and context ground-truth tokens with image content to serve as the TTB mapping target, ensuring the baked MLPs are relieved of the modality generalization challenge between context and novel-view processing. Our method achieves up to **23x** rendering speedup with quality on par with the state-of-the-art across several benchmarks. Project page: https://test-time-baking.github.io

원문 보기

AI 뉴스 & 리서치

기업/연구기관의 주요 발표와 블로그 업데이트

SO-Arena: Benchmarking the Capability Spectrum and Reliability of System One Models

Paper Hugging Face Papers

SO-Arena: Benchmarking the Capability Spectrum and Reliability of System One Models에 관한 최근 업데이트입니다. 자세한 내용은 원문 링크에서 확인할 수 있습니다.

원문 보기

BirdDreams: Joint Perspective and BEV Video Generation for a Plausible, Reactive Driving World Model

Paper Hugging Face Papers

BirdDreams: Joint Perspective and BEV Video Generation for a Plausible, Reactive Driving World Model에 관한 최근 업데이트입니다. 자세한 내용은 원문 링크에서 확인할 수 있습니다.

원문 보기

PhysLoop: Harnessing LLMs' Physics Awareness for Self-Improving Video Generation Models

Paper Hugging Face Papers

PhysLoop: Harnessing LLMs' Physics Awareness for Self-Improving Video Generation Models에 관한 최근 업데이트입니다. 자세한 내용은 원문 링크에서 확인할 수 있습니다.

원문 보기

G-STAR: Group-Scaled Token Capacity for Deeply Compressed Autoregressive Generation

Paper Hugging Face Papers

G-STAR: Group-Scaled Token Capacity for Deeply Compressed Autoregressive Generation에 관한 최근 업데이트입니다. 자세한 내용은 원문 링크에서 확인할 수 있습니다.

원문 보기

Duo: A Reciprocal World Model for Full-Duplex Interaction

Paper Hugging Face Papers

Duo: A Reciprocal World Model for Full-Duplex Interaction에 관한 최근 업데이트입니다. 자세한 내용은 원문 링크에서 확인할 수 있습니다.

원문 보기

Back to Pixels: Lifting Latent Video Diffusion into Pixel Space

Paper arXiv cs.CV (recent)

Frontier video generation models work in a VAE latent space that compresses video in both space and time. This compression made training tractable, but now limits high-resolution, controllable, and interactive generation -- doubling resolution quadruples token count, conditions apply to groups of frames rather than individual ones, and each response must wait for the rest of its multi-frame block. We present PixVid, a simple yet effective method that lifts a pretrained latent video model to directly generate every frame in pixel space. PixVid keeps the diffusion transformer and replaces the VAE decoder with a convolutional head that renders its outputs into pixels. We find that predicting clean video rather than velocity, together with a larger noise shift, is key to quality. Finetuned from Wan2.1-14B, PixVid is competitive with the latent baseline within $\sim 1.3$M training clips. Working framewise in pixels, it generates at $2\times$ higher resolution with the same per-frame token count, keeps depth-conditioned video editing aligned under fast motion, and responds to each action at the next frame. Project page: https://chenshuo20.github.io/PixVid/

원문 보기

K2V: Generative Keyframing For Physically Plausible Video Generation

Paper arXiv cs.CV (recent)

Video diffusion models are able to generate samples with high-quality appearance and textural qualities. However, content produced by these methods is known to exhibit significant physical errors. This is because, in a classical video generation setting, all physical reasoning must take place simultaneously and exclusively within the diffusion process.We propose to resolve these issues by redistributing the task of physical reasoning onto a language model-backed keyframe generation module, which allows for explicit reasoning in language, and fine-grained control over the appearance of the video at important moments in the video. We then condition a video generation model with these keyframes, reducing the role of the diffusion model to the simpler task of interpolating between physically-correct intermediate states, rather than inferring an entire sequence with diffusion. Results demonstrate that our approach significantly outperforms existing approaches, achieving state-of-the-art performance on the Physics-IQ Verified benchmark and PhyGenBench.

원문 보기

November 13, 2026 Advancing science & engineering with Antigravity’s /teamwork Education Innovation · General Science · Machine Intelligence · Natural Language Processing

News Google Research Blog

November 13, 2026 Advancing science & engineering with Antigravity’s /teamwork Education Innovation · General Science · Machine Intelligence · Natural Language Processing에 관한 최근 업데이트입니다. 자세한 내용은 원문 링크에서 확인할 수 있습니다.

원문 보기

November 11, 2026 Using wearables to enable cardiovascular-kidney-metabolic (CKM) risk profiling for personalized coaching Health & Bioscience · Machine Intelligence

News Google Research Blog

November 11, 2026 Using wearables to enable cardiovascular-kidney-metabolic (CKM) risk profiling for personalized coaching Health & Bioscience · Machine Intelligence에 관한 최근 업데이트입니다. 자세한 내용은 원문 링크에서 확인할 수 있습니다.

원문 보기

November 9, 2026 Exploring the world's languages with Language Explorer Natural Language Processing · Open Source Models & Datasets

News Google Research Blog

November 9, 2026 Exploring the world's languages with Language Explorer Natural Language Processing · Open Source Models & Datasets에 관한 최근 업데이트입니다. 자세한 내용은 원문 링크에서 확인할 수 있습니다.

원문 보기

November 5, 2026 Detecting signs of hypertension using wearable sensors Health & Bioscience · Machine Intelligence

News Google Research Blog

November 5, 2026 Detecting signs of hypertension using wearable sensors Health & Bioscience · Machine Intelligence에 관한 최근 업데이트입니다. 자세한 내용은 원문 링크에서 확인할 수 있습니다.

원문 보기

November 4, 2026 Agile Deliberation: How AI helps humans define the gray areas Data Mining & Modeling · Generative AI · Human-Computer Interaction and Visualization · Machine Intelligence

News Google Research Blog

November 4, 2026 Agile Deliberation: How AI helps humans define the gray areas Data Mining & Modeling · Generative AI · Human-Computer Interaction and Visualization · Machine Intelligence에 관한 최근 업데이트입니다. 자세한 내용은 원문 링크에서 확인할 수 있습니다.

원문 보기

Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents

News Microsoft Research Blog

Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents에 관한 최근 업데이트입니다. 자세한 내용은 원문 링크에서 확인할 수 있습니다.

원문 보기

Formally verifying agent-written full-stack web apps with PDD Kit

News Microsoft Research Blog

이 글은 PDD Kit을 활용하여 에이전트가 작성한 풀스택 웹 애플리케이션을 형식적으로 검증하는 방법에 대해 다룹니다. 이를 통해 웹 앱의 신뢰성과 안정성을 높이는 데 기여할 수 있습니다.

원문 보기

From synthetic enterprise data evaluations to real Microsoft Copilot AI confidence

News Microsoft Research Blog

이 글은 합성 기업 데이터 평가를 통해 Microsoft Copilot AI의 실제 신뢰도를 구축하는 과정을 설명합니다. 이를 통해 AI 시스템의 성능과 안정성을 향상시키는 방안을 제시합니다.

원문 보기

참고한 소스