Know-Thy-Not: Training-Free Targeted Negation in Dual-Encoder Vision-Language Models

2026-10-15 · ArXiv

arXiv:2610.17440v1 Announce Type: cross

Abstract: Dual-encoder vision-language models systematically fail at negation, scoring ''a cat without a dog'' nearly identically to ''a cat with a dog.'' We show that this failure is inherent to the single-embedding framework. Under contrastive training, single-embedding methods can at best ignore the negated concept (dilution) but never penalize its presence (rejection). We propose Know-Thy-Not (KTN), a training-free method that introduces an image-side rejection dimension to penalize the negated concept directly in the visual representation. We also introduce COCO-Confusion, the first benchmark requiring genuine rejection to succeed. Across 9 backbones, KTN improves COCO-Confusion mAP by up to +25.6 over vanilla VLMs and +9.6 over the strongest existing method.

原文链接：https://arxiv.org/abs/2610.17440

查看更多资讯请见 AI 行业资讯。