Show Me Examples: Inferring Visual Concepts from Image Sets

Apple ML Research · 1 day ago

Researchers introduced VICIS, a task that evaluates whether vision-language models can infer shared visual concepts from sets of example images and apply them to new inputs. State-of-the-art VLMs performed poorly on this benchmark, often ignoring visual context or producing biased outputs. The authors developed a training framework and architecture that learns to infer visual concepts from image sets, demonstrating improved accuracy and generalization to unseen concepts and modalities like sketches on ImageNet and WordNet data.