Beyond Color Geometry: Evaluating Human-Like Color Representations in Vision Models

arXiv cs.AI · 18 hours ago

Researchers evaluated whether vision models represent colors similarly to humans by testing eleven Vision Transformer encoders against a fuzzy perceptual model with 86 graded color categories derived from human survey data. Masked Autoencoders achieved the strongest alignment with human color organization beyond geometric color spaces like CIELAB, with non-overlapping confidence intervals compared to other encoders. The findings reveal that different vision models encode color distinctly—MAE represents surface color globally while language-supervised models tie color more strongly to foreground objects—demonstrating that human-like color representation has multiple measurable aspects.