Publication Title

Journal of Neuroscience

Document Type

Article

Department or Program

Neuroscience

Publication Date

7-1-2020

Keywords

Decoding, EEG, Encoding, Natural scenes

Abstract

Human scene categorization is characterized by its remarkable speed. While many visual and conceptual features have been linked to this ability, significant correlations exist between feature spaces, impeding our ability to determine their relative contributions to scene categorization. Here, we used a whitening transformation to decorrelate a variety of visual and conceptual features and assess the time course of their unique contributions to scene categorization. Participants (both sexes) viewed 2250 full-color scene images drawn from 30 different scene categories while having their brain activity measured through 256-channel EEG. We examined the variance explained at each electrode and time point of visual event-related potential (vERP) data from nine different whitened encoding models. These ranged from low-level features obtained from filter outputs to high-level conceptual features requiring human annotation. The amount of category information in the vERPs was assessed through multivariate decoding methods. Behavioral similarity measures were obtained in separate crowdsourced experiments. We found that all nine models together contributed 78% of the variance of human scene similarity assessments and were within the noise ceiling of the vERP data. Low-level models explained earlier vERP variability (88 ms after image onset), whereas high-level models explained later variance (169 ms). Critically, only high-level models shared vERP variability with behavior. Together, these results suggest that scene categorization is primarily a high-level process, but reliant on previously extracted low-level features.

PubMed ID

32467356

Copyright Note

This is the publisher's version of the work. This publication appears in Bates College's institutional repository by permission of the copyright owner for personal use, not for redistribution.

Required Publisher's Statement

Original version is available from the publisher at: https://doi.org/10.1523/JNEUROSCI.2088-19.2020

Download

Share

COinS