University of Illinois Urbana-Champaign

The influence of semantics on the visual processing of natural scenes

Kumar, Manoj

Permalink

https://hdl.handle.net/2142/98123

Description

Title
The influence of semantics on the visual processing of natural scenes
Author(s)
Kumar, Manoj
Issue Date
2017-06-28
Director of Research (if dissertation) or Advisor (if thesis)
Beck, Diane M.
Doctoral Committee Chair(s)
Beck, Diane M.
Committee Member(s)
  • Federmeier, Kara D.
  • Gratton, Gabriele
  • Koyejo, Oluwasanmi
Department of Study
Neuroscience Program
Discipline
Neuroscience
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Keyword(s)
  • Natural scenes
  • Semantics
  • Statistical regularities
  • Multi-voxel pattern analysis (MVPA)
  • Era
Abstract
A long standing question in cognitive science has been: is visual processing completely encapsulated and separate from semantics or can visual processing be influenced by semantics? We address this question in two ways: 1) Do pictures and words share similar representations and 2) Does semantics modulate visual processing. Using multi-voxel pattern analysis (MVPA) and fMRI decoding we examined the similarity of neural activity across pictures and words that describe natural scenes. A whole brain MVPA searchlight revealed multiple brain regions in the occipitotemporal, posterior parietal and frontal cortices that showed transfer from pictures to words and from words to pictures. In addition to sharing similar representations across pictures and words, can words dynamically influence the processing of visual stimuli? Using Event Related Potentials (ERPs) and good and bad exemplars of natural scenes, we show that top-down expectation, initiated via a category cue (e.g. the word ‘Beach’), dynamically influences the processing of natural scenes. Good and bad exemplars first evoked differential ERPs in the time-window 250-350 ms from stimulus onset, with the bad exemplars showing greater negativity over frontal electrode sites, when the cue matched the image. Interestingly, this good/bad effect disappeared when the images were mismatched to the cue. Overall, these studies taken together, provide evidence for the influence of semantics on the visual processing of natural scenes.
Graduation Semester
2017-08
Type of Resource
text
Permalink
http://hdl.handle.net/2142/98123
Copyright and License Information
Copyright 2017 Manoj Kumar

Owning Collections

Dissertations - Neuroscience
Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois