Combining brain computer interfaces with vision for object categorization

Human-aided computing proposes using information measured directly from the human brain in order to perform useful tasks. In this paper, we extend this idea by fusing computer vision-based processing and processing done by the human brain in order to build more effective object categorization systems. Specifically, we use an electroencephalograph (EEG) device to measure the subconscious cognitive processing that occurs in the brain as users see images, even when they are not trying to explicitly classify them. We present a novel framework that combines a discriminative visual category recognition system based on the Pyramid Match Kernel (PMK) with information derived from EEG measurements as users view images. We propose a fast convex kernel alignment algorithm to effectively combine the two sources of information. Our approach is validated with experiments using real-world data, where we show significant gains in classification accuracy. We analyze the properties of this information ...

• Type: Conference
• Year: 2008
• Where: CVPR
• Authors: Ashish Kapoor, Pradeep Shenoy, Desney S. Tan
• Keywords: Computer Vision, Convex Kernel Alignment, CVPR 2008, Discriminative Visual Category, Human Brain, Pyramid Match Kernel, Subconscious Cognitive Processing