The major
research outcomes of
the EPA
(Explorer of Perception and Attention) lab in recent years include the following
six topics: consciousness, attention, multisensory perception, applied research
on image processing and displays, word recognition, rubber hand illusion and
body ownership.
To understand the processes under the conscious and unconscious conditions, we have used different techniques like motion-induced-blindness, binocular rivalry, bistable figure, inattentional blindness, crowding, and continuous flash suppression to explore issues of the extent of unconscious processing and the nature and function of consciousness. These techniques allow us to block participants’ consciousness while keeping unconscious processing intact.
We first established that motion-induced-blindness and perceptual filling in share common mechanisms and further demonstrated that perceived depth induced by monocular cues affects perceptual fading in dynamic and static stimuli (Hsu, Kramer, & Yeh, 2010). As our visual world consists of multiple objects in various depth relationships, objects during prolonged viewing will undergo competition and one will win the competition and the others (especially those being occluded) might fade away from consciousness.
What can still be processed without consciousness? Word meanings can be extracted unconsciously (Yang & Yeh, 2011; Yeh, He, & Cavanagh, 2012), the one which we now make efforts for further investigation. Human eye gaze can also be processed unconsciously (Chen & Yeh, 2012). Object-based attention is operative even when the object is invisible (Chou & Yeh, 2012) or with a subliminal cue to the object (Chou & Yeh, 2011). In contrast, the Simon effect which involves stimulus-response combinability occurs only with consciousness (Lo & Yeh, 2011). This reveals the dissociation between processing time and consciousness, consistent with the possibility of separating processing streams for explicit and implicit visual perception (Lo & Yeh, 2011).
We have published several papers regarding object-based attention. Using a changed-object display modified from the double-rectangle paradigm, we found that instantaneous object inputs and past experience interact to determine the way attention selects objects (Ho & Yeh, 2009). We further demonstrated that object-based attention is an interactively evolving process between object representation and attention, rather than a serial process in which attention operates after object representation is completed (Lin & Yeh, 2011). The updating process is fragile since any disruption of the local change will break this process but it is quite tolerable to global changes, perhaps due to small and large receptive fields respectively in the hierarchy of cortical areas responsible for representation of objects (Lin & Yeh, 2012).
As to the hotly debated issue regarding what kind of stimulus captures attention, we have provided empirical tests that argue against either the stimulus-driven capture view or the contingent capture view (Jingling, Hsiao, & Yeh, 2012; Yeh & Liao, 2010; Liao & Yeh, 2011). A two-pathway hypothesis has emerged from these studies, and two theoretical papers were published (Liao & Yeh, 2013; Chou, Yeh, & Chen, 2014). One side track is the attention related emotion process, in which we found the interaction of familiarity, preference, and emotional valence when full attention is involved in the preference judgment (Liao, Shimojo, & Yeh, 2013).
We receive inputs from various sensory modalities and multisensory perception has become as important, if not more so, as unisensory perception. From previous works, we have found that an event containing multisensory information is easier to be perceived than unisensory ones. A suprathreshold sound would facilitate a weak visual event to reach consciousness and this sound-facilitatory effect is due to multisensory enhancement of the perception of visual events by auditory signals, rather than multisensory Gestalt (perceptual grouping) of a new representation formed by combining the visual and auditory inputs (Chen & Yeh, 2009). Whether the occurrence of the auditory and visual stimuli in the same space is critical for the sound-facilitation effect on visual processing depends on the reliability of the auditory information (Yang & Yeh, 2014). Without awareness, a suprathreshold auditory signal cannot affect the subliminal visual Simon effect (Lo & Yeh, 2011). Furthermore, auditory semantic congruency provides a constraint so that an interocularly suppressed visual stimulus is more likely to release from suppression if it is semantically congruent with the context that the sound provides (Chen, Yeh, & Spence, 2011; Hsiao, Chen, Spence, & Yeh, 2012). Using a noise paradigm to tease apart the mechanism of signal enhancement and uncertainty reduction, we found that the facilitation effect of sound on visual detection and discrimination is due to signal enhancement, rather than uncertainty reduction (Chen, Huang, Yeh, & Spence, 2011).
We also explored the cross-modal effect in time perception, and found asymmetric effects: a sound extends the perceived visual duration, whereas a visual object has no effect on perceived auditory duration. By carefully examining the mechanism, we infer that auditory signal increases the perceived visual duration by accelerating the pulse rate in the visual pacemaker (Chen and Yeh, 2009). High impulsive people tend to have higher threshold (i.e., larger JND) than low impulsive people in their time perception (Tsai & Yeh, 2014). We also found auditory perceptual grouping facilitates tactile identification, similar to the freezing phenomenon in audito-visual interaction. However, it is caused by intensity enhancement rather than subjective time expansion (Tsai & Yeh, 2013).
Applied Research on Image Processing and Displays
We have joined the project of NTU-Himax perceptual video lab, leading by Professor Liang-Gee Chen (陳良基) at the Department of Electrical Engineering and our vice-president at NTU. As the only human vision and attention expert in this research group, Prof. Yeh serves as a consultant on display designs and applications, and conduct related studies on human perception and attention, focusing on innovative ideas for new algorisms, experimental designs, and product evaluations. We have published several conference papers and journal papers. These include the evaluation of 3D displays design based on human perception, such as the effect of power-saving of low backlight displays on human vision (Huang, Yeh, & Chen, 2011; Huang, Shih, Yeh, & Chen, 2013), a closed-form model of user perception for rate changes for Skype calls (Chen, Chu, Yeh, Chu, & Huang, 2012; Chen, Chu, Yeh, Chu, & Huang, 2013), a model to predict attention allocation (Lee, Huang, Yeh, & Chen, 2011; Lee, Shaw, Liao, Yeh, Chen, 2011 and own five patents: Huang, Lin, Yeh, Chen, Bu, &, Huang, 2010; Chen, Huang, Lin, Yeh, Bu, & Huang, 2012; Huang, Lin, Yeh, Chen, Bu, & Huang, 2013; Chen, Yeh, Huang, Yang, Liao, & Huang, 2013; Chen, Yeh, Huang, Li, & Huang, 2013), and the relationship between display refresh rate and motion artifacts (Wang, Liu, Chien, Yang, Chen, & Yeh, 2009).
To provide a categorization basis of character structure empirically, we asked skilled Chinese readers to classify samples of Chinese characters according to their shapes. A hierarchical cluster analysis revealed five major clusters in terms of character structure: [H, L] [V, (P, E)], where H, L, V, P, E stands for Horizontal (e.g., 印), L-shaped (e.g.,建), Vertical (e.g.,念), P-shaped (e.g., 屋 ), and Enclosed (e.g., 困) structures, respectively (Yeh, et al., 1997, 1999, 2003). Following this, the effect of learning experience on the structure categorization of Chinese characters was further examined by recruiting various groups of participants with different learning experiences and ages. Our results showed that Japanese undergraduates who were familiar with Kanji (Chinese characters used in Japanese script) produced the same pattern as skilled Chinese readers, both classifying characters in relation to their structures. However, American undergraduates, Taiwanese illiterate adults and kindergartners categorized characters based on strokes or components. Although there was a trend of developmental changes from local details to more globally defined patterns, the identification of structure has to be nourished by learning experience and cannot be obtained solely through maturation (Yeh et al., 2003). Such learning experiences have to extend for years since school children up to the fifth grade still cannot perceive the character structure as stable as adults do (Yeh, Lin, & Li, 2004).
To see whether these structure categories are also perceived and processed in tasks other than shape sorting, we then used a visual search paradigm (Yeh, 2000), in which no explicit request for shape comparison was made. The participants were asked to search for a unique, different character or a pre-specified target character within a homogenous field consisting of multiples of another character. The slope of the function relating RT (reaction time) to set size (i.e., total number of stimulus items) was found shallower when a target character was different in structure from the distractors, than when both the target and the distractors had the same structure. Moreover, different slope values were obtained for the 10 target-distractor pairs formed by the five major structure types (Yeh, 2000), in accordance with the explicitly rated similarity relationships between differently-structured characters in the earlier two reports (Yeh, et al., 1997, 1999). In addition, the relative role of structure and component in similarity judgments of Chinese characters was examined by providing conflicting information of structure and component. Our results showed that skilled readers still sorted characters based on their structures, and yet in visual search both structure and component seemed to play important roles, with the semantic component correlated with the structure while the phonetic component being relatively independent of the character structure (Yeh & Li, 2002).
These studies have revealed an important aspect of Chinese characters that has long been overlooked: their structure. A character's structure is formed by arranging various components in appropriate positions relative to one another, thus the relative position of different components would have to be perceived in order for a reader to access structural information. We used the repetition blindness (RB) paradigm to provide converging evidence for the processing of radicals and characters (Yeh & Li, 2004). RB refers to the failure to detect the second occurrence of a repeated item in rapid serial visual presentation. We found RB for two repeated characters, as well as RB for two different characters sharing one repeated component. The time course for the component RB seems to be earlier than the character RB, indicating sublexical processing in Chinese character recognition (Yeh & Li, 2004). Radicals may be the processing units in reading Chinese (Chen & Yeh, 2009).
Rubber
Hand
Illusion
and
Body
Ownership