Jeesun Kim

Publications of Jeesun Kim

• Common and distinct mechanisms associated with view-specific and view-invariant recognition.

  Authors: Bronson Harry, Chris Davis, Jeesun Kim

  Consciousness and cognition. 04/2012;

• Effects of seeing the interlocutor on the production of prosodic contrasts (L).

  Authors: Erin Cvejic, Jeesun Kim, Chris Davis

  The Journal of the Acoustical Society of America. 02/2012; 131(2):1011-4.

  This study investigated whether the production of prosodic focus and phrasing contrasts was modified when interlocutors could only hear each other [auditory only (AO)], compared to when they couldThis study investigated whether the production of prosodic focus and phrasing contrasts was modified when interlocutors could only hear each other [auditory only (AO)], compared to when they could hear and see each other [face to face (FTF)]. The prosodic characteristics of utterances produced by six talkers were examined using both acoustic and perceptual measures (ratings of the degree of focus or clarity of the statement-question contrast). The acoustic measures showed a range of differences between narrow focus and between phrasing contrasts and some of these differences were greater in the AO setting than the FTF one. The listener's ratings of focus and phrasing showed a clear difference between the AO and FTF conditions, with perceptual attributes of both narrow focus and echoic question phrasing being rated as clearer in the AO condition. To explain these results it is proposed that talkers compensate for the lack of visual prosodic cues in the AO condition by taking extra care (relative to FTF conditions) to ensure the effective transmission of prosodic cues.

• Exposure in central vision facilitates view-invariant face recognition in the periphery.

  Authors: Bronson Harry, Chris Davis, Jeesun Kim

  Journal of vision. 01/2012; 12(2).

  The present study investigated the extent to which a face presented in the visual periphery is processed and whether such processing can be influenced by a recent encounter in central vision. ToThe present study investigated the extent to which a face presented in the visual periphery is processed and whether such processing can be influenced by a recent encounter in central vision. To probe face processing, a series of studies was conducted in which participants classified the sex and identity of faces presented in central and peripheral vision. The results showed that when target faces had not been previously viewed in central vision, recognition in peripheral vision was limited whereas sex categorization was not. When faces were previously viewed in central vision, recognition in peripheral vision improved even with the pose, hairstyle, and lighting conditions of these faces changed. These results are discussed with regard to possible mechanisms unpinning this exposure effect.

• Subliminal access to abstract face representations does not rely on attention.

  Authors: Bronson Harry, Chris Davis, Jeesun Kim

  Consciousness and cognition. 12/2011; 21(1):573-83.

  The present study used masked repetition priming to examine whether face representations can be accessed without attention. Two experiments using a face recognition task (fame judgement) presentedThe present study used masked repetition priming to examine whether face representations can be accessed without attention. Two experiments using a face recognition task (fame judgement) presented masked repetition and control primes in spatially unattended locations prior to target onset. Experiment 1 (n=20) used the same images as primes and as targets and Experiment 2 (n=17) used different images of the same individual as primes and targets. Repetition priming was observed across both experiments regardless of whether spatial attention was cued to the location of the prime. Priming occurred for both famous and non-famous targets in Experiment 1 but was only reliable for famous targets in Experiment 2, suggesting that priming in Experiment 1 indexed access to view-specific representations whereas priming in Experiment 2 indexed access to view-invariant, abstract representations. Overall, the results indicate that subliminal access to abstract face representations does not rely on attention.

• Recognizing prosody across modalities, face areas and speakers: examining perceivers' sensitivity to variable realizations of visual prosody.

  Authors: Erin Cvejic, Jeesun Kim, Chris Davis

  Cognition. 12/2011; 122(3):442-53.

  Prosody can be expressed not only by modification to the timing, stress and intonation of auditory speech but also by modifying visual speech. Studies have shown that the production of visual cues toProsody can be expressed not only by modification to the timing, stress and intonation of auditory speech but also by modifying visual speech. Studies have shown that the production of visual cues to prosody is highly variable (both within and across speakers), however behavioural studies have shown that perceivers can effectively use such visual cues. The latter result suggests that people are sensitive to the type of prosody expressed despite cue variability. The current study investigated the extent to which perceivers can match visual cues to prosody from different speakers and from different face regions. Participants were presented two pairs of sentences (consisting of the same segmental content) and were required to decide which pair had the same prosody. Experiment 1 tested visual and auditory cues from the same speaker and Experiment 2 from different speakers. Experiment 3 used visual cues from the upper and the lower face of the same talker and Experiment 4 from different speakers. The results showed that perceivers could accurately match prosody even when signals were produced by different speakers. Furthermore, perceivers were able to match the prosodic cues both within and across modalities regardless of the face area presented. This ability to match prosody from very different visual cues suggests that perceivers cope with variation in the production of visual prosody by flexibly mapping specific tokens to abstract prosodic types.

• What's in a mask? Information masking with forward and backward visual masks.

  Authors: Chris Davis, Jeesun Kim

  Quarterly journal of experimental psychology (2006). 03/2011; 64(10):1990-2002.

  Three experiments tested how the physical format and information content of forward and backward masks affected the extent of visual pattern masking. This involved using different types of forwardThree experiments tested how the physical format and information content of forward and backward masks affected the extent of visual pattern masking. This involved using different types of forward and backward masks with target discrimination measured by percentage correct in the first experiment (with a fixed target duration) and by an adaptive threshold procedure in the last two. The rationale behind the manipulation of the content of the masks stemmed from masking theories emphasizing attentional and/or conceptual factors rather than visual ones. Experiment 1 used word masks and showed that masking was reduced (a masking reduction effect) when the forward and backward masks were the same word (although in different case) compared to when the masks were different words. Experiment 2 tested the extent to which a reduction in masking might occur due to the physical similarity between the forward and backward masks by comparing the effect of the same content of the masks in the same versus different case. The result showed a significant reduction in masking for same content masks but no significant effect of case. The last experiment examined whether the reduction in masking effect would be observed with nonword masks--that is, having no high-level representation. No reduction in masking was found from same compared to different nonword masks (Experiment 3). These results support the view that the conscious perception of a rapidly displayed target stimulus is in part determined by high-level perceptual/cognitive factors concerned with masking stimulus grouping and attention.

• Temporal Relationship Between Auditory and Visual Prosodic Cues.

  Authors: Erin Cvejic, Jeesun Kim, Chris Davis

  INTERSPEECH 2011, 12th Annual Conference of the International Speech Communication Association, Florence, Italy, August 27-31, 2011; 01/2011

• Auditory Speech Processing is Affected by Visual Speech in the Periphery.

  Authors: Jeesun Kim, Chris Davis

  INTERSPEECH 2011, 12th Annual Conference of the International Speech Communication Association, Florence, Italy, August 27-31, 2011; 01/2011

• Visual Speech Speeds Up Auditory Identification Responses.

  Authors: Tim Paris, Jeesun Kim, Chris Davis

  INTERSPEECH 2011, 12th Annual Conference of the International Speech Communication Association, Florence, Italy, August 27-31, 2011; 01/2011

• The Effect of Seeing the Interlocutor on Speech Production in Different Noise Types.

  Authors: Michael Fitzpatrick, Jeesun Kim, Chris Davis

  INTERSPEECH 2011, 12th Annual Conference of the International Speech Communication Association, Florence, Italy, August 27-31, 2011; 01/2011

• Hearing speech in noise: seeing a loud talker is better.

  Authors: Jeesun Kim, Amanda Sironic, Chris Davis

  Perception. 01/2011; 40(7):853-62.

  Seeing the talker improves the intelligibility of speech degraded by noise (a visual speech benefit). Given that talkers exaggerate spoken articulation in noise, this set of two experiments examinedSeeing the talker improves the intelligibility of speech degraded by noise (a visual speech benefit). Given that talkers exaggerate spoken articulation in noise, this set of two experiments examined whether the visual speech benefit was greater for speech produced in noise than in quiet. We first examined the extent to which spoken articulation was exaggerated in noise by measuring the motion of face markers as four people uttered 10 sentences either in quiet or in babble-speech noise (these renditions were also filmed). The tracking results showed that articulated motion in speech produced in noise was greater than that produced in quiet and was more highly correlated with speech acoustics. Speech intelligibility was tested in a second experiment using a speech-perception-in-noise task under auditory-visual and auditory-only conditions. The results showed that the visual speech benefit was greater for speech recorded in noise than for speech recorded in quiet. Furthermore, the amount of articulatory movement was related to performance on the perception task, indicating that the enhanced gestures made when speaking in noise function to make speech more intelligible.

• Are tones phones?

  Authors: Denis Burnham, Jeesun Kim, Chris Davis, Valter Ciocca, Colin Schoknecht, Benjawan Kasisopa, Sudaporn Luksaneeyanawin

  Journal of experimental child psychology. 11/2010; 108(4):693-712.

  The psycholinguistic status of lexical tones and phones is indexed via phonological and tonological awareness (PA and TA, respectively) using Thai speech. In Experiment 1 (Thai participants,The psycholinguistic status of lexical tones and phones is indexed via phonological and tonological awareness (PA and TA, respectively) using Thai speech. In Experiment 1 (Thai participants, alphabetic script and orthographically explicit phones/tones), PA was better than TA in children and primary school-educated adults, and TA improved to PA levels only in tertiary-educated adults. In Experiment 2 (Cantonese participants, logographic script and no orthographically explicit phones/tones), children and primary-educated adults had better PA than TA, and PA and TA were equivalent in tertiary-educated adults, but were nevertheless still below the level of their Thai counterparts. Experiment 3 (English-language participants, alphabetic script and nontonal) showed better PA than TA. Regression analyses showed that both TA and PA are predicted by reading ability for Thai children but by general nonorthographic age-related variables for Cantonese children, whereas for English children reading ability predicts PA but not TA. The results show a phone>tone perceptual advantage over both age and languages that is affected by availability of orthographically relevant information and metalinguistic maturity. More generally, both the perception and the psycholinguistic representation of phones and tones differ.

• Masked speech priming: neighborhood size matters.

  Authors: Chris Davis, Jeesun Kim, Angelo Barbaro

  The Journal of the Acoustical Society of America. 04/2010; 127(4):2110-3.

  The current study investigated the robustness of priming from a masked speech priming method introduced by Kouider and Dupoux [(2005). Psychol. Sci. 16, 617-625]. In this procedure, a compressedThe current study investigated the robustness of priming from a masked speech priming method introduced by Kouider and Dupoux [(2005). Psychol. Sci. 16, 617-625]. In this procedure, a compressed spoken prime is embedded in auditory masking stimuli and presented immediately prior to an uncompressed auditory target. The degree to which spoken stimuli could be compressed without significant data loss was first determined. Using this compression level, repetition and form priming were measured for the target words with a high versus low number of phonological neighbors. The results indicated that robust masked speech priming occurred only for word targets that had few neighbors.

• Prosody for the eyes: quantifying visual prosody using guided principal component analysis.

  Authors: Erin Cvejic, Jeesun Kim, Chris Davis, Guillaume Gibert

  INTERSPEECH 2010, 11th Annual Conference of the International Speech Communication Association, Makuhari, Chiba, Japan, September 26-30, 2010; 01/2010

• Hearing a point-light talker: an auditory influence on a visual motion detection task.

  Authors: Jeesun Kim, Christian Kroos, Chris Davis

  Perception. 01/2010; 39(3):407-16.

  Parsing of information from the world into objects and events occurs in both the visual and auditory modalities. It has been suggested that visual and auditory scene perceptions involve similarParsing of information from the world into objects and events occurs in both the visual and auditory modalities. It has been suggested that visual and auditory scene perceptions involve similar principles of perceptual organisation. We investigated here cross-modal scene perception by determining whether an auditory stimulus could facilitate visual object segregation. Specifically, we examined whether the presentation of matched auditory speech would facilitate the detection of a point-light talking face amid point-light distractors. An adaptive staircase procedure (3-up 1-down rule) was used to estimate the 79% correct threshold in a two-alternative forced-choice procedure. To determine if different degrees of speech motion would show auditory influence of different sizes, two speech modes were tested (in quiet and Lombard speech). A facilitatory auditory effect on talking-face detection was found; the size of this effect did not differ between the different speech modes.

• Speech identification in noise: Contribution of temporal, spectral, and visual speech cues.

  Authors: Jeesun Kim, Chris Davis, Christopher Groot

  The Journal of the Acoustical Society of America. 12/2009; 126(6):3246.

  This study investigated the degree to which two types of reduced auditory signals (cochlear implant simulations) and visual speech cues combined for speech identification. The auditory speech stimuliThis study investigated the degree to which two types of reduced auditory signals (cochlear implant simulations) and visual speech cues combined for speech identification. The auditory speech stimuli were filtered to have only amplitude envelope cues or both amplitude envelope and spectral cues and were presented with/without visual speech. In Experiment 1, IEEE sentences were presented in quiet and noise. For in-quiet presentation, speech identification was enhanced by the addition of both spectral and visual speech cues. Due to a ceiling effect, the degree to which these effects combined could not be determined. In noise, these facilitation effects were more marked and were additive. Experiment 2 examined consonant and vowel identification in the context of CVC or VCV syllables presented in noise. For consonants, both spectral and visual speech cues facilitated identification and these effects were additive. For vowels, the effect of combined cues was underadditive, with the effect of spectral cues reduced when presented with visual speech cues. Analysis indicated that without visual speech, spectral cues facilitated the transmission of place information and vowel height, whereas with visual speech, they facilitated lip rounding, with little impact on the transmission of place information.

• Speaker discriminability for visual speech modes.

  Authors: Jeesun Kim, Chris Davis, Christian Kroos, Harold Hill

  INTERSPEECH 2009, 10th Annual Conference of the International Speech Communication Association, Brighton, United Kingdom, September 6-10, 2009; 01/2009

• The effect of viewing speech on auditory speech processing is different in the left and right hemispheres.

  Authors: Chris Davis, Daniel Kislyuk, Jeesun Kim, Mikko Sams

  Brain research. 06/2008;

  We used whole-head magnetoencephalograpy (MEG) to record changes in neuromagnetic N100m responses generated in the left and right auditory cortex as a function of the match between visual andWe used whole-head magnetoencephalograpy (MEG) to record changes in neuromagnetic N100m responses generated in the left and right auditory cortex as a function of the match between visual and auditory speech signals. Stimuli were auditory-only (AO) and auditory-visual (AV) presentations of /pi/, /ti/ and /vi/. Three types of intensity matched auditory stimuli were used: intact speech (Normal), frequency band filtered speech (Band) and speech-shaped white noise (Noise). The behavioural task was to detect the /vi/ syllables which comprised 12% of stimuli. N100m responses were measured to averaged /pi/ and /ti/ stimuli. Behavioural data showed that identification of the stimuli was faster and more accurate for Normal than for Band stimuli, and for Band than for Noise stimuli. Reaction times were faster for AV than AO stimuli. MEG data showed that in the left hemisphere, N100m to both AO and AV stimuli was largest for the Normal, smaller for Band and smallest for Noise stimuli. In the right hemisphere, Normal and Band AO stimuli elicited N100m responses of quite similar amplitudes, but N100m amplitude to Noise was about half of that. There was a reduction in N100m for the AV compared to the AO conditions. The size of this reduction for each stimulus type was same in the left hemisphere but graded in the right (being largest to the Normal, smaller to the Band and smallest to the Noise stimuli). The N100m decrease for the Normal stimuli was significantly larger in the right than in the left hemisphere. We suggest that the effect of processing visual speech seen in the right hemisphere likely reflects suppression of the auditory response based on AV cues for place of articulation.

• Being forward not backward: lexical limits to masked priming.

  Authors: Chris Davis, Jeesun Kim, Kenneth I Forster

  Cognition. 06/2008; 107(2):673-84.

  This study investigated whether masked priming is mediated by existing memory representations by determining whether nonwords targets would show repetition priming. To avoid the potential confoundThis study investigated whether masked priming is mediated by existing memory representations by determining whether nonwords targets would show repetition priming. To avoid the potential confound that nonword repetition priming would be obscured by a familiarity response bias, the standard lexical decision and naming tasks were modified to make targets unfamiliar. Participants were required to read a target string from right to left (i.e., "ECAF" should be read as "FACE") and then make a response. To examine if priming was based on lexical representations, repetition primes consisted of words when read forwards or backwards (e.g., "face", "ecaf") and nonwords (e.g., "pame", "emap"). Forward and backward primes were used to test if task instruction affected prime encoding. The lexical decision and naming tasks showed the same pattern of results: priming only occurred for forward primes with word targets (e.g., "face-ECAF"). Additional experiments to test if response priming affected the LDT indicated that the lexical status of the prime per se did not affect target responses. These results showed that the encoding of masked primes was unaffected by the novel task instruction and support the view that masked priming is due to the automatic triggering of pre-established computational processes based on stored information.

• Perceptual tests of rhythmic similarity: II. Syllable rhythm.

  Authors: Jeesun Kim, Chris Davis, Anne Cutler

  Language and speech. 02/2008; 51(Pt 4):343-59.

  To segment continuous speech into its component words, listeners make use of language rhythm; because rhythm differs across languages, so do the segmentation procedures which listeners use. For eachTo segment continuous speech into its component words, listeners make use of language rhythm; because rhythm differs across languages, so do the segmentation procedures which listeners use. For each of stress-, syllable- and mora-based rhythmic structure, perceptual experiments have led to the discovery of corresponding segmentation procedures. In the case of mora-based rhythm, similar segmentation has been demonstrated in the otherwise unrelated languages Japanese and Telugu; segmentation based on syllable rhythm, however, has been previously demonstrated only for European languages from the Romance family. We here report two target detection experiments in which Korean listeners, presented with speech in Korean and in French, displayed patterns of segmentation like those previously observed in analogous experiments with French listeners. The Korean listeners' accuracy in detecting word-initial target fragments in either language was significantly higher when the fragments corresponded exactly to a syllable in the input than when the fragments were smaller or larger than a syllable. We conclude that Korean and French listeners can call on similar procedures for segmenting speech, and we further propose that perceptual tests of speech segmentation provide a valuable accompaniment to acoustic analyses for establishing languages' rhythmic class membership.