Masato Hirano’s research while affiliated with Sony Computer Science Laboratories and other places

Complex motor skills involve intricate sequences of movements that require precise temporal coordination across multiple body parts, posing challenges to mastery based on perceived error or reward. One approach that has been widely used is to decompose such skills into simpler, constituent movement elements during the learning process, thereby aligning the task complexity with the learners’ capacity for accurate execution. Despite common belief and prevalent adoption, the effectiveness of this method remains elusive. Here we addressed this issue by decomposing a sequence of precisely timed coordination of movements across multiple fingers into individual constituent elements separately during piano practice. The results demonstrated that the decomposition training enhanced the accuracy of the original motor skill, a benefit not achieved through mere repetition of movements alone, specifically when skilled pianists received explicit visual feedback on timing error in the order of milliseconds during training. During the training, the patterns of multi-finger movements changed significantly, suggesting exploration of movements to refine the skill. By contrast, neither unskilled pianists who underwent the same training nor skilled pianists who performed the decomposition training without receiving visual feedback on the error showed improved skill through training. These findings offer novel evidences suggesting that decomposing a complex motor skill, coupled with receiving feedback on subtle movement error during training, further enhances motor expertise of skilled individuals by facilitating exploratory refinement of movements.

For trained individuals such as athletes and musicians, learning often plateaus after extensive training, known as the “ceiling effect.” One bottleneck to overcome it is having no prior physical experience with the skill to be learned. Here, we challenge this issue by exposing expert pianists to fast and complex finger movements that cannot be performed voluntarily, using a hand exoskeleton robot that can move individual fingers quickly and independently. Although the skill of moving the fingers quickly plateaued through weeks of piano practice, passive exposure to otherwise impossible complex finger movements generated by the exoskeleton robot at a speed faster than the pianists’ fastest one enabled them to play faster. Neither a training for fast but simple finger movements nor one for slow but complex movements with the exoskeleton enhanced the overtrained motor skill. The exoskeleton training with one hand also improved the motor skill of the untrained contralateral hand, demonstrating the intermanual transfer effect. The training altered patterns of coordinated activities across multiple finger muscles during piano playing but not in general motor and somatosensory functions or in anatomical characteristics of the hand (range of motion). Patterns of the multifinger movements evoked by transcranial magnetic stimulation over the left motor cortex were also changed through passive exposure to fast and complex finger movements, which accompanied increased involvement of constituent movement elements characterizing the individuated finger movements. The results demonstrate evidence that somatosensory exposure to an unexperienced motor skill allows surmounting of the ceiling effect in a task-specific but effector-independent manner.

Dexterous motor skills, like those needed for playing musical instruments and sports, require the somatosensory system to accurately and rapidly process somatosensory information from multiple body parts. This is challenging due to the convergence of afferent inputs from different body parts into a single neuron and the overlapping representation of neighboring body parts in the somatosensory cortices. How do trained individuals, such as pianists and athletes, manage this? Here, a series of five experiments with pianists and nonmusicians (female and male) shows that pianists have enhanced inhibitory function in the somatosensory system, which isolates the processing of somatosensory afferent inputs from each finger. This inhibitory function was assessed using a paired-pulse paradigm of somatosensory evoked potentials in electroencephalography, which measures the suppressive effect of a first stimulus [i.e., conditioning stimulus (CS)] on the response to a subsequent second stimulus. We found that pianists and nonmusicians showed an inhibitory response to the sequential stimuli to the peripheral somatosensory nerve at the wrist when the CS was intense. However, only pianists exhibited an inhibitory response to a weak CS, indicating enhanced inhibitory function in pianists. Additionally, the CS increased the information content segregating individual fingers represented in the cortical activity evoked by passive finger movements and improved the perception of fast multifinger sequential movements, specifically for pianists. Our findings provide the first evidence for experience-dependent plasticity in somatosensory inhibitory function and highlight its role in the expert motor performance of pianists.

Complex motor skills involve intricate sequences of movements that require precise temporal coordination across multiple body parts, posing challenges to mastery based on perceived error or reward. One approach that has been widely used is to decompose such skills into simpler, constituent movement elements during the learning process, thereby aligning the task complexity with the learners' capacity for accurate execution. Despite common belief and prevalent adoption, the effectiveness of this method remains elusive. Here we addressed this issue by decomposing a sequence of precisely timed coordination of movements across multiple fingers into individual constituent elements separately during piano practice. The results demonstrated that the decomposition training enhanced the accuracy of the original motor skill, a benefit not achieved through mere repetition of movements alone, specifically when skilled pianists received explicit visual feedback on timing error in the order of milliseconds during training. During the training, the patterns of multi-finger movements changed significantly, suggesting exploration of movements to refine the skill. By contrast, neither unskilled pianists who underwent the same training nor skilled pianists who performed the decomposition training without receiving visual feedback on the error showed improved skill through training. These findings offer novel evidences suggesting that decomposing a complex motor skill, coupled with receiving feedback on subtle movement error during training, further enhances motor expertise of skilled individuals by facilitating exploratory refinement of movements.

Accurate control of fast movements of multiple body parts characterizes experts' skills, such as playing musical instruments. While performing these movements, the somatosensory system is challenged to successively and in parallel process a large amount of somatosensory information originating from different body parts in a short period. Here, we show that pianists possess a unique inhibitory function that isolates the somatosensory processing of different fingers in the somatosensory cortex. Weak electrical stimulation to the ulnar nerve successfully augmented this inhibitory function, which also improved both the perception and production of fast and complex multifinger sequential movements in pianists. In nonmusicians, neither the inhibitory effects on the somatosensory process nor the perception of multifinger movements was enhanced by this stimulation. Together, these findings provide the first evidence of the experience-dependent plasticity of inhibition of the somatosensory system, which underlies the fine control of fast and complex multifinger movements in expert pianists.

Sensory afferent information, such as auditory and somatosensory feedback while moving, plays a crucial role in both control and learning of motor performance across the lifespan. Music performance requires skillful integration of multimodal sensory information for the production of dexterous movements. However, it has not been understood what roles somatosensory afferent information plays in the acquisition and sophistication of specialized motor skills of musicians across different stages of development. In the present preliminary study, we addressed this issue by using a novel technique with a hand exoskeleton robot that can externally move the fingers of pianists. Short‐term exposure to fast and complex finger movements generated by the exoskeleton (i.e., passive movements) increased the maximum rate of repetitive piano keystrokes by the pianists. This indicates that somatosensory inputs derived from the externally generated motions enhanced the quickness of the sequential finger movements in piano performance, even though the pianists did not voluntarily move the fingers. The enhancement of motor skill through passive somatosensory training using the exoskeleton was more pronounced in adolescent pianists than adult pianists. These preliminary results implicate a sensitive period of neuroplasticity of the somatosensory‐motor system of trained pianists, which emphasizes the importance of somatosensory‐motor training in professional music education during adolescence.

Fine-tuned sensory functions typically characterize skilled individuals. Although numerous studies demonstrated enhanced unimodal sensory functions at both neural and behavioral levels in skilled individuals, little is known about their multisensory interaction function, especially multisensory integration and selective attention that involve volitional control of information derived from multiple sensory organs. In the current study, expert pianists and musically untrained individuals performed five sets of intensity discrimination tasks at the auditory and somatosensory modalities with different conditions: (1) auditory stimulus, (2) somatosensory stimulus, (3) congruent auditory and somatosensory stimuli (i.e., multisensory integration), (4) auditory and task-irrelevant somatosensory stimuli, and (5) somatosensory and task-irrelevant auditory stimuli. In the fourth and fifth conditions, participants were instructed to ignore a task-irrelevant stimulus and to pay attention to a task-relevant stimulus (i.e., selective attention), respectively. While the discrimination perception was superior in the condition (3) compared to the better one of the individual unimodal conditions only in the pianists, the task-irrelevant somatosensory stimulus worsened the auditory discrimination more in the pianists than the nonmusicians. These findings indicate unique multisensory interactions in expert pianists, which enables pianists to efficiently integrate the auditory and somatosensory information, but exacerbates top-down selective inhibition of somatosensory information during auditory processing.

Fine-tuned sensory functions are bases of efficient motor control and learning and typically characterize skilled individuals. Although numerous studies demonstrated enhanced unimodal sensory functions at both neural and behavioral levels in skilled individuals, little is known about their multisensory interaction, especially multisensory integration and selective attention that involve volitional control of information derived from multiple sensory organs. Here, we show unique multisensory interaction functions of expert pianists. Expert pianists and musically untrained individuals performed five sets of intensity discrimination tasks at the auditory and somatosensory modalities with different conditions: (1) auditory stimulus, (2) somatosensory stimulus, (3) congruent auditory and somatosensory stimuli (i.e., multisensory integration), (4) auditory and task-irrelevant somatosensory stimuli, and (5) somatosensory and task-irrelevant auditory stimuli. In the fourth and fifth conditions, participants were instructed to ignore a task-irrelevant stimulus and to pay attention to a task-relevant stimulus (i.e., selective attention). The unimodal intensity discrimination of the pianists was superior to that of the nonmusicians at the auditory modality but not at the somatosensory modality. While the discrimination perception was superior in the condition (3) compared to the better one of the individual unimodal conditions (i.e., conditions 1 and 2) only in the pianists, the task-irrelevant somatosensory stimulus worsened the auditory discrimination more in the pianists than the nonmusicians. These findings indicate efficient processing of multisensory information in expert pianists, which enables to benefit from multisensory integration of the auditory and somatosensory information, but exacerbates top-down selective inhibition of somatosensory information during auditory processing.

We perceive the external world through both externally generated and self-generated sensory stimuli (i.e., passive and active perception). While the performance in passive perception is improved by training or repetition through the functional and structural reorganization of the central nervous system (i.e., perceptual learning), the mechanisms by which perceptual learning occurs in active perception remain unclear. Here, we sought to explore the mechanisms underlying the improvement of active somatosensory perception and sensorimotor skills through active perceptual learning. Because we previously found that active perceptual learning depends on the expertise of the motor task to be performed, this study focused primarily on trained individuals. To this end, active haptic training (AHT) that targets the active somatosensory perception during the piano keystroke was used as a means of inducing active perceptual learning. We found four main results. First, participants actively modulated the muscle coordination patterns to optimize the movements required by the task through AHT without any explicit instruction on it by the experimenter, suggesting the involvement of active exploration in active perceptual learning. Second, AHT increased the relative reliance on afferent sensory information relative to the predicted one during the piano keystroke. Third, perceptual sensitivity of externally generated keystroke motions remained unchanged through AHT. Finally, AHT improved feedback control of repetitive keystroke movements in expert pianists. These results suggest that active perceptual learning involves changes in both the predictive integration process and active exploration and that the improved feedback control of fine movements benefits from the improvement in active perception. Significant statement Our ability to perceive both externally generated and self-generated sensory stimuli can be enhanced through training, known as passive and active perceptual learning. Here, we sought to explore the mechanisms underlying active perceptual learning by using active haptic training (AHT), which has been demonstrated to enhance the somatosensory perception of a finger in a trained motor skill. First, AHT reorganized the muscular coordination during the piano keystroke. Second, AHT increased the relative reliance on afferent sensory information relative to predicted one, in contrast to no increment of overall perceptual sensitivity. Finally, AHT improved feedback movement control of keystrokes. These results suggest that active perceptual learning involves active exploration and adaptation of predictive sensory integration, which underlies the co-enhancement of active perception and feedback control of movements.