What if material interfaces could adapt physically to the user’s emotional state in order to develop a new affective interaction? By using emotional computing technologies to track facial expressions, material interfaces can help to regulate emotions. They can serve either as a tool for intelligence augmentation or as a means of leveraging an emphatic relationship by developing an affective loop with users. This paper explores how color and shape-changing can be used as an interactive design tool to convey emotional information, and is illustrated by two projects, one at the intimate scale of fashion, and one at a more architectural scale. By engaging with design, art, psychology, computer and material science, this paper envisions a world where materials can detect the emotional responses of a user and reconfigure themselves in order to enter into a feedback loop with the user’s affective state and influence social interaction.