Rendering Aesthetic Impressions of Text in Color Space.

To read the full-text of this research, you can request a copy directly from the authors.


What is an artwork and how could a machine become artist? This paper addresses the provocative question by theorizing a computational model of aesthetics and implement- ing the Aesthetiscope|a computer program that portrays aesthetic impressions of text and renders an abstract color grid artwork reminiscent of early twentieth century ab- stract expressionism. Following Dewey's psychological interpretation of \aesthetic" and Jung's ontology of fundamental psychological functions, we theorize that a viewer flnds an artwork moving and satisfying because it seduces her into rich evocations of thoughts, sensations, intuitions, and feelings. The Aesthetiscope embodies this theory and aims to generate color grids paired with inspiration texts (a word, a poem, or song lyrics), which can be received as aesthetic and artistic by a viewer. The paper describes flve Jungian aesthetic readers which are together capable of creative narrative understanding, and three color-logics that employ psycho-semantic principles to render the aesthetic read- ings in color space. Evaluations of the Aesthetiscope revealed that the program is best at portraying intuition and feeling, and that overall, the Aesthetiscope is capable of creating the aesthetic of art based on an inspiration text in a non-arbitrary way.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Some put emotion visualization in the context of generative art. For example, the Aesthetiscope [26], given a textual input, generates an artistic visual output in the form of colored squares. The authors define their endeavor as visualizing the aesthetics of reading, stressing that the reading process does not only involve logic, but also emotion and sensation. ...
... In terms of color palettes, Liu and Maes [26] extend the Munsell's color space [4] and use prescriptive color psychology theories of Berlin and Kay [14] and Goethe [36]. The authors manually annotated their affective lexicon with the descriptive vocabulary of color parameters. ...
In order to facilitate interaction in computer-mediated communication and enrich user experience in general, we introduce a novel textual emotion visualization approach, grounded in generative art and evocative visuals. The approach is centered on the idea that affective computer systems should be able to relate to, communicate, and evoke human emotions. It maps emotions identified in the text to evocative abstract animation. We examined two visualizations based on our approach and two common textual emotion visualization techniques, chat emoticons and avatars, along three dimensions: emotion communication, emotion evocation, and overall user enjoyment. Our study, organized as repeated measures within-subject experiment, demonstrated that in terms of emotion communication, our visualizations are comparable with emoticons and avatars. However, our main visualization based on abstract color, motion, and shape proved to be the best in evoking emotions. In addition, in terms of the overall user enjoyment, it gave results comparable with emoticons, but better than avatars.
... In [44], an approach for visualizing a document's affective structure represented by a colorbased navigation bar is detailed. Using affect analysis engine described in [42], sentences are annotated with six "basic" emotions [21], and then, the dynamics of affect throughout a text document is represented using color bar, in which colors used to symbolize each of these emotions are sequenced (Figure 1 [42] The Aesthetiscope introduced in [43] was developed with the aim to generate color grids paired with inspiration texts (a word, a poem, or song lyrics). This program renders aesthetic impressions of text, such as thoughts, sensations, intuitions, and feelings, as a 16x9 grid of colors and emulates the creative process of a visual artist. ...
... This program renders aesthetic impressions of text, such as thoughts, sensations, intuitions, and feelings, as a 16x9 grid of colors and emulates the creative process of a visual artist. Examples of color grids generated by the Aesthetiscope for the following texts (clockwise from upper-left corner): (1) the poem "Fire and Ice" by Robert Frost; (2) the poem "A Song of Despair" by Pablo Neruda; (3) the word 'fear'; (4) the word 'mourning'; (5) the word 'god'; and (6) the word 'envy' -are shown in [43] Expressive and "dancing" text has been widely employed by researchers to accentuate affective information communicated through written language. ...
Social interaction among people is an essential part of every society, and a strong foundation for the development and self-actualization of a person. Even in virtual environments we tend to interact in a social way. Our research addresses the tasks of recognition, interpretation, and visualization of affect communicated through text messaging. In order to facilitate sensitive and expressive interaction in computer-mediated communication, we previously introduced a novel syntactical rule-based approach to affect recognition from text. The evaluation of the developed Affect Analysis Model showed promising results regarding its capability to accurately recognize affective information in text from an existing corpus of informal online conversations. To enrich the user's experience in online communication, to make it enjoyable, exciting, and fun, we implemented a web-based Instant Messaging (IM) application, AffectIM, and endowed it with emotional intelligence by integrating the developed Affect Analysis Model. This paper describes the findings of a twenty-person study conducted with our AffectIM system. The results of the study indicate that our IM system with automatic emotion recognition function can achieve a level of affective intelligence (system is successful at conveying the user's feelings, avatar expression is appropriate) that is comparable to “gold standard”, where users select the label of the conveyed emotion manually.
... Each specimen of earth thus captures the story of the place in its essence, including the lives of people living on it and the plants growing there. In the academic paper titled "Rendering Aesthetic Impressions of Text in Color Space", the authors observe -"Terre Provençale evokes more broadly than for a single person, potentially evoking specific meaning for a whole community of people, namely, the residents of Provence, and to a lesser degree, all of mankind, who share a common experience with the various shades of yellow, brown, and red earth" (Liu & Maes, 2005, p. 2). The latter artwork is thus seen to reinforce the recurrent theme of recognition and acceptance of the inherent plurality of identity, while the former represents an aesthetic unification through the integration of seemingly disparate components. ...
A scholarly review of Art Space Germany exhibition organised by Institut für Auslandsbeziehungen, held at National Library, in February 2017, which is to be presented again in Mumbai in September 2017. This article presents a brief description of the artefacts, comments on the stylistic and technical aspects involved, contextualizes the artworks by providing relevant background information of the artists/genres, and offers insight into the overarching theme and message of the exhibition, which especially pertains to immediate socio-political concerns.
This paper proposes the iterative visual clustering (IVC) on unstructured text sequences to form and evaluate keyword clusters, based on which users can use visual analysis, domain knowledge to discover knowledge in the text. The text sequence data are broken down into a list representative keywords after textual evaluation, and the keywords are then grouped to form keyword clusters via an iterative stochastic process and are visualized as distributions over the time lines. The visual evaluation model provides shape evaluations as quantitative tools and users' interactions as qualitative tools to visually investigate the trends, patterns represented by the keyword clusters' distributions. The keyword clustering model, guided by the feedback of visual evaluations, step-wisely enumerates newer generations of keyword clusters and their patterns, therefore narrows down the search space. Then the proposed IVC is applied onto nursing narratives and is able to identify interesting keyword clusters implying hidden knowledge regarding to the working patterns and environment of registered nurses. The loop of producing next generation of keyword clusters in IVC is driven and controlled by users' perception, domain knowledge and interactions, and it is also guided by a stochastic search model. So both semantic and distribution features enable IVC to have significant applications as a text mining tool, on many other data sets, such as biomedical literatures.
Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2006. Includes bibliographical references (p. 153-163). People have rich points-of-view that afford them the ability to judge the aesthetics of people, things, and everyday happenstance; yet viewpoint has an ineffable quality that is hard to articulate in words, let alone capture in computer models. Inspired by cultural theories of taste and identity, this thesis explores end-to-end computational modeling of people's tastes-from model acquisition, to generalization, to application- under various realms. Five aesthetical realms are considered-cultural taste, attitudes, ways of perceiving, taste for food, and sense-of-humor. A person's model is acquired by reading her personal texts, such as a weblog diary, a social network profile, or emails. To generalize a person model, methods such as spreading activation, analogy, and imprimer supplementation are applied to semantic resources and search spaces mined from cultural corpora. Once a generalized model is achieved, a person's tastes are brought to life through perspective-based applications, which afford the exploration of someone else's perspective through interactivity and play. The thesis describes model acquisition systems implemented for each of the five aesthetical realms. (cont.) The techniques of 'reading for affective themes' (RATE), and 'culture mining' are described, along with their enabling technologies, which are commonsense reasoning and textual affect analysis. Finally, six perspective-based applications were implemented to illuminate a range of real-world beneficiaries to person modeling-virtual mentoring, self-reflection, and deep customization. by Xinyu Hugo Liu. Ph.D.
Full-text available
Presents a spreading-activation theory of human semantic processing, which can be applied to a wide range of recent experimental results. The theory is based on M. R. Quillian's (1967) theory of semantic memory search and semantic preparation, or priming. In conjunction with this, several misconceptions concerning Quillian's theory are discussed. A number of additional assumptions are proposed for his theory to apply it to recent experiments. The present paper shows how the extended theory can account for results of several production experiments by E. F. Loftus, J. F. Juola and R. C. Atkinson's (1971) multiple-category experiment, C. Conrad's (1972) sentence-verification experiments, and several categorization experiments on the effect of semantic relatedness and typicality by K. J. Holyoak and A. L. Glass (1975), L. J. Rips et al (1973), and E. Rosch (1973). The paper also provides a critique of the Rips et al model for categorization judgments.
Full-text available
Preexisting word knowledge is accessed in many cognitive tasks, and this article offers a means for indexing this knowledge so that it can be manipulated or controlled. We offer free association data for 72,000 word pairs, along with over a million entries of related data, such as forward and backward strength, number of competing associates, and printed frequency. A separate file contains the 5,019 normed words, their statistics, and thousands of independently normed rhyme, stem, and fragment cues. Other files provide n x n associative networks for more than 4,000 words and a list of idiosyncratic responses for each normed word. The database will be useful for investigators interested in cuing, priming, recognition, network theory, linguistics, and implicit testing applications. They also will be useful for evaluating the predictive value of free association probabilities as compared with other measures, such as similarity ratings and co-occurrence norms. Of several procedures for measuring preexisting strength between two words, the best remains to be determined. The norms may be downloaded from
Full-text available
This paper presents a novel way for assessing the affective qualities of natural language and a scenario for its use. Previous approaches to textual affect sensing have employed keyword spotting, lexical affinity, statistical methods, and hand-crafted models. This paper demonstrates a new approach, using large-scale real-world knowledge about the inherent affective nature of everyday situations (such as "getting into a car accident") to classify sentences into "basic" emotion categories. This commonsense approach has new robustness implications.
Full-text available
Since a decade ago, a person-century of effort has gone into building CYC, a universal schema of roughly 105 general concepts spanning human reality. Most of the time has been spent codifying knowledge about the concept; approximately 106 common sense axioms have been handcrafted for and entered into CYC's knowledge base, millions more have been inferred and cached by CYC. This paper studies the fundamental assumptions of doing such a large-scale project, reviews the technical lessons learned by the developers, and surveys the range of applications that are enabled by the technology.
Students are routinely asked in English courses for their reactions to texts they are reading. Sometimes there are so many different reactions that we may wonder whether everyone has read the same text. And some students respond so idiosyncratically to what they read that we say their responses are “totally off the wall.”
Open Mind Common Sense is a knowledge acquisition system designed to acquire commonsense knowledge from the general public over the web. We describe and evaluate our first fielded system, which enabled the construction of a 450,000 assertion commonsense knowledge base. We then discuss how our second-generation system addresses weaknesses discovered in the first. The new system acquires facts, descriptions, and stories by allowing participants to construct and fill in natural language templates. It employs word-sense disambiguation and methods of clarifying entered knowledge, analogical inference to provide feedback, and allows participants to validate knowledge and in turn each other.
Several suggestions for a class of theories of recognition memory have been proposed during the past decade. These models address predictions about judgments of prior occurrence of an event, not the identification of what it is. The history and current status of one of these models is discussed. The model postulates the detection of familiarity and the utilization of retrieval mechanisms as additive and separate processes. The phenomenal experience of familiarity is assigned to intraevent organizational integrative processes; retrieval depends on interevent elaborative processes. Other current theoretical options are described, and relevant supportive data from the literature are reviewed. New tests of the model involving both free recall and word pair paradigms are presented. The dual process model is extended to the word frequency effect and to the recognition difficulties of amnesic patients. (68 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
A new method for automatic indexing and retrieval is described. The approach is to take advantage of implicit higher-order structure in the association of terms with documents (“semantic structure”) in order to improve the detection of relevant documents on the basis of terms found in queries. The particular technique used is singular-value decomposition, in which a large term by document matrix is decomposed into a set of ca. 100 orthogonal factors from which the original matrix can be approximated by linear combination. Documents are represented by ca. 100 item vectors of factor weights. Queries are represented as pseudo-document vectors formed from weighted combinations of terms, and documents with supra-threshold cosine values are returned. initial tests find this completely automatic method for retrieval to be promising.
Conference Paper
A key to improving at any task is frequent feedback from people whose opinions we care about: our family, friends, mentors, and the experts. However, such input is not usually available from the right people at the time it is needed most, and attaining a deep understanding of someone else's perspective requires immense effort. This paper introduces a technological solution.We present a novel method for automatically modeling a person's attitudes and opinions, and a proactive interface called "What Would They Think?" which offers the just-in-time perspectives of people whose opinions we care about, based on whatever the user happens to be reading or writing. In the application, each person is represented by a "digital persona," generated from an automated analysis of personal texts (e.g. weblogs and papers written by the person being modeled) using natural language processing and commonsense-based textual-affect sensing.In user studies, participants using our application were able to grasp the personalities and opinions of a panel of strangers more quickly and deeply than with either of two baseline methods. We discuss the theoretical and pragmatic implications of this research to intelligent user interfaces.
Conference Paper
The traditional story understanding dogma in AI holds that there exists a singular, objective meaning implied by text, which can be uncovered by applying just the right logical inferences. But according to research into the cognition of reading, text can also be read, not objectively, but aesthetically. An aesthetic reading of text engages not only the agency of thought, but also intuition, sensation, and sentiment, and cultural interpretation. We have developed and implemented a basic computational model of aesthetic reading, which employs Carl Jung's Modes of Interpretation idea to delegate the task of aesthetic interpretation out to several different textual analysis engines, each computing a different mode of interpreting text, such as generating sentimental evocations of a text (Feeling modality), or remembering visual imagery evoked by the text (Sensation modality). The outputs of the various interpretive modalities are merged and then their unification is achieved by mapping them into color space using theories of color psychology. We built the Aesthetiscope, an artwork whose grid of colors are dynamically generated from aesthetic readings of an inputted poem or song, to illustrate the power and potential of going beyond literal understandings of text.
The rationale for a general three-dimensional approach to measuring emotions (the Pleasure-Arousal-Dominance, PAD, Emotional-State Model) is described. Evidence bearing on two early versions of the PAD scales is reviewed, and three studies report additional development and refinement of a final set of PAD scales. Factor analysis of 58 PAD items in Study 1, 41 PAD items in Study 2, and 34 PAD items in Study 3 consistently yielded three nearly orthogonal Pleasure, Arousal, and Dominance factors. The final 16-item State Pleasure, 9-item State Arousal, and 9-item State Dominance scales had alpha reliability coefficients of .97, .89, and .80, respectively. Across Studies 2 and 3, intercorrelations among the latter three scales did not exceed .09 in absolute value, showing that the scales provided a parsimonious base for the general assessment of emotional states. Discriminant and construct validity evidence for the scales were reviewed. Potential applications of the scales were noted.
Conference Paper
We present novel simple appearance and shape models that we call epitomes. The epitome of an image is its miniature, condensed version containing the essence of the textural and shape properties of the image. As opposed to previously used simple image models, such as templates or basis functions, the size of the epitome is considerably smaller than the size of the image or object it represents, but the epitome still contains most constitutive elements needed to reconstruct the image. A collection of images often shares an epitome, e.g., when images are a few consecutive frames from a video sequence, or when they are photographs of similar objects. A particular image in a collection is defined by its epitome and a smooth mapping from the epitome to the image pixels. When the epitomic representation is used within a hierarchical generative model, appropriate inference algorithms can be derived to extract the epitome from a single image or a collection of images and at the same time perform various inference tasks, such as image segmentation, motion estimation, object removal and super-resolution.
We describe ConceptNet, a freely available semantic network presently consisting of over 250,000 elements of commonsense knowledge. Inspired by Cyc, ConceptNet includes a wide range of commonsense concepts and relations, and inspired by WordNet, it is structured as a simple, easy-to-use semantic network. ConceptNet supports many of the same applications as WordNet, such as query expansion and determining semantic similarity, but it also allows simple temporal, spatial, affective, and several other types of inferences. This paper is structured as follows. We first discuss how ConceptNet was built and the nature and structure of its contents. We then present the ConceptNet toolkit, a reasoning system designed to support textual reasoning tasks by providing facilities for spreading activation, analogy, and path-finding between concepts. Third, we provide some quantitative and qualitative analyses of ConceptNet. We conclude by describing some ways we are currently exploring to improve ConceptNet.
Wie Farben Wirken: Farbpsychologie
  • E Heller
Heller, E.: 1989, Wie Farben Wirken: Farbpsychologie, Farbsymbolik, Kreative Farbgestaltung, Rowohlt Verlag, Reibek bei Hamburg.
  • J Albers
Albers, J.: 1963, Interaction of Color, Yale University Press.
Cezanne (The Portrait of Hortense), Nineteenth-Century Art Worldwide
  • S Sidlauskas
Sidlauskas, S.: 2004, Emotion, Color, Cezanne (The Portrait of Hortense), Nineteenth-Century Art Worldwide: 3(2), AHNCA Press.
  • R Barthes
Barthes, R.: 1964/1967, Elements of Semiology (Translated by Annette Lavers & Colin Smith), London: Jonathan Cape.
  • J W Goethe
Goethe, J.W.v.: 1840/1970. Theory of Colours, trans. C. L. Eastlake, Cambridge, MA: MIT Press.
  • A H Munsell
Munsell, A.H.: 1905, A Color Notation, Boston.
The 'uncanny The Penguin Freud Library
  • S Freud
Freud, S.: 1919/1990, The 'uncanny', in Dickson, A. (Ed.), The Penguin Freud Library, Vol. 14: Art and Literature: 335-6, Penguin, Harmondsworth.
The Creative Mind: An Introduction to Metaphysics (Trans: Mabelle L. Andison)
  • H Bergson
Bergson, H.: 1946/2002, The Creative Mind: An Introduction to Metaphysics (Trans: Mabelle L. Andison), New York: Citadel Press.
MontyLingua: Commonsense-Enriched NLP, Toolkit and API. Accessed at:
  • H Liu
Liu, H.: 2002, MontyLingua: Commonsense-Enriched NLP, Toolkit and API. Accessed at:
A function theory of creative reading. The Psycgrad Journal
  • K A Moorman
  • Ram
Moorman, K. & A. Ram: 1994, A function theory of creative reading. The Psycgrad Journal. Technical Report GIT-CC-94/01, Georgia Institute of Technology.
  • A G Baumgarten
Baumgarten, A.G.: 1735/1954, Reflections on Poetry, trans. K. Aschenbrenner and W.B. Holther, Berkeley: University of California Press.
  • G Mccracken
McCracken, G.: 1988, Culture & Consumption, Indiana University Press.
Algorithmic Aesthetics: Computer Models for Criticism and Design in the Arts
  • G Stiny
  • J Gips
Stiny, G., J. Gips: 1978a, Algorithmic Aesthetics: Computer Models for Criticism and Design in the Arts, Berkeley:University of California Press.
Marcel Duchamp, or the Castle of Purity
  • O Paz
Paz, O.: 1978, Marcel Duchamp, or the Castle of Purity, New York, Viking Press.
Culture & Consumption
  • G Mccracken
ThoughtTreasure: A natural language/commonsense platform. Accessed on 11 November
  • E Mueller
Mueller, E. T.: 2000, ThoughtTreasure: A natural language/commonsense platform. Accessed on 11 November 2005 from
Art as Experience ( Perigee Books
  • J Dewey