Sketches of Thought: Inside the Black Box of AI

Abstract

‘Sketches of Thought’ is a human-machine collaborative design system based on Artificial Intelligence (AI).
The first aim of ‘Sketches of Thought’ is illustrating an approach to AI-integration within the designer’s creative workflow. The system translates a hand-drawn architecture sketch into a photorealistic image that suggests a possible evolution of the design idea. The dialogue with the system happens through a visual interface whereby the designer communicates by sketching directly on a drawing monitor, while the system responds by showing the results of the image translation process on a second monitor. Interaction with the system does not end after a first iteration. Instead, the designer is encouraged to adjust the initial sketch – or even make new sketches – for several times to explore, with the aid of the machine’s feedback, different elaborations of an idea. This system does not require particular drawing skills, and therefore anyone can experience a proficient ‘exchange of ideas’ with the AI model.
The second aim of ‘Sketches of Thought’ is helping the designer familiarise with AI technology. This is achieved by unveiling the black box of the AI model functioning, that is, through a representation of its internal processes. Moreover, as the AI model simulates some aspects of human cognition, a look inside the black box of AI also means visualising a simplified version of the human mental processes. Therefore, learning about AI is an opportunity for humans to learn more about themselves.
The relevance of this virtual prototype is twofold: first, it promotes the view of AI as a means to augment rather than replace the human cognitive capabilities. Second, it challenges the current beliefs and prejudices on AI-technology by making the AI internal processes explicit through a visual representation.

Full text

1
Paul Loh
Mond Qu
David Leggett
architecture, art, engineering, culture, fashion
catalogue
FUTURE
PROTOTYPING
+
editors

+
Future Prototyping Exhibition Catalogue
Held at the Melbourne School of Design,
The University of Melbourne
24th February – 27th March 2020
This event is part of Melbourne Design Week 2020, an initiative of the Victorian
Government in collaboration with the NGV, 12th – 22th March. Call for projects
was held between the 19th September - 4th November 2019.
Edited by Paul Loh, Mond Qu and David Leggett
+
Published by Melbourne School of Design, the University of Melbourne,
Victoria 3010, Australia. Printed in Melbourne, Australia.
The moral right of the authors has been asserted
ISBN: 978-0-7340-5501-9
The copyright belongs to The University of Melbourne. Copyright of the project descriptors and
research statements contained in the catalogue remains the property of the authors. Apart from
fair dealing for the purpose of private study, research or review, as permitted under the Copyright
Act, no part of this book may be reproduced by any process without the prior permission of the
publishers, editors and authors.
Every eort has been made to seek permission to reproduce those images whose copyright does
not reside with the University of Melbourne, and we are grateful to the individuals and institutions
who have assisted and provided consent to publish the images in this book. Copyright of images
in this publication are the property of the authors or appear with permission granted to those
authors. The editors and publishers accept no responsibility where authors have not obtained the
appropriate permissions.
All the research statement of significance associated with each project is written by the individual
authors and is copy-edit by the curatorial team. All authors’ information, detail and biography
are provided by the authors. Any omissions are entirely unintentional, and the details should be
addressed to the editors.
Book cover designed by United Make.
We acknowledge the Traditional Owners of country throughout Australia and recognise their
continuing connection to land, waters and culture on whose unceded lands we conduct the research
gathered in this book. We pay our respects to their Elders past, present and emerging.

Content
Foreword
Professor Jane Burry
Introducton
Paul Loh, Mond Qu and Davd Leggett

5
Bologcal
Scoby Bear, Re.Bean Coee Stool, Bo-
Fabrcaton wth Mycelum and Kelp,
Synesthesa, Latte, Mycelum Structure
Prototype #3
Ceramc
Mcro-Hortus Conclusus - Ceramc 3d
prnted mcro-garden lattces, Uncommon
Methods Uncommon Objects
Concrete
Cloops, Parametrc Adjustable Mould,
Talored Flexblty, Topologcal Typology
Fbres
Coat of Theseus, Knttng Archtecture,
The Dark Crafts Pavlon
Metal
FabPod II, SheltAr
Synthetic
Cast Bodes, 3D Printed Soft Robotic
Prosthetic Hand, Repeatability of 3D
Scanned Transradial Stump-Sockets,
Material Disobedience, Motion Imprint -
Interpolating Polynomial Curve Robotic Hot
Wire Cutter, Oddish, Inconsistencies v0.6,
The wind at Byaduk
Timber
Steampunk, SuperSuccah, Surface-
Habitat X2 - Synthetic Ecological Fields,
Augmented Assemblies, I gave the Internet
a Wooden Heart, Digital Timber: Interactive
Design Environments for the Design for
Manufacturing and Assembly of Complex
Spatial Timber Structures, Crooked Lamella,
House #05, Suspended Remnants
Virtual
Sketches of Thought: Inside the Black Box of
AI, constructAR
Short Essay on
Thinking through Making
Paul Loh
Contributers
Projects

101
‘Sketches of Thought’ is a human-machine collaborative design system based on Artificial Intelligence
(AI).
The first aim of ‘Sketches of Thought’ is illustrating an approach to AI-integration within the designer’s
creative workflow. The system translates a hand-drawn architecture sketch into a photorealistic image
that suggests a possible evolution of the design idea. The dialogue with the system happens through a
visual interface whereby the designer communicates by sketching directly on a drawing monitor, while
the system responds by showing the results of the image translation process on a second monitor.
Interaction with the system does not end after a first iteration. Instead, the designer is encouraged to
adjust the initial sketch – or even make new sketches – for several times to explore, with the aid of the
machine’s feedback, dierent elaborations of an idea. This system does not require particular drawing
skills, and therefore anyone can experience a proficient ‘exchange of ideas’ with the AI model.
The second aim of ‘Sketches of Thought’ is helping the designer familiarise with AI technology. This is
achieved by unveiling the black box of the AI model functioning, that is, through a representation of its
internal processes. Moreover, as the AI model simulates some aspects of human cognition, a look inside
the black box of AI also means visualising a simplified version of the human mental processes. Therefore,
learning about AI is an opportunity for humans to learn more about themselves.
The relevance of this virtual prototype is twofold. First, it promotes the view of AI as a means to augment
rather than replace the human cognitive capabilities. Second, it challenges the current beliefs and
prejudices on AI-technology by making the AI internal processes explicit through a visual representation.
Sketches of Thought:
Inside the Black Box of AI
-
Gabriele Mirra + Alberto Pugnale,
University of Melbourne
2020+
Codng+
Touch+
Engneerng+
Vrtual+
Dgtal
Diagram explaining the architecture of the AI model, as well as the human-machine collaborative system.