Available via license: CC BY 4.0
Content may be subject to copyright.
Hack.VR: A Programming Game in Virtual Reality
Dominic Kao*, Christos Mousas*, Alejandra J. Magana*, D. Fox Harrell†, Rabindra Ratan‡,
Edward F. Melcer§, Brett Sherrick*, Paul Parsons*, Dmitri A. Gusev*
* Purdue University, †MIT, ‡Michigan State University, §UC Santa Cruz
kaod@purdue.edu, cmousas@purdue.edu, admagana@purdue.edu, fox.harrell@mit.edu, rar@msu.edu,
eddie.melcer@ucsc.edu, bsherrick@purdue.edu, parsonsp@purdue.edu, dgusev@purdue.edu
Figure 1: Existing environments for programming in VR: A) Block-based programming (Cubely), B) Multiple-choice code
snippet selection (Imikode), C) Block-based programming (VR-OCKS), D) World-building (NeosVR), E) Code visualization
(Primitive), F) Visual programming (Glitchspace). All gures reproduced with permission.
ABSTRACT
In this article we describe Hack.VR (
'
hæk
'
d t /vr/), an object-
oriented programming game in virtual reality. Hack.VR uses a VR
programming language in which nodes represent functions and
node connections represent data ow. Using this programming
framework, players reprogram VR objects such as elevators, robots,
and switches. Hack.VR has been designed to be highly interactable
both physically and semantically.
KEYWORDS
VR; virtual reality; programming; object-oriented; game.
ACM Reference Format:
Dominic Kao*, Christos Mousas*, Alejandra J. Magana*, D. Fox Harrell
†
,
Rabindra Ratan
‡
, Edward F. Melcer
§
, Brett Sherrick*, Paul Parsons*, Dmitri
A. Gusev*. 2020. Hack.VR: A Programming Game in Virtual Reality. In
International Conference on the Foundations of Digital Games (FDG ’20),
September 15–18, 2020, Bugibba, Malta. ACM, New York, NY, USA, 6 pages.
https://doi.org/10.1145/xxxxxxx.xxxxxxx
FDG ’20, September 15–18, 2020, Bugibba, Malta
2020. ACM ISBN 978-1-4503-8807-8/20/09. . . $15.00
https://doi.org/10.1145/xxxxxxx.xxxxxxx
1 INTRODUCTION
In the entire history of computing, programming has been a largely
physically static activity. But technologies previously inaccessible
to most users are now growing rapidly. Today, 78% of Americans
are familiar with VR (from 45% in 2015) [
9
]. As a result, experi-
ences traditionally created for desktops are now appearing in VR,
e.g., training [
2
] and automobile design [
25
]. Researchers argue
that VR increases immersion [
11
], which in turn increases engage-
ment and learning [
5
]. VR might be especially useful for teaching
programming because spatial navigation in VR helps reduce ex-
traneous cognitive load and increase germane cognitive focus on
learning content compared to text on a screen [
26
]. Further, VR al-
lows users to experience a sense of self-presence in the environment
[
33
], which facilitates an embodied-cognitive learning experience
[
27
,
35
] through which users interact with the learning content
more intuitively [
36
], potentially augmenting learning outcomes
[
4
]. Nonetheless, only a handful of environments for programming
exists in VR. In this article, we describe a programming game in
virtual reality that we created called Hack.VR.12
1Trailer video: https://youtu.be/3Mp6ISjD1mg.
2Walkthrough video: https://youtu.be/TGc8H3Nw-3M.
arXiv:2007.04495v1 [cs.HC] 9 Jul 2020
FDG ’20, September 15–18, 2020, Bugibba, Malta Kao et al.
Figure 2: In Hack.VR, programs are sets of nodes. Nodes are interconnected by blue tubes, which represent data ow.
2 PROGRAMMING IN VR
Existing environments for programming in VR can be seen in Fig-
ure 1. These include VR-OCKS and Cubely, block-based VR pro-
gramming environments [
34
,
38
], and Imikode, a multiple choice
code snippet selection environment in VR [
31
]. Other signicant
projects include NeosVR [
29
], a shared social universe that features
powerful programming tools for VR world creation, and Primitive
[
32
], an “immersive development environment” enabling 3D code
visualization. In the indie game Glitchspace [
8
], players use a visual
programming language to solve puzzles. These environments for
programming in VR have been developed for education (Cubely,
Imikode, VR-OCKS), for modifying virtual worlds (NeosVR), for
code visualization (Primitive), and for entertainment (Glitchspace).
Importantly, Hack.VR was created specically to teach object-
oriented programming (OOP) compared to the highly procedural
approaches in the systems above. OOP encapsulates logic into ob-
jects. This paradigm has a natural translation to VR, where 3D
objects can each contain internal programming that is abstracted
from observers. Hack.VR is the rst system for learning OOP in VR,
and will serve as a testbed to perform research studies. This testbed
may also be useful for studying other aspects, e.g., help facilities,
embellishment, the player avatar, feedback, and the resulting eects
on VR programming [3, 7, 10, 13–24, 30].
3 THE GAME
3.1 Engine
In Hack.VR, a program is a set of nodes. See Figure 2. Nodes contain
typical programming constructions, e.g., primitive values, objects,
arithmetic operators, conditionals, event handlers, function calls.
Nodes facilitate communication through data ow. Nodes may have
both inputs and outputs depending on the node type. For exam-
ple, see Figure 3. Nodes can also represent entire method calls, the
details of which are abstracted from the player except input and
output. Because the goal of Hack.VR is to teach the player OOP,
the inner-workings of the methods themselves are intentionally
Figure 3: Left: An arithmetic node that performs addition.
Right: An entity node contains a miniaturized representa-
tion of a virtual world object, e.g., a door, a robot, an eleva-
tor. Programs attached to an entity node will then operate
on the actual virtual world object, e.g., open the door, turn
the robot, operate the elevator.
abstracted away (and players cannot see the code) so that the player
can concentrate on higher-level representations. The engine also
supports extensions. For example, once a new function has been
dened in the engine, a node can call it. To reduce complexity, play-
ers in Hack.VR use designer-embedded nodes to solve each puzzle
instead of creating their own nodes. While the engine supports
on-the-y node creation, the UI does not currently support this.
Node-based programming, like any other type of programming,
can lead to execution errors. For example, a NOT node expects a
boolean input (true or false), and outputs the inversion of its input.
However, if a numerical value is instead used as input to a NOT
node, this results in an error. While this is valid in some procedural
languages (in C programming, 0 is false and anything else is true),
implicit conversions from numerical values to boolean data types is
not allowed in object-oriented programming (e.g., Java, C#). When
an error is detected in the program, this is indicated by a red tube.
See Figure 4 for examples.
Hack.VR: A Programming Game in Virtual Reality FDG ’20, September 15–18, 2020, Bugibba, Malta
Figure 4: Example program errors. Top: A feedback loop.
Middle: An invalid input. Bottom: Another invalid input. Er-
rors are automatically displayed in the node after the error.
Figure 5: Left: The left gun allows the player to freely tele-
port. Right: The right gun allows the player to manipu-
late objects, modify programs, and inspect nodes. Inspecting
shows a node’s output value on the square display.
Figure 6: Left: Puzzle 1 is opening a door using a boolean.
Right: Puzzle 2 is setting the height of an elevator so you
can reach the next puzzle.
Figure 7: Puzzle 3 is four mini-puzzles where the player ma-
nipulates arithmetic nodes to create a password.
Figure 8: Puzzles 4–8 teach logical operators. Left: A puzzle
using the and operator. Right: Programming a robot to drop
a cube only on the column marked 3.
Figure 9: Left: Puzzle 9 requires programming a robot to
move based on color. Arrows indicate in which direction the
robot should move. Right: Puzzle 10 is an extension of Puz-
zle 9, and now the robot moves based on a combination of
column number and color.
Figure 10: Puzzles 11, 12, and 13 explore constructors. Left:
The robot in Puzzle 11 must be constructed with the hover
movement type to get past the lava. Right: Robots in Puzzle
13 must be constructed with correct movement types and
body types matching each lane.
FDG ’20, September 15–18, 2020, Bugibba, Malta Kao et al.
Figure 11: Puzzles 14–17 explore classes and inheritance. In
Puzzle 14, the player modies a class to modify all objects
that are created from that class.
Similarities can be drawn between Hack.VR and other program-
ming paradigms inherently predisposed to visualization, e.g., ow-
based programming [
28
]. Hack.VR is inspired by node graph sys-
tems: Unreal Engine 4’s Blueprints [
6
], Unity Playmaker [
12
], and
shader editors (Shader Graph [
37
] and Amplify Shader Editor [
1
]).
3.2 Art Style and Gameplay
Hack.VR is based in a sci- futuristic setting. See Figure 12. In
Hack.VR, the player holds two “guns” that take the place of their
VR controllers. See Figure 5. Hack.VR is compatible with both HTC
Vive and Oculus. Hack.VR’s controls are found in Appendix A. Using
these controls, Hack.VR challenges players with object-oriented
programming puzzles. Hack.VR consists of 17 dierent puzzles.
Each puzzle builds upon concepts from prior puzzles. See Figures 6
through 11 for a short description of puzzles.
3.3 Design Process
The design process followed a spiral approach of design
→
im-
plementation
→
evaluation. Iterations grew in complexity and
renement over several cycles for each part of the game. Feedback
was solicited from designers, developers, and playtesters. Com-
ments armed positive design choices (e.g., “I like that you can see
the physical buttons and physical door miniaturized in the node
tree”) and highlighted potential improvements (e.g., “When I’m
connecting things, it’s hard to tell what connector I’m working
with; maybe highlight the current selected connector?”). A typical
early prototype can be seen in Figure 13.
4 CONCLUSION
In this article we described Hack.VR, a programming game in virtual
reality. We created a VR programming language that is highly visual,
while being highly semantic. Hack.VR is inspired by the possibilities
of programming in VR. Imagine these highly evocative scenarios:
•
Programming an innite stairwell taking you into the clouds.
Figure 12: Hack.VR’s sci- futuristic setting.
Figure 13: Prototyping Puzzles 9 and 10.
•
Programming a robot carrying you across vast deserts, rolling
hills, and tundras.
•
Reconguring and reprogramming the mechanical parts in
your gun to enhance your capabilities.
Given the great potential for VR to enhance learning outcomes
[
4
,
26
,
35
,
36
], we expect that Hack.VR might help teach program-
ming concepts more eectively than similar, non-immersive tools.
Although assessment research should be conducted to conrm this
expectation empirically, from a perspective that spans research, de-
sign, and play, there is reason to be excited about what the coming
decade will bring for programming in VR.
Hack.VR: A Programming Game in Virtual Reality FDG ’20, September 15–18, 2020, Bugibba, Malta
REFERENCES
[1]
Amplify Creations. 2020. Amplify Shader Editor. (2020). https://assetstore.unity.
com/packages/tools/visual-scripting/amplify- shader-editor-68570
[2]
Johanna Bertram, Johannes Moskaliuk, and Ulrike Cress. 2015. Virtual training:
Making reality work? Computers in Human Behavior 43 (2015), 284–292. https:
//doi.org/10.1016/j.chb.2014.10.032
[3]
Max V Birk, Cheralyn Atkins, Jason T Bowey, and Regan L Mandryk. 2016.
Fostering Intrinsic Motivation through Avatar Identication in Digital Games.
CHI (2016). https://doi.org/10.1145/2858036.2858062
[4]
Jongpil Cheon and Michael M Grant. 2012. The eects of metaphorical interface
on germane cognitive load in web-based instruction. Educational Technology
Research and Development 60, 3 (2012), 399–420.
[5]
Chris Dede. 2009. Immersive interfaces for engagement and learning. (2009).
https://doi.org/10.1126/science.1167311
[6]
Epic Games. 2020. Blueprints Visual Scripting. (2020). https://docs.unrealengine.
com/en-US/Engine/Blueprints/index.html
[7]
Julian Frommel, Kim Fahlbusch, Julia Brich, and Michael Weber. 2017. The
Eects of Context-Sensitive Tutorials in Virtual Reality Games. (2017), 367–375.
https://doi.org/10.1145/3116595.3116610
[8]
Glitchspace. 2016. Glitchspace. (2016). https://store.steampowered.com/app/
290060/Glitchspace/
[9]
Greenlight Insights. 2018. New Consumer Data Finds VR Headset Us-
age Expected To Increase In 2019, According To Greenlight Insights.
(2018). https://greenlightinsights.com/new-consumer- data-nds- vr-headset-
usage-expected- increase-2019- according- greenlight-insights/
[10]
Kieran Hicks, Kathrin Gerling, Graham Richardson, Tom Pike, Oliver Burman,
and Patrick Dickinson. 2019. Understanding the eects of gamication and
juiciness on players. IEEE Conference on Computatonal Intelligence and Games,
CIG 2019-Augus (2019). https://doi.org/10.1109/CIG.2019.8848105
[11]
Mustafa Hussein and Carl Nätterdal. 2015. The benets of virtual reality in
education- A comparison study. (2015).
[12]
Hutong Games LLC. 2019. Playmaker. (2019). https://assetstore.unity.com/
packages/tools/visual-scripting/playmaker- 368
[13]
Dominic Kao. 2019. Exploring the Eects of Growth Mindset Usernames in STEM
Games. American Education Research Association (2019).
[14]
Dominic Kao. 2019. Innite Loot Box: A Platform for Simulating Video Game
Loot Boxes. IEEE Transactions on Games (2019). https://doi.org/10.1109/tg.2019.
2913320
[15]
Dominic Kao. 2019. The Eects of Anthropomorphic Avatars vs. Non-
Anthropomorphic Avatars in a Jumping Game. In The Fourteenth International
Conference on the Foundations of Digital Games.
[16]
Dominic Kao. 2020. Exploring Help Facilities in Game-Making Software. In ACM
Foundations of Digital Games.
[17]
Dominic Kao. 2020. The eects of juiciness in an action RPG. Entertainment
Computing 34, November 2018 (2020), 100359. https://doi.org/10.1016/j.entcom.
2020.100359
[18]
Dominic Kao and D. Fox Harrell. 2015. Exploring the Impact of Role Model
Avatars on Game Experience in Educational Games. The ACM SIGCHI Annual
Symposium on Computer-Human Interaction in Play (CHI PLAY) (2015).
[19]
Dominic Kao and D. Fox Harrell. 2015. Mazzy: A STEM Learning Game. Founda-
tions of Digital Games (2015).
[20]
Dominic Kao and D. Fox Harrell. 2016. Exploring the Eects of Encouragement
in Educational Games. Proceedings of the 34th Annual ACM Conference Extended
Abstracts on Human Factors in Computing Systems (CHI 2016) (2016).
[21]
Dominic Kao and D. Fox Harrell. 2016. Exploring the Impact of Avatar Color on
Game Experience in Educational Games. Proceedings of the 34th Annual ACM
Conference Extended Abstracts on Human Factors in Computing Systems (CHI 2016)
(2016).
[22]
Dominic Kao and D. Fox Harrell. 2017. MazeStar: A Platform for Studying Virtual
Identity and Computer Science Education. In Foundations of Digital Games.
[23]
Dominic Kao and D. Fox Harrell. 2017. TowardUnderstanding the Impact of Visual
Themes and Embellishment on Performance, Engagement, and Self-Ecacy in
Educational Games. The annual meeting of the American Educational Research
Association (AERA) (2017).
[24]
Dominic Kao and D. Fox Harrell. 2018. The Eects of Badges and Avatar Identi-
cation on Play and Making in Educational Games. In Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems - CHI’18.
[25]
Glyn Lawson, Davide Salanitri, and Brian Watereld. 2016. Future directions for
the development of virtual reality within an automotive manufacturer. Applied
Ergonomics 53 (2016), 323–330. https://doi.org/10.1016/j.apergo.2015.06.024
[26]
Elinda Ai Lim Lee and Kok Wai Wong. 2014. Learning with desktop virtual
reality: Low spatial ability learners are more positively aected. Computers and
Education (2014). https://doi.org/10.1016/j.compedu.2014.07.010
[27]
Edward F Melcer. 2018. Learning with the body: Understanding the Design Space
of Embodied Educational Technology. Ph.D. Dissertation. New York University
Tandon School of Engineering.
[28]
J. Paul Morrison. 1994. Flow-based programming. In Proc. 1st International
Workshop on Software Engineering for Parallel and Distributed Systems. 25–29.
[29]
NeosVR. 2020. NeosVR. (2020). https://store.steampowered.com/app/740250/
Neos
[30]
E O’Rourke, Kyla Haimovitz, Christy Ballweber, Carol S. Dweck, and Zoran
Popović. 2014. Brain points: a growth mindset incentive structure boosts per-
sistence in an educational game. Proceedings of the 32nd annual ACM con-
ference on Human factors in computing systems - CHI ’14 (2014), 3339–3348.
http://dl.acm.org/citation.cfm?id=2557157
[31]
Solomon Sunday Oyelere and Violetta Cavalli-Sforza. 2019. Imikode: A VR Game
to Introduce OOP Concepts. Proceedings of the 19th Koli Calling International
Conference on Computing Education Research (2019).
[32] Primitive. 2020. Primitive. (2020). https://primitive.io/
[33]
Rabindra Ratan. 2013. Self-presence, explicated: Body, emotion, and identity
extension into the virtual self. In Handbook of research on technoself: Identity in a
technological society. IGI Global, 322–336.
[34]
Rafael J. Segura, Francisco J. del Pino, Carlos J. Ogáyar, and Antonio J. Rueda. 2019.
VR-OCKS: A virtual reality game for learning the basic concepts of programming.
Computer Applications in Engineering Education August (2019). https://doi.org/
10.1002/cae.22172
[35]
Dong Hee Shin. 2017. The role of aordance in the experience of virtual reality
learning: Technological and aective aordances in virtual reality. Telematics
and Informatics (2017). https://doi.org/10.1016/j.tele.2017.05.013
[36]
Anthony Steed, Ye Pan, Fiona Zisch, and William Steptoe. 2016. The impact of a
self-avatar on cognitive load in immersive virtual reality. In Proceedings - IEEE
Virtual Reality. https://doi.org/10.1109/VR.2016.7504689
[37]
Unity Technologies. 2020. Shader Graph. (2020). https://unity.com/shader-graph
[38]
Juraj Vincur, Martin Konopka, Jozef Tvarozek, Martin Hoang, and Pavol Navrat.
2017. Cubely: Virtual reality block-based programming environment. Proceedings
of the ACM Symposium on Virtual Reality Software and Technology, VRST Part
F1319, 2 (2017). https://doi.org/10.1145/3139131.3141785
FDG ’20, September 15–18, 2020, Bugibba, Malta Kao et al.
APPENDIX A
Hack.VR Controls
Figure 14: Movement.
Figure 15: Grabbing an object. Once an object is grabbed, it can be rotated and moved.
Grabbing a node will inspect it, displaying its output on the right gun.
Figure 16: Switch between physical manipulation and program modication mode. A blue
laser indicates physical manipulation, while a red laser indicates modication.
Figure 17: Program modication.
Figure 18: Modifying node attachments.