Content uploaded by Simon Renger
Author content
All content in this area was uploaded by Simon Renger on Jun 23, 2022
Content may be subject to copyright.
MASTER THESIS
Investigation into the criteria of embeddability of
visual scripting languages within the domain of
game development.
Author:
Simon Renger
Supervisor:
Thomas Buijtenweg MSc
Bojan Endrovski MSc
This thesis is submitted in fulfilment of the requirements
for the degree of Master of Science in Game Technology
In the programme
Professional Master Game Technology
Academy for AI Games and Media
Breda University of Applied Sciences
1
23/06/2022
2
Declaration of Authorship
I, Simon Renger, declare that this thesis titled, “Investigation into the criteria of
embeddability of visual scripting languages within the domain of game development.”
and the work presented are my own. I confirm that:
● This work was done wholly or mainly while in candidature for a research degree at this
University.
● Where any part of this thesis has previously been submitted for a degree or any other
qualification at this University or any other institution, this has been clearly stated.
● Where I have consulted the published work of others, this is always clearly attributed.
● Where I have quoted from the work of others, the source is always given. With the
exception of such quotations, this thesis is entirely my own work.
● I have acknowledged all main sources of help.
● Where the thesis is based on work done by myself jointly with others, I have made clear
exactly what was done by others and what I have contributed myself.
Signed:
Date: 23/06/2022
3
Empty page
4
Breda University of Applied Sciences
Abstract
Professional Master Game Technology
Academy for AI Games and Media
Master of Science in Game Technology
Investigation into the criteria of embeddability of visual scripting languages
within the domain of game development.
By Simon Renger
This paper investigates potential criteria to identify visual scripting languages as
embeddable within the domain of game development. We explore the criteria of
embeddability for scripting languages and use them as the base line for the study. Then,
we conduct a systematic review of GitHub repositories of visual scripting related
projects to gather a data foundation for defining possible data points about
embeddability of visual scripting languages. At the end, we conduct in depth interviews
with game industry professionals supported by the Q-Methodology to investigate the
views of them on the collected data from the systematic review.
We could identify the most important themes for embeddability of visual scripting
languages are performance related. Also, we identified the main differences between the
criteria of embeddability for textual scripting languages and visual scripting languages
are tooling related.
5
Empty page
6
Contents
Declaration of Authorship 2
Abstract 4
List of Abbreviations 9
List of Terms 10
Introduction 13
Background information and theory 15
2.1 Software development in the game industry. 15
2.2 Scripting Languages 17
2.2.1 Accessibility of scripting languages 17
2.2.2 Embeddability of Scripting Languages 18
2.3 Visual Scripting languages 20
2.3.1 Visual Scripting languages: Typical use cases 21
2.3.2 Embeddability of Visual Scripting Languages 23
2.4 Research Question 23
Materials and Methods 24
3.1 Data Collection and extraction 24
3.2 Search Term 24
3.3 Limitations 25
3.4 Data Extraction 26
3.5 Data Processing 27
3.6 Data validation 28
3.6.1 Q-Methodology 28
3.6.2 Interview Guide 28
Data and results of the systematic review 29
4.1 Data Cleaning 30
4.2 Data Overview 30
4.3 Language Observations 31
4.4 Execution Model 32
4.5 License distribution 33
4.6 Documentation and Samples 33
4.7 Dependencies 34
4.8 Form 35
7
4.9 API Complexity 35
Data and results of the interviews and the Q-Methodology 35
5.1 The participants’ general views 35
5.2 Q-Methodology results 37
Discussion of data and results 41
6.1 Data Validity 41
6.4 Ethics 42
6.3 Discussion of the results 42
6.3.1 Performance 42
6.3.2 Mechanical aspects of embedding software 44
6.3.3 Documentation 46
6.3.4 License 46
6.3.5 Tooling and workflow 46
6.3.6 Purpose of VSL 47
6.4 Summary of the results 47
Conclusion and future directions 49
7.1 Future directions 49
Appendix A 51
A.1 Tables 51
A.1.1 Overview of Game Engines and their use of VSL 51
A.1.2 Overview of VSL related research papers within the game industry 52
A.1.3 Implementation Language Overview of Game Engines 53
A.2 Figures and Graphs 54
A.2.1 Search Term Graphics 54
Bibliography 54
8
Empty page
9
List of Abbreviations
VSL Visual Scripting Languages
VPL Visual Programming Languages
TPL Textual Programming Languages
UE Unreal Engine
UE4 Unreal Engine 4
VM Virtual Machine
VSE Visual Scripting Environment or Visual Programming Environment
BP Blueprint (Unreal Engine)
SL Scripting Language
OS Operating System
GPL GNU Public License
MIT The MIT License is a permissive free software license originating at the
Massachusetts Institute of Technology
CPU Central processing unit
GPU Graphics processing unit
API Application programming interface
FFI Foreign function interface
IoT Internet of things
DCC Digital Content Creation
GLM Generalized linear model, a generalization of ordinary linear regression
GAM Generalized additive model
PL Programming Languages
SESE Software Engineering Stack Exchange
10
List of Terms
Game Engine – “A Software that is extensible and can be used as the foundation for many
different games without major modifications. […] A game engine generally consists of a tool
suite and a runtime component.” [1, pp. 11,32-62]. The tool suite often contains an editor to
compose the artistic and design game content.
(Game) Framework – “A framework is a partially constructed application – the programmer
completes the application by providing custom implementations of missing functionality within
the framework (or overriding its default behaviour)” [1, p. 343].
Textual Programming Language – A computer language that makes use of text to form its
logic. Generally, textual programming languages include words, numbers, and punctuation in a
similar way that written natural languages do [2].
Game Mechanics – “Game mechanics are the aspects of the game design, or rules, in which the
player follows within the game world” [3].
Headers (Files) – “A header file is a file with extension .h which contains C function
declarations and macro definitions to be shared between several source files. A simple practice
in C or C++ programs is that we keep all the constants, macros, system-wide global variables,
and function prototypes in the header files and include that header file wherever it is required”
[4].
Namespace (Programming) – “In computing, a namespace is a set of signs (names) that are
used to identify and refer to objects of various kinds. A namespace ensures that all of a given set
of objects have unique names so that they can be easily identified. ... Some programming
languages organize their variables and subroutines in namespaces.” [5].
Type (Programming) – Linda Torczon & Keith Cooper define a type in a programming
language as a collection of properties for each data value [6, pp. 164–166].
Type system (Programming) – Linda Torczon & Keith Cooper define a type system as the set
of types and their rules to specify the program behavior [6, pp. 164–166].
Dynamic type – “The type of an object as determined at runtime. Also known as most-derived
type” [7], [8]
Static Typed – “The type of an object as known to the compiler based on its declaration. See
also: dynamic type” [7], [8]
Git – “Git is a free and open-source distributed version control system designed to handle
everything from small to very large projects with speed and efficiency [9]
GitHub – 73+ million developers and companies build, ship, and maintain their software on
GitHub the largest and most advanced development platform in the world. Git repository
hosting platform [10]
11
Repository – “A source code repository is a file archive and web hosting facility where a large
amount of source code is kept, either publicly or privately.” [11]
Visual Studio – “Microsoft Visual Studio is an integrated development environment from
Microsoft. It is used to develop computer programs, as well as websites, web apps, web services
and mobile apps.” [12]
Visual Studio Code – “Visual Studio Code is a source-code editor made by Microsoft for
Windows, Linux and macOS. Features include support for debugging, syntax highlighting,
intelligent code completion, snippets, code refactoring, and embedded Git.” [13], [14]
Visual Studio or Visual Studio Code Extension – A software package that extends the existing
editor with a not already build in functionality.
REST API – “A REST API (also known as RESTful API) is an application programming interface
(API or web API) that conforms to the constraints of REST architectural style and allows for
interaction with RESTful web services. REST stands for representational state transfer and was
created by computer scientist Roy Fielding.” [15]
12
Empty page
13
Introduction
Video games are multimedia applications which are entertainment products that
require active participation of the user. Video games are often developed by a
multidisciplinary team. A multidisciplinary team does not only contains of programmers
but also of “non-programmers” and “semi-programmers” [16]. Other terms for non-
programmers and “semi-programmers” might be Designers and Artists of various
specializations. In the game development process this group of “non-programmers”,
“semi-programmers” and programmers with different backgrounds create the game
experience together.
During the development of a game various people of the multidisciplinary team have
influence on the game logic. Either through designing the logic and then passing it to a
programmer to implement the logic or executing the act of programming themselves
(“semi-programmers”) [16], [17]. The active participation of multiple divisions within a
team on the final game is crucial for the game development process.
Moreover, the process behind the creation of a game is iterative and based on quick
prototyping. Developers (i.e., designers, artists, programmers) make use of this iterative
approach to test different game mechanics. An iterative approach expresses the need for
ideas to be prototyped and tested quick. As part of the development cycle the iterative
workflow is applied to nearly all aspects of the whole creative process of creating a
game [18]. The iterative and rapid prototype approach of developing a multimedia
application needs to be supported also by the software that is being used.
Traditionally, what game development requires is for the game logic to be written in a
way that a computer can understand and process. We refer to this type of writing as
programming. This kind of programming is commonly done in a textual programming
language (TPL). Writing code in this way requires a general understanding of
programming concepts and the used language [19]. A result of this kind of programming
is that each line of code needs to be composed carefully and clearly. Even the smallest
mistake, e.g., forgetting a semicolon, might lead to multiple compilation errors or
runtime errors. Programming in a TPL is a complex task that requires a great syntactical
understanding of the language and it’s grammar to avoid small errors such as forgetting
a semicolon, thus programming is usually done by a trained professional [19].
Furthermore, as an alternative to TPL modern game engines make use of Visual
Scripting Languages (VSL). M. Idrees, et al, define visual scripting (VS) and the
associated workflow as "(VS) allows a programmer to develop the logic for a program by
simply dragging-and-dropping a visual element on a canvas and subsequently
connecting that visual element with other elements. A visual element hides the syntactic
complexities of the programming language from the programmer while the connections
between elements represent the logic of the program" [19]. VS helps with empowering
non-programmers to influence the game logic without touching a single line of code, and
increases their ability to work directly with the game content while reducing the need of
complicated abstract thinking [19], [20].
14
Modern game engines have started to address that in multi-disciplinary teams, game
designers and artists need to be able to prototype their ideas in a familiar and easy to
learn way. That is why major game engines such as Unity [21][22] and Unreal
[23][24][25] have been adopting visual scripting languages (VSL). Important to mention
here is that modern game engines or digital content creation tools (DCC) have adopted
VSL not just for game logic but also for shader programming or visual effect creations.
This is illustrated in the (Appendix A.1.1) given table. The table shows this development
by providing an overview of game engines and DCC tools of the industry [26]–[28].
It can be concluded that VSL support is a potential key factor for game development
when it comes to choosing the right tool for the specific job.
Furthermore, modern game engines provide a built-in editor for their VSL. When using a
VSL the user needs a Visual Scripting Environment (VSE). A VSE is an editor that
provides the user with a canvas to drag and drop visual elements such as blocks or
nodes. The visual elements can be subsequently connected with each other to represent
the general application flow in a spatially aware manner [29], [30]. For the most, a VSE
is part of the host environment for the VSL. The host environment would be the engine.
For example, in Unreal Engine 4 (UE4) Blueprints are edited in the built-in Blueprint
Editor [25]. Game engines such as Unity or UE4 support their users in scripting their VSL
in a built-in VSE within their engines.
Besides their VSE, the VSLs available to the game industry are mostly built-in
proprietary solutions that are not intended to be embedded within other engines or
frameworks. These proprietary VSLs come as part of the host game engine or game
framework. For example, Unreal Engine's Blueprints [24], [25], [31] are part of the
engine’s architecture and are designed to work only within the eco system of its host.
Another example is Unity's VSL Bolt [22]. Bolt is built as a plugin but cannot be used
outside of the engine’s ecosystem. After the acquisition of Bolt, Unity started to integrate
the VSL into the engine and renamed it to Unity Visual Scripting
1
[22], [32]. Moreover,
the same technique of integrating a VSL within a host framework can be observed in the
successful open source game engine, Godot. [33] Godot’s VSL is also limited to only work
within the framework. It can be summarized that the most popular game engines’ VSL
solutions are not designed to be embedded within other frameworks or engines.
At the time of conducting the literature review, the research provided minimal results
on suitable criteria to define an embeddable VSL in the domain of game development.
Since VSLs are mainly used within modern game engines [27], finding academic work on
the suitable criteria for a VSL that can be embedded within a game engine / framework
was challenging. Although, there are existing research materials that define suitable
criteria for designing an embeddable scripting language in the domain of game
development [34]–[43], [44, pp. 155–179], [1, pp. 956–957] [36], [37] [1, p. 52], [6, pp.
3–4], [44, pp. 155–179]. A notable example is the “Passing a Language through the Eye
of a Needle” [36] which explains the design of the Lua SL. However, the industry itself
has not been vocal about criteria for an embeddable visual scripting language. This leads
1
Out of consciousness reasons we will refer to Unity Visual Scripting as Bolt in this paper.
15
to the question what kind of criteria are needed to define a Visual Scripting Language’s
embeddability within a game engine / framework?
Background information and theory
The Visual Scripting Languages available to the game industry today are mostly built-in
proprietary domain-specific solutions that are not intended to be embedded within
other systems. These proprietary domain-specific solutions come as part of the host
game engine or game framework. Epic Games Unreal Engine 4 (UE4) Blueprints (BP)
System is intertwined with the Engine and designed to work only within the ecosystem
of its host [25]. The same level of coupling of systems can be said about Unity's Bolt [45]
as well as for Godot's Visual Scripting [33]. The major game engines and one of the most
successful open-source game engines [33] integrate their VSLs into their engine
environment which makes it hard to embed them into different environments.
The fact that major game engines integrate their VSLs into their own engine
environment leads to the appearance of different VSLs. The way they are implemented
and used differ from engine to engine. Their naming of nodes also differs from engine to
engine. Besides, the benefits and learnings from designing a VSL for the specific engine
are not shared with the game industry. Therefore, the minimal appearance of official
technical publications about the VSL design from major game engine vendors underlines
the missed opportunity of sharing knowledge.
While conducting this review the research provided minimal results on suitable criteria
to define an embeddable VSL in the domain of game development. VSLs are commonly
used within modern game engines [27] it was difficult to find academic work on what
kind of criteria are suitable for a VSL that can be embedded within a game engine /
framework. The literature available is mostly concentrated on the topic of VSL in game
development from a general point of view. An overview of the discovered papers can be
found in (Appendix A.1.2). Surprisingly, most of these papers are either master or
bachelor theses. Their focus lays mainly on the design of a visual scripting language and
how they are used or can be extended within the current framework. They neglect the
embeddability aspect. Therefore, this paper will explore the differences between SL and
VSL and based on this, define potential criteria of embeddability for VSL within the
domain of game development.
2.1 Software development in the game industry.
In order to understand the requirements for portability and embeddability of software
within the game industry it is important to understand how the game industry differs
from the software industry. The ability of creating software for the major platforms such
as Windows, Mac, Linux and Mobile Operating Systems (OS) is largely unconstrained
within the software industry [41, pp. 17–32], since development tools for these
platforms are accessible and in most cases, relatively affordable. The game industry, on
the other hand, has large constrains due to the nature of shipping often on proprietary
platforms with heavy licencing restrictions [41, pp. 17–32]. Moreover, the need of
serving different hardware constellations equally well creates an extra engineering
16
challenge. Therefore, how the game industry differs from the software industry is
though the licence restrictions on platforms such as game consoles (PlayStation or
Xbox) as well as the need to run equally smooth on multiple different hardware
specifications [1, pp. 32–54].
Besides the different platform requirements, the game industry works vastly different
than the software industry since it uses a heavy iterative methodology to develop
software [41, p. 60]. The iterative software development method can be defined in its
“…basic idea to do development for a period of time (a month or two), accomplish a set
of goals, get the software to a reasonable state, and then start again with another period
and add new features to it.” [42, p. 172]. With that being said, how the method of
iterative software development differs from traditional software development is
because it allows for rapid changes of the requirements for the used technology [42].
Moreover, the possibility of a change of technology mid project requires a certain type of
control over the used technology. Therefore, maintainability and source code access of
used software is important. The needed flexibility and control often lead to the fact that
game developers write their own software instead of using an existing one [42]. Writing
your own software can guarantee the needed flexibility required by the iterative
software development process.
Writing your own software instead of using someone else’s software is a controversial
discussed topic. There are multiple articles out there which discuss this topic [46], [47].
The main argument that is used for writing your own software from scratch is that
external libraries mostly come with a huge level of complexity and maintainability
efforts [46], [48]–[50], which reasoning literature such as “Introduction To Game
Development” by Rabin & Steve [42] agrees with. All of the reviewed articles and
research materials suggest that writing your own technology shifts the need for
maintainability from an external codebase to your own control which can be beneficial
but also more cost intensive [42].
Besides the platform requirements and work methodology, the game industry differs
heavily from the software industry due to the fact that games are traditionally not
shipped as open source. Shipping games as closed source marks the importance of
permissive software licenses. Therefore, when using external tools or middleware it is
important to make sure that the license is compatible with shipping a game as closed
source [43]. That is why software with non-permissive licenses such as the GNU Public
Licenses (GPL), which license requires that all source code is published, is possibly not
suited to be used within games. The inability to use a non-permissive license often leads
to “reinventing” the wheel [51]. Therefore, it can be concluded that the game industry
differs from the software industry since it is often impossible to make use of commonly
used libraries since they are using a non-permissive license which would require the
vendor to publish the source code [41].
17
2.2 Scripting Languages
Ousterhout [52] defines Scripting Languages (SL) as programming languages that are
intended to work with modules defined in other low-level languages such as C.
Moreover, Ousterhout states that they are not meant to be used to write a whole
application from scratch. Scripting Languages are more used for plug-and-play
situations to enable systems to communicate with each other. The use case of plug and
play separates scripting languages from system programming languages [52].
System Programming Languages are designed to construct data structures and
algorithms from scratch to represent an application. Ousterhout states that system
programming languages are usually very low level and can represent the most primitive
computer elements: words of memory [52]. Moreover, Ousterhout characterizes them as
being typed and principally compiled. At the same time, Scripting Languages are mainly
dynamic typed or untyped, and interpreted [52].
Interpreted languages differ from compiled languages in that they are not compiled
from one language into native code. Compiled languages are generally directly executed
on the CPU [6, pp. 1–23]. On the other hand, Linda Torczon & Keith Cooper define an
interpreter as a system that "…takes as input an executable specification and processes
as output the result of executing the specification"[6, p. 3]. Moreover, Linda Torczon &
Keith Cooper state that some Languages such as Java or Lua use a hybrid approach [6,
pp. 1–23]. They compile their source input to a bytecode representation which then gets
interpreted by an interpreter generally called virtual machine (VM). This mixed-method
(also called hybrid method) combines the benefits of both worlds and can be found in
many modern scripting languages [6, pp. 1–23] ,[44, pp. 155–179].
2.2.1 Accessibility of scripting languages
Aside from the performance, accessibility is an important aspect of the design of a
scripting language. Since game development teams may not consist solely of
programmers [53], the simplicity of a language is essential for picking the right scripting
language. The simplicity of a language can be determined by analysing the syntax of a
programming language, since the syntax defines how we write a piece of code much
alike as a book page for an author. In textual programming languages the syntax is a
sequence of strings in form of words, numbers or punctuation [6, pp. 11-13,25-82,85].
The correct use of the syntax is crucial for writing correct code. Sometimes a forgotten
semicolon might lead to compilation errors. This leads to the conclusion that the syntax
of a SL is an important factor to learn the language or increase its accessibility.
Another aspect of improved accessibility for “semi-programmers” [16] is the type-
system – strong or weak. A strongly typed language requires the user to define the type
for each variable. In contrast, in a weakly typed language the type might change due to
the ability of casting [7]. There is also the discussion between statically typed and
dynamically typed languages. While in a statically typed language the type needs to be
known at compile time, in a dynamically typed language the type is known at runtime
18
[7]. A dynamically typed language ensures a technical designer that all variable types are
correct. For example, in JavaScript, the keyword "var" or "let" is used to declare a
variable [54]. However, the designer does not need to mention the type, unlike in C++.
Therefore, a variable might hold a string or a number depending on the value of this
variable [7]. The correct choice of the type-system when designing an SL will have great
influence on the usability and accessibility of this language.
2.2.2 Embeddability of Scripting Languages
The topic of embeddability of scripting languages is covered in many different academic
sources [1, pp. 965–978], [36], [44, pp. 155–179], [55]. It is also discussed in non-
academic sources such as Software Engineering Stack Exchange platform (SESE) [35],
where a summary of the embeddability of scripting languages based on the views of
software engineers in different fields can be found [35]. The mentioned SESE thread
states that an SL can be considered as embedded when its interpreter and runtimes are
running in the same process as the host application. That the interpreter and runtime
are part of the host process is explained by the user “Berin Loritsch” [35] and matches
the academic definition of embeddability of SLs stated in “C APIs in extension and
extensible languages” by H. Muhammad et al. [34]. Another important remark of the
same user, “Berin Loritsch” [35], is that the complexity of the interpreter / the runtime
environments is being kept simple. Again, this matches the observations of the academic
world e.g. the description of the Lua API written by R. Ierusalimschyet al. [37] or the
description of C API’s for scripting languages written by H. Muhammad et al.[34]. As the
given example from SESE [35] and the two aforementioned papers show, the topic of
embeddability has been covered at many other platforms and the views of the
professional world and the academic world are aligned.
With the knowledge in mind that academic and professional views on the embeddability
of scripting languages overlap, it can be stated that the main focus on embeddability for
scripting languages lays on the connection of the interpreter / runtime with the host
application. The connection is mostly done through the application programming
interface (API) of the interpreter / runtime. The API allows the host application to send
data to the SL and vice versa [34], [39]. This transfer of data between the host and the
runtime allows the embedded language to register modules, functions (written in the
host’s language) and the SL interpreter / runtime environment. Based on the conducted
literature review it becomes clear that the main factor of the connection between the
host environment and the runtime / interpreter of an embedded scripting language lays
on the API [34], [36], [39].
For the host-environment and runtime/interpreter connection, the right choice of the
exposing language of the API or the whole interpreter / runtimes is important for the
embeddability. For example, the programming language C contains a minimal set of
primitive types, which makes the language perfect to be an implementation language of
the exposed SL API. Generally the programming language C is the lingua franca of
programming languages [39] since most programming languages can interact with it via
their foreign function interface (FFI) [36]. And because C qualifies as a highly
19
embeddable language due to the general support of other languages, it a great choice for
exposing the API in it.
Next to the API’s complexity, the memory footprint of the library and runtime are
important too. Most SL used in the game industry are designed as lightweight as
possible. Their runtime tends to be small in terms of binary size and in terms of memory
usage when executing. Lua or Wren are great examples for such languages, since they
both have a very small binary footprint and memory footprint [44, pp. 155–179], [56],
[57].
Furthermore, the execution model of the SL is important for the embeddability /
portability. As previously stated, an interpreted language differs from a system
programming language in the way that it is not executed directly on the CPU.
Along those lines, there are two important concepts to mention – the tree-walking
interpreter and the bytecode interpreter. First, the tree-walking interpreter [6, pp. 3–4]
[58, p. 17], where the SL is not compiled down to a different representation such as
bytecode [55], [58, pp. 16–20], [44, pp. 155–179] ,[1, pp. 52,954-958] but it is
represented in the form of a graph, which graph the interpreter executes. However, if
there is a hybrid language that compiles the SL source code to bytecode, the interpreter
will run this bytecode in the VM. The tree-walking interpreter has no limitations in the
way of compiling the language since it does not perform a compilation in the traditional
sense.
The hybrid approach, on the other hand, has two options that can be used – just in time
and ahead of time compilations. Just in time compilation can be used to compile the code
just before it is executed [6, p. 4,285], [58, p. 18,273], [59]. JIT compilation has the
disadvantage that it might not be allowed or work on all platforms the game is shipping
on. A specific platform such as a game console might forbid this technique in their
Technical Requirement Checklist (TRC) [60]
2
. The other alternative is ahead of time
(AOT) compilation which means that the compiler will compile all code in advance.
Choosing the right method has an influence on the embeddability and portability due to
possible platform restrictions which might prevent the usage of JIT compilation within a
hybrid language.
Overview of technical criteria to embed a Scripting Language:
Aspect
Details
API Complexity [34]–
[37]
Can be described as the number of
steps the user needs to take to extend
the embedded language via the API,
which can be measured by the
number of functions and data
structures.
2
Since TRC’s often are under NDA it was not possible to find official documents to prove this
general knowledge fact. The only statement that could be found was published by the mono
project stating that: “Xamarin for Xbox One leverages Mono’s JIT compiler during development,
allowing you to quickly iterate when working on your game code. Release builds of your game
will be fully statically compiled using Mono’s Full Ahead of Time Compilation engine as JIT
compilation is not allowed in published games.” [60]
20
Binary & Memory
footprint [44, pp.
155–179]
How big are the memory and binary
footprints of the language runtime? Is
the smaller the better?
Dependency,
Portability [35] [38]
Dependencies may introduce the risk
of security issues since they create a
reliance on external code.
Dependencies can also stand in the
way of porting between platforms
such as PlayStation, Xbox, Mobile,
Nintendo Consoles, Mac, PC.
Binding Language [34]
The binding language of the API
should be as universal as C. Since C is
the lingua franca of language
bindings, a C API or C interface should
be preferred.
Data transfer [34],
[39]
How is the data being transferred?
Directly via memory access (pointers)
or via a handle system?
Execution Model
How is the data interpreted: through
a tree-walking interpreter [6, pp. 3–4]
[58, p. 17] or through a bytecode
interpreter [55], [58, pp. 16–20], [44,
pp. 155–179] ,[1, pp. 52,954-958]?
Table 1 Mechanical and technical aspects of embedding a scripting language
Overview of professional criteria to embed a Scripting Languages:
Aspect
Details
Ease to use [1, pp.
956–957] [36], [37]
Customizable by the engineers to
offer game specific extensions to the
language such as error highlighting in
the editor. The simplicity of the
syntax is important as well.
Support for rapid
prototyping [41, pp.
17–32]
The ability to recompile / reinterpret
changed code without the need to
close the application (editor) and see
changes directly.
Documentation,
Guides, Tutorials
Is there documentation on how to
embed the language into a different
environment? Are there any guides
and tutorials?
License [41]–[43]
How permissive is the license?
Table 2 Professional Aspects of embedding a scripting language
2.3 Visual Scripting languages
Visual Scripting Languages (VSL) can be defined as a subcategory of SL. In contrast to
TPL, the user composes a program with the help of visual elements such as graphical or
textual objects in a graphical editor [19], [61]. In VSL sometimes the spatial arrangement
of the visual elements is important for future execution of the logic [30]. The graphical
editor used for such languages is usually called the visual scripting environment (VSE).
VSEs have been proven to be an effective working environment for non-programmers
[30], [62]. In conclusion, visual scripting languages are scripting languages that compose
21
a program with visual elements, where the spatial arrangement might matter in a visual
scripting environment.
Moreover visual scripting (VS) hides the semantic complexity of the underlying software
engineering problems from the end-user [19]. The user of a VSL does not need to worry
about the syntactical correctness of their work, unlike in a normal TPL, where a
forgotten semicolon might break the compilation process [19]. Moreover, VS enables the
human visual system and human visual processing to process multi-dimensional data
[29]. Since the human visual system is optimized for processing this kind of data, it will
reduce the given complexity of a program [29]. This reduced complexity increases the
accessibility of VS [30] which is essential in modern game development since teams
might not consist of just programmers [53].
The complexity of debugging errors can be reduced in VSL through the VSE [30], [63],
since it is harder for the user to make syntactical errors within such an environment
[30], [63]. The VSE guides the user through the program composition process step-by-
step. The step-by-step guidance allows the VSE to evaluate the visual representation of
the program and indicate errors quicker than a TPL could [30], [63].
Since VSL can be seen as a subcategory for SL, they share the same execution models.
Therefore, the same rules apply for the VSL as for SL. For VSL this means that, for
example, a graph is translated to an intermediate representation such as a tree
structure, which is than further compiled to bytecode, or tree-walked interpreted [58, p.
7,17]. This typically happens in the so-called “runtime”. The “runtime” can also have
different names such as “engine” or “interpreter”. Literature and industry are not clear
on which term is applicable. They all refer to the same concept [1], [42], [44], [58], [64].
An alternative approach not often used for SL is transpiling the source code to a
different language. In the book “Engineering A Compiler”, L. Torczon, K. Cooper define a
compiler as source-to-source translator. The transpiler translates one representation of
source code to another. In the domain of VSL this can be found more often than with SL.
A primary example is Google’s Blockly [65], which is a block based VSL framework that
allows own blocks to be compiled to different languages such as Lua, Python, JavaScript.
2.3.1 Visual Scripting languages: Typical use cases
M. Kuhail et al. “Characterizing Visual Programming Approaches for End-User
Developers: A Systematic Review” found that VL are being used in different areas such
as IoT or Virtual/Augmented Reality. Their use case within those industries varies from
allowing the end user to define applications such as web services or mobile applications
[61] to enabling the user to manage process routines. What M. Kuhail et al. found in their
study can also be observed within the game industry and 3D modelling software: like in
the other areas, VL are used with a different purpose such as logic composition to
shader and material compositions.
Tool
Use case
22
Autodesk, Maya
Materials Editing
Blender Foundation,
Blender 3D
Materials Editing
SideFX, Houdini
Materials Editing
Adobe, Substance 3D
Substance graphs,
Functions, MDL
Materials and FX-
Maps
Unity Technology,
Unity 3D
Shader editing, Visual
representation of
animation, Gameplay
Logic, state machine
Epic Games, Unreal
Engine 4
Material Editing, state
machine, animation,
Gameplay Logic
Godot
Game logic, Material
Editing
Table 3: Overview of selected digital content creation tools and game engines that support VSL and visual
representation of different aspects than game logic. [25], [33], [74], [66]–[73]
The use of node graphs (diagram-based) to express relationships or creation aspects of
3D models and their materials or shaders, can be observed in the leading 3D modelling
tools such as Autodesk Maya or Blender [71], [75]. Autodesk Maya offers the artists a
node editor [69] for editing their scene within Maya. Like Maya, Blender supports a
node-based graph (diagram-based) editor to manipulate their shaders, called the
“Shader Editor” [70]. Digital content creation tools (DCC) such as Maya and Blender
offer diagram-based graph editors to empower technical artists to manipulate the
materials or shaders in a visual way [72].
Other use cases of visual representations, which share some similarities with VSL, are
the form or iconic based languages when representing state machines. UE4 for example
provides a state machine editor with a visual representation of the actual states with
elements of an iconic, form and diagram based language [68]. Unity has a similar
approach when it comes to their state machine for animations, called “The Animator
Controller” [66]. It can be observed in the leading game engines such as Unity or UE4
that they make use of similar visual representation of data and logic such as a VSL for
representing state machines.
The difference between the VSL for manipulating shaders and materials and their logical
counterpart is that often the information within those shader graphs is baked
(compiled) to a representation, which the graphical processing unit (GPU/Graphics
Card) can process [76]. This contrasts with a VSL which is used for defining the game
logic, since such languages (BP or Godot Visual Scripting Language) are often compiled
to a bytecode representation or directly interpreted [77]. This interpretation of the
bytecode or the graph may happen every frame, opposed to the materials and shader
VSL, which are evaluated and converted to code that can run directly on the GPU.
VSL are used for different use cases such as shader, material or logic manipulation since
they ease the learning curve of programming/scripting for the end-user by hiding the
complexity of the underlying task and making it visually accessible [19]. VSL eliminate
23
the likelihood of syntax errors [62] since the VSE helps with evaluating and preventing
errors while programming [30]. The reduction of the complexity of the user interaction
with the system enables the user to learn new skills such as structurally and logically
solving problems [78]. Regarding the game industry, this ability of accessibility is
important because a game development team might not consist solely of programmers.
Novice programmers, such as artists or designers, can benefit from the reduced syntax
and the hidden system complexity and focus on the task at hand.
Outside of the game industry, VSL have been used successfully to teach the
programming fundamentals [62], [79] [80], [81]. In their journey of learning a new
language, students do not have to focus on the syntactical difficulties but rather on
iteratively learning diverse programming strategies [62], [78]. The visualization of the
algorithms can help with understanding their complexity [62]. The learnings gathered
through multiple studies about the effectiveness of visual scripting in education [79],
[80] [82] can be translated into the game industry, since its usage can lead to a more
accessible work for a broader audience.
2.3.2 Embeddability of Visual Scripting Languages
In the literature review process, only a limited number of research papers describing the
problems regarding the embeddability of visual scripting languages into a host
application have been found, and mainly in a context outside of the game industry.
The major source found is the "Taxonomies of Visual Programming" by B. Myers [29],
who argues that portability of the language can be considered an issue. Moreover, Myers
states that designing and building a sufficient VSE is a tremendously difficult operation
[29]. In Myers's paper from 1989, it is mentioned that the advantage of textual
languages such as C or C++ lies in the portability of the source code, since any text editor
can edit it. The same concerns as Myers’ (1989) are stated in Jason Gregory's "Game
Engine Architecture" (2014) [1, p. 953]. All in all, these two books, together with
Nystorm's "Game Programming Patterns" (2011) are the few limited sources tha t
discuss embeddability and portability when it comes to designing a VSL [44, p. 178].
2.4 Research Question
The existing gap in research results obtained through academia and the game industry
indicate that further research in this area needs to be conducted in order to define
possible criteria that mark a VSL as embeddable. A hypothesis that serves as a baseline
for this research is that the criteria of embeddability for scripting languages are also
applicable to visual scripting languages.
RQ: What is the criteria that a VSL needs to consider when attempting to maximise
embeddability of a VSL system within the domain of the game industry?
This paper proposes to investigate this question using a systematic analysis of GitHub
repositories to identify common themes. The results obtained through the systematic
analysis will be presented to and evaluated by industry experts in the form of an
interview, followed by a Q-Methodology application.
24
Materials and Methods
As a result of the literature review and the limited findings on criteria for embeddability
of visual scripting languages, this paper is using a three-step method to answer RQ1. The
base assumption of this research is that the criteria of embeddability for scripting
languages can also be applied to visual scripting languages since they are from a
software engineering perspective semantically similar, and differ mainly in the input
consumption of the script data. Therefore, this paper tests the described criteria in
chapter 2.2.2 Table 1-2 via a systematic review of GitHub repositories, followed by the
findings being presented to and evaluated by industry experts via an interview with a
following Q-Methodology application. The following paragraph gives a broad overview
on the steps further described in this chapter.
The first step (1) is a systematic data collection and extraction from GitHub repositories
via the GitHub[10] REST API [83]. The next step (2) is analysis of the gathered numerical
and textual data of open source or publicly available source projects on GitHub. After
this, the third step (3) consist of a series of in-depth interviews to validate the results of
the data analysis, followed by a Q-Methodology. This study follows the example set by “A
systematic review of open source clinical software on GitHub for improving software
reuse in smart healthcare” by Z .Shen and M. Spruit [11].
3.1 Data Collection and extraction
The data collection pipeline is written in a Python script following the example set by Z.
Shen and M. Spruit [11] by adjusting the search query and data collection towards the
discussed topics in this study. The used script uses the Python third-party library
PyGitHub
3
to communicate with the GitHub REST API[83]. The REST API responds with
a JSON version of the requested data or matching results of the search query. The results
are converted and filtered and translated into CSV file data that can be easily processed
in Excel. As mentioned in the paper by Z .Shen and M. Spruit, the whole process is
reproduceable by running the Python script [84] (Original Scripts). The script used for
this research can be found online on GitHub [85].
3.2 Search Term
A brief preliminary search shows that the logical search terms: “Visual Scripting
Language” and “Visual Programming Language” (“visual scripting language” OR “visual
programming language”) only produce limited results for the period 2001-2021, which
is not sufficient data. This can be demonstrated by the following example: a certain
rete.js VSL tooling from the domain web development cannot be found in an unfiltered
raw data set. Thus, the data collection has to be broadened by gathering data on topics
from the same time period. The following topics and search terms are included in the
search. See Table 4
3
https://github.com/PyGithub/PyGithub
25
Search Term or Topic
Query
Visual Scripting Language or
Visual Programming Language
(“visual scripting language” OR
“visual programming language”)
NOT Visual+Studio
Visual Scripting Language
topic:visual-scripting-language
NOT Visual+Studio
Visual Programming Language
topic:visual-programming-
language NOT Visual+Studio
Visual Programming
topic:visual-programming NOT
Visual+Studio
Visual Scripting
topic:visual-scripting NOT
Visual+Studio
Visual Programming Editor
topic:visual-programming-editor
NOT Visual+Studio
Dataflow Programming
topics:dataflow-programming
NOT Visual+Studio
Table 4: Describing the list of search queries for the GitHub REST API.
Mainly English terms are used, given that the majority of Git repositories are written in
English. The figure in the appendix A.2.1 shows the numbers returned by search for the
same search term in different languages on GitHub. English can be considered as the
main language in the game industry [86]. This is based on a collection of job offers on
jobs.gamesindustry.biz and gamesjobsdirect.com in (Appendix A.2.1). In conclusion, this
study is conducted with using English search terms exclusively.
All repositories that can be excluded with “NOT Visual+Studio” will be excluded since a
preliminary analysis of the search results shows that these are mostly an extension for
‘Visual Studio’ or ‘Visual Studio Code’. This is due to the GitHub search algorithm that
takes every word of a query and tries to find a match for ‘Visual’, which will mostly show
‘Visual Studio Extensions’ as a result. As a consequence, it is possible that Visual
Studio/Visual Studio code extensions that are Visual Programming Languages or Editors
are excluded from the findings, which is a known limitation of the study.
Besides excluding any repository aiming to be a ‘Visual Studio’ or ‘Visual Studio Code’
extension, this study further limits the search to repositories of the following
programming languages: C#, C++, C, JavaScript/TypeScript and Rust, since those
languages are more common in the game industry as implementation languages of the
engine or as scripting environments for the end user [17], [28] (Appendix A.1.3).
3.3 Limitations
This study has a few limitations. The first one is the time constrain of this research
project, which is one year. Among other limitations of contextual nature is that due to
the limited time scope, the focus is mainly on GitHub repositories since GitHub has over
200+ Million code repositories, which makes it the biggest source code hosting platform
out there [10]. Alternatives are Bitbucket and GitLab which offer similar services as
26
GitHub but with less popularity. Moreover, this study uses data from the period of 2001
to 2021, as discussed in chapter 3.1. Another contextual limitation is that a project needs
to have been updated at least once within the last 5 years. The reason lays in the claim
that technological evolvements follow Moore’s Law, which states that the number of
transistors that can be placed on an integrated circuit doubles every two years [87].
Therefore, projects that have not been updated within the last 5 years canbe seen as
outdated and irrelevant for the current technological development.
Moreover, important for the dependency analysis is a present and filled-in README file.
Otherwise, an analysis of potential dependencies is nearly impossible in C and C++
projects, since C or C++ do not have a standardized package manager
4
. This paper
excludes all other languages than C++, C#, C, JavaScript/TypeScript, Rust for the stated
reasons in chapter 3.1. These limitations apply both to the data collection process and
the data filtering process afterwards.
Projects that do not have a license will also be excluded from the research results since a
license is important for the game industry in terms of portability and ability to ship. For
example, GNU Public License [51] requires that the user of any project licensed with it to
also publish its source code. This may cause a conflict of interests with a game project,
therefore it is important for the license to be known.
Last, any archived project from the data set is discarded [41]–[43]. Moreover, since this
study is focused on embeddability, only projects that have a runtime and frontend or
just a runtime are investigated. Projects that do not have a runtime cannot be embedded
within other projects.
3.4 Data Extraction
The used REST API of GitHub exposes data to the external user. It has an extensive
search API that can be used to search for users, repositories topics and other
information for the social aspect of GitHub [83]. This paper only makes use of the search
API and the README file API end point [88]. Through the search API, basic information
of the repository on its own is collected by using the search queries mentioned above.
The data that is retrieved through the request is listed below:
4
C++ is notable among modern languages in not providing a package management system by which
developers can easily install and reuse code developed by others into their own projects. It shares this with
C, and for similar reasons. Until recently, source distribution was the exception, rather than the rule, and
pre-compiled libraries are fragile, as they must, in general, use the exact same set of dependent libraries,
compiled the same way. This largely limits reuse of packages to those provided by an OS or distro vendor. A
modern Linux or BSD system may provide 1000s of such packages, however changing any of them may
cause the entire system to fail. There is huge tension between Long Term Support versions and making
current packages available. http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p1254r0.html
(Steve Downey, 17.01.2022)
27
Data Type
Data Extracted
Description
Textual
Description
“A brief explanation of what
a repository does, what it is
for, and other items.”
README file
“A README file summarizes
the most important aspects
of the project. It generally
includes project name,
description, table of
contents, installation, usage,
contributors, credits, and
license.”
Languages
“Programming languages of
the source codes”
License
Source Code license
Numerical
Stargazers
“Stargazers refer to the
number of times a
repository is bookmarked. It
reflects an approximate
level of interest in the
repository.”
Forks
“A fork is a copy of a
repository. Forking is
necessary for developers to
contribute to a project.
Forks refers to the number
of forks.”
Issues
“Issues is the number of
open issues in a repository”
Date
Created
“The date that a repository
was created.”
Updated
“The latest date that a
repository was updated.”
Boolean
Has Wiki
Has a Wiki page.
Has pages
Has a page.
Archived
If the project is archived.
Table 5: List of data entries requested from the GitHub REST API.(Quoted explanations are from [11])
This study also checks each repository for a present doc, documentation folder, sample
or example folder. The README files or source code are analysed for dependencies and
if possible to extract them, several dependencies can be calculated.
3.5 Data Processing
The data from the systematic collection of GitHub projects is processed through the
search queries and criteria, described in Chapter 2.2.2. The data processing pipeline is
defined as follows:
1. Data filtering based on the limitations discussed in chapter 3.3.
2. Manual filtering of the collected projects to eliminate any nonvisual scripting or
programming language / visual scripting or programming environment.
3. Text analysis of the downloaded README files to extract the dependencies.
4. Source code and text analysis to categorize the execution model.
28
5. Summarizing the results and putting them into perspective to form the base for the
following interviews and Q-Methodology.
3.6 Data validation
The results of the systematic review are validated through a series of interviews with
domain experts to evaluate the relevance of the potential criteria. Validation is needed
to increase the spectrum of findings since the systematic review includes none game
industry visual scripting languages. The interviews also provide the opportunity for
other criteria to be discovered and/or discussed. Thus, the interviews are finalized with
a Q-Methodology [89] to be able to evaluate and put the opinions into perspective.
3.6.1 Q-Methodology
This study uses a Q-Methodology to provide a framework to evaluate the subjectivity of
interviewees’ opinions towards a series of statements about criteria of embeddability
[89]. These statements are listed below and derived from the criteria of embeddability
for SL in Table 1-2 in Chapter 2.2.2. Moreover, they are combined with the findings
about VSL and how they fundamentally differ from SL, as discussed in the literature
review.
1. The smaller the API to bind data types, functions (e.g. nodes) to the language, the
better.
2. When compiled, the binary footprint of the language should be small.
3. The best language has a few or no external dependencies.
4. A visual scripting language should come with an editor.
5. The API should be flexible enough to bind an external editor.
6. The visual scripting language should compile to native code.
7. The visual scripting language should transpile to a language such as C or C++.
8. Tree-walking interpretation should be avoided.
9. Script hot reloading is important when choosing a visual scripting language.
10. Direct memory access (e.g. via pointers/references) from the host language to the
visual scripting language is important.
11. The visual scripting language should have a C API.
12. Online documentation is important for embedding and extending the language.
13. Header/source file documentation is important for embedding.
14. Being able to control the memory allocation of the visual scripting language is
important.
15. Examples on how to bind and extend the language are important.
16. The license needs to be permissive.
3.6.2 Interview Guide
The first step of the data validation process is a in-depth interview with industry
professionals. The interview format is based on the methodology described in “In-depth
interviews for game research” by Amanda Cote and Julia G. Raz from the book “Game
29
Research Methods” [90, pp. 93–116]. The format of the interview follows the advice that
Lindlof and Taylor gave in their work: “Questions are potent tactics for starting
discourse along certain tracks or for switching tracks later. They can be used to open up
a shy person or to persuade a talkative respondent to speak more economically” [91, p.
194] [90, p. 103]. Thus, the interviews are “structured” like a conversation. The
following overview shows a possible interview structure based on the potential data
found.
Introduction
• The interviewer introduces themselves and briefly presents the research work.
Important here is to not give too many details. The intention of this way of
introducing is to start the conversation naturally and to take away any potential fear
or shyness of the participants.
• Overview of the timeline of the interview: 25-30 minutes.
• Discussion of privacy regulations.
• Ask about how long they work in the game industry and in what position.
This information is important data points for the assessment of the statements.
Conversation
The intent of this section is to start the actual conversation by steering the introduction
into the right direction in order to ask questions about the importance of the 16
statements from the Q-Methodology section. The order of the questions asked as well as
the way they are phrased might differ from interview to interview.
• Do you often use external dependencies within your project(s)?
• Could you name some of the common aspects of them?
• When you start a project / look for a solution, what do you consider important
when choosing external dependencies?
• Is one of your external dependencies a VSL?
• If yes, what made you choose this one?
• If no, have you ever considered choosing an external VSL?
• If you went to GitHub and looked for a VSL, what would make you choose it?
• How are you using VSL within your projects?
• What is their biggest issue for you?
• If you would implement your own VSL, how would you do that?
• What would be the things that are important to you here?
Important for this kind of interviews is to ask open questions. When the interviewed
person gives a relevant answer, subquestions are to follow.
Data and results of the systematic review
Data findings of the initial search with the queries stated in Table 2 result in 413
projects. The data was collected on December 30th, 2021. The collected data is not yet
filtered besides the basic filter attributes described in section 3.3 Limitations and in 3.4
Data Extraction. Therefore, this dataset needs to be filtered.
30
4.1 Data Cleaning
The filtering process reduced the number of projects from 413 to 239. The definitive
number of remaining projects is 15. The criteria used at the end are that most projects
need to have a runtime alongside their frontend. The filtering process described below
is exactly following the method from Chapter 3.4:
• Filter out all projects that are unlicensed: 239 projects are left
• Filter out all projects that do not have a README file: 234 projects are left
• Filter out all projects that are not made of one of these languages – C, C++, C#,
Rust, JavaScript/TypeScript: 142 Projects are left
• Filter out all projects in which the README file, project description or source
code does not provide enough information to be extracted: 133 projects are left
• Filter out all projects that are “off-topic”: 86 projects are left
• Filter out all projects that are discontinued: 68 projects are left
• Filter out all projects that do not provide a runtime: 15 projects are left.
4.2 Data Overview
This section will present the results of the systematic review of GitHub repositories of
the filtered dataset of 15 projects. These 15 projects are of three major types: language
framework, languages, and specific editor. A language framework can be used to create
your own language, while a language represents a fully featured VSL. The specific editor
indicates that this is a VSL that comes with a specific editor, but it is possible to integrate
the underlaying runtime in another project.
Project
Project
Type
Execution
Model
Frontend
Language
Runtime
Language
retejs/rete
Language
framework
Tree-Walked
Interpreted
TypeScript
C++
zenustech/zeno
Specific
Editor
NA
C++
C++
chigraph/chigraph
Language
Compiled
C++
C++
Nicolas-Constanty/Dnai.Editor
Language
framework
Bytecode
Interpreted
C++
C#
vuo/vuo
Language
Bytecode
Interpreted
C++
C++
kovacsv/VisualScriptCAD
Specific
Editor
Tree-Walked
Interpreted
C++
C++
kovacsv/VisualScriptEngine
Language
framework
Tree-Walked
Interpreted
C++
C++
fibo/dflow
Language
framework
Tree-Walked
Interpreted
TypeScript
TypeScript
bonsai-rx/bonsai
Language
Compiled
C#
C#
mocabe/yave
Language
Bytecode
Interpreted
C++
C++
kovacsv/VisualScriptEngineWeb
Language
framework
Tree-Walked
Interpreted
C
C++
31
XBagon/PIPES
Language
Bytecode
Interpreted
JavaScript
JavaScript
mehecip/mbd
Language
framework
Tree-Walked
Interpreted
C++
C++
drwhut/decision
Language
Bytecode
Interpreted
C
C
tzklab/vsxu
Language
Transpiled
C++
C++
Table 6: Overview of the projects including their name, type, execution method, runtime language and frontend
language
4.3 Language Observations
Figure 1: Language distribution between frontend and runtime
Figure 1 shows how the languages are distributed across the runtime and frontend
implementation. C++ shows the greatest number of usages. One can say that nearly all
projects that are using C++ in their frontend are also using C++ in their runtime. It can
also be observed that except for three, all projects match their runtime and frontend
implementation. The exceptions are shown in Table 7.
Project
Frontend
Language
Runtime
Language
kovacsv/VisualScriptEngineWeb
C
C++
Nicolas-Constanty/Dnai.Editor
C++
C#
retejs/rete
TypeScript
C++
Table 7: Exceptions of frontend language vs. runtime language
32
4.4 Execution Model
Figure 2: Overview of the execution models of projects with runtime.
Figure 2 gives an overview of the different execution models. The most projects (6) fall
under the category “Tree-Walked Interpreted”, followed by 5 projects that are Bytecode
Interpreted.
Figure 3: Relationship between the runtime implementation language and the execution method
33
Figure 3 shows the distribution of runtime implementation languages in relation to the
execution model. It can be noted that most C++ projects are implemented as a Tree-
Walked Interpreter while the distribution amongst Bytecode Interpreted finds a project
in all languages, except for TypeScript.
4.5 License distribution
Figure 4: Overview of the license distribution across the projects with runtime
Figure 4 shows an overview of the distribution of different licenses across the collected
projects. Most projects are using the MIT License. The MIT License is a permissive free
software license originating at the Massachusetts Institute of Technology in the late
1980s [92] [93].
4.6 Documentation and Samples
Project
Has
Wiki
Has
GitHub
Pages
Has Online
Documentation
Samples
drwhut/decision
TRUE
TRUE
TRUE
FALSE
kovacsv/VisualScriptEngineWeb
FALSE
FALSE
TRUE
FALSE
bonsai-rx/bonsai
TRUE
FALSE
TRUE
FALSE
vuo/vuo
FALSE
FALSE
TRUE
TRUE
tzklab/vsxu
TRUE
FALSE
TRUE
FALSE
mocabe/yave
TRUE
FALSE
FALSE
FALSE
XBagon/PIPES
TRUE
FALSE
FALSE
FALSE
Nicolas-Constanty/Dnai.Editor
TRUE
TRUE
TRUE
FALSE
kovacsv/VisualScriptEngine
TRUE
FALSE
TRUE
FALSE
kovacsv/VisualScriptCAD
TRUE
FALSE
TRUE
FALSE
34
chigraph/chigraph
TRUE
TRUE
TRUE
TRUE
mehecip/mbd
TRUE
FALSE
FALSE
TRUE
zenustech/zeno
TRUE
TRUE
TRUE
FALSE
fibo/dflow
FALSE
TRUE
FALSE
TRUE
retejs/rete
FALSE
FALSE
TRUE
TRUE
Table 8: Overview of all projects and the presence of documentation
Table 6 indicates that no project comes without some form of a documentation showing
how to use their project. Important to note is that README files are excluded from this
listing. A doc folder within the repository can be seen as a website, since it is part of the
GitHub repository view and is therefore considered as online documentation.
Count
Has Wiki
11
Has GitHub Pages
5
Has Samples
5
Has Online
Documentation
11
Table 9: Overview of projects with runtime and if they have any form of documentation
Table 7 shows that it is more common among the collected projects to have a wiki on
GitHub or to provide an online documentation.
4.7 Dependencies
Figure 5: Overview of dependencies of projects with runtime in relation to their language
Figure 5 lists the number of dependencies per project. Important to note here is that it
was not always possible to separate the runtime dependencies from the frontend
dependencies, since some projects have their runtime living within the project of the
frontend, while other have theirs in a separate project. Also, it was not possible to
35
evaluate the dependencies of 3 projects. This leads to the conclusion that the data
cannot deliver a reliable answer to the question of how many dependencies a project
should have.
4.8 Form
Diagram/Node based projects are the most common projects among the collected 15
projects. The only non-form is XBagon/PIPES, which is an esoteric VSL and cannot be
classified as a specific form known by literature.
4.9 API Complexity
The collected API complexity was calculated through the number of classes or functions
needed in order to bind, extend or embed the runtime. Results show that C projects have
the least class/function count while C++ projects and C# projects have the most.
TypeScript has the third highest class/function count. These findings must be seen
critically since the definition of complexity is not clearly set within literature nor in the
industry.
Data and results of the interviews and the Q-
Methodology
The sample size of domain specific professionals is 8 participants. The interviews were
conducted between the 24th of March and the 06th of April 2022. Most of the participants
have their residency within Europe. Also, the interviewees are on different positions and
specializations within the programming aspect of a game development team.
5.1 The participants’ general views
The participants in the study are listed in Table 8 and have various backgrounds and
different opinions on the topic of VSL in general.
Interviewee
Years of experience
Current Position
#1
~ 5
Senior Software Engineer
#2
~ 8
Indie Developer
#3
~ 8
3D Programmer
#4
~ 9
Senior Graphics Engineer
#5
~ 17
Chief Technology Officer
#6
~ 15
Senior Software Engineer
#7
~ 21
Games Technical Director
#8
~5
Software Engineer
Table 10: Overview of the interview participants
36
Mixed feelings towards the use of VSL
All the participants have expressed mixed feelings towards the use of VSL when
developing a game. Most of the participants stated that VSL are good for gluing code
together. Participants #2 and #3 shared that they use VSL to glue their game code with
other parts of the application such as the UI or to react on certain events.
Difficult to create a ‘one size fits all’ solution
A considerable number of the interviewed professionals are of the opinion that the
requirements of a VSL highly depend on the project the language is intended to be used
for. Participant #6 summarizes this as: “You have to be very opinionated about what you
actually support. The things you support may or may not be useful for someone else”. In
other words, they conclude that a generic solution might be difficult to achieve. This
view is shared by Participant #7, while in their formulation they spoke from a
gameplay/game programming perspective rather than a lower-level engine
programming perspective.
VSL are a great solution for domain-specific languages
Participants agreed on the point that VSL are good to be used as domain-specific
languages rather than general-purpose languages (#4). When used as DSL, VSL are “[…]
superior to text when it comes to expressing sequences of things, because you can lay
things out spatially in a way that it can’t be done in text.” (#5)
Performance and memory access is important
Another common theme among interviewees is performance and mainly the view that
performance is one of the biggest bottlenecks within the game industry. This is related
to memory access patterns as Participant #1 clearly states: “memory access is the main
bottleneck for everything in games” (#1). Therefore, when talking about memory access,
the actual question is about performance (#5).
Tools such as a debugger are critical
During the interviews, participants reported that tooling and workflows around the VSL
are as important as the capabilities of the VSL on its own. Participant #7 states that if
one creates a VSL as a library, only 1% of the whole work has been done. What they
share is that the bulk of the work for a VSL lays in the editor and the debugger. These
views are supported by Participant #2 and #3.
The main theme that can be identified lays in two aspects: discoverability of what is
happening within the graph, therefore debuggability (does the VSL come with some sort
of a debugger) (#3,#2,#4), as well as diffing tools and change tracking. Participant #4
expresses that it is not only important to be able to manage the source code of the VSL,
but it should not be of binary nature since this would make comparing differences with
common version control systems quite difficult. They state that a JSON file-based or
similar text-based file format approaches are preferred.
Execution method / Compilation method
37
Generally, most participants have an opinion about the execution method. For some, it is
not that important, as long as the requirements of the game are met. According to
Participant #2, a bad compiled bytecode can be as slow as a tree-walked interpreted
VSL. Other participants believe that the execution method is very important. From this
information, two statements can be made. First, just-in-time compilation should be
avoided due to possible platform limitations, while ahead-of-time compilation should be
preferred. Second, bytecode interpretation during production time, in combination with
hot reloading is welcome, since this may enable fast iteration times. While for shipping
some form of compilation either to bytecode that can be shipped as data or transpiled to
a lower-level language such as C++ or C, is needed to gain back the performance that is
required. Although, Participant #7 states that this highly depends on the needs of the
game. Participant #2, #6 and #7 all claim that being able to ship bytecode as a form of
patch that the application can recognize and use instead of the compiled C or C++ code,
can be very beneficial on consoles. This can be used to hot fix crucial bugs, e.g. a door not
opening, without having to ship a new executable that needs to go through validation
and TRC checks.
Summary: Generally speaking, participants from the chosen sample use VSL for gluing
different aspects of the game world together. They mostly agree with the statement that
there are many different requirements towards a VSL, which makes it hard to create a
general solution in their eyes. Moreover, the interviewed industry professionals state
that VSL are great for being a DSL. From then on, memory access is one of the most
common bottlenecks within the game industry when it comes to SL or VSL. Also, for the
experts, the tooling and workflow side of a VSL is very important. The main points
regarding tooling that were mentioned are debuggability and source control
management. Last, the execution method was a controversial topic, where views vary
from ‘it is not important’ to ‘it is crucial’ for performance aspects. The main consensus
was that a VSL should compile AOT rather than JIT and a way to compile to native code
via C or C++ should be provided, if required.
5.2 Q-Methodology results
This section presents the statements of the Q-Method ordered by relevance. The
relevance has been determined by the number of times each of the statements has been
placed along the Q-Method board. Moreover, the statements are supported by actual
comments from the participants.
Script hot reloading is important when choosing a visual scripting language
This is considered as very important during the interviews and the Q-Method.
Participant #5 states: “Hot reloading is important when choosing a visual language” (#5).
Participant #7 also says that hot reloading is important during the development cycle,
even though developer machines are quicker than the normal user’s machines, therefore
an extra step for reloading a script manually would still be an issue.
The smaller the API to bind data types, functions (e.g. nodes) to the language, the
better
38
This is considered as an important factor when it comes to embedding a VSL. Participant
#1 shares their understanding of the term ‘small API’: “A concise API means that there is
a thin layer of code between the actual execution of the visual scripting language and, for
example, the host program written in C++”.
When it comes to the specifics of binding data to the VSL interviewees have matching
views such as: “I would make the binding API really simple.” (#8). The main views on
having a small API design are reflected in the Q-Method results. However, there have
been critical comments such as “The smaller the API to bind data types, functions, nodes,
the better, does not really matter. It depends on how complex the language is and how well
integrated it is into whatever underlying system.” (#7) .
Being able to control the memory allocation of the visual scripting language is
important
Participants have identified performance as a bottleneck within the game industry,
which Participant #1 formulated as: “Memory access is the main bottleneck for everything
in games” which is reflected in the statement’s position within the sorting of the Q-
Methodology results. In most cases, participants would expect to be able to pass an
allocator via the API to the language. Participant #3 states that memory management
and access is important and they would require some form of an interface to control
eventual internal memory allocations.
There have been participants who mentioned that a VSL should not expose memory
allocations to the end user or alternatively should get a block of memory to work with.
For example, Participant #8 thinks that memory management should not be exposed in
a VSL, so the user does not have to be concerned with memory management and so a
non-expert can use the language.
Participant #6 asked the question: “Why would my graph actually even be allocating
memory?” since according to them, a VSL is more of a functional language. They argued
that a block of memory can just be passed towards the runtime which will then handle
it. As an existing example they mentioned WebAssembly, which handles memory in a
similar way. Lastly, they find this method of memory handling “… not super, super
complicated, but you basically just give it a block of memory.” (#6)
The license needs to be permissive
All interviewees agree that a license of a dependency used for a commercial product
should be permissive, which is aligned with the results of the literature review. When
asked about licensing, Participant #4 explains: “Certain licensing models, for example
GPL, impose an inherent risk that if somebody claims they want to see your source code,
you would need to comply with their demands.”. Consequently, this could expose trade
secrets or code that is under NDA. This claim is very much aligned with the views
Participant #3 expresses: “A license needs to be very permissive. Something like MIT.”.
The API should be flexible enough to bind an external editor
This statement was only discussed by three of the participants, which makes its high
scoring in the Q-Method rankings interesting. Participant #2 shares that they would
39
rather integrate the language into their own engine. They add that maybe a TCP
connection to the engine might be a good way to communicate with the VM /
Interpreter. On the other hand, Participant #3 would expect the language to come with
an editor, but under the condition that the API should be able to bind a custom editor.
Participant #5 agrees that “… the API should be flexible enough to bind an external editor.”
The visual scripting language should have a C API
The main reason why participants expect a C API is because C is de facto the lingua
franca for FFI bindings. Along those lines, Participant #2 argues that one can compile a C
program on certain types of hardware or nearly on every platform. This means that it
can be bound to most other languages. Participant #6 adds that if the language is just an
interpreted VM, then it could be portable if it is written in C and one can pass bindings to
cover other languages as well. Participant #3 shares similar views.
The visual scripting language should compile to native code
This statement led to some confusing answers since it was often mistaken for the
statement: the visual scripting language should transpile to a language such as C or C++.
Therefore it is rather difficult to make a proper conclusion based on this statement.
There are many comments on how compiling to native code is important but at the same
time, native code and compiling to C or C++ is being used interchangeably. The
statement was in fact intended to express that the VSL is compiled to actual machine
native code.
When compiled, the binary footprint of the language should be small
There has only been one comment on the binary footprint made by Participant #7,
where they say that the size of the binary footprint is not anymore as important as it
used to be, but that does not mean that the binary footprint should be ‘bloated’.
A visual scripting language should come with an editor
The overall expectation among the interviewed developers was that a VSL comes with
an editor. Participant #2 comments that a VSL should clearly come with an editor.
Participant #7 argues that a designing a language without an editor would be just 1% of
the whole VSL design process. Participant #6 states that “… (a VSL) should come with an
editor, but embeddable. Because a lot of engines have their own editor […], our editor
should be able to be embedded in their editor.”
The best language has as few or no external dependencies
The developers participating in the survey stated that not only the number of
dependencies is important but so is their use case. Using external libraries that come
with a large amount of their own dependencies is usually not recommended, as
Participant #1 stresses: “Usually, we do not like big libraries, like ‘boost’, it is a big red flag
for us.”
Moreover, the maintainability of those external dependencies is a big factor as
Participant #1 explains: “Usually, big systems we make ourselves because if a big system
has a problem, it is very hard to go in and fix it. Also, if you have made it yourself, usually
you know what it is about. In addition, you do not have a lot of features that you don't
want anyway, so that the implementation ends up being smaller as well.” (#1)
40
The same reason was given by Participant #4, and that is why they usually fork external
dependencies, which gives them back the control and enables them to have access to the
source code. #4 also express that “it is very simple when you develop a commercial
product” but one needs to keep two things in mind at any time. First, “dependencies
might be taken offline, so it is very important to have your own copy of the dependencies.”.
Second, having a local copy is important for potential certification processes or security
audits.
Another important factor of keeping no to a few dependencies is the platform, as
Participant #7 states.
Direct memory access (e.g. via pointers/references) from the host language to
the visual scripting language is important
There are mixed views on this. On the one hand, participants state that being able to
expose pointer / references directly to the VSL would be beneficial: “I have integrated
scripting engines in the past where you had to have communication going both ways. […]
let's say there is an object created in the scripting system, call it Sprite, and then that Sprite
is owned by the scripting system and a reference to it is given to some engine system. Let's
say, it's a UI thing, you have a scripting system that loads it.” (#2). In this example, the
scripting runtime would own the memory.
On the other hand, there are participants who asked: “What would you do with that
pointer?” (#6). The conclusion is that if a system can reference memory owned by the
runtime, this may lead to ownership issues and if the runtime can understand external
pointers, this might cause type safety issues.
The visual scripting language should transpile to a language such as C or C++
As mentioned before, this statement was sometimes mistaken for the native code
translation statement. Although, most participants state that as an AOT finalizing step
transpiling the graph to C++, which is then compiled, is crucial for the shipping release.
Participant #8, for example, states that one does not need to invent a new backend but
can simply compile to C or C++. Participant #7 agrees by claiming that transpiling to C or
C++ for a shipping release is crucial. Participant #6 discusses the reason for compiling to
native code more in depth. They state that if one writes a library that is just an
interpreted VM, which executes bytecodes, this will be slower by a factor of 10 than the
native code execution. Participant #6 thinks that it is important for most game engines,
to consider the execution performance difference. This is why #6 argues that having
some sort of pipeline that allows for data to be converted into native code, would
certainly be a big improvement for a visual scripted language.
Examples on how to bind and extend the language are important
Participant #3 thinks that samples are more important than online documentation, since
samples give an indication on how flexible the API or the project is. They state that
samples can show in either more complex or simple manner how an API is being used.
These examples are very important when it comes to binding the language to an existing
application such as a game engine.
Tree-walking interpretation should be avoided
41
The term “Tree-Walking Interpreter” was noticeably not known by many of the
interviewees and needed some further explanation. The general view on this topic was
that a tree-walking interpreter, as described in “Crafting an interpreter” by R.Nystrom,
should be avoided. When Participant #3 understood the meaning behind the tree-
walking interpreter, they agreed that it should be avoided.
Participant 2 stated explicitly that the result is more important than the method: “I think
the answer is no, it depends more on the outcome – if it is precompiled but at the end
terrible versus something that is runtime evaluated but great. I'm going to pick the
runtime evaluated.”.
Online documentation is important for embedding and extending the language
Not many interviewees commented on this subject. The main reaction to the statement
was that it is important but without explaining why. The only two comments given are
by Participant #5, who rates them as important for embedding a language, whereas
Participant #1 finds them less important than header files but sometimes needed to
understand how certain aspects of the API are to be used.
Header/source file documentation is important for embedding
There are no distinguishable comments to this statement, since they are always
discussed in the same commentary block as the Online documentation and samples.
Discussion of data and results
This chapter will discuss the findings of the two conducted studies, the systematic
review of GitHub repositories together with the interviews & Q-Methodology. First, the
validity of the presented results will be discussed and then the results themselves.
Finally, the chapter will end with a summary of all the findings of this research.
6.1 Data Validity
The dataset
GitHub has millions of repositories, but the platform is not the only source of project
repositories in the open-source community. There are several others, such as Bitbucket
or Gitlab, but due to time limitation reasons, these services were not used. As a result,
this study is solely based on GitHub projects. Moreover, this study’s initial dataset
consisted of 413 repositories, which is just a small fraction of all the possible
repositories on GitHub. Due to the fact that a repository had to match all requirements
described in Chapter 3.3, projects that have a license but did not state it in a GitHub
recognizable way were discarded, since they were not delivered as part of the REST API
results. Moreover, GitHub sometimes does not recognize double-licensed projects, same
as any project where the license file is not called LICENSE or LICENSE.md. Also, projects
that did not state in their description, name, or topics that they are VSL-related might
have been missed. This could not be avoided since a more thorough search requires
more manual work, which could not fit the time frame of this study. Nonetheless, some
of the filtering was done manually by the author of this study, since the limited time
frame did not allow for training an AI to run through and process studies or to use
42
extensive code analysis tools over the codebases. Therefore, the research is subjected to
errors and inaccuracies made by the author. To mitigate this threat, there have been
discussions led with the study supervisor.
The interviews
The main validity threatening factor here is the time limitation since this study required
a solid amount of time to perform a systematic review and process the data. That is why
there was only a limited amount of time to conduct in-depth interviews with a small
subset of industry experts. With the time limitation in mind, it was risky to use a
potentially better data collection approach, i.e., focus groups. In any case, the focus
group approach would have required a large amount of coordination, considering time
zone differences, for example and would have potentially altered the results by means of
the willingness of industry experts to answer certain questions in a group setting.
6.4 Ethics
Textual programming languages are predominantly languages that make use of English
keywords. This might limit the accessibility of programming exclusively to people who
speak English. An article from Wired called “Coding Is for Everyone—as Long as You
Speak English” [94] argues that learning to program depends mainly on if you are
capable of speaking English since the programming eco system is basically in English.
Some IDEs such as Visual Studio do support other languages but not all. Also, when
searching online for errors and solutions in a different language than English, the search
results are sometimes different or linked to websites/articles in English.
Moreover, the wired article states that there are not many multilingual programming
languages. All the languages the article mentions are actually visual scripting languages,
such as block-based languages like Google’s Blockly [65] or Scratch [95]. Therefore,
visual scripting languages have the potential to empower people interested in the game
industry to work in their mother tongue.
6.3 Discussion of the results
Within the dataset, comprised of the literature review, the GitHub systematic review results,
and the Q-Methodology impressions, the following themes have been identified:
performance, mechanical aspects of embedding software, documentation, licence and
tooling and workflow.
6.3.1 Performance
The most frequently reoccurring theme within the interviews and the Q-Methodology and
backed by literature, is the need for performance. In literature this can be found in the works
of R.Nystrom and J.Gregory’s ‘Game Engine Architecture, Second Edition’. R.Nystrom has
summarized in ‘Game Programming Patterns’ the following: the game industry is on a hunt
for the last drop of performance, which is why it strives for optimized solutions [44]. This is
also reflected by the results of the Q-Methodology as well as the accompanying interviews.
As it becomes clear, results can be separated into multiple layers of performance:
43
development performance, runtime performance, memory management, direct memory
access and compilation method.
Development performance
Development performance is considered essential for the game industry and is therefore an
important aspect of the potential criteria of embeddability of a visual scripting language. The
interviews and the results of the Q-Methodology support this by discussing the importance
of script hot reloading. Literature backs this up by reviewing the iterative process within the
game industry [41, p. 60]. Another aspect here is that hot reloading / hot patching might
help with avoiding unnecessary TRC check-ups on consoles since shipping a data patch
might not require a TRC check, while an application patch might do. The ability to hot reload
scripts could not be determined during the data collection of the systematic review.
However, the found literature on this supports the results of the interviews: the “ease of use”
[1, pp. 956–957] [36], [37] aspect of embeddability of scripting languages states that one
aspect of using SL is that the developers should not wait. Moreover the, “support for rapid
prototype” [1, pp. 956–957] [36], [37] aspect of embeddability of SL is an accurate
description of these findings. Therefore, it can be argued that both aspects – ease of use and
support for rapid prototyping are valid and very important for the use of VSL.
Runtime performance
The next aspect is the runtime performance during development. This is somewhat
related to the previous performance aspect since in this aspect, the choices are about
how the VSL will be executed. This leads to the question of what method should be used
to execute. There are three approaches addressed in literature and in the interviews,
namely the tree-walking interpretation, bytecode interpretation and transpiling. In tree-
walking interpretation [6, pp. 3–4] [58, p. 17] the script is not compiled to some other
form, but executed “node” by “node”. This is inherently slow [44, pp. 155–179] [64, pp.
25–26] since the graph may traverse through constantly looking up pointers. In
contrast, bytecode interpretation is faster since the opt-codes are densely packed [55],
[58, pp. 16–20], [44, pp. 155–179] ,[1, pp. 52,954-958]. The last method is to transpile
the graph to another language or directly to machine code. This approach has the
downside that recompiling a graph in a lower-level language than just to bytecode,
might be slower (#1). Interviewees argue that neither of those methods are always
preferable, since it depends entirely on the needs of the game project (#2,#3,#6,#7).
One interviewee describes that a method to mitigate any performance issues would be
to use a bytecode driven VM for during production but when publishing the game, the
bytecode gets compiled into native code of the platform (#6). The described methods
are connected to the aspect of development performance since rapid prototyping during
development can be supported. This is due to the fact that a bytecode driven VM can
compile quicker, while the performance can be leveraged with highly optimized native
code for the shipping build. The findings of the systematic review revealed that more
projects of the dataset are using the tree-walking interpreter method, in contrast to the
results of the Q-Methodology. This might be because some of the dataset projects are
dataflow libraries that can be also used as VSL frameworks. On the other hand, literature
and the interviews state that tree-walking interpreters should be avoided for runtime
performance issues. However, it can be argued that the proposed method with taking
44
the VM out of the equation by compiling at the end to native code could also work for a
system like this.
Memory management
Memory management is a very important aspect of performance, and it influences all the
aspects mentioned above. The results of the interviews and the Q-Methodology support
this claim. However, the views on whether a VSL should be able to allow the user to
allocate memory, differ. Important to mention is that the systematic review could not
determine if any of the analysed projects allow the user to allocate memory due to time
limitations. Nonetheless, most interviewees would expect to be able to either give a pre-
allocated block of memory to the runtime, where the runtime handles it, like in
WebAssembly, or pass along some form of an interface that exposes an external memory
allocator, for example, the engine to the runtime.
Direct memory access
The direct access of memory is related to memory management. This has been ranked
quite low during the Q-Methodology and the views on direct memory access are mixed.
On the one hand, having a way to communicate from the host application to the runtime
and vice versa might be important and needed. However, it can complicate things in
regard to memory ownership and lifetime. Another concern about direct memory access
is type safety. From the moment on a pointer that is owned by the host application is
passed along to the runtime, the runtime cannot know what is behind the type-erased
pointer unless it stores some form of type information, which will make the API more
complicated. The general view presented by the interviewed industry professionals is
that in the case described above, the runtime can only pass the pointer along to a
function in the language that has been assigned to the runtime.
Compilation method
The results of the systematic review are not detailed enough to draw deep conclusions
since it has not been investigated whether a bytecode driven project uses just-in-time or
ahead-of-time compilation. Literature suggests that that a JIT compilation is faster in
terms of development speed, since the runtime only compiles the functions that are
about to be called; while in an AOT compilation, everything is compiled [6, p. 285], [59].
However, what interviews show is that AOT compilation is preferred since it can run on
all possible platforms, while there might be proprietary platforms that do not support or
do not allow JIT compilation in their TRC [60]. As mentioned before in the runtime
performance part and the development performance part, the right compilation method
is important for the development speed and the execution speed and tightly linked to
the requirements of the game and to the platforms it will run on.
6.3.2 Mechanical aspects of embedding software
API requirements, the language chosen to implement or expose the API (runtime) and
the use of dependencies are the mechanical aspects of embeddability. These are the next
aspects to be discussed based on the results of the systematic review, the interviews and
the literature review.
45
API design
The Q-Methodology has shown that the design of the API should be rather small and
concise. None of the participants has expressed how a small or concise API looks like.
Ierusalimschy R. et al. and other researchers have described that the function and class
count of an API is important to determine the simplicity of an API [34], [96]. The
gathered data in the systematic review does not contribute significantly to defining
‘small and concise API’. The definition of ‘small’, ‘concise’ or ‘simple’ depends entirely on
the context and requirements of the game, which is a major point that Participant #7
stressed during their interview. The demand of a small and concise API might stand in
contrast to other preferences about the API design, such as flexibility. The API must be
flexible enough to expose the VSL to an external editor or allow an editor to be written.
Also, the ability to visualize the flow of the language, to debug actions such as
breakpoints or to step through has been expressed as important.
C API and implementation language
The interviews, the Q-Methodology and the literature review suggest that an API should
be written in C or at least expose a C interface. Literature suggests that the C
programming language is the lingua franca of programming languages [56] since most
programming languages can interact with C via their foreign function interface (FFI)
[29]. What makes C a great choice for exposing the API in it is that C qualifies as a highly
embeddable language due to the general great support of other languages. This
statement is supported by the interviewees. According to them, choosing C is a good way
to guarantee portability to certain proprietary platforms, since most platforms, even
older ones, support some subset of C. Important to note is that participants in the Q-
Methodology have expressed the opinion that the exposed API should be in C, but the
actual runtime can be written in C++, since C++ provides some nicer higher level
concepts without runtime penalties. Other modern languages such as Zig or Rust are
discarded out of portability reasons for platforms since there might be proprietary
platforms which do not allow such languages to run on them
5
. This view differs from the
gathered data from the systematic review since most projects there have been written in
C++. This can be seen critically since to determine if a project uses a particular language,
the GitHub REST API data has been used. GitHub puts a project in a certain category if
the majority of the code has been written in this language. This means it is possible that
if the runtime is part of a bigger project, the runtime could be implemented in C but if
the rest of the project is in C++, the whole project counts as C++.
Dependencies
The findings of the systematic review indicate that a C++ or a C project can have a few
dependencies. During the data gathering process the size of the dependencies in terms
of how many dependencies these projects have, has not been evaluated due to time
limitations. This stands in contrast to what the industry professionals report in the
interviews. The general opinion is that dependencies can be an operational risk. The
main points of concern are the loss of access and maintainability in case a project goes
5
Since the TRC requirements are under NDA it is not possible to provide official backing data
besides posts on internet forum: https://users.rust-lang.org/t/is-there-a-reliable-way-to-
support-mainstream-console-including-std/64451 or
https://news.ycombinator.com/item?id=14087639
46
offline. To prevent this, forking a project has been suggested. Another use of a local copy
is for potential certification processes or security audits. Moreover, the game industry
tends to write big systems to avoid the previously stated issues and be in full control of
the source code.
6.3.3 Documentation
The systematic review shows that all projects have some form of documentation, which
varies from project to project. Some projects just provide a README file and no other
information, while other come with a wiki or external documentation. Moreover, some
come with samples on how to use or bind the project. The results from the systematic
review have indicated that some form of online documentation, either a folder with
documentation files such as markdown or a wiki, is preferable. This stands in contrast to
the Q-Methodology results since participants mentioned that header file documentation
is important but sometimes that this is not enough, and then online documentation
comes into play (#1). Moreover, samples have a high value of seeing and exploring the
complexity of an API to see how much effort integrating would take.
6.3.4 License
The results of the systematic review as well as the results of the literature review
suggested that a permissive license would be beneficial. The MIT license has been used
within the dataset of this research most frequently. The interview results draw the
same conclusion, since the choice of the right license is important for the business:
“Certain licensing models for example GPL, impose an inherent risk that if somebody claims
they want to see your source codes, you would need to comply with their demands. […]”
(#4). This could expose trade secrets or code that is under NDA, as one of the
interviewees has phrased it: “no GPL, otherwise it’s getting way too much work for
Legals” (#3).
6.3.5 Tooling and workflow
As mentioned earlier in Section 6.3.2 about the API design decisions, it seems to be
important to take tooling into account when designing the API. The main questions
raised are: How will this language be integrated into my game engine, IDE or general
workflow? Will the language have its own editor?
Participant #7 summarized it with the following statement: “If you do a visual scripting
language as a library, then you are just down with like 1% of the work you need to do. The
rest of the work lays actually on tooling such as building all the debugging functionality
like stepping through things and visualizing values” (#7)
Therefore, with every decision made within planning of the VSL, potential tooling needs
to be taken the into account. There have been multiple suggestions on how a UDP/TCP
driven protocol can be used to communicate between the runtime and an external
editor, but this might not be aligned with some platform TRCs. Another aspect next to
potential editor design and communication concepts was the discussion around source
control management. Participant #5 suggested that binary blobs are bad for diffing in
47
source control and a file format such as JSON would be simpler to diff in a diffing tool.
There are more points about the tooling aspect to touch upon but to discuss them all
lays out of the scope of this research due to the time and resource limitations. Therefore,
more research is needed for exploring the needs of a VSL in terms of tooling.
6.3.6 Purpose of VSL
One common theme across all interviews was the skepticism of the participants towards
a VSL in general. However, they all see the benefits stated in literate that a VSL is great
for prototyping and quick iteration times. Also, a VSL is a perfectly fit for certain tasks
such as processing sequences while for others it might be not the right tool.
“A visual scripting language is superior to text when it comes to expressing sequences of
things, because you can lay things out spatially in a way that you can't in text-driven
programming languages. The spatial dimension you have is one dimension vertically and if
you have multiple flows somehow then you will interpret these lines with each other. So
what you do when you read something like this in code, is you start figuring out the graph
which is described in text and you start looking for the symbol used and where it is used.
But what you're really doing is you're constructing a graph in your mind.” (#5)
What the interviewees mentioned the most is that they use VSL for gluing code together,
which creates an interesting playground for designers and artist to connect different
worlds systems together and see what happens. The other use of VSL is as a domain
specific language for certain aspects such as in the animation state machine.
6.4 Summary of the results
When looking at the results discussed in Chapter 6.3, it can be summarised that
performance has the tendency to be one of the most important aspects when choosing a
language among the interviewed industry professionals. When it comes to visual
scripting languages, there is another aspect is of greater importance and that is tools and
workflows.
Depending on the point of view when looking at VSL, be it from a gameplay programmer
point of view or from an engine programmer point of view, some of the aspects of
performance will vary. From a gameplay programmer perspective hot reloading and
quick iteration times seem to be more important, while for an engine programmer the
execution method is with priority. The execution method has been discussed in various
ways during the interviews but mainly the compilation method mattered, which means
that compiling to bytecode is a given. This stands in contrast to the results of the
systematic review. In addition, the question whether the bytecode will be compiled
further to native code or to C or C++ before shipping, was discussed more often than the
actual execution method.
Another important aspect of the performance aspect is memory management: should
the memory be owned by the runtime and if yes, is there communication between the
48
runtime and the outside world. Most answers in regard to memory access are in favour
of that the runtime understands pointers just as handles and passes them along to a
bound function that understands the pointer correctly to avoid type safety issues and
memory lifetime / ownership problems.
Another side of the memory management aspect was how will the runtime manage
memory. The systematic review did not provide solid answers towards this question
while the Q-Methodology and the interviews showed that the runtime API would expose
memory allocation functions that can be overridden, for example via an interface, to
provide their own allocator.
When it comes to the API, literature expects that an API of a SL is relatively flexible and
small. This also applies to VSL based on the results of the systematic review and the
interviews. Neither provided a clear answer to the question what a small and concise
API is. Besides the size of the API in terms of function and class count, flexibility is
important. Since tooling is crucial for a VSL, it is kind of expected that a good VSL project
API comes with a flexible API that allows to bind or connect an external editor.
What literature suggest and what is also expected by the interviewed experts is that the
API should come with a C interface, which stands in contrast to the systematic review
which results in a preferred C++ interface. This can be explained by the way GitHub
determines the language of a project – it takes the major language used in a project to
determine if the language is the project language. That the language should be C is also a
category that matches with the criteria of embeddability for SL.
The more dependencies a project has, the bigger the risk for the company using it, which
is a result of the interviews and aligns with the views of the literature about SL. What
interviewees suggest is that it is a common practice to fork a dependency, if needed, to
preserve the dependency, to be able to solve vulnerability issues, or to prevent the
external project from going offline. The systematic review could not contribute to this
finding since the dependency analysis was only on the high-level of a project and it was
not possible within the limitations of this study to deduce all possible dependencies of
all projects.
When it comes to documentation, literature states that some form of documentation
should be present but does not specify what. The systematic review showed that for the
analysed projects, online documentation in form of, e.g., markdown files or a wiki, was
important. This stands in slight contrast to the interviews, in which it was discussed that
header file documentation and sample projects are more important than online
documentation.
It can be summarized that the use of VSL in projects for expensive or logic-heavy jobs,
should be seen critically. The main use case the interviewed experts see for VSL is for
gluing systems together and for quick rapid prototyping. There is however some
scepticism towards a generalized VSL since every game has different requirements. This
also reflects in the extensive need for tooling, which cannot be discussed in depth in this
study due to time limitations. In conclusion, even though participants have been critical
49
about generalized VSL, they have mentioned that VSL are great for domain specific
languages to solve certain problems such as in animation state machines.
Conclusion and future directions
The goal of this research was to determine potential criteria that describe the
embeddability of visual scripting languages. A trend was identified based on the
collective results of the systematic review of GitHub repositories and the interviews
with industry professionals with the consideration of the stated study limitations. It can
be concluded that there are common criteria that are potentially important for the
embedding of visual scripting languages. These findings can be summarized on a high
level in five aspects: 1. performance and the identified subcategories, 2. mechanical
aspects of embedding software and their respective subcategories, 3. license, 4.
documentation and 5. tooling and workflow. The hypothesis that there will be an overlap
between criteria of embeddability for scripting languages and visual scripting languages
has shown to be correct within the context of this study. The major difference between
them lays purely in the aspect of tooling and workflow, which seems to be the most
important difference between VSL and SL, besides the performance aspects.
However, important to mention is that in retrospect, the choice of a research method
might not have been the most suitable for this kind of study since the results of the
systematic review did not contribute as expected. Also, performing qualitative research
via interviews could have been done through focus groups for obtaining better results.
Nonetheless, due to the strict time limitations, it would not have been possible to
assemble a group of game developers with different backgrounds (e.g. different
continents). Moreover, in such a group they would have been less likely to share
information as freely as they did in a one-to-one situation. In conclusion, under the
current circumstances, the chosen method delivered useable results.
In addition to the previously mentioned concerns, it has to be mentioned that the used
systematic review method described by Z.Shen and M.Spruit in their paper “A
systematic review of open source clinical software on GitHub for improving software
reuse in smart healthcare” [11] has proven to be a valid method and a good step-by-step
guide on how one can execute such a review.
To conclude, the results may provide a direction which a potential visual scripting
language framework can follow that aims to be embeddable within the domain of the
game industry. It is important to note that due to the various use cases of VSLs within
the domain of the game industry, the resulting requirements of each use case may not
overlap with all the potential criteria, which may be a challenge for a language
framework.
7.1 Future directions
The results of this study may lay the foundation, so further studies can explore the
problem domain. Future research is certainly needed to investigate the practical
application of the found criteria.
50
Moreover, there seems to be a need to explore the aspect of UI/UX implications on
embeddability for VSL in the domain of game development. Another research topic are
the needs and implications of the code generation backend that allow a VSL to generate
native code, or compile to a higher-level SL or a system programming language such as C
or C++.
What the interviews showed is that the ability to generate native code or very efficient
bytecode in combination with native code, seems to be a very important aspect for the
practical implementation study.
51
Appendix A
A.1 Tables
A.1.1 Overview of Game Engines and their use of VSL
The table illustrates this development by summarizing and overview of game engines of
the industry, see [26]–[28]. Overview of major game engines and their support for
Visual Scripting Languages.
Game Engine Name
(Natively)
Supports
Visual
Scripting
Name
Unreal Engine 4 (Epic
Games)
+
Blueprints
Unreal Engine 5 (Epic
Games)
+
Blueprints
Unity 3D (Unity
Technology)
+
Unity Visual Scripting
(Bolt)
Snowdrop
(Massive/Ubisoft)
+
“Node Graph based”
CryEngine (Crytek)
+
Flow Graph
Godot
+
Visual Scripting
Forstbite (DICE/EA)
+
“Frostbite Schematics”
GameMaker Studio 2,
YoYo Games
+
Drag And Drop™
Open 3D Engine, Linux
Foundation ( open
source version of what
was known as Amazon
Lumberyard)
+
Script Canvas
UNIGINE (Unigine
Company)
-
Under research and
development at the
moment. [97]
Defold (King.com
Limited)
-
?
Rockstar Advanced
Game Engine
(Rockstar Games'
Rockstar San Diego
studio)
?
?
Clausewitz (Paradox)
?
?
REDEngine (CD
Project Red)
?
?
Construct 3 (Scirra
Ltd)
+
?
52
Stencyl (Stencyl, LLC)
+
Stencyl’s visual
programming language
[28]
The Machinery
(OurMachinery)
+
Entity Graph / Creation
Graph
A.1.2 Overview of VSL related research papers within the game
industry
Overview of a subset of matching results from Google Scholar search “Visual Scripting in
Game Development” or “Visual Scripting in Games” or “Visual Scripting Systems for
Games” that describe how to design or extend a new or existing VSL
Author
Paper
Keywords
Knutsen, K. Í.
(2021).
Visual Scripting in Game
Development. May.
http://www.theseus.fi/h
andle/10024/500439
[56]
Visual Scripting in
Game
Development,
Visual Scripting in
Games
Kok, L.
(2013).
Foundations of a visual
domain specific language
for programmable game
mechanics. [98]
Visual Scripting in
Games
Lehn, D. N.
(2015).
Designing a Visual
Scripting System.
February. [99]
Visual Scripting in
Games, Visual
Scripting Systems
for Games
Chu, E., &
Zaman, L.
(2021)
Exploring alternatives
with Unreal Engine’s
Blueprints Visual
Scripting System.
Entertainment
Computing, 36(July
2020).
https://doi.org/10.1016
/j.entcom.2020.100388
[27]
Visual Scripting in
Games, Visual
Scripting Systems
for Games
MacCormick,
D., & Zaman,
L. (2019)
SuBViS: The Use of
Subjunctive Visual
Programming
Environments for
Exploring Alternatives in
Game Development.
Proceedings of the 14th
International Conference
on the Foundations of
Digital Games. . (UE4
Based)
https://doi.org/10.1145
/3337722.3337740
[100]
Visual Scripting in
Games, Visual
Scripting Systems
for Games
53
A.1.3 Implementation Language Overview of Game Engines
Shows a overview of selected game engines and their runtime implementation language:
Engine
Implementation Language
(Runtime)
Unreal Engine 4 (Epic
Games)
C++
Unreal Engine 5 (Epic
Games)
C++
Unity 3D (Unity
Technology)
C++
Snowdrop
(Massive/Ubisoft)
C++
CryEngine (Crytek)
C++
Godot
C++
Forstbite (DICE/EA)
C++
GameMaker Studio 2,
YoYo Games
C++
Open 3D Engine, Linux
Foundation ( open source
version of what was known
as Amazon Lumberyard)
C++
UNIGINE (Unigine
Company)
C++
Defold (King.com Limited)
Rockstar Advanced Game
Engine (Rockstar Games'
Rockstar San Diego studio)
C++
Clausewitz (Paradox)
C++
REDEngine (CD Project
Red)
C++
Construct 3 (Scirra Ltd)
JavaScript
Stencyl (Stencyl, LLC)
Haxe
The Machinery
(OurMachinery)
C
54
A.2 Figures and Graphs
A.2.1 Search Term Graphics
The number of repositories with the search terms stated in the figure.
Shows job offers in which the Language of requirement / recommendation is set to English, French, German or
Spanish on the websites jobs.gamesindustry.biz and gamesjobsdirect.com. (Collected on the 30.12.2021)
Bibliography
[1] J. Gregory, Game Engine Architecture, Second Edition, 2nd ed. USA: A. K. Peters, Ltd., 2014.
[2] Wikpedia, “Wikipedia: Programming Languages.”
513
43
0
7
39
0 100 200 300 400 500 600
(“visual scripting language" OR "visual
programming language”) AND NOT
Visual+Studio
("lenguaje de secuencias de comandos visual"
OR "lenguaje de programación visual") AND
NOT Visual+Studio
(" script visuel Langue" OR "programmation
visuelle Langue") AND NOT Visual+Studio
("ビジュアルプログラミング" OR "ビジュアル
スクリプティング")
("可视化脚本 语" OR " 可视化编程语") AND
NOT Visual+Studio
English Spanish French Japanese
Chinese
(simplifie
d)
Results
2358
98
283
47
0
500
1000
1500
2000
2500
English French German Spanish
Job offers by Language
55
https://en.wikipedia.org/wiki/Programming_language
[3] D. Tyler, “Game Mechanics,” 2021. https://www.gamedesigning.org/learn/basic-game-
mechanics/
[4] T. Point, “Header Files.”
https://www.tutorialspoint.com/cprogramming/c_header_files.htm
[5] Wikpedia, “Wikipedia: Namespaces.” https://en.wikipedia.org/wiki/Namespace
[6] L. Torczon and K. Cooper, Engineering A Compiler, 2nd ed. San Francisco, CA, USA:
Morgan Kaufmann Publishers Inc., 2007.
[7] H. Sutter, “Type Inference vs. Static/Dynamic Typing.”
https://herbsutter.com/2008/06/20/type-inference-vs-staticdynamic-typing/
[8] B. Stroustrup, “Bjarne Stroustrup’s C++ Glossary,” 2012.
https://www.stroustrup.com/glossary.html
[9] S. F. Conservancy, “Git.” https://git-scm.com/
[10] GitHub, “GitHub.” https://github.com/about
[11] Z. Shen and M. Spruit, “A systematic review of open source clinical software on GitHub for
improving software reuse in smart healthcare,” Appl. Sci., vol. 9, no. 1, 2019, doi:
10.3390/app9010150.
[12] Microsoft, “Visual Studio.” https://visualstudio.microsoft.com/
[13] Mircosoft, “VS Code.” https://code.visualstudio.com/
[14] wiki, “VS Code Wikipeda.” https://en.wikipedia.org/wiki/Visual_Studio_Code
[15] redhat, “What is a REST API?” https://www.redhat.com/en/topics/api/what-is-a-rest-
api
[16] E. F. Anderson, “A classification of scripting systems for entertainment and serious
computer games,” Proc. - 2011 3rd Int. Conf. Games Virtual Worlds Serious Appl. VS-Games
2011, pp. 47–54, 2011, doi: 10.1109/VS-GAMES.2011.13.
[17] S. Aleem, L. F. Capretz, and F. Ahmed, “Game development software engineering process
life cycle: a systematic review,” J. Softw. Eng. Res. Dev., vol. 4, no. 1, p. 6, 2016, doi:
10.1186/s40411-016-0032-7.
[18] R. Victor, “L2a- Iterative Incremental Development - A Brief History [Larman,Basili
IEEE.Computer 2003] (10pp),” no. June, pp. 47–56, 2003.
[19] M. Idrees, F. Aslam, K. Shahzad, and S. M. Sarwar, “Towards a Universal Framework for
Visual Programming Languages,” Pak. J. Engg. Appl. Sci., vol. 23, no. July, pp. 55–65, 2018,
[Online]. Available:
https://www.researchgate.net/publication/328191862_Towards_a_Universal_Framewo
rk_for_Visual_Programming_Languages
[20] G. Team, “What is Visual Scripting.”
56
https://docs.godotengine.org/en/stable/getting_started/scripting/visual_script/what_is
_visual_scripting.html
[21] Unity Technology, “Unity 3D.” https://unity.com/
[22] U. T. Bowell, Andrew, “Bolt visual scripting is now included in all Unity plans.”
https://blog.unity.com/news/bolt-visual-scripting-is-now-included-in-all-unity-plans
[23] E. Games, “Unreal Engine 4.” https://www.unrealengine.com/en-US/
[24] E. Games, “Kismet Visual Scripting.”
https://docs.unrealengine.com/udk/Three/KismetHome.html
[25] E. Games, “Blueprints,” 2021. https://docs.unrealengine.com/4.27/en-
US/ProgrammingAndScripting/Blueprints/
[26] J. Sobolev, “Top 18 Game Engines To Make Video Games Without Coding,” 2020.
https://gamingshift.com/game-engines-without-coding/
[27] E. Chu and L. Zaman, “Exploring alternatives with Unreal Engine’s Blueprints Visual
Scripting System,” Entertain. Comput., vol. 36, no. July 2020, 2021, doi:
10.1016/j.entcom.2020.100388.
[28] A. Andrade, “Game engines: a survey,” EAI Endorsed Trans. Game-Based Learn., vol. 2, no.
6, p. 150615, 2015, doi: 10.4108/eai.5-11-2015.150615.
[29] B. A. Myers, “Taxonomies of Visual Programming,” 1989.
[30] M. F. Msiska and L. Van Zijl, “From visual scripting to Lua,” ACM Int. Conf. Proceeding Ser.,
pp. 94–99, 2012, doi: 10.1145/2389836.2389848.
[31] E. Games, “Exposing Gameplay Elements to Blueprints,” 2021.
https://docs.unrealengine.com/4.27/en-
US/ProgrammingAndScripting/Blueprints/TechnicalGuide/ExtendingBlueprints/
[32] L. (Ludiq) Bonin, “Unity Technologies acquires Bolt.” https://ludiq.io/blog/unity-
acquires-bolt
[33] G. Team, “Godot Visual Scripting.”
https://docs.godotengine.org/en/stable/getting_started/scripting/visual_script/getting_
started.html
[34] H. Muhammad and R. Ierusalimschy, “C APIs in extension and extensible languages,” J.
Univers. Comput. Sci., vol. 13, no. 6, pp. 839–853, 2007.
[35] Seanny123, “What makes a scripting language ‘embeddable’?”
https://softwareengineering.stackexchange.com/questions/403911/what-makes-a-
scripting-language-embeddable
[36] R. Ierusalimschy, L. H. de Figueiredo, and W. Celes, “Passing a Language through the Eye
of a Needle,” Queue, vol. 9, no. 5, pp. 20–29, 2011, doi: 10.1145/1978862.1983083.
[37] R. Ierusalimschy, L. H. De Figueiredo, and W. C. Filho, “SPE paper Lua – an extensible
57
extension language,” vol. 6, no. 1996, pp. 635–652, 2015.
[38] V. Salonen, “Automatic Portability Testing,” University of Jyväskylä, 2012. [Online].
Available: https://jyx.jyu.fi/handle/123456789/40043
[39] H. H. Muhammad, “A study on scripting language APIs,” 2006.
[40] and D. R. Ahmed BinSubaih, Steve Maddock, “A Survey of ‘Game’ Portability,” pp. 1–23,
2007, [Online]. Available:
https://www.researchgate.net/publication/242440118_A_Survey_of_’Game’_Portability
#share
[41] M. Fuchs, The video game industry: Formation, present state, and future, vol. 6, no. 1.
Routledge, 2014. doi: 10.1386/jgvw.6.1.83_5.
[42] S. Rabin, Introduction To Game Development (Game Development). USA: Charles River
Media, Inc., 2005.
[43] L. Pascarella, F. Palomba, M. Di Penta, and A. Bacchelli, “How is video game development
different from software development in open source?,” Proc. - Int. Conf. Softw. Eng., pp.
392–402, 2018, doi: 10.1145/3196398.3196418.
[44] R. Nystrom, “Game Programming Patterns,” in Game Programming Patterns, Genever
Benning, 2014. [Online]. Available: https://books.google.nl/books?id=AnvVrQEACAAJ
[45] Unity Technology, “Bolt Documentation.”
https://docs.unity3d.com/bolt/1.4/manual/index.html
[46] J. G. Guerrero, “Reinventing the wheel,” 2014.
[47] C. O’Toole-Bateman, “The History of the Game Engine: Part 5 – Reinventing the Wheel.”
https://ultimategamingparadise.com/features/series/history-of-the-game-engine/part-
5-reinventing-the-wheel/
[48] N. Gray, “Minimalist container library in C (part 1),” 2018.
[49] J. Fristrom, “Postmortem: Treyarch’s 2002 hit, Spider-Man,” 2013.
[50] M. Kissner, “Writing a Game Engine from Scratch - Part 1: Messaging,” 2015.
[51] I. Free Software Foundation, “GNU General Public License,” 2007.
https://www.gnu.org/licenses/gpl-3.0.en.html
[52] J. K. Ousterhout, “Scripting: Higher-level programming for the 21st century,” Computer
(Long. Beach. Calif)., vol. 31, no. 3, pp. 23–30, 1998, doi: 10.1109/2.660187.
[53] M. Harmon, “Building Lua into Games,” in Game Programming Gems 5, K. Pallister, Ed.
Charles River Media, 2005, pp. 115–128.
[54] MDN contributors, “Grammar and types: Variables,Declarations.”
https://developer.mozilla.org/en-
US/docs/Web/JavaScript/Guide/Grammar_and_types#declarations
[55] R. Nystrom, “A Virtual Machine,” 2021. https://craftinginterpreters.com/a-virtual-
58
machine.html
[56] K. Í. Knutsen, “Visual Scripting in Game Development,” no. May, 2021, [Online]. Available:
http://www.theseus.fi/handle/10024/500439
[57] A. Hirschi, “Traveling light, the Lua way,” IEEE Softw., vol. 24, no. 5, pp. 31–38, 2007, doi:
10.1109/MS.2007.150.
[58] R. Nystrom, Crafting Interpreters, 1st ed. Genever Benning. [Online]. Available:
https://www.craftinginterpreters.com/
[59] M. Pall, “Introducing LuaJIT.”
https://staff.fnwi.uva.nl/h.vandermeer/docs/lua/luajit/luajit_intro.html
[60] M. Project, “JIT on Xbox One,” 2022. https://www.mono-project.com/docs/about-
mono/supported-platforms/xbox-one/
[61] M. A. Kuhail, S. Farooq, R. Hammad, and M. Bahja, “Characterizing Visual Programming
Approaches for End-User Developers: A Systematic Review,” IEEE Access, vol. 9, no.
January, pp. 14181–14202, 2021, doi: 10.1109/ACCESS.2021.3051043.
[62] T. C. Ahern, “The effectiveness of visual programming for model building in middle
school,” Proc. - Front. Educ. Conf. FIE, 2008, doi: 10.1109/FIE.2008.4720634.
[63] M. M. Burnett, “Visual object-oriented programming,” Proc. Conf. Object-Oriented
Program. Syst. Lang. Appl. OOPSLA, vol. Part F1296, no. April 1994, pp. 127–129, 1993,
doi: 10.1145/260303.261240.
[64] G. Melchiorri, C. Castagna, R. Sorge, and M. Bonifazi, Game Scripting Mastery, vol. 24, no.
10. 2010. [Online]. Available: http://www.ncbi.nlm.nih.gov/pubmed/20877006
[65] Google, “Blocky.” https://developers.google.com/blockly (accessed Apr. 14, 2022).
[66] Unity Technology, “Animation Controller.” https://docs.unity3d.com/Manual/class-
AnimatorController.html
[67] Unity Technology, “Shader Graph”, [Online]. Available: https://unity.com/shader-graph
[68] “Unreal Engine 4 Animation State Maschine.” https://docs.unrealengine.com/4.27/en-
US/AnimatingObjects/SkeletalMeshAnimation/StateMachines/
[69] M. Autodesk, “Maya Node Editor.” [Online]. Available:
https://knowledge.autodesk.com/support/maya/learn-
explore/caas/CloudHelp/cloudhelp/2019/ENU/Maya-Basics/files/GUID-23277302-
6665-465F-8579-9BC734228F69-htm.html
[70] B. F. Blender 3D, “Shader Editor.”
https://docs.blender.org/manual/en/latest/editors/shader_editor.html
[71] Blender Foundation, “Blender.” https://www.blender.org/
[72] “Unreal Engine 4 Material Editor.” https://docs.unrealengine.com/4.27/en-
US/RenderingAndGraphics/Materials/Editor/
59
[73] A. Substance 3D Designer, “The Graph View.”
https://substance3d.adobe.com/documentation/sddoc/the-graph-view-
129368149.html
[74] G. Team, “VisualShaders.”
https://docs.godotengine.org/en/stable/tutorials/shading/visual_shaders.html
[75] M. Autodesk, “Autodesk Maya.”
https://www.autodesk.com/products/maya/overview?term=1-YEAR&tab=subscription
[76] T. (OurMachinery) Persson, “OurMachinery: Creation Graphs,” 2019.
https://ourmachinery.com/post/creation-graphs/
[77] E. Games, “Nativizing Blueprints.” https://docs.unrealengine.com/4.27/en-
US/ProgrammingAndScripting/Blueprints/TechnicalGuide/NativizingBlueprints/
[78] R. V Roque, “OpenBlocks: an extendable framework for graphical block programming
systems,” Massachusetts Institute of Technology, 2007. [Online]. Available:
https://dspace.mit.edu/handle/1721.1/41550
[79] S. Cooper, W. Dann, and R. Pausch, “Alice: A 3-D tool for introductory programming
concepts,” J. Comput. Sci. Coll., vol. 15, no. 5, pp. 107–116, 2000.
[80] S. Y. Lye and J. H. L. Koh, “Review on teaching and learning of computational thinking
through programming: What is next for K-12?,” Comput. Human Behav., vol. 41, pp. 51–
61, 2014, doi: 10.1016/j.chb.2014.09.012.
[81] M. Resnick et al., “Scratch: Programming for All,” Commun. ACM, vol. 52, no. 11, pp. 60–67,
Nov. 2009, doi: 10.1145/1592761.1592779.
[82] P. Y. Chao, “Exploring students’ computational practice, design and performance of
problem-solving through a visual programming environment,” Comput. Educ., vol. 95, pp.
202–215, 2016, doi: 10.1016/j.compedu.2016.01.010.
[83] GitHub, “GitHub Search API Reference,” 2021.
https://docs.github.com/en/rest/reference/search
[84] Z. Shen, “ianshan0915/clinical-opensource-projects,” 2019.
https://github.com/ianshan0915/clinical-opensource-projects
[85] S. Renger, “collect-data-from-github,” 2022. https://github.com/simonrenger/collect-
data-from-github
[86] J. Kuhn, “ELT for Video Games: Language and Video Game Careers,” 2021.
http://blog.tesol.org/elt-for-video-games-language-and-video-game-careers/ (accessed
Jun. 04, 2022).
[87] G. Moore, “Moore’s law.”
https://www.oxfordreference.com/view/10.1093/oi/authority.20110803100208256
(accessed Jan. 17, 2022).
60
[88] GitHub, “GitHub Repo API Reference,” 2021.
https://docs.github.com/en/rest/reference/repos
[89] S. R. Brown, “A primer on Q methodology,” Operant Subj., vol. 16, no. 3/4, pp. 91–138,
1993.
[90] P. Lankoski and S. Björk, Game Research Methods, no. June. 2015.
[91] T. R. Lindlof Taylor, Bryan C., Qualitative communication research methods. Thousand
Oaks; London; New Delhi: Sage, 2010.
[92] O. Initative, “MIT,” 2021. https://opensource.org/licenses/MIT
[93] G. Haff, “The mysterious history of the MIT License.”
https://opensource.com/article/19/4/history-mit-license
[94] G. MCCULLOCH, “Coding Is for Everyone—as Long as You Speak English,” Apr. 08, 2019.
[Online]. Available: https://www.wired.com/story/coding-is-for-everyoneas-long-as-
you-speak-english/
[95] MIT, “Scratch.” https://scratch.mit.edu/
[96] N. Ramsey, “Embedding an interpreted language using higher-order functions and types,”
Proc. 2003 Work. Interpret. Virtual Mach. Emulators, IVME 2003, pp. 6–14, 2003, doi:
10.1145/858570.858571.
[97] U. Team, “Migrating to UNIGINE from Unreal Engine: Programming,” 2021.
[98] L. Kok, “Foundations of a visual domain specific language for programmable game
mechanics,” 2013.
[99] D. N. Lehn, “Designing a Visual Scripting System,” no. February, 2015.
[100] D. MacCormick and L. Zaman, “SuBViS: The Use of Subjunctive Visual Programming
Environments for Exploring Alternatives in Game Development,” 2019. doi:
10.1145/3337722.3337740.
61
This template is based on a template by:
Steve Gunn (http://users.ecs.soton.ac.uk/srg/softwaretools/document/templates/)
Sunil Patel (http://www.sunilpatel.co.uk/thesis-template/)
Template license:
CC BY-NC-SA 3.0 (http://creativecommons.org/licenses/by-nc-sa/3.0/)
