PreprintPDF Available
Preprints and early-stage research may not have been peer reviewed yet.

Abstract and Figures

We present z-Tree unleashed, a novel approach and set of scripts to aid the implementation of experimental economics laboratory experiments outside of the laboratory. z-Tree unleashed enables subjects to join the experiment using a web portal that requires no additional software. Experimenters are likewise enabled to administer their experiments from anywhere in the world. Except for the experimental software itself, z-Tree unleashed is entirely based on free and open source software. In this paper, we firstly outline the motivation. Secondly, we give a high-level overview of z-Tree unleashed's advantages and its design. We then proceed to show how to set up the server and demonstrate the steps required for conducting an entire experiment. We subsequently explain how to leverage the security and routing features of a virtual private network with z-Tree unleashed, enabling servers to securely run behind routers. Finally, we conclude.
Content may be subject to copyright.
z-Tree unleashed: A novel client-integrating
architecture for conducting z-Tree experiments over
the Internet
Matthias L. Duch
University of Cologne
Max R. P. Grossmann
University of Cologne
Thomas Lauer
University of Cologne
First draft: March 30, 2020
This draft: April 26, 2020
Abstract
We present z-Tree unleashed, a novel approach and set of scripts to
aid the implementation of computerized laboratory experiments out-
side of the laboratory. z-Tree unleashed enables subjects to join the
experiment using a web portal that requires no additional software. Ex-
perimenters are likewise enabled to administer their experiments from
anywhere in the world. Except for z-Tree itself, z-Tree unleashed is en-
tirely based on free and open source software. In this paper we give a
high-level overview of z-Tree unleashed’s advantages and its design. We
also show how to set up the server and demonstrate the steps required
for conducting an entire experiment. We subsequently explain how to
leverage the security and routing features of a virtual private network
with z-Tree unleashed, enabling servers to securely run behind routers.
1 Introduction
Since its inception in 1998, many researchers have come to rely on the Zurich
Toolbox for Readymade Economic Experiments (z-Tree), cf. Fischbacher
(2007). Despite the fact that several new tools have become available in
Corresponding author: University of Cologne, Cologne Laboratory for Economic Re-
search, Albertus-Magnus-Platz, 50923 Cologne, Germany. Email: m.grossmann@uni-
koeln.de.
1
0.26
0.52
0.22
0.0
0.1
0.2
0.3
0.4
0.5
oTree z−Tree other
rel. Frequency
(Covering the period from 2019−01−17 to 2020−03−16)
Figure 1: z-Tree usage at the Cologne Laboratory for Economic Research
(CLER).
recent years, a majority of economic experiments are still conducted us-
ing z-Tree (see Figure 1 for an example). There are compelling reasons for
this persistence: z-Tree is easy to set up and can be learned in short order.
One of the most developed and widespread alternatives is oTree by Chen
et al. (2016). This Python based platform provides a exible and power-
ful programming environment for interactive experiments. This exibility,
however, comes at a cost. oTree also increases the complexity of the pro-
gramming environment that researchers have to handle in order to prepare
their experiments.1The productive use of oTree or similar platforms requires
considerable programming skills, and involves substantial installation time.2
z-Tree is based on a classic client-server architecture. The clients, i.e.
the digital computers which are used by subjects during the experiment,
are typically Personal Computers with Microsoft Windows. These PCs run
z-Leaf, the client software. z-Leaf connects to z-Tree over the Local Area
Network (LAN) and z-Tree streams the experiment over the network, while
z-Leaf returns the subjects’ input using a proprietary network protocol based
on TCP/IP (see Figure 2, panel a)). Given that z-Leaf is connected to
1The code for any oTree experiment is spread across several les, while zTree allows
researchers to use a single le for programming and conducting the experiment.
2There are several other software platforms in dierent stages of development, e.g.
BreadBoard, ConG (Pettit et al. (2014)), MobLab, NodeGame (Balietti (2017)), Psynter-
act (Henninger et al. (2017)) and UbiquityLab.
2
a) z-Tree setup
video stream
subject’s inputs
subject device
web browser
subject device
web browser
subject device
web browser
b) z-Tree unleashed setup
z-Tree unleashed server
experimenter PC
z-Tree server
subject PC
z-Leaf client
file server
parameters
results
subject’s inputs
subject PC
z-Leaf client
subject PC
z-Leaf client
virtualization
Figure 2: a) The client-server architecture of z-Tree; b) the boxed and sim-
plied architecture of z-Tree unleashed .
z-Tree over the LAN, a question that has frequently arisen is whether z-Tree
experiments could also be played over the Internet, allowing subjects to dial
in from anywhere in the world. However, it is generally understood that
one cannot expect participants to use a PC, use Microsoft Windows, install
foreign software on their computer, do not use Wi-Fi or mobile Internet to
connect to z-Tree to ensure network stability and to uninstall that software
after the experiment has concluded. Hence, the answer to this question used
to be a resounding “no”.
The global pandemic known as COVID-19 has renewed interest in the
question. As scientists struggle to continue their research activities while
many behavioral laboratories are closed, many researchers are looking for
a method to allow subjects to participate in z-Tree experiments remotely.
This reemergence has prompted us to develop z-Tree unleashed, which is a
set of techniques and scripts to address several of the issues that arise in the
discussion.
One of the most important features of z-Tree unleashed is that subjects
are able to participate in the experiment using just a web browser. No in-
stallation of additional software is required and there is no diculty when
using Wi-Fi or mobile internet instead of electrical or ber-optical signals.
Indeed, no diculty arises even if subjects switch between networks during
the experiment.3Subjects may use all operating systems and system archi-
tectures as long as they can access a graphical web browser with JavaScript
enabled. Subjects can participate from anywhere in the world.
z-Tree unleashed can be run as a virtual machine, i.e. as a completely
isolated system. It does not pollute any existing installation of z-Tree. Sub-
jects are likewise completely isolated from each other; they cannot interfere
with other subjects or the overall administration of the experiment. z-Tree
3Even if our web portal does not reconnect immediately, subjects can reload the portal
without any loss of data.
3
unleashed upholds the privacy of its users and encrypts all trac using
HTTPS or a VPN or sends it over trusted internal interfaces (such as the
loopback device).
Our entire package is permissively licensed, based on free software and
can be freely redistributed (except for z-Tree itself). We provide a set of
scripts as well as a user-friendly virtual machine image.4
2 Advantages of z-Tree unleashed
In this section we briey describe the most important features of z-Tree un-
leashed and resulting advantages for experimenters and lab administrators.
While the installation of z-Tree itself has always been as simple as possible,
the setup of a productive laboratory environment requires some additional
steps like providing a local network and a shared le storage.
With z-Tree unleashed we simplify this setup to the absolute minimum
(see Figure 2, panel b)). Every modern desktop computer or laptop can be
used to set up an entire laboratory. All z-Tree related network communica-
tion and le transfers can be handled in a single machine and all necessary
software is provided in a single virtual machine image le that can easily be
mounted in a hypervisor.
Generations of experimental economics have been trained to use z-Tree
and over time their experience and knowledge has become a valuable re-
source in many behavioral disciplines. Until now this expertise is only slowly
spreading to other software tools and there is no easy way to implement
existing z-Tree code in alternative software tools.
With z-Tree unleashed it is possible to use existing z-Tree programs
without any adjustments. Not only can experimenters use their already
designed experiments in a remote environment, but new experiments can
still be developed on the foundation of established code and procedures.
Another advantage is that all media les, predened tables or external pro-
grams that are embedded into a z-Tree le can be stored on one device. The
integrated le server obviates the need for a shared network storage that is
usually used to grant le access to the z-Leaf clients.
The second group that eminently benets from the simplied architec-
ture of z-Tree unleashed are the administrators and managers of behavioral
laboratories around the world. Not only does the compact architecture of
our tool allow for low performance, and therefore also low cost, client com-
puter hardware, it also minimizes maintenance and update eorts for lab
administrators. Since participant devices only need to handle the z-Tree
video stream in a web browser, more expensive oce-type computers can
4Our GitLab repository can be found at https://gitlab.com/cler1/
ztree-unleashed. Our documentation as well as links to our virtual machine im-
age are available on https://cler1.gitlab.io/ztree-unleashed-doc/.
4
be replaced by inexpensive and even single board computers. The same is
true for the software that is needed for the client computers. With z-Tree
unleashed it is possible to set up the clients without any licensing costs for
operating systems or other software. These ultra compact devices only need
a minimal setup to receive and display the video stream, and can easily
be replaced if necessary. Moreover, these lean single board computers have
about twice the service life of standard desktop computers and are therefore
the more environmentally friendly choice (cf. Pattinson et al. (2015)).
An additional benet for laboratory administrators comes from the fact
that all upgrades, backups and other maintenance measures can be done on
a single device. Dierent versions of z-Tree and z-Leaf can easily be managed
in a single folder without the need to distribute the z-Leafs to dozens of
client computers. Without the additional (external) le server used in classic
z-Tree setups all components of an experiment (e.g. instructions, media les,
data les and program les) can also be stored in a single folder. Hence,
organizing a laboratory with a large number of independent experimenters
becomes much easier.
3 Use cases
Our tool not only helps reduce the eorts and costs for setting up and
maintaining a traditional computerized laboratory, but at the same time
allows for a tremendous extension of z-Tree’s eld of application. In this
section we outline four important use cases with dierent levels of mobility
for organizing, conducting and participating in experimental sessions. We
start with the most common case of a (i) local laboratory setup, followed by
a (ii) mobile laboratory setup, a (iii) remote setup and a (iv) tremendously
mobile setup.
3.1 Local laboratory setup
Up until now most economic laboratories rely on a closed network with a
dedicated le server to simplify the implementation of z-Tree’s classic client-
server-architecture. The virtualized architecture of z-Tree unleashed (see
Figures 2 and 3) renders this closed setup redundant. That does not mean
that existing hardware can no longer be used. The opposite is true; all pre-
existing hardware components can still be used without major changes.
The only step that is necessary to switch from the existing setup to our
tool is the installation of the z-Tree unleashed server on a computer that
can be used as an experimenter’s device (e.g. an existing z-Tree server). On
account of the virtualization it is not even necessary to choose between the
existing setup and z-Tree unleashed. By using a virtual machine that hosts
the z-Tree unleashed server on the existing computer, both setups can be
used in parallel.
5
Figure 3: A screenshot of a live session in z-Tree unleashed.
Since probably all client computers use an operating system that comes
with a web browser, changes to the clients are not necessary. At the same
time the pre-existing le server can still be used as a backup device for
documentation, z-Tree les and raw data.
3.2 Mobile laboratory setup
The benets of virtualization become even clearer in a mobile laboratory
setup. The use of a wired local network in a portable laboratory with 20 or
more clients is logistically challenging to say the least. It also puts anachro-
nistically high requirements on the used client hardware, e.g. RJ-45 jacks
for ethernet cables.
One alternative could be the use of a private Wi-Fi network; but even
then additional hardware would be required. Moreover, many users report
issues using a wireless connection to connect the z-Leaf clients to the server.
In contrast to that z-Tree unleashed does not need a dedicated network,
neither in a wired setup nor in a wireless one. It can be used in any pub-
lic, private or university Wi-Fi. The only requirement is that the z-Tree
unleashed server is reachable via its IP address. Just as for the local labo-
ratory setup our architecture allows for minimal requirements on the part
of the client devices. It is even possible that participants use their own de-
vices, as long as these devices can connect to the Internet.
3.3 Remote setup
Given that participants can use their own devices, z-Tree unleashed is also
ready to be used remotely. That means the server can be set up in a local
6
laboratory while participants are invited to join the experiment from wher-
ever they are.
This of course brings up dierent challenges for the timing of interactive
experiments, i.e. all participants have to be connected to the experiment
server at a particular time. That, of course, is not only true for our setup,
but for all online experiments.
In contrast to other online experiment platforms our setup does not
require an explicit routine to handle temporary dropouts. z-Trees original
ability to reconnect lost clients and restore the stage where they dropped out
allows participants to simply re-establish the temporarily lost connection.
Even if participants accidentally close the browser window, they just have
to click the link from the invitation mail again in order to continue the
experiment at the exact screen they stopped at.
3.4 Tremendously mobile setup
The idea of a maximally mobile laboratory can be pushed even further, by
also virtualizing the used network. This way the whole laboratory setup can
be boxed and used on the go. Even a cellular network would be sucient to
conduct an experiment, and a computerized data collection with z-Tree can
take place almost anywhere in the eld.
The tremendously mobile setup is especially useful for cross-cultural ex-
periments, where the same z-Tree program is used in many dierent regions
of the world. Instead of setting up an identical laboratory in every country,
the researcher can simply bring her own lab with her in a briefcase. The de-
tailed requirements and architecture of such a setup with a virtual private
network (VPN) are described in Section 6.
4 Architecture
z-Tree unleashed integrates several z-Leaf clients into a single digital com-
puter (the “server”). These z-Leaf clients can then be securely streamed
over the network to subjects who participate in the experiment. The basic
architecture is depicted in Figure 4.
The z-Tree unleashed server runs on GNU/Linux. For each individual
z-Leaf client, a separate Unix user account on that server is created.5This
procedure segregates dierent users from each other and makes any unin-
tended interference impossible. Even if a subject were able to break out of
the connes of the experiment, she would not be able to disturb the over-
all administration of the experiment. These individual Unix users are the
5In Figure 4, 𝐾users were created, and up to 𝐾z-Leaf clients can hence be run
simultaneously. Currently, z-Tree unleashed requires that all such users share a common
prex which is appended with 1 to 𝐾. A zeroth user is used to administer the experiment
and run the z-Tree graphical user interface.
7
Subject
Server
User: client1
VNC server
WINE
z-Leaf
User: client2
VNC server
WINE
z-Leaf
User: clientK
VNC server
WINE
z-Leaf
noVNC
Web browser
Figure 4: The z-Tree unleashed architecture. The reverse proxy is not shown.
analogue to separate computers in a stationary experimental economics lab-
oratory.
Upon commencement of an experiment, each Unix user executes a chain
of commands. This is done automatically by the scripts that we provide. It
is essential that z-Tree and each z-Leaf are emulated on GNU/Linux through
Wine (cf. Amstadt and Johnson (1994)), an open source compatibility layer
that allows running Microsoft Windows executables on Unix-like operating
systems. Each z-Leaf automatically connects to z-Tree over the loopback
interface.6For each user, we initialize a headless X Window System session
(Scheier et al. (1997)) and emulate a physical screen. A VNC server is then
used to continually photograph the emulated screen and allow the streaming
thereof over the network. However, the VNC server accepts connections
only on the loopback device to preclude unwanted access from outside.
noVNC7is an open sourced programming library and application that
allows web browsers to connect to a VNC server. For each user, a sepa-
rate instance of noVNC is run; noVNC listens for HTTP connections on
a user-dened network address and port and serves a simple portal that
allows dialing in to the VNC session. Ordinarily, interacting with a VNC
session requires the installation of separate software, but noVNC obviates
6This eliminates all networking issues that could possibly arise and enables the exper-
iment to run uidly with the highest performance possible given the hardware layer.
7https://novnc.com/info.html
8
that requirement. Hence, z-Tree unleashed enables subjects to participate
by using only a web browser.
To ensure privacy and security, an additional layer of networking soft-
ware is used that is not shown in Figure 4. Firstly, noVNC’s HTTP server
only listens on the loopback interface. Secondly, we reverse proxy requests
on a dierent, but globally reachable port to noVNC’s HTTP server. This
is achieved through the use of a web server such as nginx that acts as the
reverse proxy combined with a properly issued X.509 certicate for HTTPS
(see Tanenbaum and Wetherall (2010), section 8.9.3).8Simply put, subjects
access a HTTPS protected website using a web browser. On that website,
they can login to the experiment and interact with z-Leaf. Their inputs
are internally forwarded from the HTTPS server/reverse proxy (nginx) to
noVNC’s HTTP server, which is listening on the loopback device.9noVNC
in turn converts and transmits the inputs to the VNC server, which per-
forms the desired action over the X Window System. New frames that
appear on the emulated screen (such as a dialog that appears in z-Leaf)
are likewise transmitted from the X Window System to the VNC server.
noVNC reads these inputs and converts and transmits them to the reverse
proxy. The reverse proxy performs cryptographic primitives and sends out
TCP/IP packets that are sent over the Internet stack to the subject’s dig-
ital computer. The subject’s digital computer ultimately displays the new
frame, allowing the participant to react.
This setup requires that the server on which z-Leaf unleashed is running
is globally reachable via its IP address. In most cases, this will require a
full-edged setup in a datacenter, such as a Virtual Private Server (VPS)
with sucient bandwidth and computing power. However, many Internet
providers oer residential customers an (ephemeral) public IPv4 address and
port forwardings to devices within the LAN. In these cases, z-Tree unleashed
can be installed and used from home networks. Additionally, it is possible
to direct subjects to a user-friendly hostname by using Dynamic DNS. In
instances where this is not feasible, we provide a method using a VPN and
a globally reachable “Thin” server in Section 6.
8X.509 certicates can be procured commercially, but research institutes and services
like Let’s Encrypt frequently permit the issuance of universally recognized certicates free
of charge. X.509 certicates can also be self-signed, but these certicates create a warning
in modern web browsers as the identity of the site using the certicate cannot be veried.
Our virtual machine image comes with a self-signed X.509 certicate for testing purposes.
9Since sensitive data is only ever transferred over the (internal) loopback device or en-
crypted and transferred over authenticated channels, z-Tree unleashed reachest the highest
standards of data safety. Indeed, z-Tree unleashed is more secure than z-Tree over switched
Ethernet (the standard lab conguration) since the TCP/IP packets sent and received by
z-Tree can be eavesdropped at any point between the z-Leafs and the z-Tree server. This
is impossible in z-Tree unleashed , which does not require a secure network.
9
5 Conducting an experiment
In both the model of Section 4 as well as that of Section 6, the basic steps are
identical. A z-Tree unleashed server provides facilities to start, run, control
and terminate an experiment. One instance of an experiment is known as a
session. As soon as a session is initiated, URLs are generated that allow both
the experimenter as well as participants to access the experiment. Internally,
all the software described in Sections 4 and/or 6 is initialized. That is,
the experimenter receives a set of 𝐾 + 1 URLs, the rst of which streams
the graphical user interface of z-Tree. All remaining 𝐾URLs may be sent
out to participants, who can subsequently access the z-Leaf interface and
hence, the experiment. No password entry is required because passwords
are included as part of the URL. We use port hopping (see Lee and Thing
(2004)) to renew all assigned ports, internal as well as external, from session
to session. This makes guessing or predicting ports unlikely. All ports lie
in the range 49152–65535, which is reserved for private or temporary usage.
This range of ports must be kept open on the rewall if the approach in
Section 4 is to be used. The necessary z-Tree program les can be transferred
into a shared directory (/share) using SFTP and the graphical interface of
the virtual machine.
Our scripts start a predened number of virtual clients and sets the
names with which they identify themselves to z-Tree (e.g. in the clients
table) accordingly. At this stage, all user passwords are changed in order
to ensure that only intended subjects participate in this specic study. The
resolution of the virtual client screens can be set during the initial setup or
at a later point by editing the settings. As z-Leafs are sensitive to changes
in screen resolution (e.g. overlapping boxes) the experimenter might dene
it to t her needs. Furthermore it is possible to set the font size of the z-Leafs
globally (i.e. for all radio button labels and OK buttons). This feature
signicantly improves the predictability of the screen output as it will not
depend on the resolution of the client computer. After a short period all
clients appear in the clients table of z-Tree. An experimental add-on allows
to display a dialog with a button on the emulated screen instead of directly
starting z-Leaf. The invocation of that button executes that client’s z-Leaf,
making clients appear in the clients’ table only as soon as subjects signal
their readiness to participate.
The next steps depend on the intended use case. What is crucial is
that the server can be accessed by participants and the server is reachable
through one of the methods outlined in Sections (4) or (6). The rst use
case is the local setup in an existing lab. In this case the experimenter sends
each stationary client computer an individual link which will be opened in
a full screen browser window by using the lab’s administration software like
10
Labcontrol.10 It is trivial to achieve that the shown usernames of z-Leafs
match the number of each seat or cabin booth. In this sense the experimenter
might give special attendance to a certain subject should they encounter
an issue and the subjects’ experience does not deviate from running z-Tree
directly and without our scripts in the lab.
In times where personal attendance to lab experiments is impossible the
second use case is vital for still conducting experiments with zTree. In this
case subjects use their own computers to partake in the experiment. The
unique and individual URL to access the z-Leaf is sent to the participants
via e-mail. Participants only have to click the link provided in the e-mail.
They will be automatically connected and logged in, and are ready to start
the experiment. Sending these personalized links gives an opportunity to
accurately track and support specic subjects in the case of questions or
problems.
Starting the experiment itself is straightforward as the experimenter just
needs to click ”Start Treatment” inside z-Tree in her administrator browser
window. During the experiment she might use z-Tree in the same way as
in the lab and view all available tables and submitted input. At this point
z-Tree produces its output in a specied folder.11 After the experiment is
nished the experimenter might retrieve her resulting Excel le and other
generated les via SFTP from inside the virtual machine.
6 The tremendously mobile laboratory
As stated earlier, the z-Tree unleashed server requires a globally reachable IP
address to act as an endpoint for the interconnection with subjects. However,
with little additional overhead, z-Tree unleashed permits the use of ordinary
hardware to achieve its ends. Networking professionals are well aware of the
challenges implied by the use of consumer-grade internetworking links, such
as the proliferation of IPv4 network address translation (NAT, cf. Tanen-
baum and Wetherall (2010), pp. 452-455). The lack of sucient IPv4 ad-
dress space makes giving each Internet-capable device its own IPv4 address
impossible. Hence, in a normal setup as in Figure 4, a server that is run
outside of an institutionally provided network infrastructure could not be
accessed by subjects as it is usually behind a router that provides NAT. On
the other hand, IPv6, where available, was designed such that each device
receives a globally unique and globally reachable IPv6 address. But router
rewalls frequently restrict IPv6 trac and shield devices from ostensibly
unwanted trac. Even where “port forwardings” are possible, setting them
10For more information, see https://www.kirchkamp.de//lab/zTree.html, accessed
March 30, 2020.
11For each session, a unique folder in /share/scratch is created. This alleviates the
well-known limitation that made opening z-Tree more than once per minute impossible;
since z-Tree’s output is redirected to the unique folder, the limitation does not apply.
11
Figure 5: The z-Tree unleashed architecture with tunneling. The reverse
proxy is not shown.
up may expose the server to unwanted and dangerous trac. Finally, setting
up Dynamic DNS or relying on port forwardings at all may be infeasible in
scenarios where the z-Tree unleashed server is used in a mobile conguration.
If consumer-grade hardware and network links are to be used with z-Tree
unleashed, we must make what subjects connect to and participate on glob-
ally reachable, although within parameters and perhaps only indirectly. We
also must provide a secure and authenticated method to allow subjects to
connect with the server. All of these ends are accomplished through the use
of a virtual private network (VPN), see Figure 5.
In this model, subjects do not establish a direct connection with the
z-Tree unleashed server. Instead, they establish a connection with a dedi-
cated “Thin” server, which in turn routes the trac to the z-Tree unleashed
server over the VPN. Conceptually, the “Thin” server has only three re-
quirements: to be globally reachable by subjects; to provide an entry and
exit point for the VPN; to route trac between the subjects and the z-Tree
unleashed server.12 Essentially, we oer a private port forwarding that is
independent of the z-Tree unleashed server’s reachability and never exposes
12In our investigation, we found that “Thin” servers with sucient functionality and
bandwidth are available for less than €3/month. This includes one static IPv4 address,
making the “Thin” server globally reachable. A “Thin” server is not just “Thin”, it is also
“Cheap”.
12
the server to insecure or unwanted trac.
WireGuard (see Donenfeld (2017)) is a recent protocol and software tool
for the establishment of virtual private networks. WireGuard enables users
to establish secure connections between at least two peers even over insecure
networks. The connection between two peers is always authenticated and
end-to-end-encrypted using the well-known Elliptic Curve Die-Hellman
key exchange and related cryptographic protocols (cf. Barker et al. (2017)).
We recommend the use of WireGuard because of its simplicity. In our set
of scripts, we provide a facility to automatically generate WireGuard con-
guration les for both the “Thin” server as well as the z-Tree unleashed
server. WireGuard assigns to each peer a private static IP address. Indeed,
this setup allows experimenters to run a virtual machine that handles all
details of z-Tree unleashed on their own digital computers at home. Since
this virtual machine and the “Thin” server are interconnected through Wire-
Guard, subjects only interact with the “Thin” server, which can have a
user-friendly domain name that points to the “Thin” server’s public static
IP. WireGuard is seamlessly integrated into an existing networking infras-
tructure and it straightforwardly makes z-Tree unleashed available through
NAT and similar complications. This setup does not require dynamic DNS.
We oer example scripts to interact with the “Thin” server. The “Thin”
server must be set up to minimally act as a router, a WireGuard peer and
a nginx server. A tutorial sketch for setting up a FreeBSD “Thin” server
is also available in our repository. The target audience for these scripts are
experienced Unix system administrators. However, once a “Thin” server is
set up, it can be easily shared between many researchers or access rented
out on a commercial basis.
As mentioned previously, the z-Tree unleashed server’s noVNC instances
create a HTTP server. We now congure each instance of noVNC such that
the HTTP server binds to the z-Tree unleashed server’s static private IP
address instead of the loopback device. As soon as the “Thin” server and the
z-Tree unleashed server are connected through WireGuard, internal trac
can ow between the two servers using these internally assigned IP addresses.
Binding to the static private IP address ensures that only other devices in the
VPN can access the service, such as the “Thin” server.13 By reverse proxying
requests to the “Thin” proxy over the WireGuard interface to the z-Tree
unleashed server, we enable subjects to securely access the interface over
HTTPS even as the actual trac is redirected from the “Thin” server to the
z-Tree unleashed server and vice versa. Subjects do not notice that the actual
server is not the “Thin” server, and hence the forwarding is transparent. The
strong encryption used by WireGuard makes reverse proxying HTTP over
the VPN completely secure and even enables experimenters to roam between
13This implies that the actual raw HTTP or VNC services are never directly made
publicly available.
13
networks without interruptions, cf. Donenfeld (2017). In other words, this
mode of operation views the reverse proxy and the z-Tree unleashed server as
distinct physical devices and conceptually moves the z-Tree unleashed server
away from the subject.
The WireGuard VPN enables any computer with sucient hardware
and uplink to act as a z-Tree unleashed server, not just globally reachable
servers. This special derivative architecture makes the lab tremendously
mobile. It only has one stationary component: The “Thin” server, which
can simultaneously be used for many other purposes. However, in contrast
to other use cases, neither the subjects’ computers nor the experimenter’s
computer must meet demanding requirements such as global reachability.
This laboratory can be put in a briefcase.
7 Performance, requirements and network usage
Running many clients on a single machine imposes signicant strain on the
hardware. We reduce each processor’s load by allocating subsets of z-Leaf
instances to dierent processor cores, but withhold one solely for the central
z-Tree instance.14 The resulting performance improvements facilitate the
productive use of z-Tree unleashed. Using a minimal graphical interface to
start the clients’ graphical user interfaces we were able to reduce the mem-
ory load to a very low level. As can be seen in Figure 6, a full setup of z-Tree
unleashed using 32 connected clients running a resource demanding exper-
iment did not use more than four gigabytes of RAM. Moreover, using our
environment for conducting a regular experiment we found a steady mem-
ory consumption which is even available in most personal computers. But
in order to provide participants a reasonably smooth user experience, we
recommend, given these observations, the use of highly parallelizable hard-
ware and at least 4 processor cores for running complex experiments. While
this appears to make the use of dedicated hardware for the single purpose
of running z-Tree unleashed more attractive, the cost of such hardware in
datacenters may be substantial. In most cases, it will also not be strictly
necessary to run the z-Tree unleashed server for 24 hours a day.
Another surprise was that the uplink bandwidth available in our tests
was low and at the same time we experienced a latency when transmitting
inputs of almost zero. We expected to need a very fast Internet connection,
but we were surprised to nd that z-Tree unleashed was highly responsive.
During testing we used a consumer-grade Internet link and did not notice
any latency or interruptions. As depicted in Figure 6 our test session created
only a total trac of a few dozen megabytes. Therefore it is well possible
to use z-Tree unleashed over a cellular data link.
14Basically we customized a command described by Oliver Kirchkamp, see https://
www.kirchkamp.de//lab/zTree.html, retrieved March 30, 2020.
14
0
5
10
15
0 250 500 750 1000 1250
MB
Incoming
Outgoing
Cumulative Network Traffic
a)
0
5
10
15
0 200 400 600
b)
0
25
50
75
100
0 250 500 750 1000 1250
Utilization (%)
Average CPU Utilization
0
25
50
75
100
0 200 400 600
1000
2000
3000
4000
0 250 500 750 1000 1250
MiB
Memory Utilization
1000
2000
3000
4000
0 200 400 600
Figure 6: On the left: system utilization while running GIMS, a heavy load
market simulation (see Palan (2015)). On the right: a regular public goods
game. Usage was recorded every half second.
15
8 Conclusion
In this paper, we presented z-Tree unleashed, a novel tool with a set of scripts
to enable the administration of economic laboratory experiments created
with z-Tree over the Internet. z-Tree unleashed enables experimenters and
lab administrators to “put the lab in a briefcase”, and it grants participants
the opportunity to rely on secure and resilient communication channels while
participating from anywhere in the world.
The two main challenges involved with z-Tree unleashed are the hard-
ware and the network bottleneck. But as outlined in the previous sections,
z-Tree unleashed has moderate processing power and bandwidth require-
ments. Other challenges that were traditionally salient with the implemen-
tation of online experiments, such as the global reachability of the server it-
self, have been overcome through the use of a VPN. We have also addressed
data privacy concerns by only using authenticated and end-to-end-encrypted
networking channels.
Our tool can not only be used to simplify existing laboratory setups and
take them to the Internet, but also to build low budget laboratories that
can be used in every place with any network connection, cellular, Wi-Fi or
LAN. z-Tree unleashed can also help to reduce downtime by allowing for
a simple replacement of server and client hardware, and by being able to
switch to another network easily. Finally our tool can also help to reduce
the carbon footprint of experimental labs by encouraging the use of “thin”
hardware with low power usage and longer service times.
z-Tree unleashed has been released under a permissive free software li-
cense, allowing anyone to use, adapt and redistribute the software package.
Our license is a derivative of the Mozilla Public License 2.0 with an addi-
tional clause requiring the citation of this paper in all academic publications
that make use of the z-Tree unleashed scripts or derivatives thereof.
In principle, the methodology outlined here applies to any stationary lab
software. With little eort, our scripts could be adopted such that other
software can be “streamed” over the Internet. We invite discussion and
contributions to our scripts on the GitLab repository and the Google Group
of z-Tree unleashed.15
On our project website, we provide a royalty-free virtual machine image
that can be used on dedicated hardware as well as for the occasional exper-
iment. This virtual image contains all z-Tree unleashed scripts and allows
lab managers and researchers the exibility of risklessly experimenting with
dierent setups. An additional advantage is that rstly, the virtual machine
image is an isolated container that can be trivially transferred between dig-
ital computers; and that secondly, many adjustments can be made such as
15See https://gitlab.com/cler1/zTree-unleashed as well as https://groups.
google.com/d/forum/ztree-unleashed.
16
an increase of the number of processor cores that are available to the virtual
machine. Our virtual machine has been created in and can be imported into
Oracle VM VirtualBox, a free software hosted hypervisor. It is essentially
“plug and play”.
9 Acknowledgments
We acknowledge funding by the European Research Council (ERC) under
the European Union’s Horizon 2020 research and innovation program under
grant agreement No. 741409, as well as by the German Research Foundation
(DFG) under Germany’s Excellence Strategy (EXC 2126/1—390838866).
We thank Urs Fischbacher for supporting our idea and allowing us to
use the name z-Tree.
References
Amstadt, B. and Johnson, M. K. (1994). “Wine”. Linux Journal, 1994(4es),
p. 3 (cited on p. 8).
Balietti, S. (2017). “nodeGame: Real-time, synchronous, online experiments
in the browser”. Behavior research methods, 49(5), pp. 1696–1715 (cited
on p. 2).
Barker, E., Chen, L., Keller, S., Roginsky, A., Vassilev, A., and Davis, R.
(2017). Recommendation for pair-wise key-establishment schemes using
discrete logarithm cryptography. Tech. rep. National Institute of Stan-
dards and Technology (cited on p. 13).
Chen, D. L., Schonger, M., and Wickens, C. (2016). “oTree—An open-source
platform for laboratory, online, and eld experiments”. Journal of Be-
havioral and Experimental Finance, 9, pp. 88–97 (cited on p. 2).
Donenfeld, J. A. (2017). “WireGuard: Next Generation Kernel Network Tun-
nel.” In: NDSS (cited on pp. 13, 14).
Fischbacher, U. (2007). “z-Tree: Zurich toolbox for ready-made economic
experiments”. Experimental economics, 10(2), pp. 171–178 (cited on p. 1).
Henninger, F., Kieslich, P. J., and Hilbig, B. E. (2017). “Psynteract: A exi-
ble, cross-platform, open framework for interactive experiments”. Behav-
ior research methods, 49(5), pp. 1605–1614 (cited on p. 2).
Lee, H. C. and Thing, V. L. (2004). “Port hopping for resilient networks”.
In: IEEE 60th Vehicular Technology Conference, 2004. VTC2004-Fal l.
2004. Vol. 5. IEEE, pp. 3291–3295 (cited on p. 10).
Palan, S. (2015). “GIMS—Software for asset market experiments”. Journal
of behavioral and experimental nance, 5, pp. 1–14 (cited on p. 15).
17
Pattinson, C., Cross, R., and Kor, A.-L. (2015). “Chapter 14 - Thin-Client
and Energy Eciency”. In: Green Information Technology. Ed. by M.
Dastbaz, C. Pattinson, and B. Akhgar. Boston: Morgan Kaufmann, pp. 279–
294 (cited on p. 5).
Pettit, J., Friedman, D., Kephart, C., and Oprea, R. (2014). “Software for
continuous game experiments”. Experimental Economics, 17(4), pp. 631–
648 (cited on p. 2).
Scheier, R. W., Gettys, J., Mento, A., and Converse, D. (1997). X Window
System: Core and Extension Protocols. Digital Press (cited on p. 8).
Tanenbaum, A. S. and Wetherall, D. J. (2010). Computer Networks. 5th ed.
Prentice Hall (cited on pp. 9, 11).
18
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
ConG is software for conducting economic experiments in continuous and discrete time. It allows experimenters with limited programming experience to create a variety of strategic environments featuring rich visual feedback in continuous time and over continuous action spaces, as well as in discrete time or over discrete action spaces. Simple, easily edited input files give the experimenter considerable flexibility in specifying the strategic environment and visual feedback. Source code is modular and allows researchers with programming skills to create novel strategic environments and displays.
Article
Full-text available
nodeGame is a free, open-source JavaScript/ HTML5 framework for conducting synchronous experiments online and in the lab directly in the browser window. It is specifically designed to support behavioral research along three dimensions: (i) larger group sizes, (ii) real-time (but also discrete time) experiments, and (iii) batches of simultaneous experiments. nodeGame has a modular source code, and defines an API (application programming interface) through which experimenters can create new strategic environments and configure the platform. With zero-install, nodeGame can run on a great variety of devices, from desktop computers to laptops, smartphones, and tablets. The current version of the software is 3.0, and extensive documentation is available on the wiki pages at http://nodegame.org .
Article
Full-text available
oTree is an open-source and online software for implementing interactive experiments in the laboratory, online, the field or combinations thereof. oTree does not require installation of software on subjects’ devices; it can run on any device that has a web browser, be that a desktop computer, a tablet or a smartphone. Deployment can be internet-based without a shared local network, or local-network-based even without internet access. For coding, Python is used, a popular, open-source programming language. http://www.oTree.org[www.oTree.org] provides the source code, a library of standard game templates and demo games which can be played by anyone.
Article
Full-text available
This chapter is based on the project Does Thin Client Mean Energy Efficient? funded under the JISC Greening information and communication technology (ICT) initiative. This project was devised to conduct actual measurements in use in a typical university environment. We identified a test area that was a mixed administrative and academic office location that supported a range of users, and we made a direct replacement of the current thick-client systems with thin-client equivalents; in addition, we exchanged a number of PCs operating in thin- and thick-client mode with devices specifically branded as low-power PCs and measured their power requirements in both thin and thick modes. We measured the energy consumption at each desktop for the duration of our experiments and measured the energy draw of the server designated to support the thin-client setup, giving us the opportunity to determine the power per user of each technology. Our results showed a significant difference in power use among the various candidate technologies and that a configuration of low-power PC in thick-client mode returned the lowest power use during our study. We were also aware of other factors surrounding the change such as the technical issues of implementation and management as well as the nontechnical or human factors of acceptance and use; all are reported in this chapter. Finally, our project was necessarily limited to a set of experiments carried out in a particular situation; therefore, we use estimation methods to draw wider conclusions and make general observations that should allow others to select appropriate thick- or thin-client solutions for their situations.
Article
Full-text available
In this article we lay out requirements for an experimental market software for financial and economic research. We then discuss existing solutions. Finally, we introduce GIMS, an open source market software which is characterized by extensibility and ease of use, while offering nearly all of the required functionality.
Article
We introduce a novel platform for interactive studies, that is, any form of study in which participants' experiences depend not only on their own responses, but also on those of other participants who complete the same study in parallel, for example a prisoner's dilemma or an ultimatum game. The software thus especially serves the rapidly growing field of strategic interaction research within psychology and behavioral economics. In contrast to all available software packages, our platform does not handle stimulus display and response collection itself. Instead, we provide a mechanism to extend existing experimental software to incorporate interactive functionality. This approach allows us to draw upon the capabilities already available, such as accuracy of temporal measurement, integration with auxiliary hardware such as eye-trackers or (neuro-)physiological apparatus, and recent advances in experimental software, for example capturing response dynamics through mouse-tracking. Through integration with OpenSesame, an open-source graphical experiment builder, studies can be assembled via a drag-and-drop interface requiring little or no further programming skills. In addition, by using the same communication mechanism across software packages, we also enable interoperability between systems. Our source code, which provides support for all major operating systems and several popular experimental packages, can be freely used and distributed under an open source license. The communication protocols underlying its functionality are also well documented and easily adapted to further platforms. Code and documentation are available at https://github.com/psynteract/ .
Article
z-Tree (Zurich Toolbox for Ready-made Economic Experiments) is a software for developing and conducting economic experiments. The software is stable and allows programming almost any kind of experiments in a short time. In this article, I present the guiding principles behind the software design, its features, and its limitations
Conference Paper
With the pervasiveness of the Internet, denial-of-service (DoS) and distributed DoS (DDoS) attacks have become important threats to servers, hosts and devices that are connected. The paper addresses the problem of mitigating DoS/DDoS attacks so as to ensure that legitimate traffic is given an acceptable level of quality of service. We propose a new technique, called port hopping, where the UDP/TCP port number used by the server varies as a function of time and a shared secret between the server and the client. The main strength of the mechanism lies in the simplification of both the detection and filtering of malicious attack packets and that it does not require any change to existing protocols. This port hopping technique is compatible with UDP and TCP and can be implemented using socket communications for UDP, and for setting up TCP communications. We performed both a theoretical analysis and empirical studies through an actual implementation to study the effectiveness of the scheme against DoS/DDoS flooding attacks. Our experiments show that the port hopping technique is effective in detecting and filtering malicious traffic, and hence improves the reliability of good traffic flow.
  • B Amstadt
  • M K Johnson
Amstadt, B. and Johnson, M. K. (1994). "Wine". Linux Journal, 1994(4es), p. 3 (cited on p. 8).