ArticlePDF Available

Why Johnny still can't encrypt: evaluating the usability of email encryption software

Authors:

Abstract

Our research seeks to understand the current usability situation of email encryption software, particularly PGP 9 in comparison to previous studies of PGP 5. We designed a pilot study to find current problems in the following areas: create a key pair, get public keys, verify public keys, encrypt an email, sign an email, decrypt an email, verify a digital signature, and save a backup of public and private keys.
Why Johnny Still Can’t Encrypt:
Evaluating the Usability of Email Encryption Software
Steve Sheng
Engineering and Public Policy
Carnegie Mellon University
shengx@cmu.edu
Levi Broderick
Electrical and Computer Engineering
Carnegie Mellon University
lpb@ece.cmu.edu
Jeremy J. Hyland
Heinz School of Public Policy and
Management
Carnegie Mellon University
jhyland@andrew.cmu.edu
Colleen Alison Koranda
HCI Institute
Carnegie Mellon University
ckoranda@andrew.cmu.edu
ABSTRACT
Our research seeks to understand the current usability situation of
email encryption software, particularly PGP 9 in comparison to
previous studies of PGP 5. We designed a pilot study to find
current problems in the following areas: create a key pair, get
public keys, verify public keys, encrypt an email, sign an email,
decrypt an email, verify a digital signature, and save a backup of
public and private keys.
1. INTRODUCTION
In the seminal paper “Why Johnny Can’t Encrypt”, Whitten and
Tygar [1] showed that users have great difficulty using email
encryption software PGP. In the study, only 4 out of 12
participants were able to correctly sign and encrypt an email
message in 90 minutes; and one quarter of them accidentally sent
the secret email in clear text. They concluded from the usability
test that “designing security software that is usable enough is a
specialized problem, and user interface strategies that are
appropriate for other types of software will not be sufficient to
solve it [1].” Garfinkel, however, interpreted these results
differently; he argued that the usability issues that Whitten and
Tygar identified were driven by the underlying key certification
model used by PGP [2].
Eight years passed, major changes have been made in PGP such
as semi-automatic key creation and distribution, opportunistic
encryption through email proxy, and automatic email decryption.
The overall key certification architecture still has not changed.
Our research seeks to understand the current usability situation of
email encryption software: What problems have the new features
solved, what problems still remain, are there new problems been
introduced? PGP claims that it is designed to support ‘first time
users,’ as encryption is much more transparent.
We ran a pilot of the study with six novice users using PGP 9 and
Outlook Express 6.0. Even though we only performed a pilot
study, several patterns emerged early to indicate major problems
in PGP 9. Users completed the following tasks: create a key pair,
get public keys, verify public keys, encrypt an email, sign an
email, decrypt an email, verify a digital signature, and save a
backup of public and private keys. We also spoofed a decrypted
email message to test user’s response to PGP’s automatic
decryption.
2. MAJOR FINDINGS
2.1 Verify Keys
We found that key verification and signing is still severely
lacking, such that no user was able to successfully verify their
keys. Similar to PGP 5, users had difficulty with signing keys.
Three of our users were not able to verify the validity of the key
successfully and did not understand the reasoning to do so. Four
users were not able to sign the key, these users attempted to but
struggled with the interface. They did not understand that in order
to ‘verify,’ they must ‘sign’ the key rather than just click ‘verify.’
2.2 Encryption
We found that the transparency of the software’s operation is
problematic. The greatest difficulty for the users was in
determining whether the software would operate as requested, as
no indication was given during message composition as to
whether or not the outgoing data would be encrypted or signed.
Notification of successful encryption only occurs after the email
has been sent. If the email is sent unencrypted, there is no visible
feedback to indicate this to the user. The fact that users kept
using the S/MIME toolbar in Outlook Express demonstrated that
they were not aware of PGP’s background automation. Thus,
none of our six users were able to encrypt. The transparency in
automatically decrypting emails also makes user susceptible to
spoofing attacks against messages that appear to be PGP verified.
2.3 Digitally Sign
Digital Signing of messages is more problematic in PGP 9 than
PGP 5 as none of the users were able to sign message using PGP
9, because there are no cues in the interface that support digital
signatures. This can only be completed by right clicking, on the
PGP system try icon.
3. ADDITIONAL FINDINGS
3.1 Create Keys
Users generally had no problem creating keys. This is an
improvement in PGP 9 because a key generation wizard.
3.2 Send Public Keys
Two users were unable to send their public keys to others. In
PGP, the ‘Email this key’ option appears only after the key is
selected and it was difficult to identify the key location.
3.3 Get Public Keys
Three out of six people were able to get all public keys. For two
of the users, the problem was that they typed in a partial name or
email address, using PGP’s ‘contains’ field but could not find the
key. In PGP, the search relies on entering the text regardless. In
addition, one user could not identify the location for key search.
3.4 Decryption
All users were able to decrypt. This is because PGP automatically
decrypts emails when they appear in Outlook Express. We
attempted to spoof emails by sending text that looked like it was
decrypted. Two out of five users were unable to correctly identify
legitimate emails manually, by comparing the correct key in the
email to the key in PGP. Even though decrypting occurs
automatically, we feel that further research should be done to
evaluate PGP’s automation decryption and spoofing decryption.
3.5 Key Backup
Four out of six people were able to create their backup keys. This
task was relatively simple compared to the previous tasks. For the
users that were unable to complete this task, one did not notice the
‘Include Private Key(s)’ checkbox at the bottom of the otherwise
standard Windows save file dialog. Another user was never able
to figure out that he needed to ‘Export’ his key to save a backup.
Users were searching for the word backup in the interface, and
those that were able to complete the task, spent a lot of time
searching for it.
4. IMPROVEMENTS TO PGP
In summary, compared with Whitten’s study of PGP 5, PGP 9
made strides in automatically encrypting emails. The key
certification process becomes the key to the issue in PGP 9 has
not made any improvements. PGP 9’s presents multiple instances
where the interface does not provide enough cues or feedback for
the user. Based on the pilot test, we suggest the following design
improvements for PGP:
a) For novice users, the location of ‘your keyneeds to be more
apparent. The actions that users want to perform with their key
should be better supported, such as emailing their key and
encryption.
b) Deeper integration or a clearer link between PGP and mail
client is required so users understand what actions can be
performed in each location.
c) The search interface for obtaining others’ keys needs to be
clearer. The ‘contains’ option is misleading and prevents users
from accomplishing their task.
d) The interface for signing an email is not apparent. The
common tasks that PGP allows should be predominant in the main
interface, and not put solely in a system tray icon.
e) More prominent cues are required for users to validate a key.
Clicking on the different options that display validity should
direct users to how they can sign the key to make the validity turn
green.
f) Give users feedback prior to encrypting. This could occur by
letting the users determine when they want an email to be
encrypted and when they do not. Users need to be able to know
ahead of time if their email will be encrypted successfully or not.
g) Users need a simple way to verify email validity. Many users
requested a button that will connect email client to PGP to find
out if the email matches the information in PGP
5. ACKNOWLEDGMENTS
We would like to thank our Usable Privacy and Security
Professors: Lorrie Cranor, Michael Reiter, and Jason Hong for
their help and inspirations.
6. REFERENCES
[1] Alma Whitten and J.D. Tygar, Why Johnny Can't Encrypt: A
Usability Case Study of PGP 5.0. Proceedings of the 8th
USENIX Security Symposium, August 1999.
[2] Simson L. Garfinkel and Robert C. Miller, Johnny 2: A User
Test of Key Continuity Management with S/MIME and
Outlook Express. Symposium On Usable Privacy and
Security (SOUPS), 2005.
... Usability and operational security. Usability is a well-known weakness in security systems [38,40,49]; if Johnny can't encrypt, how can we expect him to blow the whistle anonymously? Operational security mistakes can happen at either end of the communication channel and are more likely when journalists are doing something they do rarely, and the source is doing something for the first time in their life under great stress [3]. ...
... PGP offers the major benefit that messages are not passed in plaintext, but PGP clients are notoriously dif-ficult for even college-educated people to use [38,40,49]. There is also no onion encryption or cover traffic to thwart correlation attacks. ...
Article
Full-text available
Whistleblowing is hazardous in a world of pervasive surveillance, yet many leading newspapers expect sources to contact them with methods that are either insecure or barely usable. In an attempt to do better, we conducted two workshops with British news organisations and surveyed whistleblowing options and guidelines at major media outlets. We concluded that the soft spot is a system for initial contact and trust establishment between sources and reporters. CoverDrop is a two-way, secure system to do this. We support secure messaging within a news app, so that all its other users provide cover traffic, which we channel through a threshold mix instantiated in a Trusted Execution Environment within the news organisation. CoverDrop is designed to resist a powerful global adversary with the ability to issue warrants against infrastructure providers, yet it can easily be integrated into existing infrastructure. We present the results from our workshops, describe CoverDrop’s design and demonstrate its security and performance.
... A wide breakthrough has not happened. Usability issues of the end-user clients seem to be one reason [123] [124]. Additionally, from our point of view, the equality of peers, that is the foundation of a web of trust, and the limited radius of trusted entities are also a drawback of the scheme. ...
Thesis
Full-text available
Identity management is at the forefront of applications’ security posture. It separates the unauthorised user from the legitimate individual. Identity management models have evolved from the isolated to the centralised paradigm and identity federations. Within this advancement, the identity provider emerged as a trusted third party that holds a powerful position. Allen postulated the novel self-sovereign identity paradigm to establish a new balance. Thus, extensive research is required to comprehend its virtues and limitations. Analysing the new paradigm, initially, we investigate the blockchain-based self-sovereign identity concept structurally. Moreover, we examine trust requirements in this context by reference to patterns. These shapes comprise major entities linked by a decentralised identity provider. By comparison to the traditional models, we conclude that trust in credential management and authentication is removed. Trust-enhancing attribute aggregation based on multiple attribute providers provokes a further trust shift. Subsequently, we formalise attribute assurance trust modelling by a metaframework. It encompasses the attestation and trust network as well as the trust decision process, including the trust function, as central components. A secure attribute assurance trust model depends on the security of the trust function. The trust function should consider high trust values and several attribute authorities. Furthermore, we evaluate classification, conceptual study, practical analysis and simulation as assessment strategies of trust models. For realising trust-enhancing attribute aggregation, we propose a probabilistic approach. The method exerts the principle characteristics of correctness and validity. These values are combined for one provider and subsequently for multiple issuers. We embed this trust function in a model within the self-sovereign identity ecosystem. To practically apply the trust function and solve several challenges for the service provider that arise from adopting self-sovereign identity solutions, we conceptualise and implement an identity broker. The mediator applies a component-based architecture to abstract from a single solution. Standard identity and access management protocols build the interface for applications. We can conclude that the broker’s usage at the side of the service provider does not undermine self-sovereign principles, but fosters the advancement of the ecosystem. The identity broker is applied to sample web applications with distinct attribute requirements to showcase usefulness for authentication and attribute-based access control within a case study.
... However, before secure communication can be enabled, users must generate encryption and signature key pairs, be verified by a certificate authority (CA) and receive CA-signed certificates. Furthermore, key management issues including the key storage capacity required to archive all the private keys for distinct users and key certification and validation processes [24,25] result in major drawbacks of the public-key cryptography's practical implementation. The idea of identity-based cryptography was first proposed by Shamir in 1984 [26], putting forth the notion of using a unique string such as a user's name, email address or contact number to explicitly compute the user's private key. ...
Article
Full-text available
The Internet of Things (IoT) represents a growing aspect of how entities, including humans and organizations, are likely to connect with others in their public and private interactions. The exponential rise in the number of IoT devices, resulting from ever-growing IoT applications, also gives rise to new opportunities for exploiting potential security vulnerabilities. In contrast to conventional cryptosystems, frameworks that incorporate fine-grained access control offer better opportunities for protecting valuable assets, especially when the connectivity level is dense. Functional encryption is an exciting new paradigm of public-key encryption that supports fine-grained access control, generalizing a range of existing fine-grained access control mechanisms. This survey reviews the recent applications of functional encryption and the major cryptographic primitives that it covers, identifying areas where the adoption of these primitives has had the greatest impact. We first provide an overview of different application areas where these access control schemes have been applied. Then, an in-depth survey of how the schemes are used in a multitude of applications related to IoT is given, rendering a potential vision of security and integrity that this growing field promises. Towards the end, we identify some research trends and state the open challenges that current developments face for a secure IoT realization.
... This is especially important because a ceremony's non-use or a negative experience of an unattractive ceremony will compromise security and leave holes open for hackers to exploit. Current experiences appear to confirm their general unattractiveness (Cranor & Garfinkel, 2005;Sheng et al., 2006;Clark et al., 2011). The consequence is that users might try to circumvent the ritual (Blythe et al., 2013), especially when their so-called "compliance budget" (Beautement et al., 2008) has been depleted. ...
Article
Full-text available
When we use secure computer systems, we engage with carefully orchestrated and ordered interactions called “security ceremonies”, all of which exist to assure security. A great deal of attention has been paid to improving the usability of these ceremonies over the last two decades, to make them easier for end-users to engage with. Yet, usability improvements do not seem to have endeared end users to ceremonies. As a consequence, human actors might subvert the ceremony’s processes or avoid engaging with it. Here, we consider whether beautification could be one way of making ceremonies more appealing. To explore beautification in this context, we carried out three studies. Study 1 surveyed 250 participants to derive a wide range of potential dimensions of “beautiful ceremonies”. These statements were sorted into dominant themes and converted into statements, which fed into the second study, with 309 respondents, to reveal the dominant dimensions constituting beauty. Study 3 asked 41 participants to carry out a Q-sort, which revealed the ways that people combine the identified dimensions when characterising security ceremonies as “beautiful”. These studies have allowed us to pin down the perceived dimensions of beauty in the context of security ceremonies, and also to understand how people combine these dimensions in different ways in judging security ceremonies to be beautiful, confirming the old adage of beauty being “in the eye of the beholder”. We conclude by highlighting the constraints imposed by the overarching requirement for security to be maintained in the face of any usability improvements and beautification endeavours.
Chapter
Pretty Good Privacy (PGP) is well-known in the civil rights community – its author Phil Zimmerman was sued for violating US export controls, and Edward Snowden used PGP to secure his email communication with Glenn Greenwald. This chapter will concentrate on OpenPGP as a universal cryptographic data format. It can be used to encrypt files in an operating system, sign software updates, or communicate securely via email. Like any other cryptographic data format, it has been subject to attacks. Attacks on the data format itself are covered in this chapter, and attacks on OpenPGP-based email encryption are covered in chapter 18.
Article
Cloud-based email is one of the services offered by cloud computing, and the number of users continues to grow year after year. Because of its working environment, cloud computing raises concerns about security and privacy. User authentication in cloud computing is now predicated on the user's credentials, which are typically username and password. User authentication in cloud computing is currently predicated on the credentials possessed by the user, which are primarily username and password. With the growing usage of cloud emails and numerous allegations of large-scale email leakage occurrences, a security attribute known as forward secrecy has become desirable and necessary for both users and cloud email service providers to strengthen the security of their communications. However, due to the failure of email systems to meet both security and practicality requirements at the same time. A fine-grained revocation capacity is available to an email user. A security key will be provided by the user to prevent hacking of such email addresses. The MAES(Modified Advanced Encryption Standard) algorithm encrypts files and a user's email ID to safeguard their data from a third party or hackers to address this issue more efficiently. This proposed hybrid security method secures the content of emails before they are sent through email using an Advanced Cipher Technique (ACT). The study suggests employing substitution and permutation to secure email content, with the fronts offered by email systems acting as keys.
Chapter
E-mail is nearly 50 years old and is still one of the most used communication protocols nowadays. However, it has no support for End-to-end encryption (E2EE) by default, which makes it inappropriate for sending sensitive information. This is why two e-mail encryption standards have been developed—namely, Secure/Multipurpose Internet Mail Extensions (S/MIME) and OpenPGP. Previous studies found that bad usability of encryption software can lead to software that is incorrectly used or not at all. Both consequences have a fatal impact on users’ security and privacy. In recent years, the number of e-mails that are read and written on mobile devices has increased drastically. In this paper, we conduct to the best of our knowledge, the first usability study of e-mail encryption apps on smartphones. We tested two mobile apps, one uses OpenPGP on Android and one uses S/MIME on iOS. In our usability study, we tested both apps with eleven participants and evaluated the usability with the System Usability Scale (SUS) and the Short Version of User Experience Questionnaire (UEQ-S). Our study shows that both apps have several usability issues which partly led to unencrypted e-mails and participants sending their passphrase instead of their public key.
Article
Full-text available
Gmail’s confidential mode enables a user to send confidential emails and control access to their content through setting an expiration time and passcode, pre-expiry access revocation, and prevention of email forwarding, downloading, and printing. This paper aims to understand user perceptions and motivations for using Gmail’s confidential mode (GCM). Our structured interviews with 19 Gmail users at UNC Charlotte show that users utilize this mode to share their private documents with recipients and perceive that this mode encrypts their emails and attachments. The most commonly used feature of this mode is the default time expiration of one week, and the least used feature is the pre-expiry access revocation. Our analysis suggests several design improvements.
Conference Paper
Full-text available
As large messaging providers increasingly adopt end-to-end encryption, private communication is readily available to more users than ever before. However, misunderstandings of end-to-end encryption's benefits and shortcomings limit people's ability to make informed choices about how and when to use these services. This paper explores the potential of using short educational messages, built into messaging workflows, to improve users' functional mental models of secure communication. A preliminary survey study (n=461) finds that such messages, when used in isolation, can effectively improve understanding of several key concepts. We then conduct a longitudinal study (n=61) to test these messages in a more realistic environment: embedded into a secure messaging app. In this second study, we do not find statistically significant evidence of improvement in mental models; however, qualitative evidence from participant interviews suggests that if made more salient, such messages could have potential to improve users' understanding.
Conference Paper
Full-text available
Secure email has struggled with signifcant obstacles to adoption, among them the low usability of encryption software and the cost and overhead of obtaining public key certificates. Key continuity management (KCM) has been proposed as a way to lower these barriers to adoption, by making key generation, key management, and message signing essentially automatic. We present the first user study of KCM-secured email, conducted on naïve users who had no previous experience with secure email. Our secure email prototype, CoPilot, color-codes messages depending on whether they were signed and whether the signer was previously known or unknown. This interface makes users signicantly less susceptible to social engineering attacks overall, but new-identity attacks (from email addresses never seen before) are still effective. Also, naïve users do use the Sign and Encrypt button on the Outlook Express toolbar when the situation seems to warrant it, even without explicit instruction, although some falsely hoped that Encrypt would protect a secret message even when sent directly to an attacker. We conclude that KCM is a workable model for improving email security today, but work is needed to alert users to "phishing" attacks.
Article
User errors cause or contribute to most computer security failures, yet user interfaces for security still tend to be clumsy, confusing, or near-nonexistent. Is this simply due to a failure to apply standard user interface design techniques to security? We argue that, on the contrary, effective security requires a different usability standard, and that it will not be achieved through the user interface design techniques appropriate to other types of consumer software. To test this hypothesis, we performed a case study of a security program which does have a good user interface by general standards: PGP 5.0. Our case study used a cognitive walkthrough analysis together with a laboratory user test to evaluate whether PGP 5.0 can be successfully used by cryptography novices to achieve effective electronic mail security. The analysis found a number of user interface design flaws that may contribute to security failures, and the user test demonstrated that when our test participants were g...