Kiran Lakkaraju

Kiran Lakkaraju
Sandia National Laboratories - CA · Systems Analysis and Research

Ph.D. in Computer Science

About

114
Publications
22,912
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
2,088
Citations
Introduction
Computer scientist with 10+ years of experience leading cross-disciplinary teams developing innovative AI/ML solutions to national security problems. I use AI/ML, modeling & simulation, online experiments, and wargaming to study topics spanning consumer purchasing decisions, language and information diffusion, attitude change, cybersecurity and deterrence. I believe the best insights and progress come from rich interaction between fields.
Additional affiliations
November 2010 - present
Sandia National Laboratories
Position
  • Senior Member of Technical Staff
Education
August 2002 - August 2009
University of Illinois, Urbana-Champaign
Field of study
  • Computer Science
August 2000 - August 2002
University of Illinois, Urbana-Champaign
Field of study
  • Computer Science
August 1997 - August 2000
Northern Illinois University
Field of study
  • Computer Science

Publications

Publications (114)
Conference Paper
Full-text available
Massively-multiplayer online games (MMOGs) can serve as a unique laboratory for studying large-scale human behaviors. However, one question that often arises is whether the observed behavior is specific to the game world and its winning conditions. This paper studies the nature of conflict and communication across two game worlds that have differen...
Conference Paper
Social media has allowed researchers to induce large social networks from easily accessible online data. However, relationships inferred from social media data may not always reflect the true underlying relationship. The main question of this work is: How does the public social network reflect the private social network? We begin to address this qu...
Conference Paper
Human culture is fundamentally tied with language. We argue that the study of language change and diffusion in a society sheds light on its cultural patterns and social conventions. In addition, language can be viewed as a ”model problem” through which to study complex norm emergence scenarios. In this paper we study a particular linguistically ori...
Conference Paper
Full-text available
Security engineers are being overwhelmed with data from the network monitoring tools. A tool is needed that would allow security engineers to view information about the entire network. In addition, the tool must allow the security engineers to use their background knowledge and intuition. NVisionIP, a tool developed at the National Center for Super...
Article
Full-text available
We develop a methodology for comparing agent-based models that are developed for the same domain, but may differ in the data sets (e.g., geographical regions) to which they are applied, and in the structure of the model. Our approach is to learn a response surface in the common parameter space of the models and compare the regions corresponding to...
Article
Full-text available
The ground truth program used simulations as test beds for social science research methods. The simulations had known ground truth and were capable of producing large amounts of data. This allowed research teams to run experiments and ask questions of these simulations similar to social scientists studying real-world systems, and enabled robust eva...
Article
Full-text available
Social systems are uniquely complex and difficult to study, but understanding them is vital to solving the world’s problems. The Ground Truth program developed a new way of testing the research methods that attempt to understand and leverage the Human Domain and its associated complexities. The program developed simulations of social systems as vir...
Article
Full-text available
Measures of simulation model complexity generally focus on outputs; we propose measuring the complexity of a model’s causal structure to gain insight into its fundamental character. This article introduces tools for measuring causal complexity. First, we introduce a method for developing a model’s causal structure diagram, which characterises the c...
Chapter
There is a wealth of psychological theory regarding the drive for individuals to congregate and form social groups, positing that people may organize out of fear, social pressure, or even to manage their self-esteem. We evaluate three such theories for multi-scale validity by studying them not only at the individual scale for which they were origin...
Article
Full-text available
We develop a methodology for comparing two or more agent-based models that are developed for the same domain, but may differ in the particular data sets (e.g., geographical regions) to which they are applied, and in the structure of the model. Our approach is to learn a response surface in the common parameter space of the models and compare the re...
Chapter
Much has been written on the potential for games to enhance our ability to study complex systems. In this chapter we focus on how we can use games to study national security issues. We reflect on the benefits of using games and the inherent difficulties that we must address. As a means of grounding the discussion, we will present a case study of a...
Article
Technology enables new research designs, and more data
Chapter
The doctrine of nuclear deterrence and a belief in its importance underpins many aspects of United States policy; it informs strategic force structures within the military, incentivizes multi-billion-dollar weapon-modernization programs within the Department of Energy, and impacts international alliances with the 29 member states of the North Atlan...
Article
Cambridge Core - Computing and Society - Social Interactions in Virtual Worlds - edited by Kiran Lakkaraju
Chapter
Social Interactions in Virtual Worlds - edited by Kiran Lakkaraju July 2018
Chapter
Social Interactions in Virtual Worlds - edited by Kiran Lakkaraju July 2018
Article
Social Interactions in Virtual Worlds - edited by Kiran Lakkaraju July 2018
Chapter
Social Interactions in Virtual Worlds - edited by Kiran Lakkaraju July 2018
Chapter
Social Interactions in Virtual Worlds - edited by Kiran Lakkaraju July 2018
Chapter
Social Interactions in Virtual Worlds - edited by Kiran Lakkaraju July 2018
Chapter
Social Interactions in Virtual Worlds - edited by Kiran Lakkaraju July 2018
Conference Paper
Many interesting real-world systems are represented as complex networks with multiple types of interactions and complicated dependency structures between layers. These interactions can be encoded as having a valence with positive links marking interactions such as trust and friendship and negative links denoting distrust or hostility. Extracting in...
Article
Full-text available
The wealth of information available on seemingly every topic creates a considerable challenge both for information providers trying to rise above the noise and discerning individuals trying to find relevant, trustworthy information. We approach this information problem by investigating how passive versus interactive information interventions can im...
Data
This document contains the full survey content used in the experiments. (PDF)
Data
This document includes a csv of the survey data used to generate results. (CSV)
Data
This document is a memo stating exemption from the Sandia Labs Human Studies Board. (PDF)
Data
This document includes the content provided for the passive information study and Energy Games. (PDF)
Conference Paper
Networks extracted from social media platforms frequently include multiple types of links that dynamically change over time; these links can be used to represent dyadic interactions such as economic transactions, communications, and shared activities. Organizing this data into a dynamic multiplex network, where each layer is composed of a single ed...
Article
Full-text available
Agent-based modeling is commonly used for studying complex system properties emergent from interactions among agents. However, agent-based models are often not developed explicitly for prediction, and are generally not validated as such. We therefore present a novel data-driven agent-based modeling framework, in which individual behavior model is l...
Article
Full-text available
Networks extracted from social media platforms frequently include multiple types of links that dynamically change over time; these links can be used to represent dyadic interactions such as economic transactions, communications, and shared activities. Organizing this data into a dynamic multiplex network, where each layer is composed of a single ed...
Article
Full-text available
Most real-world social networks are inherently dynamic, composed of communities that are constantly changing in membership. To track these evolving communities, we need dynamic community detection techniques. This article evaluates the performance of a set of game theoretic approaches for identifying communities in dynamic networks. Our method, D-G...
Article
Full-text available
Attitude diffusion is where “attitudes” (general, relatively enduring evaluative responses to a topic) spread through a population. Attitudes play an incredibly important role in human decision-making (for instance, in health care decisions) and are a critical part of social psychology. However, existing models of diffusion do not account for key d...
Conference Paper
Full-text available
The aim of link prediction is to forecast connections that are most likely to occur in the future, based on examples of previously observed links. A key insight is that it is useful to explicitly model network dynamics, how frequently links are created or destroyed when doing link prediction. In this paper, we introduce a new supervised link predic...
Conference Paper
In the context of military training simulation, “semi-automated forces” are software agents that serve as role players. The term implies a degree of shared control – increased automation allows one operator to control a larger number of agents, but too much automation removes control from the instructor. The desired amount of control depends on the...
Conference Paper
Full-text available
Agent-based modeling is commonly used for studying complex system properties emergent from interactions among many agents. We present a novel data-driven agent-based modeling framework applied to forecasting individual and aggregate residential rooftop solar adoption in San Diego county. Our first step is to learn a model of individual agent behavi...
Conference Paper
We present a new platform to do online, social influence experiments – the Controlled, Online Social Experimentation (CLOSE) system. We describe it’s development, potential uses and justification for use. The CLOSE platform can be used to do long term (weeks to months) experiments in which we can manipulate the interaction networks (within the expe...
Conference Paper
Amazon Mechanical Turk (AMT) has become a powerful tool for social scientists due to its inexpensiveness, ease of use, and ability to attract large numbers of workers. While the subject pool is diverse, there are numerous questions regarding the composition of the workers as a function of when the “Human Intelligence Task”(HIT) is posted. Given the...
Conference Paper
We (Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energys National Nuclear Security Administration under contract DE-AC04-94AL85000. This document has technical report number: SAND2013-9802C.) study the effec...
Article
Amazon Mechanical Turk (AMT) has become a powerful tool for social scientists due to its inexpensiveness, ease of use, and ability to attract large numbers of workers. While the subject pool is diverse, there are numerous questions regarding the composition of the workers as a function of when the “Human Intelligence Task” (HIT) is posted. Given th...
Book
This book constitutes the refereed proceedings of the Workshop on Collaborative Agents, Research and Development, CARE 2014, and the Workshop on Agents, Virtual Societies and Analytics, AVSC 2014, held as Part of AAMAS 2014 in Paris, France, in May 2014. The 10 revised full papers presented were carefully reviewed and selected from more than 40 sub...
Article
Full-text available
Plug-in Hybrid Electric Vehicles (PHEVs) show potential to reduce greenhouse gas (GHG) emissions, increase fuel efficiency, and offer driving ranges that are not limited by battery capacity. However, these benefits will not be realized if consumers do not adopt this new technology. Several agent-based models have been developed to model potential m...
Conference Paper
Full-text available
Massively multiplayer online games (MMOGs) offer a unique laboratory for examining large-scale patterns of human behavior. In particular, the study of guilds in MMOGs has yielded insights about the forces driving the formation of human groups. In this paper, we present a computational model for predicting guild membership in MMOGs and evaluate the...
Article
Full-text available
In this paper we present a novel agent-based modeling methodology to predict rooftop solar adoptions in the residential energy market. We first applied several linear regression models to estimate missing variables for non-adopters, so that attributes of non-adopters and adopters could be used to train a logistic regression model. Then, we integrat...
Article
We consider the question of predicting solar adoption using demographic, economic, peer effect and predicted system characteristic features. We use data from San Diego county to evaluate both discrete and continuous models. Additionally, we consider three types of sensitivity analysis to identify which features seem to have the greatest effect on p...
Chapter
Full-text available
In this preliminary work, we study how public forum communication reflects and shapes virtual world behavior. We find that in-game groups have differential public posting habits; that player behavior is reflected in public communication (in particular, players who attack more are mentioned more in the public forums), and finally that public and per...
Article
Full-text available
This dissertation studies multiagent agreement problems – problems in which a population of agents must agree on some quantity or behavior in a distributed manner. Agreement problems are central in many areas, from the study of magnetism (Ising model), to understanding the diffusion of innovations (such as the diffusion of hybrid corn planting in I...
Conference Paper
Full-text available
Complex systems are of interest to the scientific community due to their ubiquity and diversity in daily life. Popularity notwithstanding, the analysis of complex systems remains a difficult task, due to the problems in capturing high-volume data. Massively Multiplayer Online Games (MMOGs) have recently emerged as a tractable way to analyze complex...
Chapter
Attitudes play a significant role in determining how individuals process information and behave. In this paper we have developed a new computational model of population wide attitude change that captures the social level: how individuals interact and communicate information, and the cognitive level: how attitudes and concept interact with each othe...
Conference Paper
Attitudes play a critical role in informing resulting behavior. Extending previous work, we have developed a model of population wide attitude change that captures social factors through a social network, cognitive factors through a cognitive network and individual differences in influence. All three of these factors are supported by literature as...
Conference Paper
In this paper we performed analysis of speech communications in order to determine if we can differentiate between expert and novice teams based on communication patterns. Two pairs of experts and novices performed numerous test sessions on the E-2 Enhanced Deployable Readiness Trainer (EDRT) which is a medium-fidelity simulator of the Naval Flight...
Conference Paper
Full-text available
Technologies are needed enabling more cost-effective military aviation training. Automated performance assessment has been advanced as one approach to enable instructors to make more effective use of simulation-based training systems. Recent experimental research will be reviewed illustrating that automated techniques produce student assessments co...
Conference Paper
Full-text available
In this paper we performed analysis of speech communications in order to determine if we can differentiate between expert and novice teams based on communication patterns. In this experiment, pairs of experts and novices performed numerous test sessions on the E-2 Enhanced Deployable Readiness Trainer (EDRT) which is a medium-fidelity simulator of...
Conference Paper
Modeling and simulation can be an important tool in helping develop techniques to better communicate safety-critical information for disaster preparation and recovery. However, these tools are only moderately useful if they do not capture both the social component (how information diffuses in a population through communication between individuals)...
Article
Full-text available
Sociolinguistic studies have demonstrated that centrally-connected and peripheral members of social networks can both propel and impede the spread of linguistic innovations. We use agent-based computer simulations to investigate the dynamic properties of these network roles in a large social influence network, in which diffusion is modeled as the p...
Article
Full-text available
We simulate the dynamics of diffusion and establishment of norms, variants adopted by the majority of agents, in a large social influence network with scale-free small-world properties. Diffusion is modeled as the probabilistic uptake of one of several competing variants by agents of unequal social standing. We find that novel variants diffuse foll...
Article
Full-text available
also compromised. Little did they realize that this was only the very smallest tip of the iceberg. Rather quickly, it was discovered that the attacker, who later started identifying himself as "Stakkato," spread his attacks across much more than the NCSA network. He exploited a number of specific vulnerabilities across many of the TeraGrid sites. T...
Chapter
It is desirable for many reasons to share information, particularly computer and network logs. Researchers need it for experiments, incident responders need it for collaborative security, and educators need this data for real world examples. However, the sensitive nature of this information often prevents its sharing. Anonymization techniques have...
Chapter
It is desirable for many reasons to share information, particularly computer and network logs. Researchers need it for experiments, incident responders need it for collaborative security, and educators need this data for real world examples. However, the sensitive nature of this information often prevents its sharing. Anonymization techniques have...
Conference Paper
Full-text available
In recent years, it has become important for researchers, se- curity incident responders and educators to share network logs, and many log anonymization tools and techniques have been put forth to sanitize this sensitive data source in or- der to enable more collaboration. Unfortunately, many new attacks have been created, in parallel, that try to...
Article
Full-text available
The basic paradigm of learning has shifted significantly, from single agents that learn in single, static environments, to collective learning: multiple, interacting agents with diverse goals learning from each other across different local environ-ments. Instances of collective learning abound in sensor net-works, peer-to-peer systems, distributed...
Article
Full-text available
It is widely assumed that, over their evolutionary history, languages increased in complexity from simple signals to protolanguages to complex syntactic structures. This paper investigates processes for increasing linguistic complexity while maintaining communicability across a pop-ulation. We assume that linguistic communicability is important for...
Article
Full-text available
Introduction A language is useless unless it is shared. Individuals and subgroups modify languages by adding new words, creating new grammatical constructions, etc., and propagating these changes through contact. To maintain communicability over time, the population as a whole must converge (possibly within some small diversity limit) to agreement...
Article
Full-text available
Anonymization is the process of removing or hiding sensitive information in logs. Anonymization allows organizations to share network logs while not exposing sensitive information. However, there is an inherent trade off between the amount of information revealed in the log and the usefulness of the log to the client (the utility of a log). There a...
Article
Full-text available
Our general objective is to explain how norms can emerge in complex, ambiguous situations: settings with large and com- plex spaces of normative options over which populations may try to agree using only limited, indirect knowledge of each others' currently preferred options, possibly gained through limited interaction samples. We study this proces...