• Home
  • David Ayman Shamma
David Ayman Shamma

David Ayman Shamma
Toyota Research Institute

Doctor of Philosophy

About

128
Publications
55,598
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
7,379
Citations
Additional affiliations
July 2017 - present
FX Palo Alto Laboratory
Position
  • Researcher
August 2016 - June 2017
Centrum Wiskunde & Informatica
Position
  • Researcher
Description
  • Principal Investigator on forthcoming Internet of Things project.
July 2006 - April 2016
Yahoo
Position
  • Managing Director
Education
September 2001 - December 2005
Northwestern University
Field of study
  • Computer Science

Publications

Publications (128)
Preprint
Full-text available
Virtual environments (VEs) can create collaborative and social spaces, which are increasingly important in the face of remote work and travel reduction. Recent advances, such as more open and widely available platforms, create new possibilities to observe and analyse interaction in VEs. Using a custom instrumented build of Mozilla Hubs to measure p...
Preprint
Photo applications offer tools for annotation via text and stickers. Ideophones, mimetic and onomatopoeic words, which are common in graphic novels, have yet to be explored for photo annotation use. We present a method for automatic ideophone recommendation and positioning of the text on photos. These annotations are accomplished by obtaining a lis...
Conference Paper
This paper reflects on the development of a multi-sensory clubbing experience which was deployed during a two-day event within the context of the Amsterdam Dance Event in October 2016 in Amsterdam. We present how the entire experience was developed end-to-end and deployed at the event through the collaboration of several project partners from indus...
Conference Paper
Humans are complex and their behaviors follow complex multi-modal patterns, however to solve many social computing problems one often looks at complexity in large-scale yet single point data sources or methodologies. While single data/single method techniques, fueled by large scale data, enjoyed some success, it is not without fault. Often with one...
Conference Paper
Recommendation systems rely on various definitions of similarities. These definitions while having numerous design factors in different domains help identify and recommend relevant content. For example, similarity between users, or items, are measured based on, but not limited to, explicit feedback such as ratings, thumbs up; or/and implicit feedba...
Conference Paper
Proliferation of GPS-enabled mobile devices has brought a plurality of location-aware applications leveraging the location characteristics in the shared content, like photos and check-ins. While these applications provide contextual and relevant information, they also assume geo-tagged contents to be representative of the geo-bounded characteristic...
Conference Paper
The CWI-ADE2016 Dataset is a collection of more than 40 million Bluetooth Low Energy (BLE) packets and of 14 million accelerometer and temperature samples generated by wristbands that people wore in a nightclub. The data was gathered during Amsterdam Dance Event 2016 in an exclusive club experience curated around human senses, which leveraged techn...
Conference Paper
This article showcases our effort to explore the music club of the future. We present the development and results of an end-to-end system which enhances the club-going experience through the use of wearable technology. Each party guest wearing one of the wristbands actively contributes to the overall experience with their movement and location patt...
Conference Paper
Social media sites are challenged by both the scale and variety of deviant behavior online. While algorithms can detect spam and obscenity, behaviors that break community guidelines on some sites are difficult because they have multimodal subtleties (images and/or text). Identifying these posts is often regulated to a few moderators. In this paper,...
Article
Full-text available
Despite progress in perceptual tasks such as image classification, computers still perform poorly on cognitive tasks such as image description and question answering. Cognition is core to tasks that involve not just recognizing, but reasoning about our visual world. However, models used to tackle the rich content in images for cognitive tasks are s...
Conference Paper
The world is a big place. At any given instant something is happening somewhere, but even when nothing in particular is going on people still find ways to generate data, such as posting on social media, taking photos, and issuing search queries. A substantial number of these actions is associated with a location, and in an increasingly mobile and c...
Conference Paper
In the past decade, location-based services have grown through geo-tagging and place-tagging. Proliferation of GPS-enabled mobile devices further enabled exponential growth in geo-tagged user content. On the other hand, location-based applications harness the abundance of geo-tagged content to further improve user experience and more relevant local...
Conference Paper
We propose a novel hashing-based matching scheme, called Locally Optimized Hashing (LOH), based on a state-of-the-art quantization algorithm that can be used for efficient, large-scale search, recommendation, clustering, and deduplication. We show that matching with LOH only requires set intersections and summations to compute and so is easily impl...
Conference Paper
Online photo collections are often presented along with numeric data, such as views, likes, or comments. Treemaps are ideal for visualizing such collections, as they present numeric values using rectangular blocks, within which photos can be presented. Despite abundant research showing that photo treemaps and similar space-filling approaches are us...
Conference Paper
People taking photographs with their tablets is an increasingly common sight. While the design of hardware is larger and more cumbersome than the smaller and more often carried cameraphones, the underlying photographic software is essentially the same. At the same time, cameraphone hardware usually contains objectively better quality cameras with m...
Conference Paper
Participants in social media systems must balance many considerations when choosing what to share and with whom. Sharing with others invites certain risks, as well as potential benefits; achieving the right balance is even more critical when sharing photos, which can be particularly engaging, but potentially compromising. In this paper, we examine...
Conference Paper
Animated GIFs have been around since 1987 and recently gained more popularity on social networking sites. Tumblr, a large social networking and micro blogging platform, is a popular venue to share animated GIFs. Tumblr users follow blogs, generating a feed or posts, and choose to "like' or to "reblog' favored posts. In this paper, we use these acti...
Conference Paper
Temporal terms, such as 'winter', 'Christmas', or 'January' are often used in search queries for personal images. But how do people's memories and perceptions of time match with the actual dates when their images were captured? We compared the temporal terms that 74 Flickr users used to search their own photo collections, and compared them to the d...
Conference Paper
Our research is delivered as Portable Document Format (PDF) documents, and very few include basic metadata to make them accessible to people with disabilities. As a result, many people are either unable to read them efficiently or at all. Over the past few years, we have tried everything from writing guidelines and giving accessibility feedback, to...
Conference Paper
Social media has begun to migrate from a predominantly text-based medium, through photography and into cinematography and edited video. Film is a vital medium through which we not only capture our world, but also seek to understand it. This workshop explores an emerging area of research within the CHI community that focuses on applying filmic techn...
Article
Full-text available
We present a multimedia system based on a novel matching signature able to perform de-duplucation, search, clustering and visual recommendations in a way that is easily implemented in generic distributed computing environments. Starting from a state-of-the-art algorithm, we propose a novel hashing-based matching system that allow for fast search an...
Article
Full-text available
The quality of user experience online is affected by the relevance and placement of advertisements. We propose a new system for selecting and displaying visual advertisements in image search result sets. Our method compares the visual similarity of candidate ads to the image search results and selects the most visually similar ad to be displayed. T...
Conference Paper
There are many cues that can be used to curate media from social networking websites. Beyond metadata, group behavior provide a strong community-based signal for surfacing images, which we show in a user-defined curatorial task. In a departure from mirco-task crowdwork, we observe that the curation inherent in online photo communities guides the di...
Article
Microtask crowdsourcing has enabled dataset advances in social science and machine learning, but existing crowdsourcing schemes are too expensive to scale up with the expanding volume of data. To scale and widen the applicability of crowdsourcing, we present a technique that produces extremely rapid judgments for binary and categorical labels. Rath...
Article
Points of interest are an important requirement for location-based services, yet they are editorially curated and maintained, either professionally or through community. Beyond the laborious manual annotation task, further complications arise as points of interest may appear, relocate, or disappear over time, and may be relevant only to specific co...
Article
We created the Yahoo Flickr Creative Commons 100 Million Dataseta (YFCC100M) in 2014 as part of the Yahoo Webscope program, which is a reference library of interesting and scientifically useful datasets. The YFCC100M is the largest public multimedia collection ever released, with a total of 100 million media objects, of which approximately 99.2 mil...
Conference Paper
The publication of the Yahoo Flickr Creative Commons 100 Million dataset (YFCC100M)--to date the largest open-access collection of photos and videos--has provided a unique opportunity to stimulate new research in multimedia analysis and retrieval. To make the YFCC100M even more valuable, we have started working towards supplementing it with a compr...
Conference Paper
This ACM MM panel aims to redefine the state of research between Academia and Industry.
Conference Paper
The Arts and Digital Culture program has offered a high quality forum for the presentation of interactive and arts-based multimedia applications at the annual ACM Multimedia conference for over a decade. This tutorial will explore the evolution of this program as a guide to new authors considering future participation in this program. By surveying...
Conference Paper
The Multimedia COMMONS workshop laid the groundwork for developing a research community around the Multimedia Genome Project (MMGP), an initiative initially focused on annotation of---and research using---the 99.2 million images and nearly 800,000 videos in the Yahoo Flickr Creative Commons 100 Million dataset (YFCC100M). Current and potential user...
Conference Paper
Full-text available
With the use of several tablet devices and a shared large display, CozyMaps is a multi-display system that supports real-time collocated collaboration on a shared map. This paper builds on existing works and introduces rich user interactions by proposing awareness, notification, and view sharing techniques, to enable seamless information sharing an...
Article
Photos are an important information carrier for implicit relationships. In this article, we introduce an image based social network, called CelebrityNet, built from implicit relationships encoded in a collection of celebrity images. We analyze the social properties reflected in this image-based social network and automatically infer communities amo...
Article
Full-text available
This special issue touches on many significant aspects of multimedia retrieval, including content analysis and understanding, content- and context-based indexing, search and retrieval, HCI technologies, and image and video summarization and visualization. It converges on the nexus of social multimedia and storytelling around real-world experiences,...
Article
Full-text available
Geotagged data can be used to describe regions in the world and discover local themes. However, not all data produced within a region is necessarily specifically descriptive of that area. To surface the content that is characteristic for a region, we present the geographical hierarchy model (GHM), a probabilistic model based on the assumption that...
Article
Full-text available
We present the Yahoo Flickr Creative Commons 100 Million Dataset (YFCC100M), the largest public multimedia collection that has ever been released. The dataset contains a total of 100 million media objects, of which approximately 99.2 million are photos and 0.8 million are videos, all of which carry a Creative Commons license. Each media object in t...
Conference Paper
Increasingly online reviews are relied upon to make choices about the purchases and services we use daily. Businesses, on the other hand, depend on online review sites to find new customers and understand people's perception of them. In order for an online review community to be effective to both users and businesses, it is important to understand...
Article
Online recommendation communities, like Yelp, are valuable information sources for people. Yet, we assert, review communities have their own dynamics behind the social interactions therein. In this work, we study the Yelp review votes of useful, funny, and/or cool to understand these social perceptions of the review. We examine the relationship bet...
Article
People take photos all over the world at all times of day; each photo depicting a place and a moment worth capturing. In the context of multimedia analysis and social computing, accurate location and time information about where and when these photos were taken is of importance for understanding event semantics, image content and many other purpose...
Article
The Arts and Digital Culture program has offered a high quality forum for the presentation of interactive and artsbased multimedia applications at the annual ACM Multimedia conference for over a decade. This tutorial will explore the evolution of this program as a guide to new authors considering future participation in this program. By surveying b...
Conference Paper
Recently, the popularity of user generated videos has highlighted efficient video indexing and browsing as an urgent problem. Points of interest (POI) detection is a technique to address this issue by establishing the implicit relationship among different media resources. The majority of existing studies detect POI by visual similarity, leveraging...
Patent
Full-text available
Techniques for displaying advertisements on web pages are provided. A revisitation rate for views of a web page by a user population is determined. A refresh rate for an advertisement space on the web page is synchronized with the determined revisitation rate. An advertisement is selected to be displayed at the advertisement space at a particular r...
Article
Full-text available
How do people keep track of their money? In this paper we present a preliminary scoping study of how 14 individuals in the San Francisco Bay Area earn, save, spend and understand money and their personal and family finances. We describe the practices we developed for exploring the sensitive topic of money, and then discuss three sets of findings. T...
Article
Full-text available
Photos are becoming prominent means of communication online. Despite photos' pervasive presence in social media and online world, we know little about how people interact and engage with their content. Understanding how photo content might signify engagement, can impact both science and design, influencing production and distribution. One common ty...
Article
Recently, geography's role in human-computer interaction has risen due in part to the popularity of social computing as well as the number of ubiquitous GPS enabled devices. We now, either explicitly or implicitly, track, store capture, and annotate our surroundings constantly through out the day. In turn this changes how one might come to understa...
Article
Full-text available
Traditional cameras and video equipment are gradually losing the race with smart phones and small mobile devices that allow video, photo, and audio capturing on the go. Users are now quickly creating movies and taking photos whenever and wherever they go, particularly at concerts and live events (e.g., shows, sport events). Still, in-situ media cap...
Patent
The disclosed embodiments of computer systems and techniques facilitate creation and retrieval of software code. Provenance tracking at the step level allows for each step to be tracked to its original creator and to various remixes of the original in the line of provenance.
Article
Full-text available
Using the ACM Multimedia 2012 panel on metadata as a jumping-off point, the authors investigate whether content can continue to play a dominant role in multimedia research in the age of social, local, and mobile media. In this article, they propose that the community now must face the challenge of characterizing the level of difficulty of multimedi...
Conference Paper
Multimedia social communication is becoming commonplace. Television is becoming smart and social; media sharing applications are transforming the way we converse and recall events and videoconferencing is a common application on our computers, phones, tablets and even televisions. The confluence of computer-mediated interaction, social networking,...
Patent
Full-text available
Embodiments are directed towards employing a classifier to determine a classification for target media content using nominally factored social interaction attributes, the classifier being trained using a training dataset that includes at least one nominally factored social interaction attribute. The trained classifier determines a classification of...
Conference Paper
This demonstration investigates data visualization as a performance through the use of disc jockey (DJs) mixing boards. We assert that the tools DJs use in-situ can deeply inform the creation of data mixing interfaces and performances. We present a prototype system, DMix, which allows one to filter and summarize information from social streams usin...
Patent
Full-text available
A system and method for using topic messages to understand media relating to an event. An event (such as a live event or broadcast event), is selected where at least a portion of the content of the event is stored as at least one media object, such as, for example a video or audio recording in digital format. A plurality of topic messages relating...
Chapter
Content-based video retrieval has been a very efficient technique with new video content, but it has not regarded the increasingly dynamic interactions between users and content. We present a comprehensive survey on user-based techniques and instrumentation for social video retrieval researchers. Community-based approaches suggest there is much to...
Conference Paper
Full-text available
Semantic video research has employed crowdsourcing techniques on social web video data sets such as comments, tags, and annotations, but these data sets require an extra effort on behalf of the user. We propose a pulse modeling method, which analyzes implicit user interactions within web video, such as rewind. In particular, we have modeled the use...
Conference Paper
Full-text available
We review the history and trends of multimedia technologies, especially focusing on audio & video products. Our lifestyles have been changing according to the developments of these technologies. We forecast the direction of multimedia technologies ...
Conference Paper
Multimedia social communication is filtering into everyday use. Videoconferencing is appearing in the living room and beyond, television is becoming smart and social, and media sharing applications are transforming the way we converse and recall events. The confluence of computer-mediated interaction, social networking, and multimedia content are r...
Article
Full-text available
What started as a field with an emphasis on optimally serving users' interactive information needs has now become dominated by methods that focus on improving the mean average precision (MAP) of a clearly defined task disconnected from its application. With the pervasiveness of the Internet and all the sensors available to derive contextual user in...
Article
In this article, we describe Device Communication: a protocol and architecture to enable new TV viewing experiences driven by a sophisticated platform between the TV and modern tablets, mobile phones, and laptops. Beyond simple keyboard and remote control navigation, the device connectivity platform provides a channel for two-way, application-speci...
Article
One of the most important aspects of multimedia research is mining data about user behavior. In a world with one or fewer clicks per piece of content, one needs a lot of users to get a meaningful signal. A monetization ecosystem now directs multimedia research and is changing how we think about people, interaction, media engagement, and data instru...
Article
New multimedia applications, such as community-created video repositories and tools for synchronous sharing, have revolutionized the ways that media is watched and shared. Effective instrumentation of these applications can enable researchers and system designers to better understand how video is being consumed: that is, how it is being watched, sh...
Article
The rebirth of mobile application stores has energized the markets of mobile ecology in both commercial and research communities alike. However, the effort involved in public application deployment via a commercial store is a trade-off of design, features, and functionality; free applications are no exception. This article discusses some features,...
Conference Paper
Full-text available
With the proliferation of app stores and the advancement of mobile devices, research that might have only been tested with a dozen participants in the past can now be released to millions. This offers huge opportunities, but also requires adaptations of existing methods in dealing with large deployments and making sense of large data sets. This wor...
Conference Paper
Full-text available
Categorization of online videos is often treated as a tag suggestion task; tags can be generated by individuals or by machine classification. In this paper, we suggest categorization can be determined socially, based on people's interactions around media content without recourse to metadata that are intrinsic to the media object itself. This work b...
Article
Full-text available
In this paper, we present a method for video category classification using only social metadata from websites like YouTube. In place of content analysis, we utilize communicative and social contexts surrounding videos as a means to determine a categorical genre, e.g. Comedy, Music. We hypothesize that video clips belonging to different genre catego...
Conference Paper
Full-text available
A microblogged stream is delivered over time, providing an ongoing commentary of topics, trends, and issues. In this article, we present two methods of finding temporal topics within these Twitter streams. Using a normalized term frequency, we demonstrate how an effective table of contents can be extracted by finding localized "peaky topics". Secon...