Sanjay Ghemawat's research while affiliated with Google Inc. and other places

Publications (53)

Preprint
Full-text available
Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number of task-specific training examples needed to adapt the model to a particular application. To further our understanding of the impact of scale on few-shot learning, we traine...
Preprint
Full-text available
We present the design of a new large scale orchestration layer for accelerators. Our system, Pathways, is explicitly designed to enable exploration of new systems and ML research ideas, while retaining state of the art performance for current models. Pathways uses a sharded dataflow graph of asynchronous operators that consume and produce futures,...
Conference Paper
Many recent machine learning models rely on fine-grained dynamic control flow for training and inference. In particular, models based on recurrent neural networks and on reinforcement learning depend on recurrence relations, data-dependent conditional execution, and other features that call for dynamic control flow. These applications benefit from...
Article
Full-text available
TensorFlow is a machine learning system that operates at large scale and in heterogeneous environments. TensorFlow uses dataflow graphs to represent computation, shared state, and the operations that mutate that state. It maps the nodes of a dataflow graph across many machines in a cluster, and within a machine across multiple computational devices...
Article
Full-text available
TensorFlow is an interface for expressing machine learning algorithms, and an implementation for executing such algorithms. A computation expressed using TensorFlow can be executed with little or no change on a wide variety of heterogeneous systems, ranging from mobile devices such as phones and tablets up to large-scale distributed systems of hund...
Patent
Methods for organizing and retrieving data values in a persistent data structure are provided. Data values are grouped into data blocks and pointers are obtained for each data block. In addition, one or more summaries, related to a properties of the data block, are created and associated with the data block's pointer. The summaries allow for a more...
Patent
A system may present information regarding a document and provide an option for removing the document. The system may also receive selection of the option and remove the document when the option is selected. The system may aggregate information regarding documents that have been removed by a group of users and assign scores to a set of documents ba...
Technical Report
TensorFlow [1] is an interface for expressing machine learning algorithms, and an implementation for executing such algorithms. A computation expressed using TensorFlow can be executed with little or no change on a wide variety of heterogeneous systems, ranging from mobile devices such as phones and tablets up to large-scale distributed systems of...
Patent
A system includes: an engaging post identifier for identifying and retrieving engaging posts; an extended network post identifier for identifying extended posts from an extended network; a combining module for creating a combined list of added posts from the engaging post and the extended posts, the combining module generating one or more ranked po...
Patent
Techniques for crawling hyperlinked documents are provided. Hyperlinked documents to be crawled are grouped by host and the host to be crawled next is selected according to a stall time of the host. The stall time can indicate the earliest time that the host should be crawled and the stall times can be a predetermined amount of time, vary by host a...
Patent
Upon receipt of a document request, a client assistant examines its cache for the document. If not successful, a server searches for the requested document in its cache. If the server copy is still not fresh or not found, the server seeks the document from its host. If the host cannot provide the copy, the server seeks it from a document repository...
Patent
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, relating to language models stored for digital language processing. In one aspect, a method includes the actions of generating a language model, including: receiving a collection of n-grams from a corpus, each n-gram of the collection having a corresp...
Patent
Full-text available
A search engine for searching a corpus improves the relevancy of the results by classifying multiple terms in a search query as a single semantic unit. A semantic unit locator of the search engine generates a subset of documents that are generally relevant to the query based on the individual terms within the query. Combinations of search terms tha...
Patent
Methods and apparatus consistent with the invention allow a user to submit an ambiguous search query and to receive relevant search results. Queries can be expressed using character sets and/or languages that are different from the character set and/or language of at least some of the data that is to be searched. A translation between these charact...
Patent
A computer system comprising one or more processors and memory groups a set of documents into a plurality of clusters. Each cluster includes one or more documents of the set of documents and a respective cluster of documents of the plurality of clusters includes respective cluster data corresponding to a plurality of documents including a first doc...
Article
Full-text available
Spanner is Google’s scalable, multiversion, globally distributed, and synchronously replicated database. It is the first system to distribute data at global scale and support externally-consistent distributed transactions. This article describes how Spanner is structured, its feature set, the rationale underlying various design decisions, and a nov...
Article
Spanner is Google’s scalable, multiversion, globally distributed, and synchronously replicated database. It is the first system to distribute data at global scale and support externally-consistent distributed transactions. This article describes how Spanner is structured, its feature set, the rationale underlying various design decisions, and a nov...
Patent
Web quotes are gathered from web pages that link to a web page of interest. The web quote may include text from the paragraphs that contain the hypertext links to the page of interest as well as text from other portions of the linked web page, such as text from a nearby header. The obtained web quotes may be ranked based on quality or relevance and...
Patent
Full-text available
Provided is a method and system for indexing documents in a collection of linked documents. A link log, including one or more pairings of source documents and target documents is accessed. A sorted anchor map, containing one or more target document to source document pairings, is generated. The pairings in the sorted anchor map are ordered based on...
Conference Paper
Full-text available
Spanner is Google's scalable, multi-version, globally-distributed, and synchronously-replicated database. It is the first system to distribute data at global scale and support externally-consistent distributed transactions. This paper describes how Spanner is structured, its feature set, the rationale underlying various design decisions, and a nove...
Article
MapReduce (MR) has emerged as a flexible data processing tool for different applications. It has emerged as a programming model for processing and generating large data sets. Users specify a map function that processes a key or value pair to generate a set of intermediate key or value pairs and a reduce function that merges all intermediate values...
Conference Paper
With the availability of large amounts of training data relevant to speech recognition scenarios, scalability becomes a very productive way to improve language model performance. We present a technique that represents a back-off n-gram language model using arrays of integer values and thus renders it amenable to effective block compression. We prop...
Article
Bigtable is a distributed storage system for managing structured data that is designed to scale to a very large size: petabytes of data across thousands of commodity servers. Many projects at Google store data in Bigtable, including web indexing, Google Earth, and Google Finance. These applications place very different demands on Bigtable, both in...
Conference Paper
MapReduce is a programming model and an associated implementation for processing and generating large datasets that is amenable to a broad variety of real-world tasks. Users specify the computation in terms of a map and a reduce function, and the underlying runtime system automatically parallelizes the computation across large-scale clusters of mac...
Conference Paper
We have designed and implemented the Google File Sys- tem, a scalable distributed file system for large distributed data-intensive applications. It provides fault tolerance while running on inexpensive commodity hardware, and it delivers high aggregate performance to a large number of clients. While sharing many of the same goals as previous dis- t...
Article
This paper describes the DIGITAL Continuous Profiling Infrastructure, a sampling-based profiling system designed to run continuously on production systems. The system supports multiprocessors, works on unmodified executables, and collects profiles for entire systems, including user programs, shared libraries, and the operating system kernel. Sample...
Article
Full-text available
technique [1, 26, 27]. In this method, client calls are directed to a single primary server, which communicates This paper describes the design and implementation of the with other backup servers and waits for them to respond Harp file system. Harp is a replicated Unix file system before replying to the client. The system masks failures by accessib...
Article
Full-text available
To provide high availability for services such as mail or bulletin boards, data must be replicated. One way to guarantee consistency of replicated data is to force service operations to occur in the same order at all sites, but this approach is expensive. In this paper, we propose lazy replication as a way to preserve consistency by exploiting the...
Article
To provide high availability for services such as mail or bulletin boards, data must be replicated. One way to guarantee consistency of replicated data is to force service operations to occur in the same order at all sites, but this approach is expensive. In this paper, we propose lazy replication as a way to preserve consistency by exploiting the...
Article
ProfileMe, which can provide significant insight into the behavior of programs running on complex microprocessors (especially out-of-order processors) [2]. ProfileMe requires only modest hardware modifications and can be used by our DCPI tools in a way that collects detailed profile information without substantial profiling overhead. The DCPI profi...
Article
Replicating various components of a system is a common technique for providing highly available services in the presence of failures. A replication scheme is a mechanism for organizing these replicas so that as a group they provide a service that has the same semantics as the original unreplicated service. Viewstamped replication is a new replicati...
Article
Full-text available
To provide high availability for services such as mail or bulletin boards, data must be replicated. One way to guarantee consistency of replicated data is to force service operations to occur in the same order at all sites, but this approach is expensive. For some applications a weaker causal operation order can preserve consistency while providing...
Conference Paper
This article describes the Digital Continuous Profiling Infrastructure, a sampling-based profiling system designed to run continuously on production systems. The system supports multiprocessors, works on unmodified executables, and collects profiles for entire systems, including user programs, shared libraries, and the operating system kernel. Samp...
Article
This paper describes the DIGITAL Continuous Profiling Infrastructure, a sampling-based profiling system designed to run continuously on production systems. The system supports multiprocessors, works on unmodified executables, and collects profiles for entire systems, including user programs, shared libraries, and the operating system kernel. Sample...
Article
This paper proposes efficient disk management techniques for such databases.
Conference Paper
Full-text available
Thor is an object-oriented database system designed for use in a heterogeneous distributed environment. It provides highly-reliable and highly-available persistent storage for objects, and supports safe sharing of these objects by applications written in different programming languages.Safe heterogeneous sharing of long-lived objects requires encap...
Article
Full-text available
This document provides an introduction to using Thor, an object oriented database, and its language Theta. This document applies to Thor Release 0.1. Topics include installation, writing Theta code to implement objects, writing C++ application code, and debugging.
Article
Object-oriented databases store many small objects on disks. Disks perform poorly when reading and writing individual small objects. This thesis presents a new storage management architecture that substantially improves disk performance of a distributed object-oriented database system. The storage architecture is built around a large modified objec...
Article
This document describes a new programming language called Theta. Theta is a sequential, strongly-typed, object-oriented language. It provides separate mechanisms for type hierarchy, inheritance, and parametric polymorphism. It also provides separate mechanisms for specifications, which define the interfaces of new abstractions, and code that implem...
Article
this paper I propose three disk management strategies for object-oriented databases. These strategies are based on earlier work on file-systems. They differ from this earlier work in their support for a large number of small objects and non-sequential access patterns.
Article
Full-text available
Thor is a new object-oriented database system being developed at MIT. It allows applications written in different programming languages, and possibly running on heterogeneous machines and operating systems, to share objects conveniently. Our goal is to provide safe sharing of objects with higher-level semantics than is typical for today's file syst...
Article
Full-text available
We propose several new techniques for resource management in a replicated object server. By coordinating cache and disk usage among the replicas, these techniques increase throughput and reduce fetch latency. Cache splitting speeds up fetches by avoiding redundant cache entries, effectively increasing the cache size. Coordinated writing coordinates...
Conference Paper
An object-oriented database provides persistent storage for a large number of objects. These objects may be very small, and the access patterns are likely to be not as uniform as the mostly sequential reads and writes seen in file-systems. For example, the 007 benchmark for object-oriented databases specifies a number of traversals that follow poin...
Article
Full-text available
Introduction Harp is a replicated Unix file system accessible via the VFS interface. It provides highly available and reliable storage for files and guarantees that file operations are executed atomically in spite of concurrency and failures. Replication enables Harp to safely trade disk accesses for network communication and thus to provide good p...
Conference Paper
Harp is a replicated Unix file system accessible via the VFS interface. It provides highly available and reliable storage for files and guarantees that file operations are executed atomically in spite of concurrency and failures. Replication enables Harp to safely trade disk accesses for network communication and thus to provide good performance bo...

Citations

... Automatic differentiation frameworks that have statistical inference packages built on top of them include Theano [5], Pytorch [44], JAX [10] and TensorFlow [1]. Some of these extend to non-trivial computational constructs such as control flow and recursion [63]. ...
... The learning rate is set to 0.3 with momentum 0.8. The training work is done with TensorFlow [50]. After 250 epochs, the model is converged. ...
... Like in MapReduce [12], Map() functions in MARbLE structure input data. AA get their input data from sensor containers. ...
... To show why this process can be well parallelized, we should rewrite the expression for the training loss (Eq. 6) as the sum of contributions from each training point: We note that the central importance of efficient evaluation of derivatives for backpropagation significantly stimulated the development of specialized frameworks implementing automatic differentiation like PyTorch [13] and TensorFlow [14], which do the job so well, that even quantum chemists start to implement quantum chemical methods using these frameworks providing straightforward access to derivatives [15][16]. Derivation of derivatives for quantum chemical methods is a notoriously difficult task for humans, but derivatives are required in many simulations such as geometry optimizations and molecular dynamics, [17] and also for parametrization of semiempirical methods [15][16]. ...
Reference: Neural networks
... Code for these models implemented using TensorFlow 2 [44,45] has been made publicly available for reproducibility of the results 1 . ...
... The results of this study are comparable to those reported previously in the literature [42][43][44], indicating that blockchain systems achieve performances far below what the traditional distributed databases achieve [76,77]. Traditional databases make use of concurrency control, for example, 2-phase locking to ensure atomicity, consistency, isolation, and durability. ...
... Sanjay Ghemawat presents a new storage management architecture that substantially improves disk performance of a distributed object-oriented database system [11]. The storage architecture is built around a large modified object buffer (MOB) that is stored in primary memory. ...
... The impact of profiling in non-deterministic systems: Most of the prediction models are forecast based on historical knowledge for short-term requests (Mallick et al., 2012). The historical knowledge token from the monitoring service, which logs information continuously as a profile in a searchable database (Anderson et al., 1997) and the whole process showed in Fig. 3. The proper analysis tool dissects the stored profile information at several levels. ...
... Edge servers communicate mainly to establish agreement on the order of transactions using fault-tolerant protocols. While large-scale data management systems such as Google's Spanner [14], Amazon's Dynamo [16], and Facebook's Tao [11] rely on crash faulttolerant (CFT) protocols, e.g., Paxos [27], consensus among edge servers requires Byzantine fault-tolerant (BFT) protocols, e.g., PBFT [13] due to the non-trustworthiness of edge infrastructures. ...
... An important class of fault-tolerance protocols is state machine replication (SMR). SMR protocols [36,39,44,48,55] have long been deployed in production systems [18,20,24,32] to ensure a set of distributed nodes behaves like a single, always available state machine, despite failures of individual machines. These protocols, however, can only tolerate node crash failures. ...