Micah Beck

University of Tennessee at Knoxville | UTK · Department of Electrical Engineering and Computer Science

In this paper we reexamine an assumption that underpinned the development of the Internet architecture, namely that a stateless and loosely synchronous point-to-point datagram delivery service would be sufficient to meet the needs of all network applications, including those which deliver content and services to a mass audience at global scale. Suc...

At the end of the 19th century the logician C.S. Peirce coined the term "fallibilism" for the "... the doctrine that our knowledge is never absolute but always swims, as it were, in a continuum of uncertainty and of indeterminacy". In terms of scientific practice, this means we are obliged to reexamine the assumptions, the evidence, and the argumen...

How to decompose a vertically integrated digital monopoly to enable competitive services based on a shared data structure.

The Internet stack is not a complete description of the resources and services needed to implement distributed applications, as it only accounts for communication services and the protocols that are defined to deliver them. This paper presents an account of the current distributed application architecture using a formal model of strictly layered sy...

Broadband Internet service is widely expected to be the fundamental universal service for the 21st century. But more than a decade of national and international struggles to close the digital divide between broadband haves and have nots suggest that reaching global universality will be a very difficult task. This paper argues that the strong guaran...

We introduce a unary coding of bosonic occupation states based on the famous balls and walls counting for the number of configurations of N indistinguishable particles on L distinguishable sites. Each state is represented by an integer with a human readable bit string that has a compositional structure allowing for the efficient application of oper...

We introduce a unary coding of bosonic occupation states based on the famous "balls and walls" counting for the number of configurations of $N$ indistinguishable particles on $L$ distinguishable sites. Each state is represented by an integer with a human readable bit string that has a compositional structure allowing for the efficient application o...

In addressing the universal deployment of digital services it is necessary to decide on the breadth of "universality" and the type of functionality of "digital services". These two decisions are linked: certain desirable functionality does not lend itself to being implemented in some environments or with certain constraints. In this paper we define...

Scientific communities naturally tend to organize around data ecosystems created by the combination of their observational devices, their data repositories, and the workflows essential to carry their research from observation to discovery. However, these legacy data ecosystems are now breaking down under the pressure of the exponential growth in th...

Exposed Buffer Architecture addresses network ossification by confronting the virtualization that is inherent to the Internet architecture. In the Internet stack below the Network Layer, the Link layer models services that are local to network nodes, or that connect them in local area networks. Aggregating these low level resources in the implement...

This chapter outlines a vision for how best to harness the computing continuum of interconnected sensors, actuators, instruments, and computing systems, from small numbers of very large devices to large numbers of very small devices. The hypothesis is that only via a continuum perspective one can intentionally specify desired continuum actions and...

In every form of digital store-and-forward communication, intermediate forwarding nodes are computers, with attendant memory and processing resources. This has inevitably stimulated efforts to create a wide-area infrastructure that goes beyond simple store-and-forward to create a platform that makes more general and varied use of the potential of t...

We live in an age of highly distributed collaboration in nearly every field of science coupled with widely disparate technological capabilities. Such heterogeneity makes the movement and management of data that supports scientific and end-user communities a global challenge, particularly when placed in the context of under-served populations withou...

Over the past four years, the Big Data and Exascale Computing (BDEC) project organized a series of five international workshops that aimed to explore the ways in which the new forms of data-centric discovery introduced by the ongoing revolution in high-end data analysis (HDA) might be integrated with the established, simulation-centric paradigm of...

Although the “big data” revolution first came to public prominence (circa 2010) in online enterprises like Google, Amazon, and Facebook, it is now widely recognized as the initial phase of a watershed transformation that modern society generally—and scientific and engineering research in particular—are in the process of undergoing. Responding to th...

In every form of digital store-and-forward communication, intermediate forwarding nodes are computers, with attendant memory and processing resources. This has inevitably given rise to efforts to create a wide area infrastructure that goes beyond simple store and forward, a facility that makes more general and varied use of the potential of this co...

The hourglass model is a widely used as a means of describing the design of the Internet, and can be found in the introduction of many modern textbooks. It arguably also applies to the design of other successful spanning layers, notably the Unix operating system kernel interface, meaning the primitive system calls and the interactions between user...

The hourglass model is a widely used as a means of describing the design of the Internet, and can be found in the introduction of many modern textbooks. It arguably also applies to the design of other successful spanning layers, notably the Unix operating system kernel interface, meaning the primitive system calls and the interactions between user...

The hourglass model is a widely used as a means of describing the design of the Internet, and can be found in the introduction of many modern textbooks. It arguably also applies to the design of other successful spanning layers, notably the Unix operating system kernel interface, meaning the primitive system calls and the interactions between user...

The primary mechanism for overcoming faults in modern storage systems is to introduce redundancy in the form of replication and/or error correcting codes. The costs of such redundancy in hardware, system availability and overall complexity can be substantial, depending on the number and pattern of faults that are handled. In this paper, we describe...

The emergence of high-resolution simulation, where simulation outputs have grown to terascale levels and beyond, raises major new challenges for the visualization community, which is serving computational scientists who want adequate visualization services provided to them on-demand. Many existing algorithms for parallel visualization were not desi...

This project focused on the use of Logistical Networking technology to address the challenges involved in rapid sharing of data from the the Center's gyrokinetic particle simulations, which can be on the order of terabytes per time step, among researchers at a number of geographically distributed locations. There is a great need to manage data on t...

It is often desirable or necessary to perform scientific visualization in geographically remote locations, away from the centralized data storage systems that hold massive amounts of scientific results. The larger such scientific datasets are, the less practical it is to move these datasets to remote locations for collaborators. In such scenarios,...

As the flood of data associated with leading edge computational science continues to escalate, the challenge of supporting the distributed collaborations that are now characteristic of it becomes increasingly daunting. The chief obstacles to progress on this front lie less in the synchronous elements of collaboration, which have been reasonably wel...

I. Introduction (Z. Lin, G. Y. Fu, J. Q. Dong) II. Role of theory and simulation in fusion sciences 1. The Impact of theory and simulation on tokomak experiments (H. R. Wilson, T.S. Hahm and F. Zonca) 2. Tokomak Transport Physics for the Era of ITER: Issues for Simulations (P.H. Diamond and T.S. Hahm) III. Status of fusion simulation and modeling 1...

To use heterogeneous and geographically distributed resources as a platform for parallel visualization is an intriguing topic of research. This is because of the immense potential impact of the work, and also because of its use of a full range of challenging technologies. In this work, we designed an execution environment for visualization of massi...

Visualization is a research tool that computational scientists use for qualitative exploration, hypothesis verification, and result presentation. Driven by needs for large user groups to collaborate across geographical distances, visualization must now also serve as an effective means to share concrete data as well as abstract ideas over the Intern...

To effectively share computation resources over the wide area network among a variety of data intensive applications, a scalable computation service needs to be provisioned. The end-to-end principles provide a scalable approach to the architecture of shared services on which these applications depend. We have shown the use of a best-effort network...

We are interested in developing the infrastructural tools that allow a distributed data intensive computing environment to be shared by a group of collaborating but geographically separated researchers in an interactive manner, as opposed to a batch mode of operation. However, without advanced reservation, it is difficult to assure a certain level...

Scientific applications often need to access remotely located files, but many remote I/O systems lack standard APIs that allow efficient and direct access from application codes. This work presents MPI-IO/L, a remote I/O facility for MPI-IO using logistical networking. This combination not only provides high-performance and direct remote I/O using...

The absence of an adequate distributed storage infrastructure for data buffering has become a significant impediment to the flow of work in the wide area, data intensive collaborations that are increasingly characteristic of leading edge research in several fields. One solution to this problem, pioneered under DOE's SciDAC program, is Logistical Ne...

This paper describes a simple scheduling procedure for use in multicast data distribution within a logistical networking infrastructure. The goal of our scheduler is to generate a distribution schedule that will exploit the best network paths by using historic network performance information. A "spanning tree" is constructed between available logis...

Grid computing is considered as the ultimate solution for the scientific computing community. The main accent was first put on the computational side, as the other factors, such as network and storage, were considered as given. Logistical Networking started to fill the gap in the research of storage aspects of grid, and is now widely accepted and k...

The persistent mood of exhilaration in the research community over exponential increases in the capacity of computational resources has been tempered recently by the realization that a torrential influx of data from instruments, sensors and simulations is growing even faster than the resources needed to analyze it. The impact of this "data deluge",...

While active networks provide new solutions for the deployment of dynamic services in the network, exposing network processing resources, logistical networking focuses on exposing storage resources inside networks, optimizing the global scheduling of data transport, data storage and computation. In this paper, we show how active and logistical envi...

Fusion energy science, like other science areas in DOE, is becoming increasingly data intensive and network distributed. We discuss data management techniques that are essential for scientists making discoveries from their simulations and experiments, with special focus on the techniques and support that Fusion Simulation Project (FSP) scientists m...

We have developed a threaded parallel data streaming approach using logistical networking (LN) to transfer multiterabyte simulation data from computers at NERSC to our local analysis/visualization cluster, as the simulation executes, with negligible overhead. Data transfer experiments show that this concurrent data transfer approach is more favorab...

During the past years it has become evident to the technical community that computational resources cannot keep up with the demands generated by some applications. As an example, particle physics experiments produce more data than can be realistically processed and stored in one location (i.e. several Petabytes/year). In such situations where inten...

While active networks provide new solutions for the deployment of dynamic services in the network by exposing network processing resources, logistical networks focus on exposing storage resources inside networks by optimizing the global scheduling of data transport, and data storage. In this paper, we show how active and logistical environments wor...

This paper addresses the possibility that IP, in the role of the common service, is not as general as is needed in order to directly address application requirements, including scalable storage services, that go beyond simple datagram delivery. We propose to generalize the view of layer 2 to include local storage and processing services, and to sit...

This paper introduces the Logistical Distributed Network (LoDN) tool based upon Logis-tical Networking technology. LoDN aims to solve a common distributed collaboration dilemma faced by end-users in the context of distributing large data sets.

The conventional view within the Internet community is that IP is the appropriate basis for interoperability in the network stack. However, recent developments in IP networking and link layer technologies that cannot be supported by the IP standard have increased the pressure towards creating non-interoperable network domains. In this paper we expl...

More than two decades of exponential growth in the supply of the basic computing resources, rapid progress in the deployment of high performance networks, and escalating requirements for distributed collaboration in Computational Science, has caused expectations of the benefits of distributed systems and grid computing environments to soar [1]. It...

In this paper, we show how pipelining and caching can be used to optimize the performance of the Internet Backplane Protocol, an overlay implementation of the Transnetworking architecture that offers exposed data transfer, storage and processing resources. Our experimental results clearly indicate that this approach can yield high performance in op...

We describe the information security aspects of logistical networking. The security model adopted by logistical networking is an end-to-end model that provides tunable security levels while maintaining the scalability of the network as a whole.

The need to provide remote visualization of large datasets with adequate levels of quality and interactively has become a major impediment to distributed collaboration in Computational Science. Although Image Based Rendering (IBR) techniques based on plenoptic functions have some important advantages over other approaches to this problem, they suff...

The extraordinary success of the Internet as a general-purpose, widely sharable, and globally scalable data com-munication network has set a very high standard for the designers of next generation cyberinfrastructure. Comparable success would be difficult to achieve under any circumstances, but today all visions of major new systems must face inten...

The three fundamental resources underlying Information Technology are bandwidth, storage, and computation. The goal of wide area infrastructure is to provision these resources to enable applications within a community. The end-to-end principles provide a scalable approach to the architecture of the shared services on which these applications depend...

In this chapter, we present two advancements in the development of algorithms and software for the mining of textual information. For large-scale indexing needs, we present the General Text Parser (GTP) software environment with network storage ca-pability. This object-oriented software (C++, Java) is designed to provide information retrieval (IR)...

At iGrid2002, members of the Logistical Computing and Internetworking (LoCI) Lab had two goals. The first was to present an application, Video IBPster, built using the tools of the Network Storage Stack that delivers DVD-quality video without dropping frames, without losing data and without specialized multi-media streaming servers. The Video IBPst...

An exposed approach in computer service architecture is one that offers client software a primitive service whose semantics are closely based on the underlying physical infrastructure. The exposed approach relies on the client to build higher-level services, with more abstract semantics, out of such primitive tools using sophisticated compilation o...

This paper explores three algorithms for high-performance downloads of wide-area, replicated data. The storage model is based on the Network Storage Stack, which allows for flexible sharing and utilization of writable storage as a network resource. The algorithms assume that data is replicated in various storage depots in the wide area, and the dat...

In this work we present the Internet Backplane Protocol (IBP), a middleware created to allow the sharing of storage resources, implemented as part of the network fabric. IBP allows an application to control intermediate data staging operations explicitly. As IBP follows a very simple philosophy, very similar to the Internet Protocol, and the result...

In this paper, we provide an overview of logistical runtime system (LoRS). LoRS is an integrated ensemble of tools and services that aggregate primitive (best effort, faulty) storage allocations to obtain strong properties such as capacity, performance, reliability, that Grid applications desire. The paper focuses on the design and implementation o...

Logistical Networking1 can be defined as the global optimisation and scheduling of data storage, data movement, and computation. It is a technology for shared network storage that allows an easy scaling in terms of the size of the user community, the aggregate quantity of storage that can be allocated, and the distribution breadth of service nodes...

The Logistical Computing and Internetworking (LoCI) project is a reflection of the way that the next generation internetworking fundamentally changes our definition of high performance wide area computing. A key to achieving this aim is the development of middleware that can provide reliable, flexible, scalable, and cost-effective delivery of data...

This paper discusses the application of end-to-end design principles, which are characteristic of the architecture of the Internet, to network storage. While putting storage into the network fabric may seem to contradict end-to-end arguments, we try to show not only that there is no contradiction, but also that adherence to such an approach is the...

The simplicity of the basic client/server model of Web services led quickly to its widespread adoption, but also to scalability and performance problems. The technological response to these problems has been the development of technology for the creation of surrogates for Web servers, starting with simple proxy caches and reverse proxies, and leadi...

This paper addresses the issue of improving performance when using multi-threading in network storage applications. An abstraction of network storage called the Network Storage Stack is detailed along with the software layers (IBP, the L-Bone, exNode, and Logistical Tools) that have been developed to implement it. These layers have been implemented...

Introduction Over the past few years, the Logistical Computing and Iinternetworking (LoCI) project at the University of Tennessee has been demonstrating the power of Logistical Networking in distributed and wide-area settings. Logistical Networking takes the rather unconventional view that storage can be used to augment data transmission as part of...

The dichotomy between the exposed and encapsulated approaches to computer systems architectures is well known in contexts such as the processor design (RISC vs. CISC) and layered network service stacks. In this paper we examine how this basic choice of approaches arises in the design of the Internet Backplane Protocol, a network storage service, an...

this paper we describe the Logistical Networking model of network storage, which we believe will, if successful, affect the synthesis of IP and Storage Networking in a similar way

This paper addresses the issue of fault-tolerance in applications that make use of network storage. A network storage abstraction called the Network Storage Stack is presented, along with its constituent parts. In particular, a data type called the exNode is detailed, along with tools that allow it to be used to implement a wide-area, striped and r...

It is commonly observed that the continued exponential growth in the capacity of fundamental computing resources — processing power, communication bandwidth, and storage — is working a revolution in the capabilities and practices of the research community. It has become increasingly evident that the most revolutionary applications of this superabun...

In this work we present the Internet Backplane Protocol (IBP), a middleware created to allow the sharing of storage resources, implemented as part of the network fabric. IBP allows an application to control intermediate data staging operations explicitly. As IBP follows a very simple philosophy, very similar to the Internet Protocol, and the result...

In this paper we present the Logistical Networking principles and the tools we developed to support our ideas. Logistical Networking is defined as the global scheduling and optimization of data movement, storage and computation based on a model that takes into account al the network's underlying physical resources. We introduce then the Internet Ba...

In this paper we analyze the problem of creating digital library services for "active content" (e.g. adaptive video, interactive visualization) that can scale. We describe how this challenge can be addressed by technologies based on logistical networking, an innovative approach to communication infrastructure that combines bandwidth and storage in...

this paper, we describe the co-design of the storage and processing elements of this fabric that will enable globally scalable and uninterruptible computing applications

Logistical Networking 1 can be defined as the global optimisation and scheduling of data storage, data movement, and computation. It is a technology for shared network storage that allows an easy scaling in terms of the size of the user community, the aggregate quantity of storage that can be allocated, and the distribution breadth of service nodes...

Checkpointing systems are a convenient way for users to make their programs fault-tolerant by intermittently saving program state to disk, and restoring that state following a failure. The main concern with checkpointing is the overhead that it adds to running time of the program. This paper describes memory exclusion, an important class of optimiz...

program analysis to optimize the performance of checkpointing. We achieve this performance gain using libckpt, a checkpointing library which implements memory exclusion in the context of user-directed checkpointing. The correctness of user-directed checkpointing is dependent on program analysis and insertion of memory exclusion calls by the program...

Common opinion holds that a precise definition of the concept of middleware is elusive because it is highly dependent on one's design perspective regarding application environments and system architectures. The approach to the mobile management of network files discussed in this paper, which involves issues of process mobility and architecture/OS i...

The Logistical Computing and Internetworking (LoCI) project is a reflection of the way that the next generation internetworking fundamentally changes our definition of high performance wide area computing. A key to achieving this aim is the development of middleware that can provide reliable, flexible, scalable, and cost-effective delivery of data...

The dichotomy between the exposed and encapsulated approaches to computer systemsarchitectures is well known in contexts such as the processor design (RISC vs. CISC) and layered networkservice stacks. In this paper we examine how this basic choice of approaches arises in the design of theInternet Backplane Protocol, a network storage service, and a...

Logistical Networking is the global scheduling and optimization of data movement, storage and computation,based on a model that takes into account all the network's underlying physical resources. Inthis paper we contrast,Logistical and Active Network as approaches,to flexible implementation,of advanced network protocols. We describe the Internet Ba...

Logistical Networking is the global scheduling and optimization of data movement, storage and computation based on a model that takes into account all the network’s underlying physical resources. In this paper we contrast Logistical and Active Network as approaches to flexible implementation of advanced network protocols. We describe the Internet B...

IBP-Mail is an improvement to the current state of the art in mailing large files over the Internet. It arises from the addition of writable storage to the pool of available Internet resources. With IBP-Mail, a sender registers a large file with an IBP-Mail agent, and stores it into a network storage depot. The sender then mails a pointer to the fi...

For distributed and network applications, efficient management of program state is critical to performance and functionality. To support domain- and application-specific optimization of data movement, we have developed the Internet Backplane Protocol (IBP) for controlling storage that is implemented as part the network fabric itself. IBP allows an...

Computational power grids are computing environments with massive resources for processing and storage. While these resources may be pervasive, harnessing them is a major challenge for the average user. NetSolve is a software environment that addresses this concern. A fundamental feature of NetSolve is its integration of fault-tolerance and task mi...