Content uploaded by Walid Taha
Author content
All content in this area was uploaded by Walid Taha on May 31, 2014
Content may be subject to copyright.
A preview of the PDF is not available
Multi-Stage Programming: Its Theory and Applications
Abstract
MetaML is a statically typed functional programming language with special support for program generation. In addition to providing the standard features of contemporary programming languages such as Standard ML, MetaML provides three staging annotations. These staging annotations allow the construction, combination, and execution of object-programs. Our thesis is that MetaML's three staging annotations provide a useful, theoretically sound basis for building program generators. This dissertation reports on our study of MetaML's staging constructs, their use, their implementation, and their formal semantics. Our results include an extended example of where MetaML allows us to produce efficient programs, an explanation of why implementing these constructs in traditional ways can be challenging, two formulations of MetaML's semantics, a type system for MetaML, and a proposal for extendin...
... For each character of the input, the algorithm computes derivatives and checks emptiness and nullability for sets of regexes. However, since the regexes and other information about the grammar are known in advance of parsing, the inefficient algorithm can be staged [Taha 1999] to produce an efficient algorithm. The idea of staging is to identify those parts of the algorithm that do depend only on static information -i.e. on the grammar -and execute them first, leaving only the parts that depend on dynamic information -i.e. on the input string -for later. ...
Lexers and parsers are typically defined separately and connected by a token stream. This separate definition is important for modularity and reduces the potential for parsing ambiguity. However, materializing tokens as data structures and case-switching on tokens comes with a cost. We show how to fuse separately-defined lexers and parsers, drastically improving performance without compromising modularity or increasing ambiguity. We propose a deterministic variant of Greibach Normal Form that ensures deterministic parsing with a single token of lookahead and makes fusion strikingly simple, and prove that normalizing context free expressions into the deterministic normal form is semantics-preserving. Our staged parser combinator library, flap, provides a standard interface, but generates specialized token-free code that runs two to six times faster than ocamlyacc on a range of benchmarks.
... In order to write programs that behave similarly to MetaML programs with escape expressions that occur under a binder inside staged code, we also need push. In Figure 6, the MetaML program on the left (due to Taha [24]) contains two escape expressions that both occur under function binders inside staged code. The result of running the program is a value that is alpha-equivalent to the code value <fn x1 => (fn x2 => x2 + x1)>. ...
The development of programming languages can be quite complicated and costly. Hence, much effort has been devoted to the modular definition of language features that can be reused in various combinations to define new languages and experiment with their semantics. A notable outcome of these efforts is the algebra-based "datatypes a la carte" (DTC) approach. When combined with algebraic effects, DTC can model a wide range of common language features. Unfortunately, the current state of the art does not cover modular definitions of advanced control-flow mechanisms that defer execution to an appropriate point, such as call-by-name and call-by-need evaluation, as well as (multi-)staging. This paper defines latent effects, a generic class of such control-flow mechanisms. We demonstrate how function abstractions, lazy computations and a MetaML-like staging can all be expressed in a modular fashion using latent effects, and how they can be combined in various ways to obtain complex semantics. We provide a full Haskell implementation of our effects and handlers with a range of examples.
... focussed on how to achieve this task in a portable fashion [44]. • The programming languages community uses the term multi stage programming to refer to the use of code generators as a language design tool [110]. ...
Numerical simulation with partial differential equations is an important discipline in high performance computing. Notable application areas include geosciences, fluid dynamics, solid mechanics and electromagnetics. Recent hardware developments have made it increasingly hard to achieve very good performance. This is both due to a lack of numerical algorithms suited for the hardware and efficient implementations of these algorithms not being available. Modern CPUs require a sufficiently high arithmetic intensity in order to unfold their full potential. In this thesis, we use a numerical scheme that is well-suited for this scenario: The Discontinuous Galerkin Finite Element Method on cuboid meshes can be implemented with optimal complexity exploiting the tensor product structure of basis functions and quadrature formulae using a technique called sum factorization. A matrix-free implementation of this scheme significantly lowers the memory footprint of the method and delivers a fully compute-bound algorithm. An efficient implementation of this scheme for a modern CPU requires maximum use of the processor’s SIMD units. General purpose compilers are not capable of autovectorizing traditional PDE simulation codes, requiring high performance implementations to explicitly spell out SIMD instructions. With the SIMD width increasing in the last years (reaching its current peak at 512 bits in the Intel Skylake architecture) and programming languages not providing tools to directly target SIMD units, such code suffers from a performance portability issue. This work proposes generative programming as a solution to this issue. To this end, we develop a toolchain that translates a PDE problem expressed in a domain specific language into a piece of machine-dependent, optimized C++ code. This toolchain is embedded into the existing user workflow of the DUNE project, an open source framework for the numerical solution of PDEs. Compared to other such toolchains, special emphasis is put on an intermediate representation that enables performance-oriented transformations. Furthermore, this thesis defines a new class of SIMD vectorization strategies that operate on batches of subkernels within one integration kernel. The space of these vectorization strategies is explored systematically from within the code generator in an autotuning procedure. We demonstrate the performance of our vectorization strategies and their implementation by providing measurements on the Intel Haswell and Intel Skylake architectures. We present numbers for the diffusion-reaction equation, the Stokes equations and Maxwell’s equations, achieving up to 40% of the machine’s theoretical floating point performance for an application of the DG operator.
... Even though this technique is inspired by advancements in partial evaluation [26] it proved useful to have it in a programming language with rst-class support. Part of the power of this programming model, comes from a regulation mechanism that attributes levels to terms [37]; these systems are type-safe in a modular way (type checking the generator ensures the validity of the generated code). Nowadays, gaining inspiration from MetaML and MetaOCaml, many programming languages provide support for similar mechanisms such as F#, Haskell (Template Haskell [34] and later Typed Template Haskell [15]), Converge [43] and others. ...
Program generation is indispensable. We propose a novel unification of two existing metaprogramming techniques: multi-stage programming and hygienic generative macros. The former supports runtime code generation and execution in a type-safe manner while the latter offers compile-time code generation.
In this work we draw upon a long line of research on metaprogramming, starting with Lisp, MetaML and MetaOCaml. We provide direct support for quotes, splices and top-level splices, all regulated uniformly by a level-counting Phase Consistency Principle. Our design enables the construction and combination of code values for both expressions and types. Moreover, code generation can happen either at runtime à la MetaML or at compile time, in a macro fashion, à la MacroML.
We provide an implementation of our design in Scala and we present two case studies. The first implements the Hidden Markov Model, Shonan Challenge for HPC. The second implements the staged streaming library Strymonas.
... Fragments and injections are not novel to this thesis. They draw inspiration from the very rich history of quotations for staging and macros [Bawden, 1999, Taha, 1999, Sheard and Peyton Jones, 2002 and have been used for several years in various tierless Web programming languages such as Hop [Serrano et al., 2006] and Eliom. These constructions are quite powerful and allow to express complex client-server interactions in convenient and safe ways, as we see in the rest of this section. ...
Eliom is a dialect of OCaml for Web programming in which server and client pieces of code can be mixed in the same file using syntactic annotations. This allows to build a whole application as a single distributed program, in which it is possible to define in a composable way reusable widgets with both server and client behaviors. Eliom is type-safe, as it ensures that communications are well-behaved through novel language constructs that match the specificity of Web programming. Eliom is also efficient, it provides static slicing which separates client and server parts at compile time and avoids back-and-forth communications between the client and the server. Finally, Eliom supports modularity and encapsulation thanks to an extension of the OCaml module system featuring tierless annotations that specify whether some definitions should be on the server, on the client, or both. This thesis presents the design, the formalization and the implementation of the Eliom language.
Lexers and parsers are typically defined separately and connected by a token stream. This separate definition is important for modularity and reduces the potential for parsing ambiguity. However, materializing tokens as data structures and case-switching on tokens comes with a cost.
We show how to fuse separately-defined lexers and parsers, drastically improving performance without compromising modularity or increasing ambiguity. We propose a deterministic variant of Greibach Normal Form that ensures deterministic parsing with a single token of lookahead and makes fusion strikingly simple, and prove that normalizing context free expressions into the deterministic normal form is semantics-preserving. Our staged parser combinator library, flap, provides a standard interface, but generates specialized token-free code that runs two to six times faster than ocamlyacc on a range of benchmarks.
Program generation is indispensable. We propose a novel unification of two existing metaprogramming techniques: multi-stage programming and hygienic generative macros. The former supports runtime code generation and execution in a type-safe manner while the latter offers compile-time code generation.
In this work we draw upon a long line of research on metaprogramming, starting with Lisp, MetaML and MetaOCaml. We provide direct support for quotes, splices and top-level splices, all regulated uniformly by a level-counting Phase Consistency Principle. Our design enables the construction and combination of code values for both expressions and types. Moreover, code generation can happen either at runtime à la MetaML or at compile time, in a macro fashion, à la MacroML.
We provide an implementation of our design in Scala and we present two case studies. The first implements the Hidden Markov Model, Shonan Challenge for HPC. The second implements the staged streaming library Strymonas.
In this chapter, we first motivate the needs of automation, focusing on three dimensions, i.e. the growth of diversity, complexity and software content in designing educational systems. Next, knowing this context, we introduce the theoretical basis to implement the automation in STEM-driven CS education. This basis includes two parts. The first is about feature-based analysis, modelling and feature model transformations typically applied to the problem domain. All these we borrowed from software engineering. The second part includes the basics of heterogeneous meta-programming techniques. We extend those techniques by introducing new types of heterogeneous meta-programs that represent our efforts to move from the component-level meta-programming to the system-level to design such systems as the generative scenario creator for STEM. However, this approach requires a more extensive research, though we implemented and tested the approach practically. A more thorough analysis of the system-level meta-programming is beyond the scope of this book.
In Chap. 8, we deal with a specific model of the smart content, i.e. the stage-based (SB) generative (smart) learning object (GLO/SLO) model. This model supports the automated adaptation of the educational content. We aim at showing how it is possible to integrate the STEM-driven CS education concepts into the SB GLO model. The model is a derivative from the initial GLO/SLO specification. Two dependent concepts, i.e. staging and context awareness, are basic to understand this model. The first means refactoring the structure of the original GLO specification into the stage-based structure without the loss of functionality. Semantically staging means the rearrangement of parameter space of an original GLO among stages so that it would be possible at the stage k (k > 1) to produce a meta-meta-program with (k–1) stages and finally at stage 1 to produce the target program. The user, making the context-dependent selection of parameter values at each stage, enables the system to narrow the parameter space according to his/her context and, in this way, to adapt gradually the generated content. We analyse a case study, the stage-based adaptation processes and learning scenarios. Finally, we discuss the capabilities of this methodology and present an overall evaluation with the focus on pedagogical aspects.
The Oregon Graduate Institute In this extended abstract, we outline some essential elements of a conceptual model for a component generation system. This model is based on an extensive study of a large number of high-level program generation systems, and the significant body of related literature. We focus our attention on the architectural elements of this model, and briefly discuss the technological and process elements. We show how the model is a useful basis for comparing component generation technologies. With a rapidly growing area like component generation, it is hard to get a truly representative sample of generators. As a workaround, we illustrate our model using seven significant component generation systems developed by various research groups, and discuss some insights that the model provides. We conclude with an overview of the current status of our investigation.
MetaML is a multi-stage functional programming language featuring three constructs that can be viewed as statically-typed reene-ments of the back-quote, comma, and eval of Scheme. Thus it provides special support for writing code generators and serves as a semantically-sound basis for systems involving multiple interdependent computational stages. In previous work, we reported on an implementation of MetaML, and on a small-step semantics and type-system for MetaML. In this pa-per, we present An Idealized MetaML (AIM) that is the result of our study of a categorical model for MetaML. An important outstanding problem is nding a type system that provides the user with a means for manipulating both open and closed code. This problem has eluded eeorts by us and other researchers for over three years. AIM solves the issue by providing two type constructors, one classiies closed code and the other open code, and exploiting the way they interact.
Program specialization can divide a computation into several computation,stages. This paper investigates the theoretical limitations and practical problems of standard specialization tools, presents multi-level specialization, and demonstrates that, in combination with the cogen approach, it is far more practical than previously supposed. The program,generator which we designed and implemented,for a higher-order functional language converts programs into very compact,multi-level generating extensions that guarantee fast successive specialization. Experimental results show a remarkable reduction of generation time and generator size compared to previous attempts of multi-level specialization by self-application. Our approach to multi-level specialization seems well-suited for applications where generation time and program,size are critical. Keywords: programming languages, program transformation, partial evaluation, generating extensions, binding-
