S. Ethier

Princeton University, Princeton, New Jersey, United States

Are you S. Ethier?

Claim your profile

Publications (98)68.69 Total impact

  • W. Tang, Bei Wang, S. Ethier
    [Show abstract] [Hide abstract]
    ABSTRACT: The primary goal of the extreme scale plasma turbulence studies described in this paper is to gain new insights on confinement scaling in magnetic fusion systems by using powerful, world-class supercomputers to carry out simulations with unprecedented resolution and temporal duration. New insights have been gained on the key question of how the turbulent transport of plasma particles and associated confinement scale from present generation devices to much larger ITER-size plasmas. In particular, new results from large-scale simulation studies have demonstrated that improvement in confinement as devices grow larger takes place far more gradually, and with significantly lower loss rates, than less-powerful computer simulations have indicated in research carried out over the past decade.
    Computing in Science and Engineering 05/2014; · 1.73 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Reliable predictive simulation capability addressing confinement properties in magnetically confined fusion plasmas is critically-important for ITER, a 20 billion dollar international burning plasma device under construction in France. The complex study of kinetic turbulence, which can severely limit the energy confinement and impact the economic viability of fusion systems, requires simulations at extreme scale for such an unprecedented device size. Our newly optimized, global, ab initio particle-in-cell code solving the nonlinear equations underlying gyrokinetic theory achieves excellent performance with respect to \textquotedblleft time to solution at the full capacity of the IBM Blue Gene/Q on 786,432 cores of Mira at ALCF and recently of the 1,572,864 cores of Sequoia at LLNL. Recent multithreading and domain decomposition optimizations in the new GTC-P code represent critically important software advances for modern, low memory per core systems by enabling routine simulations at unprecedented size (130 million grid points ITER-scale) and resolution (65 billion particles).
    Proc. ACM/IEEE Conf. on Supercomputing (SC), Denver, CO; 11/2013
  • [Show abstract] [Hide abstract]
    ABSTRACT: The Gyrokinetic Toroidal Code (GTC) uses the particle-in-cell method to efficiently simulate plasma microturbulence. This work presents novel analysis and optimization techniques to enhance the performance of GTC on large-scale machines. We introduce cell access analysis to better manage locality vs. synchronization tradeoffs on CPU and GPU-based architectures. Our optimized hybrid parallel implementation of GTC uses MPI, OpenMP, and NVIDIA CUDA, achieves up to a 2× speedup over the reference Fortran version on multiple parallel systems, and scales efficiently to tens of thousands of cores.
    International Journal of High Performance Computing Applications 10/2013; 27(4):454-473. · 1.30 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Verification and historical perspective are presented on the gyrokinetic particle simulations that discovered the device size scaling of turbulent transport and indentified the geometry model as the source of the long-standing disagreement between gyrokinetic particle and continuum simulations.
    Plasma Science and Technology 12/2012; 14(12):1125. · 0.51 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: I/O bottlenecks in HPC applications are becoming a more pressing problem as compute capabilities continue to outpace I/O capabilities. While double-precision simulation data often must be stored losslessly, the loss of some of the fractional component may introduce acceptably small errors to many types of scientific analyses. Given this observation, we develop a precision level of detail (APLOD) library, which partitions double-precision datasets along user-defined byte boundaries. APLOD parameterizes the analysis accuracy-I/O performance tradeoff, bounds maximum relative error, maintains I/O access patterns compared to full precision, and operates with low overhead. Using ADIOS as an I/O use-case, we show proportional reduction in disk access time to the degree of precision. Finally, we show the effects of partial precision analysis on accuracy for operations such as k-means and Fourier analysis, finding a strong applicability for the use of varying degrees of precision to reduce the cost of analyzing extreme-scale data.
    Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC12); 11/2012
  • David Perkins, Stephane Ethier, Weixing Wang
    [Show abstract] [Hide abstract]
    ABSTRACT: It is well-known that the level of ion transport in the National Spherical Torus eXperiment (NSTX) is close to the neoclassical level. This makes self-consistent neoclassical simulations carried out with the GTC-NEO particle code highly relevant for studying transport-related issues in NSTX. GTC-NEO, which now treats multiple species of ion impurities [1], takes as input the experimental profiles from NSTX discharges and calculates the fully non-local, self-consistent neoclassical fluxes and radial electric field. One unanswered question related to NSTX plasmas is that of possible ion temperature anisotropy, which cannot be determined experimentally with the current diagnostics. This work describes new numerical diagnostics and computational improvements that were implemented in GTC-NEO to enable the study of temperature anisotropy.[4pt] [1] R.A. Kolesnikov et al., Phy. Plasmas 17, 022506 (2010)
    10/2012;
  • [Show abstract] [Hide abstract]
    ABSTRACT: New results from global nonlinear gyrokinetic simulations with the GTS code show that strong flow shear can drive a negative compressibility mode [1-3] unstable in tokamak geometry in some experimentally relevant parameter regimes. The modes reside in a low-k range, similar to that of ITG mode, with smaller but almost constant growth rate over a wider kθ range, while the mode frequency increases strongly with kθ. More interestingly, the flow shear modes show significantly finite k//, unlike ITG and TEM. The nonlinear energy transfer to longer wavelength via toroidal mode coupling and corresponding strong zonal flow and geodestic acoustic mode (GAM) generation are shown to play a critical role in the nonlinear saturation of the instability. The associated turbulence fluctuations can produce significant momentum and energy transport, including an intrinsic torque in the co-current direction. Remarkably, strong ``resonance'' in the fluctuations and associated transport peaks at the lowest order rational surfaces with integer q-number (rather than fractional), consistent with theoretical calculation. As a consequence, local ``corrugations'' are generated in all plasma profiles (temperatures, density and toroidal rotation), potentially impacting transport barrier formation near the rational surface. Discussions on flow optimization for minimizing plasma transport will be reported.[4pt] [1] N. Mattor and P. H. Diamond, Phys. Fluids 31, 1180 (1988).[0pt] [2] P. J. Catto et al., Phys. Fluids 16, 1719 (1973).[0pt] [3] M. Artun and W. M. Tang, Phys. Fluids B4, 1102 (1992).
    10/2012;
  • [Show abstract] [Hide abstract]
    ABSTRACT: Fully-global, 5D gyrokinetic simulations of turbulent transport in tokamak devices generate a large amount of time-dependant data that contain a wealth of information about waves, particles, and their self-consistent interactions. To explore these data in spectral space, in both wave numbers and frequencies, the information needs to be written out and analyzed in a post-process stage. This work describes the development of a MATLAB-based system for the extensive analysis of gyrokinetic simulation data, with particular application to the Gyrokinetic Tokamak Simulation code (GTS), which is being used for studying experimental discharges from NSTX, DIIID, and C-MOD. Parallel FORTRAN and C routines are used in some cases to read in the large amount of data and carry out the first stage of post-processing. Advanced MATLAB functions are then used for calculating statistical quantities, correlations, etc. A graphical user interface enhances the user experience and provides advanced plotting capabilities. Examples of microturbulence data analyses are given and discussed.
    10/2012;
  • [Show abstract] [Hide abstract]
    ABSTRACT: A new visualization has been developed of the 3D vector field of plasma flow computed by global gyrokinetic simulations using the GTS code. The visualization shows the direction, magnitude, and structure of turbulence-generated intrinsic rotation in a tokamak. Vectors indicate the clockwise and counter-clockwise flows around the torus. Color-coded vectors are drawn at each grid point on the poloidal planes. A color scale was developed to maximize contrast within the most heavily populated range of data while preserving visibility of the global minimum and maximum values. Technical highlights include transferring large amounts of simulation data from NERSC to PPPL using multiple streams, parallel rendering by the VisIt software, and multiple nx client sessions connecting to a persistent server session. Each of the 1,000 time steps is rendered to a high definition image. The images are assembled into an animated movie that is compressed for efficient, high quality playback. A workflow is in place for producing visualizations of new simulations.
    10/2012;
  • [Show abstract] [Hide abstract]
    ABSTRACT: Reflectometry is widely used in magnetically-confined plasma devices to measure both the density equilibrium profile and fluctuations. In order to better understand and thereby more effectively interpret the output, it is very useful to have a ``synthetic diagnostic''/numerical emulator for the reflectometer itself -- a synthetic reflectometer. In particular, instead of considering a collection of artificially created equilibrium and fluctuation profiles to test the efficacy of this tool, it is more relevant to apply this against realistic equilibria and fluctuations produced in an actual ``numerical experiment.'' In this work, we apply the synthetic reflectometer to a modern 3D gyrokinetic particle simulation code named GTS which is interfaced against realistic shaped-cross section tokamak equilibria and performs first principle simulations of the evolution of fluctuations. Results will be presented on progress toward our goal of producing -- for the first time -- the capability to generate synthetic diagnostic results from turbulence simulations that can be compared to those from experimental reflectometry measurements under similar plasma conditions. This could be used in realistically characterizing experimental results, verification, validation and uncertainty quantification.
    10/2012;
  • W. W. Lee, S. Ethier, J. Ganesh
    [Show abstract] [Hide abstract]
    ABSTRACT: The use of a generalized weight-based particle simulation scheme suitable for simulating tokamak turbulence is reported. The scheme, which is a generalization of the perturbed distribution schemes developed earlier for PIC simulations, is now capable of handling the full distribution of the particles in the simulation. Specifically, we can simulate both the delta-f and the full-F particles within the same code. Its development [1] is based on the concept of multiscale expansion, which separates the scale lengths of the background inhomogeneity from those associated with the perturbed distributions, and on the fact that the intrinsic particle noise level is troublesome only in the beginning of the simulation, where the signal to noise ratio is low. But, when the signal to noise ratio becomes higher afterwards, we can gradually turn on the the full-F particles without interfering with the ensuing fluctuations. We will report on the simulation studies using GTC [2] for the ion temperature gradient (ITG) driven instabilities in the presence of zonal flows. The physics of steady state transport in tokamaks will be discussed.[4pt] [1] W. W. Lee, T. G. Jenkins and S. Ethier, Comp. Phys. Comm. 182, 564 (2011).[0pt] [2] Z. Lin, T. S. Hahm, W. W. Lee, W. M. Tang, R. White Science 281, 1835 (1998).
    10/2012;
  • [Show abstract] [Hide abstract]
    ABSTRACT: Energetic ion transport has been studied using a global gyrokinetic nonlinear simulation in the presence of ion temperature gradient (ITG) driven turbulence. The measured transport and its nature show dependence on the system size of the tokamak expressed as the ratio of plasma minor radius (a) to the thermal ion Larmor radius (ρi). It increases with system size initially and then tends to saturate at larger system size. The nature of transport, on the other hand, exhibits nondiffusive character for smaller system size which eventually becomes diffusive one as the system size becomes larger.
    Physics of Plasmas 04/2012; 19(4). · 2.38 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Efficient handling of large volumes of data is a necessity for exascale scientific applications and database systems. To address the growing imbalance between the amount of available storage and the amount of data being produced by high speed (FLOPS) processors on the system, data must be compressed to reduce the total amount of data placed on the file systems. General-purpose loss less compression frameworks, such as zlib and bzlib2, are commonly used on datasets requiring loss less compression. Quite often, however, many scientific data sets compress poorly, referred to as hard-to-compress datasets, due to the negative impact of highly entropic content represented within the data. An important problem in better loss less data compression is to identify the hard-to-compress information and subsequently optimize the compression techniques at the byte-level. To address this challenge, we introduce the In-Situ Orthogonal Byte Aggregate Reduction Compression (ISOBAR-compress) methodology as a preconditioner of loss less compression to identify and optimize the compression efficiency and throughput of hard-to-compress datasets.
    Proceedings of the 28th IEEE International Conference on Data Engineering (ICDE); 04/2012
  • W. W. Lee, S. Ethier
    [Show abstract] [Hide abstract]
    ABSTRACT: A generalized weight-based particle simulation schemes suitable for simulating microturbulence in magnetic fusion plasmas, where the zeroth-order inhomogeneity is important, has recently been developed [1]. The schemes is a generalization of the perturbative simulation schemes developed earlier for PIC simulations [2]. The new two-weight scheme, which can simulate both the perturbed distribution and the full distribution within the same code, has now been implemented to simulate tokamak plasmas using the GTC code [3]. Its development is based on the concept of multiscale expansion, which separates the scale lengths of the background inhomogeneity from those associated with the perturbed distributions. The code starts out as a delta-f code and gradually evolves into a full-F code, as such the delta-f part can help us with the noise issue in the linear stage and the full-F part can be useful in the fully nonlinear stage when the particle weights become too large or it becomes necessary to simulate realistic situations where sinks and sources become important.[4pt] [1] W. W. Lee, T. G. Jenkins and S. Ethier, Comp. Phys. Comm. 182, 564 (2011).[0pt] [2] S. E. Parker and W. W. Lee, Phys. Fluids B 5, 77 (1993).[0pt] [3] Z. Lin, T. S. Hahm, W. W. Lee, W. M. Tang and R. White, Science 281, 1835 (1998).
    03/2012;
  • [Show abstract] [Hide abstract]
    ABSTRACT: Exploding dataset sizes from extreme-scale scientific simulations necessitates efficient data management and reduction schemes to mitigate I/O costs. With the discrepancy between I/O bandwidth and computational power, scientists are forced to capture data infrequently, thereby making data collection an inherently lossy process. Although data compression can be an effective solution, the random nature of real-valued scientific datasets renders lossless compression routines ineffective. These techniques also impose significant overhead during decompression, making them unsuitable for data analysis and visualization, which require repeated data access.To address this problem, we propose an effective method for In situ Sort-And-B-spline Error-bounded Lossy Abatement (ISABELA) of scientific data that is widely regarded as effectively incompressible. With ISABELA, we apply a pre-conditioner to seemingly random and noisy data along spatial resolution to achieve an accurate fitting model that guarantees a ⩾0.99 correlation with the original data. We further take advantage of temporal patterns in scientific data to compress data by ≈ 85%, while introducing only a negligible overhead on simulations in terms of runtime. ISABELA significantly outperforms existing lossy compression methods, such as wavelet compression, in terms of data reduction and accuracy.We extend upon our previous paper by additionally building a communication-free, scalable parallel storage framework on top of ISABELA-compressed data that is ideally suited for extreme-scale analytical processing. The basis for our storage framework is an inherently local decompression method (it need not decode the entire data), which allows for random access decompression and low-overhead task division that can be exploited over heterogeneous architectures. Furthermore, analytical operations such as correlation and query processing run quickly and accurately over data in the compressed space. Copyright © 2012 John Wiley & Sons, Ltd.
    Concurrency and Computation Practice and Experience 01/2012; · 0.85 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: The size and scope of cutting-edge scientific simulations are growing much faster than the I/O and storage capabilities of their runtime environments. The growing gap gets exacerbated by exploratory dataâ"intensive analytics, such as querying simulation data for regions of interest with multivariate, spatio-temporal constraints. Query-driven data exploration induces heterogeneous access patterns that further stress the performance of the underlying storage system. To partially alleviate the problem, data reduction via compression and multi-resolution data extraction are becoming an integral part of I/O systems. While addressing the data size issue, these techniques introduce yet another mix of access patterns to a heterogeneous set of possibilities. Moreover, how extreme-scale datasets are partitioned into multiple files and organized on a parallel file systems augments to an already combinatorial space of possible access patterns. To address this challenge, we present MLOC, a parallel Multilevel Layout Optimization framework for Compressed scientific spatio-temporal data at extreme scale. MLOC proposes multiple fine-grained data layout optimization kernels that form a generic core from which a broader constellation of such kernels can be organically consolidated to enable an effective data exploration with various combinations of access patterns. Specifically, the kernels are optimized for access patterns induced by (a) queryâ"driven multivariate, spatio-temporal constraints, (b) precisionâ"driven data analytics, (c) compressionâ"driven data reduction, (d) multi-resolution data sampling, and (e) multiâ"file data partitioning and organization on a parallel file system. MLOC organizes these optimization kernels within a multiâ"level architecture, on which all the levels can be flexibly re-ordered by userâ"defined priorities. When tested on queryâ"driven exploration of compressed data, MLOC demon- trates a superior performance compared to any state-of-the-art scientific database management technologies.
    Parallel Processing (ICPP), 2012 41st International Conference on; 01/2012
  • [Show abstract] [Hide abstract]
    ABSTRACT: The size and scope of cutting-edge scientific simulations are growing much faster than the I/O subsystems of their runtime environments, not only making I/O the primary bottleneck, but also consuming space that pushes the storage capacities of many computing facilities. These problems are exacerbated by the need to perform data-intensive analytics applications, such as querying the dataset by variable and spatio-temporal constraints, for what current database technologies commonly build query indices of size greater than that of the raw data. To help solve these problems, we present a parallel query-processing engine that can handle both range queries and queries with spatio-temporal constraints, on B-spline compressed data with user-controlled accuracy. Our method adapts to widening gaps between computation and I/O performance by querying on compressed metadata separated into bins by variable values, utilizing Hilbert space-filling curves to optimize for spatial constraints and aggregating data access to improve locality of per-bin stored data, reducing the false positive rate and latency bound I/O operations (such as seek) substantially. We show our method to be efficient with respect to storage, computation, and I/O compared to existing database technologies optimized for query processing on scientific data.
    Parallel & Distributed Processing Symposium (IPDPS), 2012 IEEE 26th International; 01/2012
  • [Show abstract] [Hide abstract]
    ABSTRACT: The nature of transport of hot ions is studied in the presence of microturbulence generated by the trapped electron mode in a Tokamak using massively parallel, first principle based global nonlinear gyrokinetic simulation, and with the help of a passive tracer method. Passing and trapped hot ions are observed to exhibit inverse and inverse square scaling with energy, while those with isotropic pitch distribution are found to exhibit inverse dependence on energy. For all types of hot ions, namely, isotropic, passing, and trapped, the radial transport appears to be subdiffusive for the parameters considered.
    Physics of Plasmas 11/2011; 18(11):112510-112510-9. · 2.38 Impact Factor
  • W. X. Wang, S. Ethier
    [Show abstract] [Hide abstract]
    ABSTRACT: Global gyrokinetic simulations show that ion temperature gradient (ITG) and trapped electron mode (TEM) turbulence can drive a significant parallel current in meso-scale (zonal current). The underlying dynamics is closely related to nonlinear plasma flow generation [1] by turbulent residual stress [2]. However, unlike toroidal momentum which is mostly carried by ions, the turbulent current is mainly carried by electrons in the laboratory frame, and shows finer radial scale in comparison with poloidal and toroidal zonal flows. In both collisonless TEM and ITG turbulence, substantial electron current is first generated in the positive direction of magnetic field and remains quasi-stationary in post-saturation phase. The current generation by turbulence exhibits the similar characteristic dependence on plasma parameters as that of plasma flow generation [3]. Specifically, it increases with pressure gradient, decreases with equilibrium current Ip and increases with the radial variation of safety factor. Also discussed are interesting phase space structures between TEM and ITG turbulence driven current to elucidate the roles of resonant and non-resonant electrons. In collaboration with T. S. Hahm, P. H. Diamond (UCSD), F. L. Hinton (UCSD), A. H. Boozer (Columbia U.), G. Rewoldt, W. M. Tang and W. W. Lee. [1] W. X. Wang et al., Phys. Plasmas 17, 072511 (2010). [2] P. H. Diamond et al., Phys. Plasmas 15, 012303 (2008). [3] W. X. Wang et al., Phys. Rev. Lett. 106, 085001 (2011).
    11/2011;
  • W. W. Lee, S. Ethier, R. Ganesh
    [Show abstract] [Hide abstract]
    ABSTRACT: A generalized weight-based particle simulation schemes suitable for simulating microturbulence in magnetic fusion plasmas, where the zeroth-order inhomogeneity is important, has recently been developed [1]. The schemes is a generalization of the perturbative simulation schemes developed earlier for PIC simulations [2]. The new two-weight scheme, which can simulate both the perturbed distribution (δf ) and the full distribution (total-F) within the same code, has now been extended to simulate tokamak plasmas using the GTC code [3]. Its development is based on the concept of multiscale expansion, which separates the scale lengths of the background inhomogeneity from those associated with the perturbed distributions. In this paper, we will demonstrate the correctness and the usefulness of such a code, which starts out as a δf code and gradually evolves into a full-F code. The δf part would help us with the noise issue in the linear stage and the full-F part of the code could be useful when the particle weights become too large or it becomes necessary to simulate the realistic situation where the sinks and sources for the simulation become important. [1] W. W. Lee, T. G. Jenkins and S. Ethier, Comp. Phys. Comm. 182, 564 (2011). [2] S. E. Parker and W. W. Lee, Phys. Fluids B 5, 77 (1993). [3] Z. Lin, T. S. Hahm, W. W. Lee, W. M. Tang and R. White, Science 281, 1835 (1998).
    11/2011;

Publication Stats

386 Citations
68.69 Total Impact Points

Institutions

  • 1835–2013
    • Princeton University
      • Princeton Plasma Physics Laboratory
      Princeton, New Jersey, United States
  • 2004–2008
    • University of California, Davis
      • Department of Computer Science
      Davis, California, United States
    • University of California, Irvine
      • Department of Physics and Astronomy
      Irvine, CA, United States
  • 2002
    • University of California, San Diego
      • Department of Physics
      San Diego, California, United States