Conference PaperPDF Available

EFFIS: An End-to-end Framework for Fusion Integrated Simulation

Authors:
Julian Clark Cummings at MSC Software Corporation
Julian Clark Cummings
Jay Lofstead at Sandia National Laboratories
Jay Lofstead
Karsten Schwan at Georgia Institute of Technology
Karsten Schwan
Alex Sim at Lawrence Berkeley National Laboratory
Alex Sim
Show all 10 authorsHide
Download full-text PDF
Read full-text
Download citation

Abstract

EFFIS is a set of tools developed for working with large-scale simulations. EFFIS is used by researchers in the Center for Plasma Edge Simulation, as well as many other areas of science. EFFIS is composed of services including adaptable I/O, workflows, dashboards, visualization, code coupling, wide-area data movement, and provenance capturing. One of the unique aspects of EFFIS is that it transparently allows users to switch from code coupling on disk to coupling in memory, using the concept of a shared space in a staging area. The staging area is a small fraction of the compute nodes needed to run the large-scale simulation, but it is used for the construction of I/O pipelines and a code-coupling infrastructure. This allows the scientist to make minor changes for the code to work with ADIOS), and then with no changes perform complex transformations, and analytics, which all occur in situ with the simulation. In this talk, we will focus on the technologies CPES uses, which are scalable and can be used on anything from workstations to petascale machines.
Content uploaded by Julian Clark Cummings
Author content
All content in this area was uploaded by Julian Clark Cummings
Content may be subject to copyright.
!""#$%&'(&!()*+,*-()&"./0-1,.2&
3,.&"456,(&#(+-7./+-)&$6048/9,(&
:486/(&;<&;4006(75&
'4745+&=>&?@@A&
$<&B8/52C>&D<&E/./5F/.>&G<&E,)F,.5H26>&D<&I,83&
;-(+-.&3,.&E8/50/&!)7-&$6048/9,(&
"';!J$&?@@A&D--9(7&
$K6-(9LK&M5-.&N-O46.-0-(+5&
•  "456,(&5K6-(K-&K,004(6+C&54PP,.+-)&QC&+F-&
"$E&65&R-.C&Q.,/)>&6(K84)6(7&-SP-.60-(+/865+5>&
K,0P4+/9,(/8&5K6-(95+5&/()&+F-,.65+5&
–  ;,0P4+-.&K,)-&.-O46.-0-(+5&R/.C&567(6LK/(+8C&
/0,(7&)6T-.-(+&5-+5&,3&45-.5&
•  'K.,55&+F-&K,004(6+C>&K,)-&)-R-8,P0-(+&
045+&+/2-&P8/K-&6(&P/./88-8&
–  U6T-.-(+&K,)-5>&P.,7./006(7&8/(74/7-5>&90-&/()&
8-(7+F&5K/8-5>&PFC56K/8&0,)-85>&-+K<&
!""#$&M5-.&;,004(6+C&
•  ;E!$&
–  ;<&$<&;F/(7>&:<&;4006(75>&$<&B4>&V<&E/.2>&$<&E/.2-.>&
V<&U63*E./)/86-.>&I<&I/(&
•  VE$;WV$!E&
–  $<&!+F6-.>&X<&Y6(>&I<&I/(7>&Z<&[6/,&
•  ;F60-./&\54P-.(,R/&K,)-]&
–  ^<&D-55-.>&J<&D-HH/K/PP/&
•  $_U&\K,0Q459,(&0,)-86(7]&
–  N<&V.,4+>&N <&$/(2/./(>&:<&;F-(&
J-KF(,8,7C&N-O46.-0-(+5&
•  $P-K6LK/88C>&56048/9,(&K,)-5&045+&&
–  Q-&/Q8-&+,&.4(&/+&/88&P/./88-8&P.,K-556(7&5K/8-5&
–  .4(&/5&5-P/./+-&K,)-5&\3,.&5K6-(K-&.-5-/.KF&/()&
6()-P-()-(+&K,)-&)-R-8,P0-(+]&
–  Q-&K,0P,5/Q8-&6(&/&K,4P86(7&-(R6.,(0-(+&
•  '88&+F65&045+&Q-&/KF6-R-)&16+F,4+&+F-&(--)&3,.&
.-K,)6(7&,.&\6)-/88C]&.-K,0P68/9,(&
–  !()&45-.&(--)5&`-S6Q686+C&&
!""#$&N-O46.-0-(+5&
•  "./0-1,.2&045+&Q-&5K/8/Q8-&3.,0&,(-&+,&a&06886,(&
P.,K-55,.&.4(5&
–  'U#b$&F/5&/8.-/)C&5K/8-)&1-88&4P&+,&ac@B&K,.-5&
–  B-P8-.&F/5&0,(6+,.-)&.4(5&16+F&ac@B&K,.-5&
•  "./0-1,.2&045+&Q-&-/5C&+,&45->&/()&5F,48)&.-O46.-&(,&
KF/(7-5&6(&+F-&K,)-&+,&516+KF&Q-+1--(&45/7-&0,)-5&
•  "./0-1,.2&5F,48)&Q-&K,0P.65-)&,3&1-88&+-5+-)&
K,0P,(-(+5&/()&Q-&3488C&4()-.&K,(+.,8&,3&+F-&P.,d-K+&
–  'U#b$&65&/(&#Wb&K,0P,(-(9H/9,(&
–  B-P8-.&45-5&/K+,.5&1F6KF&/.-&5-P/./+-&K,0P,(-(+5&
–  U/5FQ,/.)&45-5&16)-8C&2(,1(&"8/5F&+-KF(,8,7C>&/5&),&5-R-./8&
8-/)-.5F6P&K8/55&K,)-5&
•  E.,R-(/(K-&K/P+4.6(7&3./0-1,.2&6(56)-&!""#$&045+&
5+,.-&0489P8-&+CP-5&,3&P.,R-(/(K-&6(3,.0/9,(&
J-KF(6K/8&#0P86K/9,(5&
•  #Wb&0-+F,)5&1688&Q-&+F-&0,5+&K.69K/8&6(+-.3/K-&
–  ^.6)7-&45/7-&0,)-&Q-+1--(&56(78-&K,)-&)-R-8,P0-(+&/()&
K,4P8-)&K,)-&4986H/9,(&
•  #Wb&(--)5&+,&Q-&F67F8C&,P906H-)&
–  U/+/&+./(53-.&65&+F-&),06(/(+&K,4P86(7&0-KF/(650>&/()&)/+/&
#Wb&65&/&P,+-(9/8&56048/9,(&Q,e8-(-K2&
•  D-+/)/+/*.6KF&K/P+4.-&,3&)/+/&65&F67F8C&)-56./Q8-&
–  ^,+F&3.,0&/&P.,R-(/(K-&P-.5P-K9R-&\0489P8-&-SP-.60-(+&
0/(/7-0-(+]&/()&3.,0&/&5K6-(9LK&/(/8C565&P,6(+&,3&R6-1&
•  #(+-7./+-)>&K,(9(4,45&)/+/&/KK-556Q686+C&/6)5&.-5-/.KF&
–  I-Q&?<@*5+C8-&K,004(6+C&/KK-55&+,&.4((6(7&/()&/.KF6R/8&
56048/9,(&)/+/&/()W,.&-SP-.60-(+/8&)/+/&
CS Technologies in CPES
•  From SDM center*
–  Workflow engine – Kepler
–  Provenance support
–  Wide-area data movement
•  From universities
–  Code coupling (Rutgers)
–  Visualization (Rutgers)
•  Newly developed technologies
–  Adaptable I/O (ADIOS)
(with Georgia Tech)
–  Dashboard (with SDM center)
Visualiza(on+
Code++
Coupling+
Wide‐area+
Data++
Movement+
Dashboard+
Workflow+
Adaptable+I/O+
Provenance+
and+
Metadata+
Founda(on++
Technologies+
Enabling++
Technologies+
Approach%&P8/K-&F67F8C&/((,+/+-)>&3/5+>&-/5C*+,*45-&#Wb&0-+F,)5&6(&+F-&K,)->&1F6KF&
K/(&Q-&0,(6+,.-)&/()&K,(+.,88-)>&F/R-&/&1,.2`,1&-(76(-&.-K,.)&/88&,3&+F-&
6(3,.0/9,(>&R654/86H-&+F65&,(&/&)/5FQ,/.)>&0,R-&)-56.-)&)/+/&+,&45-.f5&56+->&/()&F/R-&
-R-.C+F6(7&.-P,.+-)&+,&/&)/+/Q/5-&
g&#(59+49,(5&6(R,8R-)%&Y^GY>&G;$M>&bNGY>&$U$;>&M;&U/R65>&M<&M+/F&
I,.2`,15&+,&'KK-8-./+-&$K6-(K-&
•  I,.2`,1&5C5+-05&045+&6(+-7./+-&K8-/(8C&Q,+F&
6(+-.(/88C&/()&-S+-.(/88C&16+F&+F-&K,.-&
K,0P4+/9,(&
–  E.-*P.,K-55&6(P4+&)/+/&
–  '5565+&6(&K,)-&K,4P86(7&
–  D/(/7-&K,(9(4,45&)/+/&/KK-556Q686+C&R6/&1-Q&
–  E,5+*P.,K-55&56048/9,(&.-548+5&
–  D/(/7-&/PP86K/9,(&/()&-SP-.60-(+&P.,R-(/(K-&
What is the Kepler Workflow Framework?
Kepler is a proven DOE technology from the SDM center for
orchestrating scientific workflows, which aid construction
and automation of scientific problem-solving processes.
•  Kepler workflow framework
–  Captures provenance information for
•  Data provenance (Where did my data come from?)
•  Data movement and data replication (e.g., during code coupling)
•  Tar files stored on HPSS (at NERSC or ORNL)
•  Workflow actions saved in log files for user debugging
–  Is more powerful than Python scripts
•  Allows pipeline-parallel processing with ease
•  Allows work to continue even if some scripts/components fail
•  Allows checkpoint/restart of the workflow
•  Easy to modify workflow for a continuously changing group of scientists
–  Provides an excellent connection to databases
•  Allows for easy queries of shots from coupled simulations
•  Saves provenance data into database
I,.2`,1&/4+,0/9,(&(--)5&,3&+F-&
;-(+-.&3,.&E8/50/&!)7-&$6048/9,(&
h  '4+,0/+-&+F-&)/+/&P.,K-556(7&P6P-86(-&&
/()&56048/9,(&0,(6+,.6(7&
i  J./(53-.&,3&56048/9,(&,4+P4+&+,&.-0,+-&0/KF6(-&&
i  !S-K49,(&,3&K,(R-.56,(&.,49(-5&
i  #0/7-&K.-/9,(>&)/+/&/.KF6R6(7>&)C(/06K&0,(6+,.6(7&
h  '4+,0/+-&+F-&K,)-&K,4P86(7&P6P-86(-&
i  N4(&26(-9K&56048/9,(&,(&/&8/.7-&54P-.K,0P4+-.&
i  ;F-K2&86(-/.&DjU&5+/Q686+C&,(&/(,+F-.&0/KF6(-&
i  N-*.4(&26(-9K&56048/9,(&63&(--)-)&
h  N-O46.-0-(+5&3,.&P-+/5K/8-&K,0P49(7&&
i  E/./88-8&P.,K-556(7&
i  N,Q45+(-55&
i  ;,(L74./Q686+C&
i  !/5C&+,&45-&
i  U/5FQ,/.)&3.,(+*-()&
i  UC(/06K&0,(6+,.6(7&
Workflow + Provenance
•  Process&P.,R-(/(K-&
–  +F-&5+-P5&P-.3,.0-)&6(&+F-&1,.2`,1>&
+F-&P.,7.-55&+F.,47F&+F-&1,.2`,1&
K,(+.,8&`,1>&-+K<&&
•  Data&P.,R-(/(K-&
–  F65+,.C&/()&86(-/7-&,3&-/KF&)/+/&6+-0&
/55,K6/+-)&16+F&+F-&/K+4/8&56048/9,(&
\6(P4+5>&,4+P4+5>&6(+-.0-)6/+-&5+/+-5]&
•  Workflow&P.,R-(/(K-&
–  F65+,.C&,3&+F-&1,.2`,1&-R,849,(&/()&
5+.4K+4./8&)-R-8,P0-(+&
•  System&P.,R-(/(K-&
–  0/KF6(-&F/.)1/.-&/()&5,k1/.-&
-(R6.,(0-(+&6(3,.0/9,(&
–  K,0P68/9,(&F65+,.C&,3&+F-&K,)-5&
–  6(3,.0/9,(&/Q,4+&+F-&86Q./.6-5&
–  5,4.K-&K,)-&
–  .4(*90-&-(R6.,(0-(+&5-l(75&
•  Performance&P.,R-(/(K-&
–  F65+,.C&,3&P-.3,.0/(K-&5+/959K5&,(&
.4((6(7&K,)-5&
h  JF-&0/6(&K,0P,(-(+5&,3&+F-&5C5+-0&/.-%&&
+F-&B-P8-.&1,.2`,1&3./0-1,.2>&+F-&/K+4/8&
56048/9,(>&/4+F,.6H/9,(&/()&/4+F-(9K/9,(&
5-.R6K-5>&/&)/+/Q/5->&/()&/&I-Q&6(+-.3/K-<&&
h  B-P8-.>&+F-&56048/9,(&K,)-&/()&P8/m,.0>&&
/()&+F-&)/5FQ,/.)&45-.&6(+-./K+&16+F&+F-&&&&
)/+/Q/5-&\U/+/$+,.-]&+F.,47F&)65P8/C>&&
.-K,.)6(7&/()&0/(/7-0-(+&'E#f5<&
ADIOS Overview
•  Overview
–  Allows plug-ins for different
I/O implementations
–  Abstracts the API from the
method used for I/O
•  Simple API, almost as easy
as F90 write statement
•  Both synchronous and
asynchronous transports
supported without code changes
•  Componentization
–  No need to worry about I/O implementation
–  Components for I/O transport methods, buffering, scheduling,
and eventually feedback mechanisms
•  MDS+ will be included as
–  fusion-specific transport method that can write data directly into MDS+
–  converter that outputs data into a remote MDS+ tree (currently have
NetCDF, HDF5, and ascii converters)
•  Change I/O method by changing XML file only!
!S+-.(/8&
D-+/)/+/&
\[DY&L8-]&
$K6-(9LK&;,)-5&
'U#b$&'E#&
U'NJ&
U/+/J/P&
DE#*#b&
Eb$#[&#Wb&
jU"*n&
E*G-+;U"&
o6H&!(76(-5&
b+F-.5&\P847*6(]&
Q4T-.6(7& 5KF-)48-& 3--)Q/K2&
'U#b$&'E#5&3,.&K,)-&K,4P86(7>&#Wb&/()&in#
situ&R654/86H/9,(&
•  /)6,5p1.6+-&\F/()8->&qP.-554.-r>&(P>&-..]&
–  I.6+-5&,4+&P.-554.-&3.,0&/88&,3&+F-&P.,K-55,.5&6(&+F-&
K,004(6K/+,.&\,.&0,R-5&6+&+,&+F-&K,4P8-.]&
•  /)6,5p7-+pR/.&\7F>&qP.-554.-s>&P.-554.->&5+/.+>&
.-/)56H->&90->&-..,.]&
–  N-/)5&6(&+F-&P.-554.-&6(&/&)6T-.-(+&P.,7./0&\i.e.,&)6T-.-(+&
DE#p;bDDpIbNYU]&,(&5/0-&,.&)6T-.-(+&P8/m,.0&
•  ;,4P8-.&)-L(-5&1F/+&F/PP-(5&)4.6(7&+F-&6(*0-0,.C&
\,.&)652]&)/+/&-SKF/(7-&
–  M5-&0-0,.C*Q/5-)&/PP.,/KF&d45+&862-&/&P/./88-8&L8-&5C5+-0&
•  M56(7&,(-&'E#>&KF/(7-&+F-&-SKF/(7-&0-+F,)&3.,0&6(*
0-0,.C&+,&)652&
'U#b$&E-.3,.0/(K-&\N-/)&/()&I.6+-]&
t@V^W5-K&I.6+-&,(&d/74/.P3>&_n&V^W5-K&.-/)&\54Q5-+&,3&)/+/]&,(&d/74/.<&
;F60-./&#b&E-.3,.0/(K-&\$4P-.(,R/&K,)-]&
0.01
0.1
1
10
100
1000
10000
512
1024
2048
4096
8192
Total I/O Time per Restart Dump
number of cores
Chimera I/O Performance (weak scaling)
MPI_CIO
POSIX
ORIG_H5
?S&5K/86(7&
•  E8,+&06(6040&R/84-&3.,0&n&.4(5&16+F&A&.-5+/.+5W.4(&
•  !..,.&Q/.5&5F,1&0/S6040&90-&3,.&+F-&0-+F,)<&
•  E/./88-8&jU"n&1/5&F/()&K,)-)&QC&;F60-./&+-/0&
^E&"68-&",.0/+&
•  "/684.-&,3&56(78-&1.6+-.&\-R-(&.,,+]&(,+&3/+/8&
–  N-5686-(KC&K.69K/8&3,.&0--9(7&/88&,3&+F-&+-KF(,8,7C&
.-O46.-0-(+5&
•  !/KF&P.,K-55&F/5&5-P/./+-&/.-/&+,&1.6+-&
•  !55-(9/88C&/&54P-.5-+&,3&G-+;U"&/()&jU"*n&3,.&-/KF&
P.,K-55&7.,4P&16+F&/(&,R-./88&6()-S&
•  '88&)/+/&KF/./K+-.6H-)&
–  !S+./&5+/959K5&\06(W0/S>&-+K<]&/.-&K/8K48/+-)&1F-(&+F-&)/+/&
65&1.6e-(&,(&/&P-.*P.,K-55,.&K/8K48/9,(&
–  M5-348&3,.&P4886(7&,4+&54KF&5+/959K5&3.,0&8/.7-&L8-5&
•  '88&)/+/&/()&,4+P4+&6()-S-)&/4+,0/9K/88C&
•  "488C&=c*Q6+&-(/Q8-)&
•  ;/(&.-/)W1.6+-&54Q5-+5&,3&)/+/&&
U/+/&$+/76(7&6(&'U#b$&
•  #Wb&Q-K,0-5&/&U/+/&
$-. R6K-&6(&'U#b$&
–  M5-5&/&5+/76(7&/.-/&
6(56)-&+F-&jE;&.-5,4.K-&
+,&P-.3,.0&0,.-&
K,0P8-S&/K9,(5&
–  $+/76(7&/.-/&65&d45+&/&
54Q5-+&,3&(,)-5&,(&5/0-&
0/KF6(-&
–  #Wb&U/+/&$-.R6K-5>&54KF&
/5&)/+/&.-)4K9,(>&
.-,.7/(6H/9,(>&
/77.-7/9,(>&/(/8C565&
/()&R654/86H/9,(>&.4(&/5&
K,(5+./6(-)&1,.2`,15&
;,)-&+,&;,)-&;,4P86(7&
•  $K6-(95+5&(--)&+,&)-Q47&/()&R/86)/+-&K,)-5&6(&
6()6R6)4/8&5-l(75&
–  E4l(7&5+/9K&P,.+5&6(+,&K,0P,(-(+5&+F/+&.-O46.-&/(&-(9.-&
3./0-1,.2&,3&-S-K49,(&+,&)-Q47&65&/&(,(*5+/.+-.&3,.&0,5+&
K,)-&)-R-8,P-.5&/()&45-.5&
–  ')/P9(7&#Wb&3,.&K,)-&K,4P86(7&8-R-./7-5&/&4(6R-.5/8&
K,(5+.4K+&,3&/88&56048/9,(&K,)-5&
•  #(5+/(9/9,(&,3&K,)-*+,*K,)-&K,4P86(7&0/C&.-O46.-&
0489P8-&5+-P5&,3&/&)C(/06K&(/+4.-&
–  M(6+&0/+KF6(7>&90-&/()&8-(7+F&5K/8->&.-P.-5-(+/9,(&5-+5>&
)/+/&6(+-.P,8/9,(>&-+K<&
–  !()&45-.&5F,48)&(,+&F/R-&+,&.-K,)-&3,.&-/KF&P,556Q8-&
K,4P86(7u&8-R-./7-&2(,1(&0-+/)/+/&6(5+-/)&
'U#b$&K,)-&K,4P86(7&0-+F,)&
•  M5-5&+F-&3,88,16(7&+-KF(,8,76-5&
–  N-0,+-&)6.-K+&0-0,.C&/KK-55&,.&NUD'&\P,.+/85&,(&;./C&
[Jn]&3,.&560P8-&7-+WP4+&\.-/)W1.6+-]&/(/8,7C&&
•  I,.25&,(&/88&jE;&5C5+-05&\6(L(6Q/()>&;./C&[J>&&#^D&^VWE]&
–  I6+F&+F-&U/+/$P/K-5&0-+F,)>&NUD'&)/+/&0,R-0-(+&65&
45-)&6(5+-/)&,3&)652&.-/)W1.6+-&
–  M5-5&/&5+/76(7&/.-/&3,.&.4((6(7&+F-&vK,4P8-.f&
–  ;,4P8-.&65&)-L(-)&6(&'U#b$&[DY&L8-&
–  GSD&)/+/&K,4P86(7&
•  ;,4P8-.&.4(5&,(&5-P/./+-&5-+&,3&P.,K-55,.5&/()&P.,R6)-5&/&
E/./88-8&V8,Q/8&')).-55&$P/K-&
[V;&
'U#b$&U6.-K+;,((-K+,.&W&&
U/+/$P/K-5&
h  'U#b$&U/+/$P/K-5WU6.-K+;,((-K+&
8-R-./7-5&+F-&-S659(7&
L8-*Q/5-)&K,)-&K,4P86(7&
h  E.,R6)-5&,P906H/9,(5&3,.&
+F-&D*+,*G&/../C&.-*0/PP6(7&
ADIOS Code Coupling
DjU&
!S+-.(/8&
D-+/)/+/&
\[DY&L8-]&
$K6-(9LK&;,)-5&
ADIOS+DirectConnect+API+
U'NJ&
DataSpaces&
DE#*#b&
Eb$#[&#Wb&
jU"*n&
E*G-+;U"&
o6H&!(76(-5&
b+F-.5&\P847*6(]&
Q4T-.6(7& 5KF-)48-& 3--)Q/K2&
'U#b$&#b&'E#&
Y#o!WU/+/J/P&
IF/+&/.-&$-0/(9K&$F/.-)&U/+/$P/K-5w&
•  '&)C(/06K&/5C(KF.,(,45&6(+-./K9,(&3./0-1,.2&
–  'Q5+./K9,(&
•  '55,K6/9R-&\7-,0-+.C*Q/5-)]&5F/.-)&5P/K->&6()-S-)&)/+/&
•  ;,0P8-S&7-,0-+.C&Q/5-)&O4-.6-5&
–  )/+/&.-)65+.6Q49,(>&K,4P86(7>&P4QW54QW(,9LK/9,(&
•  N,Q45+&)-K-(+./86H-)&)/+/&/(/8C565&6(*+F-*5P/K-&
–  !xK6-(KC&/()&$K/8/Q686+C&
•  V-,0-+.C*Q/5-)&,(86(-&6()-S6(7&/()&UjJ*Q/5-)&)6.-K+,.C&
–  "8-S6Q8-&O4-.C6(7>&-xK6-(+&)-.6R/9,(&,3&K,004(6K/9,(&5KF-)48-5&
•  G,)-*+,*(,)-&\P/./88-8>&0489+F.-/)-)]&NUD'*Q/5-)&)/+/&+./(5P,.+&
–  #0P8-0-(+/9,(&
•  ;,(5+.4K+-)&,(*+F-*`C&,(&+F-&K8,4)&,3&5+/76(7&(,)-5&
–  ;45+,06H/9,(5&3,.&P/./88-8&5K6-(9LK&/PP86K/9,(5&\e.g.,&5F/.-)&
5P/K-&Q/5-)&,(&/&0489)60-(56,(/8&)65K.-9H/9,(&,3&+F-&P.,Q8-0&
),0/6(]&
•  !(/Q8-&/5C(KF.,(,45&)/+/&K,4P86(7&5K-(/.6,5&
–  ;,0P8-0-(+5&-S659(7&P/./88-8&P.,7./006(7W0-55/76(7&5C5+-05&
'U#b$&y&P.,R-(/(K-&y&1,.2`,15&
•  I,.2`,15&454/88C&8,,2&3,.&L8-5&,3&)/+/>&Q4+&1-&
1/(+&+,&2(,1&1F/+&65&7,6(7&,(&6(&5+/76(7&/()&
0-0,.C*+,*0-0,.C&K,4P86(7&
•  'U#b$&65&K,((-K+-)&+,&B-P8-.&1,.2`,1&)6.-K+8C>&
16+F&+F-&/)6,5*P.,R-(/(K-&0-+F,)&&
•  D-+/)/+/&\P.,R-(/(K-]&65&5-(+&+,&1,.2`,1&
1F-(&/&(-1&'U#b$&0-+F,)&65&45-)&3,.&#Wb&6(&
+F-&56048/9,(&
M5/7-&,3&+F-&/)6,5*P.,R-(/(K-&0-+F,)&
•  D,(6+,.&/()&K,(+.,8&K,4P86(7&
–  1/+KF&3,.&/&KF/./K+-.659K&R/84-&/()&+/2-&5,0-&
/K9,(&63&/&K,()69,(&65&0-+&
•  &\e.g.,&0/S&,3&P.-554.-&.-/KF-5&/&+F.-5F,8)]&
–  KF/(7-&'U#b$&0-+F,)&6(&56048/9,(&+,&516+KF&
3.,0&456(7&U/+/$P/K-5&+,&1.69(7&L8-5&
•  1,.2`,1&516+KF-5&+,&L8-*Q/5-)&K,4P86(7&
–  /))69,(/8&/(/8C565&(,1&K/(&Q-&),(-&,(&+F-&)/+/&6(&
L8-5&\e.g.,&+,&)-Q47&+F-&K,)-]&
•  D/KF6(-&D,(6+,.6(7&
•  $6048/9,(&D,(6+,.6(7&
•  ;,88/Q,./9,(&
•  '(/8C565&
–  ;/8K48/+,.&
–  $+/959K/8&'(/8C565&
–  o-K+,.&V./PF6K5&
–  M5-.&5K.6P+5&
•  M5-5&'),Q-&"8/5F&/()&/)R/(K-)&
1-Q*?&+-KF(,8,7C&3,.&5K/8/Q8-&
P-.3,.0/(K-&
U/5FQ,/.)&"-/+4.-5&
U/5FQ,/.)&"-/+4.-5&
U,1(8,/)&)/+/&
o-K+,.&
V./PF6K5&
'((,+/9,(5&
G,+-5&;/8K48/+,.&
'(/8C565&456(7&N&
J.--&R6-1&,3&
R/.6/Q8-5&
•  D,R6-5&P.,)4K-)&QC&+F-&
1,.2`,1&/.-&P.-*)-+-.06(-)&&
•  JF-&7,/8&16+F&R-K+,.&7./PF6K5&65&
+,&/88,1&5K6-(95+5&+,&6(+-./K+&
16+F&+F-&)/+/&
–  D,)63C&/S65&/()&90-&./(7-&
–  X,,0>&E/(&
–  ;45+,06H-&P8,+&\K,8,.5>&8/Q-85>&
-+K<]&
•  M890/+-8C&+F-&7,/8&65&+,&8-+&+F-&
45-.5&K,004(6K/+-&+,&+F-&Q/K2&
-()&+,&7-(-./+-&/&(-1&
P4Q86K/9,(&O4/86+C&K45+,06H-)&
P8,+&456(7&/&&
'(/8C565&i&o-K+,.&V./PF6K5&
P4Q865F&Q4e,(&
'(/8C565&i&M5-.&$K.6P+5&
•  $+-P5&+,&.4(&/(/8C565&d,Q&5K.6P+&3.,0&+F-&)/5FQ,/.)%&
–  MP8,/)&/&K-.9LK/+-&+,&+F-&myproxy&5-.R-.&
–  U,1(8,/)&P.,SC&3.,0&+F-&myproxy&5-.R-.&
–  MP8,/)&/(/8C565&5K.6P+&16+F&)-+/685&,(&F,1&+,&5-+&4P&/&d,Q&
–  N4(&+F-&d,Q&456(7&+F-&K-.9LK/+-&
•  MP8,/)-)&5K.6P+5&5F/.-)&Q-+1--(&K,88/Q,./+,.5&
•  I-&/.-&54PP,.9(7&D/+8/QWbK+/R->&"A@>&;&/(/8C565&d,Q5&&
•  I-&/.-&54PP,.9(7&+F-&/Q686+C&+,&5+/.+W5+,P&56048/9,(5&
3.,0&+F-&)/5FQ,/.)&456(7&K-.9LK/+-5&
•  :,Q&54Q06556,(&3-/+4.-&/85,&/88,15&45-.5&+,&5+,P&.4((6(7&
d,Q5&1F-(&/(&/(,0/8C&/PP-/.5&6(&+F-6.&)/+/&
"488*!YD&K,4P86(7&1,.2`,1&
•  ;,4P86(7&[V;@>&D_U*bDE>&!Y#J!&/()&D_U*
DEE&K,)-5&+,&5+4)C&P-)-5+/8*!YD&PFC56K5&
•  N4(&[V;@&/()&D_U*DEE&/8+-.(/+-8C&6(&KCK8-5&
–  454/88C&.4(&[V;@&,(&:/74/.WbNGY >&D_U*DEE&,(&
"./(286(WG!N$;&
•  J1,&0,)-5&,3&[V;@&+,&D_U*bDE&K,4P86(7&
–  L8-*Q/5-)&K,4P86(7&
–  0-0,.C*+,*0-0,.C&K,4P86(7&456(7&U/+/$P/K-5&
XGC0+
26(-9K&K,)-&
M3D‐OMP+
-O4686Q.640&
5,8R-&
0_)<6(&P.,L8-&)/+/&
ELITE+
KF-K2&!YD&
ELITE+
KF-K2&!YD&
ELITE+
KF-K2&!YD&
7*-O)52&
-O4686Q.640&)/+/&
$+/Q8-w&
M3D‐OMP+
0-5F&7-(<&
G&
P,5+*K./5F&-O4686Q.640&)/+/&
jU"n&
#UY&
E-)-5+/8&
Q468)4P&
5+/7-&
E-)-5+/8&
K./5F&
5+/7-&
U6/7(,59K&
R654/86H/9,(&
z&/.KF6R/8&
M3D‐MPP+
-R,8R-&P.,L8-&
/()&!YD&K./5F&
;,4P86(7&5KF-0-&
!O4686Q.640&5+-P&1,.25&)6T-.-(+8C&6(&+F-&+1,&
0,)-5&,3&,P-./9,(&\L8-&,.&0-0,.C&Q/5-)]&
[V;>&!Y#J!&/()&D_U*DEE&/.-&0,(6+,.-)&.4(5&
;,4P86(7&456(7&U/+/$P/K-5&
•  U/+/&K,4P86(7&+F.,47F&5F/.-)&)/+/&5P/K-5&
–  [V;*@&6(5-.+5&P8/50/&P.,L8-&6(&+F-&5P/K->&/()&
-S+./K+5&4P)/+-)&DjU&-O4686Q.640&)/+/&
–  D_U&-S+./K+5&P8/50/&P.,L8-&3.,0&+F-&5P/K->&/()&
6(5-.+5&+F-&4P)/+-)&DjU&-O4686Q.640&
!O4686Q.640&5+-P&
[V;a&D,(6+,.6(7&/()&'.KF6R6(7&I,.2`,1&
•  U6/7(,59K&L8-5&\aU&R/.6/Q8-5]&
-  J./(53-.&L8-5&+,&-?-&5C5+-0&,(*+F-*`C&
-  V-(-./+-&P8,+5&456(7&7./K-&86Q./.C&
-  '.KF6R-&'U#b$&^E&L8-5&/+&-()&,3&56048/9,(&
•  ?U&R/.6/Q8-5&
-  J./(53-.&+,&-?-&5C5+-0&\-xK6-(+8C]&
-  $+/.+&4P&+1,&'o$W!SP.-55&5-.R6K-5&&
-  V-(-./+-&60/7-5&16+F&'o$W!SP.-55&\?UW
_U]&
-  '.KF6R-&jU"n&L8-5&6(&8/.7-&KF4(25&+,&jE$$&
•  V-(-./+-&0,R6-5&3.,0&+F-&60/7-5&
•  $+,.-&P.,R-(/(K-&,3&/88&/K9,(5&
!""#$%&U-R-8,P0-(+&#554-5&
•  U-R-8,P0-(+&/()&#(+-7./9,(&
–  ;,)-*Q/5-&0/(/7-)&456(7&$oG&
–  ;45+,0&Q468)&K,(L74./9,(&/()&.-7.-556,(&+-59(7&6(3./5+.4K+4.-&P8/((-)&
•  4(6+>&6(+-7./9,(>&5K-(/.6,*Q/5-)&+-59(7&
•  /4+,0/+-)&P-.6,)6K&Q468)5&
•  ,(86(-&.-P,.9(7W0,(6+,.6(7&
•  U-P8,C0-(+&/()&)655-06(/9,(&5+./+-7C&
–  $,4.K-3,.7-&
–  J4+,.6/85>&1,.25F,P5&
•  ^47&.-P,.9(7&/()&+./K26(7&
–  JN';&Q/5-)&5C5+-0&4()-.&)-R-8,P0-(+&
•  U,K40-(+/9,(&
–  #()6R6)4/8&K,0P,(-(+5&.-/5,(/Q8C&),K40-(+-)&\e.g.,&B-P8-.]&
–  E.,7./00-.W)-R-8,P-.&746)-5&4()-.&)-R-8,P0-(+&
–  'R/68/Q8-&,(86(-&\/8,(7&16+F&-S/0P8-5&/()&+4+,.6/85]&
;,(K8456,(5&
•  !""#$&5+.4K+4.-&/()&3-/+4.-5&).6R-(&QC&
+-KF(6K/8&.-O46.-0-(+5&,3&3456,(&K,004(6+C&
–  !S+.-0-&K,)-&5K/8/Q686+C&
–  !xK6-(+&/()&`-S6Q8-&#Wb&5C5+-0&
–  #()-P-()-(+&K,)-&)-R-8,P0-(+&/()&+-59(7&
–  ')/P+/Q8-&K,)-&K,4P86(7&16+F&06(60/8&KF/(7-5&
–  ;,88/Q,./9,(&,3&+F-,.65+5&/()&-SP-.60-(+/865+5&
•  !""#$&/PP.,/KF&5F,48)&/85,&Q-(-L+&Q.,/)-.&
0489*PFC56K5&56048/9,(&K,004(6+C&

Supplementary resource (1)

EFFIS paper
Data
December 2015
Julian Clark Cummings · Jay Lofstead · Karsten Schwan · Alex Sim · Roselyne Barreto Tchoua
... The Center for Plasma Edge Simulation is developing a novel integrated predictive plasma edge simulation framework applicable in existing magnetic fusion facilities and next generation burning plasma experiments. This will help scientists in understanding new models for the plasma edge in kinetic regime with complex geometry [1]. As a part of Rutgers initiative, a visualization module for plasma edge simulation was developed using commercial visualization software, Advanced Visual Systems (AVS). ...
... Thus, CPES focuses its physics research to study the interplay of neoclassical transport, micro-turbulence and magnetohydrodynamic (MHD) instabilities in the plasma edge. CPES uses kinetic approach such as a particle-incell (PIC) simulation to model neoclassical transport and micro-turbulence, while MHD modes are more efficiently studied with a fluid code [1]. ...
... Schematic diagram of EFFIS components[1] One of the prime requirements of CPES research environments was the support for variety of data transport mechanisms and data formats. Thus, ADaptable I/O System (ADIOS) was developed. ...
View
... Regarding the simulation involving multiple codes developed for different purposes, it is an effective way to carry out the coupling simulation in a specified workflow within a well-developed framework, such as IMAS [29], OMFIT [30], ITM [31], FACETS [32], TRIASSIC [33], EFFIS [34][35][36], etc. This paper aims at implementing self-consistent turbulence-transport coupling simulations automatically and efficiently, which is essential in predicting the behavior of the edge plasma for future fusion devices, and so a simulation framework called edge plasma coupling simulation (EPCS) is developed. ...
Development of a code interface for coupled turbulence-transport simulations of tokamak edge plasmas
Article
Full-text available
The self-consistent simulation of the edge plasma is crucial for exploring the edge plasma solution compatible with high-performance plasma. While self-consistent edge plasma simulation is subject to the large gap between the turbulence and transport time scales, the coupling simulation of the turbulence and transport code is considered a reasonable way with both solid physics foundations and tolerable computational consumption. In this work, for the purpose of implementing the self-consistent turbulence-transport coupling simulation of the edge plasma automatically and efficiently, a simulation framework called edge plasma coupling simulation (EPCS) is developed based on Python. EPCS consists of various components to provide the interfaces for the specified turbulence and transport codes (BOUT++ and SOLPS-ITER at the present stage), the data transfer interfaces between the turbulence and transport code, the code running drivers and the function for configuration of the specified coupling simulation workflow. The inverse bilinear interpolation/bivariate spline interpolation under the flux-surface-aligned coordinate system is used to realize the accurate data transfer between different codes, and the breadth-first search algorithm is adopted to accelerate the interpolation process. A quasi-steady state identification method based on the coefficient of variation is developed to speed up the coupling simulation by terminating the turbulence simulation in time. Based on the components in EPCS, a steady-state coupling simulation workflow is developed, where the edge plasma is simulated by iterations of turbulence and transport codes. The steady-state coupling simulation workflow is validated by comparing the converged plasma profiles with EAST experiments (edge-localized-mode-free stage) at both upstream and divertor target, which implies the capability and flexibility of EPCS for the self-consistent simulation of the edge plasma under steady state.
View
... A common theme across existing workflow management systems is the focus on execution patterns and optimizing computational throughput, dynamic support constrained to task restarts but almost no support for real-time data delivery, monitoring, and workflow steering. The EFFIS [59] framework, initially designed to loosely couple multiple fusion codes running on HPC resources, is a workflow management system that uses a combination of enabling technologies, including ADIOS [60], Kepler, and eSimMon [61], a web-based dashboard. EFFIS is built upon the Cheetah-Savanna [62] suite of workflow tools and provides an API for composing and executing codesign studies for online data analysis on different supercomputers. ...
2022 Review of Data-Driven Plasma Science
Article
Full-text available
  • Jul 2023
Data-driven science and technology offer transformative tools and methods to science. This review article highlights the latest development and progress in the interdisciplinary field of data-driven plasma science (DDPS), i.e., plasma science whose progress is driven strongly by data and data analyses. Plasma is considered to be the most ubiquitous form of observable matter in the universe. Data associated with plasmas can, therefore, cover extremely large spatial and temporal scales, and often provide essential information for other scientific disciplines. Thanks to the latest technological developments, plasma experiments, observations, and computation now produce a large amount of data that can no longer be analyzed or interpreted manually. This trend now necessitates a highly sophisticated use of high-performance computers for data analyses, making artificial intelligence and machine learning vital components of DDPS. This article contains seven primary sections, in addition to the introduction and summary. Following an overview of fundamental data-driven science, five other sections cover widely studied topics of plasma science and technologies, i.e., basic plasma physics and laboratory experiments, magnetic confinement fusion, inertial confinement fusion and high-energy-density physics, space and astronomical plasmas, and plasma technologies for industrial and other applications. The final section before the summary discusses plasma-related databases that could significantly contribute to DDPS. Each primary section starts with a brief introduction to the topic, discusses the state-of-the-art developments in the use of data and/or data-scientific approaches, and presents the summary and outlook. Despite the recent impressive signs of progress, the DDPS is still in its infancy. This article attempts to offer a broad perspective on the development of this field and identify where further innovations are required.
View
... A common theme across existing workflow management systems is the focus on execution patterns and optimizing computational throughput, dynamic support constrained to task restarts, but almost no support for real time data delivery, monitoring, and workflow steering. The EFFIS [53] framework, initially designed to loosely couple multiple fusion codes running on HPC resources, is a workflow management system that uses a combination of enabling technologies, including ADIOS [54], Kepler and eSimMon [55], a web-based dashboard. EFFIS is built upon the Cheetah-Savanna [56] suite of workflow tools and provides an API for composing and executing codesign studies for online data analysis on different supercomputers. ...
2022 Review of Data-Driven Plasma Science
Preprint
Full-text available
  • May 2022
Data science and technology offer transformative tools and methods to science. This review article highlights latest development and progress in the interdisciplinary field of data-driven plasma science (DDPS). A large amount of data and machine learning algorithms go hand in hand. Most plasma data, whether experimental, observational or computational, are generated or collected by machines today. It is now becoming impractical for humans to analyze all the data manually. Therefore, it is imperative to train machines to analyze and interpret (eventually) such data as intelligently as humans but far more efficiently in quantity. Despite the recent impressive progress in applications of data science to plasma science and technology, the emerging field of DDPS is still in its infancy. Fueled by some of the most challenging problems such as fusion energy, plasma processing of materials, and fundamental understanding of the universe through observable plasma phenomena, it is expected that DDPS continues to benefit significantly from the interdisciplinary marriage between plasma science and data science into the foreseeable future.
View
... The EFFIS framework has its roots in the SciDAC program, where it was originally created by researchers from the fusion and computer science research community to loosely couple fusion codes in an easy-to-use framework (Cummings et al. 2010). EFFIS 1.0 combined the Kepler workflow engine (Altintas et al. 2004), along with ADIOS for I/O (Liu et al. 2014), eSimMon to operate as a dashboard (Tchoua et al. 2010), and a MySQL 12 database. ...
The Exascale Framework for High Fidelity coupled Simulations (EFFIS): Enabling whole device modeling in fusion science
Article
  • May 2021
  • INT J HIGH PERFORM C
We present the Exascale Framework for High Fidelity coupled Simulations (EFFIS), a workflow and code coupling framework developed as part of the Whole Device Modeling Application (WDMApp) in the Exascale Computing Project. EFFIS consists of a library, command line utilities, and a collection of run-time daemons. Together, these software products enable users to easily compose and execute workflows that include: strong or weak coupling, in situ (or offline) analysis/visualization/monitoring, command-and-control actions, remote dashboard integration, and more. We describe WDMApp physics coupling cases and computer science requirements that motivate the design of the EFFIS framework. Furthermore, we explain the essential enabling technology that EFFIS leverages: ADIOS for performant data movement, PerfStubs/TAU for performance monitoring, and an advanced COUPLER for transforming coupling data from its native format to the representation needed by another application. Finally, we demonstrate EFFIS using coupled multi-simulation WDMApp workflows and exemplify how the framework supports the project’s needs. We show that EFFIS and its associated services for data movement, visualization, and performance collection does not introduce appreciable overhead to the WDMApp workflow and that the resource-dominant application’s idle time while waiting for data is minimal.
View
... Future works will focus on finding semi-automatically optimal allocation size for j − α like use-cases, on the development of hybrid use-cases (parallelization at component and workflow levels) as well as making a more in-depth benchmark comparing the performance of the different platforms. Other interesting platforms such as EFFIS [19] or OMFIT [20], developed within similar efforts in the USA, might also be investigated. ...
Coupled Simulations in Plasma Physics with the Integrated Plasma Simulator Platform
Article
Full-text available
  • Dec 2015
Simulations of fusion plasma obtained in a Tokamak device can involve a wide range of physics phenomena occurring at different scales. Programming such a simulation is challenging and tends to increase the complexity of the code and its maintenance. Many approaches are trying to alleviate this issue by coupling several single scale components, the complexity being moved from the physics code to the coupling and execution platform. In this paper we are presenting the Integrated Plasma Simulator (IPS) platform, its advantages for running efficiently coupled simulations for different plasma physics use cases and are briefly comparing it to other platforms used in the fusion community.
View
... Data gathered within CPOs (Consistent Physical Objects) is used for "partitioned" interchange between codes in "strongly" coupled workflows. Similarly to CPOs, ADIOS system [6] provides groups that describe collections of data written at once. ADIOS as a HPC library for input/output (IO) provides simplified and configurable write strategy to parallel codes. ...
Visualization support for code development in EUROfusion integrated modelling
Conference Paper
Full-text available
  • May 2014
The need for diverse visualization tools within integrated modelling is based on the requirement of physics codes to be coupled under the tailored database model from which these visualization tools read data. We discuss these visualization requirements for fusion integrated modeling, and the existing visualization tools developed within the scope of the European Integrated Modelling (EU IM) framework. The datastructure model provides fundamental description for data exchange between codes that are developed in a variety of programming languages. From this description in XML schema definition one can generate required language interfaces and database access layer routines. The persistent storage database has to efficiently support codes running on HPC with memory caching mechanisms. Visualization tools can be treated as one of the codes running in-situ or as post-process. The complexity of the fusion datastructure and several visualization tools requires that scientists describe standard visualizations in XML schema directly in order to instruct visualization tools what are meaningful visualizations available in the very database. Besides standard visualizations there are specific visualizations that cannot be easily described (as mapping or axes linking) without algorithm. For the custom visualizations we provide Python snippets collected in user-shared library. Interfaces to scientific workflow engine Kepler for visualization with VisIt visualization tool and Matplotlib are presented.
View
View
Understanding and Leveraging the I/O Patterns of Emerging Machine Learning Analytics
Chapter
  • Mar 2022
The scientific community is currently experiencing unprecedented amounts of data generated by cutting-edge science facilities. Soon facilities will be producing up to 1 PB/s which will force scientist to use more autonomous techniques to learn from the data. The adoption of machine learning methods, like deep learning techniques, in large-scale workflows comes with a shift in the workflow’s computational and I/O patterns. These changes often include iterative processes and model architecture searches, in which datasets are analyzed multiple times in different formats with different model configurations in order to find accurate, reliable and efficient learning models. This shift in behavior brings changes in I/O patterns at the application level as well at the system level. These changes also bring new challenges for the HPC I/O teams, since these patterns contain more complex I/O workloads. In this paper we discuss the I/O patterns experienced by emerging analytical codes that rely on machine learning algorithms and highlight the challenges in designing efficient I/O transfers for such workflows. We comment on how to leverage the data access patterns in order to fetch in a more efficient way the required input data in the format and order given by the needs of the application and how to optimize the data path between collaborative processes. We will motivate our work and show performance gains with a study case of medical applications.KeywordsEmerging HPC applicationsDeep learning methodsI/O patternsI/O optimizationData management
View
View
Plasma simulation studies using multilevel physics models
Article
Full-text available
  • May 1999
The question of how to proceed toward ever more realistic plasma simulation studies using ever increasing computing power is addressed. The answer presented here is the M3D (Multilevel 3D) project, which has developed a code package with a hierarchy of physics levels that resolve increasingly complete subsets of phase-spaces and are thus increasingly more realistic. The rationale for the multilevel physics models is given. Each physics level is described and examples of its application are given. The existing physics levels are fluid models (3D configuration space), namely magnetohydrodynamic (MHD) and two-fluids; and hybrid models, namely gyrokinetic-energetic-particle/MHD (5D energetic particle phase-space), gyrokinetic-particle-ion/fluid-electron (5D ion phase-space), and full-kinetic-particle-ion/fluid-electron level (6D ion phase-space). Resolving electron phase-space (5D or 6D) remains a future project. Phase-space-fluid models are not used in favor of deltaf particle models. A practical and accurate nonlinear fluid closure for noncollisional plasmas seems not likely in the near future.
View
Storage resource managers: Essential components for the Grid
Article
Full-text available
  • Jan 2004
Storage Resource Managers (SRMs) are middleware components whose func- tion is to provide dynamic space allocation and file management of shared stor- age components on the Grid. They complement Compute Resource Managers and Network Resource Managers in providing storage reservation and dynamic information on storage availability for the planning and execution of a Grid job. SRMs manage two types of resources: space and files. When managing space, SRMs negotiate space allocation with the requesting client, and/or assign default space quotas. When managing files, SRMs allocate space for files, invoke file transfer services to move files into the space, pin files for a certain lifetime, re- lease files upon the clients request, and use file replacement policies to optimize the use of the shared space. SRMs can be designed to provide effective sharing of files, by monitoring the activity of shared files, and make dynamic decisions on which files to replace when space is needed. In addition, SRMs perform automatic garbage collection of unused files by removing selected files whose lifetime has expired when space is needed. In this chapter we discuss the design considerations for SRMs, their functionality, and their interfaces. We demon- strate the use of SRMs with several examples of real implementations that are in use today in a routine fashion or in a prototype form.
View
Numerical studies of edge localized instabilities in tokamaks
Article
Full-text available
  • Apr 2002
A new computational tool, edge localized instabilities in tokamaks equilibria (ELITE), has been developed to help our understanding of short wavelength instabilities close to the edge of tokamak plasmas. Such instabilities may be responsible for the edge localized modes observed in high confinement H-mode regimes, which are a serious concern for next step tokamaks because of the high transient power loads which they can impose on divertor target plates. ELITE uses physical insight gained from analytic studies of peeling and ballooning modes to provide an efficient way of calculating the edge ideal magnetohydrodynamic stability properties of tokamaks. This paper describes the theoretical formalism which forms the basis for the code.
View
Collaboration portal for petascale simulations
Article
Full-text available
  • Jan 2009
The emergence of leadership class computing is creating a tsunami of data from petascale simulations. Results are typically analyzed by dozens of scientists. In order for the scientist to digest the vast amount of data being produced from the simulations and auxiliary programs, it is critical to automate the effort to manage, analyze, visualize, and share this data. One aspect of this is leveraging of their collective knowledge and experiences through a scientific social network. This can be archived through a combination of parallel back-end services, provenance capturing, and an easy to use front-end tool. "eSimMon", is one such tool we developed as part of the Scientific Discovery through Advanced Computing (SciDAC) program. In this paper we describe eSimMon, discuss its ease of use, its efficiency, and its ability to accelerate scientific discovery through advanced computing.
View
Workflows in a secure environment
Chapter
Full-text available
  • Jan 2008
Petascale simulations on the largest supercomputers in the US require advanced data management techniques in order to optimize the application scientist time, and to optimize the time spent on the supercomputers. Researchers in such problems are starting to require workflow automation during their simulations in order to monitor the simulations, and in order to automate many of the complex analysis which must take place from the data that is generated from these simulations. Scientific workflows are being used to monitor simulations running on these supercomputers by applying a series of complex analysis, and finally producing images and movies from the variables produced in the simulation, or from the derived quantities produced by the analysis. The typical scenario is where the large calculation runs on the supercomputer, and the auxiliary diagnostics/monitors are run on resources, which are either on the local area network of the supercomputer, or over the wide area network. The supercomputers at one of the largest centers are highly secure, and the only method to log into the center is interactive authentication by using One Time Passwords (OTP) that are generated by a security device and expire in half a minute. Therefore, grid certificates are not a current option on these machines in the Department of Energy at Oak Ridge National Laboratory. In this paper we describe how we have extended the Kepler scientific workflow management system to be able to run operations on these supercomputers, how workflows themselves can be executed as batch jobs, and finally, how external data-transfer operations can be utilized when they need to perform authentication for their own as well.
View
Tracking Files in the Kepler Provenance Framework
Conference Paper
Full-text available
  • Jun 2009
  • Lect Notes Comput Sci
Workflow Management Systems (WFMS), such as Kepler, are proving to be an important tool in scientific problem solving. They can automate and manage complex processes and huge amounts of data produced by petascale simulations. Typically, the produced data need to be properly visualized and analyzed by scientists in order to achieve the desired scientific goals. Both run-time and post analysis may benefit from, even require, additional meta-data – provenance information. One of the challenges in this context is the tracking of the data files that can be produced in very large numbers during stages of the workflow, such as visualizations. The Kepler provenance framework collects all or part of the raw information flowing through the workflow graph. This information then needs to be further parsed to extract meta-data of interest. This can be done through add-on tools and algorithms. We show how to automate tracking specific information such as data files locations.
View
Extending I/O through high performance data services
Conference Paper
Full-text available
  • Jan 2009
The complexity of HPC systems has increased the burden on the developer as applications scale to hundreds of thousands of processing cores. Moreover, additional efforts are required to achieve acceptable I/O performance, where it is important how I/O is performed, which resources are used, and where I/O functionality is deployed. Specifically, by scheduling I/O data movement and by effectively placing operators affecting data volumes or information about the data, tremendous gains can be achieved both in the performance of simulation output and in the usability of output data. Previous studies have shown the value of using asynchronous I/O, of employing a staging area, and of performing select operations on data before it is written to disk. Leveraging such insights, this paper develops and experiments with higher level I/O abstractions, termed ldquodata servicesrdquo, that manage output data from `source to sink': where/when it is captured, transported towards storage, and filtered or manipulated by service functions to improve its information content. Useful services include data reduction, data indexing, and those that manage how I/O is performed, i.e., the control aspects of data movement. Our data services implementation distinguishes control aspects - the control plane - from data movement - the data plane, so that both may be changed separably. This results in runtime flexibility not only in which services to employ, but also in where to deploy them and how they use I/O resources. The outcome is consistently high levels of I/O performance at large scale, without requiring application change.
View
Adaptable, metadata rich IO methods for portable high performance IO
Conference Paper
Full-text available
  • May 2009
Since IO performance on HPC machines strongly depends on machine characteristics and configuration, it is important to carefully tune IO libraries and make good use of appropriate library APIs. For instance, on current petascale machines, independent IO tends to outperform collective IO, in part due to bottlenecks at the metadata server. The problem is exacerbated by scaling issues, since each IO library scales differently on each machine, and typically, operates efficiently to different levels of scaling on different machines. With scientific codes being run on a variety of HPC resources, efficient code execution requires us to address three important issues: (1) end users should be able to select the most efficient IO methods for their codes, with minimal effort in terms of code updates or alterations; (2) such performance-driven choices should not prevent data from being stored in the desired file formats, since those are crucial for later data analysis; and (3) it is important to have efficient ways of identifying and selecting certain data for analysis, to help end users cope with the flood of data produced by high end codes. This paper employs ADIOS, the ADaptable IO System, as an IO API to address (1)-(3) above. Concerning (1), ADIOS makes it possible to independently select the IO methods being used by each grouping of data in an application, so that end users can use those IO methods that exhibit best performance based on both IO patterns and the underlying hardware. In this paper, we also use this facility of ADIOS to experimentally evaluate on petascale machines alternative methods for high performance IO. Specific examples studied include methods that use strong file consistency vs. delayed parallel data consistency, as that provided by MPI-IO or POSIX IO. Concerning (2), to avoid linking IO methods to specific file formats and attain high IO performance, ADIOS introduces an efficient intermediate file format, termed BP, which can be converted, at small cost, to the standard file formats used by analysis tools, such as NetCDF and HDF-5. Concerning (3), associated with BP are efficient methods for data characterization, which compute attributes that can be used to identify data sets without having to inspect or analyze the entire data contents of large files.
View
View
Numerical study of neoclassical plasma pedestal in a tokamak geometry
Article
  • Apr 2004
The fundamental properties of steep neoclassical plasma pedestals in a quiescent tokamak plasma have been investigated with a new guiding center particle code XGC: an X-point included Guiding Center code. It is shown that the width of the steepest neoclassical pedestals is similar to an experimentally observed edge pedestal width, and that a steep pedestal must be accompanied by a self-consistent negative radial electric field well. It is also shown that a steep neoclassical pedestal can form naturally at a quiescent diverted edge as the particle source from the neutral penetration (and heat flux from the core plasma) is balanced by the sharply increasing convective ion loss toward the separatrix. The steep neoclassical pedestal and the strong radial electric field well are suppressed by an anomalous diffusion coefficient of a strength appropriate to an L-mode state; nonetheless, the E×B shearing rate increases rapidly with pedestal temperature. Additionally, the present study shows that a steep pedestal at the diverted edge acts as a cocurrent parallel momentum source. © 2004 American Institute of Physics.
View