PresentationPDF Available

HPC and Computational Chemistry: Overview of the Methods, Algorithms, & Challenges.

Authors:
  • Qubit Pharmaceuticals

Abstract

Presentation at HPC department of Universität Basel Florent Hédin
HPC and Computational Chemistry: Overview of the
Methods, Algorithms, & Challenges.
Presentation at HPC department of Universität Basel
Florent Hédin
Group Prof. M. Meuwly
Department of Chemistry
Universität Basel
June 2016
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 1 / 23
1Overview
What is computational chemistry ?
Size and Time-scale
2Molecular Dynamics: Algorithms, Complexity and Optimisations
MD: Algorithms
Complexity and optimisations
3Computational Chemistry programs for HPC
Requirements
Overview of codes and technologies used
4Case study 1 : GROMACS
5Case study 2 : Anton and DESMOND
6Conclusion
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 2 / 23
Overview What is computational chemistry ?
What is computational chemistry ?
Computational chemistry is a branch of chemistry that uses computer simula-
tion to assist in solving chemical problems.
It uses methods of theoretical chemistry, incorporated into efficient computer
programs, to calculate the structures and properties of molecules and solids.
Source: C. J. Mundy
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 3 / 23
Overview Size and Time-scale
Challenges: different time-scales and systems size
Source: T. Schlick
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 4 / 23
Molecular Dynamics: Algorithms, Complexity and Optimisations MD: Algorithms
Molecular Dynamics (MD): Newton’s laws of motion
For a system of N particles with coordinates Xand velocities V, the following pair
of first order differential equations may be written in Newton’s notation:
F(X) = −∇U(X) = M˙
V(t)
V(t) = ˙
X(t)
Source: Y. Sonavane et al.
Integration usually performed using Velocity
Verlet equation:
~
x(t+ ∆t) = ~
x(t) + ~
v(t)∆t+1
2~
a(t)∆t2
~
v(t+ ∆t) = ~
v(t) + ~
a(t) + ~
a(t+ ∆t)
2t
Global error is O(∆t2).
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 5 / 23
Molecular Dynamics: Algorithms, Complexity and Optimisations MD: Algorithms
Forcefields
V(rN) = X
bonds
kb(ll0)2+X
angles
ka(θθ0)2+X
torsions X
n
1
2Vn[1+cos(nωγ)]
+
N1
X
j=1
N
X
i=j+1ij r0ij
rij 12
2r0ij
rij 6+qiqj
4π0rij
Functional form and parameter sets
used to calculate the potential energy
of a system of atoms
Source: Wikipedia
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 6 / 23
Molecular Dynamics: Algorithms, Complexity and Optimisations MD: Algorithms
Periodic Boundary Conditions
A set of boundary conditions which are often cho-
sen for approximating a large (infinite) system by
using a small part called a unit cell. The large sys-
tems approximated by PBCs consist of an infinite
number of unit cells
One cell is the original simulation box, and others
are copies called images. Only the properties of
the original simulation box need to be recorded
and propagated.
Source : ISAACS;FH & MM
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 7 / 23
Molecular Dynamics: Algorithms, Complexity and Optimisations Complexity and optimisations
Algorithms complexity (1) : Truncation
Non-bonded interactions are the most time-consuming part of energy and forces
estimation : O(N2)
V(rN) = · · · +
N1
X
j=1
N
X
i=j+1ij r0ij
rij 12
2r0ij
rij 6+qiqj
4π0rij
First optimisation: ignore far atoms as
non-bonded potential goes to 0 for long
distances: Cut-off
Source: ChemWiki
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 8 / 23
Molecular Dynamics: Algorithms, Complexity and Optimisations Complexity and optimisations
Algorithms complexity (2) : Particle Mesh approaches
Optimised algorithms, like the Particle Mesh Ewald methods, can reduce complexity
from O(N2)to O(N·log N):
Ewald summation rewrites the interaction potential as the sum of two terms,
ϕ(r)def
=ϕsr (r) + ϕ`r(r)
where ϕsr (r)represents the short-range term whose sum quickly converges in real
space and ϕ`r(r)represents the long-range term whose sum quickly converges in
Fourier (reciprocal) space.
Optimised and efficient libraries like FFTW usually used
Source: Usenix;Wikipedia FFTW
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 9 / 23
Molecular Dynamics: Algorithms, Complexity and Optimisations Complexity and optimisations
Algorithms complexity (3) : neighbours finding
We saw previously that using a cut-off reduces computation time by skipping cal-
culations between distant atoms ; A Verlet list is used in order to maintain a list
of all particles within a given cut-off distance of each other: naively O(N2)
Optimisation: Cell lists work by subdividing the simulation domain into cells with
an edge length greater than or equal to the cut-off radius. The particles are sorted
into these cells and the interactions are computed between particles in the same or
neighbouring cells: complexity O(N)
Source: Wikipedia;arXiv
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 10 / 23
Computational Chemistry programs for HPC Requirements
HPC MD codes for Computational Chemistry
Classical molecular dynamics simulation programs are very widely used on super-
computers.
Nowadays, computational chemists rarely write their own codes but instead exploit
the opportunities offered by high quality, freely available and open source packages.
What “we” expect from a MD HPC code nowadays:
1Scalability (atom count) : from “a few” atoms to millions of atoms (ratio
1 CPU per 1000 atoms)
2Numerical accuracy, i.e. stable integration over millions of steps : usually 1
nano-second is 500k to 1 million steps, but timescale on which protein events
we want to observe occur can be of the order of milli-seconds !
3Efficient analysis tools : it is nice to generate tera-bytes of data, but
analysing them with a serial bash/python script may take some time !
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 11 / 23
Computational Chemistry programs for HPC Overview of codes and technologies used
HPC MD codes for Computational Chemistry (2)
Several open source or proprietary codes provide efficient production (MD) and
analysis codes :
GROMACS
CHARMM
NAMD
AMBER
DESMOND*
· · ·
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 12 / 23
Case study 1 : GROMACS
GROMACS : introduction
Sources for all the following slides: website;Article2013;Article2015
GROMACS is one of the most widely used open-source and free software codes in
chemistry, used primarily for dynamical simulations of biomolecules
In version 5, it reaches new performance heights, through several new and enhanced
parallelization algorithms.
These work on every level:
SIMD registers inside cores
multithreading
heterogeneous CPU–GPU acceleration
state-of-the-art 3D domain decomposition
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 13 / 23
Case study 1 : GROMACS
Multi-level parallelism in GROMACS
SIMD registers are used to parallelize cluster interaction kernels or bonded interac-
tions in each core, and OpenMP multithreading is used for parallelism inside spatial
domains while nonbonded interactions are handled by GPUs or other accelerators.
MPI with load balancing is used to decompose a single simulation into domains
over many nodes in a cluster.
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 14 / 23
Case study 1 : GROMACS
GROMACS : core GPU + CPU loop
There are lots of things that are computed every step, but the most compute-
intensive part is the nonbonded force calculation to GPU or other accelerator
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 15 / 23
Case study 1 : GROMACS
GROMACS : detailed GPU + CPU flowchart
Flowchart where the main MD loop runs both on the CPU and GPU:
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 16 / 23
Case study 1 : GROMACS
GROMACS : MPI over nodes and load balancing
Left: The protein lysozyme (24,119 atoms) in a compact unit cell representation
corresponding to a rhombic dodecahedron with close-to-spherical shape. Right:
Internally, this is represented as a triclinic cell and a load-balanced staggered 6x4x3
domain decomposition grid on 72 MPI ranks.
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 17 / 23
Case study 1 : GROMACS
GROMACS : performance and speedup
The reference setup (A) spends the majority of wall-time in short-range force eval-
uations, but as SIMD (B), OpenMP (C, D), and GPU (E, F) acceleration are
enabled this drops tremendously even on a workstation. With GPUs (E, F), the
CPU computes a relatively small amount of bonded interactions. Relying on multi-
threading (F) when using GPUs is more efficient than SPMD parallelization with
domain-decomposition (E)
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 18 / 23
Case study 1 : GROMACS
GROMACS : Piz Daint
Piz Daint: Cray XC30 architecture, with one 8-core socket per node containing
Intel Xeon E5-2670 Sandy Bridge processors and one NVIDA Tesla K20X GPU
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 19 / 23
Case study 2 : Anton and DESMOND
Anton : special-purpose supercomputer
Anton is a recently completed special-purpose supercomputer designed for molec-
ular dynamics (MD) simulations of biomolecular systems : specialized hardware.
DESMOND is the dedicated MD software package.
The HTIS performs the most demanding calculations in an MD simulation. the
flexible subsystem performs the remaining MD calculations, coordinates MD time
step activity, and manages housekeeping tasks.
Sources : Article2008
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 20 / 23
Case study 2 : Anton and DESMOND
Anton 2 : Next Generation ?
Source : Slides2014;
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 21 / 23
Case study 2 : Anton and DESMOND
Anton 1 & 2 : Performance
Source : Article2014
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 22 / 23
Conclusion
Conclusion
Just 2 points to remember :
1We need fast and reliable HPC code for computational chemistry
2Without computer science people like you, we can’t reach this level of
performance :
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 23 / 23
Conclusion
Conclusion
Just 2 points to remember :
1We need fast and reliable HPC code for computational chemistry
2Without computer science people like you, we can’t reach this level of
performance :
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 23 / 23
Conclusion
Conclusion
Just 2 points to remember :
1We need fast and reliable HPC code for computational chemistry
2Without computer science people like you, we can’t reach this level of
performance :
So ...
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 23 / 23
Conclusion
Conclusion
Just 2 points to remember :
1We need fast and reliable HPC code for computational chemistry
2Without computer science people like you, we can’t reach this level of
performance :
So ...
F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 23 / 23
ResearchGate has not been able to resolve any citations for this publication.
ResearchGate has not been able to resolve any references for this publication.