Content uploaded by Florent Hédin

Author content

All content in this area was uploaded by Florent Hédin on Sep 15, 2017

Content may be subject to copyright.

1Overview

What is computational chemistry ?

Size and Time-scale

2Molecular Dynamics: Algorithms, Complexity and Optimisations

MD: Algorithms

Complexity and optimisations

3Computational Chemistry programs for HPC

Requirements

Overview of codes and technologies used

4Case study 1 : GROMACS

5Case study 2 : Anton and DESMOND

6Conclusion

F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 2 / 23

Overview What is computational chemistry ?

What is computational chemistry ?

Computational chemistry is a branch of chemistry that uses computer simula-

tion to assist in solving chemical problems.

It uses methods of theoretical chemistry, incorporated into eﬃcient computer

programs, to calculate the structures and properties of molecules and solids.

Source: C. J. Mundy

F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 3 / 23

Molecular Dynamics: Algorithms, Complexity and Optimisations MD: Algorithms

Molecular Dynamics (MD): Newton’s laws of motion

For a system of N particles with coordinates Xand velocities V, the following pair

of ﬁrst order diﬀerential equations may be written in Newton’s notation:

F(X) = −∇U(X) = M˙

V(t)

V(t) = ˙

X(t)

Source: Y. Sonavane et al.

Integration usually performed using Velocity

Verlet equation:

~

x(t+ ∆t) = ~

x(t) + ~

v(t)∆t+1

2~

a(t)∆t2

~

v(t+ ∆t) = ~

v(t) + ~

a(t) + ~

a(t+ ∆t)

2∆t

Global error is O(∆t2).

F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 5 / 23

Molecular Dynamics: Algorithms, Complexity and Optimisations MD: Algorithms

Forceﬁelds

V(rN) = X

bonds

kb(l−l0)2+X

angles

ka(θ−θ0)2+X

torsions X

n

1

2Vn[1+cos(nω−γ)]

+

N−1

X

j=1

N

X

i=j+1ij r0ij

rij 12

−2r0ij

rij 6+qiqj

4π0rij

Functional form and parameter sets

used to calculate the potential energy

of a system of atoms

Source: Wikipedia

F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 6 / 23

Molecular Dynamics: Algorithms, Complexity and Optimisations MD: Algorithms

Periodic Boundary Conditions

A set of boundary conditions which are often cho-

sen for approximating a large (inﬁnite) system by

using a small part called a unit cell. The large sys-

tems approximated by PBCs consist of an inﬁnite

number of unit cells

One cell is the original simulation box, and others

are copies called images. Only the properties of

the original simulation box need to be recorded

and propagated.

Source : ISAACS;FH & MM

F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 7 / 23

Molecular Dynamics: Algorithms, Complexity and Optimisations Complexity and optimisations

Algorithms complexity (1) : Truncation

Non-bonded interactions are the most time-consuming part of energy and forces

estimation : O(N2)

V(rN) = · · · +

N−1

X

j=1

N

X

i=j+1ij r0ij

rij 12

−2r0ij

rij 6+qiqj

4π0rij

First optimisation: ignore far atoms as

non-bonded potential goes to 0 for long

distances: Cut-oﬀ

Source: ChemWiki

F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 8 / 23

Molecular Dynamics: Algorithms, Complexity and Optimisations Complexity and optimisations

Algorithms complexity (2) : Particle Mesh approaches

Optimised algorithms, like the Particle Mesh Ewald methods, can reduce complexity

from O(N2)to O(N·log N):

Ewald summation rewrites the interaction potential as the sum of two terms,

ϕ(r)def

=ϕsr (r) + ϕ`r(r)

where ϕsr (r)represents the short-range term whose sum quickly converges in real

space and ϕ`r(r)represents the long-range term whose sum quickly converges in

Fourier (reciprocal) space.

Optimised and eﬃcient libraries like FFTW usually used

Source: Usenix;Wikipedia FFTW

F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 9 / 23

Molecular Dynamics: Algorithms, Complexity and Optimisations Complexity and optimisations

Algorithms complexity (3) : neighbours ﬁnding

We saw previously that using a cut-oﬀ reduces computation time by skipping cal-

culations between distant atoms ; A Verlet list is used in order to maintain a list

of all particles within a given cut-oﬀ distance of each other: naively O(N2)

Optimisation: Cell lists work by subdividing the simulation domain into cells with

an edge length greater than or equal to the cut-oﬀ radius. The particles are sorted

into these cells and the interactions are computed between particles in the same or

neighbouring cells: complexity O(N)

Source: Wikipedia;arXiv

F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 10 / 23

Computational Chemistry programs for HPC Requirements

HPC MD codes for Computational Chemistry

Classical molecular dynamics simulation programs are very widely used on super-

computers.

Nowadays, computational chemists rarely write their own codes but instead exploit

the opportunities oﬀered by high quality, freely available and open source packages.

What “we” expect from a MD HPC code nowadays:

1Scalability (atom count) : from “a few” atoms to millions of atoms (ratio ∼

1 CPU per 1000 atoms)

2Numerical accuracy, i.e. stable integration over millions of steps : usually 1

nano-second is 500k to 1 million steps, but timescale on which protein events

we want to observe occur can be of the order of milli-seconds !

3Eﬃcient analysis tools : it is nice to generate tera-bytes of data, but

analysing them with a serial bash/python script may take some time !

F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 11 / 23

Computational Chemistry programs for HPC Overview of codes and technologies used

HPC MD codes for Computational Chemistry (2)

Several open source or proprietary codes provide eﬃcient production (MD) and

analysis codes :

GROMACS

CHARMM

NAMD

AMBER

DESMOND*

· · ·

F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 12 / 23

Case study 1 : GROMACS

GROMACS : introduction

Sources for all the following slides: website;Article2013;Article2015

GROMACS is one of the most widely used open-source and free software codes in

chemistry, used primarily for dynamical simulations of biomolecules

In version 5, it reaches new performance heights, through several new and enhanced

parallelization algorithms.

These work on every level:

SIMD registers inside cores

multithreading

heterogeneous CPU–GPU acceleration

state-of-the-art 3D domain decomposition

F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 13 / 23

Case study 1 : GROMACS

Multi-level parallelism in GROMACS

SIMD registers are used to parallelize cluster interaction kernels or bonded interac-

tions in each core, and OpenMP multithreading is used for parallelism inside spatial

domains while nonbonded interactions are handled by GPUs or other accelerators.

MPI with load balancing is used to decompose a single simulation into domains

over many nodes in a cluster.

F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 14 / 23

Case study 1 : GROMACS

GROMACS : MPI over nodes and load balancing

Left: The protein lysozyme (24,119 atoms) in a compact unit cell representation

corresponding to a rhombic dodecahedron with close-to-spherical shape. Right:

Internally, this is represented as a triclinic cell and a load-balanced staggered 6x4x3

domain decomposition grid on 72 MPI ranks.

F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 17 / 23

Case study 1 : GROMACS

GROMACS : performance and speedup

The reference setup (A) spends the majority of wall-time in short-range force eval-

uations, but as SIMD (B), OpenMP (C, D), and GPU (E, F) acceleration are

enabled this drops tremendously even on a workstation. With GPUs (E, F), the

CPU computes a relatively small amount of bonded interactions. Relying on multi-

threading (F) when using GPUs is more eﬃcient than SPMD parallelization with

domain-decomposition (E)

F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 18 / 23

Case study 2 : Anton and DESMOND

Anton : special-purpose supercomputer

Anton is a recently completed special-purpose supercomputer designed for molec-

ular dynamics (MD) simulations of biomolecular systems : specialized hardware.

DESMOND is the dedicated MD software package.

The HTIS performs the most demanding calculations in an MD simulation. the

ﬂexible subsystem performs the remaining MD calculations, coordinates MD time

step activity, and manages housekeeping tasks.

Sources : Article2008

F.Hédin (Uni. Basel, Meuwly Group) HPC2016 June 2016 20 / 23