ArticlePDF Available

The Black Hole Accretion Code: adaptive mesh refinement and constrained transport

Authors:

Abstract

With the forthcoming VLBI images of Sgr A* and M87, simulations of accretion flows onto black holes acquire a special importance to aid with the interpretation of the observations and to test the predictions of different accretion scenarios, including those coming from alternative theories of gravity. The Black Hole Accretion Code (BHAC) is a new multidimensional general-relativistic magnetohydrondynamics (GRMHD) module for the MPI-AMRVAC framework. It exploits its adaptive mesh refinement techniques (AMR) to solve the equations of ideal magnetohydrodynamics in arbitrary curved spacetimes with a significant speedup and saving in computational cost. In a previous work, this was shown using a Generalized Lagrange Multiplier (GLM) to enforce the solenoidal constraint of the magnetic field. While GLM is fully compatible with MPI-AMRVAC's AMR infrastructure, we found that simulations were sensible to the divergence control technique employed, resulting in an improved behavior for those using Constrained Transport (CT). However, cell-centered CT is incompatible with AMR, and several modifications were required to make AMR compatible with staggered CT. We present here preliminary results of these new additions, which achieved machine precision fulfillment of the solenoidal constraint and a significant speedup in a problem close to the intended scientific application.
The Black Hole Accretion Code: adaptive mesh
refinement and constrained transport
HR Olivares S´anchez, O Porth, Y Mizuno
Institute for Theoretical Physics, Max-von-Laue-Str. 1, 60438 Frankfurt, Germany.
E-mail: olivares@th.physik.uni-frankfurt.de
Abstract. With the forthcoming VLBI images of Sgr A* and M87, simulations of accretion
flows onto black holes acquire a special importance to aid with the interpretation of the
observations and to test the predictions of different accretion scenarios, including those
coming from alternative theories of gravity. The Black Hole Accretion Code (BHAC ) is a
new multidimensional general-relativistic magnetohydrondynamics (GRMHD) module for the
MPI-AMRVAC framework. It exploits its adaptive mesh refinement techniques (AMR) to solve the
equations of ideal magnetohydrodynamics in arbitrary curved spacetimes with a significant
speedup and saving in computational cost. In a previous work, this was shown using a
Generalized Lagrange Multiplier (GLM) to enforce the solenoidal constraint of the magnetic
field. While GLM is fully compatible with MPI-AMRVAC ’s AMR infrastructure, we found
that simulations were sensible to the divergence control technique employed, resulting in an
improved behavior for those using Constrained Transport (CT). However, cell-centered CT is
incompatible with AMR, and several modifications were required to make AMR compatible
with staggered CT. We present here preliminary results of these new additions, which achieved
machine precision fulfillment of the solenoidal constraint and a significant speedup in a problem
close to the intended scientific application.
1. Introduction
The Black Hole Accretion Code BHAC is an extension of the MPI-AMRVAC framework to perform
General Relativistic Magnetohydrodynamics (GRMHD) simulations in 1, 2, and 3 dimensions
using finite volume methods and a variety of modern numerical methods, described more in detail
in [1]. It exploits MPI-AMRVAC ’s infrastructure for parallelization and block-based automated
Adaptive Mesh Refinement (AMR), resulting in a significant saving in computational time and
resources.
In fact, despite the variety of General Relativistic Hydrodynamics and Magnetohydrodynam-
ics codes currently available [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] aside from
some exceptions as [8, 20, 17] AMR is still not a commonly exploited tool.
However, AMR capabilities can be extremely useful for some problems that are currently
computationally prohibitive for most codes. These include resolving simultaneously the
formation and propagation of relativistic jets from black holes, due to the interaction between
very different physical scales (see e.g., [21]), or tilted accretion disks, where the highly asymmetric
evolution prevents the use of the static stretched grid commonly employed to increase resolution
at the equator.
arXiv:1802.00860v1 [gr-qc] 2 Feb 2018
The most immediate application envisaged for BHAC is the simulation of Sgr A* and M87, the
two primary targets of the Event Horizon Telescope (EHT). Both these objects belong to the
class of advection dominated accretion flows (ADAFs), for which ideal magnetohydrodynamics
without radiation feedback constitutes a reasonable approximation of the plasma properties. In
order to properly study the impact of plasma and gravitational conditions on the EHT images,
BHAC is coupled to the General Relativistic Radiative Transfer (GRRT) codes BHOSS [22] and
RAPTOR [23].
An important motivation for our research is the possibility to distinguish departures from
General Relativity in the images obtained by the EHT. For this reason, BHAC is designed with
a modular structure that can handle arbitrary spacetimes, including numerical ones as well
as those coming from alternative theories of gravity. For instance, [24] successfully performed
GRMHD simulations of accretion flows onto a dilaton black hole in the Einstein-Maxwell-Axion-
Dilaton theory of gravity. As an example of another application, the code has recently been
used to study quasi-periodic-oscillations (QPOs) in accretion discs around neutron stars [25].
In a previous work [1], we tested BHAC in several standard problems and validated it by
comparing the results of some of them to those obtained in control simulations performed using
the well known code HARM3D [5, 26].
The new additions presented in this work concern changes in the AMR infrastructure,
necessary to allow AMR to operate simultaneously with Constrained Transport (CT), a
divergence control scheme which already showed considerable advantages with respect to GLM,
the technique used in [1] (see section 3.2 for more details). One of these advantages is the
ability to keep a discretization of ·Bequal to zero to machine precision. In fact, since no cell-
centered divergence-free discretization is currently known to be compatible with AMR [27], and
staggered versions require special divergence-free prolongation and restriction operators for face-
and edge-allocated quantities, GLM was the only such technique available for AMR simulations
in BHAC .
The paper is organized as follows: section 2 summarizes the equations and the formulation
of GRMHD used in the code; section 3 briefly describes the numerical methods employed in the
code focusing on the newly implemented divergence control strategies; and section 4 reports on
two numerical tests performed. Throughout this work, we employ geometrized units (G=c= 1)
and use the Einstein summation convention. Greek indices run from 0 to 3, while Latin indices
run from 1 to 3.
2. Equations of GRMHD
The equations of ideal GRMHD are those of particle conservation, local conservation of energy-
momentum and the homogeneous Maxwell equations
µ(ρuµ)=0,µTµν = 0 ,and µFµν = 0 ,(1)
where µdenotes the covariant derivative, ρis the particle number in the fluid frame, uµthe
fluid 4-velocity, Tµν the energy-momentum tensor and Fµν the dual of the Faraday tensor Fαβ.
The Faraday tensor and its dual are such that, for a frame moving at 4-velocity nν, the
electric and magnetic fields are given by
Eµ=Fµν nνand Bµ=Fµν nν.(2)
In ideal MHD, only the magnetic field is evolved (using the homogeneous Maxwell equations),
since the electric field is determined by the ideal MHD condition, which requires that the electric
field in the frame co-moving with the fluid is eµ=Fµνuν= 0.
To formulate system (1) as a set of evolution equations, we use the 3+1 decomposition of
spacetime (see for example [28] and [29]). The spacetime is sliced into spacelike 3-dimensional
hypersurfaces with metric γij. The 4-dimensional line element is expressed as
ds2=α2dt2+γij(dxi+βidt)(dxj+βjdt),(3)
where αand βiare called the lapse function and the shift vector. The 4-velocity of the Eulerian
observers is just the vector normal to each hypersurface, nµ=αµt. The 4-metric can
then be decomposed as gµν =γµν nµnν; therefore, γµνacts as a projection operator on the
hypersurface.
When projecting the equations of system (1) along nµand γµν, the result is a set of
conservation equations with geometry-dependent sources
t(γU) + i(γFi) = γS(4)
and the solenoidal constraint for the magnetic field iγBi= 0, which results from the
projection nµνFµν. Here, γis the metric determinant, and the vectors of conserved quantities
U, fluxes Fi, and sources Sare given by
U=
D
Sj
τ
Bj
,Fi=
ViD
αW i
jβiSj
α(SiviD)βiτ
ViBjBiVj
and S=
0
1
2αW ikjγik +SijβiUjα
1
2Wikβjjγik +Wj
ijβiSjjα
0
,
(5)
where Vi:= αviβiare the transport velocities, and the others variables are quantities in
the Eulerian frame: D=ρuνnνis the number density, Si=nµγνi Tµν the covariant 3-
momentum, U=nµnνTµν the total energy and Wij =γµiγνj Tµν the spatial stress tensor.
Evolving τ=UDinstead of Umakes the evolution more accurate in regions of low energy
and allows to recover the Newtonian limit.
Evolution cannot be carried out using only conservative variables, since the computation of
some quantities in the expressions for the fluxes requires the knowledge of the primitive variables
P=ρ, Γvi, p, Bi. Here, Γ is the Lorentz factor, vi=ui/Γβiand pis the pressure in the
fluid frame. While it is straightforward to find U(P), P(U) requires numerical inversion. To
this end, BHAC extends the vector U(P) by the auxiliary variables A= [Γ, ξ], where ξ:= Γ2ρh
and his the specific enthalpy. The inversion process then consists of finding Acompatible with
Uand P.
3. Numerical methods and implementation
In this section we will describe briefly the numerical methods employed in BHAC , focusing on
generalities of the finite volume implementation and the new features of staggered-mesh based
divergence control methods and adaptive mesh refinement for the staggered variables.
For an in depth description of the methods available in the code, including equations of state,
coordinates and handling of the metric data structure, reconstruction schemes, Riemann solvers
and procedures for primitive variable recovery, we refer the reader to [1].
3.1. Finite volume scheme
To obtain the finite-volume scheme used by BHAC , we discretize the domain into control volumes
Vi,j,k and integrate equation (4) over each of them. This leads to the equations of evolution
for the average of the conserved quantities inside each cell,
d¯
Ui,j,k
dt =1
Vi,j,k "F1S1i+1/2,j,k F1S1i1/2,j,k+
F2S2i,j+1/2,k F2S2i,j1/2,k+
F3S3i,j,k+1/2F3S3i,j,k1/2#+¯
Si,j,k .
(6)
The quantities as F1S1i+1/2,j,k are integrals of the fluxes over the surfaces ∆S1i+1/2,j,k
bounding the control volume and ¯
Si,j,k is the volume average of the sources. Both kinds of
integrals are approximated to second order, by assigning to Fn(n= 1,2,3) the point value of
the flux at the interface center and to ¯
Si,j,k the point value at the cell barycenter. Fnis obtained
through the approximate solution of a Riemann problem at the interface, and static integrals
such as cell volumes, interface areas and barycenter positions are calculated at initialization
using fourth-order Simpson’s rule and stored in memory. Equation 6 can then be solved using
the integrators present in the MPI-AMRVAC toolkit. These include the simple predictor-corrector,
the third order Runge-Kutta RK3 [30] and the strong-stability preserving s-step, pth-order RK
schemes SSPRK(s,p) schemes: SSPRK(4,3), SSPRK(5,4) due to [31]. (For implementation
details, see [32].)
3.2. Divergence control
Although the induction equation can also be expressed in the form of equation (4), using the finite
volume scheme of equation (6) alone to evolve the magnetic field usually results in the creation
and rapid growth of numerical magnetic monopoles, driving the evolution towards flagrantly
unphysical states. In order to keep violations to ·B= 0 small, three schemes are available in
BHAC that can be used together with AMR. The first one is the scheme known as the Generalized
Lagrange Multiplier (GLM) of the Maxwell equations, a generalization of the Dedner scheme
[33] used in Newtonian MHD. This method consists in solving an additional evolution equation
that has the effect of damping and advecting away the violations to ·B= 0. GLM has already
been applied to GRMHD by e.g., [34]. Though this technique can be straightforwardly included
in BHAC ’s algorithm, some of its Newtonian versions have been shown to suffer from spurious
oscillations in the magnetic energy and from an artificial growth of the magnetic fields, effects
attributed, respectively, to the loss of locality due to the parabolic nature of the additional
equation and to the resulting scheme being non-conservative [35, 36]. The other two available
Figure 1. Spatial location of variables for a cell
with indices (i, j, k). Line integrals of the electric
field Eare located at its edges, and magnetic and
numerical fluxes Φ and Fi(the latter used for the
BS algorithm) are located at its faces. The rest of
variables (not shown) are located at cell centers.
techniques are Constrained Transport (CT) schemes. These are obtained by integrating the
induction equation (γijµFµj = 0) on the boundary of the control volume, and their central
feature is that they fulfill to machine precision a discretized version of the solenoidal constraint
when constraint-satisfying initial data is supplied (it can be easily obtained by setting the initial
magnetic field as the curl of a potential). CT was first devised for ideal GRMHD by [37],
although a similar idea was exploited before in the Yee algorithm [38]. In these algorithms, the
electromagnetic variables are given a special space location (see Figure 1): on each face of the
cell resides a magnetic flux calculated, e.g., as
Φi+1/2,j,k =Z∂V (x1
i+1/2)
γ1/2B1dx2dx3,(7)
and on each edge resides a line integral of the electric field, e.g.,
Ei+1/2,j+1/2,k =Zx3
k+1/2
x3
k1/2
E3|x1
i+1/2,x2
j+1/2dx3.(8)
The magnetic flux at each face is updated using the integral form of Faraday’s law:
d
dtΦi+1/2,j,k =Ei+1/2,j+1/2,k − Ei+1/2,j 1/2,k − Ei+1/2,j,k+1/2+Ei+1/2,j,k1/2.(9)
Since each of the line integrals is shared by two faces, but appears with opposite sign in the
time update formula for each of them, the rate of change of (∇ · B), i.e., the sum of the rate
of change of the outgoing flux through all faces, vanishes. So far, equation (9) is exact. Each
variant of CT arises from different ways of approximating the line integrals of the electric field.
The two variants available in our code are the method of Balsara & Spicer (BS) [39] and
Upwind Constrained Transport (UCT) [40]. In BS, Ei+1/2,j+1/2,k is calculated simply as the
arithmetic average of the fluxes obtained by the Riemann solver that correspond to the electric
component E3at the faces ∆Si+1/2,j,k, ∆Si+1/2,j +1,k, ∆Si,j+1/2,k and ∆Si+1,j+1/2,k .
A cell-centered version of the BS algorithm, known as Flux-interpolated Constrained Transport
(flux-CT), was found in [41] and is widely used in the literature (see e.g., [5, 26]). Unlike the
staggered version, the cell-centered scheme is not compatible with AMR; however, it has been
reported that in otherwise identical GRMHD simulations performed using GLM and flux-CT,
the latter produced less spurious structures in the magnetic field, and was able to preserve
for a longer time an exact stationary solution [1]. This provided a strong motivation for us to
implement the staggered algorithm in BHAC to gain the advantages of AMR. Nevertheless, BS has
also known deficiencies [42] that can be overcome by upwinding the electromotive force, as it is
done in the the algorithm by Gardiner & Stone [43] and in UCT. The latter, also implemented in
BHAC , is another staggered algorithm, devised to incorporate the correct continuity and upwind
properties of the magnetic field by using limited reconstructions and by taking into account
the transport velocities. In contrast to BS, UCT has the additional property that it reduces to
the correct 1-dimensional limit when the correspondent symmetry is assumed. For details on
the specific UCT implementation in BHAC , we refer the reader to a more complete work [44],
currently in preparation.
3.3. Adaptive mesh refinement
Most of the infrastructure for AMR is inherited from the MPI-AMRVAC toolkit. The grid is a fully
adaptive block-based octree (in 3D) with a fixed refinement factor of two between successive
levels. Operations on the grid as time update, IO and problem initialization are performed
on a loop over a Morton Z-order curve. The time step is calculated globally and is the same
for all levels, thus load-balancing is simply done by cutting the space-filling curve in equal
parts and distributing them among the MPI-processes. This strategy is applied in various
astrophysical codes, for example in those employing the PARAMESH library [45, 46, 47] , or the
recent Athena++ framework [48]. Refinement can be triggered in a completely automated way
either using the L¨ohner scheme [49] or user defined prescriptions. Details on the prolongation
and restriction operations and the ghost cell exchange can be found in [50]. To ensure machine
precision conservation of U, re-fluxing is performed every (partial) time step, i.e., the fluxes on
the coarse side of coarse/fine interfaces are replaced by the sum of the co-spatial fluxes on the
fine side.
New additions specific to BHAC are divergence-free restriction and prolongation operators for
the staggered variables and an electric field fixing step to avoid producing numerical monopoles
across resolution jumps, and which also consists on replacing the electric fields on the coarse
side with their co-spatial fine representation. Details about the prolongation operator and the
electric field fixing formulas will be documented in a forthcoming work [44].
4. Numerical tests
4.1. Validation of the code
BHAC has been thoroughly validated using the GLM and the flux-CT schemes for the magnetic
field evolution. This was done by performing several test problems in 1, 2 and 3 dimensions,
as well as comparisons with simulations performed using the code HARM3D , as is reported in
[1]. The results obtained using the newly available features here described were verified against
the validated results to ensure that the implementation was correct. This will be documented
in detail in [44]. In the next sections, we will describe the results of applying AMR and the
staggered BS algorithm in two test problems.
4.2. Relativistic Orszag-Tang vortex
The Orszag-Tang vortex [51] is a common setup to highlight the impact of the violations to
the solenoidal constraint in the numerical solution. Starting from a configuration in which
∇ · B= 0 to machine precision, the problem quickly develops magnetic shocks and turbulence,
both challenging conditions for the preservation of the constraint. In this 2-dimensional, special-
relativistic realization of the test, we set ρ= 1, p= 10, vx=0.99
2sin y,vy=0.99
2sin x,
Bx=sin yand By= sin 2x. The equation of state is that of an ideal fluid with γ= 4/3. The
domain is the square x, y [0,2π] with periodic boundary conditions. We adopt three AMR
levels, where the lowest resolution is equivalent to resolve the whole domain with 64 ×64 cells.
The numerical methods to evolve the system are an RK3 integrator with HLL fluxes and the
Koren reconstruction (third order accuracy in smooth parts of the solution [52]). The divergence
control method is the staggered CT algorithm with arithmetic averaging and the CFL factor is
set to 0.4.
Figure 2 shows two snapshots of the evolution. On the left panel is the divergence of the
magnetic field at t= 2, near the time when the strongest shocks form. At that moment, all the
three AMR levels are present in the simulation, and it can be seen that the algorithm is able
to keep the largest violations to the level of 1013, in contrast to the 101100that are
produced in a similar set up with GLM (see [1]). The right panel displays the magnetic field
intensity and the magnetic field lines at the same time of the simulation, showing the formation
of current sheets at the same location of the maximum creation of divergence.
In order to perform a more quantitative comparison of how well AMR performs, Figure 3
displays the density and magnetic field strength profiles along a cut at y= 0.5 and at the same
time, for the simulation described above and for another one identical except for being run at a
uniform high resolution correspondent to that of the highest AMR level. A very good agreement
can be observed between both simulations. When evolving up to t= 10 the AMR case obtained
a modest speedup factor of 1.35. This is due to the fact that at later times most of the domain
shows large variations in the quantities that trigger refinement, thus most of the simulation
reaches the highest AMR level.
This is, however, not expected to be the case in the intended astrophysical applications of
the code, where likely large parts of the domain are emulating vacuum, as will be seen in the
next section.
Figure 2. Snapshots
of the relativistic Orszag-
Tang problem at t= 2.
Left: Divergence of the
magnetic field and AMR
blocks.
Right: Magnetic field
intensity and magnetic
field lines.
Figure 3. Density and magnetic
field strength profiles at y= 0.5 and
t= 2 for uniform resolution and 3-
level AMR. The AMR structure is
represented by the symbols ‘+’.
4.3. Magnetized accretion onto Kerr black hole
To study the speedup and accuracy achievable using AMR in a case closer to the intended
application of the code, we perform two simulations of the same 2D problem of accretion from
a magnetized torus onto a Kerr black hole, one using a grid with uniformly high resolution
and the other one using AMR, with the maximum resolution corresponding to that of the first
simulation.
The spacetime is described using logarithmic Kerr-Schild coordinates, correspondent to the
standard Kerr-Schild coordinates r[1.213,2500 M] and θ[0, π]. This allows the propagation
of the jet over a long distance and prevents signals from the boundaries to affect the inner region
when evolving until t= 5000 M. The spin parameter of the black hole is a= 0.9375, and the
event horizon is located at r= 1.348 M.
The fluid obeys an ideal equation of state with γ= 4/3. As initial condition, we set up an
equilibrium torus with inner radius at rin = 6 M, and density maximum at rmax = 12 M(orbital
period of 247 Mat the density maximum). A single-loop poloidal magnetic field is built from the
vector potential Aφmax(ρ/ρmax 0.2,0) and is normalized in such a way that the minimum
plasma β=pfluid/pmag = 100. To avoid vacuum regions, the rest of the simulation is filled with
Figure 4. Upper half: Logarithmic
density and AMR blocks at time
t= 2000 M.
Lower half: Divergence of the
magnetic field.
Figure 5. Mass accretion rate and absolute magnetic
flux through the horizon for uniform resolution and
AMR
a tenuous atmosphere with density ρ= 104r3/2and fluid pressure p= 1/3×106r5/2.
We reset the density or the pressure whenever they fall below these floor values. To perturb
this equilibrium state, random perturbations of 4% are added to the pressure. This eventually
triggers the magneto-rotational instability, allowing the plasma to accrete.
The simulations are evolved using a two-step predictor-corrector method, TVDLF fluxes, and
PPM reconstruction. The CFL number is set to 0.35. For evolving the magnetic field we use
staggered CT with arithmetic averaging. The boundary conditions at the inner and outer radial
boundaries are set to zero gradient in the primitive variables, i.e., their values at the last cell of
the physical domain are copied to fill the ghost cells, except for the ingoing component of the
velocity, which is set to zero. The numerical fluxes and line integrals of the azimuthal electric
field in contact with the polar axis are also set to zero, since they correspond to integrals over
zero-area surfaces and zero-length paths. The number of cells in the simulation with uniform
high resolution is Nr×Nθ= 800 ×400. The simulation with AMR has three levels, the highest
with the same resolution as the one with the uniform mesh. AMR is triggered automatically
using the L¨ohner scheme.
Figure 4 displays the grid blocks of the AMR simulation at time t= 2000 M, showing how
resolution increases in regions with large variations in density. To perform a more quantitative
comparison, Figure 5 shows that the mass accretion rate and absolute magnetic flux through the
horizon for both simulations have comparable magnitude and variability. While the simulation
at uniform resolution required 2324 cpu-hours to reach t= 5000 M, the simulation using three
AMR levels required only 327, yielding a significant speedup factor of 7.1.
5. Conclusion
BHAC is a new versatile tool to study magneto-hydrodynamic flows in arbitrary spacetimes in
General Relativity and other metric theories of gravity, which incorporates modern numerical
methods and an efficient AMR infrastructure inherited from MPI-AMRVAC .
We have made new additions to this infrastructure in order to allow staggered-mesh
constrained transport to run with the advantages of AMR.
In this work, we have presented the first results of simulations performed with BHAC using
AMR and a CT together. These indicate that the code is now capable of evolving efficiently the
GRMHD equations in multi-scale problems, with the solenoidal constraint fulfilled to machine
precision.
As a matter of fact, for a problem close to its intended application, BHAC was able to attain
a very significant speedup of 7.1, which can be of crucial importance, also for e.g., performing
parameter studies to contrast with the EHT data.
In a forthcoming work, we will describe in greater detail such modifications as well as other
additions to the numerical methods besides those described in [1].
Acknowledgments
We would like to express our gratitude to Luciano Rezzolla, Elias Most, Christian Fromm,
Ziri Younsi, Alejandro Cruz Osorio, David Kling, Jonas K¨ohler and Mariafelicia de Laurentis
for useful discussions. This research is supported by the ERC synergy grant ”BlackHoleCam:
Imaging the Event Horizon of Black Holes” (Grant No. 610058), by “NewCompStar”, COST
Action MP1304, by the LOEWE-Program in HIC for FAIR, and by the European Union’s
Horizon 2020 Research and Innovation Programme (Grant 671698) (call FETHPC-1-2014,
project ExaHyPE). HO is supported in part by a CONACYT-DAAD scholarship. The
simulations were performed on LOEWE at the CSC-Frankfurt and Iboga at ITP Frankfurt.
We acknowledge technical support from Thomas Coelho.
References
[1] Porth O, Olivares H, Mizuno Y, Younsi Z, Rezzolla L, Moscibrodzka M, Falcke H and Kramer M 2017
Computational Astrophysics and Cosmology 41 (Preprint 1611.09720)
[2] Hawley J F, Smarr L L and Wilson J R 1984 Astrophys. J. 277 296–311
[3] Kudoh S K D M K S T 2000 Astrophys. J 536 668–674
[4] De Villiers J P and Hawley J F 2003 Astrophys. J. 589 458–480 (Preprint arXiv:astro-ph/0210518)
[5] Gammie C F, McKinney J C and T´oth G 2003 Astrophys. J. 589 458 (Preprint astro-ph/0301509)
[6] Baiotti L, Hawke I, Montero P J, L¨offler F, Rezzolla L, Stergioulas N, Font J A and Seidel E 2005 Phys.
Rev. D 71 024035 (Preprint gr-qc/0403029)
[7] Duez M D, Liu Y T, Shapiro S L and Stephens B C 2005 Phys. Rev. D 72 024028 (Preprint
astro-ph/0503420)
[8] Anninos P, Fragile P C and Salmonson J D 2005 Astrophys. J. 635 723
[9] Ant´on L, Zanotti O, Miralles J A, Mart´ı J M, Ib´nez J M, Font J A and Pons J A 2006 Astrophys. J. 637
296 (Preprint astro-ph/0506063)
[10] Mizuno Y, Nishikawa K I, Koide S, Hardee P and Fishman G J 2006 ArXiv Astrophysics e-prints (Preprint
astro-ph/0609004)
[11] Del Zanna L, Zanotti O, Bucciantini N and Londrillo P 2007 Astron. Astrophys. 473 11–30 (Preprint
0704.3206)
[12] Giacomazzo B and Rezzolla L 2007 Class. Quantum Grav. 24 235 (Preprint gr-qc/0701109)
[13] Radice D and Rezzolla L 2012 Astron. Astrophys. 547 A26 (Preprint 1206.6502)
[14] Radice D, Rezzolla L and Galeazzi F 2014 Mon. Not. R. Astron. Soc. L. 437 L46–L50 (Preprint 1306.6052)
[15] McKinney J C, Tchekhovskoy A, Sadowski A and Narayan R 2014 Mon. Not. R. Astron. Soc. 441 3177–3208
(Preprint 1312.6127)
[16] Etienne Z B, Paschalidis V, Haas R, M¨osta P and Shapiro S L 2015 Class. Quantum Grav. 32 175009
(Preprint 1501.07276)
[17] White C J, Stone J M and Gammie C F 2016 Astrophys. J., Supp. 225 22 (Preprint 1511.00943)
[18] Zanotti O and Dumbser M 2015 Computer Physics Communications 188 110–127 (Preprint 1312.7784)
[19] Meliani Z, Mizuno Y, Olivares H, Porth O, Rezzolla L and Younsi Z 2017 Astron. Astrophys. 598 A38
(Preprint 1606.08192)
[20] Zanotti O, Fambri F and Dumbser M 2015 Mon. Not. R. Astron. Soc. 452 3010–3029 (Preprint 1504.07458)
[21] Mizuno Y, G´omez J L, Nishikawa K I, Meli A, Hardee P E and Rezzolla L 2015 Astrophys. J. 809 38
(Preprint 1505.00933)
[22] Younsi Z and et al 2017 In preparation
[23] Bronzwaer, T and et al 2017 In preparation
[24] Mizuno Y and et al 2017 In preparation
[25] de Avellar M and et al 2017 In preparation
[26] Noble S C, Krolik J H and Hawley J F 2009 Astrophys. J. 692 411–421 (Preprint 0808.3140)
[27] oth G and Roe P 2002 Journal of Computational Physics 180 736 – 750 ISSN 0021-9991 URL
http://www.sciencedirect.com/science/article/pii/S0021999102971202
[28] Alcubierre M 2008 Introduction to 3 + 1 Numerical Relativity (Oxford, UK: Oxford University Press)
[29] Rezzolla L and Zanotti O 2013 Relativistic Hydrodynamics (Oxford, UK: Oxford University Press) ISBN
9780198528906
[30] Gottlieb S and Shu C 1998 Math. Comp. 67 73–85
[31] Spiteri R J and Ruuth S J 2002 SIAM J. Numerical Analysis 40 469–491
[32] Porth O, Xia C, Hendrix T, Moschou S P and Keppens R 2014 Astrophys. J., Supp. 214 4 (Preprint
1407.2052)
[33] Dedner A, Kemm F, Kr¨oner D, Munz C D, Schnitzer T and Wesenberg M 2002 Journal of Computational
Physics 175 645–673
[34] Palenzuela C, Lehner L, Reula O and Rezzolla L 2009 Mon. Not. R. Astron. Soc. 394 1727–1740 (Preprint
0810.1838)
[35] Balsara D S and Kim J 2004 Astrophys. J. 602 1079–1090 (Preprint astro-ph/0310728)
[36] Mocz P, Pakmor R, Springel V, Vogelsberger M, Marinacci F and Hernquist L 2016 Mon. Not. R. Astron.
Soc. 463 477–488 (Preprint 1606.02310)
[37] Evans C R and Hawley J F 1988 Astrophys. J. 332 659–677
[38] Yee K 1966 IEEE Transactions on Antennas and Propagation 14 302–307 ISSN 0018-926X
[39] Balsara D S and Spicer D S 1999 J. Comput. Phys. 149 270–292
[40] Londrillo P and Del Zanna L 2004 Journal of Computational Physics 195 17–48
[41] Toth G 2000 J. Comput. Phys. 161 605–652
[42] Flock M, Dzyurkevich N, Klahr H and Mignone A 2010 Astron. Astrophys. 516 A26 (Preprint 0906.5516)
[43] Gardiner T A and Stone J M 2005 Journal of Computational Physics 205 509–539 (Preprint
astro-ph/0501557)
[44] Olivares H and et al 2017 In preparation
[45] MacNeice P, Olson K M, Mobarry C, de Fainchtein R and Packer C 2000 Computer Physics Communications
126 330–354
[46] Fryxell B, Olson K, Ricker P, Timmes F X, Zingale M, Lamb D Q, MacNeice P, Rosner R, Truran J W and
Tufo H 2000 Astrophys. J.s 131 273–334
[47] Zhang W and MacFadyen A 2006 The Astrophysical Journal Supplement Series 164 255 URL
http://iopscience.iop.org/0067-0049/164/1/255
[48] White C J, Stone J M and Gammie C F 2016 Astrophys. J.s 225 22 (Preprint 1511.00943)
[49] ohner R 1987 Computer Methods in Applied Mechanics and Engineering 61 323–338
[50] Keppens R, Meliani Z, van Marle A J, Delmont P, Vlasis A and van der Holst B 2012 Journal of Computational
Physics 231 718–744
[51] Orszag S A and Tang C M 1979 Journal of Fluid Mechanics 90 129–143
[52] Koren B 1993 Numerical methods for advection–diffusion problems Notes on numerical fluid mechanics, v.
45 (Braunschweig: Vieweg) ISBN 3528076453
... In this work we extend the force-free magnetodynamics simulations (i.e. in the limit of σ → ∞) of Lyutikov et al. (2017) to SRRMHD for high Lundquist numbers in 2D Minkowski spacetime with the Black Hole Accretion Code (BHAC). BHAC is a multidimensional framework that has been designed to solve the GRMHD equations in arbitrary space-times making use of constrained transport adaptive mesh refinement (AMR; Olivares, Porth & Mizuno 2018). The framework has recently been extended to solve the GRRMHD equations (Ripperda et al., in preparation). ...
... In both cases we use a Total Variation Diminishing Lax-Friedrichs scheme (TVDLF) and we employ a Cada reconstruction scheme (Čada & Torrilhon 2009) to compute the fluxes, and we use an RK integration with a Courant number of 0.4. The performance and accuracy of the schemes is briefly compared in Section 4. The magnetic field is kept solenoidal, obeying equation (5) to round off error by means of the staggered constrained transport scheme of Balsara & Spicer (1999) (see Olivares et al. 2018 for details on the implementation in BHAC). The charge density (6) is obtained by numerically taking either the central or the limited divergence of the evolved electric field. ...
Article
We apply the general relativistic resistive magnetohydrodynamics code bhac to perform a 2D study of the formation and evolution of a reconnection layer in between two merging magnetic flux tubes in Minkowski space–time. Small-scale effects in the regime of low resistivity most relevant for dilute astrophysical plasmas are resolved with very high accuracy due to the extreme resolutions obtained with adaptive mesh refinement. Numerical convergence in the highly non-linear plasmoid-dominated regime is confirmed for a sweep of resolutions. We employ both uniform resistivity and non-uniform resistivity based on the local, instantaneous current density. For uniform resistivity we find Sweet–Parker reconnection, from η = 10⁻² down to η = 10⁻⁴, for a reference case of magnetization σ = 3.33 and plasma-β = 0.1. For uniform resistivity η = 5 × 10⁻⁵ the tearing mode is recovered, resulting in the formation of secondary plasmoids. The plasmoid instability enhances the reconnection rate to |$v$|rec ∼ 0.03c compared to |$v$|rec ∼ 0.01c for η = 10⁻⁴. For non-uniform resistivity with a base level η0 = 10⁻⁴ and an enhanced current-dependent resistivity in the current sheet, we find an increased reconnection rate of |$v$|rec ∼ 0.1c. The influence of the magnetization σ and the plasma-β is analysed for cases with uniform resistivity η = 5 × 10⁻⁵ and η = 10⁻⁴ in a range 0.5 ≤ σ ≤ 10 and 0.01 ≤ β ≤ 1 in regimes that are applicable for black hole accretion discs and jets. The plasmoid instability is triggered for Lundquist numbers larger than a critical value of Sc ≈ 8000.
... We note that modern extensions of HLLE such as the five-wave HLLD method ( complete fan of waves is considered, are not yet implemented in BHAC. The preservation of the no magnetic monopoles constriction is achieved by using the flux constrained transport method (for more details see Olivares Sánchez et al. 2018). Primitive variables are recovered using the inversion technique 2DW from Noble et al. (2006). ...
Preprint
We build equilibrium solutions of magnetised thick discs around a highly spinning Kerr black hole and evolve these initial data up to a final time of about 100 orbital periods. The numerical simulations reported in this paper solve the general relativistic magnetohydrodynamics equations using the BHAC code and are performed in axisymmetry. Our study assumes non-self-gravitating, polytropic, constant angular momentum discs endowed with a purely toroidal magnetic field. In order to build the initial data we consider three approaches, two of which incorporate the magnetic field in a self-consistent way and a third approach in which the magnetic field is included as a perturbation on to an otherwise purely hydrodynamical solution. To test the dependence of the evolution on the initial data, we explore four representative values of the magnetisation parameter spanning from almost hydrodynamical discs to very strongly magnetised tori. The initial data are perturbed to allow for mass and angular momentum accretion on to the black hole. Notable differences are found in the long-term evolutions of the initial data. In particular, our study reveals that highly magnetised discs are unstable, and hence prone to be fully accreted and expelled, unless the magnetic field is incorporated into the initial data in a self-consistent way.
Article
Context . Worldwide very long baseline radio interferometry (VLBI) arrays are expected to obtain horizon-scale images of supermassive black hole candidates and of relativistic jets in several nearby active galactic nuclei. This, together with the expected detection of electromagnetic counterparts of gravitational-wave signals, motivates the development of models for magnetohydrodynamic flows in strong gravitational fields. Aims . The Black Hole Accretion Code ( BHAC ) is a publicliy available code intended to aid with the modeling of such sources by performing general relativistic magnetohydrodynamical simulations in arbitrary stationary spacetimes. New additions to the code are required in order to guarantee an accurate evolution of the magnetic field when small and large scales are captured simultaneously. Methods . We discuss the adaptive mesh refinement (AMR) techniques employed in BHAC , which are essential to keep several problems computationally tractable, as well as staggered-mesh-based constrained transport (CT) algorithms to preserve the divergence-free constraint of the magnetic field. We also present a general class of prolongation operators for face-allocated variables compatible with them. Results . After presenting several standard tests for the new implementation, we show that the choice of the divergence-control method can produce qualitative differences in the simulation results for scientifically relevant accretion problems. We demonstrate the ability of AMR to decrease the computational costs of black hole accretion simulations while sufficiently resolving turbulence arising from the magnetorotational instability. In particular, we describe a simulation of an accreting Kerr black hole in Cartesian coordinates using AMR to follow the propagation of a relativistic jet while self-consistently including the jet engine, a problem set up for which the new AMR implementation is particularly advantageous. Conclusions . The CT methods and AMR strategies discussed here are currently being used in the simulations performed with BHAC for the generation of theoretical models for the Event Horizon Telescope collaboration.
Preprint
Worldwide very long baseline radio interferometry arrays are expected to obtain horizon-scale images of supermassive black hole candidates as well as of relativistic jets in several nearby active galactic nuclei. This motivates the development of models for magnetohydrodynamic flows in strong gravitational fields. The Black Hole Accretion Code (BHAC) intends to aid with the modelling of such sources by means of general relativistic magnetohydrodynamical (GRMHD) simulations in arbitrary stationary spacetimes. New additions were required to guarantee an accurate evolution of the magnetic field when small and large scales are captured simultaneously. We discuss the adaptive mesh refinement (AMR) techniques employed in BHAC, essential to keep several problems computationally tractable, as well as staggered-mesh-based constrained transport (CT) algorithms to preserve the divergence-free constraint of the magnetic field, including a general class of prolongation operators for face-allocated variables compatible with them. Through several standard tests, we show that the choice of divergence-control method can produce qualitative differences in simulations of scientifically relevant accretion problems. We demonstrate the ability of AMR to reduce the computational costs of accretion simulations while sufficiently resolving turbulence from the magnetorotational instability. In particular, we describe a simulation of an accreting Kerr black hole in Cartesian coordinates using AMR to follow the propagation of a relativistic jet while self-consistently including the jet engine, a problem set up-for which the new AMR implementation is particularly advantageous. The CT methods and AMR strategies discussed here are being employed in the simulations performed with BHAC used in the generation of theoretical models for the Event Horizon Telescope Collaboration.
Article
Full-text available
We present the black hole accretion code (BHAC), a new multidimensional general-relativistic magnetohydrodynamics module for the MPI-AMRVAC framework. BHAC has been designed to solve the equations of ideal general-relativistic magnetohydrodynamics in arbitrary spacetimes and exploits adaptive mesh refinement techniques with an efficient block-based approach. Several spacetimes have already been implemented and tested. We demonstrate the validity of BHAC by means of various one-, two-, and three-dimensional test problems, as well as through a close comparison with the HARM3D code in the case of a torus accreting onto a black hole. The convergence of a turbulent accretion scenario is investigated with several diagnostics and we find accretion rates and horizon-penetrating fluxes to be convergent to within a few percent when the problem is run in three dimensions. Our analysis also involves the study of the corresponding thermal synchrotron emission, which is performed by means of a new general-relativistic radiative transfer code, BHOSS. The resulting synthetic intensity maps of accretion onto black holes are found to be convergent with increasing resolution and are anticipated to play a crucial role in the interpretation of horizon-scale images resulting from upcoming radio observations of the source at the Galactic Center.
Article
Full-text available
We present a constrained transport (CT) algorithm for solving the 3D ideal magnetohydrodynamic (MHD) equations on a moving mesh, which maintains the divergence-free condition on the magnetic field to machine-precision. Our CT scheme uses an unstructured representation of the magnetic vector potential, making the numerical method simple and computationally efficient. The scheme is implemented in the moving mesh code Arepo. We demonstrate the performance of the approach with simulations of driven MHD turbulence, a magnetized disc galaxy, and a cosmological volume with primordial magnetic field. We compare the outcomes of these experiments to those obtained with a previously implemented Powell divergence-cleaning scheme. While CT and the Powell technique yield similar results in idealized test problems, some differences are seen in situations more representative of astrophysical flows. In the turbulence simulations, the Powell cleaning scheme artificially grows the mean magnetic field, while CT maintains this conserved quantity of ideal MHD. In the disc simulation, CT gives slower magnetic field growth rate and saturates to equipartition between the turbulent kinetic energy and magnetic energy, whereas Powell cleaning produces a dynamically dominant magnetic field. Such difference has been observed in adaptive-mesh refinement codes with CT and smoothed-particle hydrodynamics codes with divergence-cleaning. In the cosmological simulation, both approaches give similar magnetic amplification, but Powell exhibits more cell-level noise. CT methods in general are more accurate than divergence-cleaning techniques, and, when coupled to a moving mesh can exploit the advantages of automatic spatial/temporal adaptivity and reduced advection errors, allowing for improved astrophysical MHD simulations.
Article
Full-text available
We have performed two-dimensional special-relativistic magnetohydrodynamic simulations of non-equilibrium over-pressured relativistic jets in cylindrical geometry. Multiple stationary recollimation shock and rarefaction structures are produced along the jet by the nonlinear interaction of shocks and rarefaction waves excited at the interface between the jet and the surrounding ambient medium. Although initially the jet is kinematically dominated, we have considered axial, toroidal and helical magnetic fields to investigate the effects of different magnetic-field topologies and strengths on the recollimation structures. We find that an axial field introduces a larger effective gas-pressure and leads to stronger recollimation shocks and rarefactions, resulting in larger flow variations. The jet boost grows quadratically with the initial magnetic field. On the other hand, a toroidal field leads to weaker recollimation shocks and rarefactions, modifying significantly the jet structure after the first recollimation rarefaction and shock. The jet boost decreases systematically. For a helical field, instead, the behaviour depends on the magnetic pitch, with a phenomenology that ranges between the one seen for axial and toroidal magnetic fields, respectively. In general, however, a helical magnetic field yields a more complex shock and rarefaction substructure close to the inlet that significantly modifies the jet structure. The differences in shock structure resulting from different field configurations and strengths may have observable consequences for disturbances propagating through a stationary recollimation shock.
Article
Full-text available
We present a new numerical tool for solving the special relativistic ideal MHD equations that is based on the combination of the following three key features: (i) a one-step ADER discontinuous Galerkin (DG) scheme that allows for an arbitrary order of accuracy in both space and time, (ii) an a posteriori subcell finite volume limiter that is activated to avoid spurious oscillations at discontinuities without destroying the natural subcell resolution capabilities of the DG finite element framework and finally (iii) a space-time adaptive mesh refinement (AMR) framework with time-accurate local time-stepping. The divergence-free character of the magnetic field is instead taken into account through the so-called 'divergence-cleaning' approach. The convergence of the new scheme is verified up to 5th order in space and time and the results for a sample of significant numerical tests including shock tube problems, the RMHD rotor problem and the Orszag-Tang vortex system are shown. We also consider a simple case of the relativistic Kelvin-Helmholtz instability with a magnetic field, emphasizing the potential of the new method for studying turbulent RMHD flows. We discuss the advantages of our new approach when the equations of relativistic MHD need to be solved with high accuracy within various astrophysical systems.
Article
Full-text available
In the extreme violence of merger and mass accretion, compact objects like black holes and neutron stars are thought to launch some of the most luminous outbursts of electromagnetic and gravitational wave energy in the Universe. Modeling these systems realistically is a central problem in theoretical astrophysics, but has proven extremely challenging, requiring the development of numerical relativity codes that solve Einstein's equations for the spacetime, coupled to the equations of general relativistic (ideal) magnetohydrodynamics (GRMHD) for the magnetized fluids. Over the past decade, the Illinois Numerical Relativity (ILNR) Group's dynamical spacetime, GRMHD code has proven itself as one of the most robust and reliable tools for theoretical modeling of such GRMHD phenomena. Despite the code's outstanding reputation, it was written "by experts and for experts" of the code, with a steep learning curve that would severely hinder community adoption if it were open-sourced. Here we present IllinoisGRMHD, which is an open-source, highly-extensible rewrite of the original closed-source GRMHD code of the ILNR Group. Reducing the learning curve was the primary focus of this rewrite, facilitating community involvement in the code's use and development, as well as the minimization of human effort in generating new science. IllinoisGRMHD also saves computer time, generating roundoff-precision identical output to the original code on adaptive-mesh grids, but nearly twice as fast at scales of hundreds to thousands of cores.
Article
We present a new general relativistic magnetohydrodynamics (GRMHD) code integrated into the Athena++ framework. Improving upon the techniques used in most GRMHD codes, ours allows the use of advanced, less diffusive Riemann solvers, in particular HLLC and HLLD. We also employ a staggered-mesh constrained transport algorithm suited for curvilinear coordinate systems in order to maintain the divergence-free constraint of the magnetic field. Our code is designed to work with arbitrary stationary spacetimes in one, two, or three dimensions, and we demonstrate its reliability through a number of tests. We also report on its promising performance and scalability. © 2016. The American Astronomical Society. All rights reserved.
Article
Context. In many astrophysical phenomena, and especially in those that involve the high-energy regimes that always accompany the astronomical phenomenology of black holes and neutron stars, physical conditions that are achieved are extreme in terms of speeds, temperatures, and gravitational fields. In such relativistic regimes, numerical calculations are the only tool to accurately model the dynamics of the flows and the transport of radiation in the accreting matter. Aims. We here continue our effort of modelling the behaviour of matter when it orbits or is accreted onto a generic black hole by developing a new numerical code that employs advanced techniques geared towards solving the equations of general-relativistic hydrodynamics. Methods. More specifically, the new code employs a number of high-resolution shock-capturing Riemann solvers and reconstruction algorithms, exploiting the enhanced accuracy and the reduced computational cost of adaptive mesh-refinement (AMR) techniques. In addition, the code makes use of sophisticated ray-tracing libraries that, coupled with general-relativistic radiation-transfer calculations, allow us to accurately compute the electromagnetic emissions from such accretion flows. Results. We validate the new code by presenting an extensive series of stationary accretion flows either in spherical or axial symmetry that are performed either in two or three spatial dimensions. In addition, we consider the highly nonlinear scenario of a recoiling black hole produced in the merger of a supermassive black-hole binary interacting with the surrounding circumbinary disc. In this way, we can present for the first time ray-traced images of the shocked fluid and the light curve resulting from consistent general-relativistic radiation-transport calculations from this process. Conclusions. The work presented here lays the ground for the development of a generic computational infrastructure employing AMR techniques to accurately and self-consistently calculate general-relativistic accretion flows onto compact objects. In addition to the accurate handling of the matter, we provide a self-consistent electromagnetic emission from these scenarios by solving the associated radiative-transfer problem. While magnetic fields are currently excluded from our analysis, the tools presented here can have a number of applications to study accretion flows onto black holes or neutron stars.
Article
We present a high order one-step ADER–WENO finite volume scheme with space–time adaptive mesh refinement (AMR) for the solution of the special relativistic hydrodynamic and magnetohydrodynamic equations. By adopting a local discontinuous Galerkin predictor method, a high order one-step time discretization is obtained, with no need for Runge–Kutta sub-steps. This turns out to be particularly advantageous in combination with space–time adaptive mesh refinement, which has been implemented following a “cell-by-cell” approach. As in existing second order AMR methods, also the present higher order AMR algorithm features time-accurate local time stepping (LTS), where grids on different spatial refinement levels are allowed to use different time steps.