Devito: Symbolic Finite Difference Computation

Devito is a Domain-specific Language (DSL) and code generation framework for the design of highly optimised finite difference kernels for use in inversion methods. Devito utilises SymPy to allow the definition of operators from high-level symbolic equations and generates optimised and automatically tuned code specific to a given target architecture.

Symbolic computation is a powerful tool that allows users to:

  • Build complex solvers from only a few lines of high-level code
  • Use automated performance optimisation for generated code
  • Adjust stencil discretisation at runtime as required
  • (Re-)development of solver code in hours rather than months

Documentation

Documentation for Devito is available here, including installation instructions, a set of tutorials and API documentation. In addition, a paper outlining the use of symbolic Python to define finite difference operators in Devito can be found here. Devito is a fast moving project so some of the documentation may lag behind development. Feel free to talk to us on slack if you have questions - PR’s are also welcome.

from devito import *

grid = Grid(shape=(nx, ny))
u = TimeFunction(name='u', grid=grid,
                 space_order=2)
u.data[0, :] = initial_data[:]

eqn = Eq(u.dt, a * (u.dx2 + u.dy2))
stencil = solve(eqn, u.forward)
op = Operator(Eq(u.forward, stencil))
op(t=timesteps, dt=dt)

Example code for a 2D diffusion operator from a symbolic definition. The full tutorial can be found here. To get more familiar with Devito, we provide a wide selection of tutorials.

Seismic Inversion using Devito

Devito is primarily designed to create wave propagation kernels for use in seismic inversion problems. A tutorial for the generation of a modelling operator using an acoustic wave equation can be found here and a paper outlining the verification procedures of the acoustic operator can be found here.

True velocity model (Marmousi-ii)

Initial velocity model for FWI

FWI inverted velocity model

DualShot

Optimisation and Performance

Devito provides a set of automated performance optimizations during code generation that allow user applications to fully utilise the target hardware without changing the model specification:

  • Vectorisation (via OpenMP)
  • Shared-memory parallelism (via OpenMP), including nested parallelism and non-affine loop support
  • Loop blocking, including hierarchical blocking
  • Auto-tuning (e.g., block-shape, threads per parallel region)
  • Symbolic optimisations:
    • Common sub-expression elimination (CSE)
    • Cross-iteration redundancy elimination (CIRE)
    • Expression hoisting
    • Factorization

Devito also supports distributed-memory parallelism via MPI. Several halo-exchange schemes are available; classic optimisations such as computation-communication overlap (relying on asynchronous progress engine) are implemented.

skl8180

Performance of acoustic wave modelling operator with different stencil sizes and auto-tuning on single-socket E5-2697 v4 CPU (Broadwell, 16 cores @ 2.3GHz).