Running C++ Numerical Simulations in Docker
Introduction: Docker and Numerical Simulations
You may have heard of Docker before. The wikipedia gives the definition of Docker as “a set of platform as a service (PaaS) products that use OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files; they can communicate with each other through well-defined channels. All containers are run by a single operating system kernel and therefore use fewer resources than virtual machines.” Docker has been widely used in industry.
On the other hand, let me brief talk about numerical simulations. In science, many physical and chemical processes can be described and predicted by a set of differential equations. A numerical simulation is to numerical solve differential equations that involve continuous systems in astrophysics, fluid dynamics, and continuum mechanics. These systems are too complex to be described by analytical solutions, so numerical simulations should be carried out to study the systems.
In academia, numerical simulations are usually performed on supercomputers or local clusters. Since Docker has become more and more popular, one question we would like to ask is, can we directly run numerical simulations in Docker containers instead of supercomputers/clusters? The short answer is yes.
This article provides a method to deploy numerical simulations in Docker containers. I am using a magnetohydrodynamic (MHD) code called Athena++ as an example, to deploy Athena++ on a Docker container.
What is Athena++
To better understand what are numerical simulations, I would like to use an astrophysical magnetohydrodynamic (MHD) code called Athena++ as an example. You can find the introduce and the public version of the simulation code on github, or a general introduction on this website.
The code supports the multi-dimensional MHD simulations of compressible gaseous systems, with physical processes described by special and general relativistic hydrodynamics.
Here is one example of using Athena++ to simulate turbulent gas in a three-dimensional box:
The above video shows the density evolution of a two-dimensional slide of a three-dimensional turbulent gaseous system. The system is initially uniform with a random velocity field, and I used Athena++ to solve a set of partial differential equation (PDE) to model the turbulent motion in the gas.
Note that the Athena++ code was written in C++, and configured by python. In order to run it in a Docker container, we need to first know how to run C++ code in Docker.
A “Hello World” run is given below.
Run you First C++ Application in Docker
I recommend the tutorial on this website or video. In an empty directory create two files: a C++ file and a Dockerfile. The C++ file helloworld.cpp is written as:
#include <iostream>int main(int argc, char const *argv[])
{
std::cout << “Hello Docker container!” << std::endl;
return 0;
}
For the local test we can use g++ compiler to run the following command in this directory
g++ -o test helloworld.cpp
./testThe output result on the terminal is:
Hello Docker container!Now we would like to run helloworld.cpp in a Docker container. I use the following Dockerfile to build the Docker container:
FROM gcc:latest COPY . /usr/src/cpp_test WORKDIR /usr/src/cpp_test RUN g++ -o test helloworld.cpp CMD ["./test"]
In this Dockerfile, I load the latest version of gcc (FROM), copy the .cpp file to the Docker image’s directory structure /usr/src/cpp_test (COPY), set up the the work directory to /usr/src/cpp_test (WORKDIR), compiles the .cpp file into an executable (RUN), and finally use CMD to run the image.
Now I can build a Docker container image by docker build:
docker build . -t cpp_test:1After the Docker container image is ready, I can run the “HelloWorld” application in the container by docker run:
docker run --rm -it cpp_test:1If everything is working, one will see the “Hello Docker container!” output on the terminal. Here — rm is used to terminate the Docker container after running the application.
Run your First Numerical Simulation with Athena++
Next we move forward to run Athena++ locally. To download the simulation code, using the git clone:
git clone https://github.com/PrincetonUniversity/athena-public-versionI rename athena-public-version to athena. Go to the directory /athena as the default directory.
The tutorial of Athena++ shows some very typical simulation problems:
In this blog I would like to look into the first problem: to simulate shock wave in a one-dimensional (1D) tube (the shock tube problem).
The instruction gives a step by step tutorial to run the 1D shock tube problem. In the default directory, first configure the code by:
python configure.py --prob shock_tubeThe following configured results give the detailed features and options of the simulation. For your first simulation, you can skip most of them, just focus on Problem generator: shock_tube, which identify the simulation problem as shock_tube, Magnetic fields: OFF, which means magnetic fields are ignored in this problem, so this is a hydrodynamic simulations, and Compilation command: g++, which means the code will be compiled and run by g++.
Problem generator: shock_tube
Coordinate system: cartesian
Equation of state: adiabatic
Riemann solver: hllc
Magnetic fields: OFF
Number of scalars: 0
Special relativity: OFF
General relativity: OFF
Frame transformations: OFF
Self-Gravity: OFF
Super-Time-Stepping: OFF
Debug flags: OFF
Code coverage flags: OFF
Linker flags:
Floating-point precision: double
Number of ghost cells: 2
MPI parallelism: OFF
OpenMP parallelism: OFF
FFT: OFF
HDF5 output: OFF
Compiler: g++
Compilation command: g++ -O3 -std=c++11Next clean up the old complier in the code and build a new one:
make clean
make allAfter code is complied, an executable file athena will be generated in the directory athena/bin.
Create a work directory, and copy the input file, which gives the initial and boundary condition to solve the set of differential equations of the 1D shock tube:
mkdir work
cd work
cp ../inputs/hydro/athinput.sod .The following step is to run the executable file, and generate output files in the work directory for further analysis and visualization.
../bin/athena -i athinput.sodYou can see the code is running in your terminal and eventually the simulation gives:
Terminating on time limittime=2.5000000000000000e-01 cycle=175
tlim=2.5000000000000000e-01 nlim=-1zone-cycles = 44800cpu time used = 3.1884000000000003e-02
zone-cycles/cpu_second = 1.4050934638062976e+06
I will visualize and analyze the result later. Before doing that I would like to introduce how to run the same simulation in a Docker container.
Run Your First Numerical Simulation in Docker
Note that for Athena++ we need to first configure it using python, then run the C++ code. The Dockerfile is written as:
FROM gcc:latest
FROM python:3.7 COPY . /usr/src/athena WORKDIR /usr/src/athena
RUN python configure.py --prob shock_tube -b
RUN make clean
RUN make all WORKDIR /usr/src/Athena/work
ENTRYPOINT ["../bin/athena","-i","athinput.sod"]
In the Dockerfile, I load both gcc and python, copy all local files (in the directory of Athena++) to the Docker directory /usr/src/athena, set up the work directory in Docker as /usr/src/athena, configure the code, clean and make the code again, then change the work directory to /usr/src/Athena/work. Note that in the last line I use ENTRYPOINT to combine the command “../bin/athena -i athinput.sod”.
Build a Docker container image:
docker build . -t athena:1and run the Athena++ application by
docker run -it athena:1If everything is working perfect, you can see the Athena++ code is complied in Docker and shock tube simulation is launched.
The next step is to copy the output files from the Docker container to local default directory. Note that I did not use “ — rm” in the above command, and you can search the Docker container ID by
docker ps -aand see the container which just completed the simulation:
In order to copy files from the container to local directory, I use the following command
docker cp ad1f2db17b00:/usr/src/athena/work .Now the work folder in the local Athena++ directory has all the output files, including all .tab files, a .hst file, and the simulation input file athinput.sod.
Data Visualization
Each .tab file gives a snapshot of the shock density, pressure, and velocity as a function of x-axis position at some time.
I select four snapshots to visualize the evolution of the shock tube.
• Sod.block0.out1.00000.tab (t=0),
• Sod.block0.out1.00008.tab (t=8),
• Sod.block0.out1.00016.tab (t=16),
• Sod.block0.out1.00024.tab (t=24).
The density evolution of the shock tube at four time snapshots is given as follows:
The code to plot the above figure:
fig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(nrows=2, ncols=2, figsize=(10, 8))df = pd.read_fwf('Sod.block0.out1.00000.tab',skiprows=1)
ax1.plot(df.x1v, df.rho, 'b', linewidth = 3); ax1.set_title('t=0')
ax1.set_ylabel('Density', fontsize=12)df = pd.read_fwf('Sod.block0.out1.00008.tab',skiprows=1)
ax2.plot(df.x1v, df.rho, 'b', linewidth = 3); ax2.set_title('t=8')
ax2.set_ylabel('Density', fontsize=12)df = pd.read_fwf('Sod.block0.out1.00016.tab',skiprows=1)
ax3.plot(df.x1v, df.rho, 'b', linewidth = 3); ax3.set_title('t=16')
ax3.set_xlabel('shock tube', fontsize=12); ax3.set_ylabel('Density', fontsize=12)df = pd.read_fwf('Sod.block0.out1.00024.tab',skiprows=1)
ax4.plot(df.x1v, df.rho, 'b', linewidth = 3); ax4.set_title('t=24')
ax4.set_xlabel('shock tube', fontsize=12); ax4.set_ylabel('Density', fontsize=12)plt.show()
Note that the x-axis is the shock propagation direction, and y-axis shows the density profile. Initially the shock has Density=1 at x<0 and Density = 0.125 at x>0. In order to better understand the shock evolution, I also plot the evolution of pressure and velocity in the shock tube:
Run Another Numerical Simulation in Docker
I have shown the evolution of a 1D shock tube simulation by running the Athena++ code. How about adding magnetic fields, and investigate the evolution of magnetized shock tube?
Here is the instruction of running a magnetized shock tube simulation. In the Dockerfile I write:
FROM gcc:latest
FROM python:3.7COPY . /usr/src/athenaWORKDIR /usr/src/athena
RUN python configure.py --prob shock_tube -b
RUN make clean
RUN make allWORKDIR /usr/src/Athena/work
ENTRYPOINT ["../bin/athena","-i","athinput.bw"]
Note that to configure the code I use
python configure.py — prob shock_tube -bI call the problem “shock_tube” with magnetic field “-b”. The simulation initial/boundary condition with magnetic field is in athinput.bw. I copy this file to the work directory.
Then do similar steps as mentioned above:
(1) Build a new docker container image
docker build . -t athena:2(2) Run Athena++ in the container by
docker run -it athena:2(3 ) Copy the simulation output files into your local Athena++ directory (replace “Container ID” by your container ID information)
docker cp Container ID:/usr/src/athena/work .Simulation Output Visualization
The evolution of shock density:
The density evolution looks more complex — that is because of the magnetic field. One can see the evolution of the magnetic field in the shock tube:
Note that magnetic field is initially set up as 1, and it evolves with the shock tube.
The pressure in the shock tube:
More information of shock tube can be seen here.
Summary
Numerical simulations are widely used to model physical and chemical processes in continuous systems. We can run numerical simulations in Docker containers instead of local clusters or supercomputers. This article gives an example to run simulation in Docker using the Athena++ code, which is a C++ code and can be used to perform multi-dimensional hydrodynamic and magnetohydrodynamic simulations. Taking shock tube with and without magnetic fields as examples, I demonstrate how to run the simulations in a Docker container, and export results for visualization. The method can be generalized to run other simulations in Docker containers.
More details of astrophysical and cosmological simulations can be found here.
My GitHub for data visualization is here.
Find me on LinkedIn.
