Prediction Performance & Compilation Guide¶

This guide details the recommended toolchains, compiler options, and runtime settings required to achieve maximum execution speed and scaling efficiency across different hardware architectures.

Simulation Toolchain & Parallelization Strategy¶

The choice of simulation engine and parallelization scheme heavily impacts scaling, particularly as the physical size of your system changes.

Toolchain Selection¶

Production / Large-Scale 2G-HDNNP Runs: Always use the LAMMPS toolchain utilizing MPI domain decomposition. This configuration provides the highest throughput and optimal memory distribution for large-scale production simulations of fully local 2G-HDNNP models.
Prototyping / Small Structures: The ASE (Atomic Simulation Environment) interface delivers comparable performance to LAMMPS for small atomic structures. However, ASE does not preform domain decomposition and hence has a worse performance for high core counts and large structures. It does not have a MPI parallelization, but uses RuNNer internal OpenMP parallelization.

Hybrid Parallelization Scheme (MPI + OpenMP)¶

For LAMMPS simulations, a hybrid parallelization strategy is highly recommended to maximize CPU utilization and minimize communication overhead:

Sizing Recommendations: For system sizes starting at a couple hundred atoms, configure your environment to use 8 to 16 OpenMP threads per MPI process.
Execution Rule: OpenMP parallelization should always work in tandem with MPI parallelization; do not rely on OpenMP exclusively.

Exception: 4G-HDNNP

For 4G-HDNNP, always favor OpenMP parallelization over MPI.

Hardware-Specific Toolchains & Tool Selection¶

Performance is highly dependent on matching your compiler, optimization flags, and mathematical libraries to your underlying processor architecture.

AMD Hardware¶

Compiler: Use the most recent GNU compilers (gcc, g++, gfortran), minimum version GCC 14.3. Older compilers are supported but may not fully exploit the latest AMD performance features, so upgrading to the latest stable release is recommended for best results.
Architecture Tuning: Optimize explicitly for the host machine's architecture by setting:
```
# make compilation
make ARCH="-march=native"
```

Critical Threading Configuration

When mixing GCC and MKL, you must explicitly enforce the GNU threading layer to avoid runtime collisions or deadlocks. Ensure your environment and library linking matches this setup:

setenv("MKL_THREADING_LAYER", "GNU")

When building, explicitly link the GNU threading library variant (libmkl_gnu_thread.so) and OpenMP (libgomp.so.1) instead of Intel's defaults.

Intel Hardware¶

Compiler: Use the most recent Intel OneAPI compilers (mpiicx, mpiicpx, mpiifx), ideally version >= 2025.3.
Architecture Tuning: Target Intel performance features by specifying the AVX-512 instruction set:
```
# make compilation
make ARCH="core-avx512"
```

C++ Compatibility Fix

To ensure compatibility with certain C++ libraries and prevent compilation errors, you should append msse4.1 (or modern msse4.2) flags:

# make compilation
make ARCH="core-avx512 -msse4.2"

Threading Layer: Configure the environment to use native Intel threading:
```
setenv("MKL_THREADING_LAYER", "INTEL")
```

Math & Transform Libraries¶

Core BLAS/LAPACK: Always link against Intel Math Kernel Library (MKL). It is critical for best performance to use modern releases—ideally MKL 2026 or, at the absolute minimum, MKL 2025.2/2025.3. Avoid the netlib reference implementations, as they will not provide the necessary performance for large-scale simulations.
Electrostatics (3G and 4G): Ensure that MKL FFTW is utilized for fast Fourier transforms (RuNNer: MKL=ON, LAMMPS: -D FFT=MKL). While this wrapper is typically selected automatically by the build system, explicitly verify its inclusion during the configuration phase to guarantee peak electrostatics calculation speeds.

Compilation Flags & Build Optimization¶

To ensure the compiler generates the most streamlined machine instructions, apply the following build configurations inside your GNUMakefile or build scripts.

Performance Flags: Enable link-time optimization (-flto) and ensure aggressive optimization flags are used (-O3):
```
# make compilation
make OPT="-O3 -flto"
```
Do NOT include flags such as -g / debug symbols!
Macro Configurations: Ensure that all debug symbols and performance-throttling prefactors are disabled. Do NOT use FEATURES=ENABLE_PREFACTORS and NOT FEATURES=ALL. For optimal feature construction, select FEATURES=FC_NO_INNER_CUTOFF.

Runtime Settings¶

Electrostatics Solvers: Fine-tuning solver combinations based on the spatial scale of your simulation dramatically reduces computational complexity. The optimal solver is determined by the total number of atoms and simulation cell dimensions:

System Size	Recommended Solver	Notes
Small Systems	Ewald / Direct Solver	Fast for localized systems; exact performance crossover point should be benchmarked on your specific cluster.
Large Systems	PW / CG (Plane Wave / Conjugate Gradient)	Highly scalable; significantly reduces memory and compute scaling limits on massive configurations.

Symmetry Functions: Do not disable symmetry function groups. Keeping symmetry function groups active allows the code to exploit spatial redundancies, significantly reducing the number of explicit calculations required per timestep. Try to reduce the number of symmetry functions (especially angular ones) if possible, since their evaluation is the most expensive part of a 2G-HDNNP.