Short-range energy training with 2G-HDNNPs¶

Training a second-generation HDNNP (2G-HDNNP) is the “standard” workflow in RuNNer: you provide a dataset (input.data), specify the network architecture + training settings (input.nn), define atom-centered symmetry functions (ACSFs), and run RuNNer in train mode.

Note

RuNNer can also train models with additional targets (forces, charges, long-range electrostatics, etc.).
This template keeps the configuration minimal and energy-only, but the dataset example may contain extra columns that are simply ignored unless you enable the corresponding targets in input.nn.

Prerequisites¶

A working RuNNer binary. Take a look at our installation instructions.
A dataset of structures formatted as input.data.
a text editor like vim or emacs to edit the settings file input.nn.
A folder where RuNNer can write output (weights, scaling, split files, logs).

Tip

Whenever you start working with on a new project, start with a small dataset (10–100 structures) to verify formatting, units, and that the run completes. Then scale up. This is especially helpful on HPC clusters!

Minimal directory layout¶

A typical training directory might look like:

train/
├── input.data
├── input.nn

Input files¶

`input.data`¶

The dataset file is a concatenation of structures, each enclosed by begin ... end. A minimal structure includes lattice (for periodic systems), atoms, and a total energy.

Example (truncated):

begin
lattice  14.78219400   0.00000000   0.00000000
lattice   0.00000000  14.78219400   0.00000000
lattice   0.00000000   0.00000000  14.78219400
atom    11.224971985  7.351033775  13.983973226  O  -0.20580917  0.0  -0.017791036  -0.027353819  -0.012263381
atom    10.847026609  5.480206056  13.946178689  H   0.13862518  0.0   0.006366116   0.028821136   0.002123689
atom     9.618706028  8.182513601  13.398157894  H   0.11655894  0.0   0.006205769  -0.000937695   0.00506912
[...]
energy  -1225.06168332
charge  0.0
end

An atom line always starts with positions and element:

atom x y z element_symbol ...

After the element symbol, RuNNer can accept additional per-atom properties depending on what you train (e.g., atomic charges, spins, forces). This has to be announced to the code in the begin line. The example above shows the default format, with the atomic charges, atomic energies, and atomic forces. Take a look at the input.data documentation for more details.

Warning

The most common source of early failure are wrong units. All RuNNer input files expect atomic units, which means

positions and lattice vectors: Bohr
energies: Hartree
forces: Hartree/Bohr

More details are given here.

`input.nn`¶

This file controls the training run and model definition. It also includes the definition of custom descriptor functions. A minimal setup (without descriptors) looks like this:

nnp_generation 2
runner_mode train

elements O H
random_seed 40

default_nodes 20 20
default_activation_nn t t l

test_fraction 0.1

epochs 5

initialization_method uniform

optimizer 1 kalman nue=0.99870 p_initial=1.0 lambda=0.98000
optimizer 2 kalman nue=0.99870 p_initial=1.0 lambda=0.98000
opt_short H 1
opt_short O 2

use_energy

Let's go through it step by step.

nnp_generation 2: selects 2G-HDNNP (local model).
runner_mode train: enables training.
elements O H: must match the chemical species present in input.data.
random_seed 40: seed for the random number generator. By default this is used for all tasks, including splitting the training dataset and weight initialization.
default_nodes 20 20: two hidden layers with 20 neurons each (per atomic neural network).
default_activation_nn t t l: activation functions for each layer (including output layer). Training continuous properties like the total energy usually requires a linear output activation function.
test_fraction 0.1: random split the dataset into 90% train / 10% test.
epochs 5: number of training epochs (increase for real runs).
optimizer ... kalman ...: Kalman filter optimizers (one per element here).
use_energy: use the loss of the total energy per structure for updating the model parameters.

Note

RuNNer supports multiple optimizers and flexible assignment via opt_short. Using separate optimizers per element can sometimes stabilize training if one species dominates the dataset. Look below for a setup where all elements share the same optimizer.

Training with Forces

including forces in the training with the keyword use_forces is highly recommended. The forces contains vital information about the curvature of the potential energy surface that the energy value alone cannot provide.

Generation of atom-centered symmetry functions¶

An important part of a 2G-HDNNP are its descriptors. Here, we will use atom-centered symmetry functions (ACSFs). RuNNer will compute ACSFs for each atom and feed them into the element-specific neural networks.

Symmetry functions can be hand-crafted, but we typically use tools like runnerase for generating them automatically for a given reference dataset. Take a look at the documentation of runnerase to learn more.

fc_cosine 1 12.0
symfunction_short H 2 H 0.001 0.0 1
symfunction_short H 2 H 0.01  0.0 1
symfunction_short H 2 H 0.03  0.0 1
symfunction_short H 2 H 0.06  0.0 1
symfunction_short H 2 H 0.15  1.9 1
symfunction_short H 2 H 0.30  1.9 1
symfunction_short H 2 H 0.60  1.9 1
symfunction_short H 2 H 1.50  1.9 1

symfunction_short H 3 O H 0.2   1.0 1.0  1
symfunction_short O 3 H H 0.07  1.0 1.0  1
symfunction_short H 3 O H 0.07  1.0 1.0  1
symfunction_short O 3 H H 0.07 -1.0 1.0  1
symfunction_short H 3 O H 0.07 -1.0 1.0  1

There are three types of keywords in this snippet:

fc_cosine 1 12.0: Specification of a common cutoff function for all following feature maps. In principle, RuNNer supports multiple cutoff functions. We assign a continuous index (starting at 1) and a cutoff radius in Bohr.
symfunction_short H 2 ...: specification of a radial symmetry function (type 2) for the central element hydrogen. Followed by the atomic number of the neighboring atom, and the two hyperparameters \(\eta\) and \(R_{\mathrm{shift}}\). The final argument is the index of the cutoff function that should be used.
symfunction_short H 3 ...: specification of an angular symmetry function (type 3) for the central element hydrogen. Followed by the atomic numbers of two neighboring atoms, and the hyperparameters \(\eta\), \(\lambda\), and \(\zeta\). The final argument is the index of the cutoff function that should be used.

Practical guidance for ACSFs

Include both radial (type 2) and angular (type 3) functions.
Cover relevant distance scales. In dense materials, it may be enough to work with a smaller cutoff radius than when e.g. describing a gas phase reaction.
Avoid an excessively huge set of descriptors at the start: it increases compute and can make early debugging harder. In the beginning, aim for a good compromise between training speed and accuracy.

Running the training¶

From inside your training folder:

# name of the executable may differ depending on how RuNNer is installed
RuNNer.x | tee training.out

During the training, RuNNer will:

read input.data
split the dataset according to the given test_fraction.
compute scaling factors for the given feature maps. This ensures that all inputs to the neural networks fall in the interval [0, 1].
train element-wise networks for epochs epochs.
write weights, scaling, and logs.

Capture STDOUT

RuNNer prints a lot of helpful information to STDOUT during training, like all settings, dataset statistics, and the evolution of cost metrics. As shown in the command above, it is usually advisable to redirect it to a file for later inspection.

Output files¶

RuNNer typically creates:

train_split.data and test_split.data: The randomized split created from input.data.
scaling_short.data: Scaling parameters (min, max, avg) of the feature maps.
opt.weight_short.[atomic_number].out: the model parameters for the epoch with the lowest cost metric. This is automatically determined and somewhat arbitrary. You can change the metric upon which the decision is based (see optimal_epoch_property).
[epoch].weights_short.[atomic_number].out: model parameters after each epoch. The frequency of writing this file can be controlled via the keyword cost_frequency.

What to look at first¶

Does RuNNer report the expected number of structures and atoms?

----------------- Statistical Information about the Train Set ------------------

--------------------------- Total Energy [Ha / atom] ---------------------------
     min          max        average      stddev        range        count    
  -2.55E+01    -2.55E+01    -2.55E+01    2.53E-04     1.39E-03        100     

--------------------------- Total Energy [eV / atom] ---------------------------
     min          max        average      stddev        range        count    
  -6.95E+02    -6.94E+02    -6.94E+02    6.89E-03     3.78E-02        100

Do training (and test errors) decrease over epochs?

                            Energy [meV / atom]
        Epoch Comm.          Train           Test
--------------------------------------------------------------------------------
RMSE        0     1    636366.5831            NaN
--------------------------------------------------------------------------------
RMSE        1     1         7.1363            NaN
UPDATES     1     1            100
TIMING      1      0.00 min
--------------------------------------------------------------------------------
RMSE        2     1         7.1356            NaN
UPDATES     2     1            100
TIMING      2      0.00 min
--------------------------------------------------------------------------------
RMSE        3     1         7.1325            NaN
UPDATES     3     1            100
TIMING      3      0.00 min
--------------------------------------------------------------------------------
RMSE        4     1         7.1193            NaN
UPDATES     4     1            100
TIMING      4      0.00 min
--------------------------------------------------------------------------------
RMSE        5     1         7.0834            NaN
UPDATES     5     1            100
TIMING      5      0.00 min
--------------------------------------------------------------------------------

Warning

If you see exploding errors, NaNs, or immediate divergence:

double-check units
check for outlier structures/energies in your training dataset
reduce model complexity temporarily
adjust optimizer settings
double-check units again (seriously)

Next steps¶

Good performance on cost metrics is a promising sign, but does not always mean a potential will be stable when applied to downstream tasks. Additionally, you may want to:

plot learning curves (train and test error vs epoch).
plot correlation between reference and predicted properties (see write_energy_out). This can help to identify outliers and inspect them.
validate on truly unseen structures, not just a random split.
run a small simulation trajectory to test the stability of your potential.
predict observables like the convex hull (for materials), the radial distribution function (for bulk systems), or spectra to validate your potential.

Advanced keywords¶

Using a single optimizer for all elements¶

If you prefer one optimizer shared across all elements, define a single optimizer and assign every element to it:

optimizer 1 kalman nue=0.99870 p_initial=1.0 lambda=0.98000
opt_short H 1
opt_short O 1

This way, cross-element terms are included in the correlation matrix of the Kalman filter, leading to stronger exchange of information between elements.

`energy_fraction` and `force_fraction`¶

These keywords (defaulting to 1.0) control how much data is used per epoch.

energy_fraction 1.0: Use all structures for energy updates.
force_fraction 0.05: Randomly select 5% of the atoms in the structure batch to perform force updates.

In the source code, force_fraction determines if specific optimization routes (lfrom) are taken. If you use a high fraction (>50%), the code automatically switches to a mode where it calculates derivatives for all neighbors, as this becomes more efficient than selecting specific targets.

`force_training_sum_neighbors_derivatives`¶

This settings influences the number of forward/backward passes that are performed during force training. If is is .true., we sum the gradient contributions of all periodic images of neighbor atoms before pushing them throught the model.

The optimal choice depends on your target system: if it is periodic and rather dense (more neighbors than atoms), it makes sense to sum up their contributions. In this case, we will perform one forward/backward pass per atom in the structure per force component update.

If, on the other hand, you have more atoms than neighbors, it is more useful to perform one forward/backward pass per neighbor explicitely.

Dynamic updates¶

These knobs can be used to influence how the loss is composed and when updates happen:

batchsize_structures 10
energy_fraction 0.9
loss_threshold_factor_energy 0.1

batchsize_structures 10: group ten energy updates together
energy_fraction 0.9: randomly select 90% of all structures in the training dataset for energy updates in each epoch.
loss_threshold_factor_energy 0.1: can be used to gate updates based on loss thresholds

Tip

If you later enable forces (use_forces) or charges (use_charge), revisit these weighting strategies are often beneficial by careful balancing of updates.

Debugging¶

Should something go wrong, here are some things you can try:

Increase verbosity to get more detailed logging in runner.out.
```
verbosity 2 # default is 1
```
If you suspect input formatting issues, reduce your dataset to a single structure and confirm RuNNer parses it.
If memory becomes a problem, adjust max_ram_size, or the symmetry function count.