Skip to content

Learning arbitrary atomic properties with 3G-HDNNPs

RuNNer supports training any scalar atomic property. This functionality is part of third generation models, since environment-dependent atomic properties are usually the basis for calculating long-range energy contributions like electrostatics or dispersion.

In RuNNer you may describe atomic properties with two different types of models: - environment-dependent HDNNP: predict a unique value for each atom based on it's atomic environment. - non environment-dependent elemental model: predict one value that is used for all atoms of that element.

No matter which you choose, the workflow for training is always the same.

Tip

We often use this feature to predict properties as the basis for the description of long-range interactions. For example

It has also been used to predict additional physical properties like the spin1.

Training

Preparation

Starting out, you only need two files:

input.data

The input.data file must contain reference values for the property for all atoms in your dataset.

RuNNer does not require the target property to have a predefined name. Instead, the name is defined in the begin line of each structure. Take a look at the input.data documentation for details.

Warning

If some structures in your input.data file do not contain the required training target columns it will lead to errors. Remove these structures before training. The target property name must always be the same in all begin lines.

As an example, here is a structure that is ready for learning the effective Hirshfeld volume hirshv:

begin position(3) element charges hirshv forces(3)
lattice    24.62351     0.00000     0.00000
lattice     0.00000    24.62351     0.00000
lattice     0.00000     0.00000    24.62351
atom     5.24652     3.39306     3.62737 H      0.12095     0.72938     0.00561     0.00002     0.00504
atom     3.55868     2.03894     1.62816 H      0.12762     0.74431     0.00018     0.00213     0.02287
atom     4.19705    22.52290    15.07469 H      0.13295     0.68801    -0.00176     0.00395    -0.01883
[...]

input.nn

Here is what a minimal input.nn file looks like for training the effective atomic Hirshfeld volume.

# General.
nnp_generation 3
runner_mode train
elements O H
random_seed 42
max_ram_size 10000

# Model architecture.
default_nodes 10 10
default_activation_nn t t l

# Training settings.
# Use milli arbitrary units since effective Hirshfeld volume is unitless.
train_atomic_properties hirshv hdnn 1000 mau
epochs 5
test_fraction 0.1
initialization_method xavier eckhoff

# Optimizer settings.
optimizer 1 kalman
opt_hirshv 1

# Define batching.
batchsize_structures 5
batchsize_elements  H 16
batchsize_elements O 8

# Feature map settings.
feature_map_default
center_feature_maps
scale_feature_maps
fc_cosine 1 0.0 11.33835676

symfunction H 2 H 0.001 0.0 1
[...]

Most of these settings are boilerplate. If you do not know what they mean yet, take a look at the Short-range energy training with 2G-HDNNPs tutorial first.

The central line in this file is 

train_atomic_properties hirshv hdnn 1000 mau
The keyword train_atomic_properties defines

  1. the name of the target property. This can be freely chosen, but must match the name specified in the input.data begin line.
  2. the type of model to be used (hdnn or elemental).
  3. The conversion factor for the cost calculation.
  4. The name of the unit that is displayed in the cost calculation header.

The chosen name also determines the suffix that is used for writing output files or for defining model-specific parameters. For example, after training our directory will contain weight files called opt.weights_hirshv.XXX.out.

Gotcha

For training a 3G electrostatic model, it is important to name the target property "charge" (not "charges", or "q", or similar). This is because the following shor-trange training step expects this property name.

Monitoring output

After starting the training, RuNNer will print some general statistical information about the target property and the chosen feature maps. The main report section starts with the name of the target property and it's unit, as defined by the train_atomic_properties keyword. RuNNer evaluates the chosen cost metric (RMSE by default) on the randomly selected train and test points and reports the values for each committee member and epoch. It also reports the number of parameter updates done in each epoch.

Tip

You can choose further cost functions with the keyword cost_functions. You can also alter the frequency of cost calculation with cost_frequency, which can significantly speed up training.

--------------------------------------------------------------------------------
                             hirshv [mau]
        Epoch Comm.          Train           Test
--------------------------------------------------------------------------------
RMSE        0     1        18.3768        21.7268
--------------------------------------------------------------------------------
RMSE        1     1         5.8926         9.3221
UPDATES     1     1            485
TIMING      1      0.02 min
--------------------------------------------------------------------------------
RMSE        2     1         6.0558         9.2290
UPDATES     2     1            532
TIMING      2      0.02 min
--------------------------------------------------------------------------------
RMSE        3     1         4.8399         8.8008
UPDATES     3     1            521
TIMING      3      0.02 min
--------------------------------------------------------------------------------
RMSE        4     1         4.0109         9.4751
UPDATES     4     1            507
TIMING      4      0.02 min
--------------------------------------------------------------------------------
RMSE        5     1         2.1698         9.1617
UPDATES     5     1            518
TIMING      5      0.02 min
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
INFO: In epoch 000003 the lowest test hirshv cost was reached. Weights and
      outputfiles starting with the prefix "opt." correspond to this epoch. NOTE
      THAT THIS IS NOT NECESSARILY THE BEST EPOCH! CHECK ALL TESTS AND TRAIN
      ERRORS TO FIND THE BEST EPOCH!

--------------------------------------------------------------------------------

Check out the cost metrics

As you can see, training ran quickly in only a few seconds for this small example. The training RMSE sinks nicely, while the test error stagnates quickly. This is due to the unrepresentative training set we chose to capture this sample output.

Do not blindly trust the 'optimal' epoch

RuNNer also makes an effort to determine the "optimal" epoch. However, be aware that this is only a rough estimate based on the cost metrics. These numbers often hide important features of the trained hyperparameter surface. Ideally, evaluate trained models from different epochs on a validation set or a physical observable to see which model truly performs best.

Advanced keywords

loss_threshold_factor_atomic_properties0.1

Use this keyword to filter irrelevant updates. Often significantly stabilizes training and gives a considerable speedup.

atomic_property_fractionH 0.05

Use this keyword to select only a random fraction of atoms for weight updates in each epoch. The keyword can be set on a per-element basis, allowing fine-grained control over update strategies.

Tip

This keyword can be tremendously helpful when you train a model for molecules on a surface or a small molecule surrounded by solvent.

batchsize_elementsO 16:

This keyword has two effects: it changes the number of samples that are averaged for a single update and also changes the size of matrix-matrix multiplications in the forward and backward pass of the model. Play with it to increase the speed and stability of the training process. For the Kalman filter, however, it is usually more helpful to present many individual samples than large batches.

Prediction

We currently do not implement a separate prediction mode for arbitrary atomic properties. Instead, run runner_mode train with

to obtain predictions for unknown structures.

  1. M. Eckhoff, K. N. Lausch, P. E. Blöchl, J. Behler, “Predicting oxidation and spin states by high-dimensional neural networks: Applications to lithium manganese oxide spinels” J. Chem. Phys. 2020, 153, 164107. DOI: https://doi.org/10.1063/5.0021452 ↩