Skip to content

The Workflow of Precomputing Features

This chapter gives an overview about the runner_modeprecompute.

Prerequisites

  • A working RuNNer binary. Take a look at our installation instructions.
  • A dataset of structures formatted as input.data.
  • a text editor like vim or emacs to edit the settings file input.nn.
  • A folder where RuNNer can write output.

Tip

Whenever you start working with on a new project, start with a small dataset (10–100 structures) to verify formatting, units, and that the run completes. Then scale up. This is especially helpful on HPC clusters!

Every precompute needs the following files:

and is run with 

# name of the executable may differ depending on how RuNNer is installed
RuNNer.x | tee precompute.out

Precompute Features

The precompute mode allows to calculate the features for the whole dataset so that they can be easily accessed for analysis. At the same time, it can calculate features and derivatives and store them for later use in training in a binary format. The latter can be benefitial performance-wise if one has much more disk space than RAM.

There are two formats in which the features can be dumped:

  • a human-readable ASCII file
  • a binary file for use in training

To precompute the features set runner_mode precompute and 'format_ascii' and/or 'format_binary' depending on which format is desired. If derivatives are desired add 'calculate_derivatives'. 

#### input.nn ####
runner_mode precompute
format_ascii
format_binary
calculate_derivatives
... #same settings as in training

During the process, RuNNer will:

  1. read input.data,
  2. split the dataset into chunks if there is not enough memory,
  3. evaluate feature maps to compute feature values and their derivatives,
  4. write the features of the structures in same the order of the 'input.data' into the ASCII-formatted file function.data.ascii and/or the binary file function.data with auxiliary function.data.info (and feature derivatives in binary function_derivative.data with auxiliary function_derivative.data.info).

Tip

In runner_mode train use read_features and read_feature_derivatives to read the precomputed values.

Format of function.data.ascii

Each structure begins with a header line containing its structure ID (corresponding to its position in input.data) and the total number of atoms in that structure.

The subsequent lines describe the atoms in the structure. Each line starts with the atomic number of the atom, followed by its associated feature values. The number of feature values can vary between elements, depending on the definitions provided in input.nn.

1 3 
8 sf_value sf_value
1 sf_value sf_value sf_value
1 sf_value sf_value sf_value

Write Scaling

One can precompute the scaling and save it. Use the keywords write_scaling and test_fraction

#### input.nn ####
runner_mode precompute
format_ascii
write_scaling
test_fraction 0.0
... #same settings as in training

During the process, RuNNer will:

  • read input.data
  • split the dataset into chunks if there is not enough memory.
  • compute feature maps
  • write the minimum, maximum and mean value of all features sorted by elements in 'scaling.data'.

Warning

The scaling is only calculated for the train fraction! So check the test_fraction keyword in the output.

Tip

Calculating scaling here helps if one wants to do committee training with different train / test splits. Then one calculates the scaling for the whole input.data and reads it during training.

Precompute Coulomb matrix

For 4G-HDNNP charge training the calculation of the Coulomb matrix is expensive. If you foresee that you have insufficient RAM to precompute and store the Coulomb matrices of all structures during the training run, it is beneficial to store them on disk instead.

Use the input.nn from the training and set runner_mode precompute and 'write_coulomb_matrix' to get the binary 'coulomb_matrix.data'. It stores the natoms+1 x natoms+1 interaction matrix without hardnesses.

#### input.nn ####
runner_mode precompute
write_coulomb_matrix
... #same settings as in training
During the process, RuNNer will:

  1. read input.data,
  2. split the dataset into chunks if there is not enough memory,
  3. calculate the contributions to the coulomb matrix,
  4. write the coulomb matrix of the structures in same the order of the 'input.data' into the binary file coulomb_matrix.data with auxiliary coulomb_matrix.data.info

Tip

In runner_mode train use read_coulomb_matrix to read the precomputed values.