Skip to content

mod_kalman_filter

Lines: 105 | Visibility: private

Implementation of the Multi Stream Kalman Filter optimization algorithm.

This module provides the Multi Stream Kalman Filter (MSKF) optimizer implementation that extends the abstract Kalman filter base class. The MSKF is designed for MPI-parallel neural network training with support for both modern matrix operations and legacy implementations.

author: Knut Nikolas Lausch, Alexander Knoll, Moritz R. Schaefer and Redouan El Haouari

@note The implementation supports both new dense matrix storage and legacy upper-triangular packed storage via preprocessor directives.

Dependencies

Uses

module~~mod_kalman_filter~~UsesGraph module~mod_kalman_filter mod_kalman_filter module~mod_abstract_kalman_filter mod_abstract_kalman_filter module~mod_kalman_filter->module~mod_abstract_kalman_filter module~mod_abstractoptimizer mod_abstractoptimizer module~mod_kalman_filter->module~mod_abstractoptimizer module~mod_io mod_io module~mod_kalman_filter->module~mod_io module~mod_mpi mod_mpi module~mod_kalman_filter->module~mod_mpi module~mod_precision mod_precision module~mod_kalman_filter->module~mod_precision module~mod_throw mod_throw module~mod_kalman_filter->module~mod_throw module~mod_abstract_kalman_filter->module~mod_abstractoptimizer module~mod_abstract_kalman_filter->module~mod_io module~mod_abstract_kalman_filter->module~mod_mpi module~mod_abstract_kalman_filter->module~mod_precision module~mod_abstract_kalman_filter->module~mod_throw module~mod_data_container mod_data_container module~mod_abstract_kalman_filter->module~mod_data_container module~mod_decorations mod_decorations module~mod_abstract_kalman_filter->module~mod_decorations module~mod_linalg mod_linalg module~mod_abstract_kalman_filter->module~mod_linalg module~mod_loss_mae mod_loss_mae module~mod_abstract_kalman_filter->module~mod_loss_mae module~mod_lossfunctions mod_lossfunctions module~mod_abstract_kalman_filter->module~mod_lossfunctions module~mod_parse_logical_keyword mod_parse_logical_keyword module~mod_abstract_kalman_filter->module~mod_parse_logical_keyword module~mod_path mod_path module~mod_abstract_kalman_filter->module~mod_path module~mod_abstractoptimizer->module~mod_io module~mod_abstractoptimizer->module~mod_mpi module~mod_abstractoptimizer->module~mod_precision module~mod_abstractoptimizer->module~mod_throw module~mod_global_warning mod_global_warning module~mod_abstractoptimizer->module~mod_global_warning module~mod_gradient_clipping mod_gradient_clipping module~mod_abstractoptimizer->module~mod_gradient_clipping module~mod_loss_abstract mod_loss_abstract module~mod_abstractoptimizer->module~mod_loss_abstract module~mod_nan_check mod_nan_check module~mod_abstractoptimizer->module~mod_nan_check module~mod_settings mod_settings module~mod_abstractoptimizer->module~mod_settings module~mod_io->module~mod_precision iso_fortran_env iso_fortran_env module~mod_io->iso_fortran_env module~mod_optional_args mod_optional_args module~mod_io->module~mod_optional_args module~mod_mpi->module~mod_precision mpi_f08 mpi_f08 module~mod_mpi->mpi_f08 iso_c_binding iso_c_binding module~mod_precision->iso_c_binding module~mod_precision->iso_fortran_env module~mod_throw->module~mod_io module~mod_throw->module~mod_mpi module~mod_throw->module~mod_precision module~mod_throw->iso_c_binding module~mod_throw->iso_fortran_env module~mod_sort mod_sort module~mod_throw->module~mod_sort module~mod_data_container->module~mod_io module~mod_data_container->module~mod_precision module~mod_data_container->module~mod_throw module~mod_decorations->module~mod_io module~mod_decorations->module~mod_precision module~mod_decorations->module~mod_throw module~mod_global_warning->module~mod_io module~mod_global_warning->module~mod_throw module~mod_abstract_warning mod_abstract_warning module~mod_global_warning->module~mod_abstract_warning module~mod_gradient_clipping->module~mod_precision module~mod_gradient_clipping->module~mod_settings module~mod_linalg->module~mod_precision module~mod_linalg->module~mod_throw module~mod_linalg->module~mod_settings module~cublas_wrapper cublas_wrapper module~mod_linalg->module~cublas_wrapper module~mod_loss_abstract->module~mod_mpi module~mod_loss_abstract->module~mod_precision module~mod_loss_mae->module~mod_precision module~mod_loss_mae->module~mod_loss_abstract module~mod_lossfunctions->module~mod_io module~mod_lossfunctions->module~mod_throw module~mod_lossfunctions->module~mod_loss_abstract module~mod_lossfunctions->module~mod_loss_mae module~mod_loss_mse mod_loss_mse module~mod_lossfunctions->module~mod_loss_mse module~mod_loss_mse_torch mod_loss_mse_torch module~mod_lossfunctions->module~mod_loss_mse_torch module~mod_loss_rmse mod_loss_rmse module~mod_lossfunctions->module~mod_loss_rmse module~mod_nan_check->module~mod_io module~mod_nan_check->module~mod_precision module~mod_nan_check->module~mod_throw ieee_arithmetic ieee_arithmetic module~mod_nan_check->ieee_arithmetic module~mod_parse_logical_keyword->module~mod_io module~mod_parse_logical_keyword->module~mod_throw module~mod_path->module~mod_io module~mod_settings->module~mod_io module~mod_settings->module~mod_precision module~mod_settings->module~mod_throw module~mod_constants mod_constants module~mod_settings->module~mod_constants module~mod_cpu_vendor mod_cpu_vendor module~mod_settings->module~mod_cpu_vendor module~mod_periodic_table mod_periodic_table module~mod_settings->module~mod_periodic_table module~mod_sort->module~mod_precision module~cublas_wrapper->iso_c_binding module~cublas_wrapper->module~mod_settings module~mod_constants->module~mod_precision module~mod_constants->iso_c_binding module~mod_cpu_vendor->iso_c_binding module~mod_loss_mse->module~mod_precision module~mod_loss_mse->module~mod_loss_abstract module~mod_loss_mse_torch->module~mod_precision module~mod_loss_mse_torch->module~mod_loss_abstract module~mod_loss_rmse->module~mod_precision module~mod_loss_rmse->module~mod_loss_abstract module~mod_periodic_table->module~mod_io module~mod_periodic_table->module~mod_precision module~mod_periodic_table->module~mod_throw module~mod_periodic_table->module~mod_constants module~mod_array_helpers mod_array_helpers module~mod_periodic_table->module~mod_array_helpers module~mod_array_helpers->module~mod_precision module~mod_array_helpers->module~mod_throw

Used By

module~~mod_kalman_filter~~UsedByGraph module~mod_kalman_filter mod_kalman_filter module~mod_optimizers mod_optimizers module~mod_optimizers->module~mod_kalman_filter module~mod_initialize mod_initialize module~mod_initialize->module~mod_optimizers module~mod_predict_mode mod_predict_mode module~mod_predict_mode->module~mod_initialize module~mod_write_outfiles mod_write_outfiles module~mod_predict_mode->module~mod_write_outfiles module~mod_runner_interface_bindings mod_runner_interface_bindings module~mod_runner_interface_bindings->module~mod_initialize module~mod_train_energy_multiple_structures mod_train_energy_multiple_structures module~mod_train_energy_multiple_structures->module~mod_initialize module~mod_train_forces_multiple_structures mod_train_forces_multiple_structures module~mod_train_forces_multiple_structures->module~mod_initialize module~mod_train_forces_multiple_structures->module~mod_train_energy_multiple_structures module~mod_train_loop_4g_charges mod_train_loop_4g_charges module~mod_train_loop_4g_charges->module~mod_initialize module~mod_cost_calculator mod_cost_calculator module~mod_train_loop_4g_charges->module~mod_cost_calculator module~mod_output_files_and_cost_4g_charges mod_output_files_and_cost_4g_charges module~mod_train_loop_4g_charges->module~mod_output_files_and_cost_4g_charges module~mod_train_loop_atomic_property mod_train_loop_atomic_property module~mod_train_loop_atomic_property->module~mod_initialize module~mod_train_loop_atomic_property->module~mod_cost_calculator module~mod_output_files_and_cost mod_output_files_and_cost module~mod_train_loop_atomic_property->module~mod_output_files_and_cost module~mod_train_loop_short_range mod_train_loop_short_range module~mod_train_loop_short_range->module~mod_initialize module~mod_train_loop_short_range->module~mod_train_energy_multiple_structures module~mod_train_loop_short_range->module~mod_train_forces_multiple_structures module~mod_train_loop_short_range->module~mod_cost_calculator module~mod_train_loop_short_range->module~mod_output_files_and_cost module~mod_output_files_and_cost_4g_short_range mod_output_files_and_cost_4g_short_range module~mod_train_loop_short_range->module~mod_output_files_and_cost_4g_short_range module~mod_train_mode mod_train_mode module~mod_train_mode->module~mod_initialize module~mod_train_mode->module~mod_train_loop_4g_charges module~mod_train_mode->module~mod_train_loop_atomic_property module~mod_train_mode->module~mod_train_loop_short_range module~mod_write_outfiles_training mod_write_outfiles_training module~mod_train_mode->module~mod_write_outfiles_training module~mod_write_outfiles->module~mod_initialize program~runner RuNNer program~runner->module~mod_initialize program~runner->module~mod_predict_mode program~runner->module~mod_train_mode module~mod_lammps_interface mod_lammps_interface module~mod_lammps_interface->module~mod_runner_interface_bindings module~mod_write_outfiles_training->module~mod_write_outfiles module~mod_cost_calculator->module~mod_write_outfiles_training module~mod_output_files_and_cost->module~mod_cost_calculator module~mod_output_files_and_cost_4g_charges->module~mod_cost_calculator module~mod_output_files_and_cost_4g_short_range->module~mod_cost_calculator

Derived Types Summary

Name Description
KalmanFilter Multi Stream Kalman Filter Optimizer. This implements an MPI-parallel version of the global...

Procedures Summary

Name Type Description
constructor_ptr_from_file_kalman Function Constructor for KalmanFilter from configuration file line. Creates a new KalmanFilter optimizer...
constructor_ptr_from_hyperparams_kalman Function Constructor for KalmanFilter from explicit hyperparameters. Creates a new KalmanFilter...
print_kalman Subroutine Display information about a KalmanFilter object. Called by the self%print_kalman_parent routine.
report_kalman Subroutine Displays current status of the KalmanFilter object for verbosity >= 2.
initialize_defaults_kalman Subroutine Initializes default name and parameters specific to the KalmanFilter object. Called by the...
set_hyperparameter_kalman Subroutine Sets the parameters specific to the KalmanFilter object. Called by the...
validate_hyperparameters_kalman Subroutine Validates paremeters specific to the KalmanFilter object.
update_kalman Subroutine Performs the the part of a KF update specific to KalmanFilter. Called by the...

Derived Types Details

type KalmanFilter

Multi Stream Kalman Filter Optimizer.

This implements an MPI-parallel version of the global extended Kalman Filter. For a single stream, it boils down to the same updates as implemented in earlier versions of RuNNer. For understanding the functionality of the Kalman filter, it might still be helpful to look at the old, non-parallel routines. The key difference here is that the p_matrix is stored as a dense vector containing only the upper triangle of the full matrix. They are wrapped in preprocessor blocks with the keyword LEGACY_KALMAN.

The old code has been ported to the current version of RuNNer such that it is fully functional. It can be activated by configuring with FEATURES=LEGACY_KALMAN.

Components

  • name : character– Inherited from AbstractOptimizer
  • loss_function : class(AbstractLossFunction)– Inherited from AbstractOptimizer
  • num_parameters : integer– Inherited from AbstractOptimizer
  • loss_function_type : integer– Inherited from AbstractOptimizer
  • comm_idx : integer– Index of the committee member to which this optimizer is assigned to. – Inherited from AbstractOptimizer
  • opt_idx : integer– Index of the committee member to which this optimizer is assigned to. – Inherited from AbstractOptimizer
  • lupdate_hyperparameters : logical– Inherited from AbstractOptimizer
  • pmat : real(DP) (dim: (:))– Parameter correlation matrix (packed upper triangular) – Inherited from AbstractKalmanFilter
  • pmat : real(DP) (dim: (:,:))– Parameter correlation matrix (dense square format) – Inherited from AbstractKalmanFilter
  • p_initial : real(DP)– Initial diagonal value for correlation matrix initialization – Inherited from AbstractKalmanFilter
  • regularize : logical– Enable eigenvalue-based matrix regularization – Inherited from AbstractKalmanFilter
  • read_restart : logical– Read covariance matrix from last fit – Inherited from AbstractKalmanFilter
  • write_restart : logical– Write covariance matrix from last fit and report hyperparameters – Inherited from AbstractKalmanFilter
  • reg_eps : real(DP)– Regularization epsilon for eigenvalue modification – Inherited from AbstractKalmanFilter
  • reg_degenaracy : real(DP)– Threshold for detecting degenerate eigenvalues – Inherited from AbstractKalmanFilter
  • lambda : real(DP)– Updated forgetting factor
  • nue : real(DP)– Initial forgetting factor

Type-Bound Procedures

Name Attributes Binding
weight_decay public ⇒ weight_decay
parameter_weight_decay public ⇒ parameter_weight_decay
initialize_from_file public ⇒ initialize_from_file_abstract_optimizer
step public ⇒ step_abstractoptimizer_mpi
update_hyperparameters public ⇒ update_hyperparameters_abstractoptimizer
initialize_defaults public ⇒ initialize_defaults_kalman_parent
set_hyperparameter public ⇒ set_hyperparameter_kalman_parent
set_num_parameters public ⇒ set_num_parameters_kalman
update public ⇒ update_kalman_parent
print public ⇒ print_kalman_parent
dump_restart public ⇒ dump_restart_kalman
initialize_defaults_kalman_child public ⇒ initialize_defaults_kalman
set_hyperparameter_kalman_child public ⇒ set_hyperparameter_kalman
validate public ⇒ validate_hyperparameters_kalman
update_kalman_child public ⇒ update_kalman
print_kalman_child public ⇒ print_kalman
report public ⇒ report_kalman

Inherits

type~~kalmanfilter~~InheritsGraph type~kalmanfilter KalmanFilter type~abstractkalmanfilter AbstractKalmanFilter type~kalmanfilter->type~abstractkalmanfilter type~abstractoptimizer AbstractOptimizer type~abstractkalmanfilter->type~abstractoptimizer type~abstractlossfunction AbstractLossFunction type~abstractoptimizer->type~abstractlossfunction loss_function

Interface Details

interface KalmanFilter

Procedure Details

function constructor_ptr_from_file_kalman

function constructor_ptr_from_file_kalman( &
    line, &
    icomm &
)

Constructor for KalmanFilter from configuration file line.

Creates a new KalmanFilter optimizer by parsing hyperparameters from a configuration file line. This is the primary method for creating optimizers from input.nn configuration files.

Arguments

  • line : character [in]– Configuration line
  • icomm : integer [in]– Committee index to which this `KalmanFilter` object is assigned to.

Called By

proc~~constructor_ptr_from_file_kalman~~CalledByGraph proc~constructor_ptr_from_file_kalman mod_kalman_filter::constructor_ptr_from_file_kalman interface~kalmanfilter mod_kalman_filter::KalmanFilter interface~kalmanfilter->proc~constructor_ptr_from_file_kalman

function constructor_ptr_from_hyperparams_kalman

function constructor_ptr_from_hyperparams_kalman( &
    lambda, &
    nue, &
    p_initial, &
    loss_function_type, &
    regularize, &
    reg_eps, &
    reg_degenaracy, &
    icomm &
)

Constructor for KalmanFilter from explicit hyperparameters.

Creates a new KalmanFilter optimizer with explicitly provided hyperparameters. This constructor is useful for programmatic creation of optimizers with specific parameter values, bypassing file parsing.

Arguments: - lambda: Initial update parameter controlling measurement noise (default: 0.98) - nue: Stepsize for lambda updates, acts as forgetting factor (default: 0.9987) - p_initial: Initial diagonal value for correlation matrix (inherited from parent) - loss_function_type: Type of loss function to use (inherited from parent) - regularize: Enable matrix regularization (inherited from parent) - reg_eps: Regularization epsilon parameter (inherited from parent) - reg_degenaracy: Degeneracy regularization parameter (inherited from parent) - icomm: Committee index for MPI parallelization

Arguments

  • lambda : real(DP) [in]– The initial update parameter.
  • nue : real(DP) [in]– The stepsize for each update of `lambda`.
  • p_initial : real(DP) [in]– The initialization value for the diagonal elements of the correlation matrix.
  • loss_function_type : integer [in]– Loss function type index
  • regularize : logical [in]– Enable regularization flag
  • reg_eps : real(DP) [in]– Regularization epsilon
  • reg_degenaracy : real(DP) [in]– Degeneracy regularization
  • icomm : integer [in]– Index of the committee member to which this optimizer is assigned to.

Called By

proc~~constructor_ptr_from_hyperparams_kalman~~CalledByGraph proc~constructor_ptr_from_hyperparams_kalman mod_kalman_filter::constructor_ptr_from_hyperparams_kalman interface~kalmanfilter mod_kalman_filter::KalmanFilter interface~kalmanfilter->proc~constructor_ptr_from_hyperparams_kalman

subroutine print_kalman

subroutine print_kalman( &
    self &
)

Display information about a KalmanFilter object.

Called by the self%print_kalman_parent routine.

Arguments

subroutine report_kalman

subroutine report_kalman( &
    self, &
    epoch &
)

Displays current status of the KalmanFilter object for verbosity >= 2.

Arguments

subroutine initialize_defaults_kalman

subroutine initialize_defaults_kalman( &
    self &
)

Initializes default name and parameters specific to the KalmanFilter object.

Called by the self%initialize_defaults_kalman_parent routine.

Arguments

subroutine set_hyperparameter_kalman

subroutine set_hyperparameter_kalman( &
    self, &
    name, &
    value_str &
)

Sets the parameters specific to the KalmanFilter object.

Called by the self%set_hyperparameter_kalman_parent routine.

Arguments

  • self : class(KalmanFilter) [inout]
  • name : character [in]
  • value_str : character [in]

subroutine validate_hyperparameters_kalman

subroutine validate_hyperparameters_kalman( &
    self &
)

Validates paremeters specific to the KalmanFilter object.

Arguments

Called By

proc~~validate_hyperparameters_kalman~~CalledByGraph proc~validate_hyperparameters_kalman mod_kalman_filter::KalmanFilter%validate_hyperparameters_kalman proc~constructor_ptr_from_hyperparams_kalman mod_kalman_filter::constructor_ptr_from_hyperparams_kalman proc~constructor_ptr_from_hyperparams_kalman->proc~validate_hyperparameters_kalman interface~kalmanfilter mod_kalman_filter::KalmanFilter interface~kalmanfilter->proc~constructor_ptr_from_hyperparams_kalman

subroutine update_kalman

subroutine update_kalman( &
    self, &
    measurement_noise, &
    forgetting_factor &
)

Performs the the part of a KF update specific to KalmanFilter.

Called by the self%update_kalman_parent routine.

Arguments

  • self : class(KalmanFilter) [inout]
  • measurement_noise : real(DP) [out]– Value added to the diagonal of the scaling matrix, here `lambda`.
  • forgetting_factor : real(DP) [out]– Value multiplied with the updated covariance matrix `self%pmat` i.e. `1/lambda`