Skip to content

mod_input_layer

Lines: 186 | Visibility: private

Implementation of input layers for a neural network. author: Alexander Knoll

Non-OpenMP parallelized forward pass implementation for the input layer.

This routine is very efficient for small batchsizes. Non OpenMP parallelized forward gradient pass implementation for the input layer.

This routine is very efficient for small batchsizes.

Dependencies

Uses

module~~mod_input_layer~~UsesGraph module~mod_input_layer mod_input_layer module~mod_constants mod_constants module~mod_input_layer->module~mod_constants module~mod_decorations mod_decorations module~mod_input_layer->module~mod_decorations module~mod_io mod_io module~mod_input_layer->module~mod_io module~mod_precision mod_precision module~mod_input_layer->module~mod_precision module~mod_throw mod_throw module~mod_input_layer->module~mod_throw module~mod_constants->module~mod_precision iso_c_binding iso_c_binding module~mod_constants->iso_c_binding module~mod_decorations->module~mod_io module~mod_decorations->module~mod_precision module~mod_decorations->module~mod_throw module~mod_io->module~mod_precision iso_fortran_env iso_fortran_env module~mod_io->iso_fortran_env module~mod_optional_args mod_optional_args module~mod_io->module~mod_optional_args module~mod_precision->iso_c_binding module~mod_precision->iso_fortran_env module~mod_throw->module~mod_io module~mod_throw->module~mod_precision module~mod_throw->iso_c_binding module~mod_throw->iso_fortran_env module~mod_mpi mod_mpi module~mod_throw->module~mod_mpi module~mod_sort mod_sort module~mod_throw->module~mod_sort module~mod_mpi->module~mod_precision mpi_f08 mpi_f08 module~mod_mpi->mpi_f08 module~mod_sort->module~mod_precision

Used By

module~~mod_input_layer~~UsedByGraph module~mod_input_layer mod_input_layer module~mod_input_layer_with_activations mod_input_layer_with_activations module~mod_input_layer_with_activations->module~mod_input_layer module~mod_layers mod_layers module~mod_layers->module~mod_input_layer module~mod_layers->module~mod_input_layer_with_activations module~mod_neural_network mod_neural_network module~mod_neural_network->module~mod_layers module~mod_hdnn mod_hdnn module~mod_hdnn->module~mod_neural_network module~mod_initialize mod_initialize module~mod_initialize->module~mod_hdnn module~mod_predict_mode mod_predict_mode module~mod_predict_mode->module~mod_initialize module~mod_write_outfiles mod_write_outfiles module~mod_predict_mode->module~mod_write_outfiles module~mod_runner_interface_bindings mod_runner_interface_bindings module~mod_runner_interface_bindings->module~mod_initialize module~mod_train_energy_multiple_structures mod_train_energy_multiple_structures module~mod_train_energy_multiple_structures->module~mod_initialize module~mod_train_forces_multiple_structures mod_train_forces_multiple_structures module~mod_train_forces_multiple_structures->module~mod_initialize module~mod_train_forces_multiple_structures->module~mod_train_energy_multiple_structures module~mod_train_loop_4g_charges mod_train_loop_4g_charges module~mod_train_loop_4g_charges->module~mod_initialize module~mod_cost_calculator mod_cost_calculator module~mod_train_loop_4g_charges->module~mod_cost_calculator module~mod_output_files_and_cost_4g_charges mod_output_files_and_cost_4g_charges module~mod_train_loop_4g_charges->module~mod_output_files_and_cost_4g_charges module~mod_train_loop_atomic_property mod_train_loop_atomic_property module~mod_train_loop_atomic_property->module~mod_initialize module~mod_train_loop_atomic_property->module~mod_cost_calculator module~mod_output_files_and_cost mod_output_files_and_cost module~mod_train_loop_atomic_property->module~mod_output_files_and_cost module~mod_train_loop_short_range mod_train_loop_short_range module~mod_train_loop_short_range->module~mod_initialize module~mod_train_loop_short_range->module~mod_train_energy_multiple_structures module~mod_train_loop_short_range->module~mod_train_forces_multiple_structures module~mod_train_loop_short_range->module~mod_cost_calculator module~mod_train_loop_short_range->module~mod_output_files_and_cost module~mod_output_files_and_cost_4g_short_range mod_output_files_and_cost_4g_short_range module~mod_train_loop_short_range->module~mod_output_files_and_cost_4g_short_range module~mod_train_mode mod_train_mode module~mod_train_mode->module~mod_initialize module~mod_train_mode->module~mod_train_loop_4g_charges module~mod_train_mode->module~mod_train_loop_atomic_property module~mod_train_mode->module~mod_train_loop_short_range module~mod_write_outfiles_training mod_write_outfiles_training module~mod_train_mode->module~mod_write_outfiles_training module~mod_write_outfiles->module~mod_initialize program~runner RuNNer program~runner->module~mod_initialize program~runner->module~mod_predict_mode program~runner->module~mod_train_mode module~mod_lammps_interface mod_lammps_interface module~mod_lammps_interface->module~mod_runner_interface_bindings module~mod_write_outfiles_training->module~mod_write_outfiles module~mod_cost_calculator->module~mod_write_outfiles_training module~mod_output_files_and_cost->module~mod_cost_calculator module~mod_output_files_and_cost_4g_charges->module~mod_cost_calculator module~mod_output_files_and_cost_4g_short_range->module~mod_cost_calculator

Derived Types Summary

Name Description
InputLayer An input layer for a feed-forward neural network. The output of this layer is calculated as $a...

Procedures Summary

Name Type Description
init_inputlayer Subroutine Construct a new InputLayer object.
reset_inputlayer Subroutine Reset a InputLayer object. Should be called before any new backward pass.
reset_prediction_inputlayer Subroutine Reset all forward-pass related arrays, but keep the collected gradients. Should be called before...
reallocate_inputlayer Subroutine Change the dimensions of all internal arrays to accomodate batchsize input and output values.
print_inputlayer Subroutine Print information about an InputLayer object.
forward_inputlayer Subroutine Entry point for input layer forward passes. This is the top-level routine for forward passes...
forward_gradient_inputlayer Subroutine Entry point for input layer forward gradient passes. See <a...
forward_inputlayer_omp_parallel Subroutine OMP-parallel input layer forward pass implementation. The OMP-parallelization in this routine...
forward_gradient_inputlayer_omp_parallel Subroutine OMP-parallel input layer forward gradient pass implementation. See <a...
forward_inputlayer_serial Subroutine
forward_gradient_inputlayer_serial Subroutine
backward_inputlayer Subroutine
backward_gradient_inputlayer Subroutine

Derived Types Details

type InputLayer

An input layer for a feed-forward neural network.

The output of this layer is calculated as \(a = f(x)\) where - \(x \in \R^(N_b, N_x)\) is the input vector to the layer - \(f: \R\to\R\) is a vector of activation functions, which can be different for each node. - \(a \in \R^(N_b, N_x)\) is the activation vector of the layer i.e. the output. This layer does not have any trainable parameters.

Components

  • num_nodes : integer– Inherited from InputLayer
  • batchsize : integer– Inherited from InputLayer
  • x : real(DP) (dim: (:,:))– The input to this layer, which is stored temporarily to be reused during a backward pass.
  • z : real(DP) (dim: (:,:))– Pre-activations, the node values before application of the activation function.
  • a : real(DP) (dim: (:,:))– Activations, the node values after application of the activation function. This is the output of the layer.
  • dz_dq : real(DP) (dim: (:,:))– Derivatives of the pre-activations with respect to `q` during forward-mode differentiation.
  • da_dq : real(DP) (dim: (:,:))– Derivatives of the activations with respect to `q` during forward-mode differentiation.
  • grad : real(DP) (dim: (:,:))– The gradient passing through the layer during a backward pass. Usually this is the derivative of the loss function with respect to the activation of the layer preceeding this one. As there is no preceeding layer, this is simply the derivative of the loss function with respect to the input of the network.
  • dgrad_dq : real(DP) (dim: (:,:))– Inherited from InputLayer
  • forward_ptr : procedure– Pointer for delegating the forward pass to the correct implementation. We have an OpenMP-parallel and a non-parallel forward pass implementation. The serial implementation is more efficient for small batchsizes. We align the pointer whenever reallocate_inputlayer is called, which is usually only once.
  • forward_gradient_ptr : procedure– Pointer for delegating the forward gradient pass. See forward_ptr for details.

Type-Bound Procedures

Name Attributes Binding
initialize public ⇒ init_inputlayer
print public ⇒ print_inputlayer
reallocate public ⇒ reallocate_inputlayer
reset public ⇒ reset_inputlayer
reset_prediction public ⇒ reset_prediction_inputlayer
forward_inputlayer_omp_parallel private ⇒ forward_inputlayer_omp_parallel
forward_inputlayer_serial private ⇒ forward_inputlayer_serial
forward_gradient_inputlayer_omp_parallel private ⇒ forward_gradient_inputlayer_omp_parallel
forward_gradient_inputlayer_serial private ⇒ forward_gradient_inputlayer_serial
forward public ⇒ forward_inputlayer
forward_gradient public ⇒ forward_gradient_inputlayer
backward public ⇒ backward_inputlayer
backward_gradient public ⇒ backward_gradient_inputlayer

Inherited By

type~~inputlayer~~InheritedByGraph type~inputlayer InputLayer type~inputlayerwithactivations InputLayerWithActivations type~inputlayerwithactivations->type~inputlayer type~neuralnetwork NeuralNetwork type~neuralnetwork->type~inputlayer input_layer type~hdnn HDNN type~hdnn->type~neuralnetwork atomic_nns

Procedure Details

subroutine init_inputlayer

subroutine init_inputlayer( &
    self, &
    num_nodes, &
    batchsize &
)

Construct a new InputLayer object.

Arguments

  • self : class(InputLayer) [inout]– The constructed layer.
  • num_nodes : integer [in]– The number of input features, i.e. the size of the layer.
  • batchsize : integer [in]– How many samples can pass through the layer concurrently.

Called By

proc~~init_inputlayer~~CalledByGraph proc~init_inputlayer mod_input_layer::InputLayer%init_inputlayer proc~initialize_nn_from_architecture mod_neural_network::NeuralNetwork%initialize_nn_from_architecture proc~initialize_nn_from_architecture->proc~init_inputlayer none~initialize mod_neural_network::NeuralNetwork%initialize none~initialize->proc~initialize_nn_from_architecture proc~initialize_from_architecture mod_hdnn::HDNN%initialize_from_architecture proc~initialize_from_architecture->none~initialize proc~initialize_from_file_hdnn mod_hdnn::HDNN%initialize_from_file_hdnn proc~initialize_from_file_hdnn->proc~initialize_from_architecture

subroutine reset_inputlayer

subroutine reset_inputlayer( &
    self &
)

Reset a InputLayer object. Should be called before any new backward pass.

Arguments

Called By

proc~~reset_inputlayer~~CalledByGraph proc~reset_inputlayer mod_input_layer::InputLayer%reset_inputlayer proc~reallocate_inputlayer mod_input_layer::InputLayer%reallocate_inputlayer proc~reallocate_inputlayer->proc~reset_inputlayer proc~reset_neuralnetwork mod_neural_network::NeuralNetwork%reset_neuralnetwork proc~reset_neuralnetwork->proc~reset_inputlayer proc~init_inputlayer mod_input_layer::InputLayer%init_inputlayer proc~init_inputlayer->proc~reallocate_inputlayer proc~init_inputlayer_act mod_input_layer_with_activations::InputLayerWithActivations%init_inputlayer_act proc~init_inputlayer_act->proc~reallocate_inputlayer proc~reallocate_neuralnetwork mod_neural_network::NeuralNetwork%reallocate_neuralnetwork proc~reallocate_neuralnetwork->proc~reallocate_inputlayer proc~reallocate_neuralnetwork->proc~reset_neuralnetwork proc~reset_hdnn mod_hdnn::HDNN%reset_hdnn proc~reset_hdnn->proc~reset_neuralnetwork proc~initialize_nn_from_architecture mod_neural_network::NeuralNetwork%initialize_nn_from_architecture proc~initialize_nn_from_architecture->proc~init_inputlayer proc~initialize_nn_from_architecture->proc~init_inputlayer_act proc~initialize_nn_from_list_of_layers mod_neural_network::NeuralNetwork%initialize_nn_from_list_of_layers proc~initialize_nn_from_architecture->proc~initialize_nn_from_list_of_layers proc~initialize_nn_from_list_of_layers->proc~reallocate_neuralnetwork proc~reallocate_hdnn mod_hdnn::HDNN%reallocate_hdnn proc~reallocate_hdnn->proc~reallocate_neuralnetwork none~initialize mod_neural_network::NeuralNetwork%initialize none~initialize->proc~initialize_nn_from_architecture none~initialize->proc~initialize_nn_from_list_of_layers proc~initialize_from_architecture mod_hdnn::HDNN%initialize_from_architecture proc~initialize_from_architecture->none~initialize proc~initialize_from_file_hdnn mod_hdnn::HDNN%initialize_from_file_hdnn proc~initialize_from_file_hdnn->proc~initialize_from_architecture

subroutine reset_prediction_inputlayer

subroutine reset_prediction_inputlayer( &
    self &
)

Reset all forward-pass related arrays, but keep the collected gradients. Should be called before any new forward pass.

Arguments

Called By

proc~~reset_prediction_inputlayer~~CalledByGraph proc~reset_prediction_inputlayer mod_input_layer::InputLayer%reset_prediction_inputlayer proc~reset_inputlayer mod_input_layer::InputLayer%reset_inputlayer proc~reset_inputlayer->proc~reset_prediction_inputlayer proc~reset_prediction_neuralnetwork mod_neural_network::NeuralNetwork%reset_prediction_neuralnetwork proc~reset_prediction_neuralnetwork->proc~reset_prediction_inputlayer proc~reallocate_inputlayer mod_input_layer::InputLayer%reallocate_inputlayer proc~reallocate_inputlayer->proc~reset_inputlayer proc~reset_neuralnetwork mod_neural_network::NeuralNetwork%reset_neuralnetwork proc~reset_neuralnetwork->proc~reset_inputlayer proc~reset_prediction_hdnn mod_hdnn::HDNN%reset_prediction_hdnn proc~reset_prediction_hdnn->proc~reset_prediction_neuralnetwork proc~init_inputlayer mod_input_layer::InputLayer%init_inputlayer proc~init_inputlayer->proc~reallocate_inputlayer proc~init_inputlayer_act mod_input_layer_with_activations::InputLayerWithActivations%init_inputlayer_act proc~init_inputlayer_act->proc~reallocate_inputlayer proc~reallocate_neuralnetwork mod_neural_network::NeuralNetwork%reallocate_neuralnetwork proc~reallocate_neuralnetwork->proc~reallocate_inputlayer proc~reallocate_neuralnetwork->proc~reset_neuralnetwork proc~reset_hdnn mod_hdnn::HDNN%reset_hdnn proc~reset_hdnn->proc~reset_neuralnetwork proc~reset_hdnn->proc~reset_prediction_hdnn proc~initialize_nn_from_architecture mod_neural_network::NeuralNetwork%initialize_nn_from_architecture proc~initialize_nn_from_architecture->proc~init_inputlayer proc~initialize_nn_from_architecture->proc~init_inputlayer_act proc~initialize_nn_from_list_of_layers mod_neural_network::NeuralNetwork%initialize_nn_from_list_of_layers proc~initialize_nn_from_architecture->proc~initialize_nn_from_list_of_layers proc~initialize_nn_from_list_of_layers->proc~reallocate_neuralnetwork proc~reallocate_hdnn mod_hdnn::HDNN%reallocate_hdnn proc~reallocate_hdnn->proc~reallocate_neuralnetwork none~initialize mod_neural_network::NeuralNetwork%initialize none~initialize->proc~initialize_nn_from_architecture none~initialize->proc~initialize_nn_from_list_of_layers proc~initialize_from_architecture mod_hdnn::HDNN%initialize_from_architecture proc~initialize_from_architecture->none~initialize proc~initialize_from_file_hdnn mod_hdnn::HDNN%initialize_from_file_hdnn proc~initialize_from_file_hdnn->proc~initialize_from_architecture

subroutine reallocate_inputlayer

subroutine reallocate_inputlayer( &
    self, &
    batchsize &
)

Change the dimensions of all internal arrays to accomodate batchsize input and output values.

Arguments

  • self : class(InputLayer) [inout]
  • batchsize : integer [in]

Called By

proc~~reallocate_inputlayer~~CalledByGraph proc~reallocate_inputlayer mod_input_layer::InputLayer%reallocate_inputlayer proc~init_inputlayer mod_input_layer::InputLayer%init_inputlayer proc~init_inputlayer->proc~reallocate_inputlayer proc~init_inputlayer_act mod_input_layer_with_activations::InputLayerWithActivations%init_inputlayer_act proc~init_inputlayer_act->proc~reallocate_inputlayer proc~reallocate_neuralnetwork mod_neural_network::NeuralNetwork%reallocate_neuralnetwork proc~reallocate_neuralnetwork->proc~reallocate_inputlayer proc~initialize_nn_from_architecture mod_neural_network::NeuralNetwork%initialize_nn_from_architecture proc~initialize_nn_from_architecture->proc~init_inputlayer proc~initialize_nn_from_architecture->proc~init_inputlayer_act proc~initialize_nn_from_list_of_layers mod_neural_network::NeuralNetwork%initialize_nn_from_list_of_layers proc~initialize_nn_from_architecture->proc~initialize_nn_from_list_of_layers proc~initialize_nn_from_list_of_layers->proc~reallocate_neuralnetwork proc~reallocate_hdnn mod_hdnn::HDNN%reallocate_hdnn proc~reallocate_hdnn->proc~reallocate_neuralnetwork none~initialize mod_neural_network::NeuralNetwork%initialize none~initialize->proc~initialize_nn_from_architecture none~initialize->proc~initialize_nn_from_list_of_layers proc~initialize_from_architecture mod_hdnn::HDNN%initialize_from_architecture proc~initialize_from_architecture->none~initialize proc~initialize_from_file_hdnn mod_hdnn::HDNN%initialize_from_file_hdnn proc~initialize_from_file_hdnn->proc~initialize_from_architecture

subroutine print_inputlayer

subroutine print_inputlayer( &
    self &
)

Print information about an InputLayer object.

Arguments

subroutine forward_inputlayer

subroutine forward_inputlayer( &
    self, &
    x &
)

Entry point for input layer forward passes.

This is the top-level routine for forward passes that is exposed as the public interface of this type. It is a wrapper routine that delegates the forward call to one of the two forward implementations. See forward_ptr for details.

Arguments

  • self : class(InputLayer) [inout]
  • x : real(DP) (dim: (self%num_nodes,self%batchsize)) [in]

Called By

proc~~forward_inputlayer~~CalledByGraph proc~forward_inputlayer mod_input_layer::InputLayer%forward_inputlayer proc~forward_neuralnetwork mod_neural_network::NeuralNetwork%forward_neuralnetwork proc~forward_neuralnetwork->proc~forward_inputlayer

subroutine forward_gradient_inputlayer

subroutine forward_gradient_inputlayer( &
    self, &
    x, &
    dx_dq &
)

Entry point for input layer forward gradient passes.

See forward_inputlayer for details.

Arguments

  • self : class(InputLayer) [inout]
  • x : real(DP) (dim: (self%num_nodes,self%batchsize)) [in]
  • dx_dq : real(DP) (dim: (self%num_nodes,self%batchsize)) [in]

Called By

proc~~forward_gradient_inputlayer~~CalledByGraph proc~forward_gradient_inputlayer mod_input_layer::InputLayer%forward_gradient_inputlayer proc~forward_gradient_neuralnetwork mod_neural_network::NeuralNetwork%forward_gradient_neuralnetwork proc~forward_gradient_neuralnetwork->proc~forward_gradient_inputlayer

subroutine forward_inputlayer_omp_parallel

subroutine forward_inputlayer_omp_parallel( &
    self, &
    x &
)

OMP-parallel input layer forward pass implementation.

The OMP-parallelization in this routine is only efficient for larger batchsizes. For small batchsizes, the overhead of thread initialization becomes too large. In those cases, use forward_inputlayer_serial.

Arguments

  • self : class(InputLayer) [inout]
  • x : real(DP) (dim: (self%num_nodes,self%batchsize)) [in]

subroutine forward_gradient_inputlayer_omp_parallel

subroutine forward_gradient_inputlayer_omp_parallel( &
    self, &
    x, &
    dx_dq &
)

OMP-parallel input layer forward gradient pass implementation.

See forward_inputlayer_omp_parallel fpr details.

Arguments

  • self : class(InputLayer) [inout]
  • x : real(DP) (dim: (self%num_nodes,self%batchsize)) [in]
  • dx_dq : real(DP) (dim: (self%num_nodes,self%batchsize)) [in]

subroutine forward_inputlayer_serial

subroutine forward_inputlayer_serial( &
    self, &
    x &
)

Arguments

  • self : class(InputLayer) [inout]
  • x : real(DP) (dim: (self%num_nodes,self%batchsize)) [in]

subroutine forward_gradient_inputlayer_serial

subroutine forward_gradient_inputlayer_serial( &
    self, &
    x, &
    dx_dq &
)

Arguments

  • self : class(InputLayer) [inout]
  • x : real(DP) (dim: (self%num_nodes,self%batchsize)) [in]
  • dx_dq : real(DP) (dim: (self%num_nodes,self%batchsize)) [in]

subroutine backward_inputlayer

subroutine backward_inputlayer( &
    self, &
    grad &
)

Arguments

  • self : class(InputLayer) [inout]
  • grad : real(DP) (dim: (self%num_nodes,self%batchsize)) [in]

subroutine backward_gradient_inputlayer

subroutine backward_gradient_inputlayer( &
    self, &
    grad, &
    dgrad_dq &
)

Arguments

  • self : class(InputLayer) [inout]
  • grad : real(DP) (dim: (self%num_nodes,self%batchsize)) [in]
  • dgrad_dq : real(DP) (dim: (self%num_nodes,self%batchsize)) [in]