3-6. Algorithm

z_i², z_i³ and z_i⁴ is the sum of the bias and product of each weight from input signal and the input signal itself. The calculations are as below:

a_i², a_i³ and a_i⁴ is the activation function for z_i², z_i³ and z_i⁴respectively. We can use any functions that can be differentiate and normalize as activation function. For example if we use ReLU function, the equations for a_i² and φ(z_i²) are as follow:

After n-th signal was passed through the neural network model, cost function is then calculated. Cost function is the sum of a square error function between the output layer′s output and supervisor. The calculations are as follow:

From here on, we will use back propagation method where the parameter of weight and bias are changing simultaneously with the cost function to optimize the calculation of new parameter from output to input. By using the C value, we can calculate the gradient descent of the weight and bias. The concept of the calculations is as shown below.

δ_jⁱ is the error value for weight and bias. From these equations, we can use it to calculate the delta value from the cost function. For example, we calculate the gradient function of ∂C/(∂w₁²₁ ) and ∂C/(∂b₁² ) . The concept is as shown in the figure below

The calculations are as below:

From here we can change the weight and bias to a new value. The calculations are as below:

After changing to a new parameter, we can start again to insert the input signal and the calculation will continue until the output result gives the nearest value i.e. the smallest differences between the output and supervisor value.

Thus, the flowchart of this neural network system are as shown below. The calculation will stop when the output values are as same as supervisor value