22nd LSI Design Contests-in Okinawa  Design Specification - 3-3

3-3. Algorithm

zi2, zi3 and zi4 is the sum of the bias and product of each weight from input signal and the input signal itself. The calculations are as below:


Equation 1

ai2, ai3 and ai4 is the activation function for zi2, zi3 and zi4respectively. We can use any functions that can be differentiate and normalize as activation function. For example if we use ReLU function, the equations for ai2 and φ(zi2) are as follow:


Equation 2


After n-th signal was passed through the neural network model, cost function is then calculated. Cost function is the sum of a square error function between the output layer′s output and supervisor. The calculations are as follow:

Equation 3


From here on, we will use back propagation method where the parameter of weight and bias are changing simultaneously with the cost function to optimize the calculation of new parameter from output to input. By using the C value, we can calculate the gradient descent of the weight and bias. The concept of the calculations is as shown below.

Equation 4

δji is the error value for weight and bias. From these equations, we can use it to calculate the delta value from the cost function. For example, we calculate the gradient function of ∂C/(∂w121 ) and ∂C/(∂b12 ) . The concept is as shown in the figure below

Neural Network 4

Fig 4 : Back propagation using Cost function


The calculations are as below:

Equation 5

From here we can change the weight and bias to a new value. The calculations are as below:

Equation 6

After changing to a new parameter, we can start again to insert the input signal and the calculation will continue until the output result gives the nearest value i.e. the smallest differences between the output and supervisor value.


Thus, the flowchart of this neural network system are as shown below. The calculation will stop when the output values are as same as supervisor value

Neural Network 5

Fig 5 : for neural network and back propagation


Example

Please download the PDF and Zip file below to understand the example.


PDF file:Neural Network example
Zip file:Neural Network source code for Octave version 3.8 (文字コード: UTF-8N)
Zip file:Neural Network source code for Octave version 4.2.x (文字コード: SHIFT-JIS)
<<Back                 Next>>