| Japanese | English |

# 22nd LSI Design Contests-in Okinawa Design Specification - 3-3

## 3-3. Algorithm

z_{i}^{2}, z_{i}^{3} and z_{i}^{4}
is the sum of the bias and product of each weight from input signal and the input signal itself.
The calculations are as below:

a_{i}^{2}, a_{i}^{3} and a_{i}^{4}
is the activation function for
z_{i}^{2}, z_{i}^{3} and z_{i}^{4}respectively.
We can use any functions that can be differentiate and normalize as activation function.
For example if we use ReLU function,
the equations for
a_{i}^{2} and φ(z_{i}^{2}) are as follow:

After n-th signal was passed through the neural network model, cost function is then calculated.
Cost function is the sum of a square error function between the output layer′s output and supervisor.
The calculations are as follow:

From here on, we will use back propagation method where the parameter of weight and bias are changing simultaneously
with the cost function to optimize the calculation of new parameter from output to input.
By using the C value, we can calculate the gradient descent of the weight and bias.
The concept of the calculations is as shown below.

δ_{j}^{i} is the error value for weight and bias. From these equations,
we can use it to calculate the delta value from the cost function.
For example, we calculate the gradient function of ∂C/(∂w_{1}^{2}_{1} ) and ∂C/(∂b_{1}^{2} ) .
The concept is as shown in the figure below

Fig 4 : Back propagation using Cost function

The calculations are as below:

From here we can change the weight and bias to a new value. The calculations are as below:

After changing to a new parameter, we can start again to insert the input signal and the calculation will continue until the output result gives the nearest value i.e. the smallest differences between the output and supervisor value.

Thus, the flowchart of this neural network system are as shown below. The calculation will stop when the output values are as same as supervisor value

Fig 5 : for neural network and back propagation

## Example

Please download the PDF and Zip file below to understand the example.

PDF file：Neural Network example

Zip file：Neural Network source code for Octave version 3.8 (文字コード: UTF-8N)

Zip file：Neural Network source code for Octave version 4.2.x (文字コード: SHIFT-JIS)