In order to understand what neural networks are, we must begin to learn from an artificial neuron called perceptron. The Perceptrons was developed around the 1950s and 1960s by scientist Frank Rosenblatt and was inspired by previous ideas by Warren McCulloch and Walter Pitts.

HOW IT WORKS

Perceptron accepts binary values ​​such as x1, x2, … and then produces the result as well in binary:

Above image depicts 1 perceptron has 3 inputs: x1, x2, x3. It may have more or less input depending on the intended use. Rosenblatt proposes a simple rule that can calculate the output. He added weights (w1, w2, …) as the values ​​that represent the importance of each input to the output. The output neuron will be 0 or 1, calculated by considering the total weight smaller or greater than the threshhold value. Just like weights, threshhold is 1 parameter of neuron. For more clarity we have the following rule:

That’s all what 1 perceptron works. The above is only a basic algorithm model. Another way you can think about perceptron is that it is like a decision making device by summarizing all the arguments and facilities that are available. I will take an example although it is not very practical but will be easier to understand. Suppose this weekend there will be a big festival in the city and you are wondering whether you should go or not. There are many factors that affect your decision, but here we take three main reasons that directly affect it:

1. Is the weather beautiful?
2. Does your boyfriend / girl want to go with you (imagine you have it: v)
3. Is the festival close to public transport?

We can consider the above 3 factors with 3 variables x1, x2, x3. Suppose, we have x1 = 1 if the weather is good, x1 = 0 if the weather is bad. Similarly, x2 = 1 if your boyfriend / girl wants to go and x2 = 0 in the opposite case. Similar to x3. Now it is as if you are fond of that festival and are ready to go alone if your boyfriend / girl can’t go. However, you hate the bad weather very badly, you will not go if the weather forecast says it will rain that day. In this case you can use perceptron to make a decision. First we choose the weight values ​​for it, w1 = 6 for the weather, w2 = 2 and w3 = 2 for other factors. The greater the value of the weights, the more likely it is to influence your final decision. Finally select the threshold value = 5 for the perceptron. With this option, perceptron will decide whether or not to go based on weather results. It will return 1 if the weather is good and 0 if the bad weather keeps your boyfriend / girl going with you, it doesn’t matter if the festival venue is close to public transport stations.

In fact, perceptron is not a complete model of human decision making. The above example is just to describe how perceptron synthesizes the elements and makes decisions based on the priority of each one. The more complex the perceptron network, the more accurate the decision is made.

The first layer of the perceptron will give 3 decisions based on the synthesis of the inputs. Each perceptron in the second layer will be synthesized from the results of the first class neurons. Thus, the perceptron in the latter layer can create more complex decisions, higher accuracy than the previous class. At the beginning of the article, I have indicated that the perceptron has only one output. However, in the above example we can see there are many outputs. In fact, they are still just an output. The output at the end of the arrows is just one way to show that the output from that perceptron is used to input one or more other perceptrons. This is easier than drawing an output and then dividing it into branches.

We will now simplify the description of a perceptron. Condition ∑ j w j x j > threshold is quite cumbersome. We can simplify it by adding 2 changes. First rewrite wx=∑ j w j x j in which w and x is the vector representation weigths and inputs. Next is to shift the threshold to the other side of the inequality, replacing it with a value we will call perceptron’s bias. Using the substitution bias threshold, perceptron can be rewritten as:

You can think of bias as a quantity used to measure the ability of the perceptron to return a value of 1 or perceptron to be activated . If a perceptron has a large bias, it means that the output of its output is equal to 1 is greater and vice versa.

This is not the only way for perceptron to synthesize the elements and draw conclusions. Another way is to calculate and use basic logic functions like AND, OR and NAND. I will talk in more detail in the following articles.

Source: http://neuralnetworksanddeeplearning.com/chap1.html

Source : viblo