Artificial

Made or produced by human beings rather than occurring naturally, especially as a copy of something natural.

Neural

Relating to a nerve or the nervous system.

Network

A group or system of interconnected people or things.

According to Wikipedia

Artificial neural networks are computing systems that are inspired by, but not necessarily identical to, the biological neural networks that constitute animal brains. Such systems “learn” to perform tasks by considering examples, generally without being programmed with any task-specific rules.

Tec4Tric explains Artificial Neural Network

An Artificial Neural Network(ANN) is an efficient information processing system. It processes a large number of highly interconnected elements, called neurons, nodes or units. Each neuron is connected with the other by a connection link. And each connection link is associated with weights, which contain information about the input. ANN’s collective behavior is characterized by their ability to learn, recall, and generalize patterns similar to that of a human brain. ANN is being used in Machine Learning and Deep Learning for more accurate results.

Each neuron has an internal state, called Activation. The activation signal of a neuron is transmitted to other neurons.

A Neuron can send only one signal at a time.

What is Biological Neuron?

A biological neuron consists of Fours Parts Mainly,
  • Dendrite – Receives signals from other neurons.
  • Synapse – Point of connection to other neurons.
  • Soma – It’s the CPU, it processes the information.
  • Axon – Transmits the output of the neuron.

Evolution of Neural Networks

Year

Neural Network

Inventor

1943

McCulloch Pitts Neuron

McCulloch & Pitts
1949
Hebb Network
Hebb
1958 – 1988

Perceptron

Frank Rosenblatt, Block, Minsky & Papert

1960
Adaline
Widrow & Hoff
1972
Kohonen self-organizing feature map
Kohonen
1986
Back-propagation
Rumelhart, Hinton & Williams
1987
Adaptive Resonance Theory
Carpenter & Grossberg
1988
Counter-propagation
Grossberg
1988
Radial basis function
Broomhead & Lowe
1988
Neo-cognition
Fukushima

Let me describe, What is McCulloch Pitts Neuron first

Wikipedia says An Artificial Neuron or The Mcculloch Pitts Neuron is a mathematical function conceived as a model of biological neurons, a neural network.

As you can see {X1, X2, X3, …, Xn∈ {0,1} are the Inputs and Y ∈ {0,1} is the Output. And f and g are the Functions.

f is the Activation Function and g is the Pre-Activation Function.

There are two types of inputs, One is Excitatory Input, which is dependent and another one is Inhibitory Input, which is independent input.

Here {X1, X2, X3, …, Xn} ∈ {0,1} are the Excitatory Inputs.

The output will be 1 if g(x) is Greater than equal to (≥) the threshold parameter and the output will be 0 if g(x) is less than (<) the threshold parameter.

A single Mcculloch Pitts neuron can be used to represent Boolean functions ( AND, OR, NOR, etc. ) which are linearly separable.

Linear Separability: There exists a line ( plane ) such that all inputs which produce a ‘1’ lie on one side of the line ( plane ) and all inputs which produce a ‘0’ lie on another side of the line ( plane ).

What is Perceptron?

Perceptron is a more general computational model than Mcculloch Pitts Neuron.

Here, (w1, w2, w3, …, wn ) are the Weights.

The main difference between Mcculloch Pitts neuron and Perceptron is, an introduction of numerical weights (w1, w2, w3, … wn ) for inputs and a mechanism for learning these weights.

More about Artificial Neural Network

An Artificial Neural Network typically consists of three layers.
  • Input Layer – This layer takes the input.
 
  • Hidden Layer – This layer process the given data.
 
  • Output Layer – This layer produces output.
The number of hidden layers varies.

If you want to download Artificial Neural Networks notes as a PDF, then download from the below button.

Watch the Artificial Neural Network tutorial on Youtube

Important Terminologies of ANN

  • Weights: In ANN, each neuron is connected to other neurons and associated with weights. The weights contain information about the input. This information is used by the neural network to solve problems. The weights can be represented in terms of a matrix. And this matrix is also known as a connection matrix. Like weights, W = [w1, w2, w3, …, wn].
    • Bias: A bias unit is an “extra” neuron added to each pre-output layer that stores the value of 1. The bias included in the network has its impact in calculating the net input. The bias is included by adding a component x0 = 1 to the input vector X. The input vector looks like this  X = (1, X1, X2, …, Xn).
    • Threshold: Threshold is a set of value used to calculate the final output of a network. It is used in the activation function. For each and every application there is a threshold limit. Based on the threshold value, the activation functions are defined and the output is calculated. A threshold value is denoted by “θ”.
    • Learning Rate: The learning rate is denoted by “α”. It is used to control the amount of weight adjustment at each step of training. The learning rate, ranging from 0 to 1, determines the rate of learning at each time step.