Artificial
Made or produced by human beings rather than occurring naturally, especially as a copy of something natural.
Neural
Network
According to Wikipedia
Artificial neural networks are computing systems that are inspired by, but not necessarily identical to, the biological neural networks that constitute animal brains. Such systems “learn” to perform tasks by considering examples, generally without being programmed with any taskspecific rules.
Tec4Tric explains Artificial Neural Network
An Artificial Neural Network(ANN) is an efficient information processing system. It processes a large number of highly interconnected elements, called neurons, nodes or units. Each neuron is connected with the other by a connection link. And each connection link is associated with weights, which contain information about the input. ANN’s collective behavior is characterized by their ability to learn, recall, and generalize patterns similar to that of a human brain. ANN is being used in Machine Learning and Deep Learning for more accurate results.
Each neuron has an internal state, called Activation. The activation signal of a neuron is transmitted to other neurons.
A Neuron can send only one signal at a time.
What is Biological Neuron?
 Dendrite – Receives signals from other neurons.
 Synapse – Point of connection to other neurons.
 Soma – It’s the CPU, it processes the information.
 Axon – Transmits the output of the neuron.
Evolution of Neural Networks
Year 
Neural Network 
Inventor 
1943 
McCulloch Pitts Neuron 
McCulloch & Pitts

1949 
Hebb Network

Hebb 
1958 – 1988

Perceptron 
Frank Rosenblatt, Block, Minsky & Papert 
1960

Adaline 
Widrow & Hoff

1972

Kohonen selforganizing feature map

Kohonen 
1986

Backpropagation

Rumelhart, Hinton & Williams

1987 
Adaptive Resonance Theory

Carpenter & Grossberg

1988

Counterpropagation

Grossberg 
1988

Radial basis function

Broomhead & Lowe

1988

Neocognition

Fukushima 
Let me describe, What is McCulloch Pitts Neuron first
Wikipedia says An Artificial Neuron or The Mcculloch Pitts Neuron is a mathematical function conceived as a model of biological neurons, a neural network.
As you can see {X_{1, }X_{2, }X_{3, }…, X_{n}} ∈ {0,1} are the Inputs and Y ∈ {0,1} is the Output. And f and g are the Functions.
f is the Activation Function and g is the PreActivation Function.
There are two types of inputs, One is Excitatory Input, which is dependent and another one is Inhibitory Input, which is independent input.
Here {X_{1, }X_{2, }X_{3, }…, X_{n}} ∈ {0,1} are the Excitatory Inputs.
The output will be 1 if g(x) is Greater than equal to (≥) the threshold parameter and the output will be 0 if g(x) is less than (<) the threshold parameter.
A single Mcculloch Pitts neuron can be used to represent Boolean functions ( AND, OR, NOR, etc. ) which are linearly separable.
Linear Separability: There exists a line ( plane ) such that all inputs which produce a ‘1’ lie on one side of the line ( plane ) and all inputs which produce a ‘0’ lie on another side of the line ( plane ).
What is Perceptron?
Perceptron is a more general computational model than Mcculloch Pitts Neuron.Here, (w1, w2, w3, …, wn ) are the Weights.
The main difference between Mcculloch Pitts neuron and Perceptron is, an introduction of numerical weights (w1, w2, w3, … wn ) for inputs and a mechanism for learning these weights.
More about Artificial Neural Network
 Input Layer – This layer takes the input.
 Hidden Layer – This layer process the given data.
 Output Layer – This layer produces output.
Important Terminologies of ANN
 Weights: In ANN, each neuron is connected to other neurons and associated with weights. The weights contain information about the input. This information is used by the neural network to solve problems. The weights can be represented in terms of a matrix. And this matrix is also known as a connection matrix. Like weights, W = [w1, w2, w3, …, wn].
 Bias: A bias unit is an “extra” neuron added to each preoutput layer that stores the value of 1. The bias included in the network has its impact in calculating the net input. The bias is included by adding a component x0 = 1 to the input vector X. The input vector looks like this X = (1, X1, X2, …, Xn).
 Threshold: Threshold is a set of value used to calculate the final output of a network. It is used in the activation function. For each and every application there is a threshold limit. Based on the threshold value, the activation functions are defined and the output is calculated. A threshold value is denoted by “θ”.
 Learning Rate: The learning rate is denoted by “α”. It is used to control the amount of weight adjustment at each step of training. The learning rate, ranging from 0 to 1, determines the rate of learning at each time step.