A neural network is a supervised learning algorithm originally developed by Frank Rosenblatt in 1957. Neural networks are one of the oldest and most primitive models of machine learning. However, the concepts developed in their design are widely applicable to complex deep neural networks.
There are billions of neurons in the human brain. Neurons are interconnected nerve cells that play a role in transmitting and processing chemical and electrical signals in the human brain. Dendrites are branches that receive information from other nerve cells. The nucleus of the cell processes the information received from the dendrites. Axon is a cable used by neurons to send information. A synapse is a connection between an axon and other nerve dendrites.
Artificial neurons, however, are mathematical functions that simulate biological neurons, where inputs are weighed separately, summed up, and outputs are calculated through non-linear functions. Below is a figure showing artificial and biological neurons.
A neuron is an elementary unit in an artificial neural network. it is a mathematical function designed based on the function of biological neurons; weights are assigned to a neuron from one or more separate inputs. The inputs are collected and passed through a nonlinear function (the activation function or ” ϕ”) to produce the output. In fact, an artificial neuron is controlled by the following function, in which Y is the output of the artificial neuron:
A neural network consists of several artificial neurons. This section goes over the different layers of an artificial neuron. Generally, neural networks are composed of three layers: the input layer, the hidden layer, and the output layer. The hidden layer may be made up of more than one layer.
Input layer: No calculations are performed in this layer, it only passes information to the next layer.
Hidden layer: In the hidden layer, intermediate processing is performed. After performing calculations, the weights (signals or information) are transferred from the input layer to the next layer (another hidden layer or output layer). It’s also possible for a neural network to not have a hidden layer.
Output layer: This layer uses an activation function to produce the desired output format. (For example, for classification problems, we use the softmax function).
Neural networks come in various flavors. It is also possible to divide main categories into subcategories. Here is a list of the most commonly used neural networks.
Feedforward neural network: A feed-forward neural network is an artificial neural network with no loops between the units. There is only one direction in which the information moves in this network: forward. That is, from input nodes through hidden nodes, if any, and to output groups. This type of network does not contain loops or cycles. Feedforward neural networks can be divided into two types:
Convolutional neural network: A convolutional neural network is similar to a regular neural network. They also consist of neurons with trainable weights and biases. In convolutional neural networks (CNNs or ConvNets), the connection pattern of units is inspired by the visual cortex. An area of space called the receptive field allows units to respond to stimuli. Each receptive field partially overlaps the other and covers the entire field of vision. The convolution operator can mathematically approximate the response of each unit. In image and video recognition, recommender systems are the main applications of these networks. A large amount of data is required to train convolutional neural networks.
Recurrent neural networks: In Recurrent Neural Networks (RNN), the connections between units form a directed loop (these networks propagate data forward and backward, from the final processing levels to the initial levels). A temporary dynamic behavior can be seen in this structure. In contrast to feed-forward neural networks, recurrent neural networks can process arbitrary input sequences. As a result, these networks can perform tasks such as handwriting recognition, voice recognition, and time series estimation.
You can enter your email address and subscribe to our newsletter and get the latest practical content. You can enter your email address and subscribe to our newsletter.