Resources
Neural Networks
Neural Networks are mathematical models inspired by the struture and function of the human brain. They consist of interconnected nodes (neurons) organized in layers, an input layer, one or more hidden layers, and an output layer, that work together to process input data and produce output. Each connection between neurons has a weight that is adjusted during the training process to improve the network’s ability to learn and make predictions.
Here’s a basic breakdown of how it works:
-
Input Layer: This is where the raw data enters the network. Each node represents a feature or piece of information from the input.
-
Hidden Layers: These layers, which can be one or more, process the inputs by performing calculations. Each neuron in these layers applies weights to the inputs, sums them up, and then applies an activation function to introduce non-linearity, helping the network learn complex patterns.
-
Output Layer: This layer produces the final result of the network’s computation, such as a classification label or a numerical value.
-
Training: Neural networks learn by adjusting the weights of the connections between neurons based on the error of their predictions. This process, called backpropagation, involves comparing the network’s output to the actual values and using optimization algorithms (like gradient descent) to minimize the error.
Neural networks are used in various applications, including image recognition, natural language processing, and autonomous systems. They come in different types, such as feedforward networks, convolutional neural networks (CNNs), and recurrent neural networks (RNNs), each suited to different types of problems.
Activation Functions
Activation functions introduce non-linearity to the network and help determine if a neuron should be activated. The output of each neuron is computed by calculating the weighted sum of its inputs, adding a bias term, and then passing this sum through the activation function. This non-linear transformation enables the network to model complex patterns and relationships in the data. Without non-linearity, a neural network would essentially behave like a linear regression model, regardless of the number of layers it has.
- Sigmoid Function: Maps input values to a range between 0 and 1. The threshold here is effectively the value around which the function switches from producing values close to 0 to values close to 1.
- ReLU (Rectified Linear Unit): Outputs the input value if it is positive; otherwise, it outputs 0. The threshold here is 0, meaning any input below 0 results in no activation.
- Tanh (Hyperbolic Tangent): Maps inputs to a range between -1 and 1. It has thresholds around the values that cause the function to change its slope.