Max Pooling and Average Pooling in CNN
Complete Explanation with Numerical Examples, Formulas and Python Implementation
- Understand pooling layers in CNN.
- Understand pooling windows and stride.
- Calculate Max Pooling numerically.
- Calculate Average Pooling numerically.
- Understand output dimensions.
1. What is Pooling in CNN?
Pooling is a downsampling operation used in Convolutional Neural Networks (CNNs) to reduce the spatial dimensions of feature maps.
Pooling helps to:
- Reduce computational complexity.
- Reduce spatial dimensions.
- Retain important image features.
- Provide some tolerance to small changes in feature positions.
Two commonly used pooling techniques are:
- Max Pooling
- Average Pooling
2. What is a Pooling Window?
A pooling window (also called a pooling filter) is a small region that moves over the input feature map to perform a pooling operation.
For example, a pooling window of size \(2\times2\) processes four values at a time.
- Window size: Dimensions of the pooling region.
- Stride: Number of positions the window moves.
- Padding: Extra values added around the feature map.
3. Max Pooling in CNN
Definition: Max Pooling selects the maximum value from each pooling window.
Numerical Example
Consider the following \(4\times4\) input feature map:
A=\begin{bmatrix}
1&3&2&4\\
5&6&1&2\\
7&8&3&4\\
2&1&9&5
\end{bmatrix}
\]
Parameters:
- Pooling window: \(2\times2\)
- Stride: \(2\)
- Padding: \(0\)
Step 1: Divide into four pooling windows
Window 1:
\[
\begin{bmatrix}1&3\\5&6\end{bmatrix}
\]
\[
\max(1,3,5,6)=\boxed{6}
\]
Window 2:
\[
\begin{bmatrix}2&4\\1&2\end{bmatrix}
\]
\[
\max(2,4,1,2)=\boxed{4}
\]
Window 3:
\[
\begin{bmatrix}7&8\\2&1\end{bmatrix}
\]
\[
\max(7,8,2,1)=\boxed{8}
\]
Window 4:
\[
\begin{bmatrix}3&4\\9&5\end{bmatrix}
\]
\[
\max(3,4,9,5)=\boxed{9}
\]
Step 2: Final Max Pooling Output
\boxed{
A’=\begin{bmatrix}
6&4\\
8&9
\end{bmatrix}
}
\]
Output dimensions: \(2\times2\)
4. Average Pooling in CNN
Definition: Average Pooling calculates the arithmetic mean of all values within each pooling window.
Numerical Example
Using the same input feature map:
\[
A=\begin{bmatrix}
1&3&2&4\\
5&6&1&2\\
7&8&3&4\\
2&1&9&5
\end{bmatrix}
\]
Pooling window: \(2\times2\), Stride: \(2\).
Step 1: Calculate the average of each window
Window 1:
\[
\frac{1+3+5+6}{4}
=\frac{15}{4}=3.75
\]
Window 2:
\[
\frac{2+4+1+2}{4}
=\frac{9}{4}=2.25
\]
Window 3:
\[
\frac{7+8+2+1}{4}
=\frac{18}{4}=4.5
\]
Window 4:
\[
\frac{3+4+9+5}{4}
=\frac{21}{4}=5.25
\]
Step 2: Final Average Pooling Output
\boxed{
A’=\begin{bmatrix}
3.75&2.25\\
4.5&5.25
\end{bmatrix}
}
\]
Output dimensions: \(2\times2\)
5. Difference Between Max Pooling and Average Pooling
| Feature | Max Pooling | Average Pooling |
|---|---|---|
| Operation | Maximum value | Mean value |
| Feature retention | Strongest activation | Average activation |
| Noise | May retain strong noise | Can smooth local variations |
| Computation | Maximum selection | Addition and division |
| Purpose | Feature extraction | Feature summarization |
6. Output Dimension Formula for Pooling
The spatial output dimension formula is:
\boxed{
O=\left\lfloor\frac{N+2P-F}{S}\right\rfloor+1
}
\]
Where:
- \(N\) = Input dimension
- \(P\) = Padding
- \(F\) = Pooling window size
- \(S\) = Stride
- \(O\) = Output dimension
Numerical Example
Given:
\[
N=8,\quad F=2,\quad P=0,\quad S=2
\]
Substituting:
\[
O=\left\lfloor\frac{8+2(0)-2}{2}\right\rfloor+1
\]
\[
O=\lfloor3\rfloor+1
\]
\[
\boxed{O=4}
\]
Therefore, an \(8\times8\) feature map becomes a \(4\times4\) feature map.
7. Global Average Pooling (GAP)
Global Average Pooling calculates the average of all spatial values in each feature map.
Consider:
\[
A=\begin{bmatrix}
2&4\\
6&8
\end{bmatrix}
\]
Global average:
\[
GAP=\frac{2+4+6+8}{4}
\]
\[
\boxed{GAP=5}
\]
Unlike ordinary average pooling, Global Average Pooling uses the entire spatial feature map as one pooling region.
It is commonly used before classification layers in CNN architectures.
8. PyTorch Implementation
import torch
import torch.nn as nn
# Input feature map
x = torch.tensor([[
[[1., 3., 2., 4.],
[5., 6., 1., 2.],
[7., 8., 3., 4.],
[2., 1., 9., 5.]]
]])
# Max Pooling
max_pool = nn.MaxPool2d(
kernel_size=2,
stride=2
)
# Average Pooling
avg_pool = nn.AvgPool2d(
kernel_size=2,
stride=2
)
print("Max Pooling:")
print(max_pool(x))
print("Average Pooling:")
print(avg_pool(x))
Output
Max Pooling:
tensor([[[[6., 4.],
[8., 9.]]]])
Average Pooling:
tensor([[[[3.7500, 2.2500],
[4.5000, 5.2500]]]])
9. Practice Question for Students
Consider the following feature map:
\[
A=\begin{bmatrix}
2&4&6&8\\
1&3&5&7\\
9&2&4&6\\
8&1&3&5
\end{bmatrix}
\]
Apply a \(2\times2\) pooling window with stride \(2\).
- Calculate the Max Pooling output.
- Calculate the Average Pooling output.
- Calculate the output dimensions.
- Explain the difference between both pooling techniques.
10. Key Takeaways
- 🔵 Max Pooling selects the maximum value from each window.
- 🟢 Average Pooling calculates the mean of each window.
- 🟣 Both techniques reduce spatial dimensions.
- 🟠 Pooling layers generally do not have learnable weights.
- 🔴 Global Average Pooling summarizes an entire feature map into one value.
PostNetwork Academy
Learn • Build • Innovate with Artificial Intelligence
Explore Machine Learning, Deep Learning, Statistics, Robotics and Generative AI.