Max Pooling and Average Pooling in CNN

Max Pooling and Average Pooling in CNN

Complete Explanation with Numerical Examples, Formulas and Python Implementation

Learning Objectives
  • Understand pooling layers in CNN.
  • Understand pooling windows and stride.
  • Calculate Max Pooling numerically.
  • Calculate Average Pooling numerically.
  • Understand output dimensions.

1. What is Pooling in CNN?

Pooling is a downsampling operation used in Convolutional Neural Networks (CNNs) to reduce the spatial dimensions of feature maps.

Pooling helps to:

  • Reduce computational complexity.
  • Reduce spatial dimensions.
  • Retain important image features.
  • Provide some tolerance to small changes in feature positions.

Two commonly used pooling techniques are:

  1. Max Pooling
  2. Average Pooling

2. What is a Pooling Window?

A pooling window (also called a pooling filter) is a small region that moves over the input feature map to perform a pooling operation.

For example, a pooling window of size \(2\times2\) processes four values at a time.

Pooling Parameters
  • Window size: Dimensions of the pooling region.
  • Stride: Number of positions the window moves.
  • Padding: Extra values added around the feature map.

3. Max Pooling in CNN

Definition: Max Pooling selects the maximum value from each pooling window.

Numerical Example

Consider the following \(4\times4\) input feature map:

\[
A=\begin{bmatrix}
1&3&2&4\\
5&6&1&2\\
7&8&3&4\\
2&1&9&5
\end{bmatrix}
\]

Parameters:

  • Pooling window: \(2\times2\)
  • Stride: \(2\)
  • Padding: \(0\)

Step 1: Divide into four pooling windows

Window 1:

\[
\begin{bmatrix}1&3\\5&6\end{bmatrix}
\]

\[
\max(1,3,5,6)=\boxed{6}
\]

Window 2:

\[
\begin{bmatrix}2&4\\1&2\end{bmatrix}
\]

\[
\max(2,4,1,2)=\boxed{4}
\]

Window 3:

\[
\begin{bmatrix}7&8\\2&1\end{bmatrix}
\]

\[
\max(7,8,2,1)=\boxed{8}
\]

Window 4:

\[
\begin{bmatrix}3&4\\9&5\end{bmatrix}
\]

\[
\max(3,4,9,5)=\boxed{9}
\]

Step 2: Final Max Pooling Output

\[
\boxed{
A’=\begin{bmatrix}
6&4\\
8&9
\end{bmatrix}
}
\]
Output dimensions: \(2\times2\)

4. Average Pooling in CNN

Definition: Average Pooling calculates the arithmetic mean of all values within each pooling window.

Numerical Example

Using the same input feature map:

\[
A=\begin{bmatrix}
1&3&2&4\\
5&6&1&2\\
7&8&3&4\\
2&1&9&5
\end{bmatrix}
\]

Pooling window: \(2\times2\), Stride: \(2\).

Step 1: Calculate the average of each window

Window 1:

\[
\frac{1+3+5+6}{4}
=\frac{15}{4}=3.75
\]

Window 2:

\[
\frac{2+4+1+2}{4}
=\frac{9}{4}=2.25
\]

Window 3:

\[
\frac{7+8+2+1}{4}
=\frac{18}{4}=4.5
\]

Window 4:

\[
\frac{3+4+9+5}{4}
=\frac{21}{4}=5.25
\]

Step 2: Final Average Pooling Output

\[
\boxed{
A’=\begin{bmatrix}
3.75&2.25\\
4.5&5.25
\end{bmatrix}
}
\]
Output dimensions: \(2\times2\)

5. Difference Between Max Pooling and Average Pooling

Feature Max Pooling Average Pooling
Operation Maximum value Mean value
Feature retention Strongest activation Average activation
Noise May retain strong noise Can smooth local variations
Computation Maximum selection Addition and division
Purpose Feature extraction Feature summarization

6. Output Dimension Formula for Pooling

The spatial output dimension formula is:

\[
\boxed{
O=\left\lfloor\frac{N+2P-F}{S}\right\rfloor+1
}
\]

Where:

  • \(N\) = Input dimension
  • \(P\) = Padding
  • \(F\) = Pooling window size
  • \(S\) = Stride
  • \(O\) = Output dimension

Numerical Example

Given:

\[
N=8,\quad F=2,\quad P=0,\quad S=2
\]

Substituting:

\[
O=\left\lfloor\frac{8+2(0)-2}{2}\right\rfloor+1
\]

\[
O=\lfloor3\rfloor+1
\]

\[
\boxed{O=4}
\]

Therefore, an \(8\times8\) feature map becomes a \(4\times4\) feature map.

Important Note: The output-size formula is the same as the standard convolution output-size formula under the same padding and stride conventions. However, convolution performs kernel multiplication and summation, whereas pooling performs maximum selection or averaging.

7. Global Average Pooling (GAP)

Global Average Pooling calculates the average of all spatial values in each feature map.

Consider:

\[
A=\begin{bmatrix}
2&4\\
6&8
\end{bmatrix}
\]

Global average:

\[
GAP=\frac{2+4+6+8}{4}
\]

\[
\boxed{GAP=5}
\]

Unlike ordinary average pooling, Global Average Pooling uses the entire spatial feature map as one pooling region.

It is commonly used before classification layers in CNN architectures.

8. PyTorch Implementation

import torch
import torch.nn as nn

# Input feature map
x = torch.tensor([[
    [[1., 3., 2., 4.],
     [5., 6., 1., 2.],
     [7., 8., 3., 4.],
     [2., 1., 9., 5.]]
]])

# Max Pooling
max_pool = nn.MaxPool2d(
    kernel_size=2,
    stride=2
)

# Average Pooling
avg_pool = nn.AvgPool2d(
    kernel_size=2,
    stride=2
)

print("Max Pooling:")
print(max_pool(x))

print("Average Pooling:")
print(avg_pool(x))

Output

Max Pooling:
tensor([[[[6., 4.],
          [8., 9.]]]])

Average Pooling:
tensor([[[[3.7500, 2.2500],
          [4.5000, 5.2500]]]])

9. Practice Question for Students

Consider the following feature map:

\[
A=\begin{bmatrix}
2&4&6&8\\
1&3&5&7\\
9&2&4&6\\
8&1&3&5
\end{bmatrix}
\]

Apply a \(2\times2\) pooling window with stride \(2\).

Solve the following questions:
  1. Calculate the Max Pooling output.
  2. Calculate the Average Pooling output.
  3. Calculate the output dimensions.
  4. Explain the difference between both pooling techniques.

10. Key Takeaways

  • 🔵 Max Pooling selects the maximum value from each window.
  • 🟢 Average Pooling calculates the mean of each window.
  • 🟣 Both techniques reduce spatial dimensions.
  • 🟠 Pooling layers generally do not have learnable weights.
  • 🔴 Global Average Pooling summarizes an entire feature map into one value.

 PostNetwork Academy

Learn • Build • Innovate with Artificial Intelligence

Explore Machine Learning, Deep Learning, Statistics, Robotics and Generative AI.

Visit PostNetwork Academy

©Postnetwork-All rights reserved.