Mastix Ed logoMastix Ed presents

Luna and the Seeing Forest

How does a computer look at a cat? Follow Luna, her robot friend Pixi and wise Professor Spark to find out how a Convolutional Neural Network (CNN) sees the world.

1The photo of the cat

One sunny morning, Luna found a photo of a cat. She turned to her robot friend Pixi with a big question.
Luna: Pixi, can a computer see this cat the way I can?
Pixi: Beep boop! Not quite. A computer doesn't see a picture. It sees a giant grid of tiny squares called pixels (that's where my name comes from!), and every square is just a number.
πŸ‘† Try it: paint with numbers
0 = white paper  Β·  9 = darkest ink

Tap any square to change its number. The computer's whole "picture" is just this table!

Prof. Spark: Exactly! A CNN is a computer brain that learns to look at all those numbers and find the cat hiding inside them.

2The magic magnifying lens

Deep in the forest, Professor Spark held up a very special magnifying glass.
Luna: But how does it find anything in all those numbers?
Prof. Spark: With a magic lens! It is a tiny 3Γ—3 window that slides over the picture, one step at a time. At every spot it asks, "Is my special pattern here?" and writes down one number.
Pixi: Beep! Let me try it. Slide the lens below and watch what I find!
πŸ‘† Try it: slide the lens
The picture
(a dark square)
The lens
(its 9 numbers)
What the lens found

light β†’ dark edgedark β†’ light edgenothing here
Prof. Spark: Grown-ups call the lens a filter. Our new little picture of "where the edges are" is called a feature map.

3The shrinking spell

As the sun set, Professor Spark showed them a tidy-up trick.
Prof. Spark: Too many numbers make a computer sleepy. So we shrink! Look at each 2Γ—2 box and keep only the loudest (biggest) number. That's called pooling.
Luna: So we keep the important clues and throw away the extra?
πŸ‘† Try it: tap a "?" to shrink a box
Before (16 numbers)
➜
After (only 4!)

Tap a question mark to see which number wins its box.

Pixi: Yes! The picture gets smaller, but the best clues survive. Faster and smarter, beep!

4The forest of layers

A real CNN stacks lots of lenses and shrinking spells, like climbing a tree one branch at a time.
Prof. Spark: Early layers spot simple things. Each next layer combines the clues into bigger ideas. Climb up and see!
πŸ‘† Try it: climb the layers
Luna: So it goes from lines, to shapes, to ears and whiskers, to "That's a cat!"
Prof. Spark: Right! The last layer works like a voting booth. It's called a fully connected layer, and it says how sure it is.

5Practice makes pawsome

But Pixi wasn't built knowing cats. At first, its lens numbers were totally random!
Luna: Pixi, how did you learn to spot cats?
Pixi: Practice! I guess, someone tells me the real answer, and I make tiny tweaks to my lens numbers. Then again, and again, with thousands of photos.
πŸ‘† Try it: train Pixi
Prof. Spark: After enough practice, Pixi can spot cats it has never seen before. That's what learning means for a CNN!

6The CNN family tree

On the hill stood a giant tree, and every lantern on it was a different kind of CNN.
Prof. Spark: All CNNs use lenses, shrinking and layers. But the family has lots of cousins with different superpowers.
Pixi: Beep boop! Some read pictures, some listen to music, some watch videos. Tap a lantern to meet them!
πŸ‘† Try it: meet the cousins

7Professor Spark's moonlight quiz

Under the stars, Professor Spark had four questions to earn a Junior Pixel Detective badge.
You earned the Junior Pixel Detective badge!
For grown-ups: story words vs. real words
In the storyReal term
Pixels as numbersAn image is a matrix of intensity values (three matrices for colour: R, G, B)
Magic lensFilter or kernel; sliding it over the image is the convolution operation
What the lens foundFeature map (activation map)
Shrinking spellMax pooling (downsampling)
Forest of layersStacked convolution and pooling layers; early layers learn edges, later ones parts and objects
Voting boothFully connected layer plus softmax, which gives class probabilities
Tiny tweaksBackpropagation with gradient descent, adjusting filter weights to reduce the error
Shortcut bridgesResidual (skip) connections