In machine learning, the term *manifold* is thrown around a lot—in

fact, there’s even a branch of machine learning dealing with learning

the structure of manifolds. Said branch is aptly named *manifold
learning*.

Inspired by fantastic visualizations of shapelet mining algorithms, this post aims to give a visual introduction to manifolds that is accessible for non-mathematicians and mathematicians alike.

Let’s start with a rough definition: a $d$-dimensional manifold is

something that *locally looks* like a $d$-dimensional Euclidean space,

i.e. like some $mathbb{R}^d$. To show that this is less

complicated than it sounds, we need to build some intuition by providing

examples. First of all, when mathematicians say that something ‘locally

looks’ like something else, they consider viewing that object from the

perspective of an extremely tiny bug that ‘lives’ on said

object (and cannot move away from it—but more about that

later).

According to the definition above a $1$-dimensional manifold would thus

be something that ‘locally looks’ like $mathbb{R}^1$. But it turns out

that we know $mathbb{R}^1$ or $mathbb{R}$ rather well: it is nothing but

the line of real numbers. This is how it looks like from ‘outside’:

And this is how it looks like from the perspective of a small bug

‘living’ on said line:

•

Hence, from the perspective of the bug, the line stretches onwards to

infinity, i.e. *into* your screen, without having any other

extents. In fact, the bug can only move ‘forward’ and ‘backward’, as

there are no other directions. The number line is thus an example of

a $1$-dimensional manifold. However, it is an exceptionally *boring*

example. We can find a better one by considering a *circle*. Here it is,

shown with its small inhabitant:

If you think about it, from the point of the bug, the circle and number

line ‘look’ the same. The bug can move forward or backward; there’s no

way of telling whether it is living on $mathbb{R}$ or on $S^1$, as

mathematicians like to call the circle. The reason for this notation is

that the circle is seen as a $1$-dimensional *sphere*. Notice that

mathematicians do *not* consider a sphere to be filled—it is only

the *outer* line that we are interested in. Think of the surface of

a balloon (we will come to that presently) rather than its interior.

Moving on to the second dimension, we are—according to our

definition—dealing with an object that locally looks like

$mathbb{R}^2$, the two-dimensional Euclidean space. It turns out that

we *do* know this space also from high school as the Euclidean plane,

or a *Cartesian coordinate system*. Usually, you denote the two axes

with $x$ and $y$:

If your education went anything like mine, you spent a lot of hours,

some happy and some unhappy, by adding mathematical objects to these

two axes. From the perspective of our small bug, $mathbb{R}^2$ looks

like this:

In other words, the bug can move in *two* independent directions here,

and its world will appear to be a large plane, much like the Great

Plains would appear to us.

Of course, the world of the bug does not have any edges. Again, this is

a somewhat boring example of a $2$-manifold. To get something

better, we can extend our previous example of the $1$-sphere into the

$2$-sphere $S^2$:

That looks awfully familiar—in fact, the $2$-sphere is nothing

else than the surface of a ‘ball’ in a $3$-dimensional space, just like

the $1$-sphere is the surface of a ‘disk’ in a $2$-dimensional

space (note the nice and symmetrical shift in dimensions here).

Interestingly, our poor bug has no easy way of checking whether it is

living on a sphere or on an infinite plane because it cannot move

*outside*, i.e. it does not have access to rocket technology.

Moreover, we assume that the bug is really small and does not notice the

curvature of the land. One might say that the bug has good reasons to

believe in a flat earth because everything is flat from its perspective.

At this point, you probably know where this is going, and you shall not

be disappointed: let us go to the third dimension, then! To represent

three dimensions, we again use a Cartesian coordinate system:

The order of the axes does not matter, but I like $y$ to point upwards

because I think of it as a ‘height’. Three dimensions should sound

somewhat familiar to you—it turns out that this time, *we* are

the bug! From our perspective, our universe has three spatial

dimensions, at least *locally*, so we are probably living

in (on?) a $3$-manifold. It is thus *our* turn to try to discover

some facts about the manifold we are living in, i.e. the shape

of the universe.

Given that our universe *might* be a $3$-manifold, what examples of

these manifolds are there? Again, the classical one would be the

$3$-sphere $S^3$. However, if you followed the discussion from above, you know

that the $3$-sphere ‘lives’ in $mathbb{R}^4$, a $4$-dimensional space.

These are typically hard to visualize—but it turns out that there

is a neat way to describe spheres of *all* dimensions. To see this,

let’s start with $d = 1$, the circle. With a radius of $r = 1$, also

known as unit radius, the circle contains all points that satisfy $x^2

+ y^2 = 1$:

These equations can be extended to higher dimensions without so much as

batting an eyelid! Hence, points on $S^2$, the $2$-sphere, are

characterized by satisfying $x^2 + y^2 + z^2 = 1$, while points on

$S^3$, the $3$-sphere, are characterized through $x^2 + y^2 + z^2 + w^2

= 1$. In other words, we need *four* coordinates to describe them. While

this is not surprising, it at least gives us an intuitive glimpse about

how to treat these objects analytically, even if we are incapable of

visualizing them correctly. At least at this point, you are now able to

describe the coordinates of *any* $d$-dimensional manifold.

All this knowledge about manifolds now needs to be put to some use.

Already in the $1$-dimensional example, we asked the question how a

bug might be able to figure out that they are living in $S^1$ or in

$mathbb{R}$. One easy algorithm requires only two ingredients:

- An infinitely long rope
- An infinite amount of time

By the description of these two ingredients we can already see that we

are now in mathematical territory! The algorithm is straightforward: a

bug that wants to figure out the shape of its manifold merely needs to

‘fasten’ the rope at one point and ‘unspool’ it. Next, it merely needs

to walk on in a randomly-selected direction, which must never be

changed, though. If the bug is living in $S^1$, at some point, it will

stumble over its rope. At this point, the algorithm can terminate and

the poor bug can finally rest.

Note that this algorithm will only terminate *if* the bug is indeed

living in some $S^1$, no matter how large its radius. If the bug is

indeed living in $mathbb{R}$, its journey will continue forever. A

somewhat dire fate. How might we extend this to $d = 2$? Obviously,

the walking is a little bit more difficult here, as there are *two*

independent directions. Nonetheless, if the bug suspects that it is

living in $S^2$, it could pick one direction at random, refer to it

as ‘North’, and follow it. At some point, it will re-cross its path

and thus figure out the shape of its universe.

There is a more elegant way, though. If you think back to high school

mathematics, you probably proved that the angles in a triangle sum to

180 degrees. It turns out that you can construct a triangle whose

angle sum is 270 degrees, provided you are living in $S^2$, and *not*

in $mathbb{R}^2$. Thus, our little bug could boldly declare that its

current location is *the* pole of its universe and follow these steps:

- Walk some distance in a straight line. This can be ensured

by (again) using copious amounts of rope. - Turn left by 90 degrees. This requires a bug-sized protractor, for

example, but we can imagine that a bug civilization that is smart

enough to develop mathematics is*also*smart enough to build tools

for their species. - Walk some distance in a straight line again.
- Turn left by 90 degrees and get back to where the bug started.

If this is repeated multiple times and the bug decides to use

a sufficiently large distance, it will be possible to describe—or

rather *inscribe*—a triangle with three right angles. Since this

cannot work in the plane, the bug is left with the conclusion that its

civilization is living in $S^2$! Here is an illustration of how such

a triangle might look:

The bug does not even have to go to *that* extreme, though, because it

is perfectly sufficient to find a *single* triangle whose angle sum is

more than 180 degrees in order to conclude that one is *not* living in

$mathbb{R}^2$. Of course, if the side lengths are sufficiently small,

the effects of living in such a curved space are negligible.

This little thought experiment illustrates *curvature*,

which is one of the fundamental concepts for describing manifolds.

If you enjoyed this foray into the fascinating world of manifolds,

I can recommend the book *The Shape of Space* by Jeffrey R. Weeks.

His writing style is very entertaining and on

geometrygames.org, he provides numerous programs to

visualize and experience numerous manifolds. If you ever wanted to fly

*through* a manifold or play mazes on a torus, *this* is the perfect website for you.

Until next time!