What Is a Manifold?

In machine learning, the term manifold is thrown around a lot—in
fact, there’s even a branch of machine learning dealing with learning
the structure of manifolds. Said branch is aptly named manifold
learning.
Inspired by fantastic visualizations of shapelet mining algorithms, this post aims to give a visual introduction to manifolds that is accessible for non-mathematicians and mathematicians alike.

Let’s start with a rough definition: a $d$-dimensional manifold is
something that locally looks like a $d$-dimensional Euclidean space,
i.e. like some $mathbb{R}^d$. To show that this is less
complicated than it sounds, we need to build some intuition by providing
examples. First of all, when mathematicians say that something ‘locally
looks’ like something else, they consider viewing that object from the
perspective of an extremely tiny bug that ‘lives’ on said
object (and cannot move away from it—but more about that
later).

According to the definition above a $1$-dimensional manifold would thus
be something that ‘locally looks’ like $mathbb{R}^1$. But it turns out
that we know $mathbb{R}^1$ or $mathbb{R}$ rather well: it is nothing but
the line of real numbers. This is how it looks like from ‘outside’:

And this is how it looks like from the perspective of a small bug
‘living’ on said line:

•

Hence, from the perspective of the bug, the line stretches onwards to
infinity, i.e. into your screen, without having any other
extents. In fact, the bug can only move ‘forward’ and ‘backward’, as
there are no other directions. The number line is thus an example of
a $1$-dimensional manifold. However, it is an exceptionally boring
example. We can find a better one by considering a circle. Here it is,
shown with its small inhabitant:

If you think about it, from the point of the bug, the circle and number
line ‘look’ the same. The bug can move forward or backward; there’s no
way of telling whether it is living on $mathbb{R}$ or on $S^1$, as
mathematicians like to call the circle. The reason for this notation is
that the circle is seen as a $1$-dimensional sphere. Notice that
mathematicians do not consider a sphere to be filled—it is only
the outer line that we are interested in. Think of the surface of
a balloon (we will come to that presently) rather than its interior.

Moving on to the second dimension, we are—according to our
definition—dealing with an object that locally looks like
$mathbb{R}^2$, the two-dimensional Euclidean space. It turns out that
we do know this space also from high school as the Euclidean plane,
or a Cartesian coordinate system. Usually, you denote the two axes
with $x$ and $y$:

If your education went anything like mine, you spent a lot of hours,
some happy and some unhappy, by adding mathematical objects to these
two axes. From the perspective of our small bug, $mathbb{R}^2$ looks
like this:

In other words, the bug can move in two independent directions here,
and its world will appear to be a large plane, much like the Great
Plains would appear to us.
Of course, the world of the bug does not have any edges. Again, this is
a somewhat boring example of a $2$-manifold. To get something
better, we can extend our previous example of the $1$-sphere into the
$2$-sphere $S^2$:

That looks awfully familiar—in fact, the $2$-sphere is nothing
else than the surface of a ‘ball’ in a $3$-dimensional space, just like
the $1$-sphere is the surface of a ‘disk’ in a $2$-dimensional
space (note the nice and symmetrical shift in dimensions here).
Interestingly, our poor bug has no easy way of checking whether it is
living on a sphere or on an infinite plane because it cannot move
outside, i.e. it does not have access to rocket technology.
Moreover, we assume that the bug is really small and does not notice the
curvature of the land. One might say that the bug has good reasons to
believe in a flat earth because everything is flat from its perspective.

At this point, you probably know where this is going, and you shall not
be disappointed: let us go to the third dimension, then! To represent
three dimensions, we again use a Cartesian coordinate system:

The order of the axes does not matter, but I like $y$ to point upwards
because I think of it as a ‘height’. Three dimensions should sound
somewhat familiar to you—it turns out that this time, we are
the bug! From our perspective, our universe has three spatial
dimensions, at least locally, so we are probably living
in (on?) a $3$-manifold. It is thus our turn to try to discover
some facts about the manifold we are living in, i.e. the shape
of the universe.

Given that our universe might be a $3$-manifold, what examples of
these manifolds are there? Again, the classical one would be the
$3$-sphere $S^3$. However, if you followed the discussion from above, you know
that the $3$-sphere ‘lives’ in $mathbb{R}^4$, a $4$-dimensional space.
These are typically hard to visualize—but it turns out that there
is a neat way to describe spheres of all dimensions. To see this,
let’s start with $d = 1$, the circle. With a radius of $r = 1$, also
known as unit radius, the circle contains all points that satisfy $x^2
+ y^2 = 1$:

These equations can be extended to higher dimensions without so much as
batting an eyelid! Hence, points on $S^2$, the $2$-sphere, are
characterized by satisfying $x^2 + y^2 + z^2 = 1$, while points on
$S^3$, the $3$-sphere, are characterized through $x^2 + y^2 + z^2 + w^2
= 1$. In other words, we need four coordinates to describe them. While
this is not surprising, it at least gives us an intuitive glimpse about
how to treat these objects analytically, even if we are incapable of
visualizing them correctly. At least at this point, you are now able to
describe the coordinates of any $d$-dimensional manifold.

All this knowledge about manifolds now needs to be put to some use.
Already in the $1$-dimensional example, we asked the question how a
bug might be able to figure out that they are living in $S^1$ or in
$mathbb{R}$. One easy algorithm requires only two ingredients:

1. An infinitely long rope
2. An infinite amount of time

By the description of these two ingredients we can already see that we
are now in mathematical territory! The algorithm is straightforward: a
bug that wants to figure out the shape of its manifold merely needs to
‘fasten’ the rope at one point and ‘unspool’ it. Next, it merely needs
to walk on in a randomly-selected direction, which must never be
changed, though. If the bug is living in $S^1$, at some point, it will
stumble over its rope. At this point, the algorithm can terminate and
the poor bug can finally rest.

Note that this algorithm will only terminate if the bug is indeed
living in some $S^1$, no matter how large its radius. If the bug is
indeed living in $mathbb{R}$, its journey will continue forever. A
somewhat dire fate. How might we extend this to $d = 2$? Obviously,
the walking is a little bit more difficult here, as there are two
independent directions. Nonetheless, if the bug suspects that it is
living in $S^2$, it could pick one direction at random, refer to it
as ‘North’, and follow it. At some point, it will re-cross its path
and thus figure out the shape of its universe.

There is a more elegant way, though. If you think back to high school
mathematics, you probably proved that the angles in a triangle sum to
180 degrees. It turns out that you can construct a triangle whose
angle sum is 270 degrees, provided you are living in $S^2$, and not
in $mathbb{R}^2$. Thus, our little bug could boldly declare that its
current location is the pole of its universe and follow these steps:

1. Walk some distance in a straight line. This can be ensured
   by (again) using copious amounts of rope.
2. Turn left by 90 degrees. This requires a bug-sized protractor, for
   example, but we can imagine that a bug civilization that is smart
   enough to develop mathematics is also smart enough to build tools
   for their species.
3. Walk some distance in a straight line again.
4. Turn left by 90 degrees and get back to where the bug started.

If this is repeated multiple times and the bug decides to use
a sufficiently large distance, it will be possible to describe—or
rather inscribe—a triangle with three right angles. Since this
cannot work in the plane, the bug is left with the conclusion that its
civilization is living in $S^2$! Here is an illustration of how such
a triangle might look:

The bug does not even have to go to that extreme, though, because it
is perfectly sufficient to find a single triangle whose angle sum is
more than 180 degrees in order to conclude that one is not living in
$mathbb{R}^2$. Of course, if the side lengths are sufficiently small,
the effects of living in such a curved space are negligible.

This little thought experiment illustrates curvature,
which is one of the fundamental concepts for describing manifolds.
If you enjoyed this foray into the fascinating world of manifolds,
I can recommend the book The Shape of Space by Jeffrey R. Weeks.
His writing style is very entertaining and on
geometrygames.org, he provides numerous programs to
visualize and experience numerous manifolds. If you ever wanted to fly
through a manifold or play mazes on a torus, this is the perfect website for you.

Until next time!