__del__( self )

Eaten by the Python.

Data Animations With Python and MoviePy

| Comments

Python has some great data visualization librairies, but few can render GIFs or video animations. This post shows how to use MoviePy as a generic animation plugin for any other library.

MoviePy lets you define custom animations with a function make_frame(t), which returns the video frame corresponding to time t (in seconds):

1
2
3
4
5
6
7
8
9
10
11
12

from moviepy.editor import VideoClip

def make_frame(t):
    """ returns an image of the frame at time t """
    # ... create the frame with any library
    return frame_for_time_t # (Height x Width x 3) Numpy array

animation = VideoClip(make_frame, duration=3) # 3-second clip

# For the export, many options/formats/optimizations are supported
animation.write_videofile("my_animation.mp4", fps=24) # export as video
animation.write_gif("my_animation.gif", fps=24) # export as GIF (slow)

In previous posts I used this method to animate vector graphics (with the library Gizeh), and ray-traced 3D scenes (generated by POV-Ray). This post covers the scientific libraries Mayavi, Vispy, Matplotlib, Numpy, and Scikit-image.

Animations with Mayavi

Mayavi is a Python module for interactive 3D data visualization with a simple interface. In this first example we animate a surface whose elevation depends on the time t:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

import numpy as np
import mayavi.mlab as mlab
import  moviepy.editor as mpy

duration= 2 # duration of the animation in seconds (it will loop)

# MAKE A FIGURE WITH MAYAVI

fig_myv = mlab.figure(size=(220,220), bgcolor=(1,1,1))
X, Y = np.linspace(-2,2,200), np.linspace(-2,2,200)
XX, YY = np.meshgrid(X,Y)
ZZ = lambda d: np.sinc(XX**2+YY**2)+np.sin(XX+d)

# ANIMATE THE FIGURE WITH MOVIEPY, WRITE AN ANIMATED GIF

def make_frame(t):
    mlab.clf() # clear the figure (to reset the colors)
    mlab.mesh(YY,XX,ZZ(2*np.pi*t/duration), figure=fig_myv)
    return mlab.screenshot(antialiased=True)

animation = mpy.VideoClip(make_frame, duration=duration)
animation.write_gif("sinc.gif", fps=20)

Another example with a wireframe mesh whose coordinates and view angle depend on the time :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

import numpy as np
import mayavi.mlab as mlab
import  moviepy.editor as mpy

duration = 2 # duration of the animation in seconds (it will loop)

# MAKE A FIGURE WITH MAYAVI

fig = mlab.figure(size=(500, 500), bgcolor=(1,1,1))

u = np.linspace(0,2*np.pi,100)
xx,yy,zz = np.cos(u), np.sin(3*u), np.sin(u) # Points
l = mlab.plot3d(xx,yy,zz, representation="wireframe", tube_sides=5,
                line_width=.5, tube_radius=0.2, figure=fig)

# ANIMATE THE FIGURE WITH MOVIEPY, WRITE AN ANIMATED GIF

def make_frame(t):
    """ Generates and returns the frame for time t. """
    y = np.sin(3*u)*(0.2+0.5*np.cos(2*np.pi*t/duration))
    l.mlab_source.set(y = y) # change y-coordinates of the mesh
    mlab.view(azimuth= 360*t/duration, distance=9) # camera angle
    return mlab.screenshot(antialiased=True) # return a RGB image

animation = mpy.VideoClip(make_frame, duration=duration).resize(0.5)
# Video generation takes 10 seconds, GIF generation takes 25s
animation.write_videofile("wireframe.mp4", fps=20)
animation.write_gif("wireframe.gif", fps=20)

As Mayavi relies on the powerful ITK visualization engine it can also process complex datasets. Here is an animation derived from a Mayavi example:

Animations with Vispy

Vispy is another interactive 3D data visualization library, based on OpenGL. As for Mayavi, we first create a figure and a mesh, that we animate with MoviePy.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

from moviepy.editor import VideoClip
import numpy as np
from vispy import app, scene
from vispy.gloo.util import _screenshot

canvas = scene.SceneCanvas(keys='interactive')
view = canvas.central_widget.add_view()
view.set_camera('turntable', mode='perspective', up='z', distance=2,
                azimuth=30., elevation=65.)

xx, yy = np.arange(-1,1,.02),np.arange(-1,1,.02)
X,Y = np.meshgrid(xx,yy)
R = np.sqrt(X**2+Y**2)
Z = lambda t : 0.1*np.sin(10*R-2*np.pi*t)
surface = scene.visuals.SurfacePlot(x= xx-0.1, y=yy+0.2, z= Z(0),
                        shading='smooth', color=(0.5, 0.5, 1, 1))
view.add(surface)
canvas.show()

# ANIMATE WITH MOVIEPY

def make_frame(t):
    surface.set_data(z = Z(t)) # Update the mathematical surface
    canvas.on_draw(None) # Update the image on Vispy's canvas
    return _screenshot((0,0,canvas.size[0],canvas.size[1]))[:,:,:3]

animation = VideoClip(make_frame, duration=1).resize(width=350)
animation.write_gif('sinc_vispy.gif', fps=20, opt='OptimizePlus')

Here are more advanced examples (derived from the Vispy gallery) where C code snippets are embedded in the Python code to fine-tune the 3D shaders:

Animations with Matplotlib

The 2D/3D plotting library Matplotlib already has an animation module, but I found that MoviePy produces lighter, better quality videos, while being up to two times faster (not sure why, see here for more details). Here is how you animate Matplotlib with MoviePy:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import matplotlib.pyplot as plt
import numpy as np
from moviepy.video.io.bindings import mplfig_to_npimage
import moviepy.editor as mpy

# DRAW A FIGURE WITH MATPLOTLIB

duration = 2

fig_mpl, ax = plt.subplots(1,figsize=(5,3), facecolor='white')
xx = np.linspace(-2,2,200) # the x vector
zz = lambda d: np.sinc(xx**2)+np.sin(xx+d) # the (changing) z vector
ax.set_title("Elevation in y=0")
ax.set_ylim(-1.5,2.5)
line, = ax.plot(xx, zz(0), lw=3)

# ANIMATE WITH MOVIEPY (UPDATE THE CURVE FOR EACH t). MAKE A GIF.

def make_frame_mpl(t):
    line.set_ydata( zz(2*np.pi*t/duration))  # <= Update the curve
    return mplfig_to_npimage(fig_mpl) # RGB image of the figure

animation =mpy.VideoClip(make_frame_mpl, duration=duration)
animation.write_gif("sinc_mpl.gif", fps=20)

Matplotlib has many beautiful themes and works well with numerical modules like Pandas or Scikit-Learn. Let us watch a SVM classifier getting a better understanding of the map as the number of training point increases.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

import numpy as np
import matplotlib.pyplot as plt
from sklearn import svm # sklearn = scikit-learn
from sklearn.datasets import make_moons
from moviepy.editor import VideoClip
from moviepy.video.io.bindings import mplfig_to_npimage

X, Y = make_moons(50, noise=0.1, random_state=2) # semi-random data

fig, ax = plt.subplots(1, figsize=(4, 4), facecolor=(1,1,1))
fig.subplots_adjust(left=0, right=1, bottom=0)
xx, yy = np.meshgrid(np.linspace(-2,3,500), np.linspace(-1,2,500))

def make_frame(t):
    ax.clear()
    ax.axis('off')
    ax.set_title("SVC classification", fontsize=16)

    classifier = svm.SVC(gamma=2, C=1)
    # the varying weights make the points appear one after the other
    weights = np.minimum(1, np.maximum(0, t**2+10-np.arange(50)))
    classifier.fit(X, Y, sample_weight=weights)
    Z = classifier.decision_function(np.c_[xx.ravel(), yy.ravel()])
    Z = Z.reshape(xx.shape)
    ax.contourf(xx, yy, Z, cmap=plt.cm.bone, alpha=0.8,
                vmin=-2.5, vmax=2.5, levels=np.linspace(-2,2,20))
    ax.scatter(X[:,0], X[:,1], c=Y, s=50*weights, cmap=plt.cm.bone)

    return mplfig_to_npimage(fig)

animation = VideoClip(make_frame, duration = 7)
animation.write_gif("svm.gif", fps=15)

Put simply, the background colors tell us where the classifier thinks the black points and white points belong. At the begining it has no real clue, but as more points appear it progressively understands that they are distributed along moon-shaped regions.

Animations with Numpy

If you are working with Numpy arrays (Numpy is the central numerical library in Python), you don’t need any external plotting library, you can feed the arrays directly to MoviePy.

This is well illustrated by this simulation of a zombie outbreak in France (inspired by this blog post by Max Berggren). France is modelled as a grid (Numpy array) on which all the computations for dispersion and infection are done. At regular intervals, a few Numpy operations tranform the grid into a valid RGB image, and send it to MoviePy.

Putting animations together

What is better than an animation ? Two animations ! You can take advantage of MoviePy’s video composition capabilities to mix animations from different libraries:

1
2
3
4
5
6

import moviepy.editor as mpy
# We use the GIFs generated earlier to avoid recomputing the animations.
clip_mayavi = mpy.VideoFileClip("sinc.gif")
clip_mpl = mpy.VideoFileClip("sinc_mpl.gif").resize(height=clip_mayavi.h)
animation = mpy.clips_array([[clip_mpl, clip_mayavi]])
animation.write_gif("sinc_plot.gif", fps=20)

Or for something more artistic:

1
2
3
4
5
6
7
8

# Make the white color transparent in clip_mayavi
clip_mayavi2 = (clip_mayavi.fx( mpy.vfx.mask_color, [255,255,255])
                .set_opacity(.4) # whole clip is semi-transparent
                .resize(height=0.85*clip_mpl.h)
                .set_pos('center'))

animation = mpy.CompositeVideoClip([clip_mpl, clip_mayavi2])
animation.write_gif("sinc_plot2.gif", fps=20)

It may be a tad too flashy, but sometimes you must give your audience something they can tweet.

You can also annotate the animations, which is useful when comparing different filters or algorithms. Let’s display four image transformations from the library Scikit-image:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

import moviepy.editor as mpy
import skimage.exposure as ske # rescaling, histogram eq.
import skimage.filter as skf # gaussian blur

clip = mpy.VideoFileClip("sinc.gif")
gray = clip.fx(mpy.vfx.blackwhite).to_mask()

def apply_effect(effect, title, **kw):
    """ Returns a clip with the effect applied and a title"""
    filtr = lambda im: effect(im, **kw)
    new_clip = gray.fl_image(filtr).to_RGB()
    txt = (mpy.TextClip(title, font="Purisa-Bold", fontsize=15)
           .set_position(("center","top"))
           .set_duration(clip.duration))
    return mpy.CompositeVideoClip([new_clip,txt])

# Apply 4 different effects to the original animation
equalized = apply_effect(ske.equalize_hist, "Equalized")
rescaled  = apply_effect(ske.rescale_intensity, "Rescaled")
adjusted  = apply_effect(ske.adjust_log, "Adjusted")
blurred   = apply_effect(skf.gaussian_filter, "Blurred", sigma=4)

# Put the clips together on a 2x2 grid, and write to a file.
finalclip = mpy.clips_array([[ equalized, adjusted ],
                             [ blurred,   rescaled ]])
final_clip.write_gif("test2x2.gif", fps=20)

If we replace CompositeVideoClip and clips_array by concatenate_videoclips we get a title-effect type animation:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

import moviepy.editor as mpy
import skimage.exposure as ske
import skimage.filter as skf

clip = mpy.VideoFileClip("sinc.gif")
gray = clip.fx(mpy.vfx.blackwhite).to_mask()

def apply_effect(effect, label, **kw):
    """ Returns a clip with the effect applied and a top label"""
    filtr = lambda im: effect(im, **kw)
    new_clip = gray.fl_image(filtr).to_RGB()
    txt = (mpy.TextClip(label, font="Amiri-Bold", fontsize=25,
                        bg_color='white', size=new_clip.size)
           .set_position(("center"))
           .set_duration(1))
    return mpy.concatenate_videoclips([txt, new_clip])

equalized = apply_effect(ske.equalize_hist, "Equalized")
rescaled  = apply_effect(ske.rescale_intensity, "Rescaled")
adjusted  = apply_effect(ske.adjust_log, "Adjusted")
blurred   = apply_effect(skf.gaussian_filter, "Blurred", sigma=4)

clips = [equalized, adjusted, blurred, rescaled]
animation = mpy.concatenate_videoclips(clips)
animation.write_gif("sinc_cat.gif", fps=15)

Finally, MoviePy will be particularly practical when dealing with video data, as it is its first job. For our last example we estimate the size of a growing bacterial population by thresholding the video frames and counting the white pixels. The third panel shows that the population size grows exponentially in time.

One library to animate them all ?

I hope to have given you enough recipes to impress your colleagues at your next presentation. Any other library could be animated with MoviePy, as long as its output can be converted to a Numpy array.

Some libraries have their own animation modules, but these are usually a pain to fix and maintain. Thanks to the many users who have tested it in very different contexts, MoviePy seems to have become stable (or people stopped reporting bugs), and can be adapted to many situations. There is still a lot to do, but it would be nice if authors started relying on it for video and GIF rendering, like Pandas and Scikit-Learn rely on Matplotlib for plotting.

For completeness, and because it may better fit your needs, I must mention ImageIO, another Python library with video writing capabilities which focuses on providing a very simple interface to read or write any kind of image, video or volumetric data. For instance you use imwrite() to write any image, mimwrite() for any video/GIF, volwrite() for volumetric data, or simply write() for streamed data.

Cheers, and happy GIFing !

Comments