Background

I made this intro cinematic for Gust of Wind, a hobby project I’ve been working on with Unity, in early 2023. The game was nearing its early access launch, and with some lengthy dialogs in the first meters of the story, I felt like it needed something to quickly show the player what they can expect the gameplay to be about.

This post is a mixed bag of notions of my workflow, things I learned, things I probably did wrong, and things I was able to improve when coming back to it a couple of years later.

Here’s the final cinematic, with music by Olli Oja.

Storyboard

I started by painting a storyboard. The main gist of it is that the character we follow was sent on a sabotage mission. He got caught while carrying it out, which is why the main character of the game was sent in his stead on a voyage that the story of the game is set on.

Setting

I had recently watched the Michael Mann film Thief from 1981 and really liked the atmosphere of its opening sequence with a combination of early morning lighting and wet streets.

I aimed for something similar for the lighting, adapted to the setting of the game, a post-apocalyptic world left behind by an Industrial Age civilization.

For the environment, I chose a long, dilapidated factory hall, since it gives a wide berth for using closer and more distant shots without being very laborious to model. Some barrels, an overhead crane and the train of compressed air carts provide some hiding places for the sneaky character.

The factory hall needed to be modeled and textured for the cinematic. However, it didn’t require that many pieces. Here are the beams and metal sheet variations I used to piece it together. The textures aren’t great, but good enough to do their job in hazy lighting.

Lighting

For the original version of the cinematic, I used real time lights since dynamic shadows were important and baking lighting is quite time-consuming.

Volumetric fog combined with a directional light radiating at a low angle contributed nicely to a mood of early morning.

The lighting consists of:

• Directional light
• The firelight of the barrel
• 11 point lights there just to fake indirect lighting, or add some clarity to some shots that would’ve otherwise been too murky.

Since the shadows, particles and post effects made the scene quite heavy, I later ended up pre-rendering the cutscene by screen recording it in half the normal playing speed, and increased to normal speed in video editing (which provided a better frame rate than playing it in normal speed).

However, fiddling around with the scene now, almost two years later, I tried baking the indirect lighting to see what it looks like. The video at the top of this post has that baking applied, with some subtle global illumination on the shadows (while the direct lighting is still handled real time).

Below are some views from the version of the cinematic the early access launched with, compared to new ones with some subtle changes I made now:

• Baked indirect lighting (It’s hard to distinguish but you can maybe see some some hint of bounced light on the barrels and the walls of the hall in shadow)
• Slightly better quality in volumetric light (less dithering)
• The shadows on characters are slightly brighter as well, probably due to having Screen Space Global Illumination disabled in the original version.

Rain Effects

For the rain, to save some performance and to get refraction to the droplets, I used this technique instead of having the rain completely consist of particle effects. The position of the rain cylinder is animated in the cinematic so that it’s always centered to the camera currently in use.

The ripples on the ground are produced by a regular particle effect with a flat box emitter. It would’ve been possible to make a mesh of the needed part of the terrain and use that as the emitter, but in this case the ground is pretty much completely flat (apart from the puddles, which are just dents on the plane added with parallax).

Working Neatly in Unity’s Timeline

Arranging things neatly in nested timelines not only saves time, but helps keep the visual result of the sequence tidy as well.

If you animate everything on a single timeline, there’s likely to be some annoying flickering when the camera angle changes and perhaps a light or a character is meant to get enabled or disabled precisely at the same time. Getting those timings synced is a pain when there’s a big number of tracks.

Unfortunately I didn’t know you can nest timelines when working on the cinematic. Only when working on a video in UE5 later the same year, I noticed to how handy it is to be able to group cameras and lights in separate shots, and I wondered why you wouldn’t be able to do the same in Unity.

But as I later found out, you can, by making your shots as separate timelines, and nesting them inside your “master” timeline. Here’s what it could look like:

Based on a bit of experimentation, this method seems like a much handier way to animate a sequence with several shots. You can also have animated elements with continuous motion in the master timeline, and when you’re modifying one of the nested timelines, that animation shows up with correct timing.

For comparison, here’s my timeline from the cinematic, looks pretty overwhelming.

Adding Character Animations

Most of the character animations are ones used in the game as well, a few of them having some minor alterations for the versions used in this cinematic.

However, I had to make one new animation for opening the bolt with a wrench. I added a rotating bone for the bolt the wrench is supposed to open and a helper object linked to the bolt bone (so that it rotates with it). The hand IK is constrained to the position and rotation of that helper object when the wrench is turning the nut. Without constraining it this way, it would’ve been impossible to get the wrench positioned just right.

This is the shot in question. In the gif below, the rotating bone of the nut and the hand target bone linked to it are highlighted:

As a handy detail about character animations, the Timeline tool lets you add an “Override Track” on an animation track. This would be a separate animation masked to a certain part of the character. A case in point is the enemy character carrying a weapon while walking. The carrying pose is a separate animation clip, which I added on an override track. That override track has an Avatar Mask applied, which defines which body parts the animation affects (in this case, both arms).

There are plenty of nifty things in Timeline, like how easy it is to add transitions between clips, and triggering functions from the timeline (which I haven’t gotten into). However, I think these were the main notions I had for this small post mortem of the cinematic. Thanks for reading!

(Originally posted May 31st 2024)

For as long as I’ve used it, After Effects has been a program mixing 2D compositions with the option to position layers in 3D space, apply lighting and move a camera in that 3D space.

Adobe has recently introduced new 3D features to a After Effects Beta. For someone like me, who sometimes animates paintings with parallax and is no stranger to 3D modeling, finding out their extents by testing them in a simple scene seemed like an interesting topic to delve into.

With the new tools, it’s possible to set up a 3D scene like the video below, directly in AE without plugins. However, there are no modeling tools beyond extruding and beveling shapes and text, so the 3D models would have to be brought from another piece of software.

Short summary of my notions:

• The new renderer called Advanced 3D Renderer seemed very fast to render a 3D composition with
• Importing 3d models with PBR materials and bone animations is possible (though some features missing, like shadows for most lights, definitely reflect that it’s still in Beta)
• Environment Light with a HDRI map can be used to light your scene, and it can cast nice looking shadows
• Extruding and beveling Text- and Shape Layers, and curved footage, which were previously available in the Cinema 4D renderer, are also supported in the Advanced 3D Renderer.

Limitations to keep in mind:

• Features that don’t work yet in Beta: shadows (except for Environment Lights), motion blur (except if you add Pixel Motion Blur on an Adjustment Layer), depth of field (except if you add it as a 3D Channel Effect, which has its drawbacks), as well as effects on 3D layers and precomps of 3D scenes
• Ability to do 3D modeling or modify geometry of imported models isn’t in the pipeline according to Adobe’s FAQ. So this won’t be a full substitute to using other 3D packages, but for something like adding a 3D object to a composited sequence, the features will give an option of a faster workflow.

In this post, I’ll go through my experiments with the new features.

1. Animated butterfly

Exporting bone animation

This is the bone animation, with which I exported the butterfly from Blender.

After Effects can currently use 3d files of OBJ, GLTF and GLB format (GLTF and GLB being just ASCII and binary version of the same format). OBJ doesn’t include skinning and animations, so I used GLB. Importing the model in After Effects was as easy as dragging and dropping it in the Composition window, and pressing OK in this prompt.

If the model has several animations, you can pick the one you want to use from the list shown below:

As a nifty detail, you can also animate Time Remapping of the imported animation, the same way you would with footage (which is how I animated the butterfly to flap its wings at different speeds throughout the video, using the one looping animation clip)

Trying to import a Shape Key animation

Just for the sake of experimentation, I tried to export the animation as a Shape Key* animation. A 3d model can have several Shape Keys, which are different arrangements of the same vertices. You can turn them on and off with a slider to adjust and animate the form of the model. They’re often used for facial expressions or features for example.

*same as BlendShape or Morph Target in other 3D software

In After Effects, the Shape Key animation appeared as an animation clip, but the model didn’t move (although checking with import to Blender, it exported correctly).

2. Grass / placing objects

I made a very simple grass asset by using a transparent grass texture I painted in Photoshop, on a plane.

For the ground below the grass patches, I used an image with noise map as a displacement map for a subdivided plane in Blender.

Positioning the grass patches on a ground plane in After Effects was quite slow, since moving the view is as difficult as it’s always been with the camera tools. In hindsight, it would’ve definitely been handier to combine the grass patches with the ground plane in Blender before exporting.

With the grass patches placed in After Effects, I noticed that the objects too far from the center of the world coordinates didn’t cast shadows:

However, I fixed this by linking all 3D layers (that weren’t linked to something else) to a single 3D null layer (placed at the center of the world in 1920, 1080, 0 since my comp was in UHD resolution). That’s a good way to do all-encompassing scaling and positioning changes in AE, even if there’s position or scale animation on some of the layers. Here’s the result, with the grass patches further away as well now casting shadows:

3. Beveled Text and Shapes

Using the Advanced Renderer, you can extrude a text layer without having to deal with the slowness of the Cinema 4D renderer. Like previously with the Cinema 4D renderer, you can extrude text and shape layers with three types of bevels. For the material, you can adjust the values shown below, but it can only be of single color, not textured.

According to the list of enabled features in Composition Settings, there’s something called “Material overrides on text/shape bevels and sides”. I didn’t find out what it means. Apparently you can add a stroke on a beveled object and make it a different color, but the stroke doesn’t have separate material settings, and there seem to be some z-fighting between the stroke mesh and the color fill mesh:

4. PBR Materials

Below is the butterfly imported to After Effects, side by side with the Albedo and Opacity maps. Full and partial transparency and double-sided materials seem to work fine. For the textures, the FAQ mentions only PNG and JPG being supported. I suppose that\’s when using OBJ as the format. The albedo texture below (with opacity map on its Alpha channel) is a TGA texture, but when embedded in a GLB file, it worked fine in After Effects.

According to the Adobe’s FAQ, the properties available for the PBR materials are those included in something called Adobe Standard Materials (ASM). Here are the details on what they are, quite a range of different properties.

I tried out these common PBR material properties in my test scene:

• Albedo (colors)
• Normal
• Metallic
• Roughness
• Opacity (partial as well)
• Emission

To show a wider range of maps of the PBR workflow in use, I made a UV mapped version of the 3D text in Blender. After cleaning up the geometry and mapping the letters, I imported the model in Substance Painter and applied different material presets to the different letters. Below you can see the effect of roughness and metallic maps, and how the normal map affects the grains of the wood for example.

5. Environment Light

Environment Lights use HDRI (High Dynamic Range Image) maps for image-based lighting. A HDRI means an image with high bit depth, but in the context of 3D graphics, it also means a 360° image of an environment. This article by Mark Segasby of lightmap.co.uk describes it in detail.

You can add a light, set it as an environment light, and apply any HDRI image (in your composition as a hidden layer). You can also tick on shadows for it, which can provide a nice looking occlusion that resembles Ray Tracing. My example scene isn’t the best setup to show it in, but the shadow below dragon on this video by SternFX gives a better idea of what it can look like.

Here you can see the effect of different HDRIs in my example scene. All the HDRI maps are from Poly Haven.

You can also make a 3D plane which only accepts shadows but is otherwise transparent, as described here.

3D Channel Effects

The effects in the 3D Channel section have existed before but I’ve never really noticed them, so testing them with the new features was a great excuse to try them out.

The workflow with these effects is making a precomp of your 3D scene, and then applying the effect to the precomp.

The fog effect lets you mask the fog with a separate black and white texture or precomp (which goes into “Gradient Layer” slot).

Here are three of the other 3D channel effects in use (the remaining effects are Cryptomatte, EXtractoR, ID Matte and Identifier, which I didn\’t test out this time).

The frame shown below has Fog and Depth Of Field applied.

However, you can see the edges of the screen being a bit messy, and that there’s some blur around the wings of the butterfly. Using the Depth of Field option in the camera instead (which is currently unavailable in the beta) wouldn’t cause these problems.

Also, as mentioned in the Composition Settings, effects on collapsed 3D precomp layers are currently not working (I also noticed this when trying to render the composition with the Fog and DoF effects through Media Encoder, fog was invisible and DoF had blur everywhere).

Testing with more complex meshes

I tried importing meshes that together amount to 1 million tris. This had no noticeable impact on the time it takes to update the frame in full resolution in preview (it was about 5 seconds both before and after).

I also added a chrome sphere by setting its material’s Metallic value to full and Roughness to 0. As you can see, it reflects the environment light but not the objects around it (since there’s no Ray Tracing).

Note that the blue sky on the background is not the HDRI map but just a solid with a gradient. CC Environment effect appeared way too dark, so I found no way to use the HDRI as a visible background.

Applying effects to a 3D object

As Adobe’s FAQ mentions, it’s currently not possible to apply effects to 3D layers directly, but you can add an adjustment layer on top of everything and select the 3D layer as its Track Matte layer. I tried recoloring the butterfly this way.

However, if there are other objects in front of the object you want to apply the effect to, you’d have to mask them out somehow.

Conclusion

Here are the new 3D features in a nutshell (introduced in 2023-2024):

• Advanced 3D Renderer, which renders the 3D content very fast. Rendering my scene in UHD resolution without particles and adjustment layers took only 6 minutes on my system.
• Image-based environment lights and shadows (HDRI Environments)
• Models with PBR Materials and possibility to use bone animations
• Curved footage layers and extruding and beveling Shape- and Text Layers now work in the Advanced 3D Renderer as well (not just Cinema 4D)

3D formats that can be imported (according to the FAQ, they’re working on adding more):

• OBJ: Contains only the static 3D mesh, but can reference .MTL files that contain materials and textures. Animations, skinning and Shape Keys can’t be included.
• GLTF: can include meshes, bones, animations, Shape Keys, materials (PBR) and textures
• GLB: binary version of GLTF (GLTF being ASCII), supports the same features

Still not included in the current Beta:

In addition to the points listed above, here are some problems I bumped to:

• In my test scene, only objects close enough to the world center cast shadows from the Environment Light
• If you have an Adjustment layer between two 3D layers, those two 3D layers don’t clip with each other
• Shape Key animations don’t get imported

Overall impression:

These 3D features don’t seem like an all-encompassing replacement for animating the more complex sequences in other 3D programs, but for simpler compositions involving 3d models (like flying and rotating coins, or a character running across a text title, or something), having it be possible to do without plugins and other programs will smoothen the workflow nicely. The HDRI environments casting shadows also seems like a low-weight solution for achieving nice lighting, although the missing shadows for regular kinds of lights is currently a big drawback with the Beta version.

(Originally posted in September 2020)

To continue my streak of fan trailers, I wanted to do something faster and more energetic this time, edited to a metal soundtrack. Since I had Doom (2016) in my Steam library, that ended up as the subject. The game has moments with a really good flow of movement in the heat of battle, and I wanted to try reflect that in the editing on my fan trailer.

After playing through the game and capturing the best parts with Nvidia Shadowplay, I started the process by extracting the dialogue audio from the game and finding the clips with some dialogue lines that might be suitable for a trailer. From those, I pieced together a voice over that served as a basis for the structure.

Next I went through the game’s soundtrack by Mick Gordon, in search of a clip that would structurally and mood-wise work well in a trailer. I ended up using a part of the track “Flesh & Metal”, which had a good variety of calm and intense moments and escalation. It provided a great basis to sync moments of impact in the heated battle footage to.

After planning the structure with voice over and music, I took the most interesting looking moments in the gameplay clips and placed them on the timeline. I captured new gameplay clips where necessary to fill out the escalation of the beginning with ominous pentagrams, hell-landscapes and other things that more or less fit the voice over.

For the latter, action-focused half of the trailer, I marked impact moments with impacts in the music. I tried to use as many match cuts as possible with continuity of motion in the edit – the end of one clip has the same direction of movement as the beginning of next clip. This worked out in some cuts better than others.

Once everything else was finished, I adjusted the gameplay audio with volume keyframes and balanced out the audio tracks to sound good with compression so that all parts, music, voice over and gameplay can be heard.

(Originally posted September 2020)

I thought I’d make another fan trailer to keep myself occupied in quiet times between freelance work. This time the subject turned out to be Grow Home by Ubisoft Reflections.

It’s been one of my favorite games from the first time I played it through a few years ago. There are many aspects of it that I really like, from the style of graphics and sense of huge scale, to the way the character moves (especially when climbing), with pad triggers “gripping” the cliff face with the corresponding hand, but otherwise the main character Bud being totally at the mercy of physics. I also like the idea of growing something huge, the form of which the player is in control over.

Here’s how the fan trailer turned out. For music, I used the game’s menu music by Lewis Griffin.

(Originally posted in September 2020)

I’ve had Bioshock Infinite in my Steam library for a long while and thought that might be a good subject for a fan trailer. Up to this point, something about the cartoony style had convinced me that the game is not my cup of tea, but I ended up being happy I gave it a whirl. I really liked how the story unraveled, and the visuals, even still in 2020, are consistantly great.

Here’s my take on a fan trailer:

I captured the gameplay with Nvidia Shadowplay, which seems to me by far the handiest way, since saving 15s clips with a shortcut after something interesting happens is makes the footage easier to manage than capturing continuously.

After the credits had rolled, I listened through all of both Elizabeth’s and Booker’s lines to find suitable comments for a trailer voice over. I also listened through the game’s soundtrack by Garry Schyman and after realizing there’s everything I need for a trailer there, I combined suitable clips to form some kind of a rough audio structure.

Rummaging around in the multitude of video clips, I took the best ones on my timeline and assembled them into a whole skeleton of a trailer. When the editing felt good to me, I polished the audio, toning up some of the gameplay and adding some sound effects to heighten the dramatic effect of some moments.

Assembling the clips and editing this trailer took me about 24 hours (not counting the playing time, for which Steam clocked 15 hours).

(Originally posted in January, 2020)

I made this video as a cutscene for a game project I’m planning, but also to get a portfolio sample that shows my process. All my videos consisting entirely of paintings so far have been hobby-based and I haven’t worried too much about the hours, but for this one I kept book of them and tried to work as efficiently as possible.

Script and Thumbnails

In this cutscene, I wanted to show the player’s ship crew attacking an enemy ship at night, given the element of surprise by a thick fog. To add an extra twist, I wanted the crew to attack dramatically from above, something like the ”Pole Cats” in Mad Max: Fury Road

In the ship design that I had drawn earlier, there’s a derrick crane (something that has been used in smaller size in steam ships, fishing boats and sail ships to load cargo in the hull) modified to also serve as a sort of siege ladder.

Based on this rough idea of what should happen in the cinematic, I thought about the clearest way to tell the story interestingly enough with as few scenes as possible without resorting to narration. The result was this first draft of the script with thumbnails, shown below.

Scene number	Duration (s)	Description	Thumbnails
1	8	Close-up of gauges and pipes, glistening in moonlight. Particles of foam or frost are floating about. Strong depth of field effect. There’s some blurry movement on the background
2	8	Camera focuses on the background. A man operating crank attached to the mast. Cogs of the mechanism are turning on the background.
3	8	Camera is now on the other side of the mast, still about waist-high. The mast is turning so that the crane will face on the right (two men are climbed on the mast). Camera moves backwards and reveals a group of men looking grimly to the right, with makeshift weapons in hand.
4	4	Fade to the men climbed atop the crane. Camera zooms into the face grille of the other one’s oven helmet. Maybe drops of sweat seeping through the holes
5	2	A quick, blurry camera move to captain giving a hand signal
6	2	A man kicks a lever in the winch.
7	2	Camera is between the hulls of the two ships (the enemy ship shown now for the first time). The beam of the crane is dropping in free fall.
8	2	Camera is behind the men on the crane, as the beam slams into the railing of the other ship and locks in place.
9	8	The guy with face grille standing on the deck of the enemy ship. Camera starts with a close up on him, and zooms out evenly. Another guy is jumping on deck, and a few others are running across the bridge now formed by the beam. As the camera gets further, it reveals the legs of some barbarians (tattoos, fur leggings, cuts, bruises, etc)

First mockup video and finding stock music

When I had it planned out what should be happening on the scenes visually, I started working in After Effects, importing the thumbnails and animating the rough camera movement, just to get an idea of how the length of the shots and general flow between the shots will work when I add them together.

I also started to focus on finding suitable music. What I originally had in mind initially was something more classical, but after numerous hours of searching I ended up using this metal track because it has a strong mood of anticipation what’s going to happen next, and clear points of ”change” (namely the drop in 25-27s), where you can place something interesting happening in the video.

”Metal Cinematic Trailer” by OttoMusic

After making the decision, I cut the mockup video to fit the structure of the music (also adding a few short scenes to make the pacing reflect the build-up). Here’s what I had at the end of day 4. Although I did end up changing the viewing angles of a couple of shots afterwards, this pretty much served as the skeleton of the video until the finish line.

Scene 3

When I had the structure of the video laid out, I started working on one of the scenes. I wanted to start with something that wasn\’t too easy or too laborous, so that it would give me some kind of an idea about how much time the rest of the scenes would take on average. Since scene 1 would be a whole lot of work because of the longer length (requiring more things happening) and 3d elements I’d have to add, I started with scene 3.

Here’s the progress of the painting from sketch to what I had at the end of day 7. I did some rough color experiments as well, but ended up thinking it’s best to go with black and white, for both efficiency and because of the night time there wouldn’t be much color anyway. Later I ended up doing some changes on the armor design and lighting.

Scene 4

In this scene, for the lighting and the look of waterdrops on skin, I referenced the ”tears in rain” scene in Blade Runner. The lighting was very suitable since it was set on early dawn wich fits my setting, and is something that I could get pretty close to with just color adjustments on a black and white painting.

Below are progress shots of the painting side, as well as the animated scene at the point where I had it at the end of day 9. It acted as a model for color adjustments to use on the rest of the scenes as well.

Scene 1

Now that I had a clear idea about the painting and coloring style I was going to use, I moved on to the laborous scene 1.

In an attempt to make the viewer curious, I made a strong depth of field effect, revealing the background little by little. The turning mechanism of the mast isn’t realistic, but I wanted to have something showing gears in movement as technology from the turn of the 20th century is an important part of the game setting.

To get the gears to turn in 3d space, I had to make them as 3d models. I used Blender to model them and Element 3d plugin for After Effects to place them as convincingly as I could in relation to the painted 2d layers. There’s a bit of a 90s CG feel on the textured 3d objects, which I could’ve avoided by spending time in painting actual mapped textures on them, but I tried to cut my losses in terms of time spent on a very little thing. Later in polish phase, I fixed some mistakes and painted some more interesting texture on the floor.

Scene 11

Most of the rest of the scenes had nothing really special about their process but in scene 11 I did a new thing I haven’t tried before. Instead of using 3d models of the figures and mast with camera rotating around them, I figured it would be much less time-consuming and look more interesting if I painted the frames. Despite the low frame rate due to not wanting to spend ages on this, I’m pretty happy with how it turned out.

Last finishes

Alongside doing the scenes, I wrote down a list about some smaller changes I need to do at the end to fix the consistency between scenes (for example the placement of figures, their outfits and weapons), and to add detail in places that looked too empty. Here you can get a picture of how those changes added some life to the scenes.

Here’s a breakdown of how long each phase in the project ended up taking: