Managing Occlusions in Mocha for Accurate Laptop Screen Tracking

When something moves in front of the object you are trying to track, that is called an occlusion. Occlusions cause problems because Mocha will start tracking the occluding object, such as a hand, instead of the surface you actually want, such as a laptop screen. The solution is to create an occlusion layer that tells Mocha to ignore the pixels covered by that object.

Using an occlusion layer does technically reduce tracking accuracy, since you are removing pixels that would otherwise contribute to the track. But it produces a far better result than allowing Mocha to track the wrong thing or fail entirely.

Step 1: Set Up Mocha at Full Resolution

Before launching Mocha, set your preview quality to full resolution. If your downsample factor is set to half, Mocha will warn you that tracking will be less precise. Change the resolution downsample factor to full, then launch Mocha.

Step 2: Create the Occlusion Layer

Navigate to the frame where the occluding object, in this case a hand, is at its largest. Using the pen tool, draw a rough shape that covers the hand. You do not need to trace it precisely. The goal is simply to cover it, especially in the area where it overlaps the surface you want to track.

Once the shape is drawn, right-click to close it and switch to the pick tool. Adjust the shape so it covers the hand with a little margin, but try to keep it as close as possible to avoid masking out too many useful tracking pixels.

Step 3: Keyframe the Occlusion Layer

Because the hand moves in a non-uniform way, the occlusion layer needs to follow it throughout the clip. Use a bisection approach rather than keyframing every frame:

1. Go to the beginning of the clip, where the hand is off screen, and move the entire layer off screen.
2. Go to the end of the clip, where the hand leaves again, and move the layer off screen there as well.
3. Jump to the midpoint between those two keyframes and reposition the layer over the hand.
4. Continue bisecting, jumping to the midpoint between each pair of existing keyframes and adjusting the layer as needed.

Every time you move the layer or adjust its vertices, Mocha creates a new keyframe automatically. You can move the entire shape by grabbing its outer bounding box, resize it from the edges, rotate near the corners, or adjust individual vertices with the pick tool. Focus your attention on the area where the hand crosses the surface you intend to track. Areas where the hand is only over the background can be less precise.

Once you are satisfied with the occlusion coverage, rename the layer to something descriptive like "Hand."

Step 4: Disable Tracking on the Occlusion Layer

To the left of each layer name in Mocha is a small gear icon that enables the layer for tracking when you press the track button. Turn this gear off for the occlusion layer. You have already manually keyframed the hand shape where you need it. You do not want Mocha to attempt to track it automatically.

Step 5: Create the Screen Tracking Layer

Go back to the first frame and draw a new layer over the screen you want to track. A rectangle tool works well for a laptop screen. Name this layer "Screen."

Occlusion layers must be positioned above the layer they are protecting in the layer stack. Drag the Screen layer below the Hand layer. Mocha treats layers higher in the stack as being in front, so anything inside the Hand layer will now be ignored when Mocha tracks the Screen layer.

Adjust the shape of the Screen layer so it covers the screen surface and a bit of the surrounding area. Having good contrast in the tracked region, such as a green screen against a dark bezel and background, will help Mocha lock on cleanly.

Step 6: Set Up the Planar Surface

Turn on the planar surface overlay and drag each corner point to align with the four corners of the laptop screen. Zoom in using the middle mouse wheel to place each point accurately at the edge of the screen. The planar surface defines where your replacement content will be mapped, so precise corner placement matters.

You can use the asterisk key (or the asterisk on the number pad) to fit everything in the window, and the slash key (or divide on the number pad) to return to 100 percent zoom.

Step 7: Track the Screen Layer

Make sure the Screen layer gear is enabled for tracking and the Hand layer gear is disabled. Set the percentage of pixels used for tracking to 90 percent for better accuracy. Enable perspective tracking even if there is no dramatic perspective shift, as it does not hurt.

Press track forward and watch the planar surface grid. If it drifts away from the screen, stop and make corrections before continuing. Some drift near the area covered by the hand is expected and can be corrected with the Adjust Track tool.

Save the Mocha project after tracking is complete.

Step 8: Review and Adjust the Track

After tracking, go back through the clip and check whether the planar surface stays aligned with the screen corners. If there is drift, use the Adjust Track tool, which is available in the classic workspace.

Adjust Track works by placing reference points on recognizable features in the image, such as the corners of the green screen, and keyframing their positions throughout the clip. Set your reference points at a clean frame, mark that as the reference frame, then move to other frames where drift has occurred and reposition the points to match. The auto option will attempt to find the reference point automatically at the new position. You can also nudge points manually.

Save the project again when the adjustment is complete, close Mocha, and save the After Effects file.

Step 9: Export Tracking Data to After Effects

Back in After Effects, create a new solid at comp size. The color does not matter. Open Mocha, go to the tracking data export panel, select the Screen layer, and choose Corner Pin as the export type. Set the target layer to the solid you just created, then click Apply.

This adds a Corner Pin effect to the solid, driven by the tracking data. Rename the solid to something like "Phone Screen" or "Laptop Screen" for clarity.

Step 10: Pre-Compose and Add Replacement Content

Pre-compose the solid layer using Command-Shift-C (or Control-Shift-C), leaving all attributes in place so the Corner Pin effect stays in the parent composition. Name the pre-comp something descriptive.

Double-click the pre-comp to enter it. Drag in the replacement screen graphic, right-click, and choose Transform then Fit to Comp. It will look distorted inside the pre-comp, but back in the parent comp it will conform correctly to the tracked screen shape.

To animate a screen change at the moment of a button tap, find the exact frame in the parent comp timeline, double-click into the pre-comp at that same time, and add a second screen graphic above the first. Press the left bracket key to trim its in point to the current time. Add a Fit to Comp transformation to that layer as well. The result is a seamless content swap timed to the interaction.

Step 11: Key Out the Green Screen

Move the screen replacement layer below the original source footage layer in the timeline. Apply the Keylight effect to the source footage. Use the eyedropper with Command-click to sample the green screen color.

In the Keylight controls, aim for pure black on the screen area and pure white everywhere else in the matte view. Adjust Screen Gain, Screen Balance, Clip Black, and Clip White as needed. Switch to Status view to check for any remaining gray areas in the matte. Once the matte looks clean, return to Intermediate Result view.

Enable the Advanced Spill Suppressor to remove any green fringe around the edges. The Key Cleaner effect can be added for additional edge refinement if needed, but for a simple rectangular screen it is often unnecessary.

The final result is a screen replacement that holds through the occlusion, with the replacement content appearing correctly behind the hand.

This article is part of a continuing series on motion tracking and compositing techniques in After Effects.