User:PhosCity/Blender Motion Tracking for Fansubbing

Blender is a free and open source 3D graphics program and among the many things Blender can do, motion tracking is one of them. Blender is a point tracker which means that it only tracks a single point in a video. In contrast, mocha, another program used for motion tracking, is a planar tracker. Blender and mocha thus being different in how they track motion will perform variably in different situations. One may track a scene much better while the other fails.

Requirements

 * Blender
 * Aegisub
 * Aegisub Motion
 * Aegisub Motion Perspective
 * Blender Plugin

Opening Movie Clip Editor
When you open Blender, you will see the default 3D viewport. All of our motion tracking will be done in another viewport called Movie Clip Editor. There are two methods to get to this view. First, click in the icon below File in the top left and click on Movie Clip Editor.

You can also use a keyboard shortcut. As soon as you open Blender, press Shift+F2 in your keyboard to directly go to the Movie Clip Editor view at once.

Open video or image sequence
In order to load your video or image sequence, click on Open in the top middle portion of the window. Then navigate to the folder where you have your video or image sequence. Select the file and click Open Clip.

As soon as you open a clip, you should make a habit of always doing two things. First click on Set Scene Frames to ensure your start frame and end frame for tracking is the same as that of your clip. Second, click on Prefetch to cache your clip into memory for faster playback and tracking.

Tracking Settings
In the left portion of your screen, you will see a menu called Tracking Settings where you can change settings that will affect your tracking.

Pattern Size
This is the size of your pattern area.

Search Size
This is the size of your search area.

Motion Model
This defines what type of motion your target has. Properly setting this will be the key to a good motion track.

Let’s say that your target only has horizontal and vertical movement. You might think, I’ll use Affine since it is designed to track movement among other things. Not only is this more computationally expensive, this might even introduce more error in terms of scale or rotation when there wasn’t any in the target. So choosing one carefully is very important.

Match
This defines the frame that blender will reference for tracking.

Previous frame is useful when the frame of reference changes due to skew or stretch however it must be used with caution as it can lead to bad track sometimes. If there is an error while tracking a frame, this error will now become the new frame of reference and thus the error will compound with each new mistake in tracking.

Normalize
Ticking Normalize will make blender not lose sight of the marker during slight light changes in the video. However, it might make tracking a bit slower than normal.

RGB
This can be used if you only want to track one channel. For example if you want to track a dot in a green background and don’t want to track green background at all, you can disable G to stop blender tracking the green channel. The same goes for Red and Blue channel.

Correlation
By default, it is 0.75 which means that blender will consider the tracking successful if it is 75% sure that the marker has been properly tracked. It is recommended to set this to 0.9 and only reduce it if strictly necessary.

Adding a marker
A marker marks the location to track. To add a marker at a target, press Ctrl in your keyboard and left click anywhere in the clip to place a marker in that location.

A rectangle must have appeared at that point. That rectangle is called pattern area which is the region of the clip that Blender uses as a reference for tracking.

There is a hidden box around the selected marker that can be activated by pressing Alt+s in your keyboard. A new box bigger than the pattern area will appear. This is called search area which is the region where Blender searches to find the marker in the next frame. You can make it bigger to search bigger area at expense of some performance if the target moves large distance each frame.

You can move the marker to another location by pressing g and dragging it to new location. You can increase or decrease the size of the marker by pressing s and using mouse to scale. You can also left click on each corner of the marker to change them individually. You can similarly rotate the marker by pressing r.

Lastly, unless you are doing plane tracking in blender, you must only use one marker because each marker will result in new tracking data and we only need one.

Note: It is paramount to understand that when you add a marker, it inherits all the tracking settings you choose. Once you add a marker, you cannot change the tracking settings again to make changes to the marker. It’s recommended that you remove that marker, change the tracking settings and add a new marker. However, if you must change the settings of a marker that has already been added, you must use the Track settings in the right panel. In this panel you’ll find the same settings you found in the left Track panel but this one will modify the setting for currently selected marker. Additionally, the Track panel in the right also has the picture of the target with a small cross-hair that can be used to fine tune the position of the marker.

Tracking
After you set the tracking settings and add a marker, you can now begin tracking. You can start from any frame but if you start tracking from a middle frame, you will have to track both backwards and forwards from that frame. Normally it is easier to start in the first frame or the last frame if the target is clearly visible in that frame. The keyboard shortcut to go to the first frame or the last frame is Shift+Left Click and Shift+Right Click respectively. You can start tracking either by clicking on buttons or by using keyboard shortcuts.

If at any point, the tracking fails, blender will stop immediately. In that case you will have to try with a new tracker preferably in a new target or new tracking settings. If you have seen tutorials of blender tracking somewhere else, you might have been told that changing size of the marker mid tracking is not a problem. That is true when you are motion tracking for blender but for fansubbing purposes, you cannot change the size of the pattern area mid-tracking as that will lead to change in size of resultant tracking data. You can however change the search area at any point without any problem.