FreeTrack is a general-purpose optical motion tracking application for Microsoft Windows, released under the GNU General Public License, that can be used with common inexpensive cameras. Its primary focus is head tracking with uses in virtual reality, simulation, video games, 3D modeling, computer aided design and general hands-free computing to improve computer accessibility. Tracking can be made sensitive enough that only small head movements are required so that the user's eyes never leave the screen.
A camera is positioned to observe a rigid point model worn by the user, the points of which need to be isolated from background light by means of physical and software filtering. Motion is tracked with up to six degrees of freedom (6DOF): yaw, pitch, roll, left/right, up/down and forward/back. Windows-compatible video devices like webcams are supported, as well as special Nintendo Wii Remote camera, iPhone Truedepth camera with Eyeware Beam, and NaturalPoint cameras (TrackIR, SmartNav and OptiTrack).
FreeTrack can output head-tracking data to programs directly using its own open interface, as well as TrackIR, SimConnect and FSUIPC interfaces. Programs that support these interfaces are regarded as being FreeTrack-compatible. FreeTrack can also emulate mouse, keyboard, and joystick (via PPJoy) if a program does not support a direct interface.
FreeTrack is coded in Delphi 7 and uses DirectShow and DirectX. Head tracking is achieved using implementations of DeMenthon's four-point iterative pose estimation algorithm (POSIT) and Alter's three point geometric algorithm.
Software
FreeTrack uses a camera to collect real-time information on the point model worn by the user. Specifically the image coordinates of the model points, which are either received directly from the camera or extracted from a video stream. These coordinates are used to generate an estimate of the real head pose, which can be transformed by the user in a number of ways to create a virtual pose. One of the most fundamental transformations involves amplifying rotation so that only small head movements are required. Finally, the virtual pose is sent to the user's choice of outputs. This is all done in the background, with tracking status displayed in the system tray.
A 3D preview is available that shows the virtual head position and orientation for a given real head pose and can be viewed from multiple perspectives, including first-person. This greatly assists with testing and makes it easier to experiment with different settings.
Each degree of freedom (axis) has a response curve that can be modified to change the way the virtual head moves for a given real head movement. This is commonly used to create a central deadzone region so that the user’s head can be more relaxed there.
Keyboard, mouse and joystick buttons can be used to toggle tracking settings, including the virtual centre location (like adjusting the seat position in a car) and individually toggle axes and outputs.
For NaturalPoint cameras, FreeTrack can provide advanced features and a level of customization that is not available with official software.
Camera
Resolution
In most cases a resolution of 320×240 is sufficient, this is capable of producing a much higher sub-pixel resolution, enough to allow accurate cursor control on a high-resolution monitor. Resolutions 640×480 and above have diminishing returns and correspond to an exponential increase in CPU usage when not sufficiently compressed before reaching the computer. Higher resolutions become more important at greater distances from the camera. The Wii utilizes a low-resolution 128×96 sensor, which is found by some to produce jittery tracking and may require smoothing to improve stability at the cost of decreased responsiveness.
Sensor
For the same resolution, monochrome sensors can resolve finer details much better than color sensors due to the lack of a color filter array.
Frame rate
FreeTrack uses interpolation with low-frame-rate video devices to improve panning smoothness. However, responsiveness is fundamentally limited to the frame rate; a 30 frame/s webcam has a maximal response delay of 33.3 milliseconds compared with 8.33 milliseconds for a 120 frame/s camera. To put this into perspective, a human’s reaction time to visual stimulus (finger reflex) is typically around 200 ms; 30 ms can be regarded as a competitive ping in online reflex-based games, and an LCD monitor refresh rate is typically 17 ms.