Video Tracking Tool

Since my undergrad projects, I required a tool for tracking points on videos. I have been tracking videos of falling objects, swimming brine shrimp, rat whiskers during social touch, squirrels, ungulate hooves and joints… the number of frames I clicked is immense (which is a burden to efficiency, see below).

The unabated need to track videos in my PhD project has led me to overhaul my tracking tool. I thought it might be useful for others, so here you go.

The Cost of Clicking

I originally started with commonly used tools, but found myself being inefficient due to the need for clicking on each frame of the video. Although clicking seems to be a tiny investment, it accumulates and gets annoying, especially when the video frame rate is sufficient for scientific analysis. Automated tracking algorithms exist, they certainly can facilitate this sort of task. However, for many of my applications, they failed - I guess because I was rarely blessed with markers that provide enough contrast for these algorithms to work reliably, or because my accuracy requirements were too high. So shortly after I got into learning Python, I started programming a tool myself.

Progressive Tracking

The unique feature of my tool: it allows the user to simply follow a point on the video with the mouse cursor. Playback speed can be adjusted (by mouse wheel and keyboard shortcut). Playing backwards is possible (press “control” key while playing). The cursor position is recorded frame by frame. No clicking!

This is what it looks like:

As you see, the interface is minimalistic. I tried to reduce dependencies to a minimum, for example by using matplotlib instead of more involved image libraries (such as opencv or scikit-image; these are very useful, for example for contrast enhancement, and can be included easily when needed).

• The default matplotlib/TkAgg navigation toolbar stays usable and is extended by a menu and status indicator.
• The video frame is central and has maximal size.
• At the bottom, there is a time axis (clickable) that indicates tracking progress.
• Many features are accessible via keyboard shortcuts (see help overlay and comments in the python file).
• The image sequence is loaded by a background process and stored in memory, enabling quick playback.

So in summary, this is a lightweight, efficient video tracking tool that has saved me lots of hours.

The Tracker Tool

The latest version of the tool can be downloaded here: ProgressiveTracker.py

I will keep the code downloads on this webpage updated, but you can follow future changes on this gitlab repository.

Update 2020/03

In parallel to extensively using the tool recently, I implemented a couple of improvements and new features. Those include some new menu options, shortcuts for navigation in time and image, and a help screen that shows the hotkeys.

Update 2020/10

Our team analyzed the progressive tracking accuracy and performance and published the results. They are open access!

Maja Mielke, Peter Aerts, Chris Van Ginneken, Sam Van Wassenbergh, Falk Mielke (2020): Progressive tracking: a novel procedure to facilitate manual digitization of videos. Biology Open; DOI: 10.1242/bio.055962

Enjoy using the tool! Let me know if you encounter bugs, or send me feedback. (falkmielke.biology@mailbox.org)