Pointing at something in a video is easy when it holds still. The moment it moves, the arrow you carefully placed is pointing at empty floor.

The manual way, and why it goes wrong

The traditional fix is keyframing: mark where the label sits at second one, mark it again at second three, let the editor interpolate. It works, and it is tedious in a way that scales badly. A ten second shot of something drifting across frame is a dozen keyframes, and the moment you re-cut the clip they are all in the wrong place.

That last part is the real cost. Manual keyframes are pinned to time, not to the thing you were pointing at.

An editing timeline: video clips, a zoom keyframe and an audio track below.

Anything a viewer has to read at speed should clear the W3C's minimum contrast guidance.

What tracking does instead

You place the callout over the object once. That placement is the starting box. From there the tracker follows the pixels under it and writes the path out for you.

Two design choices are worth knowing, because they explain how it behaves:

It runs as an analysis pass, not live. Tracking takes a few seconds and happens once. Preview and export stay real time, because by the time you play the clip the work is already done.

The result is keyframes. Not a special locked "tracked" object, just ordinary keyframes on the path. That means you can drag any point afterwards, exactly like something you had placed by hand. Where the tracker gets it slightly wrong, you correct that point and keep the rest.

It survives your edit

The track is computed against the edited timeline rather than the raw source. In practice that means a tracked label does not fall apart when you cut the clip it sits on, and it stays correct on a portrait clip or a cropped aspect, which is exactly where a naive implementation reads the wrong frames and puts your arrow sideways.

Nothing leaves the phone

The tracker is Apple's own, part of Vision, and it is already on the device. There is no model to download before the first use, no account, and no upload of the footage you are annotating.

That is a more meaningful difference than it sounds for annotation specifically, because the videos people annotate are so often the sensitive ones: a bug in an unreleased build, a walkthrough of an internal tool, a client's product before launch. Those are the clips where "we process this on our servers" is a reason not to use a tool at all.

What it is good at, and where it struggles

Honest expectations save you a wasted evening:

  • Good: a distinct object moving predictably against a background it does not match. A face, a hand, a product on a table, a cursor, a car.
  • Harder: something that leaves frame and comes back, or passes behind another object. The tracker follows pixels, and pixels that disappear are gone.
  • Hardest: an object the same colour and texture as what is behind it. This is the same limitation every tracker has, and it is why a dark object against a dark background is the classic failure case.

When it drifts, the fix is usually to correct one keyframe rather than to start again.

Where to use it

  • Product demos. Point at the button you are talking about while the screen scrolls.
  • Tutorials. Label a part of a workspace that moves as you tilt the camera.
  • Sports and how-to. Follow the thing the viewer is meant to be watching.
  • Reaction and commentary. Keep a callout on a face that will not sit still.

For the wider annotation toolkit, see how to add arrows and callouts to a video. If the label you want is a caption rather than a marker, captions on iPhone is the better starting point.

Frequently asked questions

Do I have to keyframe anything by hand? No. You place the callout once and tracking writes the path. You can then edit any keyframe it produced.

Does it need a model download or an internet connection? Neither. It uses Apple's Vision tracker, which ships with the system.

Does the track break when I cut the clip? No. Tracking runs against the edited timeline rather than the raw source, so cuts, portrait clips and cropped aspects are all accounted for.

What if the object goes behind something? Expect to fix it. A tracker follows visible pixels, so an object that is hidden for a while usually needs a corrected keyframe when it reappears.

Is it available on Mac? Not yet. This is an iPhone feature.

Does it cost anything? No. It is in the free tier.