Computer vision is the part of AI that turns images into data. In football it is used to follow every player and the ball through a match, frame by frame, and turn the pictures into coordinates on the pitch. That data powers offside decisions, physical reports and many of the graphics you see on television. Here is how it works, with FIFA's semi-automated offside technology as a concrete example, and where it still struggles.
Event data and tracking data
Football data comes in two main kinds:
| Event data | Tracking data | |
|---|---|---|
| What it records | Actions: passes, shots, tackles, fouls | Positions of every player and the ball |
| How often | Each time something happens | Many times per second |
| Typical use | Shot maps, pass networks, xG | Running distances, pressing, space, offside |
Event data says what happened on the ball. Tracking data says where everyone was while it happened, which is what you need to study space, movement and off-the-ball work.
From pictures to positions
A tracking system generally solves four problems in a row:
- Calibration. Each camera's position and angle are measured, so a point in the image can be mapped to a point on the pitch.
- Detection. In every frame, a model finds the players, referees and the ball.
- Identity. Each detection is linked to the same person across frames, even when players cross, bunch together or leave the picture.
- Pose. For decisions such as offside, the system goes further and estimates the position of body parts — feet, knees, shoulders — not just a single point per player.
Combining several cameras that see the same moment from different angles makes the positions more accurate and helps when one view is blocked.
A real example: semi-automated offside
FIFA introduced semi-automated offside technology at the 2022 World Cup in Qatar. According to FIFA:
| Measure | Value |
|---|---|
| Dedicated tracking cameras under the stadium roof | 12 |
| Data points tracked on each player | 29 |
| Position updates per second | 50 |
| Ball-sensor readings per second | 500 |
The ball itself carried an inertial measurement unit, so the system could detect the exact moment it was kicked. Combining the limb-tracking and ball data with AI, the system sent an automated offside alert to the video match officials, who checked the kick point and offside line it had selected before telling the referee. Once confirmed, a 3D animation of the decision was shown on the stadium screens and to broadcasters.
Key point: The AI does not make the decision on its own: it proposes one, and humans check the two things most likely to go wrong, the moment of the pass and the position of the line.
What tracking makes possible
- Physical data: distance covered, sprints and accelerations for every player.
- Tactical analysis: team shape, distances between lines, pressing intensity and the space each player controls.
- Officiating: offside decisions, and checks such as whether the ball crossed the goal line.
- Media: live graphics, virtual replays and second-screen statistics.
The limits
- Occlusion: when players overlap, a camera may lose sight of a body part or confuse two people.
- Identity switches: if the system swaps two players after a crowd, every statistic that follows is wrong until it is corrected.
- Broadcast video is harder: systems that work from television pictures only see the players in shot, so positions off-screen have to be estimated or are missing.
- Access: high-quality tracking data from stadium systems is mostly owned by leagues, clubs and data companies, and rarely published.
Tracking data is one of the inputs that make modern prediction models possible. To see how such data becomes a forecast, read How AI Predicts Football Matches.
Further reading
- Semi-automated offside technology to be used at FIFA World Cup 2022 — FIFA, 1 July 2022
- Articulated body pose estimation — Wikipedia: estimating the position of body parts from images
- Goal-line technology — Wikipedia: camera-based officiating in football
The AI Sports team writes about data, models and technology in sport.