PanoPed Read the paper ↗

Full-sphere pedestrian tracking

PanoPedBeyond Bounding Boxes for
Sim-to-Real Panoramic Pedestrian Tracking

Qinfeng Zhu · Weiguang Zhao · Yunxi Jiang
Anh Nguyen · Lei Fan

PanoPed brings people tracking onto the full sphere, from stationary cameras to quadruped and drone viewpoints. Sextant reads visual evidence already inside a frozen tracker to place each person more precisely.

Read paper ↗ PDF ↗ Code Explore the data
Synthetic panoramic drone-camera view
01 / DroneFull-sphere perception
Synthetic fixed-camera panorama
02 / Fixed camera
Synthetic quadruped-mounted panorama
03 / Quadruped
Watch the project film
60synthetic sequences
5real sequences
34,692Sextant parameters

Fixed · Quadruped · Drone

01 / Motivation

Why a box falls short
on the sphere.

Planar boxes are useful detector outputs, yet they cannot fully describe the visible angular support of a person on a sphere. PanoPed pairs a synthetic benchmark with consented real scenes and evaluates tracking in native spherical regions.

Video introductionUE third-person illustrations + released ERP footage · original instrumental audio

02 / The benchmark

Synthetic training.
Real-world evaluation.

Full-sphere tracking from simulated fixed and moving cameras to consented real panoramas, with each target protocol identified explicitly.

PanoPed-SUnreal Engine · v1.0.0

Controlled worlds.
Uncontrolled viewpoints.

60 one-minute sequences and 108,000 RGB frames, with fixed, quadruped-mounted and drone-mounted cameras. Released annotations include visible support, spherical regions, identities, depth and scene state.

30fixed
18quadruped
12drone

License: PanoPed-S Research-Only Data License v1.0.0. Noncommercial research/education; access requires acceptance of the release terms. Reconstructing or redistributing underlying meshes and textures is prohibited.

View release packages
PanoPed-RConsented real video · v0.9.0

Real people.
Real panoramic ambiguity.

Five fixed-camera sequences and 28,002 frames across corridors and open plazas. Three sequences have dense identity and full-sphere Box-fit annotations; the other two are released as unlabeled video.

5sequences
16,247labeled frames
3LOSO folds

License: PanoPed-R Research-Only Data License v0.9.0. Noncommercial research/education; access requires acceptance of the release terms. Identification and named-person surveillance uses are prohibited; secure storage is required.

View release packages
Auxiliary target

PanoPed-R-Support-Auto

An automatically derived visible-support target for source-only transfer analysis. It remains separate from the human-annotated real Box-fit benchmark and carries the PanoPed-R Research-Only Data License v0.9.0.

Explore Support-Auto
Three synthetic panoramas followed by three consented real panoramas with frozen automatic visible-support overlays
Fixed, quadruped, and drone views above; real corridors and plazas below. Real colored regions come from box-prompted automatic Support-Auto masks, not hand-drawn pixel labels. View full-size figure ↗

03 / The method

Sextant: localization
from frozen queries.

Sextant reuses the detector's frozen visual query. It predicts a local angular correction while the tracker's identities, scores, detections and history remain untouched.

Sextant diagram showing a frozen detector and identity stream, a 34,692-parameter angular localization head, and a native spherical region
Frozen tracker → query-conditioned readout → native spherical localization. The identity branch bypasses Sextant.
01

Read

256D detector query, initial geometry and confidence.

02

Correct

Lightweight 262→128→4 head, trained on native support targets.

03

Preserve

Final localization changes; scores, identities and tracker state do not.

04 / Paired results

Tracking gains under
defined protocols.

PanoPed-S TEST · MOTIP

Native full-sphere tracking

Geometric readout47.30
MOTIP + Sextant · seed 4249.49

HOTA ↑ · bars use a 0–100 scale · seed 43: 49.47

PanoPed-R-Support-Auto · HAT · source-only

Transfer without real-label fitting

Geometric readout31.71
HAT + Sextant · seed 4232.85

HOTA ↑ · bars use a 0–100 scale · different tracker and evaluation target from S TEST

05 / Scene library

Explore the benchmark footage.

Short excerpts from released RGB, encoded for this page. Synthetic labels come from release annotations; no generated or reconstructed people are added.

01 · PanoPed-S

Fixed camera

One full sphere, one stable viewpoint.

02 · PanoPed-S

Quadruped camera

Street-level movement through a crowd.

03 · PanoPed-S

Drone camera

Changing height and viewpoint.

04 · PanoPed-S

Released labels

L1 spherical regions over RGB.

05 · PanoPed-R

R-01 · Corridor

Seam crossing and lower-polar targets.

06 · PanoPed-R

R-02 · Open plaza

Multiple levels, stairs and scale variation.

07 · PanoPed-R

R-03 · Courtyard

Occlusion across a broad open scene.

08 · PanoPed-R · Unlabeled

R-U01 · Indoor passage

Released RGB footage, without dense identity labels.

09 · PanoPed-R · Unlabeled

R-U02 · Outdoor square

Released RGB footage, without dense identity labels.

All five real sequences are shown. R-U01 and R-U02 are released without dense tracking annotations; only their RGB appears here.

06 / Read & build

Take a closer look.
Build on PanoPed.

Read the full paper for the benchmark design, spherical geometry and paired evaluation protocols. If you use this work, please cite the paper and identify the dataset release version.

Read on arXiv ↗Explore the code ↗
arXiv:2610.08826 · 2026
@article{zhu2026panoped,
  title={PanoPed: Beyond Bounding Boxes for Sim-to-Real Panoramic Pedestrian Tracking},
  author={Zhu, Qinfeng and Zhao, Weiguang and Jiang, Yunxi and Nguyen, Anh and Fan, Lei},
  journal={arXiv preprint arXiv:2610.08826},
  year={2026},
  eprint={2610.08826},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  doi={10.48550/arXiv.2610.08826},
  url={https://arxiv.org/abs/2610.08826}
}
Download .bib ↓