industry / 03 · autonomous vehicles

Autonomous vehicle training data. Sourced to the scenes that cause disengagements.

Your fleet logs have millions of ordinary miles. What they lack is the cyclist behind the van, the lane closure marked with three hand-placed cones, the wet asphalt at dusk. eeko's data practice sources driving footage, point clouds, and expert-reviewed labels against a written scenario list, with consent, masking, and signed provenance on every item your safety team will ask about.

Scenario-list sourcing Labels to your guideline Second expert on disputed frames Releases per clip
Hero image · 840 × 1050
first reply
1business day
a data lead, with questions
fingerprinting
1hash / frame
duplicates caught before they inflate the set
disputed frames
2nd expert
occlusion and intent calls never rest on one annotator
delivery formats
6
COCO, WebDataset, JSON Lines, Parquet, PLY, Croissant

What is autonomous vehicle training data?

Autonomous vehicle training data is the recorded and labeled driving experience a perception, prediction, or planning model learns from: camera video and lidar point clouds of real roads, the boxes, masks, and lane markings drawn over them, the intent tags that say what a pedestrian is about to do, and the scenario descriptions a planner is judged against. A driving stack is only as safe as the rarest scene in that set.

The economics are upside down. Common driving is nearly free to collect and nearly useless at the margin. The scenes that matter are rare in every fleet log, expensive to label consistently, and full of people who never agreed to be recorded. A dataset that solves all three does not come from a crawler or from more miles.

eeko's data practice sources it on purpose: footage from consenting contributors matched to a scenario list, annotation to your guideline by trained reviewers, and a verifiable record on every clip. The same governance habits run through our compliance and governance work for regulated enterprises. Start at the AI training data hub or see how verification works item by item.

Driving edge-case data and AV annotation, to a written scenario list.

Six things AV teams ask for most. Each starts from a spec and a pilot batch, evaluated on your side before volume.

01 / footageCORE
Edge-case driving video
Clips gathered against your scenario list rather than general mileage: night rain, work zones, unprotected left turns, emergency vehicles, animals, unusual road users, and intersections that do not match the map. Licensed from consenting contributors. More under video datasets.
  • Scenario list written in scoping
  • Match rate reported on the pilot
02 / annotationCORE
Scene and object annotation
2D and 3D boxes, semantic and instance segmentation, occlusion and truncation flags, lane geometry, and sign labels, all drawn to your guideline by trained annotators. See data annotation services.
  • Guideline piloted before production
  • Label source recorded per object
03 / intentREVIEWED
Road-user intent labels
Pedestrian, cyclist, and driver intent tags for prediction models that have to anticipate the next two seconds. The judgment calls are reviewed, and the disputed ones go to a second expert rather than a coin flip.
  • Crossing, yielding, merging, waiting
  • Disagreement kept in the record
04 / lidarCORE
Point clouds and 3D scenes
Lidar point clouds and 3D scene assets delivered as PLY, labeled for objects and free space where the perception stack needs them, with camera-lidar calibration recorded alongside. See 3D and CAD datasets.
  • 3D boxes and free-space masks
  • Calibration files in the delivery
05 / languageCORE
Scenario descriptions for planning
Plain-language scene narratives and expert critiques, written by driving instructors and traffic engineers, for vision-language models and for grading what a planner decided to do.
  • What happened, what should have happened
  • Graded against a written rubric
06 / held-outEXCLUSIVE
Held-out test scenarios
Never-published scenario sets for regression testing, hashed frame by frame and licensed to you alone, so you can prove they never touched training. See AI evaluation benchmarks.
  • Near-duplicate check against the registry
  • Per-clip hash on the receipt
Edge-case scene image · 2100 × 900
The frame your model got wrong, sourced on purpose and labeled by someone who knows what an occlusion flag means.

Four things a bigger fleet log still cannot give you.

The long tail stays long

Doubling your miles roughly doubles your highway footage and barely moves the count of hand-placed cones at dusk. Sourcing to a scenario list spends effort where the disengagements are.

Labels drift between annotators

Is that pedestrian waiting or about to step out? Is that cyclist 40 percent occluded or 60? A written guideline, a pilot on the ambiguous cases, and a second expert on disputed frames keep the taxonomy stable at scale.

Bystanders never signed anything

Street footage is full of faces, plates, and front doors. Every eeko clip carries its releases, consent is checked before a clip with an identifiable person is offered, and masking is applied where your policy requires it.

Safety cases need a paper trail

When a reviewer asks where a training clip came from and under what terms, a signed provenance record, an independent timestamp, and a receipt listing the clip by hash are an answer. A folder named "misc_dashcam" is not.

Perception training data labeled by people who understand roads.

Annotation workstation image · 1200 × 900
One guideline, one taxonomy, every disputed frame escalated instead of guessed.

The reviewers behind a driving dataset

  • Computer vision annotators — boxes, masks, lanes, and flags to your guideline, with throughput and agreement tracked per annotator.
  • Driving instructors and traffic engineers — intent labels, scenario descriptions, and critiques of planner behavior from people who teach and design for real roads.
  • 3D technicians and engineers — point cloud labeling, free-space masks, calibration checks, and the hard occlusion rulings. See mechanical engineering experts and teleoperation experts.
  • Contributors — drivers and fleet operators who license footage through the eeko creator catalogue, each on a release with consent recorded.

Every reviewer is identity-verified before payment. Every label says who made it: a contributor, an AI suggestion, or an expert reviewer. You can filter on that when you train.

What a clip carries by the time it ships.

An illustrative clip lifecycle. Every clip in the batch goes through the same gates.

// clip-3381.log scenario-list.log audit-chain.log
tail -f
07:50 INTAKE Contributor uploads a 48-second clip: urban, night, rain, cyclist partially occluded by a stopped van. Matches scenario 12 on the list.Contributor identity verified · licence terms on file MATCHED
07:51 FINGERPRINT Every frame hashed and compared against the registry. One near-duplicate found: an earlier upload of the same drive from a second camera.Linked as a sibling, not counted twice LINKED
07:52 PRIVACY GPS track and device metadata removed. Two faces and three plates detected; masking applied per spec section 5.Consent not required after masking · property release linked CLEARED
11:20 ANNOTATE Boxes, segmentation, and occlusion flags drawn to guideline v4. Cyclist intent tagged as crossing by the first reviewer.Label source: expert · 212 objects across 1,440 frames LABELED
14:05 DISPUTE Second reviewer reads the cyclist as yielding. Frame escalated to a traffic engineer, who rules crossing and writes a one-line rationale.Both opinions kept in the hash-chained audit trail RESOLVED
16:30 DELIVER Provenance record signed, independent timestamp anchored, clip shipped as WebDataset plus COCO labels, listed on the signed receipt by ID and hash.Batch of 310 clips closed DONE

From disengagement report to delivered batch.

Fixed scope and a fixed price once the spec and guideline are signed. Nothing goes to volume until the pilot has been through your evaluation.

STEP 01
Share the failures
Send the scenarios your models miss, in whatever form you have them. Within one business day a data lead answers with questions and a first scenario list.
STEP 02
Spec and guideline
Scenario list, sensor requirements, annotation guideline, taxonomy, masking policy, formats, licence, and price in one signed document.
STEP 03
Pilot batch
A first set of clips and labels through the whole pipeline. You run it through your evaluation and tell us what to tighten before volume.
STEP 04
Production or standing order
Volume at the agreed bar, or a standing order that extends scenario coverage as the model improves. Same guideline, same reviewers.

Self-driving dataset questions.

Can you source specific driving scenarios like night rain or construction zones?

Yes, and that is the normal way we work. In scoping we convert your disengagement reports or eval failures into a written scenario list: weather, light, road type, road users, and the event itself. Sourcing then targets that list instead of general mileage. The pilot batch shows how closely the footage matches before you commit to volume, and a standing order keeps extending coverage as the model improves and the list changes.

How are faces and license plates handled in driving footage?

Street footage contains people who never agreed to be in a dataset, so the rules are strict. A clip showing an identifiable person is released for training only with consent on record, and releases are tied to the item itself. Location and personal metadata are removed at intake. If your privacy review calls for blurring or boxing faces and plates in the imagery, we write that into the spec and apply it before delivery, not after.

What annotation types do you offer for autonomous vehicle data?

2D and 3D bounding boxes, semantic and instance segmentation, occlusion flags, truncation flags, lane geometry, sign labels, pedestrian and cyclist intent tags, and free-text scene descriptions. Everything is drawn to your annotation guideline, each label carries its source, and a second expert decides disputed frames. An unusual taxonomy gets piloted on a small batch first so the ambiguous cases surface while they are still cheap to resolve.

What formats do you deliver AV datasets in?

Images and captions in COCO, video as WebDataset shards, labels and metadata as JSON Lines or Parquet, point clouds as PLY, and Croissant metadata describing the whole set. If your pipeline expects a specific frame naming or calibration layout, the spec records it and the pilot batch arrives that way. Every delivery ships with per-item provenance records and a signed receipt that names every item by ID and hash.

How much does autonomous vehicle training data cost?

Four things set the price: how rare the scenario is, how dense the annotation is, how many label types you need, and the licence. Full segmentation of a rare night-rain intersection costs far more than boxes on a clear highway. Exclusive rights cost more than shared rights, and volume or a standing order brings the unit price down. We quote a fixed price once the spec and guideline are agreed.

Do you provide held-out test scenarios for regression testing?

Yes. A held-out set is sourced and labeled for you alone, hashed frame by frame at intake, checked against the registry for near-duplicates, and licensed exclusively, so it stays out of every training corpus including your own. The signed receipt lists each clip by hash, which lets your safety team confirm later that nothing in the test set ever crossed into training. It is the cleanest answer to a benchmark contamination question.

Point cloud image · 2100 × 900
Point clouds labeled for objects and free space, with the camera-lidar calibration shipped in the same batch.

Send us the scene your stack keeps missing.

A disengagement report, an eval failure, or a one-line description is enough to start. Inside one business day a data lead comes back with questions, a first scenario list, and a route to a pilot batch. No deck. A ranked list.

Experienced within

  • SOC 2 Type II
  • ISO 27001
  • HIPAA
  • HITRUST r2
  • ISO 27701
  • PCI DSS
  • NIST CSF 2.0
  • CMMC
  • FedRAMP
  • GDPR

Markets served.

As an enterprise AI agency, eeko systems delivers production AI systems remote-first across the United States and internationally — including these markets:

New York City, New York (NY)

Los Angeles, California (CA)

Chicago, Illinois (IL)

Houston, Texas (TX)

Phoenix, Arizona (AZ)

Philadelphia, Pennsylvania (PA)

San Antonio, Texas (TX)

San Diego, California (CA)

Dallas, Texas (TX)

San Jose, California (CA)

Austin, Texas (TX)

Jacksonville, Florida (FL)

Fort Worth, Texas (TX)

Columbus, Ohio (OH)

Charlotte, North Carolina (NC)

Indianapolis, Indiana (IN)

San Francisco, California (CA)

Seattle, Washington (WA)

Denver, Colorado (CO)

Washington, District of Columbia (DC)

Boston, Massachusetts (MA)

El Paso, Texas (TX)

Nashville, Tennessee (TN)

Detroit, Michigan (MI)

Oklahoma City, Oklahoma (OK)

Portland, Oregon (OR)

Las Vegas, Nevada (NV)

Memphis, Tennessee (TN)

Louisville, Kentucky (KY)

Baltimore, Maryland (MD)

Milwaukee, Wisconsin (WI)

Albuquerque, New Mexico (NM)

Tucson, Arizona (AZ)

Fresno, California (CA)

Sacramento, California (CA)

Kansas City, Missouri (MO)

Atlanta, Georgia (GA)

Miami, Florida (FL)

Colorado Springs, Colorado (CO)

Raleigh, North Carolina (NC)

Omaha, Nebraska (NE)

Long Beach, California (CA)

Virginia Beach, Virginia (VA)

Oakland, California (CA)

Minneapolis, Minnesota (MN)

Tulsa, Oklahoma (OK)

Arlington, Texas (TX)

New Orleans, Louisiana (LA)

Wichita, Kansas (KS)

Cleveland, Ohio (OH)

Tampa, Florida (FL)

Bakersfield, California (CA)

Aurora, Colorado (CO)

Honolulu, Hawaii (HI)

Anaheim, California (CA)

Santa Ana, California (CA)

Corpus Christi, Texas (TX)

Riverside, California (CA)

Lexington, Kentucky (KY)

St. Louis, Missouri (MO)

Stockton, California (CA)

Pittsburgh, Pennsylvania (PA)

Saint Paul, Minnesota (MN)

Cincinnati, Ohio (OH)

Greensboro, North Carolina (NC)

Anchorage, Alaska (AK)

Plano, Texas (TX)

Lincoln, Nebraska (NE)

Orlando, Florida (FL)

Irvine, California (CA)

Newark, New Jersey (NJ)

Toledo, Ohio (OH)

Durham, North Carolina (NC)

Chula Vista, California (CA)

Fort Wayne, Indiana (IN)

Jersey City, New Jersey (NJ)

St. Petersburg, Florida (FL)

Laredo, Texas (TX)

Madison, Wisconsin (WI)

Chandler, Arizona (AZ)

Buffalo, New York (NY)

Lubbock, Texas (TX)

Scottsdale, Arizona (AZ)

Reno, Nevada (NV)

Glendale, Arizona (AZ)

Gilbert, Arizona (AZ)

Winston-Salem, North Carolina (NC)

North Las Vegas, Nevada (NV)

Norfolk, Virginia (VA)

Chesapeake, Virginia (VA)

Fremont, California (CA)

Garland, Texas (TX)

Richmond, Virginia (VA)

Baton Rouge, Louisiana (LA)

Boise, Idaho (ID)

San Bernardino, California (CA)

Spokane, Washington (WA)

Des Moines, Iowa (IA)

Modesto, California (CA)

Birmingham, Alabama (AL)

Tacoma, Washington (WA)

Fontana, California (CA)

Oxnard, California (CA)

Fayetteville, North Carolina (NC)

Huntsville, Alabama (AL)

Moreno Valley, California (CA)

Rochester, New York (NY)

Glendale, California (CA)

Yonkers, New York (NY)

Augusta, Georgia (GA)

Amarillo, Texas (TX)

Little Rock, Arkansas (AR)

Akron, Ohio (OH)

Shreveport, Louisiana (LA)

Grand Rapids, Michigan (MI)

Mobile, Alabama (AL)

Salt Lake City, Utah (UT)

Huntsville, Texas (TX)

Tallahassee, Florida (FL)

Overland Park, Kansas (KS)

Knoxville, Tennessee (TN)

Worcester, Massachusetts (MA)

Brownsville, Texas (TX)

New Port Richey, Florida (FL)

Jackson, Mississippi (MS)

Providence, Rhode Island (RI)

Fort Lauderdale, Florida (FL)

Sioux Falls, South Dakota (SD)

Tempe, Arizona (AZ)

Cape Coral, Florida (FL)

Springfield, Missouri (MO)

Pembroke Pines, Florida (FL)

Eugene, Oregon (OR)

Peoria, Arizona (AZ)

Corona, California (CA)

Lancaster, California (CA)

Rockford, Illinois (IL)

Salinas, California (CA)

Palmdale, California (CA)

Springfield, Massachusetts (MA)

Charleston, South Carolina (SC)

Duluth, Minnesota (MN)

London, England (ENG)

Dublin, Ireland (IRE)