An autonomous haul truck never gets tired, never gets distracted, and never quite sees the world the way a driver does. Its lidar traces geometry, its radar returns blobs, its cameras read light — and between those three, a person standing beside the haul road at 3 a.m. can be a genuinely hard problem. Dust hangs in the air at dusk shift change, headlights flare the visible cameras, and the lidar cloud says “something, approximately this size, probably not a rock” — which for a two-hundred-tonne truck means stop, wait, and burn the cycle.
This case looks at how an open-pit mine added thermal AI vision to its driverless fleet, and what a “who is it, not just where is it” channel did to night operations.
Project Background
In 2023, an open-pit coal mine in north China was a year into operating one of the region’s larger autonomous fleets: thirty driverless battery-electric haul trucks running fixed routes between the pit and the crusher, around the clock. The autonomy stack was the conventional one — lidar, millimetre-wave radar and visible cameras — and by day it performed well. Nights told a different story. The fleet’s incident log for the year included a genuine near-miss with a service pickup that had broken down unlit on a haul road, and a growing operational tax: dozens of conservative emergency stops per month where the stack detected something it could not classify — wildlife, dust devils, a shadowed rock — and chose to stop a loaded truck on a grade. The mine approved a thermal vision upgrade: AI thermal cores fused into each truck’s perception unit, forward and rear.
Pain Points of the Traditional Approach
- Lidar gives geometry, not identity. A fox, a person and a rock can present nearly the same point-cloud signature at range; the safe answer is always “stop,” and the fleet pays for it in cycles.
- Visible cameras die when they are needed most. Darkness, dust, and oncoming headlights degrade the camera channel exactly in the hours the mine most wants the trucks running.
- Radar is sparse on the small and slow. The obstacles that matter at night — a person, a dog, a stalled unlit vehicle — are precisely the ones radar resolves worst.
- Every unresolved blob costs a truck. A conservative stop on a loaded grade is minutes lost and brake wear spent; multiplied across thirty trucks, ambiguity had become a measurable production loss.
The Thermal Imaging Solution
The upgrade added AI thermal cores to each truck’s perception suite, forward and rear, fused with the existing lidar and radar rather than replacing them. What thermal contributes is the channel the stack was missing: everything alive, and everything with an engine, advertises itself by heat regardless of light. A person beside the haul road, a fox in the dust haze, a stalled pickup with a warm engine bay — each presents as a distinct, classifiable signature at ranges well beyond the lidar’s confident identification, and through dust and darkness that blank the visible channel outright.
The fusion logic uses the channels for what each does best. Lidar continues to own geometry and ranging; thermal owns the question of identity — person, animal, vehicle, or none of these — so the truck’s planner can distinguish “obstacle: stop” from “object: log and proceed” with confidence. Emergency stops became decisions with evidence behind them, and the evidence holds up at 3 a.m. in blowing dust as well as it does at noon.
What the Thermal Solution Changed
- Night classification range more than doubled. People and vehicles are identified as what they are, at distances where lidar alone reported an ambiguous cluster — braking decisions now arrive earlier and calmer.
- False emergency stops fell by over two thirds. Wildlife, dust shadows and rock formations that used to halt loaded trucks are classified and logged without breaking the cycle; the monthly stop count collapsed accordingly.
- The near-miss category went quiet. In the first full year with thermal fusion, no person-vehicle near-miss was logged in night operations — the channel that sees people best is now always watching.
- Night production reached day parity. With ambiguity-driven stops removed from the equation, the fleet’s night shift moved from a discounted shift to a full one in the mine’s production accounting.
Module Selection Notes
Vehicle autonomy duty demands a thermal core built for vibration, dust and temperature swings, with fast radiometric output the perception ECU can fuse at frame rate — and on-vehicle AI acceleration so detection runs at the edge, not over a network. The NEXUS LV0619B AI thermal module integrates the LWIR core with onboard detection for exactly this perception role. Where remote-takeover stations or escort vehicles also need darkness coverage, the low-light visible NIR module complements the thermal channel. See the vehicle application page for onboard perception patterns.
Building perception for autonomous or driver-assist platforms? Talk to our engineers about thermal fusion and edge AI integration.
