Waymo flood accident: The cognitive trap of autonomous driving

When the algorithm over-optimizes urban routine scenarios

After heavy rains in Atlanta, two Waymo self-driving cars were like robots on suicide missions. They rushed into flood areas with water depths of more than 30 centimeters three times and eventually turned into scrap iron. The official report downplays and attributes it to "sensor misjudgment." Only by tearing open the code layer can you see the true focus: when training the model, flood scenarios are filtered out as noise with a probability of one in a thousand *-this is not a technical failure at all. It is the product manager's KPIs that weld the cognitive boundary into the cage of "high-frequency scenario optimization."

Model Prisoner in the Urban Sandbox

Looking at Waymo's 2023 technical white paper,"complex urban road conditions" are clearly defined: the success rate of lane change on congested sections, the time spent on detour in construction areas, and the response delay of pedestrian ghost probes... are all quantifiable indicators. ** 90% of the training dataset are videos of sunny days in San Francisco and Phoenix, and the remaining 10% are allocated to rainy and foggy days-a once-in-three-year rainstorm and flood in Atlanta? Sorry,"probability is below statistical significance threshold."

But probability cannot fool the physical world. Check the National Weather Service data: Sudden floods in Atlanta have increased by 47% in the past five years, while Waymo's simulation test library has only expanded at a rate of 15%. What's even more ironic is that ** When the flood overflowed the roadside, the lidar point cloud data was determined by the system as "abnormally raised road"**-the algorithm excitedly enabled bump mode instead of emergency braking. The engineer found in the post log that the obstacle avoidance module even completed three perfect detours, accurately avoiding the virtual "gravel pile"(actually floating garbage cans).

Efficiencism strangles emergency instinct

After the accident, an engineer defended on Twitter: "Investing 20% R & D resources into a 0.01% probability event is commercially unsustainable. " ** These words expose the cognitive cancer of the autonomous driving industry: reducing safety decisions to games of chance **. When human drivers encounter reflections from the water in front of them, they will instinctively associate "someone drowned in the news last week." However, AI's "common sense" comes from the "water depth 0cm" tag library posted by the labeling staff.

Compared with Tesla's extreme scenario strategy, it is even more thought-provoking: during the 2022 Texas floods, a car owner photographed the Model 3 actively bypassing the stagnant area. ** The mystery lies in shadow mode-when 1000 vehicles encounter a similar scene, the system will capture the human driver's steering wheel torque change **. Even if the amount of data is small, the sudden turning right of the steering wheel 15 degrees is a more direct survival signal than a lidar point cloud. On the other hand, Waymo regards human emergency response as noise that needs to be eliminated in order to pursue the purity of "no one intervention."

** The real problem has never been technology, it is priority murder on the business ledger **. Looking at the cruel data: For every 1% increase in traffic efficiency (such as green light pass rate) Waymo can save US$24 million in operating costs every year; while developing a flood response module, it not only requires reconstructing the simulation system, but also deploying in 50 cities. Hydrological sensor networks-an initial investment of 120 million yuan, in exchange for the empty talk of "ability to cope with extreme weather" in the PR draft. When KPI only measures "the number of interventions per thousand miles," flood prevention naturally becomes a cost center.

When cities become algorithm testing grounds

Urban planners are stuck in the dilemma of boiling frogs in warm water. The Los Angeles Transportation Authority approved Waymo to test in heavy rain roads last year on the condition that "six additional water level sensors were installed."** What seems to be a rigorous measure is actually the if else logic of outsourcing public safety to technology companies-when sensors are stuck with mud, redundant mechanisms become a single point of failure.

The more hidden risk lies in the closed loop of data: Waymo vehicles scan cities every day to produce petabyte maps, but these high-precision maps instead solidify the narrow definition of "normal city." An intersection in Phoenix was temporarily diverted due to construction, and the system directly determined that it was "abnormal in the map" and stopped driving; similarly, when floods flooded road signs, AI lacking human association capabilities would only adhere to the dogma that "lane lines must be visible." Ironically, the human driver is now downgraded to a more robust mode: just follow the taillights of the front car.

Steelman: Should we sacrifice efficiency for small probability events?

Someone must refute: "Engineering rationality is to solve 99% of the scenarios first and then improve the corner cases. "Aviation is the best mirror-aircraft cruising smoothly 99% of the time, but ** simulators specialize in extreme conditions such as engine failure and wind shear **. After each upgrade of the Boeing 787, it had to use a bench to violently test the performance of the hydraulic system in the freezing state of-40 ° C, because it was known that a 0.1-second delay when the cabin lost pressure would cost hundreds of lives.

The fatal arrogance of autonomous driving lies in treating ground transportation as a fault-tolerant Internet product. Amnon Shashua, founder of Mobileye, once said: "** The safety standard of AI drivers should not be 20% better than humans, but 2000% more stable than humans **." When Waymo used the "million-mile accident rate" as a publicity stunt, it deliberately ignored tens of millions of flood avoidance samples from human drivers-these survival wisdom hidden in marginal scenarios are being washed away as dirty data by optimization algorithms.

Tools How to domesticate tool makers

The floods destroyed not only sensors, but also Silicon Valley's superstition of "absolute rationality." Waymo engineers recommended in an internal report: "Increase the number of manual monitoring of heavy rain weather."** This is tantamount to a disguised admission-pursuing the obsession of no one will actually make the system more fragile. ** When all the team's brainstorms focused on improving lane change efficiency by 0.5%, who still remembers the passenger who might have been trapped in the car during the heavy rain?

What's deeper is the emergence of the motif: ** Tools are shaping users 'cognitive boundaries **. When algorithm engineers applaud the "intervention rate decline curve" every day, when product managers use prevailing efficiency to convert KPI bonuses, when investors only ask "when to remove safety guards"-the crisis in the flood is folded into a statistical shadow. We create tools to understand the world, and ultimately the tools simplify the way we understand the world.

So don't be too busy laughing at the lidar soaking in water. The next time your navigation APP stubbornly leads into a construction dead end, or a video website desperately recommends similar content-** it's just another kind of dam for cognitive flood and silent thinking. **