Waymo flood accident: The cognitive trap of autonomous driving

Why AI cannot understand "common sense dangers"

0.6 Human drivers would turn around while cursing in meter-deep water, but Waymo's self-driving car drove through it three times in a row-until the Atlanta city department called to stop it. The sensor is not broken, the algorithm is not broken, the problem lies in the AI's natural blindness to "common sense dangers."

Survivor Bias Feed Big AI

The training data of the autonomous driving system is essentially a "normal driving documentary": 99.9% of the mileage is reasonable operation on dry roads. Engineers were busy optimizing the timing of lane changes and responding to red lights, but they forgot to teach AI one thing: ** what to do. **

The risk avoidance ability of human drivers comes from two data sources:

1. ** rolled over in person **(You were 16 years old and drifted into a tree in a heavy rain)
2. ** Watching the tragedy **(a neighbor's engine overhaul bill after his car was soaked in water)
   But in the AI's training set, there is only "water that passes successfully"-decisions to retreat, detour, and call for a trailer will not appear in the driving record.

No matter how expensive the ### sensor is, it cannot detect "not worth it"
Some people say that lidar should take the blame-it's just nonsense. Waymo's fifth-generation system uses 360° lidar + millimeter wave radar + camera, which can even model the swing amplitude of roadside dogtail grass. ** The problem is that the algorithm classifies stagnant water as a "traversable obstacle"**, just like humans wanting to squeeze in when they see a mall promotion, completely ignoring the risk of stepping on.

What's even more ironic is that the "safety redundancy design" of autonomous driving has exacerbated misjudgments:

• multi-sensor cross-verification → Water depth error < 5cm
• high-precision map marking drainage outlet → Real-time matching success rate 99.8%
• vehicle wading ability test → Electric chassis waterproof rating IP67

** Perfect data piles up to create perfect errors **: The system knows that when the water depth is 60cm, there is still a safety margin of 6 cm, but it doesn't know that the maintenance cost of the soaking water tanker is enough to buy 200 drivers.

Machine learning in the emergency room

(Embedded allegory paragraph-does not point out the concept)

The new AI triage system in the emergency department of the city hospital can identify signs of myocardial infarction in 0.2 seconds, reducing the mortality rate by 18%. Until one morning a young man reeked of alcohol came in:

• vomitus test → No toxin found
• heart rate/blood pressure → slightly higher but not exceeding the standard
• self-stated that "I just drank too much"

The system threw him into the observation area. Two hours later, the nurse made a ward round and found that something was wrong-what the man vomited was not food residue, but a black-brown liquid like coffee grounds. Manual blood draw for examination: gastric bleeding + alcohol poisoning.

Only after checking the log later found that the logical chain of AI was in perfect harmony:

1. visual model training set = food poisoning cases (yellow-green bile/undigested matter)
2. Typical symptoms of alcoholism = excitement/confusion (this person is quiet and drowsy)
3. gastric bleeding characteristics = hematochezia/abdominal pain (the patient did not complain of this)

** Key data missed **: Drunks often lie about their illness, and old nurses rely on smelling vomit without the smell of sour blood-this "unprofessional judgment" will never be written into medical records.

Why Simulation Can't Save Common Sense

Steel will definitely want to fight: "Wouldn't we just strengthen extreme weather simulation training?" The simulator can simulate the magnitude of heavy rain, water flow speed, and tire slip coefficient, but it cannot simulate the "wisdom of giving up".

The absurdity of the real world far exceeds the parameter library:

• In heavy rain in Chicago, humans will give up overtaking because the bus in front splashed the water curtain
• After Hurricane Houston, veteran drivers saw plastic bags floating on the water and took a detour (there may be a collapsed manhole cover below)
• During the typhoon in Guangzhou, the taxi actively refused to take a taxi not because it was afraid of water, but because it knew that it would not pay for it if it went to the company.

These decisions rely on three layers of tacit knowledge:

1. ** Cost awareness **(car repair fee vs late deduction)
2. ** Responsibility **(Who is responsible for soaking in water?)
3. ** Social experience **(drainage systems in some sections have been in disrepair for a long time)