Waymo flood accident: The modern tragedy of efficiency

When algorithm optimization overrides security redundancy

Waymo's self-driving cars were flying around like headless flies in the floods of Atlanta, not because sensors failed, but because the ** system was trained to be too "smart"*-too smart to sacrifice 99% of operational efficiency for a 1% extreme scenario.

1

Two hours after the red warning for heavy rain was issued, water reached knees on Peachtree Street in Atlanta. A Waymo car scanned the water ahead, slowed down and hesitated for 10 seconds, then--** turned the steering wheel to the right full, bypassing the main road and rushing into a deeper secondary road. This decision is called "path re-planning" in code, and "suicide" in reality. In a video taken by nearby residents, at least three Waymo cars were trapped in the water with double flashes, and the lidar on the roof of the car stretched out of the water like a drowning person's hand.

The accident report showed that the vehicle was not "unable to recognize the flood", but that it ** mistakenly assessed the probability of passing **. The depth camera captured reflections on the water surface, and the millimeter wave radar detected fluctuations in the water surface, but the instructions given by the decision-making layer were: "If the detour distance is less than 300 meters and the water level is less than 40cm, maintain the original path." The system won 99 bets based on the probability model, but lost the 100th bet.

2

This exposes the fatal flaw of Waymo's decision-making mechanism: ** Using statistics to cover real-world long-tail risks **. Their sensing module can identify "water", but the evaluation module only calculates "whether the current water surface height exceeds the historical passage threshold." It's like a driver only memorizes the question bank and does not practice driving on rainy nights when he takes a driver's license test.

Compare the instinctive reactions of human drivers:

• sees stagnant water and first observes the passing status of the vehicle in front
• determines whether the water flow speed carries floating objects
• flameout risk versus detour time cost

Waymo's decision tree was cut down to only two branches:

** Key gap: There is no final plan for "immediate withdrawal"**. Because every time an evacuation command is triggered, the system has to pay a triple price:

1. vehicle outage loss: $120/hour (Waymo internal operation data)
2. scheduling system re-planning cost-effective
   QKPFX8 GMV loss for QK users to cancel orders

3

This reminds me of Lao Zhou, a signal engineer for the San Francisco Bay Area Rapid Transit System. Their rainstorm emergency manual has a saying: "** When rainwater overflows the third step, the shutdown order does not require any approval **". In 2014, he took the initiative to delay the suspension of the last train for 10 minutes, resulting in the entire line being paralyzed for six hours-the maintenance cost was enough to buy three trains.

Lao Zhou later said at a security meeting: "Efficiency is the religion of peacetime, but disasters only give believers one chance to repent."

This is also the problem for Waymo. ** In their optimization objective function,"safety" is just a constraint, not the highest priority **:

# 简化版决策目标函数（根据公开专利反推）
def optimize_decision(sensors, map, mission):
    # 核心变量
    safety_score = calc_safety()   # 权重0.3
    efficiency_score = calc_efficiency() # 权重0.5
    customer_exp_score = calc_customer_exp() # 权重0.2
    
    if safety_score < threshold_low: 
        return EMERGENCY_STOP  # 仅当明显致命时触发
    else:
        return max(efficiency_score * 0.5 + customer_exp_score * 0.2) 

The logic of the aviation industry is:

if any_risk_factor_detected():
    emergency_protocol()  # 立即启动最高级预案
    report_to_human()     # 同时通知地面控制
    # 效率指标不计入决策

4

"But this is the advantage of autonomous driving!" Someone must have slapped the table,"Humans will make misjudgments in heavy rains. At least the system will be more stable!"

Okay, our steelman counterpoint view:

• ** Data level **: Waymo accident rate per million miles is 0.41, compared with 2.1 for human drivers (NHTSA 2023)
• ** Cost level **: For every 1% increase in security redundancy, operating costs increase by 3%(McKinsey estimates)
• ** Experience level **: Users can't stand "cancellation of orders due to slight water accumulation"

These are all true, but the mistake is that ** mixes risks of different dimensions into one formula to calculate **. Vehicles in the flood are not "likely to scratch", but "likely to murder"-in addition to sensors, there may also be passers-by trying to save the car. When the risk reaches devastating levels, statistics become the killer's defense.

5

The real problem lies in the board room, not in the code warehouse. Waymo just received $2.5 billion in financing last year, and investors are looking at three numbers:
Average daily order volume for 1. bicycles (KPI: 12 orders/day)
2. operating area coverage (KPI: 15% monthly increase)
3. manual takeover rate (KPI: <0.01%)

When the KPI required that "85% of orders must be completed even in heavy rain days," engineers could only plant a ticking time bomb in the code. This is not a technical failure, but a systematic hijacking of security logic by business goals.

6

Five years ago, I visited Boeing's backup control system laboratory. A yellowed memo is posted on the wall: "** Any main system failure must trigger a physical interlock, even if the misjudgment rate is as high as 90%*". The engineer pointed to the 30-kilogram mechanical safety device and said: "This thing costs US$80,000 every time it is accidentally touched and started, but no one dares to remove it-we know that the algorithm is not God."

Waymo may want to engrave this sentence on their servers:

** When optimization becomes a belief, redundancy is the cheapest redemption. **

Atlanta's water-soaked lidar was just the first dud to explode in Silicon Valley's temple of efficiency.