Normally, I’d wrap up 2020 by looking back at the predictions I made at the beginning of 2020. Except…I didn’t make any predictions at the beginning of 2020. I skipped a year, so I’ll have to dig back two years to look at the predictions I made at the start of 2019.
Following the example of Scott Alexander, I assign probabilities to my predictions. This allows a finer-grained evaluation of how accurate my predictions were. Unfortunately, scoring one-year predictions two years later kind of nullifies this exercise, but here we go.
100% Certain
✓No Level 5 self-driving cars will be deployed anywhere in the world.
90% Certain
✓Level 4 autonomous vehicles will be on the road, at least in test mode, somewhere in the US.
✓Deep learning will remain the dominant tool for image classification.
✓Human drivers will be permitted on all public roads in the US.
✓No car for sale anywhere in the world will include vehicle-to-traffic-light communication. [Maybe by now this is true in China?]
✓C++ will be the dominant programming language for autonomous vehicles.
✓Autonomous drone delivery will be available commercially somewhere in the world. [Google and Walmart have announced pilots — unclear if those pilots are currently ongoing.]
80% Certain
✓Level 4 self-driving cars will be available to the general public (with or without a safety operator) somewhere in the US.
✓Waymo will have recorded more autonomously-driven miles (all-time) than any other company.
✓Level 4 vehicles will operate, at least in test mode, without a safety operator, somewhere in the US.
✗No vehicle available for sale to the general public will come with OEM-installed lidar. [I think the Audi A8 still has a lidar, but it’s unclear. Volvo announced Luminar-equipped vehicles, but they’re not yet in production.]
✓No dominant technique will emerge for urban motion planning.
70% Certain
✗Level 4 vehicles will be available to the general public somewhere in Europe.
✓Level 4 vehicles will be available to the general public somewhere in China.
✗An autonomous shuttle running on public roads will be open to the general public somewhere in the world. [This seems like it must be true, but I’m not sure where. Public shuttles from May Mobility and Navya pop up periodically, but they always seem to be short-term engagements.]
✓A company will be acquired primarily for its autonomous vehicle capabilities with a valuation above $100M USD. [Luminar, Uber ATG, Zoox, although all of those are special cases in their own ways.]
✗Grocery delivery via autonomous vehicles, with no safety operator, will be available somewhere in the world.
60% Certain
✗No Level 4 self-driving cars will be available to the general public, without a safety operator, anywhere in the US.
✓Tesla will offer the best-performing Advanced Driver Assistance System available to the public. [“Best-performing” is subjective. Various ratings have downgraded Tesla Autopilot and Full Self-Driving, largely due to poor communication and driver monitoring. But Tesla still seems to me to be clearly in the lead with ADAS.]
✓All publicly available Level 4 vehicles will use lidar.
✗A member of the public will die in a collision involving a Level 4 autonomous vehicle (including if the autonomous vehicle is not at-fault). [Not that I’m aware of since January 1, 2019.]
✗Self-driving cars will be available to the general public somewhere in India. [Not that I know.]
50% Certain
✗A Level 3 vehicle will be for sale to the general public somewhere in the world. [Audi has pulled back on this. Volvo has announced but not yet delivered.]
✓Tesla’s full self-driving hardware will include a custom-designed computer.
✗Amazon will make routine (e.g. non-demonstration) autonomous deliveries using autonomous vehicles. [Supposedly Scout is still testing deliveries, but they’re pretty under-the-radar and I don’t consider these yet “routine.”]
✓A company will be acquired primarily for its autonomous vehicle capabilities with a valuation above $1B USD. [Zoox, although this is not quite what I expected when I made the prediction.]
✗Two of the US Big Three and German Big Three (i.e. two of six) will merge.
Evaluation
Evaluation one-year predictions over a two-year horizon isn’t really accurate, but here’s how I scored.
100% confidence = 100% accuracy
90% confidence = 100% accuracy
80% confidence = 60% accuracy
70% confidence = 40% accuracy
60% confidence = 40% accuracy
50% confidence = 40% accuracy
The graph should ideally be a straight line up and to the right. Instead, my graph looks like this.
Not terrible, but room to improve, for sure.
Looking over what I got wrong, it seems like two-year-ago-David thought there would be much more widespread public testing of Level 4 vehicles (Europe! India! Deliveries! Fatalities!) but all with safety operators. Instead, we’ve seen steady and cautious progress (more miles, removing the safety driver) by the largest companies in the markets in which they were already operating.
Better predicting next year!
David Silver 