Waymo’s Safety Report

Waymo has just published a lot of information about the safety and validation of its sytems — more than I have yet reviewed. At the top level is a blog post in which Waymo breaks its safety analysi s into three parts:

  • Hardware
  • Software
  • Operations

Within each of those parts is a fair bit more detail and structure, more than I have seen in the past. For example, regarding hardware:

A vehicle equipped with the Waymo Driver has four main subsystems, which form the ‘hardware layer’. This includes the vehicle itself; the systems used for steering and driving; the sensor suite built into the vehicle; and the computational platform used to run our software.

Undergirding these descriptions are three documents:

  • Safety Report. This is 48 pages of glossy material that seems similar to material Waymo has published in the past. There’s a lot of data, but the audience seems to be more for the public and policymakers, rather than engineers and analysts.
  • Safety Methodologies and Safety Readiness Determinations. This looks neat. Lots more detail on the three layers of Waymo’s stack (hardware, software, operations). Lighter detail on how Waymo determines the safety readiness of the layers.
  • Waymo Public Road Safety Performance Data. Academic-style analysis of Waymo self-driving data from the Phoenix metro area in 2019. Unsurprisingly, the collisions recorded tend to be the fault of human drivers in other vehicles, not Waymo AVs. 
    This sentence caught my eye: “There were 47 contact events that occurred over this time period, consisting of 18 actual and 29 simulated contact events, none of which would be expected to result in severe or life-threatening injuries.”

I’m excited to read these documents over the coming days and see what they reveal. As Waymo writes in the blog post:

“There is currently no universally accepted approach for evaluating the safety of autonomous vehicles — despite the efforts of policymakers, researchers and companies building fully autonomous technologies.”

Asked Me Anything

Today I had the privilege of taking over Udacity’s Twitter account to host a 60-minute AMA on robotics and artificial intelligence. Some blasts from my Udacity past even made an appearance 🙂

It was fun 🙂

ADAS Lidar

New cars with camera systems attached to the windshield have become commonplace. Just about every automotive brand now has a front-camera-enabled ADAS system, optional or even standard on their vehicles.

New cars with lidar systems are virtually non-existent.

There are several reasons for this, starting with cost: cameras are cheaper than lasers, all the more so because they are widely mass-produced for applications beyond automotive. Camera data is also easier for engineers to work with than lidar data, and it’s probably more helpful than lidar data.

That said, lidar data is still very helpful, particularly as a complement to camera data.

Velodyne’s H800 lidar unit, which will go into production in 2021, is designed for ADAS, and looks that way. The form factor itself appears designed to plug into a windshield.

At $500, the price point is still high. I assume that $500 becomes more like $5000 by the time the price of the whole lidar-equipped system gets sold to a car buyer. But even a $500 unit / $5000 system is in the realm of feasibility for consumers, whereas a bottom-of-the-line VLP-16 that retailed for $4,000 probably wasn’t realistic.

In an ADAS system, these sensors should facilitate features like automatic lane-switching, route adjustments, and even fleet mapping, which could lead to ever-greater autonomy.

Lasers, coming to a car near you in 2021.

Is Anybody Big Enough to Acquir

Kirsten Korosec has a blockbuster story in TechCrunch about Uber’s efforts to spin off its Advanced Technology Group (ATG). This marks the latest twist in the too-crazy-even-for-Hollywood saga of Uber’s self-driving unit.

Korosec reports that Uber is in discussions to sell ATG to Aurora, although Korosec notes that:

“Even with the expected depletion in Uber ATG’s valuation, it would be seemingly out-of-range for Aurora unless it was able to secure additional outside investment or structure the deal in a way that would allow Uber to keep some equity.”

Uber ATG has twice as many employees as Aurora, further raising the question of who would be acquiring whom.

ATG has been through all sorts of drama — starting with controversy related to hiring away a large portion of Carnegie Mellon University’s robotics team, to subsequently acquiring Otto, a startup headed by former leaders of what is now Waymo (and was then called the Google Self-Driving Car Project). Google sued Uber and Otto founder Anthony Levandowski, in particular, settling just as the case went to trial. That particular controversy was a factor in the ouster of Uber founder and CEO Travis Kalanick.

The biggest ATG crisis was the fatal collision with pedestrian Elaine Herzberg in Phoenix, Arizona, in early 2018. That collision prompted ATG to halt all on-road autonomous vehicle testing for months, and cast a shadow over the whole industry.

I have always been impressed by the caliber of the ATG team, particularly the individuals who worked with Udacity to build our Self-Driving Car Engineer Nanodegree Program.

As Korosec’s article points out, Uber is trying to divest non-core groups across the board, including its JUMP bicycle division and several international affiliates. ATG is part of that effort. But, at 1200 employees, ATG may be the single largest self-driving car organization in the world right now — perhaps only Cruise and Waymo are of comparable size. Which raises all sorts of questions about who might be able to or interested in absorbing all that talent.

The Beauty of Over-The-Air Software Updates

Ford just announced the E-Transit, an electric version of their market-leading Ford Transit commercial van. The announcement contains lots of good information about Ford’s electric ambitions, but the bit that excited me was a bit of a footnote: over-the-air updates.

Tesla pioneered OTA updates years ago, but it’s taken most of the big auto manufacturers a while to follow suit. Ford already promised that the upcoming Mustang Mach-E will have OTA capability — combined with OTA for the E-Transit, this seems like a trend.

Most pure software products that I use today is updated continuously, in most cases because the software itself is hosted in the cloud anyway. Internet of Things software, like the code in my car or my microwave, is much less likely to be connected to the cloud and easy to update.

Those updates are so crucial. Being stuck with software that was written at the time you made a purchase really dates hardware. Ford’s own press release for the Mach-E cities FOMO.

OTA helps a product get better over time, not worse. That’s a game-changer for assets that typically depreciate, like cars. OTAs don’t typically flip a depreciating asset into an appreciating one — it’s still nicer to have a new unit with the newest hardware and the newest software. But OTAs really help slow the depreciating utility of hardware like cars.

Big Day for ADAS

Two big ADAS announcements today!

  • Honda announced they will mass produce the world’s first SAE Level 3-capable vehicle.
  • Rivian posted specs and features for their upcoming electric light trucks, including the news that Driver+ will come standard on every vehicle.

Some caveats are in order.

Honda’s announcement seems to hinge closely on a recent approval from the Japanese government. Whether this level 3 system will be available outside Japan is unclear.

Audi launched an A8 model several years ago that was L3-capable. But they never enabled Level 3 autonomy, due to regulatory concerns.

Rivian’s Driver+, meanwhile, garnered comparison to GM SuperCruise, which is among the best such systems in the industry.

“With the system engaged, your Rivian will automatically steer, adjust speed, and change lanes on your command. Enabled on select highways at launch, more road types will be introduced through over-the-air updates. Like all driver assistance systems available today, Driver+ requires your full attention on the road at all times and you should not use a hand-held device behind the wheel.”

That’s a big claim! I’m excited to see some reviews of the system.

Big Money for Autonomous Vehicles, Again

After a year or more spent in the “trough of disillusionment”, autonomous vehicles seem to be riding high again. Within a few weeks, Nuro has raised $500 million and Pony.ai has raised $400 million. Luminar also continues its path towards a SPAC merger that would raise $400 million at a $3.4 billion valuation.

My old business school teacher, Mark Leslie, used to preach that for startups, “cash is oxygen, and oxygen is life.” It’s good to see that there is still lots of life in autonomous vehicles.

Software Is Eating The Automobile Industry

General Motors announced plans to hire 3,000 software engineers and tech staff to support the development of electric vehicles and new technologies — by the end of Q1. Especially interesting, that talent won’t have to come to Detroit.

“We have a lot of flexibility on where we can draw talent from,” said Ken Morris, GM’s vice president of autonomous and electric vehicle programs.

Nearly a decade ago, Marc Andreesen coined the phrase, “Software is eating the world.”

That’s still happening.

Lyft Level 5’s Machine Learning Pipeline

Lyft Level 5 recently published an amazing overview of their PyTorch-based machine learning pipeline.

They confess that a few years ago, their development process was slow:

“Our first production models were taking days to train due to increases in data and model sizes.…Our initial deployment process was complex and model developers had to jump through many hoops to turn a trained model into a “AV-ready” deployable model….We saw from low GPU and CPU utilization that our initial training framework wasn’t able to completely utilize the hardware.”

The post proceeds to described the new pipeline Lyft built to overcome these obstacles. They started with a proof-of-concept for lidar point cloud segmentation, and then grew that into a production system.

The pipeline accomplishes a lot of infrastructure wins.

Testing. The pipeline incorporates continuous integration testing, both to ensure that the models don’t regress, and also to verify that the code researchers write will run in the PyTorch-based vehicle environment.

Containerization. Lyft invested in a uniform container environment, to mitigate the distinction between local and cloud model training.

Deployment. The system relies heavily on LibTorch and TorchScript for deployment to the vehicle’s C++ runtime. Depending on existing libraries reduces the amount of custom code Lyft’s team needs to write.

Distributed Training. PyTorch provides a fair bit of built-in support for distributed training across GPU clusters.

There’s a lot more in the post. It’s a pretty rare glimpse of a machine learning team’s internal infrastructure, so check it out!