Big Money for Autonomous Vehicles, Again

After a year or more spent in the “trough of disillusionment”, autonomous vehicles seem to be riding high again. Within a few weeks, Nuro has raised $500 million and Pony.ai has raised $400 million. Luminar also continues its path towards a SPAC merger that would raise $400 million at a $3.4 billion valuation.

My old business school teacher, Mark Leslie, used to preach that for startups, “cash is oxygen, and oxygen is life.” It’s good to see that there is still lots of life in autonomous vehicles.

Software Is Eating The Automobile Industry

General Motors announced plans to hire 3,000 software engineers and tech staff to support the development of electric vehicles and new technologies — by the end of Q1. Especially interesting, that talent won’t have to come to Detroit.

“We have a lot of flexibility on where we can draw talent from,” said Ken Morris, GM’s vice president of autonomous and electric vehicle programs.

Nearly a decade ago, Marc Andreesen coined the phrase, “Software is eating the world.”

That’s still happening.

Lyft Level 5’s Machine Learning Pipeline

Lyft Level 5 recently published an amazing overview of their PyTorch-based machine learning pipeline.

They confess that a few years ago, their development process was slow:

“Our first production models were taking days to train due to increases in data and model sizes.…Our initial deployment process was complex and model developers had to jump through many hoops to turn a trained model into a “AV-ready” deployable model….We saw from low GPU and CPU utilization that our initial training framework wasn’t able to completely utilize the hardware.”

The post proceeds to described the new pipeline Lyft built to overcome these obstacles. They started with a proof-of-concept for lidar point cloud segmentation, and then grew that into a production system.

The pipeline accomplishes a lot of infrastructure wins.

Testing. The pipeline incorporates continuous integration testing, both to ensure that the models don’t regress, and also to verify that the code researchers write will run in the PyTorch-based vehicle environment.

Containerization. Lyft invested in a uniform container environment, to mitigate the distinction between local and cloud model training.

Deployment. The system relies heavily on LibTorch and TorchScript for deployment to the vehicle’s C++ runtime. Depending on existing libraries reduces the amount of custom code Lyft’s team needs to write.

Distributed Training. PyTorch provides a fair bit of built-in support for distributed training across GPU clusters.

There’s a lot more in the post. It’s a pretty rare glimpse of a machine learning team’s internal infrastructure, so check it out!

Cruise to Test Fully Driverless Vehicles in San Francisco

Cruise CEO Dan Amman announced yesterday that his company will test fully driverless vehicles on the streets of San Francisco “before the end of this year.”

The post heavily emphasizes the environmentally friendly nature of Cruise’s all-electric fleet, including its Origin prototype vehicle.

“Single occupant, human-driven, gasoline-powered cars are the second largest contributors of greenhouse gases on Earth…burning fossil fuels is no way to build the future of transportation.”

A video by Cruise founder and CTO Kyle Vogt accompanies that announcement. Vogt mentions that Cruise has accumulated “over a million miles of fully autonomous driving, in complex urban environments.”

That seems much lower than I would’ve guessed, so much so that I wonder if it was a mis-statement. Given Waymo’s record of over 20 million autonomous miles, I would assume Cruise would want something closer to that before putting fully autonomous vehicles on the road.

Combined with Waymo’s recent move to open driverless vehicles in Phoenix to the public, it’s been a great week for self-driving cars.

Tesla’s Snake Charger Prototype

In 2015, Tesla released a video demonstrating a prototype “snake charger” that (presumably autonomously) connects and charges a car. It’s amazing and honestly not as creepy as I might have imagined.

I missed this prototype at the time and apparently not much has happened with it in the interim, but recently Elon Musk confirmed on Twitter that the charger is still in the works.

This charger would make a lot of sense were Teslas able to drive autonomously, as Musk hints they will be able to do soon (note that a lot of people are skeptical on that point). Autonomous charging would really untether the vehicle from the need for human intervention.

At this point neither the snake charger nor Full Self-Driving mode are ready. But it’s a pretty awesome video.

Rough Economics for Driverless Vehicles

Waymo has begun offering (“selling”) driverless rides to members of the general public in the Phoenix area. This is super-exciting, both because of the technological achievement and also because of what this advancement will make possible. And that relates to economics.

With paying customers in the vehicle, each ride covers (or at least subsidizes) its own cost. This means that, over time, Waymo can afford to drive many more miles than it could if it had to cover the cost of each ride for testing purposes.

Let’s do some math to see how that works out.

Waymo is currently logging about 1 million self-driving miles per month. Let’s assume all of these miles are part of the Waymo One service in Arizona — just for the purpose of this exercise.

Waymo’s blog post this week shared that “5–10% of our rides in 2020 were fully driverless.” To keep the numbers easy, let’s multiply 1 million by 10%, which yields 100,000 driverless miles per month.

How much money does Waymo save by going fully driverless?

Presumably the driverless vehicles are operating in low-speed environments, at least to start. That means lots of intersections and stopping. Let’s assume the driverless vehicles average 10 miles per hour. That means that, in order to log 100,000 driverless miles in a month, Waymo would have to drive for 10,000 hours.

I’m honestly uncertain of the policies and economics at Waymo One, but let’s imagine that a vehicle normally has a single operator that costs Waymo $30 per hour (wage plus taxes and benefits). That implies an operator cost of $300,000 per month (10,000 hours times $30 per hour), or $4 million per year, to log driverless miles.

That is a lot of money to me, but it doesn’t actually feel like that much money for Waymo. But now imagine scaling up.

On average, for human-driven vehicles (not Waymo), an automotive fatality occurs once every 100 million miles. Imagine that we want Waymo to drive an order of magnitude more than that, every month, in order to validate the safety of its vehicles. That’s 1 billion miles per month. Now the driver cost becomes 1 billion miles, divided by 10 miles per hour, times $30 per hour, equals $3 billion per month. That’s prohibitive, even for Waymo.

But if the vehicle becomes driverless, those costs go away.

For me, that’s one of the really exciting aspects of Waymo opening to the general public. If riders will pay enough to cover the marginal cost of the ride without a driver, then it becomes possible to massively scale testing and validation.

Waymo Launches Public Driverless Vehicles

Only two days ago, I tweeted a question to Oliver Cameron.

I suppose it would be a little much to claim that Waymo was paying attention to my tweets, but lo and behold, today they announced that I can take a fully driverless ride, if I can get to Phoenix 😉

“Beginning today, October 8, we’re excited to open up our fully driverless offering to Waymo One riders. Members of the public service can now take friends and family along on their rides and share their experience with the world.”

This is huge! I am so excited and I can’t wait to get to Phoenix and take my first driverless ride 😀

COVID-19 Ethical Dilemmas

The Verge has a story out about the challenges facing self-driving car operators in the COVID era.

“Waymo and Cruise allow their staffing vendors, Transdev North America and Aerotek, respectively, to make decisions regarding when it is safe to test and how to respond to driver concerns. But drivers say those companies can be slow to act, communications are often contradictory, and they feel pressured to keep working despite unsafe conditions.”

Self-driving car operators, particularly at larger organizations like Waymo and Cruise, tend not to be employees, but rather are often contract workers, often employed by an outside staffing agency. This state of affairs is hardly limited to self-driving cars — contract workers employed by staffing agencies are so common in the Bay Area, for engineering, program management, sanitation, and more.

Even companies whose hourly workers are employees, for example firms with large retail or operations centers, face this COVID challenge. Salaried headquarters staff who can work productively from home, while the hourly employees have to be physically present. The division is really about whether you work with ideas or with the physical world. Doctors and dentists have to go to the office for the same reason.

The Verge reports that Waymo is paying its contractors to stay home, although the story only mentions this in passing, and then proceeds to relay other reports that Waymo contractors are pressured to come to work. So it’s hard to know what to make of that.

I confess the best response to this situation eludes me. It feels right to say that companies should pay their workers to stay home and not risk their health, but it is only a short-term solution for a private company to pay people to not work. In the long-term, those companies will struggle to fund those costs.

There is a public health case to be made for some sort of tax or penalty on firms whose employees catch COVID. That doesn’t solve the wildfire problem that also comes up in the piece, though.

Perhaps the best solution is meticulous record-keeping, so that employers can verify that their teams are in fact staying safe while working.

I can’t wait for COVID to end 😦

Truly Driverless Vehicles

Voyage CEO Oliver Cameron broke the self-driving car Internet recently (I just, but only a little), by predicting, “ I feel very confident that in the next 12-to-24 months, you’re going to see self-driving vehicles, with no person in the vehicle, moving people on a daily basis.”

Oliver hired me into Udacity four years ago to build the Self-Driving Car Engineer Nanodegree Program, so I’m naturally inclined to trust his judgement 🙂

And I think he’s right about this! It might already be true, in fact.

Waymo is supposedly moving people in fully driverless vehicles in the Phoenix area and has been for quite some time. The riders are pre-selected and under NDA, so the reporting has been quite limited. But it might already be happening daily.

In very constrained, geofenced operational domains, I suspect we’ll see more companies launch driverless vehicle services — particularly Chinese companies, which don’t get a lot of coverage in the US, but are progressing rapidly.

In the meantime, if you want to see driverless vehicles in action, follow Oliver’s Twitter feed.

Full Architectural Rewrite

I just saw that a few weeks ago Elon Musk tweeted that Tesla’s Full Self-Driving Functionality will be a “full architectural rewrite”, presumably of the code base.

Rewrites are hard. They often seem exciting at the beginning, because of the opportunity to do everything the “right” way, from the ground up. But it usually turns out that recreating existing functionality, built up over years, requires a lot of effort. If the original software took years to write, the rewrite won’t be trivial.

On the other hand, there is a reason Tesla is the most valuable car company in the world, and now one of the most valuable companies in the world, period. Elon Musk is amazing at leading his people to accomplish what would be impossible for most teams.

A rewrite hardly qualifies as impossible. But it’s usually a pretty big effort.