ADAS Lidar

New cars with camera systems attached to the windshield have become commonplace. Just about every automotive brand now has a front-camera-enabled ADAS system, optional or even standard on their vehicles.

New cars with lidar systems are virtually non-existent.

There are several reasons for this, starting with cost: cameras are cheaper than lasers, all the more so because they are widely mass-produced for applications beyond automotive. Camera data is also easier for engineers to work with than lidar data, and it’s probably more helpful than lidar data.

That said, lidar data is still very helpful, particularly as a complement to camera data.

Velodyne’s H800 lidar unit, which will go into production in 2021, is designed for ADAS, and looks that way. The form factor itself appears designed to plug into a windshield.

At $500, the price point is still high. I assume that $500 becomes more like $5000 by the time the price of the whole lidar-equipped system gets sold to a car buyer. But even a $500 unit / $5000 system is in the realm of feasibility for consumers, whereas a bottom-of-the-line VLP-16 that retailed for $4,000 probably wasn’t realistic.

In an ADAS system, these sensors should facilitate features like automatic lane-switching, route adjustments, and even fleet mapping, which could lead to ever-greater autonomy.

Lasers, coming to a car near you in 2021.

Is Anybody Big Enough to Acquir

Kirsten Korosec has a blockbuster story in TechCrunch about Uber’s efforts to spin off its Advanced Technology Group (ATG). This marks the latest twist in the too-crazy-even-for-Hollywood saga of Uber’s self-driving unit.

Korosec reports that Uber is in discussions to sell ATG to Aurora, although Korosec notes that:

“Even with the expected depletion in Uber ATG’s valuation, it would be seemingly out-of-range for Aurora unless it was able to secure additional outside investment or structure the deal in a way that would allow Uber to keep some equity.”

Uber ATG has twice as many employees as Aurora, further raising the question of who would be acquiring whom.

ATG has been through all sorts of drama — starting with controversy related to hiring away a large portion of Carnegie Mellon University’s robotics team, to subsequently acquiring Otto, a startup headed by former leaders of what is now Waymo (and was then called the Google Self-Driving Car Project). Google sued Uber and Otto founder Anthony Levandowski, in particular, settling just as the case went to trial. That particular controversy was a factor in the ouster of Uber founder and CEO Travis Kalanick.

The biggest ATG crisis was the fatal collision with pedestrian Elaine Herzberg in Phoenix, Arizona, in early 2018. That collision prompted ATG to halt all on-road autonomous vehicle testing for months, and cast a shadow over the whole industry.

I have always been impressed by the caliber of the ATG team, particularly the individuals who worked with Udacity to build our Self-Driving Car Engineer Nanodegree Program.

As Korosec’s article points out, Uber is trying to divest non-core groups across the board, including its JUMP bicycle division and several international affiliates. ATG is part of that effort. But, at 1200 employees, ATG may be the single largest self-driving car organization in the world right now — perhaps only Cruise and Waymo are of comparable size. Which raises all sorts of questions about who might be able to or interested in absorbing all that talent.

The Beauty of Over-The-Air Software Updates

Ford just announced the E-Transit, an electric version of their market-leading Ford Transit commercial van. The announcement contains lots of good information about Ford’s electric ambitions, but the bit that excited me was a bit of a footnote: over-the-air updates.

Tesla pioneered OTA updates years ago, but it’s taken most of the big auto manufacturers a while to follow suit. Ford already promised that the upcoming Mustang Mach-E will have OTA capability — combined with OTA for the E-Transit, this seems like a trend.

Most pure software products that I use today is updated continuously, in most cases because the software itself is hosted in the cloud anyway. Internet of Things software, like the code in my car or my microwave, is much less likely to be connected to the cloud and easy to update.

Those updates are so crucial. Being stuck with software that was written at the time you made a purchase really dates hardware. Ford’s own press release for the Mach-E cities FOMO.

OTA helps a product get better over time, not worse. That’s a game-changer for assets that typically depreciate, like cars. OTAs don’t typically flip a depreciating asset into an appreciating one — it’s still nicer to have a new unit with the newest hardware and the newest software. But OTAs really help slow the depreciating utility of hardware like cars.

Big Day for ADAS

Two big ADAS announcements today!

  • Honda announced they will mass produce the world’s first SAE Level 3-capable vehicle.
  • Rivian posted specs and features for their upcoming electric light trucks, including the news that Driver+ will come standard on every vehicle.

Some caveats are in order.

Honda’s announcement seems to hinge closely on a recent approval from the Japanese government. Whether this level 3 system will be available outside Japan is unclear.

Audi launched an A8 model several years ago that was L3-capable. But they never enabled Level 3 autonomy, due to regulatory concerns.

Rivian’s Driver+, meanwhile, garnered comparison to GM SuperCruise, which is among the best such systems in the industry.

“With the system engaged, your Rivian will automatically steer, adjust speed, and change lanes on your command. Enabled on select highways at launch, more road types will be introduced through over-the-air updates. Like all driver assistance systems available today, Driver+ requires your full attention on the road at all times and you should not use a hand-held device behind the wheel.”

That’s a big claim! I’m excited to see some reviews of the system.

Big Money for Autonomous Vehicles, Again

After a year or more spent in the “trough of disillusionment”, autonomous vehicles seem to be riding high again. Within a few weeks, Nuro has raised $500 million and Pony.ai has raised $400 million. Luminar also continues its path towards a SPAC merger that would raise $400 million at a $3.4 billion valuation.

My old business school teacher, Mark Leslie, used to preach that for startups, “cash is oxygen, and oxygen is life.” It’s good to see that there is still lots of life in autonomous vehicles.

Software Is Eating The Automobile Industry

General Motors announced plans to hire 3,000 software engineers and tech staff to support the development of electric vehicles and new technologies — by the end of Q1. Especially interesting, that talent won’t have to come to Detroit.

“We have a lot of flexibility on where we can draw talent from,” said Ken Morris, GM’s vice president of autonomous and electric vehicle programs.

Nearly a decade ago, Marc Andreesen coined the phrase, “Software is eating the world.”

That’s still happening.

Lyft Level 5’s Machine Learning Pipeline

Lyft Level 5 recently published an amazing overview of their PyTorch-based machine learning pipeline.

They confess that a few years ago, their development process was slow:

“Our first production models were taking days to train due to increases in data and model sizes.…Our initial deployment process was complex and model developers had to jump through many hoops to turn a trained model into a “AV-ready” deployable model….We saw from low GPU and CPU utilization that our initial training framework wasn’t able to completely utilize the hardware.”

The post proceeds to described the new pipeline Lyft built to overcome these obstacles. They started with a proof-of-concept for lidar point cloud segmentation, and then grew that into a production system.

The pipeline accomplishes a lot of infrastructure wins.

Testing. The pipeline incorporates continuous integration testing, both to ensure that the models don’t regress, and also to verify that the code researchers write will run in the PyTorch-based vehicle environment.

Containerization. Lyft invested in a uniform container environment, to mitigate the distinction between local and cloud model training.

Deployment. The system relies heavily on LibTorch and TorchScript for deployment to the vehicle’s C++ runtime. Depending on existing libraries reduces the amount of custom code Lyft’s team needs to write.

Distributed Training. PyTorch provides a fair bit of built-in support for distributed training across GPU clusters.

There’s a lot more in the post. It’s a pretty rare glimpse of a machine learning team’s internal infrastructure, so check it out!

Cruise to Test Fully Driverless Vehicles in San Francisco

Cruise CEO Dan Amman announced yesterday that his company will test fully driverless vehicles on the streets of San Francisco “before the end of this year.”

The post heavily emphasizes the environmentally friendly nature of Cruise’s all-electric fleet, including its Origin prototype vehicle.

“Single occupant, human-driven, gasoline-powered cars are the second largest contributors of greenhouse gases on Earth…burning fossil fuels is no way to build the future of transportation.”

A video by Cruise founder and CTO Kyle Vogt accompanies that announcement. Vogt mentions that Cruise has accumulated “over a million miles of fully autonomous driving, in complex urban environments.”

That seems much lower than I would’ve guessed, so much so that I wonder if it was a mis-statement. Given Waymo’s record of over 20 million autonomous miles, I would assume Cruise would want something closer to that before putting fully autonomous vehicles on the road.

Combined with Waymo’s recent move to open driverless vehicles in Phoenix to the public, it’s been a great week for self-driving cars.

Tesla’s Snake Charger Prototype

In 2015, Tesla released a video demonstrating a prototype “snake charger” that (presumably autonomously) connects and charges a car. It’s amazing and honestly not as creepy as I might have imagined.

I missed this prototype at the time and apparently not much has happened with it in the interim, but recently Elon Musk confirmed on Twitter that the charger is still in the works.

This charger would make a lot of sense were Teslas able to drive autonomously, as Musk hints they will be able to do soon (note that a lot of people are skeptical on that point). Autonomous charging would really untether the vehicle from the need for human intervention.

At this point neither the snake charger nor Full Self-Driving mode are ready. But it’s a pretty awesome video.

Rough Economics for Driverless Vehicles

Waymo has begun offering (“selling”) driverless rides to members of the general public in the Phoenix area. This is super-exciting, both because of the technological achievement and also because of what this advancement will make possible. And that relates to economics.

With paying customers in the vehicle, each ride covers (or at least subsidizes) its own cost. This means that, over time, Waymo can afford to drive many more miles than it could if it had to cover the cost of each ride for testing purposes.

Let’s do some math to see how that works out.

Waymo is currently logging about 1 million self-driving miles per month. Let’s assume all of these miles are part of the Waymo One service in Arizona — just for the purpose of this exercise.

Waymo’s blog post this week shared that “5–10% of our rides in 2020 were fully driverless.” To keep the numbers easy, let’s multiply 1 million by 10%, which yields 100,000 driverless miles per month.

How much money does Waymo save by going fully driverless?

Presumably the driverless vehicles are operating in low-speed environments, at least to start. That means lots of intersections and stopping. Let’s assume the driverless vehicles average 10 miles per hour. That means that, in order to log 100,000 driverless miles in a month, Waymo would have to drive for 10,000 hours.

I’m honestly uncertain of the policies and economics at Waymo One, but let’s imagine that a vehicle normally has a single operator that costs Waymo $30 per hour (wage plus taxes and benefits). That implies an operator cost of $300,000 per month (10,000 hours times $30 per hour), or $4 million per year, to log driverless miles.

That is a lot of money to me, but it doesn’t actually feel like that much money for Waymo. But now imagine scaling up.

On average, for human-driven vehicles (not Waymo), an automotive fatality occurs once every 100 million miles. Imagine that we want Waymo to drive an order of magnitude more than that, every month, in order to validate the safety of its vehicles. That’s 1 billion miles per month. Now the driver cost becomes 1 billion miles, divided by 10 miles per hour, times $30 per hour, equals $3 billion per month. That’s prohibitive, even for Waymo.

But if the vehicle becomes driverless, those costs go away.

For me, that’s one of the really exciting aspects of Waymo opening to the general public. If riders will pay enough to cover the marginal cost of the ride without a driver, then it becomes possible to massively scale testing and validation.