Tag: Autonomous Vehicles

Nobody Will Share An Alternative To Mileage & Disengagement Reports

Yesterday the California DMV published the 2020 Autonomous Vehicle Disengagement and Mileage Reports. The DMV grants permits to organizations that want to test autonomous vehicles on public roads in the state. Any organizations that do test on public roads must file reports about how many miles they drove, and how frequently their safety operators had to “disengage” the autonomous driving system in order to manually control the vehicle.

Headline numbers are that total autonomous miles summed from all companies actually decreased from 2019 to 2020, presumably due to the pandemic (also the summer wildfires). Cruise and Waymo recorded far and away the most miles, with Pony.ai a distant third, and then an asymptotic trend toward 0 miles.

The Miles Per Disengagement chart looks similar, although it includes a few surprises. For example, AutoX drove only ~41,000 autonomous miles in California during 2020, but they also only disengaged twice.

These numbers reflect only autonomous driving in California, on public roads, during 2020, which is a lot of caveats. That certainly explains why Waymo has so few miles. A few years ago they boasted of achieving 1 million autonomous miles per month, much of that in California. Now they’ve moved most of their driving miles to Arizona.

Perhaps the caveats also explain some of the big names who have major engineering teams in Silicon Valley but don’t appear in the report: Argo, Ford, Uber ATG (now Aurora, but how that merger is reflected here is unclear), and Baidu, for starters.

Tesla’s absence from the report is its own annual, recurring story. None of the standalone Class 8 trucking companies, like TuSimple, Embark, or Kodiak, appear on the list. I’m not sure if that’s because trucks go on a different list or they genuinely did 0 miles in California last year.

The reports themselves are only part of the story, though. For me, a fascinating angle is both how much attention everyone pays to the reports, and also how dismissive everyone is.

And yet, nobody seems willing to share any other numbers.

Waymo offers up their 48-page Safety Report as an alternative evaluation tool, and it is a great report, and it is more than any other company in the industry puts forward. But this report is entirely qualitative. There are no metrics in the report, and no real indication of how fast Waymo is progressing, or why they feel confident pulling safety operators from some streets in Arizona, but not others.

Other companies provide not even that much.

The big question, then, is what are the alternatives to these disengagement reports, will anybody be willing to share them, and will anybody demand to see them?

Big News Day For Ford

Ford reported Q4 earnings this afternoon, posting a $2.8 billion loss, or a $1.3 billion dollar gain, depending on whether how we count “special items.” That’s a $4.3 billion swing.

The bigger news seemed to be Ford’s 2021 outlook. CFO John Lawler estimated Ford would book an annual pre-tax profit of $8 billion to $9 billion dollars in the coming year. That would be a great year for Ford, and potentially its largest profit in 5 years. Although given that this past quarter’s swing due to “special items” was $4.3 billion, there’s a lot of variability here.

Ford also announced a big investment in electric autonomous vehicles. The headline number is $29 billion through 2025, of which $22 billion will go to electric vehicles and $7 billion will go to autonomous vehiicles. Various tweeters explain some of this headline number includes some expenditures from previous years, so it’s not clear how much of this is new money.

Meanwhile, The Wall Street Journal reports that Ford may under-perform expectations by a billion dollars or two, because of a global shortage of semiconductor chips. The shortage is already hitting both GM and Ford. Ford, in particular, is set to cut shifts at its F-150 plants in the coming weeks, due to the lack of chips. Since the F-150 is Ford’s profit engine, that’s expensive.

Download Luminar Lidar Data

Volvo has just published a dataset called Cirrus which includes camera data and Luminar lidar data for 6,285 frames. The dataset includes 8 categories of annotations: “Vehicle, Large Vehicle, Pedestrian, Bicycle, Animal, Wheeled Pedestrian, Motorcycle, Trailer.”

I love how many companies are publishing datasets. This seems to especially make sense for a supplier like Luminar. Engineers anywhere can try out Luminar data without having to engage a sales rep or even convince their own managers.

The name of the dataset, “Cirrus,” highlights the range of Luminar’s lidar. At 250m, Luminar lidar is high range, just as Cirrus clouds are high altitude.

Driverless Robotaxis In China

Yesterday, AutoX announced the launch of its driverless robotaxi service to the public in Shenzhen, China.

As I wrote in Forbes.com, the rollout resembles the process Waymo took to launch its driverless Waymo One service in Arizona, but AutoX is progressing much faster.

Whereas Waymo tested self-driving cars with human safety operators for a decade before advancing to driverless vehicles, AutoX was founded in only 2016 and just began testing fully driverless vehicles a few months ago.

This surprised me:

Also like Waymo, the base vehicle for the AutoX service is the Chrysler Pacifica minivan. The selection of an American automotive manufacturer for this initial program is notable because AutoX has partnerships with many Chinese manufacturers, including Dongfeng Motors, Shanghai Auto, BYD, and Chery Automobile.

There’s even an in-flight safety video you can watch. Read the whole thing.

Deep Dive on Mobileye REM Maps

Yesterday, I posted a brief overview of a couple of presentations Mobileye CEO Amnon Shashua gave at CES 2021 this month. I really enjoyed these presentations, in large part because over the years I’ve read less about Mobileye and know less about them than many other companies in the automotive technology ecosystem.

Today, I re-watched Shashua’s “deep dive” on Mobileye’s REM mapping approach. It’s quite informative, so I took notes.

  • REM is a Mobileye brand name that stands for Road Experience Management
  • The maps are generated from cameras. In the future, Mobileye’s lidar and radar will be designed to work with these camera-only maps, not the other way around.
  • In particular, even future lidar and radar systems will not use standard, point-cloud-based HD maps. Point clouds take up too much storage space to be practical, particularly for updating from a huge fleet of vehicles.
  • Instead of point clouds, REM uses “semantic” maps, that record sparse information, such as driveable paths, stop lines, and traffic signal locations.
  • Identifying this semantic segmentation and uploading it to the cloud takes 10 kb of data transfer per kilometer. This costs somebody (the manufacturer?) $1 per year, on average.
  • All of this begs a question, though — are maps even necessary?
  • In theory, maps aren’t necessary. After all, humans drive without maps (in many scenarios). Humans just figure out the road as we drive.
  • Artificial intelligence can do the same thing, but AI isn’t nearly as good as humans at this (yet). The Mean Time Between Failures (MTBF) for an AI will be low — lots of problems.
  • Solution: prepare a lot of this information in advance, and store it in the map.
  • Shashua says that everyone is using a map, even if they say they’re not. Pretty clear that this is a reference to Tesla.
  • Mobileye’s maps have three performance goals: Scale (consumer vs. robotaxi), Up-To-Dateness (real-time), Accuracy (cm-level)
  • Mobileye has a division which builds lidar-based HD Maps, so they know the pros and cons of this approach
  • Lidar-based HD maps are too detailed. The AI driver only need information for a 200m radius around the vehicle, but HD maps contain very detailed information about the entire world.
  • On the flip side, point clouds are just coordinates in space. AI needs semantic meaning: drivable paths, priority, crosswalks, stopping & yield lines.
  • Calculating this in real-time is theoretically possible, but practically impossible: too many conflicting signs and signals, too much noise, too much going on
  • Mobileye is now creating AV Map, which are not HD Maps: Scalability everywhere, Accuracy in 200m radius, Semantic features generated from wisdom of crowd
  • Map creation process: Harvesting -> Alignment -> Modeling & Semantics
  • In the photo above, only data marked by yellow lines in the photo is uploaded to the cloud. That’s the important information.
  • Mobileye extracts semantic meaning from the data and uses splines to represent driveable paths.
  • Currently, Mobileye maps 8M km of roads every day (6 countries). Unclear if this is 8M unique km, or the same 1km mapped by 8M vehicles every day.
  • By 2024, they’ll be mapping 1B km of roads every day (the whole planet).

Mobileye: Redundancy, Mapping, Safety

At this month’s virtual 2021 Consumer Electronics Show, Mobileye presented a lot. CEO Amnon Shashua sat for a friendly interview with Ed Niedermeyer, and Shashua also gave a standalone hour-long presentation about Mobileye’s technology.

Shashua highlighted three areas of differentiation that provide competitive advantages for Mobileye:

  • Redundancy
  • Mapping
  • Safety

Redundancy

The plan seems to be that Mobileye will build a camera-only driver assistance system, and then layer radar and lidar on top to get to Level 4 autonomy by 2025.

Mapping

Mobileye has worked to identify minimal amounts of high-valuable semantic mapping data that it can collect from each customer vehicle. Shashua says that uploading this data back to Mobileye costs about $1 per vehicle per year.

Safety

Several years ago Mobileye published RSS: Responsibility-Sensitive Safety. This is Mobileye’s approach to safety. Shashua views this framework as a key advantage for Mobileye. I confess I don’t understand how this approach compares to other efforts to validate AV safety.

I’m not sure how much to believe in the power of these advantages. But Mobileye is the world’s premier ADAS vendor and in the past I’ve found Mobileye a bit hard to learn about. So it’s a step forward to even get a sense of how they view their own advantages.

Argo’s 4th Generation Hardware

In Ground Truth, Argo’s autonomous vehicle publication, CTO Brett Browning provides an overview of their new hardware stack.

“Our new SDS [self-driving system] leverages customized components — not off-the-shelf stuff — including high-resolution cameras, lidar, radar, microphones, and inertial sensors, that meet rigorous industry safety standards.”

A few points struck me.

Microphones

Argo’s new setup includes three microphones, “to effectively listen for emergency responder vehicles.”

Waymo includes these sensors as well. I wonder how else Argo might be able to use audio, beyond first-responder detection.

Sensor Cleaning

“The new lidar base contains water jets for cleaning and fans for cooling, allowing the sensors to efficiently operate in extreme temperatures and for the optical windows to be automatically cleaned if they’re ever obstructed by rain or dirt.”

Making sure that all of the sensors is clean is one of those operational details that engineers could ignore a few years ago. But for production vehicles, this becomes critical. Argo must care about this even more than most companies, given its focus on operating in a wide variety of climates.

Redundancy

The post mentions several times that the new stack has computation redundancy.

“We have two independent computing systems that serve to maintain safe operations.”

The description is a bit vague on some important details. It’s unclear whether the secondary stack (labeled Complementary Autonomous Vehicle System — CAVS) is “fail-safe” or “fail-operational.” That is, if the primary system fails, can CAVS complete the vehicle’s route, or does it simply pull to the side safely and wait for assistance?

The post is also a bit unclear as to whether CAVS is a separate and redundant system, or whether it participates in the functionality of the primary system.

“… the computers use different detection algorithms so the backup computer has a unique perception ability which improves the robustness of response in an unexpected situation.”

Regardless of the nitty gritty details, it sounds like this system is a big step forward for Argo!

Microsoft Joins Cruise And Cruise Joins Microsoft

This week Microsoft and Cruise announced a $2 billion investment from the former into the latter. The focus of the partnership is squarely on cloud computing. Press releases from both companies specified Microsoft as the “preferred cloud provider” of both Cruise and General Motors.

“Microsoft, as Cruise’s preferred cloud provider…”
“As Cruise and GM’s preferred cloud, we will apply the power of Azure to help them scale…”
“GM will work with Microsoft as its preferred public cloud provider”

What does it mean to be a “preferred cloud provider?”

Preference vs. Exclusivity

For starters, it seems likely that “preferred” does not mean “exclusive.”

That’s notable because a number of recent Waymo partnerships (with auto manufacturers, not with cloud providers) have referred to Waymo as an “exclusive” partner. For example:

“Waymo is now the exclusive global L4 partner for Volvo Car Group…”

Credits vs. Cash

I also wonder whether this framing means that Microsoft didn’t invest actual cash in Cruise, or at least not the headline $2 billion.

I remember hearing rumors (or maybe it was official) after Honda’s investment in Cruise, that much of that investment came in the form of manufacturing credits at Honda plants, not cash dollars. Similarly, I wonder if any of the $2 billion take the form of Azure credits.

Valuation

This investment values Cruise at $30 billion, which is basically the same as Waymo’s recent valuation about a year ago. This is a testament to Cruise’s progress. The valuation might also indicate of how eager Microsoft is for Cruise to become a credible competitor to Waymo, and (more importantly) Alphabet.

Partnerships vs. Purchase Orders

One of my favorite quotes in the tech industry is, “Favorite partnership for me is a purchase order. Defined charter, beginning, end.”

Waymo seems to mostly adhere to this philosophy. Their “partnerships”, mostly with automotive manufacturers, seem to largely amount to vendor-customer relationships.

Cruise, as well as most other companies in the self-driving industry, tend toward more a wider range of partnerships. The Microsoft investment might fall in that category, depending on the structure. Of course, it may also be a straightforward cash-for-equity transaction.

In any case, $30 billion is pretty amazing. Go Cruise!

Donald Trump Pardons Anthony Levandowski

Donald Trump pardoned or commuted the sentences of 143 Americans on his last day as President, including Anthony Levandowski.

Levandowski is one of, and one of the youngest, foundational participants in the self-driving car industry. He was there from the start, at the original DARPA Grand Challenge, with an autonomous motorcycle called Ghost Rider.

The motorcycle fell over almost immediately, but Levandowski’s career in the industry was just beginning. He joined Sebastian Thrun at Google, working first on Google Maps and then later on the Google Self-Driving Car Project. Eventually he left to found his own start-up, Otto, which is where the trouble began.

Otto didn’t last long as a stand-alone company before it was acquired by Uber for hundreds of millions of dollars. Shortly thereafter, Google sued Uber, claiming that Levandowski had stolen tens of thousands of documents from the Google Self-Driving Car Project. Google believed that IP was illegally benefitting Otto, and which was now owned by Uber.

These events intersected lightly with my own history in the self-driving car ecosystem. I joined Udacity in the summer of 2016, working with Sebastian Thrun to build the Self-Driving Car Engineer Nanodegree Program. Sebastian quickly introduced me to Otto, whose engineers offered to help teach the program.

I only met Levandowski briefly, but when the lawsuit hit a few months later, it was surreal to find myself connected, however tangentially, to the drama.

The Google-Uber lawsuit ended with a massive settlement from Uber to Google, and led to Levandowski pleading guilty of downloading a project tracking spreadsheet from his job at Google. According to Wikipedia, “Levandowski admitted to accessing the document about one month after leaving Google.”

I never could figure out whether Levandowski was really guilty, and if he was, whether it even mattered. Co-mingling personal computers and phones with cloud emails and information presumably leads to enormous amounts of data downloaded on most corporate employees’ personal devices. Often, we don’t even know that this is happening — the emails and documents get downloaded in the background. When we do load and review something, it’s not always clear whether that information existed locally on personal device, or is stored in the cloud.

And if the worst thing Levandowski did was look at a project planning spreadsheet a month after he left a job, that seems negligible.

But it did cost Levandowski hundreds of millions of dollars, as well as jail time.

There are many more worthy recipients and potential recipients of mercy than a brilliant engineer who made and then lost a fortune, and is young enough and brilliant enough to make it all again. But neither do I begrudge Levandowski the pardon. Frankly, I’m glad he received it. I only wish that many more people, from all walks of life, would receive such forgiveness.

The official explanation of the pardon, such as it is, has already been wiped from the White House website, only hours into the next presidential administration. But there’s always the Way Back Machine, which records the justification for posterity.

“President Trump granted a full pardon to Anthony Levandowski. This pardon is strongly supported by James Ramsey, Peter Thiel, Miles Ehrlich, Amy Craig, Michael Ovitz, Palmer Luckey, Ryan Petersen, Ken Goldberg, Mike Jensen, Nate Schimmel, Trae Stephens, Blake Masters, and James Proud, among others. Mr. Levandowski is an American entrepreneur who led Google’s efforts to create self-driving technology. Mr. Levandowski pled guilty to a single criminal count arising from civil litigation. Notably, his sentencing judge called him a “brilliant, groundbreaking engineer that our country needs.” Mr. Levandowski has paid a significant price for his actions and plans to devote his talents to advance the public good.”

Autonomous Drifting

I just purchased new tires for my 2004 Toyota Highlander, which made me cringe a little bit at the rubber being chewed up in this video. Otherwise, it’s awesome 🙂

Chris Gerdes’s lab at Stanford has been working on autonomous donuts and drifting for a few years. Now they’ve partnered with Toyota Research Institute.

I imagine this work requires incredibly accurate state estimation and motion control. The former senses when when the vehicle has crossed boundaries between different states, such as “traction” and “side-slip.” These states are what an engineer or mathematician would call “non-linear.” That’s basically just a mathematical way of saying what most drivers intuitively know — the vehicle starts to handle much differently when it’s in a skid.

The motion controller must then be tuned for several different states, and respond appropriately as the vehicle transitions between states.

I might also imagine that a very finely tuned simulator, modeling the physical components of the vehicle, comes into play.

All of this is a ways away from the more common problems that self-driving cars face, like object tracking and detection.

But high-performance state estimation is necessary for both map-less driving and autonomous flight. Even though this is a car, I bet a lot of what they’re learning could translate to airborne vehicles.

The motion control advances here might eventually allow autonomous vehicles to safely and comfortably travel at higher speeds than humans have ever been able to handle.

And it’s also just cool to watch 😉