As I wrote in Forbes.com, the rollout resembles the process Waymo took to launch its driverless Waymo One service in Arizona, but AutoX is progressing much faster.
Whereas Waymo tested self-driving cars with human safety operators for a decade before advancing to driverless vehicles, AutoX was founded in only 2016 and just began testing fully driverless vehicles a few months ago.
This surprised me:
Also like Waymo, the base vehicle for the AutoX service is the Chrysler Pacifica minivan. The selection of an American automotive manufacturer for this initial program is notable because AutoX has partnerships with many Chinese manufacturers, including Dongfeng Motors, Shanghai Auto, BYD, and Chery Automobile.
Yesterday, I posted a brief overview of a couple of presentations Mobileye CEO Amnon Shashua gave at CES 2021 this month. I really enjoyed these presentations, in large part because over the years I’ve read less about Mobileye and know less about them than many other companies in the automotive technology ecosystem.
Today, I re-watched Shashua’s “deep dive” on Mobileye’s REM mapping approach. It’s quite informative, so I took notes.
REM is a Mobileye brand name that stands for Road Experience Management
The maps are generated from cameras. In the future, Mobileye’s lidar and radar will be designed to work with these camera-only maps, not the other way around.
In particular, even future lidar and radar systems will not use standard, point-cloud-based HD maps. Point clouds take up too much storage space to be practical, particularly for updating from a huge fleet of vehicles.
Instead of point clouds, REM uses “semantic” maps, that record sparse information, such as driveable paths, stop lines, and traffic signal locations.
Identifying this semantic segmentation and uploading it to the cloud takes 10 kb of data transfer per kilometer. This costs somebody (the manufacturer?) $1 per year, on average.
All of this begs a question, though — are maps even necessary?
In theory, maps aren’t necessary. After all, humans drive without maps (in many scenarios). Humans just figure out the road as we drive.
Artificial intelligence can do the same thing, but AI isn’t nearly as good as humans at this (yet). The Mean Time Between Failures (MTBF) for an AI will be low — lots of problems.
Solution: prepare a lot of this information in advance, and store it in the map.
Shashua says that everyone is using a map, even if they say they’re not. Pretty clear that this is a reference to Tesla.
Mobileye’s maps have three performance goals: Scale (consumer vs. robotaxi), Up-To-Dateness (real-time), Accuracy (cm-level)
Mobileye has a division which builds lidar-based HD Maps, so they know the pros and cons of this approach
Lidar-based HD maps are too detailed. The AI driver only need information for a 200m radius around the vehicle, but HD maps contain very detailed information about the entire world.
On the flip side, point clouds are just coordinates in space. AI needs semantic meaning: drivable paths, priority, crosswalks, stopping & yield lines.
Calculating this in real-time is theoretically possible, but practically impossible: too many conflicting signs and signals, too much noise, too much going on
Mobileye is now creating AV Map, which are not HD Maps: Scalability everywhere, Accuracy in 200m radius, Semantic features generated from wisdom of crowd
At this month’s virtual 2021 Consumer Electronics Show, Mobileye presented a lot. CEO Amnon Shashua sat for a friendly interview with Ed Niedermeyer, and Shashua also gave a standalone hour-long presentation about Mobileye’s technology.
Shashua highlighted three areas of differentiation that provide competitive advantages for Mobileye:
Redundancy
Mapping
Safety
Redundancy
The plan seems to be that Mobileye will build a camera-only driver assistance system, and then layer radar and lidar on top to get to Level 4 autonomy by 2025.
Mapping
Mobileye has worked to identify minimal amounts of high-valuable semantic mapping data that it can collect from each customer vehicle. Shashua says that uploading this data back to Mobileye costs about $1 per vehicle per year.
Safety
Several years ago Mobileye published RSS: Responsibility-Sensitive Safety. This is Mobileye’s approach to safety. Shashua views this framework as a key advantage for Mobileye. I confess I don’t understand how this approach compares to other efforts to validate AV safety.
I’m not sure how much to believe in the power of these advantages. But Mobileye is the world’s premier ADAS vendor and in the past I’ve found Mobileye a bit hard to learn about. So it’s a step forward to even get a sense of how they view their own advantages.
“Our new SDS [self-driving system] leverages customized components — not off-the-shelf stuff — including high-resolution cameras, lidar, radar, microphones, and inertial sensors, that meet rigorous industry safety standards.”
A few points struck me.
Microphones
Argo’s new setup includes three microphones, “to effectively listen for emergency responder vehicles.”
Waymo includes these sensors as well. I wonder how else Argo might be able to use audio, beyond first-responder detection.
Sensor Cleaning
“The new lidar base contains water jets for cleaning and fans for cooling, allowing the sensors to efficiently operate in extreme temperatures and for the optical windows to be automatically cleaned if they’re ever obstructed by rain or dirt.”
Making sure that all of the sensors is clean is one of those operational details that engineers could ignore a few years ago. But for production vehicles, this becomes critical. Argo must care about this even more than most companies, given its focus on operating in a wide variety of climates.
Redundancy
The post mentions several times that the new stack has computation redundancy.
“We have two independent computing systems that serve to maintain safe operations.”
The description is a bit vague on some important details. It’s unclear whether the secondary stack (labeled Complementary Autonomous Vehicle System — CAVS) is “fail-safe” or “fail-operational.” That is, if the primary system fails, can CAVS complete the vehicle’s route, or does it simply pull to the side safely and wait for assistance?
The post is also a bit unclear as to whether CAVS is a separate and redundant system, or whether it participates in the functionality of the primary system.
“… the computers use different detection algorithms so the backup computer has a unique perception ability which improves the robustness of response in an unexpected situation.”
Regardless of the nitty gritty details, it sounds like this system is a big step forward for Argo!
This week Microsoft and Cruise announced a $2 billion investment from the former into the latter. The focus of the partnership is squarely on cloud computing. Press releases from both companies specified Microsoft as the “preferred cloud provider” of both Cruise and General Motors.
“Microsoft, as Cruise’s preferred cloud provider…” “As Cruise and GM’s preferred cloud, we will apply the power of Azure to help them scale…” “GM will work with Microsoft as its preferred public cloud provider”
What does it mean to be a “preferred cloud provider?”
Preference vs. Exclusivity
For starters, it seems likely that “preferred” does not mean “exclusive.”
That’s notable because a number of recent Waymo partnerships (with auto manufacturers, not with cloud providers) have referred to Waymo as an “exclusive” partner. For example:
“Waymo is now the exclusive global L4 partner for Volvo Car Group…”
Credits vs. Cash
I also wonder whether this framing means that Microsoft didn’t invest actual cash in Cruise, or at least not the headline $2 billion.
I remember hearing rumors (or maybe it was official) after Honda’s investment in Cruise, that much of that investment came in the form of manufacturing credits at Honda plants, not cash dollars. Similarly, I wonder if any of the $2 billion take the form of Azure credits.
Valuation
This investment values Cruise at $30 billion, which is basically the same as Waymo’s recent valuation about a year ago. This is a testament to Cruise’s progress. The valuation might also indicate of how eager Microsoft is for Cruise to become a credible competitor to Waymo, and (more importantly) Alphabet.
Waymo seems to mostly adhere to this philosophy. Their “partnerships”, mostly with automotive manufacturers, seem to largely amount to vendor-customer relationships.
Cruise, as well as most other companies in the self-driving industry, tend toward more a wider range of partnerships. The Microsoft investment might fall in that category, depending on the structure. Of course, it may also be a straightforward cash-for-equity transaction.
In any case, $30 billion is pretty amazing. Go Cruise!
Alphabet is shutting down Loon. You could be forgiven for not knowing Loon was even a thing that existed for Alphabet to shut down. Forgive the incredibly uncreative pun, but Loon never really got off the ground.
“When we unveiled Loon in June 2013, we meant everything in its name. It was a way-out-there and risky venture. Not just fragile-balloons-on-the-edge-of-space risky, but risky at the core of the question it was asking. Could this be the radical idea that might finally bring abundant, affordable Internet access, not just to the next billion, but to the last billion? To the last unconnected communities and those least able to pay?”
I have very little knowledge of Loon and point you instead to Teller’s blog post and the Wikipedia page on Loon.
What interests me is this very public act of shutting down a startup. The folks who worked on Loon were probably very, very successful and smart, otherwise they wouldn’t have made it into X, Alphabet’s Moonshot Factory, in the first place.
You’ll be on y our way up! You’ll be seeing great sights! You’ll join the high fliers who soar to high heights.
You won’t lag behind, because you’ll have the speed. You’ll pass the whole gang and you’ll soon take the lead. Wherever you fly, you’ll be best of the best. Wherever you go, you will top all the rest.
Except when you don’t. Because, sometimes, you won’t.
I’m sorry to say so but, sadly, it’s true that Bang-ups and Hang-ups can happen to you.
You can get all hung up in a prickle-ly perch. And your gang will fly on. You’ll be left in a Lurch.
The good news, and it is good news, is that shutting down Loon liberates everyone to work on something else.
When things aren’t going well, inertia can keep us tethered (again with the puns) to fruitless endeavors. Conceding failure is almost physically painful.
But I’ve found that once I make the decision to move on, all sorts of opportunities open up.
I expect that’s what will happen to the folks at Loon, and we’ll all be better off for it. Them most especially.
Levandowski is one of, and one of the youngest, foundational participants in the self-driving car industry. He was there from the start, at the original DARPA Grand Challenge, with an autonomous motorcycle called Ghost Rider.
The motorcycle fell over almost immediately, but Levandowski’s career in the industry was just beginning. He joined Sebastian Thrun at Google, working first on Google Maps and then later on the Google Self-Driving Car Project. Eventually he left to found his own start-up, Otto, which is where the trouble began.
Otto didn’t last long as a stand-alone company before it was acquired by Uber for hundreds of millions of dollars. Shortly thereafter, Google sued Uber, claiming that Levandowski had stolen tens of thousands of documents from the Google Self-Driving Car Project. Google believed that IP was illegally benefitting Otto, and which was now owned by Uber.
These events intersected lightly with my own history in the self-driving car ecosystem. I joined Udacity in the summer of 2016, working with Sebastian Thrun to build the Self-Driving Car Engineer Nanodegree Program. Sebastian quickly introduced me to Otto, whose engineers offered to help teach the program.
I only met Levandowski briefly, but when the lawsuit hit a few months later, it was surreal to find myself connected, however tangentially, to the drama.
The Google-Uber lawsuit ended with a massive settlement from Uber to Google, and led to Levandowski pleading guilty of downloading a project tracking spreadsheet from his job at Google. According to Wikipedia, “Levandowski admitted to accessing the document about one month after leaving Google.”
I never could figure out whether Levandowski was really guilty, and if he was, whether it even mattered. Co-mingling personal computers and phones with cloud emails and information presumably leads to enormous amounts of data downloaded on most corporate employees’ personal devices. Often, we don’t even know that this is happening — the emails and documents get downloaded in the background. When we do load and review something, it’s not always clear whether that information existed locally on personal device, or is stored in the cloud.
And if the worst thing Levandowski did was look at a project planning spreadsheet a month after he left a job, that seems negligible.
But it did cost Levandowski hundreds of millions of dollars, as well as jail time.
There are many more worthy recipients and potential recipients of mercy than a brilliant engineer who made and then lost a fortune, and is young enough and brilliant enough to make it all again. But neither do I begrudge Levandowski the pardon. Frankly, I’m glad he received it. I only wish that many more people, from all walks of life, would receive such forgiveness.
The official explanation of the pardon, such as it is, has already been wiped from the White House website, only hours into the next presidential administration. But there’s always the Way Back Machine, which records the justification for posterity.
“President Trump granted a full pardon to Anthony Levandowski. This pardon is strongly supported by James Ramsey, Peter Thiel, Miles Ehrlich, Amy Craig, Michael Ovitz, Palmer Luckey, Ryan Petersen, Ken Goldberg, Mike Jensen, Nate Schimmel, Trae Stephens, Blake Masters, and James Proud, among others. Mr. Levandowski is an American entrepreneur who led Google’s efforts to create self-driving technology. Mr. Levandowski pled guilty to a single criminal count arising from civil litigation. Notably, his sentencing judge called him a “brilliant, groundbreaking engineer that our country needs.” Mr. Levandowski has paid a significant price for his actions and plans to devote his talents to advance the public good.”
I just purchased new tires for my 2004 Toyota Highlander, which made me cringe a little bit at the rubber being chewed up in this video. Otherwise, it’s awesome 🙂
Chris Gerdes’s lab at Stanford has been working on autonomous donuts and drifting for a few years. Now they’ve partnered with Toyota Research Institute.
I imagine this work requires incredibly accurate state estimation and motion control. The former senses when when the vehicle has crossed boundaries between different states, such as “traction” and “side-slip.” These states are what an engineer or mathematician would call “non-linear.” That’s basically just a mathematical way of saying what most drivers intuitively know — the vehicle starts to handle much differently when it’s in a skid.
The motion controller must then be tuned for several different states, and respond appropriately as the vehicle transitions between states.
I might also imagine that a very finely tuned simulator, modeling the physical components of the vehicle, comes into play.
All of this is a ways away from the more common problems that self-driving cars face, like object tracking and detection.
But high-performance state estimation is necessary for both map-less driving and autonomous flight. Even though this is a car, I bet a lot of what they’re learning could translate to airborne vehicles.
The motion control advances here might eventually allow autonomous vehicles to safely and comfortably travel at higher speeds than humans have ever been able to handle.
(Truly, we were sitting down. I know some people do the standing desk thing, and I probably should do that, too. Voyage even supplies those standing desks during Covid, if I could haul myself down to the office to pick one up. But for now, I sit. So does Jason.)
David: What is an Engineering Manager for the Perception Team?
Jason: We’re looking for someone who can lead our Perception Team. It’s a small but mighty team of about half a dozen engineers who work with the sensors on Voyage’s robotaxis in order to perceive the environment.
Perception!
What is perception?
Specifically, Voyage’s Perception Team handles three main tasks:
Detection — finding objects and agents in the environment, primarily using deep learning for computer vision and sensor fusion, but also other robust techniques
Tracking — figuring out if a car we detected one second ago is the same car we’re detecting right now
Localization — calculating our vehicle’s position with respect to the environment and our high-definition maps
The ideal candidate for this position would have really deep knowledge of at least one of these tasks, and familiarity with the others.
What’s exciting about this role? How will it help a candidate grow?
Voyage is at the cutting-edge of both computer vision and sensor fusion. We’ve implemented an Active Learning approach that automatically curates and selects the most valuable sensor data from our massive data set. For example, over time our Active Learning system might discover that we need to concentrate more on golf cart samples, relative to pedestrian samples. Training on a smaller subset of the most valuable data dramatically accelerates our development process and improves performance.
Voyage’s Active Learning system selects small data subsets for optimal performance.
One of this team’s most important deliverables is camera-based Depth Perception. We train deep learning models on both camera images and lidar point clouds, so that we can ultimately use camera images alone to infer depth — the distance to objects in a two-dimensional image. This is incredibly important for redundancy, safety, and performance. And it’s easy to generate a proof-of-concept, but what we require is fully robust performance under all manner of conditions.
Voyage trains an ensemble of different deep learning models on lidar point clouds.
We’re also working with an ensemble of multiple deep learning models for point cloud detection, tracking, and sensor fusion. Very few leaders have access to the engineering team, volume of data, and real-world validation opportunities to push the cutting edge in this domain. The Engineering Manager of the Perception Team at Voyage will have all of those tools!
Is this role more about people management or technical leadership?
More technical leadership. The engineers on the Perception Team will report to this manager, so there’s an important people management aspect. But our engineers are fairly senior and strong, so what we really need is a technical expert who can serve as a sounding board and leader for architectural design decisions.
Could this be somebody’s first managerial role?
Ideally, we’re looking for someone who has managed a team before, but it could be somebody’s first managerial role— we’re open to that. In that case, what we’d really want to see is strong technical project leadership experience. A Technical Lead who hasn’t officially managed people, but has shipped large-scale computer vision and sensor fusion projects to completion, could potentially be a good candidate for this role.
The real world is crazy! Do you see those turkeys? Voyage’s perception stack does!
Would Voyage hire somebody from outside the robotaxi industry?
We’d consider it. Computer vision and sensor fusion experience is critical for this role, so we’d want to see that. And we find that there are a lot of specific nuances to building safety-critical Level 4 autonomous driving systems. That’s an advantage for engineers who already work in AV. But this position could be a good fit for somebody from a related industry, like Drones, Computer Vision, or other areas of Robotics.
Lidar is a key component of Voyage’s perception stack.
Could the candidate be remote? Work from Hawaii?
Ideally, having someone based in the Bay Area would be preferred given the nature of the role, but we have flexibility to hire a really strong candidate remotely. Voyage supports remote work!
Jason Wong, VP of Talent at Voyage
Alright, let’s get down to brass tacks. What’s the interview process? Is it hard?
Ha. We have an amazing Perception Team at Voyage and we’re looking for an amazing leader for that team. We don’t set out to make the interview process “hard”, per se, but by the end of the process, we want to be confident that a candidate is phenomenal!
The first stage is a technical domain interview with a member of the Perception Team. In this stage, we’ll gauge a candidate’s expertise and skill in the perception domain. And the candidate can start to gauge us and the work we’re doing!
The second stage is an interview with our VP of Engineering, Davide Bacchet. This position reports to Davide, so we want to make sure there’s a strong relationship. Even more importantly, Davide will want to discuss the candidate’s vision of leadership and team growth.
The final stage is a set of panel interviews with the engineers on the team. These interviews will focus on domain expertise and managerial philosophy. It’s really important to us that every engineer on the team get to meet the candidate before we extend an offer. And we want the incoming manager to feel good about all the team members, too. The strength of this team is one of the selling points of the role.
Is there a coding interview? Does the candidate need to be able to reverse a linked list in five minutes or less?
LOL. No, there’s no coding interview for this position. The type of candidate we’re looking for does code and can reverse a linked list, but that’s not part of the selection process. We’re much more focused on deep domain expertise, thorough system design, and technical leadership.
David Silver 