I remember less than four years ago meeting the whole Voyage team was at Udacity and had never worked on self-driving cars. Now they’re following the same path and building on the same platform as Waymo.
This team is truly an inspiration that if you work hard you won’t believe what you can accomplish.
And check out this video!
Got 2 minutes to kill? Nerd out with us by watching our self-driving car navigate:
Yesterday Tesla published a blog post, “Getting Back to Work,” that emphasized the company’s desire to resume automotive production at its Fremont, California, plant.
The post links to Tesla’s “Return to Work Playbook”, a polished 38-page report on Tesla’s plans for regulation and compliance.
The document is clearly intended for government relations and press purposes, not as a how-to manual for staffing the plant properly. Those documents exist, too, and this playbook shows some examples, but this playbook is for the public, not Tesla staff.
Nonetheless, the playbook serves as a good read for those of us wondering what the near-term future of work will look like, whether in manufacturing or retail or offices.
The main steps include:
The most notable changes to me are the use of PPE, temperature checks, and eliminating large meetings.
Cleaning
There’s quite a bit of emphasis on cleaning and disinfectant. Most of what I’ve read in the news about COVID indicates person-to-person transmissions, not so much person-to-surface-to-person. Cleaning is always welcome, of course, but I wonder how much of an effect rigorous cleaning will have.
Eating
“Break/lunch areas — occupancy reduced by removing some chairs and table, posting signage and installing barriers in some areas.”
Temperature Screening
The playbook references both thermal cameras and thermometers. It sounds like the thermal cameras are passive sensors that alert whenever somebody passes by with a temperature. That seems genius, and is also much more invasive than Americans are typically used to in the workplace.
Fun
“Ensure common areas are closed (e.g. game areas, gym, bean bags, etc.).”
Fun is about to become a lot less important at work.
Carpooling
“Verify signs promoting carpooling have been removed.”
Beyond Tesla, I wonder if state-regulated HOV lanes will also go away.
Handshakes
“Do not shake hands or engage in any unnecessary physical contact”
We’ve all already stopped shaking hands, but it’s interesting to see a company explicitly prohibit the practice. I assume this will come back eventually, but I wonder how long.
Testing
“By entering into our work location, you agree that: … YOU have NOT tested positive or have been tested and are awaiting the results for COVID-19 in the last 14 days [my emphasis]”
I wonder how rigorously this will be followed. This means if the tests ever become widespread enough that you could just take them on-demand, you might not want to take them, lest you get banished from work until you receive a result. Perhaps there is a legal liability issue that drives this.
Travel
“All international business-related travel is currently curtailed and requires VP’s approval…Please refrain from booking and staying at alternative or home sharing rentals.”
Further evidence that Airbnb has a tough road ahead.
Self-Care
“Hearing about the pandemic repeatedly can cause undue stress, so consider taking a break from it.”
Sound advice, I suppose, but let’s not cross the line into willful ignorance.
Conference Rooms
“Conference Rooms — occupancy is reduced to 1/3 capacity until further notice.”
That probably means that in order to have a 2-person meeting you’ll need a room originally designed for at least six people. That’s going to be hard to come by at most companies (I don’t know about Tesla specifically). I would guess this winds up being a major factor in causing white collar employees to continue to work from home. A meeting with a colleague sitting at the next desk might be easier to arrange via web chat from home, rather than scrounging a six-person conference room in the office and wearing masks.
The Detroit Free Press reports that new features are coming to GM Super Cruise. Specifically, Super Cruise will now perform lane changes “on demand”, and will negotiate highway interchanges.
More significantly, Super Cruise will roll out to the Cadillac Escalade and the Cadillac CT4 and CT5 sedans.
Super Cruise is often touted as the closest competitor to Tesla Autopilot, and maybe even a superior alternative.
Maddeningly, however, for years Super Cruise has only been available on top-of-the-line Cadillac CT6 models. As a result, it’s really hard to find a car that has it. I’ve never been in a car equipped with Super Cruise.
The Free Press also quotes GM that Super Cruise has been activated for 5.5 million miles since it launched in 2017. That means Waymo has driven more Level 4 miles since 2017 than Super Cruise has driven in something approximating Level 3. Tesla Autopilot, which has logged billions of miles, has orders of magnitude more data.
Nonetheless, Super Cruise has the potential to roll out to the larger GM model base and begin recording data at a level far beyond anything we’ve seen so far — from any manufacturer.
The company was founded by Stanford PhD and AI wunderkind Jesse Levinson, along with Tim Kentley-Klay, a brash Australian designer.
For years Zoox was highly secretive about its technology and goals, even by the standards of the tight-lipped self-driving car industry. The Zoox website was a single HTML page sporting only the company logo.
In 2018, Zoox pulled back the curtain with awesome highlighting the its autonomous capabilities and Mad Max design ethos.
Only a month later, Kentley-Klay was ousted just after Zoox raised $500 million in venture funding. Eventually Zoox recruited Intel Chief Strategy Aicha Evans as CEO.
All of which is to say, Zoox has been quite a story.
Apparently the company terminated contractors and laid off 10% of its workforce in light of the COVID pandemic. Now it seeks either further venture capital funding, or an acquirer.
The status of being publicly for-sale is reminiscent of Drive.AI, another prominent (although much smaller) self-driving start-up. That firm was widely reported to be shopping for a buyer during the first half of 2019. Apple ultimately acquired the startup in a firesale, days before bankruptcy.
All that said, a few qualifiers.
The stories I’ve read seem to make a lot of hay that Zoox has hired an investment bank, Qatalyst Partners. Investment banks usually do spell “sale”. But they can also spell “investment”. Particularly for a company of Zoox’s size, the sources of private investment change from traditional venture capital firms to larger institutional funds that work more regularly with investment banks. Zoox may simply be working on a really large funding round, which would go hand-in-hand with also beating the bushes for any potential acquirers.
Also, the stories I’ve read indicate Zoox plans to hire back laid off employees and contractors once coronavirus subsides. That’s easy to say now, of course, but it’s worth keeping in mind that perhaps what looks bad is merely a bump in the road.
Ford’s dataset includes multiple driving environments, multiple sensor modalities, multiple vehicles, and multiple seasons, on a consistent 66 km route in the Detroit area.
Combined with datasets from Waymo, nuTonomy, Lyft, UC-Berkeley, Daimler, and KIT, there is a wealth of self-driving car data available to play with 🙂
“Soon, your Volvo will be able to drive autonomously on highways when the car determines it is safe to do so,” said Henrik Green, chief technology officer for Volvo Cars.
That sounds a lot like Level 3 autonomy. The timing of this announcement is particularly interesting, coming right on the heels of Audi retreating from Level 3 — due to liability concerns.
The Volvo ADAS systems I’ve tested out in the past have always been quite impressive — in the top tier of systems behind Tesla Autopilot. If there is one way for competitors to catch and surpass Tesla, it might be exploiting the lidar technology that Tesla has famously eschewed.
This is outside my usual wheelhouse of autonomous vehicles and robotics, but I played a significant (albeit behind-the-scenes) role teaching the first course in the three-course program.
Tom Verbiscer and I worked together to design and build a sequence of lessons on high-availability, resilience, and redundancy. Tom teaches about:
Availability Zones
Regions
Server-based architecture
Serverless architecture
Global services
DynamoDB and global tables
S3 classes and features
Uptime
Downtime
Service-Level Agreements
Recovery Time Objectives
Recovery Point Objectives
Disaster Recovery
Monitoring with CloudWatch
Alerting with Simple Notification Service
Recovery
Chaos Engineering!
And that’s just the first course of the program!
I hope everyone that enrolls learns as much from taking the program as I did working with Tom to build the program.
Now, three years later, Audi has announced Level 3 won’t be coming to the A8. Surprisingly, Audi attributes this about-face to legislation, not technology.
“Currently, there is no legal framework for Level 3 automated driving and it is not possible to homologate such functions anywhere in the world in a series production car.”
Apparently the lawyers got nervous that customers might not service and maintain their cars, then the vehicles would crash, and Audi would be on the hook.
If the real issue here is liability, not technology, that suggests the first movers toward Level 3 vehicles will be companies with very little to lose. So maybe not Tesla, which is now the most valuable American automaker, by an order of magnitude.
Maybe Level 3 will come from some small startup that doesn’t have to worry about losing billions of dollars in market capitalization that it never had in the first place.
Last fall, a wide consortium of autonomous vehicle companies published, “Safety First for Automated Driving”, a whitepaper aimed as filling the standards gap between ISO 26262 (Functional Safety), ISO 21448 (Safety of the Intended Functionality — SOTIF), and the reality of where self-driving cars are heading.
According to EE Times, this whitepaper is on its way to becoming its own ISO standard. That would provide a clear and consensus view of how to approach safety for autonomous vehicles, which has been lacking in the industry up to this point.
At 157 pages, the whitepaper is thorough but digestible. Over the coming weeks I’ll try to break down the contents chapter-by-chapter, to see what the industry consensus is.
Andrei Karpathy is one of the most impressive and celebrated computer scientists in the world, and has worked for the past several years as Senior Director, AI, at Tesla. Essentially, he leads their Autopilot team.
Reilly Brennan’sFuture of Transportation newsletter (you should subscribe) pointed to a talk Karpathy recently gave at a conference called ScaledML. It’s pretty great, so I decided to annotate it, as a way to capture all of the details for myself, as much as anything else.
[00:00] Karpathy’s title is Senior Director. I remember him joining Tesla as a Director, so I think he got a promotion. Congratulations!
[00:19] Karpathy starts by defining what Autopilot is. This seems like good presentation technique. Establish the basics before moving on to advanced topics.
[00:50] Karpathy shows 8 Tesla vehicle models, noting that some of them have “only been announced.” Models S, 3, X, Y, T(ruck), A(TV — joking?), R, and S(emi). Globally Tesla has over 1 million vehicles.
[01:35] Autopilot has 3 billion miles, “which sounds(?) like a lot.”
[01:58] “We think of it (Autopilot) as roughly autonomy on the highway.” Sounds like Level 3 to me.
[02:24] “Smart Summon is quite magical, when it works (audience laughs).” I actually don’t know, is Smart Summon unreliable?
[03:12] Euro NCAP has rated Teslas as the safest vehicles, which isn’t a surprise but also puts the Autopilot lawsuits in perspective.
[03:45] Karpathy shows some examples of Tesla safety features working, even when Autopilot is not turned on. Probably this means that Karpathy’s team is working on the broader array of safety features, not just Autopilot.
[04:43] “The goal of the team is to produce full self-driving.” Karpathy has always struck me as more reliable and realistic than Musk. “Full Self-Driving” means more coming from Karpathy.
[06:30] “We do not build high-definition maps. When we come to an intersection, we encounter it basically for the first time.” This is striking, and I don’t think I’ve heard Tesla put it quite like this before. Tesla is famous for eschewing lidar, but I wonder why they don’t build vision-based maps?
[08:00] Karpathy mentions that the neural networks on the car really have two separate tasks — (a) driving, and (b) showing the humans in the vehicle that the computer perceives the environment, so the humans trust the system.
[09:16] We see a photo of a crossing guard with a handheld stop sign, hanging loose from the guard’s limp arm. Karpathy calls this “an inactive state.” This really highlights to me how hard it is for a computer to know whether a stop sign is real or not.
[10:10] Karpathy mentions Tesla builds maps, “of course, but they’re not high-definition maps.” I wonder what kind of maps they are.
[10:35] The Autopilot team spends at least part of its day-to-day work going through the long-tail and sourcing examples of weird stop signs. And presumably other weird scenarios. Man that sounds like a grind — I would imagine they must automate or outsource a lot of that.
[11:15] Bayesian uncertainty in the neural network seems to play a role.
[12:21] When Tesla needs more data, they just send an extra neural network to their vehicle fleet and ask the cars to run that network in the background, gathering potential training images. I would be it will take traditional automotive companies a long time to develop this capability.
[13:16] Test-Driven Development! TDD for the win!
[14:37] HydraNet is a collection of 48 neural networks with a “shared backbone” and 1000 distinct predictions. This is a multi-headed neural network on steroids.
[14:59] “None of these predictions can ever regress, and all of them must improve over time.” I don’t really understand what he means here. Surely there must be times a network predicts a dog and then later realizes it’s a child, etc.
[15:15] Autopilot is maintained by “a small, elite team — basically a few dozen people.” Wow.
[15:54] The goal of the Tesla AI team is to build infrastructure that other, more tactical people can then use to execute tasks. They call this approach Operation Vacation. (ruh-ruh)
[16:46] For example, if somebody at Tesla wants to detect a new type of stop sign, they supposedly don’t even have to bother Karpathy’s team. The AI team has already built out all the infrastructure for the rest of Tesla to plug new “landmark” images into.
[17:56] Karpathy shows an occupancy tracker that looks like something out of a 2-D laser scanner from twenty years ago. I wonder if they’re basically using cameras to fake what lidars do (Visual SLAM, etc.).
[19:36] Autopilot code used to be a lot of C++ code, written by engineers. As the neural networks get better, they’re eating up a lot of that “1.0” codebase.
[19:51] Aha! The occupancy tracker is old, “1.0” code, written by people. The future is neural networks!
[20:00] There is a “neural net fusion layer, that stitches up the feature maps and projects to birds-eye view.”
[20:15] There is a “temporal module” that smoothes and a “BEV net decoder”. What is are these things? I probably need to spend a few weeks getting back up to speed on the latest neural network research.
[22:15] Karpathy shows off how well this system works, but it’s hard to follow and judge for myself.
[22:35] Tesla takes a “pseudo-lidar approach, where you predict the depth of every since pixel and you basically simulate lidar input purely from vision.” Why not just use lidar, then? The unit price is coming down. Probably Tesla can’t depend on lidar because it already has a million vehicles on the road, none of which have lidar, and many of which have paid for full self-driving already. Realistically, though, this sounds like Tesla will start to add lidar at some point.
[24:02] The gap between lidar and a camera’s ability to simulate lidar is “quickly shrinking.” What’s the gap now? Is this tracked somewhere in academic literature?
[24:36] The driving policy (the motion planning), is still human-coded. But not for long! This is where Tesla’s fleet really shines. Karpathy notes that their human drivers are basically building supervised motion planning datasets for free.
[26:17] Really nice job summarizing his own talk. It’s just amazing that one guy can be such a phenomenal computer scientist and also so skilled at communication — in a second language, no less!
[27:30] During Q&A, Karpathy notes that Tesla builds low-definition semantic maps, which somewhat contradicts his earlier statement that every intersection is basically approached as if it were a new intersection.
[29:45] The hand-coded, “software 1.0” stack is used to keep the neural network within “guardrails.”
David Silver 