For my latest Forbes article, I pondered Tesla’s use of maps, and how they communicate the need for maps to regulators and to the public.
Tesla’s use of pre-defined maps for its ADAS features has long be ambiguous and tied up in Elon Musk’s disdain for lidar, which is the principal tool for building high-definition maps.
Nonetheless, recently released communication between Tesla and the California Department of Motor Vehicles reveals that mapping is a central component of several Full Self-Driving features.
Yesterday I wrote that 2020 traffic deaths showed a big increase over previous years, probably due to distracted driver (cell phones). At the end of the post, I wondered whether advanced driver assistance systems (ADAS) are helping or hurting this situation.
I did some very light Internet searching today, and the most information I could find is this NHTSA report from 2016, which states:
“The data show that the Tesla vehicles crash rate dropped by almost 40 percent after Autosteer installation.”
In addition to this, I found some sporadic reports of Autopilot-related insurance discounts from Tesla owners and small insurance companies.
This isn’t a lot to go on — the data are dated (I forgot that “Autosteer” was a feature name), and mentioned off-hand in a NHTSA report that is really about AEB deployment. More importantly, there’s no data for ADAS options other than Tesla. Tesla is a small and controversial portion of the ADAS market.
I’ve always thought about chip manufacturing as a kind of boring and commodity endeavor, but with all the analysis of Intel vs. Samsung vs. TSMC vs. NVIDIA vs. Hauwei, I should probably start reading more about extreme ultraviolet lithography.
In 2015, Tesla released a video demonstrating a prototype “snake charger” that (presumably autonomously) connects and charges a car. It’s amazing and honestly not as creepy as I might have imagined.
I missed this prototype at the time and apparently not much has happened with it in the interim, but recently Elon Musk confirmed on Twitter that the charger is still in the works.
This charger would make a lot of sense were Teslas able to drive autonomously, as Musk hints they will be able to do soon (note that a lot of people are skeptical on that point). Autonomous charging would really untether the vehicle from the need for human intervention.
At this point neither the snake charger nor Full Self-Driving mode are ready. But it’s a pretty awesome video.
I just saw that a few weeks ago Elon Musk tweeted that Tesla’s Full Self-Driving Functionality will be a “full architectural rewrite”, presumably of the code base.
The FSD improvement will come as a quantum leap, because it’s a fundamental architectural rewrite, not an incremental tweak. I drive the bleeding edge alpha build in my car personally. Almost at zero interventions between home & work. Limited public release in 6 to 10 weeks.
Rewrites are hard. They often seem exciting at the beginning, because of the opportunity to do everything the “right” way, from the ground up. But it usually turns out that recreating existing functionality, built up over years, requires a lot of effort. If the original software took years to write, the rewrite won’t be trivial.
On the other hand, there is a reason Tesla is the most valuable car company in the world, and now one of the most valuable companies in the world, period. Elon Musk is amazing at leading his people to accomplish what would be impossible for most teams.
A rewrite hardly qualifies as impossible. But it’s usually a pretty big effort.
Elon Musk famously tweeted that Tesla vehicles will be appreciating assets, a first for automobiles, if that comes to pass. The logic stems from another controversial Musk claim, that Teslas will eventually become robotaxis, generating passive income for their owners.
Recently, Electrek and other outlets wrote Tesla has taken a baby step toward the robotaxi vision. Nothing self-driving, much more pedestrian (excuse the pun) than that.
Tesla has created an “Add Drive” feature in its app.
Tesla does not yet appear to be advertising this feature, and I don’t own a Tesla, so I can’t confirm for myself. But apparently Tesla owners can now give access to their car to anybody, just by adding an email address. No key necessary, just the Tesla app and a confirmed email address.
Even if the robotaxis are a long time coming, you could imagine this might make it a lot easier for Tesla owners to rent their vehicles to other drivers through sites like Turo.
Paul Leinert reports an amazing read in Reuters. Tesla is planning to launch breakthrough electric batteries that will make the cost of electric powertrains as economical or even better than gasoline.
As if that’s not enough, Tesla is going to build “Terafactories” in China to produce these batteries. Terafactories will be at least an order of magnitude bigger than Tesla’s Nevada “gigafactory”.
There’s also some battery chemistry in the piece, as well as an academic laboratory from Nova Scotia, of all places.
It’s almost too out there to believe, but it seems just plausible and Paul is one of the automotive industries best reporters on future technology.
Yesterday Tesla published a blog post, “Getting Back to Work,” that emphasized the company’s desire to resume automotive production at its Fremont, California, plant.
The post links to Tesla’s “Return to Work Playbook”, a polished 38-page report on Tesla’s plans for regulation and compliance.
The document is clearly intended for government relations and press purposes, not as a how-to manual for staffing the plant properly. Those documents exist, too, and this playbook shows some examples, but this playbook is for the public, not Tesla staff.
Nonetheless, the playbook serves as a good read for those of us wondering what the near-term future of work will look like, whether in manufacturing or retail or offices.
The main steps include:
The most notable changes to me are the use of PPE, temperature checks, and eliminating large meetings.
Cleaning
There’s quite a bit of emphasis on cleaning and disinfectant. Most of what I’ve read in the news about COVID indicates person-to-person transmissions, not so much person-to-surface-to-person. Cleaning is always welcome, of course, but I wonder how much of an effect rigorous cleaning will have.
Eating
“Break/lunch areas — occupancy reduced by removing some chairs and table, posting signage and installing barriers in some areas.”
Temperature Screening
The playbook references both thermal cameras and thermometers. It sounds like the thermal cameras are passive sensors that alert whenever somebody passes by with a temperature. That seems genius, and is also much more invasive than Americans are typically used to in the workplace.
Fun
“Ensure common areas are closed (e.g. game areas, gym, bean bags, etc.).”
Fun is about to become a lot less important at work.
Carpooling
“Verify signs promoting carpooling have been removed.”
Beyond Tesla, I wonder if state-regulated HOV lanes will also go away.
Handshakes
“Do not shake hands or engage in any unnecessary physical contact”
We’ve all already stopped shaking hands, but it’s interesting to see a company explicitly prohibit the practice. I assume this will come back eventually, but I wonder how long.
Testing
“By entering into our work location, you agree that: … YOU have NOT tested positive or have been tested and are awaiting the results for COVID-19 in the last 14 days [my emphasis]”
I wonder how rigorously this will be followed. This means if the tests ever become widespread enough that you could just take them on-demand, you might not want to take them, lest you get banished from work until you receive a result. Perhaps there is a legal liability issue that drives this.
Travel
“All international business-related travel is currently curtailed and requires VP’s approval…Please refrain from booking and staying at alternative or home sharing rentals.”
Further evidence that Airbnb has a tough road ahead.
Self-Care
“Hearing about the pandemic repeatedly can cause undue stress, so consider taking a break from it.”
Sound advice, I suppose, but let’s not cross the line into willful ignorance.
Conference Rooms
“Conference Rooms — occupancy is reduced to 1/3 capacity until further notice.”
That probably means that in order to have a 2-person meeting you’ll need a room originally designed for at least six people. That’s going to be hard to come by at most companies (I don’t know about Tesla specifically). I would guess this winds up being a major factor in causing white collar employees to continue to work from home. A meeting with a colleague sitting at the next desk might be easier to arrange via web chat from home, rather than scrounging a six-person conference room in the office and wearing masks.
Andrei Karpathy is one of the most impressive and celebrated computer scientists in the world, and has worked for the past several years as Senior Director, AI, at Tesla. Essentially, he leads their Autopilot team.
Reilly Brennan’sFuture of Transportation newsletter (you should subscribe) pointed to a talk Karpathy recently gave at a conference called ScaledML. It’s pretty great, so I decided to annotate it, as a way to capture all of the details for myself, as much as anything else.
[00:00] Karpathy’s title is Senior Director. I remember him joining Tesla as a Director, so I think he got a promotion. Congratulations!
[00:19] Karpathy starts by defining what Autopilot is. This seems like good presentation technique. Establish the basics before moving on to advanced topics.
[00:50] Karpathy shows 8 Tesla vehicle models, noting that some of them have “only been announced.” Models S, 3, X, Y, T(ruck), A(TV — joking?), R, and S(emi). Globally Tesla has over 1 million vehicles.
[01:35] Autopilot has 3 billion miles, “which sounds(?) like a lot.”
[01:58] “We think of it (Autopilot) as roughly autonomy on the highway.” Sounds like Level 3 to me.
[02:24] “Smart Summon is quite magical, when it works (audience laughs).” I actually don’t know, is Smart Summon unreliable?
[03:12] Euro NCAP has rated Teslas as the safest vehicles, which isn’t a surprise but also puts the Autopilot lawsuits in perspective.
[03:45] Karpathy shows some examples of Tesla safety features working, even when Autopilot is not turned on. Probably this means that Karpathy’s team is working on the broader array of safety features, not just Autopilot.
[04:43] “The goal of the team is to produce full self-driving.” Karpathy has always struck me as more reliable and realistic than Musk. “Full Self-Driving” means more coming from Karpathy.
[06:30] “We do not build high-definition maps. When we come to an intersection, we encounter it basically for the first time.” This is striking, and I don’t think I’ve heard Tesla put it quite like this before. Tesla is famous for eschewing lidar, but I wonder why they don’t build vision-based maps?
[08:00] Karpathy mentions that the neural networks on the car really have two separate tasks — (a) driving, and (b) showing the humans in the vehicle that the computer perceives the environment, so the humans trust the system.
[09:16] We see a photo of a crossing guard with a handheld stop sign, hanging loose from the guard’s limp arm. Karpathy calls this “an inactive state.” This really highlights to me how hard it is for a computer to know whether a stop sign is real or not.
[10:10] Karpathy mentions Tesla builds maps, “of course, but they’re not high-definition maps.” I wonder what kind of maps they are.
[10:35] The Autopilot team spends at least part of its day-to-day work going through the long-tail and sourcing examples of weird stop signs. And presumably other weird scenarios. Man that sounds like a grind — I would imagine they must automate or outsource a lot of that.
[11:15] Bayesian uncertainty in the neural network seems to play a role.
[12:21] When Tesla needs more data, they just send an extra neural network to their vehicle fleet and ask the cars to run that network in the background, gathering potential training images. I would be it will take traditional automotive companies a long time to develop this capability.
[13:16] Test-Driven Development! TDD for the win!
[14:37] HydraNet is a collection of 48 neural networks with a “shared backbone” and 1000 distinct predictions. This is a multi-headed neural network on steroids.
[14:59] “None of these predictions can ever regress, and all of them must improve over time.” I don’t really understand what he means here. Surely there must be times a network predicts a dog and then later realizes it’s a child, etc.
[15:15] Autopilot is maintained by “a small, elite team — basically a few dozen people.” Wow.
[15:54] The goal of the Tesla AI team is to build infrastructure that other, more tactical people can then use to execute tasks. They call this approach Operation Vacation. (ruh-ruh)
[16:46] For example, if somebody at Tesla wants to detect a new type of stop sign, they supposedly don’t even have to bother Karpathy’s team. The AI team has already built out all the infrastructure for the rest of Tesla to plug new “landmark” images into.
[17:56] Karpathy shows an occupancy tracker that looks like something out of a 2-D laser scanner from twenty years ago. I wonder if they’re basically using cameras to fake what lidars do (Visual SLAM, etc.).
[19:36] Autopilot code used to be a lot of C++ code, written by engineers. As the neural networks get better, they’re eating up a lot of that “1.0” codebase.
[19:51] Aha! The occupancy tracker is old, “1.0” code, written by people. The future is neural networks!
[20:00] There is a “neural net fusion layer, that stitches up the feature maps and projects to birds-eye view.”
[20:15] There is a “temporal module” that smoothes and a “BEV net decoder”. What is are these things? I probably need to spend a few weeks getting back up to speed on the latest neural network research.
[22:15] Karpathy shows off how well this system works, but it’s hard to follow and judge for myself.
[22:35] Tesla takes a “pseudo-lidar approach, where you predict the depth of every since pixel and you basically simulate lidar input purely from vision.” Why not just use lidar, then? The unit price is coming down. Probably Tesla can’t depend on lidar because it already has a million vehicles on the road, none of which have lidar, and many of which have paid for full self-driving already. Realistically, though, this sounds like Tesla will start to add lidar at some point.
[24:02] The gap between lidar and a camera’s ability to simulate lidar is “quickly shrinking.” What’s the gap now? Is this tracked somewhere in academic literature?
[24:36] The driving policy (the motion planning), is still human-coded. But not for long! This is where Tesla’s fleet really shines. Karpathy notes that their human drivers are basically building supervised motion planning datasets for free.
[26:17] Really nice job summarizing his own talk. It’s just amazing that one guy can be such a phenomenal computer scientist and also so skilled at communication — in a second language, no less!
[27:30] During Q&A, Karpathy notes that Tesla builds low-definition semantic maps, which somewhat contradicts his earlier statement that every intersection is basically approached as if it were a new intersection.
[29:45] The hand-coded, “software 1.0” stack is used to keep the neural network within “guardrails.”
Tesla’s Q1 2020 earnings call was Wednesday. By all accounts, the company crushed it. They turned a $16 million profit, which Car and Driver marks as the first time the company has ever turned a profit in Q1.
The Tesla roller coaster ride has been and up and down for years. The nadir was perhaps when short-sellers baited Elon Musk into tweeting that he would take the company private. That tweet violated all sorts of SEC guidelines and was a bit of a PR disaster. Around the same time, the company periodically came within months or even weeks of bankruptcy.
Flash forward a few years and today Tesla is back on top as the America’s most valuable (and most profitable) care company.
Keep in mind, of course, that by just about any other metric — revenue, units, employees — GM and Ford are an orders of magnitude bigger than Tesla.
But Wall Street seems to think Tesla’s small profit in the present is a prelude to much bigger profits in the future.
David Silver 