This strikes me as so surprising that I feel like I have to preface it by stating that I’m pretty sure it’s not an April Fool’s joke.

Tencent, the Chinese Internet giant, has a division called the Keen Security Lab, which focuses on “cutting-edge security research.” Their most recent project has been to hack Tesla vehicles, which they demonstrate in this video:

So far, so good. Tencent Keen Labs even published a 40-page write-up.

The hacks have made some press for demonstrating the potential for adversarial attacks —basically, tricking a neural network. Tencent researchers ultimately were able to place a few stickers in an intersection and trick the car into switching lanes into (potentially) oncoming traffic.

I am skeptical of adversarial attacks, at least involving self-driving cars. But that strikes me as ignoring the most interesting part of this.

In order to get this far, the researchers had to hack Tesla Autopilot, and in so doing, they appear to have discovered and published a surprising amount about how Autopilot works.

Want to know the architecture of Tesla computer vision neural network? It’s published on page 29 of the paper:

The paper states that, “for many major tasks, Tesla uses a single large neural network with many outputs, and lane detection is one of those tasks.” It seems like if you spent a little while investigating what was going on in that network, you might be able to figure out a lot about how Autopilot works.

The paper is forty pages long, and the English is good but not perfect, so it takes a little while to read. I confess I’ll need to spend more time with it to really understand the ins and outs.

But there are some more good nuggets:

“Both APE and APE-B are Tegra chips, same as Nvidia’s PX2. LB (lizard brain), is an Infineon Aurix chip. Besides, there is a Parker GPU (GP106) from Nvidia connected to APE. Software image running on APE and APE-B are basically the same, while LB has its own firmware.”

“ (By the way, we noticed a camera called “selfie” here, but this camera does not exist on the Tesla Model S.)” [DS: Driver monitoring system? On what model? Supposedly they are using a Model S 75 for all of this research.]

“Those post processors are responsible for several jobs including tracking cars, objects and lanes, making maps of surrounding environments, and determining rainfall amount. To our surprise, most of those jobs are finished within only one perception neural network.”

“Tesla uses a large class for managing those functions(about “large”: the struct itself is nearly 900MB in v17.26.76, and over 400MB in v2018.6.1, not including chunks it allocates on the heap). Parsing each member out is not an easy job, especially for a stripped binary, filled with large class and Boost types. Therefore in this article, we won’t introduce a detailed member list of each class, and we also do not promise that our reverse engineering result here is representing the original design of Tesla.”

“Finally, we figured out an effective solution: dynamically inject malicious code into cantx service and hook the “DasSteeringControlMessageEmitter::finalize_message()” function of the cantx service to reuse the DSCM’s timestamp and counter to manipulate the DSCM with any value of steering angle.”

“rather than using a simple, single sensor to detect rain or moisture, Tesla decided to use its second-generation Autopilot suite of cameras and artificial intelligence network to determine whether & when the wipers should be turned on.”

“We found that in order to optimize the efficiency of the neural network, Tesla converts the 32-bit floating point operations to the 8-bit integer calculations, and a part of the layers are private implementation [DS: emphasis mine], which were all compiled in the “.cubin” file. Therefore the entire neural network is regarded as a black box to us.”

“The controller itself is kind of complex. It will receive tracking info, locate the car’s position in its own HD-Map, and provide control instructions according to surrounding situations. Most of the code in controller is not related to computer vision and only strategy-based choices.”

If this is all true, then the team reverse-engineered Tesla’s entire software stack on the way to implementing an adversarial neural network attack. The reverse engineering strikes me as the amazing part.

On Friday, the Tesla blog announced the introduction of the Navigate feature to its Enhanced Autopilot system. Navigate will drive from exit-to-exit on the highway, and automatically change lanes to pass vehicles along the way.

Near the top of the post, Tesla writes, “until truly driverless cars are validated and approved by regulators, drivers are responsible for and must remain in control of their car at all times.”

That is a prominent disclaimer, but this feature basically looks like Level 3 partial autonomy. Depending on how aggressively Tesla requires drivers to keep their hands on the wheel, it’s not hard to imagine drivers diverting their attention elsewhere.

And that could be a great thing!

Tesla could start out by requiring drivers to basically keep their hands on the wheel at all times. Over time, as the software proves itself, Tesla could use over-the-air updates to slowly relax the requirements that drivers monitor the road.

Of course, Tesla could botch the rollout and cause lots of distracted driving accidents. But so far Tesla Autopilot has a great safety record, so I feel pretty good about this.

As the blog post notes, “Since we launched Autopilot in 2015, more than 1 billion miles of real-world driving data have been used to support the feature.”

For the past two years or so, Tesla has provided a “full self-driving” option on its vehicles. The option cost $5,000.

Tesla CEO Elon Musk announced new mid-range options for the Model 3 this week, and the associated design website renames the “Full Self-Driving” feature to “Enhanced Autopilot”.

In a tweet, Musk said:

I am a bit confused by the meaning of Musk’s tweet. Is “full self-driving” actually still available, on an alternate menu? Does he mean “full self-driving” has been unavailable for week?

The simplest explanation seems to be that Tesla decided to rebrand their Autopilot feature.

A lot of people have been skeptical about the ability of Tesla to create a fully autonomous vehicle with only cameras and one radar and no lidar. Perhaps this is a nod in that direction.

Autopilot is still the best ADAS system on the market, though.

“Musk wrote in an email obtained by Bloomberg News that Tesla needed about 100 more employees to join an internal testing program linked to rolling out the full self-driving capability. Any worker who buys a Tesla and agrees to share 300 to 400 hours of driving feedback with the company’s Autopilot team by the end of next year won’t have to pay for full self-driving — an $8,000 saving — or for a premium interior, which normally costs $5,000, Musk wrote.”

Full story here.

This is so exciting!

On the other hand, as CleanTechnica reminds, Tesla has struggled to fulfill Autopilot promises in the past. So take with a grain of salt.

Elon Musk broke the Internet a few days ago with a tweet promising that in August Tesla “will begin to enable full self-driving features.”

I am pretty excited about this but it is also worth noting that this verbiage is vague enough to drive a Tesla semi through.

“Full self-driving features” presumably means something beyond current Tesla Autopilot, but short of “full self-driving”. What are the features that make up full self-driving?

Some ideas include end-to-end routing, even if drivers still have to pay attention to the road. Or even just automatic lane change decision and execution on a highway.

Potentially this could mean that Tesla is enabling drivers to stop paying attention to the road under certain scenarios, although that seems unlikely, given the recent spate of crashes.

It will be exciting to see what, if anything, comes of this. But Elon Musk himself warns us that these tweets are not well-thought out strategy, but rather off-the-cuff remarks:

I’m a fan of Tesla, but it has been a rough month for the company.

Crash

In March, a Tesla Model X on Autopilot ran into a concrete barrier on Highway 85 in Mountain View, California. The driver was killed and the car exploded. Tesla wrote, “We have never seen this level of damage to a Model X in any other crash.”

Tesla later wrote:

“In the US, there is one automotive fatality every 86 million miles across all vehicles from all manufacturers. For Tesla, there is one fatality, including known pedestrian fatalities, every 320 million miles in vehicles equipped with Autopilot hardware. If you are driving a Tesla equipped with Autopilot hardware, you are 3.7 times less likely to be involved in a fatal accident.”

Statistics are rarely as compelling as stories, especially true stories, but I find these statistics reassuring. And, as with the Florida Autopilot crash in 2016, it makes a big difference that the only fatality here was the driver of the Tesla, not a member of the general public.

Regulation

In the aftermath of the crash, Tesla has gotten into a public disagreement with the National Transportation Safety Board, the US government agency running the main investigation. Apparently the argument is about how quickly to draw conclusions — Tesla wants to move faster than the NTSB does.

Blog

Tesla’s blog for the last month has been on the defensive. The three most recent posts are titled, “What We Know About Last Week’s Accident”, “An Update on Last Week’s Accident”, and “A Not So Revealing Story”.

That last post involves Tesla pushing back against, “an extremist organization working directly with union supporters to create a calculated disinformation campaign against Tesla.” Tesla claims it is building the “safest factory on earth”, whereas Reveal claims, “Tesla has failed to report some of its serious injuries on legally mandated reports, making the company’s injury numbers look better than they actually are.”

I have no idea who to believe in this disagreement. But at the very least it has got to be a grind to be running PR for Tesla right now, and probably for a lot of other employees, as well.

Production

Tim Higgins of the Wall Street Journal, who has been on top of the Tesla beat for quite a while, reported last week that Tesla had temporarily shut down Model 3 production. Tesla has cracked 2,000 Model 3 units per week, but has gotten nowhere near the 5,000 per week it targeted for last year.

Higgins subsequently fielded an unprompted confession from Elon Musk on Twitter:

“Humans are underrated,” is a pretty amazing quote, especially coming from Musk.

Stock

Through it all, Tesla’s stock has mostly held.

Valuation is down 25% from the highs of last summer, but this month has been pretty steady, except for a big dip and bounce-back right after the accident.

Tesla, for a time America’s most valuable car company, is now in 2nd place, behind General Motors.

But the fact that a month like this hasn’t sent investors running for the exits is a testament to the quality of the company and its cars.

Did I mention one other thing that happened this month? Popular Mechanics declared the Model 3 its Car of the Year.

Will the Tesla Model 3 bring an uptick in Autopilot-related traffic fatalities?

It seems like that question can be broken into two parts: how many new cars will be sold, and how safe will Model 3 drivers be relative to Model S and Model X drivers?

To the first question, there are approximately 300,000 Tesla Model S and Model X vehicles on the road. Meanwhile, the Model 3 waitlist is about 400,000 people long. Of course, not every person on the waitlist will ultimately purchase a Model 3, but it seems likely Tesla will at least double its installed base over the next couple of years.

Since there has been one fatality attributed to Autopilot in the past two years, maybe with a doubled installed base Tesla will experience two fatalities over the next two years?

Maybe fewer — one or zero — since presumably Autopilot has gotten better over time.

As an aside, Tesla Autopilot is amazing. It is (along with GM SuperCruise) the best Advanced Driver Assistance System on the market. My guess is that it has saved a lot of lives.

But every time there is a crash involving Autopilot, the safety of self-driving cars gets evaluated.

So the question of how safe Model 3 drivers will be seems important.

The Model S and Model X are high-price luxury vehicles, on the order of $100,000 out the door. The Model 3, on the other hand, is designed for a decidedly lower price-point buyer: $50,000 out the door.

On average, older Americans are wealthier, and my guess would be Model S and Model X buyers are quite a bit older than Model 3 buyers.

Older drivers are also, on average, safer drivers (this changes somewhere above age 65, but it’s true for most ages).

If a lot of Model 3 buyers are younger, less safe drivers, it’s possible that we’ll see an uptick in Autopilot-related fatalities in the coming years. Intuitively, think of more younger drivers watching Netflix while Autopilot drives the car.

Of course, all of this is pretty speculative. I imagine both Tesla and automotive insurers have much better models for how this is likely to play it. But it seems worth watching.

Tesla hinted at this before, but apparently its long-term plan is to build its own autonomous vehicle chips. They are taking “vertical integration” to a whole new level.

(Interestingly, when I looked up vertical integration on Wikipedia just now, the opening paragraph of the article lists Ford as an example. The more things change, the more they stay the same.)

Elon Musk apparently announced this at an event for AI researchers in Long Beach last week, concurrent with NIPS 2017.

The event was live-tweeted by Stephen Merity, who is worth a read in his own right: