I confess I was previously unfamiliar with the term “middle mile”, which apparently refers to the fixed routes between centralized distribution centers and dispersed retail locations.
The advantage, I take it, is that “middle mile” routes are limited and fixed, which would dramatically simplify technical challenge.
The box trucks should also be simpler to handle than articulated tractor-trailers.
The middle “swivel” point between a tractor and trailer adds a huge degree of complexity for control systems — not only laterally, but also vertically. The back of the trailer literally bounces up and down.
Box trucks are essentially rigid bodies in that way, like cars.
One of the big questions in autonomy is how and how much it is possible to simplify the technical challenge of autonomy. Tesla approaches this by limiting self-driving mainly to highways. Waymo limits its vehicles to specific geofences in a few metro areas. Voyage tackles this deploying relatively low-speed vehicles in gated retirement communities. Other vehicles work on sidewalks, or farms, or warehouses, or mines.
Gatik is betting that “middle mile” logistics will be a favorable niche.
I’d love to see some way to quantify how simple an environment is.
“That was a clear moment in time where the whole industry went from being a bull market to a bear market,” Mr. Cameron said. “Covid has taken us even further into the bear market.”
The Times closes with the high capital needs of self-driving companies.
With autonomous vehicles, “you may find yourself in a company that requires billions of dollars of capital,” with no clear timeline for building a large business or seeing a return on the investment, said Aaron Jacobson, a partner at NEA.
This is true, and it is one reason robotics companies are not super attractive generally, at the moment. But it would be a mistake to get too spooked by the industry’s capital needs.
The other side of that coin, as Warren Buffett has preached for years, is that capital intensive businesses offer the opportunity to deploy huge amounts of capital with attractive returns. Capital requirements also form a durable moat around the business.
What this tells me is that direct to learner businesses are very attractive. They can serve a very large number of learners very efficiently, they can lightly monetize and yet produce massive revenues because of their scale, and they don’t require a huge amount of capital to build.”
Autonomous vehicles and online education are both attractive industries, but they are very different industries, with distinct capital needs. Companies in each industry have to tailor their business plans to that reality.
The Detroit Free Press reports that new features are coming to GM Super Cruise. Specifically, Super Cruise will now perform lane changes “on demand”, and will negotiate highway interchanges.
More significantly, Super Cruise will roll out to the Cadillac Escalade and the Cadillac CT4 and CT5 sedans.
Super Cruise is often touted as the closest competitor to Tesla Autopilot, and maybe even a superior alternative.
Maddeningly, however, for years Super Cruise has only been available on top-of-the-line Cadillac CT6 models. As a result, it’s really hard to find a car that has it. I’ve never been in a car equipped with Super Cruise.
The Free Press also quotes GM that Super Cruise has been activated for 5.5 million miles since it launched in 2017. That means Waymo has driven more Level 4 miles since 2017 than Super Cruise has driven in something approximating Level 3. Tesla Autopilot, which has logged billions of miles, has orders of magnitude more data.
Nonetheless, Super Cruise has the potential to roll out to the larger GM model base and begin recording data at a level far beyond anything we’ve seen so far — from any manufacturer.
The company was founded by Stanford PhD and AI wunderkind Jesse Levinson, along with Tim Kentley-Klay, a brash Australian designer.
For years Zoox was highly secretive about its technology and goals, even by the standards of the tight-lipped self-driving car industry. The Zoox website was a single HTML page sporting only the company logo.
In 2018, Zoox pulled back the curtain with awesome highlighting the its autonomous capabilities and Mad Max design ethos.
Only a month later, Kentley-Klay was ousted just after Zoox raised $500 million in venture funding. Eventually Zoox recruited Intel Chief Strategy Aicha Evans as CEO.
All of which is to say, Zoox has been quite a story.
Apparently the company terminated contractors and laid off 10% of its workforce in light of the COVID pandemic. Now it seeks either further venture capital funding, or an acquirer.
The status of being publicly for-sale is reminiscent of Drive.AI, another prominent (although much smaller) self-driving start-up. That firm was widely reported to be shopping for a buyer during the first half of 2019. Apple ultimately acquired the startup in a firesale, days before bankruptcy.
All that said, a few qualifiers.
The stories I’ve read seem to make a lot of hay that Zoox has hired an investment bank, Qatalyst Partners. Investment banks usually do spell “sale”. But they can also spell “investment”. Particularly for a company of Zoox’s size, the sources of private investment change from traditional venture capital firms to larger institutional funds that work more regularly with investment banks. Zoox may simply be working on a really large funding round, which would go hand-in-hand with also beating the bushes for any potential acquirers.
Also, the stories I’ve read indicate Zoox plans to hire back laid off employees and contractors once coronavirus subsides. That’s easy to say now, of course, but it’s worth keeping in mind that perhaps what looks bad is merely a bump in the road.
Ford’s dataset includes multiple driving environments, multiple sensor modalities, multiple vehicles, and multiple seasons, on a consistent 66 km route in the Detroit area.
Combined with datasets from Waymo, nuTonomy, Lyft, UC-Berkeley, Daimler, and KIT, there is a wealth of self-driving car data available to play with 🙂
“Soon, your Volvo will be able to drive autonomously on highways when the car determines it is safe to do so,” said Henrik Green, chief technology officer for Volvo Cars.
That sounds a lot like Level 3 autonomy. The timing of this announcement is particularly interesting, coming right on the heels of Audi retreating from Level 3 — due to liability concerns.
The Volvo ADAS systems I’ve tested out in the past have always been quite impressive — in the top tier of systems behind Tesla Autopilot. If there is one way for competitors to catch and surpass Tesla, it might be exploiting the lidar technology that Tesla has famously eschewed.
Now, three years later, Audi has announced Level 3 won’t be coming to the A8. Surprisingly, Audi attributes this about-face to legislation, not technology.
“Currently, there is no legal framework for Level 3 automated driving and it is not possible to homologate such functions anywhere in the world in a series production car.”
Apparently the lawyers got nervous that customers might not service and maintain their cars, then the vehicles would crash, and Audi would be on the hook.
If the real issue here is liability, not technology, that suggests the first movers toward Level 3 vehicles will be companies with very little to lose. So maybe not Tesla, which is now the most valuable American automaker, by an order of magnitude.
Maybe Level 3 will come from some small startup that doesn’t have to worry about losing billions of dollars in market capitalization that it never had in the first place.
Last fall, a wide consortium of autonomous vehicle companies published, “Safety First for Automated Driving”, a whitepaper aimed as filling the standards gap between ISO 26262 (Functional Safety), ISO 21448 (Safety of the Intended Functionality — SOTIF), and the reality of where self-driving cars are heading.
According to EE Times, this whitepaper is on its way to becoming its own ISO standard. That would provide a clear and consensus view of how to approach safety for autonomous vehicles, which has been lacking in the industry up to this point.
At 157 pages, the whitepaper is thorough but digestible. Over the coming weeks I’ll try to break down the contents chapter-by-chapter, to see what the industry consensus is.
Andrei Karpathy is one of the most impressive and celebrated computer scientists in the world, and has worked for the past several years as Senior Director, AI, at Tesla. Essentially, he leads their Autopilot team.
Reilly Brennan’sFuture of Transportation newsletter (you should subscribe) pointed to a talk Karpathy recently gave at a conference called ScaledML. It’s pretty great, so I decided to annotate it, as a way to capture all of the details for myself, as much as anything else.
[00:00] Karpathy’s title is Senior Director. I remember him joining Tesla as a Director, so I think he got a promotion. Congratulations!
[00:19] Karpathy starts by defining what Autopilot is. This seems like good presentation technique. Establish the basics before moving on to advanced topics.
[00:50] Karpathy shows 8 Tesla vehicle models, noting that some of them have “only been announced.” Models S, 3, X, Y, T(ruck), A(TV — joking?), R, and S(emi). Globally Tesla has over 1 million vehicles.
[01:35] Autopilot has 3 billion miles, “which sounds(?) like a lot.”
[01:58] “We think of it (Autopilot) as roughly autonomy on the highway.” Sounds like Level 3 to me.
[02:24] “Smart Summon is quite magical, when it works (audience laughs).” I actually don’t know, is Smart Summon unreliable?
[03:12] Euro NCAP has rated Teslas as the safest vehicles, which isn’t a surprise but also puts the Autopilot lawsuits in perspective.
[03:45] Karpathy shows some examples of Tesla safety features working, even when Autopilot is not turned on. Probably this means that Karpathy’s team is working on the broader array of safety features, not just Autopilot.
[04:43] “The goal of the team is to produce full self-driving.” Karpathy has always struck me as more reliable and realistic than Musk. “Full Self-Driving” means more coming from Karpathy.
[06:30] “We do not build high-definition maps. When we come to an intersection, we encounter it basically for the first time.” This is striking, and I don’t think I’ve heard Tesla put it quite like this before. Tesla is famous for eschewing lidar, but I wonder why they don’t build vision-based maps?
[08:00] Karpathy mentions that the neural networks on the car really have two separate tasks — (a) driving, and (b) showing the humans in the vehicle that the computer perceives the environment, so the humans trust the system.
[09:16] We see a photo of a crossing guard with a handheld stop sign, hanging loose from the guard’s limp arm. Karpathy calls this “an inactive state.” This really highlights to me how hard it is for a computer to know whether a stop sign is real or not.
[10:10] Karpathy mentions Tesla builds maps, “of course, but they’re not high-definition maps.” I wonder what kind of maps they are.
[10:35] The Autopilot team spends at least part of its day-to-day work going through the long-tail and sourcing examples of weird stop signs. And presumably other weird scenarios. Man that sounds like a grind — I would imagine they must automate or outsource a lot of that.
[11:15] Bayesian uncertainty in the neural network seems to play a role.
[12:21] When Tesla needs more data, they just send an extra neural network to their vehicle fleet and ask the cars to run that network in the background, gathering potential training images. I would be it will take traditional automotive companies a long time to develop this capability.
[13:16] Test-Driven Development! TDD for the win!
[14:37] HydraNet is a collection of 48 neural networks with a “shared backbone” and 1000 distinct predictions. This is a multi-headed neural network on steroids.
[14:59] “None of these predictions can ever regress, and all of them must improve over time.” I don’t really understand what he means here. Surely there must be times a network predicts a dog and then later realizes it’s a child, etc.
[15:15] Autopilot is maintained by “a small, elite team — basically a few dozen people.” Wow.
[15:54] The goal of the Tesla AI team is to build infrastructure that other, more tactical people can then use to execute tasks. They call this approach Operation Vacation. (ruh-ruh)
[16:46] For example, if somebody at Tesla wants to detect a new type of stop sign, they supposedly don’t even have to bother Karpathy’s team. The AI team has already built out all the infrastructure for the rest of Tesla to plug new “landmark” images into.
[17:56] Karpathy shows an occupancy tracker that looks like something out of a 2-D laser scanner from twenty years ago. I wonder if they’re basically using cameras to fake what lidars do (Visual SLAM, etc.).
[19:36] Autopilot code used to be a lot of C++ code, written by engineers. As the neural networks get better, they’re eating up a lot of that “1.0” codebase.
[19:51] Aha! The occupancy tracker is old, “1.0” code, written by people. The future is neural networks!
[20:00] There is a “neural net fusion layer, that stitches up the feature maps and projects to birds-eye view.”
[20:15] There is a “temporal module” that smoothes and a “BEV net decoder”. What is are these things? I probably need to spend a few weeks getting back up to speed on the latest neural network research.
[22:15] Karpathy shows off how well this system works, but it’s hard to follow and judge for myself.
[22:35] Tesla takes a “pseudo-lidar approach, where you predict the depth of every since pixel and you basically simulate lidar input purely from vision.” Why not just use lidar, then? The unit price is coming down. Probably Tesla can’t depend on lidar because it already has a million vehicles on the road, none of which have lidar, and many of which have paid for full self-driving already. Realistically, though, this sounds like Tesla will start to add lidar at some point.
[24:02] The gap between lidar and a camera’s ability to simulate lidar is “quickly shrinking.” What’s the gap now? Is this tracked somewhere in academic literature?
[24:36] The driving policy (the motion planning), is still human-coded. But not for long! This is where Tesla’s fleet really shines. Karpathy notes that their human drivers are basically building supervised motion planning datasets for free.
[26:17] Really nice job summarizing his own talk. It’s just amazing that one guy can be such a phenomenal computer scientist and also so skilled at communication — in a second language, no less!
[27:30] During Q&A, Karpathy notes that Tesla builds low-definition semantic maps, which somewhat contradicts his earlier statement that every intersection is basically approached as if it were a new intersection.
[29:45] The hand-coded, “software 1.0” stack is used to keep the neural network within “guardrails.”
Cruise has kept at least a portion of its self-driving fleet operating in San Francisco during the COVID pandemic. Those vehicles are focused on delivering meals to vulnerable populations in the city, according to Mashable.
This is a great move by Cruise, both because it keeps the vehicles up and running, and because it contributes to a societal need.
Without a larger mission, Cruise might find it hard to justify violating shelter-in-place restrictions by driving on city streets with two vehicle operators inside a prototype autonomous vehicle. But Cruise’s mission transforms the testing operation into an “essential” service, and justifiably so.
The goal of self-driving cars is to serve our communities in dangerous times and situations, like the pandemic in which we now find ourselves. The autonomous technology may not have fully arrived yet, but Cruise shows how we can achieve some of those goals in the here and now.
David Silver 