Udacity students in attendance on January 17th, you’re invited to lunch!
I am excited to be visiting Detroit next week for the North American International Auto Show. Please come say hello if you’re attending, or send me an email at david.silver@udacity.com.
Here’s the schedule:
Wednesday, January 17, 12:30pm: I’ll be at Automobili-D, speaking on a panel about the autonomous driving industry. Come watch! If you’re a Udacity student, we’ll take you to lunch afterward.
Thursday, January 18, 6pm: RSVP for our Career Workshop and Meetup! We’ll teach you how to present yourself to recruiters in the autonomous vehicle industry, and you’ll practice pitching yourself to other attendees. Also, free food!
Saturday, January 20, 10:30am: I’ll be speaking at the Future Automotive Career Exposition at NAIAS. Introduce yourself to me, then introduce yourself to the many recruiters who will be hiring autonomous vehicle engineers!
The Consumer Electronics Show used to be the place to go to see the latest and greatest television sets, but in recent years it has essentially become a car show. And this year it was a self-driving car show.
I myself have never been, but my Udacity colleagues at CES this year report that the entire show is a tribute to self-driving cars.
The Future Automotive Career Exposition (FACE) at next week’s North American International Auto Show (NAIAS) in Detroit is a terrific opportunity to connect with a huge number of autonomous vehicle employers at once. If you’re looking for a job in the field, you should go.
FACE will take place on January 20th and 21st. Large employers will be present, like Autoliv, Delphi, Flex, General Motors, HARMAN, Magna, Magneti Marelli, Nexteer, Nissan, Valeo, and Visteon. And many of the startups who will be presenting as part of the NAIAS Automobili-D showcase will also be recruiting: DropCar, eCharge.work, Einride, Karma Automotive, Navya, ParkWhiz, RideCell, and Savari.
FACE is encouraging attendees to apply to the companies ahead of time so that they can review your CV before you introduce yourself in-person. I’m also told the first 500 entrants to the event will get free admission. After that, attendees will need to have an NAIAS pass ($14) to attend.
Udacity’s upcoming “Intro to Apollo” course will focus on the top two layers: Cloud Service and Apollo Open Software Stack.
The Course
Apollo is an incredibly exciting platform in the autonomous vehicle industry. We are thrilled to work with the Apollo team to teach students and engineers around the world how to build self-driving car software quickly using the Apollo stack.
I am especially delighted that this will be a free course, open to anyone with the desire to enter this amazing field. There is a huge demand for knowledge about how self-driving cars work, and this course will help educate the world on this topic. Our Self-Driving Car Engineer Nanodegree Program is an intense nine-month journey to becoming a self-driving car engineer, and it offers an amazing learning experience, but it is for advanced engineers. And while our Intro to Self-Driving Cars Nanodegree program is an excellent point-of-entry for aspiring learners newer to the field, it offers an equally immersive experience. This course offers adds something new and important to the range of learning options.
China
This is a special opportunity for us to collaborate with Baidu, one of the leading companies in China. China is a leader in the autonomous vehicle industry. And Chinese students currently make up 5% of enrollment in Udacity’s Self-Driving Car Engineer Nanodegree Program, and 20% of enrollments in all Udacity programs. A major focus for our Self-Driving Car Program in 2018 is to reach even more students in China.
The course will be developed jointly by Baidu’s Apollo team, the Udacity Self-Driving Car team in Mountain View, and the Udacity China team. The course will be in English, but this is a new experiment for us in developing course material in one of our offices outside of the US. I’m excited.
Did I mention I’m excited about this course? Because I’m excited!
If you put Tesla Autopilot into the mix, they appear to be in the hundreds of millions of miles, if not billions by now. It’s not exactly a fair comparison, since Autopilot is SAE Level 2, whereas Waymo, Uber ATG, and Cruise are all testing Level 4 systems. But it does show the data-collection benefit of getting a real production fleet out on the road.
So, in an industry where miles driven are the name of the game, we have:
Telsa: ~1 billion Waymo: 4 million Uber: 2 million Cruise: ~0.5 million
It’s not clear who’s next — maybe nuTonomy? — but it seems like those are the pacesetters.
Since Chris Urmson departed his perch as the head of Google’s Self-Driving Car Project in August, 2016, he’s flown pretty well below the radar. Urmson strikes me as an understated guy to begin with, so perhaps that’s not too surprising.
“Both the VW Group (the parent company to VW, Audi, and many others) and Hyundai are working with Aurora to build the startup’s self-driving technology into some of their vehicles, with the ultimate goal of creating fleets that will be available on demand in cities. Each company says it plans to make Aurora-powered vehicles commercially available by 2021, and both will start testing prototypes later this year.”
Of particular note to me at Udacity, his LinkedIn profile says “we’re hiring” and The Verge reports:
“The war for talent right now is incredible,” Urmson says. “But my experience is that if you treat people well with respect and empower them to solve interesting problems, and you give them a good mission to work at, [keeping the talent] takes care of itself.”
Here’s what I think the coming year has in store for self-driving cars.
100% Certain
No Level 5 self-driving cars will be deployed anywhere in the world. No GPS or DGPS system will reliably exceed 10cm localization accuracy on all public roads in the US.
90% Certain
Level 4 self-driving cars will be available to the general public, on public roads, somewhere in the world. Deep learning will remain the dominant tool for image classification. No US road will have a speed limit for autonomous vehicles that is faster than the speed limit for human-driven vehicles.
80% Certain
All Level 4 vehicles available to the general public will use lidar. Somebody will die in a crash while due to a failure of Tesla Autopilot. Waymo will still have driven more autonomous miles than any other company. Level 4 self-driving cars will be available to the general public somewhere other than Pittsburgh. A company will be acquired primarily for its autonomous vehicle capabilities with a valuation above $100M USD. No dominant technique will emerge for urban motion planning.
70% Certain
Level 4 self-driving cars will be available to the general public in Pittsburgh. Level 4 self-driving cars will be available to the general public somewhere in China. Tesla will sell the most advanced self-driving system available to the general public. Deep learning will not be the dominant tool for object classification from point clouds.
60% Certain
2,000 students will have graduated the Udacity Self-Driving Car Engineer Nanodegree Program. Level 4 self-driving cars will be available to the general public somewhere in Europe. Waymo will have exceeded 10 million miles driven. Tesla will produce 5,000 Model 3 vehicles in a single calendar week. No member of the general public will die in a Level 4 autonomous vehicle.
50% Certain
Cruise Automation will open its Level 4 fleet to the general public. Level 3 self-driving cars will be available for purchase by the general public. A company will be acquired primarily for its autonomous vehicle capabilities with a valuation above $1B USD. 1,000 Udacity students will have jobs in the autonomous vehicle industry. Self-driving cars will be legal for public use somewhere in India.
Many students describe the Path Planning Project as the most challenging project in the entire Udacity Self-Driving Car Engineer Nanodegree program. This is understandable. Path planning is hard! But it’s not too hard, and I’m going to tell you a bit about the project—and about path planning in general—in this post.
There are three core components to path planning: 1) Predicting what other vehicles on the road will do next, 2) Deciding on a maneuver to execute, in response to our own goals, and to our predictions about other vehicles, and 3) Building a trajectory to execute the maneuver we decide on.
This is a project that provides students a lot of freedom in how to implement their solution. Here are five approaches from our amazing students!
I love Mithi’s series of posts on the Path Planning Project. Her first post covers the project outline and her solution design process. The second post covers her data structures and pipeline. The third and final post dives into the mechanics and math required to actually produce a path. This is a great series of posts for anybody thinking about building a path planner.
The goal is to create a path planning pipeline that would smartly, safely, and comfortably navigate a virtual car around a virtual highway with other traffic. We are given a map of the highway, as well as sensor fusion and localization data about our car and nearby cars .We are supposed to give back a set of points (x , y)in a map that a perfect controller will execute every 0.02 seconds. Navigating safely and comfortably means we don’t bump into other cars, and we don’t exceed the maximum speed, acceleration, and jerk requirements. Navigating smartly means we change lanes when the car in front of us is too slow.
In contrast to Mithi’s articles, which take you through her process of building a path planner, Mohan’s writeup does a great job of describing the final result. In particular, I was interested to read about the voting system he used for deciding on lane changes.
Instead of quoting Mohan, I’ll share the flowchart he built:
This installment of Andrew’s long-running “diary” covers the Path Planning Project at a high level, and details how it fits into the third term of the Nanodegree program. Like his fellow classmates, Andrew also found this to be a challenging project.
I found the path planning project challenging, in large part due to fact that we are implementing SAE Level 4 functionality in C++ and the complexity that comes with the interactions required between the various modules.
Shyam’s post contains a particularly concise 6-point walkthrough of trajectory generation, which is both fundamental to building a path planner, and surprisingly challenging.
The trajectory generation part which is the most difficult is covered as part of the project walk-through by Aaron Brown and David Silver. LINK. They recommend using the open source C++ tk:spline() method to generate a 5th degree polynomial which help minimize jerk while accelerating, decelerating and changing lanes.
Alena touches on several interesting points with her post. She focuses on cost functions, which she identifies as the most important part of the project. The post describes her finite state machine and the associated cost functions in detail, and describes how the car decides when to shift lanes. She also touches on how she merged the two branches of her path planner — one for the Nanodegree project, and one for the Bosch Challenge — to create a more generalized planner.
The most important part of the project is to define cost functions. Let’s say we are on a highway with no other cars around. In such situation, we should stay in the same lane. Constant lane changing is extremely uncomfortable, so my first cost function is change_lane_cost. We penalized our trajectory if we want to change lane. Honestly, I did a small trick for Bosch challenge. I did not penalize the trajectory if I want to move in the middle lane. It gives me more freedom with maneuvers. Otherwise, I can be stuck in the left-most and right-most lanes when my lane and middle lanes are busy.
Seeing our students working through these challenges, experiencing their solutions, and learning about their processes fills me with so much excitement about the future of this field—these students represent the next generation of self-driving car engineers, and based on the work they’re already doing, I am certain they’re going to be making incredible contributions. I am especially moved by their generosity in taking the time to share in such detail the work they’re engaged in, and it’s a real pleasure to share their articles with you.
I haven’t seen this reported anywhere else, and even if it’s true it’s not a huge deal on its own. But it would be a sign of potentially huge changes ahead.
Why?
Lidar is a critical sensor for every major autonomous vehicle company (except Tesla).
Lidar is by far the most expensive sensor on the vehicle. The VLP-16 is “only” $8,000, but Velodyen’s top-end HDL-64E retails for about $100,000. And even at that price, historically there has been a multi-month backlog!
Exactly one year ago, I laid out a number of predictions for the self-driving car world in 2017, along with associated percentages (I lifted the percentages idea from Scott Alexander). Probably I should’ve written this post yesterday, on the last day of 2017, but better late than never.
Here are my predictions from one year ago, scored. Bold font means the prediction was correct. Plain font means the prediction was incorrect.
1 Udacity Self-Driving Car Engineer Nanodegree Program student will start a new, permanent, full-time autonomous vehicle job: 99%
Level 4 self-driving cars will be available for ridesharing on public roads somewhere in the world: 90%
Level 4 self-driving cars will be available for ridesharing on public roads somewhere in the United States: 90%
100 Udacity Self-Driving Car Engineer Nanodegree Program students will start new, permanent, full-time autonomous vehicle jobs: 90%
No US highway will have a speed limit for autonomous vehicles that is faster than the speed limit for human-driven vehicles: 90%
1000 Udacity Self-Driving Car Engineer Nanodegree Program students will start new autonomous vehicle jobs (possibly contract roles or internships): 80% [UPDATE: A year ago I did not appreciate how hard it would be for us to measure this. If you are a student, at some point Udacity will email you a survey about your job search. Please respond! In the meantime, we are looking into more automatic ways to measure students-in-jobs.]
Level 4 self-driving cars will be available for ridesharing on public roads in Singapore: 80% [UPDATE: As far as I can tell, nuTonomy is continuing to beta test their service in Singapore, with a target public launch of Q2 2018. Although the prediction was not precise, I am not going to count the current service, because, as far as I know, it’s not open to the general public.]
A company will be acquired primarily for its autonomous vehicle capabilities with a valuation above $100 MM USD: 80%
No company will sell a vehicle with autonomous technology that exceeds what Tesla offers: 80% [UPDATE: Although Business Insider just ranked GM Super Cruise ahead of Tesla Autopilot, it seems close enough and subjective enough that I score them as essentially tied.]
At least 1 company that has not done so before will complete a continuous US coast-to-coast demonstration trip in a fully autonomous vehicle: 80% [UPDATE: Torc Robotics.]
Level 4 self-driving cars will be available for ridesharing on public roads in California: 70%
Conditional on Level 3 self-driving cars being legal somewhere, Tesla will enable Level 3 autonomy in that location: 70% [UPDATE: Many (most?) US states have not specifically regulated self-driving cars, leading some lawyers to believe that there are no restrictions on self-driving cars in those jurisdictions. So I count this one as wrong, because Level 3 is presumably legal somewhere, but Tesla has not enabled it.]
Google will offer rides to the public in its self-driving cars: 70% [UPDATE: Same logic as Singapore, above.]
No member of the general public will die in a Level 4 vehicle: 70%
2500 Udacity Self-Driving Car Engineer Nanodegree Program students start new autonomous vehicle jobs (possibly contract roles or internships): 60%
Level 4 self-driving cars will be available for ridesharing on public roads somewhere in Europe: 60%
Level 4 self-driving cars will not be available for ridesharing on public roads somewhere in China: 60%
No company will be acquired primarily for its autonomous vehicle capabilities with a valuation above $500 MM USD: 60% [UDPATE: Mobileye. Oops.]
A new autonomous vehicle startup will form and raise money at a valuation above $75 MM USD: 60% [UPDATE: Argo AI]
Ford will commit to launching Level 4 vehicles before 2021: 50%
Level 4 self-driving cars will be available for ridesharing on public roads somewhere in the world, without a safety driver: 50% [UPDATE: Same logic as Singapore, above.]
Level 3 self-driving cars will be available for private ownership somewhere in the United States: 50%
Somebody will die in a Tesla Autopilot crash: 50%
Level 4 autonomous vehicles will be available for public ridesharing when snow is on the ground: 50%
99% confidence: 1 / 1 (100% correct)
90% confidence: 5 / 5 (100% correct)
80% confidence: 3/ 5 (60% correct)
70% confidence: 1 / 4 (25% correct)
60% confidence: 2 / 5 (40% correct)
50% confidence: 0 / 5 (0% correct)
Statistically, based on my confidence assessments, about 18 of the predictions should have been correct. In fact, only 12 were correct. So I was overconfident in my ability to predict the future.
Mostly, I appear to have been too optimistic, although in some cases (i.e. fatalities) I was too pessimistic. A few of my predictions were also not precise enough.
Some important stories from 2017 do not show up in my predictions:
The Waymo v. Uber lawsuit.
The success of Cruise Automation in building a Level 4 fleet.
Tesla’s manufacturing difficulties.
The 10,000+ students who have enrolled in the Nanodegree Program, many of whom will get jobs in the industry in 2018.
Voyage spinning out from Udacity.
The growing importance of lidar.
Investments by Didi, Baidu, and Chinese companies generally.
NVIDIA’s dominance of the self-driving car chip market.
David Silver 