Our partner Didi Chuxing is the largest ride-sharing service in China. And it’s looking to hire self-driving car engineers!
Didi founder and CEO Cheng Wei said he is hunting for data scientists in Silicon Valley to develop a self-driving car. Didi Chuxing bought Uber China in a $35 billion deal over the summer.
Cheng added that he’s also been in talks with Gansha Wu, the former director of Intel Labs who also founded UiSee Technology, a Beijing-based self-driving car company.
Didi is a terrific partner and we are lucky to have their support.
We are working hard to make this the world’s best training program for self-driving car engineers. The entire curriculum will consist of three terms over nine months. Here’s what in the program:
Term 1
Introduction
Meet the instructors — Sebastian Thrun, Ryan Keenan, and myself. Learn about the systems that comprise a self-driving car, and the structure of the program.
Project: Detect Lane Lines Detect highway lane lines from a video stream. Use OpenCV image analysis techniques to identify lines, including Hough transforms and Canny edge detection.
Deep Learning
Machine Learning: Review fundamentals of machine learning, including regression and classification.
Neural Networks: Learn about perceptrons, activation functions, and basic neural networks. Implement your own neural network in Python.
Logistic Classifier: Study how to train a logistic classifier, using machine learning. Implement a logistic classifier in TensorFlow.
Optimization: Investigate techniques for optimizing classifier performance, including validation and test sets, gradient descent, momentum, and learning rates.
Rectified Linear Units: Evaluate activation functions and how they affect performance.
Regularization: Learn techniques, including dropout, to avoid overfitting a network to the training data.
Convolutional Neural Networks: Study the building blocks of convolutional neural networks, including filters, stride, and pooling.
Project: Traffic Sign Classification Implement and train a convolutional neural network to classify traffic signs. Use validation sets, pooling, and dropout to choose a network architecture and improve performance.
Keras: Build a multi-layer convolutional network in Keras. Compare the simplicity of Keras to the flexibility of TensorFlow.
Transfer Learning: Finetune pre-trained networks to solve your own problems. Study cannonical networks such as AlexNet, VGG, GoogLeNet, and ResNet.
Project: Behavioral Cloning Architect and train a deep neural network to drive a car in a simulator. Collect your own training data and use it to clone your own driving behavior on a test track.
Computer Vision
Cameras: Learn the physics of cameras, and how to calibrate, undistort, and transform image perspectives.
Lane Finding: Study advanced techniques for lane detection with curved roads, adverse weather, and varied lighting.
Project: Advanced Lane Detection Detect lane lines in a variety of conditions, including changing road surfaces, curved roads, and variable lighting. Use OpenCV to implement camera calibration and transforms, as well as filters, polynomial fits, and splines.
Support Vector Machines: Implement support vector machines and apply them to image classification.
Decision Trees: Implement decision trees and apply them to image classification.
Histogram of Oriented Gradients: Implement histogram of oriented gradients and apply it to image classification.
Deep Neural Networks: Compare the classification performance of support vector machines, decision trees, histogram of oriented gradients, and deep neural networks.
Vehicle Tracking: Review how to apply image classification techniques to vehicle tracking, along with basic filters to integrate vehicle position over time.
Project: Vehicle Tracking Track vehicles in camera images using image classifiers such as SVMs, decision trees, HOG, and DNNs. Apply filters to fuse position data.
Term 2
Sensor Fusion
Our terms are broken out into modules, which are in turn comprised of a series of focused lessons. This Sensor Fusion module is built with our partners at Mercedes-Benz. The team at Mercedes-Benz is amazing. They are world-class automotive engineers applying autonomous vehicle techniques to some of the finest vehicles in the world. They are also Udacity hiring partners, which means the curriculum we’re developing together is expressly designed to nurture and advance the kind of talent they would like to hire!
Lidar Point Cloud
Below please find descriptions of each of the lessons that together comprise our Sensor Fusion module:
Sensors The first lesson of the Sensor Fusion Module covers the physics of two of the most import sensors on an autonomous vehicle — radar and lidar.
Kalman Filters Kalman filters are the key mathematical tool for fusing together data. Implement these filters in Python to combine measurements from a single sensor over time.
C++ Primer Review the key C++ concepts for implementing the Term 2 projects.
Project: Extended Kalman Filters in C++ Extended Kalman filters are used by autonomous vehicle engineers to combine measurements from multiple sensors into a non-linear model. Building an EKF is an impressive skill to show an employer.
Unscented Kalman Filter The Unscented Kalman filter is a mathematically-sophisticated approach for combining sensor data. The UKF performs better than the EKF in many situations. This is the type of project sensor fusion engineers have to build for real self-driving cars.
Project: Pedestrian Tracking Fuse noisy lidar and radar data together to track a pedestrian.
Localization
This module is also built with our partners at Mercedes-Benz, who employ cutting-edge localization techniques in their own autonomous vehicles. Together we show students how to implement and use foundational algorithms that every localization engineer needs to know.
Particle Filter
Here are the lessons in our Localization module:
Motion Study how motion and probability affect your belief about where you are in the world.
Markov Localization Use a Bayesian filter to localize the vehicle in a simplified environment.
Egomotion Learn basic models for vehicle movements, including the bicycle model. Estimate the position of the car over time given different sensor data.
Particle Filter Use a probabilistic sampling technique known as a particle filter to localize the vehicle in a complex environment.
High-Performance Particle Filter Implement a particle filter in C++.
Project: Kidnapped Vehicle Implement a particle filter to take real-world data and localize a lost vehicle.
Control
This module is built with our partners at Uber Advanced Technologies Group. Uber is one of the fastest-moving companies in the autonomous vehicle space. They are already testing their self-driving cars in multiple locations in the US, and they’re excited to introduce students to the core control algorithms that autonomous vehicles use. Uber ATG is also a Udacity hiring partner, so pay attention to their lessons if you want to work there!
Here are the lessons:
Control Learn how control systems actuate a vehicle to move it on a path.
PID Control Implement the classic closed-loop controller — a proportional-integral-derivative control system.
Linear Quadratic Regulator Implement a more sophisticated control algorithm for stabilizing the vehicle in a noisy environment.
Project: Lane-Keeping Implement a controller to keep a simulated vehicle in its lane. For an extra challenge, use computer vision techniques to identify the lane lines and estimate the cross-track error.
Term 3
Path Planning
Elective
Systems
Term 2 and Term 3 are under construction and we’ll share more details on those as we finalize the curriculum and projects.
All of this, including Term 1, is subject to change as we update the curriculum over time, because part of building a great course is taking feedback and making improvements!
If you’ve been accepted into the course, congratulations! We are excited to teach you.
If we suggested you brush up on a few topics and take a self-assessment before joining the course, please do! We are excited to teach you and want to make sure you have a great experience.
And if you haven’t yet applied, please do! We are taking applications for the 2017 cohorts and would love to have you in the class.
The quality of the students has been astounding. It’s both incredibly exciting and, honestly, intimidating to be teaching students this talented and accomplished. Many of students who applied already work on autonomous vehicles, and want to take the course to make sure they’re at the top of their game. Others are top engineers at companies like Google and Facebook, and still others are stars at universities (including professors!) in the US and abroad.
And many are just good engineers who are excited about self-driving cars, kind of like me not very long ago.
Thank you to everyone!
Dhruv has spent all of his waking hours this past week going through each application and organizing students into cohorts.
Udacity’s mission is to democratize education, so we will be trying to teach as many people as possible.
We’ll start small, with an initial cohort of 500 students, to give us a chance to scale up the program and iron out any wrinkles.
But we plan to be teaching thousands of students about self-driving cars by the end of the year.
See you in class!
(By the way, if you’re interested but haven’t applied yet, you can still do that! We’re taking applications now for our 2017 cohorts, so please apply!)
…almost four years to the day since driverless trials on public streets were first approved, a new bill has been signed off by Governor Jerry Brown that permits autonomous car tests without a human passenger overseeing proceedings.
Rather than applying throughout the Golden State, the bill is specific to a pilot project headed by the Contra Costa Transportation Authority. At San Ramon’s Bishop Ranch business park, EasyMile’s 12-seater shuttles will ferry workers around the site, which will include travelling on some public roads. The approval also covers GoMentum Station: A ghost town within the Concord Naval Weapons Station where Honda has been testing its driverless car technologies. Recently, Uber-owned Otto also signed up to test self-driving trucks on the site.
I’m actually a little surprised this wasn’t already legal. I kind of thought you could do whatever you want on your own land.
Another article I read said that disabled-persons groups were key in getting the bill passed.
I think one of the biggest areas of society that self-driving cars will change, and one of the least-appreciated areas, will be policing.
Most of the interactions I have had with police are related to driving, and once the car is driving for me, those interactions will go away. Maybe they’ll be replaced by other police interactions, or maybe not. But it will be different.
So what’s the big deal if police can no longer make traffic stops? It’s about half of what police do, says [criminology Professor Joseph] Schafer. He estimates such stops, along with traffic accidents, account for nearly 50 percent of all police-public encounters.
The end of traffic stops would have surprisingly large implications.
Another aspect of this situation might stem from a system that burdens the poor and black in Ferguson. Minor traffic offenses are the starting point, and the costs spiral up rapidly if the offenders do not pay the fines on time or do not appear in court. The income from court fines represented the second largest source of revenue for Ferguson in 2013. On October 1, 2014, the city of St. Louis cancelled 220,000 arrest warrants and gave a three-month delay to the offenders to get a new court date before the warrants would be reissued.
Ferguson might be an outlier, but traffic fines provide a huge part of the budget of many police departments or even cities.
Police may be sanguine about self-driving cars until it really becomes obvious how deeply self-driving cars can hit their funding structure.
The easiest way to think of their relationship is to visualize them as concentric circles with AI — the idea that came first — the largest, then machine learning — which blossomed later, and finally deep learning — which is driving today’s AI explosion — fitting inside both.
I guess if I had to explain, I would say that:
“artificial intelligence” refers to techniques that help computers accomplish goals
“machine learning” refers to techniques that help computers accomplish goals by learning from data
“deep learning” refers to techniques that help computers accomplish goals by using deep neural networks to learn from data
But if you’re interested in these topics, then read the NVIDIA post. It’s good.
We are so excited to have over 10,000 students apply to join the program, and we hope to teach all of them.
We’re limiting the initial cohort to 500 students to make sure we have everything ready to go to scale up the program over time, but the goal is to be able to teach everyone who wants to learn.
Here is a tentative (subject to change) overview of the first term:
Introduction: You’ll learn about the program, the student support available, and, most importantly, the ways we’ll help you land a job in autonomous vehicles. Within hours of starting, you’ll be writing code to find lane lines on the road.
Deep Learning: You’ll learn about deep neural networks and deep learning frameworks. In the final project you’ll build a deep neural network for end-to-end driving of a vehicle in a simulator.
Computer Vision: You’ll learn about how computers and cameras work together to see the world. In the final project you’ll use OpenCV and deep learning to identify vehicles on the highway.
I am super-excited about this program and I hope you are, too. Please join us!
A van in Mountain View, California, ran a red light yesterday and t-boned a Google self-driving car, resulting in what news outlets are calling, “one of the worst accidents in the history of the Google self-driving car program”.
To be clear, Google’s self-driving car was the victim — the human driver of the other vehicle was 100% at-fault.
A website called 9to5google.com has the best reporting I’ve seen, including a statement from Google:
A Google vehicle was traveling northbound on Phyllis Ave. in Mountain View when a car heading westbound on El Camino Real ran a red light and collided with the right side of our vehicle. Our light was green for at least six seconds before our car entered the intersection. Thousands of crashes happen everyday on U.S. roads, and red-light running is the leading cause of urban crashes in the U.S. Human error plays a role in 94% of these crashes, which is why we’re developing fully self-driving technology to make our roads safer.
This is just another reminder that however sexy and interesting the Trolley problem might be, it’s irrelevant, at least in the short-term. The immediate, real problem is that we humans are terrible drivers.
The context is an interview with Ken Lawson, an urban planner at MIT. Lawson argues that self-driving taxis will be great, but they won’t last for that long.
Most trips in the city, he said, involve individuals moving around their own neighborhoods far below the maximum speeds of cars.
“Why have a 4,000-pound automobile that seats five to move one person a short distance at low speed?” he said.
And this:
“It’s just like, you had the horse-and-buggy,” he said. “You got rid of the horse — it still looked like a buggy.”
Lawson doesn’t reveal what he thinks the self-driving equivalent of the Model T will be, though.
David Silver 