A pioneering training center, developed jointly with Tier IV and PIX, provides an opportunity for students to work in teams on their own self-driving car.
Last week I had the privilege of visiting Udacityâs Self-Driving Car Training Center in Guiyang, China.
This is a facility that our colleagues on Udacityâs China team have developed jointly with Tier IV (the creators of Autoware) and PIX (an autonomous vehicle startup in Guiyang). The center provides an opportunity for students from all over China to come together and work in teams for a week on their own self-driving car. Over the course of the program, they install all of their own software and get the car driving itself around a test track, stopping at traffic lights and stop lines.
This is such an amazing program!
Bringing together teams of students to work in-person on a self-driving car is a tremendous experience that was exceptionally valuable for the students at the Training Center.
Watching and participating with students working on self-driving cars in western China, I was reminded of how huge the talent pool is, all over the world, for people who want to build autonomous vehicles.
Check out this video of one of the teamâs getting their car to drive!
As we continue to move forward into the future of autonomous transportation, opportunities to experience firsthand the migration from theory to practiceâand from online to on the streetâwill become ever more valuable to then engineers engaged in making this future a reality. I am excited that Udacityâs China team is helping to make this kind of experience possible for aspiring autonomous engineers, and I am grateful to have had the opportunity to watch it all in action.
Are you interested in joining the next generation of talent building the future of autonomous transportation? Visit our School of Autonomous Systems today!
Aspiring Self-Driving Car Engineers in India, apply today for the opportunity to work on autonomous vehicles, regardless of your financial situation!
KPIT, one of Indiaâs leading automotive software suppliers, just announced they are sponsoring 500 scholarships for Indian students to take Udacityâs Self-Driving Car Engineer Nanodegree program!
Since we launched the Nanodegree program two years ago, we have seen tremendous interest from students in India who want to learn about autonomous vehicles. Many of the Indian students who have enrolled in the Nanodegree program now work on autonomous vehicles at great Indian companies like KPIT, Ola, and Hi-Tech Robotics.
The KPIT Scholarships will provide the opportunity for any student in India to work on self-driving cars, regardless of their financial situation.
KPIT is making a tremendous investment in Indian software engineers. We are delighted to be able to work with Kishor Patil and the KPIT team to make this possible!
See the prototype in actionâ10km of autonomous highway driving!
One of the great joys of teaching self-driving cars at Udacity is watching the amazing things our students build. Iâm based in Silicon Valley, but our students come from all over the world, and I have the opportunity on a daily basis to experience the truly global nature of this transportation revolution.
One of our Udacity self-driving car graduates, Shilpaj Bhalerao, is the tech lead for a small team building self-driving trucks in India. This is no small feat. India is even more of a challenge for self-driving cars than other parts of the world, due to the complexity of traffic norms, and the infrastructure quality. So itâs super-exciting to watch their 10km autonomous trucking run in action.
Check it out.
No matter where you are in the world, if your dream is to contribute to the future of autonomous transportation, then you can acquire the skills you need with Udacity. Check out our School of Autonomous Systems today.
This Thursday, August 23rd, at 9am Pacific Time, I will be hosting an online open house for Udacityâs School of Autonomous Systems. RSVP now to join me, my Udacity colleagues, our alumni, and other potential students, and learn about our many exciting programs!
At the Open House, Iâll share an overview of each program, compare them, and describe the careers for which each option prepares you.
Weâll finish with a live question-and-answer session, co-hosted by myself and several of my Udacity instructional colleagues, to answer as many of your questions as we can.
RSVP now to join us on Thursday! And donât worry if you canât make it! RSVP via the link, and weâll send you the recording of the open house so you donât miss out!
Self-Driving Fundamentals: Featuring Apollo is a terrific, free introduction to how self-driving cars work, through the lens of the Apollo open-source self-driving car project.
Itâs perfect for beginners who are in the exploratory phase of their autonomous systems journey, and it does a great job demonstrating how existing basic programming and data skills can be applied to this field.
Upon successfully completing the program, youâll be ready to combine your newly-acquired self-driving car fundamentals with your existing programming skills to enroll in our Intro to Self-Driving Cars Nanodegree program.
The course begins with an overview of self-driving cars, and how they work. From there, we dive into Apollo and itâs multi-layer architecture: reference vehicle, reference hardware, open-source software, and cloud services.
This is a great opportunity to learn the key parts of self-driving cars, and get to know the Apollo architecture. Youâll utilize Apollo HD maps, localization, perception, prediction, planning and control, and start learning the fundamentals of how to build self-driving cars.
Get the inside scoop on how self-driving cars work, and learn about career opportunities in the autonomous vehicle industry!
A few days ago I hosted a webinar with my colleague Jane Sheppard about how self-driving cars work, and about career opportunities in the autonomous vehicle industry. If youâre interested in self-driving cars, you should watch đ
I talk about business models in the self-driving car space, run down whoâs focused on hardware vs. software, and describe all the players in the arena, from the smallest startups to the biggest global organizations (you wouldnât believe how much money some of the big companies are putting into this technology!). I specifically discuss Teslaâs consumer sales model, and the impact and involvement of ride-sharing companiesâand thatâs all before I even get to the core technologies that make self-driving cars possible đ
If you want to get the low-down on perception, localization, and planning, just hit play below!
âIf sensor fusion and computer vision tell you what the world looks like, localization tells you where you are in that world.â
In these projects, students showcase initiative, creativity, and work ethic, as they build projects focused on topics like perception, deep learning, and computer vision.
Their intense interest in these topics translates directly to the high quality of their work. Today, Iâd like to share three especially impressive student projects with you that cover these areas!
I love this project! Mohammad did it on his own, and he went way beyond the requirements of the Nanodegree program to do so. Thatâs going to serve him well in the long run, because employers love it when talented students get so deep into particular subjects, that they start building their own projects to further flesh out their ideas and test their skills.
âIn this project I have used a pre-trained ResNet50 network, removed its classifier layers so it becomes a feature extractor, and then added the YOLO classifier layer instead (randomly initialized). I then trained the network on Udacityâs CrowdAI dataset to detect cars in video frames.â
Enrique used VGG-16 to create a semantic segmentation network for the Advanced Deep Learning project in the Nanodegree program. He trained that network using the KITTI dataset, and then applied the network to scenes he recorded driving in Mexico. Check out his YouTube videos!
âThe original FCN-8s was trained in stages. The authors later uploaded a version that was trained all at once to their GitHub repo. The version in the GitHub repo has one important difference: The outputs of pooling layers 3 and 4 are scaled before they are fed into the 1×1 convolutions. As a result, some students have found that the model learns much better with the scaling layers included. The model may not converge substantially faster, but may reach a higher IoU and accuracy.â
Moataz built a vehicle detection pipeline combining histogram of oriented gradients, support vector machines, and sliding window search. I particularly like the heatmap he employs to reduce false positives in vehicle detection. This is a great example of going beyond the steps outlined in the Nanodegree program, to build a truly standout project.
âNow given the simplicity of the SVM model, we expect some detections to be false positives. In order to filter out these incorrect detections, one approach is to threshold our positive windows such that we only pick areas where more than one window overlap. In essence we are generating a heatmap of the positive windows.â
Udacity practices project-based learning, which means all of our students in all of our Nanodegree programs build projects like these. This approach enables you to learn practical skills, and to build a dynamic portfolio populated with completed projects that clearly showcase your new skills and experience.
Since we launched our Self-Driving Car Engineer Nanodegree program in late 2016, nearly 2,000 students have completed the program, and more new graduates are joining them every month.
Not all students enroll in the program specifically to find a new job, but many do, and itâs exciting to see a new generation of talent entering this field. With that in mind, weâd like to introduce you to some of these alums!
One of our early graduates, Robert Ioffe, transitioned within his company, Intel, to the role of Senior Self-Driving Car Software Engineer shortly after enrolling in our Nanodegree Program. Since then, Intel has announced its plan to begin testing 100 self-driving cars in Jerusalem, and eventually in the U.S.
âThe coolest thing is that everything I learn in the class is immediately applicable to my current job, which is building a self-driving Range Rover with Intel technology inside. It is very rare where you can learn things in class one day, and the next day you can apply it in your work!â
Megha Maheshwari, who immigrated to the U.S. from India, began her career as a software engineer, but ultimately landed a position at Volvo Cars as an Autonomous Driving, Deep Learning, and Computer Vision Engineer. Volvo is investing heavily in fully electric and self-driving vehicles, which many industry analysts believe is part of its plan to go public.
âWhen I was ending Term One, I felt I had enough knowledge about classical computer vision and deep learning. Thatâs when I started looking for jobs and not long after looking and applying, I got hired.â
Udacity grads are also launching startups to capitalize on the growing opportunities in the autonomous vehicle market. After earning their Nanodegree credentials, alums David Hayes and Duncan Iglesias formed the Autonomous Vehicle Organization, or AVO, to increase safety and security by focusing on the vehicle-to-pedestrian (V2P) segment.
Using skills developed in the course, David and Duncan built a semi-autonomous Honda Civic in just 11 days!
Fellow alum Patrick Kern also co-founded a startup, with a different focus. Brighter AI was launched to develop âDeep Natural Anonymizationâ to help companies comply with new General Data Protection Regulations (GDPR) in Europe.
And just last month, we profiled Han Bin Lee, who teamed up with fellow students he met while working on the Udacity-Didi challenge to start Seoul Robotics. Expanding on the Lidar perception software, they began building during the challenge, and today, the company is already looking for a 3D Vision Researcher and a C++ Software Developer to join their crew in South Korea!
âWeâre looking for people with the willingness and ability to learn new concepts and algorithms from the latest research. And weâre a startup, so we need team players, who are able to work effectively within a fast-growing, diverse group of peopleâââwe have Korean, Vietnamese, American, and Irish people working with us so far. We are really short of people right now, and we would love to work with fellow Udacity alumni to build this company together!â
Because this is still an emerging field, many of our early students didnât start out wanting to become Self-Driving Car Engineers. Thatâs changing rapidly as people are realizing how much opportunity there is in this space, but for lifelong learners like Kyle Martin, the decision to enter this field was the culmination of a really interesting journey:
âI started looking for an industry role while I was still in the program. A lead robotics engineer role appeared with a company that was beginning to work on an autonomous shuttle. I jumped at it and got an interview! They were interested in all the areas Iâd been working onâââthings like computer vision and systems architecture. And they were really impressed Iâd kept learning and adding to my skills in the program. When they made me an offer, I said âyesâ immediatelyâââit sounded like Iâd have the opportunity to work on really groundbreaking projects.â
These are just a few examples of how alumni of our autonomous transportation programs are having an impact on this incredible field. As our alumni network continues to grow, weâre excited to help more students find positions in the industry. Stay tuned!
These successes have taught us a great deal about what you need to know in order to accomplish your goals, and to advance your career. In particular, weâve learned that by narrowing the breadth of the program, and expanding opportunities to go deep in specific areas, we can better offer a path that is expressly tailored to support your career journey.
To that end, weâre updating the curriculum for the program to focus on fundamental skills in each core area of the self-driving car stack. Iâd like to share some details with you about this important update, and about the changes weâve made.
Term 1
Introduction
Welcome In our introduction, youâll begin by meeting your instructorsâââSebastian Thrun, Ryan Keenan, and myself. Youâll learn about the systems that comprise a self-driving car, and the structure of the program as a whole.
Workspaces Udacityâs new in-browser programming editor moves you straight to programming, and past any challenges related to installing and configuring dependencies.
Computer Vision
Computer Vision Fundamentals Here, youâll use OpenCV image analysis techniques to identify lines, including Hough transforms and Canny edge detection.
Project: Detect Lane Lines This is really excitingâyouâll detect highway lane lines from a video stream in your very first week in the program!
Advanced Computer Vision This is where youâll explore the physics of cameras, and learn how to calibrate, undistort, and transform images. Youâll study advanced techniques for lane detection with curved roads, adverse weather, and varied lighting.
Project: Advanced Lane Detection
EIn this project, youâll detect lane lines in a variety of conditions, including changing road surfaces, curved roads, and variable lighting. Youâll use OpenCV to implement camera calibration and image transforms, as well as apply filters, polynomial fits, and splines.
Deep Learning
Neural Networks Here, youâll survey the basics of neural networks, including regression, classification, perceptrons, and backpropagation.
TensorFlow Next up, youâll train a logistic classifier using TensorFlow. And, youâll implement related techniques, such as softmax probabilities and regularization.
Deep Neural Networks This is where youâll combine activation functions, backpropagation, and regularization, all using TensorFlow.
Convolutional Neural Networks Next, youâll study the building blocks of convolutional neural networks, which are especially well-suited to extracting data from camera images. In particular, youâll learn about filters, stride, and pooling.
Project: Traffic Sign Classifier
For this project, youâll implement and train a convolutional neural network to classify traffic signs. Youâll use validation sets, pooling, and dropout to design a network architecture and improve performance.
Keras This will be your opportunity to build a multi-layer convolutional network in Keras. And, youâll compare the simplicity of Keras to the flexibility of TensorFlow.
Transfer Learning Here, youâll fine tune pre-trained networks to apply them to your own problems. Youâll study cannonical networks such as AlexNet, VGG, GoogLeNet, and ResNet.
Project: Behavioral Cloning
For this project, youâll architect and train a deep neural network to drive a car in a simulator. Youâll collect your own training data, and use it to clone your own driving behavior on a test track.
Career Development
GitHub For this career-focused project, youâll get support and guidance on how to polish your portfolio of GitHub repositories. Hiring managers and recruiters will often explore your GitHub portfolio before an interview. So itâs important to create a professional appearance, make it easy to navigate, and ensure it showcases the full measure of your skills and experience.
Sensor Fusion
Our terms are broken out into modules, which are in turn comprised of a series of focused lessons. This Sensor Fusion module is built with our partners at Mercedes-Benz. The team at Mercedes-Benz is amazing. They are world-class automotive engineers applying autonomous vehicle techniques to some of the finest vehicles in the world. They are also Udacity hiring partners, which means the curriculum weâve developed is expressly designed to nurture and advance the kind of talent theyâre eager to hire!
Sensors The first lesson of the Sensor Fusion Module covers the physics of two of the most import sensors on an autonomous vehicleâââradar and lidar.
Kalman Filters Kalman filters are a key mathematical tool for fusing together data. Youâll implement these filters in Python to combine measurements from a single sensor over time.
C++ Checkpoint This is a chance to test your knowledge of C++ to evaluate your readiness for the upcoming projects.
Geometry and Trigonometry Before advancing further, youâll get a refresh on your knowledge of the fundamental geometric and trigonometric functions that are necessary to model vehicular motion.
Extended Kalman Filters Extended Kalman Filters (EKFs) are used by autonomous vehicle engineers to combine measurements from multiple sensors into a non-linear model. First, youâll learn the physics and mathematics behind vehicular motion. Then, youâll combine that knowledge with an extended Kalman filter to estimate the positions of other vehicles on the road.
Project: Extended Kalman Filters in C++
For this project, youâll use data from multiple sensors to track a vehicleâs motion, and estimate its location with precision. Building an EKF is an impressive skill to show an employer.
Term 2
Localization
This module is also built with our partners at Mercedes-Benz, who employ cutting-edge localization techniques in their own autonomous vehicles. Together we show students how to implement and use foundational algorithms that every localization engineer needs to know.
Introduction to Localization In this intro, youâll study how motion and probability affect your understanding of where you are in the world.
Markov Localization Here, youâll use a Bayesian filter to localize the vehicle in a simplified environment.
Motion Models Next, youâll learn basic models for vehicle movements, including the bicycle model. Youâll estimate the position of the car over time given different sensor data.
Particle Filter Next, youâll use a probabilistic sampling technique known as a particle filter to localize the vehicle in a complex environment.
Implementation of a Particle Filter To prepare for your project, youâll implement a particle filter in C++.
Project: Kidnapped Vehicle
For your actual project, youâll implement a particle filter to take real-world data and localize a lost vehicle.
Planning
Search First, youâll learn to search the environment for paths to navigate the vehicle to its goal.
Prediction Then, youâll estimate where other vehicles on the road will be in the future, utilizing both models and data.
Behavior Planning Next, youâll model your vehicles behavior choices using a finite state machine. Youâll construct a cost function to determine which state to move to next.
Trajectory Generation Here, youâll sample the motion space, and optimize a trajectory for the vehicle to execute its behavior.
Project: Highway Driving
For your project, youâll program a planner to navigate your vehicle through traffic on a highway. Pro tip: Make sure you adhere to the speed, acceleration, and jerk constraints!
Control
Control Youâll begin by build control systems to actuate a vehicle to move it on a path.
Project: PID Control
Then, youâll implement the classic closed-loop controllerâââa proportional-integral-derivative control system.
Career Development
Build Your Online Presence Here, youâll continue to develop your professional brand, with the goal of making it easy for employers to understand why you are the best candidate for their job.
System Integration
Autonomous Vehicle Architecture Get ready! Itâs time to earn the system architecture of Carla, Udacityâs own self-driving car!
Introduction to ROS Here, youâll navigate Robot Operating System (ROS) to send and receive messages, and perform basic commands.
Packages & Catkin Workspaces Next, youâll create and prepare an ROS package so that you are ready to deploy code on Carla.
Writing ROS Nodes The, youâll develop ROS nodes to perform specific vehicle functions, like image classification or motion control.
Project: Program an Autonomous Vehicle
Finally, for your last project, youâll deploy your teamsâ code to Carla, a real self-driving car, and see how well it drives around the test track!
Graduation Congratulations! You did it!
By structuring our curriculum in this way, weâre able to offer you the opportunity to master critical skills in each core area of the self-driving car stack. Youâll establish the core foundations necessary to launch or advance your career, while simultaneously preparing yourself for more specialized and advanced study.
To try and answer this question, Iâll begin with a story. In October of 2016, Udacity welcomed the first class of students into our Self-Driving Car Engineer Nanodegree program. Since that historic debut, we have been delighted to enroll over 11,000 students around the world in this program!
Along the way, we learned that while people across the globe were thrilled at the prospect of being able to work on autonomous vehicles, not all of them were equipped to do soâmany of them needed additional training to get ready for the rigors and challenges of our curriculum.
In order to provide a viable point-of-entry for these eager learners, we built the Intro to Self-Driving Cars Nanodegree Program, and welcomed the first class of students at the end of 2017. This âIntroâ program prepares students with the fundamentals in Python, C++, calculus, linear algebra, statistics, and physics that are necessary to become a Self-Driving Car Engineer.
Both Nanodegree programs are paths to a career in the self-driving car field, but the goals of each program are distinct, as are the skills one learns.
The Self-Driving Car Engineer (SDC) Nanodegree program is an advanced program in which students write programs in Python and C++, and learn new frameworks like ROS and TensorFlow. Students entering SDC should be able to write programs from scratch, and should be comfortable with both calculus and linear algebra. SDC does not require solving differential equations by hand, but does require that students be comfortable interpreting mathematical notation and translating it into code.
The Intro to Self-Driving Cars (iSDC) Nanodegree program is an intermediate program that requires entering students to have only minimal programming and math knowledge. Students entering iSDC should be comfortable reading and modifying code in at least one language (Python helps, since that is first language the program uses). Entering students should also be comfortable with high-school algebra. From there, iSDC teaches the trigonometry, calculus, linear algebra, statistics, and physics that are necessary to succeed in the advanced SDC program.
iSDC does not require an application to enroll, and everybody is welcome. However, students with no programming experience at all might consider starting their journey with Udacityâs Intro to Programming Nanodegree program, and then proceeding on to Intro to Self-Driving Cars. A slightly more mathematical (and more challenging) alternative first step would be Udacityâs Data Analyst Nanodegree Program.
David Silver 