Tag: Autonomous Vehicles

Photorealism of Microsoft AirSim

Over the last year, a number of companies (including Udacity) have released self-driving car simulators powered by gaming engines.

The latest entrant is Microsoft, which has updated their open-source AirSim flight program to also support self-driving cars.

AirSim looks awesome. The big advantages of building off of a gaming engine (AirSim uses Unreal Engine, whereas the Udacity simulator uses Unity) include fully baked APIs, powerful physics engines, and incredibly realistic design and graphics.

That last item what will ultimately make or brake AirSim, or any other simulation engine.

The holy grail of autonomous vehicle simulation is the ability to train machine learning models in the simulator, and then port them to the real world. Once a simulator breaks that barrier, we should see incredibly fast improvements in our ability to build autonomous driving systems, as it’s exponentially faster to drive “simulated” miles compared “real” miles.

As photorealistic as AirSim is, it doesn’t yet look to me like it’s realistic enough to reliably move models between AirSims photorealistic environment and the actual, real environment.

That said, I doubt it’s possible to determine model portability with much confidence simply by eyeballing YouTube videos of the simulator, which is all I’ve done so far.

I look forward to people trying out AirSim models in the real world and seeing how they do.

Roundup of Autonomous Vehicle News

I was on vacation last week and it was delightful. But despite valiant struggles, I was not able to fully stay on top of the latest news in the autonomous vehicle world.

Here’s what I missed:

Everything we learned from the Tesla Semi and Roadster event

Zac Estrada
The Verge

The Tesla Semi drew excitement from the crowd at the Hawthorne, California facility, as people eagerly waited for Musk to emerge from the big truck. But the surprise showing of the second-generation Tesla Roadster caused explosive cheers from the second its headlights switched on.

GM Challenges Tesla With Promise of Profitable Electric Cars

Paul Lienert
Reuters

Barra said GM aims to be selling 1 million electric vehicles a year by 2026, many of them in China, which has set strict production quotas on such vehicles. On Monday, GM’s China chief said the automaker and its joint-venture partners will be able to meet the country’s 2019 electric vehicle requirements without purchasing credits from other companies.

Mercedes-Benz opens tech hub in Tel Aviv to secure lead in connected cars

Shoshanna Solomon
The Times of Israel

The Mercedes-Benz team in Israel will both develop in-house technologies and scout the ecosystem for products that could be integrated into their pipeline, either through acquisitions, long-term co-operations with startups, or investments.

Jaguar Land Rover self-driving cars hit real roads for first time

Andrew Krok
CNET

Jaguar Land Rover announced Friday that it will test its self-driving vehicles on public roads in the United Kingdom. Its vehicles will amble around Coventry as its engineers assess the systems and prepare this technology for an eventual public debut — which is still years away, it should be noted.

Training Self-Driving Car Engineers in India

Udacity and Infosys partner to teach autonomous technology!

Udacity and Infosys just announced a partnership to train hundreds of Infosys’ top software engineers in autonomous vehicle development.

Quoting Infosys President Ravi Kumar:

Udacity and Infosys are uniting the elements of education and transformative technology in this one-of-a-kind program. Trainees, with the first 100 selected through a global hackathon in late November, will immerse themselves in autonomous technology courses that require hands-on training to simulate real-life scenarios. By the end of 2018, Infosys will have trained 500 employees on the spectrum of technologies that go into building self-driving vehicles, and in doing so will help to evolve the future of transportation for drivers, commuters and even mass transit systems.

And Udacity CEO Vishal Makhijani:

This program will be part of Udacity Connect, which is Udacity’s in-person, blended learning program. Infosys engineers from around the world will participate in Udacity’s online Self-Driving Car Engineer Nanodegree program, and combine one term of online studies with two terms of being physically located together at the Infosys Mysore training facility, where the program will be facilitated by an in-person Udacity session lead.

Two aspects of this partnership are particularly exciting for me. One is simply working with a top technology company like Infosys. When we started building the Nanodegree program, our objective was to “become the industry standard for training self-driving car engineers.” This partnership moves us significantly closer to that objective. We are grateful and excited for the opportunity, and thrilled for the participating engineers.

The other exciting aspect of this partnership is that it will happen in India. The Infosys engineers will fly in from all over the world, but there is something special about conducting the program in Mysore.

For many years autonomous vehicle development has happened in just a few places: Detroit, Pittsburgh, southern Germany. Recently, we’ve seen autonomous vehicle development expand to Silicon Valley, Japan, Israel, various parts of Europe, Singapore, and beyond. Training autonomous vehicle engineers in India expands the opportunities for students worldwide.

7% of students in the Udacity Self-Driving Car Engineer Nanodegree program are from India. The Infosys partnership is an important next step in building a robust pipeline of job opportunities for our students on the subcontinent.

Dominik Nuss at Mercedes-Benz

One of the world-class experts in our Self-Driving Car Engineer Nanodegree program!

Me and Andrei Vatavu and Dominik Nuss

One of the delights of teaching at Udacity is the opportunity to work with world-class experts who are excited about sharing their knowledge with our students.

We have the great fortune of working with Mercedes-Benz Research and Development North America (MBRDNA) to build the Self-Driving Car Engineer Nanodegree Program. In particular, we get to work with Dominik Nuss, principal engineer on their sensor fusion team.

In these two videos, Dominik explains how unscented Kalman filters fuse together data from multiple sensors across time:

These are just a small part of a much larger unscented Kalman filter lesson that Dominik teaches. This is an advanced, complex topic I haven’t seen covered nearly as well anywhere else.

MBRDNA has just published a terrific profile of Dominik, along with a nifty video of him operating one of the Mercedes-Benz autonomous vehicles.

Read the whole thing and learn what it’s like to work on one of the top teams in the industry. Then, enroll in our program (if you haven’t already!), and start building your OWN future in this amazing field!

A Sunny Laboratory of Democracy

There’s a reason every company is testing self-driving cars in Arizona. It’s sunny. It’s warm. It’s flat (at least around Phoenix).

And according to The New York Times, Arizona Governor Doug Ducey is excited about making Arizona a leader in autonomous vehicle testing.

While Uber and Waymo were working through regulatory barriers testing in California, Ducey recruited them to Arizona with an “open for business” attitude.

“We responded by saying we weren’t going to hassle them,” Mr. Ducey said of Uber. “I’d be remiss if I didn’t thank my partner in growing the Arizona economy, Jerry Brown”, the Democratic governor of California.

The article closes with several anecdotes of human drivers crashing into self-driving cars, because that’s what human drivers do, and seizes on those anecdotes to suggest Arizona isn’t ready for self-driving cars.

I’m not sold.

Louis Brandeis once postulated that the beauty of American federalism is that each state is its own little laboratory of democracy, experimenting on its own, without risk to the rest of the country.

God bless Arizona for that.

Waymo Goes Driverless

With self-driving cars already being tested in cities across the United States and in several parts of the world, there have been three big questions about how quickly self-driving cars would expand:

  1. How quickly will the geofences around the (usually urban) test areas expand?
  2. When will companies open their services to the general public?
  3. How soon will companies pull the test driver from the vehicle?

Waymo just went ahead and answered #3. In a blog post and accompanying video (above), Waymo just announced that they have pulled the driver out of the seat on a subset of their test vehicles in the Phoenix, Arizona, metro area.

This looks like the latest step in a campaign by Waymo to both step forward in their self-driving efforts, and reassure the public that everything will be okay. And it looks like everything will be okay.

To that end, Waymo has invited reporters to their previously top-secret Castle test facility, and published a 43-page online safety brochure, alongside a slew of Medium posts.

A few thoughts of my own to accompany the Waymo announcement:

  1. This is awesome, and it has the potential to be huge if Waymo continues to roll this out to the rest of their test fleet in a timely manner.
  2. Waymo doesn’t say it, but I have to believe that, for now, they have test engineers near the driverless vehicles. They might be in trailing vehicles or at some sort of central command point to which the driverless vehicles are geofenced. I wouldn’t want an accident to happen (even an accident that’s not Waymo’s fault) and have civilian passengers be the first ones to talk with police and the press.
  3. As I understand it, these rides are carrying civilian, non-Waymo employees, but they’re also pre-screened for the program. The next step for Waymo will be what Uber has already done in Pittsburgh: open the program up to anybody who downloads the app.

It’s an exciting time for self-driving cars 😀

The “Introduction to Neural Networks” Lesson

Editor’s note: On November 1st of this year, David Silver (Program Lead for Udacity’s Self-Driving Car Engineer Nanodegree program) made a pledge to write a new post for each of the 67 lessons currently in the program. We check in with him today as he introduces us to Lesson 4!

The 4th lesson of the Udacity Self-Driving Car Engineer Nanodegree Program introduces students to neural networks, a powerful machine learning tool.

This is a fast lesson that covers the basic mechanics of machine learning and how neural networks operate. We save a lot of the details for later lessons.

My colleague Luis Serrano starts with a quick overview of how regression and gradient descent work. These are foundational machine learning concepts that almost any machine learning tool builds from.

Luis is great at this stuff. I love Mt. Errorest.

Moving on from these lessons, Luis goes deeper into the distinction between linear and logistic regression and then explores how these concepts can reveal the principles behind a basic neural network.

See the slash between the red and green colors there? If you ever meet Luis in person, ask him to sing you the forward-slash-backward-slash alphabet song. It’s amazing.

From here we introduce perceptrons, which historically were the precursor to the “artificial neurons” that make up a neural network.

As we string together lots of these perceptrons, or “artificial neurons”, my colleague Mat Leonard shows that we can take advantage of a process called backpropagation, that helps train the network to perform a task.

And that’s basically what a neural network is: a machine learning tool built from layers of artificial neurons, which takes an input and produces an output, trained via backpropagation.

This lesson has 23 concepts (pages), so there’s a lot more to it than the 3 videos I posted here. If some of this looks confusing, don’t worry! There’s a lot more detail in the lesson, as well as lots of quizzes to help make sure you get it.

If you find neural networks interesting in their own right, perhaps you should sign up for Udacity’s Deep Learning Nanodegree Foundation Program. And if you find them interesting for how they can help us build a self-driving car, then of course you should apply to join the Udacity Self-Driving Car Nanodegree Program!

The “Career Services Available to You” Lesson

The guiding star of the Udacity Self-Driving Car Engineer Nanodegree Program is jobs. Everything we do ultimately connects to preparing students to become autonomous vehicle engineers.

For that reason, Udacity has invested heavily in career support for students. Every student has access to optional projects where they can get personalized reviews of résumés and cover letters, as well as guidance for online profiles on sites liked LinkedIn, GitHub, and Udacity’s own Career Portal.

In this lesson, students hear from our Careers Team about the services and extracurricular professional activities that Udacity offers for students.

There are also videos from three of our content partners in the Nanodegree — Mercedes-Benz, NVIDIA, and Uber ATG — explaining what it’s like to work with them, how to get a job with them, and the value the Nanodegree Program provides.

We also have pointers to extracurricular lessons that are available to all Nanodegree students on two topics: “Job Search Strategies”, and “Networking”.

The “Job Search Strategies” lesson covers how to build a résumé and cover letter tailored to a specific job, as well as strategies for finding that job.

The “Networking” lesson offers tips for building your personal brand and developing a network that can push job opportunities in your direction. There are also optional projects through which you can get personal reviews of your GitHub, LinkedIn, and Udacity profiles.

The “Finding Lane Lines” Project

Udacity Self-Driving Car Engineer Nanodegree program

The second lesson of the Udacity Self-Driving Car Nanodegree program is actually a lesson followed by a project. In “Finding Lane Lines”, my colleague Ryan Keenan and I teach students how to use computer vision to extract lane lines from a video of a car driving down the road.

Students are able to use this approach to find lane lines within the first week of the Nanodegree program! This isn’t the only way to find lane lines, and with modern machine learning algorithms it’s no longer the absolute best way to find lane lines. But it’s pretty effective, and it’s amazing how quickly you can get going with this approach.

Here’s a photo of Interstate 280, taken from Carla, Udacity’s own self-driving car:

The first thing we’re going to do is convert the image to grayscale, which will make it easier to work with, since we’ll only have one color channel:

Next, we’ll perform “Canny edge detection” to identify edges in the image. An edge is place where the color or intensity of the image changes sharply:

Now that we have the edges of the image identified, we can use a technique called a “Hough transform” to find lines in the image that might be the lane lines we are looking for:

All of these tools have various parameters we can tune: how sharp should the edges be, how long should the lines be, what should the slope of the line be. If we tune the parameters just right, we can get a lock on our lane lines:

Apply these lane lines to the original image, and you get something like this “Finding Lane Lines” project, submitted by our student Jeremy Shannon:

Pretty awesome for the first week!

The “Welcome” Lesson

Udacity Self-Driving Car Engineer Nanodegree program

“Welcome” is the first of 20 lessons in Term 1 of the Udacity Self-Driving Car Engineer Nanodegree program.

This is an overview lesson in which we introduce:

We also cover the history of self-driving cars, the logistics of how Udacity and this Nanodegree program work, and the projects that students will build throughout the program.

I’ll let Sebastian share that last bit:

Next up, the “Finding Lane Lines” project!