Deep Fakes

A colleague had to miss a work co-presentation yesterday, so I made some deep fakes and inserted him in the presentation virtually. It was kind of a riot.

I was surprised how much effort this turned out to be – I kind of thought there would be a bunch of turn-key deepfake services to use. Instead, I found a Google Colab and wound up doing a lot of manual video and audio editing, and it turned out kind of comically bad.

Deepfakes still seem to require a fair amount of artistic ability, in a sense.

Nonetheless, it was a lot of fun.

Here’s a kind of comically bad deepfake of my brother Adam spouting something about Python.

Oliver Cameron, Halifax Lad

The Times of London has a fun (but paywalled) article on, and a podcast interview with, my former Udacity and Voyage boss, and current Cruise colleague, Oliver Cameron.

The university dropout from Halifax, West Yorkshire, learned to code as a child. As a teenager, he built an app that hit No 1 in Apple’s App Store, and another that was featured in the iPhone’s giant TV adverts. And at 21, he landed a spot at the vaunted start-up boot camp Y Combinator, which brought him to California. He never left.

The autodidact went on to create courses in flying cars and virtual reality for the billion-dollar education start-up Udacity — despite no formal training in those fields. He founded the self-driving car firm Voyage and in March sold it to Cruise, a subsidiary of motor giant GM.

I first spoke with Oliver years ago when applying for an engineering position at Udacity. I didn’t get the job, but a year later I was at Ford, working on self-driving cars, when Oliver wrote me out of the blue. He asked for advice on how to teach self-driving cars at Udacity.

I met with Oliver for a few weeks, suggesting a curriculum and projects. Eventually Oliver recruited me to join Udacity and teach the program myself, together with Sebastian Thrun, the founder of both Udacity and Waymo.

While I built the Udacity Self-Driving Car Engineer Nanodegree Program, Oliver incubated a self-driving startup called Voyage, within Udacity. Eventually, Voyage spun out as its own endeavor.

Years later, I wrapped up my own Udacity work and followed Oliver to Voyage, and now to Cruise.

I’ve had a great experience working with and learning from the lad from Halifax.

200,000 Miles

I’ve never had a car reach 200,000 miles before! The Silver Family approach to car ownership is:

  1. Change oil regularly, at the cheapest location possible.
  2. Avoid all other types of maintenance.
  3. Buy a new car upon receiving a four-digit repair estimate.

As I’ve matured into my own person, I’ve thrown off the yolk of my parents’ approach and invested in scheduled maintenance.

Maybe this 2004 Toyota Highlander can reach 300,000 miles?

Happy Thanksgiving!

Thanksgiving | The Art Institute of Chicago
Thanksgiving by Doris Lee

Happy Thanksgiving!

I find this tweet from Judea Pearl surprisingly profound:

Transitioning From Research & Development To Production


Forbes Alan Transportation editor Alan Ohnsman has a short interview with Zoox CEO Aicha Evans that is mostly about vague restructuring at Zoox.

“The company..isn’t making headcount reductions as part of the business review, though some early Zoox employees are leaving, Evans tells Forbes.”

I find the kicker of the article really interesting.

Zoox currently has about 1,500 employees and isn’t planning to reduce that number though some “early, prominent Zoox people” are leaving the company, Evans said, without identifying individual team members. 

“We’re celebrating them, we’re thanking them for that first phase,” she said. For some, the company’s initial period was probably more fun than where the company finds itself now. “This phase, it’s a little less sexy, it’s more grindy. But this is what gives you the stripes to actually build a company that ships products and changes society.”

That seems correct to me. Different people thrive at different stages of a company. The folks who are interested in taking on a science project are often different than the people who want to manage the profit and loss statement of a business.

Science projects are also a lot riskier, so if one does pay off, it should pay off big. I don’t know if that happened at Zoox or not, due to their fundraising challenges. But presumably the current leaders seeing Zoox through to shipping product have a good sense of the business’s viability.

Facebook SEER

Last spring, Facebook published SEER, a new approach to self-supervised deep learning.

One of the core challenges for most deep learning efforts is securing labeled data. The neural network needs labeled data for training, so that the network can learn when it’s right and when it’s wrong, and how wrong it is, and then improve.

Unfortunately, lots of datasets don’t come with labels. The solution is often to pay a third-party vendor to ship the data to a country with low labor costs for manual human labeling. Even in very economical locations, this effort becomes very expensive. And surprisingly error-prone.

Over time, most companies have gotten smarter about how to automatically label a lot of data, but human labeling remains important.

Facebook’s SEER approach skips the labeling entirely, using a “self-supervised” approach to learn directly from the raw data. Instead of labeling different images with “cat”, “dog”, and other descriptors, SEER learns to correlate similar images together. The basic idea is to extract features from each image and then assign images with similar features to clusters.

The second contribution of SEER is an architecture for training a network at Facebook’s scale. The Facebook AI team behind this effort documents their use of RegNets (regulator networks) to trade off compute power for memory capacity, and scale the system.

Self-supervised learning seems like it might become important for robotics, and autonomous vehicles, particularly in the planning pipeline. This is an area in which it can be hard to even know what labels to assign to raw data. If we could instead design a system to let the network learn for itself, that would be a big step forward.

Brad Templeton On Radar


In Forbes, Brad Templeton has a great article on the strengths and weaknesses of radar, relative to other sensors, and why so many companies seem to be interested in it right now. He pegs the analysis on Waymo’s recent radar announcement, but Templeton does a really nice job covering lots of different aspects of radar generally, including how Tesla and Mobileye do (and do not) use this sensor.

Because radar tells you how fast a target is moving towards or away from you, those targets stand out from all the stationary things in the world. You get reflections from stationary objects (like a stalled car in front of you) but it’s hard to tell reliably from all the other stationary things — like the road, the fences, the signs and more. Early radar users had to just ignore any returns from fixed objects, which is why you saw radar-equipped Teslas plow into the side of trucks crossing the road and emergency vehicles stopped in the left lane.

Definitely worth a read.

Autonomous Vehicle ETFs

The recent SPACs of Aurora and Embark, on top of all of the other lidar and EV and eVTOL and AV companies that have gone public over the last year, got me wondering what the financial returns might be on a basket of these stocks.

And as soon as I wondered that, I suspected Wall Street had already created such a basket.

The answer turns out to be, sort of. There are a handful of ETFs that are roughly in the “next generation mobility” space, but a big confounder seems to be exposure to Tesla stock. Tesla stock has just been so successful over the last 12 months (last 10 years, really), that it seems that the performance of these ETFs must be dominated by how much Tesla stock they hold.

Indeed, in the chart above, you can see purple Tesla ticker is just way beyond every other option, whereas the Russell 3000 (up 33% in one of its best years ever!) lags the pack. Probably the Russell 3000 is a lot less exposed to Tesla than the future mobility ETFs.

Here is a different graph that plots some of the individual new mobility stocks over the last six months. You can see it’s kind of a mixed bag, with Tesla and NIO, the only two consumer-facing companies on the chart, performing quite well.

Seems like this should almost go without saying, but I’ll say it. None of this is investment advice. My personal financial portfolio consists of none of these stocks, but rather is almost entirely built on low-cost Vanguard Target Retirement Funds.

This is all just an interesting curiosity to me.

Hello South Bay

Cruise is coming to the South Bay! A new office will open at 840 W California in Sunnyvale, right next to downtown, the Caltrain, and Walmart Labs.

This is a big win for a lot of my former Voyage colleagues who are now Cruise colleagues. Voyage had a South Bay orientation, so returning to the office in 2022 was going to mean a long haul to San Francisco, for some of those folks.

The Mercury News reports that Cruise will have 400 employees in Sunnyvale, mostly engineering, so obviously this is about a lot more than just my former Voyage colleagues. Lots of Cruisers live in the South Bay, as indeed does most of the Silicon Valley automotive industry.

If you live in the South Bay and are interested in working for Cruise, email me your CV!

Under Cruise’s Hood

Last week Cruise conducted a 2.5 hour deep-dive into our technical architecture, with a particular focus our AV stack. The event was called Under The Hood, and now it’s available on YouTube for the whole world to watch.

You’ll see everything from how our perception stack operates, to planning, to the upcoming Cruise Origin, to how we’re going to roll out a ride-hailing service in San Francisco.

This is about as much information as any AV organization has ever put out about how self-driving cars work.

Check it out!