Autonomous self-driving cars keep inching ever closer to reality. More companies are getting into the self-driving game, but one big problem remains: Self-driving cars have a really difficult time identifying people on bikes. And they are even worse at predicting where those cyclists are headed.

Autonomous cars & their issues identifying cyclists

courtesy of Waymo

A couple of new reports out since the start of 2018 have summarized the huge progress that self-driving car technologies have made in the last year or two. On one side industry group IEEE has credited progress overall, but stagnation regarding cyclists. And then there is Waymo – the evolution of Google’s self-driving car project – that released a safety report showing the strides they’ve made, specifically calling out cyclist detection.

These autonomous vehicles are becoming smarter and much more accurate at identifying, tracking, and even predicting the movement of objects in their field of view. The problem remains, though, that progress doesn’t appear to be being made with regards to cyclists.

courtesy Uber

We reported on the same issue a little over a year ago when Uber admitted their own self-driving cars were having trouble identifying bike lanes. That made it difficult, both to keep the cars out of the lanes and to recognize that they should be carefully looking there for cyclists.

Cyclist detecting technologies

courtesy of Waymo

And a lot of that comes down to the most common technology that self-driving cars utilize – namely analyzing a constant inflow of two dimensional images to identify objects and their direction of travel. Many vehicles combine that optical sensing with additional tech like lidar & radar which help identify locations of objects. But those sensing systems are less effective at determining their direction of travel, limiting their effectiveness on their own.

Essentially the autonomous car sees the world as a succession of images, and software automatically models boxes around things like cars (green & purple), pedestrians (yellow), and cyclists (red) over the vehicle’s 3D basemap data.

Then it anticipates what happens next, so it can decide how to proceed driving.

Cyclists are a difficult thing to identify and track. We are less massive than other vehicles on the road. Plus, we are able to both move speedily with traffic and react quickly, changing direction faster than almost anything else in the roadway.

But maybe the biggest obstacle for the automated machine analysis made by self-driving cars is that cyclists are so individual. Whether we are roadies in skin tight lycra, mountain bikers in baggies, or a commuter in regular street clothes, we all look very different, even though we move in similar ways. While autonomous cars have effectively learned how to identify mostly standard looking cars by studying huge numbers of images, those images haven’t had a lot of cyclists in them, so there is much more learning to be had.

How can self-driving cars and cyclists co-exist?

Volvo Leads the Way to Safety, with First Cyclist Detection Braking System

courtesy Volvo

Some cycling & car industry minds have been developing technologies, using reflective paint, specially identifiable helmets, a smartphone app, and even bikes equipped with technology that communicates directly with the self-driving cars. That seems like a band-aid over the problem to us (and many others). It certainly could help those equipped with the special tech, but would ultimately leave anyone without it more vulnerable.

For now it seems like the best solution is more machine learning. The programmers behind the development of these self-driving cars need to get more images of cyclists into the robot brains of their cars, so they can make the right decisions.

The same problems exist with regular drivers who don’t always react properly to seeing cyclists on the road. (Not helped by some cyclists not obeying the rules of the road to start with.) Much like we need to use cycling advocacy to teach human drivers how to share the road, it seems we need to do the same with self-driving cars. And as more autonomous vehicles encounter more cyclists out on the road, they can better learn to identify us and operate safely around us.

Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

25 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Craig
Craig
3 years ago

Interesting article

Paul G
Paul G
3 years ago

They should look into re-creating a full-body skeleton in a similar way video game technology does with Kinect and gesture prediction. Because just analyzing 2D images and radar data seems like a limited approach to predict the path of a fast moving agile object. Having an 3D representation and an understanding of how the cyclist can turn would help. That’s how the human brain seems to work at least. If you see the bike lean right, but the cyclist still has his body weight centered, you know it’s possible he can still flick left at any moment. But not if his body weight also leans with the bike. Good luck predicting that with a box representation.

Michael Plakus
3 years ago

If you really want to get into the weeds try this article that oddly enough was published today. http://www.bikewalknc.org/2018/02/autonomous-driving-and-collision-avoidance-technology/

Robin
Robin
3 years ago

It’s still relatively early in autonomous car development. I’m confident this problem will be resolved. I don’t see the use of a signal broadcast by a device on a bike as a bandaid. Instead, it’s making use of technology that’s available. It’s not a bandaid when military aircraft use IFF (Identification friend or foe) to identify aircraft. It’s not bandaid when aircraft use transponders. It shouldn’t be difficult for an industry to develop a low cost transponder for bicycles and integrate a receiver that transponder in cars. Developing a common spec would allow bike manufacturers to directly integrate that transponder into bikes and offer transponders for older bikes.

Chris
Chris
3 years ago
Reply to  Robin

No, that’s not a solution. If your transponder stops working you won’t have air traffic control to warm you about it, you’ll just get run over.

JBikes
JBikes
3 years ago
Reply to  Robin

I too am confident that the problem will be resolved. However, the use of a broadcast device (active) or specific reflective type device (inactive) is a bandaid solution and a poor one at that. The instances where these are used are in well controlled environments with multiple back-up solutions.

Case in point. A kid grabs a bike that hasn’t been retrofitted, or the transponder malfunctions without their knowledge. They are now extremely likely to be hit.

Self driving cars should navigate and avoid objects as a human does…that is by observing surroundings and adjusting correctly to suit. With the fact that they can use sight modes i can’t (thermal, radar) and should never be tired, intoxicated, distracted, impatient, angry, poorly trained, etc, we should eventually see a truly safe vehicle.

Robin
Robin
3 years ago
Reply to  JBikes

I’d prefer a multi-mode detection scheme, given the apparent difficulty in detecting cyclists. Active detection (with some EM radiation frequency or possibly with multiple frequencies) coupled with passive detection (the transponders) would make for a robust system.

JBikes
JBikes
3 years ago
Reply to  Robin

I think you underestimate how robust a system can be without relying on these devices. The “apparent difficulty” is only now, at current tech. 20 years ago, nobody could fathom smartphones. Proper detection will improve.

The main problem is that these devices cannot inherently provide failure detection/alert to the vehicle that relies on them to not hit said object. Imagine one failing on our commute or a kids failing on their ride to school.

All control should be inherent to the vehicle where it can detect failure, and employ redundancy as required (or safely shutdown as needed). It should not rely on devices it has no control over and no way to run diagnostics on.

Robin
Robin
3 years ago
Reply to  JBikes

So you’re in favor of single failure. Interesting.

A transponder could in fact allow a vehicle to take corrective measures. It makes little difference whether a sensor is detecting a beam on a round trip flight or a bean on a one way flight (except for time of flight sensors, diffractive or interferometric sensors). A vehicle sensor can absolutely act on that incoming transponder beam, especially given that over a few micro seconds in can calculate the velocity of that transponder.

I also believe that it will part of the natural progression of autonomous vehicles that said vehicles will in some way communication with each other, even if that communication is just a transponder signal. I don’t want to see bicycles left out of that loop.

JBikes
JBikes
3 years ago
Reply to  Robin

Single failure? What part of redundant sensing systems on the car with fault detection logic and planned compensation/action don’t you understand?

Robin
Robin
3 years ago
Reply to  JBikes

Failure to detect the bicyclist. That’s the failure I’m talking about.

JBikes
JBikes
3 years ago
Reply to  Robin

a system designed/tested to be effective based on equipment mounted to a bike will fail to be effective if that bike mounted equipment fails or is not installed and there is no way for the vehicle to detect said failure or lack of equipment.

nopers is bopes
nopers is bopes
3 years ago

ill just note, living in SF and commuting on bike daily that some automated cars see me. theres a gazillion prototypes around here. some are super dangerous and some are quite good.

remember also that most cyclists basically throw themselves into cars fully expecting the car will stop (ive seen quite a few direct hits in the last 5-6 years here on my daily commute.. which is actually quite low considering that 99% of the cyclists i see blow through lights without even a look)

Karl
Karl
3 years ago

“remember also that most cyclists basically throw themselves into cars”

NOPE.

Richard Wharton
3 years ago

Well, we could’ve had a LIDAR reflector a few years ago, but no one funded it, and the “advocates” denigrated it. You might get in touch with the developer and see where it stands rt now. http://bikerumor.com/2016/10/10/ilumaware-shield-tl-bicycle-tail-light-enhances-vehicles-crash-avoidance-radar-keeps-safer/

beezus
beezus
3 years ago

“The same problems exist with regular drivers who don’t always react properly to seeing cyclists on the road.”
Automate the flaw! Efficiency gained!

FFM
FFM
3 years ago
Reply to  beezus

Human drivers have actually developed a pretty brilliant workaround that could be easily translated to machine language and incorporated into the decision loop. Train the machine to hate cyclists and assure it that all it has to do is blame the cyclist and it’ll get off the hook every time.

Milessio
Milessio
3 years ago

Or, just do some of the testing where there are cyclists in abundance e.g. the Netherlands, Denmark, etc ?

Bob
Bob
3 years ago

why don’t we just require people to actually have responsibility and learn to drive a car? all these safety systems do is make people dumber and then when the system fails (they always will at some %) the human is too stupid to know what is going on or how to correct it. look at back up cameras/sensors. they are great 99% of the time but people no longer actually look and simply rely on the system to save them. when it fails they run over their 1 year old. That actually happened in Minneapolis.

Brenda Bell (@tmana)
3 years ago

“Some cyclists not obeying the rules of the road to begin with”. That describes about 70% of the cyclists around here. Anyone not in technical cycling clothing (or at least wearing a helmet) is more likely to be cycling on the sidewalks (note, NOT “shared use/mixed use paths”), or cycling on the road AGAINST traffic, wearing dark clothing (and no lights or reflectors) at night… It’s hard enough sometimes for another CYCLIST to avoid these road hazards. (Add to the mix, pedestrians walking in streets without sidewalks, or whose sidewalks have been rendered impassible by snow plows dumping their loads onto them to keep the ROADS clear.)

lop
lop
3 years ago

Who are you and why are you here?

Chris
Chris
3 years ago

It seems yet again that the ‘blame the victim’ brigade are out in force.
If these so-called autonomous systems cannot ‘see’ cyclists then they probably cannot ‘see’ people either. People should always have priority (whether on foot on cycles) over motor vehicles in civilised societies and should not be expected to have to carry whatever totem the motorists’ lobby is currently advocating, whether that be hi-viz, reflectives, or, heaven forbid, some kind of transponder, in order to prevent themselves being struck by an out-of-control motor vehicle!
If these devices cannot negotiate expected hazards on roads and streets then they should be banned until they can be proven to be able to do so.

Pynchonite
3 years ago
Reply to  Chris

I agree. Much of the discussion here revolves around somehow modifying bikes or cyclists. While cyclists do have responsibilities as road users, their behavior is not relevant to the discussion about the ability or inability of self-driving cars to detect bicycles. The onus is on the companies introducing the new technology to make it safe for the roads, rather than vice versa.

lihtan
3 years ago

FLIR has suggested the use of it’s thermal vision sensors as a way for self-driving cars to better see pedestrians, cyclists, animals and other road users, as thermal cameras aren’t effected by adverse weather or lighting conditions.

https://www.digitaltrends.com/cool-tech/flir-systems-thermal-imaging-tech/

Cycling is great - the Car could do better (@PanhuyzenRalph)

Any progress since Febr. 2018??