Depth and Accessibility

This month is National Disability Employment Awareness Month, which is the perfect time to discuss some accessibility solutions where depth is or could be potentially helpful. Depth cameras are designed to help computers understand and interpret the world, which makes them ideal devices for offloading a variety of tasks from and for humans.

In the USA, 1 in 5 adults, or over 53 million people have a disability of one form or another. This varies from state to state, ranging from 1 in 6 people in Minnesota, to nearly 1 in 3 people in Alabama. Many of us at some point in our lives will experience either short term or permanent disability. Over a third of adults 65 years or older report some kind of disability affecting their daily lives. With an aging world population, this means more and more individuals will need accessibility accommodations. By 2050, the world’s population aged 60 and over is expected to total 2 billion.

Accessibility for all

Removing barriers is what accessibility is all about. Many people think of accessibility as something which has limited use for specific people but in reality accessibility solutions are often useful for individuals both with and without disability. For example, Piaggio’s gita. It’s a device that can carry up to 40lbs, hands free. It uses depth to help it follow you, identifying you unambiguously, and carrying groceries or personal items with ease at speeds up to 6mph. As with many robotic devices, using depth allows for a more concrete understanding of the environment, such as detecting when the device is about to fall off a cliff (definitely something that you don’t want happening).

The gita has been designed to improve the pedestrian experience – to allow people to walk unencumbered in their environments without needing to worry about what they are carrying. This product is a great example of something that can improve quality of life and experiences for people with and without disabilities.

Piaggio gita

We often think of disability as something that is binary – either you have a disability or you do not, but it’s really a spectrum that varies from individual to individual, and even from day to day in the same individual. Closed, primarily walking communities such as retirement communities could utilize gita to help residents walk to the store instead of needing to drive. Similarly, people who need to use a walker or wheelchair some or all of the time could find gita useful to carry more than they would normally be able to. Even if someone is physically able to carry things themselves, the gita makes it easier to get moving or choose walking over riding.

Hoobox Wheelie

Depth as a control

As well as allowing a computer or device to understand environmental cues, depth cameras can also be used as a control mechanism – for example, instead of a joystick or physical controls, a depth camera can be used to control a PC, or in the case of Hoobox, a wheelchair. A small startup from Sao Paulo, Brazil, Hoobox built a system with Intel® RealSense™ Depth Cameras and software that allows people to control motorized wheelchairs with simple facial expressions. Make a kissy face to go forward, wrinkle your nose to turn left, and smile to stop, for instance (users can pick from 10 expressions to control their chair). The innovative part of this product is that the kit can be installed very easily on existing wheelchairs, using a small robotic arm to control the joystick on a motorized wheelchair. By using a depth camera, the accuracy of the expression recognition is improved, and allows the system to avoid invasive body sensors.

Similarly, Eyeware Tech uses depth cameras as part of their eye tracking technology. Eye tracking is extremely useful in streamlining human computer interactions, allowing user’s attention and intentions to be sensed or even predicted and giving people with limited or restricted hand mobility more opportunities for easy control of computers and devices.

Another potential opportunity for depth is allowing access – facial authentication using depth cameras is more secure and robust in varied lighting conditions, so instead of requiring button pushes, keys or door handles to open or unlock doors, a depth based facial authentication system could unlock or open automatically when the authorized user is detected.

Future ideas

Allowing machines to understand the environment and the people in it have many obvious applications for accessibility, but there are plenty of other situations where depth could be helpful – for example, assisted navigation for individuals with impaired sight. A depth camera can be used to identify obstacles or assist with finding the distance to a wall or object. Adding a depth camera to a mobile phone along with audio output could allow for a simple and persistent model of the world, without needing to fully explore the space physically to understand it.

Another issue that could be managed and helped by depth are accessible sidewalks – many sidewalks and curbs around the USA are not ADA compliant, to the point that many cities are being sued for their lack of compliance. The scale of the problem is not hard to imagine, given the number of sidewalks and curbs present in every city, building, monitoring and maintaining access is a huge problem. How do cities manage their sidewalk maintenance? If a pothole appears, how do they become aware of the problem? Depth cameras mounted on drones, or public vehicles like postal delivery vans or parking enforcement vehicles could be used to monitor every sidewalk and curb automatically.

The added benefit of an ongoing survey such as this would be the ability to take that data and put it into a route planning or map system. Once that data is integrated, users of wheelchairs could plan the shortest possible route to their destinations that avoid broken sidewalks or dangerous curbs via a mobile app, and cities could start to actively address their problem areas without the need for expensive citywide surveys.

As we move into the future, it’s clear that depth and computer vision devices can make our world smarter and remove barriers to many different activities. Technology is at its best when it is used to enhance our lives, to give us the power to act in the way we want to or need to or improve our experiences by removing friction. What can you build with depth to help achieve that?