Intel RealSense SDK 2.0 for depth and tracking cameras

Intel® RealSense™ SDK 2.0

It's easy to build a depth application of the future with the cross-platform open source Intel® RealSense™ SDK 2.0.

Stable: v2.35.2 | Source code | Pre‑release: v2.35.0

Different OS? Download for Windows 7, Linux or see below.

Linux

We provide our installation packages in dpkg format for Ubuntu 16.04/18.04 LTS

The Intel RealSense SDK 2.0 is cross-platform and operating system independent. In addition to the platforms above we also have installation guides for Jetson TX2, Raspberry Pi 3 and macOS.

Supported platforms and languages

Supported platforms and languages

Code samples

Get your project off the ground quickly with help from our code examples and tutorials.

'rs-hello-realsense' example demonstrates the basics of connecting to an Intel RealSense device and taking advantage of depth data by printing the distance to object in the center of camera field of view.

Code
Preview
#include <librealsense2/rs.hpp> // Include RealSense Cross Platform API

// Create a Pipeline - this serves as a top-level API for streaming and processing frames
rs2::pipeline p;

// Configure and start the pipeline
p.start();

// Block program until frames arrive
rs2::frameset frames = p.wait_for_frames();

// Try to get a frame of a depth image
rs2::depth_frame depth = frames.get_depth_frame();

// Get the depth frame's dimensions
float width = depth.get_width();
float height = depth.get_height();

// Query the distance from the camera to the object in the center of the image
float dist_to_center = depth.get_distance(width / 2, height / 2);

// Print the distance
std::cout << "The camera is facing an object " << dist_to_center << " meters away \r";

This sample demonstrates how to configure the camera for streaming and rendering Depth & RGB data to the screen. We use OpenGL for cross-platform rendering and GLFW for window management. For OpenCV users we have an imshow alternative.

Code
Preview
#include <librealsense2/rs.hpp> // Include RealSense Cross Platform API

#include "example.hpp"          // Include a short list of convenience functions for rendering

// Create a simple OpenGL window for rendering:
window app(1280, 720, "RealSense Capture Example");

// Declare depth colorizer for enhanced color visualization of depth data
rs2::colorizer color_map; 

// Declare rates printer for showing streaming rates of the enabled streams.
rs2::rates_printer printer;

// Declare the RealSense pipeline, encapsulating the actual device and sensors
rs2::pipeline pipe;

// Start streaming with default recommended configuration
// The default video configuration contains Depth and Color streams
// If a device is capable to stream IMU data, both Gyro and Accelerometer are enabled by default 
pipe.start(); 

rs2::frameset data = pipe.wait_for_frames().    // Wait for next set of frames from the camera
                     apply_filter(printer).     // Print each enabled stream frame rate
                     apply_filter(color_map);   // Find and colorize the depth data
                     
    
// Show method, when applied on frameset, break it to frames and upload each frame into a gl textures
// Each texture is displayed on different viewport according to it's stream unique id
app.show(data);

This example demonstrates a very simple facial landmark detection using Dlib's machine learning algorithms, using depth data to implement basic anti-spoofing. This is just an example intended to showcase possible applications. The heuristics used here are very simple and basic, and can be greatly improved on.

Code
Preview
// Enable usage of librealsense frame data as a dlib image
rs_frame_image< dlib::rgb_pixel, RS2_FORMAT_RGB8 > image( color_frame );

// 1. Faces are detected in two steps:
// Facial boundary rectangles are detected:
  dlib::frontal_face_detector face_bbox_detector = dlib::get_frontal_face_detector();
  ...
  std::vector< dlib::rectangle > face_bboxes = face_bbox_detector( image );

// 2. Each one is then annotated to find its landmarks:
  dlib::shape_predictor face_landmark_annotator;
  dlib::deserialize( "shape_predictor_68_face_landmarks.dat" ) >> face_landmark_annotator;
  ...
  std::vector< dlib::full_object_detection > faces;
  for( auto const & bbox : face_bboxes )
    faces.push_back( face_landmark_annotator( image, bbox ));

if( nose_depth >= eye_depth ) return false;
if( eye_depth - nose_depth > 0.07f ) return false;
if( ear_depth <= eye_depth ) return false;
if( mouth_depth <= nose_depth ) return false;
if( mouth_depth > chin_depth ) return false;

// All the distances, collectively, should not span a range that makes no sense. 
// F.e. if the face accounts for more than 20cm of depth, or less than 2cm, then something's not valid
float x = std::max( { nose_depth, eye_depth, ear_depth, mouth_depth, chin_depth } );
float n = std::min( { nose_depth, eye_depth, ear_depth, mouth_depth, chin_depth } );
if( x - n > 0.20f ) return false;
if( x - n < 0.02f ) return false;

// Align depth and color frames
rs2::align align_to_color( RS2_STREAM_COLOR );
...
rs2::frameset data = pipe.wait_for_frames();
data = align_to_color.process( data );       // Replace with aligned frames

Tools

SDK tools help you harness your depth camera’s capabilities.

Intel® RealSense™ Viewer

Quickly access your Intel RealSense depth camera to view the depth stream, visualize point clouds, record and playback streams, configure your camera settings and more.

Depth Quality Tool

Test the camera depth quality with tools including: plane fit RMS error, subpixel accuracy, fill rate, distance accuracy compared to ground truth, on‑chip self‑calibration and tare calibration.

Stay updated

By submitting this form, you are confirming you are an adult 18 years or older and you agree to Intel contacting you with marketing-related emails or by telephone. You may unsubscribe at any time. Intel’s web sites and communications are subject to our Privacy Notice and Terms of Use.

Talk to sales

Let’s talk about how Intel RealSense depth sensing cameras can enhance your solution.

Stay Updated

Keep up to date with Intel® RealSense™ Technology: product updates, upcoming announcements, and events.

By submitting this form, you are confirming you are an adult 18 years or older and you agree to Intel contacting you with marketing-related emails or by telephone. You may unsubscribe at any time. Intel’s web sites and communications are subject to our Privacy Notice and Terms of Use

You are about to leave our website and switch to intel.com.
Click here to proceed

Stay Updated

Keep up to date with Intel® RealSense™ Technology: product updates, upcoming announcements, and events.

By submitting this form, you are confirming you are an adult 18 years or older and you agree to Intel contacting you with marketing-related emails or by telephone. You may unsubscribe at any time. Intel’s web sites and communications are subject to our Privacy Notice and Terms of Use