Author: WU Jia (Engineer @ OpenCV China)
With the OpenCV VideoCapture class, users can easily access Orbbec’s 3D Cameras without relying on its SDK, similar to working with USB cameras, as demonstrated in following snippet. MacOS users also enjoy the similar convenience as Windows and Linux users now that the related pull request has been merged. In our comparisons here the OpenCV VideoCapture class and corresponding code snippet were employed to ensure an objective evaluation, devoid of any alterations or filters, thereby maintaining the integrity and authenticity of these comparisons.
import cv2 as cv
orbbec_cap = cv.VideoCapture(0, cv.CAP_OBSENSOR)
if orbbec_cap.isOpened() == False:
exit("Fail to open camera.")
while True:
if orbbec_cap.grab():
# RGB data
ret_bgr, bgr_image = orbbec_cap.retrieve(None, cv.CAP_OBSENSOR_BGR_IMAGE)
# depth data
ret_depth, depth_map = orbbec_cap.retrieve(None, cv.CAP_OBSENSOR_DEPTH_MAP)
if ret_depth and ret_bgr:
color_depth_map = cv.applyColorMap(cv.convertScaleAbs(depth_map, alpha=0.0255), cv.COLORMAP_JET)
# show the images
cv.imshow('RBG', bgr_image)
cv.imshow('Depth', color_depth_map)
else:
print("Fail to grab data from the camera.")
if cv.pollKey() >= 0:
break
orbbec_cap.release()
About The Cameras
Orbbec is a global leader in the design and manufacturing of 3D cameras. It offers a diverse range of 3D cameras tailored for various use cases. The Gemini 2 series is a collection of stereo vision 3D cameras that offer a wide field of view (FOV), an extended depth sensing range, and high-quality depth data.
Designed for easy setup and portability, the stereo vision 3D cameras RealSense D400 series also feature high depth resolution. We conducted some rapid tests with the Orbbec Gemini 2 L and RealSense D455 cameras in various environments, with the results displayed below. The output depth images are mapped to false color images using the JET colormap for visualization.
JET colormap
Depth Comparisons
-
- Comparison A: In the office
Orbbec Gemini 2 L
Intel® RealSense D455
-
- Comparison B: In the corridor
Orbbec Gemini 2 L
Intel® RealSense D455
-
- Comparison C: Reflections
Orbbec Gemini 2 L
Intel® RealSense D455
-
- Comparison D: Strong lights are within the field of view
Orbbec Gemini 2 L
Intel® RealSense D455
-
- Comparison E: In the office with various objects present
Orbbec Gemini 2 L
Intel® RealSense D455
Overall, the colored depth images from Gemini 2 L are more stable and smoother than those from RealSense D455. This is essential for 3D vision based applications. In the depth images of Gemini 2 L, the edges of objects are smooth and continuous without jaggedness or fluctuation, which ensure the integrity of the object contours. For instance, the edges of various objects in Scene E and the contour of the person in Scene B.
Moreover, the FOVs of both the depth and RGB sensors of Gemini 2 L are slightly wider than those of RealSense D455. This verifies the cameras’ technical specifications.
| FOV(HxV) | Gemini 2 L | D455 |
|---|---|---|
| Depth | 91° x 66° | 87° x 58° |
| RGB | 94° x 68° | 90° x 65° |
In the case of Scene C, reflections have a significant impact on the detection of the distance. The depth data of the TV surface is incorrect from RealSense D455 due to the reflections from the lights caused by the TV screen, while the depth image from Gemini 2 L keeps almost consistent. However RealSense has another model D455f which adds to D455 an IR pass filter pass over the lenses. This is to mitigate repetitive pattern and false object detection due to light reflections. Therefore the depth image from D455f should be much better than that of D455. However since we don’t have the D455f model we couldn’t conduct a test. In the case of Scene B, the light reflections on the floor also affect D455’s ability to detect the floor.
The above results also show that in some cases Gemini 2 L performs worse than RealSense D455. In Scene C RealSense D455 has better detection of the glasses. Strong lights from the window can also cause problems for Gemini 2 L as demonstrated in the results of Scenes D and E. This is because that in chanllenging light conditions RealSense D455 leverages passive stereo vision to improve its depth sensing capability, while Gemini 2 L doesn’t integrate the functionality.
Comparing the depth images of Scene B, it can be observed that Gemini 2 L has longer depth range. The ideal range of RealSense D455 is 0.6m to 6m, and that of Gemini 2 L is 0.2m to 10m. The depth accuracy varies depending on the distance. There are various metrics that can be defined to evaluate depth quality, and a flat white wall parallel to the camera’s front, as shown in the figure below, is often used to help measure these metrics.
-
- Comparison F: Depth image of a white wall parallel to the camera’s front at the distance of 0.6m
Orbbec Gemini 2 L
Intel® RealSense D455
-
- Comparison G: Depth image of a white wall parallel to the camera’s front at the distance of 0.3m
Orbbec Gemini 2 L
Intel® RealSense D455
The GIFs show the depth images of a white wall which is parallel to the camera’s front at the distance of 0.6m and 0.3m. Qualitatively, it can be seen that at distance of 0.6m the depth images from RealSense D455 is not entirely flat and stable, while those from Gemini 2 L are; and at the distance of 0.3m Gemini 2 L can output correct depth data, while RealSense D455 can not.
Quantitatively, root mean square error (RMSE) is used to evaluate the spatial noise of the depth data from the camera across a region of interest (ROI), while temporal noise measures this variation in depth over time.
In the figure above, z is the depth value (in mm) of a depth pixel (blue dots), the fitted plane is the best-fit plane to the depth data, and d is the distance of the depth pixel to the fitted plane. The RMSE is defined as 
. Plane fitting of the data helps reduce the sensitivity of the measurement to alignment errors between the camera and the wall.
The Gemini 2 L and RealSense D455 are set at distance ranging from 0.6m to 1.2m from the wall with intervals of 0.1m, and the RMSE is shown below:
From the figures, it can be seen that at various distances and different ROIs, the RMSEs of the Gemini 2 L are consistently smaller than those of RealSense D455. This indicates that depth data from Gemini 2 L is more spatially stable than that from RealSense D455.
Below are the results of another test that shows the spatial variation of the depth values over time. The standard deviation (STD) in depth values across an ROI is computed over consecutive 600 frames. We could see that over time and across different ROI scales, the STDs are smaller and remain consistent. However the STDs of RealSense D455 are much more volatile.
Temporal noise may be defined as a standard deviation in depth values over a specific number of frames (e.g. 50 or 100). It is typically measured per pixel and then an average or median is taken over the ROI to represent the temporal noise for that device and ROI. We take 50 depth images in a certain time frame of the white wall and compute the average value of the standard deviation matrix. The temporal noise for Gemini 2 L and RealSense D455 are 0.1488mm and 1.4102mm respectively.This shows that Gemini 2 L also has smaller temporal noise compared to RealSense D455.
Orbbec released Gemini 2 XL, a long range 3D camera suitable for outdoor scenarios, as an enhanced model following Gemini 2 L. Similar tests are also conducted using Gemini 2 XL, and indeed it yielded much better results compared to Gemini 2 L with even smoother and more stable depth images, less susceptible to strong lights and better glass object detection.
Prices
Lastly, let’s take a look at the price of these cameras.
| Orbbec | Gemini 2 | Gemini 2 L | Gemini 2 XL (POE) |
|---|---|---|---|
| Price | $267 | $326 | $447 |
| RealSense | D435i | D455/D455f | |
|---|---|---|---|
| Price | $334 | $419/$439 |
Note: An earlier version of this article incorrectly stated that RealSense cameras have been discontinued since August 2021. That is not true; Intel decided to only EOL (End of Life) the Intel RealSense Lidar and Tracking product lines in August 2021. Intel RealSense continues new development on advancing innovative technologies in the stereo depth camera product line and facial authentication solutions today.
