KR101609188B1 - Depth camera system of optimal arrangement to improve the field of view - Google Patents

Abstract

1. A depth camera system using a plurality of depth cameras, comprising: a first depth camera for acquiring a depth image having depth information; Acquiring a depth image having depth information on the first depth camera and having a view point in a direction different from a direction in which the view point of the first depth camera is directed, A second depth camera that is disposed in an intersecting manner; And a depth map image processing unit for generating a panorama depth map using the first depth image obtained by the first depth camera and the second depth image obtained by the second depth camera .

Description

[0001] DEPTH CAMERA SYSTEM OF OPTIMAL ARRANGEMENT TO IMPROVE THE FIELD OF VIEW [0002]

The present invention relates to a depth camera system having an optimal arrangement for improving the angle of view.

With the expansion of the robot industry, there is a lot of research on the depth camera that constructs the 3D map by measuring the distance in the indoor environment. Among them, Microsoft's Kinect, which is expanding in popularity at an affordable price, is not only recognizing the human body motion for the game, which is the original function of the game, but also expanding its application field as recognition of various objects in the room and recognition of plural human motions .

Kinect, which can acquire both color and depth images at the same time, has a limited horizontal angle of view, which limits the recognition of various objects in a complex room or a plurality of human gestures. Therefore, in order to solve the problem of such narrow angle of view, a method of using a plurality of Kinect has been devised.

According to the related art, a method has been proposed in which three Kinect cameras are installed at a distance from each other according to human motion, thereby carrying out skeleton tracking in a wider area. However, it should satisfy the condition that several people should come within a certain range, and if two people are attached, the recognition rate is lowered, and if the two depth cameras overlap quickly, the recognition rate also deteriorates. (C. Schonauer et al, "Wide Area Motin Tracking Using Consumer Hardware ", ACE 2011)

According to another conventional technique, a method has been proposed in which a plurality of Kinect are installed in four directions in the east, the south, and the south, and a person within a certain range is represented in 3D. However, since the four Kinect were looking at one place, the angle of field could not be improved. (Dimitrios S. et al, "Real-Time, Full 3-D Reconstruction of Moving Foreground Objects From Multiple Consumer Depth Cameras ", IEEE, 2013)

The research on the dual stereo camera was also carried out. It is a function to install two stereo cameras on one board and to extract more accurate distance information by matching the distance maps of reference objects obtained from each stereo camera. However, even if this method is used, it is possible to improve the accuracy of the distance map by disposing two stereo cameras in a day, but the overall angle of view can not be greatly improved. (Lee, Byung-Soo, "Distance Measuring Apparatus with Dual Stereo Camera", Korean Patent Application No. 10-2009-0056998)

The present invention is to provide a depth camera system capable of enlarging a horizontal angle of view to about twice using a plurality of depth cameras and reducing distortion of depth images caused by interference and interaction by a plurality of depth cameras.

According to an aspect of the present invention, there is provided a depth camera system using a plurality of depth cameras, including: a first depth camera for acquiring a depth image having depth information; Acquiring a depth image having depth information on the first depth camera and having a view point in a direction different from a direction in which the view point of the first depth camera is directed, There is provided a depth camera system including a second depth camera that is disposed in an interlaced manner.

In one embodiment, the first depth camera and the second depth camera may be arranged in an X-shape to have the same sensor center axis.

In one embodiment, the first depth camera and the second depth camera may be cross-positioned so that overlap is minimized between the depth image regions covered by the horizontal angle of view of each depth camera.

In one embodiment, the first depth camera and the second depth camera may be vertically aligned so that the vertical scan ranges in the depth image by each depth camera coincide with each other.

In one embodiment, the apparatus may further include a depth map image processor for generating a panorama depth map using the first depth image acquired by the first depth camera and the second depth image acquired by the second depth camera have.

Wherein the depth map image processing unit divides the center depth between the first depth camera and the second depth camera by a plane that faces the front surface as a reference plane, Can be corrected.

In one embodiment, the depth map image processing section may calculate a perspective distortion caused by a difference between respective view points of the first depth camera and the second depth camera after performing the depth value correction A corrective projective transformation can be performed.

In one embodiment, the depth map image processing unit may perform superimposition in which the projection depth-converted first depth image and the second depth image are connected to generate the panorama depth map.

In one embodiment, the first depth camera and the second depth camera are arranged so that the first and second depth cameras are mounted so as to obtain a depth image with respect to an omnidirectional reference background or reference object And can be installed so as to be relatively rotatable 360 degrees in relation to the substrate.

According to the embodiment of the present invention, there is an effect that the horizontal angle of view can be doubled by using a plurality of depth cameras.

Further, according to the embodiment of the present invention, there is an effect of acquiring the depth panoramic image with a depth image sensor as if by attaching depth images obtained from a plurality of depth cameras to each other naturally.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a simplified view of a depth camera system in accordance with an embodiment of the present invention.
FIG. 2 illustrates an arrangement of two depth cameras in a depth camera system according to an embodiment of the present invention. FIG.
3 is a conceptual diagram for explaining differences between different arrangements of two depth cameras and arrangements according to an embodiment of the present invention;
4 and 5 are flowcharts of a method for generating a dual panorama depth map in a depth camera system according to an embodiment of the present invention.
6 is a conceptual diagram illustrating a process of generating a dual panorama depth map by a depth camera system according to an embodiment of the present invention;
FIG. 7 illustrates dual panorama depth map images and depth images generated by a depth camera system in accordance with an embodiment of the present invention; FIG.

The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated and described in the drawings. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following description of the present invention, detailed description of known related arts will be omitted when it is determined that the gist of the present invention may be unnecessarily obscured. In addition, numerals (e.g., first, second, etc.) used in the description of the present invention are merely an identifier for distinguishing one component from another.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a view illustrating a depth camera system according to an embodiment of the present invention. Referring to FIG. FIG. 2 is a view illustrating an arrangement of two depth cameras in a depth camera system according to an embodiment of the present invention. FIG. 3 shows the difference between the different arrangement of the two depth cameras and the arrangement according to the embodiment of the present invention FIG. Hereinafter, with reference to Figs. 1 to 3, a method of arranging a plurality of depth cameras in a depth camera system according to an embodiment of the present invention will be described.

Referring to FIGS. 1 and 2, a depth camera system 100 according to an embodiment of the present invention includes a first depth camera 110 for acquiring a depth image having depth information; The first depth camera 110 and the second depth camera 110 are positioned at the first depth camera 110 so as to have a view point in a direction different from a direction in which the view point of the first depth camera 110 is directed, A second depth camera 120 cross-positioned relative to the camera 110; A depth map image processor 110 for generating a panorama depth map using the first depth image 111 obtained by the first depth camera 110 and the second depth image 121 obtained by the second depth camera 120, (130).

In the present specification, as shown in Figs. 1 and 2, two depth cameras are vertically cross-disposed. However, three or more depth cameras are arranged vertically with their respective view points oriented in different directions It is also possible to expand to the form that it is. Hereinafter, for convenience and concentration of explanation, embodiments of the present invention will be described with reference to the embodiments shown in Figs. 1 and 2. Fig.

The first arrangement method that can be considered for the expansion of the angle of view is to arrange the two depth cameras horizontally in the same manner as in Fig. 3 (a). In this case, a larger horizontal angle of view can be ensured compared to other force arrangements, but an empty space 32 is created where the two depth cameras can not scan. Also, due to the horizontal disparity that occurs when the two sensors are separated from each other, distortion of the object, which is located close to the front of the sensor, occurs in the same manner as the human eye.

Accordingly, the embodiment of the present invention proposes a layout method that can solve the distortion problem due to horizontal parallax while enlarging the horizontal angle of view. A layout scheme according to an embodiment of the present invention will be described below with reference to FIG. 2 and FIG. 3 (b).

In an embodiment of the present invention, the first depth camera 110 and the second depth camera 120 may be arranged in an X-shape to have the same sensor center axis. At this time, the first depth camera and the second depth camera can be arranged so as to minimize the overlap between the depth image regions covered according to the horizontal angle of view of each depth camera (see FIG. 3B). In addition, the first depth camera and the second depth camera are vertically aligned so that the vertical scan ranges in the depth image by the depth cameras are matched with each other.

3 (b), since the two depth cameras are vertically aligned up and down, the vertical tilt angle of the two depth cameras can be adjusted easily It is possible to obtain an effect of expanding the angle of view to about 2 times in a state in which the vertical scan region can be matched. There is no horizontal disparity, and no blind spot is generated even at a position close to the sensor. In addition, since the overlapping region 131 does not occur as shown in FIG. 3A, the horizontal angle of view can be improved and the interference can be minimized. As will be described later, since the two depth cameras have no horizontal parallax, it is easy to integrate the depth images acquired from the two depth cameras into one image.

1 to 3, according to one embodiment, the first depth camera 110 and the second depth camera 120 may be configured to have an omni-directional reference background, So that the first depth camera can be relatively rotated 360 degrees with respect to the substrate on which the first depth camera and the second depth camera are mounted. For example, the substrate may be configured to be rotated in all directions in relation to the first and second depth cameras, or vice versa, and the first and second depth cameras may be rotated in all directions.

4 and 5 are flowcharts illustrating a method for generating a dual panorama depth map in a depth camera system according to an embodiment of the present invention. 6 is a conceptual diagram illustrating a process of generating a dual panorama depth map by a depth camera system according to an embodiment of the present invention. 2 is a diagram illustrating a dual panorama depth map image.

Hereinafter, a method of generating a dual panorama depth map in a depth camera system according to an embodiment of the present invention will be described with reference to FIG. 4 to FIG. 1 to 3, using the first depth image 111 and the second depth image 121 obtained through the first depth camera and the second depth camera, respectively, A method of generating a dual panorama depth map image (refer to reference numeral 141 or FIG. 7 (c) in FIG. 6) is as follows.

First, a first depth image and a second depth image are obtained through two depth cameras, respectively (see S100 in FIG. 4 or S100-1 and S100-2 in FIG. 5).

When the first depth image and the second depth image are obtained through the two depth cameras arranged in the upper and lower X-shape, the depth map image processing unit 130 performs the geometric transformation suitable for the sensor arrangement of the depth camera (See S110 in Fig. 4). This will be described in more detail as follows.

The depth map image processing unit 130 first cuts the center intersection point between the first depth camera 110 and the second depth camera 120 and uses the plane that faces the front as a reference plane, And the depth value of the second depth image 121 (see S112-1 and S112-2 in Fig. 5). This is a preliminary work for generating a panoramic depth map image in the direction of looking forward in the future by correcting the depth value with reference to the front view plane.

Thereafter, the depth map image processing unit 130 performs a depth image processing operation to correct a perspective distortion caused by a difference between respective view points of the first depth camera 110 and the second depth camera 120 distortion (see S114-1 and S114-2 in Fig. 5).

Upon completion of the geometric transformation process as described above, the depth map image processing unit 130 generates a dual panorama depth map by superimposing the projection-transformed first depth image and the second depth image together (See S120 and S130 in Fig. 4 and Fig. 5). Accordingly, a dual panorama distance map having an improved angle of view can be generated by making two depth images into one image.

The above-described process will be described with reference to FIGS. 6 and 7 by way of example.

FIG. 6 is a conceptual diagram for processing two depth images obtained from each depth camera in the depth map image processing unit 130 according to the present invention. In FIG. 6, the two parallel lines indicate an object or background that lies parallel to the two depth cameras. Each depth camera scans the actual object or the background to generate two depth images 111 and 121. At this time, the parallel red lines are distorted as shown in the depth images 111 and 121 of FIG. 6 . This is because, in the embodiment of the present invention, since the two depth cameras are respectively turned right and left, when the initial values are set, the two depth images 111 and 121 are transformed through correction and projection transformation of the depth information considering the geometric relation, A dual panorama depth map, such as 141, can be generated.

An actual example image of this is shown in Fig. 7 (a) and 7 (b) are depth images taken at the viewpoint of each depth camera. 7 (c) shows a depth panorama image in which the depth images are processed through depth correction and projection transformation and the two images are stitched together. As shown in the example, 4-5 persons can be scanned by a single depth camera. However, according to the embodiment of the present invention, it is confirmed that the angle of view is enlarged by scanning up to about eight persons.

As described above, when a plurality of depth cameras are arranged according to the embodiment of the present invention, all the objects located in the forward direction of the sensor can be scanned while maximizing the angle of view using two depth images obtained from the plurality of depth cameras , The depth map image processor can perform the projection transformation considering the geometric arrangement of the depth camera and then generate the dual panorama depth map. By acquiring dual panorama depth maps that are about twice as high as the original depth of field of each depth camera, 3D reconstruction in the room can be more effective, and it can be applied to various applications such as object tracking or human extraction using a depth camera can do.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the following claims And changes may be made without departing from the spirit and scope of the invention.

Claims (8)

A depth camera system using a plurality of depth cameras,
A first depth camera for acquiring a depth image having depth information;
Acquiring a depth image having depth information on the first depth camera and having a view point in a direction different from a direction in which the view point of the first depth camera is directed, A second depth camera that is disposed in an intersecting manner; And
And a depth map image processor for generating a panorama depth map using the first depth image obtained by the first depth camera and the second depth image acquired by the second depth camera,
Wherein the first depth camera and the second depth camera,
The overlapping is minimized between the depth image regions covered by the horizontal view angle of each depth camera and the vertical scan ranges in the depth image by each depth camera are mutually aligned so as to have the same sensor center axis, Respectively,
Wherein the depth map image processing unit divides the center depth between the first depth camera and the second depth camera by a plane that faces the front surface as a reference plane, Correction, depth camera system.
delete delete delete delete The method according to claim 1,
Wherein the depth map image processing unit is configured to perform a projection transformation process for correcting a perspective distortion caused by a difference between respective view points of the first depth camera and the second depth camera after performing the depth value correction projective transformation.
The method according to claim 6,
Wherein the depth map image processing unit performs superimposing to connect the projection-converted first depth image and the second depth image to generate the panorama depth map.
The method according to claim 1,
Wherein the first depth camera and the second depth camera are arranged in a relationship with a substrate on which the first and second depth cameras are mounted so as to obtain depth images relating to an omnidirectional reference background or reference object Depth camera system installed to allow relative rotation of 360 degrees.

Images