KR101296365B1 - hologram touch detection method using camera - Google Patents

Abstract

The present invention relates to a holographic touch recognition method using a camera, comprising: (a) correcting an image to match a view image photographed by a plurality of stereo cameras and at least one depth camera; (b) photographing a 3D image including a hologram image generated by a hologram generator in real time with the plurality of stereo cameras and at least one depth camera; (c) recognizing the hologram image by comparing the 3D image photographed by the stereo camera and the 3D image photographed by the depth camera by a touch recognition unit, and obtaining spatial position information of the hologram image; And (d) obtaining, by the touch recognition unit, touch location information that the user touches on the space in the image of the hologram.
The present invention as described above provides a simple and easy way to directly touch and recognize the hologram in the space by using a variety of cameras, and does not use any other sensors other than the camera and to the human body Not only does it need to be attached, it also provides a way to interface naturally with the holograms in space without any additional devices.

Description

Hologram touch detection method using camera {hologram touch detection method using camera}

The present invention relates to a holographic touch recognition method, and more particularly to a touch recognition method that can recognize the holographic touch point simply and easily using a camera.

There is currently no technology for directly touching a hologram in space, and there may be a technology for controlling a hologram by an operation, but this is not by touching a hologram, but by recognizing a user's motion by using a camera or a sensor. It is a technique to manipulate.

Similarly, there is a technology that seems to touch the hologram, but this is a technology that determines whether the computer that projects the hologram knows the position of the hologram in advance and touches the hologram by judging the position of an external object.

In order to recognize and grasp such touch location information, a method of acquiring 3D information from an object or a scene can be largely divided into an active method and a passive method. Active methods include a method using a 3D scanner, a method using a structured ray pattern, and an active method such as a depth camera. In this case, three-dimensional information can be obtained relatively accurately in real time, but there is a problem that equipment is expensive and modeling of dynamic objects or scenes is impossible except for a depth camera.

Passive methods include stereo matching and binocular stereo images, silhouette-based method, volume-based modeling method, Voxel Coloring method, and 3 There are a motion based shape estimation method for calculating dimensional information (Shape from Motion) and a shape estimation method using shadow information (Shape from Shading). This passive method generates three-dimensional information by using images obtained from optical cameras of various viewpoints, and can obtain three-dimensional information at a lower cost than the active method, and has a high resolution. However, the reliability of the depth information is lower than that of the active method due to the characteristics of the image, that is, the change of illumination conditions, the texture, and the presence of the shielding area.

In order to improve this, conventionally, a technique for identifying a location of a 3D image through a device or a computer that projects a 3D image in space and recognizing touch location information through a user's location recognition module has been proposed.

1 is a block diagram of a terminal device having a conventional spatial projection and spatial touch function. As shown in FIG. 1, the terminal device 100 according to an exemplary embodiment of the present invention includes a display module 120 for projecting an image on the space 200, and a user's indicating means projecting the space on the space 200. A spatial touch recognition module 110 for recognizing which part of the image is located, a user position detection module 130 for detecting whether the user is located within a recognition distance of the spatial touch recognition module 110, and a user recognition If it is detected to be located within the distance control the display module 120 to project the image, and comprises a computing module 140 for performing a function corresponding to the position information recognized by the spatial touch recognition module 110.

However, such a conventional technology has a problem in that the system configuration for touch recognition is complicated, expensive, and the device is inferior in accuracy due to an error in that the 3D image and the location of the user's touch point are identified and synthesized by different devices. .

SUMMARY OF THE INVENTION An object of the present invention for solving the above problems is to provide a method for recognizing or acquiring position information of touching a hologram projected onto a space without external information (positioning of the hologram known by the computer projecting the hologram). To do so.

Another object of the present invention is to provide a touch recognition method capable of directly touching a hologram with a body or an arbitrary object or interacting with each other in a simple and easy manner.

Features of the present invention to solve the above problems is a) correcting the image to match the view image taken with a plurality of stereo cameras and at least one depth camera; (b) photographing a 3D image including a hologram image generated by a hologram generator in real time with the plurality of stereo cameras and at least one depth camera; (c) recognizing the hologram image by comparing the 3D image photographed by the stereo camera and the 3D image photographed by the depth camera by a touch recognition unit, and obtaining spatial position information of the hologram image; And (d) obtaining, by the touch recognition unit, touch location information that the user touches on the space in the image of the hologram.

Here, in the step (b), when there are two or more depth cameras, it is preferable that the photographing time of the depth camera is divided and real-time shooting is performed according to the divided time of each of the depth cameras. The method may include determining whether the hologram image is present by subtracting the 3D image photographed by the stereo camera and the 3D image photographed by the depth camera by a touch recognition unit; If there is the hologram image, it is preferable to include the step of acquiring spatial position information of the hologram image by using the images captured by the plurality of stereo cameras.

Also, preferably, in the step (d), when the user touches a location area in the space of the hologram image through the images of the depth camera and the stereo camera, the touch is detected by the user. Recognizing a pointer; Recognizing the position information of the recognized touch pointer as the touch position information, it may be a step of obtaining the touch position information through the image of the stereo camera.

In addition, the location information of the touch pointer may be obtained by recognizing the location information of the touch pointer when the user stays in a location area where the user touches a location area in the space of the hologram image for a predetermined time or more, The touch recognition unit may further include transmitting the acquired touch position information to an external hologram generator or a hologram control terminal.

The present invention provides a simple and easy way to directly touch and recognize the hologram in space by using a variety of cameras, do not use other sensors other than the camera and do not have to attach to the body to the human In addition, it provides a way to interface naturally with the holograms in space without any additional devices.

In addition, it can be determined even if it is difficult to determine whether the object in space is a hologram or a real object.

1 is a block diagram of a terminal device equipped with a conventional spatial projection and spatial touch function,
2 is a block diagram of a system used in the holographic touch recognition method using a camera according to an embodiment of the present invention,
3 is a view showing the flow of a holographic touch recognition method using a camera according to an embodiment of the present invention,
4 is an example of an image coming into the depth camera and the stereo camera used in the holographic touch recognition method according to the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and how to accomplish it, will be described with reference to the embodiments described in detail below with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described herein but may be embodied in other forms. The embodiments are provided so that those skilled in the art can easily carry out the technical idea of the present invention to those skilled in the art.

In the drawings, embodiments of the present invention are not limited to the specific forms shown and are exaggerated for clarity. In addition, parts denoted by the same reference numerals throughout the specification represent the same components.

The expression "and / or" is used herein to mean including at least one of the components listed before and after. Also, singular forms include plural forms unless the context clearly dictates otherwise. Also, components, steps, operations and elements referred to in the specification as " comprises "or" comprising " refer to the presence or addition of one or more other components, steps, operations, elements, and / or devices.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

2 is a block diagram of a system used in a holographic touch recognition method using a camera according to an embodiment of the present invention, Figure 3 is a view showing the flow of a holographic touch recognition method using a camera according to an embodiment of the present invention. .

As shown in FIG. 2, the configuration of the system used in the holographic touch recognition method according to the present invention includes at least two stereo cameras 10, at least one depth camera 20, and a touch recognition unit 30. . The arrangement and number of the camera may be determined in various ways as long as the view image of the 3D image including the hologram is properly installed in the ROI. In addition, the touch recognition information acquired by such a system may be transmitted to the external hologram generator 40 or the terminal to provide various changes in the hologram image due to touch recognition.

That is, the system used in the holographic touch recognition method according to the present invention captures the hologram image in real time with the stereo camera 10 and the depth camera 20, recognizes the hologram image through the captured image, and the user's touch position. By obtaining the information to transmit the touch position information to the external hologram generator 40, it is a system that can obtain a control command signal to induce a variety of changes through the touch of the holographic image.

As shown in FIG. 3, in the holographic touch recognition method using a camera according to an embodiment of the present invention, (a) a view image captured by a plurality of stereo cameras 10 and at least one depth camera 20. Correcting the image to match (S100); (b) photographing a 3D image including a hologram image generated by the hologram generator 40 in real time with the plurality of stereo cameras 10 and at least one depth camera 20 (S200); (c) the touch recognition unit 30 recognizes the hologram image by comparing the 3D image photographed by the stereo camera 10 with the 3D image photographed by the depth camera 20, and spatial position of the hologram image. Obtaining information (S300); and (d) acquiring, by the touch recognition unit 30, touch location information that the user touches on the space in the image of the hologram (S400).

Here, the depth camera 20 may be a method of measuring the distance of the pattern by measuring the time of flight (TOF) method and the pattern light by emitting light from the camera and measuring the time to return to the object reflected back. Can be. Depth camera 20 used in the present invention refers to a method of measuring the distance by all irradiating the light and processing it.

In addition, the stereo camera 10 is a method of measuring the distance by using two views of the camera (view) is different from each other even if the same point using the two general cameras.

The basic principle is that when an object (real object, an object reflecting light) exists in a space, both the depth camera 20 and the stereo camera 10 acquire and calculate the image of the object, and the object is located at a certain point in space. You can calculate what form (3D form) appears in the. However, in the case of the hologram, the stereo camera 10 may recognize the object. However, in the depth camera 20, the hologram does not appear in the depth camera 20 because the light emitted from the depth camera 20 transmits the hologram. Using both, we can determine whether the object in space is a hologram or a real object. Based on the above principle, we propose the present invention with the idea that it is possible to determine the hologram in space and determine whether a person or an object touches the hologram.

There is a hologram in space, and the camera is located around the hologram. If the camera can only exist in the region of interest and you are only interested in the front region of the hologram (touchable region), there is no need to install the camera behind. As used herein, the term “camera” refers to a depth camera 20, at least two stereo cameras 10 pairs.

It is three-dimensional information that is obtained by processing the image received through the stereo camera 10 in real time, and another object may cover the hologram, in which case another object (depth camera 20 using the information of the depth camera 20) You can solve the problem of masking to some extent by removing the objects that are only visible).

4 is an example of an image coming into the depth camera 20 and the stereo camera 10 used in the holographic touch recognition method according to the present invention. As shown in FIG. 4, in the image, the user is holding a plastic object (shown with a blue pattern) through which light is transmitted. In the image coming into the stereo camera 10, this plastic object can be distinguished, but in the depth camera 20, it can be seen that this object does not appear at all. The present invention proposes a holographic touch recognition method by applying the above principle.

Specifically, referring to the holographic touch recognition method according to an embodiment of the present invention, as shown in Figure 3, the pair of depth camera 20 and stereo camera 10 in the same position in step (a) (S100) is calibrated with each other Since it should be, the image is corrected so that the image viewed by the stereo camera 10 and the view image viewed by the depth camera 20 match.

In step (S200), the depth camera 20 and the stereo camera 10 respectively capture a 3D image in space in real time. Here, the depth camera 20 is irradiated with light, so there may be interference when using multiple depth cameras 20. In this case, the camera splits the time by camera and shoots the light by the time corresponding to each image. It is desirable to obtain.

(c) In step S300, the 3D image of the depth camera 20 and the 3D image of the stereo camera 10 are compared to determine the hologram and the real object, respectively. The position and shape of the hologram in space can be known from the information of the stereo camera 10, and the position and shape of the real object can be known from the depth camera 20.

That is, the touch recognition unit 30 determines whether the hologram image is present by subtracting the 3D image photographed by the stereo camera 10 and the 3D image photographed by the depth camera 20 (S310), and the hologram image. If there is (Yes), the spatial position information of the hologram image is obtained by using the images captured by the plurality of stereo cameras 10 (S330). Go to step b).

(d) In step S400, since the positional information of the hologram recognized in step (c) is included and the depth camera 20 can recognize another object or a person in this space, the person or the object Touching this (pointer) hologram can be recognized.

More specifically, when the user touches any position area in the space of the hologram image through the images of the depth camera 20 and the stereo camera 10 in the touch recognition unit 30, The touch pointer is recognized (S410), the position information of the recognized touch pointer is recognized as the touch position information, and the touch position information is obtained through the image of the stereo camera 10 (S430). The pointer refers to an area of an object that a person or an object points to touch the holographic image, and a spatial location point of the hologram image of the pointer corresponds to a touch location point.

(e) In step S500, when the hologram is touched, various applications may be made by connecting to various devices using touch location information. (Process using hologram rotation, touched area information, shaking hands with hologram (person), etc.) That is, the touch position information obtained by the touch recognition unit 30 is transmitted to an external hologram generator or hologram control terminal. The hologram generator 40 or the hologram control terminal recognizes the control command signal to change the image of the hologram.

While the invention has been shown and described with respect to the specific embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined by the appended claims. Anyone with it will know easily.

Claims (6)

(a) correcting an image to match a view image photographed by a plurality of stereo cameras and at least one depth camera by a touch recognition unit;
(b) capturing, in real time, a 3D image including a hologram image generated by a hologram generator by the plurality of stereo cameras and at least one depth camera;
(c) comparing the 3D image photographed by the stereo camera with the 3D image photographed by the depth camera to recognize the hologram image, and obtaining spatial position information of the hologram image;
and (d) acquiring touch position information of a user who touches on a space in an image of the hologram, by the touch recognition unit.
The method of claim 1,
The step (b)
If there are two or more depth cameras,
Holographic touch recognition method using a camera, characterized in that the step of dividing the shooting time of the depth camera to shoot in real time according to each of the divided time of the depth camera.
The method of claim 1,
The step (c)
Determining whether the hologram image is present by subtracting the 3D image photographed by the stereo camera and the 3D image photographed by the depth camera by a touch recognition unit;
And obtaining spatial position information of the hologram image by using the images photographed by the plurality of stereo cameras when the hologram image is present.
The method of claim 1,
The step (d)
Recognizing, by the touch recognition unit, the touch pointer of the user when the user touches a location area in the space of the hologram image through the images of the depth camera and the stereo camera;
And recognizing the position information of the recognized touch pointer as the touch position information and acquiring the touch position information through the image of the stereo camera.
5. The method of claim 4,
Position information of the touch pointer,
And recognizing and acquiring the location information of the touch pointer when the user stays in the location area where the user touches a location area in the space of the hologram image for a predetermined time or more.
The method according to any one of claims 1 to 5,
And transmitting the acquired touch position information from the touch recognition unit to an external hologram generating unit or a hologram control terminal.

Images