KR20140123418A

KR20140123418A - Augmented reality system using mirror

Info

Publication number: KR20140123418A
Application number: KR1020140037340A
Authority: KR
Inventors: 이문기
Original assignee: 이문기
Priority date: 2013-04-14
Filing date: 2014-03-29
Publication date: 2014-10-22
Also published as: WO2014171644A1

Abstract

The present invention relates to an augmented reality system using a mirror such as a holoflexor of the Microsoft Company. An image photographing device such as the Kinect of an existing system is installed at the upper part of a display edge. Therefore, it is difficult for a user to photograph an image when the user becomes closer to the display. To solve the problem, the system of the present invention has a camera installed in a display which is a transparent display.

Description

[0001] AUGMENTED REALITY SYSTEM USING MIRROR [0002]

The present invention relates to an improved augmented reality system using mirrors.

The present invention relates to an improvement of an augmented reality system using a mirror called a holoflector disclosed by Microsoft Corporation. An introductory video on the hollector is available at http://venturebeat.com/2012/02/28/microsoft-holoflector/ . As shown in Fig. 1,

The LCD display (FD) is placed behind the half mirror (HM) and the key (CM) is placed above the half mirror to recognize the gesture of the user (CS) in front of the half mirror (HM) And outputs the virtual object image VO to the LCD display FD in response to the virtual object image VO. Here, matching and outputting the virtual objects means that the user displays his or her appearance in the mirror and the virtual objects as if they exist in the same space. With this system, the user can feel as if his or her own image (RO) and virtual object (VO) in the mirror exist in the same space. For example, the virtual hole VO can be moved to the hand RO of the mirror

. Since the above-mentioned hollet reflector is installed on the upper edge of the semi-mirror (HM), there is a problem that it is difficult to photograph the user when the user approaches the half mirror. To measure the user's three-dimensional position with the kinematics, the gap between the kinem and the user should not be too small. For example, when configuring a digital signage with a single-pole reflector, the user can access the digital signage screen in close contact with the screen in order to view the details of the screen. In this case, There is a problem that it is difficult to do.

In order to solve the problem of the conventional holetirector described above, the present invention provides a configuration in which a key knot CM is installed at an interval IT from the display surface inside a display (FDT) as shown in Fig. It is preferable that the display (FDT) is a transparent display for capturing a user with a mounted keypad as shown in FIG. It is preferable to use a variable reflectance mirror (HMT) which can adjust the reflectance to an electric signal instead of the half mirror. These reflector mirrors are introduced at http://kentoptronics.com/solutions.html as a Switchable Mirror. With this configuration, the image is outputted to the transparent display (FDT) and the transparent state is rapidly reciprocated, and the variable reflectance mirror (HMT) is also driven to reciprocate the transparent state and the mirror state rapidly. It is desirable that the user picks up the kinetics when the transparent display and the variable reflectance mirror are in a transparent state.

In order to match a virtual object to a real object reflected on a mirror in a reflective augmented reality system such as a holetirector and output it to a display, the three-dimensional position of the user's point cloud is converted into a mirror plane, It is necessary to check the collision or contact between the surrounding virtual object and the point-symmetrically transformed point cloud to calculate the movement of the virtual object at the next moment, transform the position and shape of the virtual object according to the movement, and output it to the screen. That is, the kinetect must be able to detect not only the position of the user but also the position of the display surface. In the conventional system having the configuration shown in FIG. 1, the user manually inputs the three-dimensional relative position between the key knot and the display through trial and error several times. However, in the case of the configuration of FIG. 2 according to the present invention, It outputs a square mark of the prescribed shape and size to the position, records the mark with the key, and recognizes the mark size and the distorted shape by the image processing program to calculate the three-dimensional relative position between the key knot and the mark, It is possible to quickly and quickly find the relative position of the robot.

As described above, it is preferable that the process of calculating the relative position between the key and the display is performed only once, and the calculated value is written to the memory device, and then the recorded value is read when used next time.

Calculating the three-dimensional relative position between the camera and the mark from the distorted shape and size of the square mark on the image of a rectangular mark of known shape is a technique known as a solution to the perspective 4 point problem.

If the user is not in front of the screen, the user can change the direction of sight by driving the keyboard with a motor to capture the user. In this case, the relative position and direction between the Kinect and the display are changed. In this manner, the relative position and direction between the Kinect and the display are automatically measured in the above manner, and the reflective ARF system can be driven without interruption. That is, in the configuration of FIG. 2, there is no problem even if the photographing direction of the keynote is changed at any time according to the user's movement. However, such a configuration as shown in FIG. 1 has such a problem that the automatic measurement is impossible and the sight line of the keynote is fixed.

By concealing the fact that the Kinect is hidden inside the display, it is possible to hide the fact that the user is being photographed, thereby reducing the feeling of being watched, and can make the appearance of the system more beautiful and provide more novelty.

1 is a block diagram of an existing system;
Fig. 2 is a block diagram
FIG. 3 is a view showing an embodiment of the present invention
Fig. 4 is a view showing an embodiment of the present invention using a hologram optical element

Example 1

The mirror-based augmented reality system according to the present invention as shown in Fig. 2 is an improvement of an existing system (a holographic reflector disclosed by Microsoft Corporation) as shown in Fig. In Figure 1, the Kinect (CM) is mounted on the upper edge of the Half-Mirror (HM) so that when the user approaches very close to the half-mirror, the distance between the Kinect and the user is too narrow to measure the user's three- There is a problem that it can not. In order to solve this problem, the system of the present invention includes a transparent display (FDT) and a certain distance (IT) inside a reflectivity variable mirror (HMT) Configuration. The transparent display is driven so as to reciprocate rapidly in a state of outputting an image and a transparent state, and the reflectivity variable mirror is made to be in a transparent state at the moment of the transparent state of the display and is driven to be in a mirror state at the moment when the display outputs an image, The user takes a picture at a moment when the display and the reflectivity variable mirror are in a transparent state.

In the case of FIG. 1, there is a problem that the system becomes thick due to the gap (IT) between the key knot and the display. This problem can be solved by installing a mirror (RM) that changes the direction of the line of sight of the keynote between the key and the display as shown in Fig.

Fig. 4 shows a configuration in which the thickness of the system is further reduced by using a hologram optical element instead of the mirror of Fig. The hologram optical element has a reflection type or transmission type holographic film (HO) attached to a mirror or a plastic plate (GL) that totally reflects light incident on an optical element used in a near-eye display, And the like.

Instead of such a hologram optical element, a wedge light guide may be used. The introductory document on wedge light guides is entitled wedge optics in flat panel displays ( http://download.microsoft.com/download/4/B/4/4B49C1C2-4C7A-4CEA-ADB5-EF4E4E7F5F63/Wedge%20optics % 20in% 20flat% 20panel% 20displays.pdf ). The wedge type light guide has a structure in which light incident at one end is totally internally reflected and emitted to the other end.

Various depth cameras or stereo cameras may be used in place of the Kinect to detect the user's three-dimensional position.

CM Kinect
HM anti-mirror
FD LCD display
CS users
VO virtual object image
RO mirror user's appearance
FDT transparent display
HMT variable reflectance mirror
IT interval
RM mirror
GL plastic plate
HO holographic film

Claims

In augmented reality system
Transparent display;
A reflectivity variable mirror installed in front of or behind the transparent display;
A video photographing means provided behind the display for photographing a user;
And an image processing means for analyzing the user image photographed by the image photographing means to match the virtual object image with the user image reflected on the mirror and outputting the registered user image to the transparent display

In Item 1
The transparent display rapidly reciprocates between a state of outputting an image and a transparent state
Wherein the image capturing means photographs a user at a moment when the transparent display is transparent.

In Item 1
The reflectivity variable mirror quickly reciprocates between a mirror state and a transparent state
Wherein the image capturing means photographs a user at a moment when the reflectivity variable mirror is transparent.

In Item 1
The image processing means
Display a mark of known size and shape on the display
The mark is photographed by the image photographing means
And recognizes the size and the distorted shape of the mark on the photographed image to calculate a three-dimensional relative position between the image photographing means and the display,
And outputs the resultant image to a mirror-based augmented reality system

In Item 1
Wherein the image capturing means includes an optical element for reducing the volume of the system by changing the viewing direction of the image capturing means.

In Section 5
Wherein the optical element is a mirror, a hologram optical element, or a wedge-shaped light guide.

In Item 1
The image capturing means includes a projection unit for projecting an infrared ray pattern
And a camera unit for photographing an image projected on the object by the infrared ray pattern.

In Item 1
Wherein the image capturing means is a stereo camera.

In Item 1
Wherein the image capturing means is a depth camera.