KR20160093146A - Half mirror based focus-view matching display device and method, recording medium and device for performing the method - Google Patents

Abstract

A half mirror-based focus and view matching display device comprises: a display unit displaying a virtual object; a half mirror installed in a middle portion between a user and the display unit to reflect a mirror image to the display unit; a user tracking unit obtaining position information of the user; and a controller converting the virtual object displayed on the display unit to correspond to the users view, on the basis of the position information of the user, wherein a distance from the user to the virtual object and a distance from the user to the mirror image are equal. Thus, since a focus and a view are matched, the virtual object is able to be displayed intuitively and captively and the virtual object and the user are able to interact.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a half-mirror-based focus and point-of-view display device and method, and a recording medium for performing the method.

The present invention relates to a half mirror-based focus and point-of-view display apparatus and method, and a recording medium for performing the method. More particularly, the present invention relates to a display apparatus and method for displaying a virtual object on a mirror image of a user's eyes, And more particularly, to a focus and point-of-view matching display device and method based on a half mirror capable of displaying an immersive virtual object and interacting with a user by matching distance (focus), and a recording medium for carrying out the method.

At present, information displays are being developed in various forms as new technologies are developed. Recently, a display device based on a half mirror has been developed and a new interface has been provided to a user in combination with various contents such as an augmented reality. However, since the half mirror is installed adjacent to the screen on which the virtual object is displayed, the observation distance of the mirror image on the basis of the user is different from the observation distance of the virtual object, which causes fatigue of the user's eyes and deteriorates the immersion feeling.

In order to solve such a focus mismatch, development of a depth camera that estimates a user's position and development of a color camera for relationship estimation are suggested. However, it is necessary to construct a new equipment and time for calculating the positional relationship between a color camera and a display Thus limiting immediate interaction.

KR 2013-0059650 A KR 2013-0139280 A KR 2013-0126623 A

 Jae Seok Jang et al., "Two-Phase Calibration for a Mirror Metaphor Augmented Reality System," in Proceedings of the IEEE, Vol. 102, No. 2, February 2014, pp. 197-203

SUMMARY OF THE INVENTION Accordingly, the present invention has been made keeping in mind the above problems occurring in the prior art, and it is an object of the present invention to provide a display device in which the focus and the viewpoint coincide with each other between a mirror image reflected on the half mirror and a virtual object displayed on the display.

It is another object of the present invention to provide a display method in which the focus and the viewpoint coincide between a mirror image reflected on the half mirror and a virtual object displayed on the display.

It is still another object of the present invention to provide a recording medium on which a computer program is recorded for performing a display method in which a focus and a viewpoint are matched between a mirror image reflected on the half mirror and a virtual object displayed on the display.

According to an embodiment of the present invention, a half mirror-based focus and point-of-view matching display device includes a display unit for displaying a virtual object; A half mirror disposed at a midpoint between the user and the display unit and reflecting the mirror image to the display unit; A user tracking unit for obtaining location information of the user; And a control unit for converting a virtual object displayed on the display unit to correspond to a viewpoint of the user based on the position information of the user, wherein the distance from the user to the virtual object and the distance from the mirror image coincide with each other do.

In an embodiment of the present invention, the distance between the half mirror and the display unit may be equal to the distance between the user and the half mirror.

In an embodiment of the present invention, the half mirror may be movable within a range of the depth of field in the center of the display unit and the user.

In an embodiment of the present invention, the user tracer may include at least one of a color camera and a depth camera.

In an embodiment of the present invention, the user tracking unit may further acquire operation information of the user and provide the obtained operation information to the control unit.

In an embodiment of the present invention, the controller may perform an interaction between the user and the virtual object based on the operation information of the user.

In an embodiment of the present invention, the control unit may calculate a positional relationship between the user tracking unit, the display unit, and the half mirror.

In an embodiment of the present invention, the controller can use a homography-based positional relationship estimation technique using a marker.

In an embodiment of the present invention, the mirror image may be a shape in which the user is reflected on the half mirror.

In an embodiment of the present invention, the mirror image may be a shape in which an actual object is reflected on the half mirror.

According to another aspect of the present invention, there is provided a focus and point-of-view matching display method based on a half mirror, including a display unit for displaying a virtual object and a half mirror for reflecting the mirror image to the display unit. ) Using a display device including a user tracking unit for acquiring position information of a user, and positioning the half mirror at an intermediate point between the user and the display unit; Obtaining a positional relationship between the display unit, the half mirror, and the user tracking unit; Obtaining location information of the user; Converting the virtual object displayed on the display unit to correspond to the viewpoint of the user using the positional information of the user and the positional relationship between the display unit, the half mirror, and the user tracking unit; And displaying the virtual object and the mirror image by matching the distance from the user to the virtual object and the distance from the mirror image.

In an embodiment of the present invention, the distance between the half mirror and the display unit may be equal to the distance between the user and the half mirror.

In an embodiment of the present invention, the half mirror may be movable within a range of the depth of field in the center of the display unit and the user.

In an embodiment of the present invention, the step of positioning the half mirror at a midpoint between the user and the display unit includes moving the half mirror within a range of the depth of field in the center of the user and the display unit Step < / RTI >

In the embodiment of the present invention, the acquiring of the positional relationship between the display unit, the half mirror, and the user tracking unit may use an external camera having a marker attached thereto.

In the embodiment of the present invention, the step of acquiring the positional relationship between the display unit, the half mirror, and the user tracking unit may use a homography-based positional relationship estimation technique using a marker.

In the embodiment of the present invention, the focus and point-in-time matching display method based on the half mirror according to the above may further include obtaining the operation information of the user.

In an embodiment of the present invention, the above-described half mirror-based focus and point-of-view matching display method may further include performing an interaction between the user and the virtual object based on the operation information of the user .

In an embodiment of the present invention, the mirror image may be a shape in which the user or an actual object is reflected on the half mirror.

According to an embodiment of the present invention, a computer program for performing a focus and point-of-view matching display method based on a half mirror is recorded in a computer-readable storage medium.

According to such a half mirror-based focus and point-of-view matching display apparatus, information is transmitted through a virtual object aligned with a user's viewpoint, and the observation distance (viewpoint) of a virtual object is measured with respect to a viewing distance Focus) to reduce eye fatigue. In addition, since the viewpoint between the mirror image reflected on the half mirror and the virtual object displayed on the display coincides with each other, intuitive and immersive virtual objects can be displayed and interact with the user.

1 is a block diagram of a focus and point-in-time display device based on a half mirror according to an embodiment of the present invention.
FIG. 2 is a conceptual diagram of a focus and point-of-sight display device based on the half mirror of FIG. 1;
3 is a conceptual diagram of a focus and point-in-time display device based on a half mirror according to another embodiment of the present invention.
4 is a flowchart of a focus and point-in-time display method based on a half mirror according to an embodiment of the present invention.
5 is a flowchart of a focus and point-in-time display method based on a half mirror according to another embodiment of the present invention.
6 is a conceptual diagram for explaining calculation of a positional relationship between devices according to an embodiment of the present invention.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

1 is a block diagram of a focus and point-in-time display device based on a half mirror according to an embodiment of the present invention. FIG. 2 is a conceptual diagram of a focus and point-of-sight display device based on the half mirror of FIG. 1; 3 is a conceptual diagram of a focus and point-in-time display device based on a half mirror according to another embodiment of the present invention.

A focus and point-of-view display device 10 (hereinafter, referred to as a device) based on a half mirror according to the present invention uses a half mirror to show a virtual object on a mirror of a user, , The observation distance between them is matched to provide a display capable of displaying a virtual object with immersive feeling and interacting with the user.

Focus and viewpoint matching displays can combine real and virtual environments and provide a three-dimensional environment capable of real-time interaction. In addition, the mirror display is a digital information interface, aiming at customized information transmission, and performs controller-based user interaction. Furthermore, it can be used in fields such as entertainment, edutainment, digital signage, and health care.

Referring to FIG. 1, an apparatus 10 according to the present invention includes a half mirror 100, a display unit 300, a user tracking unit 500, and a control unit 700. The half mirror 100, the display unit 300, the user tracking unit 500, and the controller 700 may be separated from each other by a separate module or may be formed as an integrated unit .

The device 10 may be a separate terminal or some module of the terminal. For example, the device 10 may be a virtual reality display device, an augmented reality display device, a mirror display device, a transparent display device, a head mounted display (HMD) Device or the like.

The half mirror 100 is a mirror for passing a certain amount of light and reflecting a certain amount of light. The half mirror 100 reflects a user or an actual object to provide a virtual object. The half mirror 100 is positioned at an intermediate point between the user and the display unit 300 so that the viewing distance between the reflected mirror image and the virtual object displayed by the display unit 300 can be matched.

The display unit 300 is a screen for displaying a virtual object, and provides a virtual reality, an augmented reality, a 2D or a 3D screen. In addition, a mirror image reflected from the half mirror 100 can be displayed together with the virtual object.

The display unit 300 may include an organic light-emitting diode (OLED) display panel, a liquid crystal display (LCD) panel, and a plasma display panel (PDP). In addition, a touch screen function may be included in the display unit 300 or may be provided as a separate touch pad device to process user input. Alternatively, the apparatus 10 may include an input unit (not shown) such as a keypad that is formed separately from the display unit 300 and receives a user's input.

The user tracking unit 500 obtains the location information of the user and provides the obtained information to the controller 700. [ The user tracking unit 500 is movable and may include at least one of a color camera and a depth camera, and may include both a color camera and a depth camera.

For example, the position information of the user can be obtained by using the color camera and the depth camera. Also, the user tracking unit 500 can acquire operation information of a user using a depth camera.

The control unit 700 converts a virtual object displayed on the display unit 300 according to the user's viewpoint, based on the positional information of the user and the positional relationship between the devices.

Also, the controller 700 performs an interaction between the user and the virtual object based on the operation information of the user.

The controller 700 may use a homography-based positional relationship estimation technique using an external camera and a marker. The controller 700 may be a separate terminal or a module of the terminal.

The control unit 700 may be movable or fixed. The controller 700 may be in the form of a server or an engine and may be a device, an apparatus, a terminal, a user equipment (UE), a mobile station (MS) a wireless device, a handheld device, and the like.

The control unit 700 may be manufactured by various software based on an operating system (OS), i.e., a system. The operating system is a system program for allowing software to use the hardware of the apparatus, and may include a computer operating system such as a Windows system, a Linux system, a Unix system, a MAC, AIX, and HP-UX.

The control unit 700 of the present invention may be implemented with software (application) for performing a focus and point-of-view matching display based on a half mirror. The control unit 700 may control the position of the half mirror 100, The user tracking unit 500 can be controlled.

Referring to FIG. 2, information is transmitted through a virtual object aligned with a user's point of view, and the half mirror 100 is rotated by a user (not shown) to match the observation distance (focus) And the display unit 300, as shown in FIG. That is, the distance between the half mirror 100 and the display unit 300 is equal to the distance between the user and the half mirror 100. This can be expressed by the following equation (1).

[Equation 1]

d _observe = 2 xd _mirror = 2 xd _image = d _display

Here, d _observe is the mirror image observation distance, that is, indicates the distance between the user and the mirror image, d _mirror represents the distance of the user and the half mirror 100, d _image is the distance between the mirror image and the half mirror 100 And d _display represents the observation distance of the virtual object, that is, the distance between the user and the display unit 300. [

The display device based on the conventional half mirror is installed adjacent to the screen on which the virtual object is displayed and there is a problem of the focus mismatch because the mirror observation distance based on the user and the observation distance of the virtual object are different. Such a focus mismatch has a disadvantage of causing fatigue of the user's eye and decreasing the feeling of immersion.

In the present invention, since the half mirror 100 is positioned at the halfway point between the user and the display unit 300, the viewing distance between the user's viewpoint and the mirror image reflected through the half mirror 100 coincides . Accordingly, not only fatigue of the user's eyes can be reduced, but also an intuitive and immersive display can be provided.

In addition, the half mirror 100 is located at a midpoint between the user and the display unit 300, but may be moved within a certain range.

Referring to FIG. 3, the half mirror 100 may be positioned within the depth of field with respect to the center of the user and the display unit 300.

The depth of field means that when a subject at a certain distance is focused, the subject is focused within a certain distance between the front point and the backward point.

The observation distance (focus) between the user's viewpoint and the mirror image projected through the half mirror 100 can coincide with each other even when the half mirror 100 is positioned within the depth of field depth. The control unit 700 may control the half mirror 100 to move within the range of the depth of field according to the user's viewpoint.

The distance between the half mirror 100 and the display unit 300 may be more than half the distance between the user and the half mirror 100 or the distance between the user and the display unit 300 The distance may be at least twice the distance between the user and the half mirror 100.

Alternatively, the distance between the user and the display unit 300 may be at least twice the distance between the half mirror 100 and the display unit 300. That is, in the present invention, the half mirror 100 may be fluidly positioned at a predetermined interval in the central portion of the user and the display unit 300.

The present invention can be applied to a large display device that provides a virtual object display, an augmented reality, and a virtual reality.

In the present invention, since a separate device or calculation process for matching the focus of the user is omitted, efficient interaction between the user and the virtual object can be performed, thereby providing an intuitive and immersive display.

In addition, the present invention can be applied to a market and a field in which a motion map is required as a motion map system using a focus and viewpoint matching system. For example, it can be used for posture correction in Fitness, and for motion correction in Sports. In addition, it is possible to induce training in rehabilitation and apply it to edutainment fields such as entertainment field and sign language learning as motion game contents.

Further, the present invention has a remarkable technical differentiation from the prior art. Specifically, the trainer's motion is guided by the motion of the trainer to improve the expertise, and the temporal and spatial constraints are reduced, so that the accessibility can be improved regardless of the on-line and off-line applications. In addition, various visualization techniques and application of game contents are possible, so that amusement can be secured and the display can be expanded to various types of displays. In addition, it can be expected that the product can be manufactured through the commercialized equipment and the universal desktop system can be used, thereby reducing the manufacturing and management cost.

4 is a flowchart of a focus and point-in-time display method based on a half mirror according to an embodiment of the present invention. 5 is a flowchart of a focus and point-in-time display method based on a half mirror according to another embodiment of the present invention. 6 is a conceptual diagram for explaining calculation of a positional relationship between devices according to an embodiment of the present invention.

The focus and point-of-view matching display method based on the half mirror according to the present embodiment can proceed in substantially the same configuration as the apparatus 10 of Fig. Therefore, the same constituent elements as those of the apparatus 10 of FIG. 1 are denoted by the same reference numerals, and repeated description is omitted. In addition, the focus and point-in-time matching display method based on the half mirror according to the present embodiment can be executed by software (application) for performing focus and point-in-time matching display based on the half mirror.

Referring to FIG. 4, in the focus and point-of-view matching display method based on the half mirror according to the present embodiment, the half mirror is positioned at a midpoint between the user and the display unit (step S10).

The half mirror is a mirror that passes a certain amount of light and reflects a certain amount of light, and reflects a user or an actual object to provide a virtual object. The half mirror is located at an intermediate point between the user and the display unit so that the viewing distance between the reflected mirror image and the virtual object displayed by the display unit can be matched. That is, the distance between the half mirror and the display unit is equal to the distance between the user and the half mirror.

In addition, the half mirror is movable within a range of a depth of field in the center of the display unit and the user. The step of placing the half mirror at a midpoint between the user and the display unit may further include moving the half mirror within a range of the depth of field in the center of the user and the display unit.

Accordingly, a virtual object is superimposed on a mirror of a user viewed by a user, and the viewing distances between the virtual objects are matched to provide a display capable of displaying a virtual object with immersive feeling and interacting with the user.

Thereafter, the positional relationship between the display unit, the half mirror, and the user tracking unit is obtained (step S30). At this time, an external camera marked with a marker can be used.

6, the external camera senses the half mirror and the marker attached to the display unit, and calculates the positional relationship between the external camera, the half mirror, and the display unit through homography (T _cm , T _cd Where T _ab is the conversion from a to b, and T _ab = T _ba ^-1 .

Also, the user tracking unit detects a marker attached to a front part of the external camera and calculates a positional relationship between the user tracking unit and the external camera (T _tc ). This calculates the positional relationship between each device.

In addition, the location information of the user is obtained (step S50). In step S50, the position information of the user can be obtained using at least one of the color camera and the depth camera.

A virtual object displayed on the display unit corresponding to the viewpoint of the user is converted using the location information of the user and the positional relationship between the display unit, the half mirror and the user tracking unit, (Step S70).

Thereby, the distance from the user to the virtual object and the distance from the mirror image are matched with each other, and the virtual object and the mirror image are displayed (step S70). Therefore, the fatigue of the user's eyes can be reduced by matching the focus of the user with the viewpoint.

In addition, referring to FIG. 5, the method may further include acquiring operation information of the user (step S91). Thus, the interaction between the user and the virtual object can be performed based on the operation information of the user (step S92).

Accordingly, it is possible to efficiently interact with the user and the virtual object, thereby providing an intuitive and immersive display.

Such a half mirror based focus and viewpoint matching display method may be implemented in an application or implemented in the form of program instructions that may be executed through various computer components and recorded on a computer readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination.

The program instructions recorded on the computer-readable recording medium may be ones that are specially designed and configured for the present invention and are known and available to those skilled in the art of computer software.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like.

Examples of program instructions include machine language code such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules for performing the processing according to the present invention, and vice versa.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the present invention as defined by the following claims. You will understand.

The present invention provides an intuitive and immersive personalized interactive display by matching the viewpoint between the mirror image reflected on the half mirror and the virtual object displayed on the display. It is necessary to provide market and user-customized information that needs guidance of the user such as rehabilitation training, fitness, sports, edutainment as well as augmented reality or virtual reality display, head mounted display (HMD) Applicable to the market.

10: Half-mirror based focus and viewpoint matching display device
100: half mirror
300:
500: user tracking unit
700:

Claims (20)

A display unit for displaying a virtual object;
A half mirror disposed at a midpoint between the user and the display unit and reflecting the mirror image to the display unit;
A user tracking unit for obtaining location information of the user; And
And a control unit for converting a virtual object displayed on the display unit to correspond to a viewpoint of the user based on the position information of the user,
Wherein a distance from the user to the virtual object is equal to a distance from the mirror image to the virtual object.
The method according to claim 1,
Wherein the distance between the half mirror and the display unit is equal to the distance between the user and the half mirror.
The method according to claim 1,
Wherein the half mirror is movable within a range of depth of field in the middle of the user and the display unit.
The apparatus according to claim 1,
A color camera, and a depth camera. ≪ Desc / Clms Page number 24 >
The apparatus according to claim 1,
And further acquires operation information of the user and provides the operation information to the control unit.
6. The method of claim 5,
Wherein the controller performs an interaction between the user and the virtual object based on the operation information of the user.
The method according to claim 1,
Wherein the control unit calculates a positional relationship between the user tracking unit, the display unit, and the half mirror, based on a half mirror.
8. The method of claim 7,
Wherein the control unit uses a homography-based positional relationship estimation technique using a marker, and a half mirror-based focus and point matching display unit.
The method according to claim 1,
Wherein the mirror image is a shape reflected by the user to the half mirror.
The method according to claim 1,
Wherein the mirror image is a shape in which an actual object is reflected on the half mirror.
A half mirror-based focus and point matching display method using a display unit that displays a virtual object, a half mirror that reflects a mirror image to the display unit, and a user tracking unit that acquires position information of a user silver,
Positioning the half mirror at a midpoint between the user and the display;
Obtaining a positional relationship between the display unit, the half mirror, and the user tracking unit;
Obtaining location information of the user;
Converting the virtual object displayed on the display unit to correspond to the viewpoint of the user using the positional information of the user and the positional relationship between the display unit, the half mirror, and the user tracking unit; And
And displaying the virtual object and the mirror image by matching a distance from the user to the virtual object and a distance from the mirror image to the virtual object, thereby displaying the virtual object and the mirror image.
12. The method of claim 11,
Wherein the distance between the half mirror and the display unit is equal to the distance between the user and the half mirror.
12. The method of claim 11,
Wherein the half mirror is movable within a range of the depth of field in the center of the user and the display unit.
12. The method of claim 11, wherein the positioning the half mirror at an intermediate point between a user and the display unit comprises:
Further comprising moving the half mirror within a range of depth of field in the middle of the user and the display unit.
12. The method of claim 11, wherein obtaining the positional relationship between the display unit, the half mirror,
A focus and point-of-view matching display method based on a half mirror using an external camera with a marker attached thereto.
16. The method of claim 15, wherein obtaining the positional relationship between the display unit, the half mirror,
A half mirror based focus and point matching display method using homography based positional relationship estimation technique using markers.
12. The method of claim 11,
Further comprising the step of acquiring the motion information of the user.
18. The method of claim 17,
Further comprising performing interaction between the user and the virtual object based on the motion information of the user.
12. The method of claim 11,
Wherein the mirror image is a shape in which the user or an actual object is reflected on the half mirror.
A computer-readable recording medium on which a computer-readable recording medium for performing a focus and point-in-time matching display method based on a half mirror according to any one of claims 11 to 19 is provided.

Images