KR101802308B1 - Device for 3D augmented reality display based on user's vision and system for providing plural users with 3D augmented reality at same time - Google Patents

Abstract

The present invention relates to a three-dimensional augmented reality image viewing apparatus corresponding to a real-time user time and a three-dimensional augmented reality implementation system in which a plurality of users simultaneously participate, A three-dimensional augmented reality image viewing apparatus for providing real-time three-dimensional augmented reality images by synthesizing event information, and generating and providing virtual objects and event information corresponding to each of a plurality of users using the apparatus, We present a 3D realization system that can participate at the same time.

Description

TECHNICAL FIELD [0001] The present invention relates to a three-dimensional augmented reality image display apparatus and a three-dimensional augmented reality image generation apparatus, time}

The present invention relates to a three-dimensional augmented reality image viewing apparatus corresponding to a real-time user time and a three-dimensional augmented reality implementation system in which a plurality of users simultaneously participate. More specifically, the present invention relates to a three- A three-dimensional augmented reality image viewing apparatus for providing a real-time three-dimensional augmented reality image by synthesizing a virtual object and event information in one image, and generating and providing a virtual object and event information corresponding to each of a plurality of users using the apparatus, Dimensional real-time realization system in which a user can simultaneously participate in the same augmented reality.

As the performance of the smartphone device is rapidly improved, a variety of augmented reality realization methods using a smartphone have been suggested. Especially, a variety of headgears for realizing an augmented reality through applications mounted on a smartphone are being introduced.

These augmented reality realization devices are a method of presenting a pre-stored image as a three-dimensional augmented reality, rather than a real image corresponding to a user's view, with the headgear worn by the user, and simply implement an augmented reality .

Therefore, it is necessary to implement augmented reality for the real situation that real users see in real time.

Furthermore, since such augmented reality implementing devices provide the augmented reality only to the user with the stored images, there is a limit that a plurality of users can not receive the same augmented reality at the same time.

SUMMARY OF THE INVENTION The present invention has been made to solve the problems of the prior art as described above, and it is an object of the present invention to provide a method of realizing an augmented reality for a real situation corresponding to a time of a user.

In particular, the present invention proposes a method of providing a 3D augmented reality corresponding to a user's real time by using a smart phone possessed by a user.

Furthermore, a plurality of users are provided with the same augmented reality image at the same time, thereby providing a way for users to participate in a 3D augmented reality together.

According to an aspect of the present invention, there is provided a three-dimensional augmented reality image viewing apparatus comprising: two imaging means for generating two photographed images corresponding to a real-time binocular vision of a user; An augmented reality image generation unit for generating augmented reality image by synthesizing a virtual object on each of the two shot images; And an augmented reality image display unit for displaying the augmented reality image on two screens corresponding to the binaural view of the user to implement a three-dimensional augmented reality image corresponding to the real time of the user.

Preferably, the smartphone includes the augmented reality image generation unit and the augmented reality image appearance unit. And a headset to which the smartphone is attached so that the screen of the smartphone corresponds to both sides of the user and is worn on the head of the user.

More preferably, the photographing means is constituted by a combination of at least one camera mounted on the smartphone and at least one camera selected from at least one camera mounted on the headset, wherein the first camera displays one of the binocular eyes of the user And the second camera can generate the photographed image corresponding to another one of the binocular times of the user.

In one embodiment, the first camera is provided in the smart phone, the second camera is provided in the headset, and the headset is configured to transmit the photographed image of the second camera to an augmented reality image And an image transmitting unit for transmitting the image data to the generating unit.

Further comprising an event recognition unit for recognizing an event corresponding to an input signal input by the user and extracting previously stored virtual event information corresponding to the event, wherein the augmented reality image generation unit generates an augmented reality image based on the virtual event information The augmented reality image can be regenerated by adding or changing a virtual object.

Here, the event recognizer may determine the motion of the user by analyzing the photographed image, and may recognize a predetermined motion-based event.

Alternatively, the event recognition unit may receive an input signal from a remote controller operated by the user and recognize an event for each input signal.

Also, the 3D augmented reality implementation system in which a plurality of users concurrently participate according to the present invention includes: a plurality of the 3D augmented reality image viewers individually corresponding to a plurality of users; And a plurality of the three-dimensional augmented reality image displaying apparatuses, receiving the photographed images from the plurality of the three-dimensional augmented reality image displaying apparatuses, generating a common virtual object information by mutually associating the plurality of photographed images, The augmented reality participating device for providing the augmented reality participating device to the device and providing a three-dimensional augmented reality image in which a common virtual object is synthesized corresponding to each of the plurality of users.

Preferably, the augmented reality participating implementer receives the photographed images from each of the plurality of the 3D augmented reality image viewers, collects the plurality of photographed images to search for a common point in the photographed image, Dimensional coordinates of the plurality of photographed images on the basis of the three-dimensional coordinates of the plurality of photographed images; And a virtual object generation unit for generating virtual object information including at least one virtual object and position information of the virtual object based on the three-dimensional coordinates and providing the generated virtual object information to the 3D augmented reality image viewing apparatus, The image viewing apparatus can generate an augmented reality image by synthesizing a virtual object on a photographed image based on the virtual object information.

Furthermore, the augmented reality participating implementer further includes an event processor for receiving a user event from one or more of the plurality of the 3D augmented reality image viewers, and extracting previously held virtual event information corresponding to the event The virtual object generation unit may update the virtual object information based on the virtual event information and provide the virtual object information to the 3D augmented reality image viewing apparatus.

According to the present invention, a three-dimensional (3D) augmented reality corresponding to an actual reality corresponding to a user's time can be provided through a smart phone possessed by a user.

Particularly, even in the case of a smartphone equipped with only one camera, it is possible to mount an image capturing image corresponding to the binocular time of the user on a headset equipped with a camera, thereby realizing a 3D augmented reality corresponding to the real time user time do.

Furthermore, by providing a system in which a plurality of users can simultaneously participate in the same augmented reality, a plurality of users can participate in a 3D augmented reality at the same time in a game such as survival or an online video conference.

FIG. 1 illustrates an embodiment of a 3D augmented reality image viewing apparatus according to the present invention,
FIG. 2 illustrates an embodiment in which a smartphone is mounted on a 3D augmented reality image viewing apparatus according to the present invention,
3 is a block diagram illustrating an embodiment of a 3D augmented reality image viewing apparatus according to the present invention,
4 is a view showing an augmented reality implementation screen through a 3D augmented reality image viewing apparatus according to the present invention,
FIG. 5 shows a three-dimensional augmented reality realization screen corresponding to a user's event through the 3D augmented reality image viewing apparatus according to the present invention,
FIG. 6 illustrates an embodiment of a three-dimensional augmented reality implementation system in which a plurality of users concurrently participate according to the present invention,
FIG. 7 shows a configuration diagram of an embodiment of an augmented reality participating apparatus in a three-dimensional augmented reality implementation system in which a plurality of users concurrently participate according to the present invention,
8 is a flowchart illustrating an embodiment of a method for implementing a 3D AR in which a plurality of users concurrently participate in a 3D AR in a system in which a plurality of users concurrently participate according to the present invention,
9 illustrates an embodiment for collecting position and orientation information for a plurality of users through a 3D augmented reality realization system in which a plurality of users concurrently participate according to the present invention,
10 and 11 illustrate an embodiment in which three-dimensional coordinates are set by collecting photographed images corresponding to real-time times of a plurality of users through a three-dimensional augmented reality realization system in which a plurality of users simultaneously participate, according to the present invention ,
12 is a flowchart of an embodiment of updating a virtual object according to a user event in a 3D AR in a method for implementing a 3D AR in which a plurality of users concurrently participate in a 3D AR realization system in which a plurality of users simultaneously participate Respectively.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.

First, the terminology used in the present application is used only to describe a specific embodiment, and is not intended to limit the present invention, and the singular expressions may include plural expressions unless the context clearly indicates otherwise. Also, in this application, the terms "comprise", "having", and the like are intended to specify that there are stated features, integers, steps, operations, elements, parts or combinations thereof, But do not preclude the presence or addition of features, numbers, steps, operations, components, parts, or combinations thereof.

In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

The present invention proposes a three-dimensional augmented reality image viewing apparatus for providing a real-time three-dimensional augmented reality image by synthesizing a virtual object and event information on an image taken by two or more cameras corresponding to a user's sight, A three-dimensional incremental reality realization system in which a plurality of users can simultaneously participate in the same augmented reality by generating and providing virtual objects and event information corresponding to a plurality of users, respectively.

FIG. 1 shows an embodiment of a 3D augmented reality image viewing apparatus according to the present invention.

The 3D augmented reality image viewing apparatus according to the present invention includes a headset 100 that a user can wear on a head, and a smart phone (not shown) is mounted inside the headset 100.

1 (a), a visual image corresponding to both sides of a user wearing the headset 100 can be photographed through a camera provided in the smart phone, on the front portion of the body 140 of the headset 100 Holes 120 and 130 are formed in correspondence with the camera of the smartphone mounted on the headset 100 and both ends of the rear portion of the body 140 are provided on both sides of the user when the headset 100 is worn on the user's head. A wear band 150 for fixing the headset 100 is connected.

As shown in FIG. 1B, the face contact portion 160 is formed of a soft material such as rubber or silicone so that the rear portion of the headset 100 can be closely attached to the face of the user on the rear face of the headset 100 And lenses 170 and 180 corresponding to both sides of the user are installed inside the back face of the headset 100 so that the user can see the screen of the smartphone installed inside through the lenses 170 and 180. [

The smartphone installed in the headset 100 is equipped with an augmented reality application. The augmented reality application generates augmented reality image by synthesizing virtual objects on each of two captured images through two cameras, The user views the augmented reality image corresponding to the real time of the user on two screens obtained by dividing the screen into two regions, and thus the user can experience a three-dimensional augmented reality corresponding to his or her own viewpoint.

FIG. 2 illustrates an embodiment in which a smartphone is mounted on a 3D augmented reality image viewing apparatus according to the present invention. In the embodiment of FIG. 2, a smart phone is slidably inserted into an upper portion of the headsets 100a and 100b The smart phones 110a and 110b can be mounted on the head sets 100a and 100b by inserting the smart phones 110a and 110b into the smartphone mounting part.

In one embodiment, the method of mounting the smartphone in the headset may be variously modified. For example, a smartphone mount capable of slidably mounting the smartphone on the side of the headset may be formed, The front and rear portions of the headset may be separated so that the smart phone can be mounted on the smartphone mounting portion formed therebetween or the front face portion of the headset can be opened and closed by a door method and a smart phone mounting portion can be formed therein, have.

2A illustrates a case where a smartphone 110a including only one camera 125a is mounted on the headset 100a and a camera 135a is also provided on the headset 100a so that the smartphone 110a The camera 125a mounted on the headset 100a and the camera 135a mounted on the headset 100a.

When the combination of cameras is configured as described above, when the smartphone 110a is mounted on the headset 100a, the camera 125a of the smartphone 110a is moved through the opening hole 121a formed in the front portion of the headset 100a, And the camera 135a provided on the front face of the headset 100a generates the photographed image corresponding to the time of the other one of the eyes of the user.

2B illustrates a case where the smartphone 110b having two cameras 125b and 135b is mounted on the headset 100b and the two cameras 125b and 135b of the smartphone 110b Two aperture holes 121b and 131b are formed in the front face of the headset 100b and two cameras 125b and 135b of the smartphone 110b are attached to the headset 100b when the smartphone 110b is mounted on the headset 100b. Generates a photographed image corresponding to both eyes of the user.

In addition, the imaging means of the 3D augmented reality imaging apparatus according to the present invention may be modified in various ways. For example, even if a smart phone includes a camera, two cameras corresponding to the user's eyes are installed in the headset itself, Or two cameras are installed in the vicinity of the aperture hole with two aperture holes in the front face of the headset so that the camera installed in the headset and the camera provided in the smart phone can be used as a user May be selectively combined to set the photographing means.

The detailed configuration of the 3D ARF image viewing apparatus according to the present invention will be described with reference to a configuration diagram of an embodiment of a 3D ARF image viewing apparatus according to the present invention shown in FIG.

The 3D augmented reality image viewing apparatus according to the present invention basically includes a headset 200 and a smartphone 300, and a remote controller 400 may be additionally configured as needed.

The smartphone mounting unit 210, the photographing unit 230, the image transmitting unit 250, and the like may be optionally configured in the headset 200.

The smartphone mounting part 210 is formed so that the smartphone mounting part 210 can stably fix the smartphone in accordance with the shape of the smartphone mounted inside the headset 200. At this time, 200, and the camera of the smartphone is formed in a shape that can correspond to the opening hole formed in the front portion of the headset 200. [ In addition, the smartphone mounting unit 210 may be provided with a terminal for receiving power or transmitting data from the smartphone, so that a port of the smartphone may be fastened to the terminal.

The photographing means 230 includes at least one camera installed on the front face of the headset 200 and photographs a visual image corresponding to the time of the user wearing the headset 200 through the camera to generate a photographed image . The number and position of the cameras included in the photographing means 230 of the headset 200 can be adjusted as necessary and the number of cameras included in the photographing means 230 of the headset 200, Two cameras can be selected in combination.

The image transmitting unit 250 transmits the photographed image generated by the photographing unit 230 to a smart phone mounted on the headset 200. The image transmitting unit 250 may transmit the photographed image using a short distance communication method such as Bluetooth or the like, Terminal to transmit the photographed image.

As shown in FIG. 1, the headset 200 may have a secondary structure for wearing the headset 200, or a battery for supplying power to each configuration of the headset 200, as shown in FIG.

The augmented reality application 310 is installed in the smart phone 300 installed in the headset 200. The augmented reality application 310 includes an augmented reality image generation unit 320, an augmented reality image appearance unit 330, An event recognition unit 340, and the like.

The augmented reality application 310 may be a kind of computer program and each configuration of the augmented reality application 310 may be implemented by execution of a computer program through a basic configuration mounted on the smartphone 300. [

The smartphone 300 is provided with a photographing means 350 including a camera and a display 360. The augmented reality application 310 can operate in cooperation with the configurations of the smartphones 300 have.

The augmented reality image generation unit 320 generates augmented reality image by combining a plurality of virtual images with each of the two captured images. The augmented reality image generation unit 320 may store various virtual objects in advance, The object can be extracted and synthesized to the shot image, or the virtual object information can be received from the external augmented reality participation implementation device, the virtual object can be extracted from the virtual object information, and synthesized to the shot image.

The augmented reality image display unit 330 displays the augmented reality image generated by the augmented reality image generation unit 320 through the display 360 of the smartphone 300. At this time, The display 360 of the phone 300 can be divided into two regions and the three-dimensional augmented reality image corresponding to the user's real time can be displayed on each screen.

The event recognition unit 340 recognizes an event according to an input signal input by a user, extracts previously stored virtual event information corresponding to the recognized event, and provides the extracted virtual event information to the augmented reality image generation unit 320. [ Then, the augmented reality image generation unit 320 regenerates the augmented reality image by adding or modifying the virtual object based on the virtual event information provided from the event recognition unit 340. [

For this, the event recognition unit 340 may determine the motion of the user by analyzing the captured image, recognize the predetermined motion-based event, or receive the input signal from the remote controller 400 operated by the user, It is also possible to recognize a signal-specific event.

The remote controller 400 may further include an input signal generator 410 for receiving an operation signal from a user and generating an input signal. The input signal generator 410 may be an input signal generator, such as a Bluetooth The event recognition unit 340 of the smartphone 300 can be interlocked with the event recognition unit 340 of FIG.

Here, the remote controller 400 may be modified into various forms according to a situation where the 3D augmented reality image viewing apparatus according to the present invention is applied. It is simply provided with a plurality of buttons so that an input signal Or may be transmitted to the smartphone 300. Alternatively, when the 3D augmented reality image displaying apparatus according to the present invention is applied to a virtual survival game or the like, the remote controller 400 may be modified into various forms such as a firearm.

In the present invention, the above-described three-dimensional augmented reality image viewing apparatus is presented, and a user can receive a three-dimensional augmented reality image corresponding to his or her own time. In this case, An implementation example of a 3D augmented reality provided by the present invention will be described with reference to an augmented reality realization screen through a real image visualizer.

FIG. 4 illustrates an implementation screen for providing an augmented reality image corresponding to a real time of a user participating in the survival game, as a smartphone screen 500 of the 3D augmented reality image viewing apparatus.

When two cameras generate two photographed images corresponding to the binocular time of the user, a virtual object is synthesized on the photographed image and displayed on the screen 500 of the smartphone display. At this time, the screen of the smartphone display is divided into two areas And are displayed as two screens 510a and 510b corresponding to the sides of the user wearing the headset.

The three-dimensional augmented reality image displayed on the smartphone screen 500 includes virtual objects 520a and 520b synthesized on the photographed image and further includes state information 530 , 540) may be displayed.

As described above, according to the present invention, a 3D augmented reality corresponding to a real time situation corresponding to a user's real time can be provided to a user through a smart phone possessed by a user.

Furthermore, in the present invention, an augmented reality image can be reproduced and provided by recognizing an event according to an input signal input by a user and adding or modifying a virtual object corresponding to the event.

For example, the 3D augmented reality image viewing apparatus can analyze a photographed image corresponding to a real time of a user to determine a motion of a user and recognize a predetermined motion-based event. For example, When a motion of a specific hand gesture or the like is taken so that it can be seen from the user's eyes while wearing the headset of the vision device, the 3D augmented reality image vision device analyzes the photographed image to judge the motion of the user's gesture, , It is recognized as a predetermined motion-based event, and the augmented reality image is regenerated by adding or changing the virtual object set corresponding to the event.

For example, when the 3D ARF image viewing apparatus includes a remote controller, a smart phone mounted on a headset of the 3D ARF image viewing apparatus interacts with the remote controller to recognize an operation through the user's remote controller as an event You may.

In this regard, FIG. 5 illustrates a 3-dimensional augmented reality realization screen corresponding to a user's event through the 3-dimensional augmented reality image view apparatus according to the present invention.

5, the three-dimensional augmented reality image display device includes a firearm-type remote controller 650, and a smart phone installed in the headset 610 is connected to the remote controller 650, The augmented reality application installed on the smart phone recognizes the event when the remote controller 650 is operated while the headset 610 is worn by the user. When augmented reality image screen 640 is wobbled or an event occurs in which the user 600 is shot in the virtual environment of the augmented reality, a virtual object 640 corresponding to the collision is added to the augmented reality image screen 640, Can be displayed.

As described above, according to the present invention, a user can feel the virtual environment of the 3D augmented reality full of life by providing or reproducing the augmented reality image by adding or modifying the virtual object according to the event performed by the user.

In addition, the present invention proposes a 3D augmented reality realization system that realizes a 3D augmented reality in which a plurality of users simultaneously participate in a state in which a plurality of users wear the 3D augmented reality image viewing apparatus. A 3D augmented reality realization system according to the present invention will be described with reference to its embodiments.

FIG. 6 illustrates an embodiment of a three-dimensional augmented reality implementation system in which a plurality of users concurrently participate according to the present invention.

The 3D augmented reality realization system according to the present invention includes an augmented reality participant implementation apparatus 700 and a 3D augmented reality image view apparatus 810a, 810b, 810c, 810c, 810b, 810c, 810c, 810d.

The three-dimensional augmented reality image viewers 810a, 810b, 810c, and 810d worn by each of the plurality of users 820a, 820b, 820c, and 820d may be connected to the augmented reality participant 700 by a wireless communication method. A smartphone mounted on the 3D enhanced reality image display devices 810a, 810b, 810c, and 810d can be connected to the augmented reality participating device 700 by various communication methods such as Wi-Fi, Zigbee, and Bluetooth.

The augmented reality participating implementer 700 distinguishes and recognizes each user 820a, 820b, 820c, and 820d of the connected three-dimensional augmented reality image viewers 810a, 810b, 810c, and 810d. For example, Number or other unique identification information, or to distinguish the user by login information when accessing the augmented reality participation device 700 through a smart phone.

In this way, the augmented reality participating implementer 700 distinguishes and recognizes each of the 3D augmented reality image viewers 810a, 810b, 810c, and 810d and generates a virtual object corresponding to each of the users 820a, 820b, 820c, 810b, 810c, and 810d, virtual object information commonly applied to all users 820a, 820b, 820c, and 820d is generated and provided to the corresponding three-dimensional augmented reality image viewers 810a, 810b, The vision devices 810a, 810b, 810c, and 810d synthesize the virtual objects in the real time visual image of the user based on the virtual object information provided from the augmented reality participant implementing device 700, and provide the three dimensional augmented reality images.

The augmented reality participant realization device 700 may be an individual device specialized for applying the present invention, or may be implemented by a commonly-used server, a PC, or the like.

FIG. 7 shows a configuration diagram of an embodiment of an augmented reality participating apparatus of a three-dimensional augmented reality implementation system in which a plurality of users concurrently participate according to the present invention.

The augmented reality participant realization device 700 may include an image collection unit 710, a virtual object generation unit 720, an event processing unit 730, and the like, And a user recognition unit (not shown) for recognizing the user.

The image collection unit 710 receives the real time visual images of the users 820a, 820b, 820c, and 820d from the plurality of three-dimensional augmented reality image display devices 810a, 810b, 810c, and 810d, Collects the photographed images, searches common points in the photographed images, and sets three-dimensional coordinates of the plurality of photographed images based on the common points. In addition, the image collecting unit 710 receives the positional information and the visual direction information of each of the plurality of three-dimensional augmented reality image displaying apparatuses 810a, 810b, 810c, and 810d and collects them, It can also be used to set three-dimensional coordinates.

The virtual object generation unit 720 extracts at least one virtual object among the held virtual objects, generates position information of the extracted virtual object based on the three-dimensional coordinates set by the image collection unit 710, Generates virtual object information including positional information corresponding thereto, and provides the virtual object information to the corresponding three-dimensional augmented reality image display devices 810a, 810b, 810c, and 810d. At this time, the virtual object generation unit 720 may generate virtual object information individually corresponding to each of the plurality of three-dimensional augmented reality image vision devices 810a, 810b, 810c, and 810d, Virtual object information commonly applied to all of the devices 810a, 810b, 810c, and 810d may be generated.

The three-dimensional augmented reality image display devices 810a, 810b, 810c, and 810d generate augmented reality images based on the virtual object information provided from the augmented reality participating implementer 700, and generate the augmented reality images using the users 820a, 820b, 820c, ) To the three-dimensional reality.

Also, the event processor 730 receives user events from the 3D augmented reality image viewing apparatus in which a user event is generated among a plurality of the 3D augmented reality image viewers 810a, 810b, 810c, and 810d, And transmits the extracted virtual event information to the virtual object generation unit 720. The virtual object generation unit 720 updates the virtual object information based on the virtual event information, To the vision devices 810a, 810b, 810c, and 810d.

Each of the components of the augmented reality participant realization device may be included in one device or may be configured as a separate separate device as necessary. In the case where the augmented reality participated realization device is implemented as a server or a PC, May be implemented by the execution of a computer program through a basic configuration mounted on the augmented reality participant implementation device.

The present invention proposes a method for providing augmented reality in which a plurality of users participate simultaneously through the above-described three-dimensional augmented reality realization system. Referring to FIG. 8, which is a flowchart of an embodiment thereof, A method for providing augmented reality participating concurrently will be described.

In the following embodiments, the identification code of the 3D augmented reality image display device and the identification code of the headset are used in combination for convenience of explanation. In addition, in Fig. 8, the operation of two 3D augmented reality image viewers In one embodiment, three or more 3D augmented reality image viewers may be interlocked with the augmented reality participant implementer.

Although not shown in FIG. 8, it is assumed that a process of connecting the 3D enhanced-reality image display device # 1 810a and the 3D enhanced-reality image display device # 2 810b to the augmented reality participating device 700 is assumed, The augmented reality participant realization device 700 receives the 3D augmented reality image view device # 1 810a and the 3D augmented reality image view device # 2 810b when the 3D augmented reality image view device # 1 810a and the 3D augmented reality image view device # A process of recognizing each of the three-dimensional augmented reality image vision devices 810a and 810b through the unique identification information and the login information of the 3D augmented reality image view device # 2 (810b) can be performed first.

The augmented reality participant realization device 700 transmits position and direction information from the 3-dimensional augmented reality image view device # 1 810a and the 3-D augmented reality image view device # 2 810b connected by wireless communication (S110a, 110b.

At this time, the smartphone mounted on the 3D augmented reality image viewing apparatus can grasp the position information of the user through the GPS information, and also can grasp the photographing direction of the camera through a gyro sensor provided in the smart phone The three-dimensional augmented reality image viewing apparatus grasps position and direction information through a smart phone and transmits the information to the augmented reality participating implementer 700. [

The augmented reality participating implementer 700 collects the position and orientation information transmitted from the 3D augmented reality image view device # 1 810a and the 3D augmented reality image view device # 2 810b (S120) The relative position and direction between the real image display device # 1 810a and the 3D augmented reality image display device # 2 810b are grasped.

In this regard, FIG. 9 illustrates an embodiment for collecting position and orientation information for a plurality of users through a 3D AR system in which a plurality of users simultaneously participate according to the present invention.

As shown in FIG. 9, each of the three-dimensional augmented reality image viewers grasps position information and camera direction information measured by the gyroscope based on GPS information, transmits the information to the augmented reality participating implementer 700, The real participation device 700 can collect the position and orientation information transmitted from each 3D augmented reality image display device and grasp a space corresponding to the time of each user wearing the 3D augmented reality image viewing device. A three-dimensional space scale for a virtual environment in which a plurality of users participate can be set based on a position and a viewing direction for each of a plurality of users, and a relative position of each user can be confirmed in the three-dimensional space.

In FIG. 9, the three-dimensional space can be grasped although the two-dimensional space has been described as grasping for convenience of explanation.

Returning to FIG. 8 again, a method for providing augmented reality in which a plurality of users concurrently participate will be continuously examined.

The 3D ARF image viewing apparatus # 1 810a generates the captured image # 1 corresponding to the user's real time and transmits it to the augmented reality participating implementing apparatus 700 (S140a). The 3D ARF image viewing apparatus # 2 810b also generates the shot image # 2 corresponding to the user's real time and transmits it to the augmented reality participating implementer 700 (S140b).

Then, the augmented reality participating implementer 700 receives the photographed image # 1 and the photographed image # 1 based on the position and orientation information of the 3D augmented reality image view device # 1 810a and the 3D augmented reality image view device # The image # 2 is collected (S150), and a common point is searched in the captured image # 1 and the captured image # 2 (S160) to match the captured image # 1 and the captured image # 2 based on the common point. Through this, it is possible to recognize the terrain and the object in the three-dimensional space synthesizing the views of the users through the images of the 3D augmented reality image viewing apparatus # 1 (810a) and the 3D augmented reality image viewing apparatus # 2 (810b) And sets three-dimensional coordinates of the recognized three-dimensional space (S170).

As an example, a three-dimensional augmented reality realization system in which a plurality of users concurrently participate in the present invention shown in Figs. 10 and 11 collects photographed images corresponding to real time of a plurality of users and sets three-dimensional coordinates 10A is an image # 1 830a of the 3D augmented reality image displaying apparatus # 1 810a worn by the user # 1 820a, and FIG. (b) is a photographed image # 2 (830b) of the 3D augmented reality image displaying apparatus # 2 810b worn by the user # 2 820b and FIG. 10 (c) 3 (830c) of the 3-D augmented reality image preview device # 3 810c that is worn.

The augmented reality participating implementer 700 grasps the terrain and the objects from the photographed images 830a, 830b and 830c transmitted from each of the three-dimensional augmented reality image viewers 810a, 810b and 810c, 830b, and 830c in the case of FIG. 10 in which the captured images 830a, 830b, and 830c are collected based on the positions of the captured images 830a, 820b, and 820c, b2 and b3 can be searched as a common point and the objects a1, a2 and a3 etc. captured in the photographed images 830a, 830b and 830c can be searched as a common point.

Based on the searched common points, the augmented reality participating implementer 700 integrates the captured images 830a, 830b, and 830c as shown in FIG. 11 by matching the captured images 830a, 830b, and 830c, The three-dimensional coordinates are set for the three-dimensional space generated based on the common points (a, b, c, d, and e). In FIG. 11, two-dimensional coordinates are shown for convenience of explanation, but three-dimensional coordinates can be set substantially.

8, a method for providing augmented reality in which a plurality of users concurrently participate is described.

When the three-dimensional coordinates corresponding to the time of each user who wears the 3D augmented reality image displaying apparatus # 1 810a and the 3D augmenting reality image displaying apparatus # 2 810b are set, Selects and extracts one or more of the held virtual objects, generates virtual object information including the position information of the virtual object and the content of the corresponding virtual object as to where the extracted virtual object is located on the three-dimensional space S180).

Here, the virtual object information is divided into individual virtual object information to be applied only to a specific 3D ARP image viewing apparatus and common virtual object information to be applied to all 3D ARP image viewing apparatuses connected to the ARPU 700 .

Then, the augmented reality participant implementing apparatus 700 transmits the generated virtual object information to the 3D augmented reality image viewers # 1 810a and 3D augmented reality image viewers # 2 810b (S190a, S190b).

Furthermore, the augmented reality participant implementing apparatus 700 may acquire coordinate information about the user position or coordinate information about the photographed image # 1 in the three-dimensional space with the 3D augmented reality image view device # 1 810a before transmitting the virtual object information Or may be transmitted together with the virtual object information. Similarly, the coordinate information for the user position or the coordinate information for the photographing image # 2 may be transmitted to the 3D augmented reality image displaying device # 2 810b.

When the virtual object information is transmitted from the augmented reality participation implementation device 700, the 3D augmented reality image display device # 1 810a synthesizes a virtual object in S210a to generate an augmented reality image S220a, The 3D augmented reality image viewing apparatus # 1 810a extracts the contents and location information of the virtual object from the virtual object information and stores the contents of the virtual object in association with the location information in the captured image corresponding to the real time of the user Thereby generating an augmented reality image. Preferably, the three-dimensional augmented reality image viewing apparatus # 1 810a can grasp the position of the virtual object on three-dimensional coordinates based on the coordinate information on the user position and the coordinate information on the photographing image # 1, The virtual object can be synthesized on the photographed image that is changed in real time.

As described above, according to the present invention, a plurality of users simultaneously participate in the same augmented reality by implementing augmented reality as shown in FIG. 4 for each of a plurality of users by generating and providing virtual objects corresponding to each of a plurality of users .

Further, in the present invention, when an event is generated by a specific user on augmented reality in which a plurality of users participate, a three-dimensional augmented reality may be implemented by reflecting the event. In this regard, FIG. 4 is a flowchart illustrating an embodiment of updating a virtual object according to a user event in a 3D AR in a method for implementing a 3D AR in real time by a plurality of users concurrently participating in the AR.

9 is performed after the embodiment of FIG. 9 is performed. In a situation where a plurality of users simultaneously participate in the same evolved reality, the user # 1 is allowed to view the three-dimensional augmented reality image view device # 1 When the event is generated through the augmented reality participant implementing apparatus 700, the 3D augmented reality view apparatus # 1 810a detects the generated event and transmits the sensed event to the augmented reality participating apparatus 700 (S310).

In this case, as described above, the three-dimensional augmented reality image display device # 1 810a recognizes the motion of the user or the operation of the remote controller by the three-dimensional augmented reality image display device # 1 810a, And can be performed.

The augmented reality participating implementing device 700 has virtual event information corresponding to various events and searches for virtual event information corresponding to the event transmitted from the 3D augmented reality image displaying device # )do.

The augmented reality participating implementer 700 determines whether the extracted virtual event information is applied to the 3D augmented reality image view device worn by a specific user or to the 3D augmented reality image view device worn by all users And determines the user to which the event is to be applied (S330).

Thereafter, the augmented reality participating implementing device 700 updates the virtual object information based on the extracted virtual inventory information (S340). Here, the virtual object information is a state in which a new object is added or an existing object is changed And may include object content and location information for the object.

When the virtual object information is updated, the augmented reality participating implementing device 700 transmits the updated virtual object information to the 3D augmenting reality image displaying device (S350a, S350b). At this time, according to the determination of the user to which the event is to be applied, And transmits the updated virtual object information to the three-dimensional augmented reality image view device.

When receiving the virtual object information from the augmented reality participating implementer 700, the 3D augmented reality image viewers 810a and 810b synthesize the augmented reality image with the photographed image as described above with reference to FIG. And presents the 3D augmented reality image to the user. Accordingly, as shown in FIG. 5, the user experiences a three-dimensional augmented reality reflecting the event effect due to the addition or change of the virtual object.

As described above, according to the present invention, when a plurality of users participate in augmented reality, augmented reality is newly constructed and provided by reflecting the occurrence of an event for a specific user, thereby allowing a plurality of users to participate in a common augmented reality.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments of the present invention are not intended to limit the scope of the present invention but to limit the scope of the present invention. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents thereof should be construed as being included in the scope of the present invention.

100, 100a, 100b, 200, 610, 810a, 810b, 810c, 810d:
110a, 110b, 300: smart phone,
210: smartphone mounting unit, 230: photographing means,
250: image transmission unit,
310: augmented reality application, 320: augmented reality image generation unit,
330: augmented reality image display unit, 350: event recognition unit,
350: photographing means, 350: display,
400, 650: remote controller,
410: input signal generating unit,
700: augmented reality participation device,
710: image collecting unit, 720: virtual object creating unit,
730: Event processing unit.

Claims (10)

Two shooting means for generating two shot images corresponding to the user's real-time binocular time; and
An augmented reality image generation unit for generating augmented reality image by synthesizing a virtual object on each of the two shot images; And an augmented reality image display unit for displaying the augmented reality image on two screens corresponding to the user's eyes and realizing a three-dimensional augmented reality image corresponding to the real time of the user. And
And a headset to which the smartphone is attached so that a screen of the smartphone corresponds to both sides of the user and is worn on the head of the user,
Wherein,
A first camera mounted on the smart phone and a second camera mounted on the headset,
Wherein the first camera generates a photographed image corresponding to one of the binocular eyes of the user and the second camera generates a photographed image corresponding to another one of the binocular eyes of the user, 3D augmented reality image vision device.
delete delete The method according to claim 1,
The headset includes:
Further comprising an image transmission unit for transmitting an image captured by the second camera to an augmented reality image generation unit mounted on the smartphone. The method according to claim 1,
And an event recognition unit for recognizing an event according to an input signal input by the user and extracting virtual event information corresponding to the event,
Wherein the augmented reality image generation unit comprises:
Wherein the augmented reality image is regenerated by adding or modifying a virtual object based on the virtual event information. 6. The method of claim 5,
The event recognizing unit,
Wherein the motion of the user is determined by analyzing the captured image, and the predetermined motion-based event is recognized. 6. The method of claim 5,
The event recognizing unit,
Wherein the controller receives the input signal from the remote controller operated by the user and recognizes the event for each preset input signal. A plurality of 3D augmented reality image viewing devices according to any one of claims 1 and 4 to 7 corresponding individually to a plurality of users; And
A plurality of the 3D augmented reality image displaying apparatuses are respectively received from the three-dimensional augmented reality image displaying apparatuses, a plurality of the captured images are mutually associated to generate common virtual object information, And an augmented reality participation implementing device for providing the augmented reality participating device
A three-dimensional augmented reality image, in which a plurality of users are associated with a common virtual object corresponding to each time, is provided. A three-dimensional augmented reality system 9. The method of claim 8,
Wherein the augmented reality participating implementer comprises:
A plurality of the three-dimensional augmented reality image viewing apparatuses are received from each of the three-dimensional augmented reality imaging apparatuses, a plurality of the captured images are collected to search for a common point in the captured image, A video collecting unit for setting the dimensional coordinates; And
And a virtual object generation unit for generating virtual object information including at least one virtual object and position information of the virtual object based on the three-dimensional coordinate and providing the generated virtual object information to the 3D augmented reality image viewing apparatus,
Wherein the three-dimensional augmented reality image viewing apparatus generates augmented reality image by synthesizing a virtual object on a photographed image based on the virtual object information. 10. The method of claim 9,
Wherein the augmented reality participating implementer comprises:
A three-dimensional augmented reality image viewing apparatus according to claim 5,
Wherein the augmented reality participating implementer comprises:
Further comprising an event processor for receiving a user event from one or more of the plurality of 3D augmented reality image viewers and extracting virtual event information corresponding to the event,
Wherein the virtual object generation unit updates the virtual object information based on the virtual event information and provides the updated virtual object information to the 3D augmented reality image viewing apparatus.

Images