KR20120000695A - System for generating 3d mobile augmented reality based on the user - Google Patents

Abstract

PURPOSE: A three dimensional mobile augmented reality creation system and method thereof are provided to enable a user to search content by providing a mobile augmented reality service to the user. CONSTITUTION: A content storage unit(120) stores content. An actual image acquisition module(160) acquires an actual image. A 3D rendering module(200) searches predetermined content in the content storage unit through the acquired location of the actual image. The 3D rendering model changes the searched content into a 3D virtual model. An image matching module(170) matches the 3D virtual model with the actual image.

Description

System for generating 3D mobile augmented reality based on the user}

The present invention relates to a 3D mobile augmented reality generation system and method for user participation, and in detail, can provide a social network service (SNS) suitable for augmented reality, can reflect the user behavior as it is, the real world as possible Similarly, the 3D model can be prevented from being separated from reality, and it is suitable for mobile augmented reality (mixed reality) where hardware performance is greatly influenced, the content can be searched over time, and the existing short message is It relates to a 3D mobile augmented reality generation system and method for user participation that can escape the form of SNS through.

As mobile AR goes beyond providing local information, services that leave their traces in augmented reality and also connect to social network services (SNS) are beginning to emerge. However, Mobile AR and SNS, which are currently being tried, are merely a combination of two functions and are not providing SNS suitable for AR. In particular, the existing information-oriented forms will be inconvenient to use because they are shown in 2D form.

According to an embodiment of the present invention, an object of the present invention is to provide a 3D mobile augmented reality generation system and method for providing a social network service (SNS) suitable for augmented reality.

In addition, according to an embodiment of the present invention, it provides a user participation-oriented 3D mobile augmented reality generating system and method that can reflect the user's behavior of leaving a memo or picture as a post-it (attach) For that purpose.

In addition, according to an embodiment of the present invention, to provide a user participation-oriented three-dimensional mobile augmented reality generation system and method to prevent the 3D model from being separated from the reality as close to the real world as possible.

In addition, according to an embodiment of the present invention, since the values of various sensors are used as they are, a system and method for generating 3D mobile augmented reality for user participation suitable for mobile augmented reality (mixed reality) having a great effect on hardware performance. To provide that purpose.

In addition, according to an embodiment of the present invention, it is an object of the present invention to provide a 3D mobile augmented reality generation system and method for user content search over time.

In addition, according to an embodiment of the present invention, it is an object of the present invention to provide a 3D mobile augmented reality generation system and method that can escape the form of SNS through the existing short message.

3D mobile augmented reality generating system for user participation according to an embodiment of the present invention, the content storage unit for storing the content; A live action acquisition module for acquiring a live action image; A 3D rendering model that searches for a predetermined content in the content storage unit and renders the searched content as a 3D virtual model by using the position at which the live image is obtained and the direction facing toward the live image; And an image registration module for generating an augmented reality by matching the rendered 3D virtual model with the real image.

Direction, location, and time information may be mapped to each content stored in the content storage unit.

The content stored in the content storage unit may be mapped to direction and location information in the actual space.

According to an embodiment of the present invention, a 3D mobile augmented reality generating system for user participation includes: a user input module configured to receive a search time for searching for a content mapped to a specific time in contents stored in the content storage unit; The 3D rendering module may further search for a predetermined 3D content in the content storage unit by using the location where the live image is obtained, the direction in which the live image is viewed, and the search time, and the searched content may be a 3D virtual model. May be rendered as

3D mobile augmented reality generating system for user participation according to an embodiment of the present invention, the sensor measurement module for sensing the illuminance of the live-action image obtained by the live-action acquisition module; The 3D rendering module is the search When rendering a content as a 3D virtual model, the content may be rendered by reflecting the illumination sensed by the sensor measurement module.

3D mobile augmented reality generating system for user participation in accordance with an embodiment of the present invention, the positioning module for obtaining the location information of the live-action acquisition module; The 3D rendering module, the location of the live-action acquisition module The content storage unit may search for all content located within a predetermined distance from the real picture acquisition module obtained from the live action acquisition module, and render the searched content as a 3D virtual model.

According to an embodiment of the present invention, a 3D mobile augmented reality generating system for user participation includes: a user input module configured to receive a search time for searching for a content mapped to a specific time in contents stored in the content storage unit; And a positioning module for acquiring position information of the due diligence obtaining module, wherein the 3D rendering module includes: The content storage unit may search for all content located within a predetermined distance toward the obtained live-action, and render the searched content as a 3D virtual model.

The direction mapped to the contents stored in the content storage unit may be defined as a normal vector direction which is completely opposite to the direction facing the front of the content.

The content retrieved by the 3D rendering module may have a difference of 180 ± ΔD ° between a direction from the location of the live action acquisition module to the live action acquired by the live action acquisition module and a normal vector direction mapped to the content.

The absolute value of ΔD ° may be less than 90 °.

A method for generating 3D mobile augmented reality for user participation according to an embodiment of the present invention includes: storing content in a storage unit; Obtaining a live image; Searching for a predetermined content stored in the storage unit by using the position where the actual image is obtained and the direction facing toward the actual image, and rendering the retrieved content as a 3D virtual model; And generating an augmented reality by matching the rendered 3D virtual model with the real image.

Direction, position, and time information may be mapped to each content stored in the storage unit.

The content stored in the storage unit may be mapped with direction and location information in the actual space.

In accordance with an aspect of the present invention, there is provided a method of generating 3D mobile augmented reality for user participation, comprising: receiving a search time for searching for a content mapped to a specific time from contents stored in the storage unit from a user; The 3D rendering module may search for a predetermined 3D content in the storage unit by using the location where the live image is obtained, the direction of viewing the live image, and the search time, and convert the retrieved content into a 3D virtual model. May be rendering.

3D mobile augmented reality generating method according to an embodiment of the present invention, the sensor measurement module for sensing the illuminance of the live-action image obtained by the live-action acquisition module; The 3D rendering module is the search When rendering a content as a 3D virtual model, the content may be rendered by reflecting the illumination sensed by the sensor measurement module.

3D mobile augmented reality generating method according to an embodiment of the present invention, positioning module for obtaining the location information of the live-action acquisition module; The 3D rendering step, the live action acquisition module The content storage may search for all content located within a predetermined distance from the location toward the live action obtained by the live action acquisition module, and render the searched content as a 3D virtual model.

According to an embodiment of the present invention, there is provided a method for generating 3D mobile augmented reality for user participation, comprising: receiving a search time for searching for a content mapped to a specific time from contents stored in the storage unit; And acquiring location information of the location where the live action is obtained; wherein the rendering comprises: from the location of the location where the live action is obtained from among contents mapped with the same time as the search time. The storage unit may search for all contents located within a predetermined distance in a direction toward and render the searched contents as a 3D virtual model.

The direction mapped to the contents stored in the storage unit may be defined as a normal vector direction which is completely opposite to the direction facing the front of the contents.

The searched content may have a difference of 180 ± ΔD ° between a direction from the position at which the live action is obtained to the live action and a normal vector direction mapped to the content.

The absolute value of ΔD ° may be less than 90 °.

3D mobile augmented reality generating system for user participation according to an embodiment of the present invention, a live-action acquisition module for obtaining a live-action image; A user input module configured to receive content to be matched from a user; And a content storage unit that maps and stores a location obtained from the live image, a direction opposite to a direction viewed toward the live image, and a predetermined time mapped to content received by the user input module.

The predetermined time may be a time for acquiring the live image or a time defined by a user.

The user input module may provide a user interface for adjusting the direction of the content.

A method for generating 3D mobile augmented reality for user participation according to an embodiment of the present invention includes: obtaining a live-action image; Receiving content to be matched from a user; And mapping and storing a location where the live image is obtained, a direction opposite to a direction viewed toward the live image, and a predetermined time mapped to content received from the user.

The predetermined time may be a time for acquiring the live image or a time defined by a user.

The step of receiving the content to be matched from the user may provide a user interface for adjusting the direction of the content to be matched to the user.

3D mobile augmented reality generation system and method for user participation in accordance with an embodiment of the present invention can provide a social network service (SNS) suitable for mobile augmented reality.

In addition, the 3D mobile augmented reality generation system and method for user participation according to an embodiment of the present invention has an effect that can reflect the user's behavior of leaving a memo or a picture as a post-it. .

In addition, the 3D mobile augmented reality generation system and method for user participation according to an embodiment of the present invention is as close as possible to the real world, there is an effect of preventing the 3D model from being separated from the reality.

In addition, the 3D mobile augmented reality generation system and method for user participation according to an embodiment of the present invention is suitable for mobile augmented reality (mixed reality), which has a large influence on the hardware performance since the values of various sensors are used as they are. It works.

In addition, the user participation-oriented 3D mobile augmented reality generating system and method according to an embodiment of the present invention, it is possible to search the content over time.

In addition, the 3D mobile augmented reality generation system and method for user participation according to an embodiment of the present invention has the effect of escaping the form of SNS through the existing short message.

In addition, the 3D mobile augmented reality generation system and method for user participation in accordance with an embodiment of the present invention, there is an effect that can follow the place and time prep (Follow) in the form of a conventional person-centered (Follow).

1 is a configuration diagram of a 3D mobile augmented reality generating system for user participation according to an embodiment of the present invention;
2 is a coordinate system for explaining a user participation in 3D mobile augmented reality generating system or method according to an embodiment of the present invention,
3 is a camera coordinate system for explaining a 3D mobile augmented reality generation system or method for user participation according to an embodiment of the present invention;
4 is a diagram illustrating a normal vector (normal vector) in a 3D mobile augmented reality generation system or method for user participation according to an embodiment of the present invention;
5 is a diagram illustrating calculation of a normal vector (normal vector) in a 3D mobile augmented reality generation system or method for user participation according to an embodiment of the present invention;
FIG. 6 is a view illustrating a user interface in a 3D mobile augmented reality generating system or method for user participation according to an embodiment of the present invention; FIG.
7 is a flowchart illustrating a method for generating 3D mobile augmented reality for user participation according to an embodiment of the present invention;
8 is a flowchart illustrating a method for generating 3D mobile augmented reality for user participation according to an embodiment of the present invention;
9 is a flowchart illustrating a method for generating 3D mobile augmented reality user participation in accordance with an embodiment of the present invention;
10 is a photograph showing augmented reality according to a user participation in three-dimensional mobile augmented reality generating system or method according to an embodiment of the present invention,
11 is a photograph showing augmented reality according to a user participation in three-dimensional mobile augmented reality generating system or method according to an embodiment of the present invention,
12 is a photo showing a user interface according to a 3D mobile augmented reality generating system or method for user participation according to an embodiment of the present invention, and
FIG. 13 is a photograph showing a result of searching by varying time according to a 3D mobile augmented reality generating system or method for user participation according to an embodiment of the present invention.

Objects, other objects, features and advantages of the present invention will be readily understood through the following preferred embodiments associated with the accompanying drawings. However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments disclosed herein are provided so that the disclosure can be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. In this specification, when an element is referred to as being on another element, it may be directly formed on another element, or a third element may be interposed therebetween. Also in the figures, the thickness of the components is exaggerated for an effective description of the technical content.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. As used herein, the words 'comprises' and / or 'comprising' do not exclude the presence or addition of one or more other components.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. In describing the specific embodiments below, various specific details are set forth in order to explain the invention more specifically and to help understand. However, one of ordinary skill in the art can understand that the present invention can be used without these various specific details. In some cases, it is mentioned in advance that parts of the invention which are commonly known in the description of the invention and which are not highly related to the invention are not described in order to prevent confusion in explaining the invention without cause.

1 is a block diagram of a 3D mobile augmented reality generating system for user participation according to an embodiment of the present invention.

Referring to FIG. 1, the 3D mobile augmented reality generating system for user participation includes a sensor measurement module 100, a positioning module 110, a user input module 130, a time retrieval module 140, and an image acquisition module 160. , Image matching module 170, output module 180, and 3D rendering module 200. Here, the 3D rendering module 200 includes a camera matrix generation module 156, a user content module 150, a light setting module 153, a time searching module 154, and a 3D environment configuration module 155. The user content module 150 includes an input object generation module 151 and an input object matrix generation module 152. In addition, the user input module 130 includes a content input module 131 and a touch input module 133.

The sensor measurement module 100 may include a camera sensor for obtaining an image of the actual image and a focal length to the actual image, a geomagnetic sensor for measuring an absolute direction, an acceleration sensor (and / or gyro sensor) for measuring a camera view direction, and It includes an illumination sensor for sensing the brightness of the light.

The positioning module 110 obtains the location information of the location where the live action is obtained. For example, the positioning module 110 may include a GPS receiver and / or LBS based location information means. According to an embodiment of the present invention, the positioning module 110 may receive a current location value using a GPS when outdoors, and receive a current location using a Real Time Location System (RTLS) when indoors. The Real Time Location System (RTLS) includes, for example, a positioning method using a base station of a mobile communication network and a positioning method using Wi-Fi.

The user input module 130 allows the user to select content to match with the live action, and provides an input interface that can rotate or modify the content. FIG. 6 is a diagram illustrating a user interface in a 3D mobile augmented reality generating system or method for user participation in accordance with an embodiment of the present invention, and FIG. 12 is a 3D user participation orientation in accordance with an embodiment of the present invention. A picture showing a user interface according to a mobile augmented reality generation system or method. As illustrated in these figures, the user input module 130 may provide a user with the convenience of freely entering and modifying content.

The time retrieval module 140 may provide a time input interface to the user when the user wants to search for content in a specific time zone. Although the time retrieval module 140 is illustrated as being separated from the user input module 130 described above, the time retrieval module 140 may be implemented by being combined with each other.

The image acquisition module 160 is a module for acquiring the due diligence and processes the image acquired by the sensor measurement module 100 in the form of an image.

The image matching module 170 matches the 3D rendering model rendered in 3D by the 3D rendering module 200 with the image (hereinafter, referred to as "real image") configured by the image capturing module 160 to mix mixed reality (or augmented reality). (Also called). Thereafter, the output module 180 may output the mixed reality in a form that a user can see.

According to an embodiment of the present invention, the 3D rendering module 156 may include a photorealistic image currently viewed by the user obtained through the sensor measurement module 100 and the image acquisition module 160, and obtained through the positioning module 110. Content that can be viewed at the current location using the current location value (hereinafter, also referred to as 'location' or 'location information') can be retrieved from the content storage 120 and loaded, and the loaded content can be rendered as a 3D content model. . The rendered 3D content may be configured as a 3D environment and then matched with the live image by the image matching module 170.

The content store 120 according to an embodiment of the present invention stores a plurality of contents, and directions, locations, and time information are mapped to the respective contents. For example, information as shown in Table 1 may be stored.

Content ID  location direction time  Address of the physical disk where the content is stored C1 P1 D1 (north facing) T1 Address 1 C2 P2 D3 (North facing) T2 Address 2 C3 P3 D1 (north facing) T3 Address 3 C4 P1 D1 (north facing) T1 Address 4 .
.
. .
.
. .
.
. .
.
. .
.
.

Referring to Table 1, locations, directions, and times are mapped to each of the contents stored in the content repository 120. As such, using the location, direction, and time mapped for each content, the desired content may be searched among the contents stored in the content storage 120. Here, if the camera (image acquisition module) is located at position P1 and the camera direction is true south, the content at position P1 is C1 and C4, but the content having a direction different from the camera direction is 180 degrees. C1. Therefore, since the content C4 has a true south-facing direction, the content of the content C4 is not visible from the camera side, and thus the content C4 is not a search target.

As such, when the view direction of the camera at the position P1 is true south, the content to be searched for includes the content facing north north (that is, the content whose normal vector is north north), and north north ± D ° (where , D ° may be less than 90 degrees).

The user input module 130 according to an embodiment of the present invention includes a content input module 130 and a touch input module 131. Here, the content input from the user by the content input module 130 becomes a 3D virtual model by the 3D rendering module 156, and the 3D virtual model is configured by the image matching module 170 after being configured in a 3D environment. Can be matched with the image. Meanwhile, in the mixed reality image matched with the live image, the content may be directly changed by the user, and the touch input module 131 may be used for this purpose. Referring to FIG. 6, the user may adjust left / right / up / down through the touch input module 131 to register content according to a desired position and angle. For example, referring to FIG. 6, when the touch input module 131 of the user input module 130 has a request for modification of the current content, the normal vector of the current 3D model is converted into a quaternion of Equation 1 below. Convert to quaternion).

After converting to 4 enemies, the 3D model is rotated by generating a rotation matrix based on a user's touch input as shown in FIG. 6. Circles indicated by dotted lines in FIG. 6 indicate touchable areas of the user. Touching the right side of the content (500) rotates a predetermined interval 510 to the right, and touching the left side of the content (520) rotates a predetermined interval to the left (530). The same applies to the vertical rotation (540, 550, 560, 570).

Content input from the content input module 130 (eg: text, graffiti, photo, video, etc.) is given a texture ID after texture processing and is stored in the content storage 120, and is an input object generation module 151 of the 3D rendering module. ), Which is then applied to the texture of the 3D virtual model. Here, "texture processing" means 2n * 2n imaging of the content.

 The camera matrix generation module 150 measures the current camera pose using the current position of the camera (ie, camera view) and the rotation (value) received from the sensor measurement module 100 to determine the direction and position of the camera view. Decide According to an embodiment of the present invention, the camera matrix generation module 150 may generate a direction and a position of the camera view in a matrix form.

2 is a coordinate system for explaining a system or method for generating 3D mobile augmented reality user participation in accordance with an embodiment of the present invention, Figure 3 is a 3D mobile augmented reality creation of the user participation in accordance with an embodiment of the present invention Camera coordinate system for describing the system or method. A method of determining a camera position by the camera matrix generation module 150 will be described with reference to these drawings.

The camera matrix generation module 150 may determine the position of the camera in two ways, for example.

 The first method is to use absolute position received from GPS. Since the position received from the GPS is represented by the earth coordinate system (or spherical coordinate system) as shown in FIG. 2, (latitude λ, longitude φ, altitude ρ) may be used as a position vector as it is.

The second method is to use the relative position obtained through RTLS. Since the position obtained through RTLS is a relative position, a position vector can be determined by using a formula that converts the relative position into an absolute position. For example, if only (x, y) coordinates are obtained, z can determine the position vector by receiving the current user's key.

Since the position vector is determined, if only a look at vector is determined, the direction and position of the camera view may be uniquely determined. Although the looket vector has been described above, it may be determined using values sensed by the geomagnetic sensor and the acceleration sensor (and / or the gyro sensor) included in the sensor measurement module 100.

As described above, when both the looket vector and the position vector are determined, the camera matrix generation module 150 may generate the camera view metrics using the vectors.

The user content module 150 may generate an input object and determine a position and a direction of the input object. According to an embodiment of the present invention, the user content module 150 includes an input object generation module 151 and an input object matrix generation module 152.

FIG. 4 is a diagram illustrating a normal vector (normal vector) in a 3D mobile augmented reality generating system or method for user participation according to an embodiment of the present invention, and FIG. 5 is a diagram illustrating user participation according to an embodiment of the present invention. A diagram illustrating the calculation of normal vectors (normal vectors) in a 3D mobile augmented reality generation system or method. With reference to these figures, a method of determining input object metrics will be described.

The input object generation module 151 of the user content module 150 generates a new 3D model and applies content registered as a texture by the user to the surface of the new 3D model (this operation is referred to as 'texturing'). .

The input object matrix generation module 152 sets the position and direction vector of the 3D model of which the content is textured. Referring to FIGS. 4 and 5, it is assumed that the looket vector 400 and the 3D model of the camera are separated, and the values added and subtracted by the offset (for example, focal length) from the camera position vector (latitude, longitude, altitude ± offset) This can be determined by the position vector of the 3D model. In addition, the input object matrix generation module 152 may determine a vector having a direction opposite to that of the camera, as a direction vector of the 3D model. For convenience of description, in the present specification, a vector having a direction opposite to the looket vector of the camera is referred to as a direction vector (or normal vector or normal vector) of the 3D model.

As described above, the input object matrix generation module 152 may determine a position and a direction with respect to the 3D model, and generate the position and the direction in the form of a matrix.

The light setting module 153 may receive an intensity value of light from an illuminance sensor, set an illuminance value corresponding to the light at the current position, and transmit it to the 3D environment configuration module 155. The 3D environment configuration module 155 may include an illuminance value received from the light setting module 153, a camera matrix generated by the camera matrix generation module 156, an input object matrix generated by the user content module, and a time input from the user. Construct a virtual environment of 3D model using.

For example, suppose the current time is night. If 3D augmented reality is implemented without consideration of illuminance, it may not be possible to see augmented reality visually. In this case, the 3D environment configuration module 155 may set and render the illumination so that the augmented reality can be seen by the eye. On the other hand, based on the intensity of the light sensed from the illumination sensor, the contrast of the 3D virtual model included in the augmented reality may be changed according to the current position of sunlight. As a result, the user may experience a more realistic augmented reality.

 The time retrieval module 154 loads only the content that can be seen in the time interval set by the user from the content store 120, rather than loading all the content that is visible at the current location as a time filter. The output of all modules 150-154 in the 3D rendering module are combined in the 3D configuration module 155 and are in Δlatitude, Δlongitude at the user's current location and based on all the content in a particular time interval, The scene is constructed and rendered.

Subsequently, when the image matching module 170 matches the actual image obtained through the image acquisition module 160 with the rendering frame (3D virtual model) obtained through the 3D environment configuration module 155, the output module 180 may display Display in a viewable form.

FIG. 7 is a flowchart illustrating a method for generating 3D mobile augmented reality for user participation according to an embodiment of the present invention, and exemplarily illustrates a process of a user storing content used in a 3D virtual model in a content repository. .

Referring to FIG. 7, the positioning module acquires current location information from GPS or RTLS (S701), and acquires various sensor values at the current location of the camera (S703). Here, the various sensor values are values necessary to determine the direction of the camera at the camera location. For example, the camera orientation may be determined by values sensed by the geomagnetic sensor and the gyro sensor (or acceleration sensor).

When the content is input from the user (S705: Y), the received content is textured and applied to the 3D model (S707). That is, the content is textured to the 3D virtual model. Thereafter, the 3D environment configuration model constructs the 3D environment using the textured 3D virtual model and the position vector and the direction of the 3D virtual model (S709). According to an embodiment of the present invention, a 3D virtual environment is configured in consideration of the intensity of light obtained from the illuminance sensor.

Thereafter, the image matching module matches the image S704 acquired by the image capturing module with the 3D virtual environment, and the matched mixed reality image may be displayed by the output module (S713). If the user wants to store the content used in the mixed reality image (S715: Y). The 3D rendering module textifies the content, gives it an ID, and stores it in the content repository. However, when storing the content in the storage, the 3D rendering module always maps and stores the position, direction, and time for each content. Here, the position mapped to the content is a position where the real image is acquired (that is, the position of the camera), the direction is the opposite direction of the lookout vector of the camera (that is, the normal vector or the normal vector of the 3D model), and the time is It is time to create mixed reality. Meanwhile, the time may be a time input by the user, not a time of generating the mixed reality.

8 is a flowchart illustrating a method for generating 3D mobile augmented reality for user participation according to an embodiment of the present invention. The process of configuring 3D mixed reality by loading content stored in the content repository 120 is illustrated. It is shown.

Referring to FIG. 8, the positioning module acquires current position information from GPS or RTLS (S801), and acquires various sensor values at the location of the image acquisition module (S803). Here, the various sensor values are values necessary to determine the direction of the camera at the camera location. For example, the camera orientation may be determined by values sensed by the geomagnetic sensor and the gyro sensor (or acceleration sensor).

When the user wants to match only the content generated at a specific time to the live action, the user may receive a time from the user. For example, when a specific time is input through the time search module 140 (S805: Y), the 3D rendering module may determine the input time, the position obtained by the positioning module at the current position, and the camera acquired from the sensor measurement module. The content mapped to a specific time, a specific location, and a specific direction is loaded based on the direction (S809). The specific time is a time input by the user, the specific position is a position obtained by the positioning module, and the specific direction is a direction opposite to the direction of the camera. In particular, the specific direction is a direction opposite to the direction of the camera, that is, a direction that is 180 degrees different from the camera direction vector, and may also correspond to a direction that is different from the absolute value of 180 degrees by a predetermined angle (mainly less than 90 degrees).

After that, the 3D environment configuration module constructs a 3D virtual environment using a 3D sweetened model and delivers the image to the image matching module, and the image matching module matches the real-world image obtained by the image acquisition module with the 3D environment including the 3D virtual model and mixed reality. (S813). Thereafter, the output module displays the mixed reality to the user (S815).

9 is a flowchart illustrating a method for generating 3D mobile augmented reality for user participation according to an embodiment of the present invention.

Referring to FIG. 9, the operation is similar to that of FIG. 8, and further includes a case in which a user inputs content after loading content mapped from a specific time, a specific location, and a specific direction from the content repository. There is. Since the process of configuring the mixed reality by loading content mapped to a specific time, a specific location, and a specific direction from the content repository has been described with reference to FIG. 8, operations from S901 to S909 will be omitted. Hereinafter, after S911 of FIG. 9, a case in which a user wants to add another content again after loading a content mapped with a specific time, a specific location, and a specific direction from the content store is illustrated. In S911, when there is content input from the user (S911: Y), the 3D rendering module 3D virtualizes both the content input from the user and the content loaded in S909 and reconstructs the 3D environment (S917). Thereafter, the 3D environment configured in step S917 is matched with the real image (S919) and displayed externally (S921).

FIG. 10 is a photograph showing augmented reality according to a system or method for generating 3D mobile augmented reality user participation in accordance with an embodiment of the present invention, and FIG. 11 is a 3D mobile augmentation flavor according to an embodiment of the present invention. It is a picture showing augmented reality according to the reality generation system or method. Referring to these pictures, it will be better to understand a user participation-oriented 3D mobile augmented reality generating system and method according to an embodiment of the present invention. Referring to FIGS. 10 and 11, it can be seen that a mixed image is generated in consideration of the direction of the camera of the user, and moreover, it is generated in consideration of the illumination of the surroundings.

As described above, when a process of realizing a mixed image is started, image and content registration is performed in real time through the image matching module. To do this, when the camera starts, the image acquisition module reconstructs the 3D virtual environment by loading the contents that can be seen at the current location with the process of obtaining the camera image frame and the current position information acquired by the positioning module and the sensor values obtained by the sensor measurement module. The process can run as a thread.

Methods according to an embodiment of the present invention can be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts.

Although the present invention as described above has been described by way of limited embodiments and drawings, the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from this description. This is possible. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims below but also by the equivalents of the claims.

100: sensor measurement module 110: positioning module
130: user input module 131: content input module
133: touch input module 140: time search module
150: user content module 151: input object generation module
152: input object matrix generation module 153: light setting module
154: time search module 155: 3D environment configuration module
160: image acquisition module 170: image registration module
180: output module 180 200: 3D rendering module

Claims (26)

A content storage unit for storing content;
A live action acquisition module for acquiring a live action image;
A 3D rendering model that searches for a predetermined content in the content storage unit and renders the searched content as a 3D virtual model by using the position at which the live image is obtained and the direction facing toward the live image; And
And an image registration module for generating an augmented reality by matching the rendered 3D virtual model with the real image. 3. The method of claim 1,
Each content stored in the content storage unit, the direction, location, and time information, the user participation in three-dimensional mobile augmented reality generating system characterized in that the mapping. The method of claim 1,
The content stored in the content storage unit, the user participation-oriented three-dimensional mobile augmented reality generation system, characterized in that the direction and location information in the actual space is mapped. The method of claim 2,
And a user input module configured to receive a search time for searching for a content mapped to a specific time in the contents stored in the content storage unit.
The 3D rendering module searches for the predetermined 3D content in the content storage unit by using the location where the live image is obtained, the direction of viewing the live image, and the search time, and renders the retrieved content as a 3D virtual model. 3D augmented reality generation system for user participation. The method of claim 2,
And a sensor measurement module configured to sense illuminance of the live image obtained by the live image acquisition module.
And the 3D rendering module reflects the illuminance sensed by the sensor measurement module when rendering the retrieved content as a 3D virtual model. The method of claim 1,
It further includes a positioning module for obtaining position information of the live action acquisition module,
The 3D rendering module searches for all contents located within a predetermined distance from the position of the actual image acquisition module toward the actual image obtained by the actual image acquisition module in the content storage unit, and renders the retrieved content as a 3D virtual model. 3D augmented reality generation system characterized by user participation. The method of claim 2,
A user input module configured to receive a search time for searching for a content mapped to a specific time in the contents stored in the content storage unit; And
It further includes a positioning module for obtaining position information of the live action acquisition module,
The 3D rendering module searches in the content storage unit for all contents located within a predetermined distance from the location of the live-action acquisition module to the live-action acquisition module obtained from the location of the live-action acquisition module among the contents mapped with the same time as the search time. And rendering the retrieved content as a 3D virtual model. The method of claim 7, wherein
And a direction mapped to the contents stored in the content storage unit is defined as a normal vector direction which is completely opposite to a direction facing the front of the content. The method of claim 8,
The content retrieved by the 3D rendering module,
And a difference between a direction from the location of the live action acquisition module toward the live action acquired by the live action acquisition module and a normal vector direction mapped to the content is 180 ± ΔD °. The method of claim 9,
The absolute value of the ΔD ° is less than 90 ° user participation flavor three-dimensional augmented reality generating system. Storing the content in the storage unit;
Obtaining a live image;
Searching for a predetermined content stored in the storage unit by using the position where the actual image is obtained and the direction facing toward the actual image, and rendering the retrieved content as a 3D virtual model; And
And generating an augmented reality by matching the rendered 3D virtual model with the photorealistic image. The method of claim 11,
Each content stored in the storage unit, the direction, location, and time information, characterized in that the user participation orientation 3D mobile augmented reality generating method characterized in that the mapping. The method of claim 11,
The content stored in the storage unit, the user participation orientation 3D mobile augmented reality generation method characterized in that the direction and location information in the actual space is mapped. The method of claim 12,
Receiving a search time for searching for a content mapped to a specific time from contents stored in the storage, from a user;
The 3D rendering module searches for the predetermined 3D content in the storage unit by using the location where the real image is obtained, the direction of viewing the live image, and the search time, and renders the retrieved content as a 3D virtual model. 3D augmented reality generation method characterized by user participation. The method of claim 12,
And a sensor measurement module configured to sense illuminance of the live image obtained by the live image acquisition module.
And the 3D rendering module reflects the illuminance sensed by the sensor measuring module when rendering the searched content as a 3D virtual model. The method of claim 11,
It further includes a positioning module for obtaining position information of the live action acquisition module,
In the 3D rendering, the content storage unit may search for all contents located within a predetermined distance from the position of the actual image acquisition module to the actual image obtained by the actual image acquisition module, and render the searched content as a 3D virtual model. Method for generating 3D augmented reality user participation flavor, characterized in that. The method of claim 12,
Receiving a search time for searching for content mapped to a specific time from contents stored in the storage unit from a user; And
Acquiring the location information of the location where the live action is obtained;
The rendering may include searching and retrieving, in the storage unit, all content located within a predetermined distance in a direction toward the live action from a location where the live action is obtained, among the contents mapped with the same time as the search time. 3D augmented reality generation method for user participation characterized in that to render a content as a 3D virtual model. The method of claim 17,
And a direction mapped to the contents stored in the storage unit is defined as a normal vector direction which is completely opposite to a direction facing the front of the contents. The method of claim 18,
The searched content,
And a difference between a direction from the position at which the photorealistic is acquired to the photorealistic direction and a normal vector direction mapped to the content is 180 ± ΔD °. The method of claim 19,
The absolute value of the ΔD ° is less than 90 ° user participation flavor three-dimensional augmented reality generation method characterized in that. A live action acquisition module for acquiring a live action image;
A user input module configured to receive content to be matched from a user;
And a content storage unit configured to map and store a position obtained from the live image, a direction opposite to a direction viewed toward the live image, and a predetermined time mapped to the content input by the user input module. Incense 3D Augmented Reality Generation System. The method of claim 21,
The predetermined time may be a time for acquiring the live-action image or a time defined by a user. The method of claim 21,
The user input module,
3D augmented reality generation system for user participation, characterized in that to provide a user interface that can adjust the direction of the content. Obtaining a live image;
Receiving content to be matched from a user;
And mapping and storing a location where the live image is acquired, a direction opposite to a direction viewed toward the live image, and a predetermined time mapped to content received from the user. 3D Augmented Reality How to produce. 25. The method of claim 24,
The predetermined time may be a time for acquiring the live image or a time defined by a user. 3. 25. The method of claim 24,
The step of receiving the content to be matched from the user,
And a user interface for adjusting a direction of content to be matched to the user.

Images