KR20180106440A - Multimedia massage providing service based on location using augmented reality - Google Patents

Abstract

The present invention is a model combining a location-based message service with an augmented reality. The model replaces and sends location information included in a message with a marker of an augmented reality and a message receiver moves to a designated position and confirms the multimedia information inserted in the message. Since a user can use the included multimedia information as a more dynamic screen by using an augmented reality technique and a sender of the message can specify a target to which the marker can be recognized by the message receiver (a smartphone of the receiver), a photo or video file containing privacy of an individual can be protected and commercial use of the message linked with the location information are possible because the multimedia information cannot be verified without the smartphone and the location information of the message receiver even if content of the message is leaked by account hacking of the message receiver.

Description

[0001] MULTIMEDIA MASSAGE PROVIDING SERVICE BASED ON LOCATION USING AUGMENTED REALITY [0002]

The present invention relates to a message providing service, and more particularly, to a location-based multimedia message providing service using an augmented reality.

The optimal environment for using an augmented reality is to use devices such as an HMD device for enhancing reality, a globe equipped with a sensor, and a data suit capable of digitizing user's movements, but the majority It is difficult to provide the equipment of the present invention. The most notable augmented reality devices currently use network-based personalized devices such as smartphones with excellent mobility.

Smartphones are not designed to realize augmented reality professionally, but they can reduce the reality, but on the contrary, they can have the opportunity to converge augmented reality with the many applications used in smartphones. Although augmented reality brings revolutionary progress in science, industry, education, and game fields, it has the potential to be developed into a new service model by converging with a generalized application that has been used with smartphones.

Key technologies for realizing augmented reality include tracking technology for user's position, direction, and movement, registration technology for creating a virtual object through computer graphics and accurately positioning the actual image, And UI (User Interface) technology for handling interaction with a user such as a user.

The augmented reality system transmits the captured image of a real world camera to a tracking module that performs a function of tracking the position, motion, and speed direction of the object through a video interface. In the next Rendering module, an augmented image is created by creating or removing a virtual object based on the position of the object identified through the tracking module. Then, the distance between the virtual objects generated through the merging module, the distance and the direction between the generated coordinate systems are measured, and whether or not there is interference between the virtual objects is displayed, and the augmented reality is expressed through the display

Tracking module can be divided into sensor based technology and vision based technology. Vision based technology can be divided into marker based and non - marker based.

Sensor-based technology refers to a technology that visualizes relevant content by tracking the movement, position, and direction of objects using GPS, acceleration sensor, and gyro sensor available in mobile devices to identify reinforcement positions on the screen. However, there is a disadvantage in that it is difficult to increase accurate information at specific locations.

Among the vision-based technologies, marker-based technology is a technology for recognizing, tracking and mapping markers such as QR codes using a camera or the like. Non-marker based technology is also used for filtering images compared with images taken through a camera, It is a technology that provides information.

In addition, the matching technique is a technique for accurately positioning a virtual object at a position of an actual image, and is composed of three steps of positioning, rendering, and merging.

Such augmented reality service can be used in various fields. In particular, it can be used conveniently in providing a message service, but there are various complicated problems at present to apply it.

The present invention for solving the above problem is to solve the problem of leakage of a photograph or a moving picture file containing privacy of an individual due to leakage of message contents due to account hacking of a message receiver.

The present invention is characterized by comprising: information receiving means for receiving message information (A) including multimedia content information from a message sender terminal and terminal unique information (A) corresponding to a receiver wireless terminal to receive the message; (B) for reproducing the multimedia contents included in the message information (B), and storing the generated message information (B) in a storage medium; (B) transmitted from the augmented reality device of the message receiver to the message receiver terminal, and the message information (B) is transmitted from the augmented reality device of the message receiver to the message receiver terminal When the augmented reality data request corresponding to the augmented reality marker is received, the augmented reality data request corresponding to the augmented reality data corresponding to the marker stored in the storage medium And an augmented reality data transmission means for transmitting the augmented reality data to the message receiver wireless terminal and outputting the augmented reality data.

According to the present invention, it is possible to use a message using method as a dynamic screen using an augmented reality technique, and to have a message account information, a location information, and a smart phone in an environment where a message account exposure possibility is rather high, It is effective to protect personal privacy information.

In addition, the method is not so different from the method of transmitting an existing message, so there is no difficulty in using the service, and it can be easily applied through simple modification of the conventional message system. The augmented reality marker used in the present invention is generated Commercial use of the message is possible by using the location information rather than the marker.

1 is a diagram illustrating a configuration of an augmented reality system according to an embodiment of the present invention.
2 is a diagram illustrating a process of providing location-based multimedia information included in a message according to another embodiment of the present invention through an augmented reality service.

A system for providing a location-based multimedia message service using an augmented reality according to the present invention is characterized in that, when message information including multimedia content information is received from a message sender terminal, an information check means for checking whether multimedia information is included on the received message information A confirmation unit for confirming the location information corresponding to the received multimedia information when the multimedia information is included in the message information, the location information and the message information in association with the message receiver information for receiving the message, An information storing means for storing the message request information in the message D / B; and a message storing means for storing the message information corresponding to the received message request information and the message information in association with the message D / B Information extracting means for extracting the stored location information A message information transmission means for transmitting the extracted message information and position information to the message receiver terminal and outputting the extracted message information and position information to the message receiver terminal; and a controller for, when receiving the augmented reality request information corresponding to the position information, And augmented reality data transmission means for extracting multimedia information, converting the extracted multimedia information into augmented reality data, and transmitting the augmented reality data to the message receiver augmented reality device.

The system for providing a location-based multimedia message service using an augmented reality according to the present invention further comprises augmented reality device authentication means for authenticating a message recipient augmented reality device transmitting the augmented reality request information.

The operation principle of the preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings and description. It should be understood, however, that the drawings and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention, and are not to be construed as limiting the present invention. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. The terms used below are defined in consideration of the functions of the present invention, which may vary depending on the user, intention or custom of the operator. Therefore, the definition should be based on the contents throughout the present invention.

It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. The configuration is omitted as much as possible, and a functional configuration that should be additionally provided for the present invention is mainly described.

Those skilled in the art will readily understand the functions of components that have been used in the prior art among the functional configurations that are not shown in the following description, The relationship between the elements and the components added for the present invention will also be clearly understood.

In order to efficiently explain the essential technical features of the present invention, the following embodiments properly modify the terms so that those skilled in the art can clearly understand the present invention, It is by no means limited.

As a result, the technical idea of the present invention is determined by the claims, and the following embodiments are merely means for effectively explaining the technical idea of the present invention to a person having ordinary skill in the art to which the present invention belongs Only.

1 is a diagram illustrating a configuration of an augmented reality system according to an embodiment of the present invention.

In more detail, FIG. 1 shows a case where a message service and an augmented reality service are merged, location information included in a message is replaced with an augmented reality marker, and location information is tracked to a terminal of a message receiver to provide multimedia information inserted in a message 1 is a block diagram illustrating a system configuration in which a message service and an augmented reality service are merged to provide location-based multimedia information by referring to and / or modifying FIG. 1 by a person having ordinary skill in the art to which the present invention belongs It will be appreciated that various implementations of the augmented reality system configuration (e.g., some configuration portions may be omitted, or subdivided or combined) may be inferred, but the present invention encompasses all of the above- The technical features thereof are not limited only by the method shown in FIG.

For example, in FIG. 1, each means constituting the augmented reality system is shown in the augmented reality operation server 100 as a component of the augmented reality operation server 100, but the augmented reality operation server 100 The present invention is not limited to the components and may be implemented in the form of a server according to the intention of those skilled in the art or a combination of two or more means implemented in the form of a server, It is to be understood that the present invention is not limited by the implementation of the augmented reality operation server 100 and that the present invention is not limited thereto.

According to the present invention, the augmented reality system includes a message sender terminal 165 having a function of creating a message containing multimedia information by a user desiring to send a message and transmitting the message information, a message sender terminal 165, And transmits the message to the message receiver terminal 170. The message receiver terminal 170 receives the location information included in the message information from the message receiver terminal 170, An augmented reality operating server (100) for providing multimedia information corresponding to location information when requesting multimedia information using information; and a receiving server (110) for receiving the message information transmitted through the message sender terminal (165) A message receiver terminal (170) for outputting location information included in the received message information, Transmitting the requested information to the virtual reality to the AR operations server 100, and includes a message receiver augmented reality device (175) for outputting by receiving virtual reality data from the AR operations server 100. [

Referring to FIG. 1, when receiving the message information from the message sender terminal 165, the augmented reality operation server 100 checks whether the multimedia information is included in the received message information, And a message D / B 160 for processing and storing location information corresponding to the multimedia information included in the message information in association with the receiver information for the message receiver.

The augmented reality data may include augmented reality data file generated by combining a frame and a texture, rendering the generated augmented reality data file, or 3D data including rendering information for combining and rendering the texture in the frame File, or one or more frame data files and texture data files that can be combined via a message recipient augmented reality device 175 to receive the message information.

If the augmented reality data includes a frame data file and a texture data file, the augmented reality data may include combining condition information for combining the frame data and the texture data, or combining two or more frames to form one frame data It is possible to further include combining condition information and combining condition information for combining two or more pieces of texture data with one frame data. In this case, rendering information for rendering a combined texture on a frame based on the combining condition information may be combined And may be dynamically determined in the message receiver augmented reality device 175 that will receive the message information based on the condition information.

According to the embodiment of the present invention, the augmented reality data can be stored on the message D / B 160 in association with augmented reality meta information including a tag, a keyword, and a character string describing the augmented reality data .

Referring to FIG. 1, the augmented reality operation server 100 includes a message information receiving unit 105 for receiving message information from the message sender terminal 165, and a memory unit for storing the received message information, (115) for checking the location information corresponding to the multimedia information included in the received message information, a message information (115) including the multimedia information specified in the location information, And an information storage unit 120 for storing the location information on the message D / B 160 in association with the sender information on the message sender.

Here, the message information includes text (i.e., message content) input by the message sender through a wireless terminal or a wired terminal, multimedia information (e.g., video, , An image, etc.), location information (e.g., longitude, latitude), caller message information (e.g., caller message address, caller terminal address (i.e., Internet Protocol address, MAC (Media Access Control) And a recipient terminal address (e.g., recipient message address, recipient terminal address, etc.).

When the message information is received from the message sender terminal 165 via the message information receiving means 105, the location information checking means 115 checks the location information on the received message information, The multimedia information checking unit 110 checks whether the multimedia information is included in the received message information.

According to the method of the present invention, the confirmation of whether the multimedia information is included in the message information received from the message sender terminal 165 is made by checking whether the file included in the received message information is included or not, Is multimedia information.

Here, when the identified file is a compressed file, it is possible to decompress the compressed file to check whether multimedia information is included.

According to another embodiment of the present invention, whether or not the multimedia information is included in the message information received from the message sender terminal 165 may include checking whether the link information is included in the message information, Check whether it is multimedia information.

Here, it is possible to check various link information included in the message information, confirm whether the multimedia information is included on the web page corresponding to the confirmed link information, and display the multimedia information on the web page corresponding to the link information If information is included, it is possible to store the multimedia information on the message D / B 160.

Referring to FIG. 1, the augmented reality operation server 100 stores message information received from the message sender terminal 165 on the message D / B 160, and transmits a message receiver A message request information receiving means (125) for receiving message request information from the message receiver terminal (170) when transmitting message request information through a message receiver terminal (170) The message receiver 170 transmits the message information to the message receiver 170. The message receiver 170 transmits the message information to the message receiver 170. The message receiver 170 transmits the message information to the message receiver 170. The message receiver 170 transmits the message information to the message receiver 170, A message information extracting means (130) for extracting the extracted location information further including the location information if the extracted information includes the location information And message information transmitting means 135 for transmitting at least one message to the message receiver terminal 170.

When the message request information is received from the message receiver terminal 170 via the message request information receiving means 125, the message information extracting means 130 extracts the message information corresponding to the message request information from the message D / Extract the message information.

The message information transmission unit 135 replaces the multimedia information included in the message information with the augmented reality marker information and stores the augmented reality marker information in the message information storage unit 140. [ To the message receiver terminal 170.

Referring to FIG. 1, the augmented reality operation server 100 includes a message receiver terminal 170 and a message receiver terminal 170. The augmented reality operation server 100 receives the augmented reality service request from the message receiver terminal 170, An augmented reality request information receiving means 140 for receiving the augmented reality request information from the message receiver augmented reality device 175 and an augmented reality request information receiving means 140 for reading the augmented reality request information or receiving the augmented reality request information and the message D / Augmented reality data extracting means 145 for extracting the augmented reality data to be transmitted to the message receiver augmenting reality apparatus 175 by comparing the position information stored in the augmented reality receiving apparatus 175, And transmits the augmented reality data to the augmented reality data transmitting means 150. [ If augmented reality data is received from another server, means for transmitting the augmented reality data to the message receiver augmented reality device 175 may be omitted.

In addition, the augmented reality operation server 100 includes an augmented reality device authentication unit 155 for confirming a message receiver augmented reality device 175 that transmitted the augmented reality request information.

Here, the augmented reality request information may include at least one of a tag, a keyword, and a string, which is matched with positional information of any augmented reality data stored in the message D / B 160, It is possible to include an address and a data identification code for requesting specific actual reality data according to a method.

The augmented reality data provided to the message recipient augmented reality device 175 may include multimedia information included in the message information transmitted by the message sender as it is or may include multimedia information converted into an augmented reality model It is possible.

The augmented reality data extracting unit 145 extracts the augmented reality data from the message D / B auger 160 from the message receiver augmenting reality apparatus 175 through the augmented reality request information receiving unit 140, And extracts the augmented reality data corresponding to the reality request information.

When the extracted augmented reality data includes the augmented reality data file in accordance with an embodiment of the present invention, the augmented reality data transfer unit 150 transmits the augmented reality data file to the augmented reality device .

According to another embodiment of the present invention, when the extracted augmented reality data includes the 3D data file, the augmented reality data transmission unit 150 generates the augmented reality data file by rendering the 3D data file To the message recipient augmented reality device 175, or to transmit the 3D data file to the message recipient augmented reality device 175. [

If the 3D data file is transmitted to the message recipient augmented reality device 175, the message recipient augmented reality device 175 may render the 3D data file to generate an augmented reality data file.

According to another embodiment of the present invention, when the extracted augmented reality data includes the frame data file, the texture data file, and the combining condition information, the augmented reality data transmitting unit 150 transmits Generates a 3D data file by combining the frame data file and the texture data file, generates an augmented reality data file in which the 3D data file is rendered, and transmits the augmented reality data file to the message recipient augmented reality device 175, And transmits the 3D data file generated by combining the texture data file and the texture data file to the message recipient augmented reality device 175 or transmits the frame data file and the texture data file and the coupling condition information to the message recipient augmented reality device 175 Lt; / RTI >

If the augmented reality data transmission unit 150 transmits the 3D data file combining the frame data file and the texture data file to the message receiver augmented reality device 175 based on the combining condition information, The real device 175 may render the 3D data file to generate an augmented reality data file.

Alternatively, when the augmented reality data transmission unit 150 transmits the frame data file, the texture data file, and the combining condition information to the message recipient augmented reality device 175, the message recipient augmented reality device 175 transmits the frame data file, A 3D data file combining the frame data file and the texture data file is generated based on the combining condition information, and an augmented reality data file in which the 3D data file is rendered can be generated.

FIG. 2 is a diagram illustrating a process of providing location-based multimedia information included in a message according to another embodiment of the present invention through an augmented reality service.

2 illustrates a method of providing a generated message. Referring to FIG. 2 and / or modified by a person having ordinary skill in the art to which the present invention pertains, the message service and augmented reality The present invention can be embodied in all of the above-described embodiments, and it is to be understood that the present invention is not limited to the above-described embodiments, Features are not limited.

Referring to FIG. 2, a message sender creates a message content using a message sender terminal 165, and inserts a multimedia file such as an image or a moving picture into the created message content.

Here, it is possible to check whether the inserted multimedia file is to be transmitted in the augmented reality manner, and to register the smartphone number of the message recipient who can confirm the location information.

Then, when the message sender terminal 165 transmits the message information to the server, the server receiving the message designates the location information matched with the multimedia file information so that the multimedia file set by the sender of the message can be provided to the augmented reality .

Here, it is possible to further include a process of converting a multimedia file to be provided by applying an augmented reality graphical effect. When a message sender provides a message originator with a multimedia file in the application of an augmented reality service, It is possible to provide a few examples of graphics that can be made available for preview.

Then, the message receiver 170 transmits a message including positional information at the request of the message receiver terminal 170, and provides the message information by applying the augmented reality service on the included multimedia file through the provided positional information.

Here, the message that the message for the augmented reality is transmitted to the wireless terminal of the message recipient is activated, and the button for activating the augmented reality application installed in the smart phone is activated so that the user can immediately use the augmented reality service immediately It is possible to do.

100: Augmented Reality operating server 105: Message information receiving means
110: Multimedia information checking means 115: Position information checking means
120: information storing means 125: message request information receiving means
130: Message information extracting means 135: Message information transmitting means
140: augmented reality request information receiving means 145: augmented reality data extracting means
150: augmented reality data transmission means 155: augmented reality device authentication means
160: Message D / B 165: Message sender terminal
170: Message recipient terminal 175: Message recipient augmented reality device

Claims (2)

Information checking means for checking whether multimedia information is included in the received message information when message information including location information is received from a message sender terminal;
Location information checking means for checking location information corresponding to the received multimedia information when the message information includes multimedia information;
Information storage means for associating the confirmed location information and the message information with the message receiver information to receive the message and storing the processed message information in the message D / B;
Information extracting means for extracting message information corresponding to the received message request information and position information stored on the message D / B in association with the message information when the message request information is received from the message receiver terminal;
Message information transmission means for transmitting the extracted message information and location information to the message receiver terminal and processing the message information to be output;
Extracting the multimedia information corresponding to the received augmented reality request information from the message receiver augmenting reality apparatus when the augmented reality request information corresponding to the identified position information is received and converting the extracted multimedia information into augmented reality data And augmented reality data transmission means for transmitting the augmented reality data to the message recipient augmented reality device.
The method according to claim 1,
And augmented reality device authentication means for authenticating a message recipient augmented reality device that has transmitted the augmented reality request information to the augmented reality device.

Images