KR102620477B1 - Server, device and method for providing augmented reality service - Google Patents

Abstract

The augmented reality providing server that provides the augmented reality service includes a receiving unit that receives images from a plurality of cameras, an object analysis unit that detects at least one object of interest from the received images and analyzes the detected object of interest, and an object analysis unit that analyzes the detected object of interest through the received images. A situation determination unit that determines the situation in which the image was captured, a generation unit that generates augmented reality data based on information on the analyzed object of interest and the determined situation, a synthesis unit that synthesizes the augmented reality data generated in the image, and augmented reality. It may include a providing unit that provides an image composed of data to a user terminal.

Description

Server, terminal and method for providing augmented reality service {SERVER, DEVICE AND METHOD FOR PROVIDING AUGMENTED REALITY SERVICE}

The present invention relates to a server, terminal, and method for providing augmented reality services.

Augmented reality technology is a technology that synthesizes and displays virtual objects on images of the real world. This augmented reality technology can synthesize virtual objects on the basis of the real world and provide additional information that is difficult to obtain through the real world alone.

Meanwhile, as the spread of augmented reality services using mobile terminals becomes widespread, user demand for various additional services using augmented reality on mobile terminals is increasing, and efforts to provide various augmented reality contents to users of mobile terminals are actively underway. It is becoming.

Korean Patent Publication No. 10-1757884 (registered on July 7, 2017)

The present invention is intended to generate augmented reality data based on information about objects of interest detected from an image and situations determined through the image, and to provide the augmented reality data to a user terminal by combining the augmented reality data with the image. However, the technical challenges that this embodiment aims to achieve are not limited to the technical challenges described above, and other technical challenges may exist.

As a technical means for achieving the above-described technical problem, an augmented reality providing server that provides an augmented reality service according to the first aspect of the present invention includes a receiving unit that receives images from a plurality of cameras; an object analysis unit that detects at least one object of interest from the received image and analyzes the detected object of interest; a situation determination unit that determines a situation in which the image was captured based on the received image; a generator that generates augmented reality data based on information about the analyzed object of interest and the determined situation; a synthesis unit that synthesizes the generated augmented reality data with the image; and a providing unit that provides an image synthesized with the augmented reality data to a user terminal.

A user terminal providing an augmented reality service according to a second aspect of the present invention includes a request unit that requests the augmented reality service from an augmented reality providing server; a receiving unit that receives an image synthesized with augmented reality data corresponding to the augmented reality service from the augmented reality providing server; and an output unit that outputs an image synthesized from the received augmented reality data, wherein the augmented reality data includes information on an object of interest detected and analyzed from images received from a plurality of cameras and a determination that the image was captured. It can be created based on the situation.

A method of providing an augmented reality service in an augmented reality providing server according to a third aspect of the present invention includes receiving images from a plurality of cameras; Detecting at least one object of interest from the received image and analyzing the detected object of interest; determining a situation in which the image was captured through the received image; generating augmented reality data based on information about the analyzed object of interest and the determined situation; synthesizing the generated augmented reality data with the image; And it may include providing an image synthesized with the augmented reality data to a user terminal.

The above-described means for solving the problem are merely illustrative and should not be construed as limiting the present invention. In addition to the exemplary embodiments described above, there may be additional embodiments described in the drawings and detailed description of the invention.

According to one of the means for solving the problems of the present invention described above, the present invention generates augmented reality data based on information about an object of interest detected from an image and a situation determined through the image, and synthesizes augmented reality data into the image. This can be provided to the user terminal. Through this, the present invention can provide information about the store to the user through an image synthesized with augmented reality data even if the user does not visit the store directly, and can provide a route guidance service from the user's location to the store. . According to another embodiment of the present invention, in a disaster situation, an escape route can be provided to the user through an image synthesized with augmented reality data, so that the user can safely and quickly escape from the disaster site. The present invention can track the movement path of a missing child through video and provide the tracked movement route to the user terminal so that the missing child can be quickly found when a missing child report is received.

1 is a configuration diagram of an augmented reality service providing system according to an embodiment of the present invention.
FIG. 2 is a block diagram of the augmented reality providing server shown in FIG. 1 according to an embodiment of the present invention.
3A to 3B are diagrams for explaining a method of providing an augmented reality service according to an embodiment of the present invention.
4A to 4C are diagrams for explaining a method of providing an augmented reality service according to another embodiment of the present invention.
Figure 5 is a flowchart showing a method of providing an augmented reality service in an augmented reality providing server according to an embodiment of the present invention.
Figure 6 is a block diagram of the user terminal shown in Figure 1, according to an embodiment of the present invention.
Figure 7 is a flowchart showing a method of providing an augmented reality service in a user terminal, according to an embodiment of the present invention.

Below, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts that are not related to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected," but also the case where it is "electrically connected" with another element in between. . Additionally, when a part "includes" a certain component, this means that it may further include other components rather than excluding other components, unless specifically stated to the contrary.

In this specification, 'part' includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more pieces of hardware, and two or more units may be realized using one piece of hardware.

In this specification, some of the operations or functions described as being performed by a terminal or device may instead be performed on a server connected to the terminal or device. Likewise, some of the operations or functions described as being performed by the server may also be performed on a terminal or device connected to the server.

Hereinafter, specific details for implementing the present invention will be described with reference to the attached configuration diagram or processing flow diagram.

1 is a configuration diagram of an augmented reality service providing system according to an embodiment of the present invention.

Referring to FIG. 1, the augmented reality service providing system may include an augmented reality providing server 100 and a user terminal 110. However, since the augmented reality service providing system of FIG. 1 is only one embodiment of the present invention, the present invention is not limited to FIG. 1, and may be configured differently from FIG. 1 according to various embodiments of the present invention. there is.

In general, each component of the augmented reality service providing system of FIG. 1 is connected through a network (not shown). Network refers to a connection structure that allows information exchange between nodes such as terminals and servers, including Local Area Network (LAN), Wide Area Network (WAN), and World Wide Area Network (WWW). Wide Web), wired and wireless data communication networks, telephone networks, wired and wireless television communication networks, etc. Examples of wireless data communication networks include 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Bluetooth communication, infrared communication, and ultrasound. This includes, but is not limited to, communication, Visible Light Communication (VLC), LiFi, etc.

The user terminal 110 may request an augmented reality service from the augmented reality providing server 100. For example, the user terminal 110 may request an augmented reality service for an escape route from the augmented reality providing server 100 in a disaster situation.

When an augmented reality service is requested from the user terminal 110, the augmented reality providing server 100 may receive images required to provide the requested augmented reality service from a plurality of cameras.

The augmented reality providing server 100 may detect at least one object of interest from the received image and analyze the detected object of interest. For example, the augmented reality providing server 100 may analyze the movement path, emotions, and actions of the object of interest through the received image.

The augmented reality providing server 100 can determine the situation at the time the image was captured through the received image. For example, when the augmented reality providing server 100 detects the occurrence of a disaster event through analysis of a received video, it may determine that the situation in which the video was captured is a disaster situation.

The augmented reality providing server 100 may generate augmented reality data based on information about the analyzed object of interest and the determined situation. For example, when the determined situation is a disaster situation, the augmented reality providing server 100 may generate augmented reality data about the escape route based on the movement path of at least one object of interest in the image.

The augmented reality providing server 100 may synthesize the augmented reality data generated in the image and then provide the image in which the augmented reality data is synthesized to the user terminal 110.

The user terminal 110 may output an image synthesized with augmented reality data received from the augmented reality providing server 100. For example, the user terminal 110 may display and output the direction of movement for an escape route on an image synthesized with augmented reality data.

The user terminal 110 may include, for example, a mobile terminal capable of wired or wireless communication. A mobile terminal is a wireless communication device that guarantees portability and mobility, and includes not only smartphones, tablet PCs, and wearable devices, but also Bluetooth (BLE, Bluetooth Low Energy), NFC, RFID, ultrasonic, infrared, and Wi-Fi ( It may include various devices equipped with communication modules such as WiFi) and LiFi. However, the client 120 is not limited to those illustrated above.

Hereinafter, the operation of each component of the augmented reality service providing system of FIG. 1 will be described in more detail.

FIG. 2 is a block diagram of the augmented reality providing server 100 shown in FIG. 1 according to an embodiment of the present invention.

Referring to FIG. 2, the augmented reality providing server 100 includes a receiving unit 200, an object analysis unit 210, a situation determination unit 220, a generating unit 230, a combining unit 240, and a providing unit 250. may include. However, the augmented reality providing server 100 shown in FIG. 2 is only one implementation example of the present invention, and various modifications are possible based on the components shown in FIG. 2. Hereinafter, FIG. 2 will be described with reference to FIGS. 3A to 4C.

The receiving unit 200 can receive images from a plurality of cameras. For example, when receiving a request for an augmented reality service from the user terminal 110, the receiver 200 may receive the corresponding images from a plurality of cameras that have captured images necessary to provide the requested augmented reality service. For example, when a request for an augmented reality service for finding a missing child is received from the user terminal 110, images can be received from a plurality of cameras that take pictures within a preset range from the location where the child went missing. For example, when an augmented reality service for store guidance according to the location of the user terminal 110 is requested from the user terminal 110, a plurality of stores existing within a preset range from the location of the user terminal 110 are photographed. Images can be received from multiple cameras.

The object analysis unit 210 may detect at least one object of interest from the received image. Referring to FIG. 3A , the object analysis unit 210 may detect the object of interest 303 by separating the object of interest 303 from the background in the received image 301 using a full-background separation algorithm. Additionally, the object analysis unit 210 may detect the object of interest 303 from the received image 301 using an object detection method (eg, a motion detector, an object detector, etc.).

The object analysis unit 210 may analyze an object of interest detected from an image. For example, the object analysis unit 210 may track the movement path of the object of interest through the received image. For example, when a missing child report is received from the user terminal 110, the object analysis unit 210 sets the missing child as an object of interest, and sets the object of interest based on the movement path of the object of interest tracked in the received image. The final location information can be obtained. For example, when disaster information is received from an on-site control terminal (not shown), an escape route (evacuation movement path) may be analyzed in the image based on the movement path of at least one object of interest in the image. For example, the object analysis unit 210 may analyze the mood of the object of interest based on the facial expression information of the detected object of interest. For example, the object analysis unit 210 may analyze the movement line of an object of interest in images captured within a store and detect points where many people walk within the store. For example, the object analysis unit 210 may analyze consumption pattern information of the object of interest from products purchased by the object of interest or purchase receipts.

The situation determination unit 220 may determine the situation in which the image was captured through the received image. For example, when a missing child report is received from the user terminal 110, the situation determination unit 220 may determine that the situation in which the received video was captured is a situation in which a missing child has occurred. For example, when receiving a request from the user terminal 110 to provide an augmented reality service based on the location of the user terminal 110, the situation determination unit 220 may determine that the situation in which the received image was captured is a general situation. . For example, when disaster information is received from an on-site control terminal (not shown), the situation determination unit 220 determines that the situation in which the received video was captured is a disaster occurrence situation, or determines that a disaster event has occurred from the received video. When detected, the situation in which the video was captured can be determined to be a disaster situation.

The generator 230 may generate augmented reality data based on information about the analyzed object of interest and the determined situation. For example, when the determined situation is a missing child situation, the generator 230 may generate augmented reality data about the movement path of the object of interest corresponding to the missing child and the final location information of the object of interest. For example, when the determined situation is a disaster situation, the generator 230 may generate augmented reality data about the escape route based on the movement path of at least one object of interest in the image. For example, when the determined situation is a general situation (an augmented reality service request situation), the generator 230 generates information corresponding to the product selected based on the movement and consumption pattern information of the object of interest analyzed from the image. Augmented reality data including: For example, when the determined situation is a normal situation and the user's location information is received from the user terminal 110, the generator 230 stores a plurality of stores detected based on the user's location information from the received image. Augmented reality data can be generated.

The synthesis unit 240 may synthesize augmented reality data generated from the received image. For example, the synthesis unit 240 calculates the intrinsic matrix and distortion coefficient of the camera that captured the image to create an augmented reality effect that makes the augmented reality data appear to exist in reality and creates homography. Augmented reality data can be converted through homography transformation, polar coordinate transformation (log-polar transformation), and the converted augmented reality data can be synthesized into an image.

The provider 250 may provide an image synthesized with augmented reality data to the user terminal 110 .

1st Example (Store guidance method)

When the object analysis unit 210 receives the user's location information from the user terminal 110, the object analysis unit 210 may detect a plurality of stores from images received from a plurality of cameras based on the user's location information. As another example, when the object analysis unit 210 receives a request from the user terminal 110 for information about a specific place that exists near the user's location (e.g., the location of a restroom, the location of a specific restaurant desired by the user, etc.), Based on the user's location information, a specific place requested by the user can be detected from the received image.

The generator 230 may generate augmented reality data for a plurality of detected stores. For example, the generator 230 may generate augmented reality data including store information and store location information for each of the plurality of detected stores. As another example, the generator 230 may generate augmented reality data including the direction of movement to a specific location requested by the user.

The synthesis unit 240 may display information about a plurality of stores found by combining augmented reality data with an image corresponding to the user's location information. For example, the composite unit 240 may display information about a plurality of stores existing on each floor in an image corresponding to each floor of a shopping mall where the user is located, based on augmented reality. Additionally, the synthesis unit 240 may display an arrow indicating the direction of movement for a specific store searched by the user among a plurality of stores in a shopping mall by combining the image corresponding to the user's location. Additionally, the composite unit 240 may display information about a store that is to be recommended to a user among a plurality of stores in a shopping mall (for example, information about a store that is holding a sale event) on the image.

The provider 250 may provide the user terminal 110 with an image displaying information about a plurality of stores. For example, the provider 250 may provide the user terminal 110 with an image with an arrow indicating the direction of movement toward a specific store. As another example, the provider 250 may provide each user terminal 110 with augmented reality data including information on a plurality of stores along with an image corresponding to the user's location information.

2nd Example (How to visualize the movement of objects of interest and display products)

The object analysis unit 210 may analyze the movement line of at least one object of interest through images captured within the store.

The generator 230 may generate augmented reality data including a movement path of the object of interest and information on the probability of passage of the object of interest along that route.

The synthesis unit 240 may synthesize augmented reality data generated in an image. Referring to FIG. 3B, an arrow 305 indicating the movement path of each object of interest and passage probability information for each movement line may be displayed in the image 301 synthesized with augmented reality data.

In another embodiment, the object analysis unit 210 may analyze the movement line of the object of interest through images captured within the store and analyze consumption pattern information from the customer's receipt image.

The generator 230 may select a store product to recommend to a customer based on the analyzed movement path and consumption pattern information of the object of interest. Alternatively, the creation unit 230 may select a location for placing products to be sold to customers in a store based on the movement path and consumption pattern information of the object of interest. Alternatively, the creation unit 230 may select a point where the movement line of the object of interest is concentrated as a location for product placement.

The generator 230 may generate augmented reality data including a virtual product image corresponding to the selected product (or a product to be sold) and placement information of the virtual product image. For example, as shown in FIG. 4A, the generator 230 may generate a virtual image 401 corresponding to a TV product and a virtual image 403 corresponding to a washing machine product.

The synthesis unit 240 may synthesize a virtual product image into an image based on placement information of the virtual product image included in the augmented reality data. For example, referring to FIGS. 4B to 4C, the synthesis unit 240 places a virtual image corresponding to a TV product at a position corresponding to the first arrangement information of the virtual image 401 corresponding to the TV product among the pages detected in the image. The image 401 may be synthesized, and the virtual image 403 corresponding to the washing machine product may be synthesized at a position corresponding to the second arrangement information of the virtual image 403 corresponding to the washing machine product.

In addition, the synthesis unit 240 may synthesize guidance route information that guides the user to the location where the recommended product selected based on the consumption tendency of the user of the user terminal 110 requesting the augmented reality service is placed into the image.

The provider 250 may provide an image synthesized with augmented reality data to the user terminal 110.

3rd Example (How to find a lost child)

When a missing child report is received, the object analysis unit 210 may select the missing child from the video as an object of interest.

The object analysis unit 210 can track the movement path of the object of interest from the image and determine the final location information of the object of interest based on the movement path of the tracked object of interest. For example, the object analysis unit 210 may search the video for the point where the missing child and the guardian first separated and track the movement path of the object of interest from the searched point.

The generator 230 may generate augmented reality data including the movement path and final location information of the object of interest.

The synthesis unit 240 may display the movement path and final location information of the object of interest in the image based on the augmented reality data and synthesize the image.

The providing unit 250 may provide the user terminal 110 with an image combining the movement path and final location information of the object of interest.

4th Example (How to provide an escape route in a disaster situation)

The object analysis unit 210 may analyze an escape route based on the movement path of at least one object of interest from the image.

In case of a disaster situation, the generator 230 may generate augmented reality data about the escape route based on the movement route of the object of interest analyzed through the image.

The synthesis unit 240 may display the direction of movement for the escape route by combining augmented reality data with the image.

The providing unit 250 may provide an image showing the direction of movement for an escape route to at least one user terminal 110 located at a location where a disaster situation occurs.

Meanwhile, those skilled in the art will know that the receiving unit 200, the object analysis unit 210, the situation determination unit 220, the generating unit 230, the combining unit 240, and the providing unit 250 are each implemented separately, or one of them is implemented separately. It will be fully understood that more than one may be integrated and implemented.

Figure 5 is a flowchart showing a method of providing an augmented reality service in the augmented reality providing server 100 according to an embodiment of the present invention.

The method for providing an augmented reality service according to the embodiment shown in FIG. 5 includes steps processed in time series in the augmented reality providing server 100 and the user terminal 110 according to the embodiment shown in FIGS. 1 to 4C. . Therefore, even if the content is omitted below, the content described regarding the augmented reality providing server 100 and the user terminal 110 of FIGS. 1 to 4C can also be applied to the augmented reality service providing method according to the embodiment shown in FIG. 5. there is.

Referring to FIG. 5, in step S501, the augmented reality providing server 100 may receive images from a plurality of cameras.

In step S503, the augmented reality providing server 100 may detect at least one object of interest from the received image and analyze the detected object of interest.

In step S505, the augmented reality providing server 100 may determine the situation in which the image was captured through the received image.

In step S507, the augmented reality providing server 100 may generate augmented reality data based on information about the analyzed object of interest and the determined situation.

In step S509, the augmented reality providing server 100 may synthesize augmented reality data generated in the image.

In step S511, the augmented reality providing server 100 may provide an image synthesized with augmented reality data to the user terminal 110.

Although not shown in FIG. 5 , the augmented reality providing server 100 may receive the user's location information from the user terminal 110 in step S501.

In the above description, steps S501 to S511 may be further divided into additional steps or combined into fewer steps, depending on the implementation of the present invention. Additionally, some steps may be omitted or the order between steps may be changed as needed.

FIG. 6 is a block diagram of the user terminal 110 shown in FIG. 1 according to an embodiment of the present invention.

Referring to FIG. 6, the user terminal 110 may include a request unit 600, a location information transmission unit 610, a reception unit 620, and an output unit 630. However, the user terminal 110 shown in FIG. 6 is only one implementation example of the present invention, and various modifications are possible based on the components shown in FIG. 6.

The request unit 600 may request an augmented reality service from the augmented reality providing server 100. For example, the request unit 600 may request an augmented reality service for finding a missing child from the augmented reality providing server 100. For example, the request unit 600 may request an augmented reality service for an escape route from the augmented reality providing server 100 in a disaster situation. For example, the request unit 600 may request an augmented reality service for store information, recommended products for a specific store, and route guidance to a specific store from the augmented reality providing server 100.

The location information transmission unit 610 may transmit location information of the user terminal 110 to the augmented reality providing server 100. For example, when an augmented reality service for an escape route or store guide is requested, the location information transmitter 610 may transmit the location information of the user terminal 110 to the augmented reality providing server 100.

The receiving unit 620 may receive an image synthesized with augmented reality data corresponding to the augmented reality service from the augmented reality providing server 100. Here, augmented reality data includes information on objects of interest analyzed from images and situations determined from images (images received from multiple cameras that captured images necessary for providing augmented reality services) (e.g., a missing child situation, a disaster occurrence) situation, general situation, etc.). Here, information about the object of interest may include, for example, the movement path of the object of interest, the movement line of the object of interest, and consumption pattern information.

The output unit 630 may output an image synthesized from the received augmented reality data.

For example, when an augmented reality service for finding a missing child is requested from the augmented reality providing server 100, the receiver 620 may determine the movement path of the object of interest corresponding to the missing child tracked from the video. The final location information of the object of interest identified based on can be received from the augmented reality providing server 100.

The output unit 630 may display the final location information of the object of interest on the image synthesized with the augmented reality data.

For example, when an augmented reality service for store guidance is requested from the augmented reality providing server 100, the receiver 620 receives augmented reality data for a plurality of stores detected from the image and location information of the user terminal 110. You can receive the corresponding video.

The output unit 630 may display and output augmented reality data about a plurality of stores on an image corresponding to the location information of the user terminal 110.

The output unit 630 displays and outputs the direction of movement of the guide path leading from the current location of the user terminal 110 to the store selected by the user among the plurality of stores on the image corresponding to the location information of the user terminal 110. You can.

For example, when an augmented reality service for an escape route according to a disaster situation is requested from the augmented reality providing server 100, the receiver 620 provides an escape route that guides from the current location of the user terminal 110 to a safe place. An image in which augmented reality data is synthesized can be received. Here, the augmented reality data about the escape route may be data generated based on the movement path of at least one object of interest analyzed through an image.

The output unit 630 may display the direction of movement for the escape route on an image synthesized with augmented reality data and output it.

For example, when requesting an augmented reality service for a specific store, the receiver 620 receives a virtual image of the product corresponding to the product sold at the specific store (or a product to be recommended to the user, etc.) from the augmented reality providing server 100. And an image synthesized with augmented reality data including placement information of a virtual product image may be received.

The output unit 630 may display and output the virtual product image at a location corresponding to the arrangement information of the virtual product image among the images in which the augmented reality data is synthesized.

Meanwhile, those skilled in the art will fully understand that the request unit 600, the location information transmitter 610, the receiver 620, and the output unit 630 may be implemented separately, or one or more of them may be integrated and implemented. will be.

Figure 7 is a flowchart showing a method of providing an augmented reality service in the user terminal 110, according to an embodiment of the present invention.

The method for providing an augmented reality service according to the embodiment shown in FIG. 7 includes steps processed in time series in the augmented reality providing server 100 and the user terminal 110 according to the embodiment shown in FIGS. 1 to 6. . Therefore, even if the content is omitted below, the content described regarding the augmented reality providing server 100 and the user terminal 110 of FIGS. 1 to 6 can also be applied to the augmented reality service providing method according to the embodiment shown in FIG. 7. there is.

Referring to FIG. 7, in step S701, the user terminal 110 may request an augmented reality service from the augmented reality providing server 100.

In step S703, the user terminal 110 may receive an image in which augmented reality data corresponding to the augmented reality service is synthesized from the augmented reality providing server 100. Here, the augmented reality data may be data generated based on information about the object of interest analyzed from the image and the situation determined from the image.

In step S705, the user terminal 110 may output an image synthesized from the received augmented reality data.

In the above description, steps S701 to S705 may be further divided into additional steps or combined into fewer steps, depending on the implementation of the present invention. Additionally, some steps may be omitted or the order between steps may be changed as needed.

One embodiment of the present invention may also be implemented in the form of a recording medium containing instructions executable by a computer, such as program modules executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and non-volatile media, removable and non-removable media. Additionally, computer-readable media may include all computer storage media. Computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data.

The description of the present invention described above is for illustrative purposes, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential features of the present invention. will be. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. For example, each component described as single may be implemented in a distributed manner, and similarly, components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. .

100: Augmented reality provision server
110: user terminal
200: Receiving unit
202: Location information receiver
210: Object analysis unit
220: Situation judgment unit
230: Generation unit
240: synthesis part
250: Provider
610: Request section
620: Location information transmission unit
630: Receiving unit
630: output unit

Claims (18)

In the augmented reality provision server that provides augmented reality services,
A receiving unit that receives images from a plurality of cameras;
an object analysis unit that detects at least one object of interest from the received image and analyzes the detected object of interest;
a situation determination unit that determines a situation in which the image was captured based on the received image;
a generator that generates augmented reality data based on information about the analyzed object of interest and the determined situation;
a synthesis unit that synthesizes the generated augmented reality data with the image; and
It includes a providing unit that provides an image synthesized with the augmented reality data to a user terminal,
The object analysis unit analyzes the movement path and consumption pattern information of the object of interest from the received image.
According to claim 1,
The object analysis unit
Tracking the movement path of the object of interest from the image,
An augmented reality providing server that determines final location information of the object of interest based on the tracked movement path of the object of interest.
delete According to claim 1,
The generator
Select a product to recommend to customers based on the movement path and consumption pattern information of the analyzed object of interest,
An augmented reality providing server that generates augmented reality data including a product virtual image corresponding to the selected product and placement information of the product virtual image.
According to claim 4,
The synthesis part
An augmented reality providing server that synthesizes the virtual product image into the image based on placement information of the virtual product image included in the augmented reality data.
According to claim 1,
The generator
If the determined situation is a disaster situation, augmented reality data about the escape route is generated based on the movement path of the object of interest analyzed through the image,
The augmented reality providing server wherein the synthesis unit synthesizes the augmented reality data with the image and displays a movement direction for an escape route.
According to claim 1,
The object analysis unit
When receiving the user's location information from the user terminal, search for a plurality of stores from the image based on the location information,
The generator
Generating augmented reality data for the detected plurality of stores,
The synthesis part
An augmented reality providing server that displays information about the plurality of stores found by combining the augmented reality data with an image corresponding to the user's location information.
In a user terminal that provides an augmented reality service,
a request unit that requests the augmented reality service from an augmented reality providing server;
a receiving unit that receives an image synthesized with augmented reality data corresponding to the augmented reality service from the augmented reality providing server; and
An output unit that outputs an image synthesized from the received augmented reality data.
Including,
The augmented reality data is generated based on information on objects of interest detected and analyzed from images received from a plurality of cameras and the situation in which the images are determined to have been captured,
The user terminal wherein the information on the analyzed object of interest includes movement line information and consumption pattern information of the object of interest.
According to claim 8,
The receiving unit is configured to receive final location information of the object of interest identified based on the movement path of the object of interest tracked from the image from the augmented reality providing server.
According to claim 8,
The user terminal further comprising a location information transmission unit that transmits location information of the user terminal to the augmented reality providing server.
According to claim 10,
The receiving unit receives augmented reality data for a plurality of stores detected from the image and an image corresponding to location information of the user terminal.
According to claim 11,
The output unit displays and outputs augmented reality data about the plurality of stores on an image corresponding to the location information of the user terminal.
According to claim 8,
If the determined situation is a disaster situation, the receiving unit receives an image synthesized with augmented reality data about an escape route,
The augmented reality data about the escape route is generated based on the movement route of the object of interest analyzed through the image.
According to claim 13,
The output unit displays the direction of movement for the escape route on the image and outputs it.
According to claim 8,
The user terminal wherein the receiving unit receives an image in which augmented reality data including a product virtual image corresponding to the product and placement information of the product virtual image are synthesized from the augmented reality providing server.
According to claim 15,
The product is a product selected based on the movement path and consumption pattern information of the object of interest analyzed through the video.
In a method of providing augmented reality services from an augmented reality provision server,
Receiving images from a plurality of cameras;
Detecting at least one object of interest from the received image and analyzing the detected object of interest;
determining a situation in which the image was captured through the received image;
generating augmented reality data based on information about the analyzed object of interest and the determined situation;
synthesizing the generated augmented reality data with the image; and
Comprising the step of providing an image synthesized with the augmented reality data to a user terminal,
The step of analyzing the object of interest includes analyzing the movement line and consumption pattern information of the object of interest from the received image.
According to claim 17,
A method for providing augmented reality, further comprising receiving user location information from the user terminal.

Images