JP7465228B2 - Augmented Reality Systems - Google Patents

Description

The present invention relates to an augmented reality system.

Guide information in the form of augmented reality is being provided (see Patent Document 1).

Patent No. 6100496

Advertisements cannot be displayed when using augmented reality guide information.

The present invention was made in light of this background, and aims to provide a technology that can provide advertisements in an augmented reality space.

The main invention of the present invention for solving the above problem is an augmented reality system comprising a storage unit that stores content objects representing content to be displayed in an augmented reality space and advertising information, an input unit that accepts designation of the content objects, an object management unit that places the advertising information in the augmented reality space together with the designated content object, and a purchasing processing unit that performs purchasing processing for products related to the advertising objects.

Other problems and solutions disclosed in this application will be made clear in the description of the embodiments of the invention and the drawings.

The present invention makes it possible to provide advertisements in an augmented reality space.

1 is an overall configuration diagram of an augmented reality system 1 according to an embodiment of the present invention. 1 is a system configuration diagram of an augmented reality system 1 according to an embodiment of the present invention. 13 is a flowchart showing a process in which a user terminal 3 receives an advertisement. FIG. 2 is an explanatory diagram relating to the display of augmented reality content in the augmented reality system according to the present embodiment. FIG. 2 is an explanatory diagram relating to a product purchasing process in the augmented reality system according to the present embodiment. FIG. 11 is a system configuration diagram of an augmented reality system 1 according to a second embodiment.

<Summary of the Invention>
The present invention will be described below with reference to the preferred embodiments thereof.
[Item 1]
a storage unit that stores a content object representing a content to be displayed in an augmented reality space and advertisement information;
an input unit for receiving a designation of the content object;
an object management unit that places the advertisement information together with the designated content object in the augmented reality space;
a purchase processing unit that performs purchasing processing for a product related to the advertisement object;
An augmented reality system comprising:

Specific examples of an advertisement providing system according to an embodiment of the present invention will be described below with reference to the drawings. Note that the present invention is not limited to these examples, but is intended to include all modifications within the scope and meaning equivalent to the claims as set forth in the claims. In the following description, the same elements in the description of the drawings will be given the same reference numerals, and duplicate descriptions will be omitted.

Fig. 1 is an overall configuration diagram of an augmented reality system 1 according to this embodiment. Fig. 2 is a system configuration diagram of the augmented reality system 1 according to this embodiment. The augmented reality system 1 is configured to include a server 2, and the server 2 is connected to a user terminal 3 via the Internet 4 so as to be able to communicate bidirectionally.

The server 2 has the function of providing advertisements and information related to advertisements to customers, and mainly includes a server communication unit 21, a memory unit 22, an advertisement provision unit 23, and a payment unit 24.

The server communication unit 21 is a part capable of communicating with external devices such as a customer's terminal, and may be, for example, a wireless communication means.

The memory unit 22 stores databases in which various information is stored, programs, etc. The databases include an advertisement DB 22a in which advertisements used for augmented reality are stored, and a user DB 22b in which personal information of users is stored.

Advertisements used in augmented reality include at least advertisements for restaurants and retail stores. The advertisements also include visual information including still images, videos, and virtual objects, audio information, smell information, taste information, tactile information, and the like. Each advertisement is linked to at least one piece of information. For example, an advertisement for a cafe latte, which is a beverage, is linked to information such as detailed product information, features, price, and sales location. Furthermore, this information may be further linked to other advertisements. For example, the sales location, which is product information, may be linked to advertisements for other products.

In addition to personal information about users, user DB22b also contains information about the user's payment method. Payment methods include, for example, electronic money, tokens, points, etc., which may be used alone or in combination.

The advertisement providing unit 23 has a function of extracting advertisements from the advertisement DB 22a and transmitting them to the user terminal 3. The advertisement providing unit 23 also has a function of providing information according to the request when detailed information about the advertisement is requested by the user. For example, the advertisement providing unit 23 has a function of recognizing a keyword, and has a function of extracting information related to the keyword from the advertisement DB 22a based on the received keyword and transmitting it to the user terminal. The advertisement providing unit 23 may send an advertisement to the user terminal 3 randomly or in response to the user situation or the user terminal situation. The user situation may include, for example, the surrounding environmental situation such as the user's location, and the attribute situation such as the user's preferences and demands. The user terminal situation may include, for example, the behavior and attributes of the virtual character O displayed on the user terminal 3 described later. The advertisement providing unit 23 extracts information from the advertisement DB 22a, but is not limited to this. For example, the advertisement providing unit 23 may access the database (index) of another search engine to extract information.

The payment unit 24 has a function of making payments. For example, when a payment request is received from the user terminal 3, the payment unit 24 accesses the user DB 22b and determines whether the user's means of payment are sufficient. If the payment unit 24 determines that the user's means of payment are sufficient, it can make the payment. Note that the payment unit 24 can also make the payment by credit card or debit card in addition to the user's means of payment.

The user terminal 3 may be an electronic terminal such as a smartphone, tablet, or laptop, and mainly includes a terminal communication unit 31, a camera 32, an input unit 33, a display unit 34, an advertisement acquisition unit 35, an information output unit 36, an object management unit 37, and a payment unit 38.

The terminal communication unit 31 is a part capable of communicating with external devices, such as a wireless communication means.

The camera 32 is the part that performs the photographing operation, and has an imaging element such as a CCD or CMOS element that converts an optical image into an electrical signal, and a shutter. It can also take not only still images but also moving images.

The input unit 33 can input a keyword (search term) to be searched for through a user's operation, such as a touch panel or a keyboard. The input unit 33 can also be operated manually or, for example, input by voice.

The display unit 34 is, for example, a liquid crystal monitor, and is capable of displaying various information. The display unit 34 may also be integrated with the input unit 33.

The advertisement acquisition unit 35 has the function of acquiring advertisements and advertisement-related information from the server 2.

The information output unit 36 has a function to output various information and data to the display unit 34. The information output unit 36 may also have a function to output not only video information, but also audio information, smell information, tactile information, taste information, and the like.

The object management unit 37 has a function to generate virtual objects based on motifs such as virtual characters and everyday items, and to control the attributes and actions of the virtual objects. The object management unit 37 also has a function to control the virtual objects so that they are linked to the input unit 33 and the information output unit 36. For example, the object management unit 37 can control the virtual character to make a listening gesture when there is voice input from the user, or to make a speaking gesture when information is output by the information output unit 36.

The purchasing process 38 has a function to confirm the user's intention to pay the price and a function to transmit the intention to pay to the settlement unit 24 via the terminal communication unit 31. The purchasing process 38 may also be linked to the input unit 33, so that the user can input the intention to pay via the input unit 33.

Figure 3 is a flowchart showing the process by which the user terminal 3 receives an advertisement. Figure 4 is an explanatory diagram showing the display of augmented reality content in the augmented reality system according to this embodiment. Figure 5 is an explanatory diagram showing the process of purchasing a product in the augmented reality system according to this embodiment.

First, in step S1, the user U activates the augmented reality mode on the user terminal 3. The augmented reality mode is a state in which the camera 32 of the user terminal 3 is activated, and a real-space image P (information about the real space) of the surroundings captured by the camera 32 and a virtual character O (virtual object) generated by the object management unit 37 are displayed on the display unit 34.

Next, in step S2, when the user terminal 3 receives an advertisement from the server 2, the user terminal 3 causes the display unit 34 to display the advertisement. In detail, for example, when the user terminal 3 receives an advertisement, the object management unit 37 causes the display unit 34 to output, via the information output unit 36, information so that the virtual character O holds the "cafe latte" that is the advertisement.

Next, in step 3, the user U asks the virtual character O, "What are the characteristics of that cafe latte?" The object management unit 37 then controls the virtual character O to make the listening movement or reply to the voice input by the user. The input unit 33 transmits this information to the advertisement provision unit 23 of the server 2 via the device communication unit 31.

Next, in step S4, upon receiving the notification, the advertisement provision unit 23 of the server 2 extracts information about the characteristics of the caffe latte (e.g., creamy taste) based on the keyword and transmits it to the user terminal 3.

Then, in step S5, the object management unit 37 controls the virtual character O to output information about the characteristics of "cafe latte." As a method of outputting information, for example, the object management unit 37 may cause the virtual character O to output a voice message to the user saying, "It's very creamy," or to perform an action such as showing letters or a diagram to the user.

Then, in step S6, the user completes the product purchase procedure via the virtual character. In detail, for example, when purchasing a caffe latte, the user speaks to virtual character O saying, "I'll buy that caffe latte," and the purchasing process 38 outputs board b, which confirms the user's intention to pay for the caffe latte. Board b is made up of a field showing the product and its price, and a button to confirm the purchase, and the user completes the purchasing procedure by pressing that button. Then, when the purchasing procedure is completed, the purchasing process 38 transmits a notice of intention to pay to the settlement unit 24 of the server 2.

Next, in step S7, the payment unit 24 of the server 2 performs the payment upon receiving the notice of payment, and then transmits notice of the payment to the user terminal 3. Upon receiving notice of the payment, the object management unit 37 outputs a message required for the purchase, such as "OK! I bought it!", to the user as the virtual character O's line, and the process ends.

In this way, the advertising provision system 1 in this embodiment can increase the user U's interest in the advertisement, and has the effect of encouraging the user U to purchase the advertised product.

This concludes the description of the embodiment of the present invention, but the aspects of the present invention are not limited to this embodiment.

In the above embodiment, when the user U decides to purchase the product, the user U is not limited to pressing a button on the board b to confirm the intention to pay, but may also decide to purchase by voice input. For example, when the user U speaks to the virtual character O, saying "I'll buy that cafe latte!", the purchase process 38 may carry out the purchase procedure directly. In this way, the user U can purchase the advertised product without performing any manual operations.

In addition, in the above embodiment, the user U receives the advertisement after activating the augmented reality mode on the user terminal 3, but this is not limited to the above. For example, the user terminal 3 may automatically activate the augmented reality mode after receiving the advertisement. In detail, for example, when the user terminal 3 receives an advertisement from the server 2, it activates the augmented reality mode. Then, the object management unit 37 may notify the user U of the virtual character O that the advertisement has been received.

In the above embodiment, the virtual objects, advertisements, and real-space environment are displayed as images, but this is not limited to this and audio information may also be included.

Figure 6 is a system configuration diagram of the augmented reality system 1 according to the second embodiment.

The user terminal 3 is further provided with a position measurement unit 39. The position measurement unit 39 has a function of acquiring position information of the user terminal 3, for example, by using satellite positioning such as GPS or a beacon that performs positioning using short-range wireless technology. The position measurement unit 39 can also transmit the position information to the server 2 via the terminal communication unit 31.

When the user terminal 3 receives an advertisement from the server 2, the object management unit 37 outputs information about the advertisement via the information output unit 36 as if speaking to the user. The user U hears the advertisement and inputs into the user terminal 3, "Where can I buy that?", and the user terminal 3 transmits this information and its location information to the server 2.

Then, based on the keyword and the location information, the server 2 transmits to the user terminal 3 information about a place to purchase the advertised product near the user's location. The user terminal 3 then outputs the information about the place to purchase as audio. In this way, the user U has the effect of being able to hear the information he or she wants to know about the advertisement by audio without watching the video.

In addition, the information on the advertisement in the above embodiment may be odor information. For example, the information output unit 36 of the user terminal 3 includes at least one odor element, and the information output unit 36 can emit one or more odor elements. The odor element may be, for example, a single chemical substance such as furaneol (strawberry odor) or 1-hexanol (forest odor), or a combination of these. Also, for example, it may be an odor with an equivalent odor to the odor of jasmine as one unit, regardless of the composition of the chemical substance. That is, the information output unit 36 may output an equivalent odor by combining multiple chemical substances, for example, using the odor of strawberry milk as an output method, or may output a combination of an odor with an odor of strawberry as one unit and an odor with an odor of milk as one unit. Therefore, for example, when a user U speaks to the user terminal 3 saying that he wants to know the odor of an advertised product, the information output unit 36 outputs the odor of the advertised product.

In addition, the information on the advertisement in the above embodiment may be taste information. For example, the information output unit 36 of the user terminal 3 includes at least one taste element, and the information output unit 36 can release one or more taste elements. The taste element may be, for example, a single chemical substance such as inosinic acid or glutamic acid, or a combination of these. Also, for example, it may be a taste with a taste equivalent to strawberry flavor as one unit, regardless of the chemical composition. That is, the information output unit 36 may output, for example, a strawberry chocolate taste information output method by outputting an equivalent taste by combining multiple chemical substances, or may output a combination of a taste with strawberry flavor as one unit and a taste with chocolate flavor as one unit. Therefore, for example, when a user U speaks to the user terminal 3 saying that he wants to know the taste of an advertised product, the information output unit 36 outputs the taste of the advertised product.

In addition, the information relating to the advertisement in the above embodiment may be tactile information.

In addition, in the above embodiment, the user terminal 3 receives advertisements discontinuously, but this is not limited to the above. For example, the user terminal 3 may be provided with an advertisement update unit (not shown) that can automatically receive advertisements related to the surrounding environment when the surrounding environment of the user U changes.

For example, the advertisement update unit provided in the user terminal 3 has a function of updating advertisements in response to changes in the user's location, and has a function of detecting a change in location when the user U moves, etc., and having the advertisement acquisition unit 35 acquire new advertisements. Furthermore, the condition for updating advertisements by the advertisement update unit is not limited to a change in the user's location, but may also be a change in time of day. For example, the advertisement update unit may acquire advertisements during a specific time period, or at a specified time interval. Furthermore, the advertisement update unit may acquire updated advertisements every time advertisements related to search results are updated. Furthermore, the advertisement update unit may be integrated with the advertisement acquisition unit.

The above-mentioned augmented reality mode is a state in which the surrounding real space image P (real space information) captured by the camera 32 and the virtual character O (virtual object) generated by the object management unit 37 are displayed on the display unit 34, but is not limited to this. For example, it may be an augmented reality route planning service (e.g., Live View, a Google map service) that recognizes the surrounding real space (buildings, roads, objects, etc.) captured by the camera based on the position information of the user terminal and the direction in which the user terminal is facing.

In addition, in this embodiment, a virtual space using augmented reality (AR) is assumed, but this is not limited thereto, and the advertising provision system of this embodiment can be applied as is to a virtual reality (VR) space or a mixed reality (MR) space.

Second Embodiment
In the second embodiment, the storage unit 22 of the server 2 includes an object DB 22c in addition to an advertisement DB 22a.

The object DB 22c stores model information of virtual objects (hereinafter simply referred to as virtual objects) to be placed in the augmented reality space. The model information can be data in a format such as USDZ or glTF.

In the second embodiment, the providing unit 23 provides augmented reality content. The providing unit 23 can provide the augmented reality content in response to a request from the user terminal 3, for example. A virtual object (hereinafter referred to as a content object) that the user wishes to view, such as a character, is specified in the request. The providing unit 23 reads out the content object specified in the request from the storage unit 22, and responds to the user terminal 3 with augmented reality content including the content object and scene information for placing the content object in an augmented reality space. At this time, the providing unit 23 can respond to the user terminal 3 by including a virtual object (hereinafter referred to as an advertisement object) for displaying an advertisement in the augmented reality content. The providing unit 23 can, for example, attach an advertisement image as the texture of the advertisement object.

In step S2, the user terminal 3 transmits a request for augmented reality content to the server 2. The request includes a specification of a virtual object, such as a character, that the user wishes to view.

In step S3, the server 2 creates augmented reality content in response to the request. The server 2 can create the augmented reality content by, for example, reading model information corresponding to a virtual object (hereinafter referred to as a content object) such as a character specified in the request from the object DB 22b. The server 2 can also generate objects not specified by the user, such as background sets or small items, as advertising objects, and include the advertising information in the texture of the advertising object before attaching it to the augmented reality content. The server 2 can also include scene information specifying the placement positions and movements of the content objects and advertising objects in the augmented reality content.

In step S4, the user terminal 3 receives the augmented reality content and can create an augmented reality scene based on the received augmented reality content.

In this way, the augmented reality system according to the second embodiment can provide advertising objects in association with content objects that the user is about to view. Therefore, advertisements can be placed in the same augmented reality space as the content objects in which the user is interested.

In addition, in the second embodiment, the user can carry out the purchase procedure for products, etc. in the same manner as in the first embodiment.

The purchasing processing unit may also output information necessary for the purchasing process to the augmented reality space.

Furthermore, the object management unit controls the character to output a message required for the purchase form as a line when a request for purchasing the product is received from the user;
An augmented reality system that features:

Reference Signs List 1 Advertisement providing system 2 Server 3 User terminal 21 Server communication unit
22 Storage unit 22a Advertising DB
22b User DB
22c Object DB
23 Advertisement provision unit 24 Settlement unit 31 Terminal communication unit 32 Camera 33 Input unit 34 Display unit 35 Advertisement acquisition unit 36 Information output unit 37 Object management unit 38 Purchase processing unit 39 Position measurement unit

Claims (3)

a storage unit that stores a character object and an advertisement object to be displayed in an augmented reality space;
an input unit that accepts designation of the character object;
an object management unit that places the advertisement object together with the designated character object in the augmented reality space and controls the character object to output information about a product related to the advertisement object as a line ;
a purchase processing unit that performs purchasing processing for a product related to the advertisement object;
An augmented reality system comprising: 2. The augmented reality system of claim 1,
the purchase processing unit outputs information necessary for the purchase processing to the augmented reality space;
An augmented reality system that features: 2. The augmented reality system of claim 1,
when a request for purchasing the product is received from the user, the object management unit controls the character object to output a message required for the purchase as a line;
An augmented reality system that features:

Images