JP6905615B1 - Augmented reality system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an advertisement in an augmented reality space. SOLUTION: In an augmented reality system, an advertisement acquisition unit that acquires advertisement information according to a user attribute for viewing an object arranged in an augmented reality space, an object is arranged in the augmented space, and advertisement information is arranged. It is characterized by having an object management unit for the operation. [Selection diagram] Fig. 1

Description

The present invention relates to an augmented reality system.

Guide information in an augmented reality form is provided (see Patent Document 1).

Japanese Patent No. 6100496

Advertisements cannot be displayed when using augmented reality guide information.

The present invention has been made in view of such a background, and an object of the present invention is to provide a technique capable of providing an advertisement in an augmented reality space.

The main invention of the present invention for solving the above problems is an augmented reality system, which includes an advertisement acquisition unit that acquires advertisement information according to a user attribute for viewing an object arranged in the augmented reality space, and the augmented space. It is characterized in that it is provided with an object management unit for arranging the object and arranging the advertisement information.

Other problems disclosed in the present application and solutions thereof will be clarified in the columns and drawings of the embodiments of the invention.

According to the present invention, it is possible to provide an advertisement in an augmented reality space.

It is an overall block diagram of the augmented reality system 1 which concerns on this embodiment. It is a system block diagram of the augmented reality system 1 which concerns on this embodiment. It is a flowchart which shows the process which a user terminal 3 receives an advertisement. It is explanatory drawing about the display of the augmented reality content in the augmented reality system which concerns on this embodiment. It is explanatory drawing about the interaction between a virtual object and an advertisement in the augmented reality system which concerns on this embodiment.

<Outline of the invention>
The contents of the embodiments of the present invention will be described in a list. The present invention includes, for example, the following configuration.
[Item 1]
An advertisement acquisition unit that acquires advertisement information according to user attributes that view objects placed in augmented reality space,
An object management unit that arranges the object in the extended space and arranges the advertisement information,
An augmented reality system characterized by being equipped with.
[Item 2]
The augmented reality system described in item 1
Further provided with a purchase processing unit for purchasing goods or services in the augmented reality space.
The advertisement acquisition unit acquires the advertisement information according to the purchase history of the product or service.
An augmented reality system featuring.
[Item 3]
The augmented reality system described in item 1
Further provided with an action history acquisition unit for acquiring the action history information of the user in the augmented reality space.
The advertisement acquisition unit acquires the advertisement information according to the action history.
An augmented reality system featuring.
[Item 4]
The augmented reality system described in item 1
The object management unit arranges a plurality of the objects in the augmented reality space.
A browsing detection unit that identifies the object browsed by the user is further provided.
The advertisement acquisition unit acquires the advertisement information according to the browsing history of the object.
An augmented reality system featuring.
[Item 5]
The augmented reality system described in item 1
Further provided with a browsing detection unit that identifies a location in the augmented reality space browsed by the user.
The advertisement acquisition unit acquires the advertisement information according to the browsing history of the place.
An augmented reality system featuring.

<System overview>
A specific example of the augmented reality system according to the embodiment of the present invention will be described below with reference to the drawings. It should be noted that the present invention is not limited to these examples, and is indicated by the scope of claims, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims. In the following description, the same elements will be designated by the same reference numerals in the description of the drawings, and duplicate description will be omitted.

<First Embodiment>
FIG. 1 is an overall configuration diagram of the augmented reality system 1 according to the present embodiment. FIG. 2 is a system configuration diagram of the augmented reality system 1 according to the present embodiment. The augmented reality system 1 includes a server 2, and the server 2 is bidirectionally connected to the user terminal 3 via the Internet 4.

The augmented reality content providing system 1 of the present embodiment is intended to provide content (hereinafter, referred to as augmented reality content) to be displayed as so-called augmented reality (AR).

The server 2 is a computer that provides augmented reality content to the user terminal 3. The server 2 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

The user terminal 3 is a computer operated by a user who wants to browse augmented reality content. The user terminal 3 can be an electronic terminal such as a smartphone, a tablet, or a laptop computer.

<Server 2>
The server 2 mainly includes a communication unit 21, a storage unit 22, a provision unit 23, an action history acquisition unit 24, and a user attribute determination unit 25. The communication unit 21, the provision unit 23, the action history acquisition unit 24, and the user attribute determination unit 25 can be realized, for example, by executing a program stored in the storage by the CPU included in the server 2. Further, the storage unit 22 is realized as a part of the storage area provided by the storage provided by the server 2. The storage can be, for example, a hard disk, a solid state disk, a flash memory, a RAM, a ROM, or the like.

The communication unit 21 communicates with an external device such as the customer's user terminal 3. The communication unit 21 can be, for example, a wireless communication means.

The storage unit 22 stores a database, a program, or the like in which various information is stored.

For example, the storage unit 22 includes an advertisement DB 22a or the like in which advertisement information used in augmented reality is stored. Advertisements used in augmented reality can include advertisements for goods and services. Further, as a form of the advertisement, visual information including a still image or a moving image, audio information, odor information, taste information, tactile information and the like are included. The advertisement information related to the visual information may be a still image or a moving image for displaying the advertisement, or may be text data. The advertisement information related to the voice information may be, for example, voice data encoded by a format such as PCM or MP3, or text data for generating voice information by synthetic voice. In addition, the advertisement information includes advertisement attribute information directly related to the advertisement, advertisement-related information indirectly related to the advertisement, and the like. In the advertisement attribute information, in the case of product advertisement, for example, information such as product type, shape, size, feature, price, mode, use, effect, etc. is included, and in the case of service service advertisement, for example, service etc. Information such as type, feature, price, mode, effect, etc. is included. The advertisement-related information includes, for example, information such as a place where the product is sold in the case of a product advertisement, and information such as a place where the service can be received in the case of a service service.

Further, for example, the storage unit 22 includes an action history DB 22b that stores the user's personal information, the user's action history information, and the purchase history information of the product or service. The user's personal information stored in the action history DB 22b includes, for example, information such as the user's gender, age, taste, and hobby. The user's action history information stored in the action history DB 22b includes, for example, information such as an Internet browsing record by the user, the number of visits, a location history, and a participating social networking service (SNS). In the purchase history information stored in the action history DB 22b, products and services purchased by the user through online shopping and the like are recorded.

The action history acquisition unit 24 collects the user's personal information, the action history information, and the purchase history information, and stores or updates the action history DB 22b. Specifically, as a method of collecting, for example, the information collecting unit accesses a terminal storage unit provided in the user terminal 3 described later, a cookie (cookie) of a web browser, and a location specifying unit described later to collect each information. Although there is a method of obtaining the information, a general method can be adopted, and detailed description thereof will be omitted here.

The user attribute determination unit 25 determines the segment to which the user belongs. The user attribute determination unit 25 determines, for example, a tendency (advertisement range) of an advertisement of interest to the user from at least one of the user's personal information and action history information (user attribute) stored in the storage unit 22. be able to. For example, if the user is a woman in her twenties (personal information) and the behavior history information includes a history of accessing the cafe site multiple times and a history of purchasing coffee beans multiple times, the user attribute is determined. The department can determine that the user is interested in advertisements (advertisement scope) related to coffee, cafe latte, cafe facilities, and the like. In addition, the user attribute determination unit can learn about advertisements of interest to the user by machine learning or the like.

Based on the segment determined by the user attribute determination unit 25, the providing unit 23 extracts the advertisement information belonging to the segment from the advertisement DB 22a and transmits it to the user terminal 3. The providing unit 23 may transmit not only a single advertisement but also a plurality of advertisements to the user terminal 3.

<User terminal 3>
The user terminal 3 mainly includes a communication unit 31, a camera 32, an input unit 33, a display unit 34, an object management unit 35, an advertisement management unit 36, an advertisement response unit 37, an information output unit 38, and a position. The specific unit 39 and the like are included.

The terminal communication unit 31 is a part capable of communicating with an external device, and is, for example, a wireless communication means.

The camera 32 is a part that performs a shooting operation or the like, and has an image pickup element such as a CCD or CMOS element that converts an optical image into an electric signal, and a shutter. Moreover, not only still images but also moving images can be taken.

The input unit 33 can input via a user's operation, such as a touch panel or a keyboard. Further, the input unit 33 may be input not only by hand but also by voice, for example.

The display unit 34 is, for example, a liquid crystal monitor and can display various types of information. Further, the display unit 34 may be integrated with the input unit 33.

The object storage unit 35a stores, for example, model information necessary for generating a virtual object with a motif such as a virtual character or daily necessities. The model information can be, for example, data in a format such as USDZ or glTF. Further, the model information includes, for example, attribute information directly related to the virtual object, related advertisement attribute information corresponding to the attribute information, and advertisement correspondence information corresponding to the advertisement related information. The attribute information can include, for example, information such as appearance, type, mode, and component configuration.

The object management unit 35b creates augmented reality content. Specifically, the object management unit 35b creates an augmented reality image to be superimposed and displayed on an image in the real space (an image taken by the camera 32). The object management unit 35b can create a virtual object based on the model information stored in the object storage unit 35a and arrange it in the augmented reality space. The object management unit 35b can also arrange the advertisement object created by the advertisement management unit 36, which will be described later, in the augmented reality space.

The advertisement receiving unit 36 receives the advertisement information from the server 2.

The advertisement management unit 37 applies the received advertisement information to an advertisement object suitable for the advertisement information. The advertisement object is a virtual object created by the object management unit 35b. The advertising object is an object (peripheral object) delivered together with an object specified by the user, such as a character. The advertising object may be a virtual object that originally displays content on its surface, such as a picture frame or television, or around a virtual object that is the main content such as a character, such as a wall, desk, or door. It may be a virtual object representing an article to be distributed, or it may be another virtual object such as a T-shirt or a cup that is possessed by a virtual object that is the main content such as a character. The advertisement management unit 37 can display the advertisement on the appearance of the advertisement object by, for example, pasting the advertisement information of the still image or the moving image as a texture on the surface of the advertisement object. Further, for example, when the advertisement information is voice information, the voice information can be attached to the advertisement object so that the voice information can be reproduced when the advertisement object is reproduced. For example, when the advertisement management unit 37 receives the advertisement information regarding the new clothes, the advertisement management unit 37 can have the new clothes design attached to the surface of the advertisement object having the clothes as a motif. The advertisement management unit 37 can attach the advertisement information to the advertisement object of the attribute corresponding to the attribute information of the advertisement information. For example, when the advertisement management unit 37 receives the advertisement information of a new song, since the advertisement attribute information is music, it can be attached to an advertisement object having a radio cassette player as a motif, which includes the advertisement correspondence information corresponding to the music. ..

The terminal storage unit has a function of storing user's personal information, Internet browsing history, purchase history of products or services described later, and the like. Further, the terminal storage unit may include a cookie of a web browser.

The purchase processing unit has a function for a user to execute a purchase of a product or service on the Internet. Further, the purchase processing unit has a function of storing the purchase history of the product or service in the terminal storage unit.

The information output unit 38 superimposes the augmented reality image created by the object management unit 35b on the image of the camera 32 that captures the real space and outputs the image. In addition, the information output unit 38 can also output visual information such as moving images, audio information, odor information, tactile information, taste information, and the like. The information output unit can output visual information such as a video via the display unit 34.

The position specifying unit 39 has a function of acquiring and storing the position information of the user terminal 3 by, for example, a beacon for positioning using satellite positioning such as GPS or short-range wireless technology. In addition, the position specifying unit 39 can also transmit the acquired position information to the server 2.

FIG. 3 is a flowchart showing a process in which the user terminal 3 receives an advertisement. FIG. 4 is an explanatory diagram regarding the display of augmented reality content in the augmented reality system according to the present embodiment. FIG. 5 is an explanatory diagram regarding the interaction between the virtual object and the advertisement in the augmented reality system according to the present embodiment.

First, as step S1, the user activates the augmented reality mode on the user terminal 3. In the augmented reality mode, the camera 32 of the user terminal 3 is activated, the surrounding real space image P (information in the real space) taken by the camera 32, and the virtual character O generated by the object management unit 37. A virtual drink D (virtual object) with a drink as a motif is displayed on the display unit 34.

Next, in step S2, the information collecting unit of the server acquires the user's personal information, behavior history information, purchase history information, etc. from the user terminal and stores or updates the information in the storage unit. Then, the server determines the advertisement range to be provided to the user based on the personal information, the behavior history information, and the purchase history information stored in the storage unit.

Next, in step S3, the server 2 extracts the advertisement belonging to the determined advertisement range from the advertisement DB and transmits it to the user terminal 3.

Next, in step S4, the advertisement management unit 37 causes the received advertisement to be associated with the appropriate virtual drink D to generate the advertisement object C.

Next, in step S5, the object management unit 35b controls the virtual character O to react to the advertisement object C. In the present embodiment, the virtual character O raises the advertisement object C, but the present invention is not limited to this. For example, the virtual character may apply an external force to the advertising object such as touching or hitting the advertising object. Virtual characters may also refer to advertising objects. For example, when an advertisement is received or an advertisement object is generated, the virtual character may output a voice to that effect. Further, the virtual character may be used depending on its purpose. For example, if the advertising object is a drink, the virtual character can make a drinking gesture on the advertising object.

Next, in step S6, the user displays and reproduces the content of the advertisement, and the process in the present embodiment is completed.

As described above, the augmented reality system 1 in the present embodiment can output an advertisement together with the augmented reality content to the user.

<Second embodiment>
In the second embodiment, the user terminal 3 omits the advertisement receiving unit 36 and the advertisement management unit 37 so that the storage unit 22 of the server 2 includes the object DB 22b in addition to the advertisement DB 22a.

The object DB 22b stores model information (hereinafter, also simply referred to as a virtual object) of a virtual object arranged in the augmented reality space. The model information can be, for example, data in a format such as USDZ or glTF.

In the second embodiment, the providing unit 23 provides augmented reality content. The providing unit 23 can provide augmented reality content in response to a request from the user terminal 3, for example. In the request, a virtual object (hereinafter referred to as a content object) that the user wants to view, such as a character, is specified. The providing unit 23 reads the content object specified in the request from the storage unit 22, and responds to the user terminal 3 with augmented reality content including the content object and scene information for arranging the content object in the augmented reality space. .. At this time, the providing unit 23 can respond to the user terminal 3 by including a virtual object for displaying the advertisement (hereinafter, referred to as an advertisement object) in the augmented reality content. The providing unit 23 can paste an advertising image as a texture of an advertising object, for example.

In the second embodiment, prior to step S2, the user terminal 3 transmits a request for augmented reality content to the server 2. The request includes the specification of a virtual object such as a character that the user wants to view.

Further, in step S3, the server 2 creates augmented reality content in response to the request. The server 2 can create augmented reality content by reading model information corresponding to a virtual object (hereinafter, referred to as a content object) such as a character specified in the request from the object DB 22b. The server 2 can also generate an object such as a background set or an accessory that is not specified by the user as an advertisement object, attach the advertisement information to the texture of the advertisement object, and then include it in the augmented reality content. In addition, the server 2 can include scene information in which the arrangement position and movement of the content object and the advertisement object are specified in the augmented reality content.

In step S4, the user terminal 3 receives the augmented reality content and can create an augmented reality scene based on the received augmented reality content.

In the second embodiment, steps S5 and S6 can be omitted.

As described above, according to the augmented reality content providing system according to the second embodiment, the advertisement object can be provided in association with the content object that the user intends to browse. Therefore, the advertisement can be placed in the same augmented reality space as the content object that the user is interested in.

Although the description of the embodiment of the present invention is completed above, the aspect of the present invention is not limited to this embodiment.

In the above embodiment, the segment is specified according to the attribute of the user, but the segmenting for the targeting advertisement is not limited to the above method. The segmentation may be performed based on, for example, the purchase history or the behavior history. Segmentation may be performed based on the behavior history limited to the behavior in the augmented reality space. In addition, it is possible to specify where the user was looking in the augmented reality space by, for example, where the focusing point of the camera 32 was, and segmentation is performed according to the history of the place the user was looking at. You may do it. In addition, it is possible to specify which object arranged in the augmented reality space the user was looking at, and to perform segmentation according to the object that the user was looking at.

In the above embodiment, the user reproduces the advertisement by operating the input unit 33, but the present invention is not limited to this. For example, the advertisement management unit 36 has a function of detecting a keyword, and the advertisement management unit 36 has a function of displaying or playing an advertisement when it detects a keyword related to an advertisement during a voice or text conversation between a virtual character and a user. You may prepare. By doing so, there is an effect that the user has more interest and familiarity with the advertisement. (Effect of claim 5)

In the above embodiment, the advertisement providing unit 23 of the server 2 extracts the advertisement related to the location information and transmits it to the user terminal 3, but the present invention is not limited to this. For example, the advertisement providing unit 23 has a function of acquiring the attribute information of the virtual object from the object management unit 35, and may extract the advertisement related to the acquired attribute information. By doing so, the server 2 can provide advertisements that are of great interest to the user. (Effect of claim 6)

Further, in the above embodiment, the advertisement object is generated by associating the advertisement with the virtual object, but the present invention is not limited to this. For example, the object management unit 35 of the user terminal 3 may have a function of directly converting an advertisement into an advertisement object (virtual object). (Effect of claim 3)

Further, in the augmented reality mode in the above embodiment, the surrounding real space image P (information in the real space) captured by the camera 32 and the virtual character O (virtual object) generated by the object management unit 37 are used. Is displayed on the display unit 34, but the present invention is not limited to this. For example, an augmented reality draft service that recognizes the surrounding real space (buildings, roads, objects, etc.) taken by the camera based on the location information of the user terminal and the direction in which the user terminal is facing (for example, Google map). It may be a service (Live View).

Further, the object management unit 35 may have a function of converting all or a part of the surrounding real space (buildings, roads, objects, etc.) recognized by the augmented reality mode into a virtual object (real space object). (Effect of claim 4)

Further, the advertisement providing unit 23 of the server 2 may extract the advertisement related to the position information and the information about the real space object and send it to the user terminal 3. (Effect of claim 7)

Further, in the above embodiment, the user terminal 3 receives the advertisement when the augmented reality mode is activated, but the present invention is not limited to this. For example, the user terminal 3 may automatically receive (update) an advertisement related to the surrounding environment when the surrounding environment changes. (Effect of claim 11)

For example, the user terminal 3 may be provided with an advertisement updating unit (not shown) that updates the advertisement according to a change in the location of the user. Specifically, the advertisement update unit has a function of detecting when the user moves or the like and the location changes, and causing the advertisement management unit 36 to acquire a new advertisement. (Effect of claim 12)

Further, the condition for updating the advertisement by the advertisement update unit is not limited to the change in the location of the user, but may be the change in the time zone. For example, the advertisement update unit may acquire the advertisement at a specific time zone or may acquire the advertisement at a predetermined time interval. In addition, the advertisement update unit may acquire the updated advertisement every time the advertisement related to the search result is updated. Further, the advertisement update unit may be integrated with the advertisement management unit 36. (Effect of claim 13)

In the above embodiment, the advertisement is displayed as a video, but the advertisement is not limited to this and may be audio information.

Further, the advertisement in the above embodiment and the environment in the real space may be odor information. For example, the information output unit 38 of the user terminal 3 contains at least one odor element, and the information output unit 38 can emit one or more odor elements. The odorant may be a single chemical substance such as furaneol (strawberry scent) or 1-hexanol (forest scent), or a combination thereof. Further, for example, regardless of the composition of the chemical substance, an odor having a scent equivalent to that of jasmine as one unit may be used. That is, for example, the information output unit 38 may output an equivalent odor by combining a plurality of chemical substances using the odor of strawberry milk as an output method, or the odor of strawberry as one unit and the odor of milk. The odor may be output in combination with the odor of one unit.

Further, the advertisement in the above embodiment and the environment in the real space may be taste information. For example, the information output unit 38 of the user terminal 3 contains at least one taste element, and the information output unit 38 can release one or more taste elements. The taste element may be, for example, a single chemical substance such as inosinic acid or glutamic acid, or a combination thereof. Further, for example, regardless of the composition of the chemical substance, a taste equivalent to the strawberry taste may be used as one unit. That is, for example, the information output unit 38 may output the same taste by combining a plurality of chemical substances by using the taste information of strawberry chocolate as an output method, or the taste with the strawberry taste as one unit and the chocolate taste. May be output in combination with a taste having a unit of.

Further, the advertisement in the above embodiment and the environment in the real space may be tactile information. (Effect of claim 14)

Further, in the present embodiment, a virtual space based on augmented reality (AR) is assumed, but the present invention is not limited to this, and is limited to a virtual reality (VR) space or a mixed reality (MR) space. Even if there is, the advertising system of this embodiment can be applied as it is.

1 Advertising system 2 Server 3 User terminal 21 Server communication unit
22 Storage unit 22a Advertising DB
23 Advertising section 31 Terminal communication section 32 Camera 33 Input section 34 Display section 35 Object management section 36 Advertising management section 37 Advertising response section 38 Information output section 39 Location identification section

Claims (4)

An object management unit that places objects in augmented reality space,
And action history acquisition unit that acquires action history information is a behavior history of the user, including the location of the user in the augmented reality space,
Advertising information storage unit that stores advertising information,
An advertisement obtaining section that obtains the advertisement information corresponding to the action history information of the user from the advertisement information storage section,
Equipped with a,
The object management unit is an augmented reality system characterized in that the acquired advertisement information is arranged in the augmented reality space. The augmented reality system according to claim 1.
Further provided with a purchase processing unit for purchasing goods or services in the augmented reality space.
The advertisement acquisition unit acquires the advertisement information according to the purchase history of the product or service.
An augmented reality system featuring. The augmented reality system according to claim 1.
The object management unit arranges a plurality of the objects in the augmented reality space.
A browsing detection unit that identifies the object browsed by the user in the augmented reality space is further provided.
The advertisement acquisition unit acquires the advertisement information according to the browsing history of the object.
An augmented reality system featuring. The augmented reality system according to claim 1.
Further provided with a browsing detection unit for identifying a browsing location in the augmented reality space browsed by the user.
The advertisement acquisition unit acquires the advertisement information according to the browsing history of the browsing location.
An augmented reality system featuring.

Images