JP2022011070A - Event management server system, event management system, and event management method - Google Patents

Abstract

To enable visitors to an event to easily utilize various services using portable terminals.SOLUTION: An event management server causes a superimposed image 202 indicating the face photographs or records, etc., of contestants to be displayed to a real-space image that is displayed as visitors hold up portable terminals. The event management sever further causes a superimposed image 206 that represents information of sales objects correlated to the contestants and a GUI 208 that realizes commodity purchase to be displayed. When operation that indicates the GUI 208 is performed, the event management server performs order placement or payment to stores and causes a superimposed image 210 that indicates completion of purchase to be displayed by portable terminals.SELECTED DRAWING: Figure 3

Description

The present invention relates to an event management server, an event management system, and an event management method that provide services to visitors at an event venue such as a sports competition.

At event venues and tourist facilities, technologies that enhance convenience by providing various information via mobile terminals held by visitors have been put into practical use. For example, a technique has been proposed for notifying a visitor's smartphone of the availability of toilets in a stadium and the position of a seller who sells food and drink (see, for example, Patent Document 1). Further, a technique has been proposed in which auxiliary information about a player of interest is displayed by using a see-through display or the like while watching a baseball game (see, for example, Patent Document 2). On the other hand, a technique has been proposed in which a state of a game held at a stadium is photographed from a plurality of viewpoints and a video of a viewpoint corresponding to a request from a client is distributed (see, for example, Patent Document 3).

Japanese Unexamined Patent Publication No. 2018-1428080 Special Table 2019-531538 Gazette Japanese Unexamined Patent Publication No. 2019-68130

Many of the above services that can be used by event visitors using mobile terminals are relatively simple, such as unilaterally presenting additional information. Therefore, especially at events such as sports competitions and music festivals where the interest of the audience tends to concentrate on performance in the real space, there are few opportunities to dare to look at or operate the terminal, and the scenes where the service can be effectively used are limited. Was there.

The present invention has been made in view of these problems, and an object of the present invention is to provide a technique for easily using various services by an event visitor using a mobile terminal. Another object of the present invention is to provide a technique for activating an event by promoting the use of services by visitors.

One aspect of the present invention relates to an event management server system. In this event management server system, a personal information storage unit that stores personal information of a user who visits an event in association with the identification information of the terminal held by the user, and an event represented by the terminal are performed. An image data transmission unit that transmits data for displaying a superimposed image in association with an existing real space image to the terminal, a terminal information acquisition unit that acquires the content of a user operation for the superimposed image from the terminal, and a user operation. It is characterized by having an information processing unit that acts on behalf of the procedure for using the service by the operating user based on the contents and personal information of the above.

Another aspect of the invention relates to an event management system. This event management system includes a terminal held by a user who visits an event and an event management server system that provides services to the user via the terminal, and the event management server identifies the personal information of the user and the terminal. The personal information storage unit that stores information in association with each other and the data for displaying the superimposed image in association with the image of the real space where the event is being performed, which is represented by the terminal, are transmitted to the terminal. An image data transmission unit, a terminal information acquisition unit that acquires the content of user operation for the superimposed image from the terminal, and an information processing unit that takes over the procedure of service use by the operated user based on the content of user operation and personal information. , It is characterized by having.

Yet another aspect of the present invention relates to an event management method. In this event management method, the event management server system displays data for displaying a superimposed image in association with an image of the real space in which the event is being performed, which is represented by a terminal held by a user who visits the event. The step of sending to the terminal, the step of acquiring the content of the user operation for the superimposed image from the terminal, the personal information stored in the memory corresponding to the identification information of the terminal, and the operation by the user based on the content of the user operation. It is characterized by including a step of acting for the procedure of using the service.

It should be noted that any combination of the above components and the conversion of the expression of the present invention between a method, a device, a system, a computer program, a recording medium on which a computer program is recorded, and the like are also effective as aspects of the present invention. ..

According to the present invention, visitors to an event can easily use various services using a mobile terminal. In addition, the event can be activated by encouraging the use of services by visitors.

It is a figure which shows the structure of the event management system to which this embodiment is applied. It is a figure which illustrates the screen which the mobile terminal displays in this embodiment. It is a figure which illustrates the screen of the mobile terminal 12 when realizing the sales service using the superposed image in this embodiment. It is a figure which shows the internal circuit configuration of the event management server in this embodiment. It is a figure which shows the internal circuit structure of the mobile terminal in this embodiment. It is a figure which shows the structure of the functional block of the event management server in this embodiment. It is a figure which shows the structure of the functional block of the mobile terminal in this embodiment. It is a figure which illustrates the data structure of the personal information stored in the personal information storage part of the event management server in this embodiment. It is a figure which illustrates the data structure of the information stored in the related product information storage unit, the store information storage unit, and the advertisement information storage unit of the event management server in this embodiment. It is a figure which illustrates the screen for realizing the presentation from a visitor to a participant, which is displayed on a mobile terminal in this embodiment. It is a figure which illustrates the screen which shows the information of the other visitor who gave the present to the contestant, which is displayed on the mobile terminal in this embodiment. It is a figure which illustrates the transition of the screen which represented the information which concerns on the actual store, which is displayed on the mobile terminal in this embodiment. It is a figure which illustrates the screen which represented the advertisement, which is displayed on the mobile terminal in this embodiment. It is a flowchart which shows the operation of an event management server and a mobile terminal when a visitor enters an event venue in this embodiment. It is a flowchart which shows the operation of the event management server and the mobile terminal at the time of selling the product which is related to the real object in this embodiment. It is a flowchart which shows the operation of the event management server and the mobile terminal at the time of realizing the presenting of goods from a visitor to a participant in this embodiment. It is a flowchart which shows the operation of the event management server and the mobile terminal at the time of selling the product of the actual store in this embodiment. It is a flowchart which shows the operation of the event management server and the mobile terminal at the time of displaying an advertisement in this embodiment.

The present embodiment relates to a system that provides a service to visitors via a mobile terminal at an event venue. Here, an event may be one in which multiple people go to the venue to see or experience the event, for example, sports competitions, music festivals, concerts, shows, etc., the content, scale, and type of venue of the event. Not particularly limited. FIG. 1 shows a configuration of an event management system to which this embodiment can be applied. The event management system 1 includes an event management server 10, mobile terminals 12a to 12e, a virtual store management server 14, and a physical store management server 16.

The mobile terminals 12a to 12e are terminals carried by the visitors of the event, display various information related to the event, and accept user operations. In FIG. 1, mobile terminals 12a, 12b, and 12c are high-performance mobile phones (smartphones), and mobile terminals 12d and 12e are wearable terminals such as AR (Augmented Reality) glasses. As described above, the mobile terminals 12a to 12e are not limited in their form as long as they are general terminals provided with a display, an information processing device including a communication function, and an input device. Hereinafter, the mobile terminals 12a to 12e are simply referred to as "mobile terminals 12".

The event management server 10 establishes communication with the mobile terminal 12, receives the contents of the user operation, performs the corresponding information processing, and returns necessary information. In recent years, with the introduction of 5G (5th generation mobile communication system), it has become possible to transmit various data with low delay and large capacity even in a mobile terminal. Therefore, such a technique is appropriately used for communication between the mobile terminal 12 and the event management server 10.

The event management server 10 may actually be realized by a plurality of information processing devices and storage devices. For example, the event management server 10 may be distributed and arranged according to the function by MEC (Multi-access Edge Computing), which is one of the 5G technologies. In this case, functions that require high responsiveness, such as the function of transmitting image data to the mobile terminal 12, are realized by a server installed near the event venue, and other functions are realized by a server installed in another data center or the like. It is conceivable to do. Therefore, the "event management server" of the present embodiment can be read as "event management server system" regardless of the number of devices.

The virtual store management server 14 establishes communication with the event management server 10 and mediates the buying and selling between the virtual store on the network and the visitors. The physical store management server 16 establishes communication with the event management server 10 and mediates the buying and selling between the physical store inside and outside the event venue and the visitors. The physical store management server 16 further provides information on the real-time status of the physical store, such as the degree of congestion, waiting time, and product inventory, to the event management server 10 at any time.

In the present embodiment, the event management server 10 provides various services to visitors via the mobile terminal 12. The services provided here are broadly divided into selling products to visitors and supporting support for participants. In either case, in this embodiment, the service can be used at any opportunity by increasing the affinity with the main event itself and pursuing the convenience of operation.

Taking sports competitions as an example, spectators are often motivated to purchase products related to them as fans of teams and athletes. However, at the competition venue, there is not much opportunity to get information on products because they are absorbed in the game itself, and at a later date, they may regret knowing its existence. In this case, the event organizer will miss the opportunity. In the present embodiment, various information can be visually recognized along with the state of the competition via the mobile terminal 12, so that the spectators are aware of the service and can seamlessly shift to the service use.

Specifically, on the mobile terminal 12, the content of the service can be displayed at the same time as the event by superimposing and displaying an image showing information related to the service such as a sales target in correspondence with the image of the real space where the event such as a match is held. Make it visible. For example, by arranging a three-dimensional object in the real space and drawing the image as a superimposed image, fusion with the event space is achieved. In high-performance mobile phones and wearable terminals, a technique for superimposing an image of a virtual object on an image in real space is known as AR or MR (Mixed Reality).

FIG. 2 illustrates a screen displayed by the mobile terminal 12. In this example, the event is a soccer match, and visitors are watching a match on the field from their seat. When the visitor holds the mobile terminal 12 in that state, the mobile terminal 12 displays the state of the real space seen from the viewpoint, for example, as shown in (a).

This situation is realized, for example, by immediately displaying a real-time image taken by a camera mounted on the back of the mobile terminal 12 on a display. If the visitor changes the direction of the mobile terminal, the field of view of the image will naturally be linked to it. However, such a real-space image display is not limited to the one taken by the camera, and may be realized by using an optical system such as AR glass.

(B) shows a state in which a superimposed image is represented on the image in the real space shown in (a). In this example, the superimposed image 200 showing the uniform number and name of the participating player, the current position on the field, and the like, and the superimposed image 202 showing the face photograph and the battle record of the player selected from the superimposed image 200 are shown. The superimposed images 200 and 202 may be simple planar images, or may be a board-shaped object drawn based on a three-dimensional model, and may represent information necessary as a texture. In (b), a superimposed image 204 of the characters "GOAL!" Showing that the player has scored a goal is further displayed on the field.

By representing the superimposed images 200 and 202 so as to be fixed to a place other than the field in the real space, the state of the game is not hidden by the superimposed image regardless of the orientation of the mobile terminal 12. However, the purpose is not to limit the position representing the superimposed image to other than the field, and it may be appropriately set according to the display content and the like. Visitors can see information about players and the game situation at the same time as the game, so they can deepen their understanding of the game and enjoy watching the game more. The display / non-display of the superimposed image and the type of information to be displayed are switched by a user operation via a predetermined input means of the mobile terminal 12. Alternatively, the superimposed image may be displayed at all times while the visitor is holding the mobile terminal 12.

FIG. 3 illustrates the screen of the mobile terminal 12 when realizing a sales service using superimposed images. In (a), the superimposed image 202 showing the information of the individual player shown in FIG. 2B and the superimposed image 206 showing the information of the sales target associated with the player are shown. In this example, the superimposed image 206 shows an image of spikes of the same model worn by the athlete and additional information such as price. The target of sale is not particularly limited as long as it corresponds to the player, and may be the same items as those used by the player such as clothing, shoes, balls, towels, related goods of the player, and official goods of the team to which the player belongs.

In addition, the sales target is not limited to goods, but may be services such as participation rights for events to be held thereafter. Hereinafter, they may be simply referred to as "commodities". The image of the product preferably creates a situation as if it exists in a real space by drawing an object based on a three-dimensional model. Also, using MR technology, visitors can reach out to the displayed space to turn the object of the product or draw it toward themselves so that they can see the details. May be good.

In the screen example of (a), a GUI (Graphical User Interface) 208 that realizes the purchase of a product is further represented as a superimposed image. In the illustrated example, the GUI 208 is represented by a horizontally long three-dimensional object representing the characters "buy this". However, the expression of GUI is not limited to this. When the operation for instructing the GUI 208 is performed, the event management server 10 accepts the operation, and orders the corresponding product to the virtual store management server 14 or the actual store management server 16 that manages the seller, and executes the payment processing.

Here, the GUI instruction operation may be a tap operation on the screen, an instruction operation using a cursor, or the like, and a visitor reaches out to the displayed space to touch an object of the GUI or point to an object. It may be an operation. Alternatively, a predetermined gesture of the visitor may be used as an instruction operation, such as drawing a circle or a V shape with a finger in the displayed space.

When the processing of the order and the settlement is completed and the event management server 10 notifies to that effect, the mobile terminal 12 displays the superimposed image 210 indicating the completion of the purchase, as illustrated in (b). In this example, the superimposed image 210 is represented by a horizontally long three-dimensional object representing the characters "purchase completed". The products purchased in this way may be available at the venue or may be delivered at a later date, depending on the type and the place of purchase. In the former case, it may be delivered to the seat of the visitor, or the visitor may go to the store to pick it up.

As described above, in this embodiment, a mechanism is provided in which visitors can enjoy the service at any time while watching the state of the event. From this point of view, as shown in the figure, it is particularly important that necessary and sufficient information is presented in as small an area as possible, and that the desired service is executed with as little effort as possible. Therefore, the event management server 10 collects and utilizes various information related to each visitor, and optimizes the information presented for each mobile terminal 12. Further, by supplementing the personal information necessary for using the service on the event management server 10 side, the service can be used without the need for complicated input.

FIG. 4 shows the internal circuit configuration of the event management server 10. The event management server 10 includes a CPU (Central Processing Unit) 23, a GPU (Graphics Processing Unit) 24, and a main memory 26. Each of these parts is connected to each other via a bus 30. An input / output interface 28 is further connected to the bus 30. The input / output interface 28 receives data from a communication unit 32 that establishes communication with the mobile terminal 12, a storage unit 34 such as a hard disk drive or a non-volatile memory, an output unit 36 that outputs data to a display (not shown), and an input device (not shown). The input unit 38 for input, and the recording medium driving unit 40 for driving a removable recording medium such as a magnetic disk, an optical disk, or a semiconductor memory are connected.

The CPU 23 controls the entire event management server 10 by executing the operating system stored in the storage unit 34. The CPU 23 also executes various programs read from the removable recording medium and loaded into the main memory 26. The GPU 24 performs drawing processing according to a drawing command from the CPU 23 and outputs the drawing process to the output unit 36. The main memory 26 is composed of a RAM (Random Access Memory) and stores programs and data necessary for processing.

FIG. 5 shows the internal circuit configuration of the mobile terminal 12. The mobile terminal 12 includes a CPU 50, a main memory 52, a display unit 54, an audio output unit 56, a communication unit 60, a motion sensor 62, a camera 64, and a position acquisition unit 66. Each of these parts is connected to each other via a bus 58. The CPU 50 processes the information acquired from each unit of the mobile terminal 12 via the bus 58, and supplies the output data to the display unit 54 and the voice output unit 56. The main memory 52 stores programs and data necessary for processing in the CPU 50.

The display unit 54 is composed of a display panel such as a liquid crystal panel or an organic EL panel and a drive mechanism thereof, and displays an image as illustrated in FIGS. 2 and 3. The voice output unit 56 is composed of a speaker, earphones, and a control mechanism thereof, and allows the user to hear voice as needed. The communication unit 60 establishes communication with the event management server 10 and transmits / receives necessary data. In this embodiment, mobile communication of the mobile terminal 12 is realized by, for example, 5G communication technology.

The motion sensor 62 is composed of at least one of an acceleration sensor, a gyro sensor, a geomagnetic sensor, and the like, and measures the posture and movement of the mobile terminal 12 in a three-dimensional space. The camera 64 captures a moving image of the real space in the direction opposite to the screen of the display unit 54. In the present embodiment, in particular, the image is immediately displayed on the display unit 54 so that the state of the real space can be seen in real time. The image captured by the camera 64 is also used when determining the position on the screen on which the superimposed image is drawn according to the change in the visual field. The position acquisition unit 66 acquires the position information of its own device at any time by using well-known techniques such as GPS (Global Positioning System) and A-GPS (Assisted-GPS).

FIG. 6 shows the configuration of the functional block of the event management server 10 in the present embodiment. Each functional block shown in FIG. 6 and FIG. 7 to be described later can be realized by an electronic component such as a CPU or a main memory shown in FIG. 4 or FIG. 5 in terms of hardware, and mainly from a recording medium or the like in terms of software. It is realized by a program loaded in memory that exerts various functions such as data input function, data retention function, image processing function, and communication function. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof, and the present invention is not limited to any of them.

The event management server 10 carries a terminal information acquisition unit 70 that acquires necessary information such as user operation contents from the mobile terminal 12, an information processing unit 72 that performs information processing related to the provided service, and data related to an image to be displayed. It includes an image data transmission unit 74 to be transmitted to the terminal 12, an information storage unit 76 for storing information necessary for providing services, and a store communication unit 78 for communicating with the virtual store management server 14 and the actual store management server 16.

The terminal information acquisition unit 70 acquires the position information of the mobile terminal 12 and the contents of the operation by the visitor together with the identification information of the mobile terminal 12. Here, the contents of operations by the visitors include selection of participants, purchase of products, selection of stores, selection of advertisements, and the like. In the present embodiment, as described above, the superimposed image is used as a GUI that accepts these operations.

The information processing unit 72 performs information processing corresponding to the visitor who owns the mobile terminal 12 of the communication destination. That is, the information processing unit 72 optimizes the services to be provided and the information to be presented by using personal information such as attributes, preferences, payment means, and seat numbers of each visitor. In addition, the information processing unit 72 takes the opportunity of a simple operation of the visitor to the superimposed image, supplements the information necessary for using the service, and then places an order or makes a payment on behalf of the procedure for using the service. Specifically, the information processing unit 72 includes a related product sales unit 80, a presentation support unit 84, and an actual store management unit 86.

The related product sales unit 80 controls the sales of products corresponding to the participants of the event, as in the spike shown in FIG. For example, the related product sales unit 80 determines the products to be sold according to the tastes and attributes of each mobile terminal 12 and, by extension, each visitor. The price may also be optimized. When the product purchase operation is performed on the mobile terminal 12, the related product sales unit 80 executes the order processing and the settlement processing. Conventional technology can be applied to the mechanism of electronic commerce using payment methods such as credit cards and electronic money.

In this embodiment, it is also possible to use the "points" held by the visitors for payment. Here, the "point" is a substitute for money given by a store or a company as a service according to a past purchase history or the like. In this case, the event management server 10 also collects the number of points held by the visitors as personal information. Then, the related product sales department 80 carries out a procedure of deducting the possessed points at the same time as placing an order in cooperation with a company or the like that has given the points according to the purchase operation of the product. When a visitor purchases a product at an event, points may be given on the contrary. Regarding the ordering and settlement of products, the operations of the gifting support unit 84 and the physical store management unit 86 are the same.

The presentation support unit 84 supports the presentation of goods from the visitors to the event participants. Specifically, the presentation support unit 84 presents a gift candidate to the contestants, and when an operation indicating the intention to give the gift is performed on the mobile terminal 12, the delivery destination is set to the contestant, and the order processing and settlement processing are performed. To carry out. Here, the gifts are bouquets, garlands, foods and drinks, etc., and the gift support department 84 determines the products handled by the sponsoring company or the gift shop near the venue as candidates. The sender of the gift may be an individual visitor or the entire visitor who agrees.

For example, the presentation support unit 84 may determine the success of the gift when the number of visitors who have indicated the intention of the gift or the total value of the gift costs reaches the set standard. In this case, the presentation support unit 84 does not immediately place an order or settle the presentation operation on the mobile terminal 12, but accumulates it as a request. Then, when the predetermined conditions for the presentation are satisfied, the order and settlement processing are carried out for the accumulated requests.

Participants in sports competitions and concerts can improve their performance by supporting the spectators. On the other hand, the "new lifestyle" that accompanies the recent epidemic of infectious diseases may limit loud cheering. Even in such a case, by expressing the intention of support by presenting goods, the visitors can contribute to the improvement of the motivation of the participants and the excitement of the event. Visitors can also enjoy themselves with the visitor-participation-type content that unites with other visitors to complete the presentation. Therefore, the presentation support unit 84 may allow the visitor to confirm the number of people who have operated the gift purchase operation and the progress of the total value of the purchase price due to the operation.

The physical store management unit 86 manages the situation of physical stores in or near the venue and relays sales. Specifically, the actual store management unit 86 acquires store status information such as congestion, product inventory, and waiting time at any time, and causes the mobile terminal 12 to present the latest information. The physical store management unit 86 may also dynamically determine the price of the product based on at least one of the store status information and the personal information of the visitor.

For example, the physical store management unit 86 sets a discount price for a product to be sold when the store congestion is lower than the predetermined standard (the order queue is shorter than the predetermined standard). Alternatively, a discount price may be set when the member rank of the visitor operating the mobile terminal 12 is higher than the standard. Here, the "member rank" is a class of customers that is determined based on, for example, the cumulative value of the purchase price of products at related companies, and that the higher the rank, the more preferential treatment is received. The physical store management unit 86 may set the price of the product in multiple stages according to those indicators.

When the operation of purchasing the product is performed on the mobile terminal 12, the physical store management unit 86 carries out the order processing and the settlement processing. If the purchase operation is performed without specifying the store by operating the advertisement image, or if the target product is sold at a chain store having a plurality of target products, the actual store management unit 86 is transmitted from the mobile terminal 12. Select the nearest store as the supplier based on at least one of the location information and the seat number of the visitor. After placing an order, the actual store management unit 86 acquires the order number, waiting time, and the like from the store of the ordering party and displays them on the mobile terminal 12 of the purchasing source.

The image data transmission unit 74 determines the data related to the image to be superimposed on the image in the real space in the mobile terminal 12, and transmits the data to the mobile terminal 12. Here, the data transmitted by the image data transmission unit 74 may be the image itself, or a part necessary for the mobile terminal 12 to interpret and draw the image, such as the arrangement of the object in the real space and the character information represented therein. Data may be used. For example, the image data transmission unit 74 acquires the position information of the mobile terminal 12 and transmits the position information of the object to be arranged within the range of the real space that may be visible from the position. In the case of the screen example of FIG. 3, the image data transmission unit 74 is in the range visible from the mobile terminal 12 (or the audience seats of the visitors) in the three-dimensional space of the soccer stadium (stadium), and is the source of the superimposed image 206 or the like. A three-dimensional object is arranged, and its position information is transmitted to the mobile terminal 12.

The mobile terminal 12 arranges an object in a world coordinate system representing a real space based on a three-dimensional model held internally and transmitted position information, and then projects an image on a screen coordinate system composed of a display screen. As a result, even if the posture of the mobile terminal 12 changes, the object can be represented in a state of being fixed in the real space. Alternatively, the event management server 10 may generate an image with a region other than the superimposed image transparent and then transmit the image to the mobile terminal 12. In this case, the mobile terminal 12 only needs to superimpose the transmitted image on the image in the real space.

It is understood by those skilled in the art that various things can be considered regarding the division of data and processing held by the event management server 10 and the mobile terminal 12. Therefore, in the following description, the "data related to the image" may be the data of the image itself or the information necessary for drawing the superimposed image. The image data transmission unit 74 includes a basic image control unit 91, a related product image control unit 92, a gift image control unit 94, an actual store image control unit 96, and an advertisement image control unit 98.

The basic image control unit 91 generates data related to a superimposed image (hereinafter, may be referred to as a basic image) representing information related to the event itself, and transmits the data to the mobile terminal 12. The information related to the event itself is supplementary information directly related to the real space, such as the information represented by the superimposed images 200, 202, and 204 shown in FIG. 2 (b). Therefore, the basic image control unit 91 holds a database related to events such as participants. Further, the basic image control unit 91 continuously acquires the event status by the analysis result of the image taken by the camera (not shown), the manual input of the server administrator, etc., and the result is reflected in the superimposed image. , Generate necessary data and send it to the mobile terminal 12 at any time.

The related product image control unit 92 transmits data related to the image of the product associated with the real object existing in the event, which is determined by the related product sales unit 80 of the information processing unit 72, to the mobile terminal 12. The GUI for purchasing the product and the image data indicating the completion of the order may be held by the mobile terminal 12, or may be transmitted by the related product image control unit 92 each time. In the former case, the related product image control unit 92 notifies the mobile terminal 12 of the timing of their display.

The gift image control unit 94 transmits data related to the image of the gift candidate determined by the gift support unit 84 of the information processing unit 72 to the mobile terminal 12. The gift image control unit 94 transmits data related to the gift to all the mobile terminals 12 at a predetermined timing in the progress of the event, for example, before and after the game or half-time. The gift image control unit 94 may also transmit to the mobile terminal 12 data relating to an image showing the progress of the process, such as the number of people who have indicated their intention to give the gift and the total value of the gift costs.

The gift image control unit 94 may further transmit data relating to an image pointing to another visitor who agrees with the gift to the mobile terminal 12. For example, in the audience seats in the field of view of the mobile terminal 12, which visitors agree with the presentation is represented by figures or characters so that the contestants and the gifts to be presented can be understood. In this case, the gift image control unit 94 transmits the seat number of the supporter in association with the information of the contestants at the gift destination and the gift.

The actual store image control unit 96 transmits data related to an image representing the state information of the actual store acquired from the actual store management unit 86 of the information processing unit 72 to the mobile terminal 12. Since the state information of the actual store changes from moment to moment, the actual store image control unit 96 transmits the updated data at any time. Further, the actual store image control unit 96 transmits data related to a menu image representing a product sold at the actual store in response to an operation for selecting the actual store performed on the mobile terminal 12.

Here, the actual store image control unit 96 includes the price determined by the actual store management unit 86 of the information processing unit 72 in the menu screen. When the price is changed depending on how crowded the store is as described above, the actual store image control unit 96 may notify the visitor by transmitting the information to the mobile terminal 12 at the timing when the price drops. As a result, orders are promoted during the time when the store is vacant, and congestion is easily alleviated.

The advertisement image control unit 98 transmits data related to an image representing an advertisement to the mobile terminal 12. Here, the advertisement image control unit 98 may change the advertisement depending on the destination mobile terminal 12. For example, the advertisement image control unit 98 classifies the preference of the visitor based on at least one of the purchase history of the visitor who operates the mobile terminal 12 and the browsing history of the website up to that point. Then, the advertisement associated in advance as suitable for the preference classification is determined as the display target. Further, the advertisement image control unit 98 may extract a product purchased at an event by another visitor having the same type of preference as a recommendation, and transmit data related to the image of the product in the same manner as the advertisement.

The information storage unit 76 stores information necessary for the information processing unit 72 and the image data transmission unit 74 to perform the above-mentioned processing. In detail, the information storage unit 76 includes a personal information storage unit 100, a related product information storage unit 102, a store information storage unit 104, and an advertisement information storage unit 106. The personal information storage unit 100 stores the personal information of the visitor and the identification information of the terminal to be held in association with each other. Here, personal information includes both event-specific information and information obtained regardless of the event. Specific examples will be described later.

The related product information storage unit 102 stores the actual object existing in the event such as a contestant and the information of the related product in association with each other. It is desirable that the product information includes image data such as a three-dimensional model that can recognize features at a glance. The related product information storage unit 102 also stores information on gift candidates for the contestants. The store information storage unit 104 stores information on virtual stores and actual stores, which are the sellers of products, and information on products for sale.

The advertisement information storage unit 106 stores image data of advertisements of sponsor companies and the like. Here, as described above, the advertisement may be associated with the classification of human taste. In addition, by including an advertisement indicating a product sold in a physical store in the venue or in the vicinity, the product may be purchased by an operation of instructing the advertisement. On the other hand, for example, a conventional ad network technology may be introduced, and an advertisement submitted for general purposes regardless of an event may be superimposed and displayed on each mobile terminal 12.

The store communication unit 78 establishes communication with the virtual store management server 14 and the actual store management server 16, and relays the order processing and the settlement processing by the information processing unit 72. Further, the store communication unit 78 establishes communication with the actual store management server 16, periodically acquires the state information of each store, and provides it to the information processing unit 72. The status information may be manually input by an employee on the store side, or the actual store management server 16 may automatically acquire an order queue or the like. You may use POS (Point of sale system).

FIG. 7 shows the configuration of the functional block of the mobile terminal 12 in the present embodiment. All or part of the illustrated functions may be configured by downloading from the distribution server that distributes the application by user selection or autonomous request control of the mobile terminal 12, or may be installed at the time of shipment of the mobile terminal 12. It may be configured by doing so. Further, the mobile terminal 12 may be composed of a terminal in which an application such as a high-performance mobile phone is installed and a wearable display such as an AR glass for displaying an image. In this case, both transmit and receive data by short-range wireless communication such as Bluetooth (registered trademark). Alternatively, both may be connected by wire.

The mobile terminal 12 has an operation reception unit 110 that accepts operations by visitors, a terminal information transmission unit 112 that transmits operation details and location information to the event management server 10, and state information that acquires information related to the state of the mobile terminal 12 itself. It includes an acquisition unit 114, an image data reception unit 116 that receives data related to an image from the event management server 10, and a display unit 118 that displays a real-space image or a superimposed image.

The mobile terminal 12 further includes a personal information storage unit 120 that stores the personal information of the visitor to be operated, and an image data storage unit 122 that stores data related to the image to be displayed. The operation reception unit 110 accepts operations by visitors. As described above, various possible means of operation include tapping the screen, operating on an object represented by a superimposed image, and making a gesture in the field of view.

The terminal information transmission unit 112 transmits the content of the operation by the visitor and the position information of the mobile terminal 12 to the event management server 10 together with the identification information of the mobile terminal 12. Therefore, the personal information storage unit 120 stores at least the identification information of the mobile terminal 12. In addition, the personal information storage unit 120 may store personal information recorded on a general terminal such as a user's name, telephone number, login ID, and password.

The state information acquisition unit 114 includes the position acquisition unit 66 and the motion sensor 62 of FIG. 5, and acquires the position, posture, and movement of the mobile terminal 12 as state information at any time. By transmitting the information to the event management server 10 as needed, it is possible to efficiently receive the data related to the superimposed image to be displayed within the field of view of the mobile terminal 12. The image data receiving unit 116 receives data related to the superimposed image from the event management server 10.

The display unit 118 displays an image of the real space in which the event is being performed via a camera or an optical system, and superimposes and displays an image based on the data received by the image data reception unit 116. For example, as described above, the display unit 118 projects a three-dimensional model of an object to be displayed as a superimposed image onto the screen coordinate system corresponding to the position and orientation of the mobile terminal 12 acquired by the state information acquisition unit 114. Draw a superimposed image.

Further, the display unit 118 may move or deform the object so that the object represented as the superimposed image virtually interacts with the movement of a real object such as a visitor's body or a tool. For example, allow visitors to reach out and draw objects in the real space of the field of view. Here, the technology that has been put into practical use as MR can be adopted.

The image data storage unit 122 stores a part of the data necessary for drawing. As described above, the data stored in the image data storage unit 122 may vary depending on the division of processing between the mobile terminal 12 and the event management server 10. The superimposed image displayed by the display unit 118 is not limited to being fixed in the real space. For example, depending on the type and display timing of the display target such as the state information of the actual store, the product menu, and the gift candidate, the display may be performed at a predetermined position on the screen.

FIG. 8 illustrates a data structure of personal information stored in the personal information storage unit 100 of the event management server 10. The personal information 130 is information in which the basic information 132 that can be acquired regardless of the event and the information 134 about the event are associated with the "user ID" that is the identification information of the user who is a visitor. In this example, the basic information 132 includes terminal ID, password, name, gender, age, address, payment information such as credit card number and electronic money number, number of points held, member rank, product purchase history up to that point, and website. Includes browsing history and preference classification.

The basic information 132 is, for example, read information registered or collected in a system (not shown) such as a browser or an EC (Electronic Commerce site) based on a common user ID. The event management server 10 basically identifies the user ID of the visitor based on the terminal ID transmitted from the mobile terminal 12 together with the operation content and the like.

On the other hand, in this example, the information 134 about the event includes the seat number and the block number of the ticket, the information accompanying the ticket purchase, and the purchase history at the event venue. Here, the information accompanying the ticket purchase is information such as the number of purchased tickets and whether or not the child fee ticket is included. These information are, for example, read out sales information recorded in a system for selling tickets (not shown) based on a common user ID.

The personal information 130 can be used by the related product sales unit 80, the presentation support unit 84, and the advertisement image control unit 98 to optimize the information to be presented. For example, products that are likely to be liked are sorted according to the gender, age, and preference classification of the visitors who operate the mobile terminal 12, and the display priority is determined. Products in the optimal price range may be filtered based on the seat rank. Furthermore, if children's ticket is purchased at the same time, it is estimated that parents and children will come to the venue, so the products that children like may be displayed preferentially. In this way, by estimating the attributes of the companion based on the information that can be collected, the accuracy of optimizing the presented information can be improved.

As described above, the preference classification can also be used by the advertisement image control unit 98 to extract another visitor having the same preference at the event and determine the product purchased by that person at the venue as a recommendation. The member rank can be used by the physical store management department 86 to determine the price of the product. The seat number can be used by the physical store management department 86 to identify a physical store close to the visitor who purchased the product.

The seat number can also be used as auxiliary information for acquiring the position of the mobile terminal 12 when the image data transmission unit 74 determines the arrangement of objects and the data to be transmitted. That is, by integrating the position information transmitted from the mobile terminal 12 and the seat position information, the accuracy of specifying the position of the mobile terminal 12 can be improved. The seat position can also be used by the gift image control unit 94 to display an image pointing to another visitor who has agreed to the gift to the contestants.

FIG. 9 illustrates a data structure of information stored by the related product information storage unit 102, the store information storage unit 104, and the advertisement information storage unit 106 of the event management server 10. As shown in (a), the data stored in the related product information storage unit 102 is the actual object of the event, for example, the "participant ID" which is the identification information given to each contestant, and the product (product A, It is the information associated with the information of the product B, the product C, ...).

Information on each product includes image data representing the product, product name, price, identification information of the store of the seller, and the like. Further, by giving priority to products according to a person's preference classification or attribute, it is possible to determine a sales target or optimize a display order depending on the data transmission destination mobile terminal 12. The products associated with this product include those obtained by the visitors themselves and gifts to the participants.

As shown in (b), the data stored in the store information storage unit 104 corresponds to the management server corresponding to the "store ID", which is the identification information assigned to each store, among the virtual store management server 14 or the actual store management server 16. URL, image data representing the store, information on the products to be sold (product a, product b, product c, ...), in the case of an actual store, the location information, discount rules, etc. .. Information on each product includes image data representing the product, product name, price, and the like. The product associated with the contestant ID in (a) and the product associated with the store ID in (b) may be duplicated.

As shown in (c), the data stored in the advertisement information storage unit 106 is an advertisement (advertisement) suitable for the visitor of the preference in the "preference classification ID" which is the identification information given for each classification of the preference of the visitor. It is the information associated with the information of I, advertisement II, advertisement III ...). The information of each advertisement includes image data representing the advertisement and information of the store selling the product represented by the image data. The product displayed in the advertisement, the product associated with the contestant ID in (a), and the product associated with the store ID in (b) may be duplicated. The data structures shown in FIGS. 8 and 9 are merely examples, and the information and data formats to be held may be various.

FIG. 10 illustrates a screen for realizing presentation from a visitor to a participant, which is displayed on the mobile terminal 12 in collaboration with the presentation support unit 84 and the gift image control unit 94. In this example, the screen 140 superimposes an image showing information related to the gift on the image of the soccer stadium, which is the same real space as shown in FIGS. 2 and 3. Specifically, it represents a superposed image 142 composed of text information prompting the presentation to "Yamada player", superposed images 144a and 144b showing candidates for gifts, and GUI 146a and 146b accepting the intention of gifting.

The screen 140 assumes a case where a visitor-participation-type content in which fans work together to establish a gift of goods is provided during a period such as half-time when a match is not held. This allows the field to be hidden by the superimposed image. On the other hand, during a match, it is desirable to place the object at a position where the field is not hidden as shown in FIG.

In the illustrated example, a gift to a specific contestant named "Yamada" is accepted. In this case, as a pre-stage process, the operation reception unit 110 of the mobile terminal 12 may accept a user operation for selecting a contestant to be presented. Alternatively, in the case of an event such as figure skating, gymnastics, or a music festival in which participants appear in order, the recipients can be naturally limited by displaying the screen 140 at the timing of appearance or exit.

In FIG. 10, a bouquet is shown in the superimposed image 144a, and a sports drink is shown in the superimposed image 144b as a candidate for a gift. These candidates may be changed depending on the attributes of the visitors who operate the mobile terminal 12. Below the superimposed images 144a and 144b of each candidate, GUI 146a and 146b for showing the intention to present them are displayed. GUI 146a and 146b also show the prices of each gift candidate.

In this example, the points held by the visitors are used for payment, and it is shown that the bouquet is "500P (points)" and the sports drink is "1000P (points)". When a visitor who sees the screen 140 wishes to make a presentation, he / she indicates his / her intention by instructing and operating any one of GUI 146a and 146b. The presentation support unit 84 of the event management server 10 totals the totals, and determines that the presentation is successful when the predetermined conditions are satisfied. For example, when the number of visitors who have indicated their intention to present a bouquet reaches X, or when the total presentation cost (points) reaches Y points, a predetermined amount of bouquet is presented to the relevant contestants.

The progress and results until the presentation is completed may be displayed not only on the individual mobile terminal 12 but also on a large screen in the venue so that it can be shared by all the participants and visitors. The presentation support department 84 places an order for the gift to the store after setting the delivery address to the relevant contestant. The store of the ordering party preferably delivers the ordered gift to a place close to the contestants during the event, such as around the field or stage, waiting room, or flower path. As a result, the contestants can recognize the gifts on the spot and increase their motivation.

FIG. 11 illustrates a screen showing information of other visitors who have given a gift to the contestants, which is displayed on the mobile terminal 12 in collaboration with the gift support unit 84 and the gift image control unit 94. In this example, the screen 150 superimposes an image pointing to a supporter on an image of the audience seats of a soccer stadium, which is the same real space as shown in FIGS. Here, the supporter is a visitor who has performed an operation of instructing any of GUI 146a and 146b that accepts the intention of presenting on the screen 140 shown in FIG. Of these, only the supporters who contributed to the establishment at the time when the presentation is established may be pointed out, or all the supporters may be pointed out regardless of whether the presentation is established or not.

In this example, the audience seats on the opposite side of the field are displayed for the visitors who are looking at the screen 150. Among the visitors in the field of view, the balloon-shaped superimposed images 152a, 152b, and 152d represented as "flowers" are displayed for the supporters who present the bouquet. For the supporters who present sports drinks, a balloon-shaped superimposed image 152c represented as "drink" is displayed.

That is, in the illustrated example, only the type of the gift is shown in the superimposed images 152a, 152b, 152c, 152d, but information for identifying the contestant of the gift destination may be shown, and the contestant and the gift are shown. The combination of may be shown. In addition, the format of the superimposed image is not limited to the one shown in the figure as long as the position of the supporter, the recipient, and the gift can be determined. For example, in FIG. 11, the number of superimposed images is reduced for the sake of clarity, but the recipients and gifts may be distinguished by colors and figures so that they can be easily visually recognized when there are many supporters.

When a visitor who operates the mobile terminal 12 performs an operation of instructing any of the superimposed images 152a, 152b, 152c, and 152d while the screen 150 is displayed, the designated superimposed image points to the mobile phone of the visitor. The terminal 12 may be notified that such an operation has been performed. In this case, the gift image control unit 94 identifies the mobile terminal 12 to be notified according to the instruction operation, and notifies other visitors that the instruction has been given. As a result, it is possible to realize communication between visitors, as in the case of inputting a so-called "like (thumbs up)" indicating a positive intention in SNS.

FIG. 12 illustrates a transition of a screen showing information related to an actual store displayed on the mobile terminal 12 in collaboration with the store communication unit 78, the actual store management unit 86, and the actual store image control unit 96. .. In this example, an image showing various information related to the actual store is superimposed on the image of the soccer stadium, which is the same real space as shown in FIGS. 2 and 3. First, (a) shows a superimposed image 162 showing the degree of congestion as a percentage as the state information of the actual store.

FIG. 12 shows that the crowdedness of one physical store is “50%”. This actual store may be one registered in advance by the visitor, or may be selected by the visitor from the list of actual stores as the operation in the previous stage. However, the number of physical stores that display status information is not limited. In particular, when information on a plurality of actual stores is shown, an image representing the store is shown in the superimposed image 162 so that the state information of which store can be distinguished.

Further, in FIG. 12, a superimposed image 160 showing how crowded the toilet is is also displayed in parallel. These superimposed images 160 and 162 are basically always displayed during the event. Alternatively, the operation reception unit 110 of the mobile terminal 12 may accept the display / non-display operation of the superimposed images 160 and 162 by the visitors. The event management server 10 periodically acquires the state information of the store or the toilet and reflects it in the numerical values in the superimposed images 160 and 162.

When the operation of instructing the superimposed image 162 is performed on the screen of (a), the mobile terminal 12 displays the superimposed image 164 showing the product sold at the actual store and its price, as shown in (b). In the illustrated example, a menu representing light meals such as hamburgers and pasta is displayed. As described above, the price represented here may be dynamically determined according to the state of the store, the member rank, and the like. When the operation of instructing the image of any of the products of the superimposed image 164 is performed on the screen of (b), the event management server 10 orders the instructed product from the actual store.

When the order and settlement are normally completed, the event management server 10 transmits to that effect to the mobile terminal 12 together with necessary information. As a result, as shown in (c), the mobile terminal 12 displays the superimposed image 166 showing the purchase result. In the illustrated example, the order number "A10" is displayed along with the text information "Thank you for your order!" And "It will be completed around 14:30".

Visitors can save the screen as a screenshot and use it as a voucher for the product. When a visitor goes to a store and receives a product, the superimposed image 166 may indicate the location or route of the store. The order number shown in the superimposed image 166 may be represented by a code or graphic that can be read by a sensor in the store, such as a two-dimensional code, or may be represented by voice.

Further, if the visitor recognizes the store status information, the menu image, the order completion notification and the order number, the arrangement and format of the superimposed image are not limited to those shown in the figure. Although the superimposed image 164 and 166 allow the field to be concealed in FIG. 12, the object may be arranged at a position where the field is not concealed as shown in FIG.

FIG. 13 illustrates a screen showing an advertisement displayed on the mobile terminal 12 by the advertisement image control unit 98. In this example, the screen 170 superimposes an image representing an advertisement on an image of a soccer stadium, which is a real space similar to that shown in FIGS. 2 and 3. The superimposed image 200 is a basic image also represented on the screen of FIG. 2B. Further, on the screen 170, the superimposed images 172a and 172b representing the advertisement are arranged at positions where the field is not concealed, and the superimposed image 176 representing another advertisement is arranged on the field. The superimposed images 172a and 172b may be still images or moving images.

The object representing the advertisement may be fixed in the real space or may move according to a predetermined rule. Also, the displayed advertisement may be replaced with another advertisement on a regular basis. As described above, the display order of the advertisement is determined by the preference classification and attributes of the visitors who operate the mobile terminal 12. When the visitor performs an operation of instructing one of the advertisements, the product shown in the advertisement may be made available for purchase in the same manner as shown in FIG.

When the order and settlement are normally completed, the event management server 10 transmits to that effect to the mobile terminal 12 together with necessary information. As a result, the mobile terminal 12 displays a superimposed image showing the purchase result, as in the case of FIG. 12 (c). The superimposed images 172a and 172b may show images of products purchased by another visitor having the same taste at the event. Also in this case, when the visitor performs an operation of instructing the product, the event management server 10 places an order for the product and causes the mobile terminal 12 to display a superimposed image showing the purchase result.

In addition to displaying the image of the advertisement in an superimposed manner, the advertisement image control unit 98 may provide content using AR or MR as a promotion, which may lead to product sales or gift presentation. For example, during the intermission such as half-time, a show including virtual objects (music live, dance, throwing a sign ball to the audience, etc.) will be held. In this case, the company planning the event sells the right to hold the show to the advertiser in advance. During the show, the advertisement image control unit 98 transmits data such as a three-dimensional model of a virtual object such as a mascot to appear and various production tools, its arrangement, and movement to the mobile terminal 12.

At this time, the advertisement image control unit 98 causes the mobile terminal 12 to display an image of the product sold by the advertiser and a GUI for purchasing the product in the same manner as in FIG. As a result, visitors can purchase products with simple operations while enjoying the show, and it is possible to realize effective sales promotion for advertisers. The content for promotion is not limited to the show, and a person or a virtual object may explain or appeal the product itself for sale.

Further, the advertisement image control unit 98 may realize a game in which visitors can participate. For example, the advertisement image control unit 98 superimposes and displays a virtual object representing a tool for shooting a target placed on the field on the mobile terminal 12. Visitors participate in the targeting game by manipulating the virtual object. The terminal information acquisition unit 70 of the event management server 10 acquires the operation status of the mobile terminal 12 and determines the result of the game. The advertisement image control unit 98 determines a prize according to the result of the game for each mobile terminal 12, and displays the information superimposed on the mobile terminal 12. Here, the prize is an advertiser's product or the like, and the event management server 10 associates it with the visitor and stores it so that the visitor can receive it after the event is over.

Next, the operations of the event management server 10 and the mobile terminal 12 that can be realized by the above configuration will be described. FIG. 14 is a flowchart showing the operation of the event management server 10 and the mobile terminal 12 when a visitor enters the event venue. As a premise, the visitor has installed an application (hereinafter referred to as a service image display application) for realizing the present embodiment on the mobile terminal 12 owned by the visitor. The service image display application is made available only to visitors who have agreed to distribute product information and analyze and use the location information of the mobile terminal 12.

By registering the use of the service, at least a part of the personal information 130 shown in FIG. 8 is stored in association with the personal information storage unit 100 of the event management server 10. Furthermore, the visitor purchases an electronic ticket for participating in the event from the service image display application or the ticket sales site (not shown) in association with the common user ID. The information of the purchased ticket is stored in the mobile terminal 12.

The event management server 10 permits data transmission related to the service on condition that the ticket information is stored in the mobile terminal 12. At this stage, the event management server 10 can store the information 134 related to the event in the personal information 130 of FIG. 8 in the personal information storage unit 100. As for the information stored in each of the related product information storage unit 102, the store information storage unit 104, and the advertisement information storage unit 106 shown in FIG. 9, the information obtained at the planning stage of the event is manually or automatically collected.

When the service image display application is activated on the day of the event, the mobile terminal 12 accepts a login from a visitor (S10). When the visitor inputs login information such as a login ID and a password, the mobile terminal 12 transmits the information to the event management server 10. In response to this, the event management server 10 performs a login authentication process and starts acquiring the location information of the mobile terminal 12 (S12). The location information may be acquired at the required timing.

When the event management server 10 notifies the user to approve the login, the mobile terminal 12 performs a process necessary for event ticket authentication in response to an operation by a visitor or the like (S14). For example, the mobile terminal 12 displays the two-dimensional code of the ticket, and the sensor installed at the entrance gate reads it to realize the authentication process. However, any of the practical authentication technologies such as short-range communication and other non-contact authentication may be adopted. The mobile terminal 12 that has received ticket authentication, or the authenticated system, notifies the event management server 10 to that effect.

At this stage, the seat number of the visitor or the like may be transmitted from the mobile terminal 12 to the event management server 10. Then, the event management server 10 performs preprocessing necessary for starting the service provision to the mobile terminal 12, for example, checking whether normal operation is possible, and returns the result to the mobile terminal 12 (S16). When normal operation is enabled, the mobile terminal 12 shifts to a mode in which images related to various services provided by the event management server 10 are superimposed on an image in real space. After that, the mobile terminal 12 continues the display mode until it is detected that the visitor has left.

FIG. 15 is a flowchart showing the operation of the event management server 10 and the mobile terminal 12 when selling a product or the like related to a real object. This process is performed in a mode in which the image related to the service, which has been transferred through the process shown in FIG. 14, is superimposed on the image in the real space. In addition, the actual object to which the product is associated is considered to be a participant in the event. First, when a visitor arrives at a seat and holds the mobile terminal 12 over the real space where the event is held, the mobile terminal 12 displays the image of the real space and the position information of the mobile terminal 12 itself. Is transmitted to the event management server 10 (S20).

Then, the image data transmission unit 74 of the event management server 10 reads out basic image data representing information related to the event itself, such as participant information, commentary, and battle situation, from the storage device and transmits it to the mobile terminal 12 (S22). .. At this time, the event management server 10 specifies that the object is arranged at an appropriate position in the real space that can be seen from the position information of the mobile terminal 12. The appropriate position is, for example, as shown in FIG. 2 (b), an opposite audience seat where the state of the event itself is not concealed.

The mobile terminal 12 superimposes and displays a basic image using the data transmitted from the event management server 10 (S24). Then, the mobile terminal 12 accepts a selection operation for instructing an image representing a contestant among the basic images, and notifies the event management server 10 of the ID of the selected contestant (S26). In response to this, the related product sales unit 80 of the event management server 10 confirms the personal information based on the user ID of the visitor of the sender, and determines whether or not the related product can be sold (S28). For example, if an electronic payment method is not registered, it is determined that the product cannot be sold.

If it is determined that the product can be sold, the related product sales unit 80 determines the product to be presented and the order of presentation among the products related to the selected contestants based on the attribute and preference classification associated with the user ID. Then, the related product image control unit 92 transmits the data related to the determined product image to the mobile terminal 12. If it is determined that the product cannot be sold, the related product image control unit 92 notifies the mobile terminal 12 to that effect and displays it. Upon receiving the data related to the image of the related product, the display unit 118 of the mobile terminal 12 superimposes and displays the image (S30).

The mobile terminal 12 further accepts a purchase operation from a visitor by superimposing and displaying a GUI for purchasing a product. When the purchase operation is performed, the terminal information transmission unit 112 of the mobile terminal 12 transmits the operation content to the event management server 10 (S32). However, as described above, the purchase operation is not limited to the one via the GUI, and may be based on the user's gesture or the like. Upon receiving the content of the purchase operation, the related product sales unit 80 of the event management server 10 accesses the management server of the store that sells the product, such as the virtual store management server 14 or the actual store management server 16, and executes the ordering and payment processing. (S34).

When the ordering and settlement processing are normally completed, the related product image control unit 92 of the event management server 10 notifies the mobile terminal 12 that the ordering process is completed. In response to this, the display unit 118 of the mobile terminal 12 displays a superimposed image indicating that fact (S36). By the above processing, the transition of the screen as shown in FIG. 3 is realized.

In the ordering and settlement processing of S34, when the related product is sold at a plurality of physical stores, the related product sales unit 80 may select the physical store close to the mobile terminal 12 as the ordering party. In addition, the sales amount of related products can be used as an index for measuring the popularity of contestants. Therefore, the related product sales unit 80 of the event management server 10 may record the sales amount of the related product in association with the contestant ID or send it to the contestant's affiliated company. As a result, the data obtained from the sale can be utilized for the incentive calculation of the contestants.

FIG. 16 is a flowchart showing the operation of the event management server 10 and the mobile terminal 12 when the presenting of goods from the visitors to the participants is realized. This process may be performed before or after the event or during a limited period such as half-time, or may be performed periodically during the event. Further, it is assumed that the display of the real space in S20 of FIG. 15 is continuously performed. First, the presentation support unit 84 of the event management server 10 determines candidates for gifts to the contestants (S80).

At this time, the presentation support unit 84 may optimize the candidates for the gifts based on the attributes based on the user ID and the like. In this case, the displayed gift candidate may differ depending on the mobile terminal 12. Then, the gift image control unit 94 of the event management server 10 transmits the data related to the determined image of the gift candidate to the mobile terminal 12. The display unit 118 of the mobile terminal 12 superimposes and displays an image of a gift candidate based on the transmitted data (S82). At this time, the display unit 118 indicates to which contestant the candidate for the gift is, if necessary.

The mobile terminal 12 further accepts a gifting operation from a visitor by superimposing and displaying a GUI for selecting a gift and showing the intention of gifting (S84). When the presentation operation is performed, the terminal information transmission unit 112 of the mobile terminal 12 transmits the operation content and the position information of the mobile terminal 12 to the event management server 10. The presentation support unit 84 of the event management server 10 aggregates the presentation operations until the conditions for establishing the presentation are reached (N in S86). When the number of people who support the gift reaches the conditions for establishing the gift (Y in S86), the gift support unit 84 accesses the actual store management server 16 of the store that sells the gift, and performs ordering and settlement processing (S88).

When the ordering and payment processing are completed normally, the gift image control unit 94 of the event management server 10 outputs the location information of the supporter, that is, the other visitor who performed the gift operation, and the information representing the content of the gift to the mobile terminal 12. Send to. The display unit 118 of the mobile terminal 12 superimposes and displays an image indicating a supporter based on the transmitted information (S90). By the above processing, the screen as shown in FIGS. 10 and 11 is displayed.

FIG. 17 is a flowchart showing the operation of the event management server 10 and the mobile terminal 12 when selling products in a physical store. It is assumed that the display of the real space in S20 of FIG. 15 is continuously performed. First, the actual store management unit 86 of the event management server 10 starts collecting information on the actual store via the store communication unit 78 (S40). That is, the actual store management server 16 is accessed, and status information such as inventory status, congestion, and waiting time is collected for each store.

Then, the actual store image control unit 96 starts transmitting the store information including the store ID and the state information to the mobile terminal 12 (S42). At this time, the actual store image control unit 96 may transmit only the store information that the visitor who operates the mobile terminal 12 has pre-registered or designated on the spot. The display unit 118 of the mobile terminal superimposes and displays the state information together with the image representing the store based on the transmitted data (S44). For example, as shown in the superimposed image 162 of FIG. 12, a numerical value indicating the mark of the designated store and the degree of congestion is displayed.

The mobile terminal 12 waits until an operation for instructing an image showing store information is performed by a visitor, and during that time, the state information is updated as needed (N in S46). When the operation for instructing the image is performed, the operation reception unit 110 of the mobile terminal 12 accepts the operation and transmits the operation to the event management server 10 together with the store ID (Y in S46). The actual store management unit 86 of the event management server 10 determines the price of the product sold by the selected store accordingly (S48).

Then, the actual store image control unit 96 transmits data related to the menu image reflecting the determined price of the product to the mobile terminal 12. The display unit 118 of the mobile terminal 12 superimposes and displays a menu image as shown in FIG. 12 (b) based on the transmitted data (S50). Then, the operation reception unit 110 of the mobile terminal 12 accepts a purchase operation instructing any of the images of the product (S52). When the purchase operation is performed, the mobile terminal 12 transmits the content of the purchase operation, such as the identification information of the target product, to the event management server 10.

Upon receiving the content of the purchase operation, the related product sales unit 80 of the event management server 10 accesses the corresponding physical store management server 16 and executes an ordering and payment processing (S54). When the ordering and settlement processing are normally completed, the related product sales department 80 acquires the order number and the information related to the waiting time from the actual store of the ordering party. Then, the actual store image control unit 96 of the event management server 10 notifies the mobile terminal 12 of information such as the completion of the order processing, the order number, and the waiting time. Based on the transmitted information, the display unit 118 of the mobile terminal 12 displays a superposed image showing the completion of the order processing, the order number, the waiting time, and the like, as shown in FIG. 12 (c) (S56).

FIG. 18 is a flowchart showing the operation of the event management server 10 and the mobile terminal 12 when displaying an advertisement. This process may be performed before or after the event or during a limited period such as half-time, or may be performed regularly or continuously during the event. It is assumed that the display of the real space in S20 of FIG. 15 is continuously performed. First, the advertisement image control unit 98 of the event management server 10 optimizes the advertisement to be displayed based on the preference separation and attributes based on the user ID, and then transmits the data related to the image to the mobile terminal 12 (S60). ).

The display unit 118 of the mobile terminal 12 superimposes and displays the image of the advertisement based on the transmitted data (S62). The display position and the arrangement of the object for displaying the advertisement may be set in advance, or may be specified by the event management server 10 each time. As a result, the screen as shown in FIG. 13 is displayed. Then, the mobile terminal 12 accepts a purchase operation of the related product by instructing any of the advertisement images (S64). When the purchase operation is performed, the mobile terminal 12 transmits the operation content such as the target advertisement or product identification information and the location information of the mobile terminal 12 to the event management server 10.

If the advertisement is not the operation target, the processing after S64 may be omitted. Further, in the case of a corporate advertisement, when the instruction operation of the advertisement is performed in S64, the advertisement image control unit 98 transmits additional data related to the product sold by the company to the mobile terminal 12 to select the product. The superimposed image of may be displayed. The actual store management unit 86 of the event management server 10 searches for the actual store (in stock) that sells the corresponding product, and identifies the actual store closest to the mobile terminal 12 (S66).

The search target may include not only the inside of the event venue but also nearby stores outside the venue. In the case of stores outside the venue, the search target is stores that can deliver to visitors by the scheduled end time of the event. Then, the physical store management unit 86 accesses the physical store management server 16 of the specified store, and executes the ordering and settlement processing (S68). When the ordering and settlement processing are normally completed, the advertisement image control unit 98 of the event management server 10 notifies the mobile terminal 12 that the ordering process is completed.

In response to this, the display unit 118 of the mobile terminal 12 displays a superimposed image indicating that the order processing has been completed (S70). At this time, the display unit 118 may acquire information related to the order number, the waiting time, the scheduled delivery time, and the like from the event management server 10 and display the superimposed image representing them, as in S56 of FIG. ..

FIG. 18 shows an example of displaying an advertisement optimized based on individual visitor information. However, when a product purchased by another visitor with the same taste is presented as a recommendation, the same processing procedure is used. realizable. That is, in the process of S60, the advertisement image control unit 98 matches the visitors by the preference classification, and extracts the products purchased by one visitor at the venue as the products recommended to another visitor who matches the preferences. Visitor matching may be based on the identity of the purchase history at the event venue, the operation history of the superimposed image, for example, the history of operations for enlarging or attracting objects. Matching may be performed by combining the preference classification, the purchase history, and the operation history.

In the case of matching based on the preference classification, the display unit 118 of the mobile terminal 12 may further display a text message such as "a user similar to you also purchases this product" in S62. In the case of matching based on the purchase history and the operation history of the superimposed image, the display unit 118 of the mobile terminal 12 may say, "If you are interested in this, this product is also available." You may want to display more text messages such as "I'm buying." The target of matching does not have to be limited to visitors.

According to the present embodiment described above, on a terminal carried by a visitor to the event, an image for realizing various services related to the event is superimposed and displayed on an image of the real space where the event is being performed. At this time, by optimizing the information presented to individual visitors, sufficient information can be obtained in a small area. As a result, visitors can recognize the existence of the service as well as the state of the event itself such as sports and concerts. In addition, the event management server holds the personal information of the visitors and takes over the procedure for the visitors to use the service. This makes it possible to complete the service provision with an extremely simple operation. As a result, visitors are less likely to miss the opportunity to use the service. In addition, economic activities and events related to services will be activated.

Further, by drawing an object arranged in correspondence with the real space as a superimposed image, it is possible to make it appear that the object representing the service information exists in the real space. As a result, it is possible to guide the use of the service naturally and intuitively without damaging the world view of the event or distracting the attention of the visitors. In addition, various services such as selling products related to real objects such as contestants, presenting goods to contestants, placing orders at stores in the venue, displaying advertisements and selling related products can be easily developed, and various events can be held. It can be lively from the surface.

As an example, the event management server aggregates the presentation operations of the visitors and determines the success of the presentation. With this mechanism, it is possible to raise the motivation of the participants and to provide the contents that the visitors can enjoy together. Furthermore, by realizing cooperation with another system, such as using the points owned by visitors for payment, it is possible to increase the motivation and convenience of using the service.

The present invention has been described above based on the embodiments. It is understood by those skilled in the art that the above-described embodiment is an example, and that various modifications are possible for each of these components and combinations of each processing process, and that such modifications are also within the scope of the present invention. be.

For example, the image of the real space displayed by the mobile terminal 12 in the present embodiment is generated by the camera or the optical system included in the mobile terminal 12. On the other hand, an image taken by another camera in the hall and distributed by the event management server 10 to each mobile terminal 12 may be used as an image in real space. For example, the event management server 10 distributes a 360 ° image of the event and makes it visible on the client terminal from a free viewpoint. This technique is disclosed in, for example, Patent Document 2.

As a result, visitors can use the service in the same manner as in the present embodiment even in a situation where they cannot enter the venue such as a game or a concert without spectators. In this case, the event management server 10 may display information related to the presentation of goods to the contestants on a signage installed in the venue in addition to the mobile terminal 12.

For example, the event management server 10 displays the status of the presenting operation until the presenting is completed, the established presenting item, the face photograph of the presenter, and the like. Further, the event management server 10 may receive a support message from the audience via the mobile terminal 12 and display it on the signage. With such a production, even if there is no audience, the intention of cheering can be conveyed to the contestants, and the motivation of the contestants can be raised.

1 event management system, 10 event management server, 12 mobile terminal, 14 virtual store management server, 16 physical store management server, 23 CPU, 26 main memory, 32 communication unit, 50 CPU, 52 main memory, 54 display unit, 60 communication Department, 62 Motion sensor, 64 Camera, 66 Position acquisition department, 70 Terminal information acquisition department, 72 Information processing department, 74 Image data transmission department, 76 Information storage department, 78 Store communication department, 80 Related product sales department, 84 Presentation support Department, 86 Actual store management department, 91 Basic image control unit, 92 Related product image control unit, 94 Gift image control unit, 96 Actual store image control unit, 98 Advertising image control unit, 100 Personal information storage unit, 102 Related products Information storage unit, 104 store information storage unit, 106 advertisement information storage unit, 110 operation reception unit, 112 terminal information transmission unit, 114 status information acquisition unit, 116 image data reception unit, 118 display unit, 120 personal information storage unit, 122 Image data storage unit.

Claims (17)

A personal information storage unit that stores the personal information of a user who visits an event in association with the identification information of the terminal held by the user.
An image data transmission unit that transmits data for displaying a superimposed image corresponding to an image in the real space in which an event is performed, which is represented by the terminal, to the terminal.
A terminal information acquisition unit that acquires the content of a user operation for the superimposed image from the terminal, and
Based on the content of the user operation and the personal information, the information processing unit that acts on behalf of the procedure for using the service by the operated user,
An event management server system featuring. The event according to claim 1, wherein the image data transmission unit arranges a three-dimensional object represented by the superimposed image in a three-dimensional space of a stadium, which is an event venue, and transmits the information. Management server system. The image data transmission unit transmits data related to the superimposed image representing the sales target, and the image data transmission unit transmits the data.
The event management server system according to claim 1 or 2, wherein the information processing unit executes a payment process for sale in response to a purchase operation instructing the superimposed image. The event management server system according to claim 3, wherein the information processing unit determines a priority of a sales target represented in the superimposed image based on the personal information. The event according to claim 3 or 4, wherein the image data transmission unit transmits data relating to the superimposed image representing a product associated with a real object existing in the real space as a sales target. Management server system. The personal information storage unit stores the value of points held by the user as a substitute for money as the personal information.
The event management server system according to any one of claims 1 to 5, wherein the information processing unit uses the points as a consideration for using the service. Any of claims 3 to 5, wherein the information processing unit executes an order processing for sale to a store selected based on the distance from the terminal on which the operation is performed in response to the purchase operation. The event management server system described in. The image data transmission unit transmits data related to the superimposed image representing a gift to the contestants of the event.
The event according to any one of claims 1 to 7, wherein the information processing unit executes an ordering process for a gift to be delivered to the contestant in response to a gifting operation for instructing the superimposed image. Management server system. The event management server system according to claim 8, wherein the information processing unit executes the order processing when the status of the presentation operation by a plurality of users who have visited the event satisfies a predetermined condition. The event according to claim 8 or 9, wherein the image data transmission unit transmits data related to the superimposed image pointing to an image of another user who has performed the presentation operation among the images in the real space. Management server system. The information processing department acquires the state information of the store at the event venue at any time, and determines the price of the product sold at the store based on at least one of the state information and the personal information of the user.
The event management server system according to any one of claims 1 to 10, wherein the image data transmission unit transmits data relating to the superimposed image representing a menu including a determined price. The event management according to claim 11, wherein the image data transmission unit updates the display on the terminal by transmitting data related to the superimposed image representing the state information of the store designated by the user at any time. Server system. Based on the personal information, the image data transmission unit performs matching with other users who are attending the event, and transmits data related to the superimposed image showing the product purchased by the other user at the event as a recommendation. The event management server system according to any one of claims 1 to 12, characterized in that. The event according to any one of claims 1 to 13, wherein the image data transmission unit classifies a user's preference based on the personal information, and determines an advertisement to be represented as the superimposed image based on the classification. Management server system. Including a terminal held by a user who visits an event and an event management server system that provides a service to the user via the terminal.
The event management server system is
A personal information storage unit that stores the personal information of the user in association with the identification information of the terminal, and the personal information storage unit.
An image data transmission unit that transmits data for displaying a superimposed image in association with an image of the real space in which an event is performed, which is represented by the terminal, to the terminal.
A terminal information acquisition unit that acquires the content of a user operation for the superimposed image from the terminal, and
Based on the content of the user operation and the personal information, the information processing unit that acts on behalf of the procedure for using the service by the operated user,
An event management system characterized by being equipped with. A step of transmitting data for displaying a superimposed image corresponding to an image of the real space in which the event is being performed, which is represented by a terminal held by a user who visits the event, to the terminal.
The step of acquiring the content of the user operation for the superimposed image from the terminal, and
Based on the personal information stored in the memory in association with the identification information of the terminal and the content of the user operation, the step of substituting the procedure for using the service by the operated user, and the step.
An event management method by an event management server system, characterized by including. A function to send data for displaying a superposed image corresponding to an image in the real space where the event is being performed, which is displayed on the terminal held by the user who visits the event, to the terminal.
A function to acquire the contents of user operations for the superimposed image from the terminal, and
Personal information stored in the memory in association with the identification information of the terminal, and a function to act for the procedure of service use by the operating user based on the content of the user operation.
A computer program characterized by the realization of a computer.

Images