WO2022107639A1 - Program, information processing method, information processing device, and system - Google Patents

Abstract

A program executed by a first computer causes a processor to execute: a step for receiving first information about a race in the real world from a second computer; a step for generating a virtual object including a racetrack object that represents a racetrack and moving objects that represent moving bodies or participants in a prescribed race on the basis of the first information; and a step for displaying the virtual object superimposed on a real image captured by an imaging unit. The first information includes identification information for the moving bodies or participants, and color information associated with the identification information. In the step for generating the virtual object, a moving object 502 that is at least partially colored with a prescribed color is generated on the basis of the color information.

Description

Programs, information processing methods, information processing equipment, and systems

The present disclosure relates to programs, information processing methods, information processing devices, and systems.

Patent Documents 1 and 2 disclose techniques related to AR (Augmented Reality).

Japanese Unexamined Patent Publication No. 2020-58658 Japanese Unexamined Patent Publication No. 2020-77187

By the way, in the real world, various races such as boat races and horse races are carried out. Some people actually go to the race track to watch these races, but it may not be possible due to time or geographical constraints.

From the above situation, if the real world race can be displayed as a virtual race using a virtual object linked to the real world race, you can watch the race in a simulated manner without actually going to the race track. Can be beneficial.
By the way, when displaying various races such as boat races and horse races as virtual races, there is a problem that it is difficult for beginners to identify the contestants they support.

The techniques disclosed in Patent Documents 1 and 2 superimpose and display a virtual object unrelated to the real world on an image in the real world, and do not present any means for solving the above problem.

One aspect of the present disclosure is to use a virtual object linked to a real world race to determine a moving object corresponding to a race participant or a moving object when displaying the real world race as a virtual race. The purpose is to provide easy programs, information processing methods, information processing devices, and systems.

According to one embodiment presented in the present disclosure.
A program executed in a first computer equipped with a processor and an image pickup unit.
The program is attached to the processor.
The step of receiving the first information about a given race in the real world from the second computer,
A step of generating a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race based on the first information.
The step of superimposing the virtual object on the real image around the first computer captured by the imaging unit and displaying the virtual object is executed.
The first information includes identification information of the contestant or the moving body and color information associated with the identification information.
The generating step is provided with a program comprising generating the moving object, which is at least partially colored with a predetermined color, based on the color information.

According to one embodiment shown in the present disclosure, when a real world race is displayed as a virtual race by using a virtual object linked to the real world race, the movement corresponding to a race participant or a moving object is performed. The object can be easily identified.

It is a figure which shows the configuration example of the system which follows a certain embodiment. It is a block diagram which shows an example of the functional configuration of the user terminal according to a certain embodiment. It is a block diagram which shows an example of the functional configuration of the server according to a certain embodiment. It is a schematic diagram which shows an example of the race field of the real world according to a certain embodiment. It is a schematic diagram which shows an example of the virtual object displayed on the user terminal according to a certain embodiment. It is a flowchart which shows an example of the display control processing according to a certain embodiment. It is a flowchart which shows an example of the display control processing according to a certain embodiment. It is a figure which shows the race selection screen which is displayed on the user terminal according to a certain embodiment. (A) and (b) are schematic diagrams showing an example of a moving object displayed on a user terminal according to a certain embodiment. It is a schematic diagram which shows an example of the real image which was taken by the image pickup part according to a certain embodiment. It is a schematic diagram which shows an example of the screen which superposed virtual object on the real image which follows a certain embodiment. It is a schematic diagram which shows an example of the screen which superposed virtual object on the real image which follows a certain embodiment. It is a schematic diagram which shows an example of the virtual object displayed on the user terminal according to a certain embodiment. It is a schematic diagram which shows an example of the virtual object displayed on the user terminal according to a certain embodiment.

Hereinafter, embodiments of this technical idea will be described in detail with reference to the drawings. In the following description, the same elements are designated by the same reference numerals, and duplicate description will be omitted as appropriate. In one or more embodiments shown in the present disclosure, the elements included in each embodiment may be combined with each other, and the combined result shall also form part of the embodiments shown in the present disclosure.

(System configuration)
FIG. 1 is a diagram showing a configuration of a system 1 according to the present embodiment. The system 1 can display, for example, a predetermined race performed in the real world as a virtual race using a virtual object on an information processing device used by a user. In the present specification, the "predetermined race" is not particularly limited as long as it is a race carried out in the real world, and for example, a boat race (actual race or exhibition race), horse racing, bicycle racing, auto racing, etc. Car races such as F1, drone races, dog races, marathons, ekiden, etc. can be mentioned.

As shown in FIG. 1, the system 1 is a user such as a user terminal 10A, a user terminal 10B, and a user terminal 10C (hereinafter, user terminals 10A, 10B, 10C) which are information processing devices (first computers) used by each user. It includes a plurality of user terminals 10 such as (generally referred to as "user terminal 10"), a first server device (second computer) 20, a second server device 40, and a network 30.

The user terminal 10A and the user terminal 10B are connected to the network 30 by communicating with the radio base station 31. The user terminal 10C connects to the network 30 by communicating with a wireless router 32 installed in a facility such as a house. The user terminal 10 is, for example, a portable terminal provided with a touch screen, and may be a smartphone, a phablet, a tablet, or the like.

The user terminal 10 executes, for example, a program installed via a platform for distributing an application or the like, or a program including pre-installed website browsing software. The user terminal 10 communicates with the first server device 20 by executing the above program, and transmits / receives data related to a predetermined race, data related to the user, and the like to / from the first server device 20. It is possible to display a virtual race on the terminal 10.

The first server device 20 receives data related to a predetermined race from the second server device 40. The first server device 20 appropriately transmits data related to a predetermined race to the user terminal 10. The first server device 20 stores and manages data related to a predetermined race and data related to each user.

The first server device 20 includes a communication IF (Interface) 22, an input / output IF 23, a memory 25, a storage 26, and a processor (second processor) 29 as a hardware configuration, and these provide a communication bus. They are connected to each other via.

The communication IF 22 supports various communication standards such as LAN (Local Area Network) standards, and functions as an interface for transmitting and receiving data to and from the user terminal 10 and the second server device 40.

The input / output IF 23 functions as an interface for accepting input of information to the first server device 20 and outputting information to the outside of the first server device 20. The input / output IF 23 may include an input receiving unit that accepts the connection of an information input device such as a mouse and a keyboard, and an output unit that accepts the connection of an information output device such as a display for displaying an image or the like.

The memory 25 is a storage device for storing data or the like used for processing. The memory 25 provides the processor 29 with a work area for temporary use, for example, when the processor 29 performs processing. The memory 25 includes a storage device such as a ROM (ReadOnlyMemory) and a RAM (RandomAccessMemory).

The storage 26 is a storage device for storing various programs and data for the processor 29 to read and execute. The information stored in the storage 26 includes data related to a predetermined race (including identification information and color information of a moving object), data related to each user, and the like. The storage 26 may be configured to include a storage device such as an HDD (Hard Disk Drive) or a flash memory. The storage is not limited to the form included in the server device, and a cloud service can also be used.

The processor 29 controls the operation of the first server device 20 by reading and executing a program or the like stored in the storage 26. The processor 29 may be configured to include, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), and the like.

The second server device 40 stores and manages data related to a predetermined race. The second server device 40 is, for example, a server device managed by a predetermined race organizer or an organization (publisher of a race magazine, race video distributor, or radio distributor) that transmits information about a predetermined race to the outside. Etc.) is a server device managed by. The second server device 40 appropriately transmits data related to a predetermined race to the first server device 20. In certain aspects, the second server device 40 may transmit data related to a predetermined race to the user terminal 10. The hardware configuration of the second server device 40 may be the same as that of the first server device 20 as long as there is no contradiction. There may be a plurality of second server devices 40.

(User terminal)
FIG. 2 is a block diagram showing an example of the functional configuration of the user terminal 10. As shown in FIG. 2, the user terminal 10 includes an antenna 110, a wireless communication IF 120, a touch screen 130, an input / output IF 140, a storage unit 150, a voice processing unit 160, a microphone 161 and a speaker 162. It includes an image pickup unit 170 and a control unit (first processor) 190.

The antenna 110 radiates a signal emitted by the user terminal 10 into space as a radio wave. Further, the antenna 110 receives radio waves from the space and gives a received signal to the wireless communication IF 120.

Since the user terminal 10 communicates with other communication devices, the wireless communication IF 120 performs modulation / demodulation processing for transmitting / receiving signals via the antenna 110 or the like. The wireless communication IF 120 is a communication module for wireless communication including a tuner, a high frequency circuit, etc., performs modulation / demodulation and frequency conversion of the wireless signal transmitted / received by the user terminal 10, and supplies the received signal to the control unit 190.

The touch screen 130 receives input from the user and outputs information to the user on the display 132. The touch screen 130 includes a touch panel 131 for receiving user operation input and a display 132. The touch panel 131 detects that a user's finger or the like has approached by using, for example, a capacitance type touch panel. The display 132 is realized by, for example, an LCD (Liquid Crystal Display), an organic EL (electroluminescence), or other display device.

The input / output IF 140 functions as an interface for accepting input of information to the user terminal 10 and outputting information to the outside of the user terminal 10.

The storage unit 150 is composed of a flash memory, an HDD, or the like, and stores a program used by the user terminal 10 and various data received by the user terminal 10 from the first server device 20 or the like.

The voice processing unit 160 performs modulation / demodulation of the voice signal. The voice processing unit 160 modulates the signal given from the microphone 161 and gives the modulated signal to the control unit 190. Further, the voice processing unit 160 gives a voice signal to the speaker 162. The voice processing unit 160 is realized by, for example, a processor for voice processing. The microphone 161 functions as an audio input unit for receiving an input of an audio signal and outputting it to the control unit 190. The speaker 162 functions as an audio output unit for outputting an audio signal to the outside of the user terminal 10.

The image pickup unit 170 is a camera that captures a real image around the user terminal 10. The image captured by the image pickup unit 170 is image-processed by the control unit 190 and output to the display 132.

The control unit 190 controls the operation of the user terminal 10 by reading and executing a program stored in the storage unit 150. The control unit 190 is realized by, for example, an application processor.

The process of executing the application program 151 by the control unit 190 will be described in more detail. The storage unit 150 stores the application program 151, the application information 152, and the user information 153.

For example, the user terminal 10 downloads the application program 151 from the first server device 20 and stores it in the storage unit 150. Further, the user terminal 10 communicates with the first server device 20 to send and receive various data such as application information 152 and user information 153 to and from the first server device 20.

The application program 151 is a program for displaying a virtual race on the user terminal 10. The application information 152 includes various data referred to by the application program 151. The application information 152 includes the first information 152A and the event information 152B.

The first information 152A is information regarding a predetermined race transmitted from the first server device 20 and the second server device 40. The first information 152A includes, for example, race time information indicating the race time of a participant or a moving body (hereinafter, also referred to as “participant”) of a predetermined race, position information of the participant or the like during the execution of the predetermined race, and the like. The time information corresponding to the position information and the time information are included. The first information further includes identification information of participants and the like, color information associated with the identification information, and color position information for designating a position of color registration in a moving object described later.

As used herein, the term "participant" is a concept that includes not only humans but also animals such as horses and dogs. Further, the "moving body" is a main body of movement in a predetermined race, and is an animal or an aircraft on which a participant rides, an aircraft remotely controlled by a contestant, or the like. In marathons, dog races, etc., "participants" and "moving bodies" are the same.

In addition to the above, the first information 152A includes, for example, a predetermined race name, date and time, race field data, contestant data, moving object data, odds information, race prediction, race start table, and information immediately before the race. It may include pit reports, race results, race videos, race stills, past race information, and other information that may be published in information magazines or information sites about a given race.

The event information 152B is information about an event that can occur or has occurred in a predetermined race, and includes, for example, information about an effect display (effect display) and an effect sound corresponding to the event.

The event is not particularly limited, but for example, the goal of the first contestant, etc. is reached, the ranking of a predetermined race is confirmed, the ranking of the contestants, etc. is changed, and the difference in the positions of two or more contestants, etc. at a predetermined timing. May be less than or equal to a predetermined threshold value (two or more contestants or the like have a close battle), or one or more of the occurrence of rule violations. In this example, the event information 152B is information received from the first server device 20 after an event occurs in a predetermined race.

Further, the event information 152B includes, for example, information about a two-dimensional image or a three-dimensional image corresponding to a predetermined event. Specifically, the event information 152B may include information about a text image for notifying the content of the event, a predetermined character image, an effect image to be added to a virtual object described later, and the like.

The user information 153 includes information about the user of the user terminal 10. The user information 153 includes, for example, information for identifying a user, location information of a user terminal 10, a race purchase history and a hit rate of a user (for example, a history of a boat ticket purchased in the case of a boat race, and a boat ticket purchased. Hit rate) and the like may be included.

By reading and executing the application program 151, the control unit 190 includes an operation input reception unit 191, a transmission / reception unit 192, a race attribute selection unit 193, an object generation unit 194, an identification number assigning unit 195, and color information. Each function of the registration unit 196, the display control unit 197, the flat surface detection unit 198A, the event detection unit 198B, and the selection information acquisition unit 199 is exhibited.

The operation input receiving unit 191 accepts the user's operation input based on the output of the touch screen 130. Specifically, the operation input receiving unit 191 detects that the user's finger or the like touches or approaches the touch panel 131 as the coordinates of the coordinate system consisting of the horizontal axis and the vertical axis of the surface constituting the touch screen 130.

The operation input receiving unit 191 determines the user's operation on the touch screen 130. The operation input reception unit 191 is, for example, "approach operation", "release operation", "tap operation", "double tap operation", "long press operation (long touch operation)", "drag operation (swipe operation)", Determine the user's operation such as "move operation", "flick operation", "pinch-in operation", and "pinch-out operation".

The operation input receiving unit 191 may accept the movement of the user terminal 10 detected by the acceleration sensor or the like mounted on the user terminal 10 as an operation input.

The transmission / reception unit 192 transmits and receives various information to and from external communication devices such as the first server device 20 and the second server device 40 via the wireless communication IF 120 and the network 30. The transmission / reception unit 192 receives, for example, the first information 152A from the first server device 20 or the second server device 40. Further, the transmission / reception unit 192 transmits, for example, information corresponding to the operation input received by the operation input reception unit 191 and information stored in the user information 153 to the first server device 20 or the second server device 40. ..

The race attribute selection unit 193 selects the race attribute (specifically, the type of race) according to the operation input received by the operation input reception unit 191. As described above, examples of race types include boat races (actual races and exhibition races), horse races, bicycle races, auto races, car races such as F1, drone races, dog races, marathons, and relay races. ..

The object generation unit 194 generates a virtual object for presenting information about a predetermined race selected by the race attribute selection unit 193 to the user based on the first information. The object generation unit 194 generates, as virtual objects, a race field object representing a race field and a moving object representing a contestant or the like. The object generation unit 194 may generate a virtual display board for displaying the second information generated based on the first information as text. In addition to the above objects, the object generation unit 194 is a landscape object that constitutes a landscape such as a virtual screen for displaying an image of the second information generated based on the first information, various building objects, and trees. , An object or the like that becomes a user's avatar may be generated.

The identification number assigning unit 195 assigns an identification number to each of the moving objects generated by the object generation unit 194. The identification number assigning unit 195 assigns identification numbers in the order of arrangement (for example, frame order) of the moving objects according to the number of moving objects that differ depending on the attributes of the race.

The color information registration unit 196 acquires color information such as participants in a predetermined race selected by the race attribute selection unit 193. The color information registration unit 196 further associates the acquired color information with the identification number assigned to each moving object by the identification number assigning unit 195, and executes color registration to the moving object. For example, the color information registration unit 196 acquires identification information and color information from the first information 152A received from the second server device 40, and is identified by the identification number assigning unit 195 based on the acquired identification information and color information. A predetermined color is registered in a predetermined part of each numbered moving object.

The display control unit 197 superimposes an image (hereinafter, also referred to as “superimposed image”) on which a virtual object generated by the object generation unit 194 is superimposed on a real image around the user terminal 10 imaged by the image pickup unit 170. It is displayed on the display 132. Based on the race time information included in the first information 152A, the display control unit 197 moves each moving object whose predetermined portion is colored by the color information registration unit 196 on the racecourse object, and virtually performs the predetermined race. The reproduced virtual race is displayed on the display 132. It is preferable that the display control unit 197 reproduces the virtual race based on the position information of the contestants and the like included in the first information 152A and the time information corresponding to the position information in addition to the race time information. ..

It is preferable that the display control unit 197 can change the viewpoint in the superimposed image according to the operation input received by the operation input reception unit 191. The display control unit 197 displays various menu screens and GUIs (Graphical User Interfaces) on the display 132, and changes the display contents of the display 132, in response to the operation input received by the operation input reception unit 191.

Further, the display control unit 197 displays an effect corresponding to the event in response to the event detection unit 198B detecting the occurrence of the event. The content of the effect display is not particularly limited, but may be, for example, an effect display on at least one of a race field object and a moving object, and a display of a two-dimensional image or a three-dimensional image corresponding to a predetermined event. It may be one or more of displaying predetermined character images and operating these images according to the progress of the race. The display control unit 197 may simultaneously execute two or more effect displays corresponding to different events.

The flat surface detection unit 198A detects a flat surface in a real image captured by the image pickup unit 170. The detection of a flat surface is realized by a conventionally known image recognition technique. For example, when the user performs an operation of selecting a flat surface detected by the flat surface detection unit 198A, a superimposed image in which the race field object is arranged on the flat surface is displayed on the display 132.

The flat surface is preferably a horizontal surface. Further, the angle formed by the flat surface and the bottom surface constituting the race field object may be 0 degrees, but is preferably an acute angle, and can be, for example, in the range of 15 degrees to 45 degrees. The above angle may be adjusted by accepting a user's operation. Also, even if there is a convex part on a part of the flat surface in the real world, or if there is an object on the flat surface, if the convex part or the object is of a size that can be hidden by the racetrack object. , The place where the race field object can be placed may be detected as a flat surface.

The event detection unit 198B detects the occurrence of a predetermined event in a predetermined race. The event detection unit 198B receives, for example, event information 292B transmitted from the first server device 20 when a predetermined event occurs in a predetermined race in the real world, and the transmission / reception unit 192 receives the predetermined event. Detects the occurrence of.

Further, the event detection unit 198B may detect the occurrence of a predetermined event based on, for example, the first information 152A and the user information 153. Specifically, the event detection unit 198B determines from the position of the contestants, etc. during the race, such as the change of ranking of the contestants, etc., based on the position information of the contestants, etc. and the time information corresponding to the position information. Can detect possible events. The event detection unit 198B can detect an event such that a contestant or the like purchased by the user is located at the head of the race based on, for example, information on the race result and information on the race purchase history of the user.

The selection information acquisition unit 199 acquires one or more selection information indicating one or more contestants selected by the user. The "selection information" is, for example, information indicating a contestant or the like predicted by the user as a winner or the like of a predetermined race. When the predetermined race is a boat race or a horse race, the "selection information" may be information indicating a boat ticket or a betting ticket purchased by the user.

The selection information acquisition unit 199 acquires selection information based on, for example, a user's operation input on the screen for selecting a contestant or the like. The selection information acquired by the selection information acquisition unit 199 is transmitted to, for example, the first server device 20 or the second server device 40. When the predetermined race is a boat race or a horse race, the purchase process of the boat ticket or the betting ticket may be performed according to the selection information being transmitted to the second server device 40.

(1st server device)
FIG. 3 is a block diagram showing a functional configuration of the first server device 20. A detailed configuration of the first server device 20 will be described with reference to FIG. The first server device 20 exerts functions as a communication unit 220, a storage unit 250, and a control unit 290 by operating according to a program.

The communication unit 220 functions as an interface for the first server device 20 to communicate with an external communication device such as the user terminal 10 or the second server device 40 via the network 30.

The storage unit 250 stores various programs and data for realizing the system 1. In a certain aspect, the storage unit 250 stores the program 251 and the race information 252 and the user information 253.

The program 251 is a program for the first server device 20 to communicate with the user terminal 10 and the second server device 40 to realize the system 1. The program 251 is executed by the control unit 290 to send and receive data to and from the user terminal 10 and the second server device 40, a process according to the operation content performed by the user of the user terminal 10, race information 252, and the user. The first server device 20 is made to perform a process of updating the information 253 and the like.

Race information 252 contains various data related to a given race. The race information 252 includes, for example, the first information 252A and the event information 252B. The first information 252A is the source information of the first information 152A, and the first information 152A can be a part of the first information 252A. The first information 252A is, for example, information acquired from the second server device 40. Similarly, the event information 252B is the source information of the event information 152B, and the event information 152B may be a part of the event information 252B. The event information 252B is, for example, information acquired from the second server device 40.

The user information 253 is information about the user of the user terminal 10. The user information 253 includes a user management table 253A. The user management table 253A stores, for example, information for identifying the user, position information of the user terminal 10, a race purchase history of the user, a hit rate, and the like for each user.

The control unit 290 is realized by the processor 29, and by executing the program 251, the transmission / reception unit 291, the first information acquisition unit 292A, the event information acquisition unit 292B, the selection information acquisition unit 293, the data management unit 294, and the timekeeping unit 295. Demonstrate the function as.

The transmission / reception unit 291 transmits and receives various information to and from an external communication device such as a user terminal 10 and a second server device 40 via the communication unit 220 and the network 30. The transmission / reception unit 291 transmits, for example, at least a part of the first information 252A and the event information 252B to the user terminal 10. Further, the transmission / reception unit 291 receives, for example, the first information 252A and the event information 252B from the second server device 40.

The first information acquisition unit 292A acquires the first information 252A from the second server device 40 via the transmission / reception unit 291. The event information acquisition unit 292B acquires the event information 252B from the second server device 40 via the transmission / reception unit 291. The selection information acquisition unit 293 acquires the user's selection information from the user terminal 10 or the second server device 40 via the transmission / reception unit 291. The data management unit 294 performs a process of updating various data stored in the storage unit 250 according to the processing results of the first information acquisition unit 292A, the event information acquisition unit 292B, the selection information acquisition unit 293, and the like. The timekeeping unit 295 performs a process of measuring time. Various times displayed on the user terminal 10 (for example, the time until the start of the race) can be controlled based on the time measured by the time measuring unit 295.

(Operation example)
Next, an operation example in the system 1 will be described with reference to FIGS. 4 to 13. In the following, a case where a predetermined race is a boat race will be described as an example, but the following description can be applied even if the predetermined race is another race. Further, in the following description, it is assumed that data is transmitted / received between the user terminal 10 and the first server device 20 and between the first server device 20 and the second server device 40, but in a certain aspect, the user The terminal 10 and the second server device 40 may be configured to directly transmit and receive data.

FIG. 4 is a schematic diagram showing an example of a boat race field in the real world. Two turn marks 403 are installed in the boat race field 401, and races are carried out by boats 402a to 402f on which each boat racer rides. When the race is completed, the race time information indicating the race times of the boats 402a to 402f is transmitted from the second server device 40 to the first server device 20, and is transmitted from the first server device 20 to the user terminal 10.

The boat race field 401 is provided with image pickup devices (cameras) 404a to 404b. The image pickup apparatus 404a captures the boats 402a to 402f in the field of view from above the boat race field 401. The image pickup apparatus 404b keeps the boats 402a to 402f in the field of view from the side of the boat race field 401. The images of the boats 402a to 402f captured by the image pickup devices 404a to 404b are transmitted to the second server device 40. In the second server device 40, for example, image analysis of each image is performed, and position information indicating the position of each of the boats 402a to 402f in the shooting time of each image is calculated. The calculated position information and the time information related to the shooting time corresponding to the position information are transmitted to the first server device 20, and are transmitted from the first server device 20 to the user terminal 10. The location information may be calculated by the first server device 20.

Further, a position sensor such as a GPS sensor may be installed on the boats 402a to 402f instead of or in addition to the image pickup devices 404a to 404b. The position information of the boats 402a to 402f acquired by the position sensor and the time information indicating the time when the position information was acquired are finally transmitted to the user terminal 10.

FIG. 5 is a schematic diagram showing an example of a virtual object displayed on the user terminal 10. In the example of FIG. 5, as virtual objects, a race field object 501, moving objects 502a to 502f, two turnmark objects 503, and a virtual display board 505 are shown.

The race field object 501 is an object that virtually displays the boat race field 401. The race field object 501 and the turn mark object 503 are preferably created based on race field data such as course information of the boat race field 401, and preferably have a shape corresponding to the boat race field 401.

The moving objects 502a to 502f are objects that virtually display the boats 402a to 402f, respectively, and have a shape imitating a boat. The moving objects 502a to 502f move the race field object 501 based on the race time information, the position information of the boats 402a to 402f, and the time information corresponding to the position information. That is, the race field object 501 and the moving objects 502a to 502f display the race in the real world as a virtual race on the user terminal 10.

The moving objects 502a to 502f, which are virtual representations of the boats 402a to 402f, are colored by the color information registration unit 196 in a predetermined position in accordance with the boats 402a to 402f in the real world. 10 Displayed above. As shown in FIGS. 4 and 5, the moving object 502a is colored white like the real-world boat 402a (boat 1) and the moving object 502b is black like the real-world boat 402b (boat 2). Colored with, the moving object 502c is colored red like the real-world boat 402c (boat 3), the moving object 502d is colored blue like the real-world boat 402d (boat 4), and the moving object The 502e is colored yellow like the real world boat 402e (boat 5) and the moving object 502f is colored green like the real world boat 402f (boat 6).

Even if there is no position information and time information of the boats 402a to 402f, it is possible to display the virtual race only by the race time information. However, in this case, the final order of arrival will be the same as in the real world race, but it is difficult to reproduce the race progress such as the ranking during the race.

The virtual display board 505 is an object that displays text information. The virtual display board 505 is, for example, a corresponding nonexistent object in the boat race field 401. The text information displayed on the virtual display board 505 is not particularly limited, and may be, for example, ranking information, odds information, or the like. Further, the text information displayed on the virtual display board 505 may be changeable based on the operation input of the user.

6 and 7 are flowcharts showing an example of display control processing. It should be noted that the order of each process constituting the flowchart described below is random as long as there is no contradiction or inconsistency in the process contents. Further, the processing executed by each device may be executed by another device within a range that does not cause a contradiction.

The processes shown in FIGS. 6 and 7 are realized by the control unit 190 executing the application program 151 and the control unit 290 executing the program 251.

First, in step S610, the control unit 190 selects a race type (race attribute) to be displayed on the user terminal 10 as a virtual object based on the user's operation input. FIG. 8 is a diagram showing a race selection screen 800 displayed on the user terminal 10 by executing the application program 151. As shown in FIG. 8, on the race selection screen 800, a plurality of races (for example, boat race, horse race, bicycle race, auto race) corresponding to a plurality of types of races (for example, boat race, horse race, bicycle race, auto race) in which a virtual race can be displayed on the user terminal 10 by the application program 151 are displayed. Selection buttons 801a to 801d are included. The control unit 190 selects a race type based on the user inputting an operation to any of the plurality of selection buttons 801a to 801d displayed on the race selection screen 800.

In step S620, the control unit 190 creates a race field object as a virtual object according to the selected race type. In this example, for example, a race field object of a boat race is generated based on the user selecting the selection button 801a corresponding to the boat race.

In step S630, the control unit 190 creates a moving object as a virtual object according to the selected race type. In this example, based on the user selecting the selection button 801a corresponding to the boat race, a plurality of moving objects 502a to 502f corresponding to the plurality of boats 402a to 402f of the boat race are generated.

In step S640, the control unit 190 assigns an identification number to each of the plurality of generated moving objects 502a to 502f. Specifically, the control unit 190 assigns an identification number to each of the moving objects 502a to 502f in correspondence with the boats 402a to 402f of boats 1 to 6 in the boat race in the real world.

In step S650, the control unit 190 associates the identification numbers assigned to the moving objects 502a to 502f with the color information. Specifically, the control unit 190 associates the color information matching the designated colors of boats 1 to 6 in the boat race in the real world with the identification numbers of the moving objects 502a to 502f. The control unit 190 is No. 1 in the real world boat race based on the fact that the plurality of moving objects 502a to 502f generated by the object generation unit 194 are objects corresponding to the boats 402a to 402f of the real world boat race. The color information that matches the designated color from the boat to the No. 6 boat is associated with the identification number of each moving object 502a to 502f.

As mentioned above, in the real world boat race, the designated color of boat 1 is white, the designated color of boat 2 is black, the designated color of boat 3 is red, and the designated color of boat 4 is red. It is stipulated that the designated color of boat No. 5 is blue and the designated color of boat No. 6 is green. Therefore, the control unit 190 associates white color information with the identification number of the moving object 502a corresponding to the boat 402a of the first boat, and black color the identification number of the moving object 502b corresponding to the boat 402b of the second boat. The information is associated with the identification number of the moving object 502c corresponding to the boat 402c of the third boat, and the red color information is associated with the identification number. Further, the control unit 190 associates blue color information with the identification number of the moving object 502d corresponding to the boat 402d of the 4th boat, and yellow color the identification number of the moving object 502e corresponding to the boat 402e of the 5th boat. The information is associated with the identification number of the moving object 502f corresponding to the boat 402f of the No. 6 boat, and the green color information is associated with the identification number.

In step S660, the control unit 190 performs color registration at designated locations of the moving objects 502a to 502f based on the correspondence between the identification numbers of the moving objects 502a to 502f and the color information. Specifically, the control unit 190, and the control unit 190, each moving object based on the color information associated with the identification numbers of the moving objects 502a to 502f and the color position information included in the first information. Color registration is performed at predetermined positions of 502a to 502f. That is, for each moving object 502a to 502f, color registration is performed according to the frame number of the boat in the real world.

FIG. 9 (a) is a diagram showing an example of a moving object 502 that imitates a boat and a player (boat racer) of a boat race, and FIG. 9 (b) is a diagram that imitates a racehorse and a player (jockey) of horse racing. It is a figure which shows an example of a moving object 950. As shown in FIG. 9A, in the case of boat race, the color position information includes, for example, color registration being performed on the side surface 901A of the boat portion 901 and the helmet 902A of the athlete portion 902 of the moving object 502. Is done. On the other hand, as shown in FIG. 9B, in the case of horse racing, the color position information includes, for example, color registration being performed on the hat 952A of the player 952 among the moving objects 950. It is also possible to color-register the racehorse 951 number 951A and the racing uniform 952B. In this way, the color position information included in the first information is changed according to the type (attribute) of the race and the shape of the moving object. The color position information may be changed by the user's operation input. That is, the user may be able to select / change the color position information so that the color is registered at a predetermined position of the moving object desired by the user.

In step S670, the control unit 190 activates the image pickup unit 170, which is a camera. The image pickup unit 170 captures a real image around the user terminal 10.

In step S680, the control unit 190 detects a flat surface in the image captured by the image pickup unit 170. In step S690, the control unit 190 arranges the virtual object on the detected flat surface.

Here, the processes of steps S670, S680, and S690 will be specifically described with reference to FIGS. 10 to 12. FIG. 10 is a schematic diagram showing an example of a real image captured by the imaging unit 170. In the example of FIG. 10, a keyboard 1002 and a monitoring device 1003 are placed on a flat desk 1001.

When the image pickup unit 170 is activated in step S670, the real image captured by the image pickup unit 170 is displayed on the display 132. Next, in step S680, the control unit 190 detects a flat surface in the image captured by the image pickup unit 170, that is, in the image displayed on the display 132.

In FIG. 10, the region 1004 is detected as a flat surface. There is a keyboard 1002 in the area 1004, but since the keyboard 1002 is small enough to be hidden by the race field object 501, the control unit 190 detects the area 1004 as a flat surface.

If the position imaged by the imaging unit 170 is changed in the state shown in FIG. 10, the position of the region 1004 can also be changed. The area 1004 is displayed on the display 132 so as to be distinguishable from other parts, for example, by adding a predetermined color. When the user executes a tap operation or the like on the area 1004, in step S690, the control unit 190 arranges virtual objects such as the race field object 501 and the moving objects 502a to 502f on the area 1004.

FIG. 11 is a schematic diagram showing an example of a screen in which a virtual object is superimposed on a real image and displayed. In FIG. 11, the area with the dot pattern including the monitor device 1003 is the real image, and the other area is the area where the virtual object is displayed. For example, an advertisement image may be displayed in an area where the virtual object is not displayed.

In FIG. 11, racefield objects 501, a plurality of moving objects 502, two turnmark objects 503, 505, large monitor objects 506, building objects 507a to 507b, and other reference numerals are added as virtual objects. A large number of objects (tree objects, clock objects, etc.) that have not been displayed are displayed. These objects are created, for example, based on the first information 152A received from the first server device 20.

FIG. 12 is a schematic view showing an example of a screen in which a virtual object is superimposed on a real image, and shows another aspect of the race field object 501 shown in FIG. Specifically, FIG. 12 is an example in which a predetermined race is horse racing.

Also in FIG. 12, the area with the dot pattern including the monitoring device 1003 is the real image, and the other areas are the areas where the virtual objects are displayed. In FIG. 12, as virtual objects, a race field object 511, a plurality of moving objects 512, a large monitor object 513, a pond object 514, and a plurality of tree objects 515 are displayed on the display 132. These objects are also created, for example, based on the first information 152A received from the first server device 20.

The racetrack object 511, the large monitor object 513, the pond object 514, and the plurality of tree objects 515 are preferably created based on racetrack data such as, for example, course information of a predetermined racetrack in the real world. The plurality of moving objects 512 are, for example, objects that virtually display horses and jockeys running in horse racing.

Next, in step S710 of FIG. 7, the control unit 190 acquires the position information of the boats 402a to 402f in the boat race field 401 in the real world. That is, when the race by the boats 402a to 402f is started in the real world, the control unit 190 acquires the position information and the time information of the boats 402a to 402f from the first server device 20. The method of acquiring the position information and the time information is as described with reference to FIG.

In step S720, the control unit 190 controls the position information acquired in step S710 to be linked with the moving object. Specifically, using the time information and the position information, the movements of the boats 402a to 402f on the race field object 501 are controlled to be the same as the movement objects 502a to 502f on the boat race field 401.

In step S730, the control unit 190 detects the occurrence of a predetermined event in a predetermined race. When a predetermined event occurs in a predetermined race in the real world, for example, the control unit 290 is based on the image analysis of the race, the information received from the second server device 40, and the information stored in the event information 252B. , The occurrence of the event is detected, and the event information indicating that the occurrence of the event is detected is transmitted to the user terminal 10. In step S730, the control unit 190 detects the occurrence of an event, for example, based on receiving the event information during the race.

Further, in step S730, the control unit 190 generates a predetermined event based on, for example, the first information 152A and the user information 153, without depending on the transmission of the event information 252B from the first server device 20 during the race. May be detected. For example, the control unit 190 may detect the occurrence of a predetermined event based on the position information of the contestants and the like and the time information corresponding to the position information.

In step S740, the control unit 190 executes registration of the display color of the effect image corresponding to the generated event based on the identification number and the color information associated with each other in step S650. For example, in the case of a boat race, the control unit 190 registers the display color of the effect image based on the color registration corresponding to each identification number of the moving objects 502a to 502f.

In step S750, the control unit 190 executes the display of the effect image for which the registration of the display color is completed. When event information is received from the first server device 20 in step S730 and the event information includes information related to the effect display, in step S750, for example, based on the information related to the effect display, the effect image is displayed. The display is executed. It should be noted that only the event identification information such as the event ID is received from the first server device 20, and for example, the information about the effect image corresponding to the acquired event ID or the like is acquired from the event information 152B and based on the information. An effect image may be displayed.

Hereinafter, the effect display process in step S750 will be specifically described with reference to FIGS. 13 and 14. 13 and 14 are schematic views showing an example of the display screen in step S750.

The effect image 1201 is displayed differently depending on the type of event detected in step S730. In the example of FIG. 13, on the display 132, in addition to virtual objects such as the race field object 501 and the moving objects 502a to 502f, an aura-like effect image 1201 is formed around the moving object 502c which is the target of the effect display. It is displayed. This effect image 1201 is added to, for example, a contestant who is the target of the voting right purchased by the user with the occurrence of the final straight line event. The effect image 1201 displayed around the moving object 502c is registered in red, which is the same color as the display color of the moving object 502c, in step S740, and is displayed in a state of being colored red in step S750.

The addition of the effect image 1201 is not limited to the above example, and for example, detection of the occurrence of another event other than the final linear event (for example, increased acceleration, faster speed than other contestants, etc.) May be added depending on (such as when) is detected. Further, the effect display may be added to other virtual objects such as the race field object 501 instead of the moving object 502. For example, as shown in FIG. 14, the movement locus image 1301 through which the movement object 502c of the event target has passed among the movement objects 502a to 502f may be added to the race field object 501 as an effect display. Similar to the effect image 1201 described above, the movement locus image 1301 is registered in red, which is the same color as the display color of the moving object 502c to be displayed in step S740, and is displayed in a state of being colored red in step S750. Is preferable. In the example of FIG. 14, the moving objects 502d, 502e, and 502f are omitted from the moving objects 502a to 502f for the sake of simplification.

In the examples of FIGS. 13 and 14, the effect image 1201 and the movement locus image 1301 are displayed only when an event occurs. For example, in the vicinity of the moving object of the contestant who is the target of the voting right purchased by the user. The effect image may be displayed at all times. That is, in step S730, the occurrence of an event may be detected based on the user purchasing the voting right of a predetermined contestant. Examples of the effect image that is always displayed include a crown-shaped two-dimensional image displayed on the upper part of the moving object 502 that is the target of the voting right purchased by the user. The position and shape of the effect image is not limited to the illustrated example as long as it is arranged in the vicinity of the corresponding moving object, and may be a three-dimensional image.

The association between the moving object 502 and the effect image is not limited to the above example, and may be added to, for example, a contestant who is not subject to the voting right purchased by the user. In this case, the association between the moving object 502 and the effect image may be set in advance by the administrator of the first server device 20 or the user of the user terminal 10. For example, the user of the user terminal 10 may display the effect image corresponding to the moving object set as "favorite".
Further, the effect image displayed corresponding to the moving object 502 may be configured so that the user can select it from a plurality of objects set as selection candidates. Further, the types of objects that are candidates for selection may be increased according to the user's billing and the like.

The process of steps S710 to S750 is repeated at least from the start time to the end time of the race in the real world, but may be repeated before and after the start and end of the race in the real world.

If the race displayed as a virtual race is a past race, the position information of the boats 402a to 402f from the start to the end of the race may be collectively acquired before the start of the virtual race. Further, the virtual race may be displayed by acquiring only the race time information without acquiring the position information and the like. The race displayed as a virtual race may be an exhibition race.

After the end of the race in the real world, in step S760, the control unit 190 executes the result display process and ends a series of display control processes in response to receiving an operation input for terminating the application program 151.

The above embodiments are merely examples for facilitating the understanding of the present invention, and are not intended to limit the interpretation of the present invention. It goes without saying that the present invention can be modified and improved without departing from the spirit thereof, and the present invention includes its equivalents.

[Additional notes]
The contents of this disclosure are listed below.

(Item 1)
A program executed in a first computer equipped with a processor and an image pickup unit.
The program is attached to the processor.
The step of receiving the first information about a given race in the real world from the second computer,
A step of generating a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race based on the first information.
The step of superimposing the virtual object on the real image around the first computer captured by the imaging unit and displaying the virtual object is executed.
The first information includes identification information of the contestant or the moving body and color information associated with the identification information.
The generation step comprises generating the moving object, which is at least partially colored with a predetermined color, based on the color information.
program.
This makes it easier to identify moving objects that correspond to race participants and moving objects when displaying real-world races as virtual races using virtual objects linked to real-world races. The visibility of the race can be improved. As a result, it is possible to improve the satisfaction and convenience of users, especially users who are new to watching games.

(Item 2)
The correspondence between the identification information and the color information is changed based on the designated color corresponding to the frame number of the predetermined race.
The program described in item 1.
As a result, the color coding in the real world race can be reproduced on the virtual race, so that the user's satisfaction can be further improved.

(Item 3)
The program according to item 2, wherein the designated color is changed by the attribute of the predetermined race.
Since the color set for the frame number differs depending on the type of race in the real world, it is possible to improve the visibility of the virtual race by changing the color registration of the moving object on the virtual race according to the type of race. can.

(Item 4)
The first information further includes color position information for designating the position of color registration to the moving object.
The color position information is changed according to the shape of the moving object and at least one of the attributes of the predetermined race.
The program according to any one of items 1 to 3.
As a result, the shape of the moving object changes according to the type of race in the real world, so by registering the color of the moving object according to the race, the visibility and interest of the virtual race can be further improved. ..

(Item 5)
In the display step, the moving object is moved on the race field object based on the first information, the predetermined race is virtually displayed, and at least one of the race field object and the moving object is displayed. Display the effect on
Based on the color information, at least a part of the effect display is colored with the predetermined color.
The program according to any one of items 1 to 4.
As a result, the user can make the target moving object more conspicuous during the virtual race, and the user can easily grasp the progress of the race. As a result, the convenience of the user can be improved.

(Item 6)
The program according to item 5, wherein the effect display is a display showing a movement locus of the moving object on the race field object.
This makes it possible to further improve the visibility of the virtual race.

(Item 7)
An information processing method executed in a first computer equipped with a processor and an image pickup unit.
The information processing method is applied to the processor.
The step of receiving the first information about a given race in the real world from the second computer,
A step of generating a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race based on the first information.
The step of superimposing and displaying the virtual object on the real image around the first computer captured by the imaging unit is included.
The first information includes the identification information of the moving object and the color information associated with the identification information.
The generation step comprises generating the moving object, which is at least partially colored with a predetermined color, based on the color information.
Information processing method.
This makes it easier to identify moving objects that correspond to race participants and moving objects when displaying real-world races as virtual races using virtual objects linked to real-world races. The visibility of the race can be improved. As a result, it is possible to improve the satisfaction and convenience of users, especially users who are new to watching games.

(Item 8)
An information processing device including a processor and an image pickup unit, wherein the processor is
Receive the first information about a given race in the real world from the second computer,
Based on the first information, a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race is generated.
The virtual object is superimposed and displayed on a real image around the information processing apparatus captured by the image pickup unit.
The first information includes the identification information of the moving object and the color information associated with the identification information.
Creating the virtual object comprises creating the moving object, which is at least partially colored with a predetermined color, based on the color information.
Information processing equipment.
This makes it easier to identify moving objects that correspond to race participants and moving objects when displaying real-world races as virtual races using virtual objects linked to real-world races. The visibility of the race can be improved. As a result, it is possible to improve the satisfaction and convenience of users, especially users who are new to watching games.

(Item 9)
There is a system including a first computer provided with a first processor and an image pickup apparatus, and a second computer provided with a second processor and capable of communicating with the first computer.
The second processor is
It acquires the first information about a predetermined race in the real world and transmits the first information to the first computer.
The first processor is
The first information is received from the second computer,
Based on the first information, a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race is generated.
The virtual object is superimposed and displayed on a real image around the first computer captured by the image pickup device.
The first information includes identification information of the moving object and color information associated with the identification information.
Creating the virtual object comprises creating the moving object, which is at least partially colored with a predetermined color, based on the color information.
system.
This makes it easier to identify moving objects that correspond to race participants and moving objects when displaying real-world races as virtual races using virtual objects linked to real-world races. The visibility of the race can be improved. As a result, it is possible to improve the satisfaction and convenience of users, especially users who are new to watching games.

1: System, 10: User terminal, 20: First server device, 30: Network, 40: Second server device, 130: Touch screen, 150: Storage unit (of user terminal), 190: Control (of user terminal) Unit, 191: Operation input reception unit, 192: Transmission / reception unit, 193: Race attribute selection unit, 194: Object generation unit, 195: Identification number assignment unit, 196: Color information registration unit, 197: Display control unit, 198A: Flat Surface detection unit, 198B: Event detection unit, 199: Selection information acquisition unit, 250: Storage unit (of the first server device), 290: Control unit (of the first server device), 401: Boat race field, 402a to 402f : Boat, 501: Race field object, 502 (502a to 502f): Moving object, 1201: Effect image, 1301: Movement trajectory image
 

Claims (9)

1. A program executed in a first computer equipped with a processor and an image pickup unit.
   The program is attached to the processor.
   The step of receiving the first information about a given race in the real world from the second computer,
   A step of generating a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race based on the first information.
   The step of superimposing the virtual object on the real image around the first computer captured by the imaging unit and displaying the virtual object is executed.
   The first information includes identification information of the contestant or the moving body and color information associated with the identification information.
   The generation step comprises generating the moving object, which is at least partially colored with a predetermined color, based on the color information.
   program.
2. The correspondence between the identification information and the color information is changed based on the designated color corresponding to the frame number of the predetermined race.
   The program according to claim 1.
3. The program according to claim 2, wherein the designated color is changed by the attribute of the predetermined race.
4. The first information further includes color position information for designating the position of color registration to the moving object.
   The color position information is changed according to the shape of the moving object and at least one of the attributes of the predetermined race.
   The program according to any one of claims 1 to 3.
5. In the display step, the moving object is moved on the race field object based on the first information, the predetermined race is virtually displayed, and at least one of the race field object and the moving object is displayed. Display the effect on
   Based on the color information, at least a part of the effect display is colored with the predetermined color.
   The program according to any one of claims 1 to 4.
6. The program according to claim 5, wherein the effect display is a display showing a movement locus of the moving object on the race field object.
7. An information processing method executed in a first computer equipped with a processor and an image pickup unit.
   The information processing method is applied to the processor.
   The step of receiving the first information about a given race in the real world from the second computer,
   A step of generating a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race based on the first information.
   The step of superimposing and displaying the virtual object on the real image around the first computer captured by the imaging unit is included.
   The first information includes the identification information of the moving object and the color information associated with the identification information.
   The generation step comprises generating the moving object, which is at least partially colored with a predetermined color, based on the color information.
   Information processing method.
8. An information processing device including a processor and an image pickup unit, wherein the processor is
   Receive the first information about a given race in the real world from the second computer,
   Based on the first information, a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race is generated.
   The virtual object is superimposed and displayed on a real image around the information processing apparatus captured by the image pickup unit.
   The first information includes the identification information of the moving object and the color information associated with the identification information.
   Creating the virtual object comprises creating the moving object, which is at least partially colored with a predetermined color, based on the color information.
   Information processing equipment.
9. There is a system including a first computer provided with a first processor and an image pickup apparatus, and a second computer provided with a second processor and capable of communicating with the first computer.
   The second processor is
   The first processor acquires the first information about a predetermined race in the real world and transmits the first information to the first computer.
   The first information is received from the second computer,
   Based on the first information, a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race is generated.
   The virtual object is superimposed and displayed on a real image around the first computer captured by the image pickup device.
   The first information includes the identification information of the moving object and the color information associated with the identification information.
   Creating the virtual object comprises creating the moving object, which is at least partially colored with a predetermined color, based on the color information.
   system.
   

Images