WO2022014303A1 - Program, information processing method, information processing device, and system - Google Patents

Abstract

A program executed by a first computer causes a processor to execute: a step for receiving first information relating to a race in the real world from a second computer; a step for generating, on the basis of the first information, a virtual object for presenting second information relating to the race to a user; and a step for displaying the virtual object superimposed on a real image imaged by an imaging unit. The first information includes at least the race time information of a contestant or a moving vehicle in a race, and the virtual object includes at least a racecourse object 501 and a moving object 502. In the displaying step, a race is virtually displayed moving the moving object 502 on the racecourse object 501 on the basis of the race time information.

Description

Programs, information processing methods, information processing equipment, and systems

The present disclosure relates to programs, information processing methods, information processing devices, and systems.

Patent Documents 1 to 5 disclose techniques related to AR (Augmented Reality).

Japanese Unexamined Patent Publication No. 2020-58658 Special Table 2020-509505 Gazette Japanese Patent No. 6640294 Japanese Patent No. 6366808 Japanese Unexamined Patent Publication No. 2020-77187

By the way, in the real world, various races such as boat races and horse races are carried out. Some people actually go to the race track to watch these races, but it may not be possible due to time or geographical constraints.

From the above situation, if the real world race can be displayed as a virtual race using a virtual object linked to the real world race, you can watch the race in a simulated manner without actually going to the race track. Can be beneficial. However, the technology disclosed in Patent Documents 1 to 5 superimposes and displays a virtual object unrelated to the real world on an image of the real world, and makes it possible to observe an actual race in a pseudo manner. It wasn't.

One aspect of the present disclosure is to display a real world race as a virtual race using a virtual object linked to the real world race.

According to one embodiment presented in the present disclosure.
A program executed in a first computer equipped with a processor and an image pickup unit.
The program is attached to the processor.
The step of receiving the first information about a given race in the real world from the second computer,
A step of generating a virtual object for presenting the second information about the predetermined race to the user of the first computer based on the first information, and
The step of superimposing the virtual object on the real image around the first computer captured by the image pickup unit and displaying the virtual object is executed.
The first information includes at least race time information indicating the race time of a contestant or a moving body in the predetermined race.
The virtual object includes at least a race field object representing the race field and a moving object representing the contestant or the moving object.
In the display step, the moving object is moved on the race field object based on the race time information, and the predetermined race is virtually displayed.
The program is offered.

According to one embodiment shown in the present disclosure, a real world race can be displayed as a virtual race by using a virtual object linked to the real world race.

It is a figure which shows the configuration example of the system which follows a certain embodiment. It is a block diagram which shows an example of the functional configuration of the user terminal according to a certain embodiment. It is a block diagram which shows an example of the functional configuration of the server according to a certain embodiment. It is a schematic diagram which shows an example of the race field of the real world according to a certain embodiment. It is a schematic diagram which shows an example of the virtual object displayed on the user terminal according to a certain embodiment. It is a flowchart which shows an example of the display control processing according to a certain embodiment. It is a schematic diagram which shows an example of the real image which was taken by the image pickup part according to a certain embodiment. It is a schematic diagram which shows an example of the screen which superposed virtual object on the real image which follows a certain embodiment. It is a schematic diagram which shows an example of the arrangement method of the race field object according to a certain embodiment. It is a flowchart which shows an example of the race result display processing according to a certain embodiment. It is a schematic diagram which shows an example of the screen which superposed virtual object on the real image which follows a certain embodiment.

Hereinafter, embodiments of this technical idea will be described in detail with reference to the drawings. In the following description, the same elements are designated by the same reference numerals, and duplicate description will be omitted as appropriate. In one or more embodiments shown in the present disclosure, the elements included in each embodiment may be combined with each other, and the combined result shall also form part of the embodiments shown in the present disclosure.

(System configuration)
FIG. 1 is a diagram showing a configuration of a system 1 according to the present embodiment. The system 1 can display, for example, a predetermined race performed in the real world as a virtual race using a virtual object on an information processing device used by a user. In the present specification, the "predetermined race" is not particularly limited as long as it is a race carried out in the real world, and for example, a boat race (actual race or exhibition race), horse racing, bicycle racing, auto racing, etc. Car races such as F1, drone races, dog races, marathons, ekiden, etc. can be mentioned.

As shown in FIG. 1, the system 1 is a user such as a user terminal 10A, a user terminal 10B, and a user terminal 10C (hereinafter, user terminals 10A, 10B, 10C) which are information processing devices (first computers) used by each user. It includes a plurality of user terminals 10 such as (generally referred to as "user terminal 10"), a first server device (second computer) 20, a second server device 40, and a network 30.

The user terminal 10A and the user terminal 10B are connected to the network 30 by communicating with the radio base station 31. The user terminal 10C connects to the network 30 by communicating with a wireless router 32 installed in a facility such as a house. The user terminal 10 is, for example, a portable terminal provided with a touch screen, and may be a smartphone, a phablet, a tablet, or the like.

The user terminal 10 executes, for example, a program installed via a platform for distributing an application or the like, or a program including pre-installed website browsing software. The user terminal 10 communicates with the first server device 20 by executing the above program, and transmits / receives data related to a predetermined race, data related to the user, and the like to / from the first server device 20. It is possible to display a virtual race on the terminal 10.

The first server device 20 receives data related to a predetermined race from the second server device 40. The first server device 20 appropriately transmits data related to a predetermined race to the user terminal 10. The first server device 20 stores and manages data related to a predetermined race and data related to each user.

The first server device 20 includes a communication IF (Interface) 22, an input / output IF 23, a memory 25, a storage 26, and a processor (second processor) 29 as hardware configurations, and these provide a communication bus. They are connected to each other via.

The communication IF 22 supports various communication standards such as LAN (Local Area Network) standards, and functions as an interface for transmitting and receiving data to and from the user terminal 10 and the second server device 40.

The input / output IF 23 functions as an interface for accepting input of information to the first server device 20 and outputting information to the outside of the first server device 20. The input / output IF 23 may include an input receiving unit that accepts the connection of an information input device such as a mouse and a keyboard, and an output unit that accepts the connection of an information output device such as a display for displaying an image or the like.

The memory 25 is a storage device for storing data or the like used for processing. The memory 25, for example, provides the processor 29 with a work area for temporary use when the processor 29 performs processing. The memory 25 includes a storage device such as a ROM (Read Only Memory) and a RAM (Random Access Memory).

The storage 26 is a storage device for storing various programs and data for the processor 29 to read and execute. The information stored in the storage 26 includes data related to a predetermined race, data related to each user, and the like. The storage 26 may be configured to include a storage device such as an HDD (Hard Disk Drive) or a flash memory. The storage is not limited to the form included in the server device, and a cloud service can also be used.

The processor 29 controls the operation of the first server device 20 by reading and executing a program or the like stored in the storage 26. The processor 29 may be configured to include, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), and the like.

The second server device 40 stores and manages data related to a predetermined race. The second server device 40 is, for example, a server device managed by a predetermined race organizer or a server device managed by an organization (such as a publisher of a race magazine) that transmits information about a predetermined race to the outside. The second server device 40 appropriately transmits data related to a predetermined race to the first server device 20. In certain aspects, the second server device 40 may transmit data related to a predetermined race to the user terminal 10. The hardware configuration of the second server device 40 may be the same as that of the first server device 20 as long as there is no contradiction.

(User terminal)
FIG. 2 is a block diagram showing an example of the functional configuration of the user terminal 10. As shown in FIG. 2, the user terminal 10 includes an antenna 110, a wireless communication IF 120, a touch screen 130, an input / output IF 140, a storage unit 150, a voice processing unit 160, a microphone 161 and a speaker 162. It includes an image pickup unit 170 and a control unit (first processor) 190.

The antenna 110 radiates a signal emitted by the user terminal 10 into space as a radio wave. Further, the antenna 110 receives radio waves from the space and gives a received signal to the wireless communication IF 120.

Since the user terminal 10 communicates with other communication devices, the wireless communication IF 120 performs modulation / demodulation processing for transmitting / receiving signals via the antenna 110 or the like. The wireless communication IF 120 is a communication module for wireless communication including a tuner, a high frequency circuit, etc., performs modulation / demodulation and frequency conversion of the wireless signal transmitted / received by the user terminal 10, and supplies the received signal to the control unit 190.

The touch screen 130 receives input from the user and outputs information to the user on the display 132. The touch screen 130 includes a touch panel 131 for receiving user operation input and a display 132. The touch panel 131 detects that a user's finger or the like has approached by using, for example, a capacitance type touch panel. The display 132 is realized by, for example, an LCD (Liquid Crystal Display), an organic EL (electroluminescence), or other display device.

The input / output IF 140 functions as an interface for accepting input of information to the user terminal 10 and outputting information to the outside of the user terminal 10.

The storage unit 150 is composed of a flash memory, an HDD, or the like, and stores a program used by the user terminal 10 and various data received by the user terminal 10 from the first server device 20 or the like.

The voice processing unit 160 performs modulation / demodulation of the voice signal. The voice processing unit 160 modulates the signal given from the microphone 161 and gives the modulated signal to the control unit 190. Further, the voice processing unit 160 gives a voice signal to the speaker 162. The voice processing unit 160 is realized by, for example, a processor for voice processing. The microphone 161 functions as an audio input unit for receiving an input of an audio signal and outputting it to the control unit 190. The speaker 162 functions as an audio output unit for outputting an audio signal to the outside of the user terminal 10.

The image pickup unit 170 is a camera that captures a real image around the user terminal 10. The image captured by the image pickup unit 170 is image-processed by the control unit 190 and output to the display 132.

The control unit 190 controls the operation of the user terminal 10 by reading and executing a program stored in the storage unit 150. The control unit 190 is realized by, for example, an application processor.

The process of executing the application program 151 by the control unit 190 will be described in more detail. The storage unit 150 stores the application program 151, the application information 152, and the user information 153.

For example, the user terminal 10 downloads the application program 151 from the first server device 20 and stores it in the storage unit 150. Further, the user terminal 10 communicates with the first server device 20 to send and receive various data such as application information 152 and user information 153 to and from the first server device 20.

The application program 151 is a program for displaying a virtual race on the user terminal 10. The application information 152 includes various data referred to by the application program 151. The application information 152 includes the first information 152A.

The first information 152A is information regarding a predetermined race transmitted from the first server device 20 and the second server device 40. The first information 152A includes, for example, race time information indicating the race time of a contestant or a moving body (hereinafter, also referred to as “participant or the like”) of a predetermined race, position information of the contestant or the like during the execution of the predetermined race, and the like. The time information corresponding to the position information and the time information are included.

As used herein, the term "contestant" is a concept that includes not only humans but also animals such as horses and dogs. Further, the "moving body" is a main body of movement in a predetermined race, and is an animal or an aircraft on which a participant rides, an aircraft remotely controlled by a contestant, or the like. In marathons, dog races, etc., "participants" and "mobiles" are the same.

In addition to the above, the first information 152A includes, for example, a predetermined race name, date and time, race field data, contestant data, moving object data, odds information, race prediction, race start table, and information immediately before the race. It may include pit reports, race results, race videos, race stills, past race information, and other information that may be published in information magazines or information sites about a given race.

The user information 153 includes information about the user of the user terminal 10. The user information 153 includes, for example, information for identifying the user, position information of the user terminal 10, a race purchase history and a hit rate of the user (for example, in the case of a boat race, the history of the purchased boat ticket and the purchased boat ticket. Hit rate) and the like may be included.

By reading and executing the application program 151, the control unit 190 reads and executes the operation input reception unit 191, the transmission / reception unit 192, the object generation unit 193, the display control unit 194, the detection unit 195, and the selection information acquisition unit 196. And, each function is demonstrated.

The operation input receiving unit 191 accepts the user's operation input based on the output of the touch screen 130. Specifically, the operation input receiving unit 191 detects that the user's finger or the like touches or approaches the touch panel 131 as the coordinates of the coordinate system consisting of the horizontal axis and the vertical axis of the surface constituting the touch screen 130.

The operation input receiving unit 191 determines the user's operation on the touch screen 130. The operation input reception unit 191 is, for example, "approach operation", "release operation", "tap operation", "double tap operation", "long press operation (long touch operation)", "drag operation (swipe operation)", Determine the user's operation such as "move operation", "flick operation", "pinch-in operation", and "pinch-out operation".

The operation input receiving unit 191 may accept the movement of the user terminal 10 detected by the acceleration sensor or the like mounted on the user terminal 10 as an operation input.

The transmission / reception unit 192 transmits and receives various information to and from external communication devices such as the first server device 20 and the second server device 40 via the wireless communication IF 120 and the network 30. The transmission / reception unit 192 receives, for example, the first information 152A from the first server device 20 or the second server device 40. Further, the transmission / reception unit 192 transmits, for example, information corresponding to the operation input received by the operation input reception unit 191 and information stored in the user information 153 to the first server device 20 or the second server device 40. ..

The object generation unit 193 generates a virtual object for presenting the second information regarding a predetermined race to the user based on the first information. In the present specification, the "second information" is, for example, at least a part of the first information 152A or information calculated from the first information 152A. In a certain aspect, the "second information" can be information that can be presented as text. In a certain aspect, the "second information" may be ranking information indicating the ranking of the contestants or the like during a predetermined race or the ranking of the contestants or the like as a result of the race.

The object generation unit 193 generates, as virtual objects, a race field object representing a race field and a moving object representing a contestant or the like. The object generation unit 193 may generate a virtual display board for displaying the second information as text. In addition to the above objects, the object generation unit 193 also creates a virtual screen for displaying the second information as an image, various building objects, landscape objects such as trees, objects that serve as user avatars, and the like. May be generated.

The display control unit 194 superimposes an image (hereinafter, also referred to as “superimposed image”) on which a virtual object generated by the object generation unit 193 is superimposed on a real image around the user terminal 10 imaged by the image pickup unit 70. It is displayed on the display 132. The display control unit 194 moves the moving object on the race field object based on the race time information included in the first information 152A, and displays a virtual race that virtually reproduces a predetermined race on the display 132. It is preferable that the display control unit 194 reproduces the virtual race based on the position information of the contestants and the like included in the first information 152A and the time information corresponding to the position information in addition to the race time information. ..

It is preferable that the display control unit 194 can change the viewpoint in the superimposed image according to the operation input received by the operation input reception unit 191. The display control unit 194 displays various menu screens and GUIs (Graphical User Interfaces) on the display 132, and changes the display contents of the display 132, in response to the operation input received by the operation input reception unit 191.

The detection unit 195 detects a flat surface in the real image captured by the image pickup unit 170. The detection of a flat surface is realized by a conventionally known image recognition technique. For example, when the user performs an operation of selecting a flat surface detected by the detection unit 195, a superimposed image in which the race field object is arranged on the flat surface is displayed on the display 132.

The flat surface is preferably a horizontal surface. Further, the angle formed by the flat surface and the bottom surface constituting the race field object may be 0 degrees, but is preferably an acute angle, and can be, for example, in the range of 15 degrees to 45 degrees. The above angle may be adjusted by accepting a user's operation. Also, even if there is a convex part on a part of the flat surface in the real world, or if there is an object on the flat surface, if the convex part or the object is of a size that can be hidden by the racetrack object. , The place where the race field object can be placed may be detected as a flat surface.

The selection information acquisition unit 196 acquires one or more selection information indicating one or more contestants selected by the user. The "selection information" is, for example, information indicating a contestant or the like predicted by the user as a winner or the like of a predetermined race. When the predetermined race is a boat race or a horse race, the "selection information" may be information indicating a boat ticket or a betting ticket purchased by the user.

The selection information acquisition unit 196 acquires selection information based on, for example, a user's operation input on the screen for selecting a contestant or the like. The selection information acquired by the selection information acquisition unit 196 is transmitted to, for example, the first server device 20 or the second server device 40. When the predetermined race is a boat race or a horse race, the purchase process of the boat ticket or the betting ticket may be performed according to the selection information being transmitted to the second server device 40.

(1st server device)
FIG. 3 is a block diagram showing a functional configuration of the first server device 20. A detailed configuration of the first server device 20 will be described with reference to FIG. The first server device 20 exerts functions as a communication unit 220, a storage unit 250, and a control unit 290 by operating according to a program.

The communication unit 220 functions as an interface for the first server device 20 to communicate with an external communication device such as the user terminal 10 or the second server device 40 via the network 30.

The storage unit 250 stores various programs and data for realizing the system 1. In a certain aspect, the storage unit 250 stores the program 251 and the race information 252 and the user information 253.

The program 251 is a program for the first server device 20 to communicate with the user terminal 10 and the second server device 40 to realize the system 1. The program 251 is executed by the control unit 290 to send and receive data to and from the user terminal 10 and the second server device 40, a process according to the operation content performed by the user of the user terminal 10, race information 252, and the user. The first server device 20 is made to perform a process of updating the information 253 and the like.

Race information 252 contains various data related to a given race. The race information 252 includes, for example, the first information 252A. The first information 252A is the source information of the first information 152A, and the first information 152A can be a part of the first information 252A. The first information 252A is, for example, information acquired from the second server device 40.

The user information 253 is information about the user of the user terminal 10. The user information 253 includes a user management table 253A. The user management table 253A stores, for example, information for identifying the user, position information of the user terminal 10, a race purchase history of the user, a hit rate, and the like for each user.

The control unit 290 is realized by the processor 29, and by executing the program 251, functions as a transmission / reception unit 291, a first information acquisition unit 292, a selection information acquisition unit 293, a data management unit 294, and a timekeeping unit 295. ..

The transmission / reception unit 291 transmits and receives various information to and from an external communication device such as a user terminal 10 and a second server device 40 via the communication unit 220 and the network 30. The transmission / reception unit 291 transmits, for example, at least a part of the first information 252A to the user terminal 10. Further, the transmission / reception unit 291 receives, for example, the first information 252A from the second server device 40.

The first information acquisition unit 292 acquires the first information 252A from the second server device 40 via the transmission / reception unit 291. The selection information acquisition unit 293 acquires the user's selection information from the user terminal 10 or the second server device 40 via the transmission / reception unit 291. The data management unit 294 performs a process of updating various data stored in the storage unit 250 according to the processing results of the first information acquisition unit 292, the selection information acquisition unit 293, and the like. The timekeeping unit 295 performs a process of measuring time. Various times displayed on the user terminal 10 (for example, the time until the start of the race) can be controlled based on the time measured by the time measuring unit 295.

(Operation example)
Next, an operation example in the system 1 will be described with reference to FIGS. 4 to 11. In the following, a case where a predetermined race is a boat race will be described as an example, but the following description can be applied even if the predetermined race is another race. Further, in the following description, it is assumed that data is transmitted / received between the user terminal 10 and the first server device 20 and between the first server device 20 and the second server device 40, but in a certain aspect, the user The terminal 10 and the second server device 40 may be configured to directly transmit and receive data.

FIG. 4 is a schematic diagram showing an example of a boat race field in the real world. Two turn marks 403 are installed in the boat race field 401, and races are carried out by boats 402a to 402f on which each boat racer rides. When the race is completed, the race time information indicating the race times of the boats 402a to 402f is transmitted from the second server device 40 to the first server device 20, and is transmitted from the first server device 20 to the user terminal 10.

The boat race field 401 is provided with image pickup devices (cameras) 404a to 404b. The image pickup apparatus 404a captures the boats 402a to 402f in the field of view from above the boat race field 401. The image pickup apparatus 404b keeps the boats 402a to 402f in the field of view from the side of the boat race field 401. The images of the boats 402a to 402f captured by the image pickup devices 404a to 404b are transmitted to the second server device 40. In the second server device 40, for example, image analysis of each image is performed, and position information indicating the position of each of the boats 402a to 402f in the shooting time of each image is calculated. The calculated position information and the time information related to the shooting time corresponding to the position information are transmitted to the first server device 20, and are transmitted from the first server device 20 to the user terminal 10. The location information may be calculated by the first server device 20.

Further, a position sensor such as a GPS sensor may be installed on the boats 402a to 402f instead of or in addition to the image pickup devices 404a to 404b. The position information of the boats 402a to 402f acquired by the position sensor and the time information indicating the time when the position information was acquired are finally transmitted to the user terminal 10.

FIG. 5 is a schematic diagram showing an example of a virtual object displayed on the user terminal 10. In the example of FIG. 5, as virtual objects, a race field object 501, moving objects 502a to 502f, two turnmark objects 503, and a virtual display board 505 are shown.

The race field object 501 is an object that virtually displays the boat race field 401. The race field object 501 and the turn mark object 503 are preferably created based on race field data such as course information of the boat race field 401, and preferably have a shape corresponding to the boat race field 401.

The moving objects 502a to 502f are objects that virtually display the boats 402a to 402f, respectively, and have a shape imitating a boat. The moving objects 502a to 502f move the race field object 501 based on the race time information, the position information of the boats 402a to 402f, and the time information corresponding to the position information. That is, the race field object 501 and the moving objects 502a to 502f display the race in the real world as a virtual race on the user terminal 10.

Even if there is no position information and time information of the boats 402a to 402f, it is possible to display the virtual race only by the race time information. However, in this case, the final order of arrival will be the same as in the real world race, but it is difficult to reproduce the race progress such as the ranking during the race.

The virtual display board 505 is an object that displays text information. The virtual display board 505 is, for example, a corresponding nonexistent object in the boat race field 401. The text information displayed on the virtual display board 505 is not particularly limited, and may be, for example, ranking information, odds information, or the like. Further, the text information displayed on the virtual display board 505 may be changeable based on the operation input of the user.

FIG. 6 is a flowchart showing an example of the display control process. The order of the processes constituting the flowchart (including FIG. 10) described below is random as long as the processing contents do not have any contradiction or inconsistency. Further, the processing executed by each device may be executed by another device within a range that does not cause a contradiction.

The processes shown in FIGS. 6 and 10 are realized by the control unit 190 executing the application program 151 and the control unit 290 executing the program 251. First, in step S610, the control unit 190 activates the image pickup unit 170, which is a camera. The image pickup unit 170 captures a real image around the user terminal 10.

In step S620, the control unit 190 detects a flat surface in the image captured by the image pickup unit 170. In step S630, the control unit 190 arranges the virtual object on the detected flat surface.

Here, the processes of steps S610, S620, and S630 will be specifically described with reference to FIGS. 7 to 9. FIG. 7 is a schematic diagram showing an example of a real image captured by the imaging unit 170. In the example of FIG. 7, a keyboard 702 and a monitoring device 703 are placed on a flat desk 701.

When the image pickup unit 170 is activated in step S610, the real image captured by the image pickup unit 170 is displayed on the display 132. Next, in step S620, the control unit 190 detects a flat surface in the image captured by the image pickup unit 170, that is, in the image displayed on the display 132.

In FIG. 7, the region 704 is detected as a flat surface. There is a keyboard 702 in the area 704, but since the keyboard 702 is small enough to be hidden by the race field object 501, the control unit 190 detects the area 704 as a flat surface.

If the position imaged by the image pickup unit 170 is changed in the state shown in FIG. 7, the position of the region 704 can also be changed. The area 704 is displayed on the display 132 so as to be distinguishable from other parts, for example, by adding a predetermined color. When the user executes a tap operation or the like on the area 704, in step S630, the control unit 190 arranges a virtual object such as a race field object 501 on the area 704.

FIG. 8 is a schematic diagram showing an example of a screen in which a virtual object is superimposed on a real image and displayed. In FIG. 8, the area with the dot pattern including the monitor device 703 is the real image, and the other area is the area where the virtual object is displayed. For example, an advertisement image may be displayed in an area where the virtual object is not displayed.

In FIG. 8, as virtual objects, a race field object 501, a plurality of moving objects 502, two turnmark objects 503, a large monitor object 506, building objects 507a to 507b, and other reference numerals are not attached. A large number of objects (tree objects, clock objects, etc.) and are displayed. These objects are created, for example, based on the first information 152A received from the first server device 20.

In FIG. 8, the race field object 501 is arranged on the area 704 so that the angle formed by the flat surface included in the area 704 and the bottom surface constituting the race field object 501 is 0 degree. In this case, for example, an area in the depth direction such as the area X1 or moving objects 502 may overlap each other and become difficult to see. Further, in the area on the depth direction side of the large monitor object 506 or the like, a blind spot may occur in which the moving object 502 cannot be seen.

However, in the state shown in FIG. 8, the user can change the viewpoint of the superimposed image displayed on the display 132. For example, when the user inputs an operation such as a pinch-in operation or a pinch-out operation, the viewpoint position of the image pickup unit 170 may move toward or away from the race field object 501. Further, even when the user moves in the depth direction of the display 132 while holding the user terminal 10 or moves in the front direction, the viewpoint position of the image pickup unit 170 may move toward or away from the race field object 501. Further, when the user performs an operation such as designating a predetermined position on the display 132, the viewpoint position can be moved to the designated position. By such a viewpoint change, the above-mentioned visibility problem can be solved.

FIG. 9 is a schematic diagram showing an example of a method of arranging the race field object 501. In the example of FIG. 9, the race field object 501 is arranged so that the angle θ formed by the flat surface S1 and the bottom surface S2 constituting the race field object 501 is an acute angle. Such an arrangement can also alleviate the above-mentioned problems related to visibility.

Returning to the description of the flowchart of FIG. In step S640, the control unit 190 acquires the position information of the boats 402a to 402f in the boat race field 401 in the real world. That is, when the race by the boats 402a to 402f is started in the real world, the control unit 190 acquires the position information and the time information of the boats 402a to 402f from the first server device 20. The method of acquiring the position information and the time information is as described with reference to FIG.

In step S650, the control unit 190 controls so that the position information acquired in step S640 and the moving object are interlocked with each other. Specifically, using the time information and the position information, the movements of the boats 402a to 402f on the race field object 501 are controlled to be the same as the movement objects 502a to 502f on the boat race field 401.

The process of steps S640 and S650 is repeated at least from the start to the end of the race in the real world, but may be repeated before and after the start and end of the race in the real world.

If the race displayed as a virtual race is a past race, the position information of the boats 402a to 402f from the start to the end of the race may be collectively acquired before the start of the virtual race. Further, the virtual race may be displayed by acquiring only the race time information without acquiring the position information and the like. The race displayed as a virtual race may be an exhibition race.

After the end of the race in the real world, in step S660, the control unit 190 executes the result display process and ends a series of display control processes in response to receiving an operation input for terminating the application program 151.

FIG. 10 is a flowchart showing an example of the race result display process in step S660 described above. First, in step S661, the control unit 190 acquires the ranks of the moving objects 502a to 502f at the end of the race. The rank of the moving objects 502a to 502f is, for example, a rank on the virtual race reproduced based on the position information, the time information, and the race time information of the boats 402a to 402f.

Next, in step S662, the control unit 190 acquires the data of the ranking determined in the real world race from the first server device 20. The first server device 20 acquires the order data from the second server device 40.

Next, in step S663, the control unit 190 determines whether or not the rank acquired in step S661 and the rank data acquired in step S662 match.

When the rank acquired in step S661 and the rank data acquired in step S662 match (Yes in step S663), in step S664, the control unit 190 displays the rank on the virtual display board 505.

On the other hand, when the rank acquired in step S661 and the rank data acquired in step S662 do not match (No in step S663), in step S665, the control unit 190 acquired in step S662 together with the alert display. The ranking is displayed on the virtual display board 505. The case where the rank acquired in step S661 and the rank data acquired in step S662 do not match is, for example, a case where a disqualified person appears in a boat race in the real world.

FIG. 11 is a schematic diagram showing an example of a screen in which a virtual object is superimposed on a real image and displayed. Specifically, FIG. 11 is a schematic diagram showing a superimposed image after the race is completed in the real world.

In FIG. 11, in addition to the example of FIG. 8, a virtual display board 505, a virtual screen 508, a purchase information display board 509, and a ranking display board 510 are displayed as virtual objects. It is preferable that these virtual objects can be switched to non-display, the display position can be changed, the display contents can be changed, and the like by the user's operation input.

Text information such as the order of the moving objects 502a to 502f and the race time is displayed on the virtual display board 505. The ranking information displayed on the virtual display board 505 is displayed as temporary information until the processing of step S663 is performed, and is displayed as confirmation information after the processing of step S663, for example. Before the start of the race or during the race, other text information such as odds information may be displayed on the virtual display board 505.

The image of the boat race in the real world is displayed on the virtual screen 508. What is displayed on the virtual screen 508 is, for example, a video showing the state of each boat after the race, a video of an interview with a player, a video of a replay of the race, and the like. Before the start of the race or during the race, the virtual screen 508 may display an image of the exhibition race, a real-time image of the race, or the like.

The purchase information display board 509 displays information on the boat ticket purchased by the user acquired by the control unit 190 (selection information acquisition unit 196). The boat ticket purchased by the user is displayed on the right side of the purchase information display board 509, and the refund amount when each boat ticket hits is displayed on the left side. Further, the mark 509a is attached to the hit boat ticket on the purchase information display board 509, and the mark 509a is also displayed on the moving object 502b corresponding to the boat ticket.

Before the start of the race or during the race, the user can select the information of the boat ticket displayed on the purchase information display board 509. For example, when the user selects the boat ticket (five-frame-4 frame double-unit boat ticket) that is possessed third from the top, the mark 509a is displayed in the third column from the top. Further, the mark 509a is attached to the moving objects of the 5th frame and the 4th frame.

The order of the moving objects 502a to 502f is displayed on the order display board 510. During the race, the rank displayed on the rank display plate 510 may change based on the position information or the like acquired in step S640.

Further, in FIG. 11, the contestant information 512b is displayed as text on the upper part of the moving object 502b. In the example of FIG. 11, the frame number and the racer name are displayed as the contestant information 512. It is preferable that the information to be displayed as the contestant information 512 can be selected by the user. The user can also hide the contestant information 512. When displaying the contestant information 512, it may be displayed for all moving objects, or it may be displayed only for a predetermined moving object such as a moving object selected by the user.

The above embodiments are merely examples for facilitating the understanding of the present invention, and are not intended to limit the interpretation of the present invention. It goes without saying that the present invention can be modified and improved without departing from the spirit thereof, and the present invention includes its equivalents.

[Additional notes]
The contents of this disclosure are listed below.

(Item 1)
A program executed in a first computer equipped with a processor and an image pickup unit.
The program is attached to the processor.
The step of receiving the first information about a given race in the real world from the second computer,
A step of generating a virtual object for presenting the second information about the predetermined race to the user of the first computer based on the first information, and
The step of superimposing the virtual object on the real image around the first computer captured by the image pickup unit and displaying the virtual object is executed.
The first information includes at least race time information indicating the race time of a contestant or a moving body in the predetermined race.
The virtual object includes at least a race field object representing the race field and a moving object representing the contestant or the moving object.
In the display step, the moving object is moved on the race field object based on the race time information, and the predetermined race is virtually displayed.
program.
This makes it possible to display real-world races as virtual races using virtual objects. As a result, you can watch the race in a simulated manner without actually going to the race track. In addition, since the place around the user becomes a virtual race track, it is possible to provide the user with unprecedented enjoyment.

(Item 2)
The first information further includes position information of the contestant or the moving body during the execution of the predetermined race, and time information corresponding to the position information.
In the display step, the moving object is moved on the race field object based on the race time information, the position information, and the time information, and the predetermined race is virtually displayed.
The program described in item 1.
As a result, the ranking and offense and defense during the race in the real world can be reproduced on the virtual race, so that the user's satisfaction can be improved.

(Item 3)
It is possible to change the viewpoint of a predetermined race virtually displayed in the step to be displayed based on the operation input of the user.
The program according to item 1 or item 2.
As a result, the virtual race can be watched from various viewpoints, so that the user's satisfaction can be improved. In addition, it is possible to provide users with new enjoyment that they can watch virtual races from a viewpoint that is impossible in the real world race field. In addition, the visibility of the virtual race can be improved.

(Item 4)
The second information includes at least textual information about the predetermined race.
The virtual object includes at least a virtual display board for displaying the text information.
The program according to any one of items 1 to 3.
As a result, it becomes easy to provide information to the user, and the convenience of the user can be improved.

(Item 5)
The text information displayed on the virtual display board includes at least ranking information indicating the ranking of the contestants or the moving body in the predetermined race.
The ranking information varies depending on the situation of the predetermined race that is virtually displayed in the step to be displayed.
The program described in item 4.
This allows the user to easily grasp the ranking of moving objects during a virtual race. As a result, the convenience of the user can be improved.

(Item 6)
The program is applied to the processor and further to the processor.
A step of detecting a flat surface in the real image captured by the image pickup unit is executed.
The display step displays the race field object on the flat surface detected in the detection step in response to the operation input of the user.
The program according to any one of items 1 to 5.
As a result, it is possible to reduce the possibility of causing a sense of discomfort in the superimposed image and improve the visibility of the virtual race.

(Item 7)
The display step displays the race track object so that the angle formed by the flat surface and the bottom surface constituting the race track object is an acute angle.
The program according to item 6.
This makes it possible to further improve the visibility of the virtual race.

(Item 8)
An information processing method executed in a first computer equipped with a processor and an image pickup unit.
The information processing method is applied to the processor.
The step of receiving the first information about a given race in the real world from the second computer,
A step of generating a virtual object for presenting the second information about the predetermined race to the user of the first computer based on the first information, and
A step of superimposing the virtual object on a real image around the first computer captured by the image pickup unit and displaying the virtual object.
Including running
The first information includes at least race time information indicating the race time of a contestant or a moving body in the predetermined race.
The virtual object includes at least a race field object representing the race field and a moving object representing the contestant or the moving object.
In the display step, the moving object is moved on the race field object based on the race time information, and the predetermined race is virtually displayed.
Information processing method.
This makes it possible to display real-world races as virtual races using virtual objects. As a result, you can watch the race in a simulated manner without actually going to the race track. In addition, since the place around the user becomes a virtual race track, it is possible to provide the user with unprecedented enjoyment.

(Item 9)
An information processing device including a processor and an image pickup unit, wherein the processor is
Receive the first information about a given race in the real world from the second computer,
A virtual object for presenting the second information regarding the predetermined race to the user of the information processing apparatus is generated based on the first information.
The virtual object is superimposed and displayed on a real image around the information processing apparatus captured by the image pickup unit.
The first information includes at least race time information indicating the race time of a contestant or a moving body in the predetermined race.
The virtual object includes at least a race field object representing the race field and a moving object representing the contestant or the moving object.
The superimposed display includes moving the moving object on the race field object based on the race time information and virtually displaying the predetermined race.
Information processing equipment.
This makes it possible to display real-world races as virtual races using virtual objects. As a result, you can watch the race in a simulated manner without actually going to the race track. In addition, since the place around the user becomes a virtual race track, it is possible to provide the user with unprecedented enjoyment.

(Item 10)
A system realized by a first computer provided with a first processor and an image pickup unit, and a second computer provided with a second processor and capable of communicating with the first computer.
The second processor is
Acquires the first information about a predetermined race in the real world, and transmits the first information to the first computer.
The first processor is
The first information is received from the second computer,
A virtual object for presenting the second information regarding the predetermined race to the user of the first computer is generated based on the first information.
The virtual object is superimposed and displayed on a real image around the first computer captured by the image pickup unit.
The first information includes at least race time information indicating the race time of a contestant or a moving body in the predetermined race.
The virtual object includes at least a race field object representing the race field and a moving object representing the contestant or the moving object.
The superimposed display includes moving the moving object on the race field object based on the race time information and virtually displaying the predetermined race.
system.
This makes it possible to display real-world races as virtual races using virtual objects. As a result, you can watch the race in a simulated manner without actually going to the race track. In addition, since the place around the user becomes a virtual race track, it is possible to provide the user with unprecedented enjoyment.

(Item 11)
The first information further includes position information of the contestant or the moving body during the execution of the predetermined race, and time information corresponding to the position information.
The superimposed display means that the moving object is moved on the race field object based on the race time information, the position information, and the time information, and the predetermined race is virtually displayed. Including,
The position information is acquired based on the position sensor provided on the contestant or the moving body, or the image of the predetermined race captured by the image pickup device during the execution of the predetermined race.
The system according to item 10.
This makes it easy to obtain the position information of the contestants and the like. As a result, the ranking and offense and defense during the race in the real world can be reproduced on the virtual race, so that the user's satisfaction can be improved.

1: System, 10: User terminal, 20: First server device, 30: Network, 40: Second server device, 130: Touch screen, 150: Storage unit (of user terminal), 190: Control (of user terminal) Unit, 191: Operation input reception unit, 192: Transmission / reception unit, 193: Object generation unit, 194: Display control unit, 195: Detection unit, 196: Selection information acquisition unit, 250: Storage unit (of the first server device), 290: Control unit (of the first server device)

Claims (11)

1. A program executed in a first computer equipped with a processor and an image pickup unit.
   The program is attached to the processor.
   The step of receiving the first information about a given race in the real world from the second computer,
   A step of generating a virtual object for presenting the second information about the predetermined race to the user of the first computer based on the first information, and
   The step of superimposing the virtual object on the real image around the first computer captured by the image pickup unit and displaying the virtual object is executed.
   The first information includes at least race time information indicating the race time of a contestant or a moving body in the predetermined race.
   The virtual object includes at least a race field object representing the race field and a moving object representing the contestant or the moving object.
   In the display step, the moving object is moved on the race field object based on the race time information, and the predetermined race is virtually displayed.
   program.
2. The first information further includes position information of the contestant or the moving body during the execution of the predetermined race, and time information corresponding to the position information.
   In the display step, the moving object is moved on the race field object based on the race time information, the position information, and the time information, and the predetermined race is virtually displayed.
   The program according to claim 1.
3. It is possible to change the viewpoint of a predetermined race virtually displayed in the step to be displayed based on the operation input of the user.
   The program according to claim 1 or 2.
4. The second information includes at least textual information about the predetermined race.
   The virtual object includes at least a virtual display board for displaying the text information.
   The program according to any one of claims 1 to 3.
5. The text information displayed on the virtual display board includes at least ranking information indicating the ranking of the contestants or the moving body in the predetermined race.
   The ranking information varies depending on the situation of the predetermined race that is virtually displayed in the step to be displayed.
   The program according to claim 4.
6. The program is applied to the processor and further to the processor.
   A step of detecting a flat surface in the real image captured by the image pickup unit is executed.
   The display step displays the race field object on the flat surface detected in the detection step in response to the operation input of the user.
   The program according to any one of claims 1 to 5.
7. The program according to claim 6, wherein the display step displays the race track object so that the angle formed by the flat surface and the bottom surface constituting the race track object is an acute angle.
8. An information processing method executed in a first computer equipped with a processor and an image pickup unit.
   The information processing method is applied to the processor.
   The step of receiving the first information about a given race in the real world from the second computer,
   A step of generating a virtual object for presenting the second information about the predetermined race to the user of the first computer based on the first information, and
   A step of superimposing the virtual object on a real image around the first computer captured by the image pickup unit and displaying the virtual object.
   Including running
   The first information includes at least race time information indicating the race time of a contestant or a moving body in the predetermined race.
   The virtual object includes at least a race field object representing the race field and a moving object representing the contestant or the moving object.
   In the display step, the moving object is moved on the race field object based on the race time information, and the predetermined race is virtually displayed.
   Information processing method.
9. An information processing device including a processor and an image pickup unit, wherein the processor is
   Receive the first information about a given race in the real world from the second computer,
   A virtual object for presenting the second information regarding the predetermined race to the user of the information processing apparatus is generated based on the first information.
   The virtual object is superimposed and displayed on a real image around the information processing apparatus captured by the image pickup unit.
   The first information includes at least race time information indicating the race time of a contestant or a moving body in the predetermined race.
   The virtual object includes at least a race field object representing the race field and a moving object representing the contestant or the moving object.
   The superimposed display includes moving the moving object on the race field object based on the race time information and virtually displaying the predetermined race.
   Information processing equipment.
10. A system realized by a first computer provided with a first processor and an image pickup unit, and a second computer provided with a second processor and capable of communicating with the first computer.
    The second processor is
    Acquires the first information about a predetermined race in the real world, and transmits the first information to the first computer.
    The first processor is
    The first information is received from the second computer,
    A virtual object for presenting the second information regarding the predetermined race to the user of the first computer is generated based on the first information.
    The virtual object is superimposed and displayed on a real image around the first computer captured by the image pickup unit.
    The first information includes at least race time information indicating the race time of a contestant or a moving body in the predetermined race.
    The virtual object includes at least a race field object representing the race field and a moving object representing the contestant or the moving object.
    The superimposed display includes moving the moving object on the race field object based on the race time information and virtually displaying the predetermined race.
    system.
11. The first information further includes position information of the contestant or the moving body during the execution of the predetermined race, and time information corresponding to the position information.
    The superimposed display means that the moving object is moved on the race field object based on the race time information, the position information, and the time information, and the predetermined race is virtually displayed. Including,
    The position information is acquired based on the position sensor provided on the contestant or the moving body, or the image of the predetermined race captured by the image pickup device during the execution of the predetermined race.
    The system according to claim 10.

Images