JP2022073925A - Program, information processing method, information processing device and system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enliven watching of a virtual race by voice relating to the race.

SOLUTION: A program causes a processor to execute the steps of: receiving first information relating to a prescribed race in an actual world from a second computer; generating virtual objects including a racecourse object and moving objects based on the first information; superimposedly displaying the virtual objects on the actual image; and receiving voice information relating to the prescribed race from the second computer to output the voice based on the voice information. The first information includes information on positions of participants and the like during execution of the prescribed race and time information corresponding to the position information. The display step includes virtually displaying the prescribed race based on the first information, and the output step includes outputting the voice along with the virtual display of the prescribed race in the display step.

SELECTED DRAWING: Figure 9

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present disclosure relates to programs, information processing methods, information processing devices, and systems.

Patent Document 1 discloses that when providing a virtual reality to a user by using an HMD (Head Mounted Display), it is possible to output voice from the HMD and to enable voice input by a microphone. Patent Document 2 discloses a technique for performing voice communication between users in a virtual reality space.

Patent Document 3 discloses that when an augmented reality content is provided to a user using an AR (Augmented Reality) glass or the like, an audio parameter included in the augmented reality content is adjusted and output. In Patent Document 4, when the extended real space or the composite real space is provided to the user by using the HMD, a synthetic voice obtained by synthesizing the voice of the virtual reality space and the voice of the real space around the user is output. Is disclosed.

Japanese Unexamined Patent Publication No. 11-184584 Japanese Unexamined Patent Publication No. 2017-28390 International Publication No. 2019/142695 International Publication No. 2018/128161

By the way, in the real world, various races such as boat races and horse races are carried out. Some people actually go to the race track to watch these races, but it may not be possible due to time or geographical constraints.

From the above situation, if the real world race can be displayed as a virtual race using a virtual object linked to the real world race, you can watch the race in a simulated manner without actually going to the race track. Can be beneficial.

In addition, when displaying a virtual race, if the voice related to the race can excite the watching of the virtual race, it is possible to provide the user with further enjoyment. However, the techniques disclosed in Patent Documents 1 to 4 do not display the race in the real world as a virtual race, and naturally, the voice related to the race does not excite the watching of the virtual race.

One aspect of the present disclosure is intended to enliven the spectator of a virtual race with audio related to the race.

According to one embodiment presented in the present disclosure.
A program executed in a first computer equipped with a processor and an image pickup unit.
The program is attached to the processor.
The step of receiving the first information about a given race in the real world from the second computer,
A step of generating a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race based on the first information.
A step of superimposing the virtual object on a real image around the first computer captured by the image pickup unit and displaying the virtual object.
A step of receiving voice information related to the predetermined race from the second computer and outputting voice based on the voice information is executed.
The first information includes the position information of the contestant or the moving body during the execution of the predetermined race, and the time information corresponding to the position information.
In the display step, the moving object is moved on the race field object based on the first information, and the predetermined race is virtually displayed.
In the output step, the sound is output together with the virtual display of the predetermined race in the display step.
The program is offered.

According to one embodiment shown in the present disclosure, the watching of a virtual race can be enlivened by the voice related to the race.

It is a figure which shows the configuration example of the system which follows a certain embodiment. It is a block diagram which shows an example of the functional configuration of the user terminal according to a certain embodiment. It is a block diagram which shows an example of the functional configuration of the server according to a certain embodiment. It is a schematic diagram which shows an example of the race field of the real world according to a certain embodiment. It is a schematic diagram which shows an example of the virtual object displayed on the user terminal according to a certain embodiment. It is a flowchart which shows an example of the process which concerns on display control according to a certain embodiment. It is a schematic diagram which shows an example of the real image which was taken by the image pickup part according to a certain embodiment. It is a schematic diagram which shows an example of the screen which superposed virtual object on the real image which follows a certain embodiment. It is a flowchart which shows an example of the process which concerns on voice control according to a certain embodiment. It is an example of a data table according to an embodiment. It is a schematic diagram which shows an example of the display screen according to a certain embodiment. It is a schematic diagram which shows an example of the display screen according to a certain embodiment.

Hereinafter, embodiments of this technical idea will be described in detail with reference to the drawings. In the following description, the same elements are designated by the same reference numerals, and duplicate description will be omitted as appropriate. In one or more embodiments shown in the present disclosure, the elements included in each embodiment may be combined with each other, and the combined result shall also form part of the embodiments shown in the present disclosure.

(System configuration)
FIG. 1 is a diagram showing a configuration of a system 1 according to the present embodiment. The system 1 can display, for example, a predetermined race performed in the real world as a virtual race using a virtual object on an information processing device used by a user. In the present specification, the "predetermined race" is not particularly limited as long as it is a race carried out in the real world, and for example, a boat race (actual race or exhibition race), a horse race, a bicycle race, an auto race, etc. Car races such as F1, drone races, dog races, marathons, ekiden, etc. can be mentioned.

As shown in FIG. 1, the system 1 is a user such as a user terminal 10A, a user terminal 10B, and a user terminal 10C (hereinafter, user terminals 10A, 10B, 10C) which are information processing devices (first computers) used by each user. It includes a plurality of user terminals 10 such as (generally referred to as "user terminal 10"), a first server device (second computer) 20, a second server device 40, and a network 30.

The user terminal 10A and the user terminal 10B are connected to the network 30 by communicating with the radio base station 31. The user terminal 10C connects to the network 30 by communicating with a wireless router 32 installed in a facility such as a house. The user terminal 10 is, for example, a portable terminal provided with a touch screen, and may be a smartphone, a phablet, a tablet, or the like.

The user terminal 10 executes, for example, a program installed via a platform for distributing an application or the like, or a program including pre-installed website browsing software. The user terminal 10 communicates with the first server device 20 by executing the above program, and transmits / receives data related to a predetermined race, data related to the user, and the like to / from the first server device 20. It is possible to display a virtual race on the terminal 10.

The first server device 20 receives data related to a predetermined race from the second server device 40. The first server device 20 appropriately transmits data related to a predetermined race to the user terminal 10. The first server device 20 stores and manages data related to a predetermined race and data related to each user.

The first server device 20 includes a communication IF (Interface) 22, an input / output IF 23, a memory 25, a storage 26, and a processor (second processor) 29 as a hardware configuration, and these provide a communication bus. They are connected to each other via.

The communication IF 22 supports various communication standards such as a LAN (Local Area Network) standard, and functions as an interface for transmitting and receiving data to and from a user terminal 10 and a second server device 40.

The input / output IF 23 functions as an interface for accepting input of information to the first server device 20 and outputting information to the outside of the first server device 20. The input / output IF 23 may include an input receiving unit that accepts the connection of an information input device such as a mouse and a keyboard, and an output unit that accepts the connection of an information output device such as a display for displaying an image or the like.

The memory 25 is a storage device for storing data or the like used for processing. The memory 25 provides the processor 29 with a work area for temporary use, for example, when the processor 29 performs processing. The memory 25 includes a ROM (Read Only Memory) and a RAM (Random).
It is configured to include a storage device such as Access Memory).

The storage 26 is a storage device for storing various programs and data for the processor 29 to read and execute. The information stored in the storage 26 includes data related to a predetermined race (including voice information), data related to each user, and the like. The storage 26 may be configured to include a storage device such as an HDD (Hard Disk Drive) or a flash memory. The storage is not limited to the form included in the server device, and cloud services can also be used.

The processor 29 controls the operation of the first server device 20 by reading and executing a program or the like stored in the storage 26. The processor 29 is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics).
Processing Unit) etc. may be included.

The second server device 40 stores and manages data related to a predetermined race. The second server device 40 is, for example, a server device managed by a predetermined race organizer or an organization (publisher of a race magazine, race video distributor, or radio distributor) that transmits information about a predetermined race to the outside. Etc.) is a server device managed by. The second server device 40 appropriately transmits data related to a predetermined race to the first server device 20. In certain aspects, the second server device 40 may transmit data related to a predetermined race to the user terminal 10. The hardware configuration of the second server device 40 may be the same as that of the first server device 20 as long as there is no contradiction. There may be a plurality of second server devices 40.

(User terminal)
FIG. 2 is a block diagram showing an example of the functional configuration of the user terminal 10. As shown in FIG. 2, the user terminal 10 includes an antenna 110, a wireless communication IF 120, a touch screen 130, an input / output IF 140, a storage unit 150, a voice processing unit 160, a microphone 161 and a speaker 162. It includes an image pickup unit 170 and a control unit (first processor) 190.

The antenna 110 radiates a signal emitted by the user terminal 10 into space as a radio wave. Further, the antenna 110 receives radio waves from the space and gives a received signal to the wireless communication IF 120.

Since the user terminal 10 communicates with other communication devices, the wireless communication IF 120 performs modulation / demodulation processing for transmitting / receiving signals via the antenna 110 or the like. The wireless communication IF 120 is a communication module for wireless communication including a tuner, a high frequency circuit, etc., performs modulation / demodulation and frequency conversion of the wireless signal transmitted / received by the user terminal 10, and supplies the received signal to the control unit 190.

The touch screen 130 receives input from the user and outputs information to the user on the display 132. The touch screen 130 includes a touch panel 131 for receiving user operation input and a display 132. The touch panel 131 detects that a user's finger or the like has approached by using, for example, a capacitance type touch panel. The display 132 is realized by, for example, an LCD (Liquid Crystal Display), an organic EL (electroluminescence), or other display device.

The input / output IF 140 functions as an interface for accepting input of information to the user terminal 10 and outputting information to the outside of the user terminal 10.

The storage unit 150 is composed of a flash memory, an HDD, or the like, and stores a program used by the user terminal 10 and various data received by the user terminal 10 from the first server device 20 or the like.

The voice processing unit 160 performs modulation / demodulation of the voice signal. The voice processing unit 160 modulates the signal given from the microphone 161 and gives the modulated signal to the control unit 190. Further, the voice processing unit 160 gives a voice signal to the speaker 162. The voice processing unit 160 is realized by, for example, a processor for voice processing. The microphone 161 functions as an audio input unit for receiving an input of an audio signal and outputting it to the control unit 190. The speaker 162 functions as an audio output unit for outputting an audio signal to the outside of the user terminal 10.

The image pickup unit 170 is a camera that captures a real image around the user terminal 10. The image captured by the image pickup unit 170 is image-processed by the control unit 190 and output to the display 132.

The control unit 190 controls the operation of the user terminal 10 by reading and executing a program stored in the storage unit 150. The control unit 190 is realized by, for example, an application processor.

The process of executing the application program 151 by the control unit 190 will be described in more detail. The storage unit 150 stores the application program 151, the application information 152, the user information 153, and the voice information 154.

For example, the user terminal 10 downloads the application program 151 from the first server device 20 and stores it in the storage unit 150. Further, the user terminal 10 communicates with the first server device 20 to send and receive various data such as application information 152 and user information 153 to and from the first server device 20.

The application program 151 is a program for displaying a virtual race on the user terminal 10. The application information 152 includes various data referred to by the application program 151. The application information 152 includes the first information 152A.

The first information 152A is information regarding a predetermined race transmitted from the first server device 20 and the second server device 40. The first information 152A includes, for example, race time information indicating the race time of a participant or a moving body (hereinafter, also referred to as “participant”) of a predetermined race, position information of the participant or the like during the execution of the predetermined race, and the like. The time information corresponding to the position information and the time information are included.

As used herein, the term "participant" is a concept that includes not only humans but also animals such as horses and dogs. Further, the "moving body" is a main body of movement in a predetermined race, and is an animal or an aircraft on which a participant rides, an aircraft remotely controlled by a contestant, or the like. In marathons, dog races, etc., "participants" and "moving bodies" are the same.

In addition to the above, the first information 152A includes, for example, a predetermined race name, date and time, race field data, contestant data, moving object data, odds information, race prediction, race start table, and information immediately before the race. It may include pit reports, race results, race videos, race stills, past race information, and other information that may be published in information magazines or information sites about a given race.

The user information 153 includes information about the user of the user terminal 10. The user information 153 includes, for example, information for identifying the user, position information of the user terminal 10, a race purchase history and a hit rate of the user (for example, in the case of a boat race, the history of the purchased boat ticket and the purchased boat ticket. Hit rate) and the like may be included.

The voice information 154 includes voice information related to a predetermined race (hereinafter, also referred to as “race-related voice information”) transmitted from the first server device 20. The voice information 154 may include user voice information (hereinafter, also referred to as "user voice information") acquired by the voice acquisition unit 196 described later.

By reading and executing the application program 151, the control unit 190 includes an operation input reception unit 191, a transmission / reception unit 192, an object generation unit 193, a display control unit 194, a detection unit 195, and a voice acquisition unit 196. , Voice control unit 197, and each function.

The operation input receiving unit 191 accepts the user's operation input based on the output of the touch screen 130. Specifically, the operation input receiving unit 191 detects that the user's finger or the like touches or approaches the touch panel 131 as the coordinates of the coordinate system consisting of the horizontal axis and the vertical axis of the surface constituting the touch screen 130.

The operation input receiving unit 191 determines the user's operation on the touch screen 130. The operation input reception unit 191 is, for example, "approach operation", "release operation", "tap operation", "double tap operation", "long press operation (long touch operation)", "drag operation (swipe operation)", Determine the user's operation such as "move operation", "flick operation", "pinch-in operation", and "pinch-out operation".

The operation input receiving unit 191 may accept the movement of the user terminal 10 detected by the acceleration sensor or the like mounted on the user terminal 10 as an operation input.

The transmission / reception unit 192 transmits and receives various information to and from external communication devices such as the first server device 20 and the second server device 40 via the wireless communication IF 120 and the network 30. The transmission / reception unit 192 receives, for example, the first information 152A and the race-related voice information from the first server device 20 or the second server device 40. Further, the transmission / reception unit 192 uses, for example, the information corresponding to the operation input received by the operation input reception unit 191, the information stored in the user information 153, the user voice information acquired by the voice acquisition unit 196, and the like as the first server device. It is transmitted to 20 or the second server device 40.

The object generation unit 193 generates a virtual object for presenting the second information regarding a predetermined race to the user based on the first information. In the present specification, the "second information" is, for example, at least a part of the first information 152A or information calculated from the first information 152A. In a certain aspect, the "second information" can be information that can be presented as text. In a certain aspect, the "second information" may be ranking information indicating the ranking of the contestants or the like during a predetermined race or the ranking of the contestants or the like as a result of the race.

The object generation unit 193 generates, as virtual objects, a race field object representing a race field and a moving object representing a contestant or the like. The object generation unit 193 may generate a virtual display board for displaying the second information as text. In addition to the above objects, the object generation unit 193 also creates a virtual screen for displaying the second information as an image, various building objects, landscape objects such as trees, objects that serve as user avatars, and the like. May be generated.

The display control unit 194 superimposes an image (hereinafter, also referred to as “superimposed image”) on which a virtual object generated by the object generation unit 193 is superimposed on a real image around the user terminal 10 imaged by the image pickup unit 70. It is displayed on the display 132. The display control unit 194 moves the moving object on the race field object based on the race time information included in the first information 152A, and displays a virtual race that virtually reproduces a predetermined race on the display 132. It is preferable that the display control unit 194 reproduces the virtual race based on the position information of the contestants and the like included in the first information 152A and the time information corresponding to the position information in addition to the race time information. ..

It is preferable that the display control unit 194 can change the viewpoint in the superimposed image according to the operation input received by the operation input reception unit 191. The display control unit 194 displays various menu screens and GUIs (Graphical User Interfaces) on the display 132, and changes the display contents of the display 132, in response to the operation input received by the operation input reception unit 191.

The detection unit 195 detects a flat surface in the real image captured by the image pickup unit 170. The detection of a flat surface is realized by a conventionally known image recognition technique. For example, when the user performs an operation of selecting a flat surface detected by the detection unit 195, a superimposed image in which the race field object is arranged on the flat surface is displayed on the display 132.

The flat surface is preferably a horizontal surface. Further, the angle formed by the flat surface and the bottom surface constituting the race field object may be 0 degrees, but is preferably an acute angle, and can be, for example, in the range of 15 degrees to 45 degrees. The above angle may be adjusted by accepting a user's operation. Also, even if there is a convex part on a part of the flat surface in the real world, or if there is an object on the flat surface, if the convex part or the object is of a size that can be hidden by the racetrack object. , The place where the race field object can be placed may be detected as a flat surface.

The voice acquisition unit 196 acquires the voice emitted by the user via the microphone 161 or the like in a state where the superimposed image is displayed by the display control unit 194, and generates the user voice information. The voice acquisition unit 196 may start acquiring voice in response to a user's operation input, or may detect that the user has spoken and start acquiring voice. Further, the voice information generated by the voice acquisition unit 196 may include acquisition time information indicating the time when the voice is acquired. Further, the voice information generated by the voice acquisition unit 196 may include text information corresponding to the voice content. The voice information acquired by the voice acquisition unit 196 is transmitted to the first server device 20 via the transmission / reception unit 192.

The voice control unit 197 controls to output the voice from the speaker 162 based on the race-related voice information received by the transmission / reception unit 192. For example, the voice control unit 197 generates a voice signal based on the race-related voice information, demodulates the voice signal in the voice processing unit 160 as necessary, and outputs the voice from the speaker 162.

The voice output by the voice control unit 197 is performed together with the display of the superimposed image by the display control unit 194. Here, the display control unit 194 may display the text information corresponding to the voice together with the output of the voice by the voice control unit 197.

Further, the display control unit 194 may display a user image corresponding to the user and / or another user image corresponding to another user. The display control unit 194 may further display text information created based on the user voice information in the vicinity of the user image, and may display text information created based on the other user voice information in the vicinity of the other user image. It may be displayed.

(1st server device)
FIG. 3 is a block diagram showing a functional configuration of the first server device 20. A detailed configuration of the first server device 20 will be described with reference to FIG. The first server device 20 exerts functions as a communication unit 220, a storage unit 250, and a control unit 290 by operating according to a program.

The communication unit 220 functions as an interface for the first server device 20 to communicate with an external communication device such as the user terminal 10 or the second server device 40 via the network 30.

The storage unit 250 stores various programs and data for realizing the system 1. In a certain aspect, the storage unit 250 stores the program 251 and the race information 252, the user information 253, and the voice information 254.

The program 251 is a program for the first server device 20 to communicate with the user terminal 10 and the second server device 40 to realize the system 1. The program 251 is executed by the control unit 290 to send and receive data to and from the user terminal 10 and the second server device 40, a process according to the operation content performed by the user of the user terminal 10, race information 252, and the user. The first server device 20 is made to perform a process of updating the information 253 and the like.

Race information 252 contains various data related to a given race. The race information 252 includes, for example, the first information 252A. The first information 252A is the source information of the first information 152A, and the first information 152A can be a part of the first information 252A. The first information 252A is, for example, information acquired from the second server device 40.

The user information 253 is information about the user of the user terminal 10. The user information 253 includes a user management table 253A. The user management table 253A stores, for example, information for identifying a user, position information of a user terminal 10, a race purchase history of the user, a hit rate, and the like for each user.

The audio information 254 includes race-related audio information. The race-related voice information includes, for example, the user voice information of the user A transmitted from the user terminal 10A, and the user voice information of the user B transmitted from the user terminal 10B of the user B different from the user A (other for the user A). (User voice information), one of the live voice information generated from the live voice of the predetermined race, and the field sound information generated from the field sound collected at the real-world race track where the predetermined race is performed. The above may be included.

Further, the race-related voice information may include acquisition time information indicating the time when the voice that is the source of the voice information was acquired and text information corresponding to the voice content. The text information may be generated in the user terminal 10, the second server device 40, or other computer device that is the source of each voice information, or may be generated in the first server device 20.

The control unit 290 is realized by the processor 29, and by executing the program 251 as a transmission / reception unit 291, a first information acquisition unit 292, a voice information acquisition unit 293, a selection unit 294, a data management unit 295, and a timekeeping unit 296. Demonstrate function.

The transmission / reception unit 291 transmits and receives various information to and from an external communication device such as a user terminal 10 and a second server device 40 via the communication unit 220 and the network 30. The transmission / reception unit 291 transmits, for example, at least a part of the first information 252A and the voice information 254 to the user terminal 10. Further, the transmission / reception unit 291 receives, for example, the first information 252A from the second server device 40.

Further, the transmission / reception unit 291 receives, for example, user voice information from the user terminal 10. Further, the transmission / reception unit 291 receives, for example, live voice information and on-site sound information from the second server device 40 or another computer device (not shown).

The first information acquisition unit 292 acquires the first information 252A from the second server device 40 via the transmission / reception unit 291.

The voice information acquisition unit 293 acquires one or more of the user voice information, the live voice information, and the on-site sound information of one or more users as race-related voice information via the transmission / reception unit 291.

The selection unit 294 selects information to be transmitted to the user terminal 10 from the race-related voice information included in the voice information 254. For example, the selection unit 294 can select one or more of the user voice information, the live voice information, and the on-site sound information as information to be transmitted to the user terminal 10 in accordance with a request from the user terminal 10. Further, the selection unit 294 can select information to be transmitted to the user terminal 10 based on the acquisition time information included in the race-related voice information.

When other user voice information for user A is selected as information to be transmitted to the user terminal 10A, the selection unit 294 further determines one or more other users from among a plurality of other user voice information based on a predetermined selection criterion. It is preferable to select voice information. The predetermined selection criteria are not particularly limited, but may be based on, for example, one or more of the relationship between the user and another user, and the expected hit rate or recovery rate of the other user in a predetermined period. preferable. The above relationship is not particularly limited, and examples thereof include follow-up relationships between users. The predetermined period is not particularly limited, and examples thereof include a period of the latest one year, a period such as a spring season or an autumn season of the year, and the like. Instead of the predetermined period, the hit rate and the recovery rate may be calculated based on the latest predetermined number of races (for example, the latest 100 races) purchased by the user.

The follow relationship is, for example, a relationship in which comments or actions of the other user on the application can be viewed. The follow relationship may be one that can be unilaterally set by the user by selecting a specific user, or is bidirectional, which is set after the user applies to the specific user and is approved by the specific user. It may be a thing. Users in a follow-up relationship may be able to communicate with each other, for example, by sending and receiving messages. Follow relationships include so-called friend relationships.

In a certain aspect, the selection unit 294 selects other user voice information of another user with whom the user A has a follow-up relationship as information to be transmitted to the user terminal 10A. Further, in a certain aspect, the selection unit 294 selects other user voice information of another user having a high expected hit rate or recovery rate in the race as information to be transmitted to the user terminal 10A. Further, in a certain aspect, the selection unit 294 transfers the voice information of another user, who has a high expected hit rate or recovery rate in the race, to the user terminal 10A among the other users with whom the user A has a follow-up relationship. Select as information to send.

The data management unit 295 performs a process of updating various data stored in the storage unit 250 according to the processing results of the first information acquisition unit 292, the voice information acquisition unit 293, and the like. The timekeeping unit 296 performs a process of measuring the time. Various times displayed on the user terminal 10 (for example, the time until the start of the race) can be controlled based on the time measured by the time measuring unit 296.

(Operation example)
Next, an operation example in the system 1 will be described with reference to FIGS. 4 to 12. In the following, a case where a predetermined race is a boat race will be described as an example, but the following description can be applied even if the predetermined race is another race. Further, in the following description, it is assumed that data is transmitted / received between the user terminal 10 and the first server device 20 and between the first server device 20 and the second server device 40, but in a certain aspect, the user The terminal 10 and the second server device 40 may be configured to directly transmit and receive data.

FIG. 4 is a schematic diagram showing an example of a boat race field in the real world. Two turn marks 403 are installed in the boat race field 401, and races are carried out by boats 402a to 402f on which each boat racer rides. When the race is completed, the race time information indicating the race times of the boats 402a to 402f is transmitted from the second server device 40 to the first server device 20, and is transmitted from the first server device 20 to the user terminal 10.

The boat race field 401 is provided with image pickup devices (cameras) 404a to 404b. The image pickup apparatus 404a captures the boats 402a to 402f in the field of view from above the boat race field 401. The image pickup apparatus 404b keeps the boats 402a to 402f in the field of view from the side of the boat race field 401. The images of the boats 402a to 402f captured by the image pickup devices 404a to 404b are transmitted to the second server device 40. In the second server device 40, for example, image analysis of each image is performed, and position information indicating the position of each of the boats 402a to 402f in the shooting time of each image is calculated. The calculated position information and the time information related to the shooting time corresponding to the position information are transmitted to the first server device 20, and are transmitted from the first server device 20 to the user terminal 10. The location information may be calculated by the first server device 20.

Further, a position sensor such as a GPS sensor may be installed on the boats 402a to 402f instead of or in addition to the image pickup devices 404a to 404b. The position information of the boats 402a to 402f acquired by the position sensor and the time information indicating the time when the position information was acquired are finally transmitted to the user terminal 10.

FIG. 5 is a schematic diagram showing an example of a virtual object displayed on the user terminal 10. In the example of FIG. 5, as virtual objects, a race field object 501, moving objects 502a to 502f, two turnmark objects 503, and a virtual display board 505 are shown.

The race field object 501 is an object that virtually displays the boat race field 401. The race field object 501 and the turn mark object 503 are preferably created based on race field data such as course information of the boat race field 401, and preferably have a shape corresponding to the boat race field 401.

The moving objects 502a to 502f are objects that virtually display the boats 402a to 402f, respectively, and have a shape imitating a boat. The moving objects 502a to 502f move the race field object 501 based on the race time information, the position information of the boats 402a to 402f, and the time information corresponding to the position information. That is, the race field object 501 and the moving objects 502a to 502f display the race in the real world as a virtual race on the user terminal 10.

Even if there is no position information and time information of the boats 402a to 402f, it is possible to display the virtual race only by the race time information. However, in this case, the final order of arrival will be the same as in the real world race, but it is difficult to reproduce the race progress such as the ranking during the race.

The virtual display board 505 is an object that displays text information. The virtual display board 505 is, for example, a corresponding nonexistent object in the boat race field 401. The text information displayed on the virtual display board 505 is not particularly limited, and may be, for example, ranking information, odds information, or the like. Further, the text information displayed on the virtual display board 505 may be changeable based on the operation input of the user.

FIG. 6 is a flowchart showing an example of processing related to display control. The order of the processes constituting the flowchart (including FIG. 9) described below is random as long as the processing contents do not have any contradiction or inconsistency, and may be executed in parallel. Further, the process executed by each device in each flowchart may be executed by another device within a range that does not cause a contradiction.

The processes shown in FIGS. 6 and 9 are realized by the control unit 190 executing the application program 151 and the control unit 290 executing the program 251. First, in step S610, the control unit 190 activates the image pickup unit 170, which is a camera. The image pickup unit 170 captures a real image around the user terminal 10.

In step S620, the control unit 190 detects a flat surface in the image captured by the image pickup unit 170. In step S630, the control unit 190 arranges the virtual object on the detected flat surface.

Here, using FIGS. 7 to 8, step S61
The processing of 0, S620, and S630 will be specifically described. FIG. 7 is a schematic diagram showing an example of a real image captured by the imaging unit 170. In the example of FIG. 7, a keyboard 702 and a monitoring device 703 are placed on a flat desk 701.

When the image pickup unit 170 is activated in step S610, the real image captured by the image pickup unit 170 is displayed on the display 132. Next, in step S620, the control unit 190 detects a flat surface in the image captured by the image pickup unit 170, that is, in the image displayed on the display 132.

In FIG. 7, the region 704 is detected as a flat surface. There is a keyboard 702 in the area 704, but since the keyboard 702 is small enough to be hidden by the race field object 501, the control unit 190 detects the area 704 as a flat surface.

If the position imaged by the image pickup unit 170 is changed in the state shown in FIG. 7, the position of the region 704 can also be changed. The area 704 is displayed on the display 132 so as to be distinguishable from other parts, for example, by adding a predetermined color. When the user executes a tap operation or the like on the area 704, in step S630, the control unit 190 arranges a virtual object such as a race field object 501 on the area 704.

FIG. 8 is a schematic diagram showing an example of a screen in which a virtual object is superimposed on a real image and displayed. In FIG. 8, the area with the dot pattern including the monitor device 703 is the real image, and the other area is the area where the virtual object is displayed. For example, an advertisement image may be displayed in an area where the virtual object is not displayed.

In FIG. 8, as virtual objects, a race field object 501, a plurality of moving objects 502, two turnmark objects 503, 505, a large monitor object 506, building objects 507a to 507b, and other reference numerals are added. Many objects (tree objects, clock objects, etc.) that have not been displayed are displayed. These objects are created, for example, based on the first information 152A received from the first server device 20.

In FIG. 8, the racefield object 501 is arranged on the region 704 so that the angle formed by the flat surface included in the region 704 and the bottom surface constituting the racefield object 501 is 0 degrees. In this case, for example, an area in the depth direction such as the area X1 or moving objects 502 may overlap each other and become difficult to see. Further, in the area on the depth direction side of the large monitor object 506 or the like, a blind spot may occur in which the moving object 502 cannot be seen.

However, in the state shown in FIG. 8, the user can change the viewpoint of the superimposed image displayed on the display 132. For example, when the user inputs an operation such as a pinch-in operation or a pinch-out operation, the viewpoint position of the image pickup unit 170 may move toward or away from the race field object 501. Further, even when the user moves in the depth direction of the display 132 while holding the user terminal 10 or moves in the front direction, the viewpoint position of the image pickup unit 170 may move toward or away from the race field object 501. Further, when the user performs an operation such as designating a predetermined position on the display 132, the viewpoint position can be moved to the designated position. By such a viewpoint change, the above-mentioned visibility problem can be solved.

Further, by arranging the race field object 501 so that the angle θ formed by the flat surface and the bottom surface constituting the race field object 501 is an acute angle, the above-mentioned problem regarding visibility can be alleviated.

Returning to the description of the flowchart of FIG. In step S640, the control unit 190 acquires the position information of the boats 402a to 402f in the boat race field 401 in the real world. That is, when the race by the boats 402a to 402f is started in the real world, the control unit 190 acquires the position information and the time information of the boats 402a to 402f from the first server device 20. The method of acquiring the position information and the time information is as described with reference to FIG.

In step S650, the control unit 190 controls so that the position information acquired in step S640 and the moving object are interlocked with each other. Specifically, using the time information and the position information, the movements of the boats 402a to 402f on the race field object 501 are controlled to be the same as the movement objects 502a to 502f on the boat race field 401.

The process of steps S640 and S650 is repeated at least from the start to the end of the race in the real world, but may be repeated before and after the start and end of the race in the real world.

If the race displayed as a virtual race is a past race, the position information of the boats 402a to 402f from the start to the end of the race may be collectively acquired before the start of the virtual race. Further, the virtual race may be displayed by acquiring only the race time information without acquiring the position information and the like. The race displayed as a virtual race may be an exhibition race.

After the end of the race in the real world, a series of display control processes are terminated in response to the acceptance of the operation input for terminating the application program 151.

FIG. 9 is a flowchart showing an example of processing related to voice control. Each process shown in FIG. 9 may be executed before or after any of the processes shown in FIG. 6 or may be executed in parallel with any of the processes shown in FIG.

First, in step S901A, the control unit 190 of the user terminal 10A activates the application program 151 based on the user's operation input.

Next, in step S902A, the control unit 190 of the user terminal 10A accepts the selection of the audio live mode. Step S902A may be executed in response to, for example, an operation input such as tapping a virtual button for starting the voice live mode by the user. The "voice live mode" is a mode in which one or more of the race-related voice information can be output from the user terminal 10A.

Next, in step S903A, the control unit 190 of the user terminal 10A displays the avatar of the user A on the display 132. The avatar is a user image corresponding to the user. The appearance, clothes, and the like of the avatar can be selected in advance by the user, for example. The avatar may be a two-dimensional model or a three-dimensional model. The avatar may be an image representing a part of a predetermined character (for example, a face part).

Next, in step S904A, the control unit 190 of the user terminal 10A acquires the voice emitted by the user A as the microphone 161 user voice information. Next, in step S905A, the control unit 190 of the user terminal 10A transmits the user voice information of the user A to the first server device 20. In response to the execution of step S905A, in step S906, the control unit 290 of the first server device 20 receives and acquires the user voice information of the user A.

Each process of step S901B to step S905B is also executed in the user terminal 10B of the user B. Steps S901B to S905B are the same processes as steps S901A to S905A, respectively. In response to the execution of step S905B, in step S906, the control unit 290 of the first server device 20 receives and acquires the user voice information of the user B. The other user terminals 10 can also perform the same processing as the user terminals 10A and 10B.

Further, in step S904C, the control unit of the computer device C acquires the live voice information which is the voice information of the live person who is playing the race. Next, in step S905C, the control unit of the computer device C transmits the live voice information to the first server device 20. In response to the execution of step S905C, in step S906, the control unit 290 of the first server device 20 receives and acquires the live voice information.

The computer device C is not particularly limited as long as it has a sound collecting function. The computer device C may be the second server device 40. The computer device C may be the same as the server device that transmits the first information to the first server device 20, or may be a different server device.

Further, in step S904D, the control unit of the computer device D acquires the site sound information from the sound of the site sound collected in the race field in the real world. Next, in step S905D, the control unit of the computer device D transmits the on-site sound information to the first server device 20. In response to the execution of step S905D, in step S906, the control unit 290 of the first server device 20 receives and acquires the on-site sound information.

The computer device D is not particularly limited as long as it has a sound collecting function. The computer device D may be the second server device 40. The computer device D may be the same as the server device that transmits the first information to the first server device 20, or may be a different server device.

Each voice information acquired in step S906 is stored as race-related voice information. In this embodiment, an example in which three types of user voice information, live voice information, and on-site sound information are acquired is given, but the one acquired in step S906 is any one of the above three types. Anything above that is fine.

Next, in step S907, the control unit 290 selects race-related voice information to be transmitted to each user terminal 10. In step S907, for example, one or more voice information is selected from the user voice information (including other user voice information), the live voice information, and the on-site sound information in response to a request from each user terminal 10.

Specifically, when the user terminal 10A is set to output the user voice information together with the display of the virtual race, in step S907, the virtual race is displayed as the race-related voice information to be transmitted to the user terminal 10A. The user voice information acquired from the user terminal 10 is selected. Further, when the user terminal 10B is set to output the live voice information and the site sound information together with the display of the virtual race, in step S907, the race-related voice information to be transmitted to the user terminal 10B is set to the above virtual race. Corresponding live audio information and on-site sound information are selected. It is preferable that each user can set which race-related voice information is output in each user terminal 10.

Further, when the user voice information is selected in step S907, the control unit 290 further selects one or more other user voice information from the plurality of other user voice information based on a predetermined selection criterion. May be good. The selection criterion is, for example, one or more of the hit rate, the recovery rate, and the follow-up relationship obtained from the data table as shown in FIG.

FIG. 10 is an example of a data table according to this embodiment. The data table 1001 shown in FIG. 10A shows the hit rate and the recovery rate of each user. In the data table 1001, the hit rate (%) is a value calculated by "(number of hit races / number of purchased races) x 100", and the recovery rate (%) is "(collection amount / purchase amount)". It is a value calculated by "x100".

When the selection criterion is the hit rate and / or the recovery rate, for example, the user voice information of the user having the higher hit rate and / or the recovery rate in a predetermined period is preferentially selected as the other user voice information. In the example of FIG. 10A, when the hit rate is the selection criterion, the order of priority is user 001, user 003, and user 002. When the collection rate is the selection criterion, the order of priority is user 002, user 001, and user 003.

Further, in step S907, for example, only the user voice information of the user whose hit rate and / or recovery rate is equal to or higher than a predetermined threshold value may be selected. For example, when the threshold value of the hit rate is 80%, the voice information of other users selected in step S907 is only the voice information of user 001.

The data table 1002 shown in FIG. 10B shows the followers of each user. In the example of the data table 1002, the user 001 sets at least the users 002 to 004 as followers. When the selection criterion is a follow-up relationship between users, in step S907, the user voice information of the users 002 to 004 is selected as the other user voice information on the user terminal 10 of the user 001.

It should be noted that user 001 and user 002 are bidirectionally following each other. On the other hand, the user 001 and the user 003 are in a relationship in which the user 001 unilaterally follows the user 003. In step S907, the selection may be prioritized based on the type of follow-up relationship.

Further, in step S907, in place of or in addition to the follow-up relationship between users, the selection may be prioritized based on the intimacy that increases in response to the interaction between users via the application program 151. ..

Further, in step S907, the acquisition time information included in each race-related voice information may be used as one of the selection criteria. Specifically, in step S907, the time indicated by the acquisition time information may be preferentially selected within a predetermined time from the current time.

Returning to the description of the flowchart of FIG. In step S908, the control unit 290 transmits the race-related voice information selected as the transmission content to each user terminal 10 to each user terminal 10. The transmission content in step S908 may differ for each user terminal 10.

In response to the execution of step S908 for the user terminal 10A, in step S909A, the control unit 290 of the user terminal 10A receives and acquires the race-related voice information transmitted from the first server device 20.

Next, in step S910A, the control unit 290 of the user terminal 10A reflects the race-related voice information acquired in step S909A on the application program 151. Specifically, in step S910A, the control unit 290 of the user terminal 10A outputs voice based on race-related voice information. Further, in step S910A, the text information corresponding to the output voice may be displayed on the display 132.

Each process of steps S903A to S910A is repeated during the display of the virtual race. The control unit 290 of the user terminal 10A ends a series of voice control processes in response to receiving an operation input for terminating the voice live mode or the application program 151.

Further, in response to the execution of step S908 for the user terminal 10B, the processes of steps S909B and S910B are also executed for the user terminal 10B of the user B. Steps S909B and S910B are the same processes as in steps S909A to S910A, respectively. However, the race-related audio information acquired in step S909B may be different from that acquired in step S909A.

FIG. 11 is a schematic diagram showing an example of a display screen according to the present embodiment. FIG. 11 is, for example, a display screen displayed on the display 132 of the user terminal 10A in step S910A. Note that FIG. 11 is an example in which the user voice information based on the voice of the user A and the voice information of another user of another user different from the user A are received in step S909A.

In the example of FIG. 11, the virtual race using the moving objects 502a to 502e is displayed on the display 132. The moving object 502c is, for example, a moving object bet by the user A, and a frame number is displayed above the moving object 502c.

Further, on the display 132, the mode display field 1101, the text display button 1102, the voice output button 1103, and the text field 1110 are displayed together with the display of the virtual race.

The mode display field 1101 is a field for displaying that the audio live mode is selected. When the user inputs an operation to cancel the voice live mode, that fact is displayed in the mode display field 1101 or the mode display field 1101 is hidden.

Further, in the mode display field 1101, the number of users who are watching the virtual race at the same time is displayed. As a result, the user can grasp the number of people watching the virtual race, and for example, it becomes possible to select and watch a race in which a large number of users are watching and a large excitement is likely to occur. .. In addition to or in addition to the number of spectators in the virtual race, the number of users who have selected the voice live mode in the virtual race may be displayed.

The text display button 1102 is a button for switching display / non-display of the text field 1110. The voice output button 1103 is a button for switching on / off of voice output based on race-related voice information. When the text display button 1102 is in the display state and the voice output button is in the off state, the voice based on the race-related voice information is not output, and only the text information based on the race-related voice information is displayed.

In the text field 1110, the avatar 1111 of the user A, the user text field 1112 created based on the user voice information of the user A, and the other user text fields 1113a to 1113d created based on the other user voice information of the other user ( Hereinafter, (also collectively referred to as "other user text field 1113") is displayed.

The avatar 1111 is a user image set by the user A as his / her own image. If the user A has uttered "3 has been inserted !!" in step S904A, the voice "3 has been inserted !!" is output in step S910A. Further, together with the voice output, a text based on the text information corresponding to the voice content is displayed in the user text field 1112 in the vicinity of the avatar 1111.

In the other user text field 1113, a text based on the text information corresponding to the other user voice information selected in step S907 is displayed. In the example of FIG. 11, the voice output of "Do your best !!" is output based on the user voice information of the user name "USER109", and the text corresponding to the voice output is displayed in the other user text field 1113a.

In the other user text field 1113, for example, texts are displayed in chronological order in the order in which voice output is performed. That is, in the other user text field 1113, new acquisition times indicated by acquisition time information included in each other user voice information are displayed, and the oldest ones can be deleted in order. The same applies to the user text field 1112.

In the other user text field 1113, another user image set by another user as another user's image may be displayed, and text based on the text information of the other user may be displayed in the vicinity thereof. In addition, the configuration adopted in the conventional chat, message application, or the like may be applied to the text field 1110.

FIG. 12 is a schematic diagram showing an example of a display screen according to the present embodiment. FIG. 12 is, for example, a display screen displayed on the display 132 of the user terminal 10B in step S910B. Note that FIG. 12 is an example in which the live voice information and the on-site sound information are received in step S909B.

In the example of FIG. 12, a virtual race similar to that of the example of FIG. 11 is displayed on the display 132. However, in the example of FIG. 12, the viewpoint of the virtual race is different from the example of FIG.

In the example of FIG. 12, the mode display field 1101, the live person image 1201, the live text field 1202, and the on-site sound text field 1203 are displayed on the display 132.

The commentator image 1201 is an image of a commentator who is commenting on a real race corresponding to a virtual race. What is displayed in the commentary image 1201 may be a still image or a moving image. In step S904C, the live voice information is generated from the live voice of the live person shown in the live person image 1201. Further, in step S910B, the live voice based on the live voice information is output from the user terminal 10B.

In the live text field 1202, a text based on the text information corresponding to the live voice information is displayed. The text display in the live text field 1202 is executed in accordance with the output of the live voice.

In the field sound text field 1203, a text based on the text information corresponding to the field sound information is displayed. The text display in the field sound text field 1203 is executed in accordance with the output of the field sound.

In the live text field 1202 and the field sound text field 1203, for example, the texts are displayed in chronological order in the order in which the voice is output, and the oldest ones are deleted in order. Further, it is preferable that the live text field 1202 and the on-site sound text field 1203 can be switched between display and non-display according to the operation input of the user. Further, in the example of FIG. 12, the text field 1110 as shown in FIG. 11 may be further displayed.

The above embodiments are merely examples for facilitating the understanding of the present invention, and are not intended to limit the interpretation of the present invention. It goes without saying that the present invention can be modified and improved without departing from the spirit thereof, and the present invention includes its equivalents.

Even if the race displayed as a virtual race is a past race, based on the time information corresponding to the position information of each moving object in the virtual race and the acquisition time information included in each race-related audio information, It is possible to output audio based on appropriate race-related audio information while virtually displaying past races.

In addition, the audio output in the present disclosure may be executed before and after the start of the race. For example, even before and after the start of a race, user voice information (including voice information of other users) and live commentary (various types by the commentator) are provided to the user terminal of the user who virtually displays the race track of the race. Explanation) One or more of the voice information and the on-site sound information may be provided, and voice output or the like based on the respective voice information may be executed.

[Additional notes]
The contents of this disclosure are listed below.

(Item 1)
A program executed in a first computer equipped with a processor and an image pickup unit.
The program is attached to the processor.
The step of receiving the first information about a given race in the real world from the second computer,
A step of generating a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race based on the first information.
A step of superimposing the virtual object on a real image around the first computer captured by the image pickup unit and displaying the virtual object.
A step of receiving voice information related to the predetermined race from the second computer and outputting voice based on the voice information is executed.
The first information includes the position information of the contestant or the moving body during the execution of the predetermined race, and the time information corresponding to the position information.
In the display step, the moving object is moved on the race field object based on the first information, and the predetermined race is virtually displayed.
In the output step, the sound is output together with the virtual display of the predetermined race in the display step.
program.
As a result, it is possible to liven up the watching of the virtual race by the voice related to the race.

(Item 2)
In addition to the processor
A step of generating user voice information from the voice emitted by the user during the execution of the display step and transmitting the user voice information to the second computer is executed.
The voice information includes the user voice information and / or other user voice information which is the user voice information of the other user transmitted from the first computer of another user different from the user to the second computer. Is included,
The program described in item 1.
As a result, the voice of oneself and / or the voice of another user emitted while watching the virtual race is output from the user terminal, so that the watching of the virtual race can be effectively excited.

(Item 3)
The voice information includes at least the voice information of the other user.
In the output step, one or more other user voice information selected based on a predetermined selection criterion from a plurality of other user voice information is output.
The predetermined selection criteria are based on one or more of the relationship between the user and the other user, and the hit rate or recovery rate of the other user in a predetermined period.
The program described in item 2.
As a result, it is possible to select another user's voice information that is considered to be useful for that user from among a plurality of other user's voice information, so that the user's satisfaction can be enhanced.

(Item 4)
The relationship is a follow-up relationship between users.
In the output step, voice is output based on the voice information of one or more other users followed by the user from among the voice information of the other users.
The program described in item 3.
As a result, it is possible to give the user a sense of reality as if he / she is watching the race with another user who has a follow-up relationship with the user, so that the watching of the virtual race can be more effectively excited.

(Item 5)
In the display step, a user image corresponding to the user and / or another user image corresponding to the other user is further displayed.
In the display step, text information created based on the user voice information is further displayed in the vicinity of the user image, and / or created based on the other user voice information in the vicinity of the other user image. Display text information that is
The program according to any one of items 2 to 4.
As a result, even if the voice cannot be heard, the content of the voice can be confirmed, and the user's satisfaction can be enhanced. In addition, the voice emitted while watching the virtual race can give the user the feeling of chatting or playing live, and the watching of the virtual race can be more effectively excited.

(Item 6)
The audio information includes live audio information generated from the live audio of the predetermined race.
The program according to any one of items 1 to 5.
As a result, it is possible to give the user who is watching the virtual race the feeling of watching the actual race broadcast on TV or the like, and it is possible to more effectively enjoy watching the virtual race.

(Item 7)
In the display step, further, text information created based on the live voice information is displayed.
The program according to item 6.
As a result, even if the live voice cannot be heard, the content of the live voice can be confirmed, and the user's satisfaction can be enhanced.

(Item 8)
The audio information includes on-site sound information generated from on-site sounds collected at a real-world race track where the predetermined race is performed.
The program according to any one of items 1 to 7.
As a result, it is possible to give the user a sense of realism as if he / she is in an actual race field, and it is possible to more effectively enjoy watching a virtual race.

(Item 9)
An information processing method executed in a first computer equipped with a processor and an image pickup unit.
The information processing method is applied to the processor.
The step of receiving the first information about a given race in the real world from the second computer,
A step of generating a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race based on the first information.
A step of superimposing the virtual object on a real image around the first computer captured by the image pickup unit and displaying the virtual object.
The step of receiving the voice information related to the predetermined race from the second computer and outputting the voice based on the voice information includes executing the step.
The first information includes the position information of the contestant or the moving body during the execution of the predetermined race, and the time information corresponding to the position information.
In the display step, the moving object is moved on the race field object based on the first information, and the predetermined race is virtually displayed.
In the output step, the sound is output together with the virtual display of the predetermined race in the display step.
Information processing method.
As a result, it is possible to liven up the watching of the virtual race by the voice related to the race.

(Item 10)
An information processing device equipped with a processor and an image pickup unit.
The processor
Receive the first information about a given race in the real world from the second computer,
Based on the first information, a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race is generated.
The virtual object is superimposed and displayed on the real image around the first computer captured by the image pickup unit.
The voice information related to the predetermined race is received from the second computer, and the voice is output based on the voice information.
The first information includes the position information of the contestant or the moving body during the execution of the predetermined race, and the time information corresponding to the position information.
The superimposed display includes moving the moving object on the race field object based on the first information and virtually displaying the predetermined race.
Outputting the sound includes outputting the sound together with a virtual display of the predetermined race.
Information processing equipment.
As a result, it is possible to liven up the watching of the virtual race by the voice related to the race.

(Item 11)
A system including a first computer provided with a first processor and an image pickup apparatus, and a second computer provided with a second processor and capable of communicating with the first computer.
The second processor is
Get the first information about a given race in the real world,
Acquire voice information related to the predetermined race and
The first information and the voice information are transmitted to the first computer,
The first processor is
The first information is received from the second computer,
Based on the first information, a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race is generated.
The virtual object is superimposed and displayed on the real image around the first computer captured by the image pickup device.
The voice information is received from the second computer, and the voice is output based on the voice information.
The first information includes the position information of the contestant or the moving body during the execution of the predetermined race, and the time information corresponding to the position information.
The superimposed display includes moving the moving object on the race field object based on the first information and virtually displaying the predetermined race.
Outputting the sound includes outputting the sound together with a virtual display of the predetermined race.
system.
As a result, it is possible to liven up the watching of the virtual race by the voice related to the race.

1: System, 10: User terminal, 20: First server device, 30: Network, 40: Second server device, 130: Touch screen, 150: Storage unit (of user terminal), 190: Control (of user terminal) Unit, 250: Storage unit (of the first server device), 290: Control unit (of the first server device)

Claims (11)

A program executed in a first computer equipped with a processor and an image pickup unit.
The program is attached to the processor.
The step of receiving the first information about a given race in the real world from the second computer,
A step of generating a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race based on the first information.
A step of superimposing the virtual object on a real image around the first computer captured by the image pickup unit and displaying the virtual object.
A step of receiving voice information related to the predetermined race from the second computer and outputting voice based on the voice information is executed.
The first information includes the position information of the contestant or the moving body during the execution of the predetermined race, and the time information corresponding to the position information.
In the display step, the moving object is moved on the race field object based on the first information, and the predetermined race is virtually displayed.
In the output step, the sound is output together with the virtual display of the predetermined race in the display step.
program. In addition to the processor
A step of generating user voice information from the voice emitted by the user during the execution of the display step and transmitting the user voice information to the second computer is executed.
The voice information includes the user voice information and / or other user voice information which is the user voice information of the other user transmitted from the first computer of another user different from the user to the second computer. Is included,
The program according to claim 1. The voice information includes at least the voice information of the other user.
In the output step, voice is output based on one or more other user voice information selected based on a predetermined selection criterion from a plurality of other user voice information.
The predetermined selection criteria are based on one or more of the relationship between the user and the other user, and the hit rate or recovery rate of the other user in a predetermined period.
The program according to claim 2. The relationship is a follow-up relationship between users.
In the output step, voice is output based on the voice information of one or more other users followed by the user from among the voice information of the other users.
The program according to claim 3. In the display step, a user image corresponding to the user and / or another user image corresponding to the other user is further displayed.
In the display step, text information created based on the user voice information is further displayed in the vicinity of the user image, and / or created based on the other user voice information in the vicinity of the other user image. Display text information that is
The program according to any one of claims 2 to 4. The audio information includes live audio information generated from the live audio of the predetermined race.
The program according to any one of claims 1 to 5. In the display step, further, text information created based on the live voice information is displayed.
The program according to claim 6. The audio information includes on-site sound information generated from on-site sounds collected at a real-world race track where the predetermined race is performed.
The program according to any one of claims 1 to 7. An information processing method executed in a first computer equipped with a processor and an image pickup unit.
The information processing method is applied to the processor.
The step of receiving the first information about a given race in the real world from the second computer,
A step of generating a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race based on the first information.
A step of superimposing the virtual object on a real image around the first computer captured by the image pickup unit and displaying the virtual object.
The step of receiving the voice information related to the predetermined race from the second computer and outputting the voice based on the voice information includes executing the step.
The first information includes the position information of the contestant or the moving body during the execution of the predetermined race, and the time information corresponding to the position information.
In the display step, the moving object is moved on the race field object based on the first information, and the predetermined race is virtually displayed.
In the output step, the sound is output together with the virtual display of the predetermined race in the display step.
Information processing method. An information processing device equipped with a processor and an image pickup unit.
The processor
Receive the first information about a given race in the real world from the second computer,
Based on the first information, a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race is generated.
The virtual object is superimposed and displayed on the real image around the first computer captured by the image pickup unit.
The voice information related to the predetermined race is received from the second computer, and the voice is output based on the voice information.
The first information includes the position information of the contestant or the moving body during the execution of the predetermined race, and the time information corresponding to the position information.
The superimposed display includes moving the moving object on the race field object based on the first information and virtually displaying the predetermined race.
Outputting the sound includes outputting the sound together with a virtual display of the predetermined race.
Information processing equipment. A system including a first computer provided with a first processor and an image pickup apparatus, and a second computer provided with a second processor and capable of communicating with the first computer.
The second processor is
Get the first information about a given race in the real world,
Acquire voice information related to the predetermined race and
The first information and the voice information are transmitted to the first computer,
The first processor is
The first information is received from the second computer,
Based on the first information, a virtual object including a race field object representing a race field and a moving object representing a contestant or a moving body of the predetermined race is generated.
The virtual object is superimposed and displayed on the real image around the first computer captured by the image pickup device.
The voice information is received from the second computer, and the voice is output based on the voice information.
The first information includes the position information of the contestant or the moving body during the execution of the predetermined race, and the time information corresponding to the position information.
The superimposed display includes moving the moving object on the race field object based on the first information and virtually displaying the predetermined race.
Outputting the sound includes outputting the sound together with a virtual display of the predetermined race.
system.

Images