WO2022024898A1

WO2022024898A1 - Information processing device, information processing method, and computer program

Info

Publication number: WO2022024898A1
Application number: PCT/JP2021/027253
Authority: WO
Inventors: 隆今村
Original assignee: 株式会社ソニー・インタラクティブエンタテインメント
Priority date: 2020-07-28
Filing date: 2021-07-21
Publication date: 2022-02-03
Also published as: JP2022024819A

Abstract

A feedback device 22 assigns a user, who views a video of an event delivered online, to any of a plurality of positions in a venue where the event is being held. The feedback device 22 acquires, via a communication network 26, data concerning a reaction of the user transmitted from a user-side processing device 12. The feedback device 22 causes a speaker 24 installed at the venue to output audio corresponding to the user's reaction in a manner corresponding to the position to which the user is assigned at the venue.

Description

Information processing equipment, information processing methods and computer programs

The present invention relates to data processing technology, and particularly to an information processing device, an information processing method, and a computer program.

Due to the epidemic of the new coronavirus infection (COVID-19), events such as professional baseball and concerts are increasingly being held without spectators in the venue. In such an unattended event, the live video may be delivered to the viewer via the Internet or the like.

The cheers and cheers of the audience at the venue where the event is held will increase the motivation of the performers of the event (for example, athletes and musicians). Attempts have been made to deliver the appearance and voice of the audience to the performers of the event even in an event without spectators, but the present invention is effective for the performers of the event to react to the viewers who remotely watch the live video of the event. I thought there was room for improvement in order to give feedback to.

The present invention has been made based on the above-mentioned idea of the present inventor, and one object is to provide a technique for effectively feeding back the reaction of a viewer who remotely watches a live video of an event to the performer of the event. There is something in it.

In order to solve the above problems, the information processing apparatus of the present invention assigns a viewer who watches a video of an event delivered online to one of a plurality of positions in the venue where the event is held. An aspect according to the position in the venue to which the viewer is assigned, the allocation unit, the acquisition unit that acquires the data related to the viewer's reaction transmitted from the viewer's device, and the sound according to the viewer's reaction. It is equipped with an audio output control unit that outputs from a speaker installed at the venue.

Another aspect of the present invention is an information processing method. This method assigns a viewer who watches the video of the event delivered online to one of multiple locations within the venue where the event is held, and the viewer's device, which is transmitted from the viewer's device. The computer performs a step of acquiring data related to the reaction and a step of outputting the sound according to the reaction of the viewer from the speaker provided in the venue in a manner according to the position in the venue to which the viewer is assigned. Run.

It should be noted that any combination of the above components and the conversion of the expression of the present invention between a system, a computer program, a recording medium on which a computer program is readable, and the like are also effective as aspects of the present invention.

According to the present invention, the reaction of the viewer who remotely watches the live video of the event can be effectively fed back to the performer of the event.

It is a figure which shows the structure of the live streaming system of an Example. It is a block diagram which shows the functional block of the feedback device of FIG. It is a figure which shows the example of the room and the speaker provided in the baseball field. It is a figure which shows the example of a room and a speaker provided in a concert hall. It is a figure which shows the operation of the live streaming system of an Example schematically.

The outline of the live streaming system of the embodiment will be described before the detailed configuration thereof is described.
As a method of feeding back the reaction of the viewer who watches the event remotely to the performer of the event, (1) a video showing the viewer (fan, etc.) is displayed on the screen of the venue using a web conference system or the like (1). 2) Display the comments posted by the viewer on the screen of the venue, (3) Arrange the viewer's photos in the audience seats of the venue, (4) Output the cheers of the audience in the past events from the speakers of the venue. May be done.

As described in (1) above, when feeding back the video of the viewer, it is difficult to display the video of all the viewers of the event on the screen of the venue in terms of the screen area and the amount of data transfer. Therefore, at present, only the images of viewers selected at random or according to a predetermined rule are projected on the screen of the venue. Therefore, the content projected on the screen of the venue is a degraded version of the appearance of the audience that should be visible to the performers of the event when the audience is accommodated in the venue. Further, the feedback of the voice at present is to reproduce the past voice as described in (4) above. Therefore, it is not possible to feed back the real-time reaction of the audience to the event to the performers of the event.

The live streaming system of the embodiment allows viewers to remotely watch live images such as sports events and music events delivered online (that is, outside the venue where the event is held) in the venue where the event is held. Assign to one of multiple positions. The live streaming system of the embodiment outputs audio according to the viewer's real-time reaction in a manner corresponding to the position in the venue to which the viewer is assigned. As a result, it is possible to effectively feed back the reaction of the viewer to the performers of the event in the venue.

Hereinafter, the venue where various events such as sports events and music events are held is also referred to as an "event venue". In the case of a sporting event, the event venue is, for example, a baseball field or a soccer stadium. In the case of a music event, the event venue is, for example, a concert hall or a studio.

The detailed configuration of the live streaming system of the embodiment will be described.
FIG. 1 shows the configuration of the live streaming system 10 of the embodiment. In FIG. 1, a user a, a user b, and a user c are drawn as viewers who remotely view an event. The live streaming system 10 captures an actual event currently in progress, and a live image showing the state of the event is referred to as a head-mounted display (hereinafter, also referred to as “HMD”) of a plurality of users (user a, user b, user c). ) Is an information processing system to be displayed.

The live streaming system 10 includes a user-side processing device 12a, HMD14a, and a controller 16a used by the user a, a user-side processing device 12b, HMD14b, and a controller 16b used by the user b, and a user-side processing device 12c used by the user c. , HMD14c, and controller 16c. When the user-side processing device 12a, the user-side processing device 12b, and the user-side processing device 12c are generically referred to, they are simply referred to as the user-side processing device 12. Further, when HMD14a, HMD14b, and HMD14c are generically referred to, they are simply referred to as HMD14. Further, when the controller 16a, the controller 16b, and the controller 16c are generically referred to, they are simply referred to as the controller 16.

The user-side processing device 12 is an information processing device operated by the user, and may be, for example, a stationary game machine, a PC, a tablet terminal, or a smartphone. The user-side processing device 12 and the HMD 14 may be connected by wire using a cable, or may be wirelessly connected using a known wireless communication protocol. The controller 16 is a device to which a user operation for the user-side processing device 12 is input. The user-side processing device 12 and the controller 16 may be connected by wire using a cable, or may be wirelessly connected using a known wireless communication protocol.

The user-side processing device 12 controls the display of live video in the HMD 14. For example, the user-side processing device 12 receives the live video data transmitted from the video distribution device 20 described later, transmits the received live video data to the HMD 14, and displays the live video on the HMD 14.

Further, the user-side processing device 12 acquires the voice emitted by the user input to the microphone (not shown) of the HMD 14 from the HMD 14, and transmits the voice data to the feedback device 22 described later. Further, the user-side processing device 12 transmits data indicating the user operation (for example, data related to the button pressed by the user) input to the controller 16 to the feedback device 22.

The live streaming system 10 further includes a camera 18, a video distribution device 20, a feedback device 22, and a speaker 24. The user-side processing device 12, the video distribution device 20, and the feedback device 22 of FIG. 1 are connected via a communication network 26 including a LAN, a WAN, the Internet, and the like.

The camera 18 captures the current state of a sporting event, music event, etc. (for example, the performance of an athlete or a musician). The camera 18 may include a plurality of cameras that capture the state of the event from different positions. The camera 18 outputs a live image showing the current state of the event to the video distribution device 20.

The video distribution device 20 is an information processing device that streams and distributes live video data generated by the camera 18 to a plurality of user-side processing devices 12 that have requested distribution of the live video.

Speaker 24 will be installed in the event venue. The speaker 24 may include a plurality of speakers corresponding to a plurality of positions in the event venue. The feedback device 22 is an information processing device that outputs sound from the speaker 24 according to the reaction of the user who remotely views the event.

FIG. 2 is a block diagram showing a functional block of the feedback device 22 of FIG. Each block shown in the block diagram of the present specification can be realized by an element such as a CPU and a memory of a computer, an electronic circuit, and a mechanical device in terms of hardware, and can be realized by a computer program or the like in terms of software. Here, the functional blocks realized by their cooperation are drawn. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by combining hardware and software.

The feedback device 22 includes a control unit 30, a storage unit 32, and a communication unit 34. The control unit 30 executes various data processing for feeding back the reaction of the viewer to the performer of the event. The storage unit 32 stores data referenced or updated by the control unit 30. The communication unit 34 communicates with the external device according to a predetermined communication protocol. The control unit 30 transmits / receives data to / from the user-side processing device 12, the video distribution device 20, and the speaker 24 via the communication unit 34.

The storage unit 32 includes a room data storage unit 40 and a user data storage unit 42. The room data storage unit 40 shows room data indicating a correspondence relationship between a plurality of positions in the event venue (also referred to as “rooms” in the embodiment) and a plurality of speakers 24 provided at the plurality of positions in the event venue. Remember. The room can be said to be a virtual space corresponding to the actual audience seats at the event venue. Here, the room is an expression of a processing unit (processing unit) provided on a server and mainly used in an online game to process data for a user. Especially in the case of content with interaction, there is a limit on the server for reflecting the interaction and data of multiple users in the same virtual space, so by creating the concept of room, the number of users gathering in the same virtual space is limited. do. For example, users in the same room can communicate and communicate only with users in the same room, and users in other rooms cannot communicate and communicate with each other. The load can be reduced. In this case, when the user watches the live video of the event, there are cases where one room is automatically assigned to the user and cases where the user can select the room. Room data includes the maximum number of users that can be assigned to each room, in other words, the capacity of each room.

FIG. 3 shows an example of a room and a speaker provided in a baseball field. The spectator seats of the baseball stadium shown in the figure are divided into seven rooms. In the example of the figure, the speaker 24a is installed in the actual audience seat "back net back seat", and in the room data, the room "back net back seat" and the speaker 24a are associated with each other. Further, a speaker 24b is installed in the actual audience seat "1st base side infield seat", and in the room data, the room "1st base side infield seat" and the speaker 24b are associated with each other. Further, a speaker 24c is installed in the actual audience seat "1st base side middle seat", and in the room data, the room "1st base side middle seat" and the speaker 24c are associated with each other.

In addition, a speaker 24d is installed in the actual audience seat "3rd base side infield seat", and in the room data, the room "3rd base side infield seat" and the speaker 24d are associated with each other. Further, a speaker 24e is installed in the actual audience seat "third-base side middle seat", and in the room data, the room "third-base side middle seat" and the speaker 24e are associated with each other. Further, a speaker 24f is installed in the actual audience seat "light side outfield seat", and in the room data, the room "light side outfield seat" and the speaker 24f are associated with each other. Further, a speaker 24g is installed in the actual audience seat "left side outfield seat", and in the room data, the room "left side outfield seat" and the speaker 24g are associated with each other.

FIG. 4 shows an example of a room and a speaker provided in a concert hall. The audience seats in the concert hall shown in the figure are divided into 16 rooms. In the example of the figure, speakers 24h to 24o are installed in the actual audience seats "left 1st area" to "left 8th area", and in the room data, the rooms "left 1st area" to "left 8th area" are installed. The area ”is associated with the speaker 24h to the speaker 24o. In addition, speakers 24p to 24w are installed in the actual audience seats "right 1st area" to "right 8th area", and in the room data, the rooms "right 1st area" to "right 8th area" and speakers are installed. 24p to 24w are associated with each other.

Returning to FIG. 2, the user data storage unit 42 includes a plurality of viewers (user or user-side processing device 12) who remotely view the live video of the event, and a plurality of positions in the event venue (“room” in the embodiment). ) And the user data indicating the correspondence relationship is stored.

The control unit 30 includes a room allocation unit 44, a voice acquisition unit 46, an extraction unit 48, an operation acquisition unit 50, a conversion unit 52, a synthesis unit 54, and a voice output control unit 56. A computer program (for example, a reaction feedback program) in which the functions of the plurality of functional blocks are implemented may be stored in a recording medium, and may be installed in the storage of the feedback device 22 via the recording medium. Further, the computer program may be downloaded to the feedback device 22 via the network and installed in the storage of the feedback device 22. The processor (CPU or the like) of the feedback device 22 may exhibit the functions of the plurality of functional blocks by reading the computer program into the main memory and executing the program.

The room allocation unit 44 allocates each of the plurality of users to the plurality of user-side processing devices 12 that have requested the distribution of the live video to any of the plurality of locations (rooms in the embodiment) in the event venue. The room allocation unit 44 allocates any room provided on the server to each user up to the capacity of each room defined by the room data stored in the room data storage unit 40. In other words, the room allocation unit 44 allocates the room on the server to which the viewer is assigned to any of the plurality of locations in the event venue. The room allocation unit 44 may allocate any of a plurality of rooms to each user by round robin.

The room allocation unit 44 may acquire data related to the user who requested the distribution of the live video from the requesting user-side processing device 12 or the video distribution device 20. Further, when the video distribution device 20 receives the request for live video distribution from the user-side processing device 12, the video distribution device 20 may transmit data regarding the requesting user to the feedback device 22. The room allocation unit 44 stores the identification information of each of the plurality of users and the identification information of the room assigned to each user in association with each other in the user data storage unit 42.

The voice acquisition unit 46 and the operation acquisition unit 50 function as acquisition units for acquiring data related to the reactions of a plurality of users transmitted from the plurality of user-side processing devices 12. Specifically, the voice acquisition unit 46 acquires the voice data transmitted by the user from the user-side processing device 12 as the data related to the user's reaction. On the other hand, the motion acquisition unit 50 acquires data related to the user's motion as data related to the user's reaction. In the embodiment, the operation acquisition unit 50 acquires data indicating a user operation input by the user to the controller 16.

The extraction unit 48 extracts a voice indicating a reaction to an event (hereinafter, also referred to as “reaction voice”) from the voice uttered by the user acquired by the voice acquisition unit 46. The reaction voice may be a voice indicating cheers (for example, "wa"), exclamation (for example, "oh"), and cheering (for example, "do your best"). The extraction unit 48 may use a voice recognition technique using known template matching to extract a voice that is the same as or similar to a voice indicating cheers, exclamations, and cheers from the voice uttered by the user as a reaction voice. The extraction unit 48 outputs the reaction voice data extracted from the voice emitted by each user to the synthesis unit 54 in association with the identification information of the room assigned to each user in the user data of the user data storage unit 42. ..

The conversion unit 52 functions as a determination unit for determining a voice corresponding to the user's movement (hereinafter, also referred to as “converted voice”) based on the data related to the user's movement acquired by the movement acquisition unit 50.

In the embodiment, the storage unit 32 stores data of a plurality of types of converted voices in association with a plurality of types of user operations on the controller 16. The plurality of types of converted voices may be voices showing different reactions to the event, and may include, for example, cheering voices, exclamation voices, cheering voices, applause voices, and megaphone tapping voices. .. Further, the plurality of types of converted voices may include voices that are difficult to utter with the human mouth. Further, the operation of pressing the ○ button of the controller 16 may be associated with the voice indicating applause, and the operation of pressing the × button of the controller 16 may be associated with the voice of hitting the megaphone.

The conversion unit 52 selects and acquires the conversion voice data associated with the user operation acquired by the operation acquisition unit 50 from the plurality of types of conversion voice data stored in the storage unit 32. The conversion unit 52 associates the converted voice data acquired (generated) based on the actions of individual users with the identification information of the room assigned to each user in the user data of the user data storage unit 42, and synthesizes the unit. Output to 54.

The synthesis unit 54 synthesizes the reaction voice extracted by the extraction unit 48 and the converted voice acquired by the conversion unit 52 for each room. The synthesizing unit 54 outputs the voice data synthesized for each room (hereinafter, also referred to as “room voice”) to the voice output control unit 56 in association with the room identification information.

The audio output control unit 56 outputs audio according to the user's reaction from the speaker 24 provided in the event venue in an manner according to the position (room in the embodiment) in the event venue to which the viewer is assigned. .. In the embodiment, the voice output control unit 56 outputs voice according to the reaction of the user from the speaker corresponding to the room in the event venue to which the user is assigned among the plurality of speakers 24 provided at the plurality of positions of the event venue. Output. The voice output control unit 56 includes a function of outputting the reaction voice extracted by the extraction unit 48 from the speaker 24, and also includes a function of outputting the converted voice acquired by the conversion unit 52 from the speaker 24.

Specifically, the voice output control unit 56 receives the room voice data output from the synthesis unit 54 and the room identification information. The audio output control unit 56 refers to the room data stored in the room data storage unit 40, and among the plurality of speakers 24 installed at the event venue, the speaker 24 associated with the room indicated by the room identification information. To identify. The voice output control unit 56 outputs the room voice (for example, cheers and applause) associated with the room identification information from the speaker 24 associated with the room indicated by the room identification information.

As a modification, the feedback device 22 may be configured not to include the synthesis unit 54. In this case, the extraction unit 48 may output the reaction voice data based on the voice emitted by the user to the voice output control unit 56 in association with the identification information of the room assigned to the user. Similarly, the conversion unit 52 may output the data of the converted voice based on the user's operation to the voice output control unit 56 in association with the identification information of the room assigned to the user. The voice output control unit 56 outputs the reaction voice output from the extraction unit 48 from the speaker 24 corresponding to the room identification information associated with the reaction voice, and the converted voice output from the conversion unit 52 is the converted voice. It may be output from the speaker 24 according to the room identification information associated with.

The operation of the live streaming system 10 with the above configuration will be described.
The user-side processing device 12 transmits a live video distribution request to the video distribution device 20 according to the user's operation. The video distribution device 20 starts streaming distribution of the live video of the event captured by the camera 18 to the user-side processing device 12. At the same time, the video distribution device 20 transmits data regarding the user of the live video distribution destination to the feedback device 22. The user-side processing device 12 causes the HMD 14 to display a live video of the event delivered from the video distribution device 20.

FIG. 5 schematically shows the operation of the live streaming system 10 (mainly the feedback device 22). The room allocation unit 44 of the feedback device 22 provides one of a plurality of rooms corresponding to a plurality of spectator seats (watching area, etc.) at the event venue to the user of the live video distribution destination notified from the video distribution device 20. assign. The voice acquisition unit 46 of the feedback device 22 acquires the voice data transmitted by the user from the user-side processing device 12. The extraction unit 48 of the feedback device 22 extracts the voice indicating the reaction to the event from the voice emitted by the user.

The operation acquisition unit 50 of the feedback device 22 acquires the data indicating the user operation for the controller 16 transmitted from the user side processing device 12. The conversion unit 52 of the feedback device 22 converts the user operation for the controller 16 into a voice indicating a reaction to the event by acquiring the voice corresponding to the user operation for the controller 16 from a plurality of predetermined types of voice. .. The synthesis unit 54 of the feedback device 22 generates a room sound obtained by synthesizing the sound extracted by the extraction unit 48 and the sound converted by the conversion unit 52 for each room of the event venue.

The audio output control unit 56 of the feedback device 22 determines the output mode of the room audio of each of the plurality of rooms. In the embodiment, the audio output control unit 56 determines the speaker 24 associated with each room as the output destination of the room audio of each of the plurality of rooms. The audio output control unit 56 outputs the room audio of each of the plurality of rooms from the speaker 24 associated with each room.

For example, it is assumed that user a and user b watch a live video of the same baseball game, user a is assigned a room in the back of the back net, and user b is assigned a room in the left side outfield seat. .. When the user a makes a cheer, the voice output control unit 56 outputs the cheer from the speaker 24 (for example, the speaker 24a in FIG. 3) associated with the room behind the back net. Further, when the user b presses the × button of the controller, the voice output control unit 56 transmits the voice (for example, cheering) corresponding to the pressing of the × button to the speaker 24 (for example, FIG. It is output from the speaker 24f) of 3.

According to the live streaming system 10 of the embodiment, the sound according to the reaction of a plurality of viewers who remotely watch the live video of the event is played in the mode according to the position of the audience seat to which each viewer is assigned. It is output from the speaker 24 provided in. Specifically, cheers, cheers, etc. based on the reaction of each viewer are output from the speaker 24 corresponding to the audience seat to which each viewer is assigned among the plurality of speakers 24 provided at the event venue. This makes it possible to feed back the real-time reaction of the viewer who remotely watches the event to the performer of the event such as an athlete or a musician. Even if the event is unattended, the performer of the event can be made to feel as if the audience is at the venue, and the motivation of the performer can be improved.

Further, according to the live streaming system 10 of the embodiment, the voice emitted by the viewer is acquired by using the microphone and voice chat function normally provided in the HMD 14. However, remote viewing tends to reduce tension, and viewers often emit audio that is not related to the event (also referred to as "noise audio"). Therefore, by extracting the voice showing the reaction to the event from the voice uttered by the viewer, the noise voice is eliminated, and only the voice of cheers, exclamations, cheers, etc. uttered by the viewer is fed back to the performer of the event. It can effectively improve the motivation of the performers of the event.

Further, according to the live streaming system 10 of the embodiment, the operation input to the controller 16 by the viewer is converted into a voice showing a reaction to the event, and the converted voice is fed back to the performer of the event. As a result, even a viewer who does not have a voice input means or a viewer who has difficulty in speaking can deliver cheers and the like to the performer of the event. In addition, it is possible to deliver to the performer of the event a sound that is difficult for the viewer to actually emit.

The present invention has been described above based on the examples. This embodiment is an example, and it is understood by those skilled in the art that various modifications are possible in the combination of components and processing processes, and that such modifications are also within the scope of the present invention. Hereinafter, a modified example will be described.

The first modification example will be described. Although not mentioned in the above embodiment, at least one of the user-side processing device 12 and the HMD 14 may be provided with a camera for photographing the user's body (here, a hand), and the image pickup data output from the camera may be provided. A recognition unit that recognizes a user's movement (here, a hand movement) may be provided. The movement of the user's hand may be, for example, clapping, hitting a megaphone, or various gestures. The user-side processing device 12 may further transmit data indicating the user's hand movement recognized by the recognition unit to the feedback device 22.

The motion acquisition unit 50 of the feedback device 22 may further acquire data indicating the user's hand motion transmitted from the user-side processing device 12 as data related to the user's motion. The storage unit 32 of the feedback device 22 may store a plurality of types of voice data in association with a plurality of types of actions of the user's hand. For example, the action of applause may be associated with a voice indicating applause, and the action of striking a megaphone may be associated with a voice indicating that the megaphone has been hit. The conversion unit 52 of the feedback device 22 selects voice data associated with the user's hand movement acquired by the motion acquisition unit 50 from among the plurality of types of voice data stored in the storage unit 32. You may get it. Subsequent processing is the same as that of the embodiment.

According to the live streaming system 10 of the first modification, the motion recognition function (here, the hand recognition function) normally provided in the system including the HMD 14 is used to acquire the motion of the viewer's hand and perform the motion of the viewer's hand. , It is converted into a voice showing the reaction to the event, and the converted voice is fed back to the performer of the event. This makes it possible to feed back the viewer's real-time reaction to the event performer in a wider variety of ways.

The second modification example will be explained. Although not mentioned in the above embodiment, the capacity of each of the plurality of rooms corresponding to the plurality of spectator seats (watching areas) of the event venue may be arbitrarily determined by the event organizer. The capacity of a room can be said to be the maximum number of people that can be accommodated, and can also be said to be the maximum number of users that can be allocated. The organizer of an event can be said to be the organizer, organizer, or organizer of the event. The capacity of at least one of the plurality of rooms in the event venue may be set to a value exceeding the number of people that can actually be accommodated in the audience seats corresponding to the rooms. As a result, the room allocation unit 44 of the feedback device 22 may allocate a plurality of users to at least one of the plurality of rooms in the event venue, which exceeds the number of users that can actually be accommodated in the audience seats corresponding to the rooms. It will be.

For example, when the left first area of the concert hall shown in FIG. 4 can actually accommodate 100 people, the capacity of the room corresponding to the left first area may be set to 2000 people. In this case, the room allocation unit 44 of the feedback device 22 may allocate a maximum of 2000 users to the room corresponding to the left first area, regardless of the actual number of spectators that can be accommodated in the left first area. According to this variant, the reaction of a large number of viewers exceeding the actual number of spectators that can be accommodated at the event venue can be fed back to the performers of the event. For example, if the venue can accommodate 5,000 spectators, the cheers of 100,000 remote viewers can be delivered to the performers of the event.

The third modification example will be explained. In the above embodiment, the voice output control unit 56 of the feedback device 22 outputs the user's voice in a manner corresponding to the room to which the user is assigned (that is, the position in the event venue). , Of the plurality of speakers 24 provided at the event venue, the speaker 24 corresponding to the user's room was used for output. As a modification, the voice output control unit 56 uses known acoustic technology (for example, wave field synthesis technology or virtual surround technology) so that the voice of the user is emitted from the room to which the user is assigned to the performer of the event. The user's voice may be output from one or more speakers 24 installed at the event venue so that the user can hear the sound.

A fourth modification will be described. A viewer allocation priority may be set by the event organizer in each of the plurality of rooms set in the event venue. For example, the event organizer may set a relatively high priority for a room that wants to fill the viewer quickly. The room data stored in the room data storage unit 40 of the feedback device 22 may include a priority set for each room. When allocating any of a plurality of rooms to the user, the room allocation unit 44 of the feedback device 22 refers to the priority of each room defined in the room data, and prioritizes the room having a relatively high priority. May be assigned to the user in preference to a room with a relatively low value. For example, the room allocation unit 44 may determine the rooms to be allocated to each user so as to fill each room up to the capacity in descending order of priority.

According to this variant, the event organizer can arbitrarily set a room to fill the user (that is, an audience) first among a plurality of rooms at the event venue. For example, event organizers can set relatively high priorities for rooms that correspond to spectator seats that are physically close to the performers (eg, back net back seats in baseball stadiums or front row seats in concert halls). The sound output from the room can be made lively at an early stage, and the motivation of the performer can be effectively improved.

The fifth modification example will be described. In the above embodiment, when the user watches the live video of the event, one room is automatically assigned to the user. As a modification, the live streaming system 10 may be configured so that a user who watches a live video of an event can select a desired room from a plurality of rooms set in the event venue. For example, the room allocation unit 44 of the feedback device 22 provides information on a plurality of rooms set in the event venue (for example, information on the spectator seats corresponding to each room) to the user-side processing device 12 and causes the HMD 14 to display the information. You may. The user-side processing device 12 may transmit data indicating the room selected by the user to the feedback device 22.

When the room allocation unit 44 of the feedback device 22 receives data indicating a room selected by the user from the feedback device 22, the selected room is given to the user on condition that the selected room has not reached the capacity. May be assigned. According to this modification, the user who watches the live video of the event can select a desired room in the event venue, in other words, the position where the user's sound is output can be selected. As a result, the user can deliver cheers and the like to the performers of the event from the same position among like-minded friends and fans.

The sixth modification will be described. Each of the plurality of rooms set in the event venue may be priced by the organizer of the event. For example, the organizer of an event may set a relatively high price for a room corresponding to an audience seat relatively close to the performer (for example, a back net back seat in a baseball stadium or a front row seat in a concert hall). On the other hand, the price of the room corresponding to the audience seats relatively far from the performer (for example, the outfield seats in the baseball stadium or the rear seats in the concert hall) may be set relatively low. The room data stored in the room data storage unit 40 of the feedback device 22 may include the price of each room.

The room allocation unit 44 of the feedback device 22 may present the price of each room specified in the room data to the user who watches the live video of the event. When a room is selected by the user, the room allocation unit 44 selects the room on condition that the room has not reached the capacity and that the charge processing for the room price to the user (settlement processing for the room price) is successful. The reserved room may be assigned to the user. According to this variant, it is possible to realize a new business of selling a room to viewers as a means of delivering one's cheers to the performer.

The 7th modification will be described. The livestreaming system 10 of the embodiment may deliver less cheering to the performers when the number of viewers of the event is small. Therefore, the feedback device 22 may be equipped with a mechanism for amplifying the audio of the viewer. For example, the feedback device 22 storage unit 32 may store a threshold value of the number of users, which is a condition for amplifying the user's voice, which is predetermined by the performer of the event. When the number of users who watch the live video of the event is less than the above threshold value, the audio output control unit 56 of the feedback device 22 may amplify the user's audio and output it from the speaker 24.

As an example of amplifying the user's voice, the voice output control unit 56 may make the volume of the user's voice louder than usual and output it from the speaker 24 corresponding to the room to which the user is assigned. Alternatively, the voice output control unit 56 outputs the user's voice from the speaker 24 corresponding to the room to which the user is assigned, and also outputs the user's voice from the speaker 24 corresponding to another room to which the user is not assigned. You may.

The eighth modification example will be described. The room allocation unit 44 of the feedback device 22 may allocate a plurality of different rooms to the user when the user charges. The room allocation unit 44 may transmit a screen for the user to select whether or not to purchase a room different from the allocated room to the user side processing device 12 and display it on the HMD 14. The first room may be free of charge or may be charged as described in the sixth modification.

When a plurality of different rooms are assigned to a user, the audio output control unit 56 of the feedback device 22 transmits the voice of the user from a plurality of different speakers 24 corresponding to the different plurality of rooms assigned to the user. It may be output. According to this variant, the user can amplify and deliver his or her cheers to the performer by purchasing assignments to multiple rooms.

Further, the feedback device 22 of the eighth modification may further include a notification unit for notifying the terminal of the performer or the implementer of the event of information (name or handle name, comment, etc.) about the user who purchased the plurality of rooms. .. According to this configuration, it is possible to realize a mechanism in which the performers and organizers of the event express their gratitude to the users who have purchased a plurality of rooms.

The ninth modification example will be described. The user may watch the live video of the event using a display other than the HMD 14. In this case, the user-side processing device 12 may transmit the voice data of the user acquired by the external microphone or the built-in microphone to the feedback device 22.

Any combination of the above-mentioned examples and modifications is also useful as an embodiment of the present disclosure. The new embodiments resulting from the combination have the effects of each of the combined examples and variants. It is also understood by those skilled in the art that the functions to be fulfilled by each of the constituent elements described in the claims are realized by a single component or a cooperation thereof shown in the examples and modifications.

The present invention can be applied to an information processing device.

10 live streaming system, 12 user side processing device, 22 feedback device, 24 speaker, 44 room allocation unit, 46 audio acquisition unit, 48 extraction unit, 50 operation acquisition unit, 52 conversion unit, 56 audio output control unit.

Claims

An assignment unit that assigns viewers who watch the video of the event delivered online to one of multiple locations in the venue where the event is held, and
An acquisition unit that acquires data related to the viewer's reaction transmitted from the viewer's device, and
An audio output control unit that outputs audio according to the reaction of the viewer from a speaker provided in the venue in an manner corresponding to the position in the venue to which the viewer is assigned.
An information processing device characterized by being equipped with.
The audio output control unit receives audio according to the reaction of the viewer from the speaker corresponding to the position in the venue to which the viewer is assigned among the plurality of speakers provided at the plurality of positions in the venue. The information processing apparatus according to claim 1, wherein the information processing apparatus is to be output.
With more extraction section,
The acquisition unit acquires audio data emitted by the viewer as data relating to the reaction of the viewer.
The extraction unit extracts a sound showing a reaction to the event from the sound emitted by the viewer, and then extracts the sound indicating a reaction to the event.
The information processing device according to claim 1 or 2, wherein the voice output control unit outputs the voice extracted by the extraction unit from the speaker.
With more decision-making parts
The acquisition unit acquires data on the operation of the viewer as data on the reaction of the viewer.
The determination unit determines the sound corresponding to the movement of the viewer, and determines the sound.
The information processing device according to any one of claims 1 to 3, wherein the voice output control unit outputs the voice determined by the determination unit from the speaker.
Any of claims 1 to 4, wherein the allocation unit can allocate a plurality of viewers to at least one of the plurality of positions in the venue, which exceeds the number of viewers that can be actually accommodated at the position. Information processing device described in Crab.
Priority is set for each of the multiple locations in the venue.
The allocation unit according to any one of claims 1 to 5, wherein when allocating any of the plurality of positions in the venue to the viewer, the position having a high priority is preferentially assigned to the viewer. The information processing device described.
The information processing device according to any one of claims 1 to 6, wherein the allocation unit allocates a room on a server to which the viewer is assigned to any of a plurality of positions in the venue.
A step of assigning a viewer who watches a video of an event delivered online to one of multiple locations in the venue where the event is held, and
The step of acquiring the data regarding the reaction of the viewer transmitted from the device of the viewer, and
A step of outputting audio according to the reaction of the viewer from a speaker provided in the venue in an manner corresponding to the position in the venue to which the viewer is assigned.
An information processing method characterized by the execution of a computer.
A function to assign viewers who watch the video of the event delivered online to one of multiple locations in the venue where the event is held, and
A function to acquire data related to the viewer's reaction transmitted from the viewer's device, and
A function of outputting audio according to the reaction of the viewer from a speaker provided in the venue in a manner corresponding to the position in the venue to which the viewer is assigned.
A computer program to realize the above in a computer.