JP6951610B1 - Speech processing system, speech processor, speech processing method, and speech processing program - Google Patents

Abstract

The voice processing system includes a microphone that detects sounds emitted by users participating in the event, a performer position that represents the position of the performer of the event in a predetermined area where the event is held, and an audience position that represents the position of the user in the predetermined area. A sound adjustment unit that adjusts the sound detected by the microphone based on the relationship, a sound detected by the microphone corresponding to the first user who is the user and adjusted by the sound adjustment unit, and a user different from the first user. It is provided with a compositing unit that synthesizes the sound detected by the microphone corresponding to the second user and adjusted by the sound adjusting unit, and an output unit that outputs data representing the sound synthesized by the compositing unit.

Description

The present invention relates to a voice processing system, a voice processing device, a voice processing method, and a voice processing program.

Patent Document 1 below proposes a technique that can give a fan the impression that the artist is singing or playing on the spot without showing the artist in front of the fan. ing. In the field of distributing the contents of various events such as music, entertainment, and sports, a technology that can provide a new experience is desired.

Japanese Unexamined Patent Publication No. 2015-185020

According to an aspect of the present invention, a microphone that detects a sound emitted by a user participating in an event, a performer position that represents the position of the performer of the event in a predetermined area where the event is held, and a spectator that represents the position of the user in a predetermined area. A sound adjustment unit that adjusts the sound detected by the microphone based on the relationship with the position, a sound detected by the microphone corresponding to the first user who is the user, and adjusted by the sound adjustment unit, and the first user. A compositing unit that synthesizes the sound detected by the microphone corresponding to the second user who is a different user from the above and adjusted to the sound adjusting unit, and an output unit that outputs data representing the sound synthesized by the compositing unit. Provided is a sound processing system in which the performer performs a performance outside a predetermined area, and the sound data synthesized by the synthesis unit is provided to a sound output device arranged at a position where the sound reaches the performer.
According to an aspect of the present invention, a microphone that detects a sound emitted by a user participating in an event, a performer position that represents the position of the performer of the event in a predetermined area where the event is held, and a spectator that represents the position of the user in a predetermined area. A sound adjustment unit that adjusts the sound detected by the microphone based on the relationship with the position, a sound detected by the microphone corresponding to the first user who is the user, and adjusted by the sound adjustment unit, and the first user. A compositing unit that synthesizes the sound detected by the microphone corresponding to the second user who is a different user from the above and adjusted to the sound adjusting unit, and an output unit that outputs data representing the sound synthesized by the compositing unit. Provided by the sound processing system, the sound adjustment unit adjusts the first sound detected by the microphone so as to approach the second sound that reaches the performer position when the first sound is emitted at the audience position. Will be done.
According to an aspect of the present invention, a microphone that detects a sound emitted by a user participating in an event, a performer position that represents the position of the performer of the event in a predetermined area where the event is held, and a spectator that represents the position of the user in a predetermined area. A sound adjustment unit that adjusts the sound detected by the microphone based on the relationship with the position, a sound detected by the microphone corresponding to the first user who is the user, and adjusted by the sound adjustment unit, and the first user. A compositing unit that synthesizes the sound detected by the microphone corresponding to the second user who is a different user from the above and adjusted to the sound adjusting unit, and an output unit that outputs data representing the sound synthesized by the compositing unit. The sound adjusting unit is provided with a sound processing system that adjusts the loudness of the sound detected by the microphone by using the distance between the audience position and the performer position.
According to an aspect of the present invention, a microphone that detects a sound emitted by a user participating in an event, a performer position that represents the position of the performer of the event in a predetermined area where the event is held, and a spectator that represents the position of the user in a predetermined area. A sound adjustment unit that adjusts the sound detected by the microphone based on the relationship with the position, a sound detected by the microphone corresponding to the first user who is the user, and adjusted by the sound adjustment unit, and the first user. A synthesizer that synthesizes the sound detected by the microphone corresponding to the second user who is a different user from the above and adjusted to the sound adjustment unit, an output unit that outputs data representing the sound synthesized by the synthesizer, and a first A second sound adjustment unit that adjusts the sound detected by the microphone corresponding to the second user based on the relationship between the audience position corresponding to the first user and the audience position corresponding to the second user, and the second Provided is a sound processing system including a second output unit that outputs sound data including sound adjusted by the sound adjustment unit to a sound output device that transmits the sound to a first user.
According to the aspect of the present invention, the position information indicating the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area is received. It is provided with a receiving unit for adjusting the sound, a sound adjusting unit that adjusts the sound emitted from the user and detected by the microphone based on the position information, and a transmitting unit that transmits data representing the sound adjusted by the sound adjusting unit. The sound adjustment unit is provided with a sound processing device that adjusts the first sound detected by the microphone so as to approach the second sound that reaches the performer position when the first sound is emitted at the audience position. ..
According to the aspect of the present invention, the position information indicating the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area is received. It is provided with a receiving unit for adjusting the sound, a sound adjusting unit that adjusts the sound emitted from the user and detected by the microphone based on the position information, and a transmitting unit that transmits data representing the sound adjusted by the sound adjusting unit. The sound adjusting unit is provided with a sound processing device that adjusts the loudness of the sound detected by the microphone by using the distance between the audience position and the performer position.
According to the aspect of the present invention, position information indicating the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area is transmitted. Receives sound data emitted from the user and adjusted based on the position information from the transmission unit to receive the sound data from the transmission destination of the position information, and from the transmission unit corresponding to the first user who is the user. The performer is provided with a compositing unit that synthesizes the sound represented by the data and the sound represented by the data received from the destination corresponding to the second user who is a user different from the first user, and the performer is outside the predetermined area. The sound processing device is provided, in which the sound data synthesized by the synthesis unit is provided to the sound output device arranged at a position where the sound reaches the performer.
According to the aspect of the present invention, position information indicating the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area is transmitted. The sound data emitted from the user and adjusted based on the position information is received from the transmission unit that receives the sound data from the transmission destination of the position information, and from the transmission unit corresponding to the first user who is the user. A compositing unit that synthesizes the sound represented by the data and the sound represented by the data received from the destination corresponding to the second user who is a user different from the first user, and the audience position corresponding to the first user. , A second sound adjustment unit that adjusts the sound detected by the microphone corresponding to the second user based on the relationship with the audience position corresponding to the second user, and is adjusted by the second sound adjustment unit. A sound processing device is provided in which sound data including the sound is output to a sound output device that transmits the sound to the first user.
According to the aspect of the present invention, the sound emitted by the user participating in the event is detected by the microphone, the performer position representing the position of the performer of the event in the predetermined area where the event is held, and the position of the user in the predetermined area are determined. The sound detected by the microphone is adjusted by the sound adjustment unit based on the relationship with the represented audience position, and the sound detected by the microphone corresponding to the first user who is the user and adjusted by the sound adjustment unit. The synthesizer synthesizes the sound detected by the microphone corresponding to the second user, which is a different user from the first user, and adjusted to the sound adjustment unit, and outputs the data representing the sound synthesized by the synthesizer. A sound processing method in which the performer performs a performance outside a predetermined area, and the sound data synthesized by the synthesizer is provided to a sound output device arranged at a position where the sound reaches the performer. Is provided.
According to the aspect of the present invention, the sound emitted by the user participating in the event is detected by the microphone, the performer position representing the position of the performer of the event in the predetermined area where the event is held, and the position of the user in the predetermined area are determined. The sound detected by the microphone is adjusted by the sound adjustment unit based on the relationship with the represented audience position, and the sound detected by the microphone corresponding to the first user who is the user and adjusted by the sound adjustment unit. The synthesizer synthesizes the sound detected by the microphone corresponding to the second user, which is a different user from the first user, and adjusted to the sound adjustment unit, and outputs the data representing the sound synthesized by the synthesizer. The sound adjustment unit adjusts the first sound detected by the microphone so as to be closer to the second sound that reaches the performer position when the first sound is emitted at the audience position. A voice processing method is provided.
According to the aspect of the present invention, the sound emitted by the user participating in the event is detected by the microphone, the performer position representing the position of the performer of the event in the predetermined area where the event is held, and the position of the user in the predetermined area are determined. The sound detected by the microphone is adjusted by the sound adjustment unit based on the relationship with the represented audience position, and the sound detected by the microphone corresponding to the first user who is the user and adjusted by the sound adjustment unit. The synthesizer synthesizes the sound detected by the microphone corresponding to the second user, which is a different user from the first user, and adjusted to the sound adjustment unit, and outputs the data representing the sound synthesized by the synthesizer. A sound processing method is provided in which the sound adjusting unit adjusts the loudness of the sound detected by the microphone by using the distance between the audience position and the performer position.
According to the aspect of the present invention, the sound emitted by the user participating in the event is detected by the microphone, the performer position representing the position of the performer of the event in the predetermined area where the event is held, and the position of the user in the predetermined area are determined. The sound detected by the microphone is adjusted by the sound adjustment unit based on the relationship with the represented audience position, and the sound detected by the microphone corresponding to the first user who is the user and adjusted by the sound adjustment unit. The synthesizer synthesizes the sound detected by the microphone corresponding to the second user, which is a different user from the first user, and adjusted to the sound adjustment unit, and outputs the data representing the sound synthesized by the synthesizer. Based on the relationship between the spectator position corresponding to the first user and the spectator position corresponding to the second user, the sound detected by the microphone corresponding to the second user is detected by the second sound adjustment unit. A sound processing method including adjusting and outputting sound data including a sound adjusted by a second sound adjusting unit to a sound output device that transmits the sound to a first user is provided.
According to the aspect of the present invention, a position showing the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area on the computer. Receiving information, adjusting the sound emitted by the user and detected by the microphone based on the position information, and transmitting data representing the adjusted sound can be executed and adjusted. , A speech processing program is provided that includes adjusting the first sound detected by the microphone to be closer to the second sound that reaches the performer position when the first sound is emitted at the audience position.
According to the aspect of the present invention, a position showing the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area on the computer. Receiving information, adjusting the sound emitted by the user and detected by the microphone based on the position information, and transmitting data representing the adjusted sound can be executed and adjusted. , A voice processing program is provided that includes adjusting the loudness of the sound detected by the microphone using the distance between the audience position and the performer position.
According to the aspect of the present invention, a position showing the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area on the computer. Sending information, receiving sound data emitted from the user and adjusted based on the position information from the destination of the position information, and receiving from the destination corresponding to the first user who is the user. Combining the sound represented by the data and the sound represented by the data received from the destination corresponding to the second user who is a different user from the first user, the performer is out of the predetermined area. A sound processing program is provided in which the synthesized sound data is provided to a sound output device arranged at a position where the sound reaches the performer.
According to the aspect of the present invention, a position showing the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area on the computer. Sending information, receiving sound data emitted from the user and adjusted based on the position information from the destination of the position information, and receiving from the destination corresponding to the first user who is the user. Combining the sound represented by the generated data and the sound represented by the data received from the destination corresponding to the second user who is a different user from the first user, the audience position corresponding to the first user, and the audience position corresponding to the first user. Adjusting the sound detected by the microphone corresponding to the second user based on the relationship with the audience position corresponding to the second user, and executing the sound adjusted corresponding to the second user. A sound processing program is provided in which the sound data including the above is output to a sound output device that transmits the sound to the first user.
According to the first aspect of the present invention, a microphone for detecting a sound emitted by a user participating in an event, a performer position representing an event performer's position in a predetermined area where the event is held, and a user's position in a predetermined area. A sound adjustment unit that adjusts the sound detected by the microphone based on the relationship with the audience position representing the above, a sound detected by the microphone corresponding to the first user who is the user, and adjusted by the sound adjustment unit. A synthesizer that synthesizes the sound detected by the microphone corresponding to the second user who is a different user from the first user and adjusted to the sound adjustment unit, and an output unit that outputs data representing the sound synthesized by the synthesizer. And, a sound processing system is provided.
According to an aspect of the present invention, a microphone that detects a sound emitted by a user participating in an event, a performer position that represents the position of the performer of the event in a predetermined area where the event is held, and a spectator that represents the position of the user in a predetermined area. A sound adjustment unit that adjusts the sound detected by the microphone based on the relationship with the position, a sound detected by the microphone corresponding to the first user who is the user, and adjusted by the sound adjustment unit, and the first user. A compositing unit that synthesizes the sound detected by the microphone corresponding to the second user who is a different user and adjusted to the sound adjusting unit, and an output unit that outputs data representing the sound synthesized by the compositing unit. Provided by the sound processing system, the sound adjustment unit adjusts the first sound detected by the microphone so as to approach the second sound that reaches the performer position when the first sound is emitted at the audience position. Will be done.

According to the second aspect of the present invention, a position indicating the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area. A receiving unit that receives information, a sound adjusting unit that adjusts the sound emitted from the user and detected by the microphone based on the position information, and a transmitting unit that transmits data representing the sound adjusted by the sound adjusting unit. A voice processing device comprising the above is provided.

According to the third aspect of the present invention, a position indicating the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area. A transmitter that transmits information, a receiver that receives sound data emitted from the user and adjusted based on the location information from the destination of the location information, and a destination corresponding to the first user who is the user. Provided is a voice processing device including a compositing unit that synthesizes a sound represented by data received from and a sound represented by data received from a destination corresponding to a second user who is a user different from the first user. Will be done.

According to the fourth aspect of the present invention, the sound emitted by the user participating in the event is detected by the microphone, the performer position representing the position of the performer of the event in the predetermined area where the event is held, and the user in the predetermined area. The sound detected by the microphone is adjusted by the sound adjustment unit based on the relationship with the audience position representing the position of, and is detected by the microphone corresponding to the first user who is the user and adjusted to the sound adjustment unit. The sound and the sound detected by the microphone corresponding to the second user, which is a user different from the first user, and adjusted by the sound adjustment unit are synthesized by the synthesis unit, and the sound synthesized by the synthesis unit is represented. A sound processing method including outputting data is provided.

According to the fifth aspect of the present invention, the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area on the computer. Voice processing that receives the position information indicating the above, adjusts the sound emitted from the user and detected by the microphone based on the position information, and transmits the data representing the adjusted sound. The program is provided.

According to the sixth aspect of the present invention, the computer is provided with a relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area. Sending the indicated position information, receiving sound data emitted from the user and adjusted based on the position information from the destination of the position information, and a destination corresponding to the first user who is the user. A voice processing program for synthesizing the sound represented by the data received from the first user and the sound represented by the data received from the destination corresponding to the second user who is a user different from the first user and executing the data is provided. ..

It is a figure which shows the voice processing system which concerns on 1st Embodiment. It is a figure which shows the process of setting the terminal which concerns on 1st Embodiment. It is a figure which shows the voice processing system which concerns on 1st Embodiment. It is a figure which shows the example of the sound adjustment method which concerns on 1st Embodiment. It is a figure which shows the voice processing method which concerns on 1st Embodiment. It is a figure which shows the voice processing system which concerns on 2nd Embodiment. It is a figure which shows the voice processing method work which concerns on 2nd Embodiment. It is a figure which shows the voice processing system which concerns on 3rd Embodiment. It is a figure which shows the example of the group which concerns on 3rd Embodiment. It is a figure which shows the voice processing system which concerns on 4th Embodiment.

[First Embodiment]
The first embodiment will be described. FIG. 1 is a diagram showing a voice processing system according to the first embodiment. The voice processing system 1 is, for example, an information processing system used for a service for distributing contents at various events (appropriately referred to as a content distribution service). The voice processing system 1 distributes the content expressing the performance of the performer P to the user U who is an audience. The voice processing system 1 transmits the reaction of the user U who views the content to the performer P. The performer P can perform a later performance while being aware of the reactions of the plurality of users U. A plurality of users U can, for example, watch a performance that changes according to their own reaction. The plurality of users U can experience the sense of presence even when participating in an event in a remote environment, for example. In this way, the voice processing system 1 can provide the user U with a new experience in the field of distributing the contents of various events. In the following description, any user is appropriately represented by a reference numeral U, and when the user U is distinguished, it is represented by a reference numeral in which the alphabets a, b, ... Are added to the reference numeral U, such as user Ua and user Ub.

Various events in which the audio processing system 1 is used include, for example, events related to at least one of music, drama, entertainment, sports, esports, competition, lectures, lectures, speeches, dialogues, discussions, and demonstration sales. The voice processing system 1 may be used for other events. An event is, for example, a place to show the skills of performer P to the audience. Performer P may be referred to as an artist, performer, singer, actor, entertainer, speaker, or player. The audience may be referred to as the audience, or viewer. The event may be attended by at least a part of the spectators for a fee, or may be attended by at least a part of the spectators free of charge. The event may be held in a real space, in a virtual space (appropriately referred to as cyberspace), or in parallel in the real space and the virtual space.

Reference numeral AE in FIG. 1 is a predetermined area where an event is held (appropriately referred to as an event venue AE). The event venue AE may be an area of real space or an area of virtual space. The event venue AE may be a facility that exists in the real space reproduced or expanded in the cyber space. For example, the participants of the event may include an audience who participates in the venue where the performer P performs, and an audience who participates in the venue where this venue is reproduced in cyberspace. Reference numeral P1 in FIG. 1 is a performer position representing the position of the performer in the event venue AE. The performer position P1 is, for example, a position preset with respect to the stage AE1 of the event venue AE. For example, the performer position P1 may be the center of the stage AE1.

The reference numeral AP in FIG. 1 is an area where the performer P performs (appropriately referred to as a demonstration area). The demonstration area AP is, for example, outside the event venue AE. The demonstration area AP is an arbitrary area in the real space. The demonstration area AP is, for example, a studio. The demonstration area AP may be an area in the residence of the performer P, an area in a commercial facility, or another area. The demonstration area AP may be an indoor area or an outdoor area (eg, a park, the roof of a building).

The performer P may be one person or a plurality of people. When there are a plurality of performers P, the performance area AP corresponding to the first performer P may be different from the performance area AP corresponding to the second performer P. When a plurality of performers P exist in one performance area AP, the performer position P1 may be determined for each performer P or may be determined at a position representing the plurality of performers P. The representative position of the plurality of performers P may be, for example, the position of each of the plurality of performers P represented by coordinates, and may be the average of the coordinates of the plurality of performers P (eg, the center of gravity of the positions of the plurality of performers).

The demonstration area AP may be an area within the event venue AE. For example, the demonstration area AP may be a stage in a real space venue. The performer position P1 may be fixed or variable. For example, the voice processing system 1 may include a sensor that detects one or both of the position and movement of the performer P in the demonstration area AP, and may determine the performer position P1 based on the detection result of the sensor. For example, the audio processing system 1 may change the performer position P1 from the center of the stage AE1 to the outside when the performer P moves from the center of the performance area AP to the outside.

Reference numeral Q in FIG. 1 is an spectator position representing the position of the spectator in the event venue AE. In the following description, the spectator position of any user U is represented by the code Q, and the alphabets a, b, ... Are added to the code Q, such as the spectator position Qa and the spectator position Qb, when distinguishing the spectator positions. It is represented by the code. The spectator position Qa represents the position of the user Ua who is the spectator. The spectator position Qb represents the position of the user Ub who is a different spectator from the user Ua. The audience position Qb is a position farther from the performer position P1 than the audience position Qa. The distance La between the performer position P1 and the audience position Qa is shorter than the distance Lb between the performer position P1 and the audience position Qb.

The reference numeral AU in FIG. 1 is an area (appropriately referred to as a viewing area AU) in which the user U, who is an audience, views the contents of the event. The viewing area AU is an arbitrary area in the real space. The viewing area AU is, for example, outside the event venue AE. The viewing area AU is, for example, an area in the residence of the user U. The viewing area AU may be an area different from the area in the user U's residence. The viewing area AU may be an area of facilities such as a karaoke box, a movie theater, a theater, and a restaurant (eg, a sports bar). The viewing area AU may be an area of a moving body such as a railroad, an aircraft, a ship, or a vehicle. The viewing area AU may be an area within the event venue AE. For example, the event venue AE may be a drive-in theater, and the viewing area AU may be an area in a vehicle parked at the event venue AE. The viewing area AU may be indoors or outdoors.

In the following description, when an arbitrary viewing area is appropriately represented by a code AU and the viewing area AU is distinguished, a code in which the alphabets a, b, ... Are added to the code AU, such as the viewing area AUa and the viewing area AUb. It is represented by. For example, the viewing area AUa is an area where the user Ua views the content, and the viewing area AUb is an area where the user Ub views the content. The number of viewers existing in one viewing area AU may be one only for the user U or may be a plurality of viewers including the user U.

The voice processing system 1 includes a processing device 2, a terminal 3, a camera 4, a microphone 5, and a speaker 6. The camera 4, the microphone 5, and the speaker 6 are arranged in the demonstration area AP. The camera 4, the microphone 5, and the speaker 6 are connected to the processing device 2. For example, the camera 4, the microphone 5, and the speaker 6 are connected by wire or wirelessly to an information processing device (eg, a computer) including a communication unit. This information processing device is communicably connected to the processing device 2 via, for example, an internet line. The terminal 3 is arranged in the viewing area AU. The terminal 3 is communicably connected to the processing device 2 via, for example, an internet line.

The camera 4 captures the performer P performing in the demonstration area AP. The microphone 5 detects the sound emitted by the performer P who performs in the demonstration area AP. The voice produced by the performer P includes, for example, one or both of the sound produced by the performer P from the body (eg, voice, clapping) and the sound produced by the performer P by a tool (eg, musical instrument). The video data captured by the camera 4 and the audio data detected by the microphone 5 are provided to the processing device 2. The speaker 6 outputs sound to the performer P. The speaker 6 outputs voice based on the voice data provided by the processing device 2. The processing device 2 generates content using data (eg, video data, audio data) provided by devices (eg, camera 4, microphone 5) installed in the demonstration area AP. The processing device 2 provides the generated content data via, for example, an internet line.

The voice processing system 1 does not have to include at least one of a camera 4, a microphone 5, and a speaker 6. For example, at least one of the camera 4, the microphone 5, and the speaker 6 may be an external device of the voice processing system 1. The voice processing system 1 may be in the form of using the external device. The camera according to the embodiment may be called an imaging device. The microphone according to the embodiment may be referred to as a voice detection device. The speaker according to the embodiment may be referred to as an audio output device.

The terminals 3 are arranged for each viewing area AU, for example. In the following description, when the terminals 3 arranged in the viewing area AU are appropriately distinguished, they are represented by reference numerals having alphabets added, such as terminals 3a and 3b. For example, the terminal 3a is a terminal arranged in the viewing area AUa, and the terminal 3b is a terminal arranged in the viewing area AUb. The terminal 3b executes the same processing as the terminal 3a, for example. In the following description, the terminal 3b has the same reference numerals as the terminal 3a for the same configuration as the terminal 3a, and the description of the terminal 3b overlapping with the terminal 3a is omitted or simplified.

The terminal 3 acquires the data of the content provided by the processing device 2, for example, via an internet line. The terminal 3 uses the acquired data to provide content to the user. In addition, the terminal 3 detects the sound emitted by the user. The sound produced by the user includes, for example, one or both of the sound produced by the user from the body (eg, voice, clapping) and the sound produced by the user by a tool (eg, noise). In the following description, the sound emitted by the user is appropriately referred to as an audience sound. The terminal 3 adjusts the audience sound based on the relationship between the performer position P1 and the audience position Q associated with the own device. For example, the terminal 3 adjusts the audience sound so that the longer the distance between the performer position P1 and the audience position Q, the lower the volume of the audience sound. The terminal 3 provides the adjusted audience sound data, for example, via an internet line.

The processing device 2 synthesizes the spectator sounds supplied from the plurality of terminals 3 and provides the combined spectator sound data. In the demonstration area AP, the speaker 6 outputs the audience sound to the performer P based on the audience sound data supplied from the processing device 2. The audience sound R output from the speaker 6 includes, for example, a component Ra derived from the audience sound from the user Ua and a component Rb derived from the audience sound from the user Ub. The audience position Qa corresponding to the user Ua is closer to the performer position P1 than the audience position Qb corresponding to the user Ub, and the component Ra derived from the audience sound of the user Ua is the component Rb derived from the audience sound of the user Ub. The volume is louder than that.

Since the audience sound output from the speaker 6 contains a component adjusted based on the audience position Q, it contributes to, for example, transmitting a live feeling to the performer. For example, the performer P can perform a performance so as to respond to the reactions of a plurality of users U by listening to the audience sound having a live feeling. As a result, the plurality of users U can enjoy, for example, live performance content.

In the present embodiment, the user U applies for participation in the event (appropriately referred to as a participation application) before the event is held. Here, it is assumed that the application destination of the participation application is the event organizer. When the event organizer accepts the participation of the user U, the event organizer sets the terminal 3 for the user U. For example, the event organizer stores various information (eg, data, program) used for providing the service described in FIG. 1 in the terminal 3. The event organizer provides the set terminal 3 to the user U. The user U can use the terminal 3 provided by the event organizer to participate in the event, watch the content, and convey the reaction to the performer P. As described above, the terminal 3 according to the present embodiment also serves as a ticket indicating the right to participate in the event.

FIG. 2 is a diagram showing a process of setting a terminal according to the first embodiment. User Ua applies for participation in the event prior to holding the event. The user Ua operates, for example, a terminal installed in a store such as a convenience store (appropriately referred to as a reservation terminal 7), and inputs information regarding a participation application (appropriately referred to as entry information). The entry information includes, for example, information that identifies the user Ua, information that identifies the event that the user Ua wants to participate in, and information on the seats of the event venue AE that the user Ua wants to participate in.

The information that identifies the user is, for example, the user's name or identification information. Information that identifies a user as appropriate in the following description is referred to as a user ID. The information that identifies the event is, for example, the identification information (eg, number) assigned to each event, the name of the event, or the like. In the following description, information that identifies an event is appropriately referred to as an event ID. The seat information includes, for example, information for specifying the seat rank (S seat, A seat), information for specifying seat conditions (eg, upstairs seat, arena, aisle side), and information for specifying the seat number. Includes at least one of.

The entry information input to the reservation terminal 7 is provided to the application destination terminal (appropriately referred to as a reception terminal 8). For example, the reservation terminal 7 is connected to an internet line and transmits entry information. The reception terminal 8 receives the entry information transmitted by the reservation terminal 7.

The reception terminal 8 is, for example, a server that manages user information. The reception terminal 8 includes a storage unit 11 and a communication unit 12. The communication unit 12 is communicably connected to the reservation terminal 7 and receives the entry information transmitted by the reservation terminal 7. The reception terminal 8 includes an application acquisition unit for acquiring a participation application provided by a user. The reception terminal 8 determines the seat at the event venue AE of the user Ua based on the received entry information. The reception terminal 8 determines the seat by, for example, a lottery. The reception terminal 8 may determine the seat by a method different from the lottery, or may determine the seat by a method combining the lottery and other methods. For example, the reception terminal 8 may allocate seats to the user U corresponding to the entry information in the order in which the entry information is received, or may determine the seat based on the seat information shown in the entry information.

The storage unit 11 of the reception terminal 8 stores the user information D1. The user information D1 includes, for example, a user ID and identification information that identifies a seat determined as a seat of the user Ua. In the following description, the identification information that identifies the seat is referred to as a seat ID as appropriate. The user information D1 includes, for example, table data in which a user ID and a seat ID are associated with each of the plurality of users U. The user information D1 may include information different from the user ID and the seat ID. For example, the user information D1 may include a history of events in which the user Ua has participated in the past. The user information D1 may include at least one of the age, gender, and other attribute information of each user U. At least a part of the user information D1 stored in the storage unit 11 is provided to the processing device 2.

The reception terminal 8 may determine a seat when the first user participates in an event after the past event based on the history of the first user participating in the past event. For example, the user information D1 stored in the storage unit 11 includes a history of each user participating in a past event (appropriately referred to as a participation history). The reception terminal 8 determines the seat for the current participation application of the first user by using, for example, the participation history included in the user information D1. For example, in the content distribution service according to the embodiment, the user is given an attribute (eg, rank) according to the number of times he / she has participated in the event. For example, a user's rank (eg silver, gold, platinum) increases as the number of times they attend an event increases. As the rank of the user is higher, the reception terminal 8 may determine a preferential seat as the seat of the first user in this event. The preferential seats are, for example, the seats relatively close to the stage AE1 at the event venue AE shown in FIG. The rank of the user may be determined by a parameter other than the number of participations, and may be determined according to, for example, the amount of money spent attending the event. The rank of the user may be determined by one parameter, may be determined by a plurality of parameters, or may be randomly determined by lottery or the like.

The processing device 2 determines the audience position Q of the user Ua based on the participation application of the user Ua. The processing device 2 includes a processing unit 21, a storage unit 22, and a communication unit 23. The communication unit 23 is communicably connected to the reception terminal 8. The communication unit 23 receives the user information transmitted by the reception terminal 8. The storage unit 22 stores information D2 of the performer position P1 (see FIG. 1) of the event venue AE. The information D2 of the performer position P1 stored in the storage unit 22 is, for example, coordinates in cyberspace.

The processing unit 21 includes a positioning unit 24. The position determination unit 24 determines the audience position Q representing the user's position in the predetermined area where the event is held (eg, the event venue AE in FIG. 1) based on the user's application for participation in the event. The position determination unit 24 determines the spectator position Q based on, for example, the participation application acquired by the reception terminal 8 which is the application acquisition unit. For example, the position determination unit 24 derives the coordinates in cyberspace corresponding to the seat ID determined based on the participation application, and determines the derived coordinates as the spectator position Q. The position determination unit 24 derives the coordinates in the same coordinate system as the performer position P1 corresponding to the seat ID. For example, the storage unit 22 stores map data in which the seat ID and the spectator position Q are associated with each other. The position determination unit 24 acquires the seat ID included in the user information received by the communication unit 23, collates the seat ID with the map data, and determines the spectator position Q. The position determination unit 24 determines the relative positions of the plurality of spectator positions Q by, for example, determining the respective spectator positions Q of the plurality of users U.

The position determination unit 24 determines the audience position Q when the first user participates in an event after the past event based on the history of the first user (eg, user Ua) participating in the past event. You may decide. For example, the position determination unit 24 may determine the spectator position Q corresponding to this seat based on the seat of the user Ua in the current event determined by the reception terminal 8 based on the participation history of the user Ua.

The processing unit 21 generates position information indicating the relationship between the performer position P1 and the audience position Q determined by the position determination unit 24. The position information includes, for example, the distance between the performer position P1 and the audience position Q (eg, the distance La in FIG. 1). The position information may include information different from the distance from the performer position P1 to the audience position Q. For example, the position information may include information in which the performer position P1 and the audience position Q are paired. The position information may include information (eg, a vector) in the direction connecting the performer position P1 and the audience position Q. The position information may include spatial information between the performer position P1 and the audience position Q (eg, information indicating the level of sound attenuation from the audience position Q to the performer position P1, the presence or absence of obstacles). The position information includes the distance between the performer position P1 and the audience position Q, the information in which the performer position P1 and the audience position Q are paired, the information in the direction connecting the performer position P1 and the audience position Q, and the performer position P1 and the audience. It is not necessary to include at least one of the information of the space between the position Q and the information, and the information different from any of these information may be included.

The position information generated by the processing device 2 is supplied to the terminal 3a. The terminal 3 includes a storage unit 31 and a communication unit 32. The communication unit 32 is communicably connected to, for example, the communication unit 23 of the processing device 2. The communication unit 32 receives the position information transmitted by the communication unit 23 of the processing device 2. The terminal 3 stores the position information received by the communication unit 23 in the storage unit 31. The terminal 3a stores the position information D3a indicating the relationship between the performer position P1 shown in FIG. 1 and the audience position Qa determined based on the participation application of the user Ua. The terminal 3a is provided to the user Ua by, for example, mail, and is used when the user Ua participates in the event.

The terminal 3b has the same configuration as the terminal 3a. The terminal 3b stores the position information D3b indicating the relationship between the performer position P1 shown in FIG. 1 and the audience position Qb determined based on the participation application of the user Ub. The terminal 3b is delivered to the user Ub by, for example, a courier service, and is used when the user Ub participates in the event. In the following description, the position information indicating the relationship between the audience position Q corresponding to the arbitrary user U and the performer position P1 is represented by reference numeral D3 as appropriate.

The reservation terminal 7 may be a part of the voice processing system 1 or an external device of the voice processing system 1. The reservation terminal 7 may be a terminal such as a smartphone or a personal computer owned by the user Ua. The reservation terminal 7 that receives the entry information from the user Ub may be the same device as the reservation terminal 7 that receives the entry information from the user Ua, or may be a different device. The process of inputting at least a part of the entry information to the reservation terminal 7 may be executed by a person different from the user Ua. For example, the user Ua may notify the receptionist of the participation application by the store or telephone, and the receptionist may input the entry information into the reservation terminal 7. The application destination for the participation application does not have to be the event organizer. For example, the application destination of the participation application may be a trustee entrusted by the event organizer, a distributor of content related to the event, or an event manager.

The process of determining the seat of the user Ua may be manually executed by the person in charge of the application destination. For example, the user Ua may mail the application form on which the participation application is filled in to the application destination, and the person in charge of the application destination may determine the seat of the user Ua based on the information described in the application form. The person in charge of the application destination may input the seat ID determined as the seat of the user Ua into the reception terminal 8 in association with the user ID. The reception terminal 8 or the person in charge may decide whether or not to grant the user Ua the right to participate in the event (appropriately referred to as the participation right), and then decide the seat of the user Ua. For example, when there is a participation application from a number of users U exceeding the capacity, or when it is predicted that there is a participation application, the reception terminal 8 may decide whether or not to grant the participation right by lottery.

The reception terminal 8 may be a part of the voice processing system 1 or an external device of the voice processing system 1. For example, the processing device 2 may include a reception terminal 8. The reception terminal 8 may also serve as the reservation terminal 7. The function of the processing device 2 may be realized by being divided into a plurality of devices, and these plurality of devices may be collectively referred to as a processing system.

The method of storing the position information in the terminal 3 may be a method that does not use wired or wireless communication. For example, the position information may be stored in the terminal 3 via a recording medium such as a USB memory. The terminal 3 does not have to store the position information in advance before being delivered to the user U. For example, the processing device 2 may store the destination associated with the terminal 3 and transmit the location information to the destination after the user U receives the terminal 3. The processing device 2 may update the position information stored in the terminal 3 by transmitting the position information to the destination.

FIG. 3 is a diagram showing a voice processing system according to the first embodiment. Here, the operation of each part of the voice processing system 1 when transmitting the sound from the user to the performer will be described. The processing device 2 includes, for example, an information processing device such as a server computer. The processing device 2 manages each part of the voice processing system 1. The processing device 2 includes a processing unit 21, a communication unit 23, an input unit 25, and an output unit 26. A camera 4, a microphone 5, and a speaker 6 are connected to the processing device 2.

The input unit 25 includes an interface in which a signal is input from the outside of the processing device 2. A camera 4 and a microphone 5 are connected to the input unit 25. The video data captured by the camera 4 and the audio data collected by the microphone 5 are input to the processing device 2. The output unit 26 includes an interface that outputs a signal to the outside of the processing device 2. A speaker 6 is connected to the output unit 26. At least one of the camera 4, the microphone 5, and the speaker 6 may be a part of the voice processing system 1 or a part of the processing device 2. At least one of the camera 4, the microphone 5, and the speaker 6 may be an external device of the audio processing system 1, for example, the equipment of the demonstration area AP shown in FIG.

The processing unit 21 includes a synthesis unit 27 and a content generation unit 28. The content generation unit 28 generates content using the data input to the input unit 25. The content generation unit 28 generates, for example, content in which the performance of the performer P is expressed by video and audio. For example, the content generation unit 28 generates content to be distributed to the user U by using the video data input from the camera 4 to the input unit 25 and the audio data input from the microphone 5 to the input unit 25. .. The content generation unit 28 synchronizes video and audio, for example, and generates a streaming moving image as content. The content generation unit 28 may generate content by executing at least one of video processing, various sound processing, filtering, and compression processing. The communication unit 23 transmits the data of the content generated by the content generation unit 28.

The terminal 3a includes a storage unit 31, a communication unit 32, and a processing unit 33. The communication unit 32 is communicably connected to the communication unit 23 of the processing device 2. The communication unit 32 receives the data of the content transmitted by the communication unit 23. A microphone 41, a speaker 42, and a display device 43 are connected to the terminal 3a. The microphone 41 and the speaker 42 may be a headset. The display device 43 is a device that displays an image of a television set, a PC monitor, a projector, or the like.

The processing unit 33 of the terminal 3a causes the speaker 42 to output the sound of the content based on the data of the content received by the communication unit 32. The processing unit 33 of the terminal 3a causes the display device 43 to display the video of the content based on the data of the content received by the communication unit 32. The user Ua can view the content expressing the performance of the performer P by the sound output from the speaker 42 and the video displayed on the display device 43. The microphone 41 detects a sound emitted by a user who watches the performer P of the event. For example, the microphone 41 detects the cheers and applause of the user Ua who views the content. The audience sound data detected by the microphone 41 is output to the terminal 3a. The processing unit 33 of the terminal 3a includes a sound adjusting unit 34. The sound adjustment unit 34 adjusts the sound detected by the microphone 41 based on the relationship between the performer position P1 and the audience position Qa.

FIG. 4 is a diagram showing an example of a voice adjustment method according to the first embodiment. Reference numeral D5 in FIG. 4 is audio data input to the sound adjustment unit 34, and reference numeral D6 is audio data output from the sound adjustment unit 34. In each of the data D5 and the data D6, the horizontal axis is the frequency and the vertical axis is the amplitude. The frequency corresponds to the pitch and the amplitude corresponds to the volume. Reference numeral D7 is a function (eg, filter) used to convert data D5 to data D6. The function D7 is a function showing the relationship between the distance and the gain. The gain corresponds to the rate of sound attenuation with respect to distance. In this example, the sound adjusting unit 34 adjusts the loudness of the sound detected by the microphone 41 by using the distance La between the audience position Qa and the performer position P1 shown in FIG. Here, it is assumed that the gain is constant in the frequency band of audible sound (eg, 20 Hz or more and 20 kHz or less).

The gain is a value that is uniquely determined with respect to the distance. For example, for the distance La shown in FIG. 1, the gain is determined to be G1. The sound adjustment unit 34 generates the data D6 by applying the gain G1 to the amplitude of the data D5 representing the audience sound of the user Ua. For example, the sound adjustment unit 34 adjusts the first sound detected by the microphone 41 so as to approach the second sound that reaches the performer position P1 when the first sound is emitted at the audience position Q. The distance between the microphone 41 and the user Ua is, for example, the type of viewing area AUa shown in FIG. 1 (eg, home, movie theater), the type of device used for viewing content (eg, smartphone, personal computer, television). It is largely determined by the set) and the layout of the equipment.

Here, it is assumed that the distance La between the performer position P1 and the audience position Qa shown in FIG. 1 is, for example, 20 mails. It is assumed that the terminal 3a is arranged in the living room of the user Ua and is connected to the television set, and the distance between the microphone 41 and the user Ua is 2 meters. In this case, the distance La between the performer position P1 and the audience position Qa (eg, 20 meters) is longer than the distance between the microphone 41 and the user Ua (eg, 2 meters). The sound adjustment unit 34 makes the audience sound detected by the microphone 41 2 meters away from the user Ua approach the sound heard at the performer position P1 20 meters away from the audience position Qa when it is emitted at the audience position Qa. To adjust the volume. For example, the data D6 corresponds to the data of the audience sound heard at the performer position P1 when the audience sound is emitted at the audience position Qa. In the example of FIG. 4, the gain G1 corresponding to the distance La is less than 1, and the amplitude of the data D6 is smaller than that of the data D5. The sound represented by the data D6 has a lower volume than the sound represented by the data D5, and corresponds to data representing the audience sound heard at the performer position P1 when the audience sound is emitted at the audience position Qa.

The sound adjusting unit 34 may adjust a waveform representing an amplitude with respect to the frequency. For example, the sound adjusting unit 34 may adjust the audience sound by using different gains depending on the frequency band. For example, the sound adjusting unit 34 may attenuate the relatively high-pitched sound component of the audience sound more than the relatively low-pitched sound component. The sound adjustment unit 34 does not have to attenuate the audience sound, and may amplify the audience sound. Even if the sound adjustment unit 34 adjusts the audience sound detected by the microphone 41 so that the volume is louder than the audience sound that reaches the performer position P1 when the audience sound is emitted at the audience position Q. good. When the audience sound detected by the microphone 41 has a volume exceeding a predetermined value, the sound adjusting unit 34 cuts the volume exceeding the predetermined value, and the audience sound from which the volume is cut is referred to as the audience position Q and the performer position P1. It may be adjusted based on the relationship of. The sound adjustment unit 34 may adjust the audience sound so that the volume of the adjusted audience sound does not exceed a predetermined value.

The sound adjustment unit 34 may adjust the audience sound by using the information indicating the relationship between the position of the microphone 41 and the position of the user U and the information indicating the relationship between the audience position Q and the performer position P1. The information representing the relationship between the position of the microphone 41 and the position of the user U is, for example, the distance between the microphone 41 and the user Ua. The sound adjusting unit 34 may use a value detected by the sensor as the distance between the microphone 41 and the user Ua. The sound adjusting unit 34 may use a predetermined value as the distance between the microphone 41 and the user Ua. The distance between the microphone 41 and the user Ua may be, for example, a value (eg, a recommended value) within a range defined in the terms of use of the service provided by the voice processing system 1.

Returning to the description of FIG. 3, the sound adjusting unit 34 according to the present embodiment adjusts the audience sound by using the position information D3a stored in the storage unit 31. The position information D3a includes, for example, the distance La between the audience position Qa and the performer position P1 shown in FIG. The sound adjusting unit 34 acquires the gain G1 with respect to the distance La by using, for example, the function D7 shown in FIG. For example, the function D7 is stored in the storage unit 31, and the sound adjustment unit 34 reads the function D7 and the position information D3a (eg, distance La) from the storage unit 31 to derive the gain G1. The sound adjustment unit 34 generates data D6 (see FIG. 4) by applying the derived gain G1 to the data D5 (see FIG. 4) indicating the audience sound detected by the microphone 41. In the following description, the audience sound adjusted by the sound adjustment unit 34 as appropriate is referred to as the adjusted audience sound. The data D6 is data representing the adjusted audience sound.

The position information D3a may include, for example, information in which the audience position Qa and the performer position P1 represented by the coordinates in cyberspace are paired. In this case, the sound adjusting unit 34 may calculate the distance La between the audience position Qa and the performer position P1 by using the audience position Qa and the performer position P1. The function D7 may be stored outside the terminal 3, and the sound adjusting unit 34 may acquire information representing the function D7 from the outside of the terminal 3 via the communication unit 32. The function D7 may be represented by a mathematical formula or table data. The position information D3a may include information (eg, gain G1) representing a change in sound transmitted from the audience position Qa to the performer position P1. In this case, the sound adjusting unit 34 may adjust the audience sound without using the function D7 shown in FIG.

The terminal 3 provides the adjusted audience sound data to the outside of the terminal 3. For example, the processing unit 33 of the terminal 3 controls the communication unit 32 to transmit the adjusted audience sound data. The processing device 2 acquires the adjusted audience sound data provided by the terminal 3. For example, the communication unit 23 of the processing device 2 receives the adjusted audience sound data transmitted by the communication unit 32 of the terminal 3. The processing device 2 acquires the adjusted audience sound data of the user Ua from the terminal 3a corresponding to the user Ua. The processing device 2 acquires the adjusted audience sound data of the user Ub from the terminal 3b corresponding to the user Ub.

The synthesis unit 27 is detected by the microphone 41 corresponding to the user Ua and adjusted by the sound adjustment unit 34 of the terminal 3a, and is detected by the microphone 41 corresponding to the user Ub and is detected by the sound adjustment unit 34 of the terminal 3b. Combine with the adjusted audience sound. The synthesizing unit 27 superimposes the spectator sound data acquired from the terminal 3a and the spectator sound data acquired from the terminal 3b, and generates data representing the synthesized spectator sound. In the following description, the adjusted audience sound synthesized by the synthesis unit 27 is referred to as the synthesized audience sound.

The output unit 26 outputs data representing the audience sound synthesized by the synthesis unit 27. For example, the output unit 26 outputs digital data of audience sound, and this digital data is DA-converted into analog data (eg, audio signal) by an amplifier and input to the speaker 6. The speaker 6 converts analog data into vibration and outputs the synthesized audience sound. The amplifier may be provided by the processing device 2, a speaker 6, or a device connected between the processing device 2 and the speaker 6. The synthesized audience sound data provided by the processing device 2 to the outside may be digital data or analog data (eg, audio signal). The processing device 2 may output audience sound data from the communication unit 23 by communication. In this case, the communication unit 23 may be an output unit that outputs data representing the sound synthesized by the synthesis unit 27.

The performer P can perform while listening to the audience sound output from the speaker 6, for example. The microphone 5 detects the sound in synchronization with the output of the audience sound by the speaker 6, for example. The microphone 5 detects the audience sound output from the speaker 6 and the sound emitted by the performer P together. For example, the camera 4 detects the sound at a predetermined timing with respect to the timing at which the audience sound is output from the speaker 6. The camera 4 takes a picture of the performer P in synchronization with the sound detection by the microphone 5. At least two timings, that is, the timing at which the sound is output from the speaker 6, the timing at which the microphone 41 detects the sound, and the timing at which the camera 4 shoots the performer P, are controlled by the processing device 2 or other devices. You may.

The processing device 2 acquires the sound detected by the microphone 5 and the image captured by the camera 4. The content generation unit 28 generates content data using the voice detected by the microphone 5 and the video captured by the camera 4. The processing device 2 distributes the content generated by the content generation unit 28. For example, the communication unit 23 transmits the data of the content generated by the content generation unit 28. The sound included in the content generated by the content generation unit 28 includes the synthesized audience sound output from the speaker 6 and the sound emitted by the performer P. The user U can watch the video of the performer P while listening to the sound including the synthesized audience sound and the sound emitted by the performer P, for example. As described above, the communication unit 23 may be an output unit that outputs sound data in which the sound emitted by the performer P and the sound adjusted by the sound adjustment unit 34 are combined.

The microphone 5 does not have to detect the audience sound output from the speaker 6. For example, the speaker 6 may be earphones or headphones. The microphone 5 and the speaker 6 may be a headset. In this case, even if the content generation unit 28 of the processing device 2 synthesizes the sound detected by the microphone 5 (for example, the sound emitted by the performer P) and the audience sound synthesized by the synthesis unit 27 to generate the content. good. The content delivered by the processing device 2 does not have to include the audience sound after synthesis. Even in this case, the performer P can perform, for example, according to the audience sound after synthesis, and the user can experience a sense of reality by, for example, viewing the performance of the performer P who reacts to the audience. ..

Although the content generation unit 28 is provided in the same device as the synthesis unit 27 in FIG. 3, it may be provided in a device different from the composition unit 27. For example, the synthesis unit 27 may be provided in a first streaming server that processes audience sound data provided by a plurality of terminals 3. The content generation unit 28 may be provided in a second streaming server that processes data provided by devices (eg, camera 4, microphone 5) arranged in the demonstration area AP.

The terminal 3 may or may not be returned from the user to the event organizer after the end of the event. The terminal 3 may be reused in the events after this event. For example, the user Ua applies for participation in the second event after the first event. The processing device 2 determines the spectator position Qa in the second event based on the application for participation in the second event. The processing device 2 provides the terminal 3a with the position information D3a corresponding to the spectator position Qa in the second event. The sound adjustment unit 34 of the terminal 3a may adjust the audience sound detected by the microphone 41 by using the position information D3a corresponding to the second event.

In addition, in FIGS. 1 to 3 and the like, two users, user Ua and user Ub, are typically shown as the user U. The number of users U is two in FIGS. 1 to 3, but is arbitrary. For example, the number of users U may be 100, 10,000, or 10,000.

Next, the voice processing method according to the embodiment will be described based on the configuration of the voice processing system 1 described above. FIG. 5 is a diagram showing a voice processing method according to the first embodiment. For the configuration of the voice processing system 1, refer to FIGS. 1 to 3 as appropriate. Each process shown in FIG. 5 is executed by each part of the voice processing system 1 during, for example, an event. As described with reference to FIG. 2, before the event is held, the position determination unit 24 determines the spectator position Q based on the participation application of the user U, and the terminal 3 that stores the position information corresponding to the spectator position Q. Is delivered to the user.

In FIG. 5, the first user corresponds to the user Ua of FIGS. 1 to 3, and the first user terminal corresponds to the terminal 3a corresponding to the user Ua. The second user corresponds to the user Ub of FIGS. 1 to 3, and the second user terminal corresponds to the terminal 3b corresponding to the user Ub.

The processing device transmits the content in step S1. For example, the communication unit 23 of the processing device 2 of FIG. 3 transmits content data. The first user terminal receives the data of the content transmitted by the processing device. The first user terminal reproduces the content based on the received content data. The first user terminal detects the audience sound from the first user in step S2. In step S3, the first user terminal adjusts the audience sound using the position information. For example, the sound adjusting unit 34 of the terminal 3a of FIG. 3 adjusts the audience sound detected in step S2 by using the position information D3a stored in the storage unit 31. The first user terminal transmits the audience sound data adjusted in step S4. For example, the communication unit 32 of the terminal 3a of FIG. 3 transmits the adjusted audience sound data. The processing device receives the adjusted audience sound data transmitted by the first user terminal in step S4.

The processing of steps S5 to S7 is the same as the processing of steps S2 to S4. The second user terminal receives the content data transmitted by the processing device in step S1. The second user terminal detects the audience sound from the second user in step S5. In step S6, the second user terminal adjusts the audience sound using the position information. The second user terminal transmits the audience sound data adjusted in step S7. The processing device receives the adjusted audience sound data transmitted by the second user terminal in step S7.

The processing device synthesizes the adjusted audience sound in step S8. For example, the synthesizing unit 27 of the processing device 2 of FIG. 3 synthesizes the spectator sound data transmitted from the terminal 3a and the spectator sound data transmitted from the terminal 3b. In step S9, the processing device outputs the synthesized audience sound to the performer. For example, the processing device 2 outputs the synthesized audience sound data to the speaker 6, and outputs the audience sound to the performer P by the speaker 6. In step S10, the processing device transmits the content including the synthesized audience sound. For example, the microphone 5 of FIG. 3 detects a sound including a sound emitted by the performer P and a sound output from the speaker 6. Then, the content generation unit 28 of the processing device 2 generates content using the sound detected by the microphone 5. The communication unit 23 of the processing device 2 transmits the data of the content generated by the content generation unit 28.

The first user terminal and the second user terminal each receive the content data transmitted by the processing device. The first user terminal and the second user terminal reproduce the content based on the received content data. The processes of steps S2 to S10 are repeatedly executed, for example, during the holding of an event. The cycle in which the series of processes from step S2 to step S10 is repeated is arbitrarily set.

In the voice processing system 1 according to the above-described embodiment, the position determination unit 24 determines the user U in a predetermined area (eg, event venue AE) where the event is held based on the application for participation in the event from the user U. The audience position Q representing the position is determined. The microphone 41 detects the sound emitted by the user U who is viewing the performer P (eg, content) of the event. The sound adjustment unit 34 adjusts the sound detected by the microphone 41 based on the relationship between the performer position P1 representing the position of the performer of the event in the predetermined area and the audience position Q. The synthesis unit 27 includes a sound detected by the microphone 41 corresponding to the first user (eg, user Ua) who is the user U and adjusted by the sound adjustment unit 34, and a second user who is a user U different from the first user. (Example, the sound detected by the microphone 41 corresponding to the user Ub) and adjusted by the sound adjusting unit 34 is synthesized. The output unit 26 outputs data representing the sound synthesized by the synthesis unit 27.

The voice processing system 1 can convey the reaction of the user U to the performer P. The performer P can perform a performance conscious of the reaction of the user U, for example. User U can, for example, view a performance influenced by his or her reaction. In this way, the voice processing system 1 according to the embodiment can provide a new experience.

In the present embodiment, the application acquisition unit (eg, the reception terminal 8) acquires the participation application provided by the user U. The position determination unit 24 determines the audience position Q based on the participation application acquired by the application acquisition unit. In this case, for example, the process from the acquisition of the participation application to the determination of the spectator position Q can be automatically performed. The voice processing system 1 does not have to include an application acquisition unit (eg, a reception terminal 8). For example, information on a user participating in an event may be input to the processing device 2, and the position determination unit 24 may determine the position of the spectator using this information.

At least some of the users participating in the event (eg, first user) watch the performer P outside, for example, a predetermined area (eg, event venue AE). The microphone 41 corresponding to the first user (eg, user Ua) is arranged at a position (eg, viewing area AUa) where the sound emitted by the first user is detected. According to this form, the number of users who can participate in the event is less likely to be restricted by the size of the event venue as compared with the form in which the event is held only in the real space. For example, according to this form, in addition to participating in an event at a venue in real space, it is possible to participate in the event from outside the venue in real space, and the number of spectators exceeding the capacity of the venue in real space is the event. You can participate in. According to this form, for example, a larger audience can enjoy the event. According to this form, for example, it is possible to create an opportunity for a user who lives far from the venue in the real space to participate outside the venue, and to improve the convenience of the user.

Performer P performs, for example, outside a predetermined area (eg, event venue AE). The sound data synthesized by the synthesis unit 27 is provided to a voice output device (eg, speaker 6) arranged at a position where the sound reaches the performer P (eg, the demonstration area AP). According to this form, the position where the performer P actually exists is less likely to be restricted by the position of the event venue, as compared with the form in which the performer P performs at the event venue in the real space. For example, an event can be performed even when there are a plurality of performers P and some performers P are separated from other performers P.

In the present embodiment, the terminal 3 corresponds to a first voice processing device including a sound adjusting unit 34 and a transmitting unit (eg, a communication unit 32) for transmitting sound data adjusted by the sound adjusting unit 34. The processing device 2 includes a receiving unit (eg, communication unit 23) that receives data transmitted by the transmitting unit of the first voice processing device, and a synthesizing unit 27 that synthesizes the sound represented by the data received by the receiving unit. 2 Corresponds to a voice processing device. According to this form, for example, the process of adjusting the audience of a plurality of users is distributed to the plurality of first voice processing devices, and the load on the second voice processing device can be reduced. The transmission unit of the second audio processing device may transmit position information indicating the relationship between the performer position and the audience position determined by the position determination unit. The receiving unit of the first voice processing device may receive the position information transmitted by the transmitting unit of the second voice processing device. The sound adjusting unit of the first voice processing device may adjust the sound detected by the microphone of the first voice processing device based on the position information received by the receiving unit of the first voice processing device.

In the present embodiment, the voice processing system 1 may generate content by variously combining virtual information and real information. For example, the event venue AE may include an actual facility or location (concert hall, stadium, event stage, etc.). The event venue AE may include virtual facilities and environments (stages constructed in virtual space, sea, mountains, sky, etc.). The voice processing system 1 may generate content by using artificial spectator sounds and sound effects generated for the event in addition to the spectator sounds emitted from the user.

In this embodiment, one or both of the terminal 3 and the processing device 2 includes a computer. The computer according to the embodiment includes, for example, at least one of a portable device such as a smartphone, a tablet, or a laptop computer, or a stationary device such as a server device, a desktop personal computer, or a tower personal computer. The computer includes, for example, a processing unit, a storage unit, a communication unit, and an input / output interface.

The processing unit includes, for example, a general-purpose processor (eg, CPU). The storage unit includes, for example, one or both of a non-volatile storage device and a volatile storage device. The non-volatile storage device is, for example, a hard disk, a solid state drive, or the like. Volatile storage devices include, for example, random access memory, cache memory, work memory, and the like. The communication unit performs wired or wireless communication in accordance with a predetermined communication standard. The communication unit may perform communication by combining two or more types of communication having different communication standards. The predetermined communication standard may include a LAN, a short-range wireless communication standard such as Bluetooth (registered trademark), an infrared communication standard, or other standards.

The terminal 3 may be referred to by a processing device, an information processing device, an electronic device, or another name. At least a part of the processing of the terminal 3 may be executed by a digital signal processor or an integrated circuit for a specific purpose. The processing device 2 may be referred to by a terminal, an information processing device, an electronic device, or other names. At least a part of the processing of the processing device 2 may be executed by a digital signal processor or an integrated circuit for a specific application.

When the terminal 3 is a computer, the processing unit reads a program stored in the storage unit and executes various processes according to this program. This program, for example, tells a computer the audience position representing the position of the user in a predetermined area where the event is held and the position of the performer of the event in the predetermined area, which is determined based on the user's application for participation in the event. Receiving position information indicating the relationship with the represented performer position, adjusting the sound emitted by the user viewing the performer of the event and detecting by the microphone based on the position information, and participating in the event. The user includes a sound processing program for transmitting data representing the sound adjusted by the sound adjustment unit to a processing device including a sound adjusting unit for synthesizing the sound adjusted by the sound adjustment unit. The program may be provided recorded on a computer-readable storage medium.

When the processing device 2 is a computer, the processing unit reads a program stored in the storage unit and executes various processes according to this program. This program determines, for example, a spectator position representing a user's position in a predetermined area where an event is held, and a position of a performer of the event in a predetermined area, based on a user's request for participation in the event on a computer. Position information indicating the relationship between the performer position and the audience position representing the above is transmitted to a processing device provided with a sound adjustment unit that adjusts the sound emitted from the user viewing the performer of the event and detected by the microphone based on the position information. It is detected by the microphone corresponding to the first user who is the user, based on the fact that the sound is adjusted by the sound adjustment unit and the data transmitted by the processing device is received, and the data received by the reception unit is received. Data representing the combined sound by synthesizing the sound adjusted by the sound adjustment unit and the sound detected by the microphone corresponding to the second user who is a different user from the first user and adjusted by the sound adjustment unit. Includes a voice processing program that outputs and executes.

The program according to the embodiment may be a program (eg, difference file, difference program) that can realize the function according to the embodiment in combination with the program recorded in the computer system. The program according to the embodiment may be recorded and provided on a computer-readable storage medium.

[Second Embodiment]
The second embodiment will be described. In the present embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. FIG. 6 is a diagram showing a voice processing system according to the second embodiment. In the present embodiment, the terminal 3 is a terminal associated with the user U who uses the terminal 3. For example, the terminal 3a is a terminal owned by the user Ua. The terminal 3a is, for example, a smartphone, a tablet, a personal computer, or other information processing device. The terminal 3a stores, for example, information that identifies the terminal 3a and information that identifies the user Ua in the storage unit 31. In the following description, the information that identifies the terminal 3 is referred to as a terminal ID as appropriate. The information that identifies the user includes, for example, the information of the account in the content distribution service. Account information includes, for example, a user ID and password.

The terminal 3a includes a microphone 41, a speaker 42, a display unit 45, and an input unit 46. The display unit 45 corresponds to the display device 43 shown in FIG. The input unit 46 accepts input of various information. The input unit 46 includes, for example, at least one of a keyboard, a mouse, and a touchpad. The input unit 46 and the display unit 45 may be a touch panel. The user Ua inputs various information to the terminal 3a by operating the input unit 46.

In the present embodiment, the terminal 3a also serves as the reservation terminal 7 described with reference to FIG. The user Ua can input the entry information and apply for participation by operating the input unit 46 of the terminal 3a. The entry information includes, for example, a user ID and information that identifies an event. The communication unit 32 of the terminal 3a transmits the entry information. In the present embodiment, the processing device 2 also serves as the reception terminal 8 described with reference to FIG. The processing device 2 acquires the entry information transmitted by the communication unit 32 of the terminal 3a. For example, the communication unit 32 of the processing device 2 receives the entry information transmitted by the terminal 3a.

The storage unit 22 of the processing device 2 stores the user information D1 described with reference to FIG. The processing unit 21 of the processing device 2 includes a reception unit 29. The reception unit 29 executes the process by the reception terminal 8 described with reference to FIG. The reception unit 29 determines the user's seat at the event. For example, the reception unit 29 acquires the event ID included in the entry information transmitted by the terminal 3a. The reception unit 29 refers to a database that stores the event ID and the event information in association with each other, and acquires the event seat information corresponding to the event ID. The reception unit 29 acquires the user ID included in the entry information transmitted by the terminal 3a. The reception unit 29 acquires user Ua information (eg, user rank) corresponding to the user ID from the user information D1 stored in the storage unit 22. The reception unit 29 determines the seat of the user Ua (eg, the seat number) in the event by using the seat information of the event (eg, the rank of the seat, the information of the vacant seat) and the information of the user Ua.

The position determination unit 24 of the processing device 2 determines the audience position Qa of the user Ua by using the information on the seat of the user Ua determined by the reception unit 29. The processing unit 21 generates position information indicating the relationship between the performer position P1 and the audience position Qa determined by the position determination unit 24. The communication unit 23 transmits the position information generated by the processing unit 21. The communication unit 32 of the terminal 3a corresponding to the user Ua receives the position information transmitted by the communication unit 23 of the processing device 2. The processing unit 33 of the terminal 3a stores the position information received by the communication unit 32 in the storage unit 31.

As described in the first embodiment, the communication unit 32 of the terminal 3a receives the data of the content transmitted by the processing device 2. The terminal 3a reproduces this content based on the data of the content received by the communication unit 32. For example, the processing unit 33 causes the display unit 45 to display the video represented by the content data. The processing unit 33 causes the speaker 42 to output the sound represented by the content data. The microphone 41 detects the audience sound emitted by the user Ua. The sound adjusting unit 34 adjusts the audience sound detected by the microphone 41 by using the position information D3a stored in the storage unit 31. The communication unit 32 transmits the adjusted audience sound data. The communication unit 23 of the processing device 2 receives the adjusted audience sound data transmitted by the communication unit 23 of the terminal 3a. The synthesizing unit 27 synthesizes the adjusted spectator sound corresponding to the user Ua and the adjusted spectator sound corresponding to the user Ub. The output unit 26 outputs the data of the synthesized spectator sound, and causes the speaker 42 to output the synthesized spectator sound.

In this embodiment, the user U can change the audience position Q. For example, user Ua can apply for a change in spectator position Qa during the event. By operating the terminal 3a, for example, the user Ua can transmit information requesting a change of the spectator position Qa (appropriately referred to as a change application) by the terminal 3a. The input unit 46 of the terminal 3a accepts a change application or input of information (appropriately referred to as a change application, etc.) that is the basis of the change application. The change application and the like include information on the seat of the change destination desired by the user U (appropriately referred to as change front seat information).

The communication unit 32 of the terminal 3a transmits a change application based on the change application or the like input to the input unit 46. The change application includes, for example, one or both of information that identifies the user Ua (eg, user ID) and information that identifies the terminal 3a associated with the user Ua (eg, terminal ID), and change prior information. include. The storage unit 31 may store one or both of the user ID and the terminal ID in advance. When applying for a seat change, the user Ua does not have to input the change destination seat information and one or both of the user ID and the terminal ID in the terminal 3a. In this case, the processing unit 33 of the terminal 3a generates a change request using one or both of the user ID and the terminal ID stored in the storage unit 31 and the change senior information input to the input unit 46. You may. The communication unit 32 may transmit the change request generated by the processing unit 33.

The communication unit 23 of the processing device 2 receives the change application transmitted by the communication unit 32 of the terminal 3a. The position determination unit 24 changes the spectator position Qa corresponding to the user Ua based on the change request of the spectator position Q from the user Ua. For example, the position determination unit 24 determines the change destination of the spectator position Qa of the user Ua based on the change application received by the communication unit 23. For example, the position determination unit 24 determines whether or not the seat indicated in the change front seat information included in the change application is available. For example, if the seat indicated in the change destination seat information is not assigned to the user U, the position determination unit 24 determines that this seat can be used. For example, the position determination unit 24 derives the coordinates of the event venue AE in cyberspace corresponding to the seats shown in the changed seat information, and uses the derived coordinates as the changed audience position Q (as appropriate, the changed audience position). ) To be decided. The communication unit 23 transmits the position information corresponding to the updated spectator position Q. For example, the processing unit 21 specifies the terminal 3a as the transmission destination of the location information based on one or both of the user ID and the terminal ID included in the change application transmitted by the terminal 3a. The communication unit 23 transmits the position information to the destination specified by the processing unit 21.

The communication unit 32 of the terminal 3a receives the updated position information transmitted by the processing device 2. The processing unit 33 of the terminal 3a updates the position information D3a stored in the storage unit 31 to the updated position information received by the communication unit 32. The sound adjustment unit 34 adjusts the sound detected by the microphone 41 corresponding to the user Ua by using the audience position Qa changed by the position determination unit 24. For example, after the position information D3a stored in the storage unit 31 is updated, the sound adjustment unit 34 adjusts the audience sound detected by the microphone 41 using the updated position information D3a.

The process of changing the spectator position may be executed by consuming tokens. The token may include, for example, electronic money compatible with the currency of each country, or may include cryptographic assets. The token may be, for example, a point that can be used by the user in the content distribution service, or a point that can be used by the user in a service affiliated with the content distribution service. The information of the token owned by the user U may be included in the user information D1. The value of the token to be consumed may be, for example, a predetermined fixed value, a value specified by the user, or a value determined by the seat to be changed. The processing device 2 may determine the seat to be changed by, for example, an auction using the value of the token specified by the user. When tokens are consumed in the process of changing the audience position, the timing at which the tokens are consumed is arbitrary. For example, the token may be consumed when the processing device 2 accepts the change application, or may be consumed when the spectator position is changed by the change application.

The positioning unit 24 determines whether or not this seat is available, for example, depending on whether or not the seat specified by the user is in use, but whether or not the seat can be used according to other criteria or conditions. It may be determined whether or not. For example, the position determination unit 24 may determine that the seat can be used when the value of the token held by the user is equal to or greater than the value of the token consumed in the process of changing the spectator position. Seats in cyberspace may be assigned to only one user or to multiple users. For example, the spectator positions of a plurality of users may be the same, and the spectator positions of any user may be the same as the spectator positions of other users.

The voice processing system 1 executes an action reception unit that receives information specifying an action for one or both of the performer P and the user U from the user U, and a process associated with the information received by the action reception unit in advance. A processing unit may be provided. The above actions include actions (eg, video, audio) on the content in a form different from the audience sound. The action includes, for example, an action relating to one or both of video and audio in the content. The above action includes, for example, the action of expressing the user's input by one or both of the video and the audio on the content. In the following description, the information that specifies the action is referred to as an action request, and the process associated with the information received by the action receiving unit in advance is referred to as an action addition process.

The action reception unit includes, for example, an input unit 46 of the terminal 3a. The processing unit includes, for example, a content generation unit 28 of the processing device 2. For example, the user Ua inputs a comment about the event in the input unit 46 in a text format as an action request. The communication unit 32 of the terminal 3a transmits the text information input to the input unit 46. The communication unit 23 of the processing device 2 receives the text information transmitted by the communication unit 32 of the terminal 3a. The content generation unit 28 generates, for example, content including the text included in the text information received by the communication unit 23 as a video. For example, the text input by the user may be displayed while moving on the video of the content, or may be displayed in other forms.

The action reception unit may include the microphone 41 of the terminal 3a. For example, the user may input voice instead of inputting text, and the microphone 41 may accept the voice emitted by the user as an action request. In this case, the processing unit 33 of the terminal 3a may convert the voice input to the microphone 41 into text information by voice recognition. The communication unit 32 of the terminal 3a may transmit the text information generated by the processing unit 33 by voice recognition.

The action request information may be input in a format different from text or voice. For example, the terminal 3a displays an icon associated with a predetermined action addition process on the display unit 45, and requests the execution of the action addition process associated with the icon when the user's operation on this icon is detected. You may. For example, the terminal 3a displays an icon indicating the addition of the applause sound on the display unit 45. When the terminal 3a detects an operation from the user for this icon, the communication unit 32 transmits a request for adding the sound of applause to the content. The processing device 2 receives the request transmitted by the communication unit 32, and the content generation unit 28 generates, for example, content to which a pre-sampled clap sound is added.

The action addition process may include a process of making additional adjustments to the audience sound adjusted by the sound adjustment unit 34. For example, the action addition process may include a process of changing the volume of the adjusted audience sound, or may include a process of adding an effect such as making an echo. The action addition process may be a process of adding a video or a process of adding a video effect. For example, the action addition process may include a process of adding a fireworks image to the content image. The action addition process may include a process of transferring a value equivalent to a tip or a thrown money to the performer. For example, the processing unit 21 of the processing device 2 may execute a process of transferring a token having a value specified by the user Ua from the account of the user Ua to the account of the performer P, or transfer the token to the device that manages the token. The process of transferring may be executed. When the process of transferring the token is executed, the content generation unit 28 may add a video showing the transfer of the token to the content. For example, the content generation unit 28 may add the tossed money expressed in CG to the video of the content. The above icon may be one type of icon, or may be a plurality of types of icons having different action addition processes.

The process associated with the action request may be executed by consuming the token. The value of the token to be consumed may be, for example, a predetermined fixed value or a value specified by the user. The value of the token to be consumed may be a value determined by the type of action addition processing. For example, the value of the token to be consumed may differ between the process of adding text to the video of the content and the process of adding sound effects to the audio of the content.

The value of the token to be consumed may be a value determined by the level of the effect by the action addition process. For example, it is assumed that the action addition process includes a process of adding an applause sound effect. When the token to be consumed is the first value, the content generation unit 28 adds the sound effect of applause to the content at the first volume, and the token to be consumed is a second value larger than the first value. In this case, the applause sound effect may be added to the content at a second volume higher than the first volume. It is assumed that the action addition process includes a process of adding text to the video. The content generation unit 28 adds text to the content with the first font size when the token to be consumed is the first value, and when the token to be consumed is a second value larger than the first value, the content generation unit 28 adds the text to the content. Text may be added to the content in a second font size that is larger than the first font size.

The sound adjustment unit 34 of the terminal 3a or the sound adjustment unit 34 of the terminal 3b is a microphone corresponding to the user Ub based on the relationship between the audience position Qa corresponding to the user Ua and the audience position Qb corresponding to the user Ub. You may adjust the sound detected by. For example, the audience position Qb of the user Ub is assumed to be a range in which the audience sound of the user Ub reaches the audience position Qa of the user Ua when the audience sound of the user Ub is emitted at the audience position Qb. The terminal 3a can execute a process of emphasizing the spectator sound emitted at the spectator position Q around the spectator position Qa (appropriately referred to as an ambient emphasizing process). The terminal 3 can switch between, for example, a first operation mode in which the peripheral emphasis processing is not executed and a second operation mode in which the peripheral enhancement processing is executed. For example, the terminal 3 receives an input for mode switching from the user U, and when it is determined that this input has been received, the terminal 3 switches between the first operation mode and the second operation mode.

In the first operation mode, the adjusted audience sound of the user Ub is synthesized by the synthesis unit 27 of the processing device 2. The synthesized audience sound is output from the speaker 6 to the performer P. The audience sound output from the speaker 6 is detected by the microphone 5 together with the sound emitted by the performer P. The content generation unit 28 generates content using the sound detected by the microphone 5, and the audience sound of the user Ub is included in the sound of the content.

In the second operation mode, the communication unit 23 of the processing device 2 has position information indicating the relationship between the spectator position Qa of the user Ua and the spectator position Qb of the user Ub (appropriately referred to as a second position information and an inter-audience position information). To send. The inter-audience position information is, for example, the audience sound of the user Ua when the audience sound adjusted by the sound adjustment unit 34 of the terminal 3b is emitted at the audience position Qb by the audience sound detected by the microphone 41 corresponding to the user Ub. Includes a gain that converts the sound to reach position Qa. The communication unit 23 of the processing device 2 transmits the adjusted audience sound data corresponding to the user Ub. The communication unit 32 of the terminal 3a receives the adjusted audience sound data transmitted by the communication unit 23 of the processing device 2. The processing unit 33 of the terminal 3a uses the inter-audience position information to send the adjusted spectator sound corresponding to the user Ub to the sound that reaches the spectator position Qa of the user Ua when the spectator sound is emitted at the spectator position Qb. Convert. The processing unit 33 synthesizes the converted audience sound of the user Ub with the sound of the content and outputs it from the speaker 42. In the voice processing system 1 of such a form, for example, the audience sound of the second user who is around the first user in the cyber space is realistically transmitted to the first user, and the presence of the event is enhanced.

When switching from the first operation mode to the second operation mode, the terminal 3a notifies the processing device 2 that the operation is to be switched. Upon receiving the above notification, the processing unit 21 of the processing device 2 identifies the audience position Qa of the user Ua corresponding to the terminal 3a. The processing unit 21 extracts at least one spectator position Q included in a predetermined range around the spectator position Qa. The processing unit A generates inter-audience position information for each of the extracted spectator positions Q. The processing device 2 provides the generated inter-audience position information to the terminal 3a. The processing device 2 provides the adjusted spectator sound data corresponding to the inter-audience position information to the terminal 3a in association with the inter-audience position information. For example, the processing unit 21 of the processing device 2 allocates identification information for each position information between spectators, and transmits the adjustment sound data as a set with the identification information.

Next, the voice processing method according to the embodiment will be described based on the configuration of the voice processing system 1 described above. FIG. 7 is a diagram showing a voice processing method according to the second embodiment. Refer to FIG. 6 as appropriate for the configuration of the voice processing system 1. The same processing as in FIG. 5 is appropriately designated with the same reference numerals as those in FIG. 5, and the description thereof will be omitted or simplified.

The first user terminal accepts input of information regarding the participation application from the user Ua shown in FIG. The first user terminal transmits the entry information in step S21. For example, the processing unit 33 of the terminal 3a of FIG. 6 causes the communication unit 32 to transmit the entry information based on the information input to the input unit 46. The processing device receives the entry information transmitted in step S21. The processing device determines the audience position of the first user in step S22. For example, the position determination unit 24 of the processing device 2 of FIG. 6 determines the audience position Qa of the user Ua based on the entry information corresponding to the user Ua received by the communication unit 23. The processing device transmits the position information in step S23. For example, the processing unit 21 of the processing device 2 of FIG. 6 uses the spectator position Qa determined by the position determining unit 24 to generate position information indicating the relationship between the spectator position Qa and the performer position P1. The communication unit 23 transmits the position information generated by the processing unit 21.

The process from step S24 to step S26 is the same as the process from step S21 to step S23, and the description thereof will be simplified. The second user terminal accepts input of information regarding the participation application from the user Ub shown in FIG. The second user terminal transmits the entry information in step S24. Example The processing device receives the entry information transmitted in step S24. In step S25, the processing device determines the audience position of the second user (eg, user Ub in FIG. 6). The processing device transmits the position information in step S26.

The processing of steps S21 to S26 described above is executed, for example, before the start of the event. In addition, at least a part of the processing of steps S21 to S26 may be executed after the start of the event. For example, the user may apply for participation in the event and participate in the event after the distribution of the content is started. The processing of steps S1 to S11 after step S26 is the same as in FIG. The processes of steps S1 to S11 are repeatedly executed, and the process of changing the spectator position and the action addition process are executed during the period in which the processes of steps S1 to S11 are repeatedly executed, respectively. The process of changing the spectator position may be executed before the process of step S1 is executed. For example, the user U may operate the terminal 3 to execute the spectator position change process after the participation application is made and before the distribution of the content is started.

[Third Embodiment]
The third embodiment will be described. FIG. 8 is a diagram showing a voice processing system according to the third embodiment. In the present embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. The voice processing system 1 according to the present embodiment includes a plurality of first voice processing devices divided into a plurality of groups, and a synthesis unit is provided corresponding to each group of the plurality of groups. (1) A first synthesis unit that synthesizes the sound adjusted by the sound adjustment unit of the voice processing device, and a second synthesis unit that synthesizes the sound synthesized by the first synthesis unit in a plurality of groups are provided.

In the present embodiment, the plurality of terminals 3 are divided into a plurality of groups. In the following description, any group is appropriately represented by a code G, and when the group G is distinguished, it is represented by a code obtained by adding the alphabets a, b, ... To the code G such as group Ga and group Gb.

In the present embodiment, the voice processing system 1 includes a plurality of processing devices 51 and a processing device 52. The processing device 51 is provided for each group G. In the following description, when an arbitrary processing device 51 is appropriately represented by reference numeral 51 and the processing device 51 is distinguished, the reference numerals in which the alphabets a, b, ... Are added to the reference numerals 51 as in the processing device 51a and the processing device 51b. It is represented by.

The processing device 51 includes a synthesis unit 27 described with reference to FIG. 3 and the like. The processing device 51 acquires the adjusted audience sound data from two or more terminals 3 included in the corresponding group. For example, the processing device 51a receives the adjusted audience sound data transmitted by each of the terminals 3 included in the group Ga. The synthesizing unit 27 of the processing device 51a synthesizes the adjusted spectator sound acquired from each of the terminals 3 included in the group Ga by using the received spectator sound data. In the following description, the adjusted spectator sound synthesized by the processing device 51 as appropriate is referred to as the spectator sound after the primary synthesis.

The processing device 52 is provided corresponding to two or more processing devices 51. In FIG. 8, the processing device 52 corresponds to the processing device 51a and the processing device 51b. The processing device 52 acquires the data of the audience sound after the primary synthesis from each of the two or more processing devices 51 corresponding to each of the processing devices 51. The processing device 52 includes a synthesis unit 53 and a content generation unit 28. The synthesizing unit 53 synthesizes the spectator sounds represented by the spectator sound data by using the spectator sound data after the primary synthesis acquired from each of the two or more processing devices 51. The processing of the synthesis unit 53 may be the same processing as that of the synthesis unit 27, for example. The processing device 52 outputs the data of the audience sound synthesized by the synthesis unit 53. The processing device 52 causes the speaker 6 to output the audience sound synthesized by the synthesis unit 53.

As described with reference to FIG. 3 and the like, the content generation unit 28 generates content using the video captured by the camera 4 and the voice detected by the microphone 5. The processing device 52 provides the data of the generated content. The processing device 51 acquires data on the content provided by the processing device 52. The processing device 51 provides content data to the terminal 3 included in the corresponding group G. For example, the processing device 51a provides content data to each of the two or more terminals 3 included in the corresponding group Ga. The two or more terminals 3 included in the group Ga each reproduce the content using the content data provided by the processing device 51a.

As described above, in the audio processing system 1 according to the present embodiment, the plurality of terminals 3 are divided into a plurality of groups, and the adjusted audience sounds provided by the two or more terminals 3 included in each group are synthesized. The processing device 51 is provided. Since the voice processing system 1 executes the process of synthesizing the adjusted audience sound in a distributed manner among a plurality of devices, it is possible to reduce the processing load of each device, for example, to reduce the occurrence of processing delay. be able to.

The audio processing system 1 includes a plurality of processing devices (appropriately referred to as higher-level processing devices) for synthesizing the audience sound after the primary synthesis provided by two or more processing devices 51, and the processing device 52 is a plurality of higher-level processing devices. Audience sounds synthesized by each higher-level processing device (appropriately referred to as spectator sounds after secondary synthesis) may be acquired from the processing device, and the spectators after these secondary synthesis may be combined and the data may be output. For example, the layer of the device that generates intermediate data between each terminal 3 and the processing device 52 is one layer of the processing device 51 in FIG. 8, but may be two layers of the processing device 51 and the upper processing device. As described above, the number of layers of the device for generating the intermediate data is arbitrary, and may be 0 layer as shown in FIG. 3, 1 layer as shown in FIG. 8, or multiple layers as shown in FIG. The content generation unit 28 may be provided in a device different from the processing device 52, and may be provided in, for example, a device (eg, processing device 51) that generates intermediate data between the terminal 3 and the processing device 52. ..

FIG. 9 is a diagram showing an example of a group according to the third embodiment. Reference numerals EX1 to EX3 in FIG. 9 represent examples of groups, respectively. In the first example EX1, each of the plurality of groups G is a set of terminals corresponding to the audience position Q in which the distance to the performer position P1 in the event venue AE is within a predetermined range. For example, the group Ga is a set of terminals corresponding to the audience position where the distance to the performer position P1 is less than L1. The group Gb is a set of terminals corresponding to the audience positions where the distance to the performer position P1 is L1 or more and less than L2.

In the second example EX2, each of the plurality of groups G is a set of terminals corresponding to the audience position Q whose direction from the performer position P1 in the event venue AE is within a predetermined range. For example, the group Ga is a group of terminals corresponding to the audience position Q whose azimuth angle with respect to the performer position P1 is in the range θ1. The group Gb is a group of terminals corresponding to the audience position Q whose azimuth angle with respect to the performer position P1 is in the range θ2.

The rules for grouping are not limited to the first example or the second example, and are arbitrarily determined. For example, the third example EX3 is an example of grouping in which the grouping rule of the first example and the grouping rule of the second example are combined. In the third example, the plurality of groups G are determined based on the distance from the performer position P1 and the azimuth angle with respect to the performer position P1.

The plurality of groups G may be determined based on the order in which the spectator positions Q are determined. For example, the plurality of terminals 3 may be grouped based on the order in which the audience position Q is determined. For example, the terminal 3 included in the group Ga may be a terminal corresponding to the spectator position Q in which the spectator position Q is determined from the first to the 100th. The terminal 3 included in the group Gb may be a terminal corresponding to the spectator position Q from 101 to 200 in the order in which the spectator position Q is determined.

The plurality of groups G may be determined based on the priority of seats (eg, seat rank) at the event venue AE. For example, the terminal 3 included in the group Ga may be a terminal corresponding to the audience position Q corresponding to the first rank seat (eg, S seat) in the event venue AE. The terminal 3 included in the group Gb may be a terminal corresponding to the audience position Q corresponding to a second rank seat (eg, A seat) having a lower priority than the first rank in the event venue AE. The first-ranked seats may be seats for which the participation fee paid by the user is higher than that of the second-ranked seats, or may be seats assigned to preselected users (eg, invited guests). The plurality of groups G may be determined by randomly selecting the terminals 3 included in each group G from the plurality of terminals 3. The above-mentioned grouping rule may be a rule that combines two or more types of rules (eg, conditions).

In the plurality of groups G, the number of terminals 3 included in each group may be the same or different. The number of terminals 3 included in each group G may be, for example, the number of terminals 3 in which the processing device 51 corresponding to each group G is in charge of processing. Here, it is assumed that the terminal 3 included in the group Ga is a terminal corresponding to the spectator position Q corresponding to the seat of the first rank. Further, it is assumed that the terminal 3 included in the group Gb is a terminal corresponding to the spectator position Q corresponding to the second rank seat. The number of terminals 3 included in the group Ga may be smaller than the number of terminals 3 included in the group Gb. In this case, the processing device 51a corresponding to the group Ga has a reduced processing load as compared with the processing device 51b corresponding to the group Gb, and for example, the possibility of a processing delay is reduced.

In the plurality of processing devices 51, the performance of each processing device 51 may be the same or different. Here, it is assumed that the terminal 3 included in the group Ga is a terminal corresponding to the spectator position Q corresponding to the seat of the first rank. Further, it is assumed that the terminal 3 included in the group Gb is a terminal corresponding to the spectator position Q corresponding to the second rank seat. The processing device 51a corresponding to the group Ga may have higher hardware performance (eg, CPU processing speed, storage read / write speed, communication speed) than the processing device 51b corresponding to the group Gb. In this case, the processing device 51a corresponding to the group Ga is less likely to cause a processing delay than the processing device 51b corresponding to the group Gb, for example.

The process of dividing the plurality of terminals 3 into a plurality of groups (appropriately referred to as grouping process) is executed by, for example, the reception terminal 8 of FIG. The grouping process may be executed by a device different from the reception terminal 8. For example, the grouping process may be executed by the processing device 2 or may be executed by another device. The grouping process may be executed by an external device of the voice processing system 1.

[Fourth Embodiment]
A fourth embodiment will be described. FIG. 10 is a diagram showing a voice processing system according to the fourth embodiment. In the present embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. In the present embodiment, the predetermined area (eg, event venue AE) is divided into a plurality of subregions each corresponding to each group, and each group corresponds to the audience position included in the subregion corresponding to each group. Includes a set of first audio processing devices. Each of the plurality of subregions corresponds to one of a plurality of groups. For example, the plurality of terminals 3 (see FIG. 8) are divided into a plurality of groups G. The plurality of groups G are determined based on the orientation of the performer position with respect to the audience position Q, as described in the second example EX2 of FIG.

The voice processing system 1 of FIG. 10 includes a plurality of processing devices 2. In the following description, any processing device is appropriately represented by reference numeral 2, and when the processing device 2 is distinguished, the reference numerals are obtained by adding the alphabets a, b, ... To the reference numerals 2 as in the processing device 2a and the processing device 2b. show. Each of the plurality of processing devices 2 corresponds to one group G. For example, the processing device 2a corresponds to the group Ga, and the processing device 2b corresponds to the group Gb.

Each processing device 2 provides content data to a terminal 3 included in the corresponding group G. For example, the processing device 2a provides content data to each terminal 3 included in the group Ga. Each processing device 2 acquires the adjusted audience sound data from the terminal 3 included in the corresponding group G. For example, the processing device 2a receives the adjusted audience sound data transmitted by each terminal 3 included in the group Ga. Each processing device 2 synthesizes the adjusted audience sound acquired from the terminal 3 included in the corresponding group G, and outputs the data of the synthesized audience sound. For example, the processing device 2a synthesizes the adjusted audience sound acquired from each terminal 3 included in the group Ga, and outputs the data of the synthesized audience sound.

In the present embodiment, the voice processing system 1 includes a plurality of voice output devices (eg, speakers 6). In the following description, any speaker is appropriately represented by reference numeral 6, and when the speaker 6 is distinguished, it is represented by a reference numeral in which the alphabets a, b, ... Are added to the reference numeral 6 such as the speaker 6a and the speaker 6b. The plurality of speakers 6 are arranged in the demonstration area AP. Each of the plurality of speakers 6 has a corresponding relationship with the processing device 2. Here, it is assumed that the speaker 6 is provided in a one-to-one relationship with the processing device 2. For example, the speaker 6a corresponds to the processing device 2a, and the speaker 6b corresponds to the processing device 2b.

Each of the plurality of speakers 6 corresponds to a group G in charge of the corresponding processing device 2. For example, the speaker 6a corresponds to the group Ga, and the speaker 6b corresponds to the group Gb. Here, the area in which the audience position Q corresponding to the terminal 3 included in each group G is distributed is referred to as a group area.

The arrangement of the performer P and the plurality of speakers 6 in the demonstration area AP is set so as to correspond to the arrangement of the performer position P1 and the plurality of group areas in the event venue AE. Here, in the demonstration area AP, the rotation direction around the vertical direction of the performer P is considered centering on the performer P. This rotation direction is centered on the performer position P1 at the event venue AE, and corresponds to the rotation direction around the vertical direction of the performer position P1. In the demonstration area AP, the plurality of speakers 6 are arranged in the order of speaker 6a, speaker 6b, speaker 6c, speaker 6d, and speaker 6e in the clockwise direction in the rotation direction. In the event venue AE, the regions of the plurality of groups G are arranged in the order of the region of the group Ga, the region of the group Gb, the region of the group Gc, the region of the group Gd, and the region of the group Ge in the clockwise direction in the rotation direction. .. In this way, the plurality of speakers 6 are arranged so that the relationship of the azimuth angles with respect to the performer P maintains the mutual relationship of the regions of the plurality of groups G with respect to the performer position P1.

In the present embodiment, the voice processing system 1 includes a plurality of cameras 4. In the following description, any camera is appropriately represented by reference numeral 4, and when the camera 4 is distinguished, it is represented by a reference numeral in which the alphabets a, b, ... Are added to the reference numeral 4 such as the camera 4a and the camera 4b. The plurality of cameras 4 are arranged in the demonstration area AP. Each of the plurality of cameras 4 has a corresponding relationship with the processing device 2. Here, it is assumed that the camera 4 is provided in a one-to-one relationship with the processing device 2. For example, the camera 4a corresponds to the processing device 2a, and the camera 4b corresponds to the processing device 2b. Each of the plurality of cameras 4 corresponds to a group G in charge of the corresponding processing device 2. For example, the camera 4a corresponds to the group Ga and the camera 4b corresponds to the group Gb.

The arrangement of the performer P and the plurality of cameras 4 in the demonstration area AP is set so as to correspond to the arrangement of the performer position P1 and the areas of the plurality of groups G in the event venue AE. Here, in the demonstration area AP, the rotation direction around the vertical direction of the performer P is considered centering on the performer P. This rotation direction is centered on the performer position P1 at the event venue AE, and corresponds to the rotation direction around the vertical direction of the performer position P1. In the demonstration area AP, the plurality of cameras 4 are arranged in the order of camera 4a, camera 4b, camera 4c, camera 4d, and camera 4e in the clockwise direction in the rotation direction. In the event venue AE, the regions of the plurality of groups G are arranged in the order of the region of the group Ga, the region of the group Gb, the region of the group Gc, the region of the group Gd, and the region of the group Ge in the clockwise direction in the rotation direction. .. In this way, the plurality of cameras 4 are arranged so that the relationship of the azimuth angles with respect to the performer P maintains the positional relationship of the plurality of group G regions with respect to the performer position P1. When the performer P is in the performer position P1, each camera 4 is set to be oriented with respect to the performer P so as to shoot an image corresponding to the state in which the performer position P1 is viewed from the area of the corresponding group G. There is.

The processing device 2 generates content by using the sound detected by the microphone 5 and the image captured by the camera 4 corresponding to the processing device 2. For example, the processing device 2a generates content using the sound detected by the microphone 5 and the image captured by the camera 4a. The processing device 2a provides the generated content to the terminal 3 included in the corresponding group Ga.

In the present embodiment, the performer P can hear the synthesized audience sound corresponding to the group G from the same direction as the azimuth angle of the group G with respect to the performer position P1. For example, in the event venue AE, the group Ga is the area on the left side of the front of the performer position P1. It is assumed that the performer P faces the area of the group Ga in the content and requests the user for a reaction such as applause. The user U who views the content by the terminal 3 included in the group Ga can see the performer P who turns to himself and calls for a reaction. When the user U performs a reaction such as applause or cheers, the audience sound corresponding to the reaction is adjusted by the terminal 3, synthesized by the processing device 2a, and output from the speaker 6a. The performer P can hear the audience sound corresponding to the reaction from, for example, the direction in which the user U is called for the reaction. For example, the user U can experience a sense of realism even when participating in an event at a place different from the venue of the event in the real space, and the voice processing system 1 according to the embodiment can provide a new experience.

In FIG. 10, the speaker 6 is provided in a one-to-one correspondence with the processing device 2, but the number of speakers 6 corresponding to one processing device 2 may be one or a plurality. The voice processing system 1 does not have to include at least one speaker 6. For example, the plurality of speakers 6 are devices provided in a system different from the voice processing system 1 (eg, a sound system), and the voice processing system is arranged so as to correspond to the arrangement of the regions of the plurality of groups G. A system that outputs the data of the audience sound after synthesis to the speaker 6 of the above may be used.

In FIG. 10, the camera 4 is provided in a one-to-one correspondence with the processing device 2, but the number of cameras 4 corresponding to one processing device 2 may be one or a plurality. The voice processing system 1 does not have to include at least one camera 4. For example, the plurality of cameras 4 are devices provided in a system different from the voice processing system 1 (eg, a photographing system), and the voice processing systems are arranged so as to correspond to the arrangement of the regions of the plurality of groups G. A system that acquires the image of the performer P from the camera 4 of the above may be used. The number of cameras 4 may be one, and for example, the voice processing system 1 may be a system that distributes common contents among a plurality of groups G.

The voice processing system 1 according to the embodiment may generate content by using the computer graphics (appropriately referred to as CG) of the performer P created in advance. For example, the performer P is represented by a three-dimensional CG such as a polygon, and the voice processing system 1 generates content by moving the three-dimensional CG using the movement of the performer P detected by motion capture or the like. May be good. The audio processing system 1 may generate content by combining the video acquired by the camera 4 and the CG.

The voice processing system 1 according to the embodiment does not have to include a position determining unit that determines an audience position representing the position of the user in a predetermined area where the event is held based on an application for participation in the event from the user. good. For example, the position of the spectators may be determined independently of the application for participation, or may be determined by the organizer of the event. Even in such a form, the voice processing system 1 can convey the reaction of the user U to the performer P, and for example, the reaction of the user U affects the performance of the performer P, so that the user U is newly affected. Can provide an experience.

The technical scope of the present invention is not limited to the embodiments described in the above-described embodiments. One or more of the requirements described in the above embodiments and the like may be omitted. In addition, the requirements described in the above-described embodiments and the like can be combined as appropriate. To the extent permitted by law, the disclosure of all documents cited in the above embodiments and the like shall be incorporated as part of the description of this specification.

1 Audio processing system, 2 Processing device, 3 terminals, 4 cameras, 5 microphones, 6 speakers, 7 reservation terminals, 8 reception terminals, G group, P performers, Q spectator position, R spectator sound, U user, 24 position determination unit , 27 Synthesis part, 28 Content generation part, 34 Sound adjustment part

Claims (30)

A microphone that detects the sound emitted by users participating in the event,
A sound that adjusts the sound detected by the microphone based on the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user in the predetermined area. Adjustment part and
The sound detected by the microphone corresponding to the first user who is the user and adjusted by the sound adjustment unit and detected by the microphone corresponding to the second user who is a user different from the first user. A compositing unit that synthesizes the sound adjusted to the sound adjusting unit, and a compositing unit,
It is provided with an output unit that outputs data representing the sound synthesized by the synthesis unit.
The performer performs outside the predetermined area and
A voice processing system in which sound data synthesized by the synthesis unit is provided to a voice output device arranged at a position where the sound reaches the performer. The sound adjusting unit adjusts the first sound detected by the microphone so as to be close to the second sound that reaches the performer position when the first sound is emitted at the audience position.
The voice processing system according to claim 1. A microphone that detects the sound emitted by users participating in the event,
A sound that adjusts the sound detected by the microphone based on the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user in the predetermined area. Adjustment part and
The sound detected by the microphone corresponding to the first user who is the user and adjusted by the sound adjustment unit and detected by the microphone corresponding to the second user who is a user different from the first user. A compositing unit that synthesizes the sound adjusted to the sound adjusting unit, and a compositing unit,
It is provided with an output unit that outputs data representing the sound synthesized by the synthesis unit.
The sound adjustment unit adjusts the first sound detected by the microphone so as to approach the second sound that reaches the performer position when the first sound is emitted at the audience position. system. The sound adjusting unit adjusts the loudness of the sound detected by the microphone by using the distance between the audience position and the performer position.
The voice processing system according to any one of claims 1 to 3. A microphone that detects the sound emitted by users participating in the event,
A sound that adjusts the sound detected by the microphone based on the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user in the predetermined area. Adjustment part and
The sound detected by the microphone corresponding to the first user who is the user and adjusted by the sound adjustment unit and detected by the microphone corresponding to the second user who is a user different from the first user. A compositing unit that synthesizes the sound adjusted to the sound adjusting unit, and a compositing unit,
It is provided with an output unit that outputs data representing the sound synthesized by the synthesis unit.
The sound adjusting unit is a voice processing system that adjusts the loudness of the sound detected by the microphone by using the distance between the audience position and the performer position. A second adjusting the sound detected by the microphone corresponding to the second user based on the relationship between the spectator position corresponding to the first user and the spectator position corresponding to the second user. Sound adjustment part and
Any of claims 1 to 5, further comprising a second output unit that outputs sound data including sound adjusted by the second sound adjustment unit to a voice output device that transmits sound to the first user. The voice processing system described in item 1. A microphone that detects the sound emitted by users participating in the event,
A sound that adjusts the sound detected by the microphone based on the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user in the predetermined area. Adjustment part and
The sound detected by the microphone corresponding to the first user who is the user and adjusted by the sound adjustment unit and detected by the microphone corresponding to the second user who is a user different from the first user. A compositing unit that synthesizes the sound adjusted to the sound adjusting unit, and a compositing unit,
An output unit that outputs data representing the sound synthesized by the synthesis unit, and an output unit.
A second adjusting the sound detected by the microphone corresponding to the second user based on the relationship between the spectator position corresponding to the first user and the spectator position corresponding to the second user. Sound adjustment part and
A voice processing system including a second output unit that outputs sound data including sound adjusted by the second sound adjustment unit to a voice output device that transmits the sound to the first user. The voice processing system according to any one of claims 1 to 7, further comprising a position determining unit for determining the position of the spectator based on an application for participation in the event from the user. It is equipped with an application acquisition unit that acquires the participation application provided by the user.
The position determination unit determines the audience position based on the participation application acquired by the application acquisition unit.
The voice processing system according to claim 8. A position determining unit for changing the spectator position corresponding to the first user is provided based on the spectator position change request from the first user.
The sound adjusting unit adjusts the sound detected by the microphone corresponding to the first user by using the audience position changed by the positioning unit.
The voice processing system according to any one of claims 1 to 9. A position determining unit for determining the spectator position when the first user participates in the event after the past event based on the history of the first user participating in the past event is provided. The voice processing system according to any one of claims 1 to 10. A first voice processing device including the sound adjusting unit and a first communication unit for transmitting sound data adjusted by the sound adjusting unit.
A second communication unit including a second communication unit that receives the sound data transmitted by the first communication unit and a synthesis unit that synthesizes the sound represented by the sound data received by the second communication unit. The voice processing system according to any one of claims 1 to 11, further comprising a voice processing device. The second audio processing device includes a position determining unit that determines the audience position.
The second communication unit transmits position information indicating the relationship between the performer position and the audience position determined by the position determination unit.
The first communication unit receives the position information transmitted by the second communication unit, and receives the position information.
The sound adjusting unit of the first voice processing device adjusts the sound detected by the microphone corresponding to the first voice processing device based on the position information received by the first communication unit.
The voice processing system according to claim 12. It is provided with a plurality of the first voice processing devices divided into a plurality of groups, and has a plurality of first voice processing devices.
The synthesis part
A first synthesis unit that is provided corresponding to each group of the plurality of groups and that synthesizes a sound adjusted by the sound adjustment unit of the first voice processing device in each group.
The plurality of groups include a second compositing unit that synthesizes the sound synthesized by the first compositing unit.
The voice processing system according to claim 12 or 13. The predetermined region is divided into a plurality of subregions, each of which corresponds to each of the groups.
Each group includes a set of the first audio processing devices corresponding to the audience position included in the subregion corresponding to each group.
The voice processing system according to claim 14. An action reception unit that receives information from the user that specifies an action for one or both of the performer and the user.
The voice processing system according to any one of claims 1 to 15, further comprising a processing unit that executes processing associated with information received in advance by the action receiving unit. The voice processing according to any one of claims 1 to 16, further comprising a third output unit that outputs sound data obtained by combining the sound emitted by the performer and the sound adjusted by the sound adjustment unit. system. The first user watches the event outside the predetermined area and views the event.
The microphone corresponding to the first user is arranged at a position where the sound emitted by the first user is detected.
The voice processing system according to any one of claims 1 to 17. A receiver that receives position information indicating the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area.
A sound adjustment unit that adjusts the sound emitted from the user and detected by the microphone based on the position information.
A transmission unit for transmitting data representing the sound adjusted by the sound adjustment unit is provided.
The sound adjusting unit adjusts the first sound detected by the microphone so as to be close to the second sound that reaches the performer position when the first sound is emitted at the audience position.
Voice processing device. A receiver that receives position information indicating the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area.
A sound adjustment unit that adjusts the sound emitted from the user and detected by the microphone based on the position information.
A transmission unit for transmitting data representing the sound adjusted by the sound adjustment unit is provided.
The sound adjusting unit adjusts the loudness of the sound detected by the microphone by using the distance between the audience position and the performer position.
Voice processing device. A transmitter that transmits position information indicating the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area.
A receiving unit that receives sound data emitted from the user and adjusted based on the position information from the transmission destination of the position information.
The sound represented by the data received from the destination corresponding to the first user who is the user and the data received from the destination corresponding to the second user who is the user different from the first user are represented. It is equipped with a synthesizer that synthesizes sound and
The performer performs outside the predetermined area and
The sound data synthesized by the synthesis unit is provided to a voice output device arranged at a position where the sound reaches the performer.
Voice processing device. A transmitter that transmits position information indicating the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area.
A receiving unit that receives sound data emitted from the user , detected by the microphone, and adjusted based on the position information from the transmission destination of the position information.
The sound represented by the data received from the destination corresponding to the first user who is the user and the data received from the destination corresponding to the second user who is the user different from the first user are represented. The synthesizer that synthesizes the sound,
A second adjusting the sound detected by the microphone corresponding to the second user based on the relationship between the spectator position corresponding to the first user and the spectator position corresponding to the second user. Equipped with a sound adjustment unit
The sound data including the sound adjusted by the second sound adjusting unit is output to the voice output device that transmits the sound to the first user.
Voice processing device. Detecting the sound emitted by the user participating in the event with a microphone,
A sound adjustment unit that adjusts the sound detected by the microphone based on the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user in the predetermined area. To adjust by
The sound detected by the microphone corresponding to the first user who is the user and adjusted by the sound adjustment unit and detected by the microphone corresponding to the second user who is a user different from the first user. By synthesizing the sound adjusted to the sound adjustment unit by the synthesis unit,
Including the output of data representing the sound synthesized by the synthesis unit.
The performer performs outside the predetermined area and
The sound data synthesized by the synthesis unit is provided to a voice output device arranged at a position where the sound reaches the performer.
Voice processing method. Detecting the sound emitted by the user participating in the event with a microphone,
A sound adjustment unit that adjusts the sound detected by the microphone based on the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user in the predetermined area. To adjust by
The sound detected by the microphone corresponding to the first user who is the user and adjusted by the sound adjustment unit and detected by the microphone corresponding to the second user who is a user different from the first user. By synthesizing the sound adjusted to the sound adjustment unit by the synthesis unit,
Including the output of data representing the sound synthesized by the synthesis unit.
The sound adjusting unit adjusts the first sound detected by the microphone so as to be close to the second sound that reaches the performer position when the first sound is emitted at the audience position.
Voice processing method. Detecting the sound emitted by the user participating in the event with a microphone,
A sound adjustment unit that adjusts the sound detected by the microphone based on the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user in the predetermined area. To adjust by
The sound detected by the microphone corresponding to the first user who is the user and adjusted by the sound adjustment unit and detected by the microphone corresponding to the second user who is a user different from the first user. By synthesizing the sound adjusted to the sound adjustment unit by the synthesis unit,
Including the output of data representing the sound synthesized by the synthesis unit.
The sound adjusting unit adjusts the loudness of the sound detected by the microphone by using the distance between the audience position and the performer position.
Voice processing method. Detecting the sound emitted by the user participating in the event with a microphone,
A sound adjustment unit that adjusts the sound detected by the microphone based on the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user in the predetermined area. To adjust by
The sound detected by the microphone corresponding to the first user who is the user and adjusted by the sound adjustment unit and detected by the microphone corresponding to the second user who is a user different from the first user. By synthesizing the sound adjusted to the sound adjustment unit by the synthesis unit,
To output data representing the sound synthesized by the synthesis unit,
The sound detected by the microphone corresponding to the second user is adjusted to the second sound based on the relationship between the spectator position corresponding to the first user and the spectator position corresponding to the second user. Adjusting by department and
To output sound data including the sound adjusted by the second sound adjusting unit to the voice output device that transmits the sound to the first user.
Speech processing methods including. On the computer
Receiving position information indicating the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area.
Adjusting the sound emitted from the user and detected by the microphone based on the position information,
Sending data representing the adjusted sound and executing
The adjustment includes adjusting the first sound detected by the microphone so as to be close to the second sound that reaches the performer position when the first sound is emitted at the audience position. ,
Speech processing program. On the computer
Receiving position information indicating the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area.
Adjusting the sound emitted from the user and detected by the microphone based on the position information,
Sending data representing the adjusted sound and executing
The adjustment includes adjusting the loudness of the sound detected by the microphone by using the distance between the audience position and the performer position.
Speech processing program. On the computer
Transmission of position information indicating the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area.
Receiving sound data emitted from the user and adjusted based on the position information from the transmission destination of the position information.
The sound represented by the data received from the destination corresponding to the first user who is the user and the data received from the destination corresponding to the second user who is the user different from the first user are represented. To synthesize the sound and to execute,
The performer performs outside the predetermined area and
The synthesized sound data is provided to a voice output device arranged at a position where the sound reaches the performer.
Speech processing program. On the computer
Transmission of position information indicating the relationship between the performer position representing the position of the performer of the event in the predetermined area where the event is held and the audience position representing the position of the user participating in the event in the predetermined area.
Receiving sound data emitted from the user , detected by the microphone, and adjusted based on the position information from the transmission destination of the position information.
The sound represented by the data received from the destination corresponding to the first user who is the user and the data received from the destination corresponding to the second user who is the user different from the first user are represented. Combining sound and
Adjusting the sound detected by the microphone corresponding to the second user based on the relationship between the spectator position corresponding to the first user and the spectator position corresponding to the second user. To execute,
The sound data including the sound adjusted corresponding to the second user is output to the voice output device that transmits the sound to the first user.
Speech processing program.

Images