JP5553446B2 - Amusement system - Google Patents

Description

  The present invention relates to an amusement system in which a user can sing a song in accordance with music data to be reproduced.

  In karaoke, a format in which a song is sung in accordance with the performance of the music selected by the user is common. On the monitor of the karaoke apparatus, the lyrics of the music selected by the user are displayed as the performance of the music progresses. Thereby, even if the user does not remember all the lyrics of the selected music, he can enjoy karaoke. On the background of the lyrics displayed on the monitor, a video of a character that sings a song, a video that matches the image of the music, and the like are displayed.

  Karaoke is often enjoyed by small groups such as family members or friends. Japanese Patent Application Laid-Open No. 2004-151867 discloses an apparatus that can generate a sound effect or the like at a desired timing for a user who sings a song to excite karaoke.

  An electronic percussion device according to the following Patent Document 1 has a percussion instrument shape such as maracas, and includes an acceleration sensor. The electronic percussion device notifies the user of the timing at which the device can be operated by turning on the LED, such as when an interlude is being played. When the user shakes the electronic percussion device during the lighting period of the LED, the percussion sound is reproduced.

JP 2004-287020 A

  As described above, karaoke is often enjoyed by a small group. A person who listens to a song sung by the user is limited to a family member or a friend who participates in karaoke. In other words, in karaoke, you can't have many spectators listen to your song like a live singer.

  Therefore, in view of the above problems, an object of the present invention is to provide an amusement system capable of experiencing a live performance with a high sense of presence in a pseudo manner.

In order to solve the above-mentioned problem, an invention according to claim 1 is an amusement system, comprising a main unit, a voice input device held by a user and outputting a voice inputted by the user as voice data, and the voice data A speech recognition device that performs a speech recognition process on the device to generate phrase information indicating a phrase spoken by the user, and the speech input device outputs first motion information indicating a motion of the speech input device. first motion information output unit includes the main unit, a reproducing unit for reproducing the music data selected by the user, on the basis of the phrase information and the first motion information, identify performance of the user and performance specification unit that, the response level of the audience for the identified performance, said timing of the phrase information is detected first Determined based on a time difference between the timing at which the motion information is detected, from among the plurality of the reaction data indicating the response of the audience, to select the playback reaction data corresponding to response levels of the identified performance and the audience A reaction instruction unit that instructs the reproduction unit to reproduce the reproduction reaction data.

The invention according to claim 2 is an amusement system, which is a main body device, a voice input device that is held by a user and that outputs voice inputted by the user as voice data, and outputs video data by photographing the user. an imaging device for the performing speech recognition processing on audio data, and a speech recognition device for generating a phrase information indicating a phrase that the user has spoken, the main unit, the music data selected by the user and a reproduction unit for reproducing, the video analysis unit generates the first motion information indicating a motion of the user by analyzing the image data, based on the phrase information and the first motion information, the user performance and performance specifying unit configured to specify the reaction level of the audience for the identified performance, the phrase information is detected Timing Play is determined based on the time difference between the timing of grayed and the first motion information is detected, from among the plurality of the reaction data indicating the response of the audience, according to the response level of the identified performance and the audience A reaction instruction unit that selects reaction data and instructs the reproduction unit to reproduce the reproduction reaction data.

According to a third aspect of the present invention, in the amusement system according to the first or second aspect , the reaction instruction unit determines a reproduction condition of the reproduction reaction data based on a reaction level of the audience, and the reproduction A condition is instructed to the playback unit.

According to a fourth aspect of the present invention, in the amusement system according to the third aspect, the reaction instructing unit determines a reaction level of the audience based on the audience of the audience set by the user .

According to a fifth aspect of the present invention, in the amusement system according to any one of the first to fourth aspects, the performance specifying unit specifies a question format for the audience based on the phrase information. And when it is determined that the direction of the voice input device is reversed based on the first movement information, it is determined that the user has performed a performance of scolding the audience, and the reaction instruction unit includes: As the reproduction reaction data, reaction data in which video and audio in which the audience responds simultaneously to the specific phrase is selected is selected.

According to a sixth aspect of the present invention, in the amusement system according to any one of the first to fourth aspects, the performance specifying unit requests the audience to chorus based on the phrase information. When calling a specific phrase and determining that the direction of the voice input device is reversed based on the first movement information, the user determines that the user has performed a performance requesting chorus, and the response The instructing unit selects reaction data in which video and audio sung by the audience are recorded as the reproduction reaction data.

The invention according to claim 7 is the amusement system according to any one of claims 1 to 4 , wherein the performance specifying unit determines that the user is clapping based on the motion information Then, it is determined that the user has performed a performance that leads the clapping, and the reaction instruction unit selects reaction data in which video and clapping sound recorded by the audience are recorded as the reproduction reaction data.

According to an eighth aspect of the present invention, in the amusement system according to any one of the first to fourth aspects, the performance specifying unit performs an operation in which the user swings his / her arm based on the movement information. When the determination is made, it is determined that the user has performed a performance requesting an operation of waving both hands to the audience, and the reaction instruction unit is a reaction in which an image of the audience shaking hands is recorded as the reproduction reaction data. Select data.

A ninth aspect of the present invention is the amusement system according to any one of the first, third, and eighth aspects, wherein the user holds the voice input device with a hand opposite to the hand that holds the voice input device. The controller includes a second motion information output unit that outputs second motion information indicating the motion of the controller, and the performance specifying unit is configured to perform the performance of the user based on the second motion information. Is identified.

According to a tenth aspect of the present invention, in the amusement system according to any one of the third to ninth aspects, the reaction instruction unit performs all the performances performed in the first part of the music data on each performance. Scoring is based on the audience response level, and all performances performed in the second part of the music data are scored based on the audience response level for each performance.

The invention according to claim 11 is an amusement system comprising a main unit and a voice input device held by a user, and the voice input device includes a radio communication unit that performs radio communication with the main unit. the main unit includes a reproducing unit for reproducing the music data selected by the user, based on the presence or absence of a radio signal transmitted from the audio input device, and performance specifying unit for specifying the performance of the user, hearing A reaction instruction unit that selects reproduction reaction data corresponding to the specified performance from a plurality of reaction data in which audio or video indicating a public reaction is recorded, and instructs the reproduction unit to reproduce the reproduction reaction data. And including.

According to a twelfth aspect of the present invention, in the amusement system according to the eleventh aspect , when the performance specifying unit detects the radio signal after the reproduction of the music data is started, the user performs a virtual live operation. It is determined that you have entered the venue.

According to a thirteenth aspect of the present invention, in the amusement system according to the eleventh or twelfth aspect , when the performance specifying unit becomes unable to detect the radio signal after the reproduction of the music data is started, It is determined that the user has left the virtual live venue.

A fourteenth aspect of the present invention is a voice input device used in the amusement system according to any one of the first to thirteenth aspects.

The invention according to claim 15 is a voice input device that outputs voice data that is held by a user and input by the user as voice data, and outputs first motion information indicating a motion of the device itself, and voice recognition for the voice data. processing performed, a voice recognition device for generating a phrase information indicating a phrase that the user has spoken, the computer to be installed in communicable main unit reproducing unit for reproducing the music data selected by the user, the Based on at least one of the phrase information and the first motion information, a performance identifying unit that identifies the performance of the user, the audience's response level to the identified performance, the timing at which the phrase information is detected, and the first motion information is determined based on the time difference between the detected timing, a plurality of reactance indicating the reaction of the audience Among Yondeta, a program for functioning to select the reproduction reaction data corresponding to response levels of the identified performance and the audience, the reproduction of the reproduction reaction data as a reaction instruction unit, to instruct the playback unit is there.

The invention according to claim 16 is an audio input device that is held by a user and outputs audio input by the user as audio data, an imaging device that shoots the user and outputs video data,
Performing speech recognition processing on the audio data, and voice recognition device for generating a phrase information indicating a phrase that the user has spoken, the computer to be installed in communicable main device, the music data selected by the user Based on at least one of the phrase information and the first motion information, a playback unit that reproduces, a video analysis unit that analyzes the video data and generates first motion information indicating the user's motion, A performance identifying unit that identifies the audience's response level to the identified performance based on a time difference between the timing at which the phrase information is detected and the timing at which the first motion information is detected ; from a plurality of the reaction data indicating the specified performance and anti of the audience Select reproduction reaction data corresponding to the level, a program for operating the reproduction of the reproduction reaction data as a reaction instruction unit, to instruct the playback unit.

The invention of claim 17, wherein the transmission of the computer the user is mounted on the main device capable of voice input device and the wireless communication for holding, reproduction unit for reproducing the music data selected by the user, from the voice input device Based on the presence / absence of a radio signal, a performance specifying unit for specifying the performance of the user, and selecting a playback reaction data corresponding to the specified performance from a plurality of reaction data indicating an audience reaction, It is a program for functioning as a reaction instruction unit that instructs the reproduction unit to reproduce reaction data.

  In the present invention, the main device reproduces the audience status data indicating the status of the live audience and displays it on a monitor or the like. The main device identifies the live performance performed by the user based on the phrase information indicating the phrase input to the microphone by the user and the movement information indicating the user's movement, and reproduces reaction data indicating the audience's reaction to the live performance. . Accordingly, the user can enjoy karaoke while acting as a singer that provides a live performance to the displayed audience, and can experience a live performance with a high sense of reality in a pseudo manner.

1 is an overall view of a karaoke system according to an embodiment of the present invention. It is a block diagram which shows the functional structure of a main body apparatus. It is a block diagram which shows the functional structure of a microphone. It is a block diagram which shows the functional structure of a controller. It is a flowchart which shows the flow of operation | movement of a main body apparatus. It is a figure which shows the image | video of the spectator displayed on the screen of a monitor. It is a figure which shows the procedure of performing audience talk with respect to the audience in a screen. It is a figure which shows the procedure of performing audience talk with respect to the audience in a screen. It is a figure which shows the procedure which leads chorus with respect to the audience in a screen. It is a figure which shows the procedure which leads chorus with respect to the audience in a screen. It is a figure which shows the procedure of leading clapping with respect to the audience in a screen. It is a figure which shows the procedure of leading clapping with respect to the audience in a screen. It is a figure which shows the procedure of leading a wave (operation | movement which shakes both hands largely) with respect to the audience in a screen. It is a figure which shows the procedure of leading a wave with respect to the audience in a screen. It is a figure which shows the procedure in which a user virtually enters a live venue. It is a figure which shows the setting screen which sets a live customer segment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this embodiment, a karaoke system will be described as an example of an amusement system.

{1. Overall configuration of karaoke system 1}
FIG. 1 is an overall view of the karaoke system 1. The karaoke system 1 includes a main body device 2, a monitor 3, a microphone 4, and a controller 5.

  The main device 2 is a processing device that performs overall control of the karaoke system 1 and reproduces music data and video data corresponding to the music selected by the user. The user selects music to sing at karaoke using a remote controller (not shown).

  The monitor 3 is a liquid crystal display or the like, and displays an image output from the main device 2. The monitor 3 includes a speaker (not shown), and outputs music data reproduced by the main body device 2 as sound.

  The microphone 4 is a voice input device that inputs voice when the user sings a song. The microphone 4 has a motion detection function for detecting the motion of the own device, and transmits motion information indicating the user's motion to the main body device 2.

  The controller 5 has a motion detection function for detecting the motion of the device itself, and transmits motion information to the main device 2. The user sings a song with the microphone 4 in one hand and the controller 5 in the other hand. As shown in FIG. 1, the controller 5 has a rectangular parallelepiped shape, but may have a shape that can be fitted on the user's arm, such as a wristband. In this case, the user can be prevented from dropping the controller 5 by mistake.

  The main unit 2 outputs the performance sound of the music selected by the user and displays on the monitor 3 a video in which the state of the live venue where many spectators entered is recorded. That is, the monitor 3 displays images of many spectators watching a singer (user) standing on the stage of the live venue.

  The user not only sings the song of the selected music piece, but also performs various live performances for the audience by calling the audience in the screen 3a through the microphone 4 or pointing the microphone 4 at the audience. The main device 2 detects the movement of both hands of the user based on the movement information transmitted from the microphone 4 and the controller 5. The main device 2 identifies the user's live performance based on the sound input to the microphone 4 and the movement information of the microphone 4 and the controller 5. An image showing the audience reaction to the identified live performance is displayed on the monitor 3. In this way, by using the karaoke system 1, the user can behave like a singer who actually performs at a live venue, so that a live performance with a high sense of reality can be simulated.

  Next, the configuration of the main device 2 will be described. FIG. 2 is a block diagram illustrating a functional configuration of the main device 2. The main device 2 includes a wireless communication unit 21, a data acquisition unit 22, a reproduction unit 23, a voice recognition unit 24, a performance identification unit 25, a reaction instruction unit 26, a storage unit 27, and an output unit 28. Prepare.

  The wireless communication unit 21 performs wireless communication with the microphone 4 and the controller 5 and acquires motion information 42 </ b> A and 51 </ b> A from the microphone 4 and the controller 5. For wireless communication, Bluetooth (registered trademark), wireless LAN, infrared communication, or the like can be used.

  The data acquisition unit 22 acquires music data 61, audience video data 62, a plurality of reaction data 63, 63,... Corresponding to the music selected by the user from the music server via the Internet or the like. The music data 61 is data in which the performance sound of the music selected by the user is recorded. Audience video data 62 is audience status data indicating the status of a large number of spectators (audiences) who entered the live venue, and is data that records the video and audio of the audience. Here, the audience refers to people who appreciate the song that the user sings along with the music at the live venue, and is a concept that includes the audience. The reaction data 63 is data in which video and audio indicating the reaction of the audience to the live performance are recorded. The reaction data 63 corresponds to any one of a plurality of live performances and also corresponds to any one of the audience reaction levels. Details of the reaction level will be described later.

  In the present embodiment, an example will be described in which audience video data 62 is used as audience situation data indicating the situation of a large audience. However, audio data indicating the situation of a large audience (audience) may be used. Specifically, in place of the audience video data 62, audio data in which a cheer of the audience (audience) is recorded can be used.

  The reproduction unit 23 reproduces the music data 61 and the audience video data 62 acquired by the data acquisition unit 22. The voice recognition unit 24 performs voice recognition processing on the voice data 4 </ b> A transmitted from the microphone 4 and detects a phrase input to the microphone 4 by the user. The voice recognition unit 24 outputs phrase information 24A indicating the detected phrase.

  The performance specifying unit 25 specifies the live performance performed by the user based on the phrase information 24A and the motion information 42A and 51A. The performance specifying unit 25 determines that the user has performed a live performance when the phrase information 24A includes a specific phrase (a phrase that the singer calls the audience) and the user has performed an action corresponding to the specific phrase. To do.

  The reaction instruction unit 26 determines the audience reaction level for the specified live performance. Reaction data 63 indicating the reaction of the audience is selected based on the identified live performance and the audience reaction level. The reaction instruction unit 26 determines the reproduction condition of the selected reaction data 63 based on the audience reaction level. The reproduction condition is a volume when the selected reaction data 63 is reproduced. The reaction instruction unit 26 instructs the reproduction unit 23 to reproduce the selected reaction data 63 under the determined reproduction condition.

  The storage unit 27 is a hard disk device or the like, and stores music data 61, video data 62, and reaction data 63, 63,... Acquired by the data acquisition unit 22. The output unit 28 outputs the audio and video generated by the playback unit 23 to the monitor 3.

  Next, the configuration of the microphone 4 will be described. FIG. 3 is a block diagram showing a functional configuration of the microphone 4. The microphone 4 includes a voice input unit 41, a sensor unit 42, and a wireless communication unit 43.

  The voice input unit 41 inputs a voice uttered by the user, converts it into an electrical signal, and outputs voice data 4A. The sensor unit 42 includes an acceleration sensor that detects a change in the movement of the microphone 4, and outputs movement information 42 </ b> A indicating the movement of the microphone 4. The sensor unit 42 may include a geomagnetic sensor or a gyroscope in addition to the acceleration sensor. The wireless communication unit 43 performs wireless communication with the main body device 2 and transmits audio data 4A and motion information 42A to the main body device 2.

  Next, the configuration of the controller 5 will be described. FIG. 4 is a block diagram showing a functional configuration of the controller 5. The controller 5 includes a sensor unit 51 and a wireless communication unit 52. The sensor unit 51 includes an acceleration sensor similarly to the sensor unit 42 of the microphone 4, and outputs motion information 51 </ b> A indicating the motion of the controller 5. The wireless communication unit 52 performs wireless communication with the main body device 2 and transmits motion information 51 </ b> A to the main body device 2.

  The karaoke system 1 may include a camera that captures a user's movement. In this case, the main body device 2 includes an image analysis unit that analyzes video data and detects a user's movement. For example, the camera is installed on the monitor 3 and outputs video data obtained by photographing the user to the main unit 2 in real time. When the user performs a live performance in front of the monitor 3 and the camera, the image processing unit generates and outputs motion information indicating the user's motion. The performance specifying unit 25 specifies the live performance performed by the user based on the phrase information 24A and the motion information output from the image analysis unit. In this case, the karaoke system 1 does not need to include the controller 5, and the sensor unit 42 may not be provided in the microphone 4.

{2. Operation of main device 2}
Hereinafter, the process of changing the video and audio of the spectator output from the monitor 3 in accordance with the user's live performance will be described focusing on the operation of the main device 2. In the following description, an example using motion information 42A and 51A transmitted from the microphone 4 and the controller 5 will be described. Similar processing is performed even when a user's movement is detected using a camera.

  FIG. 5 is a flowchart showing an operation flow of the main device 2. First, the user operates a remote controller (not shown) to select a song that he wants to sing at karaoke. The data acquisition unit 22 acquires music data 61 corresponding to the music selected by the user, audience video data 62, reaction data 63, 63,... From the music server and stores them in the storage unit 27. (Step S1).

  The reproducing unit 23 starts reproducing the music data 61 and the audience video data 62 (step S2). Thereby, the performance sound of the music selected by the user and the video and cheers of the audience enjoying the live performance at the live venue are output from the monitor 3.

  FIG. 6 is a diagram showing the monitor 3 on which many spectator videos are displayed. The user stands in front of the monitor 3 on which images of many spectators are displayed on the screen 3a, so as to face many spectators in the screen 3a. That is, the user sings along with the performance sound of the music as a singer that performs live for the audience displayed on the monitor 3. The user can perform various live performances while singing a song as a live singer.

  The main device 2 repeatedly executes the processes of steps S4 to S9 until the reproduction of the music data 61 ends (Yes in step S3).

  The microphone 4 is transmitting the audio | voice phrase input into the audio | voice input part 41 to the main body apparatus 2 in real time as audio | voice data 4A. When the main device 2 receives the voice data 4A (Yes in Step S4), the voice recognition unit 24 executes voice recognition processing on the voice data 4A (Step S5). The voice recognition unit 24 outputs phrase information 24A in which a voice phrase issued by the user is recorded. Based on the phrase information 24A, the performance specifying unit 25 determines whether or not the user has called a specific phrase to the audience in the screen 3a (step S6).

  When the phrase information 24A does not include a specific phrase (No in step S6), it is determined that the voice input to the microphone 4 is the user's singing voice. The main device 2 returns to the process of step S3. On the other hand, when the phrase information 24A includes a specific phrase (Yes in step S6), the performance specifying unit 25 determines that the user has called the audience for the specific phrase. And the performance specific | specification part 25 confirms whether it moved according to a specific phrase based on movement information 42A, 51A.

  The microphone 4 transmits the movement information 42A output from the sensor unit 42 to the main body device 2 in real time. Similarly, the controller 5 transmits the motion information 51A output from the sensor unit 51 to the main body device 2 in real time. The performance specifying unit 25 determines whether or not the user has made a movement according to the specific phrase based on the movement information 42A and 51A (step S7).

  If no movement corresponding to the specific phrase is detected (No in step S7), main device 2 returns to the process in step S3.

  When the movement according to the specific phrase is detected (Yes in step S7), the performance specifying unit 25 determines that the user has performed the live performance on the audience in the screen 3a. Examples of live performances performed by the user include audience enthusiasts, audience chorus leaders, and clapping leaders.

  The reaction instruction unit 26 uses the phrase information 24A and the movement information 42A, 51A to determine the audience reaction level with respect to the user's live performance (step S8). The reaction instruction unit 26 selects the reaction data 63 based on the live performance performed by the user and the reaction level. For example, when the reaction level is high, the reaction instructing unit 26 selects the reaction data 63 in which an image in which the entire audience responds to the user's live performance is recorded.

  The reaction instruction unit 26 instructs the reproduction unit 23 to reproduce the selected reaction data 63. The reproducing unit 23 reproduces the instructed reaction data 63 together with the music data 61 being reproduced (step S9). As a result, the reaction of the audience with respect to the user's live performance is displayed on the screen 3a, and the performance sound of the music and the cheer of the audience overlap and are output from the monitor 3.

  The reaction instruction unit 26 may change the reproduction condition of the reaction data 63 according to the reaction level. For example, if the reaction level is high, the audience cheer volume may be increased. If the response level is low, the audience cheer volume may be reduced, or the playback speed of data recorded with the audience cheer may be reduced.

  In the flowchart shown in FIG. 5, after detecting a specific phrase from the audio data 4 </ b> A, it is determined whether or not a movement according to the detected specific phrase has been performed. However, the processing order of steps S4 to S6 and the order of motion detection (step S7) may be interchanged according to the type of live performance. Moreover, you may identify a user's live performance based on any one of a specific phrase and a user's movement.

  In the present embodiment, an example is described in which the audience video data 62 and the reaction data 63 are different data, but the present invention is not limited to this. For example, each audience in the audience video data 62 may be converted into an object. In this case, the main body device 2 changes the movement of each audience object based on the specified live forman and reaction level. For example, the main device 2 can change the behavior and cheers of each spectator object as a spectator reaction to the specified live performance. Further, the reaction instruction unit 26 can change the proportion of the audience objects that react to the user's live performance according to the reaction level.

{3. Specific example of live performance}
Hereinafter, a specific example of live performance that can be detected by the main device 2 will be described.

{3.1. Spend the audience}
FIG. 7A and FIG. 7B are diagrams illustrating a procedure in which the user strikes the audience 31 in the screen 3a. 7A and 7B, only one spectator 31 is displayed in the screen 3a in order to easily explain the movement of the spectator 31 in the screen 3a. Actually, the reproduction unit 23 reproduces the audience video data 62, so that the live venue and a large number of spectators 31 coming to the live venue are displayed on the screen 3a. 7A and 7B, the display of the main body device 2 and the controller 5 is omitted.

  Audience resentment is a live performance in which a live singer calls a question-type phrase to the audience and the audience responds to the singer all at once. The user can perform audience enthusiasm when a prelude or interlude of music is being played.

  When the main unit 2 specifies the audience excitement as the live performance, the main unit 2 uses the audio data 4A and the motion information 42A transmitted from the microphone 4, and does not use the motion information 51A transmitted from the controller 5.

  The main device 2 first detects a specific phrase when the audience performance is specified as the live performance. As shown in FIG. 7A, the user first directs the microphone 4 toward the user himself, for example, a specific phrase in a question format of “Do n’t you?” (Hereinafter, referred to as “buzz phrase”). Is input to the microphone 4. The microphone 4 transmits the beat phrase as audio data 4A to the main device 2.

  The voice recognition unit 24 performs voice recognition processing on the received voice data 41A (step S5) to generate phrase information 24A. Various phrases that the singer calls to the audience during the live performance are set in the performance specifying unit 25. The performance specifying unit 25 compares the phrase information 24A with the various set phrases, and determines that the user has called the utterance phrase to the audience (Yes in step S6).

  The user turns the microphone 4 in the direction of the screen 3a from the user's own direction after inputting the phrase to the microphone 4 directed toward the user himself / herself. That is, the user reverses the direction of the microphone 4 as indicated by the arrow 45 (see FIG. 7A). The sensor unit 42 of the microphone 4 generates motion information 42A corresponding to the user's motion. The performance specifying unit 25 determines that the direction of the microphone 4 is reversed based on the motion information 42 </ b> A transmitted from the microphone 4. The performance specifying unit 25 determines that the user has engaged in the audience as a live performance because the speaking phrase is input to the microphone 4 and the direction of the microphone 4 is reversed (Yes in step S7).

  The reaction instructing unit 26 determines the audience reaction level with respect to the audience excitement performed by the user (step S8). The reaction instruction unit 26 determines the reaction level based on the time difference (detection time difference) between the voice input timing when the beat phrase is input and the motion detection timing when the inversion of the microphone is detected. The voice input timing is included in the phrase information 24A. The motion detection timing is included in the motion information 42A.

  In the reaction instruction unit 26, a time difference (ideal time difference) between the voice detection timing and the motion detection timing at which the reaction level is maximized is set. The reaction instruction unit 26 determines the reaction level based on the magnitude of the difference between the ideal time difference and the detection time difference. For example, when the deviation is small, the reaction instruction unit 26 determines that the audience has been smoothly struck and increases the reaction level. On the other hand, if the deviation is large, such as when the timing to invert the microphone is late, it is considered that the audience's reaction to the utterance phrase varies. In this case, the reaction level decreases. In addition, the reaction instruction unit 26 may determine the reaction level in consideration of the reproduction position in the music data 61 where the user has engaged the audience.

  The reaction instruction unit 26 selects the reaction data 63 for the audience audience from the reaction data 63, 63,... Stored in the storage unit 27 based on the determined reaction level. If the reaction level is high, the reaction data 63 in which a video in which all the spectators react to the audience is recorded is selected. If the reaction level is low, the reaction data 63 in which a video with a small number of spectators who react to the audience sensation is recorded is selected.

  The reaction instruction unit 26 instructs the reproduction unit 23 to reproduce the selected reaction data 63. At this time, the playback condition of the selected reaction data 63 is also notified to the playback unit 23. The reproduction condition is determined based on the reaction level with respect to the audience. For example, if the reaction level is high, the volume when the reaction data 63 is reproduced increases. If the reaction level is low, the volume of the reaction data 63 during playback can be reduced, and the playback speed of the audience cheer can be reduced.

  The reproduction unit 23 reproduces the reaction data 63 instructed by the reaction instruction unit 26 in accordance with the music data being reproduced (step S9). As a result, as shown in FIG. 7B, the audience 31 in the screen 3a raises the right hand as a response of the audience to the audience, but is displayed on the screen 3a and the cheer of the audience “Yay!” Is reproduced. . Actually, since the screen 3a displays how many spectators react to the audience at once, the user can experience the live performance at the live venue in a pseudo manner. Depending on how the audience is enthusiastic, the reaction of the audience 31 in the live venue changes greatly, so that the user can enjoy the same tension as a singer who actually performs live.

{3.2. Leading chorus}
As a live performance, the user can lead the chorus of the music sung by the user to the audience in the screen 3a. 8A and 8B are diagrams illustrating a procedure in which the user leads the audience's chorus.

  8A and 8B show only two spectators 31 and 31 in order to explain the movement of the spectators in the screen 3a in an easy-to-understand manner, as in FIG. 7A. 8A and 8B, the display of the main body device 2 and the controller 5 is omitted.

  When determining that the live performance leading the chorus has been performed, the main device 2 uses the audio data 4A and the motion information 42A transmitted from the microphone 4. The motion information 51A transmitted from the controller 5 is not used for the determination of the chorus lead.

  As shown in FIG. 8A, while the music data 61 is being reproduced, the screen 3a displays the video of the audience in the live venue and the lyrics 32 of the music being reproduced. As shown in FIG. 8A, the lyrics 32 corresponding to the portion (for example, the chorus portion of the song) that can be sung by the audience in the reproduced music are displayed surrounded by a square frame. Thereby, the user can know the timing which leads chorus.

  When the main device 2 specifies the chorus lead as the live performance, the main device 2 first detects a voice phrase (hereinafter referred to as “choral phrase”) that calls the chorus to the audience. For example, the user inputs a chorus phrase “Let's sing together!” To the microphone 4 while pointing the microphone 4 at the user himself / herself. The microphone 4 transmits the input choral phrase to the main device 2 as audio data 4A. In the same procedure as described above, the performance specifying unit 25 determines that the user has called for a choral phrase.

  After inputting the choral phrase into the microphone 4, the user reverses the microphone 4 in the direction of the screen 3 a (the direction of the arrow 45). The performance specifying unit 25 determines that the direction of the microphone 4 is reversed based on the motion information 42 </ b> A transmitted from the microphone 4. Since the chorus phrase is input to the microphone 4 and the direction of the microphone 4 is reversed, the performance specifying unit 25 determines that the user is leading chorus as a live performance.

  The reaction instruction unit 26 determines a reaction level for the lead of the chorus performed by the user. Since the procedure for determining the reaction level is the same as described above, the description thereof is omitted.

  The reaction instruction unit 26 instructs the reproduction of the reaction data 63 corresponding to the determined reaction level among the reaction data 63, 63,. The playback condition of the reaction data 63 is also notified to the playback unit 23. Thereby, the number of spectators who sing and the volume of the chorus change according to the reaction level. As shown in FIG. 8B, as a response to the chorus request made by the user, a state in which the spectators 31, 31 in the screen 3a sing with their shoulders crossed is displayed. In addition, the singing voice of the audience corresponding to the lyrics that the user has led the chorus is output from the monitor 3. Actually, the screen 3a displays a state in which many spectators sing at the same time, so the user can experience a live performance of singing a song together with the spectator at the live venue. it can.

  The audience's chorus is continued unless the user moves the microphone 4. When the user stops the chorus of the audience, the microphone 4 is reversed from the direction of the screen 3a to the user's own direction. If the direction of the microphone 4 is reversed during the reproduction of the reaction data 63, the performance specifying unit 25 determines that the lead has been completed. Thereby, the reproduction of the music data 61 and the audience video data 62 is resumed.

{3.3. Leading hand clapping}
As a live performance, the user can perform an operation of leading clapping to the audience 31 in the screen 3a. FIG. 9A and FIG. 9B are diagrams showing a procedure in which the user leads the hand clapping to the spectators 31 and 31 in the screen 3a.

  9A and 9B show only two spectators 31 and 31 in order to explain the movement of the spectators 31 in the screen 3a in an easy-to-understand manner, as in FIG. 7A. 9A and 9B, the display of the main device 2 is omitted. The main body device 2 uses the audio data 4A, the motion information 42A transmitted from the microphone 4 and the motion information 51A transmitted from the controller 5 when detecting the lead of the clapping.

  As shown in FIG. 9A, while the music data 61 is being played back, the user can lead clapping as a live performance. When the main device 2 detects the lead of the hand time signature, the main device 2 first detects a voice phrase that leads the hand time signature (hereinafter referred to as a “hand time phrase”) (Yes in step S6).

  Specifically, the user inputs a clapping phrase into the microphone 4. The clapping phrase is, for example, “Everyone clapping!” The microphone 4 transmits the input clapping phrase to the main device 2 as audio data 4A. In the same procedure as described above, the performance specifying unit 25 determines that the user has called a clapping phrase to the audience in the screen 3a.

  The user performs an operation of leading the clapping after inputting the clapping phrase to the microphone 4. As shown in FIG. 9B, the user clapping in time with the rhythm of the music performance sound (reproduced music data 61). Since the user has the microphone 4 in the right hand and the controller 5 in the left hand, the microphone 4 and the controller 5 transmit motion information 42 </ b> A and 51 </ b> A indicating the movement of the hand clapping to the main body device 2.

  The performance specifying unit 25 detects the movement of both hands of the user based on the received movement information 42A and 51A. For example, when the microphone 4 and the controller 5 repeatedly move left and right, and the direction in which the microphone 4 moves and the direction in which the controller 5 moves are opposite directions, the performance specifying unit 25 indicates that the user is clapping. It can be judged (Yes in step S7). In other words, the performance specifying unit 25 determines that the user is leading the clapping by detecting the clapping phrase and the clapping action.

  The reaction instruction unit 26 determines the response level of the audience based on the timing at which the clapping phrase is detected and the timing at which the movement of the clapping is detected (step S8). The procedure for determining the reaction level is basically the same as that for determining the response level of the audience. Moreover, the reaction instruction | indication part 26 may calculate a user's clapping rhythm using movement information 42A, 51A, and may determine a reaction level based on the calculated rhythm. For example, the reaction instruction unit 26 can detect a deviation between the rhythm of the musical performance sound and the clapping rhythm of the user, and the reaction level can be increased as the detected deviation is smaller.

  The reaction instruction unit 26 instructs the reproduction of the reaction data 63 corresponding to the determined reaction level among the reaction data 63, 63. The playback condition of the reaction data 63 is also notified to the playback unit 23. As a result, the reaction data 63 that the audience is clapping with the music data 61 is reproduced (step S9), and as shown in FIG. 9B, the video of the audience 31, 31 in the screen 3a clapping overhead. Is displayed. Of the audience displayed on the screen 3a, the proportion of the audience clapping varies depending on the reaction level. In addition, the volume of the clapping also changes depending on the playback condition determined based on the reaction level.

  In this way, the video of the audience 31, 31 clapping in the screen 3a is displayed in accordance with the movement of the user clapping, so the user is clapping with the audience at the live venue. You can experience live performance in a simulated manner.

{3.4. Leading wave (motion of shaking both hands)}
As a live performance, the user can perform an operation of leading the wave to the audience in the screen 3a. A wave is an operation in which the audience at a live venue shake their hands greatly to the left and right according to the music being played. 10A and 10B are diagrams illustrating a procedure in which a user leads a wave.

  10A and 10B show only two spectators 31 and 31 in order to explain the movement of the spectators in the screen 3a in an easy-to-understand manner, as in FIG. 9A. 10A and 10B, the display of the main device 2 is omitted. When detecting the leading of the wave, the main body device 2 uses the audio data 4A and motion information 42A transmitted from the microphone 4 and the motion information 51A transmitted from the controller 5.

  As shown in FIG. 10A, while the music data 61 is being reproduced, the user can lead the wave as a live performance. When the main device 2 detects the leading of the wave, the main device 2 first detects an audio phrase that calls the wave (hereinafter referred to as a “wave phrase”).

  Specifically, the user inputs a wave phrase to the microphone 4. For example, the wave phrase is “Everyone shake both hands!”. The microphone 4 transmits the input wave phrase to the main unit 2 as audio data 4A. In the same procedure as described above, the performance specifying unit 25 determines that the user has called a wave phrase to the audience in the screen 3a (Yes in step S6).

  The user performs a wave operation after inputting the wave phrase to the microphone 4. As shown in FIG. 10B, the user shakes his hands greatly over the head in accordance with the rhythm of the performance sound of the music (music data 61 to be reproduced). Since the user has the microphone 4 on the right hand and the controller 5 on the left hand, the microphone 4 and the controller 5 transmit motion information 42 </ b> A and 51 </ b> A indicating wave motion to the main body device 2.

  The performance specifying unit 25 detects the movement of both hands of the user based on the received movement information 42A and 51A. For example, if the performance specifying unit 25 detects an operation in which the microphone 4 and the controller 5 repeatedly move left and right, and the direction in which the microphone 4 moves matches the direction in which the controller 5 moves, the user performs a wave operation. Judge that

  When it is determined that the user is leading the wave (Yes in step S7), the reaction instruction unit 26 determines the reaction level of the audience (step S8). The reaction level is determined based on the rhythm with which the user shakes both hands. For example, the reaction instruction unit 26 detects a deviation between the rhythm of the musical performance sound and the user's wave rhythm, and the reaction level can be increased as the detected deviation is smaller.

  The reaction instruction unit 26 instructs the reproduction of the reaction data 63 corresponding to the determined reaction level among the reaction data 63, 63,. Thereby, the reaction data 63 that the audience waves together with the music data 61 to be reproduced is reproduced (step S9), and as shown in FIG. 10B, the audience 31 and 31 in the screen 3a wave over the head. Will be displayed. Of the audiences displayed on the screen 3a, the proportion of the audiences who wave will vary depending on the reaction level.

  When a user leads a hand beat or wave as a live performance, the user does not have to call the hand beat phrase or wave phrase to the audience in the screen 3a. For example, the main device 2 may reproduce the reaction data 63 in which the audience is clapping even if the user detects only a clapping phrase without detecting a clapping phrase.

  The user may lead the clapping or wave using only the microphone 4 without using the controller 5. The performance specifying unit 25 may determine that the user is leading the clapping or wave when the microphone 4 repeatedly moves left and right. At this time, the difference between the clapping and the wave may be distinguished by the size by which the user moves the microphone 4 left and right.

{3.5. Admission and Exit}
As a live performance, the user can enter or leave the live venue displayed on the screen 3a in a pseudo manner.

  First, a case where a user enters a live venue in a pseudo manner will be described. FIG. 11 is a diagram showing a procedure for a user to pseudo-enter a live venue. The performance specifying unit 25 detects the user's movement based on the presence / absence of a radio signal from the microphone 4 instead of the movement information 42A and 51A.

  The user goes out of the room where the main device 2 is installed and stands by before the reproduction of the music data 61 is started. At this time, the wireless communication unit 21 of the main body device 2 cannot detect the wireless signal transmitted from the microphone 4.

  After the reproduction of the music data 61 is started, the user enters the room where the karaoke system 1 is installed with the microphone 4. The wireless communication unit 21 can detect a wireless signal from the microphone 4 when the user enters the room. When the performance specifying unit 25 detects a wireless signal from the microphone 4 after the reproduction of the music data 61 is started, the performance specifying unit 25 determines that the user has entered the live venue.

  When it is determined that the user has entered the live venue, the reaction instruction unit 26 determines the reaction level of the audience. The reaction instruction unit 26 determines the reaction level of the audience based on the timing when the wireless signal from the microphone 4 is detected.

  The reaction instructing unit 26 instructs the reproducing unit 23 to reproduce the reaction data 63 indicating the reaction of the audience when the singer enters. At this time, the playback condition of the reaction data 63 is also instructed to the playback unit 23. The playback unit 23 plays back the reaction data 63 in which the video that the audience is greatly excited and the cheer of the audience are recorded in accordance with the entrance of the user. The volume of the audience cheers is changed according to the playback conditions.

  Next, the user leaving the live venue will be described. When the user leaves the live venue in a pseudo manner, the user leaves the room with the microphone 4. When the wireless communication unit 21 cannot detect the wireless signal from the microphone 4, the performance specifying unit 25 determines that the user has left the live venue.

  When it is determined that the user has left the live venue, the reaction instruction unit 26 determines the response level of the audience. The reaction instruction unit 26 determines the reaction level at the time of leaving based on the timing when the user leaves the live venue and the reaction level of the live performance performed until the user leaves. If the reaction level is high, the reaction instructing unit 26 instructs the reproducing unit 23 to reproduce the reaction data 63 for which the audience requests encore. On the other hand, if the reaction level is low, the reaction instruction unit 26 instructs the reproduction unit 23 to reduce the volume of the cheer of the audience, or instructs the reproduction unit 23 to reproduce the reaction data 63 that the audience is booing. can do.

  The microphone 4 may be provided with a switch for notifying that the user has entered and exited. In this case, when the entrance switch is pressed, the microphone 4 transmits an entrance signal to the main body device 2. When the exit switch is pressed, the microphone 4 transmits an exit signal to the main body device 2. The main device 2 changes the audience reaction in response to the reception of the entrance signal and the exit signal. Thereby, the user does not need to go in and out of the room where the main body device 2 is installed.

  In this way, the user can experience a live presence in a pseudo manner even before starting to sing a song or after singing a song.

{3.6. Other live performances}
The user can perform various live performances in addition to the live performance described above.

  For example, the user may sing with the microphone 4 in the right hand while turning the left hand with the controller 5 greatly. In this case, the controller 5 transmits movement information 51A indicating the movement of the controller 5 to the main body device 2. The performance specifying unit 25 specifies the movement of the user's left hand based on the movement information 51A. In this case, the reaction data in which the video of the spectator turning his / her left hand in accordance with the user's movement is recorded together with the music data 61.

  The user may perform a performance combining the above-described live performance. For example, the user may simultaneously perform chorus lead and wave lead as a live performance. The user performs the wave operation shown in FIG. 10B after inputting the choral phrase into the microphone 4. The performance specifying unit 25 determines that the user is leading chorus based on the detected choral phrase, and determines that the wave is leading based on the detected movement of both hands of the user. As a result, an image in which the audience waves is displayed on the screen 3a. In addition, the performance sound of the music being reproduced and the singing voice sung by the audience are simultaneously output from the monitor 3.

{4. Functions other than live performance}
{4.1. Audience designation function}
When karaoke is performed using the karaoke system 1, the user can specify the audience of the audience at the live venue. The user needs to perform live performance according to the set customer segment.

  FIG. 12 is a diagram showing a setting screen for setting a live customer segment. The user operates a remote controller (not shown) to set the audience audience based on gender and age. In the gender setting, the user designates a male subject, a female subject, or an intermediate. The male subject indicates that the ratio of male spectators is set to 80%. The female subject indicates that the ratio of female audience members is set to 80%. The intermediate means that the ratio of males to females is 50%. In the age setting, the user selects from 15 to 25 years old, 25 to 45 years old, and 45 years old or older. By the age setting, the ratio of the audience in the age group selected by the user is set to 70%. As a result, together with the music data 61, audience video data 62 corresponding to the set customer segment is reproduced.

  When the audience is set, the audience reaction level changes according to the music selected by the user and the live performance performed by the user. For this reason, the user needs to perform a live performance in consideration of the audience of the audience. For example, when a user designates a female-centered audience who is over 45 years old, by selecting a relatively slow tempo music and performing a live performance (such as wave guidance) that does not require intense movement for the audience , Can increase the reaction level. On the other hand, when a hard rock music is led by the same audience and the audience is repeatedly struck as a live performance, the audience's reaction level becomes low. In this way, by setting the audience layer of the audience, the user can experience a simulated live performance for various audiences.

{4.2. Battle mode}
The karaoke system 1 can execute a battle mode in which a plurality of users compete for live performance. Hereinafter, the operation of the main device 2 when the battle mode is executed will be described.

  In the battle mode, two users perform live performance alternately. Here, the case where a battle | competition is performed using the music which has the lyrics to the 3rd is demonstrated. First, the first user sings song 1 while performing a live performance. Next, the second user sings the second piece of music while performing a live performance. At this time, the first user and the second user share the microphone 4 and the controller 5. However, a plurality of microphones 4 and controllers 5 may be prepared for the number of persons performing the battle mode.

  The live performance performed by each user is scored by the main device 2. Among the first user and the second user, a user with a high score can sing No. 3 of the music being played.

  Explain how to score live performances. As described above, the main body device 2 determines the reaction level every time the live performance performed by the user is performed. Each time the first user performs a live performance, the main device 2 scores each performance based on the determined reaction level. The total score for each live performance performed by the first user is calculated as the score of the first user. The score of the second user is calculated in the same way.

  After the second user finishes the live performance, the score of each user is displayed on the screen 3a. The winning user can perform a live performance with the third song. Thus, by providing the battle mode, it is possible to provide a new karaoke system in which a plurality of users compete for live performance.

{Modifications}
The microphone 4 and the controller 5 may have a vibration function. For example, the reaction instruction unit 26 of the main device 2 transmits a vibration instruction signal to the microphone 4 and the controller 5 when the audience reaction level exceeds a certain level. When the microphone 4 and the controller 5 receive the vibration instruction signal, the microphone 4 and the controller 5 vibrate their own devices for a certain period of time (about 3 seconds). Thereby, the user can know not only the image of the audience in the screen 3 a but also the reaction of the audience via the microphone 4 and the controller 5.

  The main device 2 may be provided with a live venue selection function. In this case, the user can select an arena, a baseball field, a concert hall, or the like as a live venue by operating a remote controller (not shown). The main device 2 reproduces the audience video data corresponding to the selected venue. Thereby, the user can experience the live performance in various live venues in a pseudo manner as a live singer.

  As illustrated in FIG. 2, the example in which the main device 2 includes the voice recognition unit 24 has been described, but the microphone 4 may include the voice recognition unit 24. The microphone 4 transmits the generated phrase information 24 to the main body device 2 in real time each time the phrase information 24A is generated by the voice recognition unit 24.

  The example in which the reaction data 63 is data in which video and audio indicating the reaction of the audience to the live performance is recorded has been described. However, the reaction data 63 may be data in which only sound indicating the reaction of the audience is recorded. For example, the data in which the cheering of the audience with respect to the audience sensation, the clapping of the clapping of the clapping, the booing of the audience, etc. can be used as the reaction data 63.

  The reaction data 63 may be data that symbolizes the reaction of the audience rather than the data that specifically indicates the audience's action on the live performance as described above. For example, as the reaction data 63, it is possible to use data in which a video in which a plurality of fireworks are recorded from a live venue is recorded. When the reaction instruction unit 26 changes the number of fireworks to fire according to the reaction level, the user can know the reaction of the audience to the live performance. Or the reaction instruction | indication part 26 may select the data by which the image | video of the live venue without a spectator was recorded, and the data by which the raging sea was recorded as the reaction data 63 when the reaction level is low.

DESCRIPTION OF SYMBOLS 1 Karaoke system 2 Main body apparatus 3 Monitor 3a Screen 4 Microphone 5 Controller 21,43,52 Wireless communication part 22 Data acquisition part 23 Reproduction part 24 Voice recognition part 25 Performance specific part 26 Reaction instruction part 42, 51 Sensor part

Claims (17)

A main unit;
A voice input device that is held by a user and outputs voice input by the user as voice data;
A speech recognition device that performs speech recognition processing on the speech data and generates phrase information indicating a phrase spoken by the user;
With
The voice input device includes:
A first motion information output unit that outputs first motion information indicating the motion of the voice input device;
Including
The main unit is
A reproducing unit for reproducing the music data selected by the user,
The phrase information and based on the first motion information, and performance specifying unit for specifying the performance of the user,
An audience response level with respect to the identified performance is determined based on a time difference between the timing at which the phrase information is detected and the timing at which the first motion information is detected, and a plurality of reaction data indicating the audience response A reaction instruction unit for selecting reproduction reaction data according to the identified performance and the audience reaction level, and instructing the reproduction unit to reproduce the reproduction reaction data;
Amusement system including. A main unit;
A voice input device that is held by a user and outputs voice input by the user as voice data;
An imaging device that images the user and outputs video data;
A speech recognition device that performs speech recognition processing on the speech data and generates phrase information indicating a phrase spoken by the user;
With
The main unit is
A reproducing unit for reproducing the music data selected by the user,
A video analysis unit that analyzes the video data and generates first motion information indicating the movement of the user;
The phrase information and based on the first motion information, and performance specifying unit for specifying the performance of the user,
An audience response level with respect to the identified performance is determined based on a time difference between the timing at which the phrase information is detected and the timing at which the first motion information is detected, and a plurality of reaction data indicating the audience response A reaction instruction unit for selecting reproduction reaction data according to the identified performance and the audience reaction level, and instructing the reproduction unit to reproduce the reproduction reaction data;
Amusement system including. In the amusement system according to claim 1 or 2 ,
An amusement system in which the reaction instruction unit determines a reproduction condition of the reproduction reaction data based on a reaction level of the audience, and instructs the reproduction unit of the reproduction condition. In the amusement system according to claim 3 ,
The said reaction instruction | indication part is an amusement system which determines the reaction level of the said audience based on the customer base of the audience set by the said user . In the amusement system according to any one of claims 1 to 4 ,
The performance specifying unit calls the user a specific phrase in a question format based on the phrase information, and the direction of the voice input device is reversed based on the first movement information. If it is determined that the user has performed a performance of scolding the audience,
The reaction instruction unit is an amusement system that selects, as the reproduction reaction data, reaction data in which video and audio in which an audience responds to the specific phrase all at once are recorded. In the amusement system according to any one of claims 1 to 4 ,
The performance specifying unit calls a specific phrase for the user to request chorus based on the phrase information, and the direction of the voice input device is reversed based on the first movement information. If it is determined that the user has performed a performance requesting chorus from the audience,
The reaction instruction unit is an amusement system that selects reaction data in which video and audio sung by the audience are recorded as the reproduction reaction data. In the amusement system according to any one of claims 1 to 4 ,
When the performance identifying unit determines that the user is clapping based on the motion information, the performance identifying unit determines that the user has performed a performance that leads the clapping,
An amusement system in which the reaction instruction unit selects reaction data in which video and clapping sounds recorded by the audience are recorded as the reproduction reaction data. In the amusement system according to any one of claims 1 to 4 ,
The performance specifying unit determines that the user has performed a performance requesting an operation of waving both hands to the audience when it is determined that the user is performing an operation of waving an arm based on the movement information. ,
The reaction instruction unit is an amusement system that selects reaction data in which a video of the audience shaking hands is recorded as the reproduction reaction data. The amusement system according to any one of claims 1 and 3 to 8 , further comprising:
A controller that the user holds with the hand opposite to the hand holding the voice input device;
With
The controller is
A second motion information output unit for outputting second motion information indicating the motion of the controller;
Including
The performance specifying unit is an amusement system that specifies the performance of the user based on the second motion information. In the amusement system according to any one of claims 3 to 9 ,
The reaction instruction unit scores all performances performed in the first part of the music data based on the audience's response level for each performance, and performs all performances performed in the second part of the music data. An amusement system for scoring a game based on the audience's level of response to each performance. A main unit;
A voice input device held by the user;
With
The voice input device includes:
A wireless communication unit for performing wireless communication with the main device,
Including
The main unit is
A reproducing unit for reproducing the music data selected by the user,
A performance identifying unit that identifies the performance of the user based on the presence or absence of a radio signal transmitted from the voice input device;
From a plurality of the reaction data that the audio or video showing the reaction was recorded auditory crowd, to select the playback reaction data corresponding to the specified performance, reaction instruction to reproduction of the reproduction reaction data to the reproduction unit And
Amusement system including. The amusement system according to claim 11,
An amusement system in which the performance specifying unit determines that the user has entered a virtual live venue when the wireless signal is detected after the reproduction of the music data is started. In the amusement system according to claim 11 or 12 ,
The performance specifying unit determines that the user has left the virtual live venue when the wireless signal cannot be detected after the music data has started to be reproduced.   A voice input device used in the amusement system according to any one of claims 1 to 13. A voice input device that holds the user and outputs the voice input by the user as voice data and outputs first motion information indicating the movement of the user apparatus, and performs voice recognition processing on the voice data, and the user A speech recognition device that generates phrase information indicating a spoken phrase and a computer mounted on a communicable main unit,
Reproducing unit for reproducing the music data selected by the user,
A performance identifying unit that identifies the performance of the user based on at least one of the phrase information and the first motion information;
An audience response level with respect to the identified performance is determined based on a time difference between the timing at which the phrase information is detected and the timing at which the first motion information is detected, and a plurality of reaction data indicating the audience response A reaction instruction unit that selects reproduction reaction data according to the identified performance and the audience reaction level, and instructs the reproduction unit to reproduce the reproduction reaction data;
Program to function as. An audio input device that is held by a user and outputs audio input by the user as audio data, an imaging device that images the user and outputs video data, and performs audio recognition processing on the audio data, and the user A speech recognition device that generates phrase information indicating a phrase spoken by a computer and a computer mounted on a communicable main unit,
Reproducing unit for reproducing the music data selected by the user,
A video analysis unit that analyzes the video data and generates first motion information indicating the movement of the user;
A performance identifying unit that identifies the performance of the user based on at least one of the phrase information and the first motion information;
An audience response level with respect to the identified performance is determined based on a time difference between the timing at which the phrase information is detected and the timing at which the first motion information is detected, and a plurality of reaction data indicating the audience response A program for selecting playback reaction data corresponding to the specified performance and the audience reaction level from among them and causing the playback unit to function as a reaction instruction unit that instructs the playback unit to play back the playback reaction data. A computer installed in the main unit capable of wireless communication with the voice input device held by the user,
Reproducing unit for reproducing the music data selected by the user,
A performance identifying unit that identifies the performance of the user based on the presence or absence of a radio signal transmitted from the voice input device;
From a plurality of the reaction data indicating the response of hearing crowd, select the reproduction reaction data corresponding to the specified performance, reaction instructing unit for instructing the reproduction of the reproduction reaction data to the reproduction portion,
Program to function as.

Images