JP2000047675A - Singing sound device and action expression deciding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a singing sound device which can increase the presence feeling and absorption feeling at the time of singing and gives effects on concentration on the singing and intoxication by employing device constitution which scores an input voice of a microphone under specific conditions and reflects the scoring result on surrounding environmental facilities (video facilities, sound facilities, etc.), and generating and providing proper video and sound reacting to the singing for users. SOLUTION: A singing skill scoring device 22 actualized by a command computing device generates scores for finding a command for video variation according to the voice inputted from a microphone device 21 and sends them out to a video and sound generating device 23. The video and sound generating device 23 generates video and auxiliary sound according to the scores (scoring result) inputted from the command computing device 22 and sends them out to a monitor device 24 and a speaker device 25.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a singing sound apparatus suitable for a karaoke facility or the like having a singing ability determination function.
Also, the present invention relates to a method for determining an acting expression such as singing and singing.

[0002]

2. Description of the Related Art A device constituting a conventional karaoke facility has a device configuration A having a function of displaying (outputting) an image in accordance with the image of the lyrics supplementarily for the purpose of presenting the lyrics. There is a karaoke facility or a karaoke facility of apparatus configuration B having a function of performing scoring according to singing ability and presenting a score according to the scoring at the end of the singing. Furthermore, there is a karaoke facility having a device configuration having both functions.

However, the above-described conventional karaoke facilities have the following problems.

That is, in the karaoke facility having the above-described apparatus configuration A, the video does not respond to the singing skill or the singing style, and the karaoke user can be satisfied with one-sided singing. It had only functions and had a limited range of services. In other words, singing affects the world (now video and audio) and has no effect of "immersion" that draws the user into a world different from reality. In addition, the video is 2
It is provided from a TV monitor of about 0 inches, the content of the image is independent of the place of the karaoke, and it can satisfy the illusion that the user is in a place, situation, etc. that is different from the real place. There was no effect to give "realism."

[0005] In the karaoke facility of the above-described apparatus configuration B, only the score is presented at the end of the singing, and there is no "immersion" or "realism" in the middle of the singing. The response to the song was only a score, and the range of services was limited.

As described above, in the conventional karaoke equipment, the singing affects the surrounding environmental equipment (here, the video equipment and the audio equipment) so that the singing is absorbed in the singing (enthusiasm effect), and It does not have the effect of becoming euphoric (euphoric effect).

[0007]

As described above, in a conventional karaoke facility which has a main purpose of presenting lyrics and additionally has a device configuration in which a video according to the image of the lyrics is played. Etc. do not respond to singing skills (good or bad singing or differences in singing style), and have only the function of providing singing singing satisfaction for karaoke users. I was Therefore, there is no so-called immersive effect (effect of being immersed in singing) in which singing affects surrounding environmental equipment (here, video equipment and audio equipment) and draws the user into a world different from reality. Furthermore, the video is provided from a TV monitor of about 20 inches, and the content of the video is not related to the place of the karaoke, and uses the sensation (illusion) of being present in a place or situation different from the real world. There is no so-called effect of giving a sense of reality (effect of falling into a state of euphoria separated from reality) which is given to a person.

[0008] The present invention has been made in view of the above circumstances,
Singing ability is converted to surrounding environmental equipment (video equipment, audio equipment, etc.)
Singing sound that has the effect of immersing in singing and the effect of falling into euphoria, which can enhance the sense of presence and immersion when singing by creating and providing appropriate images in response to singing It is intended to provide a device.

Further, the present invention reflects the performance of singing, singing, etc. on environmental equipment such as video equipment and audio equipment around the singing,
By generating and providing appropriate images and sounds in response to singing, singing and other performances, it is possible to enhance the sense of presence and immersion during singing and singing and other performances. It is an object of the present invention to provide a method for determining an acting expression that can be immediately specified to the viewers around it.

[0010]

SUMMARY OF THE INVENTION The present invention is directed to a device configuration for reacting an input sound of a microphone to surrounding environmental equipment (video equipment, audio equipment, etc.), whereby the effect of immersing in singing and the state of euphoria are achieved. It has the effect of falling.

That is, the present invention relates to a singing sound apparatus having a microphone for picking up voice, an accompaniment apparatus for reproducing accompaniment, and a loudspeaker for vocalizing the voice and accompaniment. Means, singing ability scoring means for scoring singing by accompaniment and voice, and auxiliary video means for reflecting the scoring in the video.

According to the present invention, there is provided a singing sound apparatus having a microphone for picking up voice, an accompaniment apparatus for reproducing accompaniment, and a loudspeaker for vocalizing the voice and accompaniment. It is characterized by comprising a display means, a singing ability scoring means for scoring the singing by the accompaniment and the voice, and an auxiliary sound means for reflecting the scoring on at least the voice or the accompaniment.

With the above-mentioned configuration, the singing ability is reflected on the surrounding environmental equipment (image equipment, audio equipment, etc.)
By generating and providing an appropriate image in response to singing, it is possible to provide a singing sound device that can enhance the sense of presence and immersion at the time of singing, have an effect of immersing in singing, and an effect of falling into a euphoric state .

Further, the present invention relates to a method for judging a performance expression such as singing, singing, and the like, in which a user's acting action present in an effect space created by two-dimensional or more images and sound effects is performed under predetermined conditions. The level is determined, and the result of the level determination is reflected on at least the video or audio in an expression form embodied so that the determination level can be analogized.

According to the above-described acting expression determination method,
Singing by reflecting the performance of singing, singing, etc. on the surrounding environmental equipment (video equipment, sound equipment, etc.) and generating and providing appropriate video and audio in response to singing, singing, etc. In addition, it is possible to enhance the sense of presence and immersion when performing a performance such as a singing dance, and the determination content can be immediately specified to the performer and the surrounding viewers.

[0016]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the basic configuration of the entire apparatus according to the present invention. The basic configuration of the apparatus is such that an image reacts to an input voice of a microphone.

The device shown in FIG. 1 includes a microphone device (microphone) 11 for picking up a user's voice and a microphone device 1.
1. A command calculation device 12 for obtaining a command for a video change from the user's voice obtained from Step 1 and a video / audio generation device 13 for generating a video or an auxiliary voice according to the score of the command calculation device 12.
A monitor device 14 for displaying the video generated by the video / audio generation device 13 and auxiliary information generated as necessary, and a speaker device 15 for outputting the voice generated by the video / audio generation device 13. It comprises.

The command calculation device 12 generates a score for obtaining a command for a video change based on the audio input from the microphone device 11 and sends it to the video / audio generation device 13. The video / audio generation device 13 generates a video and an auxiliary audio according to the score input from the command calculation device 12 and sends them to the monitor device 14 and the speaker device 15.

FIG. 2 shows the configuration of a karaoke facility according to an embodiment of the present invention to which the apparatus having the above-described basic configuration is applied.

The apparatus configuration of the karaoke facility shown in FIG. 2 is realized by a microphone device 21 that picks up the singer's singing voice and a command calculation device that scores the singing power from the singing voice of the singer obtained by the microphone device 21 as needed. Singing ability scoring device 22
And a video / audio generation device 23 that generates a video or an auxiliary voice according to the score of the singing ability scoring device 22, a video generated by the video / audio generation device 23, and lyrics generated as needed. A monitor device 24 for displaying, a speaker device 25 for outputting the sound generated by the video / sound generating device 23, a song selection / interruption / cancellation for karaoke, management / search of accompaniment data and lyrics data, singing voice And a karaoke basic function device 26 having functions such as amplification and output.

The singing ability scoring device 22 realized by the command calculation device scores the singing ability from the singer's singing voice input from the microphone device 21 as needed, and displays the scored score as an image / video.
It is sent to the voice generation device 23. Video / audio generator 23
Generates a video and an auxiliary sound in accordance with the score input from the singing ability scoring device 22 and sends it to the monitor device 24 and the speaker device 25.

FIGS. 3 and 3 show specific examples in which the “immersiveness” (effect of immersing in singing) and the “realism” (effect of falling into a euphoric state) are further enhanced in the karaoke facility having the system configuration shown in FIG. This will be described with reference to FIG.

FIG. 3 is a functional explanatory diagram of the karaoke facility for providing an immersive feeling (effect of immersing in singing) in the above embodiment.

In FIG. 3, by generating and displaying video and audio in response to information provided by the user, such as the skill of singing, the user's feeling of “immersion” is enhanced. Here, a case where the image is a cherry tree will be described as a specific example.

The singing voice of the user input from the microphone device 21 is input to the singing ability scoring device 22. Here, the original pitch of the designated song and the user's pitch are compared in real time, and the score is output to the video / audio generation device 23 as needed.

The singing ability scoring device 22 may use the scoring method adopted in the above-described prior art.
Depending on the purpose, a case where the loudness (sound pressure) of the voice is simply measured, a case where the pattern of the voice (a rhythm) is measured, or the like may be considered.

The video / audio generation device 23 has a reference video (eg, cherry blossoms that bloom in 5 minutes in the case of Sakurabayashi), and changes the video as the score increases. For example, when the score rises, an image in which the flowering of the cherry tree has advanced is generated. On the other hand, when the score drops, the wind blows and the cherry blossoms fall,
Play the sound of the wind in the background (so as not to disturb the song).

As a specific method of generating an image, a large number of sample images are recorded and an optimum image is automatically selected (a). A basic image model is provided as a CG model, and the model is changed. (B) of creating a video may be considered. Here, any of the above methods (a) and (b) may be adopted.

The generated video information is input to the monitor device 24, and is presented to the user as a real video in which the cherry blossoms bloom or scatter. In addition, sound effects such as auxiliary winds are input to the speaker device 25, and are played on the background of the singing voice as wind sounds that scatter flowers.

Although the above description has been given with reference to the example of Sakurabayashi, the following examples may be considered as specific examples that influence the image world by the score of the song and enhance the immersion of the user. These are merely examples, and the present invention does not limit the specific contents of the video.

Example 1 Cultivate digital pets. The higher the score, the faster the pet grows, but the lower the score, the weaker the pet. Or it becomes a variant. In the case of using a karaoke box in a group, etc., it is possible to continue raising the same pet over a plurality of stations without ending with one song.

Example 2 Village station landscape. The higher the score, the more people gather and show off. The lower the score, the less popular it is, the empty cans are blown away, and the snow turns into a blizzard.

Example 3 Video of the audience seating from the stage of the large hall. The higher the score, the more the audience stands and throws tapes, etc. with loud cheers. As the score decreases, guests begin to fall asleep, take a seat, and vacancies begin to stand out.

FIG. 4 is a functional explanatory diagram of a karaoke facility that provides a sense of presence (effect of falling into a euphoric state) in the above embodiment.

The singing situation (singing environment) is not a real karaoke box or snack stage, but a real situation such as a large hall or a stage before a judge, and an image apparatus for providing a virtual reality is used. This enhances the “realism”. By providing such an environment, it can be used for stage simulation in singer lessons, rehearsal for contests, and practice. Here, an example will be described in which a backup is performed in which a spotlight is placed on an existing large stage or a stage such as a hall, and the audience sings a song.

The singing voice of the user input from the microphone device 21 is input to the singing ability scoring device 22. Here, the original pitch of the designated song and the user's pitch are compared in real time, and the score is output to the video / audio generation device 23 as needed.

The singing ability scoring device 22 can use the scoring method employed in the above-described prior art.
For example, it is conceivable to prepare a complete pitch and strictly match the pitch by frequency analysis or the like.

The video / audio generation device 23 has a reference video (in the case of a stage, for example, a calmly packed audience) and changes the video as the score increases. For example, when the score rises, the customer gets excited, the gesture becomes large, and the person stands. Cheers are also assisted by the monitor 24. The monitor device 24 uses a large-screen wall-mounted display device or the like so that a clear image can be displayed on a large screen in order to enhance the sense of realism. From the speaker device 15, a realistic noise of the venue is played.

As specific examples, the following examples can be considered. These are merely examples, and the present invention does not limit the specific contents of the video.

Example 1 Display the judging room where many judges are lined up. The lyrics are not displayed, and the user's song is subjected to voice recognition in the singing ability scoring device 22 to take into account any mistakes in the lyrics. If the score decreases due to a mistake in the lyrics, the expression of the judge becomes cloudy, grows short, and falls asleep.

Example 2 Displays the opening ceremony venue for sports competitions. Sing "Kimi gayo" etc. in front of a large crowd.

As a technical feature, in response to the user's voice, the image and the voice are changed in real time, so that the user can be immersed in the video world presented in response to the voice. It has the characteristic of giving a sense of presence as if you were in a situation other than the place. In particular, in karaoke, the user's song is scored, and the video and audio are changed in real time according to the score, so that the user can be immersed in the video world presented while singing the song. Have a sense of presence as if singing in situations other than the place,
It has the characteristic. With these features, singing situations are virtually generated, and singing situations are produced in a large hall. The device can be realized. In addition, by making a video react to an audio input, a video device that responds to a voice for the purpose of enhancing the sense of presence and immersion in the entertainment and simulation fields can be realized.

In the above embodiment, only the singing ability of the user existing in the presentation space created by the two-dimensional or more video and the sound effect is determined, and the result of the determination is reflected in the video or the sound. However, the present invention is not limited to this. For example, singing including the choreography timing is performed by photographing the actor, performing image recognition processing, synchronizing with the singing, and performing recognition processing of the choreography timing, body action, and the like. , Singing ability including body action, acting performance such as singing and the like are determined under predetermined conditions, and the level determination result is reflected on at least the video or audio in an expression form embodied so that the determination level can be analogized. be able to.

[0045]

As described above in detail, according to the present invention, the singing ability is reflected on the surrounding environmental equipment (image equipment, audio equipment, etc.), and an appropriate image is generated and provided in response to the singing. Thus, it is possible to provide a singing sound device that can enhance the sense of presence and immersion during singing, has an effect of immersing in singing, and has an effect of falling into a euphoric state.

Further, according to the present invention, a user's singing, singing, and other acting actions existing in a presentation space created by two-dimensional or more video and sound effects are determined under predetermined conditions, and the level is determined. By using a performance expression determination method that reflects at least the video or sound in an expression form embodied so that the determination result can be analogized to the determination level,
Singing by reflecting the performance of singing, singing, etc. on the surrounding environmental equipment (video equipment, sound equipment, etc.) and generating and providing appropriate video and audio in response to singing, singing, etc. In addition, it is possible to enhance the sense of presence and immersion when performing a performance such as a singing dance, and the determination content can be immediately specified to the performer and the surrounding viewers.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a basic configuration of an entire apparatus according to the present invention.

FIG. 2 is a block diagram showing a configuration of a karaoke facility according to the embodiment of the present invention.

FIG. 3 is a functional explanatory diagram of a karaoke facility that provides an immersive feeling (effect of immersing in singing) in the embodiment.

FIG. 4 is a functional explanatory diagram of a karaoke facility that provides a sense of presence (effect of falling into a euphoric state) in the embodiment.

[Explanation of symbols]

 Reference numeral 21: microphone device (microphone), 22: singing ability scoring device (command calculation device), 23: video / audio generation device, 24: monitor device, 25: speaker device, 26: karaoke basic function device.

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Koichi Morimura 1-1-1, Wadazakicho, Hyogo-ku, Kobe-shi, Hyogo F-term in Mitsubishi Heavy Industries, Ltd. Kobe Shipyard 5D108 BE06 BF02 BF05

Claims (3)

[Claims] 1. A singing sound apparatus having a microphone for picking up sound, an accompaniment device for reproducing accompaniment, and a loudspeaker for vocalizing the sound and accompaniment, wherein an image display means for displaying an image in accordance with the accompaniment; A singing sound apparatus comprising: singing ability scoring means for scoring singing by the accompaniment and voice; and auxiliary video means for reflecting the scoring on the video. 2. A singing sound apparatus comprising a microphone for picking up a voice, an accompaniment apparatus for reproducing an accompaniment, and a loudspeaker for vocalizing the voice and the accompaniment, wherein an image display means for displaying an image corresponding to the accompaniment; A singing sound apparatus comprising: singing ability scoring means for scoring singing by the accompaniment and voice; and auxiliary sound means for reflecting the scoring on at least voice or accompaniment. 3. A level decision is made under a predetermined condition for a user's acting action present in a presentation space created by two-dimensional or more video and sound effects, and the level decision result can be analogized to the decision level. A method for judging the acting expression, characterized in that at least the image or the sound is reflected in the expression form embodied in the above.