JP2016191846A - Karaoke system responding to delayed emission of singing voice - Google Patents

Abstract

PROBLEM TO BE SOLVED: To emit substituting reproduction sound according to emission timing of karaoke performance sound, in response to delayed emission of a singing voice, and thereby enable a singer to sing without feeling discomfort.SOLUTION: A karaoke system comprises: vowel emission timing detection means 107 that detects a vowel emission timing at which a vowel of a singing voice inputted through a microphone 30 is emitted from a speaker 40; substituting reproduction sound generation means 108 that generates substituting reproduction sound to be emitted from the speaker 40 on the basis of reference data of a karaoke song; and delayed singing voice supplementation means 109 that outputs the substituting reproduction sound from a start timing of singing voice emission based on the reference data until an actual vowel emission timing, and that outputs the substituting reproduction sound and an adjusted singing voice using the singing voice inputted through the microphone 30, from the actual vowel emission timing until an end timing of singing voice output based on the reference data.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a karaoke system corresponding to a singing voice sound emission delay, and more specifically, a difference between a karaoke performance sound sounding timing and a singing voice sounding timing corresponding to a singing voice sounding delay. It is related with the karaoke performance apparatus which can eliminate the discomfort of the singer by.

  In the karaoke performance device, the user plays the music selected by the user, displays the lyric characters on the display screen of the display device according to the performance, and performs the color change display of the lyric characters to notify the singing timing. It has become. In such a conventional karaoke performance device, a commercially available digital television is used as a display device for displaying lyric characters, and a commercially available USB-connected microphone may be used as a microphone for inputting a singer's singing voice. .

  The karaoke performance device configured as described above is based on the electrical compatibility with digital TVs and USB-connected microphones, and the electrical characteristics of each peripheral device. In some cases, the singing voice is not synchronized, a delay occurs in the output of the singing voice, or the color change timing of the music performance sound and the lyrics telop is shifted. Various techniques have been proposed for such inconveniences related to output delay and the like (see, for example, Patent Document 1 and Patent Document 2).

  The technique described in Patent Document 1 uses a camera connected to a karaoke performance device to photograph the color change timing of the lyrics telop displayed on the monitor, and to the color change timing at the time of shooting and the karaoke performance device. The delay time is calculated from the color change timing executed. Then, by delaying the output timing of the music performance sound or conversely increasing the color change timing of the lyrics telop, errors in the performance timing and the color change timing of the lyrics telop are suppressed.

  In the technique described in Patent Document 2, a guide melody output from a speaker of a digital television is collected by a singing microphone and input to a karaoke performance device, and a delay value is acquired from the input guide melody. The display timing of the karaoke video and the generation timing of the karaoke performance sound are synchronized based on the acquired delay value.

JP 2013-29744 A JP 2010-276949 A

  However, since the technique described in Patent Document 1 synchronizes the music performance sound of the reserved music and the lyrics telop displayed on the display device, the delay of the singing voice output from the USB-connected microphone or the like No measures have been taken for. Therefore, the inconvenience that the singer feels the difference between the color change timing of the played music performance sound and the displayed lyrics telop and his / her singing voice has not been solved.

  In addition, since the technique described in Patent Document 2 is a technique for solving the delay in the color change timing of the music performance sound and the lyrics telop, the output delay of the singing voice cannot be solved, and the singing voice shifts. Cannot be corrected.

  The present invention has been proposed in view of the above-described circumstances, and eliminates the uncomfortable feeling of the singer due to the difference between the sound output timing of the karaoke performance sound and the sound output timing of the singing sound, corresponding to the sound output delay of the singing sound. An object is to provide a karaoke system that can be used.

  The karaoke system corresponding to the sound output delay of the singing voice according to the present invention has the following features in order to achieve the above-described object. That is, in the karaoke system corresponding to the sound output delay of the singing sound of the present invention, the karaoke singing sound input from the microphone in accordance with the karaoke performance sound of the karaoke music emitted from the speaker is the karaoke performance sound. On the other hand, in a karaoke performance device that emits sound with a predetermined range of delay time, it is characterized by comprising vowel sound emission timing detection means, proxy reproduction sound generation means, and delayed singing sound supplement means. is there.

  The vowel sound emission timing detection means is means for detecting the vowel sound emission timing at which the vowel of the singing voice input from the microphone is emitted from the speaker. The proxy reproduction sound generating means is means for generating proxy reproduction sound that is emitted from the speaker in response to the sound output delay of the singing sound. The delayed singing voice supplementing means outputs the proxy playback voice from the timing of starting the singing voice based on the reference data of the karaoke song to be played to the vowel sounding timing detected by the vowel sounding timing detecting means. In the period from the vowel sound output timing detected by the vowel sound output timing detection means to the output end timing of the singing sound based on the reference data, the adjusted singing sound using the proxy playback sound and the singing sound input from the microphone is output. Means.

  In the karaoke system corresponding to the sound output delay of the singing voice having the above-described configuration, the substitute reproduction voice can be a synthetic voice corresponding to the attribute information of the singer, and the singing is performed. It is possible to use the singing voice of the singer stored when the person sang the same karaoke song in the past.

According to the karaoke system corresponding to the sound output delay of the singing sound according to the present invention, when the singer performs the singing in accordance with the karaoke performance sound, the sound output timing of the karaoke performance sound and the singing sound input from the microphone Guide vocals, singing voices sung by the singer in the past, and the like can be output as proxy playback voices in a delay time caused by a deviation from the sound emission timing.

  Therefore, the singer can sing without a sense of incongruity according to the timing of the color change of the music performance sound output from the speaker or the lyrics telop displayed on the display device, as there is no delay. Become.

  Moreover, by using the synthesized voice corresponding to the singer's attribute information or the singing voice sound of the singer stored when the singer sang the same karaoke song in the past as the proxy playback voice, The proxy playback sound can be emitted without a sense of incongruity.

The block diagram which shows schematic structure of the karaoke system corresponding to the sound emission delay of the song voice which concerns on embodiment of this invention. Explanatory drawing for demonstrating sound emission of the proxy reproduction | regeneration audio | voice in delay time.

  Hereinafter, an embodiment of a karaoke system (hereinafter abbreviated as a karaoke system) corresponding to the sound output delay of a singing voice according to the present invention will be described with reference to the drawings. 1 and 2 illustrate a karaoke system according to an embodiment of the present invention. FIG. 1 is a block diagram showing a schematic configuration of the karaoke system. FIG. It is explanatory drawing of.

<Outline of Karaoke System>
As shown in FIG. 1, a karaoke system (hereinafter, abbreviated as karaoke system 100) corresponding to a sound release delay of singing voice according to an embodiment of the present invention is a karaoke performance sound of karaoke music emitted from a speaker 40. Is applied to the karaoke performance device 10 in which the singing voice of the karaoke song input from the microphone 30 is emitted with a delay time in a predetermined range with respect to the karaoke performance sound. In addition, the microphone 30 and the speaker 40 that are incidental thereto are the main components.

  Note that the karaoke performance device 10 can be configured by a personal computer or a game device in which dedicated software is installed, and the karaoke remote control device 20 can be configured by a personal portable terminal in which a dedicated application is installed. In personal computers and game machines, a USB microphone, which will be described later, is often used, which causes a delay in the release of singing voice.

  In the following description, a program is stored in a RAM or the like and executed by hardware such as a CPU, so that not only software that exhibits its function but also a logic circuit that can exhibit an equivalent function. It is a concept that includes.

<Karaoke performance device and incidental equipment>
As shown in FIG. 1, a karaoke remote control device 20, a microphone 30, a speaker 40, a display device 50, and a mixing amplifier 60 are attached to a karaoke performance device 10 to which a karaoke system 100 according to an embodiment of the present invention is applied. . The karaoke performance device 10 and the karaoke remote control device 20 are connected to each other via a router 70 and a LAN line 80. Although not shown, the karaoke performance device 10 and the karaoke remote control device 20 may be network-connected to a server that comprehensively manages them.

<Karaoke remote control device>
The karaoke remote control device 20 has a user interface function, and transmits / receives data to / from the karaoke performance device 10 via the router 70 and the LAN line 80. The karaoke remote control device 20 includes a program that functions as a music search means 20a, a music index database 20b, a data storage unit 20c that stores various data, and an input / output display unit 20d that inputs and outputs data.

  By operating switches attached to the karaoke remote control device 20 and various icons displayed on the input / output display unit 20d, etc., music selection operation, volume adjustment, tempo adjustment, key adjustment, reference to the music selection reservation history, etc. Processing is performed.

<Microphone / Speaker>
The microphone 30 is a device for inputting singing voice. The singing voice signal input from the microphone 30 is input to the mixing amplifier 60 after the karaoke performance device 10 detects the vowel sound output timing, generates the proxy reproduction voice, and supplements the delayed singing voice. It is mixed with the karaoke performance signal sent from the reproduction control means 110, amplified, and outputted to the speaker 40. Moreover, the audio | voice input signal from the microphone 30 can be used for calculation of a singing score value, or recording of a singing voice.

  As the microphone 30 of the present embodiment, for example, a USB microphone connected to the karaoke performance apparatus 10 by USB can be used. In this case, if no measures are taken, the sound emission timing of the music performance sound and the display timing of the lyrics telop and the sound emission timing of the singing voice of the singer input from the microphone 30 are synchronized according to the electrical characteristics of each device. Without delaying, the singing voice is emitted. Therefore, in the present invention, as will be described later, there is a means for eliminating the discomfort of the singer caused by the delay.

<Display device>
The display device 50 is a device for displaying a background video, lyrics telop, and the like related to karaoke music, and is configured by a liquid crystal display, for example.

<Karaoke performance device>
The karaoke performance apparatus 10 includes a central control unit 101, a ROM 102, a RAM 103, an HDD 104, a communication control unit 105, a reserved music management unit 106, a vowel sound emission timing detection unit 107, a proxy reproduction voice generation unit 108, a delayed singing voice supplement unit 109, A music reproduction control unit 110, a singing voice input unit 111, and a video reproduction control unit 112 are provided.

<Central control means>
The central control means 101 is an electronic device for comprehensively controlling the karaoke performance device 10, and is composed of, for example, a CPU and its peripheral devices, and the CPU or the like operates according to a program stored in the ROM 102 or the like. The control function can be demonstrated.

<ROM / RAM>
The ROM 102 is an electronic device for storing program data and numerical data for controlling each device constituting the karaoke performance apparatus 10. The RAM 103 is an electronic device that functions as a temporary storage area for temporarily storing programs and various data. In the present embodiment, the reserved music management table 103 a is temporarily stored in the RAM 103.

<HDD>
The HDD 104 functions as a mass storage device, and stores a music database 104a and a background video database 104b. Note that a large-capacity storage unit such as a DVD that can rewrite data may be used instead of the HDD 104 or together with the HDD 104.

<Music database / background video database>
The music database 104a is a database configured by associating music performance data (MIDI standard data) and lyrics rendering data in synchronization with music IDs. The performance control data is digital data for controlling the performance of each song, and the lyric rendering data is the display timing data and color change data of lyric characters synchronized with the performance, guide vocal data (details will be described later), the main melody of the performance song This includes reference data that is used as a basis for singing.

<Communication control means>
The communication control unit 105 includes an electronic device and a program for controlling transmission and reception of data and signals with the karaoke remote control device 20. The function of the communication control means 105 performs synchronization and error control of data and signals transmitted to and received from the karaoke remote control device 20.

<Reserved music management means / reserved music management table>
The reserved music management means 106 transmits the reserved music information determined by using the karaoke remote control device 20 to the karaoke performance apparatus 10 when an arbitrary user makes a reservation for the karaoke music apparatus 10. The user ID of the user and the music ID of the reserved music are associated with each other and managed by the reserved music management table 103a. That is, the reserved music management means 106 creates a reserved music management table 103 a and stores this reserved music management table 103 a in the RAM 103 for management. The reserved music management table 103a is a data table in which music IDs and user IDs of music pieces selected by the user using the function of the music search means 20a are associated with each other and arranged in order of performance.

<Vowel emission timing detection means>
The vowel sound emission timing detection means 107 includes a program for detecting the vowel sound emission timing at which the vowel of each syllable of the singing voice input from the microphone 30 is emitted from the speaker 40. That is, the singing voice signal input from the microphone 30 is delayed due to, for example, the specifications of the USB driver in the singing voice input unit 111, and there is a difference between the timing input to the microphone 30 and the timing emitted from the speaker 40. Arise.

  That is, in this embodiment, the singing voice signal input from the microphone 30 is input to the karaoke performance device 10 via the singing voice input unit 111. Then, a proxy playback sound is generated based on the vowel sound output timing detected by the vowel sound output timing detection means 107, and the proxy playback sound and the singing sound input from the microphone 30 are passed through the delayed singing sound supplement means 109. And input to the mixing amplifier 60. The mixing amplifier 60 receives the musical sound signal output from the music reproduction control unit 110, mixes the singing voice and the musical sound signal that have been subjected to the delay countermeasure process, and emits the sound from the speaker 40.

  The delay countermeasure processing related to the singing voice input from the microphone 30 may be performed before being input to the mixing amplifier 60. In this embodiment, detection of the vowel sound emission timing by the vowel sound emission timing detection means 107 is performed. Is not the timing at which the singing voice is actually output from the speaker 40 but the timing at which the singing voice input from the microphone 30 is input to the karaoke performance device 10 and output from the singing voice input unit 111 to the bus. .

  By the way, most of the syllables of the singing voice are composed of vowels alone or consonant plus vowels. For example, in the example shown in FIG. 2, the vowel “I” is pronounced after the consonant “K”, and “ki” The singing voice (syllable). The consonant “K” is a noise having no harmonic structure, so that it is difficult to detect the pitch, and the vowel “I” having the harmonic structure can be detected at the sounding portion. Therefore, in the present invention, the timing at which the vowel part is emitted from the speaker 40 is detected, and at this timing, switching from the later-described proxy playback voice to the adjusted singing voice is started. In FIG. 2, the delay time (the time difference between the timing at which the singer utters the singing voice to the microphone 30 and the timing at which the singing voice is actually emitted from the speaker 40) is about 80 msec.

  In the present invention, the vowel sound emission timing detection means 107 detects the vowel sound emission timing, so that the actual singing voice utterance timing (the sound emission timing from the speaker 40 when there is no delay) is actually used for the speaker 40. To the timing at which the vowel is emitted is set as the sound emission section of the proxy reproduction voice (described later). In addition, the utterance timing of the original singing voice (sound emission timing from the speaker 40 when there is no delay) can be detected based on the reference data, and the timing is “ki” illustrated in FIG. The start timing of the pronunciation of each syllable constituting the lyrics, the timing at which the pitch is changed during the syllable pronunciation, and the like.

<Proxy playback audio generation means>
The proxy reproduction sound generation means 108 includes an electronic device and a program for generating proxy reproduction sound to be emitted from the speaker 40 in response to the sound emission delay of the singing sound. That is, in the proxy reproduction sound generation means 108, the karaoke singing sound input from the microphone 30 in accordance with the karaoke performance sound has a predetermined range of delay time with respect to the karaoke performance sound, for example, the singer feels uncomfortable with the delay. Assuming the case where the substitute reproduction sound is emitted with a delay time of about 120 msec, which is the upper limit that the unnatural sound cannot be heard unnaturally, the silence state in the delay time is supplemented and emitted. Generate proxy playback audio.

  As the proxy playback voice, for example, a guide vocal prepared for karaoke music can be used. The guide vocal is an exemplary singing voice, and is an auxiliary singing voice that is output as an auxiliary to the karaoke singing when singing karaoke music that the singer is not good at or singing. In the present embodiment, the singing voice of the guide vocal is selected based on the reference data of the karaoke music, and the sound emission timing is determined.

  Moreover, the synthetic | combination audio | voice corresponding to the attribute information of the singer who performs a karaoke song can be used as proxy reproduction | regeneration audio | voice. That is, when a singer logs into the system, data such as gender distinction and age, which are attribute information of the singer, can be acquired. Therefore, when the singing voice of the singer and the proxy playback voice are joined together by generating a proxy playback voice that approximates the singing voice that the singer actually sings based on these attribute information, Discomfort due to the difference of can be eliminated.

  Moreover, the singing voice of the said singer memorize | stored when the singer who performs karaoke singing sings the same karaoke music in the past can be used as a substitute reproduction | regeneration audio | voice. That is, when the singer logs in to the system, the singer can be identified, and when the singer has memorized the singing voice when singing the same karaoke song in the past, the memorized singing voice Is the proxy playback audio. In this case, since the proxy reproduction voice is the singing voice of the singer, even if the proxy reproduction voice is emitted, there is no sense of incongruity.

<Delayed singing voice supplement means>
The delayed singing voice supplementing means 109 receives the proxy reproduced voice from the timing at which the singing voice is emitted based on the reference data of the karaoke music to be played to the vowel sounding timing detected by the vowel sounding timing detecting means 107. The adjusted singing using the proxy playback voice and the singing voice inputted from the microphone 30 from the vowel emitting timing detected by the vowel emitting timing detecting means 107 to the output end timing of the singing voice based on the reference data. It consists of an electronic device and a program for outputting sound.

<Sound of substitute playback sound>
The sound emission start timing of the original singing voice (the sound emission start timing from the speaker 40 when there is no delay) and the output end timing of the singing voice (the sound emission end timing from the speaker 40 when there is no delay) are It can be determined based on reference data of karaoke music. The vowel sound emission timing actually input from the microphone 30 can be detected by the vowel sound emission timing detection means 107.

  As described above, between the sound emission start timing of the original singing voice (the sound emission start timing from the speaker 40 when there is no delay) and the sound emission timing of the singing voice actually input from the microphone 30. There is a delay time. Therefore, if no measures are taken, a difference occurs between the sounding timing of the karaoke performance sound and the sounding timing of the singing voice, and the singer feels uncomfortable. Therefore, in the present invention, by generating the proxy reproduction sound corresponding to this delay time, the uncomfortable feeling of the singer is eliminated.

  That is, any countermeasure is taken between the sound emission start timing of the original singing voice (the sound emission start timing from the speaker 40 when there is no delay) and the timing at which the singing voice actually input from the microphone 30 is emitted. Otherwise, there will be no sound during this time. Therefore, in this section, the substitute reproduction sound is output to cancel the silent state.

  When the timing at which the singing voice actually input from the microphone 30 is emitted from the speaker 40 (detectable vowel emission timing) arrives, the proxy reproduction voice and the singing voice of the singer are synthesized (crossfade). ) And connect to the singer's voice of the singer. Note that the synthesis (crossfade) of the proxy playback voice and the singing voice of the singer is a predetermined time from the timing when the vowel sound is detected (for example, several cycles of the voice waveform, about 5 to 15 msec in length). Conduct during That is, by fading out the proxy reproduction sound during this predetermined time and fading in (crossfading) the singing sound of the singer himself, both can be smoothly connected.

  After that, at the output end timing of the singing voice based on the reference data (sound emission end timing from the speaker 40 when there is no delay), the amplitude of the singing voice is rapidly attenuated, and the syllable sound is released from the singing voice without a sense of incongruity. Terminate. Here, when the syllable of the singing voice is composed of consonants plus vowels (for example, “K” plus “I”), if the sound is emitted from the beginning of the syllable inputted from the microphone 30, the proxy playback voice “KI” is followed. Then, “KI” of the singing voice input from the microphone 30 is emitted, and “KIKI” is emitted, which makes it unnaturally heard. Therefore, the syllable of the singing voice input from the microphone 30 following the proxy reproduction voice “KI” is made to be heard naturally by excluding the consonant and emitting only “vowel” as “KII”.

<Music playback control means>
The music reproduction control means 110 drives a sound source (synthesizer) based on the performance control data extracted from the music data corresponding to the music ID, generates a musical sound signal constituting a karaoke performance signal, and converts it into an analog signal for mixing. 60 is an electronic device for outputting to 60. Further, the mixing amplifier 60 mixes the singing voice signal subjected to the delay countermeasure processing inputted from the delayed singing voice supplementing means 109 and the karaoke performance signal sent from the music reproduction control means 110, and also by an amplifier function. This is an apparatus for amplifying and outputting from the speaker 40.

<Video playback control means>
The video reproduction control means 112 uses the background video data extracted from the background video database 104b and the lyric characters created based on the lyric rendering data included in the music data during the performance of the karaoke music, as the music of the karaoke music. It is an electronic device for outputting to the display device 50 in synchronization with data.

DESCRIPTION OF SYMBOLS 100 Karaoke system 10 Karaoke performance apparatus 20 Karaoke remote control apparatus 20a Music search means 20b Music index database 20c Data storage part 20d Input / output display part 30 Microphone 40 Speaker 50 Display apparatus 60 Mixing amplifier 70 Router 80 LAN line 101 Central control means 102 ROM
103 RAM
103a Reserved music management table 104 HDD
104a Music database 104b Background video database 105 Communication control means 106 Reserved music management means 107 Vowel sound emission timing detection means 108 Proxy reproduction voice generation means 109 Delayed singing voice supplement means 110 Music reproduction control means 111 Singing voice input unit 112 Video reproduction control means

Claims (3)

In a karaoke performance device in which a singing voice of a karaoke song input from a microphone in accordance with a karaoke performance sound of a karaoke piece emitted from a speaker is emitted with a delay time of a predetermined range with respect to the karaoke performance sound ,
A vowel sound emission timing detection means for detecting a vowel sound emission timing at which a vowel of a singing voice input from the microphone is emitted from the speaker;
In response to the sound output delay of the singing sound, proxy playback sound generation means for generating proxy playback sound to be output from a speaker;
The proxy reproduction voice is output from the sound emission start timing of the singing voice based on the reference data of the karaoke music to the vowel sound emission timing, and the output of the singing voice based on the reference data is ended from the vowel sound emission timing. Until the timing, delayed singing voice supplementing means for outputting adjusted singing voice using the proxy playback voice and the singing voice inputted from the microphone,
A karaoke system that supports delays in the release of singing voices.   The karaoke system according to claim 1, wherein the proxy playback voice is a synthesized voice corresponding to the attribute information of the singer.   The said proxy reproduction | regeneration audio | voice is the singing audio | voice of the said singer memorize | stored when the said singer sang the same karaoke music in the past, The sound emission delay of the singing audio | voice of Claim 1 characterized by the above-mentioned Karaoke system.

Images