JP3116937B2 - Karaoke equipment - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a karaoke apparatus which can correct a deviation in pitch (frequency) of a singing voice signal, and can add a harmony voice to the corrected singing voice signal. It relates to a karaoke device.

[0002]

2. Description of the Related Art In a karaoke apparatus, harmony (for example, a melody three times higher than a melody of a singer) is applied to a singer's singing voice in order to excite the singing or to make the singing sound better. A device having a function of adding and outputting is added. As a harmony addition function, a harmony voice signal is generally formed by pitch-shifting a singer's voice signal input from a microphone in order to generate a harmony voice in accordance with the singer's singing tempo.

[0003]

However, in a karaoke apparatus having such a harmony sound signal adding function,
Since the singer's singing voice is pitch-shifted as it is, if the singer's pitch is out of sync, the harmony voice will also be out of sync with the singer, but the pitch will be out of sync. Cannot excite. It is also conceivable to add a harmony voice signal ignoring the singer's singing.However, it is not possible to match the harmony to the main melody singing. There was a drawback that could not be raised more.

[0004] It is an object of the present invention to provide a karaoke apparatus in which accompaniment and harmony are well matched to the singing voice by correcting the singing voice signal of the singer.

[0005]

The present invention relates to a karaoke apparatus having storage means (17), control means (10), sound source means (18), and sound processing means (30), wherein the storage means (17) Stores a plurality of pieces of music information, and the music information includes a music track, which is sequence data of music information,
The control means (10) reads out the tone information from the storage means (17) based on the tempo clock, and comprises a tone generator means (18). The harmony track is sequence data of harmony data and a phoneme track is sequence data of phoneme information. The sound source means (18) generates a musical tone based on the input musical sound information, and the voice processing means (30) detects a syllable segment from the input singing voice signal, and detects the detected syllable. The phoneme pointer is incremented based on the punctuation of the phoneme, and a harmony voice signal is formed based on the phoneme information and the harmony data specified by the phoneme pointer. The phoneme pointer is advanced on the basis of this.

The karaoke apparatus of the present invention detects a syllable segment from a singing voice signal input by a singer, and advances a phoneme pointer in synchronization with the syllable breaking, thereby converting a harmony voice signal synchronized with the singing voice signal. Form. However, a singer's voice signal may not be input, for example, when the singer forgets to sing.In this case, a harmony voice signal is not formed. A harmony sound signal is formed even when no sound signal is input.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A karaoke apparatus according to an embodiment of the present invention will be described with reference to the drawings. This karaoke apparatus has a communication function, a singing correction function, and a harmony adding function. A sound source karaoke device is a karaoke device that generates a karaoke performance sound by driving the sound source device with music data. The music data is sequence data composed of a plurality of tracks, such as a performance data string that specifies a pitch and a sound generation timing. The communication function is a function that is connected to the host station via a communication line, downloads music data from the host station, and stores the music data in the hard disk device 17 (see FIG. 1). The hard disk device 17 can store music data for hundreds to thousands of songs. The singing correction function is a function of correcting a singing pitch, that is, a singing voice signal input from a microphone by a singer when the singing voice signal deviates from a correct and preferable frequency. The harmony addition function is to generate an audio signal having a predetermined pitch, such as third or fifth, with respect to the singing voice of the singer as the voice of the original singer, and to output this as a harmony voice signal.

First, the configuration of music data used for karaoke performance in the karaoke apparatus will be described with reference to FIGS. FIG. 3 is a diagram showing a configuration of music data. FIGS. 4 and 5 are diagrams showing a detailed configuration of music data. FIG. 6 is a diagram illustrating the configuration of phoneme data included in the music data.

In FIG. 3, one piece of music data includes a header, a musical tone track, a main melody track, a harmony track, a lyrics track, a voice track, an effect track, a phoneme track, and a voice data section.

The header is a portion in which various data relating to the music data are written, and data such as a music title, a genre, a release date, and a music playing time (length) are written therein. The CPU 10 (see FIG. 1: the same applies hereinafter) determines a background image to be displayed on the monitor 26 based on the genre data when the main sequence program is executed, and transmits a chapter number of the image to the LD changer 24. The method of determining the background image is to select an image of a snowy country in the case of enka on the theme of winter, and to select an image of a foreign country in the case of pops.

As shown in FIGS. 4 and 5, each track from the musical tone track to the phoneme track is composed of sequence data including a plurality of event data and duration data Δt indicating a time interval between the event data. CP
U10 executes a sequence program during karaoke performance. The sequence program is a program that counts Δt at a predetermined tempo clock that determines the tempo of the song, and when Δt is counted up, determines that it is the readout timing of the event data following this, and reads out this and outputs it to a predetermined processing unit. It is.

[0012] In the musical sound track, tracks of various parts including a melody track and a rhythm track are formed. The CPU 10 outputs to the tone generator 18 the event data read by the tone sequence program. The sound source device 18 selects a sounding channel based on the channel designation data included in the event data, and executes the event for the sounding channel.

In the main melody track, the main melody of the karaoke song, that is, the sequence data of the melody to be sung by the singer is written. This data is input from the CPU 10 to the DSP 30 for audio processing. DSP 30 for audio processing
Corrects the singer's voice signal based on this data.
The configuration of the harmony track is the same as that of the main melody track, and the sequence data of the harmony melody of this karaoke song is written. This data is also stored in the CPU 10
Is input to the DSP 30 for audio processing. D for audio processing
The SP 30 determines the frequency (pitch) of the harmony audio signal based on this data.

The lyrics track is a track that stores sequence data for displaying lyrics on the monitor 26. Although this sequence data is not tone data, this track is also described in the MIDI data format in order to unify the implementation and facilitate the work process. The data type is system exclusive
It is a message. In the data description of the lyrics track, usually, one line of lyrics is treated as one piece of lyrics display data. The lyrics display data is composed of character data (character codes and display coordinates of the characters) of one line of lyrics, display time of the lyrics (usually around 30 seconds), and wipe sequence data. The wipe sequence data is sequence data for changing the display color of the lyrics in accordance with the progress of the song. The timing of changing the display color (the time from when the lyrics are displayed) and the change position (coordinates) ) Is data sequentially recorded over the length of one line.

The audio track is a sequence track for specifying the generation timing of the audio data n (n = 1, 2, 3,...) Stored in the audio data section. The voice data section stores human voices such as back chorus and harmony singing that are difficult to synthesize by the sound source device 18.
The audio track is written with duration data Δt for designating the timing at which the voice-designated data and the voice-designated data are read, that is, the timing at which the voice data is output to the voice data processing unit 19 and the voice signal is formed. The voice designation data includes a voice data number, pitch data, and volume data. The audio data number is an identification number n of each audio data recorded in the audio data section. The pitch data and the volume data are data indicating the pitch and volume of the audio data to be formed. In other words, a back chorus without words, such as "Ah" or "Wawa Wawa", can be used many times by changing the pitch or volume. The pitch and volume are shifted based on and used repeatedly. The audio data processing unit 19 sets the output level based on the volume data, and sets the interval of the audio signal by changing the reading interval of the audio data based on the interval data.

In the effect track, DSP control data for controlling the effect DSP 20 is written. The effect DSP 20 applies reverberation or other reverberation effects to signals input from the sound source device 18, the audio data processing unit 19, and the audio processing DSP 30. DSP
The control data includes data designating the kind of the effect and data of a change amount thereof.

On the other hand, the phoneme track includes phoneme data s1, s2,... Shown in FIG. 6 and duration data e1, e2,.
Are written alternately in chronological order (in the order of progress of the music). The phoneme data is data for each syllable obtained by analyzing the singing of the original singer, and is composed of consonant data and vowel data. In FIG. 6, the lyrics “Akashiya no” consist of five syllables “a”, “ka”, “shi”, “ya”, and “no”, and the phoneme data s1 is obtained by analyzing “a”. Therefore, consonant data (c1) does not exist, and vowel data v
There is only one. Also, since the phoneme data s2 is a syllable including a consonant and a vowel, the consonant data c2 and the vowel data v2
Consists of Since the consonant data has a short non-periodic waveform, the sampling data is stored as it is. The vowel data is obtained by extracting the characteristics of the "a" sound which is the vowel of the "ka" sound. Sample data obtained by encoding one to several waveforms of the vowel, average amplitude (amplitude), vibrato frequency, These are the vibrato depth (amplitude) and additional sound noise characteristics. Here, the additional sound noise characteristic is data indicating characteristics of aperiodic noise mixed with vowels. Hereinafter, s3, s4, and ‥ are data having the same configuration.

FIG. 1 is a block diagram of the karaoke apparatus. The CPU 10 for controlling the operation of the entire apparatus includes a ROM 11, a RAM 12, a hard disk storage (HDD) 17, an ISDN controller 16, a remote control receiver 13, a display panel 14, a panel switch 15,
Sound source device 18, audio data processing unit 19, effect DSP2
0, a character display unit 23, an LD changer 24, a display control unit 25, and a voice processing DSP 30.

The ROM 11 stores a system program, an application program, a loader, and font data. The system program is a program that controls the basic operation of the device and data transmission / reception with peripheral devices. The application program is a peripheral device control program, a sequence program, or the like. During a karaoke performance, a sequence program is executed by the CPU 10 to generate musical tones and reproduce images based on music data. The loader is a program for downloading music data from a host station. The font data is for displaying lyrics, song titles, and the like, and stores fonts of a plurality of character types, such as Mincho and Gothic. In the RAM 12, a work area is set. A music data file is set in the HDD 17.

The ISDN controller 16 is a controller for communicating with a host station via an ISDN line. Music data and the like are downloaded from the host station. The ISDN controller 16 has a built-in DMA circuit, and writes downloaded music data and application programs directly to the HDD 17 without passing through the CPU 10.

The remote control receiver 13 receives the infrared signal sent from the remote control 31 and restores the data. The remote controller 31 includes a command switch such as a music selection switch, a numeric key switch, and the like. When the user operates these switches, an infrared signal modulated with a code corresponding to the operation is transmitted. The display panel 14 is provided on the front of the karaoke apparatus, and displays the currently playing music code and the number of reserved music. The panel switch 15 is provided on a front operation unit of the karaoke apparatus, and includes a music code input switch, a key change switch, and the like.

The tone generator 18 forms a tone signal based on music data. The audio data processing unit 19 forms an audio signal having a specified length and a specified pitch based on audio data that is ADPCM data included in the music data. The audio data is obtained by directly digitizing and storing a signal waveform, such as a back chorus or a model singing sound, which is hardly generated electronically by the sound source device 18.

On the other hand, the singing voice signal input from the singing microphone 27 is input to the voice processing DSP 30 via the preamplifier 28 and the A / D converter 29, and the main melody information, harmony information and phonemes are also transmitted from the CPU 10. Data and the like are input. The voice processing DSP 30 corrects the frequency of the singing voice signal of the singer based on the information, and synthesizes a harmony voice signal of the original singer's tone with the corrected singing voice signal.
These audio signals are output to the effect DSP 20.

The tone signal formed by the sound source device 18, the sound signal formed by the sound data processing section 19, and the sound processing D
The singing voice signal and the harmony voice signal input from the SP 30 are input to the effect DSP 20. DSP2 for effects
A value of 0 gives effects such as reverb and echo to these input audio signals and tone signals. DSP2 for effects
The type and degree of the effect given by 0 are controlled based on the DSP control data included in the music data. The DSP control data is input to the effect DSP 20 at a predetermined timing by the CPU 10 based on the DSP control sequence program. The tone signal and audio signal to which the effect has been added are converted into analog signals by the D / A converter 21 and then output to the amplifier / speaker 22. The amplifier / speaker 22 amplifies this signal and emits sound.

The character display unit 23 generates character patterns such as song titles and lyrics based on the input character data. The LD changer 24 reproduces the background video of the corresponding LD based on the input video selection data (chapter number). The video selection data is determined based on the genre data of the karaoke song or the like. At the start of the karaoke performance, the CPU 10 reads the genre data of the music from the header of the music data. The CPU 10 determines which background video is to be reproduced based on the genre data and the content of the background video, and stores the video selection data in the LD changer 2.
4 is output. The LD changer 24 has 5 sheets (120
Scene) and a laser disk of about 120
The background video of the scene can be reproduced. One background image is selected from among them according to the image selection data,
Output as video data. The character pattern and the video data are input to the display control unit 25. The display control unit 25 superimposes these data in superimposition and displays them on the monitor 26.

FIG. 2 is a diagram for explaining the operation of the DSP 30 for audio processing. The voice processing DSP 30 executes various processing operations on an input signal based on a built-in microprogram, and each processing operation is represented by blocks as shown in FIG.

The singing voice signal input from the microphone 27 is amplified by the amplifier 28 and converted into a digital signal by the A / D converter 29, and then the syllable detecting unit 40, the pitch detecting unit 41 and the pitch correcting unit of the voice processing DSP 30. 4
2 is input. The syllable detecting unit 40 sequentially detects the level of the singing voice signal in real time, and detects a syllable break based on a level break or the appearance of a consonant. Since the consonant is a non-periodic waveform like white noise, it can be distinguished from a vowel having a periodic waveform. The level change waveform is output to the envelope assigning section 46, and the syllable delimiter signal is output to the phoneme pointer generating section 43.

The pitch detecting section 41 sequentially detects the frequency of the input singing voice signal in real time, and outputs the frequency information (frequency change waveform) to the pitch correcting section 42. The main melody information in addition to the frequency information and the singing voice signal is input to the pitch correction unit 42. The pitch correction unit 42 compares the input frequency information with the main melody information, and shifts the singing voice signal by an amount corresponding to the difference, thereby correcting the singing voice signal to match the main melody. The corrected singing voice signal is output to the adding section 47.

Here, the main melody information may be data in which the pitch is stored in note units, and the pitch change of the model singer (original singer) is written in chronological order, that is, the moderation is kept as it is. It may be written data.

FIG. 7 is a graph showing pitch correction in the case of using information in which the change of the singing pitch of the model singer is written in time series as the main melody information. The thin line in the figure shows the change in the singing pitch of the model singer. The singing pitch of the karaoke singer also changes substantially following, but there is a slight error in the pitch. Therefore, by generating a pitch correction value as shown in the lower part of the figure and adding this to the frequency of the singing voice signal, the singing pitch of the karaoke singer matches the singing pitch of the model singer.

It should be noted that, besides the method of complete correction as shown in the figure, by correcting about half the difference between the singing pitch of the karaoke singer and the singing pitch of the model singer, the moderation of the karaoke singer is reduced to some extent. You can make use of it.

On the other hand, at the start of the karaoke performance, the contents of the phoneme tracks in the music data are
Input to 0. Of these, the phoneme data is input to the phoneme data register 44, and the duration data is input to the pointer generator 43. The pointer generation unit 43 generates a phoneme pointer that specifies one phoneme data in the phoneme data register 44 in synchronization with the progress of the music. The phoneme pointer is normally advanced based on a syllable delimiter signal input from the syllable detection unit 40, but is a tempo clock when a singing voice signal is not input at all or when a syllable cannot be delimited. The increment is performed by counting the duration data according to the beat information. The phoneme data specified by the phoneme pointer is output from the phoneme data register 44 to the harmony forming unit 45. Harmony forming part 45
In addition, the harmony data is input from the CPU 10, and the harmony forming unit 45 forms a harmony sound signal of the pitch of the harmony data with the waveform of the input phoneme data. The formed harmony audio signal is output to the envelope providing unit 46. The envelope adding section 46 multiplies the harmony sound signal input from the harmony forming section 45 by the level change waveform input from the syllable detecting section 40 to add an envelope. A harmony voice signal singing to a karaoke singer can be formed by advancing phoneme data by syllable detection and adding this envelope. The harmony sound signal to which the envelope has been added is output to the adding unit 47.

The adder 47 adds the singing voice signal and the harmony voice signal whose pitch has been corrected. This signal is output to the effect DSP 20.

By the above-described operation of the voice processing DSP 30, the singing voice signal of the singer can be correctly matched, and the harmony voice signal matched with the singing of the singer can be formed by the voice of the original singer. .

In this embodiment, the syllable data extracted from the singing of the original singer is sequentially stored as phoneme data, but the phoneme data is not limited to this. That is, the pronunciation of the general 50 sounds may be stored, the singing voice of the singer may be recognized, and the corresponding phoneme may be determined and used.

At the start of the karaoke performance, only the phoneme tracks are taken into the DSP 30 for voice processing, and the main melody data and the harmony data are input to the DSP 30 in a sequence by the CPU 10. However, at the start of the karaoke, the phoneme tracks, the main melody tracks and the harmony tracks are obtained. May be taken into the DSP 30, and the pitch may be corrected in accordance with the progress of the singer's singing. Conversely, the phoneme data may also be input to the DSP 30 in a sequence by the CPU 10, so that the harmony voice signal may be formed in accordance with the progress of the karaoke song regardless of the progress of the singer's singing.

[0037]

As described above, according to the karaoke apparatus of the present invention, in synchronization with the singing voice signal input by the singer,
A harmony voice signal synthesized based on the phoneme information can be added, and harmony of a voice different from the voice of the singer can be added. In this case, even if the singer forgets to sing, the harmony voice signal is synthesized based on the tempo clock, so that both the main melody and the harmony voice do not disappear.

[Brief description of the drawings]

FIG. 1 is a block diagram of a voice conversion karaoke apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of a voice processing DSP of the voice conversion karaoke apparatus.

FIG. 3 is a diagram showing a configuration of music data used in the voice conversion karaoke apparatus.

FIG. 4 is a diagram showing a configuration of music data used in the voice conversion karaoke apparatus.

FIG. 5 is a diagram showing a configuration of music data used in the voice conversion karaoke apparatus.

FIG. 6 is a diagram showing a configuration of phoneme data included in the music data.

FIG. 7 is a diagram showing an example of main melody data and pitch correction included in the music piece data;

[Explanation of symbols]

30-DSP for voice processing, 41-Pitch detector, 42-
Pitch correction part, 45-harmony formation part.

Continuation of the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) G10L 11/00-13/08 G10L 19/00-21/06 G10K 15/00-15/12 G10H 1/00-7 / 12

Claims (1)

(57) [Claims] 1. A storage means (17), a control means (10),
A karaoke apparatus having a sound source means (18) and a voice processing means (30), wherein the storage means (17) stores a plurality of pieces of music information, and the music information is a musical sound track, a harmony, which is sequence data of musical sound information. The control means (10) reads out the tone information from the storage means (17) based on the tempo clock, and generates a sound source means (18). The harmony track is data sequence data and the phoneme track is phoneme information sequence data.
The sound source means (18) generates a musical tone based on the input musical sound information, and the voice processing means (30) detects a syllable segment from the input singing voice signal, and detects the detected syllable. The phoneme pointer is incremented based on the punctuation of the phoneme, and a harmony voice signal is formed based on the phoneme information and the harmony data specified by the phoneme pointer. A karaoke device that advances phoneme pointers based on the