JP3613859B2 - Karaoke equipment - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an audio signal processing device for adding harmony to a melody audio signal such as singing, and in particular, when a plurality of melody audio signals are input, harmony is applied only to the main melody audio signal. The present invention relates to an audio signal processing apparatus to be added.
[0002]
[Prior art]
In order to enliven the singing of karaoke, what is output by adding a voice of harmony (for example, a melody three times higher than the melody of the singing) to the singing of the singer has been proposed. As the harmony function, one that generates a harmony sound by pitch shifting the singing voice signal is generally used.
[0003]
Also, some karaoke songs are sung in plural (two people) such as so-called duet songs.
[0004]
[Problems to be solved by the invention]
However, in the case of the above-mentioned duet music, two singing voice signals are mixed and input. However, in a conventional karaoke apparatus having a harmony addition function, since all the singing voices are harmonized, there are multiple parts. However, there was a drawback that the singing voice signal of the two people was damaged.
[0005]
An object of the present invention is to provide a karaoke apparatus that can extract harmony by extracting only the main melody from a plurality of singing voice signals.
[0006]
[Means for Solving the Problems]
The invention of claim 1 is a karaoke apparatus comprising music data reproduction means, fundamental frequency information extraction means, main melody selection means, main melody separation means, pitch shift means, and singing analysis means,
The music data reproduction means reproduces the music data and outputs an accompaniment signal, outputs the main melody information included in the music data to the main melody selection means, and outputs the harmony information to the pitch shift means and the singing analysis means,
The fundamental frequency information extracting means extracts fundamental frequency information for each part from the audio signal from the microphone,
The main melody selection means outputs the basic frequency information suitable for the main melody information among the basic frequency information for each part as selection information,
The main melody separation means frequency-analyzes the sound signal from the microphone and separates it into a plurality of part sound signals, and outputs one of the part sound signals as the main melody sound signal based on the selection information from the main melody selection means,
The pitch shift means pitch-shifts the main melody audio signal based on the harmony information and outputs a harmony audio signal.
The singing analysis means analyzes the number of singing parts in the audio signal based on the fundamental frequency information. When the number of singing parts is 0, the first process is performed. When the number of singing parts is 1, the second process is performed. If the number of singing parts is plural and the harmony part corresponding to the harmony information is included, the third process is performed.
The first process pauses the main melody selection means, the main melody separation means, and the pitch shift means,
In the second process, the main melody selection means and the main melody separation means are paused, and a voice signal is input to the pitch shift means instead of the main melody voice signal,
The third process pauses the main melody separation means and the pitch shift means.
It is characterized by that.
The invention of claim 2 is a karaoke apparatus comprising music data reproduction means, fundamental frequency information extraction means, main melody selection means, main melody separation means, pitch shift means, and singing analysis means,
The music data reproduction means reproduces the music data and outputs an accompaniment signal, and outputs the harmony information included in the music data to the pitch shift means and the singing analysis means,
The fundamental frequency information extracting means extracts fundamental frequency information for each part from the audio signal from the microphone,
The main melody selection means outputs the fundamental frequency information having a predetermined frequency height among the fundamental frequency information for each part as selection information,
The main melody separation means frequency-analyzes the sound signal from the microphone and separates it into a plurality of part sound signals, and outputs one of the part sound signals as the main melody sound signal based on the selection information from the main melody selection means,
The pitch shift means pitch-shifts the main melody audio signal based on the harmony information and outputs a harmony audio signal.
The singing analysis means analyzes the number of singing parts in the audio signal based on the fundamental frequency information. When the number of singing parts is 0, the first process is performed. When the number of singing parts is 1, the second process is performed. If the number of singing parts is plural and the harmony part corresponding to the harmony information is included, the third process is performed.
The first process pauses the main melody selection means, the main melody separation means, and the pitch shift means,
In the second process, the main melody selection means and the main melody separation means are paused, and a voice signal is input to the pitch shift means instead of the main melody voice signal,
The third process pauses the main melody separation means and the pitch shift means.
It is characterized by that.
[0007]
The audio signal processing apparatus of the above invention inputs a plurality of melodic audio signals from the audio signal input means. This apparatus is applied to, for example, a karaoke apparatus. In this case, the audio signal input means is a device such as a singing microphone and an amplifier connected to the microphone. The main melody determination means determines the main melody audio signal from the plurality of input audio signals. The determination of the main melody may be based on main melody information stored in advance, and the corresponding one may be determined as the main melody. Also, based on a certain rule, for example, the sound signal of the highest tone is used as the main melody information. You may determine based on rules, such as doing. The sound signal determined as the main melody in this way is extracted from the plurality of input sound signals. If multiple audio signals are input in different systems, the main melodic system can be selected. If multiple singing audio signals are input in one system, the main melodic basics can be selected. The main melody is extracted by a method such as separating and extracting only frequency components corresponding to overtones. The extracted main melody audio signal is pitch-shifted to generate a harmony audio signal. As a pitch shift method, there is a method in which the read clock is simply changed, and there is a method in which only the frequency component is shifted without moving the formant.
[0008]
By synthesizing the generated harmony audio signal into a plurality of input audio signals, an audio signal with harmony can be output.
[0009]
DETAILED DESCRIPTION OF THE INVENTION
A karaoke apparatus according to an embodiment of the present invention will be described with reference to the drawings. This karaoke apparatus is a so-called sound source karaoke apparatus. The sound source karaoke device is a karaoke device that generates a karaoke performance sound by driving the sound source device with music data. The music data is sequence data composed of a plurality of tracks such as a performance data string for designating pitches and sound generation timings.
[0010]
Moreover, this karaoke apparatus has a harmony addition function for adding a harmony voice signal having a pitch of 3 or 5 degrees to a singing voice signal of a singer. The harmony sound signal is a signal that generates a sound signal having a pitch of 3 degrees or 5 degrees by pitch-shifting the singing sound of the singer and outputs the sound signal as a harmony sound signal. Furthermore, this karaoke apparatus determines which one is the main melody even when two people are singing simultaneously in a duet song, and adds harmony only to the singing voice signal of the main melody.
[0011]
FIG. 1 is a block diagram of a main part of the karaoke apparatus. The figure shows only the voice signal processing unit for karaoke performance sound (accompaniment sound) and singing sound, and the display processing unit and the song selection unit for lyrics and background images are not shown because they have a more general configuration than before. doing. Music data for performing karaoke performance is stored in the HDD 15. The HDD 15 stores thousands of pieces of music data. When one music piece is selected by a music selection unit (not shown), the sequencer 14 reads the selected music data. The sequencer 14 has a memory for storing the read music data and a sequence program processing unit for sequentially reading the music data based on the tempo clock. The read data is sent to a predetermined processing unit according to the track. Is output.
[0012]
Here, the composition of the music data will be described with reference to FIG. In FIG. 2A, the music data is composed of a music sound track, main melody track, harmony track, lyrics track, audio track, effect track, and audio data portion following a header in which the title, genre, etc. are written. Of these, the main melody track is composed of a plurality of event data and sequence data composed of duration data Δt indicating a time interval between the event data as shown in FIG. The sequencer 14 counts Δt with a predetermined tempo clock at the time of karaoke performance, and reads the event data that follows when Δt is counted up. The read event data of the main melody track is output to the main melody selection unit 23 described later as main melody selection data.
[0013]
Similar to the main melody track, tracks other than the main melody track, that is, a musical tone track, a harmony track, a lyrics track, an audio track, and an effect track, are composed of sequence data composed of a plurality of event data and duration data strings. The musical tone track is composed of a plurality of sequence tracks such as a melody track, a rhythm track, and a chord track for karaoke performance. When the sequencer 14 reads event data from the musical sound track during karaoke performance, the data is output to the sound source 16. The harmony track is a track that stores the harmony melody to be added to the main melody, and this event data is output to the song analysis unit 22 and the pitch shift unit 26. The sound source 16 generates a musical sound signal based on this data. The lyrics track is a sequence track for displaying lyrics on the screen. When the sequencer 14 reads the event data of the lyrics track, it outputs it to a display control unit (not shown). The display control unit controls the display of lyrics based on the event data. The audio track is a track for designating the reproduction timing of a human voice signal such as chorus sound or a matching hand that is difficult to synthesize with the sound source 16. A plurality of human voice signals are stored as voice data in the voice data section. When the sequencer 14 reads out the audio track event data during the karaoke performance, the audio data designated by the event data is output to the adder 28 described later. Thereby, this voice data is mixed as a karaoke performance. The effect track is a track for controlling an effect unit (configured by a DSP) included in the sound source 16. The effect imparted by the effect unit includes reverb. This event data is output to the sound source 16.
[0014]
Based on the musical tone event data input from the sequencer 14, the sound source 16 forms a musical tone signal having a tone color, pitch, and volume specified by the data. This musical sound signal is input to the adder 28 in the DSP 13.
[0015]
On the other hand, this karaoke apparatus has one singing microphone 10, and when two people sing a duet song or the like, the two singing sounds are input to the one microphone 10. The singing voice signal input from the microphone 10 is amplified by the amplifier 11 and converted into a digital signal by the ADC 12. The audio signal converted into the digital signal is input to the DSP 13. The DSP realizes various functions by a microprogram. The DSP 13 stores a microprogram for realizing a function as shown in the block of the figure, and the digital signal is sampled in one sampling period. The microprogram is running at a speed that performs all functions.
[0016]
In the figure, the digital audio signal input from the ADC 12 is input to the autocorrelation analyzer 21 and delays 24 and 27. The autocorrelation analyzer 21 analyzes the repetition period of each frequency component of the input sound signal, and detects the fundamental frequency of the singing sound signals of a plurality of singers from this repetition period.
[0017]
FIG. 3 is a diagram for explaining a method of autocorrelation analysis in the autocorrelation analysis unit 2. Since the autocorrelation function of the periodic signal is also a periodic function having the same period as the signal, the autocorrelation function of the signal of periodic P samples is 0, ± P, ± 2P,..., Regardless of the time origin of the signal. The maximum is reached. Therefore, the period can be estimated by finding the first local maximum of the autocorrelation function. In the figure, the maximum values appear at a plurality of positions that are not integer multiples, and it can be seen that these indicate the periods of singing signal waves of different frequencies by two singers. This determines the fundamental frequency. The autocorrelation analysis unit 21 inputs these two fundamental frequencies to the song analysis unit 22 and the main melody selection unit 23. In addition, a voiced sound has a clear periodic waveform, whereas an unvoiced sound has a noise-like waveform, so that a voiced / unvoiced sound can be identified. This identification result is input to the song analysis unit 22.
[0018]
Based on the main melody information (main melody track event data) input from the sequencer 14, the main melody selection unit 23 is one of the fundamental frequencies of the possible audio signals of a plurality of parts input from the autocorrelation analysis unit 21. Find out. This selection information is input to the main melody component separation unit 25.
[0019]
On the other hand, the singing analysis unit 22 analyzes the current singing state based on the analysis information including the fundamental frequency input from the autocorrelation analysis unit 21. The singing state is a state in which the number of singers currently singing is 0 (silent section such as an interlude), 1 (solo or cross), or 2 or more (during a duet) It is. The singing analysis unit 22 judges this, and detects a state such as whether or not an audio signal other than the main melody is in harmony when two or more people are singing. The detection of harmony is determined based on the harmony information (harmonic track event data) input from the sequencer. It is also determined whether the main melody is voiced or unvoiced.
[0020]
The singing analysis unit 22 controls the operation contents of the main melody selection unit 23 and the main melody component separation unit 25 based on the determination result. If it is determined that the singing state is a silent section, neither main melody selection nor main melody component analysis is required, and therefore the operations of the main melody selection unit 23 and the main melody component separation unit 25 are suspended during this period. Also, when one of the two sings the main melody and the other sings the harmony to it, it is not necessary to generate a harmony sound on top of it, so the main melody component separation unit 25 is suspended. . When the main melody separation unit 25 ceases operation, since there is no audio signal input to the subsequent pitch shift unit 26, the generation of the harmony sound by the pitch shift is also stopped.
[0021]
If it is detected that only one of the singers is currently singing, it is clear that the melody being sung is the main melody, so that the operation of the main melody selection unit 23 is stopped and the main melody component separation unit is stopped. The singing voice signal input to 25 is instructed to be skipped. As a result, one singing voice signal is directly input from the delay 24 to the pitch shift unit 26.
[0022]
Further, the separation algorithm of the main melody component separation unit 25 is switched depending on whether the current voice of the main melody is a voiced sound or an unvoiced sound. That is, in the case of a voiced sound, a singing voice signal is composed of harmonics of the fundamental tone (basic frequency) relatively simply, and therefore, main melody component separation is performed based on this principle. On the other hand, in the case of an unvoiced sound, since many nonlinear noise components are included, the melody singing voice signal is separated by a method different from that of the voiced sound.
[0023]
The main melody audio signal separated by the main melody component separation unit 25 or the single singing voice signal skipped by the main melody component separation unit 25 is input to the pitch shift unit 26. The pitch shift unit 26 shifts the pitch of the input audio signal based on the harmony information input from the sequencer 14 and outputs it to the adder 28 as a harmony audio signal.
[0024]
Here, as shown in FIG. 4, the pitch shift unit 26 stores the formant (envelope of the frequency component) of the audio signal input from the previous stage, and pitch-shifts only each frequency component constituting the formant. Each frequency component that has been pitch-shifted is level-adjusted to match the pitch envelope. Thereby, it is possible to shift only the pitch (frequency) without changing the sound quality.
[0025]
In FIG. 1, in addition to the harmony sound signal, the adder 28 has a karaoke performance sound input from the sound source 16, a chorus sound input directly from the sequencer 14, and a singing sound input directly from the ADC 12 via the delay 27. A signal is input. The adder 28 adds and synthesizes these singing voice signals, harmony voice signals, karaoke performance sounds, and chorus sounds, and mixes them into a stereo signal. The mixed audio signal is output from the DSP 13 and input to the DAC 17. The DAC 17 converts this digital stereo signal into an analog signal and inputs it to the amplifier 18. The amplifier 18 amplifies the analog signal and outputs it from the speaker 19.
[0026]
The two delays 24 and 27 inserted in the block of the DSP 13 are for absorbing delay times for signal processing in the signal processing autocorrelation analysis unit 21, the song analysis unit 22, the main melody selection unit 23, and the like. Is.
[0027]
In this way, in this karaoke apparatus, in order to analyze which of the plural (two) singing voices inputted from one microphone 10 is the main melody, and to add the harmony singing only to the main melody and output it. Even if you sing duet songs together, you can add harmony only to the main melody.
[0028]
FIG. 5 is a block diagram of a main part of a karaoke apparatus according to another embodiment of the present invention. The difference between this karaoke device and the karaoke device of the first embodiment shown in FIG. 1 is that this karaoke device is provided with microphones (two in the figure) for the number of singers, and the singing voice of each singer. The signal is input to the DSP in a separate system. The music data storage / reading unit for karaoke performance and the signal system after the addition of the singing voice signal and the karaoke performance signal are the same as those in the first embodiment, and thus the same reference numerals are given to the components. Is omitted.
[0029]
The two duet microphones 30 and 31 are respectively amplified by separate amplifiers 32 and 33, converted into digital signals by ADCs 34 and 35, and input to the DSP 36. In the DSP 36, the first singing voice signal (singing voice signal input from the microphone 30) is input to the autocorrelation analysis unit 41 and the addition unit 47. The second singing voice signal (singing voice signal input from the microphone 31) is input to the autocorrelation analysis unit 42 and the addition units 44 and 47. The autocorrelation analyzers 41 and 42 analyze the fundamental frequencies of the first song signal and the second song signal, respectively. In this configuration, the autocorrelation analyzers 41 and 42 do not need to separate and analyze the fundamental frequencies of a plurality of singing sounds. The analysis result is input to the song analysis unit 43. The singing analysis unit 43 performs a singing number determination operation, a main melody selection operation, and a harmony detection operation based on the fundamental frequency of the two singing voice signals input and the main melody information and harmony information input from the sequencer 14. That is, it is analyzed whether or not two people are singing at the same time, if two people are singing, which is the main melody, and the other singing sound is the harmony of the main melody. When the main melody is selected by the main melody selection operation, a corresponding select signal is input to the selector 45. The selector 45 switches the connection to input the singing voice signal selected as the main melody to the pitch shifter 46. The pitch shifter 46 pitch-shifts the input audio signal based on the harmony information input from the sequencer 14 to generate a harmony audio signal.
[0030]
The harmony audio signal is input to the adding unit 49. In addition to the harmony audio signal, the adder 49 receives a karaoke performance sound input from the sound source 16, a chorus sound input directly from the sequencer 14, and a singing audio signal input from the ADC 12 via the adder 47 -delay 48. Entered. The adder 49 adds and synthesizes these singing voice signals, harmony voice signals, karaoke performance sounds, and chorus sounds, and mixes them into a stereo signal. The mixed audio signal is output from the DSP 36 and input to the DAC 17.
[0031]
The above embodiment includes inventions other than those described in the scope of claims, and the present invention is described as follows in the subordinate form of the invention of claim 1 of the scope of claims.
[0032]
[Claim 2] The audio signal processing apparatus according to claim 1, wherein the audio signal input means is a means for inputting a plurality of singing audio signals in one system.
[0033]
[Claim 3] The main melody determination means compares the means for detecting each fundamental frequency of a plurality of input audio signals with the detected fundamental frequency and the main melody information stored in advance. The audio signal processing apparatus according to claim 1, further comprising: means for determining a main melody.
[0034]
[Claim 4] The audio signal processing apparatus according to claim 2, wherein the main melody extraction means is means for separating and extracting a main melody audio signal component from a plurality of audio signals input in one system.
[0035]
[Claim 5] The audio signal processing apparatus according to claim 1, wherein the harmony generation means is means for pitch-shifting the audio signal of the main melody based on previously stored harmony information.
[0036]
[Claim 6] Harmony detection means for detecting whether or not an audio signal other than the main melody is in harmony with the main melody, and the harmony generation means when the harmony detection means detects an audio signal in harmony. The audio signal processing apparatus according to claim 1, further comprising a means for invalidating.
[0037]
[Claim 7] Harmony detection means for determining whether or not an audio signal other than the main melody matches the harmony information, and the harmony generation means is disabled when the harmony detection means detects an audio signal in harmony. The audio signal processing apparatus according to claim 5, further comprising:
[0038]
Further, the present invention can be applied to musical instrument performance in addition to singing voice such as karaoke.
[0039]
【The invention's effect】
As described above, according to the present invention, even when a plurality of audio signals are input, the audio signal for singing / playing the main melody is determined and extracted from the input, and the harmony audio is added only to the audio signal. Only harmony that enhances the main melody can be added, and singing / performance is not impaired by adding harmony to anything that is not the main melody.
[0040]
In addition, since the main melody is determined from a plurality of input audio signals, the main melody can be extracted from the performances in which the main melody alternates.
[Brief description of the drawings]
FIG. 1 is a block diagram of the main part of a karaoke apparatus according to an embodiment of the present invention. FIG. 2 is a block diagram showing the structure of music data of the karaoke apparatus. FIG. 4 is a diagram for explaining a pitch shift method of an audio signal. FIG. 5 is a configuration diagram of a main part of a karaoke apparatus according to another embodiment of the present invention.

Claims (2)

A karaoke apparatus comprising music data reproduction means, fundamental frequency information extraction means, main melody selection means, main melody separation means, pitch shift means, and singing analysis means,
The music data reproduction means reproduces the music data and outputs an accompaniment signal, outputs the main melody information included in the music data to the main melody selection means, and outputs the harmony information to the pitch shift means and the singing analysis means,
The fundamental frequency information extracting means extracts fundamental frequency information for each part from the audio signal from the microphone,
The main melody selection means outputs the basic frequency information suitable for the main melody information among the basic frequency information for each part as selection information,
The main melody separation means frequency-analyzes the sound signal from the microphone and separates it into a plurality of part sound signals, and outputs one of the part sound signals as the main melody sound signal based on the selection information from the main melody selection means,
The pitch shift means pitch-shifts the main melody audio signal based on the harmony information and outputs a harmony audio signal.
The singing analysis means analyzes the number of singing parts in the audio signal based on the fundamental frequency information. When the number of singing parts is 0, the first process is performed. When the number of singing parts is 1, the second process is performed. If the number of singing parts is plural and the harmony part corresponding to the harmony information is included, the third process is performed.
The first process pauses the main melody selection means, the main melody separation means, and the pitch shift means,
In the second process, the main melody selection means and the main melody separation means are paused, and a voice signal is input to the pitch shift means instead of the main melody voice signal,
The third process is a karaoke apparatus that pauses the main melody separation means and the pitch shift means. A karaoke apparatus comprising music data reproduction means, fundamental frequency information extraction means, main melody selection means, main melody separation means, pitch shift means, and singing analysis means,
The music data reproduction means reproduces the music data and outputs an accompaniment signal, and outputs the harmony information included in the music data to the pitch shift means and the singing analysis means,
The fundamental frequency information extracting means extracts fundamental frequency information for each part from the audio signal from the microphone,
The main melody selection means outputs the fundamental frequency information having a predetermined frequency height among the fundamental frequency information for each part as selection information,
The main melody separation means frequency-analyzes the sound signal from the microphone and separates it into a plurality of part sound signals, and outputs one of the part sound signals as the main melody sound signal based on the selection information from the main melody selection means,
The pitch shift means pitch-shifts the main melody audio signal based on the harmony information and outputs a harmony audio signal.
The singing analysis means analyzes the number of singing parts in the audio signal based on the fundamental frequency information. When the number of singing parts is 0, the first process is performed. When the number of singing parts is 1, the second process is performed. If the number of singing parts is plural and the harmony part corresponding to the harmony information is included, the third process is performed.
The first process pauses the main melody selection means, the main melody separation means, and the pitch shift means,
In the second process, the main melody selection means and the main melody separation means are paused, and a voice signal is input to the pitch shift means instead of the main melody voice signal,
The third process is a karaoke apparatus that pauses the main melody separation means and the pitch shift means.

Images