JP3176273B2 - Audio signal processing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an audio signal processing apparatus for adding a harmony voice to a melody voice signal such as a singing voice, and more particularly to a melody voice signal when a plurality of melody voice signals are input. The present invention relates to an audio signal processing device that adds harmony to only the audio signal of the audio signal.

[0002]

2. Description of the Related Art In order to excite the singing of karaoke, it has been proposed to output a singer's singing with a harmony sound (for example, a melody three times higher than the singing melody). . As the harmony function, a harmony sound is generally generated by shifting the pitch of a singing voice signal.

Some karaoke songs, such as so-called duet songs, are sung by a plurality (two people).

[0004]

However, in the case of the above-mentioned duet music, two singing voice signals are mixed and input. In a conventional karaoke apparatus having a harmony adding function, all singing voices are input. Because it adds harmony, it becomes an unclear song with multiple parts mixed, and it is not possible to excite the karaoke song,
On the contrary, there is a disadvantage that the singing voice signal of the two is damaged.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a karaoke apparatus capable of extracting harmony by extracting only a specific part from a plurality of singing voice signals even when a plurality of singing voice signals are input.

[0006]

According to the first aspect of the present invention, a specific part selecting means (21, 23), a specific part component separating means (25), a pitch shifting means (26), a synthesizing means (2)
8) An audio signal processing device comprising 8), wherein the specific part selecting means (21, 23) extracts a fundamental frequency of each part from a mixed signal obtained by mixing sounds of a plurality of parts, and acquires melody information of the specific part or based on the level of rank of the fundamental frequency of each part, said select a specific part from among the plurality of parts, and outputs the selection information indicating the specific part to <br/> specific part component separating means (25), The specific part component separating means (25) separates the mixed signal into audio signals of the respective parts, and separates the specific part selecting means (21, 2).
Outputting a sound signal of a specific part indicated by the selection information inputted from 3) to the pitch shift means (26);
The pitch shift means (26) generates a harmony audio signal by pitch-shifting the input audio signal, and the synthesizing means (28) synthesizes and outputs the mixed signal and the harmony audio signal. .

According to a second aspect of the present invention, there is provided an audio signal processing apparatus including specific part selecting means (41, 42, 43), switching input means (45), pitch shifting means (46), and synthesizing means (47, 49). Then, specific part selection means (4
1,42,43) is separately input <br/> force audio signals of a plurality of parts, and detects the fundamental frequency of each part respectively, ranking the level of the fundamental frequency of the melody information or each part of a particular part A specific part is selected from the plurality of parts , and selection information designating the specific part is output to the switching input means (45), and the switching input means (45) is selected.
Is input from specific part selecting means (41, 42, 43)
The voice signal of the specific part specified by the selected information is output to the pitch shift means (46). The pitch shift means (46) generates a harmony voice signal by pitch-shifting the input voice signal. (47, 49)
Is characterized by combining and outputting a plurality of part audio signals and a harmony audio signal.

[0008]

The present invention is applied to , for example, a karaoke apparatus. In this case , an audio signal is input from a singing microphone and an apparatus such as an amplifier connected to the microphone.
Is done. An audio signal of a specific part is selected from the input audio signals of a plurality of parts . The specific part is, for example, a main melody part or a harmony part, a so-called hand part. As a method for determining such a specific part, a corresponding part may be determined as a specific part based on the specific part information stored in advance, or a certain rule, for example, Special audio signal
The determination may be based on rules such as a fixed part. The audio signal determined as the specific part is extracted from the input audio signals of the plurality of parts. When audio signals of a plurality of parts are input in different systems, a system to which an audio signal of a specific part is input may be selected. When audio signals of a plurality of parts are input in a single system, Then, the audio signal of the specific part is extracted by a method such as separating and extracting only the frequency component corresponding to the fundamental harmonic of the specific part. A harmony sound signal is generated by pitch-shifting the extracted sound signal of the specific part. As a pitch shift method, there is a method of simply changing a read clock, and a method of shifting only a frequency component without moving a formant.

By synthesizing the harmony audio signal generated in this way with the input audio signals of a plurality of parts, it is possible to output an audio signal with various harmony.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A karaoke apparatus according to an embodiment of the present invention will be described with reference to the drawings. This karaoke device is a so-called sound source karaoke device. A sound source karaoke device is a karaoke device that generates a karaoke performance sound by driving the sound source device with music data. The music data is sequence data composed of a plurality of tracks, such as a performance data string that specifies a pitch and a sound generation timing.

Further, the karaoke apparatus has a harmony adding function of adding a harmony voice signal having a pitch of 3rd or 5th to a singing voice signal of a singer. The harmony voice signal generates a voice signal having a pitch such as 3rd or 5th by shifting the pitch of the singer's singing voice,
This is output as a harmony audio signal.
In addition, this karaoke device uses two duet songs.
Even when a part is sung simultaneously, it is determined which of the parts is the main melody part, and harmony is added only to the singing voice signal of the main melody part.

FIG. 1 is a block diagram of a main part of the karaoke apparatus. FIG. 1 shows only a karaoke performance sound (accompaniment sound) and a singing sound signal processing unit, and a display processing unit such as lyrics and a background image and a music selection unit are omitted from the drawing since they have a more general configuration than before. are doing. Music data for performing a karaoke performance is stored in the HDD 15. HDD15
The song data is stored for thousands of songs. When one of the songs is selected by a song selection unit (not shown), the sequencer 14 reads the selected song data. The sequencer 14 has a memory for storing the read music data,
There is a sequence program processing unit for sequentially reading out the music data based on the tempo clock, and the read data is output to a predetermined processing unit according to the track.

Here, the configuration of the music data will be described with reference to FIG. In FIG. 3A, the music data is composed of a musical tone track, a main melody track, a harmony track, a lyric track, an audio track, an effect track, and an audio data section, following a header in which a music name, a genre and the like are written. Of these, the main melody track is composed of sequence data including a plurality of event data and duration data Δt indicating a time interval between the event data as shown in FIG. The sequencer 14 uses a predetermined tempo clock at the time of karaoke performance
Is counted, and when this Δt is counted up, the subsequent event data is read out. The read event data of the main melody track is output to the main melody selection unit 23 described later as data for selecting a main melody.

Tracks other than the main melody track, that is, tone tracks, harmony tracks, lyrics tracks, audio tracks, and effect tracks, like the main melody track, are composed of sequence data including a plurality of event data and duration data strings. The musical sound track is composed of a plurality of sequence tracks such as a melody track for karaoke performance, a rhythm track, and a chord track. Sequencer 1 when performing karaoke
4 reads out the event data from the musical sound track, and outputs the data to the sound source 16. The harmony track is a track in which a harmony melody to be added to the main melody is stored.
2 and output to the pitch shift unit 26. The sound source 16 generates a tone signal based on this data. The lyrics track is a sequence track for displaying lyrics on the screen. When the sequencer 14 reads out the event data of the lyrics track, it outputs this to a display control unit (not shown). The display controller controls the display of lyrics based on the event data. The audio track is a sound source 16
This is a track for designating the reproduction timing of a human voice signal such as a chorus voice or a hand that is difficult to synthesize with a.
A plurality of human voice signals are stored in the voice data section as voice data. When the sequencer 14 reads the event data of the audio track during the karaoke performance, the sequencer 14 outputs the audio data specified by the event data to the adder 28 described later. As a result, the audio data is mixed as a karaoke performance. The effect track is a track for controlling an effect section (comprising a DSP) included in the sound source 16. The effect provided by the effect section includes reverb and the like. This event data is output to the sound source 16.

The tone generator 16 forms a tone signal having a tone, pitch and volume specified by the tone event data input from the sequencer 14 based on the tone event data. This tone signal is input to the adder 28 in the DSP 13.

On the other hand, this karaoke apparatus has one singing microphone 10, and when two people sing in a duet song or the like, the singing sounds of the two are input to the one microphone 10. You. A singing voice signal input from the microphone 10 is amplified by the amplifier 11 and converted to a digital signal by the ADC 12. The audio signal converted to the digital signal is input to the DSP 13. The DSP realizes various functions by a microprogram.
Numeral 13 stores a microprogram for realizing the function shown in the block of FIG. 3, and executes this microprogram at a speed for executing all the functions shown in one sampling cycle of the digital signal. I have.

In FIG. 1, a digital audio signal input from an ADC 12 is input to an autocorrelation analyzer 21 and delays 24 and 27. The autocorrelation analyzer 21 analyzes the repetition period of each frequency component of the input audio signal, and detects the fundamental frequency of the singing voice signals of a plurality of singers from the repetition period.

FIG. 3 is a diagram for explaining a method of autocorrelation analysis in the autocorrelation analyzer 2. Since the autocorrelation function of the periodic signal is also a periodic function having the same period as that of the signal, the autocorrelation function of the signal of the period P samples is 0, ± P, ± 2P,. The maximum is reached. Therefore, by finding the first maximum point of the autocorrelation function, its period can be estimated. In the figure, it can be seen that the local maxima appear at a plurality of positions that are not integral multiples, and these indicate the periods of the singing signal waves of different frequencies by the two singers. Thus, the fundamental frequency is determined. The auto-correlation analysis unit 21
Singing analysis section 22 and main melody selection section 2
Enter 3 In addition, a voiced sound has a clear periodic waveform, whereas an unvoiced sound has a noise-like waveform, so that voiced / unvoiced sound can be distinguished. This identification result is input to the singing analysis unit 22.

Based on the main melody information (event data of the main melody track) input from the sequencer 14, the main melody selection unit 23 selects which of the fundamental frequencies of the singing voice signals of a plurality of parts input from the autocorrelation analysis unit 21. Determine whether it is a singing voice signal of the main melody part. This selection information is input to the main melody component separation unit 25.

On the other hand, the singing analysis section 22 analyzes the current singing state based on the analysis information including the fundamental frequency input from the autocorrelation analysis section 21. The singing state is a state in which the number of singers currently singing is 0 (silent section such as interlude), 1 (solo or negotiating), or 2 or more (during a duet). It is. Singing analysis part 2
2 judges this, and detects a state such as whether or not the audio signal of the part other than the main melody is in harmony when two or more people sing. Harmony is detected based on harmony information (event data of a harmony track) input from the sequencer. Also,
It is also determined whether the melody is voiced or unvoiced.

The singing analysis section 22 controls the operation of the main melody selecting section 23 and the main melody component separating section 25 based on the result of the determination. When it is determined that the singing state is a silent section, neither the main melody selection nor the main melody component analysis is required,
The operations of the main melody selecting unit 23 and the main melody component separating unit 25 are suspended during this period. When one of the two sings the main melody and the other sings the harmony for it, the main melody component separation unit 25 is paused because it is not necessary to generate a harmony sound over it. .
When the main melody separation unit 25 stops its operation, there is no audio signal input to the subsequent pitch shift unit 26, so that the generation of the harmony sound by the pitch shift also stops.

When one sings the main melody and the other sings a harmony, the pitch may be shifted by specifying a harmony sound different from the harmony singing. For example, when the other singer sings three times above the main melody, pitch shifter 26 shifts the main melody five degrees above.

If it is detected that only one of the singers is currently singing, it is clear that the melody being sung is the main melody. It instructs the component separating unit 25 to skip the singing voice signal input. Thereby, one singing voice signal is directly input from the delay 24 to the pitch shift unit 26.

Further, the separation algorithm of the main melody component separation unit 25 is switched depending on whether the voice of the current main melody is a voiced sound or an unvoiced sound. That is, for voiced sounds,
Since the singing voice signal is relatively simply composed of harmonics of the fundamental tone (fundamental frequency), the main melody component is separated based on this principle. On the other hand, since the unvoiced sound contains many non-linear noise components, the singing voice signal of the main melody is separated by a method different from the above-mentioned voiced sound.

The voice signal of the main melody separated by the main melody component separation section 25 or the voice signal of a single singing skipping the main melody component separation section 25 is input to the pitch shift section 26. The pitch shift unit 26 shifts the pitch of the input audio signal based on the harmony information input from the sequencer 14, and outputs the harmony audio signal to the addition unit 28.

Here, as shown in FIG. 4, the pitch shift section 26 stores the formants (envelopes of frequency components) of the audio signal inputted from the preceding stage, and pitch-shifts only each frequency component constituting the formants. I do. The level of each pitch-shifted frequency component is adjusted to match the envelope of the pitch. Thus, only the pitch (frequency) can be shifted without changing the sound quality.

In FIG. 1, in addition to the harmony sound signal, a karaoke performance sound input from the sound source 16, a chorus sound input directly from the sequencer 14, and a direct input from the ADC 12 via the delay 27 are added to the adder 28. Singing voice signal is input. The adder 28 adds and synthesizes the singing voice signal, the harmony voice signal, the karaoke performance sound, and the chorus sound and mixes them into a stereo signal. This mixed audio signal is DSP
13 and input to the DAC 17. DAC1
7 converts this digital stereo signal into an analog signal and inputs it to the amplifier 18. The amplifier 18 amplifies this analog signal and outputs it from the speaker 19.

The two delays 24 and 27 inserted in the block of the DSP 13 absorb the delay time for signal processing in the signal processing autocorrelation analysis unit 21, the singing analysis unit 22, the main melody selection unit 23 and the like. It is for doing.

As described above, the karaoke apparatus analyzes which of the plurality of (two parts) singing voice signals inputted from one microphone 10 is the main melody part, and analyzes the singing voice signal of the main melody part. Since harmony is added to only the melody and the melody is output, harmony can be added only to the main melody even when a duet song or the like is sung together.

FIG. 5 is a block diagram of a main part of a karaoke apparatus according to another embodiment of the present invention. The difference between the karaoke apparatus and the karaoke apparatus of the first embodiment shown in FIG. 1 is that the karaoke apparatus has microphones (two in the figure) for the number of singers and the singing voice of each singer. The point is that the signal is input to the DSP in another system. The storage / read-out unit for the music data for karaoke performance and the signal system after the singing voice signal and the karaoke performance signal are added are the same as those in the first embodiment, so the components are given the same numbers and explained. Is omitted.

The two duet microphones 30 and 31 are amplified by amplifiers 32 and 33 of different systems, respectively.
The signals are converted into digital signals at 34 and 35 and input to the DSP 36. In the DSP 36, the first singing voice signal (the singing voice signal input from the microphone 30) is input to the autocorrelation analyzing unit 41 and the adding unit 47. Also, the second
(The singing voice signal input from the microphone 31) is input to the autocorrelation analyzer 42 and the adders 44 and 47. The autocorrelation analyzers 41 and 42 analyze the fundamental frequencies of the first singing signal and the second singing signal, respectively. In this configuration, the autocorrelation analyzers 41 and 42 do not need to separate and analyze the fundamental frequencies of a plurality of singing sounds. The analysis result is input to the singing analysis unit 43. The singing analysis unit 43 outputs the fundamental melody information input from the sequencer 14 and the fundamental frequencies of the two input singing voice signals,
Based on the harmony information, a singing number determination operation, a main melody selection operation, and a harmony detection operation are executed. That is,
It is analyzed whether two people are singing at the same time, if they are singing together, which is the main melody, and whether the other singing sound is the harmony of the main melody. When the main melody is selected by the main melody selection operation, a select signal corresponding to the main melody is input to the selector 45. The selector 45 converts the singing voice signal selected as the main melody into the pitch shift unit 4.
6. Switch connection to input to 6. Pitch shift unit 4
Reference numeral 6 shifts the pitch of the input audio signal based on the harmony information input from the sequencer 14 to generate a harmony audio signal.

The harmony sound signal is input to the adder 49. In addition to the harmony sound signal, the addition unit 49 includes a karaoke performance sound input from the sound source 16, a chorus sound input directly from the sequencer 14, and the ADC 12.
, A singing voice signal input through an adder 47-delay 48. The adder 49 adds and synthesizes the singing voice signal, the harmony voice signal, the karaoke performance sound, and the chorus sound and mixes them into a stereo signal.
The mixed audio signal is output from the DSP 36 and input to the DAC 17.

In the above-described embodiment, the karaoke apparatus having the function of adding harmony to only the singing voice signal of the main melody part of the duet tune has been described. However, only the parts other than the main melody part of the duet tune, for example, only the matching hand part Harmony may be added. In addition, harmony may be added to one or both of the main melody part and other parts by selection.

For example, in the apparatus shown in FIG. 5, the selector 45 can be arbitrarily switched to the main melody part or the other parts, and the harmony information of the main melody or the harmony information of the other parts is pitch-shifted in accordance with the switching. The above function can be realized by inputting to the section 46.

FIG. 6 shows an embodiment in which a plurality of singing voice signals are input in one system. In the configuration of FIG. 2, the same components as those of the embodiment of FIG. In this embodiment, the music data stored in the sequencer 14 is provided with a specific part track instead of the main melody track. The specific part selection unit 53 inputs the data of the specific part track (specific part information) from the sequencer 14, and which of the fundamental frequencies of the singing voice signals of the plurality of parts input from the autocorrelation analysis unit 21 corresponds to the specific part. Determine whether the signal is a singing voice signal. This selection information is input to the specific part component separation unit 55. Certain part component separation unit 5 5,
Only the frequency components of a specific part are extracted and separated from the input singing voice signals of a plurality of parts. How to determine a specific part
As a rule, as mentioned earlier, the audio of multiple audio parts
Identify the highest melody part of the signal, such as the main melody
A method of forming a part or the like can be adopted. Extraction
The separated components of the singing voice signal of the specific part are input to the pitch shift unit 26. The pitch shift unit 26 shifts the pitch to a pitch that thickens the specific part.

Further, the present embodiment includes inventions other than the inventions described in the claims, and the present invention is as follows.

[0038]

[0039]

[0040]

[Structure 1 ] The voice according to claim 1 or 2 , wherein the pitch shift means is a means for pitch-shifting the voice signal of the specific part based on harmony information stored in advance. Signal processing device.

[Structure 2 ] Harmony detecting means for detecting whether or not an audio signal of a part other than the specific part is in harmony with the specific part , and detecting the audio signal in which the harmony detecting means is in harmony the audio signal processing apparatus according to claim 1 or 2 claims, characterized in that a means to disable the pitch shifting means when.

[0043]

The present invention can be applied not only to singing voices such as karaoke but also to musical instrument performances.

[0045]

As described above, according to the present invention, even if audio signals of a plurality of parts are input, the audio signals of a specific part such as the main melody part and other parts are determined and extracted from the input, and this audio signal is extracted. Since the harmony sound is added only to the signal, even when audio signals of a plurality of parts are input at the same time, only the harmony that enhances the specific part can be added, and the performance can be enhanced.

Since the main melody is determined from a plurality of input audio signals, the main melody can be extracted from the performance even when the main melody alternates.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a main part of a karaoke apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of music data of the karaoke apparatus.

FIG. 3 is a diagram illustrating an autocorrelation analysis of an input singing voice signal.

FIG. 4 is a diagram for explaining a pitch shift method of an audio signal.

FIG. 5 is a configuration diagram of a main part of a karaoke apparatus according to another embodiment of the present invention.

FIG. 6 is a configuration diagram of a main part of a karaoke apparatus according to still another embodiment of the present invention.

Claims (2)

(57) [Claims] A specific part selecting means (21, 23);
Constant part component separation means (25), pitch shift means (2
6) An audio signal processing device including a synthesizing unit (28).
The specific part selecting means (21, 23) is a
From the mixed signal with mixed voices,BasicExtract frequency
As well as the melody information of a specific part or each part
ofBasicBased on the rank of frequency,The multiple parts
From insideSpecific partToSelect, Indicating the specific part
Select informationFor specific part component separation means (25)output
Then, the specific part component separating means (25) converts the mixed signal into each part.
And a specific part selection means.
From (21,23)inputWas doneIndicated by selection information
SpecificMeans for shifting the voice signal of the part (26)
And the pitch shift means (26) picks up the input audio signal.
The harmony sound signal is generated by performing the harmony sound signal.
An audio signal processing device that synthesizes and outputs voice signals. 2. A specific part selecting means (41, 42, 4)
3), switch input means (45), pitch shift means (4)
6) an audio signal processing device provided with a synthesizing means (47, 49);
The specific part selection means (41, 42, 43) includes a plurality of parts.
Separate audio signalsToEnter each partBasicFrequency
While detecting each, the melody information of a specific part or
Is for each partBasicBased on the rank of frequency,The compound
From several partsSpecific partToSelect, The specific par
Selection informationTo switch input means (45)output
The switching input means (45) is provided with a specific part selecting means (41,
42, 43)Instructed by the entered selection information
Specific partAudio signalToTo pitch shift means (46)
Output, and the pitch shift means (46) picks up the input audio signal.
The harmony sound signal is generated by performing the shift operation, and the synthesizing means (47, 49) outputs the sound signal of a plurality of parts.
Audio signal that synthesizes and outputs the harmony audio signal
Processing equipment.