JPH10268895A - Voice signal processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a voice signal processing device that can convert a singing voice signal of a singer to that of an original singer. SOLUTION: A pitch/sound volume extracting part 31 extracts pitch (frequency) data and sound volume data from a singing voice signal of a singer. An impulse generation part 32 generates a pulse train with an interval based on the pitch data. The pulse train has a wide frequency spectrum near a white noise, but a signal of a tone of an original singer is made by cutting this with formant of an original singer in a filter 33. This formant data is stored as sequence data with music data for KARAOKE playing, read out in parallel with KARAOKE playing, and written in a formant data buffer 36. A signal of a tone of an original singer can be made quite the same as that of a song of a singer by gain-controlling the signal of timre of an original singer with the sound volume data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an audio signal processing device for converting a singer's voice into another person's voice.

[0002]

2. Description of the Related Art In order to enable men to sing original female songs, and to sing original male songs by female, the range of male voices to female voices and female voices to male voices A karaoke apparatus having a function of converting has been proposed.

In the case of converting the frequency of an audio signal, simply compressing / expanding the audio signal waveform results in a tone far apart from a normal human voice, such as advancing or slowing a recording tape. The above range conversion function uses a formant shift method.

In the formant shift method, a continuous waveform of about 30 to 60 ms is cut out from a voice signal of a singer by a Hamming function, and this is arranged at time intervals of a frequency of a conversion destination to thereby obtain a characteristic frequency component of the singer. It converts the singing frequency while preserving (formant).

[0005]

However, in the above-mentioned range conversion function, even if a man can sing with a female voice and a woman can sing with a male voice, the formants are the same, and the singing is similar to that of the original singer. There was a drawback that we could not meet the demand for singing in a voice. The demand for singing in the voice of the original singer is the same whether a man sings a male song or a woman sings a female song,
Current karaoke equipment could not respond to this.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an audio signal processing device capable of converting a singing voice of a singer into a voice of an original singer.

[0007]

According to the first aspect of the present invention, a storage means for storing a change in the formant of a specific person's singing voice as sequence data, and a frequency of an input singing voice signal is detected in real time. Frequency detecting means, and impulse generating means for generating an impulse train at intervals according to the frequency detected by the frequency detecting means,
Reading means for reading the sequence data in accordance with the progress of the singing; and converted speech synthesis means for shaping the frequency component of the impulse train generated by the impulse generating means with the formant read by the reading means and outputting as an audio signal, It is characterized by having.

According to a second aspect of the present invention, there is provided a sound volume detecting means for detecting the volume of the inputted singing voice signal in real time, and a volume of the voice signal synthesized by the converted voice synthesizing means. And sound volume control means for controlling the sound volume to the detected level.

According to the first aspect of the present invention, the storage means stores, as sequence data, a change in a formant of a voice singing a song by a specific person such as an original singer. This sequence data is obtained by extracting a singing voice formant for each predetermined frame cycle and storing the extracted singing voice in a time-series manner. The time-series data in which the characteristics of the specific person's singing voice are stored as the song progresses It is.

On the other hand, a singing voice signal of a normal singer input from a microphone or the like is processed by frequency detecting means, and the frequency is detected. The impulse generation means
An impulse train is generated at intervals according to this frequency. The impulse train has a wide frequency spectrum as shown in FIG. 4A, and includes the singing frequency of the ordinary singer as information. The shaping means forms the frequency spectrum of the impulse train so as to have the frequency characteristics of the formant read from the storage means, and synthesizes the singing voice converted to the specific person's timbre at the singing frequency of the singer. Since the storage means stores the change of the formant as sequence data as described above, the converted singing voice produces the same timbre change as the timbre change when the specific person sings, and the specific person sings again. A singing voice like that described above can be obtained. When converting a male voice into a female voice, the interval between impulse trains generated is reduced by half to increase the octave, and when converting a female voice into a male voice, the interval between impulse trains generated is performed. May be doubled to lower the octave.

Further, in the invention according to claim 2, the sound volume detecting means detects a sound volume from the singing voice signal of the ordinary singer, and the sound volume controlling means converts the sound signal output by the shaping means to the sound volume detecting means. Is controlled so that the volume becomes the detected volume. This makes it possible to synthesize a singing voice having the voice quality of the specific person at the frequency and volume of the singer.

[0012]

Embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram of a karaoke apparatus to which the present invention is applied. The karaoke apparatus includes a karaoke apparatus main body 1, a control amplifier 2, an audio signal processing apparatus 3, an LD changer 4, a speaker 5, a monitor 6, a microphone 7, and an infrared remote controller 8.
CPU of karaoke apparatus main body 1 for controlling the operation of the entire apparatus
ROM 10, RAM 12, and 10 via an internal bus.
A hard disk storage device 17, a communication control unit 16, a remote control receiving unit 13, a display panel 14, a panel switch 15,
Sound source device 18, voice data processing unit 19, character display unit 2
0, the display control unit 21 is connected, and the control amplifier 2, the audio signal processing device 3, and the LD changer 4, which are the external devices, are connected via an interface.

The ROM 11 stores a starting program and the like necessary for starting the apparatus. A system program, an application program, and the like for controlling the operation of the apparatus are stored in the hard disk storage device 17. The application program is a karaoke performance program or the like. When the power of the karaoke apparatus is turned on, a system program and a karaoke performance program are read into the RAM 12 by the above-mentioned startup program. In addition to the above system programs and application programs, the hard disk storage device 17 stores music data for about 10,000 karaoke songs.

The communication control unit 16 downloads music data and the like from the distribution center via the ISDN line, and writes the data to the hard disk storage device 17. This write operation is performed by DMA
The processing is directly performed on the hard disk storage device 17 using a circuit.

The remote controller 8 is provided with various key switches such as a numeric keypad, and when a user operates these switches, a code signal corresponding to the operation is output by infrared rays. The remote control receiver 13 receives the infrared signal transmitted from the remote controller 8, restores the code signal, and inputs the code signal to the CPU 10. The remote control device 8 has a voice conversion mode switch and a memory mode switch. The voice conversion mode is a mode in which the voice of the singer is converted into the voice of the original singer, and is turned on / off by operating the voice conversion mode switch. The hamori mode is a mode for adding a harmony singing voice to the singer's main melody singing, and is turned on / off by operating the hamori mode switch.

The display panel 14 is provided with the karaoke apparatus main body 1.
, And includes a matrix display for displaying the number of the music currently being played and the number of reserved music, an LED group for displaying currently set keys and a tempo, and the like. The panel switch 15 includes a numeric keypad, a key change switch, and a tempo change switch similar to those of the remote controller 8 for inputting a music number.

The tone generator 18 forms a tone signal based on tone track data of music data. The tone track has a plurality of tracks, and the tone generator 18 simultaneously forms tone signals of a plurality of parts based on the data. The audio data processing unit 19 forms an audio signal having a specified length and a specified pitch based on audio data included in the music data. The sound data is a signal waveform that is difficult to form electronically, such as a human voice such as a back chorus, and is stored as it is as a PCM signal. The tone signal formed by the sound source device 18 and the sound signal reproduced by the sound data processing unit 19 are input to the control amplifier 2.

A microphone 7 is connected to the control amplifier 2, and a singing voice signal of a karaoke singer is input. In the normal mode, the control amplifier 2
After giving a predetermined effect such as an echo to the karaoke performance sound, the back chorus, and the singing voice signal, the karaoke sound is mixed with a predetermined balance, amplified, and output to the speaker 5. On the other hand, in the voice conversion mode, the singing voice signal input from the microphone 7 is output to the voice signal processing device 3 without processing. Then, the converted singing voice signal re-input from the voice signal processing device 3 is given an effect as described above, mixed and amplified, and then output from the speaker 5.

The voice signal processing device 3 converts the singing voice signal input from the control amplifier 2 into a voice signal of the original singer in the voice conversion mode. This voice conversion function will be described in detail in the description of FIG. In addition, the audio signal processing device 3 detects the frequency of the singing audio signal in the hamori mode, and generates a singing audio signal of the harmonic melody part by frequency-converting the frequency. These voice conversion mode operation and hamori mode operation can be processed in parallel.

The character display unit 20 generates a character pattern such as a song title or lyrics based on the input character data. The LD changer 4 as an external device reproduces a moving image as a background image based on the image selection data input from the CPU 10. The video selection data is determined based on genre data or the like written in the header of the music data. The display control unit 21 superimposes a character pattern such as lyrics input from the character display unit 20 on the background image input from the LD changer 4 in superimposition and displays the superimposed character pattern on the monitor 6.

FIG. 2 is a diagram showing a configuration of music data used in the karaoke apparatus. The music data includes a header, a musical tone track, a guide melody track, a harmonic melody track, a lyrics track, a voice track, an effect control track, a formant sequence track, and a voice data section. The header is a part in which data relating to the attribute of the music data is written, and data such as a music title, a genre, an announcement date, and a music performance time are written therein. Each track from the musical tone track to the effect control track is composed of M pieces of event data and duration data indicating a time interval between the event data.
It is described in IDI format. Lyrics track ~
The data of the effect control tracks are not tone data, but these tracks are also described in MIDI format in order to unify the implementation and facilitate the work process.

The formant sequence track stores the formants of the singing voice when the original singer sang this song as time-series data every 10 ms. A formant is a dominant frequency component on the voice spectrum, and reflects the shape and size of the oral cavity. Formants are first in order of lower frequency
Formants, second formants,..., Mainly up to the third formant contribute to phonological properties. One formant data recorded in the formant sequence track includes first to third formant frequencies and their levels. If the formant frequency is represented by 16 bits (2 bytes) and the level is represented by 8 bits (1 byte), one formant data including the first to third formant frequencies and levels becomes 9 bytes. Therefore, the data size of the formant sequence track in which one formant data is written every 10 ms is about 150 k in the case of a song whose singing time excluding the prelude, interlude, etc. is 3 minutes.
Become bytes.

The tone track is composed of a plurality of tracks for driving the tone generator 18 to form a plurality of tone signals. In the guide melody track, data of the main melody of the karaoke song, that is, the melody to be sung by the singer is written. Harmony melody tracks include
The data of the harmony melody part is written. The lyrics track is a track that stores sequence data for displaying lyrics on the monitor 6. The event data of the lyrics track is composed of the character code of the lyrics and data indicating the display position thereof. The audio control track is a track for specifying the utterance timing of the audio data group stored in the audio data section. PCM data such as human voice is stored in the audio data section, and the event data of the audio control track specifies which audio data is reproduced at the event timing. In the effect control track, effect control data for controlling the control amplifier 2 is written. Control amplifier 2
Gives a reverberation effect such as reverb to a tone signal based on this effect control data.

When the karaoke performance starts, the data of each track is read out in parallel according to the clock and output to the corresponding processing unit. The event data of the musical sound track is output to the tone generator 18, the data of the lyrics track is output to the character display unit 20, and the data of the effect control track is output to the control amplifier 2. And the formant data of the formant sequence track is
It is read out every 10 ms and output to the audio signal processing device 3.

FIG. 3 is a block diagram showing functions of the audio signal processing device 3. The audio signal processing device 3 has a built-in DSP and processes audio signals by a microprogram. In this figure, the functions of the microprogram are shown in a block diagram. FIG. 4 is a diagram showing a signal flow in the functional block diagram of FIG. The singing voice signal input from the microphone 7 via the control amplifier 2 is converted into digital data by the A / D converter 30. This digital data
It is input to the sound volume extraction unit 31. Pitch / volume extraction unit 31
Extracts pitch data and volume data from the digital data. The pitch data is stored in the impulse generator 3
2 and the volume data is output to the gain controller 34. The impulse generator 32 is a part that functions as a sound source in this device, and is a generator that generates an impulse train at predetermined time intervals. The time interval is controlled by pitch data input from the pitch / volume extracting unit 31, and an impulse train is generated at intervals of the pitch data. That is, when pitch data of 440 Hz is input, an impulse train is generated at intervals of 1/440 seconds. In the case of converting a female voice into a male voice, when pitch data of 440 Hz is input, this frequency is halved to generate an impulse train at an interval of 1/220 seconds, and the female voice is output. Is converted to a male voice, when 220 Hz pitch data is input,
This frequency is doubled to generate an impulse train at intervals of 1/440 seconds. This time interval is determined by the pitch volume extraction unit 3
Each time new pitch data is input from 1, it is updated with this pitch data. The impulse train generated by the impulse generator 32 has the shape shown in the left column of FIG. 4A, and the amplitude spectrum of the impulse train is shown in FIG.
(A) The shape shown in the right column is obtained. This impulse train is input to the filter 33. In the filter 33, formant data read from the formant sequence track of the music data is set as a filter coefficient. The formant data input from the CPU 10 every 10 ms is stored in the formant data buffer 36, and the contents of the buffer 36 are supplied to the filter 33. As described above, the formant data is characteristic frequency component information that determines phonological characteristics in the singing voice of the original singer of the karaoke song, and is information having contents illustrated in the left column of FIG. The singing voice of the original singer is expressed by this formant change updated every 10 ms. The filter 33 is set to have a somewhat broad filtering characteristic with this frequency component as the center frequency (see the right column in FIG. 4B). By passing the impulse train through the filter 33, characteristic frequency components of the original singer are cut out from a broad spectrum and shaped into a waveform that can be heard by the voice of the original singer (see FIG. 4C). . The shaped waveform data is input to the gain controller 34. The gain controller 34 multiplies the waveform data by the volume data extracted by the pitch / volume extraction unit 31 to add an envelope having the same volume as the singing voice signal. This waveform data is input to the D / A converter 35 and converted into an analog signal. This analog signal is re-input to the control amplifier 2 as a singing voice signal converted into the voice of the original singer.

In the voice conversion mode, the control amplifier 2 does not mix the singing voice signal input from the microphone 7 with the karaoke performance sound, but mixes the singing voice signal converted into the voice of the original singer with the karaoke performance sound. Then, sound is emitted from the speaker 5.

When the voice conversion mode and the hamori mode are set at the same time, the frequency of the converted singing voice signal may be converted to generate a harmony melody voice signal.

As described above, by filtering the broad spectrum of the impulse train with the original singer's formant, a singing voice signal of the original singer's timbre can be created.

In this embodiment, an example is shown in which the audio signal processing apparatus of the present invention is applied to a karaoke apparatus.
It is also possible to apply to other devices.

[0030]

As described above, according to the first aspect of the present invention,
The change of the formant of the singing voice of a specific person such as the original singer is stored as sequence data, and the impulse train generated according to the frequency detected from the input singing voice signal is read out in accordance with the singing. By filtering with formant data that changes in a series, a singing voice signal having a timbre change accompanying the progress of the tune when the specific person sings can be created with the same singing turn as the singing voice signal.

Further, according to the invention of claim 2, by controlling the volume of this signal based on the volume data detected from the singing voice signal, the voice of a specific person singing in exactly the same way as the singing voice signal is obtained. A singing voice signal can be created, and if this signal is applied to a karaoke device or the like, karaoke singing can be further enhanced.

[Brief description of the drawings]

FIG. 1 is a block diagram of a karaoke apparatus to which the present invention is applied;

FIG. 2 is a diagram showing a configuration of music data used in the karaoke apparatus.

FIG. 3 is a functional block diagram of an audio signal processing device according to an embodiment of the present invention;

FIG. 4 is a diagram showing signals processed by respective processing units of the audio signal processing device.

[Explanation of symbols]

3 audio signal processing device, 7 microphone, 31 pitch / volume extraction unit, 32 impulse generation unit, 33 filter
36 ... Formant data buffer

Claims (2)

[Claims] 1. A storage means for storing a change in formant of a specific person's singing voice as sequence data; a frequency detecting means for detecting a frequency of an input singing voice signal in real time; a frequency detected by the frequency detecting means An impulse generating means for generating an impulse train at intervals according to the following: reading means for reading out the sequence data in accordance with the progress of the singing; a formant for reading out frequency components of the impulse train generated by the impulse generating means And a converted voice synthesizing unit configured to output the converted voice signal as a voice signal. 2. A volume detector for detecting the volume of the input singing voice signal in real time; and a volume for controlling the volume of the voice signal synthesized by the converted voice synthesizer to the volume detected by the volume detector. The audio signal processing device according to claim 1, further comprising: a control unit.