JP3173310B2 - Harmony generator - Google Patents

Abstract

PURPOSE: To provide a KARAOKE device which generates a harmony voice signal even unless the pitch of a text voice signal is detected. CONSTITUTION: An input singing voice signal to a DSP 30 for voice processing is supplied to a peak detection part 41, a phoneme detection part 42, a mean sound volume detection part 43, and a multiplier 45. The singing voice signal is multiplied by a window function through the multiplier 45 and cut waveform element data of one cycle are stored in a memory 46. A readout control part 48 for harmony data accesses the memory 46 and the signal obtained by repeatedly reading waveform element data out at intervals corresponding to a harmony frequency is the harmony voice signal. The window function is one cycle long in terms of melody data and the timing of starting the window function so controlled that the peak detected by the peak detection part 41 is at the center of the window function. A window function generation part 44 cuts the waveform element data at intervals of tens of ms and waveform element data corresponding to a timbre are written in the memory 46: when phonemes change, a phoneme detection part 42 transmits that to the window function generation part 44 to generate the window function.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a harmony generation device for generating a harmony voice signal for a singing or performance voice signal, and more particularly to a harmony generation device capable of generating an accurate harmony voice signal even when the frequency of the voice signal cannot be detected. Related to the device.

[0002]

2. Description of the Related Art In a karaoke apparatus or the like which is currently in practical use, singing voices of a singer are harmony (for example, three times higher than a melody) in order to excite the singing or to make the singing sound better. Some have a function of adding and outputting a melody sound. As a harmony addition function, there is a function that shifts the frequency of the singer's voice signal input from the microphone to form a harmony voice signal in order to create a harmony voice with a tone very similar to the singer's singing voice and at the same tempo. General.

[0003]

However, the harmony voice signal adding technique described above detects the fundamental frequency of the singing voice signal of the singer and shifts it based on this fundamental frequency. If the frequency of the signal could not be detected, there was a drawback that a harmony audio signal could not be formed.

[0004] It is an object of the present invention to provide a harmony generation device that can form a harmony audio signal even when the frequency of an input audio signal cannot be detected.

[0005]

According to a first aspect of the present invention, there is provided an audio signal input means for inputting a singing or performance audio signal, and a melody information supply for supplying melody information as frequency information of the audio signal. Means, a harmony frequency supply means for supplying a harmony frequency which is a frequency harmonizing with the frequency of the audio signal, and waveform element data extraction means for cutting out a waveform for one cycle of the melody information from the audio signal as waveform element data. Harmony forming means for forming a harmony sound signal by repeatedly reading out the waveform element data at the harmony frequency.

According to a second aspect of the present invention, there is provided an audio signal input means for inputting a singing or performance audio signal, a frequency detecting means for detecting a frequency of the audio signal input from the audio signal input means, Melody information supply means for supplying melody information which is frequency information of an audio signal, harmony frequency supply means for supplying a harmony frequency which is a frequency cooperating with the singing or performance, and the frequency detection means detects the frequency of the audio signal When it is completed, a waveform for one cycle of the frequency is cut out from the audio signal as waveform element data, and when the frequency detection means cannot detect the frequency of the audio signal, the waveform for one cycle of the melody information is converted into a waveform. Waveform element data extraction means for extracting as element data; and repeatedly reading out the waveform element data at the harmony frequency. Characterized in that a harmony forming means for forming a harmony audio signal Ri.

According to a third aspect of the present invention, there is provided a melody information supply means for storing a plurality of pieces of melody information in time series, and a melody information storage means for sequentially reading the melody information from the melody information storage means, A harmony frequency supply means, and a harmony frequency supply means for storing a plurality of harmony frequencies in a time series; and a harmony frequency reading means for sequentially reading harmony frequencies from the harmony frequency storage means. And means for supplying to the forming means.

[0008]

According to the first aspect of the present invention, a harmony generating device receives a singing or performance audio signal and cuts out a waveform for one cycle of melody information from the audio signal as waveform element data. The melody information is supplied from the melody information supply means. Although the waveform element data for one cycle has the length of the fundamental frequency, its harmonic components are included in more than one cycle (two or more cycles), so that the overtone characteristic (formant) of the input audio signal is Has been saved. The waveform element data is repeatedly read at the harmony frequency to form a harmony audio signal. The harmony frequency is a frequency that harmonizes with the frequency of the input audio signal, for example, 3 degrees (4/3 times) or 5
A frequency having a degree (3/2 times) is supplied from the harmony frequency supply means. Since the harmony sound signal repeats the waveform element data at the harmony frequency, the fundamental frequency becomes the harmony frequency, but since the harmonic component of the waveform element data is included as it is, the formant of the input sound signal is reproduced. And the same tone.

The harmony generating device according to claim 2 is
The frequency of the input audio signal is detected, and one cycle, which is a unit for extracting the waveform element data, is determined based on this frequency. If this cannot be detected, one cycle of the melody information supplied from the melody information supply means is set as one cycle of the cutout unit, as in the first aspect. If one cycle is determined based on the input voice signal, waveform element data having a more ideal length can be cut out. Even if the frequency of the singing is unstable and the frequency cannot be detected, the melody information can be obtained. It is possible to cut out waveform element data based on
The formation of the harmony audio signal is not interrupted.

According to the third aspect of the present invention, the melody information supply means stores a plurality of pieces of melody information in time series, and the melody information is sequentially read from the melody information storage means, and the waveform element data extraction means is provided. Harmony frequency supply means, and a harmony frequency storage means for storing a plurality of harmony frequencies in chronological order, and harmony frequencies are sequentially read from the harmony frequency storage means and supplied to the harmony formation means. And means. This makes it possible to supply melody information and harmony frequencies synchronized with the progress of singing or performance.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A karaoke apparatus with a harmony addition function according to an embodiment of the present invention will be described with reference to the drawings. This karaoke device is a sound source karaoke device, and has a communication function,
And it has a harmony addition function. A sound source karaoke device is a karaoke device that generates a karaoke performance sound by driving the sound source device with music data. The music data is sequence data composed of a plurality of tracks, such as a performance data string that specifies a pitch and a sound generation timing. The communication function is a function that is connected to the host station via a communication line, downloads music data from the host station, and stores the music data in the hard disk device 17 (see FIG. 1). The hard disk drive 17
Music data for hundreds to thousands of songs can be stored. The harmony adding function is a function of adding a harmony voice signal having a pitch of 3rd or 5th to the singing voice signal of the singer. The harmony voice signal is generated by pitch-shifting the singer's singing voice.

FIG. 1 is a block diagram of the karaoke apparatus. The CPU 10 for controlling the operation of the entire apparatus includes a ROM 11, a RAM 12, a hard disk storage (HDD) 17, an ISDN controller 16, a remote control receiver 13, a display panel 14, a panel switch 15,
Sound source device 18, audio data processing unit 19, effect DSP2
0, a character display unit 23, an LD changer 24, a display control unit 25, and a voice processing DSP 30.

The ROM 11 stores a system program, an application program, a loader, and font data. The system program is a program that controls the basic operation of the device and data transmission / reception with peripheral devices. The application program is a peripheral device control program, a sequence program, or the like. During a karaoke performance, a sequence program is executed by the CPU 10 to generate musical tones and reproduce images based on music data. The loader is a program for downloading music data from the host station. The font data is for displaying lyrics, song titles, and the like, and stores fonts of a plurality of character types, such as Mincho and Gothic. In the RAM 12, a work area is set. A music data file is set in the HDD 17.

The ISDN controller 16 is a controller for communicating with a host station via an ISDN line. The ISDN controller 16 downloads music data and the like from the host station. Further, the ISDN controller 16 has a built-in DMA circuit, and writes the downloaded music data and application program directly to the HDD 17 without passing through the CPU 10.

The remote control receiver 13 receives the infrared signal sent from the remote control 31 and restores the data. The remote controller 31 includes a command switch such as a music selection switch, a numeric key switch, and the like. When the user operates these switches, an infrared signal modulated with a code corresponding to the operation is transmitted. The display panel 14 is provided on the front of the karaoke apparatus, and displays the currently playing music code and the number of reserved music. The panel switch 15 is provided on a front operation unit of the karaoke apparatus, and includes a music code input switch, a key change switch, and the like.

The sound source device 18 has a CPU for performing karaoke.
A tone signal is formed on the basis of the event data input from. The event data is stored in a music track of the music data. Since a plurality of tone tracks are set as shown in FIG. 4, a plurality of event data are also input in parallel. The sound source device 18 receives these data and simultaneously forms a plurality of tone signals. The audio data processing unit 19 forms an audio signal having a specified length and a specified pitch based on audio data that is ADPCM data included in the music data. The audio data is obtained by directly digitizing and storing a signal waveform, such as a back chorus or a model singing sound, which is hardly generated electronically by the sound source device 18.

On the other hand, a singing voice signal input from a singing microphone 27 is amplified by a preamplifier 28 and converted to a digital signal by an A / D converter 29, and then converted to a digital signal for effect.
It is input to the SP 20 and the DSP 30 for audio processing. In addition, main melody information and harmony information are input from the CPU 10 to the voice processing DSP 30. DS for audio processing
P30 cuts out waveform element data from the singing voice signal of the singer based on these pieces of information, and combines the waveform element data to form a harmony voice signal. This harmony audio signal is output to the effect DSP 20.

The effect DSP 20 includes a tone signal formed by the tone generator 18, a sound signal formed by the sound data processing section 19, a singing sound signal converted by the A / D converter into a digital signal, and a harmony sound formed by the sound processing DSP 30. A signal is input. The effect DSP 20 gives effects such as reverb and echo to these input audio signals and tone signals. The type and degree of the effect provided by the effect DSP 20 are controlled based on the event data (DSP control data) of the effect track of the music data. The DSP control data is input to the effect DSP 20 at a predetermined timing by the CPU 10 based on the DSP control sequence program. The tone signal and audio signal to which the effect has been added are converted into analog signals by the D / A converter 21 and then output to the amplifier / speaker 22. The amplifier / speaker 22 amplifies this signal and emits sound.

The character display unit 23 generates character patterns such as song titles and lyrics based on the input character data.
The LD changer 24 reproduces the background video of the corresponding LD based on the input video selection data (chapter number). The video selection data is determined based on the genre data of the karaoke song or the like. The genre data is written in the header of the music data, and is read by the CPU 10 when the karaoke performance starts. CPU
10 determines which background video is to be reproduced based on the genre data, and outputs video selection data specifying the background video to the LD changer 24. The LD changer 24 contains about five (120 scenes) laser disks and can reproduce about 120 scenes of background video. One background video is selected from among them according to the video selection data, and is output as video data. The character pattern and the video data are input to the display control unit 25. The display control unit 25 superimposes these data in superimposition and displays them on the monitor 26.

Next, the structure of music data used for karaoke performance in the karaoke apparatus will be described with reference to FIGS. FIG. 3 is a diagram showing a configuration of music data. FIGS. 4 and 5 are diagrams showing a detailed configuration of music data.

In FIG. 3, one piece of music data includes a header, a musical tone track, a main melody track, a harmony track, a lyrics track, an audio track, an effect track, and an audio data section.

The header is a portion in which various data relating to the music data are written, and data such as a music title, a genre, a release date, and a music playing time (length) are written therein. The CPU 10 determines a background video to be displayed on the monitor 26 based on the genre data when executing the main sequence program, and transmits a chapter number of the video to the LD changer 24. The method of determining the background image is to select an image of a snowy country in the case of enka on the theme of winter, and to select an image of a foreign country in the case of pops.

As shown in FIGS. 4 and 5, each track from the musical tone track to the effect track is composed of sequence data including a plurality of event data and duration data Δt indicating a time interval between the event data. The event data of each track is read out by the CPU 10 based on a sequence program during a karaoke performance.
The sequence program is executed at a predetermined tempo clock at Δt.
This is a program for reading out and outputting to a predetermined processing unit, assuming that when the time Δt is counted up, it is the readout timing of the event data subsequent thereto.

On the musical sound track, tracks of various parts including a melody track and a rhythm track are formed. The CPU 10 outputs to the tone generator 18 the event data read by the tone sequence program. The sound source device 18 selects a sounding channel based on the channel designation data included in the event data, and executes the event for the sounding channel.

In the main melody track, the main melody of the karaoke song, that is, the sequence data of the melody to be sung by the singer is written. This data is input from the CPU 10 to the DSP 30 for audio processing. DSP 30 for audio processing
Extracts waveform element data from the singing voice signal based on this data. The configuration of the harmony track is the same as that of the main melody track, and the sequence data of the harmony melody of this karaoke song is written. This data is also input from the CPU 10 to the voice processing DSP 30. The voice processing DSP 30 determines the frequency (pitch) of the harmony voice signal based on the data.

The lyrics track is a track in which sequence data for displaying lyrics on the monitor 26 is stored. Although this sequence data is not tone data, this track is also described in the MIDI data format in order to unify the implementation and facilitate the work process. The data type is system exclusive
It is a message. In the data description of the lyrics track, usually, one line of lyrics is treated as one piece of lyrics display data. The lyrics display data is composed of character data (character codes and display coordinates of the characters) of one line of lyrics, display time of the lyrics (usually around 30 seconds), and wipe sequence data. The wipe sequence data is sequence data for changing the display color of the lyrics in accordance with the progress of the song. The timing of changing the display color (the time from when the lyrics are displayed) and the change position (coordinates) ) Is data sequentially recorded over the length of one line.

The audio track is a sequence track for specifying the generation timing of the audio data n (n = 1, 2, 3,...) Stored in the audio data section. The voice data section stores human voices such as back chorus and harmony singing that are difficult to synthesize by the sound source device 18.
In the audio track, the audio designation data and the reading interval of the audio designation data, that is, the duration data Δt that designates the timing at which the audio data is output to the audio data processing unit 19 and the audio signal is formed, are written. The voice designation data includes a voice data number, pitch data, and volume data. The audio data number is an identification number n of each audio data recorded in the audio data section. The pitch data and the volume data are data indicating the pitch and volume of the audio data to be formed. In other words, a back chorus without words, such as "Ah" or "Wawa Wawa", can be used many times by changing the pitch or volume. The pitch and volume are shifted based on and used repeatedly. The audio data processing unit 19 sets the output level based on the volume data, and sets the interval of the audio signal by changing the reading interval of the audio data based on the interval data.

In the effect track, DSP control data for controlling the effect DSP 20 is written. The effect DSP 20 applies reverberation or other reverberation effects to signals input from the sound source device 18, the audio data processing unit 19, and the audio processing DSP 30. DSP
The control data includes data designating the kind of the effect and data of a change amount thereof.

FIG. 2 is a diagram for explaining the operation of the DSP 30 for audio processing. The voice processing DSP 30 forms a harmony voice signal for the input singing voice signal based on a built-in microprogram, and this microprogram can be represented as shown in FIG.

The singing voice signal input from the microphone 27, amplified by the amplifier 28, and converted into a digital signal by the A / D converter 29 is converted into a period detecting section 40, a peak detecting section 41, and a phoneme detecting section 42 of the voice processing DSP 30. , Are input to the average volume detector 43 and the multiplier 45.

The cycle detecting section 40 detects the cycle T based on the waveform of the input singing voice signal (see FIG. 6A). Further, the cycle detecting unit 40 receives the main melody data from the CPU 10. The main melody data is data representing the frequency of the main melody. When the pitch of the singing voice signal becomes indefinite at a consonant part or a transition of a sound, the period detecting unit 40
Outputs period information obtained by estimating the period of the singing voice signal from the main melody data. The cycle detector 40 outputs the detected cycle information to the peak detector 41 and the window function generator 44.

The peak detector 41 detects a local peak in one cycle of the input singing voice signal (see FIG. 6A). The interval of one cycle is determined by the cycle information input from the cycle detector 40. The peak detector 41 outputs the detected peak timing information to the window function generator 44.

The phoneme detector 42 detects a phoneme break based on a level break or a change in frequency component of the input singing voice signal. Here, a phoneme refers to a section in which pronunciation is divided into individual consonants and vowels. In FIG. 6 (B), the lyrics “Ashiyano” are “A” and “K”, respectively.
It consists of five syllables, "shi", "ya", and "no", and these syllables are "a", "k", "a", "sh", "i", "y"
It can be divided into nine phonemes "a", "n" and "o". Between each syllable, there is a break in which the level decreases, and the phoneme is divided based on the fact that the consonant has a non-periodic waveform like a white noise, while the vowel has a periodic waveform. When detecting a break between phonemes, the phoneme detection unit 40 outputs information indicating that the break is a break to the window function generation unit 44.

The average volume detector 43 detects the average volume by smoothing the amplitude level of the input singing voice signal. The average volume detection unit 43 uses the detected average volume information as the volume control unit 5
Output to 0.

The window function generator 44 outputs a window function as shown in FIG. This window function is output to the multiplier 45. Since the singing voice signal is input to the multiplier 45 as described above, only the window function portion of the singing voice signal is cut off (see FIG. 6C). It is desirable to use a function that is differentially continuous from the start to the end as the window function. If a differentially continuous function is used, noise is not generated at the boundary of the cut even if only a part (one cycle) of the singing voice signal is cut. Therefore, in this DSP 30, sin ² (ωt / 2)
(T = 0 to T: T is one cycle of the singing voice signal). As is apparent from this equation, the length of the window function is one cycle of the singing voice signal. The length of one cycle is given by the cycle information input from the cycle detector 40. Further, the window function generating section 44 repeatedly generates a window function at an appropriate interval of several tens ms to 100 ms. The reason why the window function is generated with a certain time interval is that if the same waveform element data is not continued to some extent, the timbre of the waveform element data will not be recognized by the listener. On the other hand, whenever information indicating a break between phonemes is input from the phoneme detection unit 42, a window function is generated to cut out waveform element data of a new phoneme. This is because the timbre changes completely when the phoneme is switched, so that it follows the change. Also,
The start timing of the window function is controlled so that the peak input from the peak detection unit 41 is located at the center of the window function, that is, the midpoint between the peak and the next peak, that is, the lowest level point.

The waveform element data cut out by the window function as described above is obtained by storing the timbre of the singing voice signal, that is, the formant (overtone component) almost as it is.

The window function generator 44 outputs information indicating that the window function is to be generated and its length to the write controller 47 at the same time as generating the window function. The write control unit 47 inputs a write address that advances in synchronization with the sampling clock (44.1 kHz) from the start to the end of the window function in accordance with this information. By inputting the write address, the waveform element data cut out by the multiplier 45 is stored in the memory 46.

With the above configuration, the memory 46 stores the waveform element data for one cycle of the singing voice signal at that time. By repeatedly reading out the waveform element data at an arbitrary cycle, it is possible to synthesize an audio signal having the fundamental frequency of the arbitrary cycle and having the timbre (overtone configuration) of the singing voice signal. Therefore, by repeatedly reading out the waveform element data from the singing voice signal at a harmony frequency cycle having a consonant frequency relationship such as 3rd or 5th, a harmony voice signal having the same frequency and the same timbre as the singing voice signal is formed. be able to. In this embodiment, it is assumed that the frequency of the singing voice signal substantially matches the frequency of the main melody data, and that the harmony voice signal is synthesized using the frequency of the harmony melody data stored in the music data. I have. As described with reference to FIGS. 3 and 4, the harmony melody data is sequence data having a frequency relationship that harmonizes with the main melody data, which is the sequence data stored in the main melody track, and is stored in the harmony track. It is.

The read control of the memory 46 is performed by a read control unit 48. Harmony data is input from the CPU 10 to the read control unit 48. The harmony data is event data read from the harmony track of the musical sound data. The read control unit 48 repeatedly accesses the memory 46 at the frequency of the harmony data. That is, the waveform element data is repeatedly read out only for the frequency of the harmony melody data in one second. When the harmony melody has a lower frequency than the main melody, the harmony sound signal has a waveform in which the waveform element data is arranged at intervals of T1 longer than the data length T as shown in FIG. . When the harmony melody has a higher frequency than the main melody, the harmony sound signal has a waveform in which the waveform element data are arranged so as to overlap each other at intervals of T2 shorter than the data length T as shown in FIG. Becomes As a result, the fundamental frequencies of the harmony voice signal are 1 / T1 and 1 / T2, but the harmonic components in each waveform element data are stored as they are, so that a formant similar to the singing voice signal is formed. Further, since the window function is differentially continuous, no noise is generated.

The harmony sound signal formed by repeatedly reading the waveform element data from the memory 46 as described above is input to the multiplier 51 via the switch 49. The multiplier 51 receives volume control data from the volume controller 50. The volume control unit 50 is provided with the average volume detection unit 43.
And average volume information of the singing voice signal, and generates volume control data based on the average volume information. The volume control data is set to, for example, a value of 80% of the average volume information. The harmony sound signal whose volume has been controlled by the multiplier 51 is output to the effect DSP 20. The switch 49 is used when the output is forcibly set to 0 at a break between phrases.

With the above-described operation of the voice processing DSP 30, it is possible to form a harmony voice signal that preserves the timbre of the singer's singing voice signal as it is, and to detect the period (frequency) of the singing voice signal. However, the harmony sound signal can be formed without any trouble. That is, in the case of a normal singer, it is considered that there is not much difference between the frequency of the singing voice signal and the frequency given by the main melody data. By re-synthesizing the harmony voice signal, it is possible to form a harmony voice signal that substantially maintains the timbre of the singer.

As described above, in the above embodiment, the audio processing D
A period detection unit 40 is provided in the SP 30 to detect the period from the singing voice signal and determine the length of the window function. Only when this period cannot be detected, the length of the window function is determined using the main melody data input from the CPU 10. However, as shown in FIG. 8, if the determination of the length of the window function is entirely performed on the main melody data without determining the period of the singing voice signal by the DSP 30, the period detection unit 40 It becomes unnecessary. In this case, the peak detection unit 41 also detects a local peak based on a cycle (frequency) given from the main melody data. In this figure, the peak detector 41
The window function generator 44 also directly inputs the main melody data to determine the period. Since the main melody data is originally frequency data, it is extremely easy to determine the period from this, and it is not necessary to provide a separate period calculation unit. As described above, in the case of ordinary singing, there is almost no difference between the frequency of the singing voice signal and the frequency given from the main melody data, so that this configuration does not pose a practical problem.

Furthermore, even if the phoneme detecting unit 42 for detecting the switching of phonemes of the singing voice signal is omitted, the window function is repeatedly generated at an appropriate interval (for example, 50 ms), so that the auditory deviation is not so large. The harmony voice signal can be made to follow the singing voice signal without giving an impression that the harmony voice signal does. Further, since the window function is differentially continuous, it is possible to cut out the waveform element data without generating noise, regardless of the phase from which the waveform signal is cut out. Therefore, if the peak of the singing voice signal is suppressed by coming to the end of the window function and the overtone structure is allowed to slightly change, the peak detecting unit 41 is omitted and the phase relationship with the singing voice signal is taken into consideration. Instead, a window function may be generated. Further, if there is no need to follow the singing volume of the singer, the average volume detection unit 43, the volume control unit 50, and the multiplier 51 are also unnecessary. Therefore,
The minimum basic function of the present invention, "forming a harmony voice signal by extracting waveform element data from a singing voice signal using a window function and repeating this waveform element data at a predetermined cycle" is realized. Figure 9
The configuration shown in FIG.

In the above embodiment, the audio processing DSP 30 for forming the harmony audio signal is incorporated in the karaoke apparatus. However, the audio processing DSP 30, the A / D converters 29 before and after the audio processing DSP 30, and the D / A converter (see FIG. 2, see the broken line in FIG. 8) to form an external device (harmony generating device) as shown in FIG. In this case, as shown in the figure, the main melody information and the harmony information are received from the karaoke performance device 61, and the microphone 62
, The singer's analog singing voice signal is input, and a harmony voice signal is generated based on these signals. The karaoke performance device 61 performs music selection (selection of music data) and pitch shift (rewrite of pitch information of event data to be read) based on operation of the operation panel, outputs musical sound data and the like to a sound source, and lyrics data and the like. Is output to the image control unit.

As described above, in this embodiment, the karaoke apparatus for processing a singing signal has been described. However, the present invention is not limited to this, and can be applied to a tone signal generated by playing a musical instrument.

The invention disclosed in the embodiments but not described in the claims is described in the dependent form as follows.

[4] The waveform element data extracting means is a means for repeatedly executing the extraction of the waveform element data at intervals sufficiently longer than one cycle defined by the melody information. The harmony generation device according to item 1. In the present invention, the extraction of the waveform element data is repeatedly executed at intervals sufficiently longer than one cycle defined by the melody information. The harmony forming means forms a harmony sound signal with the same waveform element data until new waveform element data is cut out.Therefore, by repeating the same waveform to some extent, the listener can recognize the timbre of the waveform. It becomes easy to recognize that the tone is the same as the input voice signal.

(5) The waveform element data extracting means according to (1) or (2), wherein the waveform element data is extracted by multiplying the waveform element data by a differentially continuous window function. Harmony generator. In the present invention, the extraction of the waveform element data is performed by multiplying the waveform element data by a differentially continuous window function. As a result, the cut-out waveform element data is also differentially continuous at its start point and end point, and no noise is generated at the start point and end point even if it is reproduced.

[Claim 6] There is provided peak detecting means for detecting the peak of the waveform of the input audio signal, and the waveform element data extracting means cuts out the waveform element data so that the detected peak is located at the center thereof. The harmony generation device according to claim 5, wherein the harmony generation device is a means. In the present invention, the peak of the waveform of the input audio signal is detected,
Waveform element data is cut out such that the detected peak is at the center. Thus, a portion having a higher waveform level is located at the center of the waveform element data, and a portion having a lower waveform level is located at an end of the waveform element data. This suppresses the occurrence of noise and preserves harmonic components better. .

[Claim 7] There is provided a phoneme detecting means for detecting a switch between a consonant and a vowel of the input voice signal,
3. The harmony generating device according to claim 1, wherein said waveform element data extracting means is means for executing extraction of waveform element data when switching of phonemes is detected. According to the present invention, switching between consonants and vowels in an input audio signal is detected, and at the time of switching, waveform element data is cut out. Thereby, it is possible to quickly follow a change in the waveform of the input audio signal.

[Claim 8] A sound volume detecting means for detecting the sound volume of the input sound signal, and a sound volume control for controlling the sound volume of the harmony sound signal formed by the harmony forming means based on the detection value of the sound volume detecting means. The harmony generation device according to claim 1 or 2, further comprising means. According to the present invention, the volume of the input audio signal is detected, and the volume of the harmony audio signal is controlled based on the detected content. This makes it possible to always maintain the volume balance between the input audio signal and the harmony audio signal.

[0052]

As described above, the harmony generating apparatus of the present invention forms the harmony sound signal by repeating the waveform element data for one cycle of the input sound signal at the harmony frequency. It is possible to form a harmony sound signal of the same timbre in which the overtone component of the waveform element data of the generated sound signal is included as it is. At this time, by extracting the waveform element data based on the melody information supplied from the melody information supply means, even when the frequency of the audio signal cannot be detected, accurate waveform element data is extracted to form a harmony audio signal. can do. It is also possible to cut out the waveform element data without detecting the frequency of the audio signal at all.

[Brief description of the drawings]

FIG. 1 is a block diagram of a voice conversion karaoke apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of a voice processing DSP of the voice conversion karaoke apparatus.

FIG. 3 is a diagram showing a configuration of music data used in the voice conversion karaoke apparatus.

FIG. 4 is a diagram showing a configuration of music data used in the voice conversion karaoke apparatus.

FIG. 5 is a diagram showing a configuration of music data used in the voice conversion karaoke apparatus.

FIG. 6 is a diagram for explaining a method of extracting waveform element data from a singing voice signal.

FIG. 7 is a diagram illustrating a method of forming a harmony audio signal.

FIG. 8 is a diagram showing another embodiment of a DSP for audio processing.

FIG. 9 is a diagram showing another embodiment of a voice processing DSP.

FIG. 10 is a diagram showing a harmony generation device according to another embodiment of the present invention.

[Explanation of symbols]

30-DSP for voice processing, 44-Window function generator, 48-
Read control unit

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) G10K 15/04 302 G10H 1/00 G10H 1/10 G10L 21/04

Claims (3)

(57) [Claims] An audio signal input means for inputting a voice signal of a singing or a performance, a melody information supply means for supplying melody information which is frequency information of the audio signal, and a frequency harmonizing with the frequency of the audio signal. A harmony frequency supply unit that supplies a harmony frequency; a waveform element data extraction unit that cuts out a waveform for one cycle of the melody information from the audio signal as waveform element data;
A harmony generating means for forming a harmony audio signal by repeatedly reading the waveform element data at the harmony frequency. 2. An audio signal input means for inputting a voice signal of a singing or performance, a frequency detecting means for detecting a frequency of the audio signal input from the audio signal input means, and a melody which is frequency information of the audio signal. Melody information supply means for supplying information, harmony frequency supply means for supplying a harmony frequency that is a frequency that is in harmony with the singing or performance, and when the frequency detection means can detect the frequency of the audio signal, from the audio signal A waveform for one cycle of the frequency is cut out as waveform element data, and when the frequency detection means cannot detect the frequency of the audio signal, a waveform for one cycle of the melody information is cut out as waveform element data. Means for forming a harmony audio signal by repeatedly reading the waveform element data at the harmony frequency. Harmony generating device is characterized in that a harmony forming means for. 3. The melody information supply means includes a melody information storage means for storing a plurality of melody information in time series, and a means for sequentially reading melody information from the melody information storage means and supplying the melody information to the waveform element data extraction means. The harmony frequency supply means includes harmony frequency storage means for storing a plurality of harmony frequencies in time series, and means for sequentially reading harmony frequencies from the harmony frequency storage means and supplying the harmony frequencies to the harmony formation means. 3. The method according to claim 1, wherein
The harmony generation device according to item 1.