JPS6183599A - Singing voice synthesizer/performer - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

DETAILED DESCRIPTION OF THE INVENTION Field of the Invention The present invention relates to a singing voice synthesis performance device, and in particular generates musical instrument sounds from one of two synthesizers.
The present invention also relates to a device for generating singing voice sounds from the other synthesizer.

BACKGROUND TECHNOLOGIES AND THEIR PROBLEMS Conventionally, in electronic keyboard musical instruments, there have been known singing sound generators that generate a sound similar to a human voice (singing sound) at a pitch corresponding to the pitch of a pressed key (e.g. , Japanese Patent Publication No. 55-777
99 etc.). However, conventional singing sound generating devices cannot be used in such a way that an operator can add desired lyrics while listening to accompaniment sounds, and then generate the lyrics as a singing sound.

Therefore, the present invention uses two synthesizers, one for singing voice sounds and the other for musical instrument sounds, and while generating predetermined musical instrument sounds, the image input section inputs lyrics data. The object of the present invention is to provide a singing voice synthesis performance device that solves the above problems by generating singing voice sounds.

Means for Solving the Problems The singing voice synthesis performance device according to the present invention includes a character data input section for inputting character data, a score data input section for inputting predetermined musical score data of a specific standard, and a musical score data input section for inputting predetermined musical score data of a specific standard. A first data transfer means for sequentially outputting data, a table creation means, a pitch parameter conversion means 2, a parameter creation means, a data conversion means, a second data transfer means, a data storage means, and first and second synthesizers. , and interface means. The table creation means creates a first table by decomposing the character data into parameters having consonants and vowels as units. Further, the pitch parameter conversion means converts the pitch parameter into a pitch parameter according to the scale data in the input musical score data.

Further, the synthesis parameter creation means generates an edited and interpolated synthesis parameter by combining the pitch parameter with the formant parameter read from the memory according to the first table, and also generates an edited and interpolated synthesis parameter according to the syllable data in the musical score data. Create long tables and temporarily store them. After the synthesis parameters are converted into data of a specific standard by the data conversion means, they are stored by the storage means, while data regarding a plurality of formant frequencies are transferred by the second data transfer means, and the pitch parameters are Pitch frequency data related to the vowel part is generated and output for a period based on the time length table only during the vowel part utterance period, and is then separately supplied as a control signal to the plurality of variable frequency oscillators in the second synthesizer by the interface means. . Further, the interface means supplies the output transfer data of the first data transfer means to the first synthesizer as control signals for the plurality of variable frequency oscillators, respectively.

Operation 114: The first synthesizer reproduces musical instrument sounds at a constant speed based on the predetermined musical score data. For this desk, the second synthesizer inputs character data (voice character string data) input through the character data input section.
is converted into a parameter whose unit is the consonant and vowel of the character indicated by , and then converted into the synthesis parameter using the scale data and note value data in the musical score data, which is essentially the second is supplied to the synthesizer. Therefore, the second synthesizer produces a human voice sound, that is, a singing voice sound, with the pitch, length, and strength based on the musical score data, and of the desired character input by the character data input section. . Therefore, the first synthesizer produces musical instrument sounds according to predetermined musical score data, and the second synthesizer produces vocal sounds of arbitrary lyrics based on the user's intention. Hereinafter, the present invention will be described in more detail with reference to embodiments and drawings.

Embodiment FIG. 1 shows a block system diagram of an embodiment of the apparatus of the present invention. In the figure, an image input unit 1 is a character data input unit for inputting character strings to be lyrics into a text according to the user's intention. It consists of one arranged tablet and a light ben 20 that arbitrarily selects one character on this tablet. Light Ben 20 charges and converts the light from a single tablet on a fluorescent screen into electrical signals (character image data).
is supplied to the central processing unit (CPU) 2. The CPU 2 is connected to a random access memory (RAM>3, a read-only memory (ROM>4) and an I/O (input/output) interface 5 via bidirectional buses, respectively.The RAM 3 stores data of the CPU 2. The ROM 4 is a memory circuit for storing and working, and stores a control program for the CPU 2.

The CPU 2 performs 13 operations according to the flowchart shown in FIG. That is, the PCU 2 first scans the character image data from the input unit 1 and detects whether new data has been input (steps 22 and 23).
). When inputting new data, the user moves the light ben 20 from one character on the tablet that it had been in contact with to another character. Detect input character image data as new data. When new character image data is input, move the character indicated by the input image data to
Determine from the position indicated by the coordinates (step 24)
. After this input character image data is stored in the RAM 3 (step 25), it is output to the ■/○ board of the CPU 2 (step 26), and from this it is output to the I1 shown in FIG.
0 interface 2 to the CPU 6.

When step 26 is completed, the process returns to step 22 again.

The user can enter the above character image data using Lightben 2.
0 while listening to the musical instrument sound produced by the first synthesizer 16, which will be described later, the movement speed of the light ben 20 is synchronized with the musical instrument sound, but the character image data itself to be input is at the discretion of the user. . Therefore, the present invention is useful when used in so-called lyric writing, in which lyrics are added to songs.

The CPUs 2 and 6 shown in FIG. 1 are also supplied with signals from the operating section 7 via the ■/○ interface 5, respectively. The operating section 7 generates a signal that determines the start timing of the entire device, but can also generate a signal that arbitrarily changes the speed (tempo) of the musical instrument sound produced by the first synthesizer 16. C, PU6 is I10 interface 5. RAM8. ROM9. Memory unit 10. Video controller 11. and M.I.D.
It is connected to the I interface 15 via a bidirectional bus. The RAM 8 is a memory circuit for storing data and working for the CPU 6, and also stores pitch parameters and the like corresponding to musical scales to be described later. Also ROM
9 stores in advance a control program for the CPU 6 and a formant variation make table regarding a plurality of formant frequencies. Further, the memory unit 10 is a memory circuit for storing processed data, and also stores in advance musical score data indicating the musical instrument sounds to be produced in this embodiment, and transfers this musical score data to the CP and U6.

The CPU 6 has a configuration as shown in FIG. 4, and operates according to the flowchart shown in FIG. CPU6
is first initialized, and then sets the value of variable 11 to "○"
Step 37), it is determined whether the first synthesizer 16 (or 17) can receive (enabled) the data coming through the M'IDI interface 15 (step 38 in FIG. 5). ). When the synthesizer 16 becomes able to receive music score data from the memory unit 10 (this is M I D 'f
(Musicat Instrument D
MIDI data (hereinafter also referred to as first MIDI data) is stored in the memory unit 1o in a form that conforms to the MIDI standard. Performance data converter 32. MrD1 data transfer unit 34. The data is transferred to the first synthesizer 16 through the MIDI interface 15 (step 39 in FIG. 5).

Thereby, the synthesizer 16 generates the musical instrument sound according to the musical score data (first MIDI data) by a known operation.

Here, the MfD [standard] converts various operations performed when playing a musical instrument (pressing a key, turning a volume, etc.) into several bytes of data, and controls multiple musical instruments or them. This is a well-known standard for transmitting and receiving data between machines that operate.

~IIDI data is transmitted as 2 to 3 bytes of serial data, and the first byte is a status, and the subsequent data bytes are messages that indicate what kind of function they have. The length of the data including the status is undefined, 1 to 3 bytes, and the status and data are recognized as status when the first bit of each byte is '1' and as data when it is 'O'. The above messages can be broadly divided into channel messages for transmitting individual operation data and system messages for controlling the network.

The CPU 6 then detects whether or not character image data is coming in from the ■/○ interface 5 (step 40 in FIG. 5), and if so, the input unit 30 shown in FIG.
The input character image data is taken in by the image data converter 31 through the input character image data converter 31, and is converted into image data suitable for display. On the other hand, if no character image data has been received, the first
Based on the data from the synthesizer 16 (or 17), it is determined whether or not the synthesizer 16 (or 17) is capable of receiving data. If the synthesizer 16 (or 17) is not capable of receiving data, the process proceeds to step 40, and if it is capable of receiving data, the process proceeds to step 50, which will be described later. The process moves to operation (step 42).

The image data converter 31 writes the output image data to a video random access memory (V, RAM) 12 via the video controller 11 shown in FIG. convert it into a signal. This video signal is supplied to a cathode ray tube (CRT>14) via the three primary color signal separation circuit 13 and displayed there. Therefore, step 43 shown in FIG.
As a result of the processing operation, the character image input by the image input section 1 is displayed on the CRT 14. The input character image (@word) is displayed on the display screen of this CRT14.
You can check etc.

Next, the CPU 6 sequentially performs the processing operations of steps 44 to 47 shown in FIG. 5 in the performance data conversion section 32. That is, in step 44, the characters based on the input character image data are decomposed into first parameters in which consonants and vowels are units, thereby creating a first table, which is transferred via the RAM data transfer unit 33. and write it to RAM8. In other words, it is known that the majority of Japanese syllables consist of combinations of consonants and vowels in a broad sense, and it is thought that the sound form of a word is represented by a sequence of consonants and vowels. , the first parameter can indicate individual syllables indicating lyrics.

Next, in step 45, according to the scale data in the inputted musical score data, a tsuchibara meme indicating the fundamental frequency (pitch frequency m) Fl of the vowel that determines the scale is selected from the table stored in advance in RA~18. Convert to pitch parameter. ). Then R
Read the formant parameter from the ROM 9 while referring to the first table stored in the A fvl 8, and combine it with the pitch parameter,
In addition to creating synthesis parameters by editing parameters and interpolating them (such as adding breaks to notes so that they change smoothly), a time length table is created according to the note value data in the score data, and these is temporarily stored in the RAM 8 (step 46 in FIG. 5).

Here, the singing voice consists of individual syllables representing the lyrics and sounds generated together with the syllables, and the former are represented by consonants and vowels according to the first parameter, and formant frequency versus time characteristics as shown in FIG. is known to show. That is, it is considered that speech identification is performed based on the fundamental frequency (pitch frequency) and a plurality of formants that make up the speech waveform. 1st. The second and third formants (hereinafter also referred to as formant frequencies) are F+,
Assuming F2 and F3, in the - syllable, the consonant part is first pronounced with sequential formant frequency changes, and then the vowel part is pronounced with a substantially constant formant frequency. Note that during the vowel part utterance period, a pitch frequency is also generated together with the formant frequencies F1 to F3.

Changes in the formant frequencies F1 to F3 in the consonant part differ depending on the consonant. For example, the formant frequencies of each consonant (1, (1 and b) are shown in Fig. 7 (A), (B) and (C
) is generally known to change as shown in Here, the formant parameters in the synthesis parameters described above are parameters that determine the three types of formant frequencies FI to F3. The pitch parameter also defines the pitch frequency.

On the other hand, the above-mentioned sound generated with the syllable (lyrics) is determined by the pitch, loudness, and length of the sound, and this is inputted to the CPU 6 as musical score data as described above, but according to the scale The frequency vs. time characteristic shown in Holman 1 has a characteristic as if it were moved in parallel in the direction of the vertical axis, and the pitch parameter (pitch frequency) determines the movement 2 (pitch of the sound). Furthermore, the length of a sound can be determined by the time length of the vowel part.

Next, in step 47, the above-mentioned synthesis parameters are converted into a second MIDI in a format that complies with the MIDI standard.
converted to data. In other words, the performance data conversion section 32
constitutes the above-mentioned table creation means, pitch parameter conversion means 2, composition parameter creation means and data conversion means. This second MIDI data is
The data is supplied to the MIDI data transfer section 34 shown in the figure, where the
The MIDI data is transferred to the second synthesizer 17 via the MIDI interface 15 (step 48 in FIG. 5). Here, this second
The MIDI data transfer is based on the consonant formant frequency F.
1 to F3, etc., is transferred to the second synthesizer 17 through the MIDI interface 15, and then the vowel formant frequencies F1 to F3. This is done by transferring second data regarding the pitch frequency FQ, etc. to the second synthesizer 17 through the MIDI interface 15.

The above first data is set in order to change the frequency of each of the first to third formant frequencies F1 to F3 multiple times at regular intervals (for example, 0.33 m5) during the utterance period of the consonant part. be transferred. As a result, as shown in FIGS. 6 and 7, the first data for each formant frequency that changes the formant frequencies F1 to F3 in the consonant voicing time period in a linear approximation is output to the second synthesizer 17. That will happen. Further, the second data is sent out for the time length according to the note value based on the time length table. Note that data can be transferred from the MIDI interface 15 to one of the synthesizers 16 and 17 based on the MIDI data status, for example, a channel message.

The second MIDI data extracted in this manner is stored in the RAM 8 via the RAM data transfer section 33 shown in FIG. 4 (step 4 in FIG. 5). After that, CP
After increasing the variable 11 by "1", U6 calculates whether the value of the variable 11 after the increase is greater than or equal to the total number of notes N, and repeats steps 39 to 5 described above until it becomes greater than or equal to N.
0 operation is repeated.

As described above, it is possible to achieve the purpose by generating singing voice sounds from the second synthesizer 17 in real time in accordance with the musical instrument sounds from the first synthesizer 16, but furthermore, after the processing operations in steps 37 to 51 are completed. , the user operates the operation unit 7 to save the second MfDI in the RAM 8.
By sequentially reading out (reproducing) the data (lyrics data), the second synthesizer 17 can generate singing voices that correspond to the musical instrument sounds from the first synthesizer 16 (singing voice synthesis performance is possible).

FIG. 8 shows a circuit diagram of an example of the ~+DI zein interface 15. ~IIDI data (1st and 2nd MIDI data) from the CPU 6 is supplied to the data input/output terminal DI+"-07 of the signal processing device 58, where it is subjected to parallel-to-serial conversion, etc. ．Connector 6
2, etc., to the synthesizer 16 or 17.

Data from the synthesizer 16 or 17 is also sent to the connector 63. Signal processing device 58 via photocoupler 64
and output in parallel from its data input/output terminals Do to D7. Note that the signal processing device 58 is supplied with a clock of, for example, 500 kl-1z from the clock module l065.

Furthermore, for the network when connecting synthesizers 16 and 17, the inputs are directly connected to inverters 66 and 67.
, a resistor 68, and six connectors (THRU terminals).

FIG. 9 shows a block diagram of an embodiment of the second synthesizer 17. The second MIDI data from the MIDI interface 15 is sent to input terminals 70+ to 7 via an input/output interface (not shown) in the synthesizer 17.
06 are input sequentially. Input data at input terminals 70+ to 704 is supplied to voltage controlled oscillators (VCOs) 71 to 74 as control signals. Here, VCO71, 72 and 7
3 are respectively the above-mentioned No. 1. The VCO 74 generates and outputs signals at the second and third formant frequencies, and outputs a signal at the pitch frequency Fo. Further, data input to the input terminal 70s is supplied to a low frequency oscillator (LF◯) 75 to control its oscillation operation. The output signal of this LFO75 is VC071
74 to obtain a vibrato effect by variably controlling the output oscillation frequency, and the output signal of the LFO 75 is supplied to a voltage control filter (VCF) 78 and a voltage control amplifier (VCA) 79, respectively, to obtain a vibrato effect. You can emphasize only the overtones or create a tremolo effect.

Each output signal of VC071 to 74 is sent to the ring modulator 7.
6, where it is converted into a signal with a sum or difference frequency of each frequency, and then supplied to a mixer 77, where V
It is mixed with each output signal of C071 to C074. The ring modulator 76 is used to obtain a specific tone, and its operation is controlled by data from the terminal 70s. mixer 7717) output mixed signal LtVCF78. V
The signal passes through the CA 79 and the amplifier 80, respectively, and is converted into a signal with a waveform as shown in FIG. 10, and is supplied to the speaker 81.

In FIG. 10, the king indicates the pitch period.

Here, predetermined data is applied only to the terminals 70+ to 703 during the consonant part voicing time period, so VC.

71.72 and 73 @1. The second and third formant frequencies F+, F2 and F3 are respectively output simultaneously,
Further, although the frequency value is divided into the N stages and sequentially changed, no signal is generated or outputted from the VCO 74. On the other hand, in the vowel part voicing time period, the pitch frequency FO that determines the scale is generated and output for the first time from ■C074 according to a part of the second data, and the pitch frequency FO that determines the scale is generated and output for the first time from VC071 to
From 73 onwards, the first to third formant frequencies F+ to F
Three signals are generated, each with a constant value. As a result, the lyrics input by the image input unit 1 are output from the speaker 81.
A human voice is produced according to the musical score input by the memory unit 10. Moreover, musical changes such as vibrato and tremolo can be added to the human voice sound and the natural singing voice sound.

As a result, the singing voice becomes more nuanced and more desirable. It should be noted that the synthesizer 17 shown in FIG. 9 can also generate only musical instrument sounds by using the keyboard 82, as in the conventional case. In addition, the VC in the first synthesizer 16
The first MIDt data is supplied to O, LF○, etc. as a control signal, and musical instrument sounds are produced from the speakers.

Note that in the present invention, the number of formants may be a plurality other than three. Further, the pitch parameter conversion means is not limited to reading from a table, but can also obtain the value by calculation.

Effects of the Invention As described above, according to the present invention, the sound of a human voice (singing voice) singing lyrics arbitrarily input in correspondence with a preset musical instrument sound from one of two synthesizers is transferred to the other synthesizer. Therefore, it is suitable for use in creating lyrics that match the song, and since it synthesizes formant frequencies, it is more natural than conventional devices that use white noise as consonants. It has excellent features such as being able to generate clear singing voice sounds.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing an embodiment of the device of the present invention;
FIG. 2 is a diagram showing a schematic configuration of the main parts in the block system shown in FIG. FIG. 6 is a block diagram showing an example of the main part of the synthesized speech, FIG.
The figure shows the relationship between the formant frequency and time of each consonant, Figure 8 is a circuit diagram of an example of a MIDI interface in the block system shown in Figure 1, and Figure 9 is a diagram of the synthesizer in the block system shown in Figure 1. FIG. 10 is a block system diagram showing one embodiment, and is a diagram showing an example of an input synthesized audio signal waveform of a speaker in the block system shown in FIG. 1... Image input section, 2, 6... Central processing unit (CP
U), 3.8...Random access memory (RA
M), 4.9...Read-only memory (80M)
, 5... [10 (input/output) interface, 7...
・Operation unit, 10...Memory unit, 11...Video controller, 14...CRT, 15...MID
I interface, 16.17... synthesizer,
19...Tablet, 20...Light pen, 30.
...Input section, 31... Image data conversion section, 32...
Performance data conversion section, 33...RAM data transfer section, 3
4...MIDI data transfer unit, 58...Signal processing device, 62°63.69...Connector, 71-74...
- Voltage controlled oscillator (VCO), 75...Low frequency oscillator (LF○), 81... Speaker. Patent applicant: Victor Japan Co., Ltd. Figure 3 Figure 6 Showa 1-1jl → Figure 8 Dan

Claims (1)

[Claims] A character data input section for inputting character data, a musical score data input section for inputting predetermined musical score data of a specific standard, and a first data transfer means for sequentially outputting the musical score data from the musical score data input section. a table creation means for creating a first table by decomposing the character data from the character data input section into parameters having consonants and vowels as units; a pitch parameter conversion means for converting the score data into a synthesis parameter creation means for creating a time length table according to the note value data in the data and temporarily storing them; a data conversion means for converting the synthesis parameters into data of a specific standard; a second data transfer means that transfers data related to a plurality of formant frequencies among the synthetic parameter conversion data, and generates and outputs pitch frequency data related to the pitch parameter for a period based on the time length table only during the vowel part utterance period; means for storing output data of the second data transfer means; first and second synthesizers for mixing and outputting output signals of a plurality of variable frequency oscillators; and output transfer of the first data transfer means. The data is supplied as a control signal to the plurality of variable frequency oscillators in the first synthesizer, and data regarding the plurality of formant frequencies and the pitch frequency data are supplied among the output transfer data of the second data transfer means. 1. A singing voice synthesis performance device comprising: interface means for separately supplying a control signal to each of the plurality of variable frequency oscillators in the second synthesizer as a control signal.