JP2016206323A - Singing sound synthesis device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To move a text to a desired position with a simple operation when generating a singing sound at real time performance.SOLUTION: When a key of A is pressed K0 at time t0, velocity V0 of the key pressing k0 is detected in response to reception of a sound generation instruction. Then, a character 'sa' of a cursor position is acquired, and voice stock piece data of sa is selected. Then, based on the voice stock piece data, a singing sound of sa is generated at pitch in which a sound source is A and predetermined prescribed volume. If a key k2 is not pressed at time t2, and if a key of A is pressed at time t4 at the velocity V1, sa is acquired, sa is next to ra on a cursor position, and a singing sound of sa is generated at the pitch in which the sound source is A and predetermined prescribed volume.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a singing sound synthesizing apparatus capable of advancing lyrics to a desired position with a simple operation when singing a singing sound in real-time performance.

2. Description of the Related Art Conventionally, a singing sound synthesizer is known in which a user specifies an arbitrary section from lyrics displayed on a touch panel and outputs the specified lyrics as a singing voice at a pitch corresponding to the key that is pressed (Patent Literature). 1). This singing sound synthesizer has a first mode in which the syllable is advanced to the next syllable each time a key is pressed, and a second mode in which only the selected syllable is repeatedly read each time the key is pressed.
In such a conventional singing sound synthesizer, the user can select a desired lyrics by operating the touch panel, and each syllable of the selected lyrics can be selected at a desired pitch by pressing the key on the keyboard. It can be output as a singing voice.

JP 2014-10190 A

  In a conventional singing sound synthesizing apparatus, when a character or a syllable is automatically advanced in accordance with a key press, if there is a performance mistake or the like, the position of the lyrics may deviate from the performance content. At that time, it is difficult to control the position of the lyrics by operating a user interface such as a touch panel because real-time performance is performed.

  Therefore, an object of the present invention is to provide a singing sound synthesizing apparatus that allows a lyric to be advanced to a desired position with a simple operation when a singing sound is generated in real-time performance.

  In order to achieve the above object, the singing sound synthesizer of the present invention includes a detection means for detecting the operation start and operation intensity of the performance operator, and according to the operation start of the performance operator detected by the detection means, An acquisition means for acquiring character information from a storage device; and a pronunciation control means; the position of the character acquired by the acquisition means is controlled according to the operation intensity of the performance operator detected by the detection means; The main feature is that the pronunciation control means pronounces the characters acquired by the acquisition means as speech.

  In the singing sound synthesizer of the present invention, when a character acquired in response to the operation of the performance operator is pronounced as a voice, the character acquired in accordance with the operation intensity of the performance operator can be controlled. In this case, the lyrics can be advanced to a desired position by operating the operation intensity of the performance operator.

It is a functional block diagram which shows the hardware constitutions of the song sound synthesizer of this invention. It is a figure explaining the structure of the lyric data in the singing sound synthesizer concerning this invention, and the aspect of a velocity setting. It is a figure which shows the operation example of the singing sound synthesizer concerning this invention. It is a flowchart of the singing sound synthetic | combination process which the singing sound synthesizer concerning this invention performs.

A functional block diagram showing the hardware configuration of the singing sound synthesizer of the present invention is shown in FIG.
In the singing sound synthesizing apparatus 1 of the present invention shown in FIG. 1, a CPU (Central Processing Unit) 10 is a central processing apparatus that controls the entire singing sound synthesizing apparatus 1 of the present invention, and a ROM (Read Only Memory) 11. Is a non-volatile memory storing a control program and various data. A RAM (Random Access Memory) 3 is a volatile memory used as a work area of the CPU 10 and various buffers. A data memory 18 that is assumed to be a rewritable memory such as a flash memory stores a phonetic database that stores character information including text data of lyrics and speech segment data of singing sounds. The display unit 15 is a display unit including a liquid crystal display or the like on which an operation state, various setting screens, a message for the user, and the like are displayed. The performance operator 16 is a performance operator composed of a keyboard or the like, and generates performance information such as key-on, key-off, pitch, and velocity. The setting operator 17 is various setting operators such as operation knobs and operation buttons for setting the singing sound synthesizer 1.

  The sound source 13 has a plurality of tone generation channels, and assigns one tone generation channel in accordance with real-time performance using the user's performance operator 16 under the control of the CPU 10. In the assigned tone generation channel, the data memory 18 is assigned. The voice segment data corresponding to the performance is read from the singing sound data. The sound system 14 converts the singing sound data generated by the sound source 13 into an analog signal using a digital / analog converter, amplifies the singing sound converted into an analog signal, and outputs the amplified singing sound to a speaker or the like. Furthermore, the bus 19 is a bus for transferring data between the respective units in the singing sound synthesizer 1.

  The configuration of the lyrics data in the singing sound synthesizer 1 according to the present invention is shown in FIG. 2 (a), and the mode of velocity setting is shown in FIG. 2 (b). The lyric data is composed of text data 30 representing the contents of the lyrics, and each character constituting the text data 30 is character information 31. The character information 31 defines the text data 30 in units of one syllable read by one key press. In the case shown in FIG. 2A, “Sakura Sakura no Yamamo” is the text data 30, and c11 to c34 are the character information 31. The character information 31 of c11, c12, c13 indicates that the first three-character text data 30 of “Sakura” is the first phrase 32a, and the subsequent three-character text data 30 of “Sakura” is the second phrase 32b. That the character information of c21, c22, and c23 indicates that the subsequent four-character text data 30 of “Noyamamo” is the third phrase 32c, and the character information of c31, c32, c33, and c34 Show. That is, c11 is information that the first character “sa” belongs to the first phrase 32a, c12 is information that the second character “ku” belongs to the first phrase 32a, and c13 is three characters. This is information indicating that the eye “ra” belongs to the first phrase 32a. The same applies to c21 to c34. For example, c21 is information indicating that the fourth character “sa” belongs to the second phrase 32b, and c31 is information indicating that the seventh character “no” belongs to the third phrase 32c. Information.

  Next, one mode of velocity setting in the singing sound synthesizer 1 according to the present invention is shown in FIG. In the singing sound synthesizing apparatus 1 according to the present invention, the position of the character to be read out is controlled in accordance with the velocity V when the key is pressed. In FIG. 2 (b), the vertical axis indicates the intensity of velocity V. In the velocity setting shown in this figure, the velocity V is defined as velocity V0 when the intensity is equal to or less than Va. When V is V0, the character at the normal position is read and pronounced. When Va <Vb, the velocity V is greater than Va and less than or equal to Vb is defined as velocity V1, and when the key is depressed and the velocity V is V1, the next character in the normal position is Read and pronounce. Further, when the velocity V exceeds Vb, it is defined as velocity V2, and when the key is depressed and the velocity V is V2, the first character of the next phrase is read and pronounced. For example, Va is an intensity corresponding to mesoforte (mf), and Vb is an intensity corresponding to forte (f) or fortissimo (ff). This makes it possible to control the position of the lyrics to be read with the velocity when the key is pressed next, even when the key to be pressed is missed as described below, and the performance contents and lyrics of the real-time performance The positions of can be matched.

An operation example of the singing sound synthesizing apparatus 1 according to the present invention is shown in FIGS. 3A, 3B, and 3C, and a flowchart of the singing sound synthesizing process executed at that time is shown in FIG.
Operation example 1 shown in FIG. 3A will be described with reference to the flowchart shown in FIG. In FIG. 3A, the vertical axis indicates the pitch, and the horizontal axis indicates the time. In this case, the pitch is represented by the pitch name, and the beat timing is shown at times t0 to t12 on the time axis. Before the user performs a real-time performance, the lyric data shown in FIG. 2A among the lyric data stored in the data memory 18 is selected, and the cursor indicating the character to be read is the first character “sa” of c11. It is assumed that it is placed in the position.

  When the user starts playing and the key A of the performance operator 16 is depressed k0 at time t0, the CPU 10 detects the depressed key k0, and the first singing sound synthesis process starts. Here, the speech segment data selection process in step S14 and the sound generation process in step S15 in the singing sound synthesis process are executed in the sound source 13 under the control of the CPU 10, and the other processes are executed by the CPU 10. In step S10 of the started singing sound synthesis process, the CPU 10 accepts a sound generation instruction based on the key depression k0 of the performance operator 16 operated at time t0. At this time, the CPU 10 acquires performance information such as the timing t0 of the key depression k0, the pitch information A of the operated performance operator 16 and the velocity. In this case, the keyboard used as the performance operator 16 is, for example, a three-make keyboard provided with a first sensor, a second sensor, and a third sensor that detect the pressing operation of the keyboard in three stages. Then, the CPU 10 acquires the intensity of the velocity V calculated based on the time from when the first sensor is turned on until the second sensor is turned on. In step S11, it is determined whether or not the velocity V of the pressed key k0 is the velocity V0. If the velocity V calculated by the CPU 10 is less than Va, the velocity V is the velocity V0 strength. (Yes), the process proceeds to step S13. It is assumed that at the key pressing k0, the user intentionally presses the key at a key pressing speed at the velocity V0. As a result, the CPU 10 determines Yes in step S <b> 11, proceeds to step S <b> 13, and the CPU 10 acquires the character at the cursor position from the data memory 18. In this case, the lyric data shown in FIG. 2A is selected, and the cursor is placed on the first character c11 of the text data 30 representing the selected lyric data. Thus, when receiving a sound generation instruction based on the key press k0 at time t0, the CPU 10 reads the character “sa” of c11 on which the cursor is placed from the data memory 18.

  Next, speech segment data selection processing is performed in step S14. This speech segment data selection process is a process performed by the sound source 13 under the control of the CPU 10, and selects speech segment data for generating the acquired characters from the phoneme database stored in the data memory 18. The phoneme database stores “phoneme chain data” and “stationary partial data”. The phoneme chain data is data of phonemes when the pronunciation changes, such as silence (#) to consonant, consonant to vowel, vowel to consonant or vowel (next character). The steady part data is data of phonemes when the vowel pronunciation continues. When the sound generation instruction based on the key press k0 is received and the acquired character is “sa” of c11, the sound unit 13 uses the phoneme chain data to generate the speech segment data “# −” corresponding to “silence → consonant s”. speech unit data “sa” corresponding to “s” and “consonant s → vowel a” is selected, and speech unit data “a” corresponding to “vowel a” is selected from the steady-state partial data. . Next, in step S15, the sound source 13 performs a sound generation start process based on the speech segment data selected in step S14 under the control of the CPU 10. As described above, when the speech unit data is selected, in the sound generation start process in step S15, the sound source 13 sequentially generates the sound of the speech unit data “# -s” → “sa” → “a”. The character “sa” of c11 is pronounced. At the time of sound generation, the singing sound of “sa” is generated at a predetermined predetermined volume with the pitch A acquired at the time of receiving the sound generation instruction based on the key press k0. When sound generation is started in the sound generation start process of step S15, the process proceeds to step S16, and the CPU 10 advances the cursor to the next character “ku” of c12. Next, in step S17, the CPU 10 determines whether or not the velocity of the key being pressed has changed by Vth or more. Here, since the user is performing so as not to change the velocity of the key press k0, the CPU 10 determines No in step S17, and proceeds to step S18 to determine whether the key press k0 has been released and a sound generation stop instruction has been issued. The CPU 10 determines whether or not. If the CPU 10 determines that the key press k0 has not been released yet (No), the process returns to step S17, and the processes of steps S17 and S18 are repeated. Then, as shown in FIG. 3A, when the key press k0 is released immediately before time t1, the CPU 10 determines in step S18 that it has received a sound generation stop instruction and responds to the pronunciation of the character “sa” in c11. Silence processing is performed and the singing sound of “sa” stops. Thereby, a song sound synthesis process is complete | finished.

  Next, when the performance progresses and the user presses the key A1 at time t1, the CPU 10 detects the key press k1 and the second singing sound synthesis process is started. In this key press k1, it is assumed that the user intentionally presses the key at a key press speed at which the velocity is V0. The singing sound synthesizing process is performed in the same manner as described above. In step S11, the CPU 10 detects that the velocity V of the key press k1 is the velocity V0, and in step S13, the character “c” at the cursor position c12 is stored as data. Read from the memory 18. Next, phoneme piece data for generating “ku” in the sound source 13 is selected from the phoneme database in the data memory 18 in step S14. In this case, speech unit data “# -k”, “ku”, and “u” are selected. In step S15, the sound source 13 starts to pronounce the character “ku” of c12 based on the selected speech segment data. At the start of the sound generation, the sound segment data “# -k” → “ku” → “u” are sounded in sequence at the tone A of the key press k1 in the sound source 13 at a predetermined predetermined volume. Is done. Thereby, the singing sound of “ku” is pronounced. Next, in step S16, the cursor is advanced to the position of the next character “ra” of c13. The key press k1 is released before the time t2 is reached, and the CPU 10 detects a sound generation stop instruction of the key press k1 in step S18, and the singing sound of “ku” in c12 is muted. When the process of step S18 ends, the singing sound synthesis process ends.

  Next, when the performance progresses and the key B should be pressed k2 at time t2, it is assumed that the user fails to press the key k2. In this case, since the key press k2 has not been performed, the CPU 10 does not accept the pronunciation instruction and the singing sound synthesis process is not started. Therefore, the character “ra” at c13 at the cursor position is not read, and the sound generation start process is not performed, so that the cursor position remains at the character position “ra”.

When the performance further proceeds and the user presses the key A3 at time t4, the CPU 10 detects the key press k3, and the third singing sound synthesis process is started. In this key press k3, it is assumed that the user intentionally presses the key at a key press speed at which the velocity is V1. Thereby, in step S11 of the singing sound synthesizing process, the CPU 10 detects that the velocity V of the key press k3 is equal to or higher than the velocity V0, and branches to step S12. In step S12, a character at a position corresponding to the velocity V calculated by the CPU 10 is acquired. In this case, when the CPU 10 detects that the velocity V is the velocity V1, as shown in FIG. 2B, the character next to the character at the cursor is read, that is, the character next to the character “ra” at c13 at the cursor position. Reads out the character “sa” of c21. Next, in the sound source 13, phoneme piece data for generating “sa” is selected from the phoneme database in the data memory 18 in step S 14. In this case, speech unit data “# -s”, “s-a”, and “a” are selected. In step S15, the sound source 13 starts to pronounce the character “sa” of c21 based on the selected speech segment data. At the start of this sounding, the sound segment data “# -s” → “sa” → “a” is sounded in sequence at the tone A of the key press k3 in the sound source 13 at a predetermined predetermined volume. Is done. Thereby, the song sound of “sa” is pronounced. Next, in step S16, the cursor is advanced to the position of the next character “ku” of c22. The key press k3 is released before time t5 is reached, the CPU 10 detects a sound generation stop instruction for the key press k3 in step S18, and the singing sound of “sa” in c21 is muted. When the process of step S18 ends, the singing sound synthesis process ends.
As described above, if the key press k2 is missed, the position of the lyric to start sounding with the next key press k3 will be shifted by one character, but by pressing the key press k3 with a velocity V1 exceeding the velocity V0. Can correct. That is, by pressing the key with velocity V1, it becomes possible to read the character next to the cursor position and start sounding. As a result, if the performance is wrong (the key to be pressed is not pressed) and the next key is played at a velocity V1 exceeding the velocity V0, the character that should be pronounced by that key press is immediately displayed. Can be pronounced.

  When the performance further progresses and the user presses the key A4 at time t5, the CPU 10 detects the key press k4, and the fourth singing sound synthesis process is started. In this key press k4, it is assumed that the user intentionally presses the key at a key press speed at which the velocity is V0. The singing sound synthesizing process is performed in the same manner as described above. In step S11, the CPU 10 detects that the velocity V of the key press k4 is the velocity V0, and in step S13, the character “ku” at the cursor position c22 is stored as data. Read from the memory 18. Next, phoneme piece data for generating “ku” in the sound source 13 is selected from the phoneme database in the data memory 18 in step S14. In this case, speech unit data “# -k”, “ku”, and “u” are selected. In step S15, the sound source 13 starts to pronounce the character “ku” of c22 based on the selected speech segment data. At the start of this sounding, the sound segment data “# -k” → “ku” → “u” are sounded in sequence at the tone A of the key press k4 in the sound source 13 at a predetermined predetermined volume. Is done. Thereby, the singing sound of “ku” is pronounced. Next, in step S16, the cursor is advanced to the position of the next character “ra” in c23. The key press k4 is released before time t6 is reached, the CPU 10 detects a sound generation stop instruction for the key press k4 in step S18, and the singing sound of “ku” in c22 is muted. When the process of step S18 ends, the singing sound synthesis process ends.

  When the performance further proceeds and the user presses the key B at time t6, the CPU 10 detects the key press k5, and the fifth singing sound synthesis process is started. In this key press k5, it is assumed that the user intentionally presses the key at a key press speed at which the velocity is V0. The singing sound synthesizing process is performed in the same manner as described above. In step S11, the CPU 10 detects that the velocity V of the key press k5 is the velocity V0, and in step S13, the character "ra", which is the character c23 at the cursor position, is stored. Read from the memory 18. Next, in the sound source 13, phoneme piece data for generating “ra” is selected from the phoneme database in the data memory 18 in step S 14. In this case, “# -r”, “r-a”, and “a” speech segment data are selected. In step S15, the sound source 13 starts to pronounce the character “ra” of c23 based on the selected speech segment data. At the start of the sound generation, the sound segment data “# -r” → “r−a” → “a” is sounded in sequence at the tone B of the key press k5 in the sound source 13 at a predetermined predetermined volume. Is done. As a result, the singing sound of “ra” is pronounced. Next, in step S16, the cursor is advanced to the position of the next character “no” of c31. The key press k5 is released before the time t7 is reached after the time t7 is reached, the CPU 10 detects a sound generation stop instruction for the key press k5 in step S18, and the singing sound of “ra” in c23 is muted. When the process of step S18 ends, the singing sound synthesis process ends.

  The singing sound synthesizing apparatus 1 according to the present invention can be operated with an operation different from the above-described operation when the key is pressed with velocity V1. The operation of this variation will be described after time t8 in FIG. When the performance progresses and the user presses the key A6 at time t8, the CPU 10 detects the key press k6, and the sixth singing sound synthesis process is started. In this key press k6, it is assumed that the user intentionally presses the key at a key pressing speed at a velocity V0. The singing sound synthesizing process is performed in the same manner as described above. In step S11, the CPU 10 detects that the velocity V of the key press k6 is the velocity V0, and in step S13, the character C31 at the cursor position is “no”. Read from the memory 18. Next, phoneme piece data for generating “no” in the sound source 13 is selected from the phoneme database in the data memory 18 in step S14. In this case, “# -n”, “no”, “o” speech segment data is selected. In step S15, the sound source 13 starts to pronounce the character “no” of c31 based on the selected speech segment data. At the start of this sounding, the sound segment data “# -n” → “no” → “o” is sounded in sequence at the tone A of the key press k6 in the sound source 13 at a predetermined predetermined volume. Is done. As a result, the singing sound “no” is produced. Next, in step S16, the cursor is advanced to the position of the next character “ya” of c32. The key press k6 is released at approximately time t9, the CPU 10 detects a sound generation stop instruction for the key press k6 in step S18, and the “no” song sound of c31 is muted. When the process of step S18 ends, the singing sound synthesis process ends.

  Next, at time t9, the key B should be pressed k7, but it is assumed that the key press k7 is missed. In this case, since the key press k7 has not been performed, the CPU 10 does not accept the pronunciation instruction and the singing sound synthesis process is not started. Accordingly, the character “ya” at c32 at the cursor position is not read, and the sound generation start process is not performed, so that the cursor position remains at the character position “ya”.

When the performance further proceeds and the user presses the key C8 at time t10, the CPU 10 detects the key press k8, and the seventh singing sound synthesis process is started. In this key press k8, it is assumed that the user intentionally presses the key at a key press speed at which the velocity is V1. Thereby, in step S11 of the singing sound synthesizing process, the CPU 10 detects that the velocity V of the key press k8 is equal to or higher than the velocity V0, and branches to step S12. In step S12, a character at a position corresponding to the velocity V calculated by the CPU 10 is acquired. In the operation of this variation, when it is detected that the velocity V of the key press k8 is the velocity V1, the character at the cursor position is pronounced in a short time and then the next character is pronounced. That is, when the CPU 10 detects that the velocity V is the velocity V1, the CPU 10 reads the character “ya” of c32 at the cursor position and the next character “ma” of c33. Next, in the sound source 13, phoneme piece data for pronounced “ya” and phoneme piece data for pronounced “ma” are selected from the phoneme database of the data memory 18 in step S 14. In this case, “ma” is pronounced after “ya”, so that “# -y”, “ya”, “a” speech segment data, “am”, “m-a”. , “A” speech segment data is selected. In step S15, the sound source 13 starts to pronounce the character “ya” in c32 based on the selected speech segment data. At the start of this sounding, the sound segment data “# -y” → “ya” → “a” is sounded in sequence at the tone generator C at the pitch C of the key press k8 in the sound source 13. Is done. As a result, the singing sound of “ya” is pronounced, but this sounding period is short, and following the singing sound of “ya” whose pronunciation is stopped, the sound source 13 is based on the selected speech segment data. Then, the pronunciation of the letter “ma” in c33 is started. At the start of this sounding, the sound segment data “am” → “ma” → “a” are sequentially sounded at the tone generator C at the pitch C of the key press k8 at a predetermined predetermined volume. Is done. Thereby, the singing sound of “MA” is pronounced after the singing sound of “YA”. Next, in step S16, the cursor is advanced to the position of the next character "mo" of c34. The key press k8 is released at about time t11, the CPU 10 detects a sound generation stop instruction for the key press k8 in step S18, and the singing sound of “ma” in c33 is muted. When the process of step S18 ends, the singing sound synthesis process ends.
As described above, if the key press k7 is missed, the position of the lyric to start sounding with the next key press k8 is shifted by one character. However, by pressing the key press k8 with the velocity V1 exceeding the velocity V0. In the operation of the variation can be corrected. That is, in the variation operation, by pressing the key with velocity V1, the character at the cursor position is read, then the next character is read, and the character at the cursor position is pronounced for a short time. As a result, in the variation operation, even if the performance is wrong (note that the key to be pressed is not pressed), it is possible to make the performance easy to understand the meaning of the lyrics for the audience without missing letters. .

  When the performance further proceeds and the user presses the key B at time t11, the CPU 10 detects the key press k9, and the eighth singing sound synthesis process is started. In this key press k9, it is assumed that the user intentionally presses the key at a key press speed at which the velocity is V0. The singing sound synthesizing process is performed in the same manner as described above. In step S11, the CPU 10 detects that the velocity V of the key press k9 is velocity V0, and in step S13, the character “c” at the cursor position c34 is stored as data. Read from the memory 18. Next, in the sound source 13, phoneme piece data for generating “mo” is selected from the phoneme database in the data memory 18 in step S 14. In this case, speech unit data “# -m”, “mo”, and “o” are selected. In step S15, the sound source 13 starts to pronounce the character “c” in c34 based on the selected speech segment data. At the start of this sounding, the sound segment data “# -m” → “mo” → “o” are sounded in sequence at the tone B of the key press k9 in the sound source 13 at a predetermined predetermined volume. Is done. Thereby, the singing sound of “mo” is pronounced. Next, in step S16, the cursor is advanced to the position of the next character. In this case, if there is no next character, this step is skipped. The key press k9 is released at about time t12, the CPU 10 detects a sound generation stop instruction for the key press k9 in step S18, and the c34 "mo" singing sound is muted. When the process of step S18 ends, the singing sound synthesis process ends.

Next, FIG. 3B shows an operation example 2 of the singing sound synthesizer 1 according to the present invention. Here, the operation example 2 shown in FIG. 3B will be described with respect to the operation different from the operation example 1 shown in FIG. In FIG. 3B as well, the vertical axis represents the pitch represented by the pitch name, and the horizontal axis represents the timing of beats at times t0 to t12. Further, it is assumed that the lyric data shown in FIG. 2A is selected before the user performs a real-time performance, and the cursor indicating the character to be read is at the first character “sa” of c11.
When the user starts playing and the key A of the performance operator 16 is depressed k0 at time t0, the CPU 10 detects the depressed key k0, and the first singing sound synthesis process is started. Since the process executed at this time is the same process as in the first operation example, the description thereof is omitted. When the performance progresses and the user presses the key A1 at time t1, the CPU 10 detects the key press k1 and the second singing sound synthesis process is started. The processing executed at this time is also the same processing as in the first operation example, and the description thereof is omitted.

  When the performance further proceeds and the user presses the key B2 at time t2, the CPU 10 detects the key press k2, and the third singing sound synthesis process is started. In this key press k2, it is assumed that the user intentionally presses the key at a key press speed at which the velocity is V0. In the singing sound synthesizing process, the CPU 10 detects in step S11 that the velocity V of the key press k2 is the velocity V0, and reads “ra”, which is the character of c13 at the cursor position, from the data memory 18 in step S13. Next, in the sound source 13, phoneme piece data for generating “ra” is selected from the phoneme database in the data memory 18 in step S 14. In this case, “# -r”, “r-a”, and “a” speech segment data are selected. In step S15, the sound source 13 starts to pronounce the character “ra” of c13 based on the selected speech segment data. At the start of sounding, the sound segment data “# −r” → “r−a” → “a” is sounded in sequence at the tone B of the key press k2 in the sound source 13 at a predetermined predetermined volume. Is done. As a result, the singing sound of “ra” is pronounced. Next, in step S16, the cursor is advanced to the position of the next character “sa” of c21. The key press k2 is released before the time t3 is reached after the time t3 is reached, the CPU 10 detects a sound generation stop instruction for the key press k2 in step S18, and the singing sound of “ra” in c13 is muted. When the process of step S18 ends, the singing sound synthesis process ends.

  When the performance further proceeds and the user presses the key A3 at time t4, the CPU 10 detects the key press k3, and the fourth singing sound synthesis process is started. In this key press k3, it is assumed that the user intentionally presses the key at a key press speed at which the velocity is V0. In the singing sound synthesizing process, the CPU 10 detects in step S11 that the velocity V of the key press k3 is velocity V0, and reads “sa”, which is the character of c21 at the cursor position, from the data memory 18 in step S13. Next, in the sound source 13, phoneme piece data for generating “sa” is selected from the phoneme database in the data memory 18 in step S 14. In this case, speech unit data “# -s”, “s-a”, and “a” are selected. In step S15, the sound source 13 starts to pronounce the character “sa” of c21 based on the selected speech segment data. At the start of this sounding, the sound segment data “# -s” → “sa” → “a” is sounded in sequence at the tone A of the key press k3 in the sound source 13 at a predetermined predetermined volume. Is done. Thereby, the song sound of “sa” is pronounced. Next, in step S16, the cursor is advanced to the position of the next character “ku” of c22. The key press k3 is released before time t5 is reached, the CPU 10 detects a sound generation stop instruction for the key press k3 in step S18, and the singing sound of “sa” in c21 is muted. When the process of step S18 ends, the singing sound synthesis process ends.

Next, it is assumed that the key A should be pressed k4 at time t5, but the key press k4 is missed. In this case, since the key press k4 has not been performed, the CPU 10 does not accept the pronunciation instruction and the singing sound synthesis process is not started. Therefore, the character “ku” at the cursor position c22 is not read, and the sound generation start process is not performed, so that the cursor position remains at the character position “ku”.
Furthermore, it is assumed that the key B should be pressed k5 at time t6, but the key press k5 is also missed. In this case, since the key press k5 has not been performed, the CPU 10 does not accept the pronunciation instruction and the singing sound synthesis process is not started. Therefore, the character “ku” at the cursor position c22 is not read, and the sound generation start process is not performed, so that the cursor position remains at the character position “ku”.

  When the performance further proceeds and the user presses the key A6 at time t8, the CPU 10 detects the key press k6, and the fifth singing sound synthesis process is started. In this key press k6, it is assumed that the user intentionally presses the key at a key press speed at which velocity V2 is obtained. Thereby, in step S11 of the singing sound synthesizing process, the CPU 10 detects that the velocity V of the key press k6 is equal to or higher than the velocity V0, and branches to step S12. In step S12, a character at a position corresponding to the velocity V calculated by the CPU 10 is acquired. In this case, when it is detected that the velocity V of the key press k6 is the velocity V2, the character of the next phrase is read as shown in FIG. In this case, since the cursor is at c22 and c22 belongs to the second phrase 32b, the CPU 10 reads the character “NO” of the first c31 of the next third phrase 32c. Next, phoneme piece data for generating “no” in the sound source 13 is selected from the phoneme database in the data memory 18 in step S14. In this case, “# -n”, “no”, “o” speech segment data is selected. In step S15, the sound source 13 starts to pronounce the character “no” of c31 based on the selected speech segment data. At the start of this sounding, the sound segment data “# -n” → “no” → “o” is sounded in sequence at the tone A of the key press k6 in the sound source 13 at a predetermined predetermined volume. Is done. As a result, the singing sound “no” is produced. Next, in step S16, the cursor is advanced to the position of the next character “ya” of c32. The key press k6 is released at approximately time t9, the CPU 10 detects a sound generation stop instruction for the key press k6 in step S18, and the “no” song sound of c31 is muted. When the process of step S18 ends, the singing sound synthesis process ends.

  As described above, when the key press k4 at time t5 and the key press k5 at time t6 are continuously pressed, the position of the lyrics is shifted by two characters from the performance content. By depressing the key k6 with the velocity V2, the deviation can be corrected by reading the phrase next to the phrase where the cursor is. That is, if the performance is wrong (the note that should be pressed is not pressed), the next key press is played at velocity V2, so that the character that should originally be generated by the key press can be immediately generated. It becomes like this. The description of the operation after time t9 in FIG. 3B is omitted because it can be easily understood from the above description.

Next, FIG. 3C shows an operation example 3 of the singing sound synthesizer 1 according to the present invention. FIG. 3 (c) shows the beat timing at times t20 to t23 on the time axis on the horizontal axis, as in FIG. 3 (a). The section is a part of the performance. It is assumed that the lyric data shown in FIG. 2A is selected before the user performs a real-time performance, and the cursor indicating the character to be read is at the first character “sa” of c11.
When the user presses the key A in the performance operator 16 at time t20 during performance, the CPU 10 detects the key press k10 and the singing sound synthesis process is started. In this key press k10, it is assumed that the user intentionally presses the key at a key press speed at which the velocity is V0. In the singing sound synthesizing process, the CPU 10 detects in step S11 that the velocity V of the key press k10 is velocity V0, and reads “sa”, which is the character of c11 at the cursor position, from the data memory 18 in step S13. Next, in the sound source 13, phoneme piece data for generating “sa” is selected from the phoneme database in the data memory 18 in step S 14. In this case, speech unit data “# -s”, “s-a”, and “a” are selected. In step S15, the sound source 13 starts to pronounce the character “sa” of c11 based on the selected speech segment data. At the start of this sounding, the sound segment data “# -s” → “sa” → “a” is sounded in sequence at the tone A of the key press k10 in the sound source 13 at a predetermined predetermined volume. Is done. Thereby, the song sound of “sa” is pronounced. Next, in step S16, the cursor is advanced to the position of the next character “ku” of c12.

  It is assumed that the key A being pressed is being pressed near the time t21 while the key k10 is being pressed. In step S17, the CPU 10 determines whether or not the velocity of the key being pressed has changed by Vth or more. Here, the user presses the A key so that the velocity of the key press k10 is intentionally changed by Vth or more. Suppose you push it in. As a result, the CPU 10 determines Yes in step S <b> 17, returns to step S <b> 13, and the CPU 10 reads “ku”, which is the character of c <b> 12 at the cursor position, from the data memory 18. Next, phoneme piece data for generating “ku” in the sound source 13 is selected from the phoneme database in the data memory 18 in step S14. In this case, since “ku” is pronounced after “sa”, speech unit data of “ak”, “ku”, and “u” is selected. In step S15, the sound source 13 starts to pronounce the character “ku” of c12 based on the selected speech segment data. At the start of this sounding, the sound segment data “a−k” → “ku” → “u” are sounded in sequence at the pitch A of the key press k10 in the sound source 13 at a predetermined predetermined volume. Is done. When the pronunciation starts, the song sound of “sa” is stopped, and the song sound of “ku” is pronounced after the song sound of “sa”. Next, in step S16, the cursor is advanced to the position of the next character “ra” of c13. The key press k10 is released at approximately time t22, the CPU 10 detects a sound generation stop instruction for the key press k10 in step S18, and the singing sound of “ku” in c12 is muted. When the process of step S18 ends, the singing sound synthesis process ends.

When the performance further proceeds and the user presses the key B at time t22, the CPU 10 detects the key press k5, and the second singing sound synthesis process is started. In this key press k12, it is assumed that the user intentionally presses the key at a key press speed at which the velocity is V0. The singing sound synthesizing process is performed as described above. In step S11, the CPU 10 detects that the velocity V of the key press k12 is the velocity V0, and in step S13, "ra", which is the character of c13 at the cursor position, is stored in the data memory. 18 is read out. Next, in the sound source 13, phoneme piece data for generating “ra” is selected from the phoneme database in the data memory 18 in step S 14. In this case, “# -r”, “r-a”, and “a” speech segment data are selected. In step S15, the sound source 13 starts to pronounce the character “ra” of c13 based on the selected speech segment data. At the start of this sounding, the sound segment data “# -r” → “r−a” → “a” is sounded sequentially at the tone generator B at the pitch B of the key press k12 in the sound source 13. Is done. As a result, the singing sound of “ra” is pronounced. Next, in step S16, the cursor is advanced to the position of the next character “no” of c21. The key press k12 is released beyond time t23, the CPU 10 detects a sound generation stop instruction for the key press k12 in step S18, and the singing sound of “ra” in c13 is muted. When the process of step S18 ends, the singing sound synthesis process ends.
As described above, in the singing sound synthesizer 1 according to the present invention, when performing a performance in which the velocity is increased by a predetermined amount (Vth) or more during key depression, that is, after touching, the character next to the character being pronounced is displayed. It is possible to produce sound with the pitch of the key being pressed.

In the singing sound synthesizing apparatus of the present invention described above, various variations can be considered for the correspondence between the velocity and how to advance the characters to be pronounced. For example, by setting a velocity range in N stages and setting a character to be pronounced according to the velocity range by one character, two characters,... It can be controlled by velocity. Further, in the above description, a plurality of characters are pronounced when the velocity is high. Alternatively, a plurality of characters may be advanced when the velocity is small. Further, the character to be sounded may be returned according to the velocity value. In this case, it may be controlled to return to the beginning position of the phrase or to return one character.
By the way, the more the velocity range is set in multiple steps, the more accurate the performance is required, and the performance operation becomes more difficult. Therefore, the character to be sounded may be advanced or returned by combining the operation of the keyboard with the operation of another performance operator such as a pedal. For example, in the singing sound synthesizing apparatus of the present invention that operates as described above, when a performance is performed at velocity V1 while depressing the pedal, the character to be sounded is returned by one character, and when the performance is performed at velocity V2 while depressing the pedal. Can return the character to be played back to the beginning of the phrase. According to this, even if the velocity setting is set in three stages, it becomes possible to perform character control of five types of characters to be pronounced.

Furthermore, in the singing sound synthesizer of the present invention, the performance operator may be a performance operator that is not a keyboard. For example, in the case of a percussion instrument (pad) type, the character to be sounded is controlled according to the striking force. In the case of a stringed instrument type, the character to be sounded is controlled according to the strength of repelling the string. In this case, the character to be sounded may be controlled according to the amount of breath.
Furthermore, in the singing sound synthesizer of the present invention, the character information or the like may be stored in a hard disk, a built-in memory, an external memory, a server via a network, or the like instead of the data memory storing the character information.
In the singing sound generating device of the present invention, the key depression velocity is handled only as information for character control and is not used for volume control. However, a specific switch or knob or a specific performance operator is not used. When a key is pressed while operating, the volume may be controlled by the key pressing velocity.

DESCRIPTION OF SYMBOLS 1 Song sound synthesizer, 10 CPU, 11 ROM, 12 RAM, 13 Sound source, 14 Sound system, 15 Display part, 16 Performance operator, 17 Setting operator, 18 Data memory, 19 Bus, 30 Text data, 31 Character information , 32a 1st phrase, 32b 2nd phrase, 32c 3rd phrase

Claims (3)

Detecting means for detecting the operation start and operation intensity of the performance operator;
An acquisition means for acquiring character information from a storage device in response to the start of operation of the performance operator detected by the detection means;
Pronunciation control means,
The position of the character acquired by the acquisition means is controlled according to the operation intensity of the performance operator detected by the detection means, and the sound generation control means pronounces the character acquired by the acquisition means as speech. A singing sound synthesizer characterized by   The singing sound synthesizing apparatus according to claim 1, wherein the position of the character acquired by the acquisition unit is controlled in accordance with an operation intensity at the start of operation of the performance operator detected by the detection unit.   In response to a change in operation intensity during operation of the performance operator detected by the detection means, the acquisition means acquires a character positioned next, and the character acquired by the acquisition means is converted into the pronunciation control means. The singing sound synthesizer according to claim 1, wherein the singing sound synthesizer is generated as a voice.

Images