JP5979293B2 - Singing synthesis control device and singing synthesis device - Google Patents

Description

  The present invention relates to a song synthesis control device that controls a song synthesis device in accordance with an operation of an operation unit by a user, and a song synthesis device including the song synthesis control device.

  As a singing synthesizer, melody data and lyric data indicating lyrics to be sung in accordance with the melody indicated by the melody data are input and stored via a GUI (graphical user interface) or the like, and the stored melody data and lyric data are stored. There is a configuration in which singing synthesis is performed by reproducing the songs in parallel. Recently, there has been provided a configuration in which lyrics data is stored in advance, the user sequentially reads out the lyrics data while receiving melody data generated by a keyboard operation or the like, and performs singing synthesis (for example, Patent Document 1). reference).

JP 2008-170592 A

  By the way, the conventional singing voice synthesizing apparatus described above needs to read out the lyric data and use it for singing synthesis in a state where the lyric data is stored in advance, and is inconvenient to change the lyrics used for singing synthesis. There was a problem that the lyrics could not be changed improvised during the singing synthesis.

  The present invention has been made in view of the above-described circumstances, and an object thereof is to provide technical means for facilitating change of lyrics used for singing synthesis.

  The present invention provides melody storage means for storing melody data, which is a string of pitch data for designating the pitch of notes, and correct lyric data, which is a string of phonogram data indicating each phonogram constituting the lyrics. Each time the correct lyric storage means for storing, the operation means for accepting an operation for instructing a phonogram, the operation for instructing the phonogram to the operation means is performed, and one input phonogram is determined, Synthetic control for sequentially reading pitch data from the melody storage means and synthesizing the voice having the pitch indicated by the input phonogram and having the pitch indicated by the pitch data read from the melody storage means to the singing voice synthesis means And the phonetic text from the correct lyrics storage means while advancing the position of the phonetic character data to be read in the correct lyrics data each time the one input phonogram is determined. Sequentially reading out the data, to determine whether coincident with said input phonetic characters, provide singing synthesis control apparatus characterized by comprising a verification means for outputting a determination result.

  According to this invention, since the singing voice along the melody indicated by the melody data can be synthesized according to the operation of designating the phonetic characters constituting the lyrics, the lyrics used for the singing synthesis can be easily changed. be able to. Moreover, according to this invention, it can be determined whether the phonetic character according to correct lyric data was input.

It is a block diagram which shows the structure of the song synthesis apparatus provided with the song synthesis control part which is 1st Embodiment of the song synthesis control apparatus by this invention. It is a top view which shows the external appearance of the song synthesizing | combining apparatus in the embodiment. It is a block diagram which shows the function structure of the song synthesizing | combining apparatus. It is a figure which shows the state transition of the song synthesis | combination control part. It is a figure which shows the flow of the information in the song synthesis | combination control part in a melody reproduction | regeneration mode. It is a time chart which shows the 1st operation example in melody reproduction mode. It is a time chart which shows the 2nd operation example in melody reproduction | regeneration mode. It is a time chart which shows the 3rd operation example in melody reproduction mode. It is a time chart which shows the 4th operation example in melody reproduction mode. It is a time chart which shows the 5th operation example in melody reproduction mode. It is a figure which shows the flow of the information in the melody reproduction | regeneration mode of the singing composition control part in 2nd Embodiment of this invention.

  Embodiments of the present invention will be described below with reference to the drawings.

<First Embodiment>
FIG. 1 is a block diagram showing a configuration of a singing voice synthesizing apparatus 1000 including a singing voice synthesis control unit 1 according to the first embodiment of the present invention. As shown in FIG. 1, the singing voice synthesizing apparatus 1000 includes a singing voice synthesis control unit 1, a singing voice synthesis engine 2, and an audio interface 3 according to the present embodiment.

  The singing synthesis control unit 1 includes an operation unit, and is a device that generates and outputs a synthesis control message for instructing control of singing synthesis in response to an operation performed by the user on the operation unit. The singing voice synthesis engine 2 synthesizes and outputs singing voice data that is sample data of the singing voice waveform in accordance with the synthesis control message from the singing voice synthesis control unit 1. The audio interface 3 drives the speaker 4 according to the singing voice data output from the singing voice synthesis engine 2 and emits the singing voice from the speaker 4.

  FIG. 2 is a plan view showing the outer appearance, specifically, the upper surface of the singing voice synthesizing apparatus 1000. FIG. 3 is a block diagram showing a functional configuration of the singing voice synthesizing apparatus 1000. The singing voice synthesizing apparatus 1000 has a use as a toy for intellectual development for children. As shown in FIG. 2, the singing voice synthesis apparatus 1000 has a flat plate-shaped housing. Various operators are printed on the upper surface of the casing, which serves as an operation surface of the touch panel 100 in FIG. The touch panel 100 is provided as an operation unit of the song synthesis control device 1 according to the present embodiment. Note that the operation unit of the singing synthesis control apparatus 1 is not necessarily a touch panel, and may be an operation unit including, for example, a mechanical push button that retreats by a pressing operation as an operator.

  Inside the housing of the singing voice synthesizing apparatus 1000, a pressed position detecting unit 106, an operation intensity detecting unit 107, an LED drive control unit 108, a CPU 110, a ROM 140, and a RAM 150 of the touch panel 100 shown in FIG. Yes. Here, when the user's finger presses the operation surface of the casing of the singing voice synthesizing apparatus 1000, the pressed position detection unit 106 is a device that detects the pressed position on the operation surface and outputs the pressed position information. The operation strength detection unit 107 is a device that detects the strength of the pressing operation, specifically the pressing force of the pressing operation, and outputs the operation strength information when the user's finger presses the operation surface. In addition, the operation intensity detection unit 107 outputs an on-timing signal indicating the timing when the pressing is started and an off-timing signal indicating the timing when the pressing is completed.

  As shown in FIG. 2, the touch panel 100 is provided with a 50 sound button unit 101, a 50 sound auxiliary button unit 102, a mode control button unit 103, a scale reading position control button unit 104, and a key button unit 105. ing. Of these, the 50-sound button unit 101, the 50-sound auxiliary button unit 102, the mode control button unit 103, and the scale reading position control button unit 104 have characters and symbols such as “A” displayed in a substantially square frame. It is composed of buttons. In the following, for convenience of explanation, for each of these buttons, for example, for the button indicated by “A” in the 50 sound button unit 101, the characters indicated in the button frame are indicated as “A” button. May be used to describe.

  The 50 sound button section 101 is composed of 46 50 sound buttons for designating hiragana characters from “A” to “N”. The 50-sound auxiliary button unit 102 is an operator used to assist the input of hiragana by the 50-sound button unit 101, and includes a “small” button, a “°” button, and a “” button. Here, the “small” button is an operator that is used when a character string having a small hiragana “i” in the latter half portion, such as “Tei”, is designated by the operation of the 50 sound button unit 101. In order to instruct “Tei” in this example, the “Small” button is pressed, the “I” button of the 50 sound button unit 101 is pressed, and the “T” button is further pressed. The “°” button is an operator that is used when a hiragana having a semi-voice point such as “Pa”, “Pi”, “Pu”, etc. is designated by operating the 50 sound button unit 101. For example, when it is necessary to designate “Pa”, this “°” button may be pushed and the “ha” button of the 50 sound button unit 101 may be pushed. The ““ ”button is an operator used when a hiragana having a cloud point such as“ BA ”,“ BI ”,“ BU ”, etc. is designated by operating the 50 sound button unit 101. For example, when it is necessary to designate “B”, this “゛” button may be pressed and the “H” button of the 50 sound button unit 101 may be pressed.

  The mode control button unit 103 is an operator for giving an instruction for mode switching control to the singing synthesis control unit 1. On the back side of each button in the mode control button unit 103, two-color LEDs having red and green emission colors are arranged, and “kashi” displayed on each button by the lighting control of the two-color LEDs. It is possible to make the background of characters such as red (when red light is emitted), green (when green light is emitted), and white (when light is not emitted). The LED drive control unit 108 in FIG. 3 is a device that performs drive control of the two-color LEDs on the back side of the mode control button unit 103.

  The scale reading position control button unit 104 is an operator used when, for example, synthesizing a singing song in the melody playback mode, and includes a “jump” button, a “recommend” button, and a “repeat” button. . Here, in the melody reproduction mode, melody data in which scale data indicating the scale of a note is arranged in time series is stored in the melody storage area of the RAM 150 in advance, and the 50 sound button unit 101 and the 50 sound auxiliary button unit 102 are stored. In response to the designation of the phonetic character by the operation, the scale data is read one by one from the melody storage area, and the singing voice corresponding to the designated phonetic character and the read scale data is synthesized by the song synthesis engine 2. Mode. The “skip” button is a button for instructing that when a certain scale data is to be read out in the melody data, the scale data is not to be read out and the next scale data is to be read out. is there. The “Recommend” button is a button for instructing to read out the scale data to be read out and set the next scale data to be read out when the scale data in the melody data is to be read out. . The “Repeat” button is used to select the scale data to be read when the next phonetic character is specified, after the scale data is read in the melody data according to the specification of the phonetic character. It is a button for instructing.

  The key button unit 105 is an operator simulating a keyboard, and is an array of black key buttons each representing a black key and white key buttons representing each white key. The key button unit 105 is used to specify the scale of each note constituting the melody.

  In FIG. 3, the ROM 140 stores various control programs. The CPU 110 performs a process for performing the function as the song synthesis engine 2 according to the control program in the ROM 140, and controls the song synthesis engine 2 according to the operation of the touch panel 100 as the operation unit. Various processes are performed. At that time, the CPU 110 uses the RAM 150 as a work area.

  In FIG. 3, in addition to the singing synthesis engine 2, a user interface unit 120 and a synthesis engine control unit 130 that are part of a control program executed by the CPU 110 are shown in the box indicating the CPU 110. The user interface unit 120, the synthesis engine control unit 130, and the touch panel 100 as an operation unit constitute the singing synthesis control unit 1 according to the present embodiment.

  The user interface unit 120 includes a 50 sound designation user interface unit 121, a scale designation user interface unit 122, and a control user interface unit 123. When the pressing position information indicating the position in the region of any one of the 50 sound button unit 101 and the 50 sound auxiliary button unit 102 is output from the pressing position detection unit 106, the 50 sound designation user interface unit 121 The phonetic character designated by the designated button is recognized, and phonetic character data indicating the phonetic character is output.

  For example, the position in the area of any one of the buttons (for example, “A” button) of the 50 sound button unit 101 is displayed without outputting the pressed position information indicating the position in the area of the 50 sound auxiliary button section 102. When the pressing position information is output, the input phonetic character is determined by this single button operation. In this case, the 50-sound designation user interface unit 121 outputs phonetic character data indicating the phonetic character (in this example, “A”) corresponding to the independently operated button.

  Further, for example, after the pressing position information indicating the position in the area of the “」 ”button of the 50-sound auxiliary button unit 102 is output, one of the buttons of the 50-sound button unit 101 (for example,“ ha ”button) When the pressed position information indicating the position in the area is output, the input phonetic character is determined when the latter button operation (in this example, the operation of the “ha” button) is performed. In this case, the 50-sound designation user interface unit 121 uses a phonetic character (in this example, “B”) obtained by adding a cloud point to the phonetic character corresponding to the latter button as an input phonetic character, The phonogram data shown is output. The same applies to the case where the pressing position information indicating the position in the area of any one of the 50 sound button portions 101 is output after the pressing position information indicating the position in the area of the “°” button is output.

  Further, for example, after the pressing position information indicating the position in the area of the “small” button of the 50 sound auxiliary button unit 102 is output, a 50 sound button (for example, “I”) that indicates a vowel in the 50 sound button unit 101 is output. Press position information indicating a position in the area of the button), and a press indicating a position in the area of any one of the 50 sound buttons (for example, the “te” button) in the 50 sound button portion 101. When the position information is output, the input phonetic character is determined when the third button operation is performed. In this case, the 50-sound designation user interface unit 121 arranges a phonetic character (in this example, “tei”) in which hiragana represented by the second 50-sound button is arranged after the hiragana represented by the third 50-sound button. ) As an input phonetic character, and output phonetic character data indicating the input phonetic character.

  In either case, the 50-sound designation user interface unit 121 outputs pressing position information indicating the position in the area of any one of the buttons of the 50-sound button unit 101, and when the input phonetic character is confirmed, the input is performed. Output phonogram data indicating phonograms.

  Also, the 50-sound designation user interface unit 121 determines the input phonetic character based on the pressed position information generated by the button operation of the 50-sound button unit 101 from the pressed position detection unit 106, and displays the input phonetic character. When outputting the phonetic character data, the operation intensity information output from the operation intensity detecting unit 107 is output in response to the button operation in which the input phonetic character is confirmed, and the start and end of the pressing period in the button operation are indicated. The on-timing signal and the off-timing signal are received from the operation intensity detector 107 and output.

  In addition, the 50-note designating user interface unit 121, when the pressed position detection unit 106 performs a button operation on the scale reading position control button unit 104 and outputs the pressed position information, the pressed “skip” button, “recommend” Button name information indicating the "" button or the "repeat" button is output. Also, when the button name information is output in response to the button operation of the “skip” button, the “recommend” button, or the “repeat” button, the operation intensity information output from the operation intensity detector 107 is output in response to the button operation. In addition, an on-timing signal and an off-timing signal indicating the start and end of the pressing period in the button operation are received from the operation intensity detector 107 and output.

  When the pressed position information indicating the position in the area of any key button of the key button section 105 is output from the pressed position detection unit 106, the scale designation user interface unit 122 associates with the key button including the pressed position. Output the scale data. In addition, the scale designation user interface unit 122 performs an operation output from the operation intensity detection unit 107 when the key button is operated to output the pressing position information and outputs scale data corresponding to the operated key button. In addition to outputting the strength information, an on-timing signal and an off-timing signal indicating the start and end of the pressing period in the key button operation are received from the operation strength detector 107 and output.

  When the pressing position information indicating the position in the area of any button of the mode control button unit 103 is output from the pressing position detection unit 106, the control user interface unit 123 displays button name information indicating the pressed button. Is output to the mode control unit 133 of the synthesis engine control unit 130. Further, the control user interface unit 123 is instructed by the command information when the command information on the lighting control of the two-color LED below any button of the mode control button unit 103 is given from the mode control unit 133. The LED drive control unit 108 is caused to perform lighting control (red light emission, green light emission, or off).

  The synthesis engine control unit 130 is a module that generates a synthesis control message for controlling the singing synthesis engine 2, and includes a 50 sound / phonetic symbol conversion unit 131, a performance control information generation unit 132, a mode control unit 133, And a synthesis control message assembly unit 134.

  The synthesis engine control unit 130 has a real-time performance mode, a lyrics recording mode, a melody recording mode, a lyrics playback mode, and a melody playback mode. Here, the real-time performance mode allows the user to perform lyrics input using the 50 sound button unit 101 and the 50 sound auxiliary button unit 102 and melody input using the key button unit 105 in parallel. In this mode, the singing voice synthesizing engine 2 synthesizes singing voice data corresponding to the phonetic character input by the operation of the 50 sound auxiliary button unit 102 and corresponding to the scale data input by the operation of the key button unit 105. . The lyrics recording mode is a string of phonogram data that indicates the lyrics input by the user, specifically the lyrics, by allowing the user to input lyrics using the 50 sound button unit 101 and the 50 sound auxiliary button unit 102. In this mode, the lyrics data is written in a lyrics storage area provided in the RAM 150. In the melody recording mode, the user inputs a melody using the key button unit 105, and the melody provided in the RAM 150 includes scale data indicating the scale associated with the key button pressed by the user in the key button unit 105. In this mode, data is written to the storage area.

  In the lyric reproduction mode, the user operates the key button of the key button unit 105, and in response to the key button operation, each phonogram data constituting the lyric data stored in the lyric storage area is sequentially read one by one. This is a mode in which the singing synthesis engine 2 performs singing synthesis using scale data and operation intensity information generated by button operation and phonetic symbols determined by phonogram data read from the lyrics storage area. In the melody playback mode, the user inputs a phonetic character by operating the 50-sound button unit 101 and the 50-sound auxiliary button unit 102, and the melody data stored in the melody area in response to the input of the phonetic character. Are sequentially read one by one, and phonetic symbols corresponding to phonetic characters specified by button operation, scale data read from the melody storage area, and operation intensity information determined by the intensity of the button operation. This is a mode in which the singing synthesis engine 2 performs singing synthesis.

  The mode control unit 133 in the synthesis engine control unit 130 makes transition between the above modes in accordance with the button name information output from the control user interface unit 123, and displays information indicating the current mode as 50 sounds / phonetic symbols. The data is supplied to the conversion unit 131 and the performance control information generation unit 132. Further, the mode control unit 133 outputs command information related to lighting control of the two-color LED below any one of the buttons of the mode control button unit 103 at the time of mode transition.

  FIG. 4 is a diagram illustrating mode transition of the synthesis engine control unit 130. When the synthesis engine control unit 130 is set to the real-time performance mode, the mode control unit 133 controls the lighting control command for turning off the two-color LEDs of the “scarecrow” button and the “melody” button of the mode control button unit 103. The data is sent to the user interface unit 123.

  In this real-time performance mode, when the button name information indicating the “Kashi” button is output from the control user interface unit 123, the mode control unit 133 shifts the synthesis engine control unit 130 to the lyrics recording mode, The lighting control command for causing the two-color LED of the “Shi” button to emit red light and turning off the two-color LED of the “Melody” button is sent to the control user interface unit 123.

  In this lyrics recording mode, when the button name information indicating the “Kashi” button is output from the control user interface unit 123, the mode control unit 133 causes the synthesis engine control unit 130 to transition to the lyrics reproduction mode, and “ The lighting control command for causing the two-color LED of the “Shi” button to emit green light and turning off the two-color LED of the “Melody” button is sent to the control user interface unit 123.

  In this lyrics playback mode, when the button name information indicating the “Kashi” button is output from the control user interface unit 123, the mode control unit 133 causes the synthesis engine control unit 130 to transition to the real-time performance mode, A lighting control command for turning off the two-color LEDs of both the “Shi” button and the “Melody” button is sent to the control user interface unit 123. The same applies to other cases, and the mode control unit 133 causes the mode transition as shown in FIG. 4 to be performed according to the button operation of the mode control button unit 103, and the “Kashiwa” button and Switches the display color of the “Melody” button.

  The 50 sound / phonetic symbol conversion unit 131 converts the phonetic character data output from the 50 sound designating user interface unit 121 or the phonetic character data read from the lyrics storage area of the RAM 150 into phonetic symbols, and a synthesis control message assembly unit. 134 is a module to be supplied to 134. Which information the 50 sound / phonetic symbol conversion unit 131 generates based on which information depends on the operation mode of the synthesis engine control unit 130. Therefore, in order to avoid duplication of explanation, the specific processing contents of the 50 sound / phonetic symbol conversion unit 131 will be clarified in the explanation of the operation of the present embodiment.

  The performance control information generation unit 132 uses the output information of the scale designation user interface unit 122, the read data from the melody storage area of the RAM 150, and the output data of the 50 tone designation user interface unit 121 to incorporate scale data to be incorporated into the pronunciation instruction message. This is a module that generates scale data to be incorporated into the operation intensity information or the mute instruction message and supplies it to the synthesis control message assembly unit 134. Here, the pronunciation instruction message is a synthesis control message that instructs the singing synthesis engine 2 to start synthesis of the singing voice, and the mute instruction message is a synthesis control message that instructs the singing synthesis engine 2 to mute the singing voice being synthesized. It is. How the performance control information generation unit 132 generates parameters to be included in the sound generation instruction message and the mute instruction message depends on the operation mode of the synthesis engine control unit 130. Therefore, in order to avoid duplication of explanation, the specific processing contents of the performance control information generation unit 132 will be clarified in the explanation of the operation of the present embodiment.

  When the scale reading position control button unit 104 is operated in the melody reproduction mode, the performance control information generating unit 132 performs control to change the reading position of the scale data to be read from the current one. The contents of this control are also clarified in the description of the operation of this embodiment in order to avoid duplication of explanation.

  The synthesis control message assembly unit 134 is a module that assembles a synthesis control message for controlling the song synthesis engine 2. More specifically, when the synthesis control message assembling unit 134 receives scale data and operation intensity information to be incorporated into the pronunciation instruction message from the performance control information generating unit 132, the synthesis control message assembling unit 134 assembles the pronunciation instruction message using these pieces of information, Output to engine 2. When the synthesis control message assembling unit 134 receives the scale data to be incorporated into the mute instruction message from the performance control information generating unit 132, the synthesis control message assembling unit 134 assembles the mute instruction message using this information and outputs it to the singing synthesis engine 2. When the synthesis control message assembling unit 134 receives the phonetic symbol from the 50 sound / phonetic symbol conversion unit 131, the synthesis control message assembling unit 134 assembles a phonetic symbol message indicating the phonetic symbol and outputs it to the singing synthesis engine 2.

  The song synthesis engine 2 has a function of performing song synthesis in accordance with the synthesis control message supplied in this way. More specifically, the singing voice synthesis engine 2 includes a speech segment database in which waveform data indicating waveforms of speech segments collected from various sounds collected in advance are associated with phonetic symbols. When the singing synthesis engine 2 receives the phonetic symbol message, it reads out the waveform data of the speech unit corresponding to the phonetic symbol indicated by the phonetic symbol message from the speech unit database.

  When the singing synthesis engine 2 receives the pronunciation instruction message, the singing synthesis engine 2 performs pitch conversion on the waveform data of the speech unit read from the speech unit database so as to be the scale indicated by the scale data included in the pronunciation instruction message. . In addition, the singing synthesis engine 2 adjusts the attack feeling and amplitude of the waveform data after pitch conversion according to the operation intensity information included in the pronunciation instruction message, and outputs it to the audio interface 3 to output the singing sound from the speaker 4. Start sound emission. When the singing synthesis engine 2 receives the mute instruction message, the singing synthesis engine 2 attenuates the waveform data corresponding to the same scale data as the scale data included in the mute instruction message, out of the waveform data of the speech element emitted from the speaker 4. The singing sound corresponding to the scale data is muted.

  Next, the operation of this embodiment will be described. Since the operation for setting various operation modes and the processing of the mode control unit 133 corresponding to the operations have already been described, hereinafter, the 50 sound / phonetic symbol conversion unit 131 and the performance control information generation unit 132 in each operation mode will be described. The operation will be mainly described.

a. Real-time performance mode In the real-time performance mode, the 50-sound / phonetic symbol conversion unit 131 outputs the phonetic character data in the RAM 150 each time the phonetic character data is output from the 50-sound designation user interface unit 121. Write to the area. If the phonetic character data is already stored in the current character area, the phonetic character data output from the 50-sound designating user interface unit 121 is overwritten on the phonetic character data. Further, when the on-timing signal, the scale data, and the operation intensity information are output from the scale designation user interface unit 122, the 50 tone / phonetic symbol conversion unit 131 reads the phonetic character data from the current character area of the RAM 150, and generates the phonetic symbol. To the composition control message assembling unit 134.

  On the other hand, when the on-timing signal, the scale data, and the operation intensity information are output from the scale designation user interface unit 122, the performance control information generation unit 132 supplies the scale data and the operation intensity information to the synthesis control message assembly unit 134. Causes the pronunciation instruction message to be assembled and output. Then, the performance control information generation unit 132 performs synthesis control on the mute instruction message corresponding to the sound generation instruction message assembled by the synthesis control message assembly unit 134 immediately before the off timing signal is output from the scale designation user interface unit 122. The message assembly unit 134 assembles and outputs the message.

  The above is the operation of the present embodiment in the real-time performance mode. In this real-time performance mode, the user inputs, for example, “Kare no Uta” by operating the button of the 50 sound button unit 101, and simultaneously inputs “Doremi Family” by operating the key button unit 105. Thus, the singing synthesis of the “frog song” can be performed by the singing synthesis engine 2.

b. Lyric recording mode In this lyric recording mode, each time the phonogram data is output from the phonograph designation user interface unit 121, the phonogram / phonetic symbol conversion unit 131 stores the phonogram data in the RAM 150. Write to storage area. Here, if a string of phonogram data is already stored in the lyric storage area, the 50 sound / phonetic symbol conversion unit 131 converts the phonogram data output from the 50 sound designation user interface unit 121 into the lyrics storage area. Are written in the lyrics storage area as phonogram data next to the last phonogram data in the existing phonogram data string. The string of phonogram data in the lyrics storage area is the lyrics data.

  Further, when writing one phonetic character data into the lyrics storage area, the 50 sound / phonetic symbol conversion unit 131 converts the phonetic character data into a phonetic symbol and generates a phonetic symbol message indicating the phonetic symbol as a synthesis control message. The assembly unit 134 outputs the data.

  Further, the 50 sound / phonetic symbol conversion unit 131 sends a generation control message generation instruction for lyrics confirmation to the performance control information generation unit 132. The performance control information generation unit 132 that has received the generation instruction of the lyrics confirmation synthesis control message receives a sound generation instruction message including predetermined scale data and predetermined operation intensity information and a mute instruction message including the same scale data for a predetermined time. The composition control message assembly unit 134 outputs the information at intervals.

  The above is the operation of the present embodiment in the lyrics recording mode. In this lyric recording mode, the user inputs lyric data indicating the lyric of “Kaeru no Uta” in the lyric storage area in the RAM 150 by inputting, for example, “Kaeru no Uta” by operating the button of the 50 sound button unit 101. Can write. At this time, for example, when writing the phonetic character data of “ka” in the lyrics storage area, the phonetic symbol message indicating the phonetic symbol corresponding to the “ka” is sent from the synthesis control message assembly unit 134 to the singing synthesis engine 2. Next, a pair of a sound generation instruction message and a mute instruction message including predetermined scale data is supplied from the synthesis control message assembling unit 134 to the singing synthesis engine 2, and the sound of “ka” is synthesized and emitted from the speaker 4. The The same applies to phonetic characters such as “e” and “ru” that follow. Therefore, the user can confirm the lyrics inputted by himself / herself as voice.

c. Melody Recording Mode In this melody recording mode, the performance control information generation unit 132 writes the scale data in the melody storage area provided in the RAM 150 each time scale data is output from the scale designation user interface unit 122. When the scale data string is already stored in the melody storage area, the performance control information generation unit 132 converts the scale data output from the scale designation user interface unit 122 to the existing scale data in the melody storage area. Write to the melody storage area as the scale data next to the last scale data in the row. The string of scale data in this melody storage area becomes melody data.

  Further, when the performance control information generating unit 132 writes one scale data in the melody storage area, the performance control information generating unit 132 transmits a sound generation instruction message including the scale data, predetermined operation intensity information, and a mute instruction message including the same scale data at predetermined time intervals. The information is output to the synthesis control message assembling unit 134.

  Further, the performance control information generating unit 132 performs control for causing the synthesis control message assembling unit 134 to output a phonetic symbol message prior to the pronunciation instruction message. Specifically, the performance control information generation unit 132 sets the scale data written in the melody storage area and the phonetic symbol transmission command to 50 sounds / sound before performing the control to output the sound generation instruction message to the synthesis control message assembly unit 134. This is given to the phonetic symbol converter 131. Upon receiving the scale data and the phonetic symbol transmission command, the 50 tone / phonetic symbol conversion unit 131 generates a phonetic symbol representing the scale indicated by the scale data, and the phonetic symbol message indicating the phonetic symbol is preceded by the phonetic indication message. The combination control message assembly unit 134 outputs the message. In order to enable such processing, for example, a table for associating the scale data with the phonetic symbols representing the names of the scales indicated by the scale data is stored in the ROM 140 in advance, and the 50-tone / phonetic symbol conversion unit 131 performs this processing. The scale data may be converted into phonetic symbols by referring to the table.

  The above is the operation of the present embodiment in the melody recording mode. In this melody recording mode, the user inputs, for example, “Doremi Family” by operating the button of the key button unit 105, so that the melody data indicating the melody of “Frog Song” is written in the melody storage area in the RAM 150. it can. At this time, for example, when the scale data of “do” is written in the melody storage area, the phonetic symbol message indicating the phonetic symbol corresponding to the “do” is supplied from the synthesis control message assembly unit 134 to the singing synthesis engine 2. Next, a pair of a sound generation instruction message and a mute instruction message including the scale data of “do” is supplied from the synthesis control message assembly unit 134 to the singing synthesis engine 2, and the voice of “do” having the scale of “do” is synthesized. The sound is emitted from the speaker 4. The same applies to the subsequent scales such as “Le” and “Mi”. Therefore, the user can confirm the melody input by the user as voice.

d. Lyrics playback mode In this lyrics playback mode, the 50 tone / phonetic symbol conversion unit 131 outputs lyrics in the lyrics storage area of the RAM 150 each time an on-timing signal, scale data, and operation intensity information are output from the scale designation user interface unit 122. One phonogram data at the current lyric reading position is read out from each phonogram data constituting the data. The 50 sound / phonetic symbol conversion unit 131 converts the read phonetic character data into phonetic symbols, and causes the synthesis control message assembly unit 134 to output a phonetic symbol message indicating the phonetic symbols. Here, in the initial state, the lyrics reading position is the position of the first phonetic character data in each phonetic character data constituting the lyrics data in the lyrics storage area. Therefore, for example, when the lyrics data of “Kaeru no Uta” is stored in the lyrics storage area, the phonetic character data indicating the head phonetic character “ka” is read out first.

  On the other hand, when the on-timing signal, the scale data, and the operation intensity information are output from the scale designation user interface unit 122, the performance control information generation unit 132 causes the 50 tone / phonetic symbol conversion unit 131 to output the above phonetic symbol message. Is then output to the synthesis control message assembly unit 134, and the synthesis control message assembly unit 134 is caused to output a pronunciation instruction message including scale data and operation intensity information output from the scale designation user interface unit 122. The 50 sound / phonetic symbol conversion unit 131 advances the lyric reading position by 1 in response to the output of the pronunciation instruction message by the synthesis control message assembly unit 134.

  Then, the performance control information generation unit 132 performs synthesis control on the mute instruction message corresponding to the sound generation instruction message assembled by the synthesis control message assembly unit 134 immediately before the off timing signal is output from the scale designation user interface unit 122. The message assembly unit 134 assembles and outputs the message.

  The above is the operation of the present embodiment in the lyrics playback mode. In this lyrics playback mode, the user inputs “Doremi Family” by operating the key button of the key button unit 105 in a state where the lyrics data indicating the lyrics of “Frog Song” is stored in the lyrics storage area in the RAM 150, for example. In response to the input of “do”, the phonetic character data of the phonetic character “ka” is read from the lyrics storage area, the phonetic symbol message indicating the phonetic symbol of “ka”, and the pronunciation of the scale “do” Is issued to the singing voice synthesis engine 2. In addition, the phonetic character data of the phonetic character “e” is read from the lyrics storage area in response to the next input of “le”, and the phonetic symbol message indicating the phonetic symbol of “e” and the scale “le”. A pronunciation instruction message for instructing the pronunciation of the song is supplied to the singing synthesis engine 2. The same applies hereinafter. Therefore, the user performs a key button operation on the key button unit 105 to cause the singing synthesis engine 2 to synthesize the singing sound sung in accordance with the melody played by the key button operation on the lyrics indicated by the lyrics data in the lyrics storage area. be able to.

e. Melody Reproduction Mode FIG. 5 is a diagram showing the flow of information in the singing synthesis control unit 1 in this melody reproduction mode. The operation in this melody reproduction mode is premised on that melody data, which is a string of scale data for designating the scale of notes constituting a song, is stored in advance in the melody storage area in the RAM 150. In this melody playback mode, the 50 tone / phonetic symbol conversion unit 131 outputs the output phonetic character data each time the on-timing signal, the phonetic character data, and the operation intensity information are output from the 50 sound designation user interface unit 121. The phonetic symbol is converted to a phonetic symbol, and a phonetic symbol message indicating the phonetic symbol is output to the synthesis control message assembling unit 134.

  On the other hand, when the on-timing signal, phonogram data, and operation intensity information are output from the 50-sound designation user interface unit 121, the performance control information generation unit 132 causes the 50-sound / phonetic symbol conversion unit 131 to respond to this. After outputting the phonetic symbol message to the synthesis control message assembling unit 134, one scale data at the current scale reading position is read out from each scale data constituting the melody data in the melody storage area of the RAM 150. Here, in the initial state, the scale reading position is the position of the first scale data in each scale data constituting the melody data in the melody storage area. Then, the performance control information generation unit 132 sends the scale data read from the melody storage area and the operation intensity information output from the 50-sound designation user interface unit 121 to the synthesis control message assembly unit 134 to assemble the pronunciation instruction message and its Cause output to occur. When the sound generation instruction message is output, the performance control information generation unit 132 advances the scale reading position by one.

When the off-timing signal is output from the 50-sound designation user interface unit 121, the performance control information generation unit 132 synthesizes a mute instruction message corresponding to the sound generation instruction message assembled by the synthesis control message assembly unit 134 immediately before. The control message assembly unit 134 assembles and outputs it.
The above is the basic operation in the melody playback mode.

  Next, a specific operation example of the melody playback mode will be described. FIGS. 6A and 6B and FIGS. 7 to 10 are time charts each showing an operation example of the singing synthesis control unit 1 in the melody reproduction mode. 6A and 6B and FIGS. 7 to 10, the horizontal axis is a time axis. In these drawings, the pressing operation of each button of the 50 sound button unit 101, the 50 sound auxiliary button unit 102, or the scale reading position control button unit 104 is shown in time series. For example, in FIG. 6A, in the waveform indicating the pressing operation of the “ka” button, the section at the L level indicates the section where the “ka” button is pressed and turned on. The same applies to waveforms indicating other button operations. In addition, for example, in FIG. 6A, in the waveform indicating the operation of the “ka” button, the section that is at the L level (the section in which the “ka” button is ON) is “ka (do)”. There is a notation. This notation indicates that a voice corresponding to the phonetic symbol “ka” and having a scale of “do” is generated by operating the button in this section. The same applies to other notations such as “e” and “ru”. In these figures, the generation timing of a phonetic symbol message, a sound generation instruction message, and a mute instruction message generated in response to a button operation is shown in time series. Further, in these drawings, the scale reading positions that are switched in response to the generation of the sound generation instruction message or the operation of the control button unit 104 are shown in time series.

  In the operation example shown in FIGS. 6A and 6B, melody data indicating a melody of “Frog's Song” “Doremi Family ...” is stored in advance in the melody storage area. In this operation example, a string of phonograms “Kaeda no Utaga”, which is the lyrics of “Kaeru no Uta”, is input by operating the 50 sound button unit 101 and the 50 sound auxiliary button unit 102.

  First, when the “ka” button is pressed and an on-timing signal, phonogram data indicating “ka”, and operation intensity information are output from the 50-sound designating user interface unit 121, the 50-sound / phonetic symbol conversion unit 131 The phonetic character data indicating “ka” is converted into a phonetic symbol, and the synthesis control message assembling unit 134 assembles and outputs a phonetic symbol message indicating the phonetic symbol of “ka”.

  The performance control information generating unit 132 waits for the phonetic symbol message to be assembled and output, and the scale data at the current scale reading position among the scale data constituting the melody data in the melody storage area. The composition control message assembling unit 134 reads out and assembles the pronunciation instruction message including the scale data and the phonetic character data indicating “ka” and the operation intensity information output from the 50-sound designating user interface unit 121. Let it be done. Here, in the initial state, the scale reading position is the position of the first scale data of the melody data of “Kaeru no Uta”. Therefore, the scale data of “do” is read from the melody storage area and used for assembling the pronunciation instruction message.

  When the sound generation instruction message is output in this way, the performance control information generation unit 132 advances the scale reading position by one. Thus, the scale reading position becomes the position of the scale data of “Le” which is the second scale data in the melody data.

  Next, when the user lifts his finger from the “ka” button and an off timing signal is output from the 50-sound designation user interface unit 121, the performance control information generation unit 132 causes the synthesis control message assembly unit 134 to assemble it immediately before. The mute instruction message corresponding to the pronunciation instruction message, that is, the mute instruction message including the scale data of “do” is caused to be assembled by the synthesis control message assembling unit 134 and output. Therefore, the singing voice synthesis engine 2 synthesizes the voice of “ka” that has a scale of “do” and continues for the period during which the “ka” button is pressed.

  Next, when the “e” button is pressed and the on-timing signal, the phonetic character data indicating “e”, and the operation intensity information are output from the 50-sound designation user interface unit 121, the 50-sound / phonetic symbol conversion unit 131 , The synthesis control message assembly unit 134 assembles and outputs a phonetic symbol message indicating the phonetic symbol of “e”. Further, the performance control information generation unit 132 reads the scale data of “Le” at the current scale reading position among the scale data constituting the melody data in the melody storage area, and the scale data of “Le”, The composition control message assembling unit 134 assembles and outputs a pronunciation instruction message including the phonetic character data indicating “e” and the operation intensity information output from the 50-sound designation user interface unit 121. Then, the performance control information generation unit 132 advances the scale reading position by one. Next, when the user releases his / her finger from the “e” button and an off timing signal is output from the 50 sound designation user interface unit 121, the performance control information generation unit 132 causes the synthesis control message assembly unit 134 to assemble immediately before. The mute instruction message corresponding to the pronunciation instruction message, that is, the mute instruction message including the scale data of “R” is made to be assembled by the synthesis control message assembly unit 134 and output. Thereafter, the same operation is performed when the “RU” button, the “NO” button, the “U” button, and the “TA” button are pressed.

  Then, when the user presses the “ka” button while pressing the “ボ タ ン” button, the input phonetic character “ga” is fixed by the pressing operation of the “ka” button. For this reason, at the pressing start timing of the “ka” button, the on-timing signal, the phonetic character data indicating “ga”, and the operation intensity information about the “ka” button are output from the 50-sound designation user interface unit 121. In response to this, the 50 tone / phonetic symbol conversion unit 131 causes the synthesis control message assembly unit 134 to assemble and output a phonetic symbol message indicating the phonetic symbol of “ga”. The performance control information generation unit 132 reads the scale data of “do” at the current scale reading position among the scale data constituting the melody data in the melody storage area, and the scale data of “do”, The composition control message assembling unit 134 assembles and outputs a pronunciation instruction message including the phonetic character data indicating “ga” and the operation intensity information output from the 50-sound designation user interface unit 121. Then, the performance control information generation unit 132 advances the scale reading position by one.

  When the pronunciation instruction message is given, the singing voice synthesis engine 2 starts synthesizing the voice of “GA” having the scale of “DO”. In order to increase the duration of “A”, which is the extended sound part of the voice “GA”, the user may keep pressing the “KA” button. When the user removes the finger from the “ka” button, the performance control information generating unit 132 causes the synthesis control message assembling unit 134 to output a mute instruction message including the scale data of “do”. As a result, the singing voice synthesis engine 2 silences the extended sound portion “A” of “GA” which has been synthesized in the scale of “DO”.

  Each of the operation examples shown in FIGS. 7 to 10 is an operation example for synthesizing a singing voice of a percussion song in the melody reproduction mode. Also in these operation examples, melody data indicating the melody of “Frog song” “Doremi Family R ...” is stored in advance in the melody storage area.

  In the operation example shown in FIG. 7, the three-character lyrics “Butano” is input as the lyrics for the part “Doremifa” consisting of four notes of the melody of “Kaeru no Uta”. In order to correspond to the situation where the lyrics are “not enough characters” for the melody, in the operation example shown in FIG. 7, the user operates the “jump” button of the scale reading position control button unit 104.

  More specifically, in the operation example shown in FIG. 7, the user presses the “゛” button and presses the “F” button during the period when the “「 ”button is ON. In this case, the 50-sound designating user interface unit 121 receives the on-timing signal, the phonetic character data indicating “bu”, and the operation intensity information corresponding to the pressing force of the “fu” button at the timing of starting the “fu” button. Output. As a result, the 50 sound / phonetic symbol conversion unit 131 causes the synthesis control message assembly unit 134 to output a phonetic symbol message indicating the phonetic symbol of “bu”. The performance control information generating unit 132 reads the scale data of “do” at the current scale reading position in the melody data in the melody storage area, and the scale data of “do” and the phonetic sound indicating “bu”. The synthesis control message assembling unit 134 is caused to output a sound generation instruction message including the character data and the operation intensity information output from the 50 sound designation user interface unit 121. Then, the performance control information generation unit 132 advances the scale reading position by one. As a result, the scale reading position becomes the position of the second scale data of the melody data in the melody storage area, that is, the scale data position of “R”.

  Next, the user releases his / her finger from the “F” button. As a result, an off timing signal is output from the 50 sound designation user interface unit 121, and the performance control information generation unit 132 causes the synthesis control message assembly unit 134 to output a mute instruction message corresponding to the sound output instruction message output immediately before, The synthesis of the voice of “bu” in the scale of “do” in the singing voice synthesis engine 2 is terminated.

  Then, after the user releases the finger from the “fu” button to finish synthesizing the voice of “bu” in this way, the user presses the “skip” button. As a result, the button name information indicating the “jump” button, the on-timing signal indicating the start of pressing the “jump” button, and the operation intensity information corresponding to the pressing force of the “jump” button are output from the 50 sound designation user interface unit 121. The In this case, the performance control information generation unit 132 advances the scale reading position by 1 in response to the output of the button name information indicating the “jump” button from the 50-sound designation user interface unit 121. As a result, the scale reading position is changed from the position of the second “le” scale data in the melody data to the position of the third “mi” scale data.

  Next, the user presses the “ta” button. As a result, the on-timing signal, phonogram data indicating “ta”, and operation intensity information are output from the 50-sound designation user interface unit 121. As a result, the 50 sound / phonetic symbol conversion unit 131 causes the synthesis control message assembly unit 134 to output a phonetic symbol message indicating the phonetic symbol of “ta”. Further, the performance control information generation unit 132 reads the scale data of “M” at the current scale reading position in the melody data in the melody storage area, and the scale data of “M” and the phonetic sound indicating “TA”. The synthesis control message assembling unit 134 is caused to output a sound generation instruction message including the character data and the operation intensity information output from the 50 sound designation user interface unit 121. Then, the performance control information generation unit 132 advances the scale reading position by one. Thus, the scale reading position becomes the position of the fourth scale data of the melody data in the melody storage area, that is, the scale data position of “F”.

  As described above, in the operation example shown in FIG. 7, after the synthesis of the “bu” voice by the singing voice synthesis engine 2 is finished, the “jump” button is pressed, so that the scale reading position is the second “le” scale. When the position of the data is changed to the position of the third “Mi” scale data, and the next phonetic character “Ta” after “B” is instructed, the third “Mi” scale data is read out. The voice of “Ta” in the scale of “Mi” is synthesized by the singing voice synthesis engine 2.

  Also in the operation example shown in FIG. 8, the user operates the “jump” button of the scale reading position control button unit 104 in order to deal with the situation where the lyrics are “not enough characters” for the melody. The difference between the operation example shown in FIG. 8 and the operation example shown in FIG. 7 is in the timing of pressing the “skip” button. In the operation example shown in FIG. 7, the user releases his / her finger from the “fu” button, finishes synthesizing the voice of “bu” by the singing voice synthesis engine 2, and then presses the “jump” button. On the other hand, in the operation example shown in FIG. 8, the user presses the “fu” button with a finger and presses the “skip” button during the period when the singing synthesis engine 2 is synthesizing the voice of “bu”. Yes. Therefore, within the period in which the singing voice synthesis engine 2 is synthesizing the voice of “bu”, the scale reading position is the third “mi” scale from the position of the second “le” scale data in the melody data. It is changed to the data position.

  In the operation example shown in FIG. 9, the user operates the “recommend” button of the scale reading position control button unit 104 in order to cope with the situation where the lyrics are “not enough characters” for the melody. More specifically, in the operation example shown in FIG. 9, the user presses the “fu” button with a finger and presses the “recommend” button within the period in which the singing voice synthesis engine 2 synthesizes the voice of “bu”. Is pushing.

  Thus, the “recommend” button is pressed during the period in which the “bu” voice synthesis is performed by the singing synthesis engine 2, and the on-timing signal, the button name signal, and the operation intensity information about the “recommend” button are 50. When output from the sound designating user interface unit 121, the performance control information generating unit 132 reads the scale data of “Re” at the current scale reading position in the melody data in the melody storage area, and the scale of “Re” is read. The synthesis control message assembling unit 134 is caused to output a sound generation instruction message including data and operation intensity information output from the 50-sound designating user interface unit 121 together with button name information indicating a “recommend” button. Next, the performance control information generation unit 132 generates a mute instruction message corresponding to the pronunciation instruction message output to the synthesis control message assembly unit 134 immediately before, that is, a mute instruction message including scale data of “do”. Output to the unit 134. Then, when the user releases his / her finger from the “fu” button and an off timing signal for the “fu” button is output from the 50-sound designating user interface unit 121, the performance control information generating unit 132 immediately assembles the synthesis control message. The synthesis control message assembling unit 134 outputs a mute instruction message corresponding to the pronunciation instruction message output to the unit 134, that is, a mute instruction message including the scale data of “R”. The operation when the “ta” button is pressed is the same as the operation example shown in FIG. 7 and the operation example shown in FIG. 8, and the voice of “ta” having the scale of “mi” is generated by the singing synthesis engine 2. Synthesized.

  In the operation example shown in FIG. 9, the singing synthesis engine 2 includes a phonetic symbol message indicating a phonetic symbol of “bu” and scale data of “do” when the user starts pressing the “fu” button. Upon receipt of the pronunciation instruction message, synthesis of the voice of “bu” in the scale of “do” is started. Then, when the user starts pressing the “recommend” button, a sound generation instruction message including the scale data “re” and a mute instruction message including the scale data “do” are received. At this time, the singing voice synthesis engine 2 is synthesizing the voice of “U”, which is the extended sound part of the voice of “bu”, but is synthesizing by receiving the pronunciation instruction message including the scale data of “le”. Switch the scale of the voice of “u” from “do” to “le”. When the user releases his / her finger from the “fu” button, the singing synthesis engine 2 receives a mute instruction message including the scale data of “re”, and synthesizes the sound of “u” in the scale of “le”. finish. Therefore, in the operation example shown in FIG. 9, the phonetic characters “bu” and “ta” are input to the melody “doremi”, and the “recommend” button is used during the synthesis of the speech corresponding to the phonetic character “bu”. By pressing, the voices of bu (do), u (re), and ta (mi) (in parentheses are the scale of the voice) are synthesized.

  Strictly speaking, in this example of operation, the singing voice synthesis engine 2 performs the voice in the scale of “do” from the time when the “le” pronunciation start message is given until the “do” mute instruction message is given. And the voices in the scale of “Le” are overlapped and synthesized. Therefore, in order to make the transition from the scale “do” to the scale “le” more natural, the singing synthesis engine 2 uses the pronunciation duration period (in this example, “ ) ”And the pronunciation continuation period specified by the pair of the later pronunciation instruction message and the mute instruction message (in this example, the pronunciation continuation period of“ Le ”) overlap, At the point of time when the pronunciation instruction message (in this example, “le” pronunciation instruction message) is given, the volume of the sound that has been pronounced by the pronunciation instruction message in the preceding pair (in this example, “do” pronunciation instruction message) May be faded out.

  In the operation example shown in FIG. 10, the five-character lyrics “Sumamano” are input as the lyrics for the part “Doremifa” consisting of four notes in the melody of “Kaeru no Uta”. In order to correspond to the situation where the lyrics are “remaining characters” with respect to the melody, in the operation example shown in FIG. 10, the user operates the “repeat” button of the scale reading position control button unit 104.

  More specifically, in the operation example shown in FIG. 10, when the “shi” button is pressed, a phonetic symbol message indicating the phonetic symbol of “shi” is sent to the song synthesis engine 2 and the first “ The scale data of “do” is read out, and a pronunciation instruction message including the scale data of “do” is sent to the singing voice synthesis engine 2. Then, the scale reading position is advanced from the position of the scale data of “do” to the position of the scale data of “re”.

  Next, when the “MA” button is pressed, a phonetic symbol message indicating the phonetic symbol of “MA” is sent to the singing voice synthesis engine 2 and the scale data of “RE” at the current scale reading position in the melody data is displayed. The pronunciation instruction message including the scale data of “L” is sent to the singing voice synthesis engine 2. Then, the scale reading position is advanced from the position of the scale data of “Le” to the position of the scale data of “Mi”.

  In the operation example shown in FIG. 10, the “repeat” button is pressed during the period in which the “ma” button is ON. In this case, the 50-sound designation user interface unit 121 outputs an on timing signal, button name information, and operation intensity information for the “repeat” button at the pressing start timing of the “repeat” button.

  When the button name information about the “repeat” button is output in this way, the performance control information generation unit 132 changes the scale reading position from the position of the scale data of “mi” to the position of the scale data of “re”. Only returned.

  Then, when the “u” button is pressed after the “ma” button, a phonetic symbol message indicating the phonetic symbol of “u” is sent to the singing voice synthesis engine 2, and at the current scale reading position in the melody data. The scale data of “R” is read out, and a pronunciation instruction message including the scale data of “L” is sent to the singing synthesis engine 2. Then, the scale reading position is advanced from the position of the scale data of “Le” to the position of the scale data of “Mi”.

  Next, when the “MA” button is pressed after the “U” button, a phonetic symbol message indicating the phonetic symbol of “MA” is sent to the singing synthesis engine 2 and at the current scale reading position in the melody data. The scale data of “M” is read, and a pronunciation instruction message including the scale data of “M” is sent to the singing voice synthesis engine 2. Then, the scale reading position is advanced from the position of the scale data of “Mi” to the position of the scale data of “F”.

  Therefore, in the operation example shown in FIG. 10, the phonetic character “shi”, “ma”, “u”, “ma”, which is one character larger than the melody “doremi”, is input and the second phonetic character “ Pressing the “Repeat” button while synthesizing the voice corresponding to “ma” will synthesize the voice of “do”, “ma”, “le”, “ma”, and “ma” (in parentheses the scale of the voice). Will be. As described above, since one scale data can be repeatedly applied to two consecutive phonetic characters in the lyrics by pressing the “repeat” button, the situation where the lyrics are “character remainder” with respect to the melody is dealt with. be able to. In the example of operation shown in FIG. 10, in order to repeatedly apply the scale data of “L” to the second phonetic character “MA” and the third phonetic character “U”, the former phonetic character “MA” is used. ”During the voice synthesis, but before the latter phonetic character“ U ”is input, the voice synthesis of the former phonetic character“ MA ”ends. You may press the “Repeat” button later.

  As described above, according to the melody playback mode in the present embodiment, melody data, which is a sequence of scale data, is stored in advance in the melody storage area of the RAM 150, and the operation of the 50 sound button unit 101 and the 50 sound auxiliary button unit 102 is performed. By inputting the phonetic character, the scale data is sequentially read from the melody storage area while the scale reading position is advanced in accordance with the input of the phonetic character, corresponding to the input phonetic character, and from the melody storage area. A voice having a scale corresponding to the read scale data can be synthesized by the singing synthesis engine 2.

  By the way, an apparatus for synthesizing a voice in response to an operation of an operation unit has been provided so far. For example, as a toy for children's educational development, a toy that has a button with 50 sounds displayed, for example, synthesizes the sound of “A” and outputs it from the speaker in response to the occurrence of an “A” button on-event. Is provided. According to this type of toy, the child who is a user can find a character by reading and pressing the button of a certain character, so that the user can remember the character while enjoying the pronunciation. it can.

  However, in this toy, the 50 sounds designated by the button operation are always sounded in a certain scale, so even if 50 sounds are continuously input, the sound becomes dry and there is no musical element.

  On the other hand, according to the melody reproduction mode of the present embodiment, when a phonetic character is input by operating the 50 sound button unit 101 and the 50 sound auxiliary button unit 102, the melody reproduction mode is in a scale according to the stored melody data. Since each phonetic character is pronounced, the child who is the user can remember the characters entered as lyrics while listening to the synthesized phonetic characters being synthesized as a song.

  In addition, according to the present embodiment, scale data to be applied to phonograms sequentially input by operation of the operation unit is determined according to the melody data stored in the melody storage area, so that lyrics input for the same song is repeated. You can go and synthesize the singing voice of the same song. Therefore, the practice of inputting lyrics can be repeated.

  Further, in the melody playback mode of the present embodiment, the sound of the phonetic character is generated by the pressing force (operation intensity) of the button operation when inputting the phonetic character by the button operation of the 50 sound button unit 101. Operation intensity information is determined. Therefore, the musical expression of the singing voice can be performed by the button operation of the 50 sound button unit 101 for inputting lyrics.

  Further, according to the melody reproduction mode of the present embodiment, the user can freely input the lyrics that are pronounced in accordance with the melody data stored in advance. Therefore, the singing composition engine 2 can synthesize the singing of the replacement song. It is also possible to sing a song by improvising a part of the lyrics with other lyrics.

  When the singing composition engine 2 performs singing of a replacement song, the lyrics input by operating the 50 sound button unit 101 and the 50 sound auxiliary button unit 102 compared to the number of notes indicated by the melody data stored in the melody storage area There may be a situation where the number of phonetic characters is insufficient, or a situation where there is an excess of letters. According to the present embodiment, it is possible to deal with the “character shortage” situation by the operation of the “skipping” button and “recommend”, and it is possible to deal with the “character remainder” situation by the operation of the “repeat” button. Also, in the “character shortage” situation, by turning on the “Recommend” button while a certain voice is sounding, the scale data is read from the melody storage area, and the scale of the extended sound part in the sound being pronounced is read. The scale data indicated by the read scale data can be changed. Therefore, it is possible to realize a smooth and natural singing even in a situation where the character is insufficient. In the operation example of FIG. 9 accompanied by the operation of the “recommend” button, when the “recommend” button is pressed during the sounding of the sound of “bu”, the operation intensity information generated by the depressing operation of this “recommend” button. Is applied to the pronunciation instruction message of the extended sound part “U” of “BU”, but the operation intensity information generated by pressing the “FU” button that has confirmed the input phonetic character “BU”, that is, “ The operation intensity information applied to the pronunciation instruction message of the voice “” may be applied.

Second Embodiment
FIG. 11: is a block diagram which shows the function structure of the song synthesis control part which is 2nd Embodiment of this invention. The song synthesis control unit according to the present embodiment also has a melody reproduction mode similar to that in the first embodiment. FIG. 11 shows each part in the singing voice synthesis control section related to the melody reproduction mode, and also shows a flow of information between the parts in the melody reproduction mode.

  As shown in FIG. 11, the singing composition control unit according to the present embodiment is a program executed by the CPU 110 (see FIG. 3) as a 50-sound designation user interface unit 121, scale designation similar to that of the first embodiment. In addition to the user interface unit 122, the control user interface unit 123, the 50 sound / phonetic symbol conversion unit 131, the performance control information generation unit 132, and the synthesis control message assembly unit 134, a collation unit 135 is provided. In the melody playback mode, the collation unit 135 collates the phonogram data output from the 50-sound designation user interface unit 121 with the correct lyric data given in advance, and the phonogram data is correct as the correct lyric. This is a module for determining whether or not the data is character data and whether or not the output timing of the phonetic character data is appropriate.

  In the present embodiment, a pair of melody data indicating the melody of each song and correct lyric data constituting a correct lyric of the song is stored in the ROM 140 (see FIG. 3) for a plurality of types of songs. ing. The singing voice synthesizing device provided with the singing voice synthesizing unit is provided with a tune selection operation unit. The user operates the operation unit to set a pair of melody data and correct lyric data of a desired song. Is read from the ROM 140, the melody data is stored in the melody storage area of the RAM 150, and the correct lyric data is stored in the correct lyric storage area of the RAM 150.

  In the melody playback mode, the collation unit 135 reads out phonetic character data indicating each phonetic character of the correct lyrics one by one from the correct lyrics region at predetermined time intervals. Note that the time interval for reading out the phonetic character data may be specified by, for example, operating the operator. Or the period data which shows the reading time interval of the phonetic character data defined for every music are included in correct answer lyrics data, and the collation part 135 makes phonetic character data be the time interval which the period data in this correct answer data show. You may make it read.

  When the phonetic character data that matches the phonetic character data is output from the 50-sound designating user interface unit 121 within a predetermined allowable time after reading out one phonetic character data, 135 determines that the user has correctly input the lyrics. On the other hand, when the phonogram data is not output from the 50-tone designation user interface unit 121 within a predetermined allowable time after reading one phonogram data, or the phonogram data is output within the allowable time. However, if the phonogram data does not match the phonogram data read from the correct lyric area, the collation unit 135 determines that the user has entered an incorrect lyric.

  The collation unit 135 supplies the determination result obtained in this way to the control user interface unit 123. The casing of the singing voice synthesizing apparatus according to the present embodiment is provided with a correct answer lamp for indicating that correct lyrics have been input and an error lamp for indicating that incorrect lyrics have been input. The control user interface unit 123 turns on the correct lamp when receiving the determination result that the correct lyrics input has been performed, and turns on the error lamp when receiving the determination result that the incorrect lyrics input has been performed. Light up.

  Also, the collation unit 135 sends a warning sound generation instruction to the synthesis control message assembly unit 134 when a determination result indicating that an incorrect lyrics input has been performed is obtained. Upon receiving this warning sound generation instruction, the synthesis control message assembling unit 134 assembles a synthesis control message for synthesizing a warning sound such as a buzzer sound, and supplies it to the singing voice synthesis engine 2 (see FIG. 3) to output the warning sound. .

  Note that when the collation unit 135 reads out one phonogram data from the correct lyric region in order to inform the user when to input the phonograms constituting the lyrics, the synthesis control message assembly unit 134, for example, A pronunciation instruction message for synthesizing the metronome sound may be assembled and supplied to the song synthesis engine 2.

  According to the present embodiment, the user has not only input lyrics but synthesizes a song with a melody corresponding to melody data stored in advance, and has also input an appropriate phonetic character at an appropriate timing. The determination result about whether or not can be received. Therefore, there is an effect that the interest of singing synthesis is enhanced. In the present embodiment, the correct answer is determined when a phonetic character according to the correct lyrics is input for all types of input phonetic characters. However, in order to input the phonetic character “bu” having a cloud point, it is necessary to operate the “” and “fu” buttons. , “°” and “ha” buttons are required. To enter the phonetic character “Tii” that contains a small hiragana in the latter half, enter the “small” button, “i” button, and “te”. ”Button operation is required. In this way, depending on the type of input phonetic character, it may be harsh to request a complete button operation at an appropriate timing. Therefore, for example, for input phonograms that require multiple button operations for input, the last button operation (in the above example, “F”, “HA”, “TE” button operations) is performed correctly. Therefore, it may be determined that correct lyrics have been input, and singing synthesis corresponding to correct lyrics may be performed. That is, for example, if the phonetic character of the correct lyrics is “BA”, if “HA” is input, it is treated as a correct answer, and the singing voice using the phonetic symbol “BA” is synthesized. In this embodiment, it is determined whether or not the input timing of the phonetic character is appropriate and whether or not the input phonetic character is appropriate. Instead, it may be determined only whether or not the input phonetic character is appropriate.

<Other embodiments>
Although the first and second embodiments of the present invention have been described above, other embodiments can be considered in addition to this. For example:

(1) In the first embodiment, melody data for various songs such as “Inu no Omarisan” and “Tulip” is stored in the ROM 140 in advance, and an operator for selecting a song is a housing of the singing synthesizer 1000. The melody data of the song selected by the operation of the operator may be read from the ROM 140 and stored in the melody storage area of the RAM 150.

(2) In the operation example shown in FIG. 9 of the first embodiment, the scale of the sound “Bu” that is continuously sounding when the “recommend” button is pressed is “do”, and the “recommend” button is pressed. Therefore, the scale applied to the extended sound portion “U” of the voice “BU” is “R”, and the pitch difference between the two is small. For this reason, even when the pronunciation instruction message including the scale data of “L” is sent to the singing synthesis engine 2 at the time of pressing the “recommend” button, the singing is smooth.

  However, the pitch difference between the scale data read from the melody storage area and the scale data previously read from the melody storage area by pressing the “recommend” button may increase. In view of this situation, the following configuration may be adopted.

  First, in the melody playback mode, when scale data is read from the melody area, the scale data is stored in the read scale data buffer. Then, when the performance control information generation unit 132 reads scale data from the melody storage area in response to the pressing operation of the “recommend” button, the performance control information generation unit 132 calculates the scale data read and the preceding scale data in the read scale data buffer. Find the pitch difference. Then, based on this pitch difference, it continuously takes a predetermined time from the scale indicated by the preceding scale data in the read scale data buffer to the scale indicated by the scale data read in response to the pressing operation of the “recommend” button. A pitch bend message that changes the pitch is output to the synthesis control message assembly unit 134. According to this aspect, it is possible to make the song synthesized by the lyric input accompanied by the operation of the “recommend” button in the “character shortage” situation smoother.

(3) The speech segment waveform data collected from the voices of various singers is stored in the speech segment database, and a desired singer's speech segment is selected by operating the operation unit of the singing synthesis control unit for singing synthesis. You may make it use.

(4) In each of the above embodiments, the singing composition control unit 1, the singing composition engine 2, and the audio interface 3 are integrated, but the singing composition control unit having the singing composition control unit 1, and the singing It is good also as a separate apparatus from the song synthesis apparatus which has the synthesis engine 2 and the audio interface 3. In this case, the former song synthesis control device outputs a MIDI message as a synthesis control message, and the latter song synthesis device receives a MIDI message which is a synthesis control message via the MIDI interface, and synthesizes a song sound. May be. The latter singing voice synthesizing apparatus may be one in which application software that is a singing voice synthesis engine is installed in a personal computer.

(5) In the above aspect (4), the singing synthesis control device may be connected to the singing synthesis device via a network such as the Internet. Here, for example, it is assumed that the singing synthesis control device is owned by a grandchild, and the singing synthesis device is installed in the grandparent's house. Further, it is assumed that speech segment data obtained from a grandchild's speech that has been collected in advance is stored in the speech segment database of the singing synthesizer. According to this aspect, the grandchild sets the singing composition control device in the melody reproduction mode, generates a MIDI message (synthetic control message) for singing composition by inputting lyrics, and sends the MIDI message to the grandparents' home via the network. By supplying it to the singing synthesizer, the grandparents can hear the singing of their own voice.

(6) In each of the above embodiments, the program executed by the CPU 110 of the song synthesis control unit may be provided as an application program for a tablet computer or the like having a GUI (graphical user interface) using a touch panel.

(7) In the second embodiment, the collation unit 135 may be configured not to advance the reading position of the phonetic character data in the correct lyrics data when the user inputs wrong lyrics. Alternatively, a configuration may be adopted in which it is possible to specify whether or not to advance the reading position of the phonetic character data when the user inputs wrong lyrics by operating the operator.

(8) The singing composition control apparatus takes in data with lyrics in the SMF (Standard MIDI File) format from the outside, and stores it in the RAM 150 as the melody data and lyrics data in the first embodiment or the correct lyrics data in the second embodiment. You may make it store.

(9) In the real-time performance mode, after the scale input by the key button unit 105 is performed, the phonetic character is input by the 50 sound button unit 101, and the input timing of the phonetic character by the 50 sound button unit 101 is performed. The voice may be pronounced at. Or you may comprise so that it can select by operation of an operation element by which method sounding is performed.

(10) In each of the above embodiments, the 50 sound designation user interface unit 121, the scale designation user interface unit 122, the control user interface unit 123, the 50 sound / phonetic symbol conversion unit 131, the performance control information generation unit 132, and the mode control unit 133, information exchange between the modules such as the composition control message assembling unit 134 and the collating unit 135 may be performed directly between the modules, or between the modules via an API (Application Programming Interface). Information may be exchanged.

DESCRIPTION OF SYMBOLS 1000 ... Singing synthesis apparatus, 1 ... Singing synthesis control part, 2 ... Singing synthesis engine, 3 ... Audio interface, 4 ... Speaker, 100 ... Touch panel, 101 ... 50 sound button part, 102 ... 50 sound auxiliary button part, 103 ... Mode Control button section 104 ... Scale reading position control button section 105 ... Key button section 106 ... Pressing position detection section 107 ... Operation intensity detection section 108 ... LED drive control section 120 ... User interface section 121 ... 50 sounds Designated user interface unit, 122 ... scale designation user interface unit, 123 ... control user interface unit, 130 ... synthesis engine control unit, 131 ... 50 sound / phonetic symbol conversion unit, 132 ... performance control information generation unit, 133 ... mode control Part 134... Composition control message assembly part 135 135 collation part 140. M, 150 ... RAM.

Claims (5)

Melody storage means for storing melody data, which is a string of pitch data for designating the pitch of notes,
Correct lyric storage means for storing correct lyric data which is a string of phonogram data indicating each phonogram constituting the lyrics;
An operation means for receiving an operation for instructing a phonetic character;
Each time the operation means is instructed to indicate a phonetic character and one input phonetic character is confirmed, the pitch data is sequentially read from the melody storage means, and the voice indicated by the input phonetic character is used. Synthesis control means for synthesizing the singing voice synthesizing voice having the pitch indicated by the pitch data read from the melody storage means;
Each time the input phonogram of one character is determined, the phonogram data is sequentially read from the correct lyric storage means while advancing the position of the phonogram data to be read in the correct lyric data, and the input phonogram A singing composition control apparatus comprising: a collating unit that determines whether or not the two match, and outputs a determination result. Transmitting means for transmitting a synthesis control message for instructing control of song synthesis to the song synthesis means;
An operation intensity detecting means for detecting an operation intensity of the operation means that has confirmed the input phonetic character;
The synthesis control unit causes the transmission unit to transmit a synthesis control message including a phonetic symbol indicated by the input phonogram, and detects the pitch and the operation intensity detected by the pitch data read from the melody storage unit. 2. The singing composition control apparatus according to claim 1, wherein a composition control message for instructing sound generation at an intensity corresponding to the operation intensity of the operation means detected by the means is transmitted to the transmission means.   In response to an operation instructing the operation unit to change the position of reading the pitch data in the melody storage unit, the synthesis control unit reads the pitch data according to the operation. The singing composition control apparatus according to claim 1, further comprising a position control unit. A singing synthesis control device according to any one of claims 1 to 3,
A singing voice synthesizing device comprising: singing voice synthesizing means for synthesizing a singing voice under the control of the singing voice synthesis control device.   5. The singing voice synthesizing apparatus according to claim 4, wherein the singing voice synthesizing unit lowers the volume of the preceding voice when starting to synthesize the later voice while synthesizing the preceding voice. 6. .

Images