JP5935815B2 - Speech synthesis apparatus and program - Google Patents

Description

  The present invention relates to speech synthesis.

  A technology for synthesizing voice of a singing sound according to a character string and sound when a character string such as lyrics and pitches and lengths of a plurality of notes (hereinafter referred to as “note string”) are input as data. Known (for example, Patent Document 1).

JP 2006-259768 A

In Patent Document 1, in order to check whether a character string and a note string are input as intended, the user needs to perform an operation for reproducing the synthesized voice after inputting lyrics. Met.
On the other hand, the present invention provides a technique that makes it easier to check a voice according to a newly input note and a character assigned to the note.

  The present invention relates to a character string storage means for storing a character string, and for each of a plurality of notes, the beginning of the pronunciation period, the pitch and the length of the note, and the character assigned to the note among the character string In accordance with a coordinate system having attribute storage means for storing attributes including a first axis that represents pitch and a second axis that represents time, the beginning, pitch, and length of the pronunciation period of each of the plurality of notes are determined. Display control means for displaying a figure to be displayed on the screen of the display means, acquisition means for acquiring the pitch of a new note and the beginning of the pronunciation period designated on the screen, and the pronunciation period acquired by the acquisition means And a part of the character string stored in the character string storage means as the new note based on the context of the beginning of the pronunciation period of the plurality of notes stored in the attribute storage means. To assign An assigning means for rewriting the attribute stored in the attribute storage means, a character string assigned to the new note by the assigning means, and a sound corresponding to the pitch obtained by the obtaining means is synthesized as a confirmation sound. And a confirmation sound output means for outputting the confirmation sound synthesized by the speech synthesis means from the sound output means in accordance with the timing at which the acquisition means acquires the beginning of the pronunciation period. A synthesizer is provided.

  In a preferred aspect, the acquisition means acquires the end of the new note pronunciation period after acquiring the start of the sound generation period, and the acquisition means acquires the end of the new note generation period. When the pitch of the new note acquired by the acquisition unit is changed before, the voice synthesis unit synthesizes a voice according to the changed pitch as the confirmation sound, and the confirmation sound The output means may cause the sound output means to output the confirmation sound according to the changed pitch.

  In another preferred embodiment, the voice synthesizing unit synthesizes, as the confirmation sound, a sound corresponding to a character string assigned to a plurality of sounds including the new note and a designated pitch as the confirmation sound. May be.

  In still another preferred embodiment, the acquisition means further acquires the end of the pronunciation period of the new note, and the speech synthesis means uses the confirmation period as the end of the pronunciation period and the start of the pronunciation period. You may synthesize | combine the audio | voice which has the sound length according to the difference.

  Further, the present invention provides a computer, a character string storage means for storing a character string, and for each of a plurality of notes, the beginning of the sound generation period, the pitch and the length of the note, and the note of the character string. According to a coordinate system having attribute storage means for storing attributes including characters assigned to and a first axis representing pitch and a second axis representing time. , And a display control means for displaying a graphic representing the sound length on the screen of the display means, an acquisition means for acquiring the pitch of the new note and the beginning of the pronunciation period specified on the screen, and acquired by the acquisition means A part of the character string stored in the character string storage means based on the context of the start time of the generated sound period and the start time of the sound generation period of the plurality of notes stored in the attribute storage means. The new An assigning means for rewriting the attribute stored in the attribute storage means so as to assign it to a note; a character string assigned to the new note by the assigning means; and a sound corresponding to the pitch obtained by the obtaining means Voice confirmation means for synthesizing as a confirmation sound, and a confirmation sound output for outputting the confirmation sound synthesized by the voice synthesis means from the voice output means according to the timing at which the acquisition means acquires the beginning of the pronunciation period A program for functioning as a means is provided.

  According to the present invention, compared with a case where the confirmation sound is not output according to the timing at which the beginning of the pronunciation period is acquired, the user can input the newly input note and the character assigned to the note. It is possible to more easily confirm the sound corresponding to the above.

It is a block diagram which shows the function structure of the speech synthesis apparatus 1 which concerns on one Embodiment. It is a figure which illustrates a segment library. It is a figure which illustrates musical score data. 2 is a diagram illustrating a hardware configuration of the speech synthesizer 1. FIG. 1 is a diagram illustrating an appearance of a speech synthesizer 1. FIG. It is a figure which illustrates a screen when the speech synthesis application is being executed. 3 is a flowchart showing the operation of the speech synthesizer 1. It is a figure which illustrates a mode that a new note is input. It is a figure which illustrates a mode that the end of the pronunciation period is designated about a new note. It is a figure which illustrates processing when a note figure is dragged. It is a figure which illustrates processing when a note figure is dragged in the direction of a pitch axis.

1. Configuration FIG. 1 is a block diagram showing a functional configuration of a speech synthesizer 1 according to an embodiment. The speech synthesizer 1 is a device that synthesizes speech based on musical score data including a character string and a note string and outputs the synthesized speech. The speech synthesizer 1 roughly has a function of editing score data and a function of synthesizing speech based on the score data. More specifically, the speech synthesizer 1 includes a storage unit 11, a display control unit 12, a display unit 13, an acquisition unit 14, an assignment unit 15, a speech synthesis unit 16, a confirmation sound output unit 17, And an audio output means 18. The storage means 11 stores a segment library, lyrics (character strings), and score data. The musical score data has, for each of a plurality of notes, an attribute including the beginning of the sound generation period, the pitch, and the length of the note, and the character assigned to the note in the character string. The display control means 12 displays a graphic representing the beginning of the pronunciation period, the pitch, and the length of each of a plurality of notes in accordance with a coordinate system having a first axis representing the pitch and a second axis representing the time. On the screen. The display unit 13 displays an image under the control of the display control unit 12. The acquisition means 14 acquires the pitch of the new note and the start of the pronunciation period specified on the displayed screen. The allocating unit 15 is stored in the storage unit 11 based on the front and back relationship between the beginning of the pronunciation period acquired by the acquisition unit 14 and the beginning of the pronunciation period of a plurality of notes stored in the storage unit 11. The attribute stored in the storage means 11 is rewritten so that a part of the character string is assigned to a new note. The voice synthesizing unit 16 synthesizes a voice corresponding to the character string assigned to the new note by the assigning unit 15 and the pitch acquired by the acquiring unit 14 as a confirmation sound. The confirmation sound output means 17 causes the sound output means 18 to output the confirmation sound synthesized by the voice synthesis means 16 in accordance with the timing at which the acquisition means 14 acquires the beginning of the pronunciation period. The sound output means 18 outputs sound.

  FIG. 2 is a diagram illustrating a fragment library. The segment library is a database including musical segments (singing voice fragments) sampled from, for example, a human voice. The segment library is divided into individual databases corresponding to each of a plurality of singers. In the example shown in FIG. 2, the segment library includes personal databases 303a-c corresponding to three singers, respectively. The individual database 303 corresponding to each singer includes a plurality of pieces of segment data collected from the singing voice waveform of the singer. The segment data is voice data obtained by extracting and encoding a phonetic feature from a singing voice waveform.

  Here, the case of singing the lyrics of “sai” with respect to the segment data will be described as an example. The lyrics “Saita” are expressed by phonetic symbols “saita”. When analyzing the waveform of the sound represented by the phonetic symbol “saita”, the rising portion of the sound of “s” → the sound of “s” → the transition portion from the sound of “s” to the sound of “a” → “ The sound of “a” is continued, and ends with the attenuation part of the sound of “a”. Each piece of data is audio data corresponding to these phonetic features.

  In the following description, segment data corresponding to a rising portion of a sound represented by a phonetic symbol is represented as “#s” by adding “#” in front of the phonetic symbol. The segment data corresponding to the sound attenuation part represented by a phonetic symbol is represented as “a #” by adding “#” after the phonetic symbol. Also, segment data corresponding to a transition portion from a sound represented by a certain phonetic symbol to a sound represented by another phonetic symbol is inserted between those phonetic symbols, and “sa” is entered. ". The segment data group 3030 of the segment library stores segment data regarding all sounds and combinations of sounds collected from a singing voice waveform when a singer normally sings. In the segment data groups 3031H to L, elements related to all sounds and combinations of sounds collected from a singing voice waveform when a singer sings with a strong accent, a medium accent, and a weak accent, respectively. One piece of data is stored. In the segment data groups 3032H to L, elements related to all sounds and sound combinations collected from the singing voice waveform when the singer sings with strong legato, medium legato and weak legato added, respectively. One piece of data is stored.

  FIG. 3 is a diagram illustrating score data. The score data includes one or more part data representing a singing performance. In addition to the part data, the musical score data includes data indicating the time and tempo used in the performance, and data indicating the resolution. The part data includes note data which is a set of data indicating basic attributes and additional attributes for each of a plurality of notes. The basic attribute data is data indicating attributes indispensable for instructing sound generation, and includes a pitch, a sound generation period (the start and end of the sound generation period), and a sound generation symbol. The additional attribute data is data for giving an expression to the sound, such as giving a musical modification. In this example, the correspondence between the note and the lyrics, the strength of the sound, the strength of the accent, the legato Strength, vibrato strength, and vibrato duration.

  Next, the outline of the speech synthesis process by the speech synthesizer 16 will be described. Here, the process for the phonetic symbol string “sakura” included in the score data will be described as an example. The speech synthesizer 16 decomposes the phonetic symbol string into units of segment data. For example, “sakuura” is “#s”, “s”, “sa”, “a”, “ak”, “k”, “ku”, “u”, “ur”. ”,“ R ”,“ r−a ”,“ a ”, and“ a # ”. The speech synthesizer 16 reads the segment data corresponding to each of the dissociated phonetic symbol strings from the segment data group 3030. The voice synthesizer 16 adjusts the pitch of the read segment data based on the pitch indicated by each note. Furthermore, the speech synthesizer 16 processes the segment data according to the additional attribute data. The speech synthesizer 16 adjusts the duration of the segment based on the pronunciation period indicated by the note sequence for the segment data for which the pitch has been adjusted. The voice synthesizer 16 adjusts the volume of the segment data whose duration has been adjusted. The voice synthesizing unit 16 sequentially joins the segment data whose volume has been adjusted to generate synthesized voice data. The voice synthesis unit 16 stores the generated synthesized voice data in the storage unit 11.

  When the user inputs a music playback instruction and the acquisition unit 14 acquires the instruction, the audio output unit 18 reads out the synthesized voice data stored in the storage unit 11 and outputs a sound corresponding to the voice. As a result, the user can listen to the singing performance indicated by the score data.

  FIG. 4 is a diagram illustrating a hardware configuration of the speech synthesizer 1. In this example, the speech synthesizer 1 is a computer, a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a storage unit 104, an input unit 105, The display unit 106, a DAC (Digital Analog Converter) 107, an amplifier 108, and a speaker 109 are included. The CPU 101 is a microprocessor that performs general-purpose data processing. The ROM 102 is a non-volatile memory that stores a control program such as BIOS (Basic Input / Output System). The RAM 103 is a volatile memory that stores data. The storage unit 104 is a non-volatile storage device such as an HDD (Hard Disk Drive) or a flash memory. The storage unit 104 stores an OS (Operating System), application programs, and various data. The CPU 101 controls other components of the speech synthesizer 1 according to the BIOS, OS, or application program.

  The input unit 105 is a device for inputting instructions or data, for example, a touch panel (touch screen), a keypad, or a mouse. The display unit 106 includes a display device such as a liquid crystal display or an organic EL (Electro-Luminescence) display, and a drive circuit that drives the display device, and displays characters, graphics, and the like. The DAC 107 acquires audio data such as synthesized audio data and converts it into an analog audio signal. The DAC 107 outputs an analog audio signal to the amplifier 108. The amplifier 108 amplifies the analog audio signal and outputs it to the speaker 109. The speaker 109 outputs a sound wave corresponding to the analog audio signal.

  In this example, the storage unit 104 stores an application program for causing a computer to function as a speech synthesizer (hereinafter, this program is referred to as “speech synthesis application”). When the CPU 101 executes the speech synthesis application, the function shown in FIG. The CPU 101 executing the speech synthesis application is an example of the display control unit 12, the acquisition unit 14, the allocation unit 15, the speech synthesis unit 16, and the confirmation sound output unit 17. The ROM 102, the RAM 103, or the storage unit 104 is an example of the storage unit 11. The display unit 106 under the control of the CPU 101 is an example of the display unit 13. The DAC 107, the amplifier 108, and the speaker 109 under the control of the CPU 101 are examples of the audio output unit 18.

  FIG. 5 is a diagram illustrating the appearance of the speech synthesizer 1. In this example, the speech synthesizer 1 is a touch panel type information display device, and includes a housing 110 and a touch panel 111. The housing 110 is provided with a speaker 109 and a touch panel 111. The touch panel 111 has a structure in which a light transmissive touch sensor is stacked on a screen of a display device. The user inputs an instruction to the speech synthesizer 1 by touching the touch panel 111 while watching the displayed image or by tracing the touch panel 111 with a finger. That is, the touch panel 111 is a unit in which the input unit 105 and the display unit 106 are integrated.

2. Operation FIG. 6 is a diagram illustrating a screen when the speech synthesis application is being executed. This screen includes an input box 201, a window 202, a guide graphic 203, a note graphic 204, a note graphic 205, a note graphic 206, a note graphic 207, a note graphic 208, a play button 209, and a stop button 210. The input box 201 is an area for inputting and displaying lyrics. In this example, the character string “Higaru Agaru” comes as the lyrics. The window 202 is an area for inputting and displaying a musical note string in accordance with a coordinate system having a first axis representing the pitch (vertical axis in this example) and a second axis representing the time (horizontal axis). As for the pitch axis, the upward direction in FIG. 6 is the positive direction (the sound becomes higher). In the time axis, the right direction in FIG. 6 is the positive direction (time is later). The guide graphic 203 is a graphic indicating the pitch, and is displayed along the pitch axis of the window 202. In this example, a figure imitating a piano keyboard is used as the guide figure 203. Therefore, the display of the note string using the window 202 is called “piano roll display”. The guide graphic 203 includes an image (in this example, characters “C3” and “C4”) that specify the pitch. The notebook figures 204-208 are figures showing each note constituting the note string. In this example, note figures 204-208 have a rectangular shape, with the left end indicating the beginning of the sounding period and the right end indicating the end of the sounding period. The vertical position of the note graphic 204-208 indicates the pitch. Characters (part of the lyrics) assigned to the note are displayed inside the note graphic 204-208. In this example, the letters “a”, “sa”, “ga”, “ku”, and “ru” are assigned to the notes indicated by the note graphics 204-208, respectively. The playback button 209 is a button for instructing playback of the music displayed on the piano roll. The stop button 210 is a button for stopping the reproduction of music.

  FIG. 7 is a flowchart showing the operation of the speech synthesizer 1. The flow in FIG. 7 starts, for example, in response to an instruction from the user to start the speech synthesis application. In step S <b> 100, the CPU 101 acquires a character string indicating lyrics in accordance with the operation of the touch panel 111 by the user. That is, the user operates the touch panel 111 to input lyrics. The CPU 101 stores the character string acquired as lyrics in the storage unit 104. Here, as shown in FIG. 6, an example will be described in which the user inputs the lyrics “Asakura Hiru comes and comes”.

  In step S110, the CPU 101 determines whether the input of a new note has been detected, that is, whether a new note has been acquired. More specifically, the CPU 101 determines whether the start of a new note generation period has been acquired. If it is determined that the start of a new note has been input (S110: YES), the CPU 101 proceeds to step S120. When it is determined that a new note start time is not input (S110: NO), the CPU 101 waits until a new note start time is input. In this example, since the speech synthesizer 1 is a touch panel type information display device, the start of a new note is input via the touch panel 111.

  FIG. 8 is a diagram illustrating a state in which a new note is input. In the window 202, the user inputs a new note by touching a position at which a new note is to be added, more specifically, a position corresponding to the pitch of the new note and the beginning of the pronunciation period. FIG. 8 shows an example in which the user is located between the note graphic 204 and the note graphic 205 on the time axis and touches the position corresponding to “D3” with the finger F on the pitch axis. At this time, the touch panel 111 outputs a signal indicating the position touched by the user to the CPU 101. When the signal from the touch panel 111 indicates that a certain position in the window 202 is touched, the CPU 101 determines that a new note has been input. At this time, the CPU 101 displays a note graphic 211 indicating a new note at a position corresponding to the position touched by the user in the window 202. The width of the note pattern of the new note in the time axis direction is set to a predetermined initial value (for example, a quarter note). Hereinafter, this new note is referred to as a “processing target note” as necessary.

  Refer to FIG. 7 again. In step S120, the CPU 101 assigns lyrics to the processing target note. Details are as follows. First, the CPU 101 specifies the order of the processing target notes based on the position of the processing target notes on the time axis, particularly the context with other notes. In the example of FIG. 8, since the position of the processing target note is specified between the note graphic 204 and the note graphic 205, the CPU 101 specifies that the processing target note is the second sound. Next, the CPU 101 determines a character to be assigned to the processing target note based on the identified order. In this example, since the processing target note is the second sound, the CPU 101 assigns the second character “sa” of the lyrics “Asagari Hirugaru Kurugarakuru” to the processing target note. Furthermore, the CPU 101 changes the assignment of characters to other notes as new characters are assigned. Since the note to be processed is the second sound, the previous second sound is the third sound, and the order is lowered one by one. In this example, the character assigned to the sound indicated by the note graphic 205 is changed from “sa” to “ga”. The same applies to other notes. The CPU 101 stores data indicating the correspondence relationship between the note and the lyrics after the change in the RAM 103.

  In step S130, the CPU 101 synthesizes a confirmation sound. Here, the “confirmation sound” refers to a sound synthesized according to the pitch of the processing target note and the character assigned to the note. Here, a case where a musical note indicated by the note graphic 211 ("sa" is assigned) in FIG. The CPU 101 converts the character assigned to the note into a phonetic symbol. The storage unit 104 stores a database for converting hiragana to phonetic symbols, and the CPU 101 performs conversion with reference to this database. In this example, “sa” is converted to “sa”. The CPU 101 decomposes the phonetic symbols into unit data units. In this example, “sa” is decomposed into “#s”, “s”, “sa”, “a”, and “a #”. The CPU 101 reads out segment data corresponding to each of the decomposed phonetic symbol strings from the segment data group 3030 of the segment database stored in the storage unit 104. The CPU 101 adjusts the pitch of the read segment data based on the pitch indicated by the musical note. In this example, the segment data is adjusted so that the pitch is “D3”. The CPU 101 adjusts the duration of the segment based on the initial value of the sound generation period (for example, a quarter note) for the segment data on which the pitch is adjusted. The CPU 101 adjusts the volume of the segment data whose duration has been adjusted. The CPU 101 sequentially joins the piece data whose volumes are adjusted, and generates confirmation sound data indicating the confirmation sound. The CPU 101 stores the generated confirmation sound data in the RAM 103.

  In step S140, the CPU 101 causes the speaker 109 to output a confirmation sound. That is, the CPU 101 controls the DAC 107 so as to output a confirmation sound. The DAC 107 reads the confirmation sound data from the RAM 103, converts it into an analog audio signal, and outputs it to the amplifier 108. The amplifier 108 amplifies this analog audio signal and outputs it to the speaker 109. The speaker 109 outputs a sound wave corresponding to the analog audio signal. In this way, the sound “sa” is output from the speaker 109. This sound has a pitch corresponding to the pitch indicated by the note graphic 211 and a pronunciation period corresponding to a predetermined initial value (for example, a quarter note).

  Thereafter, when an operation for determining the end of the pronunciation period (for example, an operation for releasing the touched finger from the touch panel 111) is detected, the CPU 101 determines the allocation of the lyrics to the note string. That is, the CPU 101 rewrites the musical score data stored in the storage unit 104 based on the correspondence between the notes and lyrics stored in the RAM 103. In this example, note data in which the pitch is “D3”, the beginning of the sound generation period is the time indicated by the note graphic 211, and the end of the sound generation period is the time corresponding to a quarter note is the second sound. Added. Further, the order of the second and subsequent sounds is lowered, and the assigned phonetic symbols are also changed.

  Thus, according to the speech synthesizer 1, when the user touches the window 202, a note graphic corresponding to the touched position is displayed. At this time, a confirmation sound corresponding to the touched position and the character assigned to the note is output. The user can easily confirm the sound corresponding to the pitch and the assigned character for the newly input note.

  When the user touches a position corresponding to the playback button 209 on the touch panel 111, the CPU 101 synthesizes voice according to the score data stored in the storage unit 104 and causes the speaker 109 to output the synthesized voice. When the user touches a position corresponding to the stop button 210 on the touch panel 111, the CPU 101 stops the sound reproduction.

3. Other Embodiments The present invention is not limited to the above-described embodiments, and various modifications can be made. Hereinafter, some modifications will be described. Two or more of the following modifications may be used in combination.

3-1. Modification 1
The length of the confirmation sound may be dynamically determined according to the user's operation. In the above embodiment, the length of the confirmation sound is set to a predetermined value (quarter note). However, the confirmation sound may be continuously output while the user continues to touch the touch panel 111. Here, as in the embodiment, a case will be described as an example in which a musical note indicated by the note graphic 211 (assigned “sa”) in FIG. 8 is a processing target. Conversion to phonetic symbols, decomposition of phonetic symbols, reading of segment data, pitch adjustment, duration adjustment, and volume adjustment are as already described. Next, the CPU 101 joins the piece data and generates confirmation sound data. In this example, the CPU 101 generates confirmation sound data without including the last segment data in the segment data. Among the segment data of “#s”, “s”, “sa”, “a”, and “a #”, the confirmation sound data includes “#s”, “s”, “sa”, And “a”, but not “a #”. The CPU 101 writes the generated confirmation sound data in the RAM 103.

  In this example, the CPU 101 monitors a signal from the touch panel 111 at a predetermined time interval. While it is detected that the touch panel 111 is being touched, the CPU 101 uses the second piece data from the end, in this example, the piece data “a” as the confirmation sound data to be continuously output. Write to the RAM 103 repeatedly. While the user keeps touching the touch panel 111, the segment data “a” continues to be repeatedly written, so that the speaker 109 emits a long sound such as “soooo”. When the end condition is satisfied, such as when the user lifts the finger from the touch panel 111, the CPU 101 uses the last piece data in the order, in this example “a #”, as confirmation sound data to be output next. Write to the RAM 103. That is, when the user removes his / her finger from the touch panel 111, the output of the sound “soooo” stops.

3-2. Modification 2
As another example of the first modification, while the user continues to touch the touch panel 111, a confirmation sound having a predetermined sound length (for example, a quarter note) may be repeatedly output. In this case, while the user continues to touch the touch panel 111, a confirmation sound is repeatedly output from the speaker 109, such as “smallness”.

3-3. Modification 3
The timing for outputting the confirmation sound is not limited to that described in the embodiment. In the above-described embodiment, the confirmation sound is output when the user touches the touch panel 111, that is, when the start period of the sound generation period of the processing target note is designated. However, the confirmation sound may be output at a timing different from this, for example, when the end of the pronunciation period of the processing target note is designated.

  FIG. 9 is a diagram illustrating a state in which the end of the pronunciation period is specified for the processing target note. In this example, after the processing target note is generated, if the finger is moved sideways (so-called “dragging”) while touching the touch panel 111, the note graphic changes according to the trajectory touched by the finger. . In this example, the end of the sound generation period changes according to the drag while the start time of the sound generation period is fixed. That is, after detecting the input of the processing target note, the CPU 101 changes the end of the pronunciation period in accordance with the locus of the touched part, and displays a note graphic corresponding to the change. FIG. 9 illustrates a state in which the notebook graphic 211 is dragged in the right direction (time axis positive direction) from the state of FIG. In this case, the notebook graphic 211 extends in the time axis direction according to the trajectory of the finger. When the user removes his / her finger from touch panel 111, the length of the note is determined. That is, when the CPU 101 detects that the user's finger has left the touch panel, the CPU 101 determines the note length of the processing target note. After the sound length is determined, the CPU 101 synthesizes speech to generate confirmation sound data, and writes the generated confirmation sound data in the RAM 103.

  In this example, the confirmation sound is output after the sound length is determined. Therefore, the user can easily confirm the sound corresponding to the pitch, the sound length, and the assigned character for the newly input note. In addition, since the start period is determined before the end of the pronunciation period is confirmed, even when a confirmation sound is output after the sound length has been confirmed, the confirmation sound is `` according to the timing when the start period of the pronunciation period is acquired ''. It can be said that it is outputting.

3-4. Modification 4
The processing when the user drags the note graphic is not limited to that described in the second modification. In this example, when the note graphic is dragged, the start time of the sound generation period changes according to the dragging while the sound length remains constant. That is, after detecting the input of the processing target note, the CPU 101 changes the start period of the sound generation period according to the locus of the touched portion, and displays a note graphic corresponding to the change. In order to keep the sound length constant, the CPU 101 changes the end period according to the change of the start period of the sound generation period. The start period changes in units of a predetermined resolution (for example, an eighth note). The CPU 101 checks the positional relationship in the time axis direction with other sounds every time the start of the sound generation period changes, and changes the allocation of lyrics as necessary.

  FIG. 10 is a diagram illustrating processing when a note graphic is dragged according to the fourth modification. FIG. 10 illustrates a state where the notebook graphic 211 is dragged in the right direction (time axis positive direction) from the state of FIG. In this case, the CPU 101 specifies the note that appears next in the movement direction (time axis positive direction) as the monitoring target note with reference to the note graphic 211. Immediately after starting to move rightward from the state of FIG. 8, the note indicated by the note graphic 205 is specified as the monitoring target note. The CPU 101 determines whether the start period of the sound generation period of the processing target note is after the start period of the sound generation period of the monitoring target note. When it is determined that the start of the sound generation period of the processing target note is after the start of the sound generation period of the monitoring target note, the CPU 101 switches the assigned character between the processing target note and the monitoring target note. FIG. 10 shows a state in which the note graphic 211 is moved behind the note graphic 205 and the characters assigned to the note graphic 211 and the note graphic 205 are switched from the state of FIG. When the user removes his / her finger from touch panel 111, the beginning of the pronunciation period and the assigned character are determined. That is, when the CPU 101 detects that the user's finger has left the touch panel, the assigned character is determined. After the assigned character is determined, the CPU 101 performs voice synthesis to generate confirmation sound data, and writes the generated confirmation sound data in the RAM 103.

  In this example, in a configuration in which the start of the sound generation period can be moved, a confirmation sound is output after the start is confirmed. Therefore, the user can easily confirm the sound corresponding to the pitch and the assigned character for the newly input note.

3-5. Modification 5
The direction in which the note graphic can be dragged is not limited to the time axis direction. In this example, when a note graphic is dragged in the pitch axis direction, the pitch changes according to the dragging while the pitch is kept constant. That is, after detecting the input of the processing target note, the CPU 101 changes the pitch according to the locus of the touched portion, and displays a note graphic corresponding to the change. During dragging in the pitch axis direction, the beginning of the sounding period and the sound length are kept constant.

  FIG. 11 is a diagram illustrating processing when a note graphic is dragged in the pitch axis direction. FIG. 11 illustrates a state where the note graphic 211 is dragged upward (pitch axis positive direction) from the state of FIG. FIG. 11 shows a state in which the note graphic 211 is moved to the treble side and the pitch is changed from “D3” to “E3” from the state of FIG. When the user removes his / her finger from touch panel 111, the pitch is fixed. That is, when the CPU 101 detects that the user's finger has left the touch panel, the pitch is determined. During this time, as described in the first modification, the confirmation sound is continuously output while the user is touching the touch panel 111. That is, every time the pitch changes, the CPU 101 adjusts the pitch, adjusts the duration, and generates confirmation sound data for the already read segment data, and stores the generated confirmation sound data in the RAM 103. Write. According to this example, the user touches a position corresponding to the pitch “D3”, the pitch is changed from “D3” to “D # 3”, and “D # 3” to “E3” while touching the finger. When dragging to, a confirmation sound corresponding to the pitch “D # 3” is output at the timing when the pitch is changed from “D3” to “D # 3”, and the pitch is further changed from “D # 3” to “E3”. A confirmation sound corresponding to the pitch “E3” is output at the timing when “ In another example, as described in the second modification, the confirmation sound may be repeatedly output in a certain unit, and the pitch may be changed according to the drag.

3-6. Modification 6
The method of designating the start and end of the sound generation period of the processing target note is not limited to that described in the embodiment. When the touch panel 111 is a so-called multi-touch capable device that can detect a plurality of positions at the same time, the start and end of the sound generation period may be specified simultaneously using two fingers. In this case, the CPU 101 acquires the coordinates of two points from the touch panel 111. The CPU 101 processes a point having a small time axis coordinate as a starting point and a point having a large time axis coordinate as an end point among the two acquired coordinates. When the user removes his / her finger from touch panel 111, the start and end of the sound generation period are fixed. In this case, the CPU 101 may continuously output a confirmation sound while detecting a touch as in the first and second modifications, or the sound generation period as in the third and fourth modifications. A confirmation sound may be output after the start and end times are determined. For example, when the user first touches the index finger (first finger) to input the beginning of the pronunciation period, and then touches the middle finger (second finger) to input the end of the pronunciation period, the CPU 101 The confirmation sound may be output at the timing when the touch is detected, or the confirmation sound may be output at the timing when the touch of the middle finger is detected. When outputting the confirmation sound at the timing when the touch of the index finger is detected, the CPU 101 may stop outputting the confirmation sound at the timing when the touch of the middle finger is detected, or output the confirmation sound even after detecting the touch of the middle finger. May be continued. When outputting the confirmation sound after detecting the middle finger touch, the CPU 101 may stop outputting the confirmation sound at a timing when the touch of the index finger or the middle finger is not detected.

3-7. Modification 7
The sound output as the confirmation sound is not limited to that described in the embodiment. A plurality of notes including the processing target note and a sound corresponding to the character string assigned to them may be synthesized and output as a confirmation sound. For example, the CPU 101 generates confirmation sound data by using a note sequence of a certain number of notes before and after the processing target note (for example, one note before and after). For example, in FIG. 8, when a note graphic 211 is input as a processing target note, the CPU 101 generates confirmation sound data of a total of three sounds (“a”, “sa”, and “ga”) including one sound before and after. To do. In this case, a voice “Asa” is output from the speaker 109, and the user can easily make a voice corresponding to the pitch and the assigned character of the newly input note, including the relationship with the preceding and following notes. Can be confirmed. In the confirmation of singing synthesis, it is important to confirm the connection with the sound before and after. For example, when outputting only one note to be processed to which the lyrics “sa” is assigned as a confirmation sound, the elements “#s”, “s”, “sa”, “a”, and “a #” are output. Although piece data is used, when reproducing an actual song, the piece data corresponding to the combination with the preceding and following lyrics is used instead of the piece data of “#s” and “a #”. For this reason, as in this modification, a more accurate synthesized sound can be confirmed by generating a confirmation sound using a plurality of note strings including the processing target notes.

3-8. Modification 8
The method for designating the pitch of the processing target note and the start of the sound generation period, that is, the input interface is not limited to that described in the embodiment. For example, in the piano roll display of FIG. 6, the user may specify the pitch by touching the guide graphic 203 and drag to the window 202 as it is to specify the start of the sound generation period. The guide figure 203 is not limited to a figure imitating a piano keyboard. Any figure may be used as long as it indicates the pitch.

3-9. Modification 9
Although an example in which a new note is a processing target note has been described in the above-described embodiment, the processing target note is not limited to a new note. Already input notes may be processed as processing target notes. For example, in the example of FIG. 6, when a touch to a position corresponding to the note graphic 205 is detected, the CPU 101 may generate a confirmation sound using the note indicated by the note graphic 205 as a processing target note. In this case, since the tone length of the processing target note is fixed, the CPU 101 synthesizes and outputs a confirmation tone having a length indicated by the tone length recorded in the score data. Alternatively, in another example, when the speech synthesizer 1 is operating in the edit mode for editing the input notes, it is assumed that the sound length has not been determined, and the confirmation sound is described as described in the embodiment or the modification. May be synthesized and output. More specifically, when the end of the sound generation period can be changed by dragging in the edit mode, a confirmation sound may be output at the timing when the touch is detected as described in the embodiment, or the end of the sound generation period may be A confirmation sound may be output after confirmation. When a confirmation sound is output at the timing when a touch is detected, a confirmation sound of a predetermined length may be output, or the confirmation sound may be continuously output until no touch is detected. .

3-10. Other Modifications The speech synthesizer 1 may not have the touch panel 111. For example, the speech synthesizer 1 may have a mouse, keypad, or pen tablet as the input unit 105. The speech synthesizer 1 is not limited to a touch panel type information display device. The voice synthesizer 1 may be a personal computer, a mobile phone, a portable game machine, a portable music player, or an electronic book reader.

  The hardware configuration of the speech synthesizer 1 is not limited to that described with reference to FIG. As long as the functions shown in FIG. 1 can be implemented, the speech synthesizer 1 may have any hardware configuration. For example, the speech synthesizer 1 may have dedicated hardware (circuit) corresponding to each of the functional elements shown in FIG. In another example, some of the hardware components of the speech synthesizer 1 illustrated in FIG. 4 may be a so-called external device. For example, the display unit 106 or the speaker 109 may be an external device.

The character string is not limited to hiragana. An alphabet or a phonetic symbol may be used as a character string indicating lyrics.
The characters displayed in the note graphic in the piano roll display are not limited to those described in the embodiment. In addition to hiragana, which is part of the lyrics, a corresponding phonetic symbol may be displayed together.
The structure of the score data is not limited to that illustrated in FIG. Data having any structure may be used as long as the correspondence between the notes and the lyrics and the attributes of the notes can be specified. In the embodiment, the example in which the lyrics (character string) and the score data are separate data sets has been described. However, the lyrics may be a part of the score data.
Details of the speech synthesis process are not limited to those described in the embodiment. As long as a note and a phonetic symbol (character) are given, any processing may be used as long as it synthesizes a sound corresponding to the note and the phonetic symbol.
The function of synthesizing and outputting the confirmation sound may be switched on / off according to a user instruction.

  The speech synthesis program described in the above embodiment includes a magnetic recording medium (magnetic tape, magnetic disk (HDD, FD (Flexible Disk)), etc.), an optical recording medium (optical disk (CD (Compact Disk)), DVD (Digital Versatile Disk). )), Etc.), a magneto-optical recording medium, and a computer-readable recording medium such as a semiconductor memory (flash ROM or the like). The program may be downloaded via a network such as the Internet.

DESCRIPTION OF SYMBOLS 1 ... Speech synthesizer, 11 ... Memory | storage means, 12 ... Display control means, 13 ... Display means, 14 ... Acquisition means, 15 ... Assignment means, 16 ... Speech synthesis means, 17 ... Confirmation sound output means, 18 ... Voice output means , 101 ... CPU, 102 ... ROM, 103 ... RAM, 104 ... storage unit, 105 ... input unit, 106 ... display unit, 107 ... DAC, 108 ... amplifier, 109 ... speaker, 110 ... housing, 111 ... touch panel, 201 ... Input box, 202 ... Window, 203 ... Guide figure, 204 ... Note figure, 205 ... Note figure, 206 ... Note figure, 207 ... Note figure, 208 ... Note figure, 209 ... Play button, 210 ... Stop button, 211 ... Notebook figure, 303 ... Individual database, 3030 ... Segment data group

Claims (5)

A character string storage means for storing a character string;
Attribute storage means for storing, for each of a plurality of notes, an attribute including a character assigned to the note in the character string, and an initial period, a pitch, and a tone length of the note generation period;
A display for displaying on the screen of the display means a graphic representing the beginning, pitch, and length of the pronunciation period of each of the plurality of notes according to a coordinate system having a first axis representing pitch and a second axis representing time. Control means;
Acquisition means for acquiring the start of the pronunciation period of the processing target note designated to be newly added in the screen;
Characters stored in the character string storage unit based on the context of the start period of the pronunciation period acquired by the acquisition unit and the start period of the pronunciation period of a plurality of notes stored in the attribute storage unit A speech synthesizer comprising: an assigning unit that rewrites an attribute stored in the attribute storage unit so that a part of a sequence is assigned to the processing target note. A character string storage means for storing a character string;
Attribute storage means for storing, for each of a plurality of notes, an attribute including a character assigned to the note in the character string, and an initial period, a pitch, and a tone length of the note generation period;
A display for displaying on the screen of the display means a graphic representing the beginning, pitch, and length of the pronunciation period of each of the plurality of notes according to a coordinate system having a first axis representing pitch and a second axis representing time. Control means;
Obtaining means for obtaining the start of the pronunciation period of the note to be processed, designated on the screen;
Characters stored in the character string storage unit based on the context of the start period of the pronunciation period acquired by the acquisition unit and the start period of the pronunciation period of a plurality of notes stored in the attribute storage unit Assigning means for rewriting the attribute stored in the attribute storage means so as to assign a part of the column to the processing target note;
An instruction to move the first figure on the screen while keeping the length of the first note constant for the first figure representing the first note among the plurality of notes to which characters are assigned by the assigning means. Input means for inputting, and
The display control means moves the first graphic according to an instruction input by the input means,
If the first graphic exceeds a position corresponding to the beginning of the pronunciation period of the second note represented by the second graphic positioned next to the first graphic, the assigning means assigns the first graphic to the first graphic. features and to Ruoto voice synthesizing apparatus that replacing the character and the corresponding letter assigned to the second graphic that is. The operation according to claim 1 or 2, wherein when an operation for determining the end of the pronunciation period is detected for the processing target note, the assigning unit allocates a part of the character string to the processing target note. Voice synthesizer. Computer
A character string storage means for storing a character string;
Attribute storage means for storing, for each of a plurality of notes, an attribute including a character assigned to the note in the character string, and an initial period, a pitch, and a tone length of the note generation period;
A display for displaying on the screen of the display means a graphic representing the beginning, pitch, and length of the pronunciation period of each of the plurality of notes according to a coordinate system having a first axis representing pitch and a second axis representing time. Control means;
Acquisition means for acquiring the start of the pronunciation period of the processing target note designated to be newly added in the screen;
Characters stored in the character string storage unit based on the context of the start period of the pronunciation period acquired by the acquisition unit and the start period of the pronunciation period of a plurality of notes stored in the attribute storage unit A program for functioning as an assigning means for rewriting an attribute stored in the attribute storage means so as to assign a part of a row to the processing target note. Computer
A character string storage means for storing a character string;
Attribute storage means for storing, for each of a plurality of notes, an attribute including a character assigned to the note in the character string, and an initial period, a pitch, and a tone length of the note generation period;
A display for displaying on the screen of the display means a graphic representing the beginning, pitch, and length of the pronunciation period of each of the plurality of notes according to a coordinate system having a first axis representing pitch and a second axis representing time. Control means;
Obtaining means for obtaining the start of the pronunciation period of the note to be processed, designated on the screen;
Characters stored in the character string storage unit based on the context of the start period of the pronunciation period acquired by the acquisition unit and the start period of the pronunciation period of a plurality of notes stored in the attribute storage unit Assigning means for rewriting the attribute stored in the attribute storage means so as to assign a part of the column to the processing target note;
An instruction to move the first figure on the screen while keeping the length of the first note constant for the first figure representing the first note among the plurality of notes to which characters are assigned by the assigning means. Function as an input means to input,
The display control means moves the first graphic according to an instruction input by the input means,
If the first graphic exceeds a position corresponding to the beginning of the pronunciation period of the second note represented by the second graphic positioned next to the first graphic, the assigning means assigns the first graphic to the first graphic. A program characterized by exchanging the assigned character with the character assigned to the second graphic.

Images