JP2016004189A - Synthetic information management device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To adjust positional relationship (separation/continuity) between phonemes while a user visually and intuitively confirms the positional relationship between front and rear phonemes.SOLUTION: A instruction reception unit 22 receive an instruction from a user. A display control unit 24 displays a time sequence of phoneme symbols of plural phonemes corresponding to a pronunciation content that synthesis information designates; and displays a connection indicator indicating positional relation ship on a time axis between a first phoneme and a second phoneme, between the phoneme symbol of a first phoneme and the phoneme symbol of a second phoneme right after the first phoneme among the plural phonemes, according to the instruction from the user received by the instruction reception unit 22. An information management unit 26 edits synthesis information so that the first phoneme and the second phoneme become positional relationship based on the connection indicator on the time axis.

Description

  The present invention relates to a technique for managing synthesis information applied to speech synthesis.

  Conventionally, a unit connection type speech synthesis technique for synthesizing speech of a desired pronunciation content by connecting a plurality of speech units to each other has been proposed. For example, Patent Literature 1 discloses a technique for synthesizing speech of pronunciation characters arbitrarily designated by a user.

JP 2012-022121 A

  However, under the technique of Patent Document 1, the user can only specify the pronunciation period and the pronunciation character for each note, and the continuity (degree of separation / proximity) between the phonemes corresponding to the pronunciation character. Cannot be adjusted. Therefore, there is a problem of synthesizing unnaturally or unclear speech that is extremely close to or separated from the phonemes before and after the user's intention, and the individuality according to the user's intention and preference. There was also a problem that it was not possible to synthesize speech with a characteristic expression. In view of the above circumstances, the present invention is a voice that allows the user to adjust the positional relationship (separation / proximity) between phonemes while visually and intuitively confirming the positional relationship between previous and subsequent phonemes. The purpose is to achieve synthesis.

  In order to solve the above problems, a synthesis information management apparatus according to the first aspect of the present invention is a synthesis information management apparatus that manages synthesis information that specifies the pronunciation content of synthesized speech, and that receives instructions from a user. An instruction receiving means for receiving, and means for displaying a time series of phoneme symbols of a plurality of phonemes corresponding to the pronunciation content specified by the synthesis information on a display device, wherein the instruction receiving means responds to an instruction received from a user The positional relationship on the time axis between the first phoneme and the second phoneme between the phoneme symbol of the first phoneme and the phoneme symbol of the second phoneme immediately after the first phoneme among the plurality of phonemes Display control means for displaying a connection indicator indicating, and information management means for editing the composite information so that the first phoneme and the second phoneme have a positional relationship corresponding to the connection indicator on a time axis It comprises. In the above configuration, in accordance with an instruction received from the user, a connection indicator indicating a positional relationship on the time axis is displayed between the phoneme symbol of the first phoneme and the phoneme symbol of the second phoneme, Since the synthesized information is edited so that the first phoneme and the second phoneme have a positional relationship according to the connection indicator on the time axis, the user visually and intuitively confirms the positional relationship between the preceding and following phonemes. It is possible to adjust the positional relationship between phonemes.

  In a preferred example of the composite information management device according to the first aspect, the display control means is a proximity indicating the proximity of the first phoneme and the second phoneme in accordance with an instruction received from a user by the instruction receiving means. An indicator is displayed as the connection indicator between the phoneme symbol of the first phoneme and the phoneme symbol of the second phoneme, and the information management means is configured to display the first phoneme and the phoneme according to the proximity indicator. The composite information is edited so that the second phoneme approaches on the time axis. In the above aspect, a connection indicator indicating the proximity of the first phoneme and the second phoneme is displayed between the phoneme symbol of the first phoneme and the phoneme symbol of the second phoneme in accordance with an instruction received from the user. In addition, since the synthesis information is edited so that the first phoneme and the second phoneme approach in accordance with the proximity indicator on the time axis, the user can visually and intuitively determine the proximity of the adjacent phonemes. There is an effect that the phonemes can be brought close to each other while checking. In addition, since the user can adjust the degree of continuity between phonemes according to the intention or preference, there is an advantage that it is possible to synthesize voices with unique or characteristic facial expressions.

  In a preferred example of the composite information management apparatus according to the first aspect, the display control means is a separation indicating a separation between the first phoneme and the second phoneme in accordance with an instruction received from a user by the instruction receiving means. An indicator is displayed as the connection indicator between the phoneme symbol of the first phoneme and the phoneme symbol of the second phoneme, and the information management means is configured to display the first phoneme and the phoneme according to the separation indicator. The synthesis information is edited so that the second phoneme is separated on the time axis. In the above aspect, in accordance with an instruction received from the user, a separation indicator indicating the separation between the first phoneme and the second phoneme is displayed between the phoneme symbol of the first phoneme and the phoneme symbol of the second phoneme. In addition, since the synthesis information is edited so that the first phoneme and the second phoneme are separated on the time axis according to the separation indicator, the user can visually and intuitively determine the separation of the adjacent phonemes. While checking, there is an effect that it becomes possible to separate phonemes. In addition, since the user can adjust the degree of continuity between phonemes according to the intention or preference, there is an advantage that it is possible to synthesize voices with unique or characteristic facial expressions.

  In a preferred example of the composite information management device according to the first aspect, the display control means and the phoneme symbol of the first phoneme out of the plurality of phonemes according to the instruction received by the instruction receiving means from the user. An index value indicating the degree of proximity or separation between the first phoneme and the second phoneme on the time axis is displayed between the phoneme symbol of the second phoneme immediately after one phoneme, and the information management unit includes: The synthesis information is updated so that the first phoneme and the second phoneme approach or separate on the time axis according to the index value. In the above aspect, the index indicating the degree of proximity or separation between the first phoneme and the second phoneme between the phoneme symbol of the first phoneme and the phoneme symbol of the second phoneme according to an instruction received from the user. Since the composite information is edited so that the first phoneme and the second phoneme approach or separate according to the index value on the time axis while displaying the value, the degree of proximity or separation of the adjacent phonemes There is an effect that the user can adjust the positional relationship between phonemes while visually and intuitively checking. In addition, since the user can adjust the degree of continuity between phonemes according to the intention or preference, there is an advantage that it is possible to synthesize voices with unique or characteristic facial expressions.

  The synthesis information management device according to the second aspect of the present invention is a synthesis information management device for managing synthesis information for designating the pronunciation content of synthesized speech, and comprising a plurality of phonemes corresponding to the pronunciation content designated by the synthesis information. Means for displaying a time series of phoneme symbols on a display device, wherein the first phoneme symbol of the plurality of phonemes and the second phoneme symbol immediately after the first phoneme are between the first phoneme symbol and the first phoneme symbol. Display control means for displaying an operation indicator for designating the degree of separation between the phoneme and the second phoneme on the time axis, instruction accepting means for accepting an operation of the operation indicator from a user, and the first phoneme And information management means for updating the composite information so that a positional relationship with the second phoneme on the time axis is separated according to an operation amount of the operation indicator. In the above aspect, in accordance with an instruction received from the user, an operation instruction that specifies the degree of separation between the first phoneme and the second phoneme between the phoneme symbol of the first phoneme and the phoneme symbol of the second phoneme. Since the synthesis information is edited so that the first phoneme and the second phoneme are separated according to the operation amount of the operation indicator on the time axis, the degree of separation of the phonemes before and after is displayed. There is an effect that the user can adjust the positional relationship between phonemes while visually and intuitively checking. For example, the degree of separation may be increased when the operation amount of the operation indicator is large. In addition, since the user can adjust the degree of continuity between phonemes according to the intention or preference, there is an advantage that it is possible to synthesize voices with unique or characteristic facial expressions.

  The synthesis information management device according to each of the above aspects is realized by hardware (electronic circuit) such as DSP (Digital Signal Processor) dedicated to editing synthesis information and generating audio signals, and also a CPU (Central Processing Unit) ) Etc., and can also be realized by the cooperation of a program. The program of the present invention can be provided in a form stored in a computer-readable recording medium and installed in the computer. The recording medium is, for example, a non-transitory recording medium, and an optical recording medium (optical disk) such as a CD-ROM is a good example, but a known arbitrary one such as a semiconductor recording medium or a magnetic recording medium This type of recording medium can be included. For example, the program of the present invention can be provided in the form of distribution via a communication network and installed in a computer. The present invention is also specified as an operation method (composite information management method) of the composite information management apparatus according to each aspect described above.

1 is a block diagram of a speech synthesizer 100 according to a first embodiment. It is a schematic diagram of synthetic information. It is a schematic diagram of a part of music. It is a schematic diagram of an edit screen. It is explanatory drawing of a speech segment. 3 is a flowchart of a schematic operation of the speech synthesizer 100 according to the first embodiment. It is a flowchart of the edit process which concerns on 1st Embodiment. It is explanatory drawing of the example of a display of the separation indicator CS. It is explanatory drawing of the example of a display of proximity indicator CC. 4 is a waveform diagram of a voice signal V generated by a voice synthesizer 28. FIG. It is a flowchart of the speech synthesis process which concerns on 1st Embodiment. It is explanatory drawing of adjustment (separation) of the space | interval between phonemes. It is explanatory drawing of adjustment (proximity) of the space | interval between phonemes. It is explanatory drawing of another aspect of the example of a display of the space | interval indicator CS. It is a flowchart of the edit process which concerns on 2nd Embodiment. It is explanatory drawing of the example of a display of the separation indicator CS and the index value I. It is a flowchart of the speech synthesis process which concerns on 2nd Embodiment. It is explanatory drawing of fusion | fusion of an audio | voice element. It is a flowchart of the edit process which concerns on 3rd Embodiment. It is explanatory drawing of the example of a display of the operation indicator. It is a flowchart of the speech synthesis process of 3rd Embodiment. It is explanatory drawing of the example of a display of the operation indicator. It is explanatory drawing of the example of a display of the operation indicator. It is explanatory drawing of the example of a display of the operation indicator. It is explanatory drawing of the example of a display of the operation indicator. It is explanatory drawing of the example of a display of the operation indicator. It is explanatory drawing of the example of a display of the operation indicator. It is explanatory drawing of the example of a display of the operation indicator. It is explanatory drawing of the edit of the control variable in contrast. It is an example of the score of mezzo staccato.

<First Embodiment>
FIG. 1 is a block diagram of a speech synthesizer 100 according to the first embodiment of the present invention. The speech synthesizer 100 according to the first embodiment generates a speech signal V of a singing voice of an arbitrary piece of music (hereinafter referred to as “synthetic music”) by a unit connection type speech synthesis that connects a plurality of speech units. Device. In the first embodiment, an audio signal V is generated in which the mutual positional relationship between adjacent phonemes on the time axis is adjusted according to an instruction received from a user.

  As illustrated in FIG. 1, the speech synthesizer 100 includes a computer system (for example, a mobile phone or a personal computer) that includes an arithmetic processing device 10, a storage device 12, a display device 14, an input device 16, and a sound emitting device 18. Information processing device). The display device 14 (for example, a liquid crystal display panel) displays an image instructed from the arithmetic processing device 10. The input device 16 is an operation device (for example, a pointing device such as a mouse or a keyboard) operated by the user for various instructions to the speech synthesizer 100, and includes a plurality of operators operated by the user, for example. Is done. Note that a touch panel configured integrally with the display device 14 may be employed as the input device 16. The sound emitting device 18 (for example, a speaker or headphones) reproduces sound according to the audio signal V.

  The storage device 12 stores a program executed by the arithmetic processing device 10 and various data used by the arithmetic processing device 10. A known recording medium such as a semiconductor recording medium or a magnetic recording medium or a combination of a plurality of types of recording media is arbitrarily employed as the storage device 12. The storage device 12 of the first embodiment stores the speech element group L and the synthesis information S as illustrated below.

  The speech segment group L is a set (speech synthesis library) of a plurality of speech segments P collected in advance from the recorded speech of a specific speaker. As illustrated in FIG. 5, the speech element group L in the first embodiment includes a phoneme chain (a diphone) in which a phoneme pA and a phoneme pB are connected. The phoneme (back phoneme) pB is located behind the phoneme (front phoneme) pA. Each speech element P is expressed by a time series of a time domain speech waveform sample sequence or a frequency domain spectrum calculated for each frame of the speech waveform. In the following description, silence is positioned as one phoneme for the sake of convenience and is represented by the symbol “Sil”.

  In the speech element group L of the first embodiment, for each combination of two phonemes, a plurality of types of speech elements P in which the positional relationship of each phoneme is different are stored. For example, for a speech unit corresponding to a specific combination of the phoneme pA and the phoneme pB, the speech unit P (hereinafter referred to as “standard segment P0” in which the interval between the phoneme pA and the phoneme pB is set to a standard reference value. ), And a speech unit P (hereinafter referred to as a “separated unit”), in which the interval between the phoneme pA and the phoneme pB exceeds the reference value (that is, the phoneme pA and the phoneme pB are separated from the standard unit P0). P1 ”and the interval between the phoneme pA and the phoneme pB is less than the reference value (that is, the phoneme pA and the phoneme pB are close to each other compared to the standard segment P0). P2 ") is included in the speech element group L. However, multiple types of speech elements (separated type element P1 and proximity type element P2) having different positional relationships between phonemes for all combinations of two phonemes are not prepared in advance. There is a combination of two phonemes in which a speech unit other than the unit P0 (a separated unit P1, a proximity unit P2) is not prepared in the speech unit group L.

  As illustrated in FIG. 2, the synthesis information S is time-series data for designating the singing voice of the synthesized music, and the pitch (for example, note number) X1, the pronunciation period X2, and the voice code for each note constituting the synthesized music. Designate X3 in time series. The sound generation period X2 is the time length (note value) of a note, and is defined by, for example, the start time T1 of sound generation and the time length (continuation length) T2. A configuration in which the sound generation period X2 is defined by a sound generation start time T1 and an end time (a configuration in which the time length between both times can be calculated as the time length T2) is also suitable. As can be understood from the above description, the synthesis information S can be rephrased as time-series data for designating the score of the synthesized music. The voice code X3 designates the pronunciation content of the voice to be synthesized (that is, the lyrics of the synthesized music). Specifically, the phonetic code X3 is information for designating a voice unit (for example, a syllable or a mora) that is pronounced for one note of the synthesized music, and the phonetic character QA corresponding to the voice unit and the voice unit. The phoneme symbol QB of each phoneme that constitutes. The phonetic character QA corresponds to a character (grapheme) constituting the lyrics of the synthesized music. In the first embodiment, in response to an instruction received from the user, the phoneme immediately after an arbitrary phoneme (first phoneme) in a time series of a plurality of phonemes and a phoneme immediately after the phoneme ( Interphoneme information QC defining the positional relationship with the second phoneme) is added. More specifically, the inter-phoneme information QC defines the separation or proximity of the first phoneme and the second phoneme on the time axis.

  The arithmetic processing unit 10 (CPU) in FIG. 1 executes a program stored in the storage unit 12 to thereby edit a plurality of functions (instruction receiving unit 22, display) for editing the synthesis information S and generating the audio signal V. The control unit 24, the information management unit 26, and the speech synthesis unit 28) are realized. A configuration in which each function of the arithmetic processing device 10 is distributed to a plurality of devices, or a configuration in which a dedicated electronic circuit (for example, DSP) realizes a part of the functions of the arithmetic processing device 10 may be employed. The instruction receiving unit 22, the display control unit 24, and the information management unit 26 are realized by, for example, music editing software (editor), and the voice synthesis unit 28 is realized by, for example, voice synthesis software (voice synthesis engine). The However, the relationship between each function of the arithmetic processing device 10 and the separation of software that realizes each function is not limited to the above examples.

  The instruction receiving unit 22 receives an instruction from a user according to an operation on the input device 16. The display control unit 24 displays various images on the display device 14. Specifically, the display control unit 24 of the first embodiment causes the display device 14 to display the editing screen 40 of FIG. 4 for the user to confirm the content of the composite music specified by the composite information S. The editing screen 40 is a piano roll coordinate plane in which a time axis (horizontal axis) and a pitch axis (vertical axis) intersecting each other are set.

  The display control unit 24 arranges the note image 42, the phonetic character QA, and the phoneme symbol QB in time series on the editing screen 40 for each note specified by the synthesis information S. FIG. 4 is a voice code of each voice unit “I”, “wan-”, “ted”, “to”, “see” of the lyrics (character string) “I wanted to see” of the music shown in FIG. An edit screen 40 assigned to five notes is illustrated as X3. In the example shown in the figure, the speech code X3 corresponding to a plurality of character strings is assigned to one note, but the speech code X3 corresponding to one character may be assigned to one note. The musical note iconic image 42 is an image representing each musical note of the synthesized music. Specifically, the position of the note image 42 in the direction of the pitch axis is set according to the pitch X1 specified by the synthesis information S. The position of the musical note iconic image 42 in the direction of the time axis is set according to the start time T1 of the sounding period X2 designated by the synthesis information S, and the display length (size) of the musical note iconic image 42 in the direction of the time axis is It is set according to the time length T2 of the sound generation period X2 designated by the information S. That is, the longer the time length T2, the longer the display length of the note image 42 on the time axis. The phonetic character QA and the phoneme symbol QB are arranged inside the musical note iconic image 42. As understood from the above description, the editing screen 40 is an image in which the phonetic characters QA corresponding to the lyrics (pronunciation content) of the synthesized music and the phoneme symbols QB of each phoneme are arranged in time series. Note that the positions of the phonetic character QA and the phoneme symbol QB are appropriately changed. For example, one or both of the phonetic character QA and the phoneme symbol QB can be arranged in the vicinity (outside) of the musical note iconic image 42. A configuration in which the display of the phonetic character QA is omitted or a configuration in which the display of the phoneme symbol QB is omitted may be employed.

  The user can instruct addition or movement of the note image 42 and addition or change of the pronunciation character QA by appropriately operating the input device 16 while confirming the editing screen 40 of FIG. Further, by appropriately operating the input device 16, the user can select a phoneme symbol QB of an arbitrary phoneme (first phoneme) from the time series of a plurality of phoneme symbols QB displayed on the editing screen 40, and the phoneme. It is possible to indicate the positional relationship between the first phoneme and the second phoneme on the time axis between the phoneme symbol QB of the phoneme (second phoneme) immediately after.

  The information management unit 26 in FIG. 1 edits the composite information S in response to an instruction from the user with respect to the editing screen 40. For example, the information management unit 26 changes the pitch X1 of the note corresponding to the musical note iconic image 42 in the synthesis information S in response to an instruction to move the musical note iconic image 42 in the direction of the pitch axis. Further, the information management unit 26 changes the start time T1 of the note sound generation period X2 corresponding to the note image 42 in the synthesis information S in accordance with the position of the note image 42 in the direction of the time axis. The time length T2 of the note production period X2 corresponding to the note image 42 in the synthesis information S is changed according to the display length on the time axis. That is, the instruction to change the display length of the musical note iconic image 42 corresponds to an instruction to change the time length T2 of the sound generation period X2. When the phonetic character QA of an arbitrary note is changed, the information management unit 26 changes the phonetic character QA corresponding to the note in the synthesis information S and changes the phoneme symbol QB of the note. Update as appropriate. Further, the information management unit 26 immediately follows the phoneme symbol QB of the first phoneme so that the positional relationship between the first phoneme and the second phoneme on the time axis becomes a positional relationship according to an instruction received from the user. In addition, interphoneme information QC indicating the positional relationship according to the user's instruction is added.

  The speech synthesizer 28 in FIG. 1 generates a speech signal V using the speech element group L and the synthesis information S stored in the storage device 12. Specifically, the speech synthesizer 28 sequentially selects a speech unit P corresponding to the speech code X3 for each note specified by the synthesis information S from the speech unit group L, and selects each speech unit P as a pitch. The voice signal V of the singing voice is generated by adjusting and connecting to X1 and the sound generation period X2. The voice signal V generated by the voice synthesizer 28 is supplied to the sound emitting device 18 so that the singing voice of the synthesized music is reproduced.

  FIG. 6 is a flowchart of a schematic operation of the speech synthesizer 100 according to the first embodiment. For example, when editing of the composite information S is instructed by the user, the processing in FIG. 6 is started. The arithmetic processing device 10 (the display control unit 24, the information management unit 26) causes the display device 14 to display an editing screen 40 corresponding to the composite information S (SA1), and in response to an instruction received from the user by the instruction receiving unit 22. While editing the composite information S, an editing process SA2 for updating the editing screen 40 according to the editing content is executed. When the editing process SA2 is executed, the arithmetic processing unit 10 determines whether or not speech synthesis is instructed by the user (SA3). When speech synthesis is instructed (SA3: YES), the synthesis information S is used. The voice synthesis process SA4 for generating the voice signal V of the designated singing voice is executed. When the voice synthesis is not instructed (SA3: NO), the voice synthesis process SA4 is not executed.

  The arithmetic processing unit 10 determines whether or not the user has instructed the end of the process (SA5). When the process end is not instructed (SA5: NO), the arithmetic processing unit 10 executes the editing process SA2. On the other hand, when the process end is instructed (SA5: YES), the arithmetic processing unit 10 ends the process of FIG.

  For the note image 42 of the note that includes the desired phonetic code X3 in the editing screen 40, the user immediately follows the first phoneme and the first phoneme symbol QB included in the phonetic code X3. It is possible to instruct the two phonemes to be separated or close to each other on the time axis. FIG. 7 is a flowchart of processing executed when an instruction to separate or approach the first phoneme and the second phoneme is received from the user in the editing processing SA2 of FIG. When the instruction receiving unit 22 receives an instruction to separate or approach between phonemes, the display control unit 24 displays the connection indicator C immediately after the phoneme symbol QB of the first phoneme on the editing screen 40 (SB1). The information management unit 26 defines the separation or proximity so that the first phoneme [n] and the second phoneme [t] are separated or approached on the time axis according to the separation indicator CS or the proximity indicator CC. The interphoneme information QC is added to the synthesis information S (SB2). Specific examples of processing by the display control unit 24 and the information management unit 26 will be described in detail below.

<Separation between adjacent phonemes>
FIG. 8 is an explanatory diagram of one aspect of the connection indicator (part of FIG. 4 is shown again). By appropriately operating the input device 16, the user can set each phoneme corresponding to each successive phoneme symbol QB in the time series of the plurality of phoneme symbols QB displayed on the editing screen 40 on the time axis. It is possible to give an instruction to move away with. For example, the user can select any one phoneme symbol QB and then instruct the separation of the phoneme (first phoneme) of the phoneme symbol QB from the immediately following phoneme (second phoneme). In FIG. 8, the phoneme [n] (first phoneme) included in the “wan-” speech code X3 and the [t] (second phoneme) included in the “ted” speech code X3 are time axes. It is assumed that the instruction receiving unit 22 has received an instruction for separating from the user. When the above instruction is accepted, the display control unit 24 represents the separation of the first phoneme and the second phoneme on the time axis immediately after the phoneme symbol QB of the first phoneme [n] on the editing screen 40. The separation indicator (dot “.”) CS is displayed as the connection indicator C. In addition, the information management unit 26 determines the distance between the phonemes that defines the separation on the time axis between the first phoneme [n] and the second phoneme [t] immediately after the first phoneme [n] in accordance with the separation indicator CS. Add information QC. As understood from the above description, in the first embodiment, the time series of a plurality of phoneme symbols QB and the connection indicator C (separation indicator CS) corresponding to the instruction from the user are displayed on the editing screen 40. Since the user is displayed, the user adjusts (separates) the degree of continuity between the first phoneme [n] and the second phoneme [t] while confirming a plurality of phonemes included in the lyrics of the synthesized music. It is possible.

  The display mode of the separation indicator CS is arbitrary. For example, in addition to the symbols illustrated in FIG. 8, an aspect in which an arbitrary character string, an arbitrary image, or the like is displayed as the separation indicator CS may be employed. Further, in the above description, the “wan-” speech code X3 including the first phoneme [n] and the “ted” speech code X3 including the second phoneme [t] are represented as separate note images 42. Although it corresponded, the structure by which a 1st phoneme and a 2nd phoneme are included by one note interval (note image 42) can also be employ | adopted. For example, as illustrated in FIG. 14, the connection indicator C may be displayed between the first phoneme [t] and the second phoneme [I].

<Proximity between neighboring phonemes>
FIG. 9 is an explanatory diagram of one aspect of the connection indicator (part of FIG. 4 is shown again). In FIG. 9, the phoneme [n] (first phoneme) included in the “wan-” speech code X3 and the [t] (second phoneme) included in the “ted” speech code X3 are time axes. It is assumed that the instruction accepting unit 22 accepts an instruction to make the above approach from the user. When the above instruction is accepted, the display control unit 24 represents the separation of the first phoneme and the second phoneme on the time axis immediately after the phoneme symbol QB of the first phoneme [n] on the editing screen 40. The proximity indicator (hyphen “-” and backslash “\”) CC is displayed as the connection indicator C. Further, the information management unit 26 determines the interphoneme that defines the proximity of the first phoneme [n] and the second phoneme [t] on the time axis immediately after the first phoneme [n] according to the proximity indicator CC. Add information QC. As understood from the above description, in the first embodiment, a time series of a plurality of phoneme symbols QB and a connection indicator C (proximity indicator CC) according to an instruction from the user are displayed on the editing screen 40. The user can adjust (approach) the degree of continuity between the first phoneme [n] and the second phoneme [t] while confirming the multiple phonemes included in the lyrics of the synthesized music. Is possible. As the display mode of the proximity indicator CC, an arbitrary character string, an arbitrary image, or the like can be adopted in addition to the symbols illustrated in FIG.

  FIG. 10 is a waveform diagram of the voice signal V generated by the voice synthesis unit 28 in the voice synthesis process SA4. Specifically, the waveform of the audio signal V generated under the synthesis information S having the content illustrated in FIG. 4 is illustrated in FIG. A portion surrounded by a rectangle in FIG. 10 indicates a section including the phoneme [n] and the phoneme [t] in the speech element group L. In the following description, particular attention is paid to the interval between the phoneme [n] and the phoneme [t].

  FIG. 11 is a flowchart of a specific example of the speech synthesis process SA4 illustrated in FIG. When the speech synthesis process SA4 is started, the speech synthesizer 28 adds inter-phoneme information QC for the front phoneme among the speech units to be selected corresponding to each speech code X3 designated by the synthesis information S for each note. It is determined whether or not (SC1). When the inter-phoneme information QC is not added (SC1: NO), the speech synthesizer 28 selects the standard segment P0 corresponding to the speech code X3 from the speech unit group L (SC2). On the other hand, when the inter-phoneme information QC is added (SC1: YES), the speech synthesizer 28 determines whether the inter-phoneme information QC defines separation / proximity (SC3).

  For example, as illustrated in FIG. 8, when the interphoneme information QC that defines the separation is set (SC3: separation), the speech synthesis unit 28 uses the first phoneme [n] and the second phoneme [t]. It is determined whether or not a separated [nt] speech segment P (separated segment P1) having a spacing with respect to the reference value exists in the speech segment group L (SC4). When the separated segment P1 exists in the speech unit group L (SC4: YES), the speech synthesis unit 28 selects the separated unit P1 from the speech unit group L (SC5). On the other hand, when the separated-type segment P1 does not exist in the speech segment group L (SC4: NO), the speech synthesizer 28 sets the interval between the first phoneme [n] and the second phoneme [t] as a reference value. [Nt] speech segment P (standard segment P0) is selected from the speech segment group L, and the interval between the first phoneme [n] and the second phoneme [t] in the standard segment P0 is extended. (SC6). Specifically, as illustrated in FIG. 12, the interval D from the end of the first phoneme [n] to the start of the second phoneme [t] is extended to D1 (D1> D). The interval D1 is, for example, a time length obtained by multiplying the initial interval D by a predetermined ratio (> 1).

  On the other hand, as illustrated in FIG. 9, when the interphoneme information QC defining proximity is set (SC3: proximity), the speech synthesizer 28 performs the first phoneme [n] and the second phoneme [t]. It is determined whether or not a proximity type [n− \ t] speech segment P (proximity segment P2) having an interval with the reference value is present in the speech segment group L (SC7). When the proximity unit P2 exists in the speech unit group L (SC7: YES), the speech synthesizer 28 selects the proximity unit P2 (SC8). On the other hand, when the proximity unit P2 does not exist in the speech unit group L (SC7: NO), the speech synthesizer 28 selects the standard unit P0 from the speech unit group L, and the first unit P0 in the standard unit P0. The interval between the phoneme [n] and the second phoneme [t] is shortened (SC9). Specifically, as illustrated in FIG. 13, the interval D from the end of the first phoneme [n] to the start of the second phoneme [t] is shortened to D2 (D2 <D). The interval D2 is, for example, a time length obtained by multiplying the initial interval D by a predetermined ratio (<1).

  The speech synthesizer 28 adjusts the speech segment P selected or expanded / contracted by the above-described processes (SC2, SC5, SC6, SC8, SC9) to the pitch X1 and the pronunciation period X2 specified by the synthesis information S. In addition, the audio signal V is generated by connecting them together (SC10). The speech synthesizer 28 determines whether or not all phonemes included in the synthesis information S have been processed (SC11). If the processing is not complete (SC11: NO), the process proceeds to step SC1. Then, similar processing is executed for other phonemes designated by the synthesis information S. On the other hand, when the process is completed (SC11: YES), the speech synthesizer 28 ends the speech synthesis process SA4.

  As described above, in the first embodiment, in response to an instruction from the user, the time series of a plurality of phoneme symbols QB and the first phoneme [n] and the second phoneme [t included in the speech code X3 The connection indicator C (separation indicator CS or proximity indicator CC) that defines separation or proximity to the display is displayed. Accordingly, the user can visually confirm the plurality of phonemes (phoneme symbols QB) arranged in time series on the editing screen 40 while visually confirming the first phoneme [n] and the second phoneme [t]. The degree of continuity can be adjusted (separated or approached). According to the first embodiment, it becomes possible for a user to synthesize a voice with a unique or characteristic expression according to his / her intention or preference, and the sensation that front and rear phonemes are extremely close or separated from each other. Therefore, there is an advantage that it is possible to avoid a situation where unnatural or unclear voice is synthesized.

  Further, in the first embodiment, the separation type element P1 ([n.t] speech element P1) in which the interval between the first phoneme and the second phoneme is equal to or greater than the reference value, or the first phoneme and the second phoneme. When the proximity unit P2 (the speech unit P2 of [n- \ t]) whose distance from the phoneme is equal to or less than the reference value does not exist in the speech unit group L, the existing standard unit P0 ([nt] The interval (section D) between the first phoneme [n] and the second phoneme [t] in the speech unit). Therefore, it is not necessary to store a plurality of types of speech elements (separated type element P1, proximity type element P2) in which the positional relationship of each phoneme is different for all combinations of two phonemes. There is an advantage that the storage capacity of the device 12 can be reduced.

Second Embodiment
A second embodiment of the present invention will be described below. In the first embodiment, the configuration in which the first phoneme and the second phoneme are separated or approached according to the connection indicator C (separation indicator CS, proximity indicator CC) has been exemplified. In the second embodiment, an indication of an index value indicating the degree of separation or proximity between the first phoneme and the second phoneme is received from the user, and the first phoneme and the second phoneme are separated or approached according to the index value. .

  FIG. 15 is a flowchart of a specific example of processing executed when an instruction to separate or approach the first phoneme and the second phoneme and an index value are received from the user in the editing processing SA2. In the editing process SA2 of the second embodiment, the process of SB2 in the editing process SA2 of the first embodiment is replaced with the processes of SD1 and SD2. When the instruction receiving unit 22 receives an instruction for an index value following an instruction for separation or proximity between phonemes, the display control unit 24 sets the connection indicator C immediately after the phoneme symbol QB of the first phoneme on the editing screen 40. An index value I indicating the degree of separation or proximity is displayed in the vicinity (SD1). The information management unit 26 determines whether the first phoneme [n] and the second phoneme [t] are separated on the time axis according to the connection indicator C (the separation indicator CS or the proximity indicator CC) and the index value I. The interphoneme information QC that defines the separation or proximity and the index value I is added to the synthesis information S so as to approach (SD2). A specific example of processing by the display control unit 24 and the information management unit 26 in the second embodiment will be described in detail below.

  FIG. 16 is an explanatory diagram of a display example of the connection indicator C and the index value I. By appropriately operating the input device 16, the user can separate the phonemes corresponding to each successive phoneme symbol QB in the time series of the plurality of phoneme symbols QB displayed on the editing screen 40 on the time axis or It is possible to indicate an index value I that defines the degree of proximity. For example, the user selects an arbitrary phoneme symbol QB, and then indicates an index value indicating the degree of separation or proximity between the phoneme of the phoneme symbol QB (first phoneme) and the immediately following phoneme (second phoneme). Can be directed. The index value is an integer that relatively defines the degree of separation (proximity) between the first phoneme and the second phoneme in the range from 0 to 100. The correspondence between the numerical value of the index value I and the degree of separation is arbitrarily determined. For example, when the index value I is the maximum value (100) (that is, when the degree of separation is the maximum), the first phoneme and the second phoneme are reduced to the same extent as when only the separation indicator CS is designated. Separate on the time axis. On the other hand, when the index value I is the minimum value (0) (that is, when the degree of separation is the minimum), the first phoneme and the second phoneme are reduced to the same extent as when only the proximity indicator CC is instructed. Make them close on the time axis. In FIG. 16, the phoneme [n] (first phoneme) included in the “wan-” speech code X3 and the [t] (second phoneme) included in the “ted” speech code X3 are time axes. It is assumed that the instruction receiving unit 22 receives an instruction to separate the index and the index value I (80) from the user. When the above instruction is accepted, the display control unit 24 represents the separation of the first phoneme and the second phoneme on the time axis immediately after the phoneme symbol QB of the first phoneme [n] on the editing screen 40. The separation indicator CS is displayed as the connection indicator C, and the index value I (80) is displayed immediately after the separation indicator CS. Further, the information management unit 26 sets the time axis of the first phoneme [n] and the second phoneme [t] immediately after the first phoneme [n] according to the separation indicator CS and the index value I (80). The interphoneme information QC that defines the separation above and the degree of separation (index value I) is added. As understood from the above description, in the second embodiment, a time series of a plurality of phoneme symbols QB, a connection indicator C (separation indicator CS) according to an instruction from the user, and an index value I are provided. Since it is displayed on the editing screen 40, the user can confirm the intervals between the phonemes included in the lyrics of the synthesized music and the phonemes that follow each other, and the first phoneme [n] and the second phoneme [ It is possible to adjust (separate) the degree of continuity with t].

  FIG. 17 is a flowchart of a specific example of the speech synthesis process SA4 in the second embodiment. In the speech synthesis process SA4 of the second embodiment, the processes from SC4 to SC9 in the speech synthesis process SA4 of the first embodiment are replaced with the processes of SE1 and SE2. When the inter-phoneme information QC that defines the separation is set (SC3: separation), the speech synthesis unit 28 extracts the standard unit P0 and the separation-type unit P1 corresponding to the speech code X3 from the speech unit group L. In addition to the selection, the standard segment P0 and the separated segment P1 are fused according to the index value I (SE1).

  FIG. 18 is an explanatory diagram of speech unit fusion in the second embodiment. [N.t] standard segment P0 and [n. t], the speech synthesis unit 28 selects [n. The separated piece P1 of t] and the standard piece P0 of [n−t] are fused at a ratio according to the index value I (80). More specifically, the separation-type element P1 and the standard element P0 are fused at a ratio of 8: 2 so that the separation-type element P1 has a ratio corresponding to the index value (80). A speech unit P3 of (80) t] is generated. As understood from the above description, the index value I indicates the degree of separation between the first phoneme [n] and the second phoneme [t], and the separation type in the fusion of the separation unit P1 and the standard unit P0. It is defined in the form of the ratio of the piece P1 (80/100). As illustrated in FIG. 18, the interval D3 from the end of the first phoneme [n] to the start of the second phoneme [t] in the speech unit P3 is equal to the interval D1 between phonemes in the separated unit P1 and the standard. The time length corresponding to the index value I is set within a range between the phoneme interval D in the segment P0.

  Returning to FIG. 17, when the interphoneme information QC defining proximity is set (SC3: proximity), the speech synthesizer 28 selects the standard segment P0 and the proximity segment P2 from the speech segment group L. Then, the standard segment P0 and the proximity segment P2 are fused according to the index value I (SE2). Although the illustration of the combination of the standard segment P0 and the proximity type segment P2 is omitted, for example, when the index value I (80) is instructed, the proximity type segment P2 is similar to the example of FIG. [N-\ (80) t] speech segment by fusing the proximity segment P2 and the standard segment P0 at a ratio of 8: 2 so that the ratio is in accordance with the index value I (80). P is generated. The speech synthesizer 28 adjusts the speech segment P selected or generated in each process (SC2, SE1, SE2) described above to the pitch X1 and the sound generation period X2 specified by the synthesis information S, and then mutually. The audio signal V is generated by the connection (SC10). Since the subsequent processing is the same as that of the first embodiment, description thereof is omitted.

  As described above, according to the second embodiment, in accordance with an instruction from the user, a time series of a plurality of phoneme symbols QB, a connection indicator C (a separation indicator CS or a proximity indicator CC), and a separation or proximity An index value I that defines the degree of is displayed. Accordingly, the user visually and intuitively confirms a plurality of phonemes (phoneme symbols QB) arranged in time series on the editing screen 40, while the first phoneme [n] and the second phoneme [t]. Can be finely adjusted (separated or approached) according to the index value I. Specifically, the effect is that the user can synthesize voices with unique or characteristic facial expressions according to their intentions and preferences, and the auditory sense that front and back phonemes are extremely close or separated. The effect that it is possible to avoid the situation where unnatural or unclear voice is synthesized is remarkable in the second embodiment.

<Third Embodiment>
A third embodiment of the present invention will be described below. In 2nd Embodiment, the structure which a user instruct | indicates the index value which prescribes | regulates the isolation | separation or proximity | contact degree of a 1st phoneme and a 2nd phoneme was illustrated. In the third embodiment, a time series of a plurality of phoneme symbols QB and an operation indicator (slider) are displayed on the editing screen 40, and the first phoneme and the second phoneme are separated according to the operation amount of the operation indicator. An index value indicating the degree is set.

  FIG. 19 is a flowchart of a specific example of processing executed when an instruction to display an operation indicator is received from the user in the editing processing SA2. The display control unit 24 of the third embodiment displays an operation indicator 44 on the edge line corresponding to the end point of the pronunciation period in the note image 42 including the phoneme symbol QB of the first phoneme included in the phonetic code X3. (SF1). The instruction receiving unit 22 waits until an operation is received from the user via the operation indicator 44 (SF2: NO), and when an operation for instructing the degree of separation is received from the user via the operation indicator 44 ( SF2: YES), the information management unit 26 sets the operation amount M so that the first phoneme [n] and the second phoneme [t] are separated on the time axis according to the operation amount M of the operation indicator 44. The prescribed interphoneme information QC is added to the synthesis information S (SF3). Specific examples of processing by the display control unit 24 and the information management unit 26 in the third embodiment will be described in detail below.

  FIG. 20 is an explanatory diagram of a display example of the operation indicator 44. The operation indicator 44 is arranged on the line of the edge corresponding to the end point of the pronunciation period in the musical note image 42 corresponding to the phonetic code X3 including the first phoneme [n] on the editing screen 40. The user can move the operation indicator 44 along the edge (that is, in the direction of the pitch axis) within a range from the upper side to the lower side of the musical note image 42. The correspondence between the operation amount M of the operation indicator 44 and the degree of separation is arbitrarily determined. In the third embodiment, the degree of separation increases as the operation amount M from the starting point 0 to the lower pitch axis starts from the point O on the upper side of the musical note iconic image 42 where the operation indicator 44 is initially arranged. Is configured to be large. The user slides the operation indicator 44 from the starting point O in the downward direction of the pitch axis to thereby determine the separation and degree of separation of the first phoneme [n] and the second phoneme [t] on the time axis. It is possible to instruct by one operation. In FIG. 20, the phoneme [n] (first phoneme) included in the “wan-” phoneme symbol QB and the phoneme [t] (second phoneme) included in the “ted” phoneme symbol QB are operated. A case is assumed in which an instruction to separate the amount by an amount corresponding to the amount M (40) is given. When the above instruction is received, the display control unit 24 changes the musical note iconic image 42 including the first phoneme [n] to a point on the edge corresponding to the end point of the pronunciation period and the upper side according to the operation amount M. The shape of the musical note iconic image 42 is changed so that a notch defined by a straight line connecting the two points is formed. As will be understood from FIG. 20, the notch is increased in conjunction with the increase in the operation amount M.

  When the instruction from the user is accepted, the information management unit 26 adds the index value I corresponding to the operation amount M to the interphoneme information QC. The index value I of the third embodiment is an integer that relatively defines the degree of separation between the first phoneme and the second phoneme in the range from 0 to 100. For example, when the index value I is the maximum value (100) (that is, when the degree of separation is the maximum), the first phoneme and the second phoneme are reduced to the same extent as when only the separation indicator CS is designated. Separate on the time axis. On the other hand, when the index value I is the minimum value (0) (that is, when the degree of separation is minimum), the positional relationship between the first phoneme and the second phoneme on the time axis is not adjusted. On the other hand, when the index value I exceeds the minimum value and falls below the maximum value (index value I: 0 <I <100), the first phoneme and the second phoneme are separated on the time axis according to the index value I. .

  FIG. 21 is a flowchart of the speech synthesis process SA4 of the third embodiment. In the speech synthesis process SA4 of the third embodiment, the processes of SC3, SE1, and SE2 in the speech synthesis process SA4 of the second embodiment are replaced with the processes of SG1 to SG3. The speech synthesizer 28 determines whether the index value I defined by the interphoneme information QC corresponds to either the maximum value (100) or the minimum value (0) (SG1). When the index value I is the maximum value (I = 100), the speech synthesis unit 28 selects the separation type segment P1. On the other hand, when the index value I is the minimum value (I = 0), the speech unit 28 selects the standard segment P0. When the index value I does not correspond to either the maximum value or the minimum value [index value I: 0 <I <100)] (SG1: NO), the speech synthesizer 28 performs the separation-type segment P1 and the standard segment P0. Are selected from the speech element group L, and the separated element P1 and the standard element P0 are fused at a ratio corresponding to the index value I. As illustrated in FIG. 20, when the index value I = 40 is set according to the operation amount M (40), the speech synthesizer 28 determines that the ratio of the separated segment P1 is the ratio according to the index value I. Then, the separated piece P1 and the standard piece P0 are fused at a ratio of P1: P0 = 4: 6 (SG2). The speech synthesizer 28 generates the speech signal V by adjusting the speech units selected or generated in SC2, SG2, and SG3 to the pitch X1 and the pronunciation period X2 specified by the synthesis information S and connecting them to each other. (SC10). Since the subsequent processing is the same as that of the first embodiment described above, description thereof is omitted.

  As described above, in the third embodiment, it is possible to instruct the index value I by an operation of moving the operation indicator 44. Therefore, compared with the second embodiment in which the index value I is designated by a numerical value, there is an advantage that the user can instruct the index value I intuitively and easily.

  In the third embodiment, the operation indicator 44 is arranged on the edge line corresponding to the end point of the pronunciation period in the musical note image 42 corresponding to the phonetic code X3 including the first phoneme [n] on the editing screen 40. Although the case where the musical note iconic image 42 is moved within the range from the upper side to the lower side is illustrated, the vicinity of the boundary of each successive musical note iconic image 42 is enlarged as exemplified by the area surrounded by the dotted line in FIG. It is also possible to display and assist the operation of moving the operation indicator 44 by the user.

<Modification>
Each of the aforementioned embodiments can be variously modified. Specific modifications are exemplified below. Two or more modes arbitrarily selected from the following examples can be appropriately combined.

(1) In the third embodiment, the configuration in which the degree of separation between the first phoneme [n] and the second phoneme [t] is instructed according to the operation amount M of the operation indicator 44 by the user is exemplified. The structure which added the scale which prescribes | regulates the operation amount M to the above structure can also be employ | adopted. Specifically, as illustrated in FIG. 23, a scale 46 may be displayed near the operation indicator 44. In the aspect mentioned above, there exists an advantage that a user is easy to visually recognize the degree of separation.

(2) In the third embodiment, the movable range of the operation indicator 44 is set within the range from the upper side to the lower side of the musical note iconic image 42 including the first phoneme, and the index value I (the degree of separation) corresponding to the operation amount M However, the movable range of the operation indicator 44 is outside the region of the musical note graphic image 42 including the first phoneme (the pitch axis direction is lower than the lower side of the musical note graphic image 42). Or the upper side of the musical note iconic image 42 in the pitch axis direction).

  For example, as illustrated in FIG. 24, when the operation indicator 44 is operated over the lower side in the pitch axis direction than the lower side of the musical note iconic image 42 including the first phoneme, the index value I exceeds 100. A numerical value (I> 100) may be set. When the index value I is 100 or more, the degree of separation between the first phoneme and the second phoneme is extremely high (that is, the interval between the preceding and following phonemes is the first phoneme and the first phoneme in the separated unit P1). The positional relationship between the first phoneme and the second phoneme is adjusted so that it is separated from the interval between the two phonemes. FIG. 24 assumes that the operation indicator 44 is operated below the lower side of the musical note iconic image 42 including the first phoneme [n], and the index value I corresponding to the operation amount M is set to 150. Yes. As described above, when the degree of separation between phonemes is sufficiently large, the speech synthesizer 28 generates a silent phoneme “Sil” between the first phoneme [n] and the second phoneme [t] in the standard segment P0. [N Sil t] speech segment P is inserted, and [n Sil t] speech segment P is synthesized with [nt] standard segment P0, according to index value I (150). Generate a speech segment.

  Similarly, as illustrated in FIG. 25, when the operation indicator 44 is operated over the upper side in the pitch axis direction from the upper side of the musical note iconic image 42 including the first phoneme, the index is indicated according to the operation amount M. The value I may be defined as a negative number (I <0). When the index value I is a negative number, the positional relationship between the first phoneme and the second phoneme is adjusted so that the degree of proximity between the first phoneme and the second phoneme becomes extremely high. In FIG. 25, among the speech units “got” and “up”, the phoneme [t] included in each phoneme symbol QB of “gh Q t” is set as the first phoneme, and the phoneme symbol QB of “V p” is obtained. When the included phoneme [V] is the second phoneme, the operation indicator 44 is operated upward in the pitch axis direction from the upper side of the note image 42 including the first phoneme [t], and the operation amount M It is assumed that the index value I corresponding to is set to -100. As described above, when the degree of proximity between phonemes is sufficiently large, the speech synthesizer 28 determines that the interval on the time axis between the first phoneme [t] and the second phoneme [V] is in the proximity unit P2. The interval on the time axis between the first phoneme [t] and the second phoneme [V] is brought closer to each other so as to be closer than the interval between the first phoneme and the second phoneme. Here, since the first phoneme [t] is a plosive sound, the speech synthesizer 28 [tV so that the interval on the time axis between the first phoneme [t] and the second phoneme [V] is close. ] And a speech element intermediate between the speech element P of [QV]. According to the above configuration, it is possible to realize colloquial pronunciation in which phoneme dropout (assimilation) occurs, such as “go up (gh Q V p)”. In addition, the structure which added the structure which allocates a specific speech unit to the index value I (-100) to the above structure, and the structure which allocates a predetermined | prescribed synthetic | combination ratio may be employ | adopted.

(3) As illustrated in FIG. 26, when the operation indicator 44 is allowed to be operated even outside the area of the musical note iconic image 42 including the first phoneme, the operation indicator 44 has the first phoneme. When operated upward in the pitch axis direction from the upper side of the note image 42, the note image 42 including the first phoneme [t] and the note image 42 of “V p” including the second phoneme [V] A configuration is also adopted in which the boundary line is inclined with respect to the pitch axis. In FIG. 26, the operation indicator 44 is operated upward in the pitch axis direction from the upper side of the musical note iconic image 42 including the first phoneme [t], and the index value I is set to a negative number (I <0). Is assumed. In the above configuration, the angle of the boundary line of each musical note iconic image 42 is changed according to whether the index value I is positive or negative (it is highlighted that the index value I is a negative number). There is an advantage that a person can grasp intuitively.

(4) In the third embodiment, the configuration in which the degree of separation between the first phoneme and the second phoneme is instructed according to the operation amount M in the pitch axis direction of the operation indicator 44 is exemplified. In addition to this, it is possible to adopt a configuration in which the characteristics (sound volume, timbre, intonation, etc.) of the synthesized sound are instructed according to the operation amount M of the operation indicator 44 in the time axis direction. For example, as illustrated in FIGS. 27 and 28, the positional relationship between the first phoneme and the second phoneme is adjusted according to the operation amount M1 of the operation indicator 44 in the pitch axis direction, and the time axis direction It is possible to control the characteristics of each phoneme according to the operation amount M2 of the operation indicator 44 in FIG. While visually and intuitively confirming a plurality of phonemes (speech code X3) arranged in time series on the time axis, the user can determine the positional relationship between the first phoneme and the second phoneme and the characteristics of the synthesized speech. Can be instructed by a single operation.

  As illustrated in FIG. 29, a configuration in which the user can arbitrarily specify the time change of the control variable indicating the characteristics of the synthesized speech specified on the edit screen 40 by an operation on the control variable specification screen 70 (hereinafter “comparative”). Has been proposed in the past. In contrast, even if an arbitrary note image 42 is moved and the sound generation period X2 (sound generation time and duration) of each note is changed, the time change itself of the control variable does not change. In order to provide the same characteristic, the user had to correct the time variation of the control variable so as to be consistent with the change in the sound generation period X2. On the other hand, in the above-described configuration in which the characteristics (sound volume, timbre, inflection, etc.) of the synthesized sound are instructed according to the operation amount M2 in the time axis direction of the operation indicator 44, the operation indicator 44 is provided for each musical note image 42. By operating, the characteristics are controlled for each note, and even when the note image 42 is moved in the time axis direction or the pitch axis direction, the characteristics of the notes can be maintained. That is, the user does not need to readjust the characteristics of each note. Therefore, there is an advantage that the user's operation is simplified. Even when the musical note iconic image 42 including either the first phoneme or the second phoneme is moved in the time axis direction or the pitch axis direction, the characteristics of each note can be maintained.

(5) As illustrated in FIG. 30, a music expression method (meso staccato) is known in which successive notes are slightly separated while the entire phrase is smoothly pronounced. In order to reproduce the meso-staccato singing sound under the conventional configuration in which the sound generation period X2 of each note is specified only by adjusting the length of the note image 42 in the time axis direction, the note images are arranged at appropriate intervals. Thus, it is necessary for the user to individually adjust the length of each musical note image 42 on the time axis. According to each of the above-described embodiments, the positional relationship (separation / proximity) of each phoneme can be specified independently of the length of the musical note iconic image 42 on the time axis. By specifying according to the score and adjusting the interval of each phoneme by attaching the connection indicator C or operating the operation indicator 44, it is possible to reproduce a delicate expression such as the meso staccato illustrated in FIG. Is possible.

(6) In each of the above-described forms, the phonetic symbol QA and the phoneme symbol QB are arranged inside the musical note image 42 displayed on the editing screen 40, and a connection instruction is provided immediately after the phoneme symbol QB of the first phoneme [n]. Although the configuration in which the child C (the separation indicator CS or the proximity indicator CC) is displayed has been illustrated, a configuration in which the display of the phoneme symbol QB is omitted may be employed. In the configuration in which the display of the phoneme symbol QB is omitted, for example, immediately after the phonetic character QA corresponding to the first phoneme, the connection indicator C or the index value I corresponding to the user's instruction is displayed.

(7) In each of the above-described embodiments, the index value I is exemplified by a configuration in which the degree of separation (proximity) between the first phoneme and the second phoneme is an integer that relatively defines the range from 0 to 100. A configuration in which the range of the index value I is narrower than the above-described configuration (for example, 0.0 to 1.0) or a configuration in which the range of the index value I is wider than the above-described configuration (for example, 0 to 300) may be employed. In addition to these configurations, a configuration in which the range of the index value I is defined by a relative value (−2.0 to 2.0) with 0 as a reference value may be employed.

(8) In each of the above-described embodiments, the unit connection type speech synthesis process SA4 using the speech unit P is exemplified. However, a known technique is used for speech synthesis using the synthesis information S generated by the editing process SA2. Is arbitrarily adopted. For example, it is also possible to synthesize the singing voice of the synthesized music specified by the synthesis information S using a probabilistic model such as a Hidden Markov Model (HMM). For example, the speech synthesizer 28 calculates a time transition (pitch curve) of the pitch according to the pitch X1 and the sound generation period X2 of the synthesis information S, and a basic signal (for example, vocal cords) whose pitch changes with the time transition. A sine wave signal representing the uttered sound) and using a filter process (for example, a filter process approximating intraoral resonance) corresponding to the voice code X3 designated by the synthesis information S after the execution of the editing process SA2 as a basic signal On the other hand, the audio signal V is generated by executing it.

(9) In each of the above-described embodiments, the speech unit P (diphone) in which two phonemes are connected is illustrated, but a speech unit P in which three or more phonemes are connected may be used. In a speech unit P in which three or more phonemes are connected, the first phoneme of one speech unit P corresponds to the phoneme pA of each of the above-described forms, and the last phoneme of one speech unit P is described above. Correspond to the phoneme pB of each form.

(10) In each of the embodiments described above, the storage device 12 that stores the speech segment group L and the synthesis information S is mounted on the speech synthesizer 100. However, an external device (for example, a server device) independent of the speech synthesizer 100. Is also used to store the speech element group L and the synthesis information S. The speech synthesizer 100 acquires the speech element group L or the synthesis information S via, for example, a communication network, and executes the editing process SA2 and the speech synthesis process SA4. As understood from the above description, the elements that store the speech element group L and the synthesis information S are not essential elements of the speech synthesizer 100.

(11) In each of the above-described embodiments, the generation of the voice signal V of the singing voice of the synthesized music has been exemplified. However, the present invention is also applied to the generation of the voice signal V of the voice other than the singing voice (for example, conversation sound). Is possible. Therefore, the pitch X1 suitable for singing voice synthesis can be omitted from the synthesis information S. As understood from the above description, the synthesis information S exemplified in each of the above aspects is comprehensively expressed as information specifying the pronunciation content of the speech to be synthesized. In addition, since the necessity of controlling the presence or absence of a sound change for every phoneme becomes especially obvious in the scene of synthesizing a singing voice, the present invention is particularly suitable for synthesizing a singing voice.

(12) In each of the above-described embodiments, the synthesis of English speech has been exemplified, but the speech language to be synthesized is arbitrary. For example, the present invention can be applied to the case of generating speech in an arbitrary language such as Japanese, Spanish, Chinese, or Korean.

DESCRIPTION OF SYMBOLS 100 ... Voice synthesizer, 10 ... Arithmetic processing unit, 12 ... Memory | storage device, 14 ... Display device, 16 ... Input device, 18 ... Sound emission device, 22 ... Instruction reception part, 24 ... Display Control unit, 26... Information management unit, 28... Speech synthesis unit, 40... Editing screen, 42.

Claims (5)

A synthesis information management device for managing synthesis information for designating pronunciation of synthesized speech,
An instruction receiving means for receiving an instruction from the user;
A means for displaying a time series of phoneme symbols of a plurality of phonemes corresponding to the pronunciation content specified by the synthesis information on a display device, the plurality of phonemes in accordance with an instruction received from a user by the instruction receiving means; A connection indicator indicating a positional relationship on the time axis between the first phoneme and the second phoneme between the phoneme symbol of the first phoneme and the phoneme symbol of the second phoneme immediately after the first phoneme. Display control means for displaying;
A composite information management device comprising: information management means for editing the composite information so that the first phoneme and the second phoneme have a positional relationship corresponding to the connection indicator on a time axis. The display control means includes a proximity indicator indicating proximity of the first phoneme and the second phoneme, a phoneme symbol of the first phoneme and the first phoneme according to an instruction received from a user by the instruction receiving means. Displayed as the connection indicator between two phoneme symbols,
The composite information management apparatus according to claim 1, wherein the information management unit edits the composite information so that the first phoneme and the second phoneme approach on a time axis according to the proximity indicator. The display control means includes a separation indicator indicating separation of the first phoneme and the second phoneme, a phoneme symbol of the first phoneme, and the first phoneme according to an instruction received from a user by the instruction reception means. Displayed as the connection indicator between two phoneme symbols,
The composite information management apparatus according to claim 1, wherein the information management unit edits the composite information so that the first phoneme and the second phoneme are separated on a time axis according to the separation indicator. The display control means, between the phoneme symbol of the first phoneme among the plurality of phonemes and the phoneme symbol of the second phoneme immediately after the first phoneme, according to the instruction received from the user by the instruction receiving means To display an index value indicating the degree of proximity or separation between the first phoneme and the second phoneme on the time axis,
The composite information management according to claim 2 or 3, wherein the information management means updates the composite information so that the first phoneme and the second phoneme approach or separate on the time axis according to the index value. apparatus. A synthesis information management device for managing synthesis information for designating pronunciation of synthesized speech,
A means for displaying a time series of phoneme symbols of a plurality of phonemes corresponding to the pronunciation content specified by the synthesis information on a display device, wherein the phoneme symbol of the first phoneme and the first phoneme immediately after the plurality of phonemes Display control means for displaying an operation indicator for designating the degree of separation of the first phoneme and the second phoneme on the time axis between the phoneme symbols of the second phoneme;
An instruction receiving means for receiving an operation of the operation indicator from a user;
A composite information management apparatus comprising: information management means for updating the composite information so that a positional relationship between the first phoneme and the second phoneme on a time axis is separated according to an operation amount of the operation indicator. .

Images