JP2005345699A - Device, method, and program for speech editing - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a speech editing device capable of more easily editing a synthesized speech that an operator desires, without increasing the operation load on the operator. <P>SOLUTION: A speech editing device is equipped with correction means 110 and 112 of correcting a synthesized sound generated for a text; a correction information storage means 131 of storing correction information, indicating a corrected sound obtained by the correction means 110 and 112; a history structure generating means 136 of generating history structure information, indicating the relation between the correction information stored in the correction information storage means 131 and the synthesized sound before the correction; a display means 140 of displaying correspondence relation between the correction information and synthesized sound, made to correspond to each other in the history structure information generated by the history structure generating means 136; a choice-accepting means 120 of accepting a choice of correction information displayed by the display means 140, and a synthesized sound generating means 114 of generating a synthesized sound, based on the correction information selected by the choice-accepting means 120. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a voice editing device, a voice editing method, and a voice editing program for editing synthesized sound.

  In general, in the text-to-speech synthesis process, first, input text is subjected to language processing to obtain phonetic information indicating the pronunciation of the text. The phonetic information includes parts of speech, dependency, reading, accent type, break position, break type, and the like that are output as a result of language processing. Then, prosody information is generated by performing prosody control based on phonetic information. The prosody information includes a pitch pattern, a phoneme duration, and the like. A synthesized sound is generated based on this prosodic information, and finally the synthesized sound is output via a DA converter.

  In such a text-to-speech synthesis process, there may occur problems such as a mistake in reading a word, an obvious error in language processing, and an unnatural prosody.

  The speech synthesis editing device is a device that corrects errors such as the result of text-to-speech synthesis as described above and correction of prosody unnaturalness. Furthermore, these processes can be performed interactively by the operator. Thus, it is an effective tool for solving the above-described problems in text-to-speech synthesis.

  As a speech synthesis editing device, one that outputs phonetic information as a phonetic symbol string and interactively corrects an error that has occurred in language processing is known (for example, see Patent Document 1).

  A speech synthesis editing apparatus that can interactively change prosodic information is also known. In this apparatus, more detailed correction is possible than correction of only phonetic information. Specifically, it is possible to change the intonation subtly and adjust the reading speed. As a result, it is possible to create synthetic sounds with higher naturalness or various prosody.

  As described above, the operator can adjust the phonetic information or the prosodic information interactively using a conventional voice synthesis editing apparatus to adjust to a desired synthesized sound. In this process, it is indispensable to perform modification and audition work in which the phonetic information or prosody information is corrected and the operator himself / herself confirms the corrected result.

JP 2002-351486 A

  However, there are cases where it is desired to use a previously modified synthesized sound in the process of repeating the above-described correction / trial listening. For example, this may be the case where the previously obtained correction sound is close to the desired synthesized sound, or when it is desired to redo the correction again at a certain point.

  In such a case, the conventional speech synthesis / editing apparatus needs to recall the correction operation from the synthesized sound and reproduce the desired correction state or perform correction again from the beginning. As described above, since the editing environment by the operator is limited such that the synthesized sound cannot be corrected from a desired state, it is desired to provide an editing environment that can more easily correct the synthesized sound desired by the operator. It was.

  The present invention has been made in view of the above, and it is an object of the present invention to provide a voice editing apparatus that can more easily edit a synthesized sound desired by an operator without increasing an operation burden on the operator. Objective.

  In order to solve the above-described problems and achieve the object, the present invention stores correction means for correcting a synthesized sound generated for text and correction information indicating the correction sound obtained by the correction means. Correction information storage means, history structure generation means for generating history structure information indicating the relationship between the correction information stored in the correction information storage means and the synthesized sound before correction, and the history structure generation means generated by the history structure generation means Display means for displaying the correspondence between the correction information and the synthesized sound associated in the history structure information, a selection receiving means for receiving selection of the correction information displayed on the display means, and the selection receiving means Synthetic sound generating means for generating a synthetic sound based on the selected correction information is provided.

  The voice editing apparatus according to the present invention stores correction information indicating a corrected sound after correction, and can display the correspondence between a plurality of correction information in an identifiable manner, so that the correction is performed again from the previously performed correction stage. Even if it is desired to perform the correction, it is possible to easily specify the desired correction information based on the displayed correspondence relationship, and to perform correction again from the correction state corresponding to the correction information. As a result, the synthesized sound desired by the operator can be edited without increasing the operation burden on the operator.

  Hereinafter, embodiments of a voice editing device, a voice editing method, and a voice editing program according to the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiments.

(Embodiment)
FIG. 1 is a block diagram showing a speech synthesis / editing apparatus 10 as an embodiment of a speech editing apparatus according to the present invention. The speech synthesis editing apparatus 10 includes a text acquisition unit 100, a language processing unit 102, a phonetic information storage unit 104, a prosody control unit 106, a prosody information storage unit 108, a phonetic information correction unit 110, The prosody information correction unit 112, the synthetic sound generation unit 114, the synthetic sound output unit 116, the instruction reception unit 120, the correction management unit 130, and the correction display unit 140 are provided.

  In the present embodiment, when generating synthesized speech from text, phonetic information obtained by language processing of text, and prosodic information generated by prosody control from this phonetic information, these phonetic information and Each parameter to be corrected in the prosodic information is corrected based on an instruction input from the user while viewing the display screen, and this correction history is retained. When there is a correction instruction from the user again, a synthesized sound is generated by correcting the phonetic information and the prosodic information using this correction history.

  The text input unit 101 is an external interface for inputting text to be converted into synthesized sound. Text is input by keyboard input or by reading a text file. The text acquisition unit 100 acquires text to be converted into synthesized sound via a text input unit 101 that is an external interface.

  The language processing unit 102 performs language processing on the text acquired by the text acquisition unit 100 to generate phonetic information. Here, the language processing is processing for dividing text into accent phrase units and outputting phonetic information corresponding to each accent phrase. The phonetic information is information generated in the language processing process and includes a phonetic symbol string generally called an intermediate language. Specifically, it is morphological information such as parts of speech obtained from the result of morphological analysis, syntax information such as dependency, information representing reading, accent type, delimiter position, delimiter type, and the like.

  The phonetic information storage unit 104 stores the phonetic information generated by the language processing unit 102 in association with each accent phrase. FIG. 2 schematically shows the data structure of the phonetic information stored in the phonetic information storage unit 104. FIG. 2 shows, as an example, a data structure of the phonetic information 1040 for the text “Today is good weather”.

  As shown in FIG. 2, the phonetic information storage unit 104 stores accent accent IDs in order from the top accent phrase for each accent phrase included in the text. Also, phonetic information for each accent phrase is associated with each accent phrase ID. In this way, each accent phrase and phonetic information are stored in a one-to-one correspondence.

  The description returns to FIG. 1 again. The prosody control unit 106 performs prosody control based on the phonetic information stored in the phonetic information storage unit 104 to generate prosody information. Here, the prosodic information is information indicating a phoneme duration and a pitch pattern. The prosody control unit 106 generates prosody information for each accent phrase divided by the language processing unit 102. Specifically, for example, for all three accent phrases “Kyowa”, “Good”, and “Tenki is” in the text “Today is a good weather.” Generate duration and pitch pattern.

  The prosody information storage unit 108 stores the prosody information generated by the prosody control unit 106. Prosodic information is represented by a one-dimensional array Pitch [i]. Here, i is a frame number corresponding to time, and Pitch [x] indicates the pitch of the frame number x. In the prosodic information correction unit 112 described later, a pitch pattern is displayed with time (frame) on the horizontal axis and pitch (octave) on the vertical axis. FIG. 3 schematically shows the data structure of the prosodic information stored in the prosodic information storage unit 108. FIG. 3 shows a data structure of the prosodic information 1080 for the text “Today is a good weather” as an example.

  As shown in FIG. 3, prosodic information expressed by Pitch [i] is stored for each accent phrase included in the text. That is, the prosody information storage unit 108 stores the value of Pitch [i] as the pitch. In addition, a time length corresponding to the number of Pitch [i] is stored.

  As another example, a pointer may be set as to what frame each accent phrase starts from. In this case, the prosodic information of a predetermined accent phrase is stored in a form that can be extracted from the entire prosodic information.

  The description returns to FIG. 1 again. The instruction input unit 121 inputs an instruction from the user with a mouse, a keyboard, or the like. The instruction receiving unit 120 receives an instruction from an operator via an instruction input unit 121 that is a user interface. Then, the instruction receiving unit 120 sends instruction information indicating an instruction from the operator to the phonetic information correcting unit 110, the prosody information correcting unit 112, and the synthesized sound generating unit 114. A specific correction instruction is given while viewing the correction screen displayed on the correction display unit 140 described later with reference to FIGS.

  The phonetic information correction unit 110 interactively observes the phonetic information stored in the phonetic information storage unit 104 based on the instruction information received from the instruction receiving unit 120 while viewing the correction screen displayed on the correction display unit 140. Correct it. Then, the corrected phonetic information is stored in the correction state information file storage unit 131 described later. The prosody information correction unit 112 interactively corrects the prosody information stored in the prosody information storage unit 108 based on the instruction information received from the instruction reception unit 120 while viewing the correction screen displayed on the correction display unit 140. Then, the modified prosodic information is stored in the modified state information file storage unit 131 described later.

  The phonetic information correction unit 110 and the prosody information correction unit 112 perform processing in units of accent phrases divided by the language processing unit 102. Note that the phonetic information correction unit 110 and the prosody information correction unit 112 according to the present embodiment constitute correction means of the present invention.

  The synthesized sound generation unit 114 generates a synthesized sound based on the phonetic information stored in the phonetic information storage unit 104 and the prosody information stored in the prosody information storage unit 108. The synthesized sound output unit 116 performs DA conversion on the synthesized sound generated by the synthesized sound generating unit 114 and outputs the resultant to the outside.

  The correction management unit 130 manages the modified phonetic information corrected by the processing of the phonetic information correcting unit 110 and the prosody information correcting unit 112 and the synthesized sound synthesized based on the corrected prosody information, that is, the corrected sound. To do. The correction display unit 140 displays a correction sound and the like managed by the correction management unit 130.

  FIG. 4 is a block diagram illustrating a detailed functional configuration of the correction management unit 130. The modification management unit 130 includes a modification state information file storage unit 131, a modification history file generation unit 132, a modification history file storage unit 134, a history structure data generation unit 136, and a history structure data storage unit 138. Yes.

  The modification state information file storage unit 131 stores modification state information. Here, the correction state information is a pair of the phonetic information corrected by the phonetic information correcting unit 110 and the prosody information corrected by the prosody information correcting unit 112. That is, a correction sound can be generated based on the correction state information. The correction state information file storage unit 131 according to the embodiment constitutes a correction information storage unit of the present invention. Further, the correction state information according to the embodiment corresponds to the correction information of the present invention.

  The correction state information includes a phonetic symbol string that is phonetic information and one-dimensional array data that is prosodic information. The correction state information is generated as one file every time correction is performed. Further, it is generated in units of a character string up to one sentence in the text, that is, the punctuation mark “.”. Numbers from # 1 are assigned to the modification status information file in the order of modification such as “EditData0_ # 1.dat”, “EditData0_ # 2.dat”, and “EditData0_ # 3.dat”. More specifically, a number indicating the order is assigned when a correction is made and a correction state information file is generated. Here, “0” included in “EditData0_ # 1.dat” indicates the first sentence in the text.

  The modification state information file storage unit 131 also sends the phonetic information and prosody information corresponding to the modification state information to the phonetic information storage unit 104 and the prosody information storage unit 108 based on the instruction from the instruction receiving unit 120, respectively.

  FIG. 5 schematically shows a data structure of the prosodic information after correction stored in the correction state information file storage unit 131. As the prosodic information after correction, a difference value from the initial value of the prosodic information is stored. In the example shown in FIG. 5, a difference value “+5” from the initial pitch value and a time length difference value “0” are stored for the accent phrase “Kyowa”. By storing only the difference value in this way, the data amount can be reduced.

  As another example, the modified prosodic information itself may be retained. The correction state information according to the present embodiment includes all the phonetic information and prosody information corrected for one sentence. As another example, the corrected accent phrase, the correction item, and the correction value are included. Only the correction status information may be used.

  The correction history file generation unit 132 generates a correction history file indicating the correction history by the phonetic information correction unit 110 and the prosody information correction unit 112 and stores the correction history file in the correction history file storage unit 134. The correction history file is generated for each sentence of text.

  FIG. 6 schematically shows the data structure of the correction history file stored in the correction history file storage unit 134. The correction history file has tables 1341, 1342,... For each sentence of the text. The table 1341 is a correction history file for each sentence identified by “EditData (i)”. The table 1341 has a history registration number field, a history structure data field, an initial data field, and a modification status information field.

  In the history registration number field, the history registration number is stored. Here, the number of history registrations is the number of times that the synthesized sound generated for one corresponding sentence is corrected and the correction state information is generated. The number of history registrations is changed every time it is modified.

  The history structure data field stores the data name of the history structure data. The history structure data is data indicating a correspondence relationship between a synthesized sound generated for a corresponding sentence and a synthesized sound obtained by correcting the synthesized sound.

  The initial data field stores a synthesized sound generated for a corresponding sentence, that is, initial data. The correction status information field stores correction status information for the initial data. Correction state information is added to the correction state information field every time correction is performed.

The initial data field and the corrected data field store various parameter values relating to the corrected date / time, correction order, and the entire synthesized sound at the time of correction. The parameter values related to the entire synthesized sound include the speaker, the overall pitch, the volume, and the speaking speed.
The initial data field and the correction data field store a corrected accent phrase and a correction item corresponding to the accent phrase as a correction history of phonetic information and prosodic information. Here, the correction items are, for example, an accent type, a break, a pitch pattern shape, a duration time, and the like.

  As described above, the modification history file storage unit 134 stores the synthesized state as one initial data in association with the modified state information obtained by modifying the synthesized sound. Further, correction state information obtained by further correcting the correction sound obtained by correcting the synthesized sound is also stored in association with each other.

  The history structure data generation unit 136 generates history structure data managed by the correction history file storage unit 134 and stores it in the history structure data storage unit 138. Specifically, when the synthesized sound is modified, the history structure data generation unit 136 associates the modification state information with the initial value of the synthesized sound.

  FIG. 7 schematically shows a data structure of history structure data stored in the history structure data storage unit 138. The history structure data storage unit 138 stores history structure data in which a correspondence relationship between one initial data and the modification state information for the initial data is expressed in a tree structure. The parent node in the tree structure is initial data. Each node is modification state information obtained by modifying the initial data of the parent node.

  In the history structure data shown in FIG. 6, the modification status information “EditData0_ # 1” and “EditData0_ # 3” are both information obtained by modifying the initial data “EditData0_ # 0”. Further, the modification state information “EditData0_ # 2” and “EditData0_ # 4” are both information obtained by modifying the modification state information “EditData0_ # 1”. The modification state information “EditData0_ # 5” is information obtained by modifying the modification state information “EditData0_ # 4”. As described above, the history structure data indicates the correspondence between such pieces of correction state information.

  Also, “#i” given to each data name is a number indicating the order of correction. That is, the history structure data shown in FIG. 7 indicates that the modification state information “EditData0_ # 1” is first generated using the initial data “0EditData0_ # 0” as the modification source. Subsequently, it is indicated that the correction state information “EditData0_ # 2” is generated using the correction state information “EditData0_ # 1” as a correction source. Further, it is shown that the modification status information “EditData0_ # 3” is generated using the initial data “EditData0_ # 0” as the modification source.

  As described above, the history structure data storage unit 138 stores history structure data including information for identifying the synthesized sound of the correction source of each correction state information and the correction order.

  In the present embodiment, the correction state information and the correction history are stored as files. However, as another example, such information may be expanded in a memory, and each address may be stored and specified. .

  FIG. 8 shows a phonetic information correction screen 200 displayed by the correction display unit 140 when correcting phonetic information. FIG. 9 shows a prosodic information correction screen 210 displayed by the correction display unit 140 when correcting prosodic information.

  The operator can correct the phonetic information and the prosody information while looking at the phonetic information correction screen 200 and the prosody information correction screen 210. More specifically, when the operator inputs an instruction to the contents displayed on the phonetic information correction screen 200 and the prosody information correction screen 210 using a user interface such as a mouse, the instruction receiving unit 120 An instruction from the user is accepted, and the instruction information is sent to the phonetic information correction unit 110 and the prosody information correction unit 112. And the phonetic information correction part 110 and the prosody information correction part 112 correct based on instruction information.

  The upper part of the phonetic information correction screen 200 is a common area 220 provided in both the phonetic information correction screen 200 and the prosody information correction screen 210. The common area 220 includes phonetic / prosody modification mode switching buttons 221 and 222, a text input / display portion 223, a language processing result display portion 224, a reading button 225, and a history registration button 226.

  When the phonetic / prosodic correction mode switching buttons 221 and 222 are selected by the operator, the phonetic correction mode and the prosody correction mode are switched. Corresponding to this mode switching, the correction display unit 140 switches and displays the phonetic information correction screen 200 and the prosody information correction screen 210. That is, when the phonetic correction button 221 is selected, the lower part of the screen is switched to the phonetic correction area 230 shown in FIG. 8, and when the prosody correction button 222 is selected, the lower part of the screen is shown in FIG. Switch to the prosody modification area 240 shown.

  As another example, the phonetic / prosodic correction mode switching buttons 221 and 222 are not provided, and the phonetic correction area 230 shown in FIG. 8 and the prosody correction area 240 shown in FIG. 9 are displayed on the same screen. May be.

  The text input / display portion 223 is a field for receiving text input from the operator. In the text input / display portion 223 shown in FIG. 8, a mixed text of kanji kana “Today is good weather” is input.

  The language processing result display part 224 displays the result of language processing by the language processing unit 102. More specifically, the language processing result display portion 224 does not display the phonetic symbol string obtained as a result of the language processing as it is, but converts the phonetic symbol string to accentuate the reading, accent type, and delimiter type. It is divided into phrases and displayed in a form that is understandable to general operators.

  In the language processing result display portion 224 shown in FIG. 8, the text is divided into three accent phrases “Kyowa”, “Good”, and “Tenki is”. Furthermore, in the first accent phrase “Kyowa”, “Kyo” is underlined to indicate that there is an accent. In addition, “(small)” is added to the end of the word to indicate that the type of separation is a small pause.

  Furthermore, the operator can select the accent phrase displayed in the language processing result display portion 224. For example, the accent phrase “Kyowa” can be selected by designating an area where “Kyowa” is displayed by a user interface such as a mouse. In the language processing result display portion 224 shown in FIG. 8, the first accent phrase “Kyowa” is selected, and a mouse cursor (arrow) is placed on the selected accent phrase “Kyowa”. It is displayed.

  The lower part of the screen displays the accent phrase reading, accent type, accent strength, and division type selected by the operator in the language processing result display portion 224 based on the language processing result. Further, an input area for receiving an instruction to correct these from the operator is provided. When the accent phrase reading or the like is corrected by an instruction from the operator, the phonetic information stored in the phonetic information storage unit 104 is updated with the corrected result, and the language processing result display part 224 is updated. The display of is also updated.

  The phonetic correction area 230 at the bottom of the screen has a reading correction portion 231, a reading change button 232, an accent correction portion 234, and a break correction portion 235.

  When the reading displayed in the reading correction portion 231 is changed by an instruction from the operator, the reading of the selected accent phrase is corrected. When a reading change button 232 is pressed after a reading is input in the reading correction portion 231, the reading of the selected accent phrase is changed to the input reading.

  At this time, the instruction receiving unit 120 acquires the corrected reading of the accent phrase and sends it to the phonetic information correcting unit 110. Then, the phonetic information correction unit 110 corrects the phonetic information by reading the acquired corrected accent phrase. Then, the correction state information file storage unit 131 stores the phonetic information after correction as correction state information.

  Further, when the content of the accent correction portion 234 is changed according to the operator's instruction, the accent type and the accent strength are corrected. The accent correction portion 234 shown in FIG. 8 indicates that the accent type of “Kyowa” is type 1, and “Kyo” has an accent and is a strong accent.

  There are three types of accent type correction: “Kyowa” has 0 to 2 accent types. The accent type can be corrected by inputting a desired numerical value in a column 2341 displayed as 1. Also, the numerical value in the column 2341 can be changed by the up and down arrow buttons 2342 and 2345 on the right side of the input column. Further, the accent strength can be changed by selecting the selection buttons 2344 and 2345 on the right side.

  When the content of the break correction portion 235 is changed according to an instruction from the operator, the break type and break position are corrected. The delimiter type can be selected from strong coupling 2351, weak coupling 2352, small pose 2353, and large pose 2354. Here, when the division type is small or large pause, it indicates the end of the exhalation paragraph. Accent phrases can be connected by pressing the connection button 2355. As shown in FIG. 8, when the first accent phrase “Kyowa” is selected and the link button 2355 is pressed, the second accent phrase “Good” is linked to “Kyowayo”. Is an accent phrase.

  The language processing result display portion 224 of the common area 220 is a button for registering the corrected content as a history. When the language processing result display portion 224 is pressed, the correction state information is stored in the correction state information file storage unit 131. The timing of history registration, that is, the timing of storing the modified content in the modified status information file storage unit 131 is not limited to the method of preparing the history registration button 226 and causing the operator to perform it as in this embodiment. , Reading, accent type, delimiter, and each time it is changed in each correction interface of pitch pattern shape and duration length described later, or when new text is input, it is automatically performed Good.

  When both the phonetic information correction and the later-described prosodic information correction are performed, the correction state information in which the correction contents are paired is stored in the correction state information file storage unit 131.

  The reading button 207 is a button for reading aloud with the modified synthesized sound. When the reading button 207 is pressed, a synthesized sound is output through the synthesized sound generation unit 114 and the synthesized sound output unit 116 based on the corrected phonetic information or prosody information.

  In the prosody mode, a prosody information correction screen 210 shown in FIG. 9 is displayed. The upper part of the prosody information correction screen 210 is the phonetic information correction screen 200. A pitch pattern correction portion 240 is provided at the bottom of the screen. The pitch pattern correction portion 240 displays the pitch pattern of the accent phrase selected by the operator in the language processing result display portion 224 based on the result of prosodic control. The horizontal axis in the prosody modification area 240 indicates time (frame). The vertical axis represents pitch (octave).

  When receiving an instruction from the operator for the prosody modification area 240, the instruction receiving unit 120 acquires the instruction, and the prosody information correcting unit 112 is based on the instruction from the instruction receiving unit 120. And correct the pitch pattern. In the pitch pattern correction portion 209, the accent phrase selected in the language processing result display portion 224 is displayed with the pitch pattern as the center, and an instruction to correct the phoneme duration and pitch pattern shape is accepted.

  In FIG. 9, the first accent phrase “Kyowa” is selected, the pitch pattern is displayed at the center, and the pitch pattern of the second accent phrase “good” is displayed on the right side.

  In the prosody information correction screen 210 shown in FIG. 9, the prosody information stored in the prosody information storage unit 108, that is, data represented by a one-dimensional array can be corrected indirectly. The phoneme duration can be modified for each mora in the accent phrase, for the entire accent phrase, or for the length of the break. The boundaries of each mora in the accent phrase are indicated by vertical lines (including dotted lines). When the operator moves the position of the vertical line from side to side with the mouse, the phoneme duration of each mora is corrected. When the vertical line at the beginning (left end) of the accent phrase is moved left and right with the mouse, the phoneme duration of the entire accent phrase is expanded or contracted.

  When the phoneme duration time length of the selected accent phrase is corrected, the time lengths of other accent phrases are also shifted by the correction value in the selected accent phrase. Thereby, the consistency of the whole prosodic information can be maintained.

  Further, when the vertical line at the head of the second accent phrase “good” in FIG. 9, that is, the accent phrase adjacent to the selected accent phrase is moved left and right with the mouse, the time length of the separation interval is corrected.

  The shape of the pitch pattern can be corrected by drawing a free curve on the pitch pattern with the mouse. 10-1 and 10-2 show examples of pitch pattern shape correction. As shown in FIG. 10A, the entire pitch pattern can be corrected with a free curve (dotted line). Further, as shown in FIG. 10-2, a part of the pitch pattern can be corrected with a free curve (dotted line). Further, by moving the slider bar at the left end in the pitch pattern correction portion 209 up and down with the mouse, the pitch pattern of the selected accent phrase can be moved up and down to correct the pitch level.

  In this way, the prosody information of the prosody control result is displayed, and an instruction from the operator can be received, and the operator can interactively correct the synthesized sound.

  FIG. 11 shows a correction history file screen 250 displayed by the correction display unit 140. The correction history file screen 250 includes a text display portion 251, a history structure display portion 252, an overall parameter display portion 253, and a correction state information display portion 254.

  The text display portion 251 displays a sentence number indicating the number of sentences in the text, a text, and a history registration number indicating the number of history registrations.

  The history structure display part 252 displays the history structure data stored in the history structure data storage unit 138. As shown in FIG. 11, the correspondence between the synthesized sound and the modified sound is shown in a tree structure. In addition, in FIG. 11, the date when each correction status information was produced | generated is displayed as identification information of correction status information. The root node is initial data.

  In this way, in the history structure display portion 252, the derivation relationship between each modified state information and the initial data is displayed in a tree structure, so that the operator can easily display the correspondence between each state information and the initial data. I can grasp it.

  When the operator selects a desired node in the history structure display part 252, the overall parameters corresponding to the overall parameter display part 253 are displayed, and the correction state information corresponding to the correction state information display part 254 is displayed. . In the history structure display portion 252 in FIG. 11, “03/07 / 23_ # 5” is selected.

  In the present embodiment, the date when the correction state information was generated is displayed as the identification information of each correction state information in the history structure display portion 252, but as another example, A file name for identifying the modification status information assigned by the operator may be displayed.

  The overall parameter display portion 253 displays the overall parameters indicated by the modification state information selected in the history structure display portion 252.

  The correction state information display part 254 displays the corrected phonetic information and prosodic information indicated by the correction state information selected in the history structure display part 252. Specifically, the corrected accent phrase and its correction item are displayed. In FIG. 11, the pitch pattern shape of the accent phrase “good” and the correction state information when the duration time of the accent phrase “Tenki is” are corrected are displayed. In other words, “3/07 / 23_ # 5” is a correction in which the pitch pattern shape of the accent phrase “good” and the duration of the accent phrase “Tenki is” are corrected from the correction state of “03/07 / 23_ # 4” It turns out that it is status information.

  In addition, although the correction state information display part 254 concerning this Embodiment displayed the correction item from the last node, as another example, the correction state information display part 254 displays the correction item from initial data. May be.

  In the history structure display part 252, when the operator selects a node of the correction state information, the correction state information corresponding to the node is specified. Then, the phonetic information included in the correction state information is updated and stored in the phonetic information storage unit 104. In addition, the prosody information included in the correction state information is updated and stored in the prosody information storage unit 108. Further, it is reflected in the display contents of the phonetic information correction screen 200 and the prosody information correction screen 210. That is, the contents for the updated phonetic information and the updated prosodic information are displayed.

  In this way, by selecting one of the correction state information displayed in the history structure display portion 252, the correction state that the operator desires to reproduce is reproduced, and the phonetic information correction screen 200 and the prosody are further reproduced. It can be displayed on the information correction screen 210. Thus, the operator can make corrections from a desired correction stage.

  For example, when the operator selects “03/07 / 21_ # 2”, the phonetic information included in the modification state information for “03/07 / 21_ # 2” is the content of the phonetic information storage unit 104. And the prosody information included in the correction state information for “03/07 / 21_ # 2” is updated and stored in the prosody information storage unit 108. Then, the correction can be performed again on the basis of the correction state with respect to “03/07 / 21_ # 2”.

  In the history structure display portion 252, the node notation of each modification state information can be changed. For example, the notation “03/07 / 23_ # 5” can be changed to “modified Fix version”. The changed notation is stored as a notation to be given to the correction state in the information regarding each correction state information in the history management information. Thus, since it can change to the description which an operator desires, recognition by an operator can be made still easier.

  In the history structure display portion 252, each modification state information can be deleted. When deletion is instructed by the operator, the correction state information file storage unit 131 deletes the corresponding correction state information file. Then, the correction history file generation unit 132 updates the contents of the history management information file.

  12 to 14 are flowcharts showing processing of the speech synthesis editing apparatus 10. First, when the text acquisition unit 100 acquires text to be synthesized (step S1, Yes), the process proceeds to steps S2 to S8.

  That is, first, the language processing unit 102 performs language processing on the text received from the text acquisition unit 100 (step S2). As a result, phonetic information divided into accent phrases is generated. Next, each divided accent phrase is numbered (step S3). In the example shown in FIG. 2, “Kyowa” is No. 1, “Good” is No. 2, and “Tenki is” is No. 3. Next, phonetic information is stored in the phonetic information storage unit 104 in a form corresponding to each accent phrase (step S4).

  Next, the correction display unit 140 displays the phonetic information of the language processing result stored in the phonetic information storage unit 104 (step S5). Next, the prosody control unit 106 performs prosody control based on the phonetic information stored in the phonetic information storage unit 104, and generates prosody information (step S6). Next, prosodic information is stored in the prosodic information storage unit 108 in a form corresponding to each accent phrase (step S7). Next, the correction display unit 140 displays the prosody information stored in the prosody information storage unit 108 (step S8). Then, it returns to A and if the text acquisition part 100 acquires a new text (step S1, Yes), it will progress to step S2-S8 again. If no new text is acquired (step S1, No), the process proceeds to step S11.

  In step S11, the phonetic information correction unit 110 determines whether the phonetic information has been corrected. And when it corrects (step S3, Yes), it progresses to step S3. If not corrected (No at Step S3), the process proceeds to Step S12.

  In step S12, the prosody information correction unit 112 determines whether the prosody information has been corrected. And when it corrects (step S12, Yes), it progresses to step S7. If not corrected (No at step S12), the process proceeds to step S13.

  In step S13, it is determined whether or not a history registration of a correction state has been instructed. Here, the history registration instruction is an instruction to register the correction state at the current stage as the correction state information. If instructed (step S13, Yes), the process proceeds to B shown in FIG. If no instruction is given (No at step S13), the process proceeds to step S14.

  When the process proceeds to B in FIG. 13, first, a correction state information file is generated and stored in the correction state information file storage unit 131 (step S20). Next, a correction history file is generated by the correction history file generation unit 132 and stored in the correction history file storage unit 134 (step S21). Then, the contents of the history registration are reflected in the display contents of the phonetic information correction screen 200 and the prosody information correction screen 210. That is, the display content is updated (step S22). And it returns to A shown in FIG.

  In step S14 in FIG. 12, it is determined whether or not a predetermined correction state has been selected by the operator from those displayed by the correction display unit 140 as the correction history. If it is selected (step S14, Yes), the process proceeds to C shown in FIG. If not selected (No at Step S14), the process proceeds to Step S15.

  When the process proceeds to C in FIG. 7, first, correction state information selected by the operator, that is, phonetic information and prosodic information are acquired from the correction state information file storage unit 131 (step S30). Next, the display content of the correction display unit 140 is updated with the acquired phonetic information (step S31). Furthermore, the acquired phonetic information is stored in the phonetic information storage unit 104 (step S32). Next, the display content of the correction display unit 140 is updated with the acquired prosodic information (step S33). Furthermore, the acquired prosodic information is stored in the prosodic information storage unit 108 (step S34). And it returns to A shown in FIG.

  In step S15 of FIG. 12, when the output of the synthesized sound is instructed (step S15, Yes), the process proceeds to step S9. When the output of the synthesized sound is not instructed (No at Step S15), the process proceeds to Step S16.

  In step S <b> 9, the synthesized sound generation unit 114 generates a synthesized sound based on the prosodic information stored in the prosody information storage unit 108. Next, the synthesized sound output unit 116 performs DA conversion on the synthesized sound generated by the synthesized sound generation unit 114 and outputs the result (step S10). Then return to A.

  In step S16, when the termination is instructed (step S16, Yes), the system is terminated. If correction is not instructed (No at step S16), the process returns to A. Thus, the process of the speech synthesis / editing apparatus 10 is completed.

  FIG. 15 is a diagram illustrating a hardware configuration of the speech synthesis editing apparatus 10 according to the embodiment. The voice synthesis editing apparatus 10 controls, as a hardware configuration, a ROM 52 that stores a voice editing program for executing voice editing processing in the voice synthesis editing apparatus 10, and each unit of the voice synthesis editing apparatus 10 according to a program in the ROM 52. Then, a CPU 51 that executes buffering time changing processing, a RAM 53 in which a work area is formed and stores various data necessary for control of the speech synthesis editing apparatus 10, and a communication I connected to a network for communication / F57 and a bus 62 for connecting each part.

  The speech editing program in the speech synthesis / editing apparatus 10 described above is a file in an installable format or an executable format, and is a computer-readable recording such as a CD-ROM, floppy (registered trademark) disk (FD), or DVD. It may be provided by being recorded on a medium.

  In this case, the speech editing program is loaded onto the main storage device by being read from the recording medium and executed by the speech synthesis / editing device 10, and each unit described in the software configuration is generated on the main storage device. It is like that.

  Further, the audio editing program of the present embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network.

  As described above, the present invention has been described using the embodiment, but various changes or improvements can be added to the above embodiment.

  As described above, the speech editing apparatus, speech editing method, and speech editing program according to the present invention are useful for editing synthesized speech, and are particularly suitable for interactive speech editing.

1 is a block diagram showing a speech synthesis / editing apparatus 10. FIG. It is a figure which shows typically the data structure of the phonetic information stored in the phonetic information storage part. It is a figure which shows typically the data structure of the prosodic information which the prosodic information storage part 108 stores. 3 is a block diagram illustrating a detailed functional configuration of a correction management unit 130. FIG. It is a figure which shows typically the data structure of the prosodic information after correction | amendment stored in the correction state information file storage part 131. FIG. It is a figure which shows typically the data structure of the correction history file stored in the correction history file storage part. It is a figure which shows typically the data structure of the history structure data stored in the history structure data storage part 138. It is a figure which shows the phonetic information correction screen 200 which the correction display part 140 displays when correcting phonetic information. It is a figure which shows the prosodic information correction screen 210 which the correction display part 140 displays when correcting prosodic information. It is a figure which shows the example of shape correction of a pitch pattern. It is a figure which shows the example of shape correction of a pitch pattern. It is a figure which shows the correction history file screen 250 which the correction display part 140 displays. 4 is a flowchart showing processing of the speech synthesis editing apparatus 10. 4 is a flowchart showing processing of the speech synthesis editing apparatus 10. 4 is a flowchart showing processing of the speech synthesis editing apparatus 10. 2 is a diagram illustrating a hardware configuration of the speech synthesis editing apparatus 10. FIG.

Explanation of symbols

10 Speech synthesis editing device 51 CPU
52 ROM
53 RAM
57 Communication I / F
62 Bus 100 Text acquisition unit 101 Text input unit 102 Language processing unit 104 Phonetic information storage unit 106 Prosody control unit 108 Prosody information storage unit 110 Phonetic information correction unit 112 Prosody information correction unit 114 Synthetic sound generation unit 116 Synthetic sound output unit DESCRIPTION OF SYMBOLS 120 Instruction reception part 121 Instruction input part 130 Correction management part 131 Correction state information file storage part 132 Correction history file generation part 134 Correction history file storage part 136 History structure data generation part 138 History structure data storage part 140 Correction display part

Claims (10)

A correction means for correcting the synthesized sound generated for the text;
Correction information storage means for storing correction information indicating the correction sound obtained by the correction means;
History structure generation means for generating history structure information indicating a relationship between the correction information stored in the correction information storage means and the synthesized sound before correction;
Display means for displaying a correspondence relationship between the correction information and the synthesized sound associated with each other in the history structure information generated by the history structure generation means;
Selection accepting means for accepting selection of the correction information displayed on the display means;
A speech editing apparatus comprising: a synthesized sound generating means for generating a synthesized sound based on the correction information selected by the selection accepting means. The correction information storage means stores a plurality of correction information obtained by each correction when the correction means corrects the same synthesized sound a plurality of times,
The history structure generating means generates the history structure information indicating a relationship between a plurality of the correction information for the same synthetic sound stored in the correction information storage means and the synthetic sound before correction. The voice editing device according to claim 1.   The speech editing apparatus according to claim 2, wherein the display unit displays a correspondence relationship between the correction information and the synthesized sound in a tree structure. When the correction means corrects the synthetic sound, the correction is performed on the correction information indicating the correction sound obtained by the correction, and the number of corrections the correction means performs on the synthetic sound. Correction order information giving means for giving correction order information indicating whether or not
4. The voice editing apparatus according to claim 1, wherein the display unit displays the correction order information provided by the correction order information addition unit in association with the correction information. 5. .   5. The correction information storage unit stores a difference value between the corrected sound obtained by the correction by the correction unit and the synthesized sound before the correction as the correction information. The voice editing device according to any one of the above.   The display means displays a difference value between the correction sound indicated by the correction information and the synthesized sound before the correction sound is corrected based on the correction information stored in the correction information storage means. The voice editing apparatus according to claim 5. The correction means includes phonetic information correction means for correcting the phonetic information of the synthesized sound,
The said correction information storage means stores the said correction information including the difference value of the said phonetic information before and after the correction by the said phonetic information correction means, The correction information storage means as described in any one of Claim 1 to 6 characterized by the above-mentioned. Voice editing device. The correction means includes prosody information correction means for correcting the prosody information of the synthesized sound,
The voice editing according to any one of claims 1 to 7, wherein the correction information storage unit stores the correction information including a difference value between the prosodic information before and after correction by the prosody information correction unit. apparatus. A modification step for modifying the synthesized sound generated for the text;
A correction information storage step of storing correction information indicating the correction sound obtained by the correction means in the correction information storage means;
A history structure generation step for generating history structure information indicating a relationship between the correction information stored in the correction information storage step and the synthesized sound before correction;
A display step of displaying a correspondence relationship between the correction information and the synthesized sound associated with each other in the history structure information generated in the history structure generation step;
A selection reception step for receiving selection of the correction information displayed in the display step;
And a synthetic sound generation step of generating a synthetic sound based on the correction information selected by the selection receiving step. A voice editing program for causing a computer to execute voice editing processing,
A modification step for modifying the synthesized sound generated for the text;
A correction information storage step of storing correction information indicating the correction sound obtained by the correction means in the correction information storage means;
A history structure generation step for generating history structure information indicating a relationship between the correction information stored in the correction information storage step and the synthesized sound before correction;
A display step of displaying a correspondence relationship between the correction information and the synthesized sound associated with each other in the history structure information generated in the history structure generation step;
A selection reception step for receiving selection of the correction information displayed in the display step;
A speech editing program that causes a computer to execute a synthetic sound generation step of generating a synthetic sound based on the correction information selected in the selection receiving step.

Images