JP6398523B2 - Speech synthesizer, method, and program - Google Patents

Description

  The present invention relates to a technique for performing speech synthesis by selecting speech segments from a speech corpus.

  For a synthesis target generated from input text data, a speech unit is selected by referring to a speech corpus, which is a large-scale digitized language / speech data, and synthesized by connecting the speech unit A voice synthesis technique for outputting voice is known (for example, the technique described in Non-Patent Document 1).

  In such a speech synthesis technology, the following technology is conventionally known as a method for selecting a speech segment sequence that best matches a synthesis target from a speech corpus (for example, the technology described in Non-Patent Document 1). ). First, for each phoneme segment extracted from the input text data, speech segment data having the same phoneme as that phoneme (hereinafter referred to as “segment data”) is extracted from the speech corpus as segment candidate data. The Next, the best segment candidate data set (best segment data string) that minimizes the cost over the entire input text data is determined by a DP (Dynamic Programming) algorithm. Costs include acoustic parameters (feature vector data) such as phoneme sequences and prosodic differences between input text data and each piece of data in the speech corpus, and spectral envelopes between adjacent piece data that are candidate pieces of data. The discontinuity of is used.

  In order to obtain a more natural sensation in the speech synthesis of the unit connection type as described above, it is faithful to the specified prosodic transition, the connection part between the units is smoothly continuous, etc. Is essential.

  In order to realize these simultaneously, a dictionary in which a large amount of original speech information can be used or a dictionary in which phoneme breaks are accurately defined so that continuous sections can be adopted as much as possible is required. In general, segmentation for creating a dictionary is required to be expensive because, for example, the accuracy of automatic segmentation is low, and humans actually cut out after listening to the recorded speech.

  The following technology is known as a conventional speech synthesis technology of the unit connection type considering the continuity of speech (for example, the technology described in Patent Document 1). In connection point search for searching for the optimum connection point when connecting speech units, the optimal connection is made within the predetermined range from the end of the preceding speech unit or within the predetermined range from the beginning of the connected speech unit. This is a technique for setting a point search range.

JP 2008-191334 A

Tsuyoshi Kawai, “Knowledge Base 3-4 Corpus-Based Speech Synthesis”, [online], ver.1 / 2011.1.7, IEICE, [December 25, 2013 search], Internet <URL: http: / /27.34.144.197/files/02/02gun_07hen_03.pdf#page=6>

  However, in the above-described prior art, only the predetermined range of the connection part in the connection between the speech units is set as the search range of the optimum connection point. In the evaluation of the prosodic information when selecting the speech unit from the speech corpus, The segmentation range of the speech segment is not considered. In addition, when evaluating the adaptability of speech units at the time of selecting a segment and the adaptability of connection points between speech units using a predetermined weighted function, the speech is detected when the optimum connection point search range is changed. The degree of fit of the piece also changes, but the change is not taken into account in the above-described conventional technology. From these points, the above-described conventional technology has a problem that it is not possible to select an optimum segment that is suitable for the originally intended prosodic information.

  Furthermore, in order to make the voice suitable for the specified prosody, it is conceivable that the waveform is deformed by a synthesis unit or the like, but the risk of sound quality degradation due to the waveform deformation at this time is also a problem.

  An object of the present invention is to make it possible to correctly select an optimal speech segment from a speech corpus.

  In an example of the aspect, an extraction unit that extracts a segment data string in which phonemes and target prosody are associated with each other from the input text data, and for each segment data extracted by the extraction unit, a speech unit acquired from the speech corpus When the segment data has a duration longer than the target prosody, a generation unit that generates a new speech unit by deleting a part longer than the duration of the target prosody from the acquired speech unit, and the acquired speech unit In addition, a unit cost indicating the degree of inconsistency between each new speech unit and the segment data is calculated, and the speech to the segment data from each speech unit and each new speech unit obtained based on the calculated unit cost A segment list-up unit that lists speech units that are segment candidate data, and a segment data that constitutes a segment of the extracted segment data For each data, the connection cost indicating the discontinuity between the speech segment candidate data corresponding to the segment data and the speech segment candidate data corresponding to the segment data adjacent to the segment data is calculated. Phonemes for generating a speech unit data string by selecting one of speech units listed as speech unit candidate data based on the connected cost and the unit cost of each adjacent speech unit candidate data A column selection unit; and a speech generation unit that generates synthesized speech based on the generated speech segment data sequence.

  According to the present invention, it is possible to correctly select an optimal speech segment from a speech corpus.

1 is a block diagram of an embodiment of a speech synthesizer according to the present invention. It is a block diagram of a waveform selection part. It is explanatory drawing of phoneme sequence cost, prosodic cost, and connection cost. It is explanatory drawing of this embodiment. It is a figure which shows the hardware structural example of the computer which can implement | achieve a speech synthesizer as software processing. It is a figure which shows the data structural example of a control variable. It is a figure which shows the data structural example of segment data. It is a figure which shows the data structural example of prosodic data. It is a figure which shows the data structural example of segment candidate data. It is a figure which shows the example of a data structure of audio | voice dictionary data. It is a figure which shows the data structural example of segment data. It is a figure which shows the example of a data structure of phoneme data. It is a figure which shows the data structural example of feature-value vector data. It is a flowchart which shows the example of a segment selection process. It is explanatory drawing of selection operation | movement of the best segment candidate data. It is a flowchart (the 1) which shows the example of a segment list-up process. It is a flowchart (the 2) which shows the example of a segment list-up process. It is a flowchart which shows the example of a segment cost calculation & candidate addition process. It is a flowchart which shows the example of a phoneme sequence selection process.

  Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram of an embodiment of a speech synthesizer 100 according to the present invention, in which a text input unit 101, a morphological analysis unit 102, a prosody prediction unit 103, a prosody dictionary 104, a waveform selection unit 105, a speech dictionary 106, and a waveform A synthesis unit 107 is provided.

  The text input unit 101 inputs input text data.

  The morpheme analyzer 102 extracts a phoneme string corresponding to the input text data by executing a morpheme analysis process on the input text data input by the text input unit 101. The input text data is segmented for each phoneme in the phoneme string, and the phoneme data indicating each phoneme is registered in the segment data constituting the synthesis target obtained by the segmentation.

  The prosody prediction unit 103 refers to the prosodic dictionary 104 that stores a statistical model related to prosody based on actual speech data based on the linguistic information obtained by the morphological analysis unit 102, so that For each phoneme, a prosody represented by the pitch height, duration length, and intensity (amplitude), which is the fundamental frequency of the vocal cords, is predicted. As a result, the prosody prediction unit 103 generates target prosody data that is prosody information for each phoneme segment, and registers the target prosody data in the segment data constituting the synthesis target.

  That is, in a segment data string generated as a synthesis target from input text data, each segment data has phoneme data and target prosody data.

  The waveform selection unit 105 lists the segment candidate data from the speech corpus in the speech dictionary 106 by evaluating the segment cost for each segment data including the target prosody data and the phoneme data. Then, the waveform selection unit 105 selects the best segment candidate data from the listed segment candidate data by evaluating the connection cost and the segment cost for each segment data.

  The waveform synthesis unit 107 generates and outputs synthesized speech based on the segment data string selected from the speech dictionary 106 by the waveform selection unit 105 for each segment data.

  2 is a block diagram showing a detailed configuration of the waveform selection unit 105 in FIG. 1. The waveform selection unit 105 includes the target prosody data 201, the prosody input unit 202, and the element output from the prosody prediction unit 103 in FIG. A piece selection unit 207 and an evaluation unit 208 are provided. The segment selection unit 207 includes a segment list-up unit 207a, segment candidate data 209 output therefrom, and a phoneme string selection unit 207b. The evaluation unit 208 includes an element evaluation unit 208a and a connection evaluation unit 208b.

  The prosody input unit 202 inputs the target prosody data 201 output by the prosody prediction unit 103 of FIG.

  In the segment selection unit 207, the segment list-up unit 207a performs processing target segment data for each segment data output from the prosody prediction unit 103 in FIG. 1 (hereinafter referred to as “processing target segment data”). Is selected from the speech corpus in the speech dictionary 106 (hereinafter referred to as “processing target segment data”). To describe).

  The unit evaluation unit 208a in the evaluation unit 208 of FIG. 2 includes a phoneme string composed of a phoneme of processing target segment data and a phoneme of each segment data of two segments before and after that, and a phoneme of processing target segment data. The phoneme sequence cost is calculated by comparing the phoneme sequence composed of the phoneme of each of the two segment data before and after that. This phoneme string cost indicates the degree of disagreement between phoneme strings. The phoneme string cost is calculated such that the higher the degree of coincidence between the phoneme string between adjacent segment data and the phoneme string between adjacent segment data, the lower the phoneme string cost. This is because natural synthesized speech can be obtained by selecting segment data in which the preceding and following phoneme sequences match.

Ctxt_distance shown in FIG. 3 indicates the phoneme string cost. In FIG. 3, segment _k-2 , segment _k-1 , segment _k , segment _{k + 1} , and segment _{k + 2} indicate discrete time series of segment data that constitutes a synthesis target corresponding to input text data, Assume that segment _k is processing target segment data. unit _u-2 , unit _u-1 , unit _u , unit _{u + 1} , and unit _{u + 2} represent discrete time series of segment data extracted from a certain position in the speech corpus of the speech dictionary 106, It is assumed that unit _u is processing target segment data. Phoneme sequence cost ctxt_distance for processed fragment data Unit _u in the processing target segment data segment _k is centered on the respective processing target segment data segment _k processed fragment data Unit _u, including itself and two by two front and rear It is calculated as the degree of inconsistency between a total of five consecutive phoneme strings (indicated by “●” in FIG. 3).

In FIG. 2, the segment list-up unit 207a sends to the segment evaluation unit 208a prosodic data of processing target segment data (hereinafter referred to as “segment prosodic data”) and target prosody data of processing target segment data. The prosodic cost between 201 and 201 is calculated and evaluated. Specifically, the segment evaluation unit 208a calculates the prosody cost based on the difference between the target prosody data 201 of the processing target segment data and the segment prosodic data of the processing target segment data. The prosodic cost indicates the distance between the target prosodic data 201 and the segment prosodic data. The pros_distance shown in FIG. 3 indicates the prosodic cost, and is calculated based on the difference between the target prosody data 201 of the processing target segment data segment _{k and} the segment prosodic data of the processing target segment data unit _u .

Here, the segment list-up unit 207a obtains, for each processing target segment data, when the processing target segment data acquired from the speech corpus has a longer duration than the target prosody data 201 of the processing target segment data. New segment data is generated by deleting a portion longer than the continuation length of the target prosody data 201 from the processed target segment data, and the acquired target segment data and new segment data, the target segment data, The element cost indicating the degree of inconsistency is calculated, and based on each calculated element cost, the obtained element data to be processed and the new element data are processed object candidate data for the object segment data. To list. Specifically, as shown in FIG. 4A, as the first pattern, a head portion 401 longer than the duration of the target prosodic data segment _k is deleted from the obtained processing target segment data unit _u. A new speech segment that is cut out is generated and listed. As a second pattern, as shown in FIG. 4 (b), a new voice cut out by deleting the end portion 402 longer than the duration of the target prosody data segment _k from the acquired processing target segment data unit _u. Segments are created and listed. As the third pattern, as shown in FIG. 4 (c), the beginning portion 403 and the end portion 404 longer than the duration of the target prosodic data segment _k are equally deleted from the acquired processing target segment data unit _u. A new speech segment cut out is generated and listed. In addition to the above three patterns, the original process target segment data acquired from the speech corpus is also listed.

  The segment evaluation unit 208a calculates the cost value of the weighted sum of the phoneme sequence cost and the prosodic cost as the segment cost corresponding to each segment data in the current segmentation section and the segmentation section of the above three patterns. The phoneme sequence cost, prosodic cost, and segment cost values obtained in this way are the segment data start position 202s (top_shift in FIG. 9 described later) and segment end position 202e (tail_shift in FIG. 9 described later). ) Is recorded in the memory as segment candidate data 209 (candidate [0] in FIG. 9 to be described later).

  At this time, if the processing target segment data acquired from the speech corpus has a longer duration than the target prosody data 201 of the processing target segment data, one processing target acquired by the segment list-up unit 207a from the speech corpus For each piece of data, the original processing target piece candidate data 209 and the newly generated piece candidate data 209 of the three patterns shown in FIGS. 4A, 4B, and 4C, A total of four segment candidate data 209 is output.

  The segment list-up unit 207a rearranges the segment candidate data 209 in ascending order of segment cost evaluated by the segment evaluation unit 208a, and links and outputs the segment candidate data 209.

  The phoneme string selection unit 207b selects each segment candidate data 209 listed for each segment data (with respect to the processing target segment data), and the segment data immediately before the segment data (hereinafter “front segment data”). Connection indicating the discontinuity of the acoustic parameters between the two segment candidate data of each segment candidate data 209 (hereinafter referred to as “front segment candidate data 209”) listed for Calculate the cost (cont_distance in FIG. 3), recalculate and evaluate the segment cost of the two segment candidate data, select the best segment candidate data, generate a segment data string, and generate a waveform synthesizer It outputs to 107.

  FIG. 5 is a diagram illustrating a hardware configuration example of a computer that can implement the speech synthesis apparatus 100 of FIG. 1 as software processing. In the computer shown in FIG. 5, a CPU 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, an input device 504, an output device 505, an external storage device 506, and a portable recording medium 510 are inserted. A portable recording medium driving device 507 and a communication interface 508, which are connected to each other by a bus 509. The configuration shown in the figure is an example of a computer that can implement the above system, and such a computer is not limited to this configuration.

  A ROM 502 is a memory that stores programs including a speech synthesis program for controlling the computer. The RAM 503 is a memory that temporarily stores a program or data stored in the ROM 502 when each program is executed.

  The external storage device 506 is, for example, an SSD (solid state drive) storage device or a hard disk storage device, and is used for storing input text data and synthesized voice data.

  The CPU 501 controls the entire computer by reading each program from the ROM 502 to the RAM 503 and executing it.

  The input device 504 detects an input operation by a user using a keyboard, a mouse, or the like, and notifies the CPU 501 of the detection result. The input device 504 executes the function of the text input unit 101 in FIG. 1 to input input text data from the outside, and stores the input text data in the RAM 503 or the external storage device 506.

  The output device 505 outputs data sent under the control of the CPU 501 to a display device or a printing device. The output device 505 emits the synthesized voice data output from the waveform synthesis unit 107 of FIG. 1 to the external storage device 506 or the RAM 503 as voice.

  The portable recording medium driving device 507 accommodates a portable recording medium 510 such as an optical disk, SDRAM, or compact flash, and has an auxiliary role for the external storage device 506.

  The communication interface 508 is a device for connecting, for example, a LAN (local area network) or WAN (wide area network) communication line.

The system according to the present embodiment is realized by reading a voice synthesis program having the functions of the processing units shown in FIGS. 1 and 2 from the ROM 502 to the RAM 503 and executing it by the CPU 501. The program may be recorded and distributed in the external storage device 506 or the portable recording medium 510, for example, or may be acquired from the network by the communication interface 508 .

  Next, various data stored in the RAM 503 or the external storage device 506 in order for the computer of FIG. 5 to operate as the speech synthesizer 100 having the functions of FIGS. 1 and 2 will be described.

  FIG. 6 is a diagram illustrating a data configuration example of the control variable WavSel held in the RAM 503. The control variable WavSel holds variable data of unitdb, seg_count, and segment. unitdb holds a pointer to speech dictionary data stored in the speech dictionary 106 on the external storage device 506. seg_count holds the total number of segment data. The segment holds a pointer to the first segment data (the start address of segment [0] in FIG. 7 described later).

  FIG. 7 is a diagram illustrating a data configuration example of segment data segment [0] to segment [seg_count] that is referred to from the segment pointer in the control variable WavSel of FIG. 6 and held in the RAM 503 or the external storage device 506. For each segment data, the prosody prediction unit 103 in FIG. 1 is provided for each total number of seg_count (number stored in seg_count of the control variable WavSel) obtained by morphological analysis of the input text data in the morpheme analysis unit 102 in FIG. To obtain segment [0], segment [1],..., Segment [seg_count-1]. The storage address of the segment data is indicated by the segment of the control variable WavSel. Each segment data segment [i] (i = 0,..., Seg_count-1) holds variable data of seg_id, phone_id, target_prosody, candidate, best_cand, prev, and next. seg_id holds a segment ID (identifier). phone_id holds a phoneme ID. target_prosody holds a pointer to the head of the target prosody data 201 held in the RAM 503 or the external storage device 506. candidate holds a pointer to the first segment candidate data 209 (the leading address of candidate [0] in FIG. 9 described later). best_cand is the best segment candidate data 209 selected corresponding to the current segment data by the processing corresponding to the phoneme string selection unit 207b in FIG. 2 (candidate [0] to candidate [N] in FIG. 9 described later). ,... Is held. prev holds a pointer to the previous segment data, and next holds a pointer to the next segment data. If the current segment data is, for example, segment [1], prev holds the start address of segment [0], and next holds the start address of segment [2]. Also, if the current segment data is, for example, the top data segment [0], prev holds a null value that is an undefined value. If the current segment data is, for example, end data segment [seg_count], next holds a NULL value.

  FIG. 8 shows prosodic data prosody [0], prosody [0] that are referenced from the target_prosody pointer in each segment data of FIG. 7 or the prosody pointer in each segment data of FIG. 11 described later and stored in the RAM 503 or the external storage device 506. FIG. 3 is a diagram illustrating a data configuration example of 1],..., Prosody [N],. Each prosodic data prosody [i] (i = 0,..., N,...) Holds time, pitch, power, prev, and next variable data. time holds the time when the prosody occurs. pitch holds the pitch of the prosody (pitch frequency). power holds the strength of the prosody. prev holds a pointer to the previous prosodic data, and next holds a pointer to the next prosodic data. If the current prosodic data is head data, prev holds a null value, and if it is end data, next holds a null value.

  9 is a segment candidate data candidate [0], candidate [1], which is the segment candidate data 209 of FIG. 2 that is referenced from the target_prosody pointer in the segment data of FIG. 7 and stored in the RAM 503 or the external storage device 506. .., Candidate [N],... Each piece candidate data candidate [i] (i = 0,..., N,...) Is generated by the list-up unit of FIG. Each variable data of best_cand, top_shift, tail_shift, prev, next is stored. unit_id holds a segment ID (see FIG. 12) for identifying segment data in the speech dictionary 106, and is set by the segment list-up unit 207a in FIG. ctxt_distance holds the above-described phoneme string cost (phoneme string disagreement degree), and is calculated and set by the segment evaluation unit 208a in FIG. pros_distance holds the above-mentioned prosody cost (distance between target prosody data 201 and segment prosody data), and is calculated and set by the segment evaluation unit 208a in FIG. unit_distance holds the above-described unit cost, which is the weighted sum of the phoneme sequence cost and the prosody cost, and is calculated and set by the unit evaluation unit 208a in FIG. cont_distance holds the above-described connection cost (feature amount distance at the phoneme connection point), and is calculated and set by the connection evaluation unit 208b in FIG. prev_total_cost holds the total cost determined from the first segment data to the segment data (front segment data) immediately before the segment data to which the segment candidate data belongs. total_cost holds the total cost determined from the first segment data to the segment data to which the segment candidate data belongs, and is calculated and set by the phoneme string selection unit 207b of FIG. 2 as described above. best_cand holds a pointer to the best forward segment candidate data connected to this segment candidate data, and is calculated and set by the phoneme string selection unit 207b described above. Here, the best front segment candidate data is the segment data (front segment data) immediately before the segment data (processing target segment data) to which the segment candidate data (processing target segment candidate data) including best_cand belongs. Is a weighted sum of the total cost determined by the front segment candidate data and the connection cost between the target segment candidate data. Is the smallest (best) forward segment candidate data. “top_shift” is the movement time of the cutout start position when this segment candidate data 209 is selected, and is set by the segment list-up process (FIG. 17) described later. The starting point of this movement time is the start time start in the fragment data of FIG. 11 indicated by unit_id. tail_shift is the movement time 1 of the cutout end position when this segment candidate data 209 is selected, and is set by the segment list-up process (FIG. 17) described later. The starting point of this movement time is (start time start + duration duration) in the fragment data of FIG. 11 indicated by unit_id. prev holds a pointer to the previous piece candidate data, and next holds a pointer to the next piece candidate data. If the current segment candidate data is the top data, prev holds a NULL value, and if the current segment candidate data is end data, next holds a NULL value.

  FIG. 10 is a diagram showing a data configuration example of the speech dictionary data unitdb stored in the RAM 503 or the external storage device 506 constituting the speech dictionary 106 of FIG. 1, and is referenced from the unitdb pointer of the control variable WavSel of FIG. . The voice dictionary data unitdb holds variable data of phone_count, phoneme, unit_count, unit, and fval_count. phone_count holds the number of phonemes defined in the voice dictionary data unitdb. The phoneme holds a pointer to the head phoneme data (the head address of phoneme [0] in FIG. 12). unit_count holds the number of segment data mounted in the speech dictionary data unitdb. unit holds a pointer to the head segment data (head address of unit [0] in FIG. 12) mounted in the speech dictionary data unitdb. fval_count holds the number of feature vectors.

  11 is a diagram showing a data configuration example of the segment data unit [0] to unit [unit_count-1] stored in the RAM 503 or the external storage device 506 constituting the speech dictionary 106 of FIG. Referenced from the unit pointer of the voice dictionary data unitdb. The number of units unit_count mounted in the speech dictionary 106 is registered as unit_count data of the speech dictionary data unitdb in FIG. Each unit data unit [i] (i = 0, ..., unit_count-1) is unit_id, phone_id, start, duration, prosody, prev, next variable data, and featvalue [0] to featvalue [fval_count -1] each array variable data is retained. unit_id holds a unit ID for identifying unit data. phone_id holds a phoneme ID for specifying the phoneme associated with the segment data from the phoneme data described later with reference to FIG. start holds the start time of the segment data. duration holds a duration indicating how long the segment data lasts. The prosody holds a pointer to the head of the segment prosodic data held in the RAM 503 or the external storage device 506 having the data configuration example of FIG. featvalue [0] to featvalue [fval_count-1] hold pointers to the first data of feature vector data having the data configuration example shown in FIG. prev holds a pointer to the previous segment data, and next holds a pointer to the next segment data. If the current segment data is head data, prev holds a null value, and if it is end data, next holds a null value.

  FIG. 12 is a diagram illustrating a data configuration example of phoneme data phoneme [0] to phoneme [phone_count-1] which is referred to from the phoneme pointer in the speech dictionary data unitdb of FIG. 10 and stored in the RAM 503 or the external storage device 506. . The number of phoneme data is set in phone_count data of the voice dictionary data unitdb. Each phoneme data phoneme [i] (i = 0,..., Phone_count-1) holds variable data of phone_id, phomene, prev, and next. phone_id holds a phoneme ID for identifying a phoneme. The segment data shown in FIG. 7 or the segment data shown in FIG. 11 is unitphone in the control variable WavSel in FIG. 6 → phoneme in the speech dictionary data unitdb in FIG. 10 → phoneme in FIG. Of the data phoneme [0] to phoneme [phone_count-1], the phoneme data in which the value of the phone_id is stored is followed and associated with the phoneme name phomene in the phoneme data. phomene holds the phoneme name. prev holds a pointer to the previous phoneme data, and next holds a pointer to the next phoneme data. If the current phoneme data is head data, prev holds a null value, and if the current phoneme data is end data, next holds a null value.

  FIG. 13 shows the feature vector data featvalue [] referenced from the featvalue [i] (i = 0,..., Fval_count-1) pointer in each piece of data in FIG. 12 and stored in the RAM 503 or the external storage device 506. FIG. 6 is a diagram illustrating a data configuration example of 0], featvalue [1],..., Featvalue [N],. Each feature vector data featvalue [i] (i = 0,..., N,...) Is variable data of time, dimension, prev, and next and value [0] to value [dimension-1] Each array variable data of is held. time holds the time corresponding to the feature vector data. dimension holds the number of dimensions of the feature vector data. value [0] to value [dimension-1] hold the feature quantities from the first to the dimension. prev holds a pointer to the previous feature vector data, and next holds a pointer to the next feature vector data. If the current feature vector data is head data, prev holds a null value, and if it is end data, next holds a null value. As described above, this feature vector data is obtained by the connection evaluation unit 208b of FIG. 2 in each piece of piece data at the phoneme connection point between the piece candidate data 209 to be processed and the front piece candidate data 209. Used to calculate the spectral envelope distance.

  FIG. 14 shows an example of the segment selection processing when the CPU 501 of the computer having the hardware configuration example of FIG. 5 realizes the function corresponding to the segment selection unit 207 of FIG. 2 by the processing of the software program. It is a flowchart. The processes described below are all executed by the CPU 501.

  First, the undefined value NULL is stored in the variable data segprev on the RAM 503, and the value of the segment data in the control variable WavSel having the above-described data configuration example in FIG. 6 is stored in the variable data seg (step S1401). . This value is a pointer to the head address of the first segment data segment [0] of the segment data having the data configuration example of FIG. Variable data seg indicates processing target segment data, and variable data segprev indicates forward segment data.

  Next, it is determined whether or not the value of the seg variable is not the undefined value NULL, that is, whether or not all the segment data segment [0] to segment [seg_count] in FIG. 7 have been processed (step S1402).

  While the processing of all the segment data is not completed and the value of the seg variable is not the undefined value NULL and the determination in step S1402 is YES, the value of the seg variable is stored in the segprev variable in step S1405. While the pointer value to the next segment data indicated by the next pointer (see FIG. 7) in the segment data indicated by the variable is newly set to the seg variable, the segment listing process in step S1403 and the phoneme string selection in step S1404 The process is executed repeatedly.

  When the processing of all the segment data is completed and the value of the seg variable is the undefined value NULL and the determination in step S1402 is NO, the segprev variable moves from the rear to the beginning of the entire segment data indicated at the end. Then, while tracing the determined best candidate, the final selection candidate of the segment data of each segment data is designated and output as a segment data string (step S1406).

FIG. 15 is an explanatory diagram of the selection operation of the best segment candidate data realized by the flowchart of FIG. In this figure, ..., segment _k-2 , segment _k-1 , segment _k , segment _{k + 1} , segment _{k + 2} , ... are the segment data constituting the synthesis target corresponding to the input text data. A discrete time series is shown, and segment _k is processing target segment data. In the segment data format of FIG. 7, ..., segment [k-2], segment [k-1], segment [k], segment [k + 1], segment [k + 2], ...・ It becomes. Also, for example, connected to segment _k-2 with a solid line ..., candidate _k-2,0 , candidate _k-2,1 , candidate _k-2,2 , candidate _k-2,3 , candidate _{k-2 , 4} ,... Indicate segment candidate data 209 listed for the segment data segment _k-2 . Similarly, for example, candidate _k-1,0 , candidate _k-1,1 , candidate _k-1,2 , candidate _k-1,3 , candidate _k-1,4 , candidate _k-1,4 , connected to segment _k-1 by a solid line .. Indicate segment candidate data 209 listed for the segment data segment _k-1 . Similarly, for example, candidate _{k, 0} , candidate _{k, 1} , candidate _{k, 2} , candidate _{k, 3} , candidate _{k, 4} ,... Connected to segment _k with a solid line is for segment data segment _k . The segment candidate data 209 to be listed is shown.

Each piece candidate data 209 described above is generated by the piece list-up process in step S1403 of FIG. This segment list-up process realizes the function of the segment list-up unit 207a in FIG. At this time, as described above in the description of the segment list-up unit 207a, for example, the segment candidate data candidate _{k, 0} , candidate _{k, 1} , candidate _{k, 2} , candidate _{k, 3} , candidate _{k, 4} ,. Are listed, the segment cost is calculated with the corresponding segment data segment _k, and the arrangement order of the segment candidate data is determined based on the calculated segment cost.

Then, the segment data segment dark piece candidate data within _k-2 candidate _k-2,4, the dark segment candidate data of the next segment data segment _k-1 candidate _k-1,1 Shows the best segment candidate data 209 detected when this is executed as the processing target segment candidate data 209. Similarly, the segment data segment _k-1 in the dark segment candidate data candidate _k-2,1, the next segment data segment _k dark segment candidate data candidate _{k within, 2} processed element The best segment candidate data 209 detected when it is executed as the segment candidate data 209 is shown. Now, if the processing target segment data is segment _k and the processing target segment candidate data 209 is candidate _{k, 2} , each segment candidate data (forward segment candidate data) in the forward segment data segment _k-1 candidate _{k A} connection cost is calculated among _-1,0 , candidate _k-1,1 , candidate _k-1,2 , candidate _k-1,3 , candidate _k-1,4 ,. Then, the connection cost calculated in this way, the two unit costs, and the weighting of the total cost up to the best unit candidate data 209 that is one further forward determined for the front unit candidate data 209 The sum is calculated, and the result value is taken as the total cost for the candidate element candidate data _{k, 2} . This calculation is, in the process target segment candidate data candidate _{k, 2,} all the front piece candidate data _{candidate k-1,0, candidate k-} 1,1, candidate k-1,2, candidate k-1,3 , candidate _k−1,4 , and the front segment candidate data 209 having the smallest total cost is determined as the best front segment candidate data 209 for the processing target segment candidate data candidate _{k, 2} . For example, the dark-colored front segment candidate data candidate _k-1,1 in the front segment data segment _k-1 is determined as the best front segment candidate data 209 for the processing target segment candidate data candidate _{k, 2} . . Also, a connection cost 1502 _k between the processing target segment candidate data candidate _{k, 2} and the best forward segment candidate data candidate _k-1,1 is calculated. Then, the weighted sum of the unit costs 1501 _k and 1501 _k−1 and the connection cost 1502 _{k and} the sum of the total costs determined for the best forward segment candidate data candidate _k−1,1 are processed. Calculated as the total cost for the target segment candidate data candidate _{k, 2} .

  As described above, in this embodiment, the segment candidate data listed in the processing target segment data while the set of the processing target segment data and the front segment data is sequentially advanced from the head to the end of the segment data. For each segment 209 (processing target segment candidate data 209), the route of the best segment candidate data 209 for each segment data from the beginning to the current processing target segment candidate data 209 in the current processing target segment data is searched. (Repeat processing from step S1402 to S1405). Further, the total cost for each piece candidate data 209 searched in the current processing target segment data is compared, and the best piece candidate data 209 in the processing target segment data is determined. Then, when the process is completed up to the end segment data (NO in step S1402), a search process for sequentially tracing the best front segment candidate data 209 from the end segment data to the start segment data is executed. Fragment data is determined. Finally, when the search process reaches the first segment data, the segment data corresponding to all the segment data are determined, and are output to the waveform synthesis unit 107 in FIG. 2 as a segment data string. In this way, in the present embodiment, an optimum segment data string can be output by a so-called Viterbi algorithm.

  FIGS. 16 and 17 are flowcharts showing a detailed example of the segment list-up process in step S1403 in FIG. 15, and realize the function of the segment list-up unit 207a in FIG.

  First, a pointer from the speech corpus in the speech dictionary 106 to the first segment data is stored in the variable u on the RAM 503 (step S1601). This pointer value can be obtained as the unit pointer value of the speech dictionary data unitdb in FIG.

  Next, while it is determined in step S1602 that the search for the last segment data has not ended (while the determination is YES), a pointer to the next segment data is stored in the variable u in step S1608. For each piece of data, a series of processes from the following steps S1603 to S1620 are executed. Here, the pointer to the next segment data is obtained as the next pointer in the segment data of FIG. 11 indicated by the variable u. The determination in step S1602 can be realized by determining whether or not the value of the next pointer set in step S1608 is NULL.

  The phoneme label sequence of the segment data referenced by the variable u and the segment data of two segment data before and after it, the segment data referenced by the variable seg and the phoneme label sequence of the segment data of two segments before and after the segment data are compared. The phoneme sequence cost is calculated and stored in the variable context on the RAM 503 (step S1603). This calculation method is as described above with reference to FIG. Note that the phoneme label of the segment data referenced by the variable u refers to the phoneme name data phomene in the phoneme data in FIG. 12 from the phoneme ID data phone_id in the segment data in FIG. 9 referenced by the variable u. Is obtained. Further, the phoneme label string of the two segment data before and after the segment data is obtained from the previous segment data obtained by sequentially tracing the prev pointer and the next pointer in the segment data of FIG. 9 referenced by the variable u. It is obtained in the same manner as above. The phoneme label of the segment data referenced by the variable seg is obtained by referring to the phoneme name data phomene in the phoneme data in FIG. 12 from the phoneme ID data phone_id in the segment data in FIG. 7 referenced by the variable seg. Further, the phoneme label string of the segment data of two segments before and after the segment data is the same as described above from the previous segment data obtained by sequentially tracing the prev pointer and the next pointer in the segment data of FIG. 7 referenced by the variable seg. Is obtained.

  Next, an entry candidate [i] (i is the number of the entry added next to the end) of the newly selected candidate is generated at the end of the segment candidate data 209 in FIG. The unit_id is newly assigned, and the value of the variable context on the RAM 503 set in step S1603 is substituted as the phoneme string cost ctxt_distance (step S1604).

  Next, a value of 0 is substituted for the cutout start position top_shift and cutout end position tail_shift of the entry of the segment candidate data 209 of FIG. 9 generated on the RAM 503 in step S1604 (step S1605). The cutout start position and cutout end position indicate the start position and end position of the cutout section when the segment data of the three patterns shown in FIG. 4 are registered, and are processed in steps S1603 to S1606. This is registration processing of the segment data itself acquired from the speech corpus, and no value is cut out, so that each value is 0.

  Next, the prosody cost and the segment cost are further calculated for the entry of the segment candidate data 209 generated in steps S1603 to S1605, and stored in the variable seg on the RAM 503 in step S1405 of FIG. It is registered in the entry of the target segment (step S1606). Details of this processing will be described later with reference to the flowchart of FIG.

  Next, the current process target segment data acquired from the speech corpus has a longer duration than the target prosody data 201 of the process target segment data stored in the variable seg on the RAM 503 in step S1405 of FIG. It is determined whether or not it has (step S1607). The prosodic data length of the processing target segment data can be acquired as the length of the prosodic data referenced from the prosody pointer in the corresponding segment data entry in FIG. The length of the target prosodic data 201 of the processing target segment data can be acquired as the length of the prosodic data referenced from the target_prosody pointer in the corresponding segment data entry in FIG.

  If the determination in step S1607 is NO, the pointer to the next segment data is stored in the variable u (step S1608), and the process returns to step S1602.

  If the determination in step S1607 is YES, the process proceeds to the process of the flowchart of FIG. 17 and the above-described process of listing the segment data of the first pattern (step S1609 to S1611) in FIG. The second pattern segment data listing process (steps S1612 to S1614) in b) and the third pattern segment data listing process (steps S1615 to S1620) in FIG. 4C are executed. Is done.

  First, as registration of the segment data of the first pattern (FIG. 4A), as in step S1604 of FIG. 16, the entry “candidate” of the new selection candidate at the end of the segment candidate data 209 of FIG. [i] is generated, the element ID unit_id is newly assigned to the entry, and the value of the variable context on the RAM 503 set in step S1603 is substituted as the phoneme string cost ctxt_distance (step S1609).

  Next, in the segment start data top_shift of the segment candidate data 209 in FIG. 9 generated on the RAM 503 in step S1609, the processing target is determined from the prosodic data length of the processing target segment data calculated in step S1607 in FIG. The value of the difference in continuation length obtained by subtracting the prosodic data length of the segment data (a value corresponding to 401 in FIG. 4A) is substituted (step S1610). Note that 0 is assigned to the cutout end position tail_shift.

  Next, the prosody cost and the segment cost are further calculated for the entries of the segment candidate data 209 generated in steps S1609 and S1610, and stored in the variable seg on the RAM 503 in step S1405 of FIG. It is registered in the entry of the target segment (step S1611). Details of this processing will be described later with reference to the flowchart of FIG.

  Subsequently, as registration of the segment data of the second pattern (FIG. 4B), as in step S1604 in FIG. 16, the entry of the new selection candidate at the end of the segment candidate data 209 in FIG. candidate [i] is generated, a unit ID unit_id is newly assigned to the entry, and the value of the variable context on the RAM 503 set in step S1603 is substituted as the phoneme string cost ctxt_distance (step S1612).

  Next, at the segmentation end position tail_shift of the entry of the segment candidate data 209 in FIG. 9 generated on the RAM 503 in step S1612, the processing target is calculated from the prosodic data length of the processing target segment data calculated in step S1607 in FIG. The value of the difference in continuation length obtained by subtracting the prosodic data length of the segment data (a value corresponding to 402 in FIG. 4B) is substituted (step S1613). Note that 0 is assigned to the cut start position top_shift.

  Next, the prosody cost and the segment cost are further calculated for the entry of the segment candidate data 209 generated in steps S1612 and S1613, and stored in the variable seg on the RAM 503 in step S1405 of FIG. It is registered in the entry of the target segment (step S1614). Details of this processing will be described later with reference to the flowchart of FIG.

  Finally, as registration of the segment data of the third pattern (FIG. 4C), as in step S1604 in FIG. 16, the entry of the new selection candidate at the end of the segment candidate data 209 in FIG. candidate [i] is generated, a unit ID unit_id is newly assigned to the entry, and the value of the variable context on the RAM 503 set in step S1603 is substituted as the phoneme string cost ctxt_distance (step S1615).

  Next, a value obtained by dividing the continuation length difference value obtained by subtracting the prosodic data length of the processing target segment data from the prosodic data length of the processing target segment data calculated in step S1607 of FIG. (A value corresponding to 403 in FIG. 4C) is registered in the variable shift on the RAM 503 (step S1616).

  Next, the value obtained for the variable shift in step S1616 is substituted into the cutout start position top_shift of the entry of the segment candidate data 209 in FIG. 9 generated on the RAM 503 in step S1615 (step S1617).

  Further, from the value of the continuation length obtained by subtracting the prosodic data length of the processing target segment data from the prosodic data length of the processing target segment data calculated in step S1607 of FIG. 16, the variable shift is set in step S1616. A value obtained by subtracting the obtained value is overwritten on the variable shift (step S1618). Here, the reason why the value of the variable shift obtained in step S1616 is not used as it is is to absorb the remaining slight error.

  Then, the value obtained for the variable shift in step S1618 is substituted into the cutout end position tail_shift of the entry of the segment candidate data 209 in FIG. 9 generated on the RAM 503 in step S1615 (step S1619).

  Finally, the prosody cost and the segment cost are further calculated for the entry of the segment candidate data 209 generated in steps S1615 to S1619, and stored in the variable seg on the RAM 503 in step S1405 of FIG. It is registered in the entry of the target segment (step S1620). Details of this processing will be described later with reference to the flowchart of FIG.

  After the process illustrated in the flowchart of FIG. 17, the process proceeds to the process of step S <b> 1608 of FIG. 16, the pointer to the next segment data is stored in the variable u (step S <b> 1608), and the process of step S <b> 1602 is performed. Return.

  FIG. 18 is a flowchart showing details of the segment cost calculation & candidate addition process executed in step S1606 in FIG. 16 and steps S1611, S1614, or S1620 in FIG.

  First, the prosody transition of the target prosody data 201 of the processing target segment data (phoneme segment) and the processing target segment data is compared, and the prosodic cost is calculated (step S1801). Specific processing is described above in the description of the segment list-up unit 207a in FIG. Then, this prosodic cost value is registered as the prosodic cost pros_distance of the entry of the segment candidate data 209 of FIG. 9 generated on the RAM 503 in step S1604, S1609, S1612, or S1615.

  Next, a cost value obtained by weighted addition of the phoneme string cost calculated in step S1603 and the prosodic cost calculated in step S1801 is calculated as a segment cost. Then, the value of the element cost is registered as the element cost unit_distance of the entry of the element candidate data 209 of FIG. 9 generated on the RAM 503 in step S1604, S1609, S1612, or S1615 (step S1802).

  Finally, the segment candidate data 209 of FIG. 9 generated on the RAM 503 in step S1604, S1609, S1612 or S1615 is added to the entry of the processing target segment stored in the variable seg on the RAM 503 in step S1405 of FIG. Are linked and registered (step S1803). In this link, the unit_id of the entry of the segment candidate data 209 in FIG. 9 is linked from the candidate pointer of the segment data entry in FIG. The unit_id of the entry of the candidate data 209 is linked. At this time, links are established in ascending order of segment cost calculated in step S1802. At this time, the possibility of addition may be limited by the number of candidates and the unit cost value.

  In FIGS. 16 and 17, when the value of the variable u is updated, the search for the last speech unit is completed and the determination in step S1602 is NO, the processing of the flowcharts of FIGS. 16 and 17 is completed. Thus, the one unit list-up process in step S1403 of FIG.

  FIG. 19 is a flowchart showing a detailed example of the phoneme string selection processing in step S1404 in FIG. 14, and implements the function of the phoneme string selection unit 207b in FIG.

  First, the pointer seg.best_cand to the best segment candidate data 209 in the segment data of FIG. 7 referred to by the variable seg is initialized (cleared) (step S1901).

  Next, the address of the head data of the segment candidate data 209 listed for the current segment data (processing target segment data) indicated by the variable seg is set in the variable ct on the RAM 503 (step S1902). . Specifically, this address is obtained as the value of the pointer candidate to the first segment candidate data in the segment data of FIG. 7 referenced by the variable seg.

  Next, while it is determined in step S1903 that the search for the segment candidate data 209 has not ended (while the determination is YES), a pointer to the next segment candidate data 209 is stored in the variable ct in step S1915. For each piece candidate data 209 (processing target piece candidate data 209), a series of processes from the following steps S1904 to S1914 are executed. Here, the pointer to the next processing target segment candidate data 209 is obtained as the next pointer in the segment candidate data 209 of FIG. 9 indicated by the variable ct. The determination in step S1903 can be realized by determining whether or not the value of the next pointer set in step S1915 is NULL.

  First, the address of the head data of the segment candidate data 209 (front segment candidate data 209) listed for the front segment data of the processing target segment data is set in the variable cp on the RAM 503 (step S1904). ). Specifically, this address is obtained as the value of the pointer candidate to the first segment candidate data in the segment data of FIG. 7 referenced by the prev pointer in the segment data of FIG. 7 referenced by the variable seg.

  Subsequently, the best_cand pointer (see the processing target segment candidate data 209 in FIG. 9) in the structure data referred to by the variable ct is initialized (cleared) (step S1905). best_cand is a pointer that refers to the best front segment candidate data 209 connected to the processing target segment candidate data 209.

  Thereafter, while it is determined in step S1906 that the search for the front segment candidate data 209 has not ended (while the determination is YES), a pointer to the next front segment candidate data 209 is stored in the variable cp in step S1911. However, a series of processes from the following steps S1907 to S1910 are executed for each front segment candidate data 209. Here, the pointer to the next front segment candidate data 209 is obtained as the next pointer in the segment candidate data 209 of FIG. 9 indicated by the variable cp. The determination in step S1906 can be realized by determining whether or not the value of the next pointer set in step S1911 is NULL.

  First, the connection cost of the target segment candidate data 209 indicated by the variable ct and the front segment candidate data 209 indicated by the variable cp is calculated (step S1907). Specifically, the feature vector data corresponding to each of the segment candidate data 209 is referred to by following the unit_id data in FIG. 9 → featvalue [0] to featvalue [fval_count-1] in FIG. Feature vector data corresponding to the start position and cutout end position is extracted. The cutout start position and cutout end position are obtained as top_shift and tail_shift data of the entry in FIG. Then, the distance of each spectral envelope (for example, the Euclidean distance of the mel cepstrum) of each piece data calculated by the set of these feature vector data is calculated as the connection cost.

  Next, the sum of the determined total cost cp.total_cost to the front in the front segment candidate data 209 shown in FIG. 9 and the connection cost calculated in step S1907 is multiplied by the weighting coefficient is calculated. The Then, the addition result is stored as a total_cost value (ct.total_cost) in the structure of the processing target segment candidate data 209 indicated by the variable ct on the RAM 503 (step S1908).

  Subsequently, whether or not the above ct.total_cost value is smaller than the total_cost value shown in FIG. 9 in the processing target segment candidate data 209 on the original RAM 503 indicated by the variable ct, that is, the current ct.total_cost value. It is determined whether or not is the best (step S1909). If the determination in step S1909 is YES, all member variable values shown in FIG. 9 in the original processing target segment candidate data 209 indicated by the variable ct are all member variable values of the structure indicated by the variable ct. (Step S1910). If the determination in step S1909 is NO, the replacement in step S1910 is not executed.

  Thereafter, the pointer next to the next front segment candidate data 209 is stored in the variable cp, and the processing returns to step S1906. After the determination in step S1906, the series of processing from step S1907 to S1910 described above is repeatedly executed. Is done.

  As a result of updating the value of the variable cp, when the search of the last front segment candidate data 209 is completed and the determination in step S1906 is NO, first, the best forward information indicated by the variable ct is set in step S1910. It is rewritten with the data of the best information storage variable (step S1912).

  Thereafter, the total_cost value newly set in the processing target segment candidate data 209 indicated by the variable ct is the total_cost value in the segment candidate data 209 in FIG. 9 indicated by the best_cand pointer in the segment data in FIG. 7 indicated by the variable seg. It is determined whether it is smaller than (step S1913). If the determination in step S1913 is YES, the value of variable ct is set to the best_cand pointer in the segment data of FIG. 7 indicated by variable seg (step S1914). If the determination in step S1913 is NO, step S1724 is not executed.

  Thereafter, a pointer to the next processing target segment candidate data 209 is stored in the variable ct (step S1915), and the process returns to step S1903.

  As a result of the update of the value of the variable ct, when the search of the last candidate segment data 209 to be processed is completed and the determination in step S1903 is NO, the processing of the flowchart in FIG. 19 ends, and step S1404 in FIG. The one phoneme string selection process is completed.

  Through the above series of processing, when the processing of all segment data is completed and the value of the seg variable is the undefined value NULL and the determination in step S1402 in FIG. 14 is NO, the segment data indicated by the segprev variable is the last. 9 is referred to by referring to the best_cand pointer in FIG. 7, and one piece of data is determined by unit_id data. Thereafter, a search process for sequentially tracing the best_cand pointer in FIG. 9 from the last segment data to the first segment data is executed, and the unit_id data in the best front segment candidate data is referenced for each front segment data, Each piece of data is determined. Finally, when the search process reaches the first segment data, the segment data corresponding to all the segment data are determined, and are output to the waveform synthesis unit 107 in FIG. 2 as a segment data string.

  According to the above embodiment, it is possible to select from the speech corpus a segment that is faithful to the specified prosodic transition and in which the connected portions of the segments are smoothly connected.

Regarding the above embodiment, the following additional notes are disclosed.
(Appendix 1)
An extraction unit that extracts a string of segment data in which phonemes and target prosody are associated with each other from input text data;
For each segment data extracted by the extraction unit, when the speech segment acquired from the speech corpus has a duration longer than the target prosody of the segment data, the duration of the target prosody from the acquired speech segment A generation unit that generates a new speech segment in which a longer part is deleted;
A unit cost indicating a degree of disagreement between the acquired speech unit and the new speech unit and the segment data is calculated, and the acquired speech unit and the new unit are calculated based on the calculated unit cost. A speech unit list-up unit that lists speech units to be speech unit candidate data for the segment data from each speech unit;
For each of the segment data constituting the extracted segment data row, the speech unit candidate data corresponding to the segment data and the speech unit candidate data corresponding to the segment data adjacent to the segment data A speech cost listed as the speech segment candidate data on the basis of the calculated connection cost and the segment cost of each of the adjacent speech segment candidate data. A phoneme string selection unit that selects one of the pieces and generates the speech element data string;
A speech generation unit that generates synthesized speech based on the generated speech segment data string;
A speech synthesizer comprising:
(Appendix 2)
The generation unit, as a new speech segment from which the portion longer than the target prosody continuation length has been deleted from the acquired speech segment, a new speech segment from which the top location of the acquired speech segment has been deleted, The speech according to appendix 1, which generates a new speech unit from which the end portion of the acquired speech unit has been deleted, and a new speech unit from which the beginning and end portions of the acquired speech unit have been uniformly deleted. Synthesizer.
(Appendix 3)
The segment data and the speech segment include phoneme data and prosodic data,
The segment cost includes a phoneme string composed of a phoneme of the segment data to be processed and a phoneme of each segment data of a predetermined number of segments before and after the segment data, and a phoneme of the speech segment candidate data to be processed. And the phoneme sequence cost calculated by comparing the phoneme sequence composed of the predetermined number of phoneme data of each speech unit data before and after the speech unit candidate data, and the segment data to be processed The speech synthesizer according to appendix 1 or 2, wherein the speech synthesizer calculates a weighted sum of the prosodic data and the prosodic cost calculated based on a difference between the prosody data of the speech segment candidate data to be processed.
(Appendix 4)
The connection cost is a phoneme connection between the speech segment candidate data to be processed corresponding to the segment data to be processed and the speech segment candidate data corresponding to the segment data immediately before the segment data. The speech synthesizer according to any one of appendices 1 to 3, wherein the speech synthesizer is calculated as a distance between the feature vector data at points.
(Appendix 5)
A method for synthesizing speech with a speech synthesizer having a speech corpus that stores speech segments, the speech synthesizer comprising:
Extract a segment data string that associates phonemes and target prosody from the input text data,
For each of the extracted segment data, when the speech segment acquired from the speech corpus has a duration longer than the target prosody of the segment data, it is longer than the duration of the target prosody from the acquired speech segment. Generate a new speech segment with the part removed,
Calculating a unit cost indicating a degree of mismatch between the acquired speech unit and the new speech unit and the segment data;
List speech units that are speech unit candidate data for the segment data from the acquired speech unit and the new speech unit based on each calculated unit cost,
For each of the segment data constituting the extracted segment data row, the speech unit candidate data corresponding to the segment data and the speech unit candidate data corresponding to the segment data adjacent to the segment data Calculate the connection cost that shows the discontinuity between
Based on the calculated connection cost and the unit cost of each of the adjacent speech unit candidate data, the speech unit data is selected by selecting one of speech units listed as the speech unit candidate data. Generate columns,
A speech synthesis method for generating synthesized speech based on the generated speech unit data string.
(Appendix 6)
In a computer used as a speech synthesizer having a speech corpus for storing speech segments,
Extracting a segment data string associated with phonemes and target prosody from input text data;
For each of the extracted segment data, when the speech segment acquired from the speech corpus has a duration longer than the target prosody of the segment data, it is longer than the duration of the target prosody from the acquired speech segment. Generating a new speech segment with the location removed;
Calculating a unit cost indicating a degree of mismatch between the acquired speech unit and the new speech unit and the segment data;
Listing speech units that are speech unit candidate data for the segment data from the acquired speech unit and the new speech unit based on the calculated unit costs, respectively;
For each of the segment data constituting the extracted segment data row, the speech unit candidate data corresponding to the segment data and the speech unit candidate data corresponding to the segment data adjacent to the segment data Calculating a connection cost indicating a discontinuity between;
Based on the calculated connection cost and the unit cost of each of the adjacent speech unit candidate data, the speech unit data is selected by selecting one of speech units listed as the speech unit candidate data. Generating a column;
Generating synthesized speech based on the generated speech segment data sequence;
A program that executes

100 speech synthesis apparatus 101 text input unit 102 morphological analysis unit 103 prosody prediction unit 104 prosody dictionary 105 waveform selection unit 106 speech dictionary 107 waveform synthesis unit 201 target prosody data 202 prosody input unit 207 segment selection unit 207a segment list up unit 207b Phoneme string selection unit 208 Evaluation unit 208a Segment evaluation unit 208b Connection evaluation unit 209 Segment candidate data 210 Composition unit 501 CPU
502 ROM (Read Only Memory)
503 RAM (Random Access Memory)
504 Input device 505 Output device 506 External storage device 507 Portable recording medium driving device 508 Communication interface 509 Bus 510 Portable recording medium

Claims (6)

An extraction unit that extracts a string of segment data in which phonemes and target prosody are associated with each other from input text data;
For each segment data extracted by the extraction unit, when the speech segment acquired from the speech corpus has a duration longer than the target prosody of the segment data, the duration of the target prosody from the acquired speech segment A generation unit that generates a new speech segment in which a longer part is deleted;
A unit cost indicating a degree of disagreement between the acquired speech unit and the new speech unit and the segment data is calculated, and the acquired speech unit and the new unit are calculated based on the calculated unit cost. A speech unit list-up unit that lists speech units to be speech unit candidate data for the segment data from each speech unit;
For each of the segment data constituting the extracted segment data row, the speech unit candidate data corresponding to the segment data and the speech unit candidate data corresponding to the segment data adjacent to the segment data A speech cost listed as the speech segment candidate data on the basis of the calculated connection cost and the segment cost of each of the adjacent speech segment candidate data. A phoneme string selection unit that selects one of the pieces and generates the speech element data string;
A speech generation unit that generates synthesized speech based on the generated speech segment data string;
A speech synthesizer comprising:   The generation unit, as a new speech segment from which the portion longer than the target prosody continuation length has been deleted from the acquired speech segment, a new speech segment from which the top location of the acquired speech segment has been deleted, The new speech unit from which the end part of the acquired speech unit is deleted, and the new speech unit from which the beginning part and the end part of the acquired speech unit are equally deleted are generated. Speech synthesizer. The segment data and the speech segment include phoneme data and prosodic data,
The segment cost includes a phoneme string composed of a phoneme of the segment data to be processed and a phoneme of each segment data of a predetermined number of segments before and after the segment data, and a phoneme of the speech segment candidate data to be processed. And the phoneme sequence cost calculated by comparing the phoneme sequence composed of the predetermined number of phoneme data of each speech unit data before and after the speech unit candidate data, and the segment data to be processed The speech synthesizer according to claim 1, wherein the speech synthesizer is calculated as a weighted sum of prosodic data and prosodic cost calculated based on a difference between prosodic data of the speech segment candidate data to be processed.   The connection cost is a phoneme connection between the speech segment candidate data to be processed corresponding to the segment data to be processed and the speech segment candidate data corresponding to the segment data immediately before the segment data. The speech synthesizer according to claim 1, wherein the speech synthesizer is calculated as a distance between feature vector data at points. A method for synthesizing speech with a speech synthesizer having a speech corpus that stores speech segments, the speech synthesizer comprising:
Extract a segment data string that associates phonemes and target prosody from the input text data,
For each of the extracted segment data, when the speech segment acquired from the speech corpus has a duration longer than the target prosody of the segment data, it is longer than the duration of the target prosody from the acquired speech segment. Generate a new speech segment with the part removed,
Calculating a unit cost indicating a degree of mismatch between the acquired speech unit and the new speech unit and the segment data;
List speech units that are speech unit candidate data for the segment data from the acquired speech unit and the new speech unit based on each calculated unit cost,
For each of the segment data constituting the extracted segment data row, the speech unit candidate data corresponding to the segment data and the speech unit candidate data corresponding to the segment data adjacent to the segment data Calculate the connection cost that shows the discontinuity between
Based on the calculated connection cost and the unit cost of each of the adjacent speech unit candidate data, the speech unit data is selected by selecting one of speech units listed as the speech unit candidate data. Generate columns,
A speech synthesis method for generating synthesized speech based on the generated speech unit data string. In a computer used as a speech synthesizer having a speech corpus for storing speech segments,
Extracting a segment data string associated with phonemes and target prosody from input text data;
For each of the extracted segment data, when the speech segment acquired from the speech corpus has a duration longer than the target prosody of the segment data, it is longer than the duration of the target prosody from the acquired speech segment. Generating a new speech segment with the location removed;
Calculating a unit cost indicating a degree of mismatch between the acquired speech unit and the new speech unit and the segment data;
Listing speech units that are speech unit candidate data for the segment data from the acquired speech unit and the new speech unit based on the calculated unit costs, respectively;
For each of the segment data constituting the extracted segment data row, the speech unit candidate data corresponding to the segment data and the speech unit candidate data corresponding to the segment data adjacent to the segment data Calculating a connection cost indicating a discontinuity between;
Based on the calculated connection cost and the unit cost of each of the adjacent speech unit candidate data, the speech unit data is selected by selecting one of speech units listed as the speech unit candidate data. Generating a column;
Generating synthesized speech based on the generated speech segment data sequence;
A program that executes