US20170004821A1 - Voice synthesizer, voice synthesis method, and computer program product - Google Patents
Voice synthesizer, voice synthesis method, and computer program product Download PDFInfo
- Publication number
- US20170004821A1 US20170004821A1 US15/266,065 US201615266065A US2017004821A1 US 20170004821 A1 US20170004821 A1 US 20170004821A1 US 201615266065 A US201615266065 A US 201615266065A US 2017004821 A1 US2017004821 A1 US 2017004821A1
- Authority
- US
- United States
- Prior art keywords
- content
- voice
- unit
- text
- tagged text
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004590 computer program Methods 0.000 title claims description 4
- 238000001308 synthesis method Methods 0.000 title claims description 3
- 238000003860 storage Methods 0.000 claims abstract description 64
- 230000015572 biosynthetic process Effects 0.000 claims abstract description 41
- 238000003786 synthesis reaction Methods 0.000 claims abstract description 41
- 239000003550 marker Substances 0.000 claims description 88
- 238000000034 method Methods 0.000 claims description 15
- 238000000605 extraction Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 description 50
- 238000010586 diagram Methods 0.000 description 30
- 230000007704 transition Effects 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 230000014509 gene expression Effects 0.000 description 5
- 230000008451 emotion Effects 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000001174 ascending effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/08—Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
- G10L13/10—Prosody rules derived from text; Stress or intonation
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/02—Methods for producing synthetic speech; Speech synthesisers
- G10L13/033—Voice editing, e.g. manipulating the voice of the synthesiser
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/02—Methods for producing synthetic speech; Speech synthesisers
- G10L13/033—Voice editing, e.g. manipulating the voice of the synthesiser
- G10L13/0335—Pitch control
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/02—Methods for producing synthetic speech; Speech synthesisers
- G10L13/04—Details of speech synthesis systems, e.g. synthesiser structure or memory management
Definitions
- Embodiments described herein relate generally to a voice synthesizer, a voice synthesis method, and a computer program product.
- a voice synthesis field as effective methods for achieving desired synthesized voices including expressions such as various expressions of feelings, methods are known that produce voice waveforms of synthesized voices based on of tagged texts.
- the tagged text is composed of text serving as a target of voice synthesis and tag information described by a markup language and added to the text.
- the tag information is used for controlling voice synthesis of the text interposed with tags.
- a voice synthesis engine can obtain a desired synthesized voice by selecting a dictionary used for voice synthesis and adjusting a prosody parameter based on the tag information, for example.
- a user can produce the tagged text by adding the tag information to text using an editor. This manner, however, requires the user to perform complicated work. It is, thus, common to produce the tagged text by applying a preliminarily produced template to the text serving as the target of voice synthesis.
- FIG. 1 is a block diagram illustrating an overall structure of a voice synthesizer in a first embodiment
- FIG. 2 is a block diagram illustrating an exemplary structure of a content selection unit
- FIG. 3 is a block diagram illustrating an exemplary structure of a content generation unit
- FIG. 4 is a block diagram illustrating an exemplary structure of a content registration unit
- FIG. 5 is a schematic diagram conceptually illustrating exemplary content registered in a content storage unit
- FIG. 6 is a schematic diagram to explain a storage form of the content in the content storage unit
- FIG. 7 is a schematic diagram to explain transition of screens in a user interface (UI) screen displayed on a user terminal;
- UI user interface
- FIG. 8 is a schematic diagram illustrating an example of a marker content list screen
- FIG. 9 is a schematic diagram illustrating an example of a relevant content list screen
- FIG. 10 is a schematic diagram illustrating an example of a content detail screen
- FIG. 11 is a schematic diagram illustrating an example of a content generation screen
- FIG. 12 is a flowchart illustrating an exemplary procedure of the processing performed by the content selection unit
- FIG. 13 is a flowchart illustrating an exemplary procedure of the processing performed by the content generation unit
- FIG. 14 is a flowchart illustrating an exemplary procedure of the processing performed by the content registration unit
- FIG. 15 is a block diagram illustrating an exemplary structure of a content selection unit in a second embodiment
- FIG. 16 is a schematic diagram to explain transition of screens in the UI screen displayed on the user terminal.
- FIG. 17 is a schematic diagram illustrating an example of a content search screen
- FIG. 18 is a flowchart illustrating an exemplary procedure of the processing performed by the content selection unit in the second embodiment.
- FIG. 19 is a block diagram schematically illustrating an exemplary hardware structure of the voice synthesizer.
- a voice synthesizer includes a content selection unit, a content generation unit, and a content registration unit.
- the content selection unit determines selected content among a plurality of pieces of content registered in a content storage unit.
- the content includes tagged text in which tag information for controlling voice synthesis is added to text serving as a target of the voice synthesis.
- the content generation unit applies the tag information in the tagged text included in the selected content to designated text to generate new content.
- the content registration unit registers the generated new content in the content storage unit.
- a voice synthesizer performs voice synthesis based on tagged text in which tag information is added to text serving as the target of the voice synthesis, and particularly has a mechanism that efficiently generates the tagged text.
- a combination of the tagged text and a voice waveform of a synthesized voice generated based on the tagged text is called a piece of “content” hereinafter.
- the content may include other information such as identification information about a voice synthesis dictionary used for voice synthesis besides the tagged text and the voice waveform of the synthesized voice.
- Any known method is adoptable for voice synthesis. Examples of the known method include phoneme based voice synthesis and voice synthesis using a hidden Markov model (HMM). The detailed descriptions thereof are omitted.
- HMM hidden Markov model
- FIG. 1 is a block diagram illustrating an overall structure of a voice synthesizer 1 according to a first embodiment.
- the voice synthesizer 1 in the first embodiment can be achieved as a server on a network that provides web-based services to a user terminal 2 connected to the network as a client, for example.
- the user terminal 2 is information equipment used by a user, such as a personal computer, a tablet terminal, or a smartphone.
- the user terminal 2 includes various resources constituting a computer system, such as a central processing unit (CPU) and a memory, hardware such as various input devices, and various types of software such as an operating system (OS) and a web browser.
- CPU central processing unit
- OS operating system
- the voice synthesizer 1 in the embodiment is not necessary structured as a single apparatus.
- the voice synthesizer 1 may be structured as a system including a plurality of linked apparatuses.
- the voice synthesizer 1 may be achieved as a virtual machine that operates in a cloud computing system.
- the voice synthesizer 1 includes a content selection unit 10 , a content generation unit 20 , a content registration unit 30 , a content storage unit 40 , and a voice synthesis dictionary 50 .
- the content selection unit 10 causes the user terminal 2 to display a user interface (UI) screen, receives operation input performed by the user while using the UI screen, and determines selected content based on the user's operation among a plurality of pieces of content registered in the content storage unit 40 .
- the selected content is the content selected from among the pieces of content in accordance with the user's operation.
- the content generation unit 20 causes the user terminal 2 to display the UI screen, receives operation input performed by the user while using the UI screen, and applies the tag information in the tagged text included in the selected content determined by the content selection unit 10 to text designated by the user to generate new content.
- the content registration unit 30 registers the new content generated by the content generation unit 20 in the content storage unit 40 .
- the content storage unit 40 stores therein marker content serving as a marker and the content generated by the content generation unit 20 .
- the marker content which emphasizes a specific feature, is preliminarily registered in the content storage unit 40 .
- the content registration unit 30 registers the content generated by the content generation unit 20 in the content storage unit 40 in association with the marker content in accordance with a similarity with the marker content.
- the content storage unit 40 may be provided outside the voice synthesizer 1 .
- the content registration unit 30 accesses the content storage unit 40 outside the voice synthesizer 1 via a network and registers the content produced by the content generation unit 20 in the content storage unit 40 , for example.
- the content selection unit 10 accesses the content storage unit 40 outside the voice synthesizer 1 via the network and acquires necessary content from the content storage unit 40 in accordance with the user's operation, for example.
- the voice synthesis dictionary 50 is used when the content generation unit 20 generates a voice waveform of a synthesized voice based on the tagged text.
- the voice synthesis dictionary 50 is classified based on the features of the respective synthesized voices to be generated, for example. Based on the tag information in the tagged text, an optimum dictionary is selected.
- the voice synthesis dictionary 50 may be provided outside the voice synthesizer 1 . In this case, the content generation unit 20 accesses the voice synthesis dictionary 50 outside the voice synthesizer 1 via the network and acquires necessary information from the voice synthesis dictionary 50 , for example.
- FIG. 2 is a block diagram illustrating an exemplary structure of the content selection unit 10 .
- the content selection unit 10 includes a marker content presentation unit 11 , a relevant content presentation unit 12 , a selected content determination unit 13 , and a reproduction unit 14 .
- the marker content presentation unit 11 presents a list of marker content registered in the content storage unit 40 to the user.
- the marker content presentation unit 11 generates a marker content list screen SC 1 (refer to FIG. 8 ), which is described later, as the UI screen displayed by the user terminal 2 , and causes the user terminal 2 to display the marker content list screen SC 1 , for example.
- the relevant content presentation unit 12 presents, to the user, a list of relevant content associated with the marker content selected by the user from the list of marker content.
- the relevant content presentation unit 12 generates a relevant content list screen SC 2 (refer to FIG. 9 ), which is described later, as the UI screen displayed by the user terminal 2 , and causes the user terminal 2 to display the relevant content list screen SC 2 , for example.
- the selected content determination unit 13 determines the relevant content selected from the list of relevant content to be the selected content.
- the selected content determination unit 13 determines the relevant content selected by the user from the relevant content list screen SC 2 displayed on the user terminal 2 to be the selected content, for example.
- the reproduction unit 14 reproduces the voice waveform of the synthesized voice included in the marker content or the voice waveform of the synthesized voice included in the relevant content in accordance with the user's operation, and causes the user terminal 2 to output the reproduced voice waveform as a voice from a speaker of the user terminal 2 , for example.
- the reproduction unit 14 reproduces the voice waveform of the synthesized voice included in the marker content designated by the user from the marker content list screen SC 1 displayed on the user terminal 2 or the voice waveform of the synthesized voice included in the relevant content designated by the user from the relevant content list screen SC 2 displayed on the user terminal 2 , and causes the user terminal 2 to output the reproduced voice waveform as a voice from a speaker of the user terminal 2 , for example.
- FIG. 3 is a block diagram illustrating an exemplary structure of the content generation unit 20 .
- the content generation unit 20 includes a tag information extraction unit 21 , a tagged text generation unit 22 , a tagged text amendment unit 23 , a voice waveform generation unit 24 , and a reproduction unit 25 .
- the tag information extraction unit 21 extracts the tag information from the tagged text included in the selected content determined by the selected content determination unit 13 .
- the tag information includes a start tag located forward in the text to which the tag information is applied and an end tag located backward in the text to which the tag information is applied.
- an element name is described in the start tag and the end tag.
- an attribute value of the element represented by the element name is described in the start tag.
- Examples of the element included in the tag information include gender (attribute values are male and female), emotion (attribute values are pleasure, grief, anger, warmth, and the like), and prosody (attribute values are pitch of a voice, speaking speed, and the like).
- prosody is an element name that represents the prosody
- pitch is the attribute value (attribute value is +5%) that represents the pitch of the voice in the element of prosody
- rate is the attribute value (attribute value is ⁇ 2%) that represents the utterance speed in the element of prosody.
- the tagged text generation unit 22 applies the tag information extracted by the tag information extraction unit 21 to text designated by the user to generate the tagged text.
- the text designated by the user is assumed to be “hello” and the tag information described above is assumed to be extracted by the tag information extraction unit 21 .
- the tagged text amendment unit 23 amends the tagged text generated by the tagged text generation unit 22 based on the user's operation. For example, the tagged text amendment unit 23 amends the attribute value (in the example, the values +5% or ⁇ 2%) of the tag information included in the tagged text generated by the tagged text generation unit 22 based on the user's operation.
- the voice waveform generation unit 24 generates the voice waveform of the synthesized voice corresponding to the tagged text generated by the tagged text generation unit 22 using the voice synthesis dictionary 50 .
- the voice waveform generation unit 24 generates the voice waveform of the synthesized voice corresponding to the amended tagged text.
- the reproduction unit 25 reproduces the voice waveform of the synthesized voice generated by the voice waveform generation unit 24 , and causes the user terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof, for example.
- FIG. 4 is a block diagram illustrating an exemplary structure of the content registration unit 30 .
- the content registration unit 30 includes a similarity calculation unit 31 , a classification unit 32 , and a use frequency updating unit 33 .
- the similarity calculation unit 31 calculates a similarity of the new content generated the content generation unit 20 with the marker content for registering the new content in the content storage unit 40 in association with the marker content.
- the marker content which emphasizes a specific feature, is preliminarily registered in the content storage unit 40 .
- the attribute value of the attribute representing emotion e.g., pleasure, romance, anger, or warmth
- the attribute value of each of the pitch of the voice and the speaking speed (rate) is assumed to be capable of being set to a range from ⁇ 10% to +10%.
- marker content M 1 , M 2 , . . . , Mk are preliminarily registered in the content storage unit 40 .
- FIG. 5 is a schematic diagram conceptually illustrating exemplary content registered in the content storage unit 40 .
- the similarity calculation unit 31 calculates the similarity of the new content to each of the pieces of marker content preliminarily registered in the content storage unit 40 .
- the similarity between two pieces of content ci and cj can be obtained by calculating an inter-content distance D (ci,cj) represented by expressions (1) and (2), for example.
- A ⁇ pleasure( ci ) ⁇ pleasure( cj ) ⁇ 2 + ⁇ sorrow( ci ) ⁇ sorrow( cj ) ⁇ 2 + ⁇ anger( ci ) ⁇ anger( cj ) ⁇ 2 + . . . + ⁇ warmth( ci ) ⁇ warmth( cj ) ⁇ 2 + ⁇ pitch( ci ) ⁇ pitch( cj ) ⁇ 2 + ⁇ rate( ci ) ⁇ rate( cj ) ⁇ 2 (2)
- the two pieces of content ci and cj are more similar to each other.
- the two pieces of content having the same gender attribute value are used for calculating the distance therebetween.
- a member relating to the gender attribute value may be added to expression (2) so as to calculate the inter-content distance D (ci,cj) between the different genders.
- the classification unit 32 classifies the content generated by the content generation unit 20 based on the similarity calculated by the similarity calculation unit 31 .
- the classification is processing that registers the content generated by the content generation unit 20 in the content storage unit 40 in association with the marker content similar to the content (e.g., the marker content having an inter-content distance equal to or smaller than a certain threshold with respect to the content).
- the content registration unit 30 registers the content in the content storage unit 40 in association with each of the pieces of marker content. Every time the content generation unit 20 generates new content, the classification unit 32 classifies the generated new content.
- the pieces of content associated with the marker content are stored in the content storage unit 40 in the order of the similarity for each piece of marker content, for example.
- FIG. 6 is a schematic diagram explaining a storage form of the content in the content storage unit 40 .
- content C 1 , C 2 , . . . , Cm generated by the content generation unit 20 are stored in the content storage unit 40 in such a manner that the respective pieces of content C 1 , C 2 , . . . , Cm are classified into corresponding classes typified by respective pieces of marker content M 1 , M 2 , . . . , Mk.
- the content C 1 , C 2 , . . . , Cm are similar to the marker content M 1 , M 2 , . . . , Mk, respectively.
- Each piece of content is associated with information about the use frequency of the content.
- the use frequency represents the number of times the content is used as the selected content. Every time the content is used as the selected content by the content generation unit 20 for generating new content, the value of the use frequency of the content used as the selected content is incremented by one.
- the use frequency of the content serves as an index that presents whether the content is popular content to the user.
- the use frequency updating unit 33 increments the value of the use frequency of the content used as the selected content for generating new content to update the use frequency when the new content generated by the content generation unit 20 is registered.
- the following describes specific examples of the UI screen the voice synthesizer 1 in the embodiment causes the user terminal 2 to display with reference to FIGS. 7 to 11 .
- FIG. 7 is a schematic diagram to explain transition of screens in the UI screen displayed on the user terminal 2 .
- the voice synthesizer 1 in the embodiment causes the user terminal 2 to sequentially display, as the UI screen, the marker content list screen SC 1 , the relevant content list screen SC 2 , a content detail screen SC 3 , and a content generation screen SC 4 in accordance with the screen transition illustrated in FIG. 7 , for example.
- FIG. 8 is a schematic diagram illustrating an example of the marker content list screen SC 1 .
- the marker content list screen SC 1 is the UI screen that presents a list of marker content preliminarily registered in the content storage unit 40 to the user. As illustrated in FIG. 8 , the marker content list screen SC 1 is provided with a “title” column 101 , a “gender” column 102 , a “parameter” column 103 , a gender switching button 104 , an up-down button 105 , a “reproduce” button 106 , a “content” button 107 , and a “close” button 108 .
- the “title” column 101 displays the name of each piece of marker content.
- the “gender” column 102 displays the gender attribute value (male or female) of each piece of marker content.
- the “parameter” column 103 displays the attribute such as emotion or prosody and the attribute value (parameter) of each piece of marker content.
- the marker content list screen SC 1 illustrated in FIG. 8 the list of marker content is presented for each of the genders of male and female.
- the gender of marker content to be presented is switched by operating the gender switching button 104 .
- FIG. 8 illustrates the presentation of the list of marker content in relation to the gender of male.
- the up-down button 105 is used for designating any marker content from the list of marker content by moving a cursor (not illustrated) upward and downward.
- the “reproduce” button 106 is used for reproducing the voice waveform of the synthesized voice included in the designated marker content to output the reproduced voice waveform as the voice.
- the “reproduce” button 106 is operated in a state where any marker content is designated from the presented list of marker content, the synthesized voice of the designated marker content is output from the speaker of the user terminal 2 .
- the user can perform trail listening on the synthesized voice of desired marker content using the “reproduce” button 106 .
- the “content” button 107 is used for selecting desired marker content from the list of marker content.
- the “content” button 107 is operated in a state where any marker content is designated from the presented list of marker content, the UI screen displayed on the user terminal 2 changes from the marker content list screen SC 1 to the relevant content list screen SC 2 , in which a list of relevant content associated with the designated marker content is presented.
- the “close” button 108 is used for closing the marker content list screen SC 1 .
- the “close” button 108 is operated, the display of the UI screen on the user terminal 2 ends.
- FIG. 9 is a schematic diagram illustrating an example of the relevant content list screen SC 2 .
- the relevant content list screen SC 2 is the UI screen that presents a list of relevant content registered in the content storage unit 40 in association with the marker content selected by the user using the marker content list screen SC 1 to the user.
- the relevant content list screen SC 2 is provided with a “title” column 201 , a “distance” column 202 , a “use frequency” column 203 , an up-down button 204 , a “reproduce” button 205 , a “return” button 206 , a “detail” button 207 , and a “close” button 208 .
- the “title” column 201 displays the name of the marker content selected in the marker content list screen SC 1 and the name of each piece of relevant content.
- the “distance” column 202 displays the inter-content distance D (ci,cj) between the marker content and the relevant content for each relevant content.
- the “use frequency” column 203 displays the use frequency of the marker content and the use frequency of the relevant content for each piece of relevant content. As illustrated in FIG.
- a plurality of pieces of relevant content associated with the marker content are displayed in a list in ascending order of the values of the inter-content distances D (ci,cj), i.e., in such manner that with an increase in similarity of the relevant content with the marker content, the relevant content is displayed at the upper position in the list.
- the pieces of relevant content having the same value of the inter-content distance D (ci,cj) are displayed in the list in descending order of the values of the use frequencies of the pieces of relevant content.
- the order of relevant content in the list is not limited to the example illustrated in FIG. 9 .
- a plurality of pieces of relevant content may be displayed in a list in descending order of the values of the use frequencies of the respective pieces of relevant content.
- the up-down button 204 is used for designating any relevant content from the list of relevant content by moving a cursor (not illustrated) upward and downward.
- the “reproduce” button 205 is used for reproducing the voice waveform of the synthesized voice included in the designated relevant content to output the reproduced voice waveform as the voice.
- the synthesized voice of the designated relevant content is output from the speaker of the user terminal 2 .
- the user can perform trail listening on the synthesized voice of desired relevant content using the “reproduce” button 205 .
- the “return” button 206 is used for returning the UI screen displayed on the user terminal 2 from the relevant content list screen SC 2 illustrated in FIG. 9 to the marker content list screen SC 1 illustrated in FIG. 8 .
- the “detail” button 207 is used for checking the details of desired relevant content.
- the “detail” button 207 is operated in a state where any relevant content is designated from the presented list of relevant content, the UI screen displayed on the user terminal 2 changes from the relevant content list screen SC 2 to the content detail screen SC 3 , in which detailed information about the designated relevant content is displayed.
- the “close” button 208 is used for closing the relevant content list screen SC 2 .
- the “close” button 208 is operated, the display of the UI screen on the user terminal 2 ends.
- FIG. 10 is a schematic diagram illustrating an example of the content detail screen SC 3 .
- the content detail screen SC 3 is the UI screen that presents the detailed information about the relevant content selected by the user using the relevant content list screen SC 2 to the user.
- the content detail screen SC 3 is provided with a content name column 301 , a “used dictionary” column 302 , a “text” column 303 , a “tag information” column 304 , a “reproduce” button 305 , a “return” button 306 , a “copy” button 307 , and a “close” button 308 .
- the content name column 301 displays the name of the content.
- the “used dictionary” column 302 displays the name of the voice synthesis dictionary 50 that was used in generating the voice waveform of the synthesized voice included in the content.
- the “text” column 303 displays text (the whole of the text) in the tagged text included in the content.
- the “tag information” column 304 displays the tagged text corresponding to the designated range in the text displayed in the “text” column 303 . The user designates any range in the text displayed in the “text” column 303 and can check the tag information about the designated range in the “tag information” column 304 .
- the “reproduce” button 305 is used for reproducing the voice waveform of the synthesized voice corresponding to the tagged text displayed in the “tag information” column 304 to output the reproduced voice waveform as the voice.
- the “reproduce” button 305 is operated in a state where the tagged text corresponding to the range designated by the user is displayed in the “tag information” column 304 , the synthesized voice corresponding to the tagged text is output from the speaker of the user terminal 2 .
- the user can perform trail listening on the synthesized voice of the desired range using the “reproduce” button 305 .
- the “return” button 306 is used for returning the UI screen displayed on the user terminal 2 from the content detail screen SC 3 illustrated in FIG. 10 to the relevant content list screen SC 2 illustrated in FIG. 9 .
- the “copy” button 307 is used for determining the content to be the selected content.
- the UI screen displayed on the user terminal 2 changes from the content detail screen SC 3 to the content generation screen SC 4 .
- the “close” button 308 is used for closing the content detail screen SC 3 .
- the “close” button 308 is operated, the display of the UI screen on the user terminal 2 ends.
- FIG. 11 is a schematic diagram illustrating an example of the content generation screen SC 4 .
- the content generation screen SC 4 is the UI screen used for producing new content by applying the tag information in the selected content to text designated by the user.
- the content generation screen SC 4 is provided with a “title” column 401 , a “used dictionary” column 402 , a “text” column 403 , a “tag information” column 404 , an “apply” button 405 , a “reproduce” button 406 , an “edit” button 407 , a “return” button 408 , a “register” button 409 , and a “close” button 410 .
- the “title” column 401 displays the name of new content to be generated using the content generation screen SC 4 .
- the user can set a desired name to the new content by writing any name into the “title” column 401 .
- the “used dictionary” column 402 displays the name of the voice synthesis dictionary 50 that was used in generating the voice waveform of the synthesized voice included in the selected content.
- the user can change the name of the voice synthesis dictionary 50 used in generating the voice waveform of the synthesized voice included in the new content by changing the name of the voice synthesis dictionary 50 displayed in the “used dictionary” column 402 .
- the “text” column 403 displays the text serving as the target of voice synthesis.
- the user can designate the text serving as the target of voice synthesis by writing any text into the “text” column 403 .
- the “tag information” column 404 displays the tagged text generated by applying the tag information in the tagged text included in the selected content to the text displayed in the “text” column 403 .
- the “apply” button 405 is used for reproducing the voice waveform of the synthesized voice corresponding to the tagged text displayed in the “tag information” column 404 .
- the voice waveform of the synthesized voice is generated based on the tagged text displayed in the “tag information” column 404 .
- the voice synthesis dictionary 50 displayed in the “used dictionary” column 402 is used.
- the “reproduce” button 406 is used for reproducing the voice waveform of the synthesized voice generated based on the tagged text displayed in the “tag information” column 404 to output the reproduced voice waveform as the voice.
- the synthesized voice generated as a result of the operation of the “apply” button 405 is output from the speaker of the user terminal 2 .
- the user can perform trail listening on the synthesized voice of content to be newly generated using the “reproduce” button 406 .
- the “edit” button 407 is used for amending the tagged text displayed in the “tag information” column 404 .
- the tagged text displayed in the “tag information” column 404 can be edited.
- the user can amend the tagged text of the content to be newly generated by operating the “edit” button 407 and performing amendment operation on the tagged text displayed in the “tag information” column 404 .
- the attribute value in the example illustrated in FIG. 11 , +5%
- the “return” button 408 is used for returning the UI screen displayed on the user terminal 2 from the content generation screen SC 4 illustrated in FIG. 11 to the content detail screen SC 3 illustrated in FIG. 10 .
- the “register” button 409 is used for registering the generated new content in the content storage unit 40 .
- the “register” button 409 is operated, a combination of the tagged text displayed in the “tag information” column 404 and the voice waveform of the synthesized voice generated based on the tagged text is registered in the content storage unit 40 as new content.
- the “close” button 410 is used for closing the content generation screen SC 4 .
- the “close” button 410 is operated, the display of the UI screen on the user terminal 2 ends.
- the following describes exemplary operation of the voice synthesizer 1 that generates content and registers the produced content while causing the user terminal 2 to display the UI screens exemplarily illustrated in FIGS. 7 to 11 .
- FIG. 12 is a flowchart illustrating an exemplary procedure of the processing performed by the content selection unit 10 .
- the marker content presentation unit 11 causes the user terminal 2 to display the marker content list screen SC 1 exemplarily illustrated in FIG. 8 (step S 101 ).
- the gender switching button 104 in the marker content list screen SC 1 is operated after the user terminal 2 displays the marker content list screen SC 1 , the gender of marker content to be presented as a list is switched, the illustration of which is omitted in the flowchart in FIG. 12 .
- the “close” button 108 is operated at any timing, the processing ends.
- step S 102 It is determined whether the “reproduce” button 106 is operated in a state where any of the marker content in the list displayed on the marker content list screen SC 1 is designated. If the “reproduce” button 106 is operated (Yes at step S 102 ), the reproduction unit 14 reproduces the voice waveform of the synthesized voice included in the designated marker content, and causes the user terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof (step S 103 ), and then the processing returns to step S 102 .
- step S 104 it is determined whether the “content” button 107 is operated in a state where any of the marker content displayed in the list is designated. If the “content” button 107 is not operated (No at step S 104 ), the processing returns to step S 102 . If the “content” button 107 is operated (Yes at step S 104 ), the relevant content presentation unit 12 causes the user terminal 2 to display the relevant content list screen SC 2 exemplarily illustrated in FIG. 9 (step S 105 ).
- step S 101 the processing returns to step S 101 , at which the user terminal 2 displays the marker content list screen SC 1 again, the illustration of which is omitted in the flowchart in FIG. 12 .
- the “close” button 208 is operated at any timing, the processing ends.
- step S 106 It is determined whether the “reproduce” button 205 is operated in a state where any of the relevant content in the list displayed on the relevant content list screen SC 2 is designated. If the “reproduce” button 205 is operated (Yes at step S 106 ), the reproduction unit 14 reproduces the voice waveform of the synthesized voice included in the designated relevant content, and causes the user terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof (step S 107 ), and then the processing returns to step S 106 .
- step S 106 it is determined whether the “detail” button 207 is operated in a state where any of the relevant content displayed in the list is designated (step S 108 ). If the “detail” button 207 is not operated (No at step S 108 ), the processing returns to step S 106 . If the “detail” button 207 is operated (Yes at step S 108 ), the selected content determination unit 13 causes the user terminal 2 to display the content detail screen SC 3 exemplarily illustrated in FIG. 10 (step S 109 ).
- step S 105 At which the user terminal 2 displays the relevant content list screen SC 2 again, the illustration of which is omitted in the flowchart in FIG. 12 .
- the “close” button 308 is operated at any timing, the processing ends.
- step S 110 It is determined whether the “reproduce” button 305 is operated in a state where the tagged text is displayed in the “tag information” column 304 in the content detail screen SC 3 (step S 110 ). If the “reproduce” button 305 is operated (Yes at step S 110 ), the reproduction unit 14 reproduces the voice waveform of the synthesized voice corresponding to the tagged text displayed in the “tag information” column 304 , and causes the user terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof (step S 111 ), and then the processing returns to step S 110 .
- step S 110 it is determined whether the “copy” button 307 is operated in a state where the tagged text is displayed in the “tag information” column 304 (step S 112 ). If the “copy” button 307 is not operated (No at step S 112 ), the processing returns to step S 110 . If the “copy” button 307 is operated (Yes at step S 112 ), the selected content determination unit 13 determines the content containing detail information displayed by the content detail screen SC 3 to be the selected content (step S 113 ). The processing is, then, passed to the content generation unit 20 . The processing performed by the content selection unit 10 , thus, ends.
- FIG. 13 is a flowchart illustrating an exemplary procedure of the processing performed by the content generation unit 20 .
- the tag information extraction unit 21 causes the user terminal 2 to display the content generation screen SC 4 exemplarily illustrated in FIG. 11 (step S 201 ).
- the user writes the text serving as the target of voice synthesis into the “text” column 403 in the content generation screen SC 4 .
- the tag information extraction unit 21 extracts the tag information from the tagged text included in the selected content.
- the tagged text generation unit 22 applies the tag information extracted by the tag information extraction unit 21 to the text written in the “text” column 403 to generate the tagged text.
- the tagged text generated by the tagged text generation unit 22 is displayed in the “tag information” column 404 in the content generation screen SC 4 .
- step S 202 It is determined whether the “edit” button 407 is operated in a state where the tagged text is displayed in the “tag information” column 404 (step S 202 ). If the “edit” button 407 is operated (Yes at step S 202 ), the tagged text amendment unit 23 receives the user's operation to amend the tagged text, and amends the tagged text displayed in the “tag information” column 404 (step S 203 ), and then the processing returns to step S 202 .
- step S 204 it is determined whether the “apply” button 405 is operated in a state where the tagged text is displayed in the “tag information” column 404 (step S 204 ). If the “apply” button 405 is not operated (No at step S 204 ), the processing returns to step S 202 . If the “apply” button 405 is operated (Yes at step S 204 ), the voice waveform generation unit 24 generates the voice waveform of the synthesized voice using the voice synthesis dictionary 50 displayed in the “used dictionary” column 402 based on the tagged text displayed in the “tag information” column 404 (step S 205 ).
- step S 206 It is determined whether the “reproduce” button 406 is operated (step S 206 ). If the “reproduce” button 406 is operated (Yes at step S 206 ), the reproduction unit 25 reproduces the voice waveform of the synthesized voice generated at step S 205 and causes the user terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof (step S 207 ), and then the processing returns to step S 206 .
- step S 208 it is determined whether the “register” button 409 is operated. If the “register” button 409 is not operated (No at step S 208 ), the processing returns to step S 206 . If the “register” button 409 is operated (Yes at step S 208 ), the processing is passed to the content registration unit 30 . The processing performed by the content generation unit 20 , thus, ends.
- FIG. 14 is a flowchart illustrating an exemplary procedure of the processing performed by the content registration unit 30 .
- the similarity calculation unit 31 calculates the inter-content distance D (ci,cj) between the new content generated by the content generation unit 20 and the marker content for each marker content registered in the content storage unit 40 (step S 301 ).
- the classification unit 32 classifies the new content generated by the content generation unit 20 based on the inter-content distance D (ci,cj) calculated at step S 301 in such a manner that the classification unit 32 registers the new content in the content storage unit 40 in association with the marker content similar to the content (step S 302 ).
- the new content registered in the content storage unit 40 serves as a candidate of the selected content used for generating other content in the future.
- the use frequency updating unit 33 updates the use frequency of the content used as the selected content when the content generation unit 20 generates the new content (step S 303 ). The processing performed by the content registration unit 30 , thus, ends.
- the voice synthesizer 1 in the embodiment determines the selected content used for generating new content out of the pieces of content registered in the content storage unit 40 in accordance with operation performed by the user while using the UI screen.
- the voice synthesizer 1 applies the tag information in the tagged text included in the selected content to the text designated by the user to generate new content.
- the voice synthesizer 1 registers the generated new content in the content storage unit 40 as a candidate of the selected content.
- the voice synthesizer 1 in the embodiment thus, does not need to preliminarily prepare a number of templates for generating the tagged text and prepare training data and correct answer data for automatically generating the templates.
- the voice synthesizer 1 can generate the tagged text from any text using the content generated in the past, thereby making it possible to efficiently generate the tagged text.
- the voice synthesizer 1 in the embodiment allows the user to generate the tagged text by selecting the tag information to be applied to the text or to amend the tagged text if necessary while performing trial listening on the synthesized voice of the content generated in the past or the synthesized voice generated by applying the desired tag information to the text. As a result, the voice synthesizer 1 can efficiently obtain the synthesized voice the user desires.
- a voice synthesizer in the second embodiment includes a content selection unit having a different structure from that in the voice synthesizer in the first embodiment.
- the voice synthesizer in the second embodiment is described as “a voice synthesizer 1 ′” and the content selection unit specific to the voice synthesizer 1 ′ is described as a content selection unit 60 , so that the voice synthesizer 1 ′ and the content selection unit 60 are distinguished from those in the first embodiment.
- the structure other than the above is the same as that of the first embodiment. The duplicate descriptions are thus appropriately omitted.
- the following describes the content selection unit 60 .
- FIG. 15 is a block diagram illustrating an exemplary structure of the content selection unit 60 .
- the content selection unit 60 includes a content search unit 61 , a retrieved content presentation unit 62 , a selected content determination unit 63 , and a reproduction unit 64 .
- the content search unit 61 retrieves the content including the tagged text matching an input keyword from among the pieces of content registered in the content storage unit 40 .
- the content search unit 61 causes the user terminal 2 to display a content search screen SC 5 (refer to FIG. 17 ), which is described later, as the UI screen displayed on the user terminal 2 , and retrieves the content including the tagged text matching a keyword input by the user using the content search screen SC 5 from among the pieces of content registered in the content storage unit 40 .
- the retrieved content presentation unit 62 presents a list of retrieved content retrieved by the content search unit 61 to the user.
- the retrieved content presentation unit 62 causes the user terminal 2 to display a list of retrieved content retrieved by the content search unit 61 on the content search screen SC 5 displayed as the UI screen, for example.
- the selected content determination unit 63 determines the retrieved content selected from the list of retrieved content to be the selected content.
- the selected content determination unit 63 determines the retrieved content selected by the user from the list of retrieved content displayed on the content search screen SC 5 to be the selected content, for example.
- the reproduction unit 64 reproduces the voice waveform of the synthesized voice included in the retrieved content, and causes the user terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof, for example.
- the reproduction unit 64 reproduces the voice waveform of the synthesized voice included in the retrieved content designated by the user from the list of retrieved content displayed on the content search screen SC 5 , and causes the user terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof, for example.
- FIG. 16 is a schematic diagram to explain transition of screens in the UI screen that the voice synthesizer 1 ′ in the second embodiment causes the user terminal 2 to display.
- the voice synthesizer 1 ′ in the embodiment causes the user terminal 2 to sequentially display, as the UI screen, the content search screen SC 5 , the content detail screen SC 3 , and the content generation screen SC 4 in accordance with the screen transition illustrated in FIG. 16 , for example.
- FIG. 17 is a schematic diagram illustrating an example of the content search screen SC 5 .
- the content search screen SC 5 is the UI screen that receives input of a keyword for retrieving the content and presents the list of retrieved content serving as the result of the search to the user.
- the content search screen SC 5 is provided with a “keyword” input column 501 , a “title” column 502 , a “use frequency” column 503 , a “search” button 504 , an up-down button 505 , a “reproduce” button 506 , a “detail” button 507 , and a “close” button 508 .
- the “keyword” input column 501 serves as an area in which a keyword used for the search is input.
- the user can input any text such as text serving as the target of voice synthesis in the “keyword” input column 501 as a keyword.
- the “title” column 502 displays the name of each piece of retrieved content serving as the result of the search.
- the “use frequency” column 503 displays the use frequency of each piece of retrieved content serving as the result of the search.
- the “search” button 504 is used for the search using the keyword input in the “keyword” input column 501 .
- the “search” button 504 is operated in a state where the keyword is input in the “keyword” input column 501 , the retrieved content including the tagged text matching the keyword is retrieved from the content storage unit 40 , and the name of the retrieved content is displayed in the “title” column 502 while the use frequency of the retrieved content is displayed in the “use frequency” column 503 .
- the up-down button 505 is used for designating any retrieved content from the list of retrieved content by moving a cursor (not illustrated) upward and downward.
- the “reproduce” button 506 is used for reproducing the voice waveform of the synthesized voice included in the designated retrieved content to output the reproduced voice waveform as the voice.
- the “reproduce” button 506 is operated in a state where any retrieved content is designated from the presented list of retrieved content, the synthesized voice of the designated retrieved content is output from the speaker of the user terminal 2 .
- the user can perform trail listening on the synthesized voice of desired retrieved content using the “reproduce” button 506 .
- the “detail” button 507 is used for checking the details of desired retrieved content.
- the “detail” button 507 is operated in a state where any retrieved content is designated from the presented list of retrieved content, the UI screen displayed on the user terminal 2 changes from the content search screen SC 5 to the content detail screen SC 3 (refer to FIG. 10 ), in which detailed information about the designated retrieved content is displayed.
- the “close” button 508 is used for closing the content search screen SC 5 .
- the “close” button 508 is operated, the display of the UI screen on the user terminal 2 ends.
- FIG. 18 is a flowchart illustrating an exemplary procedure of the processing performed by the content selection unit 60 .
- the content search unit 61 causes the user terminal 2 to display the content search screen SC 5 exemplarily illustrated in FIG. 17 (step S 401 ).
- the “close” button 508 is operated at any timing after the user terminal 2 displays the content search screen SC 5 , the processing ends, the illustration of which is omitted in the flowchart in FIG. 18 .
- step S 402 It is determined whether the “search” button 504 is operated in a state where the keyword is input in the “keyword” input column 501 in the content search screen SC 5 (step S 402 ). If the “search” button 504 is not operated (No at step S 402 ), the processing returns to step S 402 , at which the determination is repeated. If the “search” button 504 is operated (Yes at step S 402 ), the content search unit 61 retrieves the retrieved content including the tagged text matching the keyword input in the “keyword” input column 501 from among the pieces of content registered in the content storage unit 40 (step S 403 ). The content search unit 61 causes the list of searched content obtained as the search result to be displayed on the content search screen SC 5 (step S 404 ).
- step S 405 It is determined whether the “reproduce” button 506 is operated in a state where any piece of the retrieved content in the list displayed on the content search screen SC 5 is designated (step S 405 ). If the “reproduce” button 506 is operated (Yes at step S 405 ), the reproduction unit 64 reproduces the voice waveform of the synthesized voice included in the designated retrieved content, and causes the user terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof (step S 406 ), and then the processing returns to step S 405 .
- step S 405 it is determined whether the “detail” button 507 is operated in a state where any piece of the retrieved content displayed in the list is designated (step S 407 ). If the “detail” button 507 is not operated (No at step S 407 ), the processing returns to step S 405 . If the “detail” button 507 is operated (Yes at step S 407 ), the selected content determination unit 63 causes the user terminal 2 to display the content detail screen SC 3 exemplarily illustrated in FIG. 10 (step S 408 ).
- step S 401 at which the user terminal 2 displays the content search screen SC 5 again, the illustration of which is omitted in the flowchart in FIG. 18 .
- the “close” button 308 is operated at any timing, the processing ends
- step S 409 It is determined whether the “reproduce” button 305 is operated in a state where the tagged text is displayed in the “tag information” column 304 in the content detail screen SC 3 (step S 409 ). If the “reproduce” button 305 is operated (Yes at step S 409 ), the reproduction unit 64 reproduces the voice waveform of the synthesized voice corresponding to the tagged text displayed in the “tag information” column 304 , and causes the user terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof (step S 410 ), and then the processing returns to step S 409 .
- step S 409 it is determined whether the “copy” button 307 is operated in a state where the tagged text is displayed in the “tag information” column 304 (step S 411 ). If the “copy” button 307 is not operated (No at step S 411 ), the processing returns to step S 409 . If the “copy” button 307 is operated (Yes at step S 411 ), the selected content determination unit 63 determines the retrieved content containing detail information displayed by the content detail screen SC 3 to be the selected content (step S 412 ). The processing is, then, passed to the content generation unit 20 . The processing performed by the content selection unit 60 , thus, ends.
- the voice synthesizer 1 ′ in the embodiment retrieves the content including the tagged text matching the keyword from among the pieces of content registered in the content storage unit 40 in accordance with operation performed by the user using the UI screen.
- the voice synthesizer 1 ′ determines the selected content used for generating new content from among the retrieved content.
- the voice synthesizer 1 ′ applies the tag information in the tagged text included in the determined selected content to the text designated by the user to generate new content.
- the voice synthesizer 1 ′ registers the generated new content in the content storage unit 40 as a candidate of the selected content.
- the voice synthesizer 1 ′ in the embodiment can generate the tagged text from any text using content generated in the past in the same manner as the voice synthesizer 1 in the first embodiment.
- the voice synthesizer 1 ′ can, thus, efficiently generate the tagged text.
- the voice synthesizer 1 ′ in the embodiment can narrow down the candidates of the selected content using the keyword, thereby making it possible to more efficiently generate the tagged text.
- the respective functional components in the voice synthesizer 1 in the first embodiment can be achieved by a program (software) executed using a general purpose computer system as basic hardware, for example.
- FIG. 19 is a block diagram schematically illustrating an exemplary hardware structure of the major part of the voice synthesizer 1 .
- the major part of the voice synthesizer 1 is structured as a general purpose computer system that includes a processor 71 such as a CPU, a main storage unit 72 such as a random access memory (RAM), an auxiliary storage unit 73 using various storage devices, a communication interface 74 , and a bus 75 that connects each one to another.
- the auxiliary storage unit 73 may be connected to the respective units with a local area network (LAN) in a wired or wireless manner, for example.
- LAN local area network
- the respective functional components of the voice synthesizer 1 are achieved by the processor 71 executing a program stored in the auxiliary storage unit 73 using the main storage unit 72 , for example.
- the program is recorded and provided as a computer program product on a computer-readable recording medium such as a compact disc read only memory (CD-ROM), a flexible disk (FD), a compact disc recordable (CD-R), and a digital versatile disc (DVD), as an installable or executable file, for example.
- a computer-readable recording medium such as a compact disc read only memory (CD-ROM), a flexible disk (FD), a compact disc recordable (CD-R), and a digital versatile disc (DVD), as an installable or executable file, for example.
- the program may be stored in another computer connected to a network such as the Internet and provided by being downloaded via the network.
- the program may be provided or distributed via a network such as the Internet.
- the program may be embedded and provided in a ROM (the auxiliary storage unit 73 ) in the computer.
- the program has a module structure including the functional components (the content selection unit 10 , the content generation unit 20 , and the content registration unit 30 ) of the voice synthesizer 1 .
- the processor 71 reads the program from the recording medium and executes the program. Once the program is executed, the respective components are loaded into the main storage unit 72 , so that the respective components are formed in the main storage unit 72 .
- Part of or the whole of the respective components of the voice synthesizer 1 can be achieved using dedicated hardware such as an application specific integrated circuit (ASIC) and a field-programmable gate array (FPGA).
- ASIC application specific integrated circuit
- FPGA field-programmable gate array
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Description
- This application is a continuation of PCT international application Ser. No. PCT/JP2015/075638 filed on Sep. 9, 2015, which designates the United States, and which claims the benefit of priority from Japanese Patent Application No. 2014-221770, filed on Oct. 30, 2014; the entire contents of which are incorporated herein by reference.
- Embodiments described herein relate generally to a voice synthesizer, a voice synthesis method, and a computer program product.
- In a voice synthesis field, as effective methods for achieving desired synthesized voices including expressions such as various expressions of feelings, methods are known that produce voice waveforms of synthesized voices based on of tagged texts. The tagged text is composed of text serving as a target of voice synthesis and tag information described by a markup language and added to the text. The tag information is used for controlling voice synthesis of the text interposed with tags. A voice synthesis engine can obtain a desired synthesized voice by selecting a dictionary used for voice synthesis and adjusting a prosody parameter based on the tag information, for example.
- A user can produce the tagged text by adding the tag information to text using an editor. This manner, however, requires the user to perform complicated work. It is, thus, common to produce the tagged text by applying a preliminarily produced template to the text serving as the target of voice synthesis.
- The conventional common manner, however, requires enormous man-hours for preparation because a large number of templates need to be produced for various types of tag information. A technique is available that automatically produces templates by machine learning. This technique, however, needs to additionally prepare training data and correct answer data for the machine learning, thereby requiring complicated work. It is, thus, desired to establish a new mechanism for efficiently produce the tagged text.
-
FIG. 1 is a block diagram illustrating an overall structure of a voice synthesizer in a first embodiment; -
FIG. 2 is a block diagram illustrating an exemplary structure of a content selection unit; -
FIG. 3 is a block diagram illustrating an exemplary structure of a content generation unit; -
FIG. 4 is a block diagram illustrating an exemplary structure of a content registration unit; -
FIG. 5 is a schematic diagram conceptually illustrating exemplary content registered in a content storage unit; -
FIG. 6 is a schematic diagram to explain a storage form of the content in the content storage unit; -
FIG. 7 is a schematic diagram to explain transition of screens in a user interface (UI) screen displayed on a user terminal; -
FIG. 8 is a schematic diagram illustrating an example of a marker content list screen; -
FIG. 9 is a schematic diagram illustrating an example of a relevant content list screen; -
FIG. 10 is a schematic diagram illustrating an example of a content detail screen; -
FIG. 11 is a schematic diagram illustrating an example of a content generation screen; -
FIG. 12 is a flowchart illustrating an exemplary procedure of the processing performed by the content selection unit; -
FIG. 13 is a flowchart illustrating an exemplary procedure of the processing performed by the content generation unit; -
FIG. 14 is a flowchart illustrating an exemplary procedure of the processing performed by the content registration unit; -
FIG. 15 is a block diagram illustrating an exemplary structure of a content selection unit in a second embodiment; -
FIG. 16 is a schematic diagram to explain transition of screens in the UI screen displayed on the user terminal; -
FIG. 17 is a schematic diagram illustrating an example of a content search screen; -
FIG. 18 is a flowchart illustrating an exemplary procedure of the processing performed by the content selection unit in the second embodiment; and -
FIG. 19 is a block diagram schematically illustrating an exemplary hardware structure of the voice synthesizer. - According to an embodiment, a voice synthesizer includes a content selection unit, a content generation unit, and a content registration unit. The content selection unit determines selected content among a plurality of pieces of content registered in a content storage unit. The content includes tagged text in which tag information for controlling voice synthesis is added to text serving as a target of the voice synthesis. The content generation unit applies the tag information in the tagged text included in the selected content to designated text to generate new content. The content registration unit registers the generated new content in the content storage unit.
- Embodiments will be described in detail below with reference to the accompanying drawings. A voice synthesizer according to each of the embodiments performs voice synthesis based on tagged text in which tag information is added to text serving as the target of the voice synthesis, and particularly has a mechanism that efficiently generates the tagged text. A combination of the tagged text and a voice waveform of a synthesized voice generated based on the tagged text is called a piece of “content” hereinafter. The content may include other information such as identification information about a voice synthesis dictionary used for voice synthesis besides the tagged text and the voice waveform of the synthesized voice. Any known method is adoptable for voice synthesis. Examples of the known method include phoneme based voice synthesis and voice synthesis using a hidden Markov model (HMM). The detailed descriptions thereof are omitted.
-
FIG. 1 is a block diagram illustrating an overall structure of avoice synthesizer 1 according to a first embodiment. Thevoice synthesizer 1 in the first embodiment can be achieved as a server on a network that provides web-based services to auser terminal 2 connected to the network as a client, for example. Theuser terminal 2 is information equipment used by a user, such as a personal computer, a tablet terminal, or a smartphone. Theuser terminal 2 includes various resources constituting a computer system, such as a central processing unit (CPU) and a memory, hardware such as various input devices, and various types of software such as an operating system (OS) and a web browser. - The
voice synthesizer 1 in the embodiment is not necessary structured as a single apparatus. Thevoice synthesizer 1 may be structured as a system including a plurality of linked apparatuses. Thevoice synthesizer 1 may be achieved as a virtual machine that operates in a cloud computing system. - As illustrated in
FIG. 1 , thevoice synthesizer 1 includes acontent selection unit 10, acontent generation unit 20, acontent registration unit 30, acontent storage unit 40, and avoice synthesis dictionary 50. - The
content selection unit 10 causes theuser terminal 2 to display a user interface (UI) screen, receives operation input performed by the user while using the UI screen, and determines selected content based on the user's operation among a plurality of pieces of content registered in thecontent storage unit 40. The selected content is the content selected from among the pieces of content in accordance with the user's operation. - The
content generation unit 20 causes theuser terminal 2 to display the UI screen, receives operation input performed by the user while using the UI screen, and applies the tag information in the tagged text included in the selected content determined by thecontent selection unit 10 to text designated by the user to generate new content. - The
content registration unit 30 registers the new content generated by thecontent generation unit 20 in thecontent storage unit 40. - The
content storage unit 40 stores therein marker content serving as a marker and the content generated by thecontent generation unit 20. The marker content, which emphasizes a specific feature, is preliminarily registered in thecontent storage unit 40. Thecontent registration unit 30 registers the content generated by thecontent generation unit 20 in thecontent storage unit 40 in association with the marker content in accordance with a similarity with the marker content. - The
content storage unit 40 may be provided outside thevoice synthesizer 1. In this case, thecontent registration unit 30 accesses thecontent storage unit 40 outside thevoice synthesizer 1 via a network and registers the content produced by thecontent generation unit 20 in thecontent storage unit 40, for example. Thecontent selection unit 10 accesses thecontent storage unit 40 outside thevoice synthesizer 1 via the network and acquires necessary content from thecontent storage unit 40 in accordance with the user's operation, for example. - The
voice synthesis dictionary 50 is used when thecontent generation unit 20 generates a voice waveform of a synthesized voice based on the tagged text. Thevoice synthesis dictionary 50 is classified based on the features of the respective synthesized voices to be generated, for example. Based on the tag information in the tagged text, an optimum dictionary is selected. Thevoice synthesis dictionary 50 may be provided outside thevoice synthesizer 1. In this case, thecontent generation unit 20 accesses thevoice synthesis dictionary 50 outside thevoice synthesizer 1 via the network and acquires necessary information from thevoice synthesis dictionary 50, for example. - The following describes each unit included in the
voice synthesizer 1 in the embodiment in detail. -
FIG. 2 is a block diagram illustrating an exemplary structure of thecontent selection unit 10. As illustrated inFIG. 2 , thecontent selection unit 10 includes a marker content presentation unit 11, a relevantcontent presentation unit 12, a selected content determination unit 13, and a reproduction unit 14. - The marker content presentation unit 11 presents a list of marker content registered in the
content storage unit 40 to the user. The marker content presentation unit 11 generates a marker content list screen SC1 (refer toFIG. 8 ), which is described later, as the UI screen displayed by theuser terminal 2, and causes theuser terminal 2 to display the marker content list screen SC1, for example. - The relevant
content presentation unit 12 presents, to the user, a list of relevant content associated with the marker content selected by the user from the list of marker content. The relevantcontent presentation unit 12 generates a relevant content list screen SC2 (refer toFIG. 9 ), which is described later, as the UI screen displayed by theuser terminal 2, and causes theuser terminal 2 to display the relevant content list screen SC2, for example. - The selected content determination unit 13 determines the relevant content selected from the list of relevant content to be the selected content. The selected content determination unit 13 determines the relevant content selected by the user from the relevant content list screen SC2 displayed on the
user terminal 2 to be the selected content, for example. - The reproduction unit 14 reproduces the voice waveform of the synthesized voice included in the marker content or the voice waveform of the synthesized voice included in the relevant content in accordance with the user's operation, and causes the
user terminal 2 to output the reproduced voice waveform as a voice from a speaker of theuser terminal 2, for example. The reproduction unit 14 reproduces the voice waveform of the synthesized voice included in the marker content designated by the user from the marker content list screen SC1 displayed on theuser terminal 2 or the voice waveform of the synthesized voice included in the relevant content designated by the user from the relevant content list screen SC2 displayed on theuser terminal 2, and causes theuser terminal 2 to output the reproduced voice waveform as a voice from a speaker of theuser terminal 2, for example. -
FIG. 3 is a block diagram illustrating an exemplary structure of thecontent generation unit 20. As illustrated inFIG. 3 , thecontent generation unit 20 includes a taginformation extraction unit 21, a taggedtext generation unit 22, a taggedtext amendment unit 23, a voice waveform generation unit 24, and areproduction unit 25. - The tag
information extraction unit 21 extracts the tag information from the tagged text included in the selected content determined by the selected content determination unit 13. The tag information includes a start tag located forward in the text to which the tag information is applied and an end tag located backward in the text to which the tag information is applied. In the start tag and the end tag, an element name is described. In the start tag, an attribute value of the element represented by the element name is described. When the element includes a plurality of attributes, the respective attributes and the attribute values of the respective attributes are described in the start tag. Examples of the element included in the tag information include gender (attribute values are male and female), emotion (attribute values are pleasure, sorrow, anger, warmth, and the like), and prosody (attribute values are pitch of a voice, speaking speed, and the like). - For example, the tagged text included in the selected content determined by the selected content determination unit 13 is assumed as follows: <gender=“female”><prosody pitch=“+5%” rate=“−2%”>good morning</prosody></gender>. In this case, the tag
information extraction unit 21 extracts the tag information in the tagged text as follows: <gender=“female”><prosody pitch=“+5%” rate=“−2%”></prosody></gender>. In the example, prosody is an element name that represents the prosody, pitch is the attribute value (attribute value is +5%) that represents the pitch of the voice in the element of prosody, and rate is the attribute value (attribute value is −2%) that represents the utterance speed in the element of prosody. - The tagged
text generation unit 22 applies the tag information extracted by the taginformation extraction unit 21 to text designated by the user to generate the tagged text. For example, the text designated by the user is assumed to be “hello” and the tag information described above is assumed to be extracted by the taginformation extraction unit 21. In this case, the taggedtext generation unit 22 generates the tagged text as follows: <gender=“female”><prosody pitch=“+5%” rate=“−2%”>hello</prosody></gender>. - The tagged
text amendment unit 23 amends the tagged text generated by the taggedtext generation unit 22 based on the user's operation. For example, the taggedtext amendment unit 23 amends the attribute value (in the example, the values +5% or −2%) of the tag information included in the tagged text generated by the taggedtext generation unit 22 based on the user's operation. - The voice waveform generation unit 24 generates the voice waveform of the synthesized voice corresponding to the tagged text generated by the tagged
text generation unit 22 using thevoice synthesis dictionary 50. When the taggedtext amendment unit 23 amends the tagged text generated by the taggedtext generation unit 22, the voice waveform generation unit 24 generates the voice waveform of the synthesized voice corresponding to the amended tagged text. - The
reproduction unit 25 reproduces the voice waveform of the synthesized voice generated by the voice waveform generation unit 24, and causes theuser terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof, for example. -
FIG. 4 is a block diagram illustrating an exemplary structure of thecontent registration unit 30. As illustrated inFIG. 4 , thecontent registration unit 30 includes a similarity calculation unit 31, a classification unit 32, and a usefrequency updating unit 33. - The similarity calculation unit 31 calculates a similarity of the new content generated the
content generation unit 20 with the marker content for registering the new content in thecontent storage unit 40 in association with the marker content. - As described above, the marker content, which emphasizes a specific feature, is preliminarily registered in the
content storage unit 40. For example, the attribute value of the attribute representing emotion (e.g., pleasure, sorrow, anger, or warmth) is assumed to be capable of being set to a range from 0% to 100%, and the attribute value of each of the pitch of the voice and the speaking speed (rate) is assumed to be capable of being set to a range from −10% to +10%. In this case, as exemplarily illustrated inFIG. 5 , marker content M1, M2, . . . , Mk, each of which emphasizes a specific feature, are preliminarily registered in thecontent storage unit 40.FIG. 5 is a schematic diagram conceptually illustrating exemplary content registered in thecontent storage unit 40. - When the
content generation unit 20 generates new content, the similarity calculation unit 31 calculates the similarity of the new content to each of the pieces of marker content preliminarily registered in thecontent storage unit 40. The similarity between two pieces of content ci and cj can be obtained by calculating an inter-content distance D (ci,cj) represented by expressions (1) and (2), for example. -
D(ci,cj)=√{square root over (A)} (1) -
A={pleasure(ci)−pleasure(cj)}2+{sorrow(ci)−sorrow(cj)}2+{anger(ci)−anger(cj)}2+ . . . +{warmth(ci)−warmth(cj)}2+{pitch(ci)−pitch(cj)}2+{rate(ci)−rate(cj)}2 (2) - With a decrease in the inter-content distance D (ci,cj) calculated using expressions (1) and (2), the two pieces of content ci and cj are more similar to each other. In this example, the two pieces of content having the same gender attribute value are used for calculating the distance therebetween. A member relating to the gender attribute value may be added to expression (2) so as to calculate the inter-content distance D (ci,cj) between the different genders.
- The classification unit 32 classifies the content generated by the
content generation unit 20 based on the similarity calculated by the similarity calculation unit 31. The classification is processing that registers the content generated by thecontent generation unit 20 in thecontent storage unit 40 in association with the marker content similar to the content (e.g., the marker content having an inter-content distance equal to or smaller than a certain threshold with respect to the content). When a plurality of pieces of marker content are present that are similar to the content generated by thecontent generation unit 20, thecontent registration unit 30 registers the content in thecontent storage unit 40 in association with each of the pieces of marker content. Every time thecontent generation unit 20 generates new content, the classification unit 32 classifies the generated new content. As a result, the pieces of content associated with the marker content are stored in thecontent storage unit 40 in the order of the similarity for each piece of marker content, for example. -
FIG. 6 is a schematic diagram explaining a storage form of the content in thecontent storage unit 40. As illustrated inFIG. 6 , content C1, C2, . . . , Cm generated by thecontent generation unit 20 are stored in thecontent storage unit 40 in such a manner that the respective pieces of content C1, C2, . . . , Cm are classified into corresponding classes typified by respective pieces of marker content M1, M2, . . . , Mk. The content C1, C2, . . . , Cm are similar to the marker content M1, M2, . . . , Mk, respectively. Each piece of content is associated with information about the use frequency of the content. The use frequency represents the number of times the content is used as the selected content. Every time the content is used as the selected content by thecontent generation unit 20 for generating new content, the value of the use frequency of the content used as the selected content is incremented by one. The use frequency of the content serves as an index that presents whether the content is popular content to the user. - The use
frequency updating unit 33 increments the value of the use frequency of the content used as the selected content for generating new content to update the use frequency when the new content generated by thecontent generation unit 20 is registered. - The following describes specific examples of the UI screen the
voice synthesizer 1 in the embodiment causes theuser terminal 2 to display with reference toFIGS. 7 to 11 . -
FIG. 7 is a schematic diagram to explain transition of screens in the UI screen displayed on theuser terminal 2. Thevoice synthesizer 1 in the embodiment causes theuser terminal 2 to sequentially display, as the UI screen, the marker content list screen SC1, the relevant content list screen SC2, a content detail screen SC3, and a content generation screen SC4 in accordance with the screen transition illustrated inFIG. 7 , for example. -
FIG. 8 is a schematic diagram illustrating an example of the marker content list screen SC1. The marker content list screen SC1 is the UI screen that presents a list of marker content preliminarily registered in thecontent storage unit 40 to the user. As illustrated inFIG. 8 , the marker content list screen SC1 is provided with a “title”column 101, a “gender”column 102, a “parameter”column 103, agender switching button 104, an up-down button 105, a “reproduce”button 106, a “content”button 107, and a “close”button 108. - The “title”
column 101 displays the name of each piece of marker content. The “gender”column 102 displays the gender attribute value (male or female) of each piece of marker content. The “parameter”column 103 displays the attribute such as emotion or prosody and the attribute value (parameter) of each piece of marker content. In the marker content list screen SC1 illustrated inFIG. 8 , the list of marker content is presented for each of the genders of male and female. The gender of marker content to be presented is switched by operating thegender switching button 104.FIG. 8 illustrates the presentation of the list of marker content in relation to the gender of male. - The up-
down button 105 is used for designating any marker content from the list of marker content by moving a cursor (not illustrated) upward and downward. - The “reproduce”
button 106 is used for reproducing the voice waveform of the synthesized voice included in the designated marker content to output the reproduced voice waveform as the voice. When the “reproduce”button 106 is operated in a state where any marker content is designated from the presented list of marker content, the synthesized voice of the designated marker content is output from the speaker of theuser terminal 2. The user can perform trail listening on the synthesized voice of desired marker content using the “reproduce”button 106. - The “content”
button 107 is used for selecting desired marker content from the list of marker content. When the “content”button 107 is operated in a state where any marker content is designated from the presented list of marker content, the UI screen displayed on theuser terminal 2 changes from the marker content list screen SC1 to the relevant content list screen SC2, in which a list of relevant content associated with the designated marker content is presented. - The “close”
button 108 is used for closing the marker content list screen SC1. When the “close”button 108 is operated, the display of the UI screen on theuser terminal 2 ends. -
FIG. 9 is a schematic diagram illustrating an example of the relevant content list screen SC2. The relevant content list screen SC2 is the UI screen that presents a list of relevant content registered in thecontent storage unit 40 in association with the marker content selected by the user using the marker content list screen SC1 to the user. As illustrated inFIG. 9 , the relevant content list screen SC2 is provided with a “title”column 201, a “distance”column 202, a “use frequency”column 203, an up-down button 204, a “reproduce”button 205, a “return”button 206, a “detail”button 207, and a “close”button 208. - The “title”
column 201 displays the name of the marker content selected in the marker content list screen SC1 and the name of each piece of relevant content. The “distance”column 202 displays the inter-content distance D (ci,cj) between the marker content and the relevant content for each relevant content. The “use frequency”column 203 displays the use frequency of the marker content and the use frequency of the relevant content for each piece of relevant content. As illustrated inFIG. 9 , in the relevant content list screen SC2, a plurality of pieces of relevant content associated with the marker content are displayed in a list in ascending order of the values of the inter-content distances D (ci,cj), i.e., in such manner that with an increase in similarity of the relevant content with the marker content, the relevant content is displayed at the upper position in the list. The pieces of relevant content having the same value of the inter-content distance D (ci,cj) are displayed in the list in descending order of the values of the use frequencies of the pieces of relevant content. The order of relevant content in the list is not limited to the example illustrated inFIG. 9 . For example, a plurality of pieces of relevant content may be displayed in a list in descending order of the values of the use frequencies of the respective pieces of relevant content. - The up-
down button 204 is used for designating any relevant content from the list of relevant content by moving a cursor (not illustrated) upward and downward. - The “reproduce”
button 205 is used for reproducing the voice waveform of the synthesized voice included in the designated relevant content to output the reproduced voice waveform as the voice. When the “reproduce”button 205 is operated in a state where any relevant content is designated from the presented list of relevant content, the synthesized voice of the designated relevant content is output from the speaker of theuser terminal 2. The user can perform trail listening on the synthesized voice of desired relevant content using the “reproduce”button 205. - The “return”
button 206 is used for returning the UI screen displayed on theuser terminal 2 from the relevant content list screen SC2 illustrated inFIG. 9 to the marker content list screen SC1 illustrated inFIG. 8 . - The “detail”
button 207 is used for checking the details of desired relevant content. When the “detail”button 207 is operated in a state where any relevant content is designated from the presented list of relevant content, the UI screen displayed on theuser terminal 2 changes from the relevant content list screen SC2 to the content detail screen SC3, in which detailed information about the designated relevant content is displayed. - The “close”
button 208 is used for closing the relevant content list screen SC2. When the “close”button 208 is operated, the display of the UI screen on theuser terminal 2 ends. -
FIG. 10 is a schematic diagram illustrating an example of the content detail screen SC3. The content detail screen SC3 is the UI screen that presents the detailed information about the relevant content selected by the user using the relevant content list screen SC2 to the user. As illustrated inFIG. 10 , the content detail screen SC3 is provided with acontent name column 301, a “used dictionary”column 302, a “text”column 303, a “tag information”column 304, a “reproduce”button 305, a “return”button 306, a “copy”button 307, and a “close”button 308. - The
content name column 301 displays the name of the content. The “used dictionary”column 302 displays the name of thevoice synthesis dictionary 50 that was used in generating the voice waveform of the synthesized voice included in the content. The “text”column 303 displays text (the whole of the text) in the tagged text included in the content. The “tag information”column 304 displays the tagged text corresponding to the designated range in the text displayed in the “text”column 303. The user designates any range in the text displayed in the “text”column 303 and can check the tag information about the designated range in the “tag information”column 304. - The “reproduce”
button 305 is used for reproducing the voice waveform of the synthesized voice corresponding to the tagged text displayed in the “tag information”column 304 to output the reproduced voice waveform as the voice. When the “reproduce”button 305 is operated in a state where the tagged text corresponding to the range designated by the user is displayed in the “tag information”column 304, the synthesized voice corresponding to the tagged text is output from the speaker of theuser terminal 2. The user can perform trail listening on the synthesized voice of the desired range using the “reproduce”button 305. - The “return”
button 306 is used for returning the UI screen displayed on theuser terminal 2 from the content detail screen SC3 illustrated inFIG. 10 to the relevant content list screen SC2 illustrated inFIG. 9 . - The “copy”
button 307 is used for determining the content to be the selected content. When the “copy”button 307 is operated, the UI screen displayed on theuser terminal 2 changes from the content detail screen SC3 to the content generation screen SC4. - The “close”
button 308 is used for closing the content detail screen SC3. When the “close”button 308 is operated, the display of the UI screen on theuser terminal 2 ends. -
FIG. 11 is a schematic diagram illustrating an example of the content generation screen SC4. The content generation screen SC4 is the UI screen used for producing new content by applying the tag information in the selected content to text designated by the user. As illustrated inFIG. 11 , the content generation screen SC4 is provided with a “title”column 401, a “used dictionary”column 402, a “text”column 403, a “tag information”column 404, an “apply”button 405, a “reproduce”button 406, an “edit”button 407, a “return”button 408, a “register”button 409, and a “close”button 410. - The “title”
column 401 displays the name of new content to be generated using the content generation screen SC4. The user can set a desired name to the new content by writing any name into the “title”column 401. The “used dictionary”column 402 displays the name of thevoice synthesis dictionary 50 that was used in generating the voice waveform of the synthesized voice included in the selected content. The user can change the name of thevoice synthesis dictionary 50 used in generating the voice waveform of the synthesized voice included in the new content by changing the name of thevoice synthesis dictionary 50 displayed in the “used dictionary”column 402. The “text”column 403 displays the text serving as the target of voice synthesis. The user can designate the text serving as the target of voice synthesis by writing any text into the “text”column 403. The “tag information”column 404 displays the tagged text generated by applying the tag information in the tagged text included in the selected content to the text displayed in the “text”column 403. - The “apply”
button 405 is used for reproducing the voice waveform of the synthesized voice corresponding to the tagged text displayed in the “tag information”column 404. When the “apply”button 405 is operated in a state where the tagged text is displayed in the “tag information”column 404, the voice waveform of the synthesized voice is generated based on the tagged text displayed in the “tag information”column 404. In the production, thevoice synthesis dictionary 50 displayed in the “used dictionary”column 402 is used. - The “reproduce”
button 406 is used for reproducing the voice waveform of the synthesized voice generated based on the tagged text displayed in the “tag information”column 404 to output the reproduced voice waveform as the voice. When the “reproduce”button 406 is operated after the operation of the “apply”button 405, the synthesized voice generated as a result of the operation of the “apply”button 405 is output from the speaker of theuser terminal 2. The user can perform trail listening on the synthesized voice of content to be newly generated using the “reproduce”button 406. - The “edit”
button 407 is used for amending the tagged text displayed in the “tag information”column 404. When the “edit”button 407 is operated, the tagged text displayed in the “tag information”column 404 can be edited. The user can amend the tagged text of the content to be newly generated by operating the “edit”button 407 and performing amendment operation on the tagged text displayed in the “tag information”column 404. For example, the attribute value (in the example illustrated inFIG. 11 , +5%) in the tag information is amended. - The “return”
button 408 is used for returning the UI screen displayed on theuser terminal 2 from the content generation screen SC4 illustrated inFIG. 11 to the content detail screen SC3 illustrated inFIG. 10 . - The “register”
button 409 is used for registering the generated new content in thecontent storage unit 40. When the “register”button 409 is operated, a combination of the tagged text displayed in the “tag information”column 404 and the voice waveform of the synthesized voice generated based on the tagged text is registered in thecontent storage unit 40 as new content. - The “close”
button 410 is used for closing the content generation screen SC4. When the “close”button 410 is operated, the display of the UI screen on theuser terminal 2 ends. - The following describes exemplary operation of the
voice synthesizer 1 that generates content and registers the produced content while causing theuser terminal 2 to display the UI screens exemplarily illustrated inFIGS. 7 to 11 . - The processing performed by the
content selection unit 10 is described with reference toFIG. 12 .FIG. 12 is a flowchart illustrating an exemplary procedure of the processing performed by thecontent selection unit 10. - At the start of the processing illustrated in the flowchart in
FIG. 12 , the marker content presentation unit 11 causes theuser terminal 2 to display the marker content list screen SC1 exemplarily illustrated inFIG. 8 (step S101). When thegender switching button 104 in the marker content list screen SC1 is operated after theuser terminal 2 displays the marker content list screen SC1, the gender of marker content to be presented as a list is switched, the illustration of which is omitted in the flowchart inFIG. 12 . When the “close”button 108 is operated at any timing, the processing ends. - It is determined whether the “reproduce”
button 106 is operated in a state where any of the marker content in the list displayed on the marker content list screen SC1 is designated (step S102). If the “reproduce”button 106 is operated (Yes at step S102), the reproduction unit 14 reproduces the voice waveform of the synthesized voice included in the designated marker content, and causes theuser terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof (step S103), and then the processing returns to step S102. - If the “reproduce”
button 106 is not operated (No at step S102), it is determined whether the “content”button 107 is operated in a state where any of the marker content displayed in the list is designated (step S104). If the “content”button 107 is not operated (No at step S104), the processing returns to step S102. If the “content”button 107 is operated (Yes at step S104), the relevantcontent presentation unit 12 causes theuser terminal 2 to display the relevant content list screen SC2 exemplarily illustrated inFIG. 9 (step S105). - When the “return”
button 206 is operated at any timing after theuser terminal 2 displays the relevant content list screen SC2, the processing returns to step S101, at which theuser terminal 2 displays the marker content list screen SC1 again, the illustration of which is omitted in the flowchart inFIG. 12 . When the “close”button 208 is operated at any timing, the processing ends. - It is determined whether the “reproduce”
button 205 is operated in a state where any of the relevant content in the list displayed on the relevant content list screen SC2 is designated (step S106). If the “reproduce”button 205 is operated (Yes at step S106), the reproduction unit 14 reproduces the voice waveform of the synthesized voice included in the designated relevant content, and causes theuser terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof (step S107), and then the processing returns to step S106. - If the “reproduce”
button 205 is not operated (No at step S106), it is determined whether the “detail”button 207 is operated in a state where any of the relevant content displayed in the list is designated (step S108). If the “detail”button 207 is not operated (No at step S108), the processing returns to step S106. If the “detail”button 207 is operated (Yes at step S108), the selected content determination unit 13 causes theuser terminal 2 to display the content detail screen SC3 exemplarily illustrated inFIG. 10 (step S109). - When the “return”
button 306 is operated at any timing after theuser terminal 2 displays the content detail screen SC3, the processing returns to step S105, at which theuser terminal 2 displays the relevant content list screen SC2 again, the illustration of which is omitted in the flowchart inFIG. 12 . When the “close”button 308 is operated at any timing, the processing ends. - It is determined whether the “reproduce”
button 305 is operated in a state where the tagged text is displayed in the “tag information”column 304 in the content detail screen SC3 (step S110). If the “reproduce”button 305 is operated (Yes at step S110), the reproduction unit 14 reproduces the voice waveform of the synthesized voice corresponding to the tagged text displayed in the “tag information”column 304, and causes theuser terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof (step S111), and then the processing returns to step S110. - If the “reproduce”
button 305 is not operated (No at step S110), it is determined whether the “copy”button 307 is operated in a state where the tagged text is displayed in the “tag information” column 304 (step S112). If the “copy”button 307 is not operated (No at step S112), the processing returns to step S110. If the “copy”button 307 is operated (Yes at step S112), the selected content determination unit 13 determines the content containing detail information displayed by the content detail screen SC3 to be the selected content (step S113). The processing is, then, passed to thecontent generation unit 20. The processing performed by thecontent selection unit 10, thus, ends. - The following describes the processing performed by the
content generation unit 20 with reference toFIG. 13 .FIG. 13 is a flowchart illustrating an exemplary procedure of the processing performed by thecontent generation unit 20. - At the start of the processing illustrated in the flowchart in
FIG. 13 , the taginformation extraction unit 21 causes theuser terminal 2 to display the content generation screen SC4 exemplarily illustrated inFIG. 11 (step S201). The user writes the text serving as the target of voice synthesis into the “text”column 403 in the content generation screen SC4. At the time, the taginformation extraction unit 21 extracts the tag information from the tagged text included in the selected content. The taggedtext generation unit 22 applies the tag information extracted by the taginformation extraction unit 21 to the text written in the “text”column 403 to generate the tagged text. The tagged text generated by the taggedtext generation unit 22 is displayed in the “tag information”column 404 in the content generation screen SC4. - When the “return”
button 408 is operated at any timing after theuser terminal 2 displays the content generation screen SC4, the processing returns to step S109 inFIG. 12 , at which theuser terminal 2 displays the content detail screen SC3 again, the illustration of which is omitted in the flowchart inFIG. 13 . When the “close”button 410 is operated at any timing, the processing ends. - It is determined whether the “edit”
button 407 is operated in a state where the tagged text is displayed in the “tag information” column 404 (step S202). If the “edit”button 407 is operated (Yes at step S202), the taggedtext amendment unit 23 receives the user's operation to amend the tagged text, and amends the tagged text displayed in the “tag information” column 404 (step S203), and then the processing returns to step S202. - If the “edit”
button 407 is not operated (No at step S202), it is determined whether the “apply”button 405 is operated in a state where the tagged text is displayed in the “tag information” column 404 (step S204). If the “apply”button 405 is not operated (No at step S204), the processing returns to step S202. If the “apply”button 405 is operated (Yes at step S204), the voice waveform generation unit 24 generates the voice waveform of the synthesized voice using thevoice synthesis dictionary 50 displayed in the “used dictionary”column 402 based on the tagged text displayed in the “tag information” column 404 (step S205). - It is determined whether the “reproduce”
button 406 is operated (step S206). If the “reproduce”button 406 is operated (Yes at step S206), thereproduction unit 25 reproduces the voice waveform of the synthesized voice generated at step S205 and causes theuser terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof (step S207), and then the processing returns to step S206. - If the “reproduce”
button 406 is not operated (No at step S206), it is determined whether the “register”button 409 is operated (step S208). If the “register”button 409 is not operated (No at step S208), the processing returns to step S206. If the “register”button 409 is operated (Yes at step S208), the processing is passed to thecontent registration unit 30. The processing performed by thecontent generation unit 20, thus, ends. - The following describes the processing performed by the
content registration unit 30 with reference toFIG. 14 .FIG. 14 is a flowchart illustrating an exemplary procedure of the processing performed by thecontent registration unit 30. - At the start of the processing illustrated in the flowchart in
FIG. 14 , the similarity calculation unit 31 calculates the inter-content distance D (ci,cj) between the new content generated by thecontent generation unit 20 and the marker content for each marker content registered in the content storage unit 40 (step S301). - The classification unit 32 classifies the new content generated by the
content generation unit 20 based on the inter-content distance D (ci,cj) calculated at step S301 in such a manner that the classification unit 32 registers the new content in thecontent storage unit 40 in association with the marker content similar to the content (step S302). The new content registered in thecontent storage unit 40 serves as a candidate of the selected content used for generating other content in the future. - The use
frequency updating unit 33 updates the use frequency of the content used as the selected content when thecontent generation unit 20 generates the new content (step S303). The processing performed by thecontent registration unit 30, thus, ends. - As described above in detail based on the specific examples, the
voice synthesizer 1 in the embodiment determines the selected content used for generating new content out of the pieces of content registered in thecontent storage unit 40 in accordance with operation performed by the user while using the UI screen. Thevoice synthesizer 1 applies the tag information in the tagged text included in the selected content to the text designated by the user to generate new content. Thevoice synthesizer 1 registers the generated new content in thecontent storage unit 40 as a candidate of the selected content. Thevoice synthesizer 1 in the embodiment, thus, does not need to preliminarily prepare a number of templates for generating the tagged text and prepare training data and correct answer data for automatically generating the templates. Thevoice synthesizer 1 can generate the tagged text from any text using the content generated in the past, thereby making it possible to efficiently generate the tagged text. - The
voice synthesizer 1 in the embodiment allows the user to generate the tagged text by selecting the tag information to be applied to the text or to amend the tagged text if necessary while performing trial listening on the synthesized voice of the content generated in the past or the synthesized voice generated by applying the desired tag information to the text. As a result, thevoice synthesizer 1 can efficiently obtain the synthesized voice the user desires. - The following describes a second embodiment. A voice synthesizer in the second embodiment includes a content selection unit having a different structure from that in the voice synthesizer in the first embodiment. In the following, the voice synthesizer in the second embodiment is described as “a
voice synthesizer 1′” and the content selection unit specific to thevoice synthesizer 1′ is described as acontent selection unit 60, so that thevoice synthesizer 1′ and thecontent selection unit 60 are distinguished from those in the first embodiment. The structure other than the above is the same as that of the first embodiment. The duplicate descriptions are thus appropriately omitted. The following describes thecontent selection unit 60. -
FIG. 15 is a block diagram illustrating an exemplary structure of thecontent selection unit 60. As illustrated inFIG. 15 , thecontent selection unit 60 includes a content search unit 61, a retrieved content presentation unit 62, a selected content determination unit 63, and a reproduction unit 64. - The content search unit 61 retrieves the content including the tagged text matching an input keyword from among the pieces of content registered in the
content storage unit 40. For example, the content search unit 61 causes theuser terminal 2 to display a content search screen SC5 (refer toFIG. 17 ), which is described later, as the UI screen displayed on theuser terminal 2, and retrieves the content including the tagged text matching a keyword input by the user using the content search screen SC5 from among the pieces of content registered in thecontent storage unit 40. - The retrieved content presentation unit 62 presents a list of retrieved content retrieved by the content search unit 61 to the user. The retrieved content presentation unit 62 causes the
user terminal 2 to display a list of retrieved content retrieved by the content search unit 61 on the content search screen SC5 displayed as the UI screen, for example. - The selected content determination unit 63 determines the retrieved content selected from the list of retrieved content to be the selected content. The selected content determination unit 63 determines the retrieved content selected by the user from the list of retrieved content displayed on the content search screen SC5 to be the selected content, for example.
- The reproduction unit 64 reproduces the voice waveform of the synthesized voice included in the retrieved content, and causes the
user terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof, for example. The reproduction unit 64 reproduces the voice waveform of the synthesized voice included in the retrieved content designated by the user from the list of retrieved content displayed on the content search screen SC5, and causes theuser terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof, for example. -
FIG. 16 is a schematic diagram to explain transition of screens in the UI screen that thevoice synthesizer 1′ in the second embodiment causes theuser terminal 2 to display. Thevoice synthesizer 1′ in the embodiment causes theuser terminal 2 to sequentially display, as the UI screen, the content search screen SC5, the content detail screen SC3, and the content generation screen SC4 in accordance with the screen transition illustrated inFIG. 16 , for example. -
FIG. 17 is a schematic diagram illustrating an example of the content search screen SC5. The content search screen SC5 is the UI screen that receives input of a keyword for retrieving the content and presents the list of retrieved content serving as the result of the search to the user. As illustrated inFIG. 17 , the content search screen SC5 is provided with a “keyword”input column 501, a “title”column 502, a “use frequency”column 503, a “search”button 504, an up-down button 505, a “reproduce”button 506, a “detail”button 507, and a “close”button 508. - The “keyword”
input column 501 serves as an area in which a keyword used for the search is input. The user can input any text such as text serving as the target of voice synthesis in the “keyword”input column 501 as a keyword. The “title”column 502 displays the name of each piece of retrieved content serving as the result of the search. The “use frequency”column 503 displays the use frequency of each piece of retrieved content serving as the result of the search. - The “search”
button 504 is used for the search using the keyword input in the “keyword”input column 501. When the “search”button 504 is operated in a state where the keyword is input in the “keyword”input column 501, the retrieved content including the tagged text matching the keyword is retrieved from thecontent storage unit 40, and the name of the retrieved content is displayed in the “title”column 502 while the use frequency of the retrieved content is displayed in the “use frequency”column 503. - The up-
down button 505 is used for designating any retrieved content from the list of retrieved content by moving a cursor (not illustrated) upward and downward. - The “reproduce”
button 506 is used for reproducing the voice waveform of the synthesized voice included in the designated retrieved content to output the reproduced voice waveform as the voice. When the “reproduce”button 506 is operated in a state where any retrieved content is designated from the presented list of retrieved content, the synthesized voice of the designated retrieved content is output from the speaker of theuser terminal 2. The user can perform trail listening on the synthesized voice of desired retrieved content using the “reproduce”button 506. - The “detail”
button 507 is used for checking the details of desired retrieved content. When the “detail”button 507 is operated in a state where any retrieved content is designated from the presented list of retrieved content, the UI screen displayed on theuser terminal 2 changes from the content search screen SC5 to the content detail screen SC3 (refer toFIG. 10 ), in which detailed information about the designated retrieved content is displayed. - The “close”
button 508 is used for closing the content search screen SC5. When the “close”button 508 is operated, the display of the UI screen on theuser terminal 2 ends. - The following describes the processing performed by the
content selection unit 60 that determines the selected content while causing theuser terminal 2 to display the content search screen SC5 exemplarily illustrated inFIG. 17 and the content detail screen SC3 exemplarily illustrated inFIG. 10 with reference toFIG. 18 .FIG. 18 is a flowchart illustrating an exemplary procedure of the processing performed by thecontent selection unit 60. - At the start of the processing illustrated in the flowchart in
FIG. 18 , the content search unit 61 causes theuser terminal 2 to display the content search screen SC5 exemplarily illustrated inFIG. 17 (step S401). When the “close”button 508 is operated at any timing after theuser terminal 2 displays the content search screen SC5, the processing ends, the illustration of which is omitted in the flowchart inFIG. 18 . - It is determined whether the “search”
button 504 is operated in a state where the keyword is input in the “keyword”input column 501 in the content search screen SC5 (step S402). If the “search”button 504 is not operated (No at step S402), the processing returns to step S402, at which the determination is repeated. If the “search”button 504 is operated (Yes at step S402), the content search unit 61 retrieves the retrieved content including the tagged text matching the keyword input in the “keyword”input column 501 from among the pieces of content registered in the content storage unit 40 (step S403). The content search unit 61 causes the list of searched content obtained as the search result to be displayed on the content search screen SC5 (step S404). - It is determined whether the “reproduce”
button 506 is operated in a state where any piece of the retrieved content in the list displayed on the content search screen SC5 is designated (step S405). If the “reproduce”button 506 is operated (Yes at step S405), the reproduction unit 64 reproduces the voice waveform of the synthesized voice included in the designated retrieved content, and causes theuser terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof (step S406), and then the processing returns to step S405. - If the “reproduce”
button 506 is not operated (No at step S405), it is determined whether the “detail”button 507 is operated in a state where any piece of the retrieved content displayed in the list is designated (step S407). If the “detail”button 507 is not operated (No at step S407), the processing returns to step S405. If the “detail”button 507 is operated (Yes at step S407), the selected content determination unit 63 causes theuser terminal 2 to display the content detail screen SC3 exemplarily illustrated inFIG. 10 (step S408). - When the “return”
button 306 is operated at any timing after theuser terminal 2 displays the content detail screen SC3, the processing returns to step S401, at which theuser terminal 2 displays the content search screen SC5 again, the illustration of which is omitted in the flowchart inFIG. 18 . When the “close”button 308 is operated at any timing, the processing ends - It is determined whether the “reproduce”
button 305 is operated in a state where the tagged text is displayed in the “tag information”column 304 in the content detail screen SC3 (step S409). If the “reproduce”button 305 is operated (Yes at step S409), the reproduction unit 64 reproduces the voice waveform of the synthesized voice corresponding to the tagged text displayed in the “tag information”column 304, and causes theuser terminal 2 to output the reproduced voice waveform as the voice from the speaker thereof (step S410), and then the processing returns to step S409. - If the “reproduce”
button 305 is not operated (No at step S409), it is determined whether the “copy”button 307 is operated in a state where the tagged text is displayed in the “tag information” column 304 (step S411). If the “copy”button 307 is not operated (No at step S411), the processing returns to step S409. If the “copy”button 307 is operated (Yes at step S411), the selected content determination unit 63 determines the retrieved content containing detail information displayed by the content detail screen SC3 to be the selected content (step S412). The processing is, then, passed to thecontent generation unit 20. The processing performed by thecontent selection unit 60, thus, ends. - As described above, the
voice synthesizer 1′ in the embodiment retrieves the content including the tagged text matching the keyword from among the pieces of content registered in thecontent storage unit 40 in accordance with operation performed by the user using the UI screen. Thevoice synthesizer 1′ determines the selected content used for generating new content from among the retrieved content. Thevoice synthesizer 1′ applies the tag information in the tagged text included in the determined selected content to the text designated by the user to generate new content. Thevoice synthesizer 1′ registers the generated new content in thecontent storage unit 40 as a candidate of the selected content. Thevoice synthesizer 1′ in the embodiment can generate the tagged text from any text using content generated in the past in the same manner as thevoice synthesizer 1 in the first embodiment. Thevoice synthesizer 1′ can, thus, efficiently generate the tagged text. Furthermore, thevoice synthesizer 1′ in the embodiment can narrow down the candidates of the selected content using the keyword, thereby making it possible to more efficiently generate the tagged text. - The respective functional components in the
voice synthesizer 1 in the first embodiment can be achieved by a program (software) executed using a general purpose computer system as basic hardware, for example. -
FIG. 19 is a block diagram schematically illustrating an exemplary hardware structure of the major part of thevoice synthesizer 1. As illustrated inFIG. 19 , the major part of thevoice synthesizer 1 is structured as a general purpose computer system that includes a processor 71 such as a CPU, a main storage unit 72 such as a random access memory (RAM), anauxiliary storage unit 73 using various storage devices, acommunication interface 74, and a bus 75 that connects each one to another. Theauxiliary storage unit 73 may be connected to the respective units with a local area network (LAN) in a wired or wireless manner, for example. - The respective functional components of the
voice synthesizer 1 are achieved by the processor 71 executing a program stored in theauxiliary storage unit 73 using the main storage unit 72, for example. The program is recorded and provided as a computer program product on a computer-readable recording medium such as a compact disc read only memory (CD-ROM), a flexible disk (FD), a compact disc recordable (CD-R), and a digital versatile disc (DVD), as an installable or executable file, for example. - The program may be stored in another computer connected to a network such as the Internet and provided by being downloaded via the network. The program may be provided or distributed via a network such as the Internet. The program may be embedded and provided in a ROM (the auxiliary storage unit 73) in the computer.
- The program has a module structure including the functional components (the
content selection unit 10, thecontent generation unit 20, and the content registration unit 30) of thevoice synthesizer 1. In actual hardware, the processor 71 reads the program from the recording medium and executes the program. Once the program is executed, the respective components are loaded into the main storage unit 72, so that the respective components are formed in the main storage unit 72. Part of or the whole of the respective components of thevoice synthesizer 1 can be achieved using dedicated hardware such as an application specific integrated circuit (ASIC) and a field-programmable gate array (FPGA). - While certain embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the inventions. Indeed, the novel embodiments described herein may be embodied in a variety of other forms; furthermore, various omissions, substitutions and changes in the form of the embodiments described herein may be made without departing from the spirit of the inventions. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the inventions.
Claims (12)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2014-221770 | 2014-10-30 | ||
JP2014221770A JP6415929B2 (en) | 2014-10-30 | 2014-10-30 | Speech synthesis apparatus, speech synthesis method and program |
PCT/JP2015/075638 WO2016067766A1 (en) | 2014-10-30 | 2015-09-09 | Voice synthesis device, voice synthesis method and program |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2015/075638 Continuation WO2016067766A1 (en) | 2014-10-30 | 2015-09-09 | Voice synthesis device, voice synthesis method and program |
Publications (2)
Publication Number | Publication Date |
---|---|
US20170004821A1 true US20170004821A1 (en) | 2017-01-05 |
US10217454B2 US10217454B2 (en) | 2019-02-26 |
Family
ID=55857112
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/266,065 Active 2035-10-21 US10217454B2 (en) | 2014-10-30 | 2016-09-15 | Voice synthesizer, voice synthesis method, and computer program product |
Country Status (4)
Country | Link |
---|---|
US (1) | US10217454B2 (en) |
JP (1) | JP6415929B2 (en) |
CN (1) | CN106688035B (en) |
WO (1) | WO2016067766A1 (en) |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108062955A (en) * | 2017-12-12 | 2018-05-22 | 深圳证券信息有限公司 | A kind of intelligence report-generating method, system and equipment |
US10276149B1 (en) * | 2016-12-21 | 2019-04-30 | Amazon Technologies, Inc. | Dynamic text-to-speech output |
US20190349640A1 (en) * | 2017-01-02 | 2019-11-14 | Samsung Electronics Co., Ltd. | Method and device for providing information on content |
US20200265335A1 (en) * | 2019-02-15 | 2020-08-20 | Wipro Limited | Method and a system for context based clustering of object |
US20200380976A1 (en) * | 2018-01-26 | 2020-12-03 | Samsung Electronics Co., Ltd. | Electronic apparatus and control method thereof |
CN114863907A (en) * | 2022-07-07 | 2022-08-05 | 北京中电慧声科技有限公司 | Marking method and device for text-to-speech processing |
CN114863906A (en) * | 2022-07-07 | 2022-08-05 | 北京中电慧声科技有限公司 | Method and device for marking alias of text-to-speech processing |
CN116092477A (en) * | 2023-03-30 | 2023-05-09 | 北京中电慧声科技有限公司 | Voice synthesis system mark memory library-based audio generation method and device |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6791669B2 (en) * | 2016-07-12 | 2020-11-25 | Supership株式会社 | Information processing equipment and programs |
US10580457B2 (en) * | 2017-06-13 | 2020-03-03 | 3Play Media, Inc. | Efficient audio description systems and methods |
CN108053696A (en) * | 2018-01-04 | 2018-05-18 | 广州阿里巴巴文学信息技术有限公司 | A kind of method, apparatus and terminal device that sound broadcasting is carried out according to reading content |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020052733A1 (en) * | 2000-09-18 | 2002-05-02 | Ryo Michizuki | Apparatus and method for speech synthesis |
US20050027532A1 (en) * | 2000-03-31 | 2005-02-03 | Canon Kabushiki Kaisha | Speech synthesis apparatus and method, and storage medium |
US6928408B1 (en) * | 1999-12-03 | 2005-08-09 | Fujitsu Limited | Speech data compression/expansion apparatus and method |
US20050197839A1 (en) * | 2004-03-04 | 2005-09-08 | Samsung Electronics Co., Ltd. | Apparatus, medium, and method for generating record sentence for corpus and apparatus, medium, and method for building corpus using the same |
US20060287861A1 (en) * | 2005-06-21 | 2006-12-21 | International Business Machines Corporation | Back-end database reorganization for application-specific concatenative text-to-speech systems |
US7630898B1 (en) * | 2005-09-27 | 2009-12-08 | At&T Intellectual Property Ii, L.P. | System and method for preparing a pronunciation dictionary for a text-to-speech voice |
US8086456B2 (en) * | 1999-04-30 | 2011-12-27 | At&T Intellectual Property Ii, L.P. | Methods and apparatus for rapid acoustic unit selection from a large speech corpus |
US20120072223A1 (en) * | 2002-06-05 | 2012-03-22 | At&T Intellectual Property Ii, L.P. | System and method for configuring voice synthesis |
US20150325248A1 (en) * | 2014-05-12 | 2015-11-12 | At&T Intellectual Property I, L.P. | System and method for prosodically modified unit selection databases |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH09160583A (en) * | 1995-12-11 | 1997-06-20 | Canon Inc | Method and device for processing voice information |
CN1156751C (en) * | 2001-02-02 | 2004-07-07 | 国际商业机器公司 | Method and system for automatic generating speech XML file |
JP2002268664A (en) * | 2001-03-09 | 2002-09-20 | Ricoh Co Ltd | Voice converter and program |
JP2003295882A (en) | 2002-04-02 | 2003-10-15 | Canon Inc | Text structure for speech synthesis, speech synthesizing method, speech synthesizer and computer program therefor |
JP2004325692A (en) * | 2003-04-23 | 2004-11-18 | Sharp Corp | Text speech synthesis apparatus, text processing apparatus and program, and recording medium for program thereof |
JP4885569B2 (en) | 2006-03-03 | 2012-02-29 | 日本放送協会 | Style sheet generating apparatus and style sheet generating program |
JP2009186498A (en) * | 2008-02-01 | 2009-08-20 | Brother Ind Ltd | Speech synthesis device and speech synthesis program |
DE112010005226T5 (en) * | 2010-02-05 | 2012-11-08 | Mitsubishi Electric Corporation | Recognition dictionary generating device and speech recognition device |
CN102385858B (en) * | 2010-08-31 | 2013-06-05 | 国际商业机器公司 | Emotional voice synthesis method and system |
JP5742482B2 (en) * | 2011-06-03 | 2015-07-01 | ヤマハ株式会社 | Sequence data editing device and program |
CN103403797A (en) * | 2011-08-01 | 2013-11-20 | 松下电器产业株式会社 | Speech synthesis device and speech synthesis method |
JP5596649B2 (en) | 2011-09-26 | 2014-09-24 | 株式会社東芝 | Document markup support apparatus, method, and program |
-
2014
- 2014-10-30 JP JP2014221770A patent/JP6415929B2/en active Active
-
2015
- 2015-09-09 WO PCT/JP2015/075638 patent/WO2016067766A1/en active Application Filing
- 2015-09-09 CN CN201580046128.4A patent/CN106688035B/en not_active Expired - Fee Related
-
2016
- 2016-09-15 US US15/266,065 patent/US10217454B2/en active Active
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8086456B2 (en) * | 1999-04-30 | 2011-12-27 | At&T Intellectual Property Ii, L.P. | Methods and apparatus for rapid acoustic unit selection from a large speech corpus |
US6928408B1 (en) * | 1999-12-03 | 2005-08-09 | Fujitsu Limited | Speech data compression/expansion apparatus and method |
US20050027532A1 (en) * | 2000-03-31 | 2005-02-03 | Canon Kabushiki Kaisha | Speech synthesis apparatus and method, and storage medium |
US20020052733A1 (en) * | 2000-09-18 | 2002-05-02 | Ryo Michizuki | Apparatus and method for speech synthesis |
US20120072223A1 (en) * | 2002-06-05 | 2012-03-22 | At&T Intellectual Property Ii, L.P. | System and method for configuring voice synthesis |
US20050197839A1 (en) * | 2004-03-04 | 2005-09-08 | Samsung Electronics Co., Ltd. | Apparatus, medium, and method for generating record sentence for corpus and apparatus, medium, and method for building corpus using the same |
US20060287861A1 (en) * | 2005-06-21 | 2006-12-21 | International Business Machines Corporation | Back-end database reorganization for application-specific concatenative text-to-speech systems |
US7630898B1 (en) * | 2005-09-27 | 2009-12-08 | At&T Intellectual Property Ii, L.P. | System and method for preparing a pronunciation dictionary for a text-to-speech voice |
US20150325248A1 (en) * | 2014-05-12 | 2015-11-12 | At&T Intellectual Property I, L.P. | System and method for prosodically modified unit selection databases |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10276149B1 (en) * | 2016-12-21 | 2019-04-30 | Amazon Technologies, Inc. | Dynamic text-to-speech output |
US20190349640A1 (en) * | 2017-01-02 | 2019-11-14 | Samsung Electronics Co., Ltd. | Method and device for providing information on content |
US10911833B2 (en) * | 2017-01-02 | 2021-02-02 | Samsung Electronics Co., Ltd. | Method and device for providing information on content |
CN108062955A (en) * | 2017-12-12 | 2018-05-22 | 深圳证券信息有限公司 | A kind of intelligence report-generating method, system and equipment |
US20200380976A1 (en) * | 2018-01-26 | 2020-12-03 | Samsung Electronics Co., Ltd. | Electronic apparatus and control method thereof |
US11721333B2 (en) * | 2018-01-26 | 2023-08-08 | Samsung Electronics Co., Ltd. | Electronic apparatus and control method thereof |
US20200265335A1 (en) * | 2019-02-15 | 2020-08-20 | Wipro Limited | Method and a system for context based clustering of object |
US11537938B2 (en) * | 2019-02-15 | 2022-12-27 | Wipro Limited | Method and a system for context based clustering of object |
CN114863907A (en) * | 2022-07-07 | 2022-08-05 | 北京中电慧声科技有限公司 | Marking method and device for text-to-speech processing |
CN114863906A (en) * | 2022-07-07 | 2022-08-05 | 北京中电慧声科技有限公司 | Method and device for marking alias of text-to-speech processing |
CN116092477A (en) * | 2023-03-30 | 2023-05-09 | 北京中电慧声科技有限公司 | Voice synthesis system mark memory library-based audio generation method and device |
Also Published As
Publication number | Publication date |
---|---|
US10217454B2 (en) | 2019-02-26 |
JP2016090664A (en) | 2016-05-23 |
CN106688035B (en) | 2020-11-10 |
WO2016067766A1 (en) | 2016-05-06 |
JP6415929B2 (en) | 2018-10-31 |
CN106688035A (en) | 2017-05-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10217454B2 (en) | Voice synthesizer, voice synthesis method, and computer program product | |
JP6799574B2 (en) | Method and device for determining satisfaction with voice dialogue | |
US9330657B2 (en) | Text-to-speech for digital literature | |
US20200035209A1 (en) | Automatic song generation | |
US10535335B2 (en) | Voice synthesizing device, voice synthesizing method, and computer program product | |
US8972265B1 (en) | Multiple voices in audio content | |
JP6361351B2 (en) | Method, program and computing system for ranking spoken words | |
JPWO2008016102A1 (en) | Similarity calculation device and information retrieval device | |
US10685644B2 (en) | Method and system for text-to-speech synthesis | |
CN115082602B (en) | Method for generating digital person, training method, training device, training equipment and training medium for model | |
JP6064629B2 (en) | Voice input / output database search method, program, and apparatus | |
JP2014038282A (en) | Prosody editing apparatus, prosody editing method and program | |
KR102177768B1 (en) | System for providing cloud based cusomized multimedia production service using voice mix | |
US11176943B2 (en) | Voice recognition device, voice recognition method, and computer program product | |
JP2018169434A (en) | Voice synthesizer, voice synthesis method, voice synthesis system and computer program for voice synthesis | |
CN109977294B (en) | Information/query processing device, query processing/text query method, and storage medium | |
US20200013409A1 (en) | Speaker retrieval device, speaker retrieval method, and computer program product | |
JP7243106B2 (en) | Correction candidate presentation method, correction candidate presentation program, and information processing apparatus | |
JP2006236037A (en) | Voice interaction content creation method, device, program and recording medium | |
KR20230015489A (en) | Apparatus for managing minutes and method thereof | |
JP2014232145A (en) | Pause application model selection apparatus, pause application device, methods thereof, and program | |
JP7057229B2 (en) | Evaluation device, evaluation method and evaluation program | |
JP6309852B2 (en) | Enhanced position prediction apparatus, enhanced position prediction method, and program | |
He et al. | LLMs Meet Multimodal Generation and Editing: A Survey | |
Nanda et al. | Story Visualization: Generation of Scenes sequentially for a given fable using NLP and Image Processing |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: KABUSHIKI KAISHA TOSHIBA, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HIRANO, KAORU;SUZUKI, MASARU;MIZUTANI, HIROYUKI;SIGNING DATES FROM 20161011 TO 20161115;REEL/FRAME:040357/0861 Owner name: TOSHIBA SOLUTIONS CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HIRANO, KAORU;SUZUKI, MASARU;MIZUTANI, HIROYUKI;SIGNING DATES FROM 20161011 TO 20161115;REEL/FRAME:040357/0861 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: TOSHIBA DIGITAL SOLUTIONS CORPORATION, JAPAN Free format text: CHANGE OF NAME;ASSIGNOR:TOSHIBA SOLUTIONS CORPORATION;REEL/FRAME:048547/0215 Effective date: 20190226 |
|
AS | Assignment |
Owner name: TOSHIBA DIGITAL SOLUTIONS CORPORATION, JAPAN Free format text: CHANGE OF NAME;ASSIGNOR:TOSHIBA SOLUTIONS CORPORATION;REEL/FRAME:055185/0728 Effective date: 20170704 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |