TWM621764U - A system for customized speech - Google Patents

Abstract

The invention disclose a system for generating customized speech, which let users enjoying personalized speech for data likes news or story. The data might be downloaded from server 2. Importing data to the server 1 via the app, then the data would be translated to the speech by text-to-speech services. Adjusting the speech conform parameters likes tone or timbre, users finally acquire customized speech for data.

Description

Customized voice service system

This new model relates to a system for voice service, especially a system for customized voice service, which obtains text through an application program and transmits it to a server for text recognition to generate the first voice data of a simulated character. The user chooses to customize the text The voice parameter playback is the second voice data of the simulated character, and the customized voice service is provided for the text for broadcast and dubbing.

The current technology has the technology of converting text into speech, such as the speech reading function provided by Google Translate.

The more widely used languages in Google Translate have the function of "reading aloud". For multi-centric languages, the accent used for reading aloud depends on the region, for example: English: America, Asia Pacific (except Hong Kong, Malaysia, Singapore) and West Asia are mostly used American English (female) and British English (female) for the rest, but Australia, New Zealand and Norfolk Island use a special Oceanian accent (female) and India uses an Indian accent (female). French: With the exception of Canada, which uses a Quebec accent (female), most other regions use a standard continental accent (female). Spanish: American Spanish (female voice) is used in the Americas (except the United States), and Castilian Spanish (male voice) is used in most other regions. Standard Chinese: Traditional Chinese (female voice) is used, and Mandarin Chinese (female voice) is used for simplified Chinese. Portuguese: Except for Portugal, which uses the national accent (female voice), most of the rest of the region uses the São Paulo accent (female voice).

Among them, the technology of converting text to speech is called Text To Speech, or TTS for short. The goal of Text-to-speech (TTS) is to synthesize a given text (text) into a natural speech (speech) for humans.

There are traditionally two approaches to implement TTS, concatenative speech synthesis and parametric speech synthesis.

Concatenative speech synthesis: Concatenative synthesis combines pre-recorded sentences, text and speech into a database. When the text to generate voice comes in, select the appropriate example in the database to splicing it into a new voice.

However, how to choose sentences to synthesize speech is a big problem. If a long sentence is used, the synthesized voice may be more natural, but it will require many examples to be stored in the database, so there will be the problem of memory space and the manpower problem of how to obtain so many examples, so the flexibility is low; if using Short sentences have higher flexibility, but the labeling of text and the selection of sounds will become more difficult, and the synthesized sounds will be less natural.

Parametric speech synthesis: Parametric synthesis uses a trained model to synthesize sound waveforms. The advantage is that it does not require a huge database like splicing synthesis, which is more flexible than splicing synthesis. ,save human effort. However, past experimental results show that the speech synthesized by the parametric method is relatively flat, poor quality and unnatural.

Parametric synthesis extracts vocoder parameters o={o1,...,oN} from speech signals x={x1,...,xT} and linguistic features l from text W.

A generative model is used during training, which can be HMM, neural network, etc. It is hoped to generate an appropriate vocoder parameter o according to the language feature l, and finally let the vocoder generate a sound waveform according to this parameter o.

In the training phase, update the model parameter λ:

In the Synthesis stage, the vocoder parameter o is obtained from the linguistic features l and the model λ:

Finally, the vocoder uses this o to synthesize the sound waveform.

With the development of artificial intelligence and deep learning, Google DeepMind proposed WaveNet in 2016 to make the synthesized speech more natural and more human-like.

Deep Learning: In recent years, with the development of deep learning technology, it has had a significant impact on the field of TTS. In 2016, Google DeepMind took the lead in proposing the deep learning architecture of WaveNet, which achieved great success and put it into commercial applications, including Google Assistant, Google Translate, etc.

However, the current voice broadcast service technology focuses more on the correctness of the voice generated, or makes the voice more natural and more human-like when generating sentences, and has not made any efforts to generate customized voice for the voice desired by the user. For voice broadcast, improving its user experience and serving the public is a problem to be overcome by technical personnel in related fields.

Based on the above problems, the present invention provides a customized voice service system, which executes an application program to generate customized voices that meet customized parameters, so that users can broadcast specific events with their favorite customized voices. It can greatly improve its user experience.

One object of the present invention is to provide a customized voice service system, which uses a text recognition module to generate voice data of a simulated character according to the text obtained by an application program, and then adjusts it to a customized voice that conforms to the customized voice parameters , so that the text is dubbed with a customized voice, and the custom voice is used for the text dubbing service. Compared with the previous technology of voice broadcast text technology, the user's freedom of voice selection in the broadcast service is increased.

For the above purpose, the present invention discloses a customized voice service system. When a user selects a text, the system is used to broadcast the text in customized voice. The system includes a first server, a second server, and a client device; here, the first server includes a text recognition module, the text recognition module uses text-to-speech technology (TTS) to convert text into speech; and the client device communicates with the first server and the second server; The two servers include a database that stores text; the client device uses an application program to connect to the first server through a network and execute an event, and the application program connects to the second server to obtain the text according to the event, and the application program converts the text to the second server. Sent to the first server, the first server receives the text and uses the text recognition module to convert the text into a first voice data of an analog character, and the client device selects the first voice data or a second voice of the analog character The data is played, wherein the first voice data has a preset voice parameter different from the voice parameter of the second voice data. The new model can generate a customized voice service that conforms to the customized parameters through the server to broadcast the input text, so as to improve the user experience of the voice broadcast service.

The present invention provides an embodiment, wherein the speech parameter includes a timbre, or includes a timbre and a tone.

The present invention further discloses a customized voice service system. When a user inputs a user voice, the system is used to play the voice with the customized voice, and the system includes a first server and a client device; Here, the first server includes a speech recognition module, and the speech recognition module adopts speech-to-text recognition (STT) to recognize speech; and the client device communicates with the first server; the client device uses an application program through a network The first server is connected to the first server and an event is executed, and the user's voice is sent to the first server according to the event application program. The first server receives the user's voice and uses the voice recognition module to convert the user's voice into an analog character A voice data, the client device selects the first voice data or a second voice data of a simulated character to play, wherein the first voice data has a preset voice parameter different from the voice parameter of the second voice data. As can be seen from the above, the present invention can recognize the input voice through the server, generate customized voice according to the customized parameters and play it, so as to improve the user experience of the voice broadcast service.

The present invention provides another embodiment, wherein the speech parameter includes a timbre, or a timbre and a tone.

1, 2: Customized voice service system

10: The first server

102: Text recognition module

104: Speech recognition module

12: Simulated characters

122: The first voice data

124:Second voice data

14: Voice parameters

142: Tone

144: Tone

20: Second server

22:Database

30: Client Device

32: Text

34: Events

36: User Voice

40: Internet

APP: application

S10-S26: Steps

Figure 1: It is a schematic block diagram of an embodiment of the new model; Figure 2: It is a flow chart of steps S10-S18 of an embodiment of the new model; Figure 3: It is another implementation of the new model The block diagram of the example; and FIG. 4 : it is a flow chart of the method of steps S20-S26 of another embodiment of the novel.

In order to enable your reviewers to have a further understanding and understanding of the features and effects of this new model, I would like to provide a preferred embodiment and a detailed description. The description is as follows:

The conventional method of broadcasting text by voice is to connect voice and text to generate voice data so as to realize voice-to-text broadcast, and the customized voice service system provided by this new model can also provide a user-defined Customized voice service, so as to use customized voice to broadcast text to provide voice broadcast service with more user experience.

Hereinafter, the present invention will be described in detail by illustrating various embodiments of the present invention by means of the drawings. The inventive concepts may, however, be embodied in many different forms and should not be construed as limited to the exemplary embodiments set forth herein.

First, please refer to FIG. 1 , which is a block diagram of an embodiment of the novel. As shown in the figure, the customized voice service system 1 of the present invention includes a first server 10, a second server 20 and a client device 30. The customized voice service system 1 of the present invention provides Services, such as: when a user selects a text 32, the system 1 is used to broadcast the text 32 to a customized voice.

Here, the first server 10 includes a text recognition module 102 , and the text recognition module 102 converts text into speech using text-to-speech technology (TTS). Text-to-speech technology (TTS) is a conventional technology, which can synthesize voice materials with acoustic features into voices or sentences that conform to text or words.

The client device 30 is communicatively connected to the first server 10 and the second server 20, wherein the communication connection may include any wired or wireless transmission technology for transmitting messages; currently known wireless transmission technology, such as CDMA, 3G or 4G, LTE, Wi-Fi, WiMax, WWAN, WLAN, WPAN, Bluetooth, and more.

The client device 30 is installed with an application program APP. Here, the application program APP may be a mobile application (mobile application, referred to as mobile app or app) or an online application (web application, referred to as web app or app). Here, the client device 30 may execute the mobile operating system (Mobile operating system; referred to as Mobile OS), and use the mobile operating system to execute the application program APP to interact with the corresponding server. Here, the mobile operating system is, for example, an Android (Android) operating system, an iOS operating system, and the like.

The second server 20 includes a database 22 that stores text 32 . The second server 20 can be used to store various kinds of files (eg, documents, pictures or combinations thereof). Here, the second server 20 may be a cloud hard disk platform, a network platform, a browser or an instant messaging platform. The cloud drive platform may be Google Drive, Dropbox, SugarSync, SkyDrive, and the like. The network platform may be Facebook (Facebook), Google+, Twitter (Twitter), or Weibo. The browser may be Chrome, Safari, or the like. The instant messaging platform can be QQ, Skype, WeChat, WhatsApp or LINE, etc. Text 32 means Text, a representation of written language, or a type of document, or any written material. Text refers to the expression of written language. From a literary point of view, it is usually a sentence or a combination of multiple sentences with complete and systematic meaning (Message). A text can be a sentence (Sentence), a paragraph (Paragraph) or a chapter (Discourse).

Please refer to Figure 1 and Figure 2 at the same time. Fig. 2 is a flowchart illustrating a method for performing customized voice service in the customized voice service system of the present invention.

Step S10: The client device uses the application program to connect to the first server via a network and execute an event.

Step S12: Connect to the second server according to the event application to obtain text.

Step S14: The application program sends the text to the first server.

Step S16 : The first server receives the text and converts the text into a first voice data of a simulated character by the text recognition module.

Step S18 : The client device selects the first voice data or a second voice data of a simulated character to play, wherein the first voice data has a preset voice parameter different from the voice parameter of the second voice data. The new model can generate a customized voice service that conforms to the customized parameters through the server to broadcast the input text, so as to improve the user experience of the voice broadcast service.

As shown in step S10 , the client device 30 uses the application APP to connect to the first server 10 via the network 40 and execute the event 34 , that is, execute the customized voice service of broadcasting the text with the customized voice.

As shown in step S12 , according to the event 34 , the application program APP is connected to the database 22 of the second server 20 through the network 40 to obtain the text 32 . Here, the text 32 is stored in the database 22 of the second server 20 as an example, and the text 32 can also be stored in the client device 30, so it is not necessary to connect the application APP to the second server 20 to obtain the text 32 , but not limited to this.

As shown in step S14 , the application program APP sends the text 32 to the first server 10 through the network 40 .

As shown in step S16 , the first server 10 receives the text 32 and uses the text recognition module 102 to convert the text 32 into the first voice data 122 of the simulated character 12 by applying text-to-speech technology (TTS), which is based on the text 32 The combination of corresponding phonetic elements or pinyin generates the first phonetic data 122 . Here, the first voice data 122 is the voice data corresponding to the text 32 with a voice attribute, that is, the first voice data 122 can be played. Here, the first voice data 122 has preset voice parameters 14 . The speech parameter 14 refers to the properties of the sound waves that constitute the speech, such as: loudness, that is, the amplitude of the sound waves, tone, that is, the frequency of the sound waves, and timbre, that is, the waveform of the sound waves. Here, the generated voice data must have a preset attribute, that is, the voice parameter 14 .

As shown in step S18 , the client device 30 can select the first voice data 122 or play the second voice data 124 of the simulated character 12 , wherein the voice parameters 14 of the second voice data 124 are different from the first voice data 122 There are preset voice parameters 14 . Here, the present invention provides that the user can obtain a customized voice that meets the customized requirements by adjusting the voice parameters 14 , that is, adjusting a timbre 142 of the first voice data 122 or including the timbre 142 and a tone 144 .

The present invention also provides a customized voice service system. Please refer to FIG. 3 , which is a block diagram of an embodiment of the new model. As shown in the figure, the customized voice service system 2 of the present invention includes a first server 10 and a client device 30. The services provided by the customized voice service system 2 of the present invention, for example, when the user inputs the user's voice 36. Use the system 2 to play the user's voice 36 in the customized voice. The input user voice 36 here refers to the user inputting the words spoken by the user into the client device 30 by uploading a digital file or in the form of recording by the client device 30 with a tape recorder. Here, if the user voice 36 is cached in the client device 30 in the form of a file, the client device 30 should have a database 22 for storing the file.

Here, the first server 10 includes a speech recognition module 104, and the speech recognition module 104 uses speech-to-text recognition (STT) to recognize speech into text. Speech-to-text recognition (STT), which is related to text-to-speech technology (TTS), and speech-to-text recognition (SST) is an algorithm that converts speech content into corresponding text, which is implemented by converting human speech. After input into the computer system, the digital signal processor (Digital Signal Processor) first decomposes the sound into different frequency bands (Frequency Bands), these frequency bands must be analyzed by the expert system program in the computer to determine each sound segment, called It is called Phonemes, and then combined with the judgment of other programs, the phonemes are combined into single words, and then the knowledge base of grammar is used to judge whether the grammatical relationship of each word conforms to human usage, and finally the whole result is put in Output on the computer screen, the simple description is by comparing the sound features with the database 22 and converting the speech content into possible words.

Here, the speech recognition module 104 is used to recognize the user's voice 36 as text, rather than directly adjusting the user's voice 36. The first reason is that the voice can be disassembled into several segments according to the speech-to-text recognition (SST), and then each segment Acoustic adjustments are made separately, and the result will be more accurate. The second reason is that according to the result of recognizing the user's voice 36 as text, it can be used in combination with the technology of voice control.

The client device 30 is communicatively connected to the first server 10, wherein the communication connection may include any wired or wireless transmission technology for transmitting messages; currently known wireless transmission technologies, such as CDMA, 3G or 4G, LTE, Wi- Fi, WiMax, WWAN, WLAN, WPAN, Bluetooth, and more.

The client device 30 is installed with an application program APP. Here, the application program APP may be a mobile application (mobile application, referred to as mobile app or app) or an online application (web application, referred to as web app or app). Here, the client device 30 can execute a mobile operating system (Mobile OS for short), and use the mobile operating system to execute an application program APP to interact with the corresponding server. Here, the mobile operating system is, for example, an Android (Android) operating system, an iOS operating system, and the like.

Please refer to Figure 3 and Figure 4 at the same time. FIG. 4 is a flow chart illustrating a method for performing customized voice service in the customized voice service system of the present invention.

Step S20: The client device uses the application program to connect to the first server via the network and execute the event.

Step S22: The application program sends the user's voice to the first server.

Step S24: The first server receives the user's voice and converts the user's voice into the first voice data of the simulated character by using the voice recognition module.

Step S26: The client device selects the first voice data or the second voice data of the simulated character to play, wherein the first voice data has preset voice parameters different from the words of the second voice data sound parameters. The new model can identify the input voice through the server, generate customized voice that meets the customized parameters and play it, and improve the user experience of the voice broadcast service.

As shown in step S20, the client device 30 uses the application program APP to connect to the first server 10 via the network 40 and execute the event 34, that is, execute the customized voice service of changing the voice of the input voice.

As shown in step S22 , the application program APP sends the user voice 36 to the first server 10 through the network 40 .

As shown in step S24, the first server 10 receives the user's voice 36 and uses the voice recognition module 104 to convert the user's voice 36 into the first voice data 122 of the simulated character 12 using speech-to-text recognition (SST). The voice 36 disassembles the voice into segments and reproduces the first voice material 122 . Here, the first voice data 122 is the voice data corresponding to the user's voice 36 with voice attributes, that is, the first voice data 122 can be played. Here, the first voice data 122 has preset voice parameters 14 . For example, the acoustic features of the speech parameter 14, such as tone and timbre, should correspond to the user's acoustic features, ie, the user's original voice, but not limited thereto.

As shown in step S26, the client device 30 can select the first voice data 122, or play the second voice data 124 of the simulated character 12, wherein the voice parameters 14 of the second voice data 124 are different from the first voice data 122 There are preset voice parameters 14 . Here, the present invention provides that the user can obtain a customized voice that meets the customized requirements by adjusting the voice parameters 14 , that is, adjusting the timbre 142 of the first voice data 122 or including the timbre 142 and the pitch 144 .

Some specific embodiments of the present invention are provided here as a reference for implementation, but are not limited thereto:

The client device 30 in this example can be a mobile phone; now, the application program APP is activated on the client device 30 and connected to the first server 10 through the network 40 to execute the event 34, and the event 34 is a customized voice broadcast Service, the customized voice broadcast service in this example takes the user manually enabling the application APP and enabling the guide reading function as an example, or the user's voice-controlled application APP to enable the reading function, but not limited to this; this example is performed on the news page. Take the customized voice broadcast service as an example, it can also be used to implement the customized voice broadcast service for the article or the user requests the application APP to read the game guide in the game, but this is not limited; first, the application APP is connected through the network 40 Obtain the text 32 from the second server 20. The text 32 in this example takes the text content (text or .txt file) on a news webpage (eg: .html file) as an example, but not limited to this; the client device 30 stores the text 32 is transmitted to the first server 10 through the network 40. The first server 10 has the text recognition module 102 through the text-to-speech technology to convert the text 32 into the first voice data 122 generated by the voice corresponding to the simulated character 12. For example, there may be a plurality of simulated characters 12 for the user to select; then the preset voice parameters 14 of the first voice data 122 are used to play, and the voice parameters 14 include the timbre 142 or the timbre 142 and the tone 144, and the client device 30 can select the simulation The second voice data 124 of the character 12 is played, and the voice parameters 14 of the second voice data 124 are different from the voice parameters 14 of the first voice data 122 .

The present invention also provides a specific embodiment. The client device 30 in this example can be a mobile phone or a personal computer, or other application APP. The client device 30 enables the application APP and connects to the first server 10 that provides services. In this example, a game program is used to activate the application APP to provide the voice of the game character; and event 34 is executed. Event 34 is a customized voice broadcast service. In this example, the program executes the service to provide different dialogues for different scenarios (eg game characters). The content is taken as an example, but not limited to this; other steps are not repeated as described above. The feature of this specific embodiment is that different speech parameters 14 are set for different dialogue contents in different scenarios, and the speech parameters 14 in this example are set by other applications. Provided by the APP, the text 32 is the character dubbing and the lines are also provided by other application APPs, but not limited to this; through the customized voice broadcast system, other application APPs can provide different dialogue content for different scenarios, which can optimize the utilization of file capacity Rate.

The present invention also provides a specific embodiment, please refer to FIG. 4 , the client device 30 in this example can be a mobile phone or a personal computer, but not limited to this. Now, the application program APP is enabled on the client device 30 and the application is connected through the network 40 . The first server 10 of the service provider of the program APP executes event 34. Event 34 is a customized voice service. In this example, the user reads out the content to be sent and changes the voice as an example to illustrate the customized voice service. this limit. The application APP uses the user's voice 36 . In this example, the file of the content read out by the user (for example: .mp3 or .aac file, the application APP can also provide the function of real-time recording) is the user's voice 36 , but it is not limited to this; The device 30 transmits the user's voice 36 to the first server 10, and the first server 10 has a voice recognition module 104 using speech-to-text recognition (STT) to generate a first voice data 122 corresponding to the user's voice 36, which corresponds to a simulated character 12. In this example, the user corresponds to the simulated character 12, but it is not limited to this; the voice parameter 14 of the preset first voice data 122 is used to play, and the voice parameter 14 includes the timbre 142 or the timbre 142 and the tone 144. The user-end device 30. The second voice data 124 of the simulated character 12 can be selected to be played, and the voice parameters 14 of the second voice data 124 are different from the voice parameters 14 of the first voice data 122. In this example, the first voice data 122 and the second voice data 124 are the results of the voice-changed content spoken by the user, but not limited to this.

To sum up, the new customized voice service system of the present invention allows users to customize the voice parameters and adjust the voice to a customized voice accordingly. The possibility of using voice to perform voice broadcast service improves the user experience.

Therefore, this new model is truly novel, progressive and available for industrial use. It should meet the patent application requirements of my country's patent law. It is absolutely necessary to file a new patent application in accordance with the law.

However, the above descriptions are only preferred embodiments of the present invention and are not intended to limit the scope of implementation of the present invention. All changes and modifications made in accordance with the shape, structure, features and spirit described in the scope of the patent application of the present invention are equivalent. , shall be included in the scope of the patent application of this new model.

1: Customized voice service system

10: The first server

102: Text recognition module

12: Simulated characters

122: The first voice data

124:Second voice data

14: Voice parameters

142: Tone

144: Tone

20: Second server

22:Database

30: Client Device

32: Text

34: Events

40: Internet

APP: application

Claims (4)

A customized voice service system, when a user selects a text, the system is used to broadcast the text with customized voice, the system includes: a first server, which includes a text recognition module, the text recognition module adopts Text-to-speech technology (TTS) converts text into speech; a second server including a database storing the text; and a client device communicating with the first server and the second server, for using an application program to connect to the first server via a network and execute an event, according to the event the application program connects to the second server to obtain the text, the application program sends the text to the first server the first server receives the text and converts the text into a first voice data of an analog character by the text recognition module, and the client device selects the first voice data or a second voice data of the analog character The voice data is played, wherein a preset voice parameter of the first voice data is different from the voice parameter of the second voice data. The customized voice service system according to claim 1, wherein the voice parameter includes a timbre, or the timbre and a tone. A customized voice service system, when a user inputs a user voice, use the system to play the voice as a customized voice, the system includes: a first server, which includes a voice recognition module, the voice recognition module The group adopts speech-to-text recognition (STT) to recognize speech; and a client device communicatively connected to the first server for using an application program to connect to the first server via a network and execute an event, according to the event The application program sends the user voice to the first server, the first server receives the user voice and uses the voice recognition module to convert the user voice into a first voice data of an analog character, the client The device selects the first voice data or a second voice data of the simulated character to play, wherein the first voice data has a preset voice parameter different from the voice parameter of the second voice data. The customized voice service system according to claim 3, wherein the voice parameter includes a timbre, or the timbre and a tone.

Images