KR20210014526A - Server, device and method for providing speech systhesis service - Google Patents

Abstract

The present invention relates to a server for providing a voice synthesis service comprising: a receiving unit for receiving voice recording data of a plurality of users for at least one of a plurality of pre-registered sample scripts; a learning unit for learning a synthetic sound prediction model for a pitch value related to a tone of each user based on the received voice recording data; a generating unit for generating sample synthesis data for at least one of the sample scripts based on the learned synthetic sound prediction model when a sample request command is received from a user terminal; a providing unit for providing the sample synthesis data for the user terminal; and a voice registering unit for registering user voice information on the user terminal based on whether or not the quality of the sample synthesis data received from the user terminal is satisfied. Accordingly, a user can check the quality of expected sample synthesis data.

Description

Server, device and method that provide speech synthesis service {SERVER, DEVICE AND METHOD FOR PROVIDING SPEECH SYSTHESIS SERVICE}

The present invention relates to a server, a terminal, and a method for providing a speech synthesis service.

The text-to-speech (TTS) system converts the input text into 　speech using the speech synthesis 　 method to synthesize 　speech of natural and high quality sound. Such speech synthesis 　 methods include a corpus method, a hidden markov mode (HMM) method, and a deep learning method.

A large-capacity corpus-based speech synthesis system is expensive because it requires a large-scale database to store musical pieces, and it is impossible to expand speech synthesis using voice actor recording in various areas requiring artificial intelligence services, and is not stored in the database. There is a drawback that it is not natural to synthesize speech for unsound musical pieces.

HMM (Hidden Markov Mode)-based speech synthesis system can provide even speech synthesis quality regardless of whether or not phonetic pieces are stored in the database, and is suitable for navigation speech synthesis with many proper nouns like place names, but is not suitable for large-capacity terminal-based embedded. However, it has the disadvantage of unnatural speech synthesis like a mechanical sound.

The deep learning-based speech synthesis system enables natural speech synthesis even with recorded voice data within a short period of time, and is thus used for various services.

In recent years, it is expanding from the existing professional voice actor-centered voice synthesis service to the voice synthesis service using the voices of general customers.

In the case of speech synthesis services for general customers, it is difficult for general customers to maintain vocalization and reading speed, unlike professional voice actors. In addition, in order to obtain the voice synthesis quality desired by the general customer, there is an inconvenience of repeatedly recording voice until the general customer is satisfied.

Korean Registered Patent Publication No. 1996-0007129 (registered on May 27, 1996)

The present invention is to solve the problems of the prior art described above, by using the user's voice recording data for a previously registered sample script to learn a synthesized tone prediction model related to the user's tone, and based on the learned synthesized tone prediction model It is intended to generate sample synthesis data and provide it to the user terminal. However, the technical problem to be achieved by the present embodiment is not limited to the technical problems as described above, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, the server providing a speech synthesis service according to the first aspect of the present invention receives voice recording data of a plurality of users for at least one of a plurality of previously registered sample scripts. Receiver; A learning unit for learning a synthesized sound prediction model for a pitch value related to the tone of each user based on the received voice recording data; A generator configured to generate sample synthesis data for at least one of the plurality of sample scripts based on the learned synthesized sound prediction model when receiving a sample request command from a user terminal; A providing unit for providing the sample synthesis data to the user terminal; And a voice registration unit for registering voice information for a user of the user terminal based on whether or not the quality of the sample synthesis data received from the user terminal is satisfied.

A user terminal receiving a speech synthesis service according to a second aspect of the present invention includes: a transmission unit for transmitting a sample request command for speech synthesis to a speech synthesis service providing server; A receiver for receiving sample synthesis data for at least one generated based on the learned synthesized sound prediction model from the speech synthesis service providing server; And an input unit for receiving quality satisfaction information for the sample synthesis data. The transmission unit transmits the input quality satisfaction information to the voice synthesis service providing server, and the synthesized sound prediction model is based on voice recording data of a plurality of users for at least one of a plurality of previously registered sample scripts. It is a model trained on a pitch value related to a tone color, and voice information for a user of the user terminal may be registered in the voice synthesis service providing server based on whether or not the sample synthesis data is satisfied with quality.

A method for providing a speech synthesis service by a speech synthesis service providing server according to a third aspect of the present invention includes the steps of: receiving voice recording data of a plurality of users for at least one of a plurality of previously registered sample scripts; Learning a synthesized sound prediction model for a pitch value related to each user's tone based on the received voice recording data; Generating sample synthesis data for at least one of the plurality of sample scripts based on the learned synthesized sound prediction model when receiving a sample request command from a user terminal; Providing the sample synthesis data to the user terminal; And registering voice information for a user of the user terminal based on whether or not the quality of the sample synthesis data received from the user terminal is satisfied.

The above-described problem solving means are merely exemplary and should not be construed as limiting the present invention. In addition to the above-described exemplary embodiments, there may be additional embodiments described in the drawings and detailed description of the invention.

According to any one of the above-described problem solving means of the present invention, the present invention is to learn a synthesized sound prediction model related to the user's tone by using the user's voice recording data for a previously registered sample script, and the learned synthesized sound prediction model Based on the sample synthesis data, it may be provided to the user terminal. Through this, since the user can check the quality of the sample synthesis data expected when the user's voice is registered, the user can save the trouble of repeatedly recording a script as in the past.

1 is a block diagram of a system for providing a speech synthesis service according to an embodiment of the present invention.
FIG. 2 is a block diagram of a voice synthesis service providing server shown in FIG. 1 according to an embodiment of the present invention.
3A to 3D are diagrams for explaining a method of registering voice information for a user according to an embodiment of the present invention.
4 is a block diagram of the user terminal shown in FIG. 1 according to an embodiment of the present invention.
5 is a diagram illustrating a method of providing a speech synthesis service according to an embodiment of the present invention.
6 is a flowchart illustrating a method of providing a speech synthesis service according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary.

In the present specification, the term "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Further, one unit may be realized using two or more hardware, or two or more units may be realized using one hardware.

In the present specification, some of the operations or functions described as being performed by the terminal or device may be performed instead by a server connected to the terminal or device. Likewise, some of the operations or functions described as being performed by the server may also be performed by a terminal or device connected to the server.

Hereinafter, with reference to the accompanying configuration diagram or processing flow chart, specific details for the implementation of the present invention will be described.

1 is a block diagram of a system for providing a speech synthesis service according to an embodiment of the present invention.

Referring to FIG. 1, a system for providing a speech synthesis service may include a server 100 for providing a speech synthesis service and a user terminal 110. However, since the speech synthesis service providing system of FIG. 1 is only an embodiment of the present invention, the present invention is not limitedly interpreted through FIG. 1, and may be configured differently from FIG. 1 according to various embodiments of the present invention. have.

In general, components of the system for providing a speech synthesis service of FIG. 1 are connected through a network (not shown). A network refers to a connection structure that enables information exchange between nodes such as terminals and servers, and is a local area network (LAN), a wide area network (WAN), and the Internet (WWW: World). Wide Web), wired/wireless data communication networks, telephone networks, wired/wireless television networks, etc. Examples of wireless data networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, and ultrasound. Communication, Visible Light Communication (VLC), LiFi, etc. are included, but are not limited thereto.

The user terminal 110 may record the user's voice recording data for at least one sample script among a plurality of previously registered sample scripts, and then transmit the voice recording data to the voice synthesis service providing server 100.

The voice synthesis service providing server 100 may learn a synthesized sound prediction model for a pitch value related to the user's tone based on the voice recording data received from the user terminal 110.

The user terminal 110 sends a sample request command for the user's speech synthesis to the speech synthesis service providing server 100 after a preset period has passed from the time when the user's voice recorded data is registered in the speech synthesis service providing server 100. Can be transmitted.

When receiving a sample request command from the user terminal 110, the speech synthesis service providing server 100 generates sample synthesis data for at least one of a plurality of sample scripts based on the learned synthesized sound prediction model of the user terminal 110. It is generated, and the generated sample synthesis data may be provided to the user terminal 110.

The user terminal 110 may check quality satisfaction with respect to the sample synthesis data received from the speech synthesis service providing server 100, and then transmit quality satisfaction information with respect to the sample synthesis data to the speech synthesis service providing server 100. .

The voice synthesis service providing server 100 may register voice information for a user of the user terminal 110 based on whether or not the quality of the sample synthesis data received from the user terminal 110 is satisfied.

Thereafter, when receiving a content request command from the user terminal 110, the voice synthesis service providing server 100 registers the content synthesis data for the content script corresponding to the content request command among a plurality of previously registered content scripts. It is generated based on the voice information of, and the generated content synthesis data may be provided to the user terminal 110.

Hereinafter, the operation of each component of the speech synthesis service providing system of FIG. 1 will be described in more detail.

FIG. 2 is a block diagram of the voice synthesis service providing server 100 shown in FIG. 1 according to an embodiment of the present invention.

2, the voice synthesis service providing server 100 includes a receiving unit 200, a learning unit 210, a generating unit 220, a providing unit 230, a voice registering unit 240, and a speed adjusting unit 250 , A verification unit 260, a pause adjustment unit 270, and a voice input unit 280. However, the voice synthesis service providing server 100 shown in FIG. 2 is only an example of implementation of the present invention, and various modifications are possible based on the components shown in FIG. 2.

The receiving unit 200 may receive voice recording data of a plurality of users for at least one of a plurality of previously registered sample scripts from terminals of a plurality of users.

The receiver 200 receives voice recording data in which a user's voice is recorded for at least one sample script among a plurality of sample scripts previously registered from each of a plurality of users who intend to use the service of moving picture content containing the user's voice. can do. In addition, the receiving unit 200 may receive each voice recording data and identification information for each user terminal together from a plurality of user terminals. For example, referring to FIG. 3A, a sample script (301, for example,'children go to school early today') extracted at random from among a plurality of sample scripts previously registered with the user terminal 110 is a user terminal ( When provided to 110), the receiving unit 200 may receive voice recording data of the user who read the sample script 301 from the user terminal 110 and identification information of the user terminal 110.

The verification unit 260 may verify pronunciation accuracy of voice recording data received from a plurality of user terminals.

The verification unit 260 performs voice recognition on the voice recording data received from a plurality of user terminals, and the similarity between the result of performing voice recognition on the received voice recording data and the sample script corresponding to the received voice recording data is determined. If it appears below the set threshold, it may be determined that the pronunciation accuracy of the received voice recording data is incorrect. For example, referring to FIG. 3A, when the similarity between the result of performing the speech recognition on the voice recording data received from the user terminal 110 and the sample script 301 is equal to or greater than 80%, the user It is determined that the pronunciation of the voice recorded data received from the terminal 110 is correct, and if the similarity is less than 80%, it may be determined that the pronunciation of the voice recorded data received from the user terminal 110 is incorrect.

The voice input unit 280 guides the user terminal, which is determined to be inaccurate in pronunciation accuracy for the voice recording data for each user terminal, to re-input new voice recording data for the sample script, and provides a new voice for the sample script from the corresponding user terminal. You can re-enter the recorded data. For example, referring to FIG. 3A, when the voice input unit 280 does not match the similarity between the result of performing voice recognition on the voice recording data received from the user terminal 110 and the sample script 301 by 80% or more, New voice recording data for the sample script 301 may be re-inputted from the user terminal 110. If the similarity between the voice recording data re-inputted from the user terminal 110 and the sample script corresponding to the voice recording data appears to be less than 80% more than a preset number of times (eg, 3 times), the voice input unit 280 May provide a new sample script different from the sample script 301 to the user terminal 110, and re-input the user's voice recording data for the new sample script.

The speed control unit 250 may provide a speed control interface to the user terminal 110 so that the reading speed of each user with respect to at least one sample script among the plurality of sample scripts is adjusted according to a preset reading speed. For example, referring to FIGS. 3A to 3B together, the speed control unit 250 may provide a speed control interface 303 (eg, a cursor) so that the user can read the sample script 301 according to a preset reading speed. It can be provided to the user terminal 110. Thereafter, the speed control unit 250 of the speed control interface 303 that guides the reading speed of the sample script corresponding to the next sentence of the sample script 301 in consideration of the user's reading speed of the sample script 301 Movement speed can be adjusted.

The voice input unit 280 may request a user to read the sample script again according to a comparison between a reading rate for at least one sample script among a plurality of sample scripts and a preset threshold (eg, 20%) for each user. For example, the voice input unit 280 may request the user terminal of the user whose reading rate for the user's sample script exceeds a preset threshold to read the sample script again according to the preset reading rate.

The speed control unit 250 may measure the speech speed of voice recorded data received from a plurality of user terminals for each user.

On the other hand, in order to improve the quality of speech synthesis, it is important to minimize the pause between voice sections in the voice recording data.

The pause adjustment unit 270 may detect at least one voice section from each of the voice recording data received from a plurality of user terminals. For example, the pause adjustment unit 270 may detect a voice section from voice recording data using a Voice Activity Detection (VAD) algorithm.

The pause adjustment unit 270 may calculate the reliability of the pause estimation between the detected at least one voice interval and adjust the pause between the speech intervals based on the calculated reliability of the pause estimation. Here, if the reliability of the pause estimation is high, the pause between the voice intervals is long, and if the reliability of the pause estimation is low, it may mean that the pause between the voice intervals is short. For example, the pause adjustment unit 270 may delete the pause between voice sections or adjust it to an average value so that the pause between voice sections is minimized.

The learning unit 210 may learn a synthesized sound prediction model related to the voice characteristics of each user for each user based on the received voice recording data of each of the plurality of users.

Specifically, the learning unit 210 may learn a synthesized sound prediction model for a pitch value related to the tone of each user for each user based on the received voice recording data of each of the plurality of users. For example, referring to FIG. 3C, the learning unit 210 derives a pitch value related to the user's tone from the user's voice recording data using a neural network algorithm 305, and uses the derived pitch value to obtain a synthesized sound. Predictive models can be trained. Here, the neural network algorithm 305 is configured based on multilayer RNNs (Recurrent Neural Networks) capable of learning a high-dimensional continuous distribution like audio data.

)를 구성하는 각 음절의 확률값은 [수학식 1]과 같이 표현될 수 있다. The learning unit 210 may train the synthesized sound prediction model so that the probability value of each syllable constituting the voice recording data of each of the plurality of users is maximum in relation to the pitch value. User's voice recording data (

The probability value of each syllable constituting) can be expressed as [Equation 1].

[Equation 1]

Referring to FIG. 3D, the learning unit 210 inputs a pitch value related to the user's tone included in the voice recording data for each user, a sample script corresponding to the voice data, and a user ID into the synthesized sound prediction model, The synthesized sound prediction model can be trained so that the probability value of each syllable constituting the is maximized.

When receiving a sample request command from at least one user terminal 110 among a plurality of user terminals (terminals that transmit the user's voice recorded data to the voice synthesis service providing server), the learned user terminal ( Sample synthesis data for at least one of a plurality of sample scripts may be generated based on the synthesized sound prediction model of 110). For example, the generation unit 220 may determine the user's pitch value and the user's speech speed measured by the speed controller 250 in at least one sample script based on the learned synthesized sound prediction model of the user terminal 110. The reflected sample synthesis data can be generated.

The providing unit 230 may provide the generated sample synthesis data to the user terminal 110.

The voice registration unit 240 may register voice information for a user of the user terminal 110 based on whether or not the quality of the sample synthesis data received from the user terminal 110 is satisfied. For example, when a user receives a registration approval message from the user terminal 110 indicating that the quality of the sample synthesis data is satisfied, the voice registration unit 240 registers the voice information for the user of the user terminal 110 can do. If the user receives a registration unacknowledged message from the user terminal 110 indicating dissatisfaction with the quality of the sample synthesized data, the learning unit 210 receives the user's voice recording data from the user terminal 110 again. The composite sound prediction model can be retrained. Here, the voice information may include a pitch value for each user derived based on the synthesized sound prediction model, and may further include a measured voice speed of the user.

When receiving a content request command (including information on a content script selected by the user) from the user terminal 110, the generation unit 220 sends a content script corresponding to the content request command among a plurality of previously registered content scripts. Content synthesis data for the registered voice information may be generated based on a pitch value included in the registered voice information.

The generator 220 may generate content synthesis data for a content script corresponding to a content request command based on a pitch value and a speech speed included in the registered voice information.

The providing unit 230 may provide the content synthesis data to the user terminal 110.

On the other hand, those skilled in the art, the receiving unit 200, the learning unit 210, the generating unit 220, the providing unit 230, the voice registration unit 240, the speed control unit 250, the verification unit 260, the pause adjustment unit ( It will be fully understood that each of the 270 and the voice input unit 280 may be implemented separately, or one or more of them may be integrated and implemented.

4 is a block diagram of the user terminal 110 shown in FIG. 1 according to an embodiment of the present invention.

Referring to FIG. 4, the user terminal 110 may include a transmission unit 400, a reception unit 410, an input unit 420, and a speed control interface providing unit 430. However, the user terminal 110 shown in FIG. 5 is only an example of implementation of the present invention, and various modifications are possible based on the elements shown in FIG. 5.

The transmission unit 400 may transmit the user's voice recording data for at least one of a plurality of previously registered sample scripts to the voice synthesis service providing server 100. For example, referring to FIG. 5, when a user who wants to use the service of moving picture content containing the user's voice selects the voice recording 501 through the'My Voice Fairy Tale' application, the receiver 410 provides a voice synthesis service. The screen of the user terminal 110 receives a sample script randomly extracted from a plurality of sample scripts previously registered from the server 100 (reference numeral 301 in FIG. 3A, for example,'Children go to school early today'). Can be printed on. When the user records the user's voice with respect to the sample script, the transmission unit 400 may transmit identification information of the user terminal 110 together with the user's voice recording data to the voice synthesis service providing server 100.

Here, the speech speed of the user's voice recorded data transmitted to the voice synthesis service providing server 100 may be measured by the voice synthesis service providing server 100.

The speed control interface providing unit 430 provides a speed control interface (reference numeral 303 in FIG. 3B) for guiding the user's reading speed for at least one sample script provided to the user terminal 110 to be adjusted according to a preset reading speed. It may be provided from the voice synthesis service providing server 100. The user can read the sample script according to the speed of the speed control interface displayed in the sample script.

The speed control interface providing unit 430 considers the reading speed of the user's sample script and adjusts the reading speed of the sample script corresponding to the next sentence provided to the user terminal 110 after provision of the sample script. The speed control interface can be controlled to be adjusted through the speed of movement.

When the user's voice recording data is transmitted to the voice synthesis service providing server 100, the voice synthesis service providing server 100 may learn a synthesized sound prediction model related to the user's voice characteristic based on the user's voice recorded data. Specifically, the voice synthesis service providing server 100 may derive a pitch value related to the user's tone from the user's voice recording data, and learn a synthesized tone prediction model using the derived pitch value.

The transmission unit 400 may transmit a sample request command for speech synthesis to the speech synthesis service providing server 100. For example, referring to FIG. 5, when the user selects the interface 503 for listening to sample synthesis data through the'My Voice Fairy Tale' application, the transmission unit 400 sends a voice to the speech synthesis service providing server 100 A sample request command for synthesis can be transmitted.

Thereafter, when receiving the sample request command from the user terminal 110, the speech synthesis service providing server 100 synthesizes a sample for at least one of the plurality of sample scripts based on the learned synthesized sound prediction model of the user terminal 110. Data may be generated and transmitted to the user terminal 110. Here, the sample synthesis data for at least one may be data in which a pitch value related to a user's tone and a user's vocal speed are reflected.

The receiver 410 may receive sample synthesis data for at least one generated based on the learned synthesized sound prediction model from the speech synthesis service providing server 100.

The input unit 420 may receive quality satisfaction information for the sample synthesis data. For example, referring to FIG. 5, when the output voice of the sample synthesis data is finished, the input unit 420 satisfies the quality satisfaction information for the sample synthesis data through the'My Voice Fairy Tale' application or a dissatisfaction button 505. ) Can be entered.

The transmission unit 400 may transmit quality satisfaction information input for the sample synthesis data to the speech synthesis service providing server 100.

Voice information for the user of the user terminal 110 may be registered in the voice synthesis service providing server 100 based on whether or not the sample synthesis data is satisfied with quality. Here, the voice information may include a pitch value for a user derived based on the synthesized sound prediction model. In addition, the voice information may further include a voice speed of the user's voice recording data.

For example, if the user satisfies the quality of the sample synthesis data, voice information for the user is registered in the speech synthesis service providing server 100, but if the user does not satisfy the quality of the sample synthesis data, the user The voice recording data is re-entered, and the voice synthesis service providing server 100 may retrain the synthesized sound prediction model with the re-input voice recording data.

The transmission unit 400 may transmit a content request command to the voice synthesis service providing server 100. For example, referring to FIG. 5, after the voice information for the user is registered in the voice synthesis service providing server 100 with quality satisfaction for the sample synthesis data, the transmission unit 400 is configured to provide a plurality of previously registered content scripts. In 507, a content request command for a content script selected by a user may be transmitted to the voice synthesis service providing server 100.

The receiver 410 may receive content synthesis data generated based on a pitch value included in voice information for a registered user from the voice synthesis service providing server 100. Here, the content synthesis data may be content synthesis data for a content script corresponding to a content request command among a plurality of previously registered content scripts. The content synthesis data may be generated based on a pitch value and a speech speed included in the registered user's voice information.

On the other hand, those skilled in the art will fully understand that each of the transmission unit 400, the reception unit 410, the input unit 420, and the speed control interface providing unit 430 may be implemented separately, or one or more of them may be integrated and implemented. will be.

6 is a flowchart illustrating a method of providing a speech synthesis service according to an embodiment of the present invention.

Referring to FIG. 6, in step S601, the voice synthesis service providing server 100 may receive voice recording data of a plurality of users for at least one of a plurality of sample scripts previously registered from a plurality of user terminals.

In step S603, the voice synthesis service providing server 100 may learn a synthesized sound prediction model for a pitch value related to the tone of each user based on the voice recording data received for each of the plurality of user terminals.

In step S605, when receiving a sample request command from the user terminal 110, the speech synthesis service providing server 100 may use at least one of the plurality of sample scripts based on the synthesized sound prediction model corresponding to the learned user terminal 110. Sample synthesis data can be generated.

In step S607, the voice synthesis service providing server 100 may provide the generated sample synthesis data to the user terminal 110.

In step S609, the voice synthesis service providing server 100 may register voice information for a user of the user terminal 110 based on whether or not the quality of the sample synthesis data received from the user terminal 110 is satisfied.

In the above description, steps S601 to S609 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, and the order between steps may be changed.

An embodiment of the present invention may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Further, the computer-readable medium may include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. .

100: voice synthesis service providing server
110: user terminal
200: receiver
210: learning department
220: generation unit
230: provision
240: voice register
250: speed control unit
260: verification unit
270: pause adjustment unit
280: voice input unit
500: transmission unit
510: receiver
520: input unit
530: speed control interface providing unit

Claims (18)

In the server providing a speech synthesis service,
A receiver for receiving voice recording data of a plurality of users for at least one of a plurality of previously registered sample scripts;
A learning unit for learning a synthesized sound prediction model for a pitch value related to the tone of each user based on the received voice recording data;
A generator configured to generate sample synthesis data for at least one of the plurality of sample scripts based on the learned synthesized sound prediction model when receiving a sample request command from a user terminal;
A providing unit for providing the sample synthesis data to the user terminal; And
A voice registration unit that registers voice information for a user of the user terminal based on quality satisfaction with respect to the sample synthesis data received from the user terminal
That includes, a voice synthesis service providing server.
The method of claim 1,
The voice information includes a pitch value for the user derived based on the synthesized sound prediction model.
The method of claim 2,
When receiving a content request command from the user terminal, the generation unit determines content synthesis data for a content script corresponding to the content request command among a plurality of previously registered content scripts based on a pitch value included in the registered voice information. And create it,
The providing unit to provide the content synthesis data to the user terminal, the voice synthesis service providing server.
The method of claim 3,
The speech synthesis service providing server further comprising a speed control unit for providing a speed control interface to a user terminal so that the reading speed for the at least one sample script is adjusted to fit a preset reading speed.
The method of claim 4,
The speed control unit measures the speech speed of the received voice recording data for each user,
The voice information further includes a speech speed measured for each user.
The method of claim 5,
The generation unit generates content synthesis data for a content script corresponding to the content request command based on a pitch value and a speech speed included in the registered voice information.
The method of claim 1,
Wherein the learning unit trains the synthesized sound prediction model so that a probability value of each syllable constituting the received voice recording data becomes maximum in relation to the pitch value.
The method of claim 1,
Verification unit for verifying accuracy of pronunciation of the received voice recording data
That further comprises a voice synthesis service providing server.
The method of claim 8,
The verification unit performs voice recognition on the received voice recording data, and a similarity between the result of performing voice recognition on the received voice recording data and a sample script corresponding to the received voice recording data is less than a preset threshold. If it appears, it is to determine that the pronunciation accuracy of the received voice recording data is incorrect.
The method of claim 9,
When it is determined that the pronunciation accuracy of the received voice recording data is inaccurate, the voice synthesis service providing server further comprises a voice input unit configured to re-input new voice recording data for the sample script.
The method of claim 1,
Providing a speech synthesis service further comprising a pause adjustment unit for calculating a pause estimation reliability between at least one voice interval detected from the voice recording data, and adjusting a pause between the speech intervals based on the calculated pause estimation reliability server.
In a user terminal receiving a speech synthesis service,
A transmission unit for transmitting the user's voice recording data for at least one of a plurality of previously registered sample scripts to a voice synthesis service providing server;
A receiver configured to receive, from the speech synthesis service providing server, sample synthesis data for at least one generated based on the learned synthesized sound prediction model when a sample request command for speech synthesis is transmitted to the speech synthesis service providing server; And
An input unit for receiving quality satisfaction information for the sample synthesis data;
The transmission unit transmits the input quality satisfaction information to the voice synthesis service providing server,
The synthesized sound prediction model is a model trained on a pitch value related to the user's tone based on the user's voice recording data for the at least one,
The user terminal, wherein the voice information for the user is registered in the voice synthesis service providing server based on whether or not the quality of the sample synthesis data is satisfied.
The method of claim 12,
The voice information includes a pitch value for the user derived based on the synthesized sound prediction model.
The method of claim 13,
The transmission unit transmits a content request command to the voice synthesis service providing server,
The receiver receives content synthesis data generated based on a pitch value included in the voice information for the registered user from the voice synthesis service providing server,
The content synthesis data is content synthesis data for a content script corresponding to the content request command among a plurality of previously registered content scripts.
The method of claim 14,
The user terminal further comprising a speed control interface providing unit provided from the voice synthesis service providing server with a speed control interface guiding the user's reading speed with respect to the at least one sample script to be adjusted according to a preset reading speed. .
The method of claim 15,
The speech speed of the voice recording data of the user transmitted to the voice synthesis service providing server is measured by the voice synthesis service providing server,
The voice information further includes a speech speed of the user's voice recording data.
The method of claim 16,
The content synthesis data is generated based on a pitch value and a speech speed included in the registered user's voice information.
In the method of providing a speech synthesis service by a speech synthesis service providing server,
Receiving voice recording data of a plurality of users for at least one of a plurality of previously registered sample scripts;
Learning a synthesized sound prediction model for a pitch value related to each user's tone based on the received voice recording data;
Generating sample synthesis data for at least one of the plurality of sample scripts based on the learned synthesized sound prediction model when receiving a sample request command from a user terminal;
Providing the sample synthesis data to the user terminal; And
And registering voice information for a user of the user terminal based on whether or not the quality of the sample synthesis data received from the user terminal is satisfied.

Images