KR20220050342A - Apparatus, terminal and method for providing speech synthesizer service - Google Patents

Abstract

Disclosed are a device, terminal and method for providing a voice synthesis service. According to one aspect of the present invention, a voice synthesis service providing device comprises: a voice database; a voice feature extraction unit configured to receive user voice sample information for a suggested word, extract voice feature information of the user from the voice sample information, and store the voice feature information in the voice database; and a voice synthesis service unit configured to extract, when receiving a voice synthesis request signal including text from the user, voice feature information of the user from the voice database and apply the voice feature information and the text to machine learning to generate voice data for the text. According to the present invention, the voice synthesis service providing device is capable of converting input text into a user's voice.

Description

Apparatus, terminal and method for providing speech synthesis service {APPARATUS, TERMINAL AND METHOD FOR PROVIDING SPEECH SYNTHESIZER SERVICE}

The present invention relates to an apparatus, terminal and method for providing a speech synthesis service, and more particularly, to an apparatus, terminal and method for providing a speech synthesis service capable of converting input text into a user's voice and providing it. .

With the recent development of speech synthesis technology, speech synthesis technology is widely used in various voice guidance and education fields. Speech synthesis is a technology that generates sounds similar to human speech, and is commonly known as a text-to-speech (TTS) system.

TTS (Text To Speech) is a technology that converts characters or symbols into voice and plays them. TTS builds a pronunciation database for phonemes and connects them to create a continuous voice. At this time, the key is to synthesize a natural voice by adjusting the volume, length, and height of the voice.

That is, the TTS is a text-to-speech conversion device that converts character strings (sentences) into speech, and is largely divided into three stages: language processing, prosody generation, and waveform synthesis. The structure is analyzed, and a rhyme similar to a human read is generated by the analyzed document structure, and a synthesized sound is generated by collecting the basic units of the stored voice DB according to the generated rhyme.

Since TTS has no target vocabulary and converts general text-type information into speech, phonetics, speech analysis, speech synthesis, and speech recognition technologies are combined to output more natural and diverse speech when implementing the system.

However, when a terminal providing such a conventional TTS outputs a voice such as a text message, it always outputs the same preset voice regardless of who the counterpart is, so there is a problem in that it cannot satisfy the needs of various users.

The background technology of the present invention is disclosed in Korean Patent Application Laid-Open No. 2011-0032256 (published on March 30, 2011, title of invention: TTS announcement apparatus and method of in-store sound broadcasting system).

The present invention has been devised to improve the above problems, and an object of the present invention is to provide an apparatus, a terminal, and a method for providing a voice synthesis service that converts an input text into a user's voice and provides it. .

The problem to be solved by the present invention is not limited to the problem(s) mentioned above, and another problem(s) not mentioned will be clearly understood by those skilled in the art from the following description.

An apparatus for providing a speech synthesis service according to an aspect of the present invention receives a voice database and user's voice sample information for a suggested word, extracts the user's voice characteristic information from the voice sample information, and uses the voice characteristic information as the voice A voice feature extraction unit for storing in a database, when receiving a voice synthesis request signal including text from the user, extracts the user's voice feature information from the voice database, and applies the voice feature information and the text to machine learning and a voice synthesis service unit generating voice data for the text.

In the present invention, the voice synthesis service unit provides a design template for editing the video input from the user or a pre-stored video, receives editing information of the video through the design template, and includes text and dubber information When receiving a video dubbing request, the voice characteristic information of the dubbing person is extracted from the voice database, the voice characteristic information and the text are applied to machine learning to generate voice data for the text, and the voice data is converted into the moving image. can be applied to

In the present invention, when receiving an e-book dubbing request including an e-book and reader information from the user, the voice synthesis service unit extracts voice characteristic information of the reader from the voice database, and extracts the voice characteristic information and the By applying the e-book to machine learning, voice data for the e-book can be generated.

According to another aspect of the present invention, in a method for providing a voice synthesis service, a voice feature extraction unit receives voice sample information of a user for a suggested word, extracts voice feature information of the user from the voice sample information, and extracts the voice feature information. Storing in a voice database, when the voice synthesis service unit receives a voice synthesis request signal including text from the user, extracts the user's voice characteristic information from the voice database, and applies the voice characteristic information and the text to machine learning and generating voice data for the text by applying it.

According to the present invention, when the voice synthesis service unit receives a video editing command from the user, it provides a design template for editing a video input from the user or a pre-stored video, and edits the video editing information through the design template. receiving and editing the video; and when the voice synthesis service unit receives a video dubbing request including text and dubbing person information, extracting the voice characteristic information of the dubbing person from the voice database, the voice characteristic information and the The method may further include generating voice data for the text by applying the text to machine learning, and applying the voice data to the moving picture.

In the present invention, when the voice synthesis service unit receives an e-book dubbing request including an e-book and reader information from the user, the voice characteristic information of the reader is extracted from the voice database, the voice characteristic information and the The method may further include generating voice data for the e-book by applying the e-book to machine learning.

A terminal according to another aspect of the present invention includes a storage unit, a voice recorder configured to detect a connection of a voice call, convert a voice during a call into voice data, and generate a recording file corresponding to the voice data, a preset period of time and a voice feature extracting unit for extracting the user's voice characteristic information from the recorded file generated during the operation and storing the voice characteristic information in the storage unit, wherein the voice recording unit converts voice data corresponding to the recorded file into text data. Thus, it can be stored in the storage unit together with the recorded file.

In the present invention, the voice recorder may create a recording file for the user's voice by filtering the voice of the callee from the call contents.

The present invention extracts the user's voice characteristic information from the storage unit when a command to convert the voice signal provided by the terminal into a user's voice is received, and applies the voice characteristic information and the voice signal to machine learning to apply the voice signal It may further include a voice synthesis service unit for generating voice data for the.

In the present invention, the voice synthesis service unit, when receiving a voice synthesis request signal including text from the user, extracts the user's voice characteristic information from the storage unit, and applies the voice characteristic information and the text to machine learning. Voice data for the text may be generated.

An apparatus, a terminal, and a method for providing a speech synthesis service according to an embodiment of the present invention may convert an input text into a user's voice and provide the converted text.

The apparatus, terminal and method for providing a voice synthesis service according to another embodiment of the present invention may provide the user's own voice by dubbing the video, such as YouTube, SNS promotional video, and corporate promotional video.

In an apparatus, terminal, and method for providing a voice synthesis service according to another embodiment of the present invention, a user can listen to an e-book in the voice of a person desired by the user.

An apparatus, terminal, and method for providing a voice synthesis service according to another embodiment of the present invention can deliver the voice of a deceased person to acquaintances and deliver information about family events and the like by voice.

On the other hand, the effects of the present invention are not limited to the above-mentioned effects, and various effects may be included within the range obvious to those skilled in the art from the description below.

1 is a diagram for explaining a system for providing a voice synthesis service according to an embodiment of the present invention.
2 is a block diagram schematically showing the configuration of an apparatus for providing a speech synthesis service according to an embodiment of the present invention.
FIG. 3 is a block diagram illustrating the voice synthesis service unit shown in FIG. 2 .
4 is a diagram for explaining a method of extracting voice characteristic information from a user's voice sample information according to an embodiment of the present invention.
5 is a flowchart illustrating a method of providing a voice synthesis service according to an embodiment of the present invention.
6 is a block diagram schematically showing the configuration of a terminal providing a voice synthesis service according to another embodiment of the present invention.

Hereinafter, an apparatus, a terminal and a method for providing a voice synthesis service according to an embodiment of the present invention will be described with reference to the accompanying drawings. In this process, the thickness of the lines or the size of the components shown in the drawings may be exaggerated for clarity and convenience of explanation. In addition, the terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of the user or operator. Therefore, definitions of these terms should be made based on the content throughout this specification.

Implementations described herein may be implemented in, for example, a method or process, an apparatus, a software program, a data stream, or a signal. Although discussed only in the context of a single form of implementation (eg, discussed only as a method), implementations of the discussed features may also be implemented in other forms (eg, as an apparatus or program). The apparatus may be implemented in suitable hardware, software and firmware, and the like. A method may be implemented in an apparatus such as, for example, a processor, which generally refers to a computer, a microprocessor, a processing device, including an integrated circuit or programmable logic device, and the like. Processors also include communication devices such as computers, cell phones, portable/personal digital assistants (“PDA”) and other devices that facilitate communication of information between end-users.

1 is a diagram for explaining a system for providing a voice synthesis service according to an embodiment of the present invention.

Referring to FIG. 1 , a system for providing a voice synthesis service according to an embodiment of the present invention may include a user terminal 100 and a service providing apparatus 200 , which may be connected through a communication network.

The user terminal 100 may receive voice sample information for a suggested word from the user, and transmit the received voice sample information to the service providing apparatus 200 . In this case, the user may read the suggestion provided from the service providing apparatus 200 , and the user terminal 100 may transmit voice sample information generated by the user's utterance of the suggested word to the service providing apparatus 200 . The user may record the voice for the suggested word through the microphone of the terminal possessed by the user rather than in a limited space (eg, a recording studio, a studio, etc.).

The user terminal 100 may be implemented in a smart phone, a computer, or a mobile phone. The text-to-speech synthesis terminal 100 may communicate with an external device (eg, a server device) including a communication unit.

The user terminal 100 may be an electronic device that can use a voice synthesis service program (application or applet) provided by the service providing apparatus 200 and can be applied to various wired and wireless environments. For example, the user terminal 100 may include a Personal Digital Assistant (PDA), a smart phone, a cellular phone, a Personal Communication Service (PCS) phone, a Global System for Mobile (GSM) phone, a Wideband CDMA (W-CDMA) phone, or a CDMA-2000 phone. It includes a phone, a Mobile Broadband System (MBS) phone, and the like.

The service providing apparatus 200 may extract the user's voice characteristic information from the voice sample information received from the user terminal 100 , and store the extracted voice characteristic information in the voice database 320 . In this case, the service providing apparatus 200 may implement a user's own voice as a TTS.

When the service providing apparatus 200 receives a voice synthesis request signal including text from the user, the user's voice characteristic information is extracted from the voice database 320, and the extracted voice characteristic information and text are applied to machine learning to apply the text can generate voice data for

The service providing apparatus 200 may be implemented as a computer capable of accessing a remote server or terminal through a communication network. Here, the computer may include, for example, navigation, a laptop equipped with a web browser, a desktop, and a laptop.

Meanwhile, in the embodiment of the present invention, it has been described that the service providing apparatus 200 converts text into voice data, but when a voice synthesis service program (application or applet) is installed in the user terminal 100, the user terminal 100 can be used to convert text into voice data.

Accordingly, a device for converting text into voice data will be referred to as a voice synthesis service providing device 300 and will be described.

2 is a block diagram schematically showing the configuration of an apparatus for providing a speech synthesis service according to an embodiment of the present invention, and FIG. 3 is a block diagram illustrating the speech synthesis service unit shown in FIG. 2 .

Referring to FIG. 2 , the apparatus 300 for providing a voice synthesis service according to an embodiment of the present invention includes a communication unit 310 , a voice database 320 , a voice feature extraction unit 330 , a voice synthesis service unit 340 , and a control unit 350 .

The communication unit 310 is configured to communicate with an external device through a communication network, and may transmit/receive various information such as voice sample information and a voice synthesis request signal. In this case, the communication unit 310 may be implemented in various forms, such as a short-range communication module, a wireless communication module, a mobile communication module, and a wired communication module.

The voice database 320 may store the user's voice feature information extracted by the voice feature extractor 330 . In this case, the voice database 320 may store user identification information and voice characteristic information, and the user identification information may include a user ID/password, and a phone number. The user's voice characteristic information (eg, an embedding vector representing the user's voice characteristic) stored in the voice database 320 may be provided to the voice synthesis service unit 340 during voice synthesis.

The voice database 320 may store a suggestion word (text) and a user's voice corresponding to the suggestion word. The suggestion word may be written in at least one language, and may include at least one of words, phrases, and sentences that can be understood by humans. Also, the voice stored in the voice database 320 may include voice data in which a plurality of speakers read a suggested word. The suggestion word and voice data may be previously stored in the voice database 320 or may be received from the communication unit 310 .

The voice feature extractor 330 may receive voice sample information based on the user's utterance of a suggested word, extract the user's voice feature information from the voice sample information, and store the voice feature information in the voice database 320 . Here, the voice sample information may include voice spectrum data representing information related to the user's voice characteristics. Voice characteristic information includes formant information, frequency (Log f0) information, LPC (Linear Predictive Coefficient) information, spectral envelope information, energy (Energy) information, firing rate (Pitch Period) information, and a log It may include spectrum (Log Spectrum) information and the like.

The voice feature extractor 330 may extract voice feature information using a voice processing method such as Mel Frequency Septral (MFC). Also, the voice feature extractor 330 may extract voice feature information by inputting voice sample information into a learned voice feature extraction model (eg, an artificial neural network).

The voice feature extractor 330 may represent the voice feature information as an embedding vector.

The voice feature extractor 330 may remove a noise component from the voice sample information before extracting the voice feature information from the voice sample information.

The voice feature extractor 330 may store the extracted user's voice feature information in the voice database 320 . Accordingly, when synthesizing the input text, the speaker's voice characteristic information stored in advance in the voice database 320 may be selected or designated, and the selected or designated speaker's voice characteristic information may be used for voice synthesis.

When receiving a voice synthesis request signal including text from the user, the voice synthesis service unit 340 extracts the user's voice characteristic information from the voice database 320, and applies the voice characteristic information and the text to machine learning to add the text to the text. It is possible to generate voice data for Here, machine learning may include a speech synthesis model (eg, pre-net, CBHG module, DNN, CNN+DNN, etc.). The speech synthesis model may convert the input text into a feature vector of speech data. In this case, the voice data may be voice data in which text is uttered according to the speaker's voice characteristic information. In other words, the speech synthesis model may output speech data in which the input text is uttered according to speech feature information of a specific speaker or a feature vector of the speech data.

The speech synthesis model may be an encoder-decoder-based neural network. Here, the neural network may include a plurality of layers, and each of the layers may include a plurality of neurons. Neurons of neighboring layers may be connected by synapses. According to learning, weights may be assigned to synapses, and parameters may include these weights.

As such, the speech synthesis model may be a text-to-feature vector or text-to-speech model composed of a sequence-to-sequence model. can The feature vector or speech data of speech data determined in the speech synthesis model may be used together with text as training data of the speech recognition model. Accordingly, it is possible to easily obtain learning data having various speech characteristics.

When the voice synthesis request signal is received, the voice synthesis service unit 340 may analyze the voice synthesis request signal to extract a speaker and text. The speech synthesis service unit 340 may configure a sentence input by embedding the extracted text. Here, embedding is a process for converting discontinuous symbol data such as sentences and speakers into feature vectors having continuous and various characteristics. Specifically, the speech synthesis service unit 340 decomposes the text into Hangul Jamo units to generate a Jamo unit input, and can number each of the split Jamo units. They are numbered according to the table. Through this, the speech synthesis service unit 340 may convert the text into numeric data. The speech synthesis service unit 340 generates a one-hot encoded vector sequence by one-hot encoding the text mapped to numeric data, and multiplies the one-hot encoded vector sequence with a sentence embedding matrix. It can be converted to a text feature vector.

Thereafter, the voice synthesis service unit 340 extracts voice characteristic information corresponding to the speaker from the voice database 320, and applies the extracted voice characteristic information and text to a voice synthesis model to convert the text into voice data. . That is, the speech synthesis service unit 340 may input the text feature vector and the embedding vector representing the speech feature information into the speech synthesis model to convert the text into speech data.

The voice synthesis service unit 340 may include an image template service unit 352 , a video editing service unit 354 , a voice message service unit 356 , and a dubbing service unit 358 as shown in FIG. 3 . there is.

The image template service unit 352 may provide an image template for each theme, receive editing information for an image template selected by a user, and store the edited image template. A user may set and edit a design of a shopping mall, etc. by using an image template for each theme provided through the image template service unit 352 .

The video editing service unit 354 provides a design template for editing a video input from a user or a pre-stored video, receives editing information of the video through the design template, and receives a video dubbing request including text and a dubber At the same time, it is possible to extract voice characteristic information of a dubber from the voice database 320, apply the extracted voice characteristic information and the text to machine learning to generate voice data for the text, and apply the generated voice data to a moving picture. there is.

That is, when receiving a video input from the user, the video editing service unit 354 may provide a design template for editing the video and store the edited video through the design template. At this time, the video editing service unit 354 uses a video template DB (not shown) based on a simple shortcut of about 30 seconds to perform simple modifications such as adding images, inserting effects, and adjusting positions, editing texts, and mixing sound sources. You can provide an editing tool that can do that.

The video editing service unit 354 may support the user's own voice or the voice of the voice actor character, and may dub the video using the user's own voice or the voice of the voice actor character.

The user can easily edit the user's own voice when producing YouTube for promotion through the video editing service unit 354 . In addition, when producing a promotional video within the SNS, the user can easily insert and edit comments through the video editing service unit 354 . In addition, through the video editing service unit 354, it is possible to effectively explain the product promotion of the company. In addition, the voice of the deceased may be delivered to acquaintances through the video editing service unit 354 . In addition, through the video editing service unit 354, information on family events, etc. may be delivered by voice.

When a text message is input by the user, the voice message service unit 356 may extract the user's voice characteristic information from the voice database 320 and apply the voice specific information to the text message to provide the text message as voice.

When a user needs a more emotional notification than when sending and receiving a text message, the user may synthesize his/her own voice in an image/video/music and the like through the voice message service unit 356 and transmit it. For example, through the voice message service unit 356, the wedding invitation may be transmitted to acquaintances by putting the voice of the groom/bride.

The dubbing service unit 358, upon receiving a request for dubbing an e-book including an e-book and a reader from the user, extracts the voice characteristic information of the reader from the voice database 320, and extracts the extracted voice characteristic information and the e-book can be applied to machine learning to generate voice data for e-books. The user may output the e-book in the voice of a person desired by the user through the dubbing service unit 358 .

For example, when text of a children's book is input, the dubbing service unit 358 may synthesize and provide the content of the children's book with the voice of the father or mother.

The apparatus 300 for providing a speech synthesis service according to an embodiment of the present invention may further include an input/output unit (I/O device; not shown). The input/output unit may directly receive an input from a user. Also, the input/output unit may output at least one of a voice, an image, and a text to the user.

Meanwhile, the voice feature extraction unit 330 and the voice synthesis service unit 340 may be implemented by a processor required to execute a program on a computing device, respectively. As such, the voice feature extraction unit 330 and the voice synthesis service unit 340 may be implemented as physically independent components, or may be implemented in a functionally separate form within one processor.

The control unit 350 controls operations of various components of the apparatus 300 for providing a voice synthesis service including the communication unit 310 , the voice database 320 , the voice feature extraction unit 330 , and the voice synthesis service unit 340 . , it may include at least one arithmetic unit, where the arithmetic unit is a general-purpose central processing unit (CPU), a programmable device device (CPLD, FPGA) implemented appropriately for a specific purpose, an application-specific semiconductor processing unit ( ASIC) or microcontroller chip.

4 is a diagram for explaining a method of extracting voice characteristic information from a user's voice sample information according to an embodiment of the present invention.

Referring to FIG. 4 , when the user terminal 100 receives a voice recording command through a voice synthesis service application and transmits the voice recording command to the service providing device (S410), the service providing device provides a suggestion (S420) .

The user terminal 100 receives voice sample information according to the user's utterance of a suggested word through a microphone (S430), and transmits the received voice sample information to the service providing apparatus (S440).

The service providing apparatus extracts voice characteristic information from the user's voice sample information (S450) and stores the voice characteristic information in the voice database 320 (S460). Here, the voice characteristic information includes formant information, frequency (Log f0) information, linear predictive coefficient (LPC) information, spectral envelope information, energy information, pitch period information, and log spectrum (Log Spectrum) information.

Meanwhile, in the embodiment of the present invention, it has been described that the service providing apparatus extracts the voice characteristic information from the voice sample information, but the user terminal 100 may also extract the voice characteristic information from the voice sample information.

5 is a flowchart illustrating a method of providing a voice synthesis service according to an embodiment of the present invention.

Referring to FIG. 5 , when a voice synthesis request signal including text is received in the voice synthesis service providing apparatus 300 ( S410 ), the voice synthesis service providing apparatus 300 receives the user's voice characteristics from the voice database 320 . Information is extracted (S420). Here, the voice synthesis request signal may include text, user identification information, and speaker information. Accordingly, the voice synthesis service providing apparatus 300 may extract speaker information by analyzing the voice synthesis request signal, and may extract voice characteristic information corresponding to the speaker information from the voice database 320 .

When step S420 is performed, the apparatus 300 for providing speech synthesis service applies the extracted speech feature information and the text to machine learning to generate speech data for the text (S430 and S440). At this time, the speech synthesis service providing apparatus 300 decomposes the text into Hangul Jamo units to generate a Jamo unit input, and may number each of the split Jamo units, and configure an index table in which the numbers for each Jamo correspond one-to-one. You can convert text to numeric data by numbering according to a table. The speech synthesis service providing apparatus 300 generates a one-hot encoded vector sequence by one-hot encoding the text mapped to numeric data, and combines the one-hot encoded vector sequence with a sentence embedding matrix. It can be multiplied into a text feature vector. Thereafter, the apparatus 300 for providing speech synthesis service extracts speech characteristic information corresponding to the speaker from the speech database 320, and inputs a text characteristic vector and an embedding vector representing the speech characteristic information to the speech synthesis model, can be converted into voice data.

When the user selects a video editing command, the voice synthesis service providing device 300 provides a design template for editing a video input from the user or a pre-stored video, and receives editing information of the video through the design template to view the video. Can be edited. In addition, when receiving a video dubbing request including text and a dubber, the voice synthesis service providing apparatus 300 extracts the voice characteristic information of the dubber from the voice database 320, and applies the extracted voice characteristic information and the text to machine learning. It is applied to generate voice data for the text, and the generated voice data can be applied to a moving picture.

In addition, when receiving a request for dubbing an e-book including an e-book and a reader from the user, the voice synthesis service providing apparatus 300 extracts voice characteristic information of the reader from the voice database 320 and extracts the extracted voice characteristics The information and the e-book may be applied to machine learning to generate voice data for the e-book.

6 is a block diagram schematically showing the configuration of a terminal providing a voice synthesis service according to another embodiment of the present invention.

Referring to FIG. 6 , a terminal 600 providing a voice synthesis service according to another embodiment of the present invention includes a terminal 600 , a voice recorder 620 , a voice feature extractor 630 , and a voice synthesis service unit ( 640 ) and a control unit 650 .

The voice recorder 620 may detect a connection of a voice call, convert voice during a call into voice data, and generate a recording file corresponding to the voice data. In this case, the voice recorder 620 may generate a recording file for the user's voice by filtering the voice of the callee from the call contents. The voice recorder 620 may register the user's voice in advance in order to filter the voice of the call party. In addition, the voice recorder 620 may analyze the frequency characteristics between the user's voice and the voice of the calling party, and may filter the voice of the called party by processing the voice as silence. In addition, the voice recorder 620 may occur when the user and the call partner speak at the same time, and the callee's voice may be similar to the user's voice. If it is out of the frequency range of , it can be recognized as the above-mentioned case and filtered.

The voice recorder 620 may not record in the call waiting state before the voice call connection, and may detect the voice call connection and convert the user's voice and the voice of the other party's voice into voice data during the call. For example, an analog audio signal may be converted into a digital signal.

The voice recorder 620 may temporarily create a recording file corresponding to the voice data. In this case, the voice recorder 620 may automatically generate the converted voice data as a recording file. For example, in the call waiting state before the recording file is created, a pre-information about the creation of the recording file can be output, and the information about the situation in which the recording file is being created while the audio call is connected and the recording file is being created may be output to a part of the display of the terminal 600 .

When the voice call is terminated, the voice recorder 620 may inquire of the user of the terminal 600 whether to store the recorded file, and may store the recorded file according to the user's response to the inquiry.

The voice recorder 620 may convert voice data corresponding to the recorded file into text data and store it in the terminal 600 together with the recorded file. In this case, the voice recorder 620 may convert the voice data into text data by applying a Speech to Text (STT) function to the recorded file.

The voice feature extractor 630 may extract the user's voice feature information from the recorded file generated for a preset period, and store the voice feature information in the terminal 600 . In this case, the voice feature extractor 630 may extract voice feature information by using a voice processing method such as Mel Frequency Septrel (MFC). Also, the voice feature extractor 630 may extract voice feature information by inputting voice sample information into a learned voice feature extraction model (eg, an artificial neural network). The voice feature extractor 630 may represent the voice feature information as an embedding vector. Since the voice feature extractor 630 performs the same operation as the voice feature extractor 330 of FIG. 2 , a detailed description thereof will be omitted.

The voice synthesis service unit 640 extracts the user's voice characteristic information from the terminal 600 when receiving a command to convert the voice signal provided from the terminal 600 into the user's voice, and machine learning the voice characteristic information and the voice signal. can be applied to generate voice data for a voice signal.

For example, a voice signal for various notifications, such as a message reception notification and a call reception notification provided by the terminal 600 , may be provided as a user's voice.

In addition, the voice synthesis service unit 640 extracts the user's voice characteristic information from the terminal 600 when receiving a voice synthesis request signal including text from the user, and applies the voice characteristic information and the text to machine learning to apply the text can generate voice data for That is, the speech synthesis service unit 640 may convert the text into speech data by inputting the text feature vector and the embedding vector representing the speech feature information to the speech synthesis model.

The terminal 600 according to an embodiment of the present invention may further include an input/output unit (I/O device; not shown). The input/output unit may directly receive an input from a user. Also, the input/output unit may output at least one of a voice, an image, and a text to the user.

Meanwhile, the voice recorder 620 , the voice feature extractor 630 , and the voice synthesis service unit 640 may be implemented by a processor required to execute a program on a computing device, respectively. As such, the voice recorder 620, the voice feature extraction unit 630, and the voice synthesis service unit 640 may be implemented as physically independent components, and may be implemented in a functionally separate form within one processor. may be

The control unit 650 controls operations of various components of the terminal 600 including the storage unit 610 , the voice recorder 620 , the voice feature extraction unit 630 , and the voice synthesis service unit 640 . The configuration may include at least one arithmetic unit, wherein the arithmetic unit is a general-purpose central processing unit (CPU), a programmable device device (CPLD, FPGA) implemented to suit a specific purpose, and an application-specific semiconductor processing unit (ASIC). ) or a microcontroller chip.

The terminal 600 configured as described above may be an electronic device that can use a voice recording and voice synthesis service program (application or applet) and can be applied to various wired and wireless environments. For example, the terminal 600 may include a Personal Digital Assistant (PDA), a smart phone, a cellular phone, a Personal Communication Service (PCS) phone, a Global System for Mobile (GSM) phone, a Wideband CDMA (W-CDMA) phone, or a CDMA-2000 phone. , including a Mobile Broadband System (MBS) phone and the like.

As described above, the apparatus, terminal, and method for providing a speech synthesis service according to an embodiment of the present invention may convert an input text into a user's voice and provide the converted text.

The apparatus, terminal and method for providing a voice synthesis service according to another embodiment of the present invention may provide the user's own voice by dubbing the video, such as YouTube, SNS promotional video, and corporate promotional video.

In an apparatus, terminal, and method for providing a voice synthesis service according to another embodiment of the present invention, a user can listen to an e-book in the voice of a person desired by the user.

An apparatus, terminal, and method for providing a voice synthesis service according to another embodiment of the present invention can deliver the voice of a deceased person to acquaintances and deliver information about family events and the like by voice.

Although the present invention has been described with reference to the embodiment shown in the drawings, this is merely an example, and those skilled in the art to which various modifications and equivalent other embodiments are possible. will understand

Accordingly, the true technical protection scope of the present invention should be defined by the following claims.

100: user terminal
200: service providing device
300: Speech synthesis service providing device
310: communication department
320: voice database
330, 630: voice feature extraction unit
340, 640: speech synthesis service department
350, 650: control unit
600: terminal
610: storage
620: voice recording unit

Claims (10)

voice database;
a voice feature extracting unit for receiving voice sample information of a user for a suggested word, extracting voice characteristic information of the user from the voice sample information, and storing the voice characteristic information in the voice database; and
When receiving a voice synthesis request signal including text from the user, extracting the user's voice characteristic information from the voice database, and applying the voice characteristic information and the text to machine learning to generate voice data for the text speech synthesis service department
A voice synthesis service providing device comprising a.
According to claim 1,
The voice synthesis service unit,
Provides a design template for editing a video input from the user or a pre-stored video, receives editing information of the video through the design template, and receives a video dubbing request including text and dubbing person information, the voice Speech synthesis, characterized in that extracting voice characteristic information of the dubbing person from a database, applying the voice characteristic information and the text to machine learning to generate voice data for the text, and applying the voice data to the moving picture Service providing device.
According to claim 1,
The voice synthesis service unit,
When receiving an e-book dubbing request including an e-book and reader information from the user, extracting voice characteristic information of the reader from the voice database, and applying the voice characteristic information and the e-book to machine learning, A voice synthesis service providing apparatus, characterized in that for generating voice data for the e-book.
receiving, by a voice feature extracting unit, voice sample information of a user for a suggested word, extracting voice feature information of the user from the voice sample information, and storing the voice feature information in a voice database; and
When the voice synthesis service unit receives a voice synthesis request signal including text from the user, extracts the user's voice characteristic information from the voice database, and applies the voice characteristic information and the text to machine learning to make a voice for the text Steps to generate data
A method of providing a speech synthesis service comprising a.
5. The method of claim 4,
When the voice synthesis service unit receives a video editing command from the user, it provides a design template for editing a video input from the user or a pre-stored video, and receives editing information of the video through the design template to receive the video editing information. editing the video; and
When the voice synthesis service unit receives a request for dubbing a video including text and dubber information, extracts voice characteristic information of the dubber from the voice database, and applies the voice characteristic information and the text to machine learning to add to the text The method for providing voice synthesis service, characterized in that it further comprises the step of generating voice data for the video, and applying the voice data to the moving picture.
5. The method of claim 4,
When the voice synthesis service unit receives an e-book dubbing request including an e-book and reader information from the user, extracts voice characteristic information of the reader from the voice database, and stores the voice characteristic information and the e-book The method for providing a voice synthesis service, characterized in that it further comprises the step of generating voice data for the e-book by applying it to machine learning.
storage;
a voice recorder for detecting a connection of a voice call, converting voice during a call into voice data, and generating a recording file corresponding to the voice data; and
A voice feature extracting unit for extracting the user's voice characteristic information from the recorded file generated for a predetermined period of time and storing the voice characteristic information in the storage unit;
The voice recording unit,
and converting the voice data corresponding to the recorded file into text data and storing the converted voice data into the storage unit together with the recorded file.
8. The method of claim 7,
The voice recording unit,
A terminal, characterized in that by filtering the voice of the called party from the call contents, and generating a recording file for the voice of the user.
8. The method of claim 7,
When receiving a command to convert a voice signal provided by the terminal into a user voice, the user's voice characteristic information is extracted from the storage unit, and the voice characteristic information and the voice signal are applied to machine learning to respond to the voice signal. The terminal characterized in that it further comprises a voice synthesis service unit for generating data.
10. The method of claim 9,
The voice synthesis service unit,
When receiving a voice synthesis request signal including text from the user, extracting the user's voice characteristic information from the storage unit, and applying the voice characteristic information and the text to machine learning to generate voice data for the text A terminal, characterized in that.

Images