JP5787794B2 - Speech synthesis system, speech conversion support device, and speech conversion support method - Google Patents

Description

Embodiments described herein relate generally to a speech synthesis system , a speech conversion support apparatus, and a speech conversion support method used for a service that converts characters such as text (characters, symbols, etc.) and graphics into speech.

  In recent years, for example, on the Internet, a service for converting characters into speech has been started, and a speech synthesizer is used for this service.

  In general, a speech synthesizer converts a text (character string) of text input by a user from a terminal into speech synthesis waveform data and returns a speech signal or a speech file to the terminal.

  By the way, if the text entered by the user is Hiragana, for example, there are homonyms, that is, words that are the same in the text but different in pronunciation due to semantic differences, and speech conversion is performed for texts containing such words When the process is performed, an audio signal or an audio file having a sound different from the user's intention may be returned to the terminal.

Japanese Utility Model Publication No. 06-28900

  In this case, the user causes the speech synthesizer to perform re-conversion of the corrected text, that is, the speech conversion, but such a re-processing does not only impose a great load on the speech synthesizer. Since it takes a certain amount of time, it is a process that you do not want to do.

The problem to be solved by the present invention is to provide a speech synthesis system, a speech conversion support apparatus, and a speech conversion support method that can eliminate an error in pronunciation of speech obtained as a result of speech conversion of a text of a text. .

The speech synthesis system and the speech conversion support apparatus according to the embodiment include a storage unit, a search unit, a test playback unit, and a control unit. The storage unit stores voice data that has been converted by the voice conversion device and conversion source text data corresponding to the voice data. The accepting unit uses the text data of the designated test playback portion among the text data to be converted newly input from the terminal as a keyword, and whether or not the voice data of the test playback location exists in the storage unit Search for. If the audio data exists in the storage unit as a result of the search by the search unit, the test reproduction unit performs test reproduction of the audio data. When the control unit receives an instruction from the terminal that the audio data test-reproduced by the test reproduction unit is correct, the control unit is obtained by converting the text data excluding the test reproduction part to the audio conversion device. The voice data and the existing voice data that have been test-reproduced are combined and transferred to the terminal.

1 is a diagram illustrating an overall configuration of a speech synthesis system according to an embodiment. It is a block diagram of an application server. It is a figure which shows an example of index information. It is a flowchart which shows operation | movement of this speech synthesis system. It is a flowchart which shows operation | movement of this speech synthesis system. It is a flowchart which shows the search operation | movement of the existing audio | voice file.

Hereinafter, embodiments will be described in detail with reference to the drawings.
(First Embodiment) FIG. 1 is a diagram showing the configuration of a speech synthesis system according to an embodiment.

  As shown in FIG. 1, the speech synthesis system of this embodiment includes a computer 1 (hereinafter referred to as “user PC1”), which is a terminal operated by a service user (hereinafter referred to as “user”), and a speech conversion apparatus. A speech synthesis server 3 which is a computer, a computer 2 (hereinafter referred to as “application server 2”) as a speech conversion support device, and these devices are connected via a network 4.

  The speech synthesis server 3 has a speech synthesis engine and converts text data into speech data. Specifically, the speech synthesis server 3 converts the intermediate file (a pair of text data and an accent symbol) sent from the application server 2 into speech data (hereinafter referred to as “speech file”), and then converts the intermediate file into the application server. Return to 2.

  The application server 2 is interposed between the voice synthesis server 3 and the user PC 1 and exchanges text data, intermediate files, and voice files. Specifically, the application server 2 transfers to the user PC 1 the voice data obtained by converting the text data input from the user PC 1 to the voice synthesis server 3.

  As shown in FIG. 2, the application server 2 includes a graphic user interface unit 21 (hereinafter referred to as “GUI unit 21”), a memory 22, an intermediate file generation unit 23, and an index information storage unit for managing cached data. 24, a cache data storage unit 25, a search unit 26, a communication processing unit 27, a data processing unit 28, a registration unit 29, a test reproduction unit 30, and the like.

  The application server 2 searches the audio data cached (stored) in the cache data storage unit 25 using the index information in the index information storage unit 24 using the text data input from the user PC 1 as a keyword (search key). If there is a hit, the cached voice file is read out and returned to the requesting user PC 1 without requesting the voice synthesis server 3 for voice synthesis.

  In other words, the application server 2 checks whether or not it is cached in its own hard disk device, and if it is not cached, it outputs the text data input from the user PC 1 to the speech synthesis server 3, and As a response, the voice file converted (voice synthesized) by the voice synthesis server 3 is acquired and sent to the user PC 1.

  The GUI unit 21 displays a screen for logging in to the application server 2 from the user PC 1, a search screen, a registration screen, and the like, accepts a voice synthesis request from the user PC 1, input of text data, etc., and receives a voice as a response to the request. Send the file to user PC1. For example, on the search screen, the GUI unit 21 functions as a reception unit that receives a designated test reproduction portion of text data to be converted newly input from the user PC 1.

  That is, the GUI unit 21 realizes an input / output interface between the user PC 1 and the application server 2.

  The memory 22 is used as a work area when the data processing unit 28, the search unit 26, the registration unit 29, etc. execute the respective processes, and as a temporary storage area for the intermediate file created when a conversion request is made.

  The intermediate file generation unit 23 divides the text data to be converted input from the user PC 1 into units of words or phrases, and the divided text data is not cached, or the pronunciation is not correct by the user in the test playback. Refer to the speech conversion dictionary using what is designated as a keyword, read the corresponding accent symbol from the speech conversion dictionary, generate an intermediate file of text data and accent symbol pairs, and generate a speech synthesis server 3 is stored in the memory 22 as data for requesting conversion to 3. This intermediate file is transmitted to the speech synthesis server 3 as original data for speech synthesis.

  For text data that has already been approved by the user and that has been flagged as not requiring conversion (confirmed), no intermediate file is generated because voice conversion is not performed.

  The cache data storage unit 25 stores files (voice data files, text data files, intermediate data files, etc.) that have been converted into the previous (past) format.

  That is, the cache data storage unit 25 stores the voice data converted by the voice synthesis server 3 and the conversion source text data corresponding to the voice data.

  The index information storage unit 24 includes a plurality of pieces of index information with different narrowing ranges for searching past data (files) cached (saved) in the cache data storage unit 25 in descending order of affinity with the user. They are ranked and stored.

  The index information storage unit 24 stores a voice conversion dictionary for each user, a conversion history, and the like. The dictionary stores index information indicating the storage destination of the voice data uniquely registered by the user. The conversion history is obtained by leaving the index information when each user performs voice conversion as the conversion history.

  The index information is a pair of text data of the conversion source corresponding to the information (link destination) indicating the storage destination of the audio data and an accent symbol (accent information) that specifies the strength of the sound when this text data is converted into audio. This is a reference dictionary in which (set) is stored. The voice data may be voice data converted (voice synthesis) by the voice synthesis server 3. This voice conversion dictionary is used by the intermediate file generation unit 23.

  That is, the index information storage unit 24 stores index information for managing the data cached in the cache data storage unit 25. The index information will be specifically described with reference to FIG.

  The search unit 26 uses the text data of the designated test playback location among the text data to be converted newly input from the user PC 1 as a keyword, and whether or not the voice data of the test playback location exists in the cache data storage unit 25. Search for.

  The search unit 26 searches the registration dictionary 42 for each user ID, the conversion history 43 for each user ID, and the server usage history 44 in the order of preset priority. The search unit 26 ranks the plurality of pieces of index information in the index information storage unit 24 based on the text data input from the request source (first file collation, second file collation, third file collation, fourth Search in the order of (file verification) and check whether there is an existing audio file (audio data).

  In the first file collation, the search unit 26 searches the voice file (voice data) at the test playback location with reference to the registered dictionary 42 (User1) of the user ID of the logged-in user using the text data at the test playback location as a keyword. To do.

  In the second file collation, the search unit 26 searches the voice file (voice data) at the test playback location with reference to the conversion history 43 (User1) of the user ID of the logged-in user using the text data at the test playback location as a keyword. To do.

  In the third file collation, the search unit 26 searches the voice file (voice data) at the test playback location by referring to the registered dictionaries 42 (User2...) Of a plurality of users using the text data at the test playback location as a keyword.

  In the fourth file collation, the search unit 26 searches the audio file (audio data) at the test reproduction location by referring to the conversion history 43 (User2...) Of the user IDs of a plurality of users using the text data at the test reproduction location as a keyword. To do.

  In addition, when the test playback location is specified in the text data input from the user PC 1 on the screen displayed on the user PC 1 by the GUI unit 21, the search unit 26 stores the past of the test playback locations accepted by the GUI unit 21. It is searched whether converted data, that is, an audio file (audio data) exists in the cache data storage unit 25.

  As a result of the search by the search unit 26, the test playback unit 30, when past converted data cached (saved) in the cache data storage unit 25, that is, an existing audio file (audio data) exists, Test play an audio file (audio data).

  The communication processing unit 27 exchanges data with the speech synthesis server 3 by TCP (HTTP) communication.

  As a result of the search by the search unit 26, the data processing unit 28 is cached from the past data storage destination indicated by the index information in which text data (also referred to as a search key or keyword) exists, that is, from the cache data storage unit 25. The audio file included in the existing file is read out and transferred to the user PC 1 as the conversion request source.

  When the user gives an instruction on the screen of the user PC 1 that the sound of the test data reproduced by the test reproduction unit 30 is correct and the data processing unit 28 receives the instruction from the user PC 1, the data The processing unit 28 functions as a control unit that combines the voice data obtained by converting the text data excluding the test playback portion into the voice synthesis server 3 and the existing voice data that has been tested and played back to the user PC 1. .

  On the other hand, if no keyword exists in any index information, the data processing unit 28 sends the intermediate file generated by the intermediate file generation unit 23 based on the input text data and stored in the memory 22 to the speech synthesis server 3. Then, the voice file converted (voice synthesized) by the voice synthesis server 3 is transferred to the requesting user PC 1. It is also possible to send input text data instead of an intermediate file.

  The registration unit 29 uses the dictionary registration screen displayed by the GUI unit 21 to store information (voice data, text data, and accent marks) that the user has entered or edited in the registration dictionary 42 for each user ID in the memory 22. At the same time, the management information of the registration dictionary 42 in the memory 22 is registered in the index information storage unit 24.

  As shown in FIG. 3, the index information storage unit 24 stores a plurality of pieces of index information (registered dictionaries 42 for each user ID (User 1, 2...)) Associated with each user ID in the user ID table 41, for each user ID. Conversion history 43 (User 1, 2...) And usage index 44 (hereinafter referred to as “server usage history 44”) of all users of the application server 2) are stored. The conversion history 43 of one user who has logged into the application server 2 is referred to as a first conversion history. A conversion history of a plurality of users who have logged into the application server 2 (for example, all users who have logged in so far) is referred to as a second conversion history.

  In the user ID table 41, a user ID that is identification information of a user who can log in to the application server 2 is set. The user identification information includes not only the user ID but also a password.

  The registration dictionary 42 (User 1, 2,...) For each user ID includes text data (this is referred to as “text”) uniquely registered by the user, an intermediate file that is a pair of text and its access symbol, and The storage destination index indicating the storage destination of the audio file corresponding to these data is stored.

  The registration dictionary 42 (User 1, 2...) For each user ID is set to the first and second ranks (first and second ranks) as the ranks for the search, and the search unit 26 searches the cache data. When you refer to it first. In the dictionary, the first rank is an intermediate file, and the second rank is text. The search order as index information is first.

  The conversion history 43 for each user ID (User 1, 2,...) Includes text when this voice conversion function is used for each logged-in user ID, an intermediate file that is a pair of text and its access symbol, and The storage destination index indicating the storage destination of the audio file corresponding to these data is stored. Text, intermediate files, save destination indexes, etc. are called usage history.

  The conversion history 43 (User 1, 2,...) For each user ID is set to the third and fourth ranks (third and fourth ranks) as the ranks for the search, and the search unit 26 searches the cache data. Sometimes referred to 3rd and 4th. In the history, the third rank is an intermediate file, and the fourth rank is text. The search order as index information is second.

  The server usage history 44 stores usage histories of all users when the user logs in to the application server 2 and uses the voice conversion function. The server usage history 44 is set to the fifth and sixth ranks (fifth and sixth ranks) as the rank in the search, and is referred to the fifth and sixth when the search unit 26 searches the cache data. Is done. In the history, the fifth rank is an intermediate file, and the sixth rank is text. The search order as index information is the third.

Next, the operation of the speech synthesis system of this embodiment will be described with reference to the flowcharts of FIGS. First, the operation of the entire system will be described with reference to the flowcharts of FIGS.
In the case of the speech synthesis system of this embodiment, when the user inputs a predetermined URL from the user PC 1 and accesses the application server 2, the GUI unit 21 displays a login screen on the user PC 1, so that the user displays the displayed login screen. The login information such as “User1”, for example, is input in the input field (step S101 in FIG. 4). In addition, a password may be input as login information.

  Then, the GUI unit 21 determines whether the login information has been registered by checking the input login information with the user ID in the user ID table 41 of the memory 22 (step S102). In the case (Yes in step S102), login to the application server 2 is permitted and a voice conversion screen is displayed (step S103).

  When the user moves the cursor to the character input field on the voice conversion screen and inputs a character to be converted (text data) by key input (step S104), the GUI unit 21 accepts the input and receives the input text data. It is displayed in the character input field (step S105).

  When the button for voice file creation instruction displayed on the voice conversion screen is pressed (Yes in step S106), the routine proceeds to a routine for voice file creation processing.

  On the other hand, the test playback location in the text data displayed in the character input field is specified by the user by an operation such as range specification (such as a mouse drag operation), and the test playback button displayed on the voice conversion screen is pressed. Then (Yes in step S107), the GUI unit 21 accepts the pressing of the test playback button and notifies the search unit 26 of it.

  The search unit 26 searches whether or not an existing audio file exists in the device (memory 22 or index information storage unit 24) (step S108).

  More specifically, with reference to the index information storage unit 24, whether or not the index information of the file at the test reproduction location exists by using the text data at the designated test reproduction location among the text data to be converted as a keyword. If the index information exists, “existing audio file exists” is determined (step S109), and the audio file at the test playback location is read from the cache data storage unit 25 according to the index information and cached in the playback work area of the memory 22 ( Step S110). This audio file is <data 1>.

  When there is an existing file as a result of the search by the search unit 26, the test playback 30 reads the existing audio file at the test playback location cached in the playback work area and plays the audio from the speaker of the user PC 1 (step S30). S111).

  When the user who has heard this voice confirms the pronunciation and selects a button such as “OK” on the voice conversion screen on the assumption that the pronunciation is correct (Yes in step S112), the data processing unit 28 determines the portion of the test playback portion. The text data is confirmed (step S113), and a conversion unnecessary flag is attached to the confirmed text data. If there is the next text data to be tested, the process returns to S104 and the above process is repeated.

  In this way, when there is no next text data to be test-played, test playback of all test playback locations is completed, and the button for audio file creation instruction is pressed (step S115), the data processing unit 28 The text data of the text input in the character input field is divided into words or clauses (step S116), and the text data is transferred to the search unit 26 in the divided units, and is stored in this apparatus in the same manner as in step S108. The existing audio file is searched (step S117). Details of the search operation for the existing audio file will be described later with reference to FIG. Further, in this case, text data that has already been approved by the user and has been subjected to test reproduction with a conversion unnecessary (confirmed) flag added is not subject to search because it is not converted to speech.

If there is an existing audio file as a result of the search, the search unit 26 determines that “existing audio file exists” (Yes in step S118), reads out the audio file from the cache data storage unit 25 according to the index information, and reproduces the work area of the memory 22 (Step S119). This audio file is <data 2>.
And

  As a result of the search, if the existing voice file does not exist in the apparatus (No in step S118), the data processing unit 28 generates an intermediate file of the searched divided text data and transfers it to the voice synthesis server 3. (Step S120), the voice file converted by the voice synthesis server 3 is acquired (Step S121).

  Then, the obtained audio file (product) is cached in the reproduction work area of the memory 22 (step S122). This audio file is <data 3>.

  Then, the data processing unit 28 combines the audio files cached in the playback work area (<data 1>, <data 2>, <data 3>) according to the divided order (step S123), and the audio file as written. And the combined audio file is reproduced from the speaker of the user PC 1 (step S124).

  The user who has heard the sound confirms the pronunciation of the reproduced sound, and if the pronunciation is incorrect, an editing screen displaying the intermediate file (text data and accent mark) of the sound file is displayed (step S126). A message such as “Please correct the accent symbol in the wrong place” is displayed to prompt the user to correct it. When the user corrects the accent symbol (step S127), the process returns to step S124 to reproduce the voice.

  As a result of the user confirming the pronunciation, if the pronunciation is correct and the sound saving button on the screen is selected and operated (step S128), the data processing unit 28 displays a screen for designating the saving destination, and the saving destination is designated. If so (step S129), the audio file is transferred to the storage location of the designated user PC 1 and stored (step S130).

  Thereafter, the process is changed depending on whether or not to continue the process. For example, if the process end operation is not performed, it is assumed that the process is continued (Yes in step S131), the process returns to the process in step S104 and waits for the next text input.

  When the process end operation is performed, the data processing unit 28 stores the audio file in the cache data storage unit 25 that is a history recording area (step S132), assuming that the process is not continued (No in step S131). Information on the storage destination is registered in the index information storage unit 24 (step S133), and a series of text / speech conversion processing ends.

Here, the details of the search operation (step S108, step S117) of the existing audio file by the search unit 26 will be described with reference to FIG.
The search unit 26 uses the text data as a keyword to search the index information in the order of preset priorities (first file verification, second file verification, third file verification, fourth file verification). The storage location of the audio file is specified, and the existing audio file is read from the specified storage location.

  In this case, as shown in FIG. 6, the search unit 26 uses the text data as a keyword, and the user of the login user who is the target of the first file verification among the index information stored in the index information storage unit 24. By searching the index (intermediate file, text, etc.) of the registration dictionary 42 (see FIG. 3) of the ID (User1) (step S201) and searching the storage destination that is the link destination of the searched index (step S202). Read audio file.

  If no matching information is found as a result of the first file collation, the retrieval unit 26 records history data (intermediate) of the conversion history 43 (see FIG. 3) of the user ID (User1) of the login user who is the second file collation target. (File and text etc.) are searched (step S203), and the storage destination that is the link destination of the searched history data is searched (step S204), and the audio file is read out.

  If no matching information is found as a result of the second file collation, the search unit 26 indexes the registration dictionary 42 (see FIG. 3) of the user ID (User2...) Of another user who is the third file collation target (see FIG. 3). An intermediate file, text, etc.) are searched (step S205), and a storage destination that is a link destination of the searched index is searched (step S206), thereby reading out an audio file.

  If no matching information is found as a result of the third file collation, the retrieval unit 26 records the history data of the conversion history 43 (see FIG. 3) of the user ID (User2...) Of another user who is the fourth file collation target. (Intermediate file and text, etc.) are searched (step S207), and the storage destination that is the link destination of the searched history data is searched (step S208), thereby reading the audio file.

  As described above, according to this embodiment, a part where erroneous conversion has occurred partially in the voice conversion processing so far is confirmed and confirmed in advance by test reproduction instead of after the final conversion processing. In this way, it is possible to eliminate an error in pronunciation of speech obtained as a result of speech conversion of the text of the text. Further, this can reduce the time required for voice conversion.

  In addition, by obtaining voice data from the past converted data stored in the application server 2 without requesting the voice synthesis server 3 for voice conversion, the load on the voice synthesis server is reduced and the processing speed is increased. Can be improved. In addition, the amount of data to be converted by the speech synthesis server 3 can be reduced. In addition, the correction location can be minimized, which ultimately shortens the time required for voice conversion by the user.

  The described embodiments are presented by way of example and are not intended to limit the scope of the invention. The novel embodiment can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. The above-described embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and equivalents thereof.

  Further, each component shown in the above embodiment may be realized by a program installed in a storage such as a hard disk device of a computer, and the program is stored in a computer-readable electronic medium: electronic media, The computer may realize the functions of the present invention by causing a computer to read a program from an electronic medium. Examples of the electronic medium include a recording medium such as a CD-ROM, flash memory, and removable media. Further, the configuration may be realized by distributing and storing components in different computers connected via a network, and communicating between computers in which the components are functioning.

  DESCRIPTION OF SYMBOLS 1 ... User PC, 2 ... Application server, 3 ... Speech synthesis server 31 ... Graphic user interface part (GUI part), 22 ... Memory, 23 ... Intermediate file generation part, 24 ... Index information storage part, 25 ... Cache data storage part , 25 ... cache data storage unit, 26 ... search unit, 27 ... communication processing unit, 28 ... data processing unit, 29 ... registration unit, 30 ... test playback unit, 41 ... user ID table, 42 ... registration dictionary for each user ID 43 ... Conversion history for each user ID, 44 ... Server usage history.

Claims (7)

In a voice conversion support device connected via a network to a user terminal and a voice conversion device that converts text data into voice data,
A storage unit in which the voice data converted by the voice converter and the text data of the conversion source corresponding to the voice data are stored;
Retrieval for searching whether or not voice data of the test playback location exists in the storage unit, using the text data of the specified test playback location as a keyword among the text data to be converted newly input from the terminal And
As a result of the search by the search unit, when the audio data is present in the storage unit, a test playback unit that performs test playback of the audio data;
When receiving an instruction from the terminal that the audio data test-reproduced by the test reproduction unit is correct, the audio data obtained by converting the text data excluding the test reproduction location to the audio conversion device and the test A voice conversion support device comprising: a control unit that combines the reproduced existing voice data and transfers the data to the terminal. The storage unit is provided with a dictionary for each user in which index information indicating a storage destination of voice data registered by a user who has logged in to the device is stored.
The search unit
2. The voice conversion support device according to claim 1, wherein the voice data of the test reproduction portion is searched by referring to the dictionary of the logged-in user by using the text data of the test reproduction portion as a keyword. The storage unit is provided with a first conversion history in which index information indicating a storage destination of converted audio data that has been previously converted by a user who has logged in to the device is stored;
The search unit
The speech conversion support device according to claim 1 or 2, wherein the text data of the test reproduction part is searched for the voice data of the test reproduction part with reference to the first conversion history using the text data of the test reproduction part as a keyword. The storage unit is provided with a plurality of user dictionaries in which index information indicating storage destinations of voice data registered by a plurality of users logged in to the apparatus is stored,
The search unit
4. The speech conversion support device according to claim 1, wherein the text data of the test playback portion is searched for the voice data of the test playback portion with reference to the plurality of user dictionaries using the text data of the test playback portion as a keyword. The storage unit is provided with a plurality of second conversion histories in which index information of a plurality of users indicating storage destinations of the converted voice data that the plurality of users who have logged in to the apparatus previously instructed to convert is stored. And
The search unit
5. The speech conversion support device according to claim 1, wherein the speech data of the test playback location is searched by referring to the plurality of second conversion histories using the text data of the test playback location as a keyword. 6. A user terminal, a voice conversion device that converts text data into voice data, and a voice conversion support device that transfers voice data obtained by converting text data input from the terminal to the voice conversion device, to the terminal In a speech synthesis system in which and are connected via a network,
The voice conversion support device
A storage unit that stores the voice data converted by the voice conversion support device and the text data of the conversion source corresponding to the voice data;
Retrieval for searching whether or not voice data of the test playback location exists in the storage unit, using the text data of the specified test playback location as a keyword among the text data to be converted newly input from the terminal And
As a result of the search by the search unit, when the audio data is present in the storage unit, a test playback unit that performs test playback of the audio data;
When receiving an instruction from the terminal that the audio data test-reproduced by the test reproduction unit is correct, the audio data obtained by converting the text data excluding the test reproduction location to the audio conversion device and the test A speech synthesis system comprising a control unit that combines the reproduced existing voice data and transfers the combined data to the terminal. In a speech conversion support method in a speech conversion support device connected via a network to a user terminal and a speech conversion device that converts text data into speech data,
The voice conversion support device stores the voice data converted by the voice conversion device and the conversion source text data corresponding to the voice data,
The text data of the specified test playback location among the text data to be converted newly input from the terminal is used as a keyword, and the voice data of the test playback location exists in the converted audio data stored. Whether or not the voice conversion support device searches,
As a result of the search, when the audio data at the test reproduction location is present in the stored converted audio data, the audio conversion support device performs test reproduction of the audio data,
When an instruction is received from the terminal that the test-played voice data is correct, the voice data obtained by converting the text data excluding the test playback portion into the voice converter and the existing voice that has been test-played A voice conversion support method in which the voice conversion support device combines data and transfers the data to the terminal .

Images