JP2008180750A - Voice labeling support system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a voice labeling support system capable of unifying quality of labeling operation in a specified level without depending on labeler's know-how. <P>SOLUTION: A label image server 200 includes a search section 203 which receives a search label string, and which searches a label image corresponding to that from a label image DB (database) 204a. An operation terminal 300 includes a search request section 301 which requests searching of the label image to the search section 203, and a display section 302. The search request section 301 transmits the label string of voice for performing labeling operation to a label image server 200, and a search section 203 receives the label string from the search request section 301, and searches entry corresponding to the label string from the label image database 204a, and returns the label image associated with the label string to the operation terminal 300. The search request section 301 makes the display section 302 to perform screen display of the label image. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a system that supports labeling work performed when speech synthesis is performed by a corpus-based speech synthesis method.

When speech synthesis is performed by a corpus-based speech synthesis method, a speech database is constructed in advance by a set of speech segments that are read out by a certain speaker as an arbitrary word or sentence. When executing speech synthesis, a suitable speech segment is selected from the speech database, and a final synthesized speech is obtained by waveform connection processing.
In this way, in the corpus-based speech synthesis method, since the quality of speech units affects the quality of the final synthesized speech, it is important to obtain speech units with good quality.

As a method of obtaining a speech unit, a speech is recorded in advance by a certain speaker, and a delimiter code is given to a position preferable as a speech unit in the speech waveform while referring to the speech waveform and the actual speech ( There is a method of obtaining a speech segment by performing a labeling operation.
This labeling work is roughly classified into manual labeling in which an experienced worker manually performs labeling and automatic labeling by a computer or the like.

Here, “the boundary error of automatic labeling is made small. As a technology for the purpose of the above, “the input audio signal is divided into a plurality of bands for each frame by Mel frequency division (S1), the power of each band is obtained, and the audio signal energy of each frame is obtained and included. An acoustic feature vector is generated (S2), an HMM (Hidden Markov Model) for each phoneme or phoneme boundary is created in advance using this type of acoustic feature vector, and a previously known phoneme in the input speech signal is generated. Alternatively, the HMM sequence corresponding to the phoneme boundary, the previously obtained feature vector sequence, and the likelihood are calculated so as to maximize the likelihood (S3), and information representing the phoneme or phoneme boundary for each frame of the speech signal at that time (Label) is assigned (S4). Is proposed (Patent Document 1).
In addition, it has been reported that label errors due to manual labeling are small between labelers having sufficient experience (2 to 8 years) (Non-patent Document 1).

JP 2004-77901 A (summary) Tsuyoshi Kawai, Tomoki Toda, "Evaluation of automatic phoneme segmentation for waveform-connected speech synthesis", IEICE Transactions, Vol. SP2002-170, pp. 5-10, 01.2003 (accepted)

Although the accuracy of labeling by the automatic labeling technology as described above has been improved, there may be places where the accuracy is still insufficient. In such cases, manual labeling by experienced labelers is performed.
In general, manual labeling is a very time-consuming operation, and there are cases where multiple labelers work together. In such cases, labeling is largely based on personal know-how. Position accuracy will vary.
A decrease in the accuracy of the label position leads to a decrease in the quality of the speech segment, and finally affects the quality of the synthesized speech.
Therefore, an audio labeling support system that can unify the quality of labeling work at a certain level without relying on the know-how of the labeler has been desired.

An audio labeling support system according to the present invention includes:
A label image server comprising storage means for storing a label image DB for holding a label image for each label row of the phonemic environment;
A work terminal for labeling work,
An audio labeling support system comprising:
The label image server
A search unit that receives a search label string and searches the label image DB for a corresponding label image;
The work terminal is
A search request unit that requests the search unit to search for a label image;
A display for displaying the label image on the screen;
With
The search request unit
Send the label sequence of the audio to be labeled to the label image server,
The search unit
A label string is received from the search request unit, an entry corresponding to the label string is searched from the label image DB, and a label image associated with the label string is returned to the work terminal,
The search request unit
A label image is received from the search unit, and the label image is displayed on the screen on the display unit.

According to the audio labeling support system according to the present invention, the label image held in the label image DB can be displayed on the work terminal and the labeling work can be performed while referring to the label image. Regardless of the difference, the accuracy of the label position can be improved uniformly.
As a result, the speech DB in the corpus-based speech synthesis method and the quality of synthesized speech generated using the speech DB can be improved.

Embodiment 1 FIG.
FIG. 1 shows a configuration of an audio labeling support system according to Embodiment 1 of the present invention.
The voice labeling support system of FIG. 1 includes a label image registration terminal 100, a label image server 200, and a labeling work terminal 300. These are connected via the network 400.

The label image registration terminal 100 is a terminal for registering a label image in the label image server 200, and includes a label string transmission unit 101 and a label image transmission unit 102.
The label sequence transmission unit 101 transmits the phoneme label sequence to the label image server 200 and requests the label image server 200 to check whether or not a label image corresponding to the phoneme label sequence has already been registered.
The label image transmission unit 102 transmits a label image to the label image server 200 and requests to register the label image.

The label image server 200 is a server for holding a label image and returning the corresponding label image when receiving a label image acquisition request from the labeling work terminal 300. The label image determination unit 201, the label image registration unit 202, a label image search unit 203, and a storage unit 204.
The label registration determination unit 201 receives a phoneme label string, determines whether a label image corresponding to the phoneme label string is held in a label image DB (abbreviation of Database, hereinafter the same) 204a described later, and the result. Reply.
The label image registration unit 202 receives a label image and stores the label image in a label image DB 204a described later.
The label image search unit 203 receives a phoneme label string, searches for a label image corresponding to the phoneme label string from a label image DB 204a described later, and returns the label image.
The storage unit 204 stores a label image DB 204a described later with reference to FIG.

The labeling work terminal 300 is a terminal for performing a labeling work, and includes a label image search request unit 301 and a display unit 302.
The label image search request unit 301 transmits a phoneme label string to the label image server 200, searches the label image DB 204a for a label image corresponding to the phoneme label string, and requests to return it.
The display unit 302 displays the label image acquired by the label image search request unit 301 from the label image server 200 in a screen configuration as described later with reference to FIG.

The label string transmission unit 101, the label image transmission unit 102, the label registration determination unit 201, the label image registration unit 202, the label image search unit 203, and the label image search request unit 301 are like circuit devices that execute these functions. It can be realized by hardware, or can be realized as software operating on an arithmetic device such as a microcomputer or CPU.
The storage means 204 is preferably composed of a storage device having a relatively large capacity, such as an HDD (Hard Disk Drive).
The display unit 302 can be configured by a screen display device such as a display device and a control function such as driver software for controlling the screen display device.
Each terminal and server is assumed to have a necessary network interface.

  Here, before entering the explanation of the operation of the voice labeling support system of FIG.

First, an example of a “phoneme (environment) label string” in the present invention is shown.
For example, a label string corresponding to “good morning” in text is a representation of “good morning” with a phoneme symbol, such as “o−h + a”, “ha−y +”, “a−y + lo”, and “y−lo + slt”. Can be expressed as
Here, “−” and “+” indicate the connection before and after phonemes, “lo” indicates the long vowel of “o”, and “slt” indicates the end silence.

FIG. 2 shows an example of a label image.
The label image referred to in the first embodiment is obtained by dividing a speech waveform image when a certain text is generated by a speaker at a phoneme boundary and recording the waveform image and the phoneme boundary together as an image image. The format of the image is set so that it can be displayed on the display unit 302 of the labeling work terminal 300 described later (for example, a standard format such as JPEG or bitmap).
FIG. 2 shows a waveform image when a label string “i-xsh + lu” is generated in a certain phonemic environment, and the boundary when dividing this into phonemes “i”, “xsh”, and “lu” is a vertical line. It is represented by a line.

In manual labeling work, the work of defining the boundary of phonemes is performed in this way, and it is dependent on the labeler's individual know-how which part is set as the boundary position.
Therefore, when a skilled labeler performs a labeling operation, the label image as shown in FIG. 2 is acquired and accumulated, and when another labeler performs the labeling operation, the labeling operation is performed with reference to this labeling operation. Think about it.
The present invention is based on such an idea, and the label image server 200 has the storage function described above.

FIG. 3 shows the configuration and data example of the label image DB 204a.
The label image DB 204a has a “label column” column and a “label image” column.
The “label string” column stores a phoneme label string.
The “label image” column stores a label image corresponding to the phoneme label column specified by the value of the “label column” column. The label image stored in this column is image data indicating both the waveform data and the label position as described in FIG.

  Next, the operation of the voice labeling support system according to the first embodiment will be described.

FIG. 4 illustrates a processing sequence when the label image registration terminal 100 registers a label image. Here, a sequence when registering a label image corresponding to the phoneme label string “ka−i + i” is taken as an example.
Hereinafter, each step will be described.

(0) Preliminary work Here, it is assumed that the operator of the label image registration terminal 100 has sufficient labeling work experience, and as the pre-work, the labeling work for the phoneme label string “ka + i” is performed on the terminal. .

(1) Transmit the label sequence of “k−a + i” The operator operates an operation unit of the label image registration terminal 100 (not shown) to the label image server 200 to provide a label image corresponding to the phoneme label sequence “ka−i + i”. Request to determine whether or not is already registered.
Upon receiving the above operation instruction, the label string transmitting unit 101 transmits the phoneme label string “ka−i + i” to the label image server 200, and the label image corresponding to the phoneme label string “ka−i +” is stored in the label image DB 204a. Request to search if it is registered.
The search request reaches the label image server 200 as a packet transmitted through the network 400.

(2) Search for a Label Sequence “k−a + i” The label registration determination unit 201 receives the request packet transmitted by the label sequence transmission unit 101 in step (1).
Next, the label registration determination unit 201 searches the label image DB 204a using the received label string “k−a + i” as a key, and determines whether or not the corresponding label image is registered.
Here, it is assumed that the corresponding label image does not hit the search and is determined not to be registered.

(3) Label String Search Result The label registration determination unit 201 returns the search result for the label string “ka−i” to the label image registration terminal 100.

(4) Transmission Request The label sequence transmission unit 101 requests the label image transmission unit 102 to transmit a label image corresponding to the phoneme label sequence “k−a + i” to the label image server 200.

(5) Label image transmission of “k−a + i” The label image transmission unit 102 transmits to the label image server 200 a label image corresponding to the phoneme label string “ka−i”, which has been labeled by the operator as a preliminary operation. To do.
The label image reaches the label image server 200 as a packet transmitted through the network 400.

(6) Label image registration of “k−a + i” The label image registration unit 202 receives the label image packet transmitted by the label image transmission unit 102 in step (5).
Next, the label image registration unit 202 registers the received label image in the label image DB 204a. “Register” here means adding a new entry with the configuration described in FIG.

Through the above processing, the label image corresponding to the phoneme label string “ka + i” is registered in the label image DB 204a.
Next, a procedure for using a label image registered in the label image DB 204a will be described.

  FIG. 5 is a configuration example of a labeling work screen displayed on the display unit 302 of the labeling work terminal 300. Hereinafter, the work performed by the operator of the labeling work terminal 300 will be described with reference to FIG.

(1) Reading voice waveform data The operator operates the operation unit (not shown) to give instructions to the labeling work terminal 300 so as to read voice waveform data to be labeled.
A waveform image corresponding to the read audio waveform data is displayed in a portion “1” in FIG.

(2) Enlarging the Labeling Location The operator gives an instruction to the labeling work terminal 300 so as to enlarge the speech waveform image at the location where labeling is performed.
An enlarged waveform image of the location where the enlargement is instructed is displayed in a portion “2” in FIG.

(3) Transmission of label string The operator inputs a phoneme label string in the portion “5” in FIG. 5 and presses the “Send” button.

(4) Acquisition and display of label image The labeling work terminal 300 acquires a label image corresponding to the phoneme label string input by the operator in step (3) from the label image server 200 by the process of FIG. 5 is displayed at a portion “4”.

(5) Setting of Label Position The operator moves the part “3” by operating an operation unit (not shown) while referring to the label image displayed in the part “4” in FIG. This position is set as the label position.

  In this way, labeling can be performed while referring to the label image corresponding to (or closest to) the phoneme label string to be labeled as a kind of work manual, so the accuracy and accuracy of the label position can be unified and improved. Can be planned.

FIG. 6 illustrates an internal processing sequence executed by the labeling work terminal 300 when the operator of the labeling work terminal 300 performs the labeling work on the screen as described with reference to FIG.
Hereinafter, each step will be described.

(1) Sending a Label Sequence of “k−a + i” The operator operates an operation unit of a labeling work terminal 300 (not shown) and sends a label image corresponding to the phoneme label sequence “ka−i” to the label image server 200. Request to send.
In response to the above operation instruction, the label image search request unit 301 transmits the phoneme label string “ka−i + i” to the label image server 200 and searches for the label image corresponding to the phoneme label string “ka−i + i”. Request.
The search request reaches the label image server 200 as a packet transmitted through the network 400.
This step corresponds to the internal operation of the labeling work terminal 300 in step (3) described with reference to FIG.

(2) Searching for a Label Sequence of “k−a + i” The label image search unit 203 receives the request packet transmitted by the label image search request unit 301 in step (1).
Next, the label image search unit 203 searches the label image DB 204a using the received label string “k−a + i” as a key, and acquires a corresponding label image.
Here, it is assumed that the corresponding label image has been registered in the label image DB 204a.

(3) Label image transmission of “k−a + i” The label image search unit 203 transmits the label image acquired in step (2) to the labeling work terminal 300.
The label image reaches the labeling work terminal 300 as a packet transmitted through the network 400.

(4) Display Request The label image search request unit 301 requests the display unit 302 to display the acquired label image on the screen.
The display unit 302 displays the label image acquired by the label image search request unit 301 on the screen.
This step corresponds to the internal operation of the labeling work terminal 300 in step (4) described with reference to FIG.

As described above, according to the first embodiment, the label image held in the label image DB can be displayed on the work terminal and the labeling work can be performed while referring to the label image. Regardless of the difference in know-how, the accuracy of the label position can be improved uniformly.
As a result, the speech DB in the corpus-based speech synthesis method and the quality of synthesized speech generated using the speech DB can be improved.

Embodiment 2. FIG.
FIG. 7 shows a configuration of an audio labeling support system according to Embodiment 2 of the present invention.
The label image server 200 in the voice labeling support system of FIG. 7 is newly provided with a default label image storage unit 205 in addition to the configuration of FIG. Other configurations are the same as those in FIG.
The default label image storage unit 205 is preferably configured by a storage device having a relatively large capacity, such as an HDD (Hard Disk Drive).

The default label image storage unit 205 stores a default label image DB 205a.
The configuration of the default label image DB 205a is the same as that of the label image DB 204a described in FIG. 3, but the label image held by the label image DB 204a is transmitted from the label image registration terminal 100, whereas the default label image DB 205a holds the label image. The difference is that the label image is a label image generated in advance based on the utterance of a prescribed speaker.
In FIG. 7, the storage unit 204 and the default label image storage unit 205 are provided separately. However, these storage units may be integrally configured to store two DBs together. The same applies to the embodiments described later.

  The “specified label image DB” in the second embodiment corresponds to the default label image DB 205a.

FIG. 8 illustrates an internal processing sequence executed by the labeling work terminal 300 when the operator of the labeling work terminal 300 performs the labeling work on the screen as described in FIG. 5 in the second embodiment. To do.
The processing in each step is substantially the same as that described with reference to FIG. 6, but the processing in steps (2) to (3) is different and will be described.

(2) Searching for a Label Sequence of “k−a + i” The label image search unit 203 receives the request packet transmitted by the label image search request unit 301 in step (1).
Next, the label image search unit 203 searches the label image DB 204a using the received label string “k−a + i” as a key, and acquires a corresponding label image.
Here, it is assumed that the corresponding label image has not been registered in the label image DB 204a.
In this case, the label image search unit 203 searches the label image stored in the default label image DB 205a for a phoneme label string that is the same as or closest to “k−a + i”, and the corresponding label. Get an image.

(3) Label image transmission closest to “k−a + i” The label image search unit 203 transmits the label image acquired in step (2) to the labeling work terminal 300.
The label image reaches the labeling work terminal 300 as a packet transmitted through the network 400.

As described above, according to the second embodiment, even if the label image corresponding to the label image DB 204a is not stored, the default label image DB 205a is constructed in advance based on the utterance of a standard speaker. By doing so, it is possible to avoid a situation in which there is no label image to be referred to when performing the labeling operation.
For example, even if the labeling operation is unavoidable when the number of label images registered in the label image DB 204a from the label image registration terminal 100 is insufficient, a label based on at least a standard speaker's utterance Since an image is obtained, the accuracy of the labeling operation can be maintained at a certain level.

Embodiment 3 FIG.
In the third embodiment of the present invention, a configuration example in which label images generated based on the utterances of a plurality of speakers are stored in the label image server 200 will be described.
The configuration of the audio labeling support system according to the third embodiment is the same as that described in the second embodiment except for the configurations of the label image DB 204a and the default label image DB 205a, and thus the description thereof is omitted.

FIG. 9 shows the configuration and data example of the label image DB 204a in the third embodiment.
In FIG. 9, a “speaker name” column is newly added in addition to the configuration described in FIG. 3.
The “speaker name” column stores information for identifying a speaker, such as a name.
The “label string” column stores a phoneme label string.
The “label image” column stores a label image corresponding to the speaker specified by the value of the “speaker name” column and the phoneme label column specified by the value of the “label column” column.

That is, the label image DB 204a according to the third embodiment stores label images generated based on the utterances of a plurality of speakers, and stores a plurality of label images even in the same phoneme label string. In some cases.
The reason why the label images of a plurality of speakers are stored in this way is that even if the speech is made for the same phoneme label string, the appropriate label position may differ depending on the speaker. Therefore, it is desirable to store a label image for each voice speaker who performs labeling work in the label image DB 204a, and this is realized by a data structure as shown in FIG.
The default label image DB 205a can have the same configuration as that in FIG.

FIG. 10 illustrates a processing sequence when the label image registration terminal 100 registers a label image. Here, a sequence when registering label images corresponding to the speaker “A” and the phoneme label string “ka + i” is taken as an example.
Hereinafter, each step will be described.

(0) Pre-work Here, the operator of the label image registration terminal 100 has sufficient labeling work experience, and as a pre-work, the phoneme label string “k−a + i” by the utterance of the speaker “A” at the terminal is used. It is assumed that labeling work has been performed.

(1) Transmit the label string of the speaker names “A” and “ka + i” The operator operates the operation unit of the label image registration terminal 100 (not shown) to the label image server 200 for the speaker “A”. A request is made to determine whether or not a label image corresponding to the phoneme label string “ka + i” by utterance has already been registered.
In response to the above operation instruction, the label string transmission unit 101 transmits the speaker name “A” and the phoneme label string “ka−i + i” to the label image server 200, and the phoneme label string generated by the utterance of the speaker “A”. A request is made to search whether a label image corresponding to “k−a + i” is registered in the label image DB 204a.
The search request reaches the label image server 200 as a packet transmitted through the network 400.

(2) Search for the label string of “ka + i” with the speaker name “A” The label registration determination unit 201 receives the request packet transmitted by the label string transmission unit 101 in step (1).
Next, the label registration determination unit 201 searches the label image DB 204a using the received speaker name “A” and the label string “ka + i” as keys, and determines whether or not the corresponding label image is registered. To do.
The determination is based on whether or not a combination of the speaker name “A” and the label string “k−a + i” is registered in the label image DB 204a. That is, even if only one of them exists, it is not considered that the search condition is met.
Here, it is assumed that the corresponding label image does not hit the search and is determined not to be registered.

(3) Label String Search Result The label registration determination unit 201 returns a search result for the speaker name “A” and the label string “ka−i +” to the label image registration terminal 100.

(4) Transmission Request The label sequence transmission unit 101 transmits to the label image server 102 a label image corresponding to the phoneme label sequence “k−a + i” produced by the speaker “A”. To ask.

(5) Label image transmission of speaker names “A” and “ka + i” The label image transmission unit 102 performs labeling as a pre-operation by the operator, and includes a phoneme label string “k−” by the utterance of the speaker “A”. The label image corresponding to “a + i” is transmitted to the label image server 200.
The label image reaches the label image server 200 as a packet transmitted through the network 400.

(6) Label image registration of “k−a + i” The label image registration unit 202 receives the label image packet transmitted by the label image transmission unit 102 in step (5).
Next, the label image registration unit 202 registers the received label image in the label image DB 204a. At the time of registration, a new entry having a value “A” in the “speaker name” column and a value “ka−i” in the “label column” column in FIG. Store in the "Label Image" column.

FIG. 11 illustrates an internal processing sequence executed by the labeling work terminal 300 when the operator of the labeling work terminal 300 performs the labeling work on the screen as described in FIG. 5 in the third embodiment. To do.
Hereinafter, each step will be described.

(1) Transmitting the label string of the speaker names “A” and “ka + i” The operator operates the operation unit of the labeling work terminal 300 (not shown) and utters the speaker “A” to the label image server 200. Is requested to transmit a label image corresponding to the phoneme label string “ka−i + i”.
In response to the above operation instruction, the label image search request unit 301 transmits the speaker name “A” and the phoneme label string “ka−i + i” to the label image server 200, and the phoneme label generated by the utterance of the speaker “A”. Request to retrieve the label image corresponding to the column “k−a + i”.
The search request reaches the label image server 200 as a packet transmitted through the network 400.
This step corresponds to the internal operation of the labeling work terminal 300 in step (3) described with reference to FIG. In this case, a field for inputting “speaker name” is provided in the portion “5” of the screen of FIG.

(2) Search for a label string of “ka + i” with the speaker name “A” The label image search unit 203 receives the request packet transmitted by the label image search request unit 301 in step (1).
Next, the label image search unit 203 searches the label image DB 204a using the received speaker name “A” and the label string “ka + i” as keys, and acquires a corresponding label image. When the corresponding label image is not registered in the label image DB 204a, the speaker name is “A” and the phoneme label string is “k−a + i” among the label images held in the default label image DB 205a. ”Is the same as or closest to this, and the corresponding label image is obtained.

(3) Label image transmission of “k−a + i” The label image search unit 203 transmits the label image acquired in step (2) to the labeling work terminal 300.
The label image reaches the labeling work terminal 300 as a packet transmitted through the network 400.

(4) Display Request The label image search request unit 301 requests the display unit 302 to display the acquired label image on the screen.
The display unit 302 displays the label image acquired by the label image search request unit 301 on the screen.

  As described above, according to the third embodiment, a label image close to the label image of the voice actually performing the labeling work is referred to by holding the label image for each speaker of the voice performing the labeling work. Since the labeling operation can be performed, it is possible to set the label position with higher accuracy in a unified manner regardless of the difference in know-how between the labelers.

Embodiment 4 FIG.
In the fourth embodiment of the present invention, a mel cepstrum information is stored for each label image in the label image server 200, and a configuration example using this mel cepstrum information when searching for a label image will be described.
The configuration of the audio labeling support system according to the fourth embodiment is the same as that of FIG. 1 described in the first embodiment except for the configuration of the label image DB 204a, and thus the description thereof is omitted.

“Mel cepstrum” is defined as an inverse Fourier transform of a logarithmic power spectrum on a mel scale, which is a scale representing a sense of pitch. In general, by using a mel cepstrum, it is possible to compress information in accordance with auditory characteristics.
In the fourth embodiment, the mel cepstrum information is extracted at 5 ms intervals in the waveform data section of the corresponding label sequence, and averaged at four equal parts in each label section. These numerical values are designed, and the designer may set them appropriately.

In the fourth embodiment, when searching for the corresponding label image from the label image DB 204a, the mel cepstrum information is used as a search condition in combination with the search label string.
As a result, even when a plurality of entries match the search condition or when there is no entry that matches the search condition, the label image with the closest mel cepstrum information can be acquired. A label image suitable for reference when performing a labeling operation at 300 can be reliably acquired.

  The configuration and operation of the fourth embodiment will be described below.

FIG. 12 shows the configuration and data example of the label image DB 204a according to the fourth embodiment.
In FIG. 12, in addition to the configuration described in FIG. 3, a “Mel cepstrum” column is newly added.
The “label string” column stores a phoneme label string.
The “label image” column stores a label image corresponding to the phoneme label column specified by the value of the “label column” column.
The “mel cepstrum” column stores the mel cepstrum information calculated from the waveform data corresponding to the label image stored in the “label image” column. Here, an example is shown in which twelve pieces of mel cepstrum information are stored for each label image, but the number of pieces of mel cepstrum information is not limited to this.

FIG. 13 illustrates a processing sequence when the label image registration terminal 100 registers a label image. Here, a sequence when registering a label image corresponding to the phoneme label string “ka−i + i” is taken as an example.
Hereinafter, each step will be described.

(0) Preliminary work to (3) Label row search results Steps (0) to (3) are the same as those described in FIG.

(4) Transmission Request The label sequence transmission unit 101 sends the label image corresponding to the phoneme label sequence “k−a + i” and the mel cepstrum information obtained from the waveform data to the label image transmission unit 102. Ask to send to.

(5) Transmit “k−a + i” label image and mel cepstrum information The label image transmission unit 102 labels the label image corresponding to the phoneme label string “ka−i”, which has been labeled as a preliminary operation by the operator. It transmits to the image server 200.
Further, the mel cepstrum information corresponding to the label image is obtained from the voice waveform data, and transmitted to the label image server 200 together with the label image.
The label image and the mel cepstrum information reach the label image server 200 as a packet transmitted through the network 400.

(6) Register “k−a + i” label image and mel cepstrum information The label image registration unit 202 receives the packet transmitted by the label image transmission unit 102 in step (5).
Next, the label image registration unit 202 registers the received label image and mel cepstrum information in the label image DB 204a.

  In FIG. 13, the mel cepstrum information corresponding to the label image to be registered is obtained by the label image registration terminal 100. Instead, the waveform data is transmitted to the label image server 200, and the label image server 200 sends the mel cepstrum information. Information may be obtained and registered.

FIG. 14 illustrates an internal processing sequence executed by the labeling work terminal 300 when the operator of the labeling work terminal 300 performs the labeling work on the screen as described in FIG. 5 in the fourth embodiment. To do.
Hereinafter, each step will be described.

(1) Transmit “k−a + i” label sequence and mel cepstrum information The operator operates an operation unit of the labeling work terminal 300 (not shown) to correspond to the phoneme label sequence “ka−i” to the label image server 200. Request to send the label image to be sent.
In response to the operation instruction described above, the label image search request unit 301 obtains mel cepstrum information from waveform data corresponding to the phoneme label string “ka−i + i”.
Next, the mel cepstrum information is transmitted to the label image server 200 together with the phoneme label string “k−a + i”, and a request is made to search for the phoneme label string “k−a + i” and the label image corresponding to the mel cepstrum information. .
The search request reaches the label image server 200 as a packet transmitted through the network 400.

(2) Searching for the label string of “k−a + i” and mel cepstrum information The label image search unit 203 receives the request packet transmitted by the label image search request unit 301 in step (1).
Next, the label image search unit 203 searches the label image DB 204a using the received label string “k−a + i” as a key, and acquires a corresponding entry. Here, it is assumed that a plurality of entries hit the search.
Furthermore, the label image search unit 203 calculates the distance between the mel cepstrum information held by the entry obtained by the search using “ka + i” as a key and the mel cepstrum information received from the labeling work terminal 300, and the most distance An entry having a small size is specified, and a label image of the entry is acquired.

(3) Label image transmission of “k−a + i” to (4) display request Steps (3) to (4) are the same as steps (3) to (4) described in FIG. 6 of the first embodiment. Therefore, the description is omitted.

In step (2), the case where there are a plurality of entries corresponding to the label string “k−a + i” has been described. However, as described above, even if there is no corresponding entry at all, the same applies. Alternatively, the entry having the shortest distance from the mel cepstrum information received from the labeling work terminal 300 may be searched.
Further, the label string and the mel cepstrum information are used together as search conditions, but the mel cepstrum information alone may be used as the search conditions.
In any case, by using the mel cepstrum information, it is possible to acquire a label image that is close to the feature of the voice that performs the labeling operation.

Although the configuration of the label image DB 204a has been described in FIG. 12 of the fourth embodiment, the default label image DB 205a can have the same configuration.
12 illustrates the configuration in which the “Mel cepstrum” column is added to the configuration in FIG. 3, but the configuration in which the “Mel cepstrum” column is added to the configuration in FIG. There is no difference in effect.

  As described above, according to the fourth embodiment, since the mel cepstrum information is stored in the label image DB 204a and the mel cepstrum information is used when searching for the label image, a plurality of entries are searched. Even if the conditions are met or there are no entries that match the search conditions, the label image with the closest mel cepstrum information can be acquired, and the label is suitable for reference when performing labeling work. The image can be acquired reliably.

Embodiment 5. FIG.
In the fifth embodiment of the present invention, a configuration having a function of calculating mel cepstrum information by the label image server 200 will be described. As a result, the processing load for retrieving the label image is collected in the label image server 200.
The configuration of the voice labeling support system according to the fifth embodiment is the same as that described in the fourth embodiment, and thus the description thereof is omitted.

FIG. 15 illustrates an internal processing sequence executed by the labeling work terminal 300 when the operator of the labeling work terminal 300 performs the labeling work on the screen as described in FIG. 5 in the fifth embodiment. To do.
Hereinafter, each step will be described.

(1) Transmit “k−a + i” label sequence and waveform data The operator operates an operation unit of a labeling work terminal 300 (not shown) to correspond to the phoneme label sequence “k−a + i” to the label image server 200. Request to send a label image.
Upon receiving the above operation instruction, the label image search request unit 301 transmits the waveform data together with the phoneme label string “ka−i +” to the label image server 200, and the phoneme label string “ka−i +” and its mel cepstrum. Requests to retrieve the label image corresponding to the information.
The search request reaches the label image server 200 as a packet transmitted through the network 400.

(2) Obtaining Mel Cepstrum Information The label image search unit 203 receives the request packet transmitted by the label image search request unit 301 in step (1).
Next, the label image search unit 203 calculates mel cepstrum information from the received waveform data.

(3) Search for the label string and mel cepstrum information of “ka + i” The label image search unit 203 searches the label image DB 204a using the received label string “ka + i” as a key, and acquires the corresponding entry. To do. Here, it is assumed that a plurality of entries hit the search.
Further, the label image search unit 203 calculates the distance between the mel cepstrum information held in the entry obtained by the search using “ka + i” as a key and the mel cepstrum information calculated in step (2), and the most distance is obtained. An entry having a small size is specified, and a label image of the entry is acquired.

(4) “k−a + i” label image transmission to (5) display request Steps (4) to (5) are the same as steps (3) to (4) described in FIG. 6 of the first embodiment. Therefore, the description is omitted.

In addition, although the case where there are a plurality of entries corresponding to the label string “k−a + i” in step (2) has been described, as in the fourth embodiment, even if there is no corresponding entry at all, Similarly, an entry having the shortest distance from the mel cepstrum information received from the labeling work terminal 300 may be searched.
In addition, the label string and the mel cepstrum information are used together as search conditions. However, similarly to the fourth embodiment, the mel cepstrum information alone may be used as a search condition.

As described above, according to the fifth embodiment, when the labeling work terminal 300 acquires the label image, the label image server 200 executes the process for obtaining the mel cepstrum information. The CPU 200 and the memory of the labeling work terminal 300 can be reduced in size by concentrating on the server 200.
In addition, it is preferable from the viewpoint of management of server assets and the like to consolidate the calculation load in the label image server 200 because it also consolidates investment targets.

In the first to fifth embodiments, the communication method between the label image registration terminal 100 and the label image server 200 and the communication method between the labeling work terminal 300 and the label image server 200 are particularly referred to. There is no, but any scheme can be used.
For example, it may be configured as a client-server type system that transmits and receives data and commands using an arbitrary port on TCP, or the label image server 200 is provided with a Web server function, and further a label image The registration terminal 100 and the labeling work terminal 300 may be provided with a Web browser function and configured as a Web application.

In addition, although it has been described that the label image DB 204a is stored in the storage unit 204 and the default label image 205a is stored in the default label image storage unit 205, an optimal storage format may be used as appropriate.
As an example, a database management system (DBMS) that stores data files in each storage means is configured on the label image server 200, and the data structure in the table format as shown in FIGS. 3, 9, and 12 under the DBMS. It is possible to define and store a data entry as described in each figure in each row.
When the image data size of the label image is large, the label image may be stored as an image file separately from the DB, and only the file path may be held in the “label image” column.

1 shows a configuration of an audio labeling support system according to Embodiment 1. An example of a label image is shown. It shows the configuration and data example of the label image DB 204a. A processing sequence when the label image registration terminal 100 registers a label image will be described. It is a structural example of a labeling work screen displayed on the display unit 302 of the labeling work terminal 300. An internal processing sequence of the labeling work terminal 300 in the first embodiment will be described. It is a structure of the audio | voice labeling assistance system which concerns on Embodiment 2. FIG. An internal processing sequence of the labeling work terminal 300 according to the second embodiment will be described. The structure and data example of label image DB204a in Embodiment 3 are shown. A processing sequence when the label image registration terminal 100 registers a label image will be described. An internal processing sequence of the labeling work terminal 300 according to the third embodiment will be described. The structure and data example of label image DB204a in Embodiment 4 are shown. A processing sequence when the label image registration terminal 100 registers a label image will be described. An internal processing sequence of the labeling work terminal 300 according to the fourth embodiment will be described. An internal processing sequence of the labeling work terminal 300 according to the fifth embodiment will be described.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 Label image registration terminal, 101 Label row transmission part, 102 Label image transmission part, 200 Label image server, 201 Label registration determination part, 202 Label image registration part, 203 Label image search part, 204 Storage means, 204a Label image DB, 205 default label image storage means, 205a default label image DB, 300 labeling work terminal, 301 label image search request unit, 302 display unit, 400 network.

Claims (5)

A label image server comprising storage means for storing a label image DB for holding a label image for each label row of the phonemic environment;
A work terminal for labeling work,
An audio labeling support system comprising:
The label image server
A search unit that receives a search label string and searches the label image DB for a corresponding label image;
The work terminal is
A search request unit that requests the search unit to search for a label image;
A display for displaying the label image on the screen;
With
The search request unit
Send the label sequence of the audio to be labeled to the label image server,
The search unit
A label string is received from the search request unit, an entry corresponding to the label string is searched from the label image DB, and a label image associated with the label string is returned to the work terminal,
The search request unit
An audio labeling support system that receives a label image from the search unit and causes the display unit to display the label image on a screen. The storage means
Stores the specified label image DB generated based on the utterance of the specified speaker,
The search unit
If the entry corresponding to the label string received from the search request unit does not exist in the label image DB,
Search the label image DB corresponding to the label column closest to the label column received from the search request unit from the specified label image DB,
The voice labeling support system according to claim 1, wherein the label image is returned to the work terminal. The label image DB is
Holding the label image generated based on the utterances of multiple speakers,
The search request unit
Along with the label sequence of the voice that performs the labeling operation, information for identifying the speaker is sent to the label image server,
The search unit
Receive information for identifying a label string and a speaker from the search request unit,
Search the label image DB for an entry corresponding to the label string and the speaker,
The voice labeling support system according to claim 1 or 2, wherein a label image associated with the label string and a speaker is returned to the work terminal. The label image DB is
Holds mel cepstrum information for each label image,
The search request unit
Find the mel cepstrum information of the voice that performs the labeling work, send the result to the label image server,
The search unit
Receive mel cepstrum information from the search request section,
For the mel cepstrum information held in the label image DB, calculate the distance from the mel cepstrum information transmitted by the search request unit,
The mel cepstrum information with the shortest distance is searched from the label image DB, and the label image corresponding to the mel cepstrum information is returned to the work terminal. Voice labeling support system. The label image DB is
Holds mel cepstrum information for each label image,
The search request unit
Send the waveform data of the voice to be labeled to the label image server,
The search unit
Receiving voice waveform data from the search request unit, obtaining mel cepstrum information from the waveform data,
For the mel cepstrum information held in the label image DB, calculate the distance from the mel cepstrum information obtained by the search unit,
4. The mel cepstrum information with the shortest distance is searched from the label image DB, and a label image corresponding to the mel cepstrum information is returned to the work terminal. Voice labeling support system.

Images