JP2006154568A - Logging system having voice recognition function, terminal device in same system and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve recognition rate and to improve convenience while conducting word retrieval without being adversely affected by the presence or the absence of ciphering. <P>SOLUTION: In the logging system having a voice recognition function, recognition texts, which are respectively generated by using specific speaker voice recognition engines (specific speaker voice recognition sections 13 and 14) and respectively held in terminal devices (IP telephones 11 and 12), or learning data of the specific speaker voice recognition engines are respectively transmitted to the respectively calling terminal devices with a prescribed timing, voice recognition of the voice data being transmitted and received at the terminal devices is conducted and the results are stored (information storing regions 15 and 16). <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a logging system with a voice recognition function suitable for use in storing and utilizing voice information exchanged using an IP (Internet Protocol) connection environment, and a terminal device and a program in the system.

  VoIP (Voice over Internet Protocol) telephones have become widespread for the purpose of reducing communication costs, and their application to CTI (Computer Telephony Information) systems that fuse computers and telephones has attracted attention.

For example, in a call center, there is a voice logging system in which logs of a plurality of telephones are collected by a logging server to obtain logs of the telephones. Among these voice logging systems, there are a system that transparently stores the voice of a telephone and a voice server that relays the voice to a target telephone by a logging server and stores the voice in the relay process. In addition, as a system for recognizing and using a voice recorded by a call recording device, there is a voice command or the like that uses the recognized voice for device operation (for example, see Non-Patent Documents 1 and 2).
http://advanced-media.co.jp/prooducts/1502.html <Internet> Viewed November 11, 2004, “AmiVoice series product information” http://www.logit.co.jp/products/nicelog/voip.html <Internet> Viewed November 11, 2004, Logit Corporation Products [Product Introduction] “Latest IP Recording that Enables VoIP Recording”

By the way, there is no mechanism for preventing eavesdropping by a third party by encrypting the voice data exchanged by the IP phone. On the other hand, there is a high possibility that an encryption mechanism will be incorporated from the viewpoint of privacy protection. In the United States, there is a movement to legally obligate preservation of IP phone voice as a measure for preserving evidence such as litigation.
However, when encryption is considered, when audio is stored transparently on a single server, the audio stored on the server is already encrypted data, so that it is not easy to use. In addition, in the logging system by relaying voice, there is a possibility that the encrypted data can be decrypted and stored. However, when considering privacy protection, it is necessary to encrypt and relay the decrypted data again. There is a risk that smooth communication may be hindered by processing required for relay processing. In addition, since the load is concentrated on the server, there is a risk of causing a logging error or the like.

On the other hand, for speech recognition, since the target is not uniquely determined by telephone communication, a speech recognition engine for unspecified majority is required. The unspecified speaker speech recognition engine has a lower recognition rate than the specific speaker speech recognition engine that can absorb individual differences in speech.
In addition, the quality of voice exchanged by telephone varies depending on the state of the device and the state of the speaker. The recognition result is adversely affected by the microphone with low sensitivity, the distance between the microphone and the vocal organ, and the amount of speech. Further, when referring to the voice and the recognized text, there is no correlation between the voice and the recognized text, so that it is difficult to reproduce the voice halfway from the location where the word occurs when a word search is performed.

  The present invention has been made based on the above circumstances, and is a speech recognition system that improves convenience when performing word search without being affected by conditions such as the presence or absence of encryption, speech input, and recognition environment. It is an object of the present invention to provide a logging system with a function and a terminal device and a program in the system.

  In order to solve the above problems, a logging system with a voice recognition function according to the present invention is a logging system with a voice recognition function that logs call voice between terminal devices connected to a network, and each of the terminal devices has The recognition text created by using a specific speaker voice recognition engine that performs speech recognition for a specific speaker or the learning data of the specific speaker voice recognition engine is transmitted to each other terminal device at a predetermined timing. And means for performing voice recognition of voice data transmitted and received in the counterpart terminal apparatus and storing the result.

  Further, the present invention is the terminal device in the logging system with a voice recognition function for logging call voice between terminal devices connected to a network, and receives and stores voice data transmitted from a partner terminal device And means for storing voice data to be transmitted to the destination terminal device, generating a recognition text from the voice data using a voice recognition engine that performs voice recognition on itself, and storing the voice data in association with the voice data; And means for transmitting the recognized text to a call partner at an arbitrary timing.

  Further, the present invention is characterized in that the recognition text is transmitted when it is detected that the call with the counterpart terminal device has ended.

  In the present invention, a session for transmitting the recognized text is established separately from the call session, and the stored recognized text is transmitted for each session.

  Further, the present invention provides the terminal device in the logging system with a voice recognition function for logging call voice between the terminal devices connected to the network, storing voice data to be transmitted to the counterpart terminal device, and storing the voice data Generating a recognized text using a speech recognition engine that performs speech recognition on the speech data, storing the correlated text in association with the speech data, and receiving and storing the speech data transmitted from the counterpart terminal device And learning data of the speech recognition engine of the counterpart terminal device transmitted at an arbitrary timing from the counterpart terminal device, and generating a recognition text from the stored received voice data using the learning data And means.

  Further, the present invention is a program used for the terminal device in a logging system with a voice recognition function for logging a call voice between terminal devices connected to a network, which receives voice data transmitted from a destination. Save processing, save voice data to be transmitted to the destination terminal device, generate recognition text from the voice data using a voice recognition engine that performs voice recognition on the voice data, and associate with the voice data And storing the recognized text and transmitting the recognized text to the other party at an arbitrary timing.

  Further, the present invention is a program used for the terminal device in a logging system with a voice recognition function for logging call voice between terminal devices connected to a network, and stores voice data to be transmitted to a destination terminal device. From the voice data, a process of generating a recognition text using a voice recognition engine that performs voice recognition on the voice data, storing the text in association with the voice data, and voice data transmitted from the counterpart terminal device Receive and store processing, and receive learning data of the speech recognition engine of the counterpart terminal device transmitted at an arbitrary timing from the counterpart terminal device, and recognize from the stored received voice data using the learning data A process for generating a text is executed by a computer.

According to the present invention, since it is possible to use speech recognition text using a specific speaker speech recognition engine that performs speech recognition for a specific speaker in each terminal device, the recognition rate is improved and the processing speed is increased. Improvement is planned. In addition, since transmission / reception data and created recognition text are stored in the terminal device, the logging error that occurs when the load is concentrated on a specific device is eliminated without being affected by encryption of voice data exchanged.
Further, according to the present invention, the end of the call is detected or the recognition text is transmitted for each session set separately from the call session, and the specific speaker recognition result is transmitted to the other party in real time. Although the performance is impaired, a result with a high recognition rate can be obtained. Furthermore, according to the present invention, by storing the correlation between the speech and the recognized text, it is possible to realize partial reproduction in which a portion hit by the word search can be heard from a phrase having the hit word in the search. it can. This can provide convenience in searching.

Hereinafter, embodiments of the present invention will be described with reference to FIGS. FIG. 1 is a system configuration diagram of a logging system with a voice recognition function according to an embodiment of the present invention. Here, an example is shown in which IP telephones 11 and 12 are used to talk between users.
The IP telephone 11 (12), which functions as the terminal device of the present invention, includes a specific speaker voice recognition unit 13 (14) equipped with a specific speaker voice recognition engine. The information is stored in the information storage areas 15 and 16 in association with the transmission data. In either case, the received voice data is only temporarily stored, and is stored in association with the recognized text transmitted from the other IP telephone 12 (11).

FIG. 2 is a block diagram showing a functional development of the internal configuration of the logging system with a voice recognition function shown in FIG.
In FIG. 2, the blocks with the same numbers as the blocks shown in FIG. 1 have the same names and functions as those shown in FIG. Here, the IP telephone 11 (12) is configured to have only the voice transmission unit 111 (122) and the voice reception unit 112 (121).
The voice data transmitted by the voice transmission unit 111 (122) is temporarily stored in the transmission voice storage unit 151 (161) of the information storage area 15, and is also supplied to the specific speaker voice recognition unit 13 (14). Here, speech recognition based on the user's own speech is performed by the specific speaker recognition engine. Here, since the learning data is abundant, the speech recognition rate is high and results can be obtained quickly. The specific speaker voice recognition unit 13 (14) holds the digital voice data until an amount necessary for recognition is accumulated, and performs the recognition process when the recognition process becomes possible. The digital voice data and the recognition text after the recognition processing are continuously stored in the transmission voice recognition text storage unit 152 (162) in the information storage area 15 even after the digital voice data recognition processing for the entire call is completed.

On the other hand, the packetized digital voice data from the IP telephone set 11 (12) is received by the voice receiving section 121 (112) of the IP telephone set 12 (11) and temporarily stored in the received voice storage section 163 (153). If it has been encrypted at this time, a decryption process is performed. In the same manner as described above, the IP telephone 12 (11) simultaneously performs voice recognition (specific speaker voice recognition unit 14 (13)) of the digital voice data uttered by itself, and sends the result to the transmission voice recognition text storage unit 162 ( 152).
In the recognized text synchronization unit 18 (17), the received text stored in the information storage area 15 of the IP telephone 11 (12) having previously transmitted the voice data is used as the recognized text stored in the transmission voice recognition text storage unit 162 (152). The transmission timing to the recognized text storage unit 154 is monitored. Here, it is assumed that transmission is waited for when the end of the call is detected and voice recognition processing is completed after the end of the call. In the case of VoIP (Voice over IP) communication, the recognized text synchronizer 18 (17) knows the host name in advance, so that the IP telephones 11 (12) communicate with each other when the call termination timing is detected. It is possible to easily synchronize the contents of calls exchanged in

Here, the timing for transmitting the recognized text to the destination IP telephone 11 (12) is the case where the end of the call is detected, but the recognized text is transmitted separately from the VoIP call session, for example, FTP A (File Transfer Protocol) connection may be established and transmitted for each session. In addition, various changes such as detecting a silent section and transmitting a recognized text can be considered.
Here, only one specific speaker recognition engine is installed in the specific speaker voice recognition unit 13 (14). However, it is conceivable that a plurality of such specific speaker recognition engines are provided and used by the speaker using a switch. Here, the “specific speaker” refers to a speaker in which individual characteristics are stored through a learning process for a predetermined period.

FIG. 3 is a conceptual diagram of speech recognition processing used in the embodiment of the present invention. Here, the phrase separation process is performed from the digital audio data stored in the data temporary storage area 51 (corresponding to the transmission audio storage units 151 and 161 and the reception audio storage units 153 and 163 in the information storage areas 15 and 16 in FIG. 2). The section 52 generates phrase-delimited audio data 53.
Next, speech recognition processing is performed on the phrase-separated speech data 53 by the speech recognition unit 54 (corresponding to the specific speaker speech recognition units 13 and 14 in FIG. 2), and the recognition text 55 is generated. The recognized text 55 and the phrase-delimited speech data 53 are paired with the data / recognized text storage area 56 (corresponding to the transmission speech recognition text storage units 152 and 162 and the reception speech recognition text storage units 154 and 164 in FIG. 2). And save. This operation is performed until the digital audio data in the data temporary storage area 51 runs out.

4 and 5 are flowcharts showing the processing procedure of the phrase segmentation process shown in FIG.
In FIG. 4, silence control of RTP (Real-time Transport Protocol), which is a standard media stream transmission protocol in IP telephones, is used for the phrase delimiting process. RTP is a protocol that controls stream transmission in an IP network that is rich in fault tolerance and does not consider data arrival in real time. Specifically, packet loss on the transmission path is detected with a sequence number incremented by one each time a voice packet is sent, and the playback timing is controlled by a time stamp expressed by the total of sent data.
Silence control is control that does not transmit data in order to save bandwidth when there is no voice input from a voice input device such as a microphone for a certain period of time. When silence control occurs, the sequence number becomes the packet +1 before the silence control occurs, and the time stamp is added in the same manner as when data is sent in the silence period. Therefore, the time stamp is greatly increased with no packet loss.

This will be described with reference to the flowchart shown in FIG. First, in step S61, it is determined whether sufficient digital audio data for recognition processing is stored in the temporary data storage area 51. If a sufficient amount is not stored, it is determined in step S62 whether the stored data has reached the end of the call. If the end of the call has been reached, a recognition process is performed in step S66. If not, the process proceeds to step S61 after waiting for a predetermined time in step S63.
If it is determined in step S61 that the stored data is stored in a sufficient amount for recognition, in step S64, a portion where the RTP timestamp is separated by a certain threshold or more, that is, silence control has occurred. Determine whether the part exists. If no silence control has occurred, it is determined whether or not the data stored in step S62 has reached the end of the call. Here, if the end of the call has been reached, control is transferred to the recognition process in step S66. If not, the stored data is likely to continue, so after waiting for a certain time in step S63. Control is transferred to step S61.

When silence control is detected in step S64, in step S65, the data from the beginning of the stored data to the silence section is regarded as audio data divided into one phrase and acquired. The acquired data is removed from the saved portion. The speech data divided into phrases is recognized by the recognition engine (speech recognition processing unit 54) in step S66, and stored in an area for storing the recognition result.
After the recognition process, it is determined whether or not the data stored in step S67 remains. If it remains, control is transferred to step S61 after waiting for a predetermined time in step S63. Is regarded as having been recognized and finished.

Next, a description will be given with reference to the flowchart shown in FIG. FIG. 5 shows a technique in which the audio level of digital audio data is checked and a continuous portion having a low audio level for a certain period of time is regarded as a phrase.
First, in step S71, it is determined whether or not sufficient digital audio data for recognition processing is stored in the temporary data storage area 51. If a sufficient amount is not stored, it is determined in step S72 whether the stored data has reached the end of the call. If the end of the call has been reached, a recognition process is performed in step S77. If not, the process proceeds to step S71 after waiting for a predetermined time in step S73.
If it is determined in step S71 that the stored data is stored in a sufficient amount for recognition, noise is removed by a noise removal filter and white noise is smoothed in step S74.

Next, in step S75, it is determined whether or not there is a section having a low audio level for a certain period of time. If there is no section with a low voice level, it is determined in step S72 whether or not the data has reached the end of the call. If so, control is passed to step S77. After waiting, control is passed to step S71.
Here, if there is a section with a low audio level, in step S76, the data from the beginning of the stored data to the low audio level section is regarded as audio data divided into one clause and acquired. The acquired data is removed from the saved portion. The speech data divided into phrases is subjected to recognition processing by the recognition engine (specific speaker speech recognition units 13 and 14) in step S77, and stored in an area for storing the recognition result. After the recognition process, it is determined whether or not the data stored in step S78 remains. If it remains, control is transferred to step S71 after waiting for a predetermined time in step S73. Is regarded as having been recognized and finished.

FIG. 6 is an operation conceptual diagram conceptually showing the search processing. Here, a method of using what is considered as a misrecognition result of speech recognition as a search word using an ambiguous search dictionary will be described.
First, a word is input into a word input field of a search GUI (graphical user interface) 81 created by browser software, and a search button is pressed. As a result, the input word is passed to the search system 82 (S81).
The search system 82 passes from the fuzzy search dictionary 83 a misrecognizable word list considered as a result of erroneous recognition of the input word to the search system 82 (S82). Subsequently, the search system 82 searches for all words in the list whether there is a match with the speech recognition text stored in the data / recognition text storage area 84 (S83). The search results are displayed on the result display interface 85 according to the rank previously assigned to the misrecognizable word list (S84).

FIG. 7 is an operation conceptual diagram conceptually showing the fuzzy dictionary creation processing. Here, focusing on the fact that speech recognition performs pattern matching based on the features of the input speech waveform, the fact that the result of erroneous recognition is limited to a certain number is used.
First, in the voice / word input unit 91, a word is input to the word input field, and a voice corresponding to the word is input by a voice input device such as a microphone. Next, the input word is held in the ambiguous dictionary creation unit 92. Next, the speech is actually subjected to speech recognition processing in the speech recognition processing unit 93, the recognition result is passed to the ambiguous dictionary creation processing unit 92, and is associated with the first input word. At this time, if the word input through the input field is the same as the speech recognition result text, the result text is discarded. If the search result text is already associated with the input word, the number of appearances of the result text is increased and used for ranking when displaying the result of the search process in FIG.

  Next, the speech that has undergone the speech recognition processing is subjected to effect processing such as speech level adjustment, noise addition, and tempo adjustment in the speech adjustment processing unit 94, and speech recognition processing is performed again in the speech recognition processing unit 93. By performing the above operation, the word recognition result list of the search dictionary is increased.

As described above, according to the present invention, it is possible to perform speech recognition using a specific speaker speech recognition engine that performs speech recognition for a specific speaker in each terminal device. Speed can be improved. In addition, since transmission / reception data and created recognition text are stored in the terminal device, the logging error that occurs when the load is concentrated on a specific device is eliminated without being affected by encryption of voice data exchanged.
In addition, according to the present invention, by detecting the end of a call or transmitting a recognition text for each session set separately from a call session, a specific speaker recognition result by itself can be transmitted to the other party. Although the real-time property is impaired, a result with a high recognition rate can be obtained.

  In the present invention, the speech recognition process is performed after the speech is divided into phrases, and this improves the recognition rate. Furthermore, by storing the correlation between the speech and the recognized text, it is possible to realize partial reproduction in which a portion hit by the word search can be heard from a phrase having the hit word at the time of the search. Further, by performing a search using an ambiguous search dictionary, an effect of reducing the occurrence rate of search omission in the search process can be obtained even if the voice recognition result is not accurate. In addition, by using what was created using an actual speech recognition engine when creating an ambiguous search dictionary, it is possible to create a search that conforms to the actual misrecognition pattern and use it to realize search processing with fewer search omissions. be able to.

It is a figure which shows the system configuration | structure of the logging system with a speech recognition function which concerns on this invention embodiment. It is the block diagram which expanded and showed the internal structure of the logging system with a speech recognition function which concerns on this invention embodiment. It is an operation | movement conceptual diagram of the audio | voice phrase division | segmentation process which concerns on this invention embodiment, and a speech recognition process. It is a flowchart which shows an example of the process sequence of the phrase division | segmentation process shown in FIG. It is a flowchart which shows the other example of the process sequence of the phrase division | segmentation process shown in FIG. It is an operation | movement conceptual diagram which shows notionally the operation | movement of the search system using a fuzzy search. It is an operation | movement conceptual diagram which shows notionally the operation | movement at the time of creating the dictionary used for a fuzzy search.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11, 12 ... IP telephone, 13, 14 ... Specific speaker voice recognition part, 15, 16 ... Information storage area, 17, 18 ... Recognition text synchronization part,

Claims (7)

A logging system with a voice recognition function for logging call voice between terminal devices connected to a network,
Each of the terminal devices has a recognition text created using a specific speaker voice recognition engine that performs voice recognition for a specific speaker, or learning data of the specific speaker voice recognition engine at a predetermined timing, respectively. Means for performing voice recognition of voice data transmitted to the partner terminal device and transmitted / received in the partner terminal device, and storing the result;
A logging system with a voice recognition function. The terminal device in a logging system with a voice recognition function for logging call voice between terminal devices connected to a network,
Means for receiving and storing audio data transmitted from the counterpart terminal device;
Means for storing voice data to be transmitted to a destination terminal device, generating a recognition text from the voice data using a voice recognition engine that performs voice recognition on the subject, and storing the text in association with the voice data;
Means for transmitting the recognized text to a call destination terminal device at an arbitrary timing;
A terminal device comprising:   The terminal device according to claim 2, wherein the recognition text is transmitted when it is detected that the call with the counterpart terminal device has ended.   The terminal device according to claim 2, wherein a session for transmitting the recognized text is established separately from a call session, and the stored recognized text is transmitted for each session. The terminal device in a logging system with a voice recognition function for logging call voice between terminal devices connected to a network,
Means for storing voice data to be transmitted to a destination terminal device, generating a recognition text from the voice data using a voice recognition engine that performs voice recognition on the voice data, and storing the text in association with the voice data; ,
Means for receiving and storing audio data transmitted from the counterpart terminal device;
Means for receiving learning data of the speech recognition engine of the counterpart terminal device transmitted at an arbitrary timing from the counterpart terminal device, and generating recognition text from the stored received voice data using the learning data;
A terminal device comprising: A program used for the terminal device in a logging system with a voice recognition function for logging call voice between terminal devices connected to a network,
A process of receiving and storing audio data transmitted from the counterpart terminal device;
Processing for storing voice data to be transmitted to a partner terminal device, generating a recognition text from the voice data using a voice recognition engine that performs voice recognition on the voice data, and storing the text in association with the voice data; ,
A process of transmitting the recognized text to a call destination terminal device at an arbitrary timing;
A program that causes a computer to execute. A program used for the terminal device in a logging system with a voice recognition function for logging call voice between terminal devices connected to a network,
Processing for storing voice data to be transmitted to a partner terminal device, generating a recognition text from the voice data using a voice recognition engine that performs voice recognition on the voice data, and storing the text in association with the voice data; ,
A process of receiving and storing audio data transmitted from the counterpart terminal device;
Processing for receiving learning data of the speech recognition engine of the counterpart terminal device transmitted at an arbitrary timing from the counterpart terminal device, and generating recognition text from the stored received voice data using the learning data;
A program that causes a computer to execute.

Images