JP2020060735A - Voice recognition system - Google Patents

Abstract

To convert sound files into texts using a learning type server via the internet and identify speakers.SOLUTION: A voice recognition system 1 includes: a sound collecting unit 2 into which sounds are input; a processing unit 3 that generates sound files F from the sounds, transmits the sound files F to a text conversion server and a speaker identification server, and receives text files W and identification results for speakers H; and a monitor unit 4 that displays processing results from the processing unit 3. Sound data V that is separated by silent states each between speeches by the speakers H is transmitted as the sound files F to a text conversion server 5 and a speaker identification server 6 that are cloud services performing data analysis through self-learning based on the data that is collected from many users via the Internet. Then, the received text files W and identification results for the speakers H are displayed in the monitor unit 4 in chronological order, in association with the sound files F.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a voice recognition system having high recognition accuracy even for a plurality of speakers, for example.

  A voice recognition device for converting voice data input from a microphone into text has been widely used. Then, Patent Document 1 discloses a voice recognition device that increases the accuracy of voice recognition by learning the utterance feature for each speaker.

  In addition, learning systems using deep learning have been constructed in various cloud services. These learning systems perform data analysis and analysis by self-learning by a neural network-based processing device based on data collected from many users via the Internet.

  By performing self-learning without waiting for human instruction, it is possible to efficiently improve the output accuracy of the processing device, and the user can use the analysis results of data analysis performed by the cloud service that utilizes deep learning. are doing.

JP, 2002-2151848, A

  However, since the voice recognition device of Patent Document 1 learns from only the voice collected from the microphone, there is a limit to the voice data that can be collected. Further, in the cloud service described above, a large-scale system in which a large number of arithmetic processing units are arranged in parallel is constructed, whereas the speech recognition device of Patent Document 1 has a small system scale. Therefore, the speech recognition apparatus of Patent Document 1 has a problem that the learning accuracy is slow to be improved, and the accuracy of text conversion and speaker identification is not easily improved.

  An object of the present invention is to solve the above-mentioned problems and use a learning server that is a cloud service via the Internet to accurately convert collected voices into texts and to accurately identify a speaker. To provide a recognition system.

  A voice recognition system according to the present invention for achieving the above object includes a sound collecting unit for inputting ambient sound, and a processing unit for generating a sound file by processing data of sound data input from the sound collecting unit. And a monitor unit that displays a processing result of the processing unit, wherein the processing unit is connected to a character conversion server having a self-learning function and a speaker identification server via the Internet. The voice file is transmitted to the character conversion server, a text file obtained by converting the voice file into a text is received from the character conversion server, and the voice file and the user ID information of the speaker are received by the speaker identification server. And the speaker identification result for the audio file is received, and the sentence file and the speaker identification result corresponding to the audio file are transmitted. And displaying the serial monitor.

  According to the voice recognition system of the present invention, a character conversion server and a speaker identification server, which are cloud services that perform data analysis and analysis by self-learning, based on data collected from many users via the Internet are used. By doing so, it is possible to perform character conversion and speaker identification with high accuracy for a voice file without providing a character conversion function and a speaker identification function.

  Further, the voice file can be associated with the text file and the speaker of the specific result, and can be displayed on the monitor unit in chronological order in substantially real time. The speaker and the content of the utterance can be confirmed by characters, and past utterances can be easily confirmed by scrolling the screen.

It is a system configuration diagram of a voice recognition system. It is a flowchart figure in the case of generating a voice file from voice data. It is explanatory drawing which represented the voice data of a speaker by the waveform. It is a list figure of the voice data for every speaker. It is explanatory drawing which represented the voice data of another speaker by the waveform. It is explanatory drawing which determines the voice data for every speaker. It is explanatory drawing of the text sentence displayed on a monitor part.

The present invention will be described in detail with reference to the illustrated embodiments.
The voice recognition system 1 generates a voice file F by processing the sound collecting unit 2 that inputs a surrounding sound and the sound data input from the sound collecting unit 2, and generates the voice file F. And a processing unit 3 for transmitting the text file W and the identification result of the speaker H to the speaker identification server, and a monitor unit 4 for displaying the processing result of the processing unit 3.

  A commercially available notebook computer or desktop computer may be used as the voice recognition system 1, and an external microphone or the like is used as the sound collection unit 2. The sound collecting unit 2 uses a monaural type instead of a stereo type in which left and right are double-recorded, and is preferably of high quality. This monaural type sound collecting unit 2 is installed in the center between speakers such as a table.

  The processing unit 3 includes a calculation unit 3a, a memory unit 3b, and a storage unit 3c, and executes various data processing by activating software stored in the storage unit 3c. The processing unit 3 and the sound collection unit 2 are connected by wire or wirelessly.

  The monitor unit 4 is connected to the processing unit 3 and is composed of, for example, a liquid crystal display, and various processing results and the like in the processing unit 3 can be displayed on the monitor unit 4. The monitor unit 4 may be a monitor of another PC or mobile terminal connected via a network.

  The character conversion server 5 is a neural network-based API (Application Programming Interface) existing on the Internet IN, and is connected to the voice recognition system 1 via the Internet IN.

  The character conversion server 5 is outside the voice recognition system 1, and when the voice file F is uploaded from the processing unit 3 of the voice recognition system 1, a text conversion process for converting into a text file W is performed, and the processing unit 3 generates it. The text file W can be downloaded. When a long voice file F for several minutes is uploaded, it takes time for the text conversion process. Therefore, it is preferable to divide the voice data V into tens of seconds or less and upload it to the character conversion server 5.

  In addition, the character conversion server 5 performs deep learning based on voice files uploaded by a large number of users, and self-corrects the text conversion processing. Therefore, as time passes, the conversion accuracy of the text conversion process improves.

  At the same time, the speaker identification server 6, which is separate from the character conversion server 5, is a neural network-based API existing on the Internet IN, and is connected to the voice recognition system 1 via the Internet IN.

  This speaker identification server 6 registers a voice sample for each speaker in advance, and uploads the voice data V from the voice recognition system 1 to the speaker identification server 6, and based on the registered speaker data, It is possible to specify the speaker of the voice data V. For example, when the voice data of the speaker Ha is uploaded, if the speaker Ha is already registered in the speaker identifying server 6, the voice data voice owner is identified as the speaker Ha. Further, since many speakers are registered in the speaker identification server 6, it is possible to efficiently recognize the user IDs by uploading the group of registered user IDs together with the voice data V. is there.

  The speaker identification server 6 also identifies the speaker while performing self-analysis using deep running based on the audio files uploaded by a large number of users. Accuracy is improved.

  For example, by uploading the user IDs of the speakers Ha, Hb, and Hc, which are a plurality of speakers, to the speaker identification server 6 together with the voice data V in which the conversations of the speakers Ha, Hb, and Hc are recorded, The speakers Ha, Hb, and Hc are specified by being selected from three user IDs. Therefore, the speaker-specific processing speed is increased, and the speaker-specific accuracy is improved.

  FIG. 2 is a flow chart of sound file generation for sound data input to the processing unit 3 via the sound collection unit 2. The sound data is the data input to the sound collection unit 2, and the processed sound data is processed as the sound data V.

  As shown in FIG. 1, for example, a process of the processing unit 3 when one sound collecting unit 2 is arranged in the center of the speakers Ha, Hb, and Hc and a conference is started will be described. When the conference is started, the speakers Ha, Hb, and Hc make respective sounds in time series as shown in FIG. 3, and one sound data in which these are synthesized is obtained.

  In step S1, only the human utterance frequency is extracted from the sound data stored in the storage unit 3c and stored as the sound data V. In this extraction processing, for example, when a sound of moving a chair, a siren sound of an ambulance, or the like is mixed in the sound data, the sound range of which is cut as noise is stored as the sound data V.

  FIG. 3 is an explanatory diagram showing the audio data V, which is recorded at time t11, in a simplified waveform. For example, the speaker Ha first says, "I'm going to start the meeting.", The speaker Hb says "OK," and the speaker Hc says "I understand." This is the waveform of the voice when Ha says, "I will move to the agenda."

  Then, the process proceeds to step S2 of FIG. 2, and the silent time m, which is the silent state between the remarks of the voice data V, is measured. For example, when the threshold value of the silent time m is set to 1 second and the silent time m1 of 1 second or more occurs, the process proceeds to step S3. If there is a silent time m0 of 1 second or less in step S2, the process of step S2 is repeated.

  In step S3, it is determined whether or not there are a plurality of speakers H for the voice data V immediately before being separated by the silent time m1. The process of identifying the speaker H utilizes the fact that the speaker H has different center frequencies with respect to the voice data V sampled at predetermined intervals. From the displacement of the center frequency, it is possible to determine the number of speakers H of the voice data immediately before being separated by the silent time m1.

  After determining the number of speakers H, the process proceeds to step S4. When the number of the speakers H is plural, the process proceeds to step S5. When the number of the speakers H is one, step S4 is omitted and the process proceeds to step S6. .

  In the voice data V shown in FIG. 3, since the voice data V immediately before divided by the times t12, t13, t14, and t15 have only one frequency characteristic for each person, the step S5 is omitted in the step S4. Control goes to step S6.

  The process of step S5 will be described later, and the process of step S6 will be described first. As shown in FIG. 4, the separated voice data V is the voice of "The conference will be started." File F1: t11, voice file F2: t12 of speaker Hb saying "OK.", Voice file F3: t13 of speaker Hc saying "I understand.", Speaker Ha saying "OK. It will be saved as a voice file F4: t14 of "Go to agenda." It should be noted that the processing unit 3 cannot specify who the voice file F is.

  Then, each generated voice file F is transmitted to the character conversion server 5 and the speaker identification server 6. After the transmission, the process returns to step S2, and the processes of steps S2 to S6 are repeated.

  As shown in FIG. 3, when the speakers Ha, Hb, and Hc have a conversation as described above, after the conversation of the first speaker Ha ends, the silence of the first speaker Ha occurs, and then the voice of the next speaker Hb starts. FIG. 5 is an explanatory diagram showing a simplified waveform of the audio data V in the case where the silent time is less than the threshold m0 during the conversation between the speakers Ha, Hb, and Hc. Is.

  In the case where the first silent time m1 occurs after the speaker Ha saying “Then, I will move to the agenda.” Shown in FIG. 5, with respect to the immediately preceding voice data V0 separated by the time t21 in step S3. , The center frequency is measured by sampling at a predetermined time. Then, when there are a plurality of center frequencies in step S4, that is, when there are a plurality of speakers H, the process proceeds to step S5.

  In step S5, the voice file F for each of the determined speakers H is generated. FIG. 6 is an explanatory diagram of a case where the speaker H is discriminated from the displacement of the center frequency with respect to the immediately preceding voice data V0 divided by the occurrence of the silent time m1 at time t25. By determining the center frequency of the voice data V0, it is possible to classify the voice data VH1 starting at time t21, the voice data VH2 starting at time t22, and the voice data VH3 starting at time t23.

  Even when two speakers H are uttered and recorded in a part of the voice data V0, the sampling time is shortened, for example, 10 msec to occupy each sampling time. It is possible to specify the speaker H to be used, and it is possible to classify the voice data V0 which is uttered redundantly into individual voice data V.

  Further, the voice data VH1 composed of two utterances can generate the voice files F1: t21 and F4: t24 of the two voice files F because the silent time m1 exists between the utterances.

  By performing the above determination process, the audio files F1: t21 to F4: t24 similar to the audio files F1: t11 to F4: t14 shown in FIG. 4 can be generated. Although the processing unit 3 can determine that the speakers of the audio files F1: t21 to F4: t24 are different, it cannot specify who is speaking.

  Alternatively, the process of generating the two audio files F from the audio data VH1 may not be performed, and only one audio file F1: t21 may be generated. In this case, in order to clarify the time series of the statements in the latter half of the audio file F1: t21 and the statements in the audio files F2: t22 and F3: t23, it is necessary to store the information of times t21 to t24 in each audio file. There is. That is, by storing the times t21 and t24 in the audio file F1: t21, it is possible to display each utterance on the monitor unit 4 described later in a time series of times t21 to t24.

  The difference between the processing of the audio data V shown in FIG. 3 and the processing of the audio data V shown in FIG. 5 is that the audio file Fa is generated every time the silent time m1 occurs in the audio data V of FIG. The four audio files F1: t11 to F4: t14 are sequentially generated from the top, and each time they are generated, the process proceeds to step S6. On the other hand, in the audio data V of FIG. 5, the audio files F1: t21 to F4: t24 are generated almost at the same time, and the process proceeds to step S6.

  When the generated voice file F is transmitted to the character conversion server 5 in step S6, the voice files F1: t11 to F4: t14 and the voice files F1: t21 to F4: t24 are converted into text sentence files W1: t11, respectively. ~ W4: t14 and the text files W1: t21 to W4: t24 are converted, and the voice recognition system 1 receives these files.

  In addition, when transmitting the generated voice file F to the speaker identification server 6, in addition to the voice files F1: t11 to F4: t14 and the voice files F1: t21 to F4: t24, the speakers configuring the conversation The user IDs Ha to Hc are also transmitted. The speaker identification server 6 selects the user IDs of the voice files F1: t11 to F4: t14 and the voice files F1: t21 to F4: t24 transmitted from the processing unit 3 from the user IDs transmitted together. The speaker H is specified, and the processing unit 3 receives the specified speakers Ha to Hc corresponding to the audio file F.

  Then, the processing unit 3 associates the voice file F with the text file W and the speaker H of the specific result, and displays them on the monitor unit 4 in chronological order. In other words, the voice file F1: t11 of "I will start the meeting." That the speaker H could not specify is identified with the sentence file W1: t11 of "I will start the meeting." And the speaker Ha, It is displayed as shown in FIG.

  The voice file F is stored with the end of the file name as a serial number for the time t, and the text file W and the speaker H are displayed in chronological order of the time t as shown in FIG. In FIG. 7, the speaker Ha is displayed on the left side and the speakers Hb and Hc are displayed on the right side in order to easily identify the speaker H.

  Although a slight time lag occurs in the generation of each file and the transmission / reception to / from the cloud service as described above, the speech date and time for the latest audio file F, the text file W and the speaker H are sequentially displayed from the bottom of the screen of the monitor unit 4 in real time. Will be displayed.

  Note that the display order is determined based on the file name of the audio file F. However, in addition to the file name, the times t11 to t14 are stored as the speech date and time in the file header or the like, and based on these informations. It can be displayed in chronological order.

  Further, the processing date and time of the display processing may be displayed on the screen instead of the displayed speech date and time. In this case, the voice files F are transmitted to the cloud service in the order in which they were generated from the processing unit 3 without storing the above-mentioned statement date and time, and the next voice file F is transmitted to the cloud service on condition that they are received. May be.

  By being displayed in a conversational format as shown in FIG. 7, it is possible to easily confirm who made what kind of speech at a later date. Further, by making the screen shown in FIG. 7 viewable on a PC or a mobile terminal to which another terminal device is connected, the contents of the conference can be visually confirmed from another place in almost real time.

  In particular, when the meeting is heard by sound at another place, the speaker H cannot be specified and it is difficult to grasp the entire content. It is easy to understand the content of the meeting because it can be confirmed by letters.

  Furthermore, the content of the conference can be easily grasped when confirming the content of the conference in a place where it is difficult to output a voice or when confirming the content of the conference by a hearing impaired person. You can easily check past comments by scrolling the screen.

  In addition, although the sound data collected by the sound collecting unit 2 installed in the center of the speakers Ha to Hc such as the table of the voice recognition system 1 has been described, the sound including the sound data recorded in another place is included. A data file may be stored in the storage unit 3c via a network, a storage medium, or the like, or may be read directly and the processing of the above-described flowchart may be performed by the calculation unit 3a.

  As described above, the voice recognition system 1 includes the character conversion server 5 and the speaker identification server 6 that are cloud services that perform data analysis and analysis by self-learning based on data collected from many users via the Internet IN. By using it, it is possible to perform accurate character conversion and speaker identification without providing a character conversion function and a speaker identification function.

  In addition, since the conference contents can be written in almost real time, and the recorded voice file can be written in a posterior manner, it can be useful for grasping the conference contents promptly.

1 voice recognition system 2 sound collection unit 3 processing unit 4 monitor unit 5 character conversion server 6 speaker identification server IN Internet

Claims (4)

It is composed of a processing unit that processes a sound data including sound data input from a sound collection unit that inputs a surrounding sound to generate a sound file, and a monitor unit that displays a processing result of the processing unit. A voice recognition system,
The processing unit is connected to a character conversion server having a self-learning function and a speaker identification server via the Internet,
The voice file is transmitted to the character conversion server, and a text file obtained by converting the voice file into a text is received from the character conversion server,
Transmitting the voice file and the speaker user ID information to the speaker identification server, and receiving the speaker identification result for the voice file,
A voice recognition system, wherein the text file corresponding to the voice file and the speaker identification result are displayed on the monitor.   Generates voice data by extracting only the frequency of human utterance from the sound data, divides the voice data when the silent state between the utterance of the speaker is more than a predetermined time, and immediately before the break. The voice recognition system according to claim 1, wherein the voice file is generated based on voice data.   The voice recognition system according to claim 2, wherein the voice data is sampled at a predetermined interval, and the voice file for each speaker is generated by determining a characteristic of a center frequency.   The voice according to any one of claims 1 to 3, wherein the monitor unit displays the sentence file corresponding to the voice file and the speaker of the specific result in association with each other in chronological order. Recognition system.

Images