JP7095569B2 - Speech recognition system and speech recognition method - Google Patents

Description

The present invention relates to a voice recognition system and a voice recognition method.

In recent years, speech recognition technology has been widely used. For example, voice recognition technology is widely used for applications such as accepting a user's voice operation on an AI (Artificial Intelligence) speaker and recording the content of a conversation with a customer at a call center.
In these applications, the distance between the mouth of the user who speaks and the microphone that picks up the voice spoken by the user is close. Therefore, the S / N ratio (signal-noise ratio), which indicates the ratio of voice and noise, is in a high state, and voice recognition can be performed with high accuracy.

On the other hand, for example, for the purpose of creating minutes in a conference, a table microphone may be arranged in the center of the conference table to collect the utterances of the conference participants at once. In this case, the distance between the speaker and the microphone is several tens of centimeters or more. Therefore, there arises a problem that the S / N ratio becomes low and the accuracy of speech recognition is lowered.
An example of a technique considering such a problem is disclosed in Patent Document 1. In the technique disclosed in Patent Document 1, each user who participates in the conference is made to wear a microphone having directivity toward the user's mouth. Then, the voice data collected by each of these plurality of microphones is appropriately selected and the voice recognition process is performed. As a result, voice recognition can be performed without causing the problem that the distance between the speaker and the microphone is increased.

However, the technique disclosed in Patent Document 1 does not consider the case where the user speaks in a direction different from the direction of the microphone and the user's voice deviates from the direction range of the microphone. When the user's voice is out of the direction range of the microphone in this way, the user's voice is buried in the voice or noise of another user and is picked up, so that the accuracy of voice recognition is lowered.

The present invention has been made in view of such a situation, and is a voice recognition system capable of suppressing a decrease in voice recognition accuracy even when the user's voice is out of the direction range of the microphone. , And a voice recognition method.

In order to solve the above-mentioned problems and achieve the object, the voice recognition system provided by the present invention is equipped with voice data of voices spoken by a plurality of users in the same space by each of the plurality of users. The acquisition means acquired from the plurality of sound collecting devices, the determination means for determining whether or not the volume of the acquired plurality of audio data is less than the threshold value, and the determination means, whichever the volume of the plurality of audio data is. Also includes a voice recognition processing means for performing voice recognition processing together with a predetermined process using the plurality of voice data when it is determined that the data is less than the threshold value.

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide a voice recognition system and a voice recognition method capable of suppressing a decrease in voice recognition accuracy even when the user's voice is out of the direction range of the microphone. ..

It is a schematic diagram which took a bird's-eye view of the whole structure of the voice recognition system which concerns on one Embodiment of this invention, and the space where a user who uses a voice recognition system exists. It is a schematic diagram which shows the wearing example when the microphone in one Embodiment of this invention is realized by the neck-hanging wearable microphone. It is a timing chart which shows an example of the utterance situation of each user. It is a timing chart which shows an example of the utterance situation of each user. It is a block diagram which shows the hardware composition of each microphone, a conference terminal, and a voice recognition server included in a voice recognition system. It is a functional block diagram which shows the functional configuration for executing a plurality of voice combined processing among the functional configurations of each microphone, a conference terminal, and a speech recognition server included in a speech recognition system. It is a schematic diagram which shows the appearance composition and display example of a conference terminal. It is a flowchart explaining the flow of the 1st plurality of voice combined processing in one Embodiment of this invention. It is a flowchart explaining the flow of the 2nd plurality of voice combined processing in one Embodiment of this invention. It is a schematic diagram which shows the appearance composition and display example of the conference terminal in 1st modification. It is a schematic diagram which shows about the image analysis in the 1st modification and the 2nd modification. It is a schematic diagram which shows the display example in the conference terminal in the 1st modification. It is a schematic diagram which shows the structural example of the voice recognition part in the 2nd modification. It is a flowchart explaining the flow of process in 2nd modification.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[System configuration]
FIG. 1 is a schematic view of the overall configuration of the voice recognition system S according to the present embodiment and a space in which a user using the voice recognition system S exists (here, a conference room as an example). As shown in FIG. 1, the voice recognition system S includes a plurality of microphones 10 (here, microphones 10A to 10F as an example), a conference terminal 20, and a voice recognition server 30.

Further, a table is arranged in the center of the conference room including windows and doors, and a plurality of users U (here, user UA to user UF as an example) who participate in the conference are located around the table. Each of these users U is equipped with a microphone 10 whose alphabet at the end of the code is the same as its own code. The number of users U and the number of microphones 10 and the like are merely examples, and are not particularly limited.

Each microphone 10 functions as a sound collecting device that collects the voice of each user U. The shape of each microphone 10 is not particularly limited, but each microphone 10 can be realized by, for example, a wearable microphone mounted on the user U as a neck-hanging type or a badge type. With such a wearable microphone, for example, even a user U who feels annoyed with a headset or a pin microphone or a user U who dislikes reusing a microphone worn by another person can wear it without worrying about it.

An example of mounting the microphone 10 will be described with reference to FIG. FIG. 2 is a schematic view showing a mounting example when the microphone 10 is realized by a neck-mounted wearable microphone. In this example, the microphone 10 has directivity toward the mouth in order to intensively pick up the sound uttered above the microphone 10, that is, from the mouth of the user U wearing the microphone 10.

Therefore, as shown in FIG. 2A, when the user U speaks facing the front, the voice spoken by the user U can be appropriately picked up. On the other hand, as shown in FIG. 2B, when the user U turns sideways or upward and speaks in a direction different from the directivity direction of the microphone, and the voice of the user U deviates from the directivity range of the microphone. The voice of this user U is buried in the voice and noise of another user U and is picked up. In the present embodiment, by performing the "multiple voice combined processing", as shown in FIG. 2B, the accuracy of voice recognition is reduced even when the voice of the user U is out of the direction range of the microphone. Suppress. The details of this combined voice processing will be described later.

Each microphone 10 creates audio data of a digital signal by analog-digital conversion of an analog signal corresponding to the picked-up sound by an A / D conversion circuit. Then, each microphone 10 transmits the created voice data to the conference terminal 20 by communication. Such a communication method is not particularly limited, but can be realized by, for example, wireless communication such as Bluetooth (registered trademark) capable of many-to-many connection. In this way, by using a communication method capable of many-to-many connection, it is possible to collect the voices spoken by each user U in parallel at the same time.

The conference terminal 20 transmits the voice data received from each microphone 10 to the voice recognition server 30. Such a communication method is not particularly limited, but can be realized by, for example, wired or wireless communication via a network such as the Internet or a LAN (Local Area Network). The conference terminal 20 may be realized by a relay device or the like having a communication function, or may be realized by a device such as an electronic information board on which the user U can input characters or the like by an input operation on the display. In the following description, a case where the conference terminal 20 is realized by an electronic information board will be described as an example. The electronic information board may also be referred to as an interactive whiteboard (IWB), an electronic blackboard, or the like.

The voice recognition server 30 is a server that performs voice recognition processing on a plurality of voice data received from each microphone 10. In the voice recognition server 30, a plurality of voice recognition engines corresponding to a plurality of voice data received from the microphone 10 operate in parallel. As a result, parallel voice recognition processing can be realized in real time for the voice data of each user U. The result of the voice recognition process is used, for example, by converting it into text and displaying it in real time on a display unit such as a conference terminal 20 or printing it on a paper medium after the conference is completed. The voice recognition server 30 can be realized by, for example, a cloud server provided on the cloud.

[Multiple voice processing]
The voice recognition system S having such a configuration performs a plurality of voice combined processing as described above. Here, the plurality of voice combined processing is a series of processes for suppressing a decrease in the accuracy of voice recognition by using the voice data of a plurality of users U in combination.

Specifically, in the plurality of voice combined processing, the voice recognition system S acquires the voice data of the voice spoken by the plurality of users U in the same space from the plurality of microphones 10 worn by each of the plurality of users U. Further, the voice recognition system S determines whether or not the volume of the acquired plurality of voice data is less than the threshold value. Here, the case where the volume of the acquired plurality of voice data is less than the threshold value is, for example, the case where the voice of the user U is out of the pointing range of the microphone, as shown in FIG. 2B described above.

Then, when it is determined that the volume of the plurality of voice data is less than the threshold value, the voice recognition system S performs the voice recognition process together with the predetermined process in which the plurality of voice data are used in combination. As a predetermined process in which the plurality of voice data are used in combination, for example, two processes, a first plurality of voice combination process and a second plurality of voice combination process, can be mentioned.

(1st multiple voice combined processing)
In the first plurality of voice combined processing, the voice recognition system S adds up a plurality of voice data and performs voice recognition processing on the combined voice data. As a result, it is possible to perform the voice recognition process on the voice data whose S / N ratio is improved by the summation, so that it is possible to suppress the deterioration of the accuracy of the voice recognition.

The first plurality of voice combined processing will be described with reference to FIGS. 3 and 4. 3 and 4 are timing charts showing an example of the utterance situation of each user U.
In this example, as shown in FIG. 4, it is assumed that three user Us (user UA, user UB, and user UC) speak in order at different timings (including partially overlapping timings). do. Specifically, it is assumed that the utterances are made in the order of "user UA-> user UB-> user UC-> user UB-> user UA" in chronological order. However, it is assumed that the user UA who made the last utterance in this utterance turned sideways and deviated from the pointing range of the microphone as shown in FIG. 2 (B) described above.

The microphones 10 (microphones 10A, 10B, and microphones 10C) worn by each of these three users U each pick up the voice spoken by the user U wearing the microphone 10 to the maximum. So, the voice spoken by the other user U is also picked up, albeit in a small amount. For example, between T1 and T2 in the figure, the microphone 10A picks up a large amount of the voice spoken by the user UA, and the microphones 10B and 10C also pick up the voice spoken by the user UA in a small amount. Although not shown in the figure, the microphone 10 actually collects a small amount of noise in the conference room (for example, operating noise of an air conditioner, a projector, or the like).

In such a situation, in the present embodiment, a threshold value is set for the volume as described above, and if the volume of the audio data of at least one microphone 10 is equal to or higher than the threshold value, the audio data having a volume equal to or higher than this threshold value is used. Perform voice recognition. That is, when the voice spoken by the wearing user U can be appropriately picked up by at least one microphone 10, voice recognition is performed on the voice data of this voice. Then, voice recognition is not performed on the voice data having a volume lower than the threshold value collected by the other microphone 10.

For example, during the period from T1 to T2, voice recognition is performed on the voice data of the voice spoken by the user UA picked up by the microphone 10A. Further, during the period from T2 to T3, voice recognition is performed on the voice data of the voice spoken by the user UB picked up by the microphone 10B. Further, during the period from T3 to T4, voice recognition is performed on the voice data of the voice spoken by the user UC collected by the microphone 10C. Further, during the period from T4 to T5, voice recognition is performed on the voice data of the voice spoken by the user UB picked up by the microphone 10B. Then, voice data having a volume lower than the threshold value other than that is excluded from the target of voice recognition.

During the period from T1 to T2, the voice spoken by the user UA picked up by the microphone 10A and the voice spoken by the user UB picked up by the microphone 10B partially overlap in time, but each of them. The sound can be picked up by the microphone 10 of the above at a volume equal to or higher than the threshold value. Therefore, the voice data of the voice spoken by the user UA and the voice data of the voice spoken by the user UB during this overlapping period are used for voice recognition, respectively.

Here, the predetermined process using the plurality of voice data in the present embodiment is applied during the period from T6 to T7 in which the user UA is speaking sideways. During the period from T6 to T7, all of the microphones 10A, the microphones 10B, and the microphones 10C pick up only the sound having a volume lower than the threshold value. Therefore, the S / N ratio of the voice picked up by each microphone 10 is low, and even if voice recognition is performed on the voice data of the voice picked up by each microphone 10 as it is, the voice spoken by the user UA is appropriate. Voice recognition is not possible.

In this way, when the user U speaks sideways or upward, the voice of the user U is out of the direction range of the microphones, and the voice data of any of the microphones 10 has a volume below the threshold value, a plurality of voice data. Performs a predetermined process in combination with. Specifically, as shown in FIG. 4, the voice data of the voice picked up by the plurality of microphones 10 is added up to improve the S / N ratio. Then, voice recognition is performed on the voice data in which the voice is emphasized by improving the S / N ratio.
As a result, even when the voice of the user U deviates from the pointing range of the microphone, it is possible to suppress a decrease in the accuracy of voice recognition.

In this case, the uttered user U may be estimated based on the comparison result of the volumes of the plurality of voice data. For example, when the volume of the voice data of the microphone 10A is relatively louder than the volume of the voice data of the microphone 10B or the microphone 10, it may be estimated that the user UA corresponding to the microphone 10A has spoken. However, if it is difficult to estimate, as shown in FIG. 4, the speaker may be unspecified and output from the voice summing unit 230 described later.

(Second multiple voice combined processing)
Although it is possible to suppress a decrease in the accuracy of voice recognition by the above-mentioned first multiple voice combined processing, strict timing control (for example, μsec order) of the voice data acquired from each microphone 10 is performed in order to add up the voice data. Strict synchronization control, etc.) is required. In addition, a processing unit for performing addition processing is also required. Therefore, as another simpler process, it is conceivable to perform the second plurality of voice combined process.

Also in the second plurality of voice combination processing, a predetermined process using a plurality of voice data is performed in the same manner as in the first plurality of voice combination processing. That is, as in the period from T6 to T7 shown in FIG. 3, the user U speaks sideways or upward, the voice of the user U is out of the direction range of the microphones, and the voice data of any microphone 10 is less than the threshold value. In the case of volume, a predetermined process is performed using a plurality of voice data in combination.

Specifically, the voice recognition system S performs voice recognition processing on each of the plurality of voice data, and corrects the voice recognition result based on the comparison of the voice recognition results of each of the plurality of voice data. As a result, correction can be performed based not only on the voice recognition result of a certain voice data but also on the voice recognition results of each of the plurality of voice data, so that it is possible to suppress a decrease in the accuracy of voice recognition.

Here, as an example of the correction process, when the voice recognition results for a plurality of voice data are common in a certain section (for example, a section corresponding to a phrase specified based on the voice recognition result). , Make corrections assuming that this common speech recognition result is correct.

Further, in this case, for example, if two voice recognition results are the same among the voice recognition results for three voice data, the so-called majority decision is made so that the voice recognition result is corrected as correct. It may be a process based on this.

Alternatively, if the results of a plurality of voice recognition processes are all different, the volume of each of the plurality of voice data is compared, and the voice recognition result of the voice data having the highest volume is corrected as correct. good.
As a result, even in the second plurality of voice combined processing, it is possible to suppress a decrease in the accuracy of voice recognition even when the voice of the user U is out of the pointing range of the microphone.

[Hardware configuration]
Next, the hardware configuration of each device in the present embodiment will be described with reference to FIG. FIG. 5 is a block diagram showing the hardware configurations of each microphone 10, the conference terminal 20, and the voice recognition server 30 included in the voice recognition system S.

The microphone 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a communication I / F (Interface) 14, an operation unit 15, and a sound collection unit 16. Each of these parts is connected so as to be able to communicate with each other by a bus connection.

The CPU 11 controls the entire microphone 10. Specifically, the CPU 11 uses the RAM 13 as a work area (that is, a work area) to perform arithmetic processing based on the firmware stored in the ROM 12 or the like, the OS (Operating System), and various programs. Then, the CPU 11 controls each hardware included in the microphone 10 based on the result of this arithmetic processing. Here, the various programs are, for example, programs for transmitting voice data to the conference terminal 20 in the above-mentioned plurality of voice combined processing.

The ROM 12 stores firmware, an OS, various programs, and various data used in these programs (for example, voice data used in the above-mentioned combined voice processing).

As described above, the RAM 13 functions as a work area of the CPU 11.
The communication I / F 14 is an interface for the microphone 10 to communicate with each other device included in the voice recognition system S.

The operation unit 15 is realized by, for example, various buttons and the like, and accepts the operation of the user U. For example, the operation unit 15 receives an operation of switching the power of the microphone 10 on and off and an operation of instructing the start of sound collection. Then, the operation unit 15 outputs the contents of the received operation of the user U to the CPU 11.

The sound collecting unit 16 includes a device that converts the collected sound into an analog electric signal, and an A / D conversion circuit for digitally converting the sound. The sound collecting unit 16 outputs the digitally converted voice data to the CPU 11.

The conference terminal 20 includes a CPU 21, a ROM 22, a RAM 23, an HDD 24, a communication I / F 25, an operation unit 26, a display unit 27, and an image pickup unit 28. Each of these parts is connected so as to be able to communicate with each other by a bus connection.

The CPU 21 controls the entire conference terminal 20. Specifically, the CPU 21 uses the RAM 23 as a work area to perform arithmetic processing based on the firmware stored in the ROM 22, the HDD 24, etc., the OS, and various programs. Then, the CPU 21 controls each hardware included in the conference terminal 20 based on the result of this arithmetic processing. Here, the various programs are, for example, a program for realizing the above-mentioned multiple voice combined processing and a program for realizing the function of the electronic information board.

The ROM 22 and HDD 24 store firmware, an OS, various programs, and various data used in these programs (for example, various data used in the above-mentioned multiple voice combined processing and electronic information board functions). do.

As described above, the RAM 23 functions as a work area of the CPU 21.
The communication I / F 25 is an interface for the conference terminal 20 to communicate with each other device included in the voice recognition system S.

The operation unit 26 is realized by, for example, various buttons and the like, and accepts the operation of the user U. For example, the operation unit 15 receives an operation of switching the power of the conference terminal 20 on / off, an operation of instructing the start of sound collection, and an operation related to the function of the electronic information board. Then, the operation unit 26 outputs the contents of the received operation of the user U to the CPU 21.

The display unit 27 is realized by a liquid crystal display (LCD: Liquid Crystal Display), an organic EL display (Organic Electro Luminescence Display), or the like, and displays predetermined information output from the CPU 21 to the user U. The display unit 27 displays, for example, information in which the voice recognition result is converted into text and various user interfaces to the user U as predetermined information.
The operation unit 26 and the display unit 27 may be integrally realized by a pen for an electronic information board or a touch panel capable of receiving a touch operation by the user U.
The image pickup unit 28 is realized by various devices for realizing a camera, and captures, for example, a user U participating in a conference at a place where a conference terminal 20 is installed. The image pickup unit 28 outputs the image data created by the image pickup to the CPU 21.

The voice recognition server 30 includes a CPU 31, a ROM 32, a RAM 33, an HDD 34, and a communication I / F 35. Each of these parts is connected so as to be able to communicate with each other by a bus connection.

The CPU 31 controls the entire voice recognition server 30. Specifically, the CPU 31 uses the RAM 33 as a work area to perform arithmetic processing based on the firmware, the OS, and various programs stored in the ROM 32, the HDD 34, and the like. Then, the CPU 31 controls each hardware included in the voice recognition server 30 based on the result of this arithmetic processing. Here, the various programs are, for example, programs for realizing the above-mentioned multiple voice combined processing.

The ROM 32 and the HDD 34 store firmware, an OS, various programs, and various data used in these programs (for example, various data used in the above-mentioned combined voice processing).

As described above, the RAM 33 functions as a work area of the CPU 31.
The communication I / F 35 is an interface for the voice recognition server 30 to communicate with each other device included in the voice recognition system S.

[Functional configuration]
Next, with reference to FIG. 5, the functional configuration realized by each of the above-mentioned hardware will be described with reference to FIG. FIG. 6 is a functional block diagram showing a functional configuration for executing a plurality of voice combined processing among the functional configurations of each microphone 10, the conference terminal 20, and the voice recognition server 30 included in the voice recognition system S. be.

It should be noted that these functional blocks are each hardware included in each device based on a program for each CPU included in each microphone 10, the conference terminal 20, and the voice recognition server 30 described above to realize a plurality of voice combined processing. It is realized by controlling the wear. It should be noted that, even if not specifically mentioned below, data necessary for realizing multiple voice combined processing is appropriately transmitted and received between these functional blocks at an appropriate timing.

Further, in the present embodiment, in order to process each voice data in parallel, a plurality of some functional blocks are provided in parallel. However, the functions of a plurality of functional blocks having the same name provided in parallel are common to each other. Therefore, in the following description, the alphabet at the end of each functional block will be omitted.

First, the functional blocks of each microphone 10 will be described.
When the plurality of voice combined processing is executed, as shown in FIG. 6, the voice pick-up unit 110 and the voice transmission unit 120 function in each microphone 10.

The voice pick-up unit 110 creates voice data of a digital signal by picking up the voice of each user U and analog-digital conversion.
The voice transmission unit 120 transmits the voice data created by the voice sound collection unit 110 to the conference terminal 20.

Next, the functional block of the conference terminal 20 will be described.
When the plurality of voice combined processing is executed, as shown in FIG. 6, in the conference terminal 20, the voice acquisition unit 210, the volume determination unit 220, the voice summing unit 230, and the character string display unit 240 function.

The voice acquisition unit 210 acquires voice data by receiving the voice data transmitted by the voice transmission unit 120.

The volume determination unit 220 determines whether or not the volume of the voice data received by the voice acquisition unit 210 is less than the threshold value. The value of this threshold value shall be set in advance according to the environment or the like in which the present embodiment is implemented. Further, the value of this threshold value may be appropriately changed based on the average value of the volume of each voice data and the like.
In addition, the volume determination unit 220 considers the case where the utterance of the user U is temporarily interrupted (for example, about several seconds), and the average value of the volume of the voice data in a certain period longer than this temporarily interrupted period is set. , It is good to judge whether or not it is less than the threshold value.

The voice summing unit 230 functions exclusively when performing the first plurality of voice combined processing. When the first plurality of voice combined processing is performed, the voice summing unit 230 sums up the plurality of voice data when the volume determination unit 220 determines that the volume of the plurality of voice data is less than the threshold value. Then, the voice summing unit 230 transmits the summed voice data to the voice recognition server 30. On the other hand, when the volume determination unit 220 determines that the volume of any of the voice data of the plurality of voice data is equal to or higher than the threshold value, the voice summing unit 230 determines that the volume is equal to or higher than this threshold value. The voice data is transmitted to the voice recognition server 30, and the voice data determined to be less than the threshold value is not transmitted.
When performing the second plurality of voice combined processing, the voice summing unit 230 transmits all of the plurality of voice data to the voice recognition server 30 regardless of the determination result of the volume determination unit 220.

The character string display unit 240 displays the voice recognition result received from the voice recognition server 30. The voice recognition result is displayed as, for example, a character string converted into text. FIG. 7 shows an example of display by the character string display unit 240. As shown in FIG. 7, the conference terminal 20 is realized as, for example, an electronic information board. In this case, the operation unit 26 and the display unit 27 are realized as a touch panel. Then, the display unit 27 is provided with, for example, a display area 271 as a display area for processing. The character string display unit 240 displays, for example, a character string in the display area 271 along a time series spoken by the user U.

In this case, information for identifying the user U (the microphone 10 worn by the user U) (for example, the name of the user U registered in advance, the number of the microphone 10 and the like) may be displayed together with the corresponding text. In the case of displaying in this way, a plurality of character string display units 240 may be provided corresponding to the plurality of microphones 10 in the same manner as the voice summing unit 230 and the like.
By performing such a display, the utterance history of who made what kind of remark in a conference scene in which a plurality of speakers exist is displayed.

Next, the functional block of the voice recognition server 30 will be described.
When the plurality of voices combined processing is executed, as shown in FIG. 6, the voice recognition unit 310 and the recognition result correction unit 320 function in the voice recognition server 30.

The voice recognition unit 310 performs voice recognition processing on the voice data received from the conference terminal 20. The voice recognition engine used for the voice recognition processing is not particularly limited, and a voice recognition engine peculiar to the present embodiment may be used, or a general-purpose voice recognition engine may be used.

The recognition result correction unit 320 functions exclusively when performing the second plurality of voice combined processing. When the second plurality of voices are used together, the recognition result correction unit 320 recognizes the voices of the plurality of voice data when the volume determination unit 220 determines that the volume of the plurality of voice data is less than the threshold value. The speech recognition result is corrected (ensemble) based on the comparison of the results. Then, the recognition result correction unit 320 transmits the corrected voice recognition result to the conference terminal 20. On the other hand, the voice summing unit 230 is determined to be equal to or higher than this threshold when the volume determination unit 220 determines that the volume of any of the voice data is equal to or higher than the threshold. The voice recognition result regarding the voice data is transmitted to the conference terminal 20. The voice recognition result is transmitted as, for example, a text-converted character string.

When performing the first multiple voice combined processing, only voice data that can be appropriately recognized, such as voice data whose volume is equal to or higher than the threshold value and voice data whose S / N ratio is improved by being added up, is the target of voice recognition. It has become. Therefore, the recognition result correction unit 320 transmits all the voice recognition results by the voice recognition unit 310 to the conference terminal 20 regardless of the determination result of the volume determination unit 220.

[motion]
Next, the flow of the multiple voice combined processing in the present embodiment will be described. Even if it is not particularly mentioned in the following description, each functional block described above with reference to FIG. 6 appropriately executes a process required for a plurality of voice combined processes. It should be noted that which of the first plurality of voice combined processing and the second plurality of voice combined processing is performed is determined according to the setting made in advance and the selection operation by the user U.

(1st multiple voice combined processing)
FIG. 8 is a flowchart illustrating the flow of the first plurality of voice combined processing. The first plurality of voice combined processing is executed, for example, when the sound collection by the microphone 10 is started and the acquisition of voice data is started, or in response to the start instruction operation by the user U.

In step S11, the loop processing of the first voice combined processing is started.
In step S12, each voice acquisition unit 210 of the conference terminal 20 acquires voice data from each microphone 10.

In step S13, each volume determination unit 220 of the conference terminal 20 determines whether or not the average value of the volume of the audio data in a certain period is less than the threshold value.

In step S14, the voice summing unit 230 of the conference terminal 20 determines whether or not the average value of the volumes of all the voice data in a certain period is less than the threshold value based on the determination result in step S13. If the average value of the volumes of all the voice data is less than the threshold value, it is determined as Yes in step S14, and the process proceeds to step S15. On the other hand, if the average value of the volume of at least one of the voice data is equal to or higher than the threshold value, it is determined as No in step S14, and the process proceeds to step S17.

In step S15, the voice summing unit 230 of the conference terminal 20 selects each voice data collected by each microphone 10. This process is performed in parallel for each voice data (here, steps S15A to S15C are performed as an example).
In step S16, the voice summing unit 230 of the conference terminal 20 sums up the voice data selected in step S15.
On the other hand, in step S17, the voice summing unit 230 of the conference terminal 20 selects voice data having a volume equal to or higher than the threshold value.

In step S18, the voice recognition unit 310 of the voice recognition server 30 adds voice in step S16 to improve the S / N ratio, or voice data having a volume equal to or higher than the threshold value selected in step S17. Perform recognition processing.

In step S19, the character string display unit 240 of the conference terminal 20 outputs a character string in which the voice recognition result in step S18 is converted into text. In this case, the output is performed, for example, by displaying as described above with reference to FIG. 7, printing on a paper medium, or the like.

If the condition for ending the loop processing of the first plurality of voice combined processing is not satisfied in step S20, the above-mentioned loop processing is repeated from step S11. On the other hand, when the condition for ending the loop processing of the first plurality of voice combined processing is satisfied, the loop processing ends. The end conditions are, for example, that the sound collection by the microphone 10 is completed and the acquisition of the voice data is completed, and that the end instruction operation by the user U is accepted.
By the first plurality of voice combined processing described above, it is possible to suppress a decrease in the accuracy of voice recognition even when the voice of the user U deviates from the pointing range of the microphone.

(Second multiple voice combined processing)
FIG. 9 is a flowchart illustrating the flow of the second plurality of voice combined processing. The second plurality of voice combined processing is executed, for example, when the sound collection by the microphone 10 is started and the acquisition of voice data is started, or in response to the start instruction operation by the user U.

In step S31, the loop processing of the second voice combined processing is started.
In step S32, each voice acquisition unit 210 of the conference terminal 20 acquires voice data from each microphone 10.

In step S33, each volume determination unit 220 of the conference terminal 20 determines whether or not the average value of the volume of the audio data in a certain period is less than the threshold value.
In step S34, the voice summing unit 230 of the conference terminal 20 determines whether or not the average value of the volumes of all the voice data in a certain period is less than the threshold value based on the determination result in step S33. If the average value of the volumes of all the voice data is less than the threshold value, it is determined as Yes in step S34, and the process proceeds to step S35. On the other hand, if the average value of the volume of at least one of the voice data is equal to or higher than the threshold value, No is determined in step S34, and the process proceeds to step S39.

In step S35, the voice summing unit 230 of the conference terminal 20 selects each voice data collected by each microphone 10. This process and the subsequent processes of steps S36 and S37 are performed in parallel for each voice data. Here, as an example, steps S35A to S35C, steps S36A to S36C, and steps S37A to S37C are performed.

In step S36, the voice recognition unit 310 of the voice recognition server 30 performs voice recognition processing on each voice data selected in step S35.
In step S37, the voice recognition unit 310 of the voice recognition server 30 outputs the result of the voice recognition process in step S36 as a text string.

In step S38, the recognition result correction unit 320 of the voice recognition server 30 corrects the voice recognition result based on the comparison of the voice recognition results of each of the plurality of voice data.

On the other hand, in step S39, the voice summing unit 230 of the conference terminal 20 selects voice data having a volume equal to or higher than the threshold value.
In step S40, the voice recognition unit 310 of the voice recognition server 30 performs voice recognition processing on voice data having a volume equal to or higher than the threshold value selected in step S39.

In step S41, the character string display unit 240 of the conference terminal 20 outputs the corrected character string in step S38 or the character string obtained by converting the voice recognition result in step S40 into text. In this case, the output is performed, for example, by displaying as described above with reference to FIG. 7, printing on a paper medium, or the like.

If the condition for ending the loop processing of the second plurality of voice combined processing is not satisfied in step S42, the above-mentioned loop processing is repeated from step S31. On the other hand, when the condition for ending the loop processing of the second plurality of voice combined processing is satisfied, the loop processing ends. The end conditions are, for example, that the sound collection by the microphone 10 is completed and the acquisition of the voice data is completed, and that the end instruction operation by the user U is accepted.
By the second plurality of voice combined processing described above, it is possible to suppress a decrease in the accuracy of voice recognition even when the voice of the user U deviates from the pointing range of the microphone.

[Modification example]
The present invention is not limited to the above-described embodiment, and modifications, improvements, and the like to the extent that the object of the present invention can be achieved are included in the present invention.

[First modification]
In the above-described embodiment, the character string is displayed based on the voice recognition process. Not limited to this, other information may be displayed. For example, the image of the user U who made the utterance corresponding to the character string may be displayed in association with each other.

This modification will be described with reference to FIGS. 10, 11, and 12. First, as a premise of display, as shown in FIG. 10, the imaging unit 28 is arranged at a position where each user U participating in the conference can be imaged. For example, the image pickup unit 28 is arranged at the upper part of the front surface of the conference terminal 20 realized as an electronic information board. As a result, each user U having a conference at a position facing the conference terminal 20 can be imaged.

Next, as shown in FIG. 11, each user U is created by performing image analysis on an image (or moving image) in which each user U is created by taking an image. Detects the face and the like of the user, and identifies who is speaking among the users U. This identification can be realized by face detection or motion detection by image analysis based on a generally known algorithm. These image analysis functions are implemented, for example, in the voice summing unit 230 of the conference terminal 20. Then, the voice summing unit 230 associates the face image of the uttering user U identified in this way with the voice picked up at the same time and transmits it to the voice recognition server 30.

Then, the recognition result correction unit 320 of the voice recognition server 30 displays the character string as a result of the voice recognition process and the associated face image of the spoken user U on the character string display unit 240 of the conference terminal 20. Send to. Then, the character string display unit 240 of the conference terminal 20 displays the character string and the face image of the spoken user U in association with each other in the display area 271. For example, it is displayed as shown in FIG. As a result, the user U who refers to the display improves the understanding of the utterance content and the sense of presence. That is, the convenience of the voice recognition system S is improved.

By registering data such as images and features of the user U participating in this conference in advance, it is possible to identify the participating user U with higher accuracy.

[Second modification]
When the user U participating in the conference is specified as in the first modification described above, the accuracy of voice recognition is improved by switching to the voice recognition model specialized for the specified user U individual. Can be made to. In this case, even if the individual user U cannot be identified, if the attributes such as male or female are known, the accuracy of voice recognition can be improved by using each voice recognition model.

This modification will be described with reference to FIGS. 13 and 14. In this modification, the voice recognition unit 310 of the voice recognition server 30 includes a plurality of functional blocks. Specifically, as shown in FIG. 13, a face recognition result receiving unit 311, a first voice recognition model 312, a second voice recognition model 313, a third voice recognition model 314, and a voice recognition processing unit 315 are included.

Then, in step S18 in FIG. 8, each process shown in FIG. 14 is performed. Specifically, in step S181, the face authentication result receiving unit 311 receives the result of face authentication by the voice summing unit 230 of the conference terminal 20. Then, based on the received face authentication result, the face authentication result receiving unit 311 switches the voice recognition model by switching the logical switch. For example, switch as follows.

In step S182, it is determined whether or not the result of face recognition is "male". If it is a "male", it is determined to be Yes in step S182, and the process proceeds to step S183. In step S183, the face recognition result receiving unit 311 switches to the first voice recognition model 312, which is a voice recognition model for men, and then the voice recognition processing unit 315 performs voice recognition processing. On the other hand, if it is not "male", it is determined as No in step S182, and the process proceeds to step S184.

In step S184, it is determined whether or not the result of face recognition is "female". If it is a "female", it is determined to be Yes in step S184, and the process proceeds to step S185. In step S185, the face recognition result receiving unit 311 switches to the second voice recognition model 313, which is a voice recognition model for women, and then the voice recognition processing unit 315 performs voice recognition processing. On the other hand, if it is not a "female", it is determined as No in step S184, and the process proceeds to step S186. In step S185, the face recognition result receiving unit 311 switches to the third voice recognition model 314, which is a general-purpose voice recognition model, and then the voice recognition processing unit 315 performs voice recognition processing.

As described above, the accuracy of voice recognition can be improved by using an appropriate voice model based on the result of face recognition.
It should be noted that the present modification may be applied not only to step S18 but also to steps S36 and S40 in FIG. 9, and the voice model may be switched as described above.

[Third variant]
The device configuration and the separation of functional blocks in the above-described embodiment are merely examples, and are not limited to these. For example, the function of the voice recognition server 30 may be mounted on the conference terminal 20 and realized as a single device. Alternatively, the conference terminal 20 which is an edge device may be realized by a mere communication relay device, and the function of the conference terminal 20 may be implemented in the voice recognition server 30. In this case, for example, the result of the voice recognition process may be displayed by a device other than the conference terminal 20.
Alternatively, each of the conference terminal 20 and the voice recognition server 30 may be realized by a plurality of devices. For example, the voice recognition server 30 may be realized by the cooperation of a plurality of cloud servers.

That is, the functional block included in each of the above-mentioned devices or an alternative functional block may be realized by any of the devices included in the voice recognition system S. In other words, the functional configuration of FIG. 6 is merely an example and is not particularly limited. That is, it is sufficient that each device included in the speech recognition system S has a function capable of executing the above-mentioned series of processes as a whole, and what kind of functional block is used to realize this function is particularly determined in FIG. Not limited to the example of.

As an example, when the functional block is realized by the functional configuration in the above-described embodiment, the voice recognition system S corresponds to the "voice recognition system" in the present invention. In this case, the microphone 10 corresponds to the "sound collecting device" in the present invention. Further, in this case, the voice acquisition unit 210 corresponds to the "acquisition means" in the present invention. Further, in this case, the volume determination unit 220 corresponds to the "determination means" in the present invention. Further, in this case, the voice summing unit 230, the voice recognition unit 310, and the recognition result correction unit 320 correspond to the "voice recognition processing means" and the "discrimination means" in the present invention.

[Other variants]
The series of processes described above can be executed by hardware or software. Further, one functional block may be configured by a single piece of hardware, a single piece of software, or a combination thereof. For example, the functional configuration in this embodiment is realized by a processor that executes arithmetic processing.

The processor that can be used in this embodiment includes a processor composed of various processing units such as a single processor, a multiprocessor, and a multicore processor. In addition, the present invention includes a combination of these various processing devices and a processing circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array).

When a series of processes are executed by software, a program constituting the software is installed in a computer or the like from a network or a recording medium. The computer may be a computer embedded in dedicated hardware. Further, the computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

The recording medium containing such a program is not only composed of removable media distributed separately from the device main body in order to provide the program to the user, but is also provided to the user in a state of being preliminarily incorporated in the device main body. It is composed of recording media and the like. The removable media is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like.

The optical disk is composed of, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), a Blu-ray (registered trademark) Disc (Blu-ray disc), or the like. The magneto-optical disk is composed of MD (Mini-Disk) or the like. Further, the recording medium provided to the user in a state of being preliminarily incorporated in the apparatus main body is composed of, for example, ROM 12, ROM 22, and ROM 32, HDD 24, HDD 34, or the like shown in FIG. 5 in which a program is recorded. To.

In the present specification, the steps for describing a program to be recorded on a recording medium are not only processed in chronological order but also in parallel or individually, even if they are not necessarily processed in chronological order. It also includes the processing to be executed. Further, in the present specification, the term of the system means an overall device composed of a plurality of devices, a plurality of means, and the like.

Although some embodiments of the present invention have been described above, these embodiments are merely examples and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and further, various modifications such as omission and substitution can be made without departing from the gist of the present invention. These embodiments and variations thereof are included in the scope and gist of the invention described in the present specification and the like, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

S printing system 10 microphone 20 conference terminal 30 voice recognition server 11, 21, 31, 41 CPU
12, 22, 32, 42 ROM
13, 23, 33, 43 RAM
14, 25, 35 Communication I / F
16 Sound pickup unit 24, 34 HDD
26 Operation unit 27 Display unit 28 Imaging unit 110 Voice pick-up unit 120 Voice transmission unit 210 Voice acquisition unit 220 Volume determination unit 230 Voice summing unit 240 Character string display unit 310 Voice recognition unit 311 Face recognition result reception unit 312 First voice recognition Model 313 2nd voice recognition model 314 3rd voice recognition model 315 Voice recognition processing unit 320 Recognition result correction unit

Japanese Unexamined Patent Publication No. 2017-167318

Claims (9)

An acquisition means for acquiring voice data of voices spoken by a plurality of users in the same space from a plurality of sound collecting devices worn by each of the plurality of users.
A determination means for determining whether or not the volume of the acquired plurality of voice data is less than the threshold value, and
When the determination means determines that the volume of the plurality of voice data is less than the threshold value, the voice recognition processing means that performs the voice recognition processing together with the predetermined processing using the plurality of voice data in combination.
A voice recognition system equipped with. The voice recognition processing means is
As a predetermined process using the plurality of voice data in combination, the plurality of voice data are added up.
Performs voice recognition processing on the summed voice data.
The voice recognition system according to claim 1. The voice recognition processing means is
A voice recognition process is performed on each of the plurality of voice data, and the voice recognition process is performed.
As a predetermined process using the plurality of voice data in combination, the voice recognition result is corrected based on the comparison of the voice recognition results of each of the plurality of voice data.
The voice recognition system according to claim 1. The voice recognition processing means corrects the voice recognition result based on the most voice recognition result when the voice recognition result of each of the plurality of voice data is different in the comparison.
The voice recognition system according to claim 3. When the determination means determines that at least one of the volumes of the plurality of voice data is equal to or higher than the threshold value, the voice recognition processing means performs voice recognition processing for the voice data determined to be equal to or higher than the threshold value. On the other hand, voice recognition processing is not performed on other voice data.
The voice recognition system according to any one of claims 1 to 4. When it is determined that the volume of the plurality of voice data is less than the threshold value, the voice recognition processing means estimates the user who has spoken based on the comparison result of the volume of each of the plurality of voice data.
The voice recognition system according to any one of claims 1 to 5. Further equipped with the plurality of sound collecting devices,
The plurality of sound collecting devices are neck-mounted or badge-type sound collecting devices.
The voice recognition system according to any one of claims 1 to 6. Further provided with an identification means for identifying the user based on an image in which the plurality of users are subjects.
The voice recognition processing means makes a difference in the method of voice recognition processing for each of the users based on the identification result of the user.
The voice recognition system according to any one of claims 1 to 7. It is a voice recognition method performed by a voice recognition system.
An acquisition step of acquiring voice data of voice uttered by a plurality of users in the same space from a plurality of sound collecting devices worn by each of the plurality of users.
A determination step for determining whether or not the volume of the acquired plurality of voice data is less than the threshold value, and
A voice recognition processing step that performs voice recognition processing together with a predetermined process using the plurality of voice data when it is determined by the determination step that the volume of the plurality of voice data is less than the threshold value.
Voice recognition method to have.

Images