JP2022120164A - Voice recognition system, voice recognition method, and voice processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a voice recognition system and a voice recognition method capable of suppressing an accuracy deterioration in voice recognition even in the case that user's voice is out of a directional range of a microphone.

SOLUTION: A voice recognition system provided by the present invention includes: acquisition means for acquiring voice data of voices uttered by a plurality of users within the same space from a plurality of sound collection devices respectively attached to the plurality of users; determination means for determining whether or not voice volumes of the plurality of pieces of acquired voice data are less than a threshold; and voice recognition means for performing voice recognition processing with prescribed processing using the plurality of pieces of voice data in combination in the case that the determination means determines that any of the voice volumes of the plurality of pieces of voice data is less than the threshold.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to a speech recognition system and a speech recognition method.

In recent years, speech recognition technology has been widely used. For example, speech recognition technology is widely used for applications such as accepting user voice operations on AI (Artificial Intelligence) speakers and recording conversations with customers at call centers.

In these uses, the distance between the mouth of the user who speaks and the microphone that picks up the voice spoken by the user is close. As a result, the S/N ratio (signal-noise ratio), which indicates the ratio of voice to noise, is high, and voice recognition can be performed with high accuracy.

On the other hand, for example, there is a case where a table microphone is placed in the center of the conference table to collectively collect the utterances of the conference participants for the purpose of creating the minutes of the conference. In this case, the distance between the speaker and the microphone is several tens of centimeters or more. As a result, the S/N ratio becomes low, and the problem arises that the accuracy of speech recognition is lowered.

An example of technology that takes such problems into account is disclosed in Japanese Patent Application Laid-Open No. 2002-200010. In the technology disclosed in Patent Literature 1, each user participating in a conference is made to wear a microphone that is directional with respect to the mouth of the user. Then, voice data picked up by each of the plurality of microphones is appropriately selected and voice recognition processing is performed. As a result, speech recognition can be performed without causing the problem that the distance between the speaker and the microphone increases.

However, the technique disclosed in Patent Document 1 does not consider the case where the user speaks in a direction different from the direction of the microphone and the user's voice is out of the directional range of the microphone. When the user's voice is out of the directional range of the microphone in this manner, the user's voice is picked up while being buried in other users' voices and noise, resulting in deterioration in voice recognition accuracy.

The present invention has been made in view of such circumstances, and a voice recognition system capable of suppressing deterioration in voice recognition accuracy even when the user's voice is out of the directional range of the microphone. , and to provide a speech recognition method.

In order to solve the above-described problems and achieve the object, the speech recognition system provided by the present invention provides a speech recognition system in which speech data of speech uttered by a plurality of users in the same space is received by each of the plurality of users. acquisition means for acquiring from a plurality of sound collecting devices; determination means for determining whether or not the volume of the plurality of acquired audio data is less than a threshold; voice recognition processing means for performing predetermined processing using the plurality of voice data together and voice recognition processing when it is determined that the voice data is less than the threshold.

ADVANTAGE OF THE INVENTION According to the present invention, it is possible to provide a speech recognition system and a speech recognition method capable of suppressing deterioration in accuracy of speech recognition even when the user's speech is out of the directional range of the microphone. .

1 is a schematic view of the overall configuration of a speech recognition system according to an embodiment of the present invention and a space where a user using the speech recognition system exists; FIG. FIG. 2 is a schematic diagram showing an example of wearing when the microphone of the embodiment of the present invention is realized by a wearable neck-type microphone. It is a timing chart which shows an example of the situation of each user's utterance. It is a timing chart which shows an example of the situation of each user's utterance. 2 is a block diagram showing the hardware configuration of each microphone, conference terminal, and speech recognition server included in the speech recognition system; FIG. FIG. 2 is a functional block diagram showing a functional configuration for executing multi-voice combined processing among the functional configurations of each microphone, conference terminal, and voice recognition server included in the voice recognition system; FIG. 3 is a schematic diagram showing an external configuration and a display example of a conference terminal; 4 is a flowchart for explaining the flow of first multiple-speech combined processing in one embodiment of the present invention. FIG. 11 is a flow chart for explaining the flow of a second plural-speech combination processing in one embodiment of the present invention; FIG. FIG. 11 is a schematic diagram showing an external configuration and a display example of a conference terminal in a first modified example; It is a schematic diagram showing about the image analysis in the 1st modification and the 2nd modification. FIG. 11 is a schematic diagram showing a display example on the conference terminal in the first modified example; FIG. 11 is a schematic diagram showing a configuration example of a speech recognition unit in a second modified example; FIG. 11 is a flow chart for explaining the flow of processing in a second modified example; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[System configuration]
FIG. 1 is a schematic view of the overall configuration of a speech recognition system S according to this embodiment and a space (here, a conference room as an example) where users who use the speech recognition system S are present. As shown in FIG. 1, the speech recognition system S includes a plurality of microphones 10 (here, as an example, microphones 10A to 10F), a conference terminal 20, and a speech recognition server 30. FIG.

A table is placed in the center of the conference room including the windows and doors, and a plurality of users U (here, users UA to UF as an example) participating in the conference are positioned around the table. Each of these users U wears a microphone 10 having the same alphabet at the end of the code as their own code. Note that the number of users U and the number of microphones 10 and the like are merely examples, and are not particularly limited.

Each microphone 10 functions as a sound pickup device that picks up each user's U sound. Although the shape of each microphone 10 is not particularly limited, each microphone 10 can be realized by a wearable microphone that is attached to the user U as a neck type or a badge type, for example. With such a wearable microphone, for example, even a user U who finds a headset or a pin microphone annoying, or a user U who dislikes using a microphone worn by another person can wear it without worrying about it.

An example of mounting the microphone 10 will be described with reference to FIG. FIG. 2 is a schematic diagram showing an example of wearing when the microphone 10 is realized by a neck-type wearable microphone. In this example, the microphone 10 has directivity toward the mouth, in order to collect intensively the voice uttered from the mouth of the user U wearing the microphone 10 above the microphone 10 .

Therefore, as shown in FIG. 2A, when the user U speaks while facing the front, the voice uttered by the user U can be appropriately picked up. On the other hand, as shown in FIG. 2B, when the user U turns sideways or upwards and speaks in a direction different from the directivity direction of the microphone, and the voice of the user U is out of the directivity range of the microphone, The voice of this user U is picked up while being buried in voices of other users U and noise. In this embodiment, as shown in FIG. 2(B), by performing the "multi-speech combination processing", even if the user U's speech is out of the directional range of the microphone, the accuracy of speech recognition can be prevented from deteriorating. Suppress. The details of this multiple voice combination processing will be described later.

Each microphone 10 converts an analog signal corresponding to the collected sound from analog to digital in an A/D conversion circuit, thereby creating audio data of a digital signal. Then, each microphone 10 transmits the created voice data to the conference terminal 20 by communication. Although such a communication method is not particularly limited, for example, it can be realized by wireless communication such as Bluetooth (registered trademark) that allows many-to-many connection. In this way, by using a communication method that allows many-to-many connections, voices uttered by each user U can be collected in parallel.

The conference terminal 20 transmits the voice data received from each microphone 10 to the voice recognition server 30 . Such a communication method is not particularly limited, but can be realized by wired or wireless communication via a network such as the Internet or a LAN (Local Area Network), for example. The conference terminal 20 may be realized by a relay device or the like having a communication function, or may be realized by a device such as an electronic information board on which the user U can input characters or the like by performing an input operation on the display. In the following description, an example in which the conference terminal 20 is implemented by an electronic information board will be described. The electronic information board is also called an interactive whiteboard (IWB), an electronic blackboard, or the like.

The voice recognition server 30 is a server that performs voice recognition processing on a plurality of voice data received from each microphone 10 . In the voice recognition server 30, multiple voice recognition engines corresponding to multiple voice data received from the microphone 10 operate in parallel. As a result, parallel speech recognition processing can be realized for the speech data of each user U in real time. The result of speech recognition processing is used by, for example, converting it into text and displaying it on the display unit of the conference terminal 20 or the like in real time, or by printing it on a paper medium after the conference is over. The voice recognition server 30 can be realized by, for example, a cloud server provided on the cloud.

[Multi-sound combination processing]
The speech recognition system S having such a configuration performs multiple speech combination processing as described above. Here, the multiple-speech combined processing is a series of processing for suppressing deterioration in accuracy of speech recognition by using speech data of a plurality of users U together.

Specifically, in the multiple voice combination processing, the voice recognition system S acquires voice data of voices uttered by multiple users U in the same space from multiple microphones 10 worn by each of the multiple users U. Also, the speech recognition system S determines whether or not the volume of the plurality of acquired speech data is less than the threshold. Here, the case where the volume of the plurality of acquired audio data is less than the threshold is, for example, the case where the voice of the user U is out of the directivity range of the microphone as shown in FIG. 2(B) described above.

Then, when it is determined that the volumes of all of the plurality of speech data are less than the threshold value, the speech recognition system S performs speech recognition processing together with predetermined processing using the plurality of speech data. As the predetermined processing using a plurality of voice data together, for example, there are two types of processing: a first multiple voice combined processing and a second multiple voice combined processing.

(First multiple voice combination processing)
In the first multiple-speech combined processing, the speech recognition system S adds up a plurality of speech data, and performs speech recognition processing on the added speech data. As a result, speech recognition processing can be performed on the speech data whose S/N ratio has been improved by the summation, so that deterioration in accuracy of speech recognition can be suppressed.

The first multiple voice combination processing will be described with reference to FIGS. 3 and 4. FIG. 3 and 4 are timing charts showing an example of the state of each user U's speech.

In this example, as shown in FIG. 4, it is assumed that three users U (user UA, user UB, and user UC) sequentially speak at different timings (including partially overlapping timings). do. Specifically, it is assumed that utterances are made in the chronological order of “user UA→user UB→user UC→user UB→user UA”. However, it is assumed that the user UA who made the last utterance among these utterances turned sideways and fell out of the directional range of the microphone as shown in FIG. 2B described above.

Each of the microphones 10 (microphone 10A, microphone 10B, and microphone 10C) worn by each of these three users U picks up the voice uttered by the user U wearing the microphone 10 the loudest. Also, voices uttered by other users U are also picked up, albeit in small amounts. For example, between T1 and T2 in the figure, the microphone 10A picks up the voice uttered by the user UA loudly, and the microphones 10B and 10C also pick up the voice uttered by the user UA, albeit in small amounts. Although not shown in the drawing, the microphone 10 actually picks up a small amount of noise in the conference room (for example, the operating sound of air conditioners, projectors, etc.).

In such a situation, in this embodiment, a threshold is set for the volume as described above, and if the volume of the audio data of at least one microphone 10 is equal to or higher than the threshold, Perform voice recognition. That is, when at least one microphone 10 can appropriately pick up the voice uttered by the user U who is wearing the device, voice recognition is performed on the voice data of this voice. Then, voice recognition is not performed for voice data with a volume less than the threshold value picked up by the other microphones 10 .

For example, during a period from T1 to T2, voice recognition is performed on the voice data of the voice uttered by the user UA picked up by the microphone 10A. During the period from T2 to T3, voice recognition is performed on the voice data of the voice uttered by the user UB picked up by the microphone 10B. Furthermore, during the period from T3 to T4, voice recognition is performed on the voice data of the voice uttered by the user UC picked up by the microphone 10C. Furthermore, during the period from T4 to T5, voice recognition is performed on the voice data of the voice uttered by the user UB picked up by the microphone 10B. Other audio data whose volume is less than the threshold is excluded from speech recognition targets.

In the period from T1 to T2, the voice uttered by the user UA picked up by the microphone 10A and the voice uttered by the user UB picked up by the microphone 10B partially overlap in terms of time. , the microphone 10 can pick up sound at a volume equal to or higher than the threshold. Therefore, the voice data of the voice uttered by the user UA and the voice data of the voice uttered by the user UB in this overlapping period are used for voice recognition.

Here, in the present embodiment, predetermined processing using a plurality of voice data is applied during the period from T6 to T7 when the user UA speaks while looking sideways. During the period from T6 to T7, all of the microphones 10A, 10B, and 10C pick up only sounds with volumes below the threshold. Therefore, the S/N ratio of the voice picked up by each microphone 10 is low, and even if voice recognition is performed on the voice data of the voice picked up by each microphone 10 as it is, the voice uttered by the user UA will not be appropriate. voice recognition is not possible.

In this way, when the user U speaks while facing sideways or upward, the voice of the user U is out of the directional range of the microphone, and the volume of the voice data of any microphone 10 is less than the threshold, a plurality of voice data Performs a predetermined process in combination with Specifically, as shown in FIG. 4, voice data of voices picked up by a plurality of microphones 10 are combined to improve the S/N ratio. Then, speech recognition is performed on speech data in which the speech is emphasized by improving the S/N ratio.

As a result, even when the voice of the user U is out of the directional range of the microphone, it is possible to prevent the accuracy of voice recognition from deteriorating.

In this case, the uttering user U may be estimated based on the result of comparing the volumes of a plurality of voice data. For example, when the volume of the audio data of the microphone 10A is relatively higher than the volume of the audio data of the microphones 10B and 10, it may be estimated that the user UA corresponding to the microphone 10A has spoken. However, if the estimation is difficult, as shown in FIG. 4, the speaker may be output from the voice summation unit 230, which will be described later, as an unspecified speaker.

(Second multiple voice combination processing)
Although the above-described first multiple voice combination processing can suppress deterioration in accuracy of voice recognition, in order to perform summation of voice data, strict timing control (for example, microsecond order) of voice data acquired from each microphone 10 is required. strict synchronous control, etc.). Also, a processing unit for performing addition processing is required. Therefore, it is conceivable to perform a second plural-speech combined process as another simpler process.

In the second plural-voice combination processing as well, a predetermined process using a plurality of voice data is performed in the same manner as in the first plural-voice combination processing. That is, as in the period T6 to T7 shown in FIG. 3, the user U speaks while looking sideways or upward, the voice of the user U is out of the directivity range of the microphone, and the voice data of any microphone 10 is less than the threshold. In the case of volume, predetermined processing is performed using a plurality of audio data.

Specifically, the speech recognition system S performs speech recognition processing on each of the plurality of speech data, and corrects the speech recognition result based on a comparison of the speech recognition results of each of the plurality of speech data. As a result, correction can be performed based not only on the speech recognition result of a certain piece of speech data, but also on the basis of the speech recognition results of each of a plurality of speech data, thereby suppressing deterioration in accuracy of speech recognition.

Here, as an example of correction processing, in a certain section (for example, a section corresponding to a clause specified based on the speech recognition result), if the speech recognition results for a plurality of speech data are common, , correction is performed assuming that this common speech recognition result is correct.

Also, in this case, for example, if two out of the speech recognition results for three speech data are the same result, the speech recognition result is regarded as correct and corrected. It may be processed based on

Alternatively, if the results of multiple voice recognition processes are all different, the volume of each of the multiple voice data is compared, and the voice recognition result for the voice data with the highest volume is corrected as the correct one. good.

As a result, even in the second multiple voice combination processing, even if the voice of the user U is out of the directional range of the microphone, it is possible to suppress a decrease in accuracy of voice recognition.

[Hardware configuration]
Next, the hardware configuration of each device in this embodiment will be described with reference to FIG. FIG. 5 is a block diagram showing the hardware configuration of each microphone 10, conference terminal 20, and speech recognition server 30 included in the speech recognition system S. As shown in FIG.

The microphone 10 includes a CPU (Central Processing Unit) 11 , a ROM (Read Only Memory) 12 , a RAM (Random Access Memory) 13 , a communication I/F (Interface) 14 , an operation section 15 and a sound pickup section 16 . These units are connected so as to be able to communicate with each other through a bus connection.

The CPU 11 controls the microphone 10 as a whole. Specifically, the CPU 11 uses the RAM 13 as a work area (that is, a work area) to perform arithmetic processing based on firmware, an OS (Operating System), and various programs stored in the ROM 12 or the like. Then, the CPU 11 controls each hardware included in the microphone 10 based on the result of this arithmetic processing. Here, the various programs are, for example, programs for transmitting voice data to the conference terminal 20 in the multiple voice combination processing described above.

The ROM 12 stores firmware, an OS, various programs, and various data used in these programs (for example, voice data used in the above-described multiple voice combined processing).

The RAM 13 functions as a work area for the CPU 11 as described above.

Communication I/F 14 is an interface for microphone 10 to communicate with other devices included in speech recognition system S. FIG.

The operation unit 15 is implemented by, for example, various buttons and the like, and receives user's U operations. For example, the operation unit 15 receives an operation for switching on/off the power of the microphone 10 and an instruction operation for starting sound collection. Then, the operation unit 15 outputs to the CPU 11 the details of the received user U operation.

The sound pickup unit 16 includes a device that converts the picked-up sound into an analog electrical signal, and an A/D conversion circuit that converts the sound into a digital signal. The sound pickup unit 16 outputs the digitally converted sound data to the CPU 11 .

The conference terminal 20 includes a CPU 21 , ROM 22 , RAM 23 , HDD 24 , communication I/F 25 , operation section 26 , display section 27 and imaging section 28 . These units are connected so as to be able to communicate with each other through a bus connection.

The CPU 21 controls the entire conference terminal 20 . Specifically, the CPU 21 uses the RAM 23 as a work area to perform arithmetic processing based on firmware, OS, and various programs stored in the ROM 22, HDD 24, and the like. Then, the CPU 21 controls each hardware included in the conference terminal 20 based on the result of this arithmetic processing. Here, the various programs are, for example, a program for realizing the above-described multiple voice combined processing and a program for realizing the function of the electronic information board.

The ROM 22 and HDD 24 store firmware, OS, various programs, and various data used in these programs (for example, the above-described multiple voice combined processing and various data used in the function of the electronic information board). do.

The RAM 23 functions as a work area for the CPU 21 as described above.

The communication I/F 25 is an interface for the conference terminal 20 to communicate with other devices included in the speech recognition system S. FIG.

The operation unit 26 is implemented by various buttons, for example, and receives user's U operations. For example, the operation unit 15 accepts an on/off switching operation of the power of the conference terminal 20, an instruction operation to start collecting sound, and an operation related to the functions of the electronic information board. Then, the operation unit 26 outputs to the CPU 21 the content of the accepted user U operation.

The display unit 27 is implemented by a liquid crystal display (LCD), an organic EL display (organic electro luminescence display), or the like, and displays predetermined information output from the CPU 21 to the user U. The display unit 27 displays to the user U, for example, information obtained by converting speech recognition results into text and various user interfaces as predetermined information.

Note that the operation unit 26 and the display unit 27 may be realized integrally by a touch panel capable of accepting a touch operation by a pen for an electronic information board or by the user's U hand.

The image capturing unit 28 is realized by various devices for realizing a camera, and captures an image of, for example, the user U participating in the conference at the place where the conference terminal 20 is installed. The imaging unit 28 outputs image data created by imaging to the CPU 21 .

The voice recognition server 30 includes a CPU 31, a ROM 32, a RAM 33, an HDD 34, and a communication I/F 35. These units are connected so as to be able to communicate with each other through a bus connection.

The CPU 31 controls the voice recognition server 30 as a whole. Specifically, the CPU 31 uses the RAM 33 as a work area to perform arithmetic processing based on firmware, OS, and various programs stored in the ROM 32, HDD 34, and the like. Then, the CPU 31 controls each hardware included in the speech recognition server 30 based on the result of this arithmetic processing. Here, the various programs are, for example, programs for realizing the above-described multiple voice combined processing.

The ROM 32 and HDD 34 store firmware, an OS, various programs, and various data used in these programs (for example, various data used in the above-described multiple voice combined processing).

The RAM 33 functions as a work area for the CPU 31 as described above.

Communication I/F 35 is an interface for voice recognition server 30 to communicate with other devices included in voice recognition system S. FIG.

[Functional configuration]
Next, the functional configuration realized by each hardware described above with reference to FIG. 5 will be described with reference to FIG. FIG. 6 is a functional block diagram showing a functional configuration for executing multiple voice combined processing among the functional configurations of the microphones 10, the conference terminal 20, and the voice recognition server 30 included in the voice recognition system S. be.

It should be noted that these functional blocks are implemented by each CPU included in each microphone 10, conference terminal 20, and voice recognition server 30 described above, based on a program for realizing combined processing of multiple voices, and each hardware included in each device. It is realized by controlling hardware. In addition, data necessary for realizing the multi-voice combined processing is appropriately transmitted and received between these functional blocks at appropriate timings, including cases not particularly mentioned below.

Further, in this embodiment, a plurality of partial functional blocks are provided in parallel in order to process each audio data in parallel. However, the functions of functional blocks with the same names provided in parallel are common. Therefore, in the following description, alphabetical letters at the end of each functional block are omitted.

First, functional blocks of each microphone 10 will be described.

When the multiple voice processing is executed, as shown in FIG. 6, in each microphone 10, the voice pickup section 110 and the voice transmission section 120 function.

The voice recording unit 110 collects the voice of each user U and converts it from analog to digital to create voice data of a digital signal.

The voice transmission unit 120 transmits voice data created by the voice recording unit 110 to the conference terminal 20 .

Next, functional blocks of the conference terminal 20 will be described.

When multiple voice processing is executed, as shown in FIG. 6, in the conference terminal 20, the voice acquisition unit 210, the volume determination unit 220, the voice summation unit 230, and the character string display unit 240 function.

The voice acquisition unit 210 acquires voice data by receiving the voice data transmitted by the voice transmission unit 120 .

Volume determination unit 220 determines whether the volume of the audio data received by audio acquisition unit 210 is less than a threshold. The value of this threshold shall be set in advance according to the environment etc. in which this embodiment is implemented. Also, the threshold value may be appropriately changed based on the average value of the volume of each audio data or the like.

Note that the volume determination unit 220 considers the case where the user U's speech is temporarily interrupted (for example, for several seconds), so that the average value of the volume of the audio data in a certain period longer than the temporary interruption period is , is less than a threshold.

The speech summation unit 230 functions only when the first plural speech processing is performed. When performing the first plural-voice combined processing, voice summing section 230 sums the plurality of voice data when volume determination section 220 determines that the volume of each of the plurality of voice data is less than the threshold value. Then, the voice combining unit 230 transmits the combined voice data to the voice recognition server 30 . On the other hand, when the volume determining unit 220 determines that the volume of any one of the plurality of audio data is equal to or greater than the threshold, the audio combining unit 230 determines that the volume is equal to or greater than the threshold. The voice data is transmitted to the voice recognition server 30, and the voice data determined to be less than the threshold value is not transmitted.

Note that when performing the second plural-speech combination processing, the speech summation unit 230 transmits all of the plurality of speech data to the speech recognition server 30 regardless of the determination result of the volume determination unit 220 .

The character string display section 240 displays the speech recognition result received from the speech recognition server 30 . A speech recognition result is displayed as a text string, for example. An example of display by the character string display unit 240 is shown in FIG. As shown in FIG. 7, the conference terminal 20 is implemented as an electronic information board, for example. In this case, the operation unit 26 and the display unit 27 are implemented as touch panels. For example, a display area 271 is provided in the display unit 27 as a display area for processing. The character string display unit 240 displays character strings in the display area 271 in chronological order of the user U's utterances, for example.

In this case, information identifying (the microphone 10 worn by) the user U (for example, the name of the user U registered in advance, the number of the microphone 10, etc.) may be displayed together with the corresponding text. When displaying in this manner, a plurality of character string display sections 240 may be provided corresponding to a plurality of microphones 10 in the same manner as the voice combining section 230 and the like.

By performing such a display, in a conference scene in which a plurality of speakers exist, the speech history of who said what kind of speech is displayed.

Next, functional blocks of the speech recognition server 30 will be described.

When multiple voice processing is executed, as shown in FIG. 6, the voice recognition unit 310 and the recognition result correction unit 320 function in the voice recognition server 30 .

The voice recognition unit 310 performs voice recognition processing on voice data received from the conference terminal 20 . A speech recognition engine used for speech recognition processing is not particularly limited, and a speech recognition engine unique to this embodiment may be used, or a general-purpose speech recognition engine may be used.

Recognition result correcting section 320 functions only when performing the second plural voice combination processing. When performing the second multiple voice combined processing, recognition result correcting unit 320 performs voice recognition of multiple voice data when volume determination unit 220 determines that the volume of each of the multiple voice data is less than the threshold value. Correct (ensemble) the speech recognition results based on the comparison of the results. The recognition result correction unit 320 then transmits the corrected speech recognition result to the conference terminal 20 . On the other hand, when the volume determining unit 220 determines that the volume of any one of the plurality of audio data is equal to or greater than the threshold, the audio combining unit 230 determines that the volume is equal to or greater than the threshold. A speech recognition result regarding the speech data is transmitted to the conference terminal 20 . A speech recognition result is transmitted as a text string, for example.

Note that when performing the first multiple voice combination processing, only voice data that can be properly recognized, such as voice data whose volume is equal to or greater than a threshold value, voice data whose S/N ratio has been improved by summing, etc., is the object of voice recognition. It has become. Therefore, the recognition result correction unit 320 transmits all speech recognition results by the speech recognition unit 310 to the conference terminal 20 regardless of the determination result of the volume determination unit 220 .

[motion]
Next, the flow of processing for using multiple voices in this embodiment will be described. It should be noted that each functional block described above with reference to FIG. 6 appropriately executes the processing necessary for the multiple voice combination processing, even if it is not specifically mentioned in the following description. It should be noted that which of the first multiple-voice combination processing and the second multiple-voice combination processing is to be performed is determined according to preset settings and a selection operation by the user U. FIG.

(First multiple voice combination processing)
FIG. 8 is a flow chart for explaining the flow of the first plural-sound combination processing. The first multiple voice processing is executed, for example, when sound collection by the microphone 10 is started and acquisition of voice data is started, or when the user U performs a start instruction operation.

In step S11, loop processing of the first voice combined processing is started.

In step S<b>12 , each voice acquisition unit 210 of the conference terminal 20 acquires voice data from each microphone 10 .

In step S13, each volume determining unit 220 of the conference terminal 20 determines whether or not the average value of volume of voice data in a certain period is less than a threshold.

In step S14, the voice summation unit 230 of the conference terminal 20 determines whether or not the average value of volume of all voice data for a certain period of time is less than a threshold based on the determination result in step S13. If the average value of volume of all audio data is less than the threshold value, it is determined as Yes in step S14, and the process proceeds to step S15. On the other hand, if the average value of the volume of at least one piece of audio data is equal to or greater than the threshold value, it is determined No in step S14, and the process proceeds to step S17.

In step S<b>15 , the voice combining unit 230 of the conference terminal 20 selects each voice data picked up by each microphone 10 . This processing is performed in parallel for each audio data (here, steps S15A to S15C are performed as an example).

In step S16, the voice summation unit 230 of the conference terminal 20 sums up each voice data selected in step S15.

On the other hand, in step S17, the voice combining unit 230 of the conference terminal 20 selects voice data whose volume is equal to or greater than the threshold.

In step S18, the speech recognition unit 310 of the speech recognition server 30 performs speech recognition on the speech data whose S/N ratio has been improved by summing in step S16, or the speech data whose volume is equal to or greater than the threshold value selected in step S17. Perform recognition processing.

In step S19, the character string display unit 240 of the conference terminal 20 outputs a character string obtained by converting the speech recognition result in step S18 into text. In this case, the output is performed by, for example, the display as described above with reference to FIG. 7, printing on a paper medium, or the like.

In step S20, if the condition for terminating the loop processing of the first plural-sound combination processing is not satisfied, the loop processing described above is repeated from step S11. On the other hand, when the condition for terminating the loop processing of the first multiple-voice combination processing is satisfied, the loop processing is terminated. The termination condition is, for example, that the pickup of sound by the microphone 10 has ended and the acquisition of voice data has ended, or that the end instruction operation by the user U has been received.

With the first multiple voice combination processing described above, even when the voice of the user U is out of the directional range of the microphone, it is possible to suppress a decrease in accuracy of voice recognition.

(Second multiple voice combination processing)
FIG. 9 is a flow chart for explaining the flow of the second plural-sound combination processing. The second multiple voice processing is executed, for example, when sound collection by the microphone 10 is started and acquisition of voice data is started, or when the user U performs a start instruction operation.

In step S31, loop processing of the second voice combined processing is started.

In step S<b>32 , each voice acquisition unit 210 of the conference terminal 20 acquires voice data from each microphone 10 .

In step S33, each volume determining unit 220 of the conference terminal 20 determines whether or not the average value of volume of voice data in a certain period is less than a threshold.

In step S34, the voice summation unit 230 of the conference terminal 20 determines whether or not the average value of volume of all voice data for a certain period of time is less than the threshold based on the determination result in step S33. If the average volume value of all audio data is less than the threshold value, the determination in step S34 is Yes, and the process proceeds to step S35. On the other hand, if the average value of the volume of at least one piece of audio data is equal to or greater than the threshold value, the determination in step S34 is No, and the process proceeds to step S39.

In step S<b>35 , the voice combining unit 230 of the conference terminal 20 selects each voice data picked up by each microphone 10 . This process and subsequent processes in steps S36 and S37 are performed in parallel for each audio data. Here, as an example, steps S35A to S35C, steps S36A to S36C, and steps S37A to S37C are performed.

In step S36, the speech recognition unit 310 of the speech recognition server 30 performs speech recognition processing on each piece of speech data selected in step S35.

In step S37, the speech recognition unit 310 of the speech recognition server 30 outputs the result of the speech recognition processing in step S36 as a text string.

In step S38, the recognition result correction unit 320 of the speech recognition server 30 corrects the speech recognition result based on the comparison of the speech recognition results of each of the plurality of speech data.

On the other hand, in step S39, the voice combining unit 230 of the conference terminal 20 selects voice data with a volume equal to or greater than the threshold.

In step S40, the speech recognition unit 310 of the speech recognition server 30 performs speech recognition processing on the speech data whose volume is greater than or equal to the threshold value selected in step S39.

In step S41, the character string display unit 240 of the conference terminal 20 outputs the character string after correction in step S38 or the character string obtained by textualizing the voice recognition result in step S40. In this case, the output is performed by, for example, the display as described above with reference to FIG. 7, printing on a paper medium, or the like.

In step S42, if the condition for terminating the loop processing of the second plural-sound combination processing is not satisfied, the loop processing described above is repeated from step S31. On the other hand, when the condition for terminating the loop processing of the second multiple-voice combination processing is satisfied, the loop processing is terminated. The termination condition is, for example, that the pickup of sound by the microphone 10 has ended and the acquisition of voice data has ended, or that the end instruction operation by the user U has been received.

With the second multiple voice combination processing described above, even when the voice of the user U is out of the directional range of the microphone, it is possible to suppress a decrease in accuracy of voice recognition.

[Modification]
The present invention is not limited to the above-described embodiments, but includes modifications, improvements, and the like within the scope of achieving the object of the present invention.

[First modification]
In the above-described embodiments, character strings are displayed based on speech recognition processing. Not limited to this, other information may be displayed. For example, an image of the user U who made the utterance corresponding to the character string may be associated and displayed.

This modification will be described with reference to FIGS. 10, 11, and 12. FIG. First, as a premise of display, as shown in FIG. 10, the imaging unit 28 is arranged at a position where each user U participating in the conference can be imaged. For example, the imaging unit 28 is arranged in the front upper part of the conference terminal 20 implemented as an electronic information board. Thereby, each user U who is having a meeting at a position directly facing the conference terminal 20 can be imaged.

Next, as shown in FIG. 11, by performing image analysis on images (or moving images) in which each user U is a subject created by imaging by the imaging unit 28, each user U , and identify which of the users U is speaking. This identification can be realized by face detection or motion detection by image analysis based on a generally known algorithm. These image analysis functions are implemented, for example, in the audio combining unit 230 of the conference terminal 20 . Then, the voice summation unit 230 associates the identified facial image of the user U who is speaking with the voice picked up at the same time, and transmits them to the voice recognition server 30 .

Then, the recognition result correcting unit 320 of the speech recognition server 30 displays the character string, which is the result of the speech recognition processing, and the associated facial image of the user U who has spoken, on the character string display unit 240 of the conference terminal 20. send to. Then, the character string display unit 240 of the conference terminal 20 associates and displays the character string and the face image of the user U who has spoken in the display area 271 . For example, it is displayed as shown in FIG. This improves the understanding of the utterance content and the sense of presence of the user U who refers to the display. That is, the convenience of the speech recognition system S is improved.

In addition, by registering in advance data such as images and feature amounts of the users U participating in the current conference, the participating users U can be identified with higher accuracy.

[Second modification]
As in the first modified example described above, when a user U participating in a conference is identified, the accuracy of speech recognition is improved by switching to a speech recognition model specialized for the identified user U. can be made In this case, even if the individual user U cannot be specified, the accuracy of speech recognition can be improved by using the respective speech recognition models if attributes such as male or female are known.

This modification will be described with reference to FIGS. 13 and 14. FIG. In this modification, the speech recognition unit 310 of the speech recognition server 30 is made to include a plurality of functional blocks. Specifically, as shown in FIG. 13, a face authentication result receiving unit 311, a first speech recognition model 312, a second speech recognition model 313, a third speech recognition model 314, and a speech recognition processing unit 315 are included.

Then, in step S18 in FIG. 8, each process shown in FIG. 14 is performed. Specifically, in step S<b>181 , the face authentication result receiving section 311 receives the result of face authentication by the voice combining section 230 of the conference terminal 20 . Then, based on the received face recognition result, the face recognition result receiving unit 311 switches the voice recognition model by switching a logical switch. For example, switch as follows.

In step S182, it is determined whether or not the face authentication result is "male". If the person is "male", a determination of Yes is made in step S182, and the process proceeds to step S183. In step S183, the face recognition result receiving unit 311 switches to the first voice recognition model 312, which is a voice recognition model for men, and then the voice recognition processing unit 315 performs voice recognition processing. On the other hand, if the person is not "male", a determination of No is made in step S182, and the process proceeds to step S184.

In step S184, it is determined whether or not the face authentication result is "female". If it is a "female", a determination of Yes is made in step S184, and the process proceeds to step S185. In step S185, the face recognition result receiving unit 311 switches to the second voice recognition model 313, which is a voice recognition model for women, and then the voice recognition processing unit 315 performs voice recognition processing. On the other hand, if it is not "female", it is determined No in step S184, and the process proceeds to step S186. In step S185, the face recognition result receiving unit 311 switches to the third voice recognition model 314, which is a general-purpose voice recognition model, and then the voice recognition processing unit 315 performs voice recognition processing.

In this way, the accuracy of speech recognition can be improved by using an appropriate speech model based on the result of face recognition.

Note that this modification may be applied not only to step S18 but also to steps S36 and S40 in FIG. 9 to switch voice models as described above.

[Third Modification]
The device configuration and division of functional blocks in the above-described embodiments are merely examples, and are not limited to these. For example, the functions of the voice recognition server 30 may be implemented in the conference terminal 20 and realized as a single device. Alternatively, the conference terminal 20, which is an edge device, may be realized by a mere communication relay device, and the functions of the conference terminal 20 may be implemented in the voice recognition server 30. FIG. In this case, for example, the result of speech recognition processing may be displayed by a device other than the conference terminal 20. FIG.

Alternatively, each of the conference terminal 20 and the speech recognition server 30 may be realized by a plurality of devices. For example, the voice recognition server 30 may be realized by cooperation of a plurality of cloud servers.

In other words, any of the devices included in the speech recognition system S may realize the functional blocks provided in the respective devices described above or alternative functional blocks. In other words, the functional configuration of FIG. 6 is merely an example and is not particularly limited. That is, it is sufficient if each device included in the speech recognition system S has a function capable of executing the above-described series of processes as a whole. is not limited to the example of

Note that, as an example, when the functional blocks are realized by the functional configuration in the above-described embodiment, the speech recognition system S corresponds to the "speech recognition system" in the present invention. Further, in this case, the microphone 10 corresponds to the "sound pickup device" of the present invention. Furthermore, in this case, the voice acquisition unit 210 corresponds to the "acquisition means" in the present invention. Furthermore, in this case, the volume determination section 220 corresponds to the "determination means" in the present invention. Furthermore, in this case, the speech combining unit 230, the speech recognition unit 310, and the recognition result correction unit 320 correspond to the "speech recognition processing means" and the "identification means" in the present invention.

[Other Modifications]
The series of processes described above can be executed by hardware or by software. Also, one functional block may be composed of hardware alone, software alone, or a combination thereof. For example, the functional configuration in this embodiment is implemented by a processor that executes arithmetic processing.

Processors that can be used in this embodiment include processors configured by various single processing units such as single processors, multiprocessors, and multicore processors. In addition, it also includes a combination of these various processing devices and a processing circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array).

When executing a series of processes by software, a program constituting the software is installed in a computer or the like from a network or a recording medium. The computer may be a computer built into dedicated hardware. Also, the computer may be a computer capable of executing various functions by installing various programs, such as a general-purpose personal computer.

A recording medium containing such a program is not only composed of a removable medium that is distributed separately from the main body of the device in order to provide the program to the user, but is also provided to the user in a state pre-installed in the main body of the device. It consists of a recording medium, etc. Removable media are composed of, for example, magnetic disks (including floppy disks), optical disks, or magneto-optical disks.

Optical discs are composed of, for example, CD-ROMs (Compact Disk-Read Only Memory), DVDs (Digital Versatile Disks), Blu-ray (registered trademark) Discs (Blu-ray Discs), and the like. The magneto-optical disk is composed of an MD (Mini-Disk) or the like. Further, the recording medium provided to the user in a state of being pre-installed in the apparatus main body is composed of, for example, the ROM 12, ROM 22, and ROM 32, or the HDD 24, and HDD 34 in FIG. 5, in which the program is recorded. be.

It should be noted that in this specification, the steps of writing a program recorded on a recording medium are not only processes that are performed chronologically in that order, but also processes that are not necessarily chronologically processed, parallel or individually. It also includes the processing to be executed. Further, in this specification, the term "system" means an overall device composed of a plurality of devices, a plurality of means, or the like.

Although several embodiments of the present invention have been described above, these embodiments are merely examples and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the gist of the present invention. These embodiments and their modifications are included in the scope and gist of the invention described in this specification and the like, and are included in the scope of the invention described in the claims and equivalents thereof.

S printing system 10 microphone 20 conference terminal 30 voice recognition server 11, 21, 31, 41 CPU
12, 22, 32, 42 ROMs
13, 23, 33, 43 RAM
14, 25, 35 Communication I/F
16 sound pickup unit 24, 34 HDD
26 operation unit 27 display unit 28 imaging unit 110 sound pickup unit 120 sound transmission unit 210 sound acquisition unit 220 volume determination unit 230 sound summation unit 240 character string display unit 310 sound recognition unit 311 face authentication result reception unit 312 first sound recognition Model 313 Second speech recognition model 314 Third speech recognition model 315 Speech recognition processing unit 320 Recognition result correction unit

JP 2017-167318 A

The present invention relates to a speech recognition system, a speech recognition method, and a speech processing device .

However, the technique disclosed in Patent Document 1 does not take into consideration the case where the user speaks in a direction different from the direction of the microphone and the user's voice goes out of the directional range of the microphone. If the user's voice is picked up while being buried in other users' voices and noise, the accuracy of voice recognition will be degraded.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a speech recognition system, a speech recognition method, and a speech processing apparatus capable of suppressing deterioration in accuracy of speech recognition. .

In order to solve the above-described problems and achieve the object, the present invention provides acquisition means for acquiring voice data of voices uttered by a plurality of users in the same space ; Speech recognition processing means for recognizing speech, and the speech recognition processing means provide a speech recognition system that corrects the speech recognition result based on a comparison of the speech recognition results of each of the plurality of speech data .

Advantageous Effects of Invention According to the present invention, it is possible to provide a speech recognition system, a speech recognition method, and a speech processing device capable of suppressing deterioration in accuracy of speech recognition.

Claims (9)

Acquisition means for acquiring audio data of voices uttered by a plurality of users in the same space from a plurality of sound collecting devices worn by each of the plurality of users;
determination means for determining whether or not the volume of the plurality of acquired audio data is less than a threshold;
voice recognition processing means for performing a predetermined process using the plurality of voice data together with a voice recognition process when the determination means determines that the volume of each of the plurality of voice data is less than a threshold value;
A speech recognition system with The speech recognition processing means is
Summing the plurality of audio data as a predetermined process using the plurality of audio data together,
performing speech recognition processing on the combined speech data;
A speech recognition system according to claim 1. The speech recognition processing means is
performing speech recognition processing on each of the plurality of speech data;
correcting a speech recognition result based on a comparison of the speech recognition results of each of the plurality of speech data as a predetermined process using the plurality of speech data together;
A speech recognition system according to claim 1. The speech recognition processing means corrects the speech recognition result based on the most common speech recognition result when the speech recognition results of the plurality of speech data are different in the comparison.
4. A speech recognition system according to claim 3. When the determining means determines that at least one of the volume levels of the plurality of audio data is equal to or greater than a threshold, the voice recognition processing means performs voice recognition processing on the audio data determined to be equal to or greater than the threshold. while performing speech recognition processing for other speech data,
5. The speech recognition system according to claim 1. The speech recognition processing means, when it is determined that the volume of each of the plurality of audio data is less than a threshold value, estimates the user who made the utterance based on the comparison result of the volume of each of the plurality of audio data.
A speech recognition system according to any one of claims 1 to 5. Further comprising the plurality of sound collecting devices,
The plurality of sound collecting devices are neck-mounted or badge-type sound collecting devices,
A speech recognition system according to any one of claims 1 to 6. Further comprising identification means for identifying the user based on the image in which the plurality of users are subjects,
The speech recognition processing means varies a method of speech recognition processing for each of the users based on the identification result of the user.
A speech recognition system according to any one of claims 1 to 7. A speech recognition method performed by a speech recognition system,
an acquisition step of acquiring audio data of voices uttered by a plurality of users in the same space from a plurality of sound collecting devices worn by each of the plurality of users;
a determination step of determining whether or not the volume of the acquired plurality of audio data is less than a threshold;
a voice recognition processing step of performing voice recognition processing along with predetermined processing using the plurality of voice data together when it is determined by the determination step that the volume of each of the plurality of voice data is less than a threshold value;
voice recognition method.

Images