TWI687917B - Voice system and voice detection method - Google Patents

Abstract

A voice detection method comprises: receiving an initial voice; a current voice channel comprised in a speaker is opened for receiving the initial voice, and other voice channels comprised in the speaker is closed; capturing a frame; detecting a mouth type with opening and closing state in the frame; recognizing a mouth position corresponding to the mouth type; and outputting a main voice according to the mouth type and the current voice channel.

Description

Voice system and sound detection method

This case relates to a voice system and sound detection method, and particularly to a voice system and sound detection method using a camera device.

Today's smart voice assistant devices rely on speakers to translate keywords into system understanding commands before they can perform radio microphones, voice processing, voice recognition engines, and various application services on the cloud. Among them, the design of the radio microphone is the first level of whether the smart speaker can accurately recognize the user's command. For example, in a multi-person meeting, the radio microphone is often prone to receive ambient noise or other than the presenter. Human speech, for example, a voice assistant device placed next to a TV, may be accidentally triggered by the sound of advertisements or news broadcast on the TV, and execute application services not directed by the user.

Therefore, how to avoid the interference of environmental noise on the recognition and how to avoid the human voiceprint characteristics in the case of mixed speakers Extraction, and how to prevent the voice assistant from being accidentally triggered by other non-user-instructed instructions, has become one of the problems to be solved.

This case provides a voice system, including: a speaker, a camera device and a processor. The speaker contains a plurality of speaker channels. The speaker opens a current audio channel in the speaker channel according to a volume of an initial audio, a sound source position, or a frequency, and closes other speaker channels in the speaker channel. The camera device is used to take a picture before the speaker. A processor is used to detect a mouth shape presenting an open and closed state on the screen, identify a mouth shape position corresponding to the mouth shape, and output a main audio according to the mouth shape position and the current audio channel.

According to one aspect of the present case, a sound detection method includes: receiving an initial audio, wherein a speaker opens a current audio channel according to a volume, an audio source position, or a frequency of the initial audio, and closes other speaker channels; A picture before the speaker; a mouth shape showing an open and closed state in the detection screen; identifying a mouth shape position corresponding to the mouth shape; and outputting a main audio according to the mouth shape position and the current audio channel.

To sum up, in this case, through the voice system and the sound detection method, by detecting the opening and closing of the mouth and the position of the mouth in the picture and judging the sound source, it is possible to filter out the noise and make the sound detection more accurate Effect, and can avoid the interference of environmental noise on the main audio, and can also analyze the voice of the presenter in the case of multiple speakers in conferences or lectures. In addition, the present invention can also be applied to voice assistants In the system, it is necessary to recognize the opening and closing of the mouth shape to confirm the operator of the voice assistant, thereby preventing the voice assistant system from being accidentally triggered by other non-user-instructed commands (such as TV advertisement sound).

100: voice system

10: Camera device

20: processor

30: Speaker

SC1~SC5: Speaker channels

USR1~USR3: user

200: sound detection method

210~250, 410~470: steps

In order to make the above and other objects, features, advantages and embodiments of the present disclosure more obvious and understandable, the attached drawings are explained as follows: FIG. 1 is a speech system according to an embodiment of the present case; FIG. 2 Is a flow chart of a sound detection method according to an embodiment of the present case; FIGS. 3A~3B are schematic diagrams of usage scenarios of a speech system according to an embodiment of the present case; and FIG. 4 is an implementation according to the present case An example flowchart of a sound detection method is shown.

Please refer to FIGS. 1-2. FIG. 1 is a speech system 100 according to an embodiment of the present invention. Figure 2 is based on the actual case A flowchart of a method 200 for sound detection shown in the embodiment. In one embodiment, the voice system 100 includes a speaker 30, a camera 10, and a processor 20. The speaker 30 includes a plurality of speaker channels SC1~SC5. The speaker channels SC1~SC5 are not limited to five. This is only an example, as long as there are a plurality of them.

In an embodiment, the voice system 100 may be a conference phone device, a smart voice assistant device, a power bank, a desktop computer, a mobile phone, a tablet, or other device with a display function.

In one embodiment, the speaker channels SC1 to SC5 are structural designs for receiving sound, which can be opened and closed because of the size, direction, or frequency of the sound to identify the user's position and filter noise.

In one embodiment, the camera device 10 is composed of at least one Charge Coupled Device (CCD) or a Complementary Metal-Oxide Semiconductor (CMOS) sensor.

In step 210, a current audio channel SC5 among the speaker channels SC1~SC5 is opened to receive an initial audio.

Please refer to FIGS. 3A~3B. FIGS. 3A~3B are schematic diagrams illustrating a usage situation of a speech system 100 according to an embodiment of the present case. For convenience of description, FIGS. 3A~3B only show the speaker channels SC1~SC5 and the camera device in the voice system 100 of FIG. 1 10. However, those with ordinary knowledge in the art should understand that the speaker channels SC1~SC5 in Figures 3A~3B are located in the voice system 100, and the usage scenarios of Figures 3A~3B can be applied to the voice system 100 in Figure 1. To achieve.

For example, in Figure 3A, users USR1~USR3 are in a meeting, and user USR3 begins to speak. At this time, since speaker channel SC5 in speaker channels SC1~SC5 is closest to user USR3, the received volume and /Or vibration is also larger than other speaker channels SC1~SC4, so the speaker 30 turns on the speaker channel SC5 to receive the initial audio (speaking sound) from the user USR3, while the other speaker channels SC1~SC4 remain closed; For convenience of description, in the present invention, the speaker channel SC5 that is turned on to receive the initial audio is defined as the current audio channel SC5.

In one embodiment, the speaker 30 determines the current audio channel SC5 according to a volume, a source position, or a frequency of the initial audio.

In step 220, the camera device 10 is used to take a picture. For example, in FIG. 3A, the camera device 10 is used to capture a screen of a conference site, and users USR1~USR3 can be captured in this screen. In actual implementation, step 210 may be reversed with step 220.

In step 230, the processor 20 is used to detect a mouth shape in an open and closed state. For example, in Figure 3A, at The processor 20 determines the mouth shape of the user USR3 in the open and closed state on the screen according to one or more frames captured by the camera device 10, which represents that the user USR3 is speaking. The detection of the face shape of the face can be implemented using known algorithms related to face detection, so it will not be repeated here.

In step 240, the processor 20 is used to identify a mouth shape position corresponding to the mouth shape. For example, when the processor 20 detects the mouth shape of the user USR3 in the opening and closing state on the screen, the processor can describe the position of the mouth shape according to the coordinates of the mouth of the user USR3 located on the screen.

In step 250, the processor 20 outputs a main audio according to the mouth position and the current audio channel SC5. For example, after the processor 20 receives the initial audio from the current audio channel SC5, it further determines the multiple users USR1~3 in the screen according to the position of the mouth that is opening and closing, because the user USR3 is speaking. The processor 20 uses the voice of the initial audio (user USR3) received by the current audio channel SC5 as the main audio.

In one embodiment, the processor 20 adjusts a weight used to weight the initial audio according to the volume of the initial audio received by the current audio channel SC5, thereby adjusting the initial audio to be louder and used as the main audio, and Output this main audio to the storage device. In one embodiment, the main audio can be transmitted Send to another voice system, and broadcast by another voice system.

In one embodiment, the voice system 100 further includes a storage device coupled to the camera device 10, the processor 20, and/or the speaker 30. The storage device can be implemented as a flash memory, a floppy disk, a hard disk, an optical disk, a flash disk, a magnetic tape, or a storage medium with the same function that can be easily thought of by those skilled in the art. The storage device is used to store the main audio as a recording file of the meeting record.

In one embodiment, the processor 20 is further used to identify the main audio content, for example, using a known speech recognition algorithm (using any existing technology that can be used to recognize speech, so it will not be repeated here), to The content of the main audio is displayed on at least one text on a display. For example, the processor 20 determines that the content of the main audio is "Hello everyone", and then displays the text of "Hello everyone" on the display (for example, a screen). This allows all users USR1~USR3 in the meeting to hear and/or watch what the presenter user USR3 wants to express at the same time, so that the main audio can be recorded as a recording file of the meeting record, and the text can be used as the meeting Document file for record.

Please refer to FIGS. 3B and 4. FIG. 4 is a flowchart of a sound detection method 400 according to an embodiment of the present invention. Steps 410 to 440 and 470 in FIG. 4 are similar to steps 210 to 250 in FIG. 2 respectively, so details are not described here. Steps 450 to 460 in FIG. 4 are described in detail below.

In step 450, the processor 20 is used to determine whether the initial audio contains noise. For example, the processor 20 can distinguish the initial audio frequency, waveform, known audio analysis method (such as signal-to-noise ratio (SNR) or other known noise determination methods Whether the initial audio contains noise, if the initial audio contains noise, then go to step 460, if the initial audio does not contain noise, then go to step 470.

In step 460, the processor 20 is used to filter another audio sent from a position other than the mouth position. For example, as shown in Figure 3B, in the conference room, except that the presenter (user USR3) is speaking (the mouth position in the screen), user USR1 and user USR2 are Voice discussion (at another location in the screen), at this time, the initial audio received by the current audio channel SC5 will contain the louder voice of user USR3 and the smaller between user USR1 and user USR2 The sound of the discussion, the current audio channel SC5 can send the initial audio to the processor 20, the processor 20 can distinguish the use of the initial audio frequency, waveform, known audio analysis methods (such as signal to noise ratio) The USR3's speech is the main audio, and the other discussion sounds are noise, and then the noise is removed and only the main audio is retained.

In one embodiment, since the presenter (user USR3) usually speaks louder, his mouth shape changes greatly, and the sounds of users USR1 and USR2 who are discussing in a low voice The sound is small, and its mouth shape changes little. Therefore, the processor 20 can be supplemented by the size of the mouth opening and closing at various positions in the frame captured by the camera 10 to further serve as a reference basis for the processor 20 to determine which sound is the main audio, so that the processor 20 can Analysis is more correct.

In one embodiment, the processor 20 may increase the volume weight of the main audio (for example, the presenter) and reduce the volume weight of the noise (for example, the low-talker or ambient noise) to highlight the main Audio.

As can be seen from the foregoing, the processor 20 records what each speaker (such as user USR3) tells in the storage device. Since unnecessary noise in the initial audio is filtered out, only the audio of the speaker is stored, thereby The load of the processor 20 can be reduced during the subsequent audio processing. In addition, in some embodiments, the position where the mouth opening and closing situation in the corresponding picture is more obvious may be increased by adjusting the volume weight of the position to calculate an adjustment result, and the adjustment result is regarded as the main audio. In some embodiments, due to the limitation of the volume or distance of speech, as shown in FIG. 3B, the current audio channel SC5 receives the USR3 (presenter) closer to the user and the speaker’s voice is usually louder, so the current The audio channel SC5 receives a louder voice from the user USR3, so the processor 20 can determine that the user USR3 is speaking, and filter out the voices of the user USR1 and the user USR2 (person who is talking in private).

To sum up, in this case, through the voice system and the sound detection method, by detecting the opening and closing of the mouth and the position of the mouth in the picture and judging the sound source, the effect of filtering noise and making sound detection more accurate can be achieved. And it can avoid the interference of the environmental noise on the main audio, and it can also analyze the voice of the presenter in the case of conferences or lectures in the case of multiple speakers. In addition, the present invention can also be applied to a voice assistant system, because the opening and closing of the mouth shape is required to recognize the operator of the voice assistant, thereby preventing the voice assistant system from being accidentally instructed by other non-users (such as TV commercials Sound) triggers.

Although this case has been disclosed above with examples, it is not intended to limit this case. Anyone who is familiar with this skill can make various changes and retouching without departing from the spirit and scope of this case, so the scope of protection of this case should be considered The scope of the attached patent application shall prevail.

200: sound detection method

210~250: steps

Claims (6)

A voice system includes: a speaker, including: a plurality of speaker channels, the speaker opens a current audio channel closest to the sound source position among the speaker channels according to a source position of an initial audio, and closes the speaker channels Other speaker channels in the; a camera device to shoot a picture before the speaker; and a processor to detect a mouth shape showing an open and closed state in the picture and identify a mouth corresponding to the mouth shape Type position, and according to the mouth position and the current audio channel, to output a main audio. The voice system according to claim 1, further comprising: a storage device for storing the main audio; wherein the processor is further used for recognizing the content of the main audio to display the content of the main audio in at least one text on On a monitor. The speech system according to claim 1, wherein the processor is further configured to filter another audio signal sent from a location other than the mouth position. A sound detection method, including: Receive an initial audio, one of the speakers according to the initial audio source position to open a current audio channel closest to the location of the audio source, and close the other speaker channels; take a picture before the speaker; detect the presence of the picture A mouth shape in an open and closed state; identifying a mouth shape position corresponding to the mouth shape; and outputting a main audio according to the mouth shape position and the current audio channel. The sound detection method according to claim 4, further comprising: storing the main audio; identifying the content of the main audio; and displaying the content of the main audio on at least one text on a display. The sound detection method according to claim 4, further comprising: filtering another audio signal sent from a position other than the mouth position.

Images