KR20200036820A - Apparatus and Method for Sound Source Separation based on Rada - Google Patents

Abstract

The present invention relates to an apparatus for a voice recognition service based on radar. The apparatus capable of exactly recognizing a user command including a wake-up word comprises: a radar; a sound source location tracking unit analyzing a transmission/reception signal of the radar to search a location of a speaker; a sound acquisition unit receiving ambient sound; a sound source separation unit separating voice of the speaker from the sound inputted through the sound acquisition unit based on information on the location of the speaker tracked by the sound source location tracking unit; and a postprocessing unit recognizing the voice of the speaker separated by the sound source separation unit.

Description

Apparatus and Method for Sound Source Separation based on Rada}

The present invention relates to a speech recognition technology, and more particularly to a speech recognition service device using radar technology.

Speech recognition refers to having a machine understand human everyday speech and perform tasks according to the understood speech. With the development of computer and information communication, the technology of speech recognition can easily obtain information from a long distance without a human being directly moving, leading to the development of devices composed of systems that operate according to voice.

With recent advances in speech recognition technology, various speech recognition services such as Apple's Siri, Google's Now, Microsoft's Cortana, and Amazon's Alexa have been released. In order to accurately recognize the user command according to the provision of the speech recognition service, the location of a target sound source such as a speaker's voice is tracked in real time, and each sound source is separated from the signal received by the microphone in an environment where noise is present to locate their sound source. It is required to grasp.

However, in the existing voice recognition service, since a sound signal is analyzed and then a sound signal is searched for source localization, a time delay due to sound source search is generated. For example, when the wake-up word 'Alexa' is called, the voice recognition service starts. At this time, the voice recognition service device separates the speaker and performs voice recognition only after the 'Alexa' sound source search is performed. However, since the 'al' is not recognized due to the time delay due to the search for the sound source, and only the 'lexa' is recognized, the voice recognition service device cannot recognize whether a wake-up is performed. Therefore, it may be cumbersome because the user has to re-name 'Alexa'. In addition, since sound source tracking is continuously performed as the user moves, voice recognition is performed, so the user command may not be accurately recognized.

The present invention provides a radar-based speech recognition service device and method capable of accurately recognizing a user command including a wake-up word.

In addition, a radar-based speech recognition service apparatus and method capable of preventing a speech recognition delay due to sound source search are provided.

In addition, a radar-based speech recognition service apparatus and method capable of performing accurate sound source recognition even when a user moves.

The present invention is a radar-based speech recognition service device, a radar, a sound source location tracking unit that analyzes a radar's transmission / reception signal to search for a speaker's location, a sound acquisition unit that receives ambient sound, and a sound source location tracking unit It includes a sound source separation unit for separating the speaker's voice from the sound input through the sound acquisition unit based on the speaker location information tracked by, and a post-processing unit for recognizing the speaker's voice separated by the sound source separation unit.

The present invention is a radar-based speech recognition service method, the step of tracking the speaker's location by analyzing the radar's transmit / receive signal, and separating the speaker's voice from the input sound based on the tracked speaker location information, And recognizing the voice of the separated speaker.

According to the present invention, since a delay according to a sound source search is not generated by quickly searching a sound source location based on a radar in advance, a user command including a wake-up word can be accurately recognized. In addition, since the location search of the sound source is quickly performed even when the user moves, accurate speech recognition can be performed.

The technology that tracks the location of sound sources is used not only in household appliances, but also in devices such as service robots that support housekeeping, surveillance cameras that track sound sources to monitor intruders, and video cameras used in multi-party video conferencing.

1 is a schematic block diagram of a radar-based speech recognition service device according to an embodiment of the present invention.
2 is an example of a top view of a radar-based speech recognition service device according to the present invention.
3 is a flowchart illustrating a radar-based speech recognition service method according to an embodiment of the present invention.

Hereinafter, a radar-based speech recognition service apparatus and method according to a preferred embodiment will be described in detail with reference to the accompanying drawings. Here, the same reference numerals are used for the same components, and repeated descriptions and detailed descriptions of well-known functions and components that may unnecessarily obscure the subject matter of the invention are omitted. Embodiments of the invention are provided to more fully describe the invention to those skilled in the art. Accordingly, the shape and size of elements in the drawings may be exaggerated for a more clear description.

Combinations of each block in the accompanying block diagrams and steps of the flow charts may be performed by computer program instructions (execution engines), these computer program instructions being incorporated into a processor of a general purpose computer, special purpose computer, or other programmable data processing equipment. Since it can be mounted, the instructions executed through a processor of a computer or other programmable data processing equipment create a means to perform the functions described in each block of the block diagram or in each step of the flowchart.

These computer program instructions can also be stored in computer readable or computer readable memory that can be oriented to a computer or other programmable data processing equipment to implement functionality in a particular way, so that computer readable or computer readable memory The instructions stored in it are also possible to produce an article of manufacture containing instructions means for performing the functions described in each block of the block diagram or in each step of the flowchart.

And since computer program instructions may be mounted on a computer or other programmable data processing equipment, a series of operational steps are performed on the computer or other programmable data processing equipment to create a process that is executed by the computer to generate a computer or other programmable It is also possible for instructions to perform data processing equipment to provide steps for performing the functions described in each block of the block diagram and in each step of the flowchart.

In addition, each block or each step can represent a module, segment, or portion of code that includes one or more executable instructions for executing specified logical functions, and in some alternative embodiments referred to in blocks or steps It should be noted that it is possible for functions to occur out of sequence. For example, two blocks or steps shown in succession may in fact be performed substantially simultaneously, and it is also possible that the blocks or steps are performed in the reverse order of the corresponding function as necessary.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the embodiments of the present invention exemplified below may be modified in various other forms, and the scope of the present invention is not limited to the embodiments described below. Embodiments of the present invention are provided to more fully describe the present invention to those skilled in the art.

1 is a schematic block diagram of a radar-based speech recognition service device according to an embodiment of the present invention, and FIG. 2 is an example of a top view of a radar-based speech recognition service device according to the present invention.

Referring to FIG. 1, a radar-based speech recognition service device (hereinafter referred to as 'device') 100 includes a sound acquisition unit 110, a sound source separation unit 120, a post-processing unit 130, and a radar 141, It includes a sound source location tracking unit 142 and the control unit 150.

The radar-based speech recognition service device 100 is a terminal installed in a user's living space, such as a living room at home, receives sound including a speaker's service request voice through the microphone 100, and separates a sound source The speaker 120 identifies the speaker by analyzing the corresponding sound, and the post-processing unit 130 performs a service corresponding to the meaning of the identified speaker's voice. Here, the service corresponding to the meaning of the voice may be, for example, to control on / off of the lighting in the living room when the voice “turn on the living room light” is input. However, this is only an example, and the present invention is not limited thereto. The radar-based speech recognition service device 100 is a device such as a household appliance to which a sound source location tracking technology is applied, as well as a service robot that supports lyrics, a surveillance camera that monitors an intruder by tracking a sound source, and a video camera used in a multi-party video conference. It might be.

According to an embodiment, the sound acquiring unit 110 may include a microphone array 111 in which a plurality of microphones 111a, 111b, .. are disposed. Referring to FIG. 2, the microphone array 111 may be formed in a shape in which a plurality of microphones 111a, 111b, .. are provided in a radial form surrounding the periphery of the cylindrical housing of the device 100. However, this is only an example, and the present invention is not limited thereto. The radar-based speech recognition apparatus 100 may have various forms, and the microphone array 110 may have a separate configuration for wired / wireless communication with the radar-based speech recognition apparatus 100. In addition, according to another embodiment, a plurality of microphones (111a, 111b, ..) is attached to the housing of the device 100 by a rotatable support (not shown), the support is controlled by the control of the control unit 150 It may be configured to be redirected.

A variety of sound sources (Sound Source: SS 1, .., SS i, .., SS N) may exist around the microphone array 111. Here, the sound sources may include one or more users, and electronic devices that generate various noises, including nearby TV devices. Each of the plurality of microphones 111a, 111b, .. constituting the microphone array 111 may receive and output sound generated from various sound sources centering on a sound source present in a oriented position. In addition, the microphone array 111 is connected to the amplification unit 112, so that the sound input by the microphone array 111 can be amplified. At this time, according to an embodiment of the present invention, the amplification unit 112 is a plurality of amplifiers for amplifying the sound output from each of the plurality of microphones (111a, 111b, ..) constituting the microphone array 111 By configuring, the gain of each of the plurality of amplifiers can be independently controlled.

Referring back to FIG. 1, the radar 141 transmits a radar signal to the surroundings as the device 100 is powered on, and the transmitted radar signal is reflected back to an object and fed back. Reflected signal is received. According to an embodiment of the present invention, a radar 141 is also formed with a plurality of radars (not shown) in a radial form surrounding the housing of the device 100, such as a microphone array 111. It can take the form. Accordingly, the radar 141 may transmit a plurality of radar signals radially around it to receive the reflected signal. In addition, according to another embodiment, the plurality of radars may be configured to be attached to the housing of the device 100 by a rotatable support (not shown), so that the support can be switched by control of the control unit 150.

The sound source location tracking unit 142 analyzes the transmission / reception signal of the radar 141 to search for the speaker's location. That is, the sound source location tracking unit 142 calculates the distance from the reflector through the transmit / receive signal, and tracks the sound source location according to the calculated distance from the reflector. At this time, the sound source location tracking unit 142 may be stored in advance, the distance information of the static reflector in the surroundings in the absence of a person. Here, the static reflector may be a reflector that does not move, including, for example, a TV device in a living room, a wall, and a sofa.

Therefore, when the distance information of the dynamic reflector different from the distance information of the surrounding static reflector stored in advance is calculated, the sound source location tracking unit 142 may determine the dynamic reflector as a speaker. Therefore, even if the TV device is driven to generate sound, the TV device corresponds to a static reflector stored in advance, so the sound source location tracking unit 142 may not determine the TV device as a speaker. Also, the sound source location tracking unit 142 may determine more than one speaker. In addition, the sound source location tracking unit 142 may also detect the shape contour information of the speaker. For example, height, posture (whether sitting or standing), and human and TV can be distinguished.

The sound source separation unit 120 separates the speaker's voice from the sound input from the sound acquisition unit 110. That is, the sound acquisition unit 110 is input in a form in which sounds generated by sound sources of various locations are mixed, and the sound source separation unit 120 separates the speaker's voice from the sound. To this end, it is important to locate the speaker. In the past, a lot of efforts have been made in research and development to locate the sound source by constructing a microphone array by arranging a number of microphones in series or in parallel and analyzing the signals input to the microphone array. lost. However, such a conventional method requires a process of searching for all directions to track the sound source position, converting it from the time domain to the frequency domain, and inversely transforming it, thereby increasing the computation amount when tracking the sound source.

Therefore, in the present invention, the signal input to the microphone array is not analyzed to determine the sound source location, but the radar 141 is transmitted / transmitted before the sound source location tracking unit 142 inputs sound from the sound acquisition unit 110. Analyze the received signal to track the location of the sound source.

Accordingly, the sound source separation unit 120 separates the speaker's voice from the sound input through the sound acquisition unit 110 based on the tracked sound source location information. At this time, various algorithms can be adopted for sound source separation. For example, the sound source separation unit 120 estimates the sound generated at the tracked sound source location so that the Active Noise Cancellation technology can be applied, and applies the ANC technology to the ambient sound from the estimated sound. Remove noise by removing it. At this time, among the microphones (111a, 111b, ..) constituting the microphone array 111, the speaker's voice is most often included in the sound input from the microphone toward the speaker, so it is important to consider the noise removal considerations. The noise signal coming from the front of the microphone is not lost as much as possible, and the noise signal coming from the direction other than the front is removed.

To this end, the control unit 150 is connected to an amplifier and other microphones connected to a microphone facing a speaker position among a plurality of microphones 111a, 111b, .. constituting the microphone array 111 of the sound acquisition unit 110 The gain of the connected amplifiers is controlled differently. That is, the gain of the amplifier connected to the microphone corresponding to the location of the sound source is increased, and the gain of other amplifiers is adjusted to be small. As a result, the sound source separation unit 120 may allow the noise signal coming from a direction other than the front to be removed without maximally losing the voice signal coming from the front of the corresponding microphone.

In addition, according to another embodiment, the control unit 150 may use an algorithm such as signal-to-noise ratio enhancement (SNR enhancement), microphone array orientation, or the like to remove the noise signal without maximally losing the speaker sound source. . For example, as described above, the directional positions of the plurality of microphones 111a, 111b, .. of the microphone array 111 may be configured to be adjustable, and the control unit 150 may be controlled by the sound source position tracking unit 142. It is also possible to adjust the directional position of the microphone so that the plurality of microphones 111a, 111b, .. are directed to the corresponding sound source position based on the tracked sound source position.

In addition, as described above, when the individual gain of the amplifier 120 connected to the microphone array 111 is controlled or the direction in which the microphone array 111 faces is adjusted, the sound source separation by the sound source separation unit 120 (Source) Separation) is omitted, and the speech recognition unit 131 may perform speech recognition through a simple & fast algorithm.

The post-processing unit 130 includes a voice recognition unit 131 and a corresponding processing unit 132. The voice recognition unit 131 may recognize the meaning of the voice as the voice of the speaker separated from the sound source separation unit 120 is input. Thereafter, the corresponding processing unit 132 allows the service operation corresponding to the recognized meaning of the voice to be performed. For example, a service corresponding to the meaning of voice controls an external electronic device, and the post-processing unit 130 may perform a function of transmitting a corresponding control command to the external electronic device through wired / wireless communication. Here, the external electronic device (not shown) is a device operated according to a control command, and various Internet of Things (IoT) such as a smart TV installed in the home, a lighting device interlocked with the service providing server 200, a heating device, and an air conditioner ).

At this time, although not shown in the drawing, the post-processing unit 130 may output a service provision message through a speaker (not shown). In addition, the radar-based voice recognition service device 100 transmits a service providing request message recognized in voice to an external service providing server (not shown) through wired / wireless communication to receive a customized service proposal message fed back from the service providing server. Upon reception, a corresponding service may be provided.

3 is a flowchart illustrating a radar-based speech recognition service method according to an embodiment of the present invention.

Referring to FIG. 3, as the device 100 is powered on, a radar signal is transmitted to the surroundings, and the transmitted radar signal is reflected by an object in the surroundings to receive a feedback reflected signal (S210). According to an embodiment of the present invention, the radar 141 may receive a reflected signal by transmitting a plurality of radar signals radially around.

The device 100 analyzes the radar transmission / reception signal to search for the speaker's location (S220). That is, the distance from the reflector is calculated through the transmission / reception signal, and the sound source location is tracked according to the calculated distance from the reflector. At this time, the device 100 may store distance information of a static reflector existing in the surroundings in the absence of a person. Here, the static reflector may be a reflector that does not move, including, for example, a TV device in a living room, a wall, and a sofa. Accordingly, when the distance information of the dynamic reflector different from the distance information of the surrounding static reflector stored in advance is calculated, the apparatus 100 may determine the corresponding dynamic reflector as a speaker. Therefore, even if the TV device is driven to generate sound, the TV device corresponds to a static reflector stored in advance, so the device 100 may not determine the TV device as a speaker. In addition, at this time, the device 100 may determine more than one speaker.

Next, the device 100 may control the components based on the searched speaker's location information (S230). According to one embodiment, the device 100 has a gain of each of a plurality of amplifiers for amplifying the sound output from each of the plurality of microphones 111a, 111b, .. constituting the microphone array 111. It can be controlled independently. That is, the gain of the amplifier connected to the microphone corresponding to the searched sound source position is increased, and the gain of other amplifiers is adjusted to be small. According to another embodiment, the directional position of the microphone may be adjusted so that the plurality of microphones 111a, 111b, .. face the corresponding sound source position based on the sound source position.

When the speech recognition preparation according to the speaker position is completed through the above-described S210 to S230, the device 100 detects whether a voice is input (S240). In this way, for example, the entire wake-up word called 'Alexa' called by the user can be recognized from the beginning.

If the voice input is not detected as a result of the detection of S240, the device 100 returns to step S210 and continuously searches for a sound source location. And, while performing S250 to S270, S210 to S230 may be continuously performed. This is to ensure that the sound source is separated quickly even if the speaker moves. That is, it is possible to detect and track the movement of a person around, so that the context of the conversation with the person continues.

On the other hand, when a voice input is detected as a result of the detection of S240, the device 100 separates the sound source based on the detected speaker position. That is, the speaker's voice is separated from the input sound. That is, sounds generated by sound sources of various locations are input in a mixed form, which separates the speaker's voice from these sounds. At this time, various algorithms can be adopted for sound source separation. For example, in order to enable active noise cancellation technology, noise generated in the tracked sound source is estimated, and ANC technology is applied to remove ambient sound from the estimated sound. Remove it. At this time, among the microphones (111a, 111b, ..) constituting the microphone array 111, the speaker's voice is most often included in the sound input from the microphone toward the speaker, so it is important to consider the noise removal considerations. The noise signal coming from the front of the microphone is not lost as much as possible, and the noise signal coming from the direction other than the front is removed. In addition, as described above, when the individual gain of the amplifier 120 connected to the microphone array 111 is controlled or the direction in which the microphone array 111 is directed is adjusted, the source separation step (S250) is omitted. In S260, speech recognition may be performed through a simple and fast algorithm.

The device 100 recognizes the meaning of the voice of the separated speaker (S260). Then, a service operation corresponding to the meaning of the cooled voice is performed (S270). For example, a service corresponding to the meaning of voice controls an external electronic device, and may perform a function of transmitting a corresponding control command to the external electronic device through wired / wireless communication.

Claims (5)

With radar,
A sound source location tracking unit that analyzes the radar's transmit / receive signal to search for the speaker's location;
A sound acquisition unit that receives ambient sound,
And a sound source separation unit for separating the speaker's voice from the sound input through the sound acquisition unit based on the speaker position information tracked by the sound source location tracking unit,
Radar-based speech recognition service device including a post-processing unit for recognizing the speaker's speech separated by the sound source separation unit.
According to claim 1, The sound acquisition unit
A microphone array consisting of a plurality of microphones arranged radiatedly,
It includes a plurality of amplification units for amplifying a plurality of sound signals output from each of the plurality of microphones to output to the sound source separation unit,
A radar-based speech recognition service device further comprising a control unit for differentially adjusting the gain of each of the plurality of amplification units based on the speaker location information searched by the sound source location tracking unit.
According to claim 1, The sound source location tracking unit
A radar that stores radar transmit / receive signal information in advance on at least one static reflector in the pre-stored environment, and determines the speaker's position if the analyzed radar transmit / receive signal does not match the radar transmit / receive signal of the stored static reflector. Based speech recognition service device.
Analyzing the radar transmit / receive signals to track the speaker's location,
Separating the speaker's voice from the input sound based on the tracked speaker location information,
A radar-based speech recognition service method comprising the step of recognizing the speech of the separated speaker.
According to claim 4,
The radar-based speech recognition service method further comprising the step of differentially adjusting the gain of each of the plurality of amplifying units for amplifying a plurality of sound signals output from each of the plurality of microphones based on the tracked speaker location information.

Images