KR101761312B1 - Directonal sound source filtering apparatus using microphone array and controlling method thereof - Google Patents

Directonal sound source filtering apparatus using microphone array and controlling method thereof Download PDF

Info

Publication number
KR101761312B1
KR101761312B1 KR1020100133002A KR20100133002A KR101761312B1 KR 101761312 B1 KR101761312 B1 KR 101761312B1 KR 1020100133002 A KR1020100133002 A KR 1020100133002A KR 20100133002 A KR20100133002 A KR 20100133002A KR 101761312 B1 KR101761312 B1 KR 101761312B1
Authority
KR
South Korea
Prior art keywords
sound source
area
source signal
delay value
image
Prior art date
Application number
KR1020100133002A
Other languages
Korean (ko)
Other versions
KR20120071452A (en
Inventor
현경학
Original Assignee
삼성전자주식회사
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 삼성전자주식회사 filed Critical 삼성전자주식회사
Priority to KR1020100133002A priority Critical patent/KR101761312B1/en
Priority to US13/373,300 priority patent/US9226070B2/en
Publication of KR20120071452A publication Critical patent/KR20120071452A/en
Application granted granted Critical
Publication of KR101761312B1 publication Critical patent/KR101761312B1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Health & Medical Sciences (AREA)
  • Otolaryngology (AREA)
  • Signal Processing (AREA)
  • General Health & Medical Sciences (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Obtaining Desirable Characteristics In Audible-Bandwidth Transducers (AREA)
  • Multimedia (AREA)

Abstract

A directional sound source filtering apparatus using a microphone array and a control method thereof are disclosed. The apparatus for filtering a directional sound source using a microphone array according to the present invention includes an image detecting unit for detecting an image of an object region, a collecting unit including a microphone array in which a plurality of microphones are arranged to detect a sound source together with an image detected by the image detecting unit, And a controller for previously calculating a time delay value of the sound source in the image to extract a sound source in the image from the sound source detected in the sound source and performing beam forming through the calculated time delay value.
Therefore, according to the present invention, it is possible to selectively amplify only a voice signal in an image using a plurality of beamforming, thereby improving a signal-to-interference ratio (SIR) have.

Figure R1020100133002

Description

[0001] The present invention relates to a directional sound source filtering apparatus using a microphone array and a control method thereof,

The present invention relates to a directional sound source filtering apparatus using a microphone array that selectively amplifies a sound source by beamforming a sound source signal detected by a microphone array, and a control method thereof.

The time has come to become commonplace when using a hand-held device to make phone calls, record external sound sources, or acquire moving images.

In various digital devices such as CE (Consumer Electronics) devices, mobile phones and digital camcorders, and in-vehicle sound recognition devices, microphones are used as means for acquiring sound sources.

In such an environment in which a sound source is recorded or a sound source signal is inputted through such a digital device, there are many environments in which various noise and peripheral interference sounds are included rather than a quiet environment without surrounding interference.

In particular, when a sound source and an image are simultaneously collected through a digital device, only the sound source corresponding to the image area should be amplified and transmitted. However, since a sound source signal has a strong diffraction characteristic, even a sound source outside the image region may be combined with a sound source in the image region to cause interference or exist as a noise source. Therefore, it is necessary to develop a method and an apparatus for collecting only the sound in the image area while effectively removing the sound outside the camera image area.

To this end, a method and apparatus have been devised for recognizing the face of the speaker in the camera image information to grasp the position information of the speaker, and amplifying only the sound source information obtained from the speaker's position information. However, this method requires image processing such as face recognition, and the performance of face recognition affects the performance of selective sound source amplification.

An aspect of the present invention is to provide a directional sound source filtering apparatus and a control method thereof using a microphone array for selectively amplifying only an infinite sound source signal detected in an object area according to angle of view information of a camera.

According to an aspect of the present invention, there is provided an apparatus for filtering a directional sound source using a microphone array, including: an image detector for detecting an image of an object area; a plurality of microphones for detecting a sound source, A controller for calculating a time delay value of the sound source in the image in advance to extract a sound source in the image from the sound source detected by the pick and place part composed of the arranged microphone array and performing the beam forming through the calculated time delay value .

The control unit sets the range of the in-field area, which is an object area, and the out-field area, which is an area outside the ink area, according to an angle of view of a predetermined image detector, And a time delay value of the sound source detected in the outfield area, and a sound source in the inphosphate area and the outfield area in the sound source detected using the calculated time delay value, And a beamforming unit for performing frequency conversion after compensating the time delay value.

The delay value calculating unit may set an in-field region, which is an area within the image, and an out-field region, which is an area outside the image, which are located within the angle of view of the image detecting unit.

The delay value calculator may calculate a first delay value, which is a time delay value of the input area, and a second delay value, which is a time delay value of the outfield area.

The beamforming unit extracts a sound source having a first delay value in the detected sound source, compensates the first delay value to the extracted sound source, and performs frequency conversion to perform a beamforming operation on the sound source in the in- And a second beamforming unit for extracting a sound source having a second delay value from the detected sound source, compensating for the second delay value to the extracted sound source, and performing frequency conversion to beamform the sound source in the outfield area have.

The delay value calculation unit may set at least one outfield area according to the arrangement of a plurality of microphones set in advance and calculate at least one second delay value corresponding to the set outfield area.

In addition, the second beamforming unit may be provided in at least one or more than one corresponding to the set outfield area.

The control unit may further include an operation unit for removing the beamformed sound source from the second beamforming unit and detecting only the sound source of the beamformed in-focus area in the first beamforming unit.

In addition, the arithmetic unit may perform addition calculation of the beamformed sound source in the first beamforming unit, and subtracting the beamformed sound source in the second beamforming unit to detect only the sound source in the input area.

In addition, the control unit may further include a filter unit that removes omnidirectional noise from the sound source of the infield area detected by the arithmetic unit.

In addition, the filter unit is composed of an LMS (Least Mean Square) filter to remove the omnidirectional noise signal.

The apparatus may further include an output unit outputting the detected image of the target area and the sound source detected in the image of the target area.

According to another aspect of the present invention, there is provided a method for controlling a directional sound source filtering apparatus using a microphone array, the method comprising: detecting an image of an object region; A method for controlling a directional sound source filtering apparatus using a microphone array including a microphone array having a plurality of microphones arranged therein, the method comprising the steps of: And performs beamforming using the calculated time delay value.

In order to calculate the time delay value, the range of the In-Field area which is the target area and the Out-field area which is the area outside the ink area is set according to the angle of view of the preset image detecting part And the time delay values of the sound sources detected in the set in the infield area and the outfield area can be calculated in advance.

In addition, the setting of the infield area and the outfield area can be performed by setting an in-field area, which is an area within an image, and an out-field area, which is an area outside the image, located within the angle of view of the image detecting part.

Also, the calculation of the time delay value can calculate the first delay value, which is the time delay value of the infield area, and the second delay value, which is the time delay value of the outfield area.

The beamforming is performed by extracting a sound source having a first delay value in the detected sound source, compensating the first delay value to the extracted sound source, performing frequency conversion to perform beamforming of the sound source in the in-focus region, The sound source having the second delay value is extracted from the sound source, the second delay value is compensated for the extracted sound source, and frequency conversion is performed to beam-form the sound source in the out-field area.

Also, the outfield area may be set to at least one or more than one according to the arrangement of a plurality of microphones set in advance, and the second delay value may be calculated to correspond to the set outfield area.

The method may further include detecting a sound source in the beamformed in-focus region by removing the sound source in the beam-formed out-field region.

If a sound source in the outfield area exists, the sound source in the infield area is additionally operated, and the sound source in the outfield area is subtracted It is possible to detect only the sound source in the infield area.

The method may further include removing omnidirectional noise from the sound source in the detected infield region.

The method may further include outputting the detected image of the target area and the sound source detected in the image of the target area together.

According to the directional sound source filtering apparatus and the control method thereof using the microphone array according to the present invention, there is an effect of spatial filtering for selectively amplifying only a sound source signal in an image region by a camera.

In addition, the signal-to-interference ratio (SIR) of the sound source signal in the image region can be improved by the ambient noise.

1 is an external view of a directional sound source filtering apparatus using a microphone array according to an embodiment of the present invention.
2 is a control block diagram of a directional sound source filtering apparatus using the microphone array shown in FIG.
3 is a circuit diagram of a directional sound source filtering apparatus using the microphone array shown in FIG.
4 is a flowchart of a directional sound source filtering method using a microphone array according to another embodiment of the present invention.

Hereinafter, preferred embodiments of a directional sound source filtering apparatus and method using the microphone array of the present invention will be described with reference to the accompanying drawings.

1 is an external view of a directional sound source filtering apparatus using a microphone array according to an embodiment of the present invention.

Referring to FIG. 1, the directional sound source filtering apparatus 100 using a microphone array according to the present embodiment includes an image detecting unit 110 for capturing an image on the front surface, And a collector 120 for collecting a sound source.

The sound collector 120 includes a microphone array in which a plurality of microphones MA1 to MA4 are linearly arranged.

In the meantime, although the microphone array is constituted by four microphones MA1 to MA4 in the present embodiment, it is also included in the scope of the present invention to construct a microphone array with fewer or more microphones. In this embodiment, a microphone array in which a plurality of microphones are arranged in a line is constituted. However, this is only one embodiment, and a microphone array composed of a plurality of microphones may be included in the scope of the present invention even if they are not linearly arranged.

A directional sound source filtering apparatus 100 using a microphone array collects sound sources and images at the same time, and amplifies and outputs only a target sound source, which is a sound source in the image, from the collected sound sources. Specifically, the directional sound source filtering apparatus 100 using a microphone array filters a target sound source in an in-field region, which is a target area, which is imaged by the image detection unit 100, using a beam forming technique.

The directional sound source filtering apparatus 100 using the microphone array can be used in a video call and video conference system to more clearly convey a speaker's voice.

Hereinafter, a directional sound source filtering apparatus using a microphone array for filtering only the target sound source in the infiltration area will be described in detail using a control block and a circuit diagram.

FIG. 2 is a control block diagram of a directional sound source filtering apparatus using the microphone array shown in FIG. 1, and FIG. 3 is a circuit diagram of a directional sound source filtering apparatus using the microphone array shown in FIG.

The directional sound source filtering apparatus 100 using the microphone array according to the present embodiment is fixedly installed in an area where a picture and a sound source are to be simultaneously collected, such as a specific space such as a terminal or a conference room. The directional sound source filtering apparatus 100 includes an image detecting unit 110, A control unit 130, an output unit 140, and a memory unit 150.

The image detecting unit 110 is composed of a camera and collects images of a specific space. The image detecting unit 110 may detect only an image of an in-field region, which is a predetermined region, according to the angle of view information of the camera. That is, the infield area is an area within the image collected by the image detecting unit 110. [

The sound collector 120 comprises a microphone array, which detects the sound waves of the sound source and generates electrical signals corresponding to the sound waves. The generated electrical signal can be defined as a sound source signal.

The microphone array may be composed of a plurality of microphones, and a plurality of microphones may be provided around the image detecting unit 110 at regular intervals or at equal intervals. On the other hand, the interval and position information between the microphones adjacent to each other are stored in advance in the memory unit 150 and used for beamforming of the sound source.

The sound collector 120 detects a sound source in an Out-Field region that is an area outside the image as well as an ink field by the microphone array.

The control unit 130 performs control for outputting only the sound source in the infield area using the beam forming technique.

The control unit 130 includes a delay value calculating unit 131, a sound source amplifying unit 132, a determining unit 133, a beam forming unit 134, a calculating unit 135, and a filter unit 136.

The delay value calculator 131 sets an in-field area, which is an object area, and an out-field area, which is a filtering area outside the object area, using the field angle information of the image detector 110 stored in the memory unit 150 in advance. The infield area is an area that can be imaged by the camera, and is determined in advance by the angle of view information of the camera. That is, the infield area is located in front of the camera and is located within the angle of view area of the camera.

On the other hand, the outfield area may be set to at least one or more according to the arrangement of the plurality of microphones. For example, when a plurality of microphones are arranged in a straight line around the camera, the right outfield area and the left outfield area are set. In addition, when a plurality of microphones are arranged in a straight line around the camera, the left and right outfield areas as well as the upper and lower outfields can be further set.

The delay value calculator 131 calculates the time delay value using the time information required for the sound source detected in the set input and output fields to reach the sound collector 120. [

Specifically, the delay value calculating section 131 calculates a first delay value t1 to be compensated for the excitation signal detected from the input region. Also, the delay value calculating unit 131 calculates at least one second delay value t2, t3, ..., tn to be compensated for the excitation signal detected from at least one of the outfield areas.

The calculated time delay values t1, t2 ... tn are stored in advance in the memory unit 150 and the beamforming unit 134 uses the previously stored time delay values t1, t2 ... tn Beamforming the sound source.

The sound source amplifying unit 132 is connected to a plurality of microphones of the sound collecting unit 120, and is provided by the number of the plurality of microphones. The sound source amplifying unit 132 amplifies the sound source signals transmitted from the plurality of microphones.

When the detected sound source is amplified through the sound source amplification unit 132, the determination unit 133 determines whether a specific signal among the sound sources exists. If the determination unit 133 determines that a specific signal exists, the determination unit 133 transmits a specific signal to the beamforming unit 134 to perform beamforming.

On the other hand, the above-mentioned specific signal may be a voice signal. Accordingly, the determination unit 133 determines whether or not a sound signal having a frequency range of 20 to 20000 Hz and a sound pressure level of 0 to 130 dB is present in the detected sound source signal.

When it is determined that a specific signal exists, the beamforming unit 134 uses the calculated first delay value t1 and second delay values t2, t3, ..., tn to calculate a sound source signal detected in a specific direction, Foaming.

The beamforming unit 134 includes a delay-and-sum beamformer for beamforming a sound source signal detected in a specific direction.

The delay and island beamformer uses the time difference of the signal arriving at each microphone to find the direction of the sound and to strengthen only the sound source signal located in a specific direction, or vice versa, to eliminate unnecessary interference noise.

Using this beamforming technology, it is possible to improve the performance of sound separation or speaker localization, which eliminates or separates other noise sources around the speaker, and it is possible to improve the performance by post- It may also reduce non-existent noise or reverberation.

That is, by acquiring sound source signals existing at a long distance by using a microphone array, it is possible to emphasize or suppress sound source signals inputted from a specific direction, and to remove sounds other than specific directions.

Such a beamformer can serve as a spatial filter for filtering only signals in a specific area in space.

The beamforming unit 134 according to the present embodiment uses the time delay values t1, t2, ..., tn corresponding to the inffield area and the outfield area calculated by the delay value calculating unit 131 to calculate Only the existing sound source signal is selectively output and the sound source signal existing in the other direction is removed.

The beamforming unit 134 includes a first beamforming unit 134a for beamforming a sound source signal in an inffield area and a second beamforming unit 134b for beamforming a sound source signal in an outfield area.

The first beamforming unit 134a outputs only the sound source signal in the image detected by the image detecting unit 110, and removes the sound source signal in the other direction.

The second beamforming unit 134b is provided corresponding to at least one outfield area set by the delay value calculating unit 131, and outputs only the excitation signal in the corresponding outfield area.

Hereinafter, the process of outputting a sound source signal by the beam forming unit 134 will be described in detail.

The beamforming unit 134 includes a buffer unit for storing the respective sound source signals X1 and X2 .... Xn transmitted from the sound source amplifying unit 132 and a buffer unit for receiving the sound source signals X1, A frequency converting unit for performing frequency conversion on the sound source signal extracted by the extracting unit and decomposing the signal according to the frequency, And an inverse frequency conversion unit for performing inverse frequency conversion on the sound source signal converted into the frequency domain to convert the sound source signal into a time domain.

The first beamforming unit 134a extracts only a tone generator signal having a time delay corresponding to the first delay value t1 in the tone generator signal, compensates the extracted tone generator signal for the first delay value t1, And reverse frequency conversion.

The second beamforming unit 134b extracts the excitation signal having the time delay corresponding to the second delay values t2, t3, ..., tn in the excitation signal, and outputs the corresponding second delay values t2, t3 ... tn) and performs frequency conversion and inverse frequency conversion.

Thus, each of the beamforming units 134a and 134b selectively outputs a sound source signal detected from a predetermined direction by using time delay information required for the sound source signal to arrive, and removes the sound source signal in the other direction.

The excitation source signal beamformed in the beamforming section 134 is transmitted to the arithmetic operation section 135. The operation unit 135 extracts only a sound source signal corresponding to a specific frequency using Spectral Subtraction or the like.

More specifically, the operation unit 135 of the present embodiment performs addition operation on the excitation signal in the first beamforming unit 134a and subtraction operation on the excitation signal in the second beamforming unit 134b, Only the computation signal in the forming section 134a is output by the output section 140. [

That is, as a result of the signal processing in the arithmetic unit 135, the excitation signal in the in-focus region is enhanced and the excitation signal in the out-field region can be removed.

The sound source signal in the inffield area output from the operation unit 135 is transmitted to the filter unit 136 again.

The filter unit 136 is composed of a Least Mean Square (LMS) filter such as Wiener Filter, and removes the omnidirectional noise signal from the sound source signal in the inffield area.

Here, the omnidirectional noise signal is a signal having the same intensity of signals detected in all directions, and corresponds to a low frequency sound such as an air conditioner sound.

This omnidirectional noise signal is removed by the filter unit 136 because there is no specific directionality and beamforming is impossible.

The output unit 140 outputs the sound source signal y in the inffield area from which the omni-directional noise signal is removed together with the image signal detected by the image detection unit 110. [ The output unit 140 may include a display for outputting a video signal and a speaker for outputting a sound source signal.

In particular, the speaker only detects the sound source signal y in the infield area by the control unit 130, converts the sound source signal having the reverse frequency converted to the vibration of the diaphragm, and outputs sound waves to the air.

In addition, the speaker outputs a sound wave in such a manner that when the sound signal is outputted, the sound signal which is reverse-frequency-converted is converted into the vibration of the diaphragm to generate a small-sized wave in the air.

Accordingly, only the speech signal in the noise canceled image is output together with the image, so that the performance of the directional sound source filtering apparatus 100 using the microphone array according to the present embodiment, which is the ratio of the sound source signal in the image region to the ambient noise, have.

4 is a flowchart of a directional sound source filtering method using a microphone array according to another embodiment of the present invention.

First, the delay value calculator 131 sets an in-field area and an out-field area using the angle of view information of the image detector 110 stored in advance (step 210) .

The infield area is a target area in which the amplified sound source of the directional sound source filtering method using the microphone array according to the present embodiment is detected. That is, the infield area is a region located in front of the image detecting unit 110, and is an area within the image that is located within the view angle area and is detected by the image detecting unit 110.

The outfield area is an area outside the image detected by the image detector 110, and may be set to at least one or more depending on the arrangement of the plurality of microphones.

When the fill area and the out field area are set (210), the delay value calculating unit 131 calculates the time delay value of the sound source signal arriving from the fill area and the out field area (220).

That is, the delay value calculating unit 131 calculates the time delay value using the directionality of the sound source signal.

Specifically, the delay value calculating section 131 calculates a first delay value t1 that is a time delay corresponding to the fill area.

Further, the delay value calculating unit 131 calculates the second delay values t2, t3, ..., tn, which are time delays corresponding to each of the at least one outfield area.

The first delay value t1 and the second delay values t2, t3, ..., tn, which are the time delay values of the respective regions thus calculated, are transmitted to the beamforming section 134. [

The control unit 130 of the directional sound source filtering apparatus 100 using the microphone array determines whether a specific one of the detected sound sources is present (230) when a sound source is detected through the sound collector 120. The control unit 130 controls the operation of the beamforming unit 134 when it is determined that a specific signal exists ('Yes' at 230).

Here, the specific signal may be a voice signal. In the case of a voice signal, the control unit 130 determines whether a sound signal having a frequency range of 20 to 20000 Hz and a sound pressure level of 0 to 130 dB is present in the detected sound source signal.

If it is determined that a specific signal is present (Yes at 230), the beamforming unit 134 uses the calculated first delay value t1 and the second delay values t2, t3 ... tn to determine Beamforming is performed on the excitation signal detected in the direction.

The first beamforming unit 134a outputs only the sound source signal in the in-fill region using the first delay value t1, and removes the sound source signal in the remaining direction (240).

Specifically, the first beamforming unit 134a extracts only the tone generator signal having the first delay value t1 from the tone generator signal detected and amplified by the sound collector 120 composed of a plurality of microphone arrays, And performs frequency conversion and inverse frequency conversion on the first delay value t1 and transmits the result to the operation unit 135. [

When the excitation signal in the infield area is beamformed by the first beamformer 134a, the second beamformer 134b beamforms the excitation signal in the outfield area (step 250).

More specifically, the second beamforming unit 134b generates a sound source signal having the second delay values t2, t3, ..., tn in the sound source signal detected by the sound collector 120 and amplified by the sound source amplifying unit 132 . The extracted sound source signal is compensated for the corresponding second delay values t2, t3, ..., tn, and frequency conversion and inverse frequency conversion are performed and transmitted to the operation unit 135. [

When the excitation signal in the outfield area is present, the arithmetic unit 135 determines whether the excitation signal in the outfield area exists (step 260) 'Yes') Spectral Subtraction or the like is used to remove the excitation signal in the outfield region (270).

More specifically, the operation unit 135 adds the excitation signal transmitted from the first beamforming unit 134a and subtracts the excitation signal transmitted from the second beamforming unit 134b to generate a excitation signal .

The sound source signal in the enhanced in-focus region is transmitted to the filter unit 136, and the filter unit 136 removes the omni-directional noise signal from the sound source signal in the in-focus region 280 (280).

Specifically, the filter unit 136 removes a noise signal that is low-frequency sound such as a sound of an air conditioner and can not be beamformed because the intensity of signals detected in all directions is the same.

The noise-removed sound source signal is finally stored together with the image signal in the infield area detected by the image detection unit 110, and is transmitted to the output unit 140.

The output unit 140 includes a display and a speaker, and outputs a video signal and a sound source signal in the infield area (290).

As a result, the audio signal in the area not output to the video is interrupted, and only the audio signal in the video can be output together with the video.

In addition, unlike the conventional method of recognizing the face of a speaker and locating a speaker and outputting only a voice signal at a recognized position, the present invention selectively amplifies only a voice signal in an image using a relatively simple beamforming can do.

This can improve the signal-to-interference ratio (SIR) of the source signal in the video region compared with the ambient noise.

100: directional sound source filtering device using microphone array
110:
120:
130:
140:
150:

Claims (22)

An image detector for detecting an image of a target area;
A sound collector configured by a microphone array in which a plurality of microphones are arranged to detect a sound source signal together with an image detected by the image detector; And
A controller for previously calculating a time delay value of a sound source signal in the image to extract a sound source signal in the image from the sound source signal detected by the sound collector and performing beam forming through the calculated time delay value, ,
Wherein the control unit sets an in-field area that is the target area and an out-field area that is an area outside the input area according to an angle of view of the image detector, And a delay value calculating unit for previously calculating a time delay value of the sound source signal detected in the outfield area.
The method according to claim 1,
Wherein the control unit extracts a sound source signal in the infield area and the outfield area from the detected sound source signal using the calculated time delay value, compensates the time delay value for the extracted sound source signal, And a beamforming unit that performs a beamforming operation on the microphone array.
3. The method of claim 2,
Wherein the delay value calculating unit sets the outfield area which is located in the image angle of the image detector and which is an area of the image and an area outside the image.
3. The method of claim 2,
Wherein the delay value calculator calculates a first delay value that is a time delay value of the infield area and a second delay value that is a time delay value of the outfield area.
5. The method of claim 4,
The beamforming unit includes:
Extracting a sound source signal having the first delay value from the detected sound source signal, compensating the first delay value for the extracted sound source signal, performing the frequency conversion, and beamforming the sound source signal in the infield area A first beamforming unit; And
Extracts a sound source signal having the second delay value from the detected sound source signal, compensates the second delay value for the extracted sound source signal, and performs frequency conversion to perform a sound source signal in the out- And a second beamforming unit for performing a second beamforming operation on the second beamforming unit.
6. The method of claim 5,
The delay value calculator may include at least one outfield area according to the arrangement of the plurality of microphones and a directional sound source filtering device using a microphone array for calculating at least one second delay value corresponding to the set outfield area. .
The method according to claim 6,
Wherein the second beamforming unit includes at least one microphone corresponding to the set outfield area.
6. The method of claim 5,
Wherein the controller further comprises an arithmetic unit for removing the beamformed sound source signal from the second beamforming unit and detecting only the sound source signal of the beamformed in the infield area in the first beamforming unit, .
9. The method of claim 8,
Wherein the arithmetic unit performs an addition operation on a beamformed sound source signal in the first beamforming unit and a subtraction operation on a beamformed sound source signal in the second beamforming unit to detect only a sound source signal in the infield area A directional sound source filtering device using a microphone array.
9. The method of claim 8,
Wherein the control unit further includes a filter unit for removing the omnidirectional noise signal from the sound source signal of the infield area detected by the arithmetic unit.
11. The method of claim 10,
Wherein the filter unit comprises a Least Mean Square (LMS) filter to remove the omnidirectional noise signal.
The method according to claim 1,
And an output unit outputting a sound source signal detected in the image of the object area and the image of the object area.
There is provided a directional sound source filtering apparatus using a microphone array including an image detecting unit for detecting an image of a target area and a collecting unit including a microphone array in which a plurality of microphones are arranged to detect a sound source signal together with the image detected by the image detecting unit. In the control method,
Calculating in advance a time delay value of the sound source signal in the image to extract the sound source signal in the image from the sound source signal detected by the sound collector;
Performing beamforming using the calculated time delay value,
Calculating the time delay value in advance may comprise:
Setting an in-field area that is the target area and an out-field area that is an area outside the infield area according to an angle of view of the image detector;
Wherein the time delay value of the sound source signal detected in the infield area and the outfield area is calculated in advance.
delete 14. The method of claim 13,
Wherein the setting of the infield area and the outfield area is performed by setting directional sound source filtering using a microphone array that is located within an angle of view of the image detector and that is an area in the image and the outfield area that is an area outside the image A method of controlling a device.
14. The method of claim 13,
Wherein the calculating of the time delay value in advance calculates a first delay value which is a time delay value of the infield area and a second delay value which is a time delay value of the outfield area, wherein the control of the directional sound source filtering device using the microphone array Way.
17. The method of claim 16,
Performing the beamforming comprises:
Extracting a sound source signal having the first delay value from the detected sound source signal, compensating the first delay value to the extracted sound source signal, performing frequency conversion to perform beamforming of the sound source signal in the infield region,
Extracts a sound source signal having the second delay value from the detected sound source signal, compensates the second delay value for the extracted sound source signal, and performs frequency conversion to perform a sound source signal in the out- And controlling the directional sound source filtering apparatus using the microphone array.
18. The method of claim 17,
Wherein at least one of the outfield areas is set according to the arrangement of the plurality of microphones and the second delay value is calculated to correspond to the set outfield area.
18. The method of claim 17,
Further comprising detecting a source signal in the beamformed in-field region by removing the source signal in the beam-formed outfield region, and detecting only a source signal in the beamformed infield region.
20. The method of claim 19,
Detecting only the excitation signal in the infield zone is:
Determining whether a sound source signal in the outfield area exists;
If it is determined that there is a sound source signal in the outfield area, an addition operation is performed on the sound source signal in the infield area and a subtraction operation is performed on the sound source signal in the outfield area to detect only the sound source signal in the infield area A method for controlling a directional sound source filtering apparatus using a microphone array.
20. The method of claim 19,
And removing the omnidirectional noise signal from the detected sound source signal in the infield area using the microphone array.
14. The method of claim 13,
And outputting the sound source signal detected in the image of the object area and the image of the object area together.
KR1020100133002A 2010-12-23 2010-12-23 Directonal sound source filtering apparatus using microphone array and controlling method thereof KR101761312B1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
KR1020100133002A KR101761312B1 (en) 2010-12-23 2010-12-23 Directonal sound source filtering apparatus using microphone array and controlling method thereof
US13/373,300 US9226070B2 (en) 2010-12-23 2011-11-10 Directional sound source filtering apparatus using microphone array and control method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020100133002A KR101761312B1 (en) 2010-12-23 2010-12-23 Directonal sound source filtering apparatus using microphone array and controlling method thereof

Publications (2)

Publication Number Publication Date
KR20120071452A KR20120071452A (en) 2012-07-03
KR101761312B1 true KR101761312B1 (en) 2017-07-25

Family

ID=46316846

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020100133002A KR101761312B1 (en) 2010-12-23 2010-12-23 Directonal sound source filtering apparatus using microphone array and controlling method thereof

Country Status (2)

Country Link
US (1) US9226070B2 (en)
KR (1) KR101761312B1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022025376A1 (en) * 2020-07-31 2022-02-03 삼성전자 주식회사 Method and device for providing ar audio data
WO2023102089A1 (en) * 2021-12-01 2023-06-08 Rensselaer Polytechnic Institute Multi-sensor systems and methods for providing immersive virtual environments

Families Citing this family (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
BR112012031656A2 (en) * 2010-08-25 2016-11-08 Asahi Chemical Ind device, and method of separating sound sources, and program
WO2013093187A2 (en) * 2011-12-21 2013-06-27 Nokia Corporation An audio lens
US9531948B2 (en) 2013-01-09 2016-12-27 Lg Electronics Inc. Voice tracking apparatus and control method therefor
TW201443875A (en) * 2013-05-14 2014-11-16 Hon Hai Prec Ind Co Ltd Method and system for recording voice
KR102150013B1 (en) 2013-06-11 2020-08-31 삼성전자주식회사 Beamforming method and apparatus for sound signal
US9565493B2 (en) 2015-04-30 2017-02-07 Shure Acquisition Holdings, Inc. Array microphone system and method of assembling the same
US9554207B2 (en) 2015-04-30 2017-01-24 Shure Acquisition Holdings, Inc. Offset cartridge microphones
US9530426B1 (en) * 2015-06-24 2016-12-27 Microsoft Technology Licensing, Llc Filtering sounds for conferencing applications
US9747920B2 (en) * 2015-12-17 2017-08-29 Amazon Technologies, Inc. Adaptive beamforming to create reference channels
CN107026934B (en) 2016-10-27 2019-09-27 华为技术有限公司 A kind of sound localization method and device
KR102534768B1 (en) 2017-01-03 2023-05-19 삼성전자주식회사 Audio Output Device and Controlling Method thereof
US10367948B2 (en) 2017-01-13 2019-07-30 Shure Acquisition Holdings, Inc. Post-mixing acoustic echo cancellation systems and methods
CN110447238B (en) 2017-01-27 2021-12-03 舒尔获得控股公司 Array microphone module and system
WO2018207453A1 (en) * 2017-05-08 2018-11-15 ソニー株式会社 Information processing device
CN109036448B (en) 2017-06-12 2020-04-14 华为技术有限公司 Sound processing method and device
US10178490B1 (en) * 2017-06-30 2019-01-08 Apple Inc. Intelligent audio rendering for video recording
US10110994B1 (en) * 2017-11-21 2018-10-23 Nokia Technologies Oy Method and apparatus for providing voice communication with spatial audio
KR20190086214A (en) 2018-01-12 2019-07-22 서울과학기술대학교 산학협력단 System and method for maximizing realistic watch using directional microphone
WO2019231632A1 (en) 2018-06-01 2019-12-05 Shure Acquisition Holdings, Inc. Pattern-forming microphone array
US11297423B2 (en) 2018-06-15 2022-04-05 Shure Acquisition Holdings, Inc. Endfire linear array microphone
EP3854108A1 (en) 2018-09-20 2021-07-28 Shure Acquisition Holdings, Inc. Adjustable lobe shape for array microphones
US11109133B2 (en) 2018-09-21 2021-08-31 Shure Acquisition Holdings, Inc. Array microphone module and system
CN113841419A (en) 2019-03-21 2021-12-24 舒尔获得控股公司 Housing and associated design features for ceiling array microphone
JP2022526761A (en) 2019-03-21 2022-05-26 シュアー アクイジッション ホールディングス インコーポレイテッド Beam forming with blocking function Automatic focusing, intra-regional focusing, and automatic placement of microphone lobes
US11558693B2 (en) 2019-03-21 2023-01-17 Shure Acquisition Holdings, Inc. Auto focus, auto focus within regions, and auto placement of beamformed microphone lobes with inhibition and voice activity detection functionality
USD967076S1 (en) * 2019-03-28 2022-10-18 Sony Group Corporation Microphone
CN114051738A (en) 2019-05-23 2022-02-15 舒尔获得控股公司 Steerable speaker array, system and method thereof
EP3977449A1 (en) 2019-05-31 2022-04-06 Shure Acquisition Holdings, Inc. Low latency automixer integrated with voice and noise activity detection
JP2022545113A (en) 2019-08-23 2022-10-25 シュアー アクイジッション ホールディングス インコーポレイテッド One-dimensional array microphone with improved directivity
US11552611B2 (en) 2020-02-07 2023-01-10 Shure Acquisition Holdings, Inc. System and method for automatic adjustment of reference gain
WO2021243368A2 (en) 2020-05-29 2021-12-02 Shure Acquisition Holdings, Inc. Transducer steering and configuration systems and methods using a local positioning system
JP7459779B2 (en) * 2020-12-17 2024-04-02 トヨタ自動車株式会社 Sound source candidate extraction system and sound source exploration method
WO2022165007A1 (en) 2021-01-28 2022-08-04 Shure Acquisition Holdings, Inc. Hybrid audio beamforming system
CN114913868B (en) * 2022-05-17 2023-05-05 电子科技大学 Acoustic array directional pickup method based on FPGA

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060104458A1 (en) * 2004-10-15 2006-05-18 Kenoyer Michael L Video and audio conferencing system with spatial audio

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8139029B2 (en) * 2006-03-08 2012-03-20 Navisense Method and device for three-dimensional sensing
US8319858B2 (en) 2008-10-31 2012-11-27 Fortemedia, Inc. Electronic apparatus and method for receiving sounds with auxiliary information from camera system
US8174932B2 (en) * 2009-06-11 2012-05-08 Hewlett-Packard Development Company, L.P. Multimodal object localization
US8558894B2 (en) * 2010-11-16 2013-10-15 Hewlett-Packard Development Company, L.P. Support for audience interaction in presentations

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060104458A1 (en) * 2004-10-15 2006-05-18 Kenoyer Michael L Video and audio conferencing system with spatial audio

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022025376A1 (en) * 2020-07-31 2022-02-03 삼성전자 주식회사 Method and device for providing ar audio data
WO2023102089A1 (en) * 2021-12-01 2023-06-08 Rensselaer Polytechnic Institute Multi-sensor systems and methods for providing immersive virtual environments

Also Published As

Publication number Publication date
KR20120071452A (en) 2012-07-03
US9226070B2 (en) 2015-12-29
US20120163624A1 (en) 2012-06-28

Similar Documents

Publication Publication Date Title
KR101761312B1 (en) Directonal sound source filtering apparatus using microphone array and controlling method thereof
US10097921B2 (en) Methods circuits devices systems and associated computer executable code for acquiring acoustic signals
CN107534725B (en) Voice signal processing method and device
KR101715779B1 (en) Apparatus for sound source signal processing and method thereof
KR101456866B1 (en) Method and apparatus for extracting the target sound signal from the mixed sound
JP5857674B2 (en) Image processing apparatus and image processing system
US20100165071A1 (en) Video conference device
KR101449433B1 (en) Noise cancelling method and apparatus from the sound signal through the microphone
JP5855571B2 (en) Audio zoom
JP4929685B2 (en) Remote conference equipment
US20100123785A1 (en) Graphic Control for Directional Audio Input
KR101812862B1 (en) Audio apparatus
JP5259622B2 (en) Sound collection device, sound collection method, sound collection program, and integrated circuit
JP4670682B2 (en) Audio apparatus and directional sound generation method
JP5772151B2 (en) Sound source separation apparatus, program and method
JP2005250397A (en) Robot
CN110875056B (en) Speech transcription device, system, method and electronic device
KR20170063618A (en) Electronic device and its reverberation removing method
JP2015023508A (en) Sound gathering device and program
JP3332143B2 (en) Sound pickup method and device
JP5086768B2 (en) Telephone device
JP3341815B2 (en) Receiving state detection method and apparatus
Lin et al. Development of novel hearing aids by using image recognition technology
KR101159239B1 (en) Apparatus for sound filtering
CN109920442B (en) Method and system for speech enhancement of microphone array

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
E701 Decision to grant or registration of patent right
GRNT Written decision to grant