KR20190016683A - Apparatus for automatic conference notetaking using mems microphone array - Google Patents

Abstract

The present invention relates to an apparatus for automatically taking minutes using a microphone array. The apparatus may identify a position of a speaker by detecting reception coordinates of a sound signal sensed by a microphone and recognize voice by speaker, convert the voice into text, output the text as an image, and generate log information to automatically take minutes. The apparatus has an effect of discriminating and confirming conversation content per speaker by sensing sound through a microphone and identifying the position of the speaker through a sensed sound signal and recognizing voice per speaker to convert the voice into the text. In addition, the apparatus has an effect of automatically taking minutes by sequentially recording the recognized text per speaker, generating the log information, and taking the minutes.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to an apparatus for automatically generating minutes using microphone arrays,

[0001] The present invention relates to a device for recognizing a speaker's voice by the detection of a sound signal and automatically creating a meeting record, more particularly, to detecting a position of a speaker by detecting reception coordinates of a sound signal sensed by a microphone, The present invention relates to an apparatus for automatic recording of minutes using a microphone array capable of automatically generating minute notes by outputting images and outputting log information.

When the distance between the speaker and the audience is close to each other in an environment such as a meeting, a lecture, or a discussion, conversation between them can proceed smoothly. However, as the distance between the speaker and the audience increases, the communication between them may not be smooth. Especially, it is difficult for all audiences to hear the lecturer's voice without amplifying the voice like a microphone in a large and large space like a large lecture room. In addition, in the method of creating the minutes, minutes are recorded separately from recorded voice files after recording the contents of meetings, or professional workers such as stenographers are used.

Korean Patent No. 10-0936244 (hereinafter referred to as " Prior Art Document ") uses a stereo camera to specify a position of a human mouth and directs a directional microphone to the specified position to acquire an accurate voice from which ambient noise has been removed To an intelligent voice input device for a robot. The directional microphone is horizontally and vertically rotated so that the sound source is specified through the image recognition and the specified sound source is directed. Thus, it is possible to obtain an accurate sound with minimized peripheral noise.

The preceding document recognizes the user through the video and controls the microphone in the direction of the user in order to receive the user's voice efficiently, and recognizes one user through the camera.

These prior art documents are difficult to use in situations such as meetings or lectures. Therefore, there is a need for a system capable of recognizing a large number of audiences and automatically recording the minutes of the audiences by recognizing the audiences' voices.

Korean Patent No. 10-0936244 entitled " Intelligent Speech Input Device for Robot and Its Operation Method, Date of Registration: 2010.01.04.

In order to solve the above problems, the present invention has an object of detecting a sound through a microphone, recognizing the position of the speaker through a sound signal, and recognizing the speaker's voice to convert the sound into text.

It is another object of the present invention to generate log information and record minutes by sequentially recording text recognized per speaker.

The apparatus for automatic recording of minutes using a microphone array according to the present invention includes a sound sensing unit for sensing sound through a microphone array, a sensing area for sensing the sound sensing unit, A sound region identifying unit for identifying a sound position based on a positional coordinate of the sound detected in the sensing region, a field manager for designating the sound position as a speaker, A text conversion unit for recognizing a sound of a speaker designated by the picture language unit as a speaker voice and converting the sound into a text, an image capturing unit for capturing a specific area, a mapping unit for mapping the sound position and the image information captured by the image capturing unit And the text is added to the image information mapped with the sound position in real time By recording the video output unit, the text is displayed in sequence comprises the minutes writing unit configured to generate information minutes over a log generation unit, and the log information to generate the log information.

The acoustic position identification unit according to the present invention includes a beam power calculation unit for calculating a beam power level corresponding to each of the grid position coordinates by summing the sound signals received for each of the grid position coordinates, And a sound position designator for designating the coordinates of a location where the intensity of the beamforming output value is strongest as the sound position.

In order to solve the above-mentioned problems, the present invention provides a method for detecting a sound through a microphone, recognizing the position of the speaker through a sound signal, and recognizing the speaker's voice and converting the sound into text, .

In addition, the present invention has the effect of automatically creating a meeting record by sequentially generating text recognized per speaker and generating log information and creating a meeting record.

FIG. 1 is a block diagram of an apparatus for automatically generating minutes of minutes using a microphone array according to the present invention.
FIG. 2 is a block diagram of an apparatus for automatically generating a meeting list using a microphone array according to an embodiment of the present invention. Referring to FIG.
3 is a diagram for explaining a method of identifying a speaker according to sound detection according to the present invention.
4 is a diagram for explaining a method of mapping location information and image information of a speaker according to the present invention.
FIG. 5 is a diagram for explaining automatic generation of log information and recording of minutes according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

FIG. 1 is a block diagram of an apparatus for automatically generating minutes of minutes using a microphone array according to the present invention. Referring to FIG. 1, the apparatus for automatically recording a meeting list using a microphone array may include a sound sensing unit 100, an image sensing unit 200, a sound processing unit 300, and an image output unit 400.

The sound sensing unit 100 is a device that senses sound through a microphone array. The sound sensing unit 100 is formed in the form of an array in which at least one microphone is arranged in a specific form. In the form of an array of microphones, a plurality of microphones may be arranged in the form of a horizontal, vertical, diagonal, or matrix. The microphone of the sound sensing unit 100 is preferably a compact device such as a MEMS (Micro Electro Mechanical Systems) microphone. The microphone arrays may be arranged at regular intervals or may be arranged with different spacing distances by the user.

The image capturing unit 200 is a device for capturing a specific area and generating image information. The image capturing unit 200 captures the same area as the area where sound can be sensed by the sound sensing unit 100.

Referring to FIG. 1, the sound processing unit 300 includes a sensing area setting unit 310, a sound position identifying unit 320, a characterizing unit 330, a text converting unit 340, a mapping unit 350, Unit 360, and a meeting record preparing unit 370. [

The sensing area setting unit 310 sets the sensing area of the sound sensing unit 100 and sets the grid position coordinates of the sensing area in order to recognize the position of the sound generated in the sensing area. The area of interest refers to a virtual area where the sound can be identified through the microphone.

The sound position identifying unit 320 is a device for identifying the sound position through the position coordinates of the sound sensed in the sensing area. The sound position identifying unit 320 may include a beam power calculating unit 321, a beam forming unit 322, and a sound position specifying unit 323. [

The beam power calculator 321 is a device for calculating a beam power level corresponding to each grid position coordinate by summing the sound signals received for each grid position coordinate.

For example, if a total of 15 position coordinates are set in the form of a grid of 3 x 5 grid coordinates set by the sensing area setting unit 310, the beam power calculator 321 calculates Adds the sound signal. The beam power level is calculated through the summed signals at the respective position coordinates.

Meanwhile, since the microphones are spaced apart from each other by a predetermined distance, the receiving angles of the sounds received by the respective microphones become different from each other. Accordingly, the beam power calculating unit 321 sets the time delay in accordance with the reception angle of the sound to each position coordinate so as to correct the time difference of sound reception. A time delay is applied to each of the position coordinates with reference to the position coordinate having the highest beam power level calculated by the beam power calculating section 321 to correct the phase angle of the sound signal equally.

The method of extracting the beam power level can be expressed by the following equation.

[Formula 1]

[Formula 1]

The beamforming generator 322 generates a beamforming output value for extracting a sound source by applying a weight to minimize the analysis of the output to the beam power level. This is to suppress interference signals other than the desired direction as the sound received by the microphone includes not only voice information of the speaker but also noise such as noise and noise. Thus, the variance of the beamforming output is minimized so that the sound can be focused only on the sound signal in the received direction.

The weighting that minimizes dispersion of the beamforming output can be expressed by the following equation.

[Formula 2]

[Formula 2]

The beamforming output values for the sound source extraction through the weights are as follows.

[Formula 3]

The sound position designation unit 323 is a device for designating the coordinates of the position where the intensity of the beam forming output value is the strongest as the sound position.

The tone-mapping unit 330 is a device for specifying a sound position as a speaker. That is, the position where the beamforming output value is strongest becomes the position where the voice is uttered from the speaker, and the image indexer 330 determines that the speaker is located at the designated sound position.

The text conversion unit 340 is a device for recognizing the sound of the speaker designated by the image-mapping unit 330 as a speaker's voice and converting the sound into a text. As a method of converting a speaker voice to text, an application that can convert voice to text, such as a speech recognition API provided by Google, is used. The text conversion unit 340 may be connected to the voice recognition API of Google via a wireless network.

 The mapping unit 350 is a device for mapping the sound position and the image information photographed by the image photographing unit 200.

The video output unit 400 is a device in which the converted text is displayed in real time on the video information to which the sound position is mapped.

Hereinafter, a method of identifying the position of the speaker according to the sound detection and outputting it as image information will be described with reference to FIGS. The configuration of the automatic recording device for recording minutes using the microphone array according to the present invention can be made more clear according to the following description.

FIG. 2 is a block diagram of an apparatus for automatically generating a meeting list using a microphone array according to an embodiment of the present invention. Referring to FIG. Referring to FIG. 2, the speaker is composed of P1, P2, and P3, and the sound sensing unit 100 receives sound from the speaker. The image capturing unit 200 captures an area where sound can be received by the sound sensing unit 100. [ The sound information sensed by the sound sensing unit 100 and the image information sensed by the image sensing unit 200 are transmitted to the sound processing unit 300.

The sound processing unit 300 identifies the speaker from the received sound information. A method for identifying a speaker according to sound detection will be described with reference to FIG.

Referring to the 'grid position coordinate setting' of FIG. 3, it can be seen that a virtual sensing area recognizable through the sound sensing unit 100 is set. The position coordinates of the grating are set in the sensing area. In the case of the embodiment of Fig. 3, the grid position coordinates of 3 x 5 are set, and each position coordinate is a position at which the sound can be identified. On the other hand, the arrangement of the microphone array and the arrangement of the position coordinates are not the same and are set by the user.

The beam power calculator 321 calculates the beam power level by summing the received sound signals for each grid position coordinate. The voice position identification unit 320 identifies the position coordinate at which the strongest beam power level is calculated among the beam power levels calculated at the respective position coordinates, as the position at which the sound (speaker voice) is received. On the other hand, a beamforming output value for sound source extraction is generated by applying a weight that minimizes the analysis of the output to the beam power level in order to minimize the interference of noise (noise) received outside the position where the sound (speaker voice) is received.

Referring to the 'speaker position designation' of FIG. 3, the sound position identifying unit 320 has identified the position coordinate '3' as the position at which the sound was received. The character-mapping unit 330 specifies that the speaker is located at the position coordinate '3' at which the sound is received.

4 is a diagram for explaining a method of mapping sound position and image information according to the present invention.

Referring to the 'speaker position and image information mapping' of FIG. 4, the image information photographed by the image capturing unit 200 by the mapping unit 350 and the grid position coordinates set by the sensing region setting unit 310 are mapped do. On the other hand, the position of the speaker is identified in the image information as the sound position is designated by the image pitcher 330 in the grid position coordinates.

The video output unit 400 displays a text in which the speaker's voice is converted into the video information to which the sound position is mapped. Referring to FIG. 4, 'Text Display by Speaker', it is seen that speech information of a speaker is converted into text and displayed. FIG. 4 is a screen in which a speaker is designated at the position coordinate '3', and the sound sensed at the position coordinate '3' is recognized as speech information of the speaker and output as text.

The image output unit 400 according to the present invention can display the speaker's voice by text and display it by speaker. In the case of 'display of text by speaker', when position coordinate '3' is designated as 'user 1' and 'speaker 2' which is another speaker is designated in position coordinate '1' Can be displayed separately from 'User 1'.

FIG. 5 is a diagram for explaining automatic generation of log information and recording of minutes according to the present invention.

The log generation unit 360 is a device for sequentially generating texts to generate log information. In the chat log of FIG. 5, it can be seen that conversation contents and conversation time are recorded for each user (speaker). Logs recorded for each user (speaker) are sequentially recorded according to the dialog sequence.

The meeting record creating unit 370 generates the meeting record information through the log information. 5, a save button is provided. When the user clicks the save button, the log information recorded in the chat log is stored to generate the minutes.

100: sound detection unit 200:
300: sound processing unit 310: detection area setting unit
320: sound position identifying unit 321: beam power calculating unit
322: beam forming unit 323: sound position determining unit
330: a text input unit 340: a text conversion unit
350: mapping unit 360: log generation unit
370: minutes recording section 400: video output section

Claims (4)

A sound sensing unit for sensing sound through the microphone array;
A sensing region setting unit for setting a sensing region of the sound sensing unit and setting a grid position coordinate of the sensing region to recognize a position of a sound generated in the sensing region;
A sound position identifying unit for identifying a sound position through position coordinates of the sound sensed in the sensing region;
A dialogue unit for designating the sound position as a speaker; And
And a text conversion unit for recognizing the sound of the speaker designated by the speech output unit as a speaker's voice and converting the sound into a text.
The method according to claim 1,
An image photographing unit for photographing a specific area;
A mapping unit for mapping the sound position and the image information photographed by the image photographing unit; And
And an image output unit for displaying the text in real time on the image information to which the sound position is mapped.
3. The method of claim 2,
A log generation unit for sequentially writing the text to generate log information; And
And a meeting record preparing unit for generating meeting record information through the log information.
2. The apparatus of claim 1, wherein the acoustic location identifier
A beam power calculation unit for calculating a beam power level corresponding to each of the grid position coordinates by summing the sound signals received for each of the grid position coordinates;
A beamforming generator for generating a beamforming output value for extracting a sound source by applying a weight to minimize the analysis of the output to the beam power level; And
And a sound position designating unit for designating a coordinate of a position having the strongest intensity of the beam forming output value as the sound position.

Images