KR101889809B1 - Automatic direction selection speech recognition system using important topic selection and Method - Google Patents

Abstract

The present invention relates to an automatic direction selection sound recognition system and a sound recognition method using important topic selection. More specifically, the present invention relates to an automatic direction selection sound recognition system and a sound recognition method using important topic selection, which separate sound input by multi-channel into a sound source, perform sound recognition for each separated sound, analyze recognition result content for each channel, automatically select a channel microphone to which the sound fit to a topic desired by a user is input, increase sensitivity of the corresponding channel microphone and ensure high recognition performance, thereby reproducing a cocktail party effect. According to the present invention, the present invention selects a place for conversation on a topic related to a predetermined topic, separate the sound source and recognize the corresponding direction, thereby reproducing the cocktail party effect. The automatic direction selection sound recognition system comprises: at least one channel microphone (100); a multi-channel sound source input part (210); a multi-channel sound source separating part (220); a multi-channel sound recognizing part (230); a multi-channel content analyzing part (240); a topic model part (250); a topic conformity degree determining part (260); an important topic selection automatic direction selection sound recognizing device (200); and a content server (300).

Description

Technical Field [0001] The present invention relates to an automatic direction selection speech recognition system and an automatic direction selection speech recognition system,

The present invention relates to an automatic directional speech recognition system and a speech recognition method using important agent selection, and more particularly, to an automatic directional speech recognition system and a speech recognition method, By analyzing the result contents, the channel microphone to which the voice corresponding to the topic desired by the user is automatically selected, thereby increasing the sensitivity of the corresponding channel microphone, thereby securing high recognition performance. Thus, it is possible to reproduce the cocktail party effect An automatic direction selection speech recognition system, and a speech recognition method.

Speech recognition technology is a technology that converts the acoustic speech signal obtained by a computer through a sound sensor such as a microphone into a word or a sentence. Generally, the acoustic signal is extracted and then the noise is removed. And then the feature of the voice signal is extracted and compared with the voice model database (DB).

That is, the input voice is analyzed and the features are extracted, and the similarity with the pre-collected voice model database (DB) is measured and the most similar thing is converted into text or command.

Although the above speech recognition technology has been used only for a special purpose in the past, the technology and the service utilizing the voice recognition technology have been rapidly developed due to the spread of the portable computing terminal represented by the smart phone and the cloud infrastructure to support it Speech recognition technology has limitations in accurately analyzing the user's intention and keyword.

Accordingly, deep network learning technology is rapidly developing. Deep network learning technology (hereinafter, referred to as deep learning) is a technique of combining a large amount of data or complex data Is defined as a machine learning algorithm that attempts to abstract the core contents or functions, that is, a machine learning algorithm that attempts to abstraction. In a wide range, it mimics the mechanism of neurons constituting the human brain, An artificial intelligence system implemented as a computer program to perform similar functions.

Such deep-learning techniques are applied to various fields such as computer vision, speech recognition, natural language processing, or signal processing.

However, there is no technology that focuses on specific topics and recognizes speech recognition technology utilizing deep network learning technology.

For example, in an environment where a large number of people are simultaneously voiced, there is no skill to better recognize the voice of a particular person, and thus the cocktail party effect can not be realized.

The cocktail party effect described above is an effect that shows that the human auditory ability has the ability to concentrate on a specific sound, and there is no technology yet to implement this method.

Accordingly, an automatic directional speech recognition system using an important topic selection that can reproduce a cocktail party effect by analyzing contents while simultaneously recognizing speech of a plurality of channels and adjusting the speech recognition more closely to a predetermined topic model And a speech recognition method.

(Prior Art 1) Korean Patent Registration No. 10-0655489

SUMMARY OF THE INVENTION Accordingly, the present invention has been made keeping in mind the above problems occurring in the prior art, and it is a primary object of the present invention to provide a speech recognition apparatus, a speech recognition method, Analyzing the contents, and automatically selecting a channel microphone to which a voice corresponding to a topic desired by the user is input, thereby increasing the sensitivity of the corresponding channel microphone, thereby ensuring high recognition performance.

The second object of the present invention is to provide a topic modeling unit 250 for crawling external contents from the contents server 300 and periodically performing analysis and learning of external contents using a deep learning model, By performing the update, it is intended to improve the accuracy of topic modeling.

In order to achieve the object of the present invention, there is provided an automatic directional speech recognition system using important agent selection,

And a channel microphone control unit 110 for performing at least one of a microphone direction control, a microphone distance movement control, and a microphone sensitivity adjustment control. The at least one channel microphone 100 )Wow,

A multi-channel sound source input unit 210 for receiving a plurality of sound sources from the plurality of channel microphones and providing the multi-channel sound source separation unit to the multi-channel sound source separation unit;

A multi-channel sound source separation unit 220 for separating a plurality of sound sources provided from the multi-channel sound source input unit for each channel and providing a plurality of separated sound sources to the multi-channel sound recognition unit;

A multi-channel speech recognition unit (230) for recognizing a plurality of sound sources provided from the multi-channel sound source separation unit for each channel;

Acquires speech recognition information for each channel from the multi-channel speech recognition unit, performs natural language processing for each channel, performs content analysis for each channel, generates speech recognition summary information for each channel by referring to the performed content analysis, A multi-channel content analyzing unit 240 for providing the topic matching degree to the judging unit;

A topic modeling unit 250 for modeling topic information to be carefully recognized by crawling external contents from the content server 300 and providing the modeled topic model information to the topic matching degree determining unit;

The method comprising: calculating a probabilistic distance value between the speech recognition summary information for each channel and the modeled topic model information provided by the multi-channel content analyzing unit, determining a channel providing a topic match degree with a probabilistic distance value equal to or greater than a reference value, To the speech input sensitivity enhancement adjusting unit (260);

And an audio input sensitivity enhancement adjusting unit (270) for increasing the audio input sensitivity of the channel microphone when the channel microphone information is obtained from the topic match degree determining unit (200). Wow,

And a contents server 300 for periodically providing external contents so as to generate topic model information in the automatic direction selection speech recognition apparatus for sorting important information.

Meanwhile, in the automatic direction selection speech recognition method using the important agent selection method according to the present invention,

The multi-channel sound source input unit 210 of the speech recognition apparatus 200 includes a channel microphone control unit 110 for performing at least one of a microphone direction adjustment control, a microphone distance movement control, and a microphone sensitivity adjustment control, A multi-channel sound source input step (S100) for receiving a plurality of sound sources from at least one channel microphone (100) configured to acquire an ambient sound source and providing the sound sources to the multi-channel sound source separation unit,

The multi-channel sound source separation unit 220 of the speech recognition apparatus 200 separates a plurality of sound sources provided from the multi-channel sound source input unit 210 for each channel, and separates a plurality of separated sound sources into multi- A multi-channel sound source separation step S200 for providing a sound recognition unit,

The multi-channel speech recognition unit 230 of the speech recognition apparatus 200 recognizes a plurality of sound sources provided from the multi-channel sound source separation unit 220 by a multi-channel speech recognition step S300)

The multichannel content analyzer 240 of the speech recognition apparatus 200 acquires speech recognition information for each channel from the multichannel speech recognition unit 230 and performs natural language processing for each channel, A multi-channel content analyzing step (S400) for analyzing the content of each channel, generating speech recognition summary information for each channel by referring to the analyzed content analysis,

The topic model unit 250 of the important topic selection automatic direction selection unit 210 crawls the external content from the content server 300 to model the topic information to be carefully recognized and transmits the modeled topic model information to the topic matching information A topic modeling step (S500) for providing to the judgment unit,

The topic matching degree determination unit 260 of the speech recognition apparatus 200 selects a probabilistic distance value between the speech recognition summary information for each channel and the modeled topic model information provided by the multi-channel content analysis unit 240 (S600) for determining a channel providing a topic match degree of which the probabilistic distance value is equal to or greater than a reference value and providing the corresponding channel microphone information to the speech input sensitivity enhancement adjusting unit,

When the speech input sensitivity enhancement adjusting unit 270 of the speech recognition apparatus 200 acquires the channel microphone information from the topic match degree determining unit 260, the voice input sensitivity enhancement unit 270 selects a voice for increasing the voice input sensitivity of the corresponding channel microphone And an input sensitivity enhancement adjustment step S700.

The automatic directional speech recognition system and the speech recognition method according to the present invention having the above-described constitution and action according to the present invention are capable of separating a sound input through a multi-channel into sound sources, performing speech recognition for each separated sound, By analyzing the contents of the recognition result, the channel microphone to which the voice corresponding to the topic desired by the user is automatically selected, thereby increasing the sensitivity of the corresponding channel microphone, thereby securing a high recognition performance, thereby exhibiting the effect of reproducing the cocktail party effect do.

In addition, by providing a topic model unit, external content is crawled from an external content server, and analysis and learning of external contents are periodically performed by using a deep learning model to continuously perform topic modeling and updating, thereby improving the accuracy of topic modeling So as to provide intensive sensitivity enhancement accuracy for channels with high topic similarity.

That is, through the present invention, it is possible to provide a real-time analyzing result of speech recognition using multi-channel sound source separation and direction detection, multi-channel real-time speech input and speech recognition, natural language processing and machine learning model, Generation effect, similarity matching with a predetermined topic, and effect of performing a speech input compensation action on a channel having a high topic similarity according to comparison, thereby reproducing a cocktail party effect that only humans have. Thus, It is possible to intensively perceive only the voice of the important direction to be performed.

In summary, it is possible to reproduce a cocktail party effect by selecting a place where a conversation on a topic related to the topic determined by the present invention is performed, separating a sound source, and recognizing the corresponding direction.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a general schematic diagram of an automatic directional speech recognition system using an important agent selection according to a first embodiment of the present invention; FIG.
FIG. 2 is a conceptual diagram of an automatic directional speech recognition system using important agent selection according to the first embodiment of the present invention. FIG.
FIG. 3 is a block diagram of an automatic direction selection speech recognition apparatus for sorting an important agent in an automatic direction selection speech recognition system using important agent selection according to the first embodiment of the present invention. FIG.
FIG. 4 is a block diagram of a multi-channel content analysis unit included in an automatic direction selection speech recognition apparatus for selecting an important agent in an automatic direction selection speech recognition system according to the first embodiment of the present invention.
FIG. 5 is a block diagram of a topic model block included in an automatic direction selection speech recognition apparatus for selecting an important agent in an automatic direction selection speech recognition system using important agent selection according to the first embodiment of the present invention.
FIG. 6 is a block diagram of a topic matching degree determination unit included in the automatic direction selection speech recognition apparatus for selecting the important agent in the automatic direction selection speech recognition system using the important agent selection according to the first embodiment of the present invention.
FIG. 7 is a block diagram of a speech input sensitivity enhancement adjusting unit included in an automatic directional speech recognition apparatus for selecting an important speech signal in an automatic directional speech recognition system using important agent selection according to a first embodiment of the present invention; FIG.
FIG. 8 is an overall flowchart of an automatic direction selection speech recognition method using an important agent selection according to the first embodiment of the present invention. FIG.
FIG. 9 is a flowchart of a multi-channel content analysis step of an automatic direction selection speech recognition method using an important agent selection method according to the first embodiment of the present invention. FIG.
10 is a flowchart of a topic model step of an automatic directional speech recognition method using an important agent selection according to the first embodiment of the present invention.
11 is a flowchart of a topic matching degree determination step of an automatic directional selection speech recognition method using important agent selection according to the first embodiment of the present invention.

The following merely illustrates the principles of the invention. Therefore, those skilled in the art will be able to devise various apparatuses which, although not explicitly described or illustrated herein, embody the principles of the invention and are included in the concept and scope of the invention.

Furthermore, all of the conditional terms and embodiments listed herein are, in principle, only intended for the purpose of enabling understanding of the concepts of the present invention, and are not to be construed as limited to such specifically recited embodiments and conditions do.

In describing the present invention, the terms first, second, etc. may be used to describe various elements, but the elements may not be limited by terms.

For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

It is to be understood that when an element is referred to as being connected or connected to another element, it may be directly connected or connected to the other element, but it may be understood that other elements may be present in between .

It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention, the singular forms of which may include a plurality of representations, unless the context clearly indicates otherwise.

It is to be understood that the term " comprising, " or " comprising " as used herein is intended to specify the presence of stated features, integers, But do not preclude the presence or addition of steps, operations, elements, components, or combinations thereof.

≪ Embodiment 1 >

The automatic direction selection speech recognition system using the important agent selection according to the first embodiment of the present invention,

And a channel microphone control unit 110 for performing at least one of a microphone direction control, a microphone distance movement control, and a microphone sensitivity adjustment control. The at least one channel microphone 100 )Wow,

A multi-channel sound source input unit 210 for receiving a plurality of sound sources from the plurality of channel microphones and providing the multi-channel sound source separation unit to the multi-channel sound source separation unit;

A multi-channel sound source separation unit 220 for separating a plurality of sound sources provided from the multi-channel sound source input unit for each channel and providing a plurality of separated sound sources to the multi-channel sound recognition unit;

A multi-channel speech recognition unit (230) for recognizing a plurality of sound sources provided from the multi-channel sound source separation unit for each channel;

Acquires speech recognition information for each channel from the multi-channel speech recognition unit, performs natural language processing for each channel, performs content analysis for each channel, generates speech recognition summary information for each channel by referring to the performed content analysis, A multi-channel content analyzing unit 240 for providing the topic matching degree to the judging unit;

A topic modeling unit 250 for modeling topic information to be carefully recognized by crawling external contents from the content server 300 and providing the modeled topic model information to the topic matching degree determining unit;

The method comprising: calculating a probabilistic distance value between the speech recognition summary information for each channel and the modeled topic model information provided by the multi-channel content analyzing unit, determining a channel providing a topic match degree with a probabilistic distance value equal to or greater than a reference value, To the speech input sensitivity enhancement adjusting unit (260);

And an audio input sensitivity enhancement adjusting unit (270) for increasing the audio input sensitivity of the channel microphone when the channel microphone information is obtained from the topic match degree determining unit (200). Wow,

And a content server (300) for periodically providing external content to generate topic model information in the automatic direction selection speech recognition apparatus for sorting important information.

At this time, the multi-channel content analyzer 240 analyzes the multi-

A channel-specific natural language processing module 241 for acquiring speech recognition information for each channel from the multi-channel speech recognition unit and performing natural language processing for each channel;

Extracting intention information and key words of the utterance for each channel with reference to the natural language processing result processed for each channel by the natural language processing module for each channel and providing the speech recognition summary information to the speech recognition summary information generation module, (242);

The speech recognition summary information for each channel is generated by referring to the per-channel intention information and the keyword intention information and key words provided for each channel provided by the key word extraction module, and provides the speech recognition summary information to the topic consistency determination unit 260 And a generation module (243).

At this time, the topic modeling unit 250,

A content collection module 251 for crawling external content from the content server 300 and collecting data;

A topic parameter adjustment module 252 for setting intention information and key words to be carefully recognized to generate a topic model;

A topic natural language processing module (253) for acquiring contents containing intention information and key words set by the topic parameter adjustment module by the content acquisition module and performing natural language processing;

A topic model generation module 254 for modeling the topic information including the intention information and key words set by the user with reference to the natural language processing result and providing the modeled topic model information to the topic match degree determination unit 260 .

At this time, the topic match degree determining unit 260,

A channel-based match degree determination information acquisition module 261 for acquiring speech recognition summary information and modeled topic model information for each channel provided by the multi-channel content analysis unit and providing the obtained speech recognition summary information to the topic match degree determination module;

Calculates a probabilistic distance value between the speech recognition summary information for each channel and the modeled topic model information provided by the per-channel match degree information acquisition module, and determines a channel for which the calculated probabilistic distance value is greater than or equal to a reference value A topic match degree judging module 262 for judging a match degree;

And a topic correspondence degree output module 263 for acquiring a channel for providing a topic agreement degree equal to or higher than a reference value in the topic match degree determination module and providing the microphone information of the corresponding channel to the speech input sensitivity enhancement adjuster 270 .

At this time, the voice input sensitivity enhancement adjusting unit 270 adjusts,

A channel microphone information acquisition module 271 for acquiring channel microphone information from the topic matching degree determination unit and providing corresponding channel microphone information to the voice input sensitivity enhancement adjustment module 272;

In order to increase the voice input sensitivity of the channel microphone when the channel microphone information is acquired from the channel microphone information acquisition module, information on at least one of the microphone direction adjustment information, the microphone distance movement information, and the microphone sensitivity adjustment information is generated, And an audio input sensitivity enhancement adjustment module (272) for transmitting the audio input sensitivity enhancement adjustment module (272) to the control unit (110).

At this time, the topic modeling unit 250,

In order to select important topic information desired by the user, a machine learning model of a deep learning model, a machine learning, a support vector machine (SVM), and a neural network is applied to a topic model Information is generated.

At this time, the topic modeling unit 250,

Crawling external content from the content server 300 and periodically performing analysis and learning of external content using a deep learning model to perform continuous topic modeling and updating to improve the accuracy of topic modeling.

Meanwhile, the automatic direction selection speech recognition method using the important agent selection according to the first embodiment of the present invention,

The multi-channel sound source input unit 210 of the speech recognition apparatus 200 includes a channel microphone control unit 110 for performing at least one of a microphone direction adjustment control, a microphone distance movement control, and a microphone sensitivity adjustment control, A multi-channel sound source input step (S100) for receiving a plurality of sound sources from at least one channel microphone (100) configured to acquire an ambient sound source and providing the sound sources to the multi-channel sound source separation unit,

The multi-channel sound source separation unit 220 of the speech recognition apparatus 200 separates a plurality of sound sources provided from the multi-channel sound source input unit 210 for each channel, and separates a plurality of separated sound sources into multi- A multi-channel sound source separation step S200 for providing a sound recognition unit,

The multi-channel speech recognition unit 230 of the speech recognition apparatus 200 recognizes a plurality of sound sources provided from the multi-channel sound source separation unit 220 by a multi-channel speech recognition step S300)

The multichannel content analyzer 240 of the speech recognition apparatus 200 acquires speech recognition information for each channel from the multichannel speech recognition unit 230 and performs natural language processing for each channel, A multi-channel content analyzing step (S400) for analyzing the content of each channel, generating speech recognition summary information for each channel by referring to the analyzed content analysis,

The topic model unit 250 of the important topic selection automatic direction selection unit 210 crawls the external content from the content server 300 to model the topic information to be carefully recognized and transmits the modeled topic model information to the topic matching information A topic modeling step (S500) for providing to the judgment unit,

The topic matching degree determination unit 260 of the speech recognition apparatus 200 selects a probabilistic distance value between the speech recognition summary information for each channel and the modeled topic model information provided by the multi-channel content analysis unit 240 (S600) for determining a channel providing a topic match degree of which the probabilistic distance value is equal to or greater than a reference value and providing the corresponding channel microphone information to the speech input sensitivity enhancement adjusting unit,

When the speech input sensitivity enhancement adjusting unit 270 of the speech recognition apparatus 200 acquires the channel microphone information from the topic match degree determining unit 260, the voice input sensitivity enhancement unit 270 selects a voice for increasing the voice input sensitivity of the corresponding channel microphone And an input sensitivity enhancement adjustment step (S700).

At this time, the multi-channel content analysis step (S400)

Channel natural language processing module 241 of the multi-channel content analyzer 240 acquires the speech recognition information for each channel from the multi-channel speech recognition unit and performs natural language processing for each channel, Processing step S410;

Channel intention information of the multi-channel content analyzer 240 and the keyword extraction module 242 refer to the natural language processing result processed for each channel by the channel-by-channel natural language processing module 241, (Step S420) of extracting keyword information and keyword for providing information to the speech recognition summary information generation module by extracting information and key words;

The speech recognition summary information generation module 243 of the multi-channel content analysis unit 240 refers to the per-channel degree information and the intention information and key words for each channel provided by the key word extraction module 242, And a speech recognition summary information generating step (S430) for generating speech recognition summary information and providing the generated speech recognition summary information to the topic match degree determining unit 260. [

At this time, the topic model step (S500)

A content collecting step (S510) for the content collecting module 251 of the topic modeling unit 250 to collect data by crawling external contents from the contents server 300;

A topic parameter adjustment step (S520) for setting the intention information and key words that the topic parameter adjustment module 252 of the topic model unit 250 intends to carefully recognize to generate the topic model;

A topic natural language processing module 253 of the topic model unit 250 acquires the contents including the intention information and key words set by the topic parameter adjustment module 252, Processing step S530;

The topic model generation module 254 of the topic model unit 250 models the topic information including the intention information and key words set by the user with reference to the natural language processing result and outputs the modeled topic model information to the topic match degree determination unit 260 (S540) of providing a topic model for providing a topic model.

In this case, the topic match degree determination step S600 may include:

The channel matching degree information acquisition module 261 of the topic matching degree determination unit 260 acquires the speech recognition summary information for each channel and the modeled topic model information provided by the multi-channel content analysis unit and provides the same to the topic matching degree determination module A step S610 of acquiring information on the degree of match of each channel;

The topic matching degree judging module 262 of the topic matching degree judging unit 260 calculates the probabilistic distance values between the speech recognition summary information for each channel and the modeled topic model information provided by the channel matching degree judgment information acquiring module 261 A topic matching degree determination step (S620) for determining a channel in which the calculated probabilistic distance value provides a topic matching degree equal to or higher than a reference value;

The topic matching degree result output module 263 of the topic matching degree judging unit 260 obtains the channel providing the topic matching degree higher than the reference value in the topic matching degree judging module 262 and transmits the microphone information of the corresponding channel to the voice input sensitivity enhancing adjusting unit 270 (Step S630) of outputting a topic match degree result for providing the topic match result.

Hereinafter, embodiments of the automatic direction selection speech recognition system and the speech recognition method using the important agent selection according to the present invention will be described in detail.

FIG. 1 is an overall schematic diagram of an automatic directional speech recognition system using important agent selection according to a first embodiment of the present invention.

As shown in FIG. 1, the automatic directional speech recognition system using important agent selection according to the present invention includes a plurality of channel microphones 100a to 100n, an important direction selection automatic direction selection speech recognition device 200, a content server 300, .

Through the above-described configuration, the voice input through the multiple channels is separated from the sound source, the voice recognition is performed for each separated voice, and the contents of the recognition result for each channel are analyzed to provide a channel microphone And the sensitivity of the corresponding channel microphones is increased to secure a high recognition performance, so that the cocktail party effect can be reproduced.

As shown in FIG. 2, in order to achieve the above-mentioned effects, the key recognition automatic speech recognition apparatus 200 for identifying important speech, which is a core component, is provided with at least one channel microphone 100a to 100n, After receiving the conversation sound source, the speech recognition is performed for each channel. After analyzing the content through the natural language processing, the degree of matching between the analyzed content information per channel and the topic model information determined by the user is measured, .

Thereafter, the sensitivity of the channel microphone to which a voice corresponding to a topic desired by the user is input is increased under the control of the channel microphone control unit 110. [

At least one of the channel microphones 100 is installed at a specific location, and a plurality of channel microphones 100a to 100n are used to acquire voice sound sources of various people.

Here, the channel microphone may include a channel microphone controller 110 for performing at least one of a microphone direction adjustment control, a microphone distance movement control, and a microphone sensitivity adjustment control.

In order to receive various people's voices in the space where the video is frequently used, such as video conferencing, seminars, broadcasting, subway history, and the general public, a channel microphone is provided with a channel microphone control unit for adjusting a microphone direction, moving a microphone distance, (110). Through this, conversation contents of a specific direction and a specific distance are obtained through sensitivity adjustment.

Accordingly, in the present invention, a channel microphone control unit is installed for each channel microphone in order to recognize the specific sound source, thereby acquiring an ambient sound source.

On the other hand, according to another embodiment, it is obvious that only the sensitivity adjustment control information for adjusting only the sensitivity can be provided if it is introduced into a microphone which can not perform direction adjustment and microphone distance movement.

In addition, the content server 300 periodically performs the function of providing the external contents so as to generate the topic model information to the automatic direction selecting speech recognition apparatus for sorting important matters.

At this time, the topic model generation is performed periodically, and the generated topic model information is stored in a database and stored and managed so that continuous performance update is possible.

Hereinafter, the specific constituent elements of the automatic speech recognition apparatus 200 for selecting importance sorting automatic speech selection will be described in detail with reference to the following drawings.

FIG. 3 is a block diagram of an automatic direction selection speech recognition apparatus for selecting an important agent in the automatic direction selection speech recognition system using the important agent selection according to the first embodiment of the present invention.

3, the important speech recognizing automatic directional voice recognition apparatus 200 according to the present invention includes a multi-channel sound source input unit 210, a multi-channel sound source separation unit 220, a multi-channel speech recognition unit 230, A multi-channel content analysis unit 240, a topic model unit 250, a topic matching degree determination unit 260, and a voice input sensitivity enhancement adjustment unit 270.

According to the constitutional feature of the automatic direction selecting speech recognizing apparatus 200, the voice inputted through the multiple channels is separated into the sound sources, the voice recognition is performed for each separated voice, and the contents of the recognition result for each channel are analyzed A channel microphone to which a voice corresponding to a topic desired by a user is inputted is automatically selected to increase the sensitivity of the corresponding channel microphone, thereby securing a high recognition performance, thereby demonstrating the effect of reproducing a cocktail party effect.

The components of the important speech recognition automatic directional voice recognition apparatus 200 for providing the above effect will be described in detail below.

The multi-channel sound source input unit 210 receives a plurality of sound sources from the plurality of channel microphones 100a, 100b, 100b, ..., 100n, and provides the multi-channel sound source separation unit.

For example, the first sound source information is transmitted from the first channel microphone 100a, the second sound source information is transmitted from the second channel microphone 100b, the third sound source information is transmitted from the third channel microphone 100c, And receives the n-th sound source information from the second sound source 100n, and provides them to the multi-channel sound source separation unit.

Then, the multi-channel sound source separation unit 220 separates a plurality of sound sources provided from the multi-channel sound source input unit for each channel, and provides a plurality of separated sound sources to the multi-channel sound recognition unit.

For example, the three sound source information may include first sound source information # 01_VOICE, second sound source information # 02_VOICE, and third sound source information # 03_VOICE, which are separately provided to the multi-channel speech recognition unit.

At this time, the multi-channel speech recognition unit 230 recognizes a plurality of sound sources provided from the multi-channel sound source separation unit for each channel.

For example, the first sound source information # 01_VOICE is acquired by the first multi-channel speech recognition unit 230a to perform speech recognition, and the second sound source information # 02_VOICE is acquired by the second multi-channel speech recognition unit 230b. And the third sound source information # 03_VOICE is acquired by the third multi-channel speech recognition unit 230c to perform speech recognition.

The above-described speech recognition technology refers to a technique of converting an acoustic speech signal obtained by a computer through a sound sensor such as a microphone into a word or a sentence. In general, a technique of extracting a sound signal and then removing noise do.

Accordingly, speech recognition information including a word or a text of a result of speech recognition for each channel is generated and provided to the multi-channel content analyzer 240. [

For example, in the first multi-channel speech recognition unit 230a, # 01_TEXT, in which # 01_VOICE is recognized as the first tone source information, and # 02_VOICE in the second multi-channel speech recognition unit 230b, And the third multi-channel speech recognition unit 230c provides the multi-channel content analysis unit 240 with # 03_TEXT, which is voice recognition of # 03_VOICE, which is the third source information, to the multi-channel content analysis unit 240.

Thereafter, the multi-channel content analyzer 240 acquires speech recognition information for each channel from the multi-channel speech recognition unit, performs natural language processing for each channel, performs content analysis for each channel, Thereby generating speech recognition summary information for each channel and providing the speech recognition summary information to the topic matching degree determination unit.

For example, the first multi-channel speech recognition unit 230a provides # 01_TEXT provided by the first multi-channel speech recognition unit 230a to the first multi-channel content analysis unit 240a to perform natural language processing, And provides the provided # 02_TEXT to the second multi-channel content analyzer 240b to perform natural language processing. The # 03_TEXT provided from the third multi-channel speech recognizer 230c is transmitted to the third multi-channel content analyzer 240c So that natural language processing is performed.

At this time, each of the multi-channel content analyzing units performs content analysis, generates speech recognition summary information for each channel by referring to the performed content analysis, and provides the generated speech recognition summary information to the topic match degree determination unit .

For example, from the first multi-channel content analyzer 240a, speech recognition summary information 'Lee Myung-bak, Moon Jae-in, Park Geun-hye, , Coupon ', generates speech recognition summary information' English, score, and mathematics' from the third multi-channel content analyzer 240c, and outputs the generated speech recognition summary information for each channel to the topic agreement And provides it to the judgment unit.

In addition, the topic modeling unit 250 crawls external content from the content server 300 to model the topic information to be carefully recognized, and provides the modeled topic model information to the topic matching degree determining unit .

When the user provides the topic information to be carefully recognized, the external content related thereto is crawled and the corresponding topic information is modeled. For example, if the user selects 'election' as a topic topic, And to determine a channel that has uttered high-agreement contents.

The Topic Model described in the present invention is a model for extracting topics from a document set, and is used for natural language processing and the like.

LSA and other vector-based models use word vectors to represent documents in a multidimensional manner, while topic models express the topics contained in a document as a probability distribution, based on the distribution of words varying according to a particular topic.

Using the topic model allows you to represent the document in low dimensions and also extract potential topics.

Latent Dirichlet Allocation (LDA), a representative topic model in natural language processing, is a probability model that assigns topics to documents.

By estimating the distribution of themed words from a given document and analyzing the distribution of the words found in a given document, you can observe what topics the document deals with.

Potential Dirichlet allocation is a concise and practical topic model that is widely applied to various researches and products, and Chinese IT company Tencent has commercialized Peacock, a large-scale potential topic extraction project using potential Dirichlet allocation.

Peacock studied 101 million topics through a parallel computation method that decomposes and computes a matrix of 1 billion × 100 million.

The learned topics are used in areas such as text word meaning extraction, recommendation system, user taste appreciation, and advertisement recommendation.

In addition to the LDA, there is a topic extraction method using word clustering method in topic extraction, and a topic clustering method is used to extract regional topic using regional news.

For example, suppose you look at the subject of a newspaper article over a period of time, and sections of election stories, international affairs, sports, etc. are related to topics, Elections, mayoral elections, and parliamentary elections.

Also, in relation to the presidential election, various topics such as presidential candidates and elections are linked.

In this way, not only newspaper articles, but also numerous articles contain various themes and they change continuously.

However, not only are they not explicitly exposed, they can not be manually attached to a subject.

Topic modeling is the process of analyzing the words in the original text to automatically find the themes in the document, to find out how the topics are linked to each other, and how these topics change with time .

Among these, Latent Dirichlet Allocation (LDA) is one of the generative models and is the most commonly used method in topic models.

A document is made up of a number of topics, and assuming that the words appearing in the document are generated by the topics, for example, the document was created by several themes, such as 'computer' or 'prediction' There may be topics in the document, such as 'data analysis' topics, 'evolutionary biology' topics such as 'life' or 'organism'.

The LDA is a tool for formalizing this concept. A document is a generation model related to a topic. A topic is defined as a distribution based on a specified vocabulary. As a result, a topic is predicted by the air relation of words appearing in the document.

In the present invention, topic modeling is performed in order to select a topic desired by the user, and any topic model used may be applied.

At this time, the topic match degree determining unit 260 calculates a probabilistic distance value between the speech recognition summary information for each channel and the modeled topic model information provided by the multi-channel content analyzing unit, And provides the channel microphone information to the voice input sensitivity enhancement adjusting unit.

The technique of calculating the probabilistic distance value is a technology that is mainly used in statistical or speech recognition technology and is generally applied to a topic model, Opinion mining, text summary, data analysis, opinion survey, etc. Therefore, It is obvious that those skilled in the art can fully understand the above-mentioned meaning.

(Reference: Aletras, N. and M. Stevenson, 'Evaluating topic

Proceedings of the 10th International Conference on Computational Semantics (IWCS), Potsdam, Germany, 2013, Blei, D., 'Probabilistic topic models,' Communications of the ACM, Vol.55, No.4, 2012), 77-84)

For example, if the reference value is set to 0.5, the probabilistic distance value between the speech recognition summary information provided from the first multi-channel content analyzer 240a and the modeled topic model information is 0.7, and the second multi- 240b) is 0.4, the probability distance value between the speech recognition summary information provided from the third multi-channel content analyzer 240c and the modeled topic model information Is 0.1, the first channel microphone 100a providing the sound source to the first multi-channel content analyzer 240a having a reference match degree of 0.5 or more is provided to the sound input sensitivity enhancement controller.

When acquiring the channel microphone information from the topic match degree determining unit, the speech input sensitivity enhancement adjusting unit 270 provides control information for increasing the voice input sensitivity of the corresponding channel microphone to the channel microphone control unit of the corresponding channel microphone .

FIG. 4 is a block diagram of a multi-channel content analysis unit included in an automatic direction selection speech recognition apparatus for selecting an importance agent in an automatic direction selection speech recognition system using important agent selection according to the first embodiment of the present invention.

4, the multi-channel content analyzer 240 includes a channel-by-channel natural language processing module 241, channel-specific degree information, keyword extraction module 242, and speech recognition summary information generation module 243 .

More specifically, the channel-by-channel natural language processing module 241 acquires speech recognition information for each channel from the multi-channel speech recognition unit and performs natural language processing for each channel.

That is, if all sound sources are integrated to perform natural language processing, it is impossible to provide a cocktail party effect, so that speech recognition information is acquired for each channel and natural language processing is performed based on each speech recognition information.

In the field of information retrieval, the natural language processing described above is used to grasp the intention of the user through natural conversation with the user and to collect and provide more accurate information from various types of data.

Detailed techniques of natural language processing techniques include speech analysis, information extraction, information summarization, and document classification techniques based on structural analysis such as morphological analysis and high-precision syntax analysis. And a technique of measuring similarity with the model DB and converting the most similar thing into text or command.

The per-channel intention information and key word extraction module 242 extracts intention information and keyword of each speech by referring to the natural language processing result processed for each channel by the natural language processing module for each channel, .

For example, it extracts intention information and key words of utterances such as 'Lee Myung-bak, Moon Jae-in, Park Geun-hye, DGE, and President' from the sound source obtained from the first channel micro. , Coupon 'and the like, and extracts intention information and key words of utterance such as' English, score, and mathematics' from sound sources obtained from the third channel microphones.

The utterance intention of the sound source obtained from the first channel micro is related to the election, the utterance intention of the sound source obtained from the second channel micro is the content related to the food, .

For example, the intention of the user who has uttered the 'guide the road, the restaurant nearby' recommends the restaurant around the current location and requests the guide to the restaurant, Since it is a well-known technology in the speech recognition technology, it is obvious that a person skilled in the art can easily understand what the intention information of the speech is based on the above description.

At this time, the speech recognition summary information generation module 243 generates speech recognition summary information for each channel by referring to the per-channel perception information and the intention information and the keyword of each channel provided by the keyword extraction module, And provides it to the determination unit 260.

For example, the speech recognition summary information of the sound source obtained from the first channel microphone is' # 01-POINT-election / Lee Myung-bak / Moon Jae-in / Park Geun-hye / , The speech recognition summary information of the sound source obtained from the third channel micro is generated as '# 02POINT-food / winning / coupons', '# 03POINT-academy / English / score / mathematics' ).

FIG. 5 is a block diagram of a topic model block included in an automatic direction selection speech recognition apparatus for selecting an importance agent in an automatic direction selection speech recognition system using important agent selection according to the first embodiment of the present invention.

5, the topic model unit 250 includes a content acquisition module 251, a topic parameter adjustment module 252, a topic natural language processing module 253, and a topic model generation module 254, do.

The Topic model described above is one of the text mining techniques used to discover the hidden semantic structure of the text body as one of statistical models for discovering the abstract "subject" of the document set.

In the literature on a particular topic, the word about the topic will appear more often than the other words.

For example, the words 'dog' and 'bone' appear more often in documents about dogs, whereas 'cat' and 'meons' appear more often in documents about cats, 'he', '~' 'Will often appear on both sides.

The words that often appear together often have a similar meaning, which can be defined as a potential 'subject'.

In other words, we can conceive a model that binds 'dog' and 'bone' as one subject, and 'cat' and 'meow' as another theme. This is a rough concept of the topic model.

In fact, it is possible to find out what kind of topics are included in the literature and how the weight among them is mathematically analyzed in terms of word statistics in the document set.

Thus, a topic model is also called a stochastic topic model, which is also used to refer to statistical algorithms for finding potential semantic structures in a broad text body.

With the advent of the information age, the daily generated text goes far beyond the amount that humans can handle directly, and the topic model can be used to automatically organize and organize a collection of unstructured texts.

In addition, the topic model is not only based on isolated word-based speech recognition, which outputs a result of a predetermined command such as' navigation, 'but also uttered in whole sentence units such as' I want to watch navigation, Based on a machine learning based language processing technique for predicting a speaker's intention and behavior, the present invention uses a topic model to select a topic desired by the user and analyzes the sound source having a high degree of matching with the topic.

More specifically, the content acquisition module 251 performs a function of collecting data by crawling external content from the content server 300. [

Content information is crawled in a content server connected using an external network or communication, and is collected through linguistic refinement.

The topic parameter adjustment module 252 performs functions for setting intention information and key words to be carefully recognized in order to generate a topic model. For example, I want to know the contents related to 'the key word' election, president, presidential candidate name, 'and so on.

That is, the intention and key words desired by the user can be adjusted through the topic parameter adjustment module, and the level of the semantic information required by the topic and the threshold value can be set.

This is disclosed in Korean Patent No. 10-1229108 and Korean Patent Registration No. 10-0362292, and a technique for parameter adjustment is well known to those skilled in the art, so that the above description can be fully understood.

Thereafter, the topic natural language processing module 253 provides the collection request information to the content collection module to collect the content including the intention information and the key word set by the topic parameter adjustment module, Processing is performed.

As described above, what the user desires is election-related content, but because there are many election-related contents, there are many words related to the election, so that it is applied as a model for extracting intention information and keywords.

For example, in relation to the election, there are various forms such as the president, the parliamentarian, the superintendent, the governor, and so on.

Accordingly, the topic model generation module 254 models the topic information including the intention information and the key word set by the user with reference to the natural language processing result, and provides the modeled topic model information to the topic match degree determination unit 260 .

That is, the topic model generation module 254 plays a role of setting what the user wants to listen to more carefully. The topic does not exist in a simple keyword form but exists in a form of a kind of model, The topic matching degree can be calculated by the probabilistic distance value with the result.

FIG. 6 is a block diagram of a topic matching degree determination unit included in an automatic direction selection speech recognition apparatus for selecting an important agent in an automatic direction selection speech recognition system using important agent selection according to the first embodiment of the present invention.

6, the topic match degree determining unit 260 includes a channel match degree information acquiring module 261, a topic match degree determining module 262, and a topic match degree result output module 263 .

Specifically, the per-channel match degree information acquisition module 261 acquires the speech recognition summary information for each channel and the modeled topic model information provided by the multi-channel content analysis unit, and provides them to the topic match degree determination module.

For example, the speech recognition summary information of the sound source obtained from the first channel microphone is' # 01-POINT-election / Lee Myung-bak / Moon Jae-in / Park Geun-hye / '# 03POINT-Academic / English / Score / Mathematics', which is the speech recognition summary information of the sound source obtained from the third channel microphone, and obtains the topic model information .

At this time, the topic match degree determining module 262 calculates the probabilistic distance value between the speech recognition summary information for each channel and the modeled topic model information provided by the channel-based match degree information acquiring module, And determines a channel providing the topic matching degree equal to or higher than the reference value.

For example, when the reference value is set to 0.5 and the speech recognition summary information provided from the first multi-channel content analyzer 240a is modeled with '# 01-POINT-election / Lee Myung-bak / Moon Jae-in / Park Geun-hye / The probabilistic distance value between the model information items is 0.7, and the probabilistic distance value between the speech recognition summary information '# 02POINT-food / winning / coupon' provided from the second multi-channel content analysis unit 240b and the modeled topic model information 0.4, and if the probabilistic distance value between the speech recognition summary information '# 03POINT-Academic / English / Score / Mathematics' provided from the third multi-channel content analyzer 240c and the modeled topic model information is 0.1, And provides the first channel microphone 100a providing the sound source to the first multi-channel content analyzer 240a having the above-mentioned topic agreement.

Then, the topic match degree result output module 263 acquires a channel providing a topic match degree higher than a reference value in the topic match degree determination module, and provides microphone information of the corresponding channel to the speech input sensitivity enhancement adjuster 270.

That is, the first channel microphone 100a providing the sound source to the first multi-channel content analyzer 240a having the reference match degree of 0.5 or more is provided to the speech input sensitivity enhancement adjuster 270. [

FIG. 7 is a block diagram of a voice input sensitivity enhancement adjusting unit included in an automatic directional voice selection apparatus for selecting an important agent in an automatic directional selection voice recognition system using important agent selection according to the first embodiment of the present invention.

7, the voice input sensitivity enhancement adjusting unit 270 includes a channel microphone information acquisition module 271 and a voice input sensitivity enhancement adjustment module 272. [

According to the above configuration, when acquiring channel microphone information from the topic matching degree determining unit, control information for increasing the voice input sensitivity of the corresponding channel microphone is provided to the channel microphone control unit of the corresponding channel microphone.

More specifically, the channel microphone information acquisition module 271 acquires channel microphone information from the topic matching degree determination unit and provides corresponding channel microphone information to the voice input sensitivity enhancement adjustment module 272, When acquiring the channel microphone information from the channel microphone information acquiring module, the adjustment module 272 acquires information on at least one of the microphone direction adjustment information, the microphone distance movement information, and the microphone sensitivity adjustment information in order to increase the voice input sensitivity of the corresponding channel microphone And transmits it to the corresponding channel microphone controller 110.

For example, when the information of the first channel microphone 100a is acquired, the information on the first channel microphone 100a is transmitted to the first channel microphone 100a in any one of the following directions such as 'direction of the microphone -30 degrees clockwise rotation' And transmits the generated information to the corresponding channel microphone controller 110.

At this time, the channel microphone control unit included in the first channel microphone acquires control information, transmits an operation signal to the rotation motor to rotate the microphone direction by 30 degrees in the clockwise direction, and transmits an operation signal And sends an adjustment signal to the sensitivity adjustment module to adjust the microphone sensitivity by 30 decibels.

This allows for better recognition performance by adjusting the microphone direction, moving the microphone distance, and adjusting the microphone sensitivity.

Generally, the direction of the microphone can be estimated according to the parallax into which each sound source is introduced, and the distance of the microphone can be adjusted by measuring the difference in sensitivity variation of the reference value.

The sensitivity of the microphone can also be adjusted so that the voice input of the corresponding channel can be received in the vicinity of the optimal sensitivity for speech recognition.

Meanwhile, in accordance with an additional aspect of the present invention, the topic modeling unit 250 may include a deep learning model, a machine learning, a support vector And a neural network, to generate topic model information by applying the machine learning model.

This means that language model information can be generated using various machine learning modeling techniques in addition to the topic model.

The deep learning model is defined as a machine learning algorithm that abstracts a core content or function in a large amount of data or complex data through a combination of various nonlinear transformation techniques. In the wide range, the human brain It refers to an artificial intelligence system implemented by a computer program so as to imitate the mechanism of the nerve cell constituting it and to perform functions similar to human brain activities such as cognition, learning and reasoning.

The deep learning model technique is applied to various fields such as computer vision, speech recognition, natural language processing, or signal processing. In the present invention, the deep learning model technique is used for modeling work for selecting important topics.

In addition, according to another additional aspect, the topic modeling unit 250 may crawl external content from the content server 300, periodically analyze and learn external content using a deep learning model, Updating is performed, thereby improving the accuracy of topic modeling.

This improves the accuracy of topic modeling, thereby providing intensive sensitivity enhancement for channels with high topic similarity.

For example, the content related to the election is acquired from the content server 300 at 5:00 PM on February 20, 2018, and the new content is requested to be connected again at 6:00 PM. If no new content exists The connection is terminated.

Then, the user is reconnected at 10:00 am on Feb. 21, 2018 to request whether there is new content. When new content exists, the content is requested and the process is repeated.

That is, continuously tuning and updating the content information or various contents information related to the main topic desired by the user, the continuous performance of the topic model is continuously improved in the long term.

Therefore, it is possible to provide high accuracy compared with statistical methods used in the conventional speech recognition field, and it is expected that continuous improvement of performance can be expected.

Hereinafter, a method for recognizing an automatic direction selection speech using the selection of important agents according to the present invention will be described in detail with reference to FIGS. 8 to 11. FIG.

FIG. 8 is an overall flowchart of an automatic directional selection speech recognition method using important agent selection according to the first embodiment of the present invention.

As shown in FIG. 8, the automatic direction selection speech recognition method using the importance agent sorting method includes a multi-channel sound source input step S100, a multi-channel sound source separation step S200, a multi-channel speech recognition step S300, A multi-channel content analysis step S400, a topic modeling step S500, a topic matching degree determination step S600, and a voice input sensitivity enhancement adjustment step S700.

More specifically, in the multi-channel sound source input step S100, the multi-channel sound source input unit 210 of the automatic sound direction selection voice recognition apparatus 200 selects a microphone direction adjustment control, a microphone distance movement control, a microphone sensitivity adjustment control And a channel microphone controller 110 for performing at least one of the control of the channel microphone 100 and the channel microphone controller 100. The channel microphone controller 100 receives a plurality of sound sources from at least one channel microphone 100 configured to acquire an ambient sound source, .

For example, the first sound source information is transmitted from the first channel microphone 100a, the second sound source information is transmitted from the second channel microphone 100b, the third sound source information is transmitted from the third channel microphone 100c, And receives the n-th sound source information from the second sound source 100n, and provides them to the multi-channel sound source separation unit.

In the multi-channel sound source separation step S200, the multi-channel sound source separation unit 220 of the automatic direction selection speech recognition apparatus separates a plurality of sound sources provided from the multi-channel sound source input unit 210 for each channel, And a plurality of separated sound sources are provided to the multi-channel speech recognition unit.

For example, the three sound source information may include first sound source information # 01_VOICE, second sound source information # 02_VOICE, and third sound source information # 03_VOICE, which are separately provided to the multi-channel speech recognition unit.

Then, in the multi-channel speech recognition step S300, the multi-channel speech recognition unit 230 of the automatic direction selection speech recognition apparatus for sorting important texts audits a plurality of sound sources provided from the multi-channel sound source separation unit 220 .

For example, the first sound source information # 01_VOICE is acquired by the first multi-channel speech recognition unit 230a to perform speech recognition, and the second sound source information # 02_VOICE is acquired by the second multi-channel speech recognition unit 230b. And the third sound source information # 03_VOICE is acquired by the third multi-channel speech recognition unit 230c to perform speech recognition.

Then, in the multi-channel content analyzing step S400, the multi-channel content analyzing unit 240 of the automatic direction selecting speech recognition apparatus obtains the speech recognition information for each channel from the multi-channel speech recognizing unit 230, Processing of the natural language is performed for each channel to analyze the contents of each channel, and the speech recognition summary information is generated for each channel by referring to the performed content analysis, and is provided to the topic matching determination unit.

For example, the first multi-channel speech recognition unit 230a provides # 01_TEXT provided by the first multi-channel speech recognition unit 230a to the first multi-channel content analysis unit 240a to perform natural language processing, And provides the provided # 02_TEXT to the second multi-channel content analyzer 240b to perform natural language processing. The # 03_TEXT provided from the third multi-channel speech recognizer 230c is transmitted to the third multi-channel content analyzer 240c So that natural language processing is performed.

Thereafter, the topic modeling step S500 models the topic information to be carefully recognized by the topic modeling unit 250 of the automatic direction selecting speech recognition apparatus by crawling the external contents from the contents server 300, The topic model information is provided to the topic matching degree judgment unit.

For example, when the user provides the topic information to be carefully recognized, the related external content is crawled and the corresponding topic information is modeled. For example, if the user selects 'election' as a topic topic, And the related topic information is modeled to judge the channel which is uttering contents having high agreement degree.

In step S600, the topic matching degree determination unit 260 of the automatic direction selection speech recognition apparatus selects the important speech recognition summary information for each channel provided by the multi-channel content analysis unit 240, And calculates the probabilistic distance value between the model information and determines the channel providing the topic matching degree at which the probabilistic distance value is equal to or higher than the reference value, and provides the corresponding channel microphone information to the voice input sensitivity enhancement adjusting unit.

For example, if the reference value is set to 0.5, the probabilistic distance value between the speech recognition summary information provided from the first multi-channel content analyzer 240a and the modeled topic model information is 0.7, and the second multi- 240b) is 0.4, the probability distance value between the speech recognition summary information provided from the third multi-channel content analyzer 240c and the modeled topic model information Is 0.1, the first channel microphone 100a providing the sound source to the first multi-channel content analyzer 240a having a reference match degree of 0.5 or more is provided to the sound input sensitivity enhancement controller.

Finally, in step S700, the speech input sensitivity enhancement adjustment unit 270 acquires the channel microphone information from the topic match degree determination unit 260, The voice input sensitivity of the corresponding channel microphone is increased.

FIG. 9 is a flowchart of a multi-channel content analyzing step of an automatic directional selection speech recognition method using an important agent selection according to the first embodiment of the present invention.

As shown in FIG. 9, the multi-channel content analyzing step S400 includes steps S410 to S410, step S420 of extracting key word information, key word extraction step S420, .

In more detail, the natural language processing step (S410) of the channel-by-channel natural language processing module 241 of the multi-channel content analyzer 240 acquires speech recognition information for each channel from the multi-channel speech recognition unit, And performs natural language processing for each channel.

Then, the per-channel intention information and key word extraction step S420 are performed by the per-channel natural language processing module 241 by the per-channel intention information of the multi-channel content analyzer 240 and the key word extraction module 242, By referring to the natural language processing result processed for each channel, intention information and key words of utterance are extracted for each channel and provided to the speech recognition summary information generation module.

The speech recognition summary information generating module 243 of the multi-channel content analyzer 240 finally generates the speech recognition summary information by using the channel-specific degree information and the angle information provided by the keyword extraction module 242 The speech recognition summary information is generated for each channel by referring to the intention information and the key word of each channel-based speech, and the speech recognition summary information is provided to the topic match degree determining unit 260.

For example, the speech recognition summary information of the sound source obtained from the first channel microphone is' # 01-POINT-election / Lee Myung-bak / Moon Jae-in / Park Geun-hye / , The speech recognition summary information of the sound source obtained from the third channel micro is generated as '# 02POINT-food / winning / coupons', '# 03POINT-academy / English / score / mathematics' ).

FIG. 10 is a flowchart of a topic model step of an automatic directional speech recognition method using important agent selection according to the first embodiment of the present invention.

10, the topic model step S500 includes a content collection step S510, a topic parameter adjustment step S520, a topic natural language processing step S530, and a topic model generation step S540 .

More specifically, in the content collection step S510, the content acquisition module 251 of the topic model unit 250 crawls external content from the content server 300 to collect data, and in a topic parameter adjustment step S520 ) Will set the intention information and key words to be carefully recognized in order to generate the topic model by the topic parameter adjustment module 252 of the topic model unit 250. [

The topic natural language processing step S530 is a step in which the topic natural language processing module 253 of the topic model unit 250 transmits content including the intention information and key words set by the topic parameter adjustment module 252, And processed in a natural language.

Then, the topic model generation step (S540) models the topic information including the intention information and key word set by the user by referring to the natural language processing result by the topic model generation module 254 of the topic model unit 250, And provides the topic model information to the topic matching degree determination unit 260. [

11 is a flowchart of a topic matching degree determination step of an automatic directional selection speech recognition method using an important agent selection method according to the first embodiment of the present invention.

As shown in FIG. 11, the topic matching degree determination step S600 includes a step S610 of obtaining the degree-of-matching-of-channel determination information S610, a step S620 of determining a topic matching degree, and a step S630 of outputting a topic matching result.

More specifically, the step of obtaining the per-channel match degree determination information S610 is performed by the per-channel match degree information acquisition module 261 of the topic match degree determining unit 260, And the modeled topic model information to provide a topic matching degree determination module.

The topic matching degree determination step S620 is a step for determining whether the topic matching degree is determined by the topic matching degree determining module 262 of the topic matching degree determining unit 260 and the voice recognition summary information for each channel provided by the channel matching degree determination information acquiring module 261, And calculates a probabilistic distance value between the topic model information and a channel in which the computed probabilistic distance value provides a topic matching degree equal to or higher than a reference value. The topic matching result output step (S630) The topic matching degree result output module 263 acquires a channel providing the topic matching degree higher than the reference value in the topic matching degree judging module 262 and provides microphone information of the corresponding channel to the voice input sensitivity enhancing adjuster 270.

For example, when the reference value is set to 0.5 and the speech recognition summary information provided from the first multi-channel content analyzer 240a is modeled with '# 01-POINT-election / Lee Myung-bak / Moon Jae-in / Park Geun-hye / The probabilistic distance value between the model information items is 0.7, and the probabilistic distance value between the speech recognition summary information '# 02POINT-food / winning / coupon' provided from the second multi-channel content analysis unit 240b and the modeled topic model information 0.4, and if the probabilistic distance value between the speech recognition summary information '# 03POINT-Academic / English / Score / Mathematics' provided from the third multi-channel content analyzer 240c and the modeled topic model information is 0.1, And provides the first channel microphone 100a providing the sound source to the first multi-channel content analyzer 240a having the above-mentioned topic agreement.

Then, the topic match degree result output module 263 obtains a channel providing a topic match degree higher than a reference value in the topic match degree determination module and provides the microphone information of the corresponding channel to the speech input sensitivity enhancement adjuster 270. The first channel microphone 100a providing the sound source to the first multi-channel content analyzer 240a having a reference level equal to or higher than 0.5, which is a reference value, is provided to the speech input sensitivity enhancement adjuster 270.

According to the present invention as described above, the sound input from the multi-channel is separated from the sound source, the speech recognition is performed for each separated sound, and the contents of the recognition result for each channel are analyzed to obtain a channel microphone It is possible to reproduce the cocktail party effect by securing high recognition performance by automatically selecting and increasing the sensitivity of the corresponding channel microphone.

In addition, by providing a topic model unit, external content is crawled from an external content server, and analysis and learning of external contents are periodically performed by using a deep learning model to continuously perform topic modeling and updating, thereby improving the accuracy of topic modeling So as to provide intensive sensitivity enhancement accuracy for channels with high topic similarity.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It should be understood that various modifications may be made by those skilled in the art without departing from the spirit and scope of the present invention.

100: Channel microphone
200: Select the important direction Automatic direction selection Speech recognition device
300: Content server

Claims (10)

An automatic directional speech recognition system using important topic selection,
And a channel microphone control unit 110 for performing at least one of a microphone direction control, a microphone distance movement control, and a microphone sensitivity adjustment control. The at least one channel microphone 100 )Wow,

A multi-channel sound source input unit 210 for receiving a plurality of sound sources from the plurality of channel microphones and providing the multi-channel sound source separation unit to the multi-channel sound source separation unit;
A multi-channel sound source separation unit 220 for separating a plurality of sound sources provided from the multi-channel sound source input unit for each channel and providing a plurality of separated sound sources to the multi-channel sound recognition unit;
A multi-channel speech recognition unit (230) for recognizing a plurality of sound sources provided from the multi-channel sound source separation unit for each channel;
Acquires speech recognition information for each channel from the multi-channel speech recognition unit, performs natural language processing for each channel, performs content analysis for each channel, generates speech recognition summary information for each channel by referring to the performed content analysis, A multi-channel content analyzing unit 240 for providing the topic matching degree to the judging unit;
A topic modeling unit 250 for modeling topic information to be carefully recognized by crawling external contents from the content server 300 and providing the modeled topic model information to the topic matching degree determining unit;
The method comprising: calculating a probabilistic distance value between the speech recognition summary information for each channel and the modeled topic model information provided by the multi-channel content analyzing unit, determining a channel providing a topic match degree with a probabilistic distance value equal to or greater than a reference value, To the speech input sensitivity enhancement adjusting unit (260);
And an audio input sensitivity enhancement adjusting unit (270) for increasing the audio input sensitivity of the channel microphone when the channel microphone information is obtained from the topic match degree determining unit (200). Wow,

And a content server (300) for periodically providing the external content so as to generate topic model information in the automatic direction selection speech recognition apparatus for sorting important information,

The multi-channel content analyzing unit 240,
A channel-specific natural language processing module 241 for acquiring speech recognition information for each channel from the multi-channel speech recognition unit and performing natural language processing for each channel;
Extracting intention information and key words of the utterance for each channel with reference to the natural language processing result processed for each channel by the natural language processing module for each channel and providing the speech recognition summary information to the speech recognition summary information generation module, (242);
The speech recognition summary information for each channel is generated by referring to the per-channel intention information and the keyword intention information and key words provided for each channel provided by the key word extraction module, and provides the speech recognition summary information to the topic consistency determination unit 260 And a generating module (243) for selecting an important direction.
The method according to claim 1,
The topic modeling unit 250,
A content collection module 251 for crawling external content from the content server 300 and collecting data;
A topic parameter adjustment module 252 for setting intention information and key words to be carefully recognized to generate a topic model;
A topic natural language processing module (253) for acquiring contents containing intention information and key words set by the topic parameter adjustment module by the content acquisition module and performing natural language processing;
A topic model generation module 254 for modeling the topic information including the intention information and key words set by the user with reference to the natural language processing result and providing the modeled topic model information to the topic match degree determination unit 260 Automatic Speech Recognition System using Automatic Speech Recognition. The method according to claim 1,
The topic match degree determining unit 260,
A channel-based match degree determination information acquisition module 261 for acquiring speech recognition summary information and modeled topic model information for each channel provided by the multi-channel content analysis unit and providing the obtained speech recognition summary information to the topic match degree determination module;
Calculates a probabilistic distance value between the speech recognition summary information for each channel and the modeled topic model information provided by the per-channel match degree information acquisition module, and determines a channel for which the calculated probabilistic distance value is greater than or equal to a reference value A topic match degree judging module 262 for judging a match degree;
And a topic agreement degree output module 263 for acquiring a channel for providing a topic match degree higher than a reference value in the topic match degree determination module and providing the microphone information of the corresponding channel to the speech input sensitivity enhancement adjustment unit 270. [ Automatic Directional Speech Recognition System Using Topic Selection.
The method according to claim 1,
The voice input sensitivity enhancement adjusting unit 270 adjusts,
A channel microphone information acquisition module 271 for acquiring channel microphone information from the topic matching degree determination unit and providing corresponding channel microphone information to the voice input sensitivity enhancement adjustment module 272;
In order to increase the voice input sensitivity of the channel microphone when the channel microphone information is acquired from the channel microphone information acquisition module, information on at least one of the microphone direction adjustment information, the microphone distance movement information, and the microphone sensitivity adjustment information is generated, And an audio input sensitivity enhancement adjustment module (272) for transmitting the audio signal to the control unit (110).
The method according to claim 1,
The topic modeling unit 250,
In order to select important topic information desired by the user, a machine learning model of a deep learning model, a machine learning, a support vector machine (SVM), and a neural network is applied to a topic model Wherein the automatic direction selection speech recognition system comprises:
The method according to claim 1,
The topic modeling unit 250,
Crawling external content from the content server 300 and periodically performing analysis and learning of external content using a deep learning model to continuously perform topic modeling and updating to improve the accuracy of topic modeling Automatic Directional Speech Recognition System Using Topic Selection.
delete delete delete delete

Images