KR20200109841A - A speech recognition apparatus - Google Patents

Abstract

According to an embodiment of the present invention, an operation method of a device for automatic speech recognition comprises the steps of: receiving a voice signal of a speaker from a terminal device; obtaining speech data according to analysis of the speech signal; and automatically recognizing and processing the speech data while a speech end timer window is continuing to convert and output text information. The speech end timer window is adaptively variable according to at least one of value information of the speech data detected according to the analysis of the speech signal, classification information of the speech data, and disfluency information of the speech data.

Description

Automatic speech recognition device {A SPEECH RECOGNITION APPARATUS}

The present invention relates to an automatic speech recognition device. More specifically, the present invention relates to a speech recognition apparatus capable of shortening a response time and a waiting time through a speech end time detection process.

The voice signal is the most common, simple, easily and quickly used means or medium for communication between users.

However, there is a problem in that it is difficult to transmit communication over a medium to long distance in a natural state, and thus an automatic speech recognition (ASR) technology has been proposed for automatic recognition processing of such a speech signal.

Such an automatic speech recognition (ASR) technology enables an automated response by detecting a SPEECH section from an AUDIO signal and obtaining and processing context data for communication from the speech section.

In recent years, such ASR technology has been constructed as a separate server-based learning system, and is provided as a service of interpreting natural language according to user voice input and outputting similar text data. The learning system may be constructed by, for example, a probability-based proximity text search diagram using a language model based on a deep neural network (DNN), and various databases and training data are used for this.

However, the current ASR system may be useful for accurately extracting context data, but there is a problem in that it is inefficient in detecting an END OF SPEECH (EOS) time point.

More specifically, the current ASR system automatically detects the end of speech (EOS) time within the automated speech recognition process, which is a factor that causes the EOS time to be determined too early or too late.

For example, silence due to the disfluency of the voice may lead, but the current ARS system judges this as the end of speech and returns it as meaningless data, or, conversely, it is not necessary to determine the end of the speech normally. There is a problem of increasing the waiting time.

Due to such a problem, there are problems in that meaningful text data cannot be extracted normally or that user convenience is impaired due to an increase in waiting time.

Korean Patent Laid-Open Patent 10-2009-0054642 (Name of invention: voice recognition method and apparatus)

The present invention was devised to solve the above-described problems, and independently of the ASR system, by adaptively determining the end of speech (EOS) time according to the characteristics of speech data according to the analysis of the speech signal. It is an object of the present invention to provide an automatic speech recognition apparatus and a method of operating the same, which enables rapid and accurate speech termination to be detected.

An automatic speech recognition apparatus according to an embodiment of the present invention for solving the above-described problems includes: a voice receiver configured to receive a voice signal of a talker from a terminal device; An audio analysis module for acquiring speech data according to the analysis of the speech signal; An automatic speech recognition processing unit for automatically recognizing speech data while the speech end timer window is sustained, and converting and outputting text information; And adaptively varying the speech end timer window according to at least one of value information of the speech data detected according to the analysis of the speech signal, classification information of the speech data, and disfluency information of the speech data. It includes an EOS matching unit that determines a speech end point.

Meanwhile, a method of operating an automatic speech recognition apparatus according to an embodiment of the present invention for solving the above-described problems includes the steps of: receiving a speech signal of a talker from a terminal device; Obtaining speech data according to the analysis of the speech signal; And automatically recognizing and processing speech data while the speech end timer window is continuing, and converting and outputting the text information, wherein the speech end timer window includes value information of the speech data detected according to the analysis of the speech signal. And adaptively variable according to at least one of classification information of the speech data and disfluency information of the speech data.

Meanwhile, the method according to an embodiment of the present invention for solving the above-described problems may be implemented with a program for executing the method on a computer and a computer-readable recording medium in which the program is recorded.

According to an embodiment of the present invention, by adaptively variable determination of the speech end timer window according to the characteristics of speech data according to analysis of a speech signal, it is possible to detect a more rapid and accurate speech end (EOS) time point. .

In addition, the characteristics of the speech data are determined according to at least one of the value information of the speech data detected according to the analysis of the speech signal, the classification information of the speech data, and the disfluency information of the speech data. While maintaining the accuracy of, the waiting time for a voice recognition response can be shortened, and accordingly, user convenience can be greatly improved.

1 is a block diagram schematically showing an entire system according to an embodiment of the present invention.
2 and 6 are diagrams for explaining speech recognition and waiting time reduction effects according to an embodiment of the present invention according to a section of voice data.
7 to 9 are flowcharts illustrating a speech recognition method according to an embodiment of the present invention in more detail for each process.
10 is a ladder diagram for explaining a difference in waiting time reduction between a speech recognition process and an existing process according to an embodiment of the present invention.

The following content merely illustrates the principles of the present invention. Therefore, those skilled in the art can implement the principles of the present invention and invent various devices included in the concept and scope of the present invention, although not clearly described or illustrated herein. In addition, it is understood that all conditional terms and examples listed in this specification are, in principle, expressly intended only for the purpose of making the concept of the present invention understood, and are not limited to the embodiments and states specifically listed as such. Should be.

In addition, it is to be understood that all detailed descriptions listing specific embodiments as well as principles, aspects and embodiments of the present invention are intended to include structural and functional equivalents of these matters. It should also be understood that these equivalents include not only currently known equivalents, but also equivalents to be developed in the future, that is, all devices invented to perform the same function regardless of structure.

Thus, for example, the block diagrams herein are to be understood as representing a conceptual perspective of exemplary circuits embodying the principles of the invention. Similarly, all flowcharts, state transition diagrams, pseudocodes, etc. are understood to represent various processes performed by a computer or processor, whether or not the computer or processor is clearly depicted and that can be represented substantially in a computer-readable medium. Should be.

In addition, the explicit use of terms presented as processor, control, or similar concepts should not be interpreted exclusively by referring to hardware capable of executing software, and without limitation, digital signal processor (DSP) hardware, ROM for storing software. It should be understood to implicitly include (ROM), RAM, and non-volatile memory. Other commonly used hardware may also be included.

The above-described objects, features, and advantages will become more apparent through the following detailed description in connection with the accompanying drawings, whereby those of ordinary skill in the technical field to which the present invention pertains can easily implement the technical idea of the present invention. There will be. In addition, in describing the present invention, when it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted.

Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically showing an entire system according to an embodiment of the present invention.

Referring to FIG. 1, a system according to an embodiment of the present invention includes a terminal device 100, an audio analysis module 120, a local index module 130, a matching module 140, and an automatic speech recognition (ASR) system ( 200) may be included.

As shown in FIG. 1, the terminal device 100 may be a user terminal having one or more voice receivers for receiving a spoken voice from a talker and outputting a voice signal, and receiving a voice signal corresponding to the input spoken voice. As a device delivered to the audio analysis module 120, examples such as mobile phones, smart phones, laptop computers, terminals for digital broadcasting, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), navigation, etc. It may be a variety of devices capable of user input and information display.

In addition, the automatic speech recognition apparatus according to an embodiment of the present invention includes an audio analysis module 120, a local index module 130, a local database 110, and a matching module 140, which are locally connected to the terminal device 100. It may be configured as a local recognition system including.

Here, the local recognition system may be provided separately from the automatic speech recognition system 200 provided at a remote location, and audio or text information that can be matched with a voice signal is obtained through the local database 110 and transferred to the terminal device 100. Provides or is connected to the automatic speech recognition system 200 by wire or wirelessly through a network, and provides speech voice data extracted from the voice signal to the automatic speech recognition system 200 and recognizes from the automatic speech recognition system 200 The result data may be provided to the terminal device 100.

In particular, by the local recognition system according to an embodiment of the present invention, audio and text information that can be matched according to local audio analysis and conversion may be obtained and preferentially provided to the terminal device 100, and furthermore, end of speech (EOS). Since the forced EOS processing for processing may be performed, a delay in response processing of the terminal device 100 may be reduced and user convenience may be improved.

Meanwhile, speech speech data that is not matched in the local recognition system may be transferred to the automatic speech recognition system 200, and the automatic speech recognition system 200 performs a normal speech recognition process based on learning data on the speech data. , Speech recognition data according to the recognition result may be transmitted to the terminal device 100. The speech recognition data may include text-based context data, for example.

The terminal device 100 receives matched audio/text information from the matching module 140 or the automatic speech recognition system 200 or speech recognition data of the automatic speech recognition system 200, and processes a response corresponding to the received information. You can do it.

For example, the terminal device 100 identifies instruction information corresponding to a user's voice input from matched audio/text information or speech recognition data, processes a user interface output corresponding to the instruction information, or processes other users It can transmit a message to the phone or process a telephone connection, or process a response voice output.

Also, for example, the terminal device 100 may determine a query requested from a talker based on word information of the speech data, and provide a corresponding process.

In addition, for example, based on the call record information recorded in the local database 110, the terminal device 100 may provide initial guide information for the speech data-based conversation service to the talker when there is no existing usage history. May be.

As described above, the local recognition system connected to the terminal device 100 may be connected to the automatic speech recognition system 200 by wire or wirelessly through a network. For communication between networks, each terminal device 100 and modules of the local recognition system and the automatic speech recognition system 200 include Internet network, LAN, WAN, PSTN (Public Switched Telephone Network), PSDN (Public Switched Data Network). , Cable TV network, WIFI, mobile communication network and other wireless communication networks, etc. can transmit and receive data. In addition, each terminal device 100 and the automatic speech recognition system 200 may include respective communication modules for communicating with a protocol corresponding to each communication network.

In addition, the local database 110, the audio analysis module 120, the local index module 130, and the matching module 140 of the local recognition system according to an embodiment of the present invention described herein are inside the terminal device 100 It is provided as an internal module located at, or can be implemented as an external module that is locally connected to the terminal device 100 by an external interface. Or, it may be configured as a local recognition system connected externally, or may be configured as a whole system including the automatic speech recognition system 200, but is not limited to the configuration form.

More specifically, the audio analysis module 120 of the automatic speech recognition apparatus according to an embodiment of the present invention, when a voice signal is output from the spoken voice received by the voice receiving unit of the terminal device 100, analyzes the voice signal. According to the speech data can be obtained.

Here, the speech data may include speech data classified as speech speech by a local classification process of speech signals.

The audio analysis module 120 may first request local conversion of voice data to the index module 130, and the index module 130 uses the local database 110 to obtain audio or text information matching the voice data. Indexing may be performed, and the index result may be transmitted to the matching module 140.

In addition, the matching module 140 may provide the matched audio/text to the terminal device 100 as a response result corresponding to the voice signal when the matched audio/text information exists according to the index result of the voice data. have.

Meanwhile, when the matched audio/text information does not exist as a result of the index, the matching module 140 may transmit the voice data classified as the speech voice to the automatic speech recognition system 200.

Accordingly, the automatic speech recognition system 200 may provide automatic speech recognition data that has been subjected to an ordinary automated speech recognition process to the terminal device 100.

Meanwhile, speech data acquired by the audio analysis module 120 according to the analysis of a speech signal may include speech analysis information. Here, the speech analysis information includes at least one of validity (Value) information of the speech data detected according to the analysis of the speech signal, classification information of the speech data, and non-disfluency information of the speech data. May be included, and each analysis information will be described in detail later.

Accordingly, the audio analysis module 120 can perform a variety of analysis processing of the speech information according to the speech speech classification and the speech signal for generating the speech analysis information. In particular, speech according to an embodiment of the present invention The data may be used in the process of determining the end of speech (EOS) timer window in the matching module 140.

Here, the EOS timer window according to an embodiment of the present invention may indicate time interval information set and managed by the matching module 140 in order to detect the end of local speech independently from the automatic speech recognition system 200 at a remote location. have.

Accordingly, the matching module 140 automatically recognizes the speech data during the duration of the speech end timer window through the local index module 130 or the automatic speech recognition system 200 to process a response to the terminal device 100. However, when the speech end timer window has expired, forced EOS information is generated and provided to the terminal device 100, thereby enabling local speech termination independent of the automatic speech recognition system 200.

In particular, the matching module 140 is the speech end timer window according to at least one of validity information of the speech data, classification information of the speech data, and disfluency of the speech data detected according to the analysis of the speech signal. The speech end point is adaptively varied to determine the speech end point, which can bring about a quick and effective response delay reduction effect compared to the existing system using only the automatic speech recognition system 200.

In addition, the local database 110 may store a probability model based on previous query and response information of the terminal device 100, and the index module 130 may provide response information matching the text information extracted from the audio analysis module 120. May be indexed and transmitted to the matching module 140.

Here, such a local database 110 accumulates input voice data and response command data, and updates the probability model to further increase accuracy.

In addition, the matching module 140 may provide response information when matching is confirmed to the terminal device 100, so that the voice inquired by the user is not transmitted to the remote automatic speech recognition system 200 at an early stage. Response results can be provided, and EOS according to response result matching can be determined early.

Accordingly, according to an embodiment of the present invention, it is possible to process a command much faster than an indexing process using only the automatic speech recognition system 200 at a remote location.

2 to 6 are diagrams for explaining speech recognition and waiting time reduction processes according to an exemplary embodiment of the present invention according to a section of voice data.

For example, since it is not easy to process a language model only with hardware specifications of a typical user terminal, an automatic speech recognition system provided as a remote server is used. This general automatic speech recognition system provides an intermediate result corresponding to the speech voice data according to the request of the terminal, but the speech recognition system determines the time of the end of speech (EOS) call, and provides final sentence information according to the EOS call.

A general automatic speech recognition system is set to call EOS when a predetermined silence time frame is identified, and the automatic speech recognition system can interpret the voice data until the EOS is called as text data. A delay time due to the processing time of the automatic speech recognition system for determining EOS is inevitably generated, and user inconvenience may be increased. In addition, there is also the possibility that the wrong EOS will be called because the voice can be temporarily suspended for any reason while the user is speaking the command.

For example, referring to FIG. 2, the total waiting time according to the user experience of a typical automatic speech recognition system may be expressed by an equation based on a log linear model.

Here, (μ) may be a response variable that is network delay (13), (α) may be an independent variable that is automatic speech recognition (ASR) processing time (12), and (β) is ASR processing for EOS detection It may be a blocking variable that is time (11), and (e) may indicate an error rate indicating a result that does not match what the user predicted. In addition, (X) may represent the speech length 14 determined by the EOS call time point. For example, if the utterance is repeated due to unexpected results, the overall X may increase.

In addition, in such a typical automatic speech recognition system, when silence of an audio input is detected, an EOS time-out may be started. For example, if audio is detected before the EOS time limit (100 seconds, section 11 in Fig. 2), the time limit is reset. If audio is not detected within the EOS time limit period 11, EOS may be called. Thereafter, a time point 15 is formed when the result of processing the audio by the automatic speech recognition system is provided to the terminal.

3 shows a method of confirming voice input data in a conventional automatic speech recognition system, and the voice data may include bursts and silence patterns 21, 28, 22, 29, and may include a plurality of time frames. It can be configured as continuous data. In addition, in a typical automatic speech recognition system, the end of speech (EOS) time point may be called at a time point 27 after the plurality of time frames have ended for complete sentence construction.

However, the matching module 140 according to an embodiment of the present invention can quickly call the initial EOS in the middle silent period 31 according to the analysis process of the audio analysis module 120, and thus the user experience waiting time This can be shortened. In addition, the matching module 140 may compensate for an analysis error due to initial EOS detection by providing the index result of the local database 110 of the index module 130. To this end, the local recognition system may be provided as a module residing inside the terminal device 100, and the local database 110 is a cache database or a local TTS/STT conversion engine that stores driving history information of the terminal device 100 May contain databases.

In this way, when the initial EOS is quickly called by the local process, the waiting time felt by the user can be greatly reduced due to the reduction in delay, and an ideal effect based on the above-described Equation 1 model can be derived.

However, since initial EOS detection may cause a potential misinterpretation, it may take an additional waiting time due to the material. Therefore, the local database 110 according to an embodiment of the present invention is used to index the results desired by the user in advance. , Previous query information (QUERY) of the terminal device 100 may be recorded, and response information corresponding to frequently asked queries may be cached and stored in the local database 110 in advance.

Accordingly, the local database 110 may build a probability model to reduce the possibility of an incorrect response, and may provide indexed response information to the matching module 140 through the indexing module 130.

Meanwhile, the audio analysis module 120 may process filtering of non-context data through noise analysis and voice activity detection in order to compensate for a potential error that may cause an unwanted initial EOS by misinterpreting noise. Accordingly, voice data not classified as speech data may be blocked before being transmitted to the index module 130 or the automatic speech recognition system 200. By this local filtering process, unnecessary data transmission to the automatic speech recognition system 200 can be reduced and resource usage can be optimized.

For this processing, the audio analysis module 120 first processes the noise level monitoring of the input voice data to initially identify the noise section and the voice section, and detects a burst pattern of the voice section or continuously By detecting the noise level data, only speech audio data can be classified and extracted from the audio signal.

Further, the audio analysis module 120 may classify and extract only speech data that can be regarded as human speech by setting a range of speech variables in the frequency domain.

Meanwhile, FIGS. 4 and 5 show waveforms of voice data when a non-fluency section is included.

In the speech recognition system according to an embodiment of the present invention, the matching module 140 checks the existence of a non-fluency section so that there is no additional delay in returning meaningless data or repeating the voice command again. And, when non-fluency is confirmed, a process of increasing the importance weight of the next word to be recognized may be performed.

In particular, in the case of Korean, the format of "subject-object-verb" is followed, and the subject is completely omitted. In this case, the non-fluency information can be used as ideal information for detecting initial EOS while providing appropriate text and context recognition. have. In the data collection and test processing for 3 months based on the system according to an embodiment of the present invention, it has been confirmed that more than 90% were in the form of "subject-object-verb" or "object-verb".

More specifically, referring to FIG. 4, in the case of a conventional automatic speech recognition system, when the non-fluency sections 400 and 402 such as'ah','he', and silent stuttering are repeated, an interpretation lacking in relation to the situation is performed. Will be calculated. In addition, it is possible to call the wrong EOS in the silent section (401, 403) according to the preset size of the time frame. In addition, there are cases in which the automatic speech recognition system does not allow additional input while incorrect input or EOS is processed and the final result is returned. Accordingly, the user repeats the desired utterance, and there is a problem that the overall waiting time is increased.

To solve this, the matching module 140 according to an embodiment of the present invention may locally set an EOS time window for EOS detection, and may change a corresponding window size. In particular, the EOS time window when detecting a non-fluency section It is possible to increase the recognition weight of the next word while expanding. For example, the matching module 140 according to an embodiment of the present invention may extend the EOS time window in the silent periods 401 and 403 due to the non-fluency speech in FIG. 4, and thus, EOS detection Can be disabled.

In addition, the matching module 140 may reset the end time of the EOS time window to the time point at which the voice data of consecutive frames ends, and thus the EOS call extension time point 408 may be reset.

In addition, FIG. 5 shows the reduction in time delay due to the dynamic size of the EOS time window. When the voice data includes burst data according to influency, the EOS time window is dynamically adjusted according to the speech length. Change can be handled.

Referring to FIG. 5, in the case of general EOS detection, a fixed length time frame period 410 after the end of the speech must pass, but in the case of the matching module 140 according to an embodiment of the present invention, voice data including influency With respect to, by processing the dynamic reduction change 411 of the EOS time window corresponding to the speech length, the user can receive the effect of reducing the waiting time 412 by the difference.

Meanwhile, FIG. 6 is a diagram for explaining a reduction in processing delay time of voice data including disfluency compared to the prior art according to an embodiment of the present invention.

As shown in Fig. 6A, in the prior art, when the speech sections 500 and 501 and the silent section 503 including influency in the talker's voice are generated, the speech recognition system generates EOS due to the silent section. The section 515 is detected, and after the recognition processing time 505 elapses, an incorrect context recognition result 506 is provided.

As a result, the user does not get a response to the actual query voice 502 and has to speak again (504). Accordingly, additional EOS section 516 detection and recognition time 514 are required, resulting in an additional delay time ( After 517) is taken, the response result is provided.

On the other hand, referring to FIG. 7(B) according to the process according to an embodiment of the present invention, even if a voice input including the non-fluency sections 508 and 509 is received from the talker, the matching module 140 sets the EOS time window. It is set locally, but it is possible to wait until the actual query voice 510 is received by extending the time window end time according to detection of influency.

In addition, after the early EOS due to the EOS window expiration 511 is detected, and only the corresponding processing time 512 is appropriately spent, the recognition result may be provided to the user.

In this way, it can be seen that the difference in waiting time 513 compared to the prior art can be clearly shortened, and this can be a major factor in improving user convenience.

7 to 9 are flowcharts illustrating a speech recognition method according to an embodiment of the present invention in more detail for each process.

7 is a flowchart illustrating a process for recognizing local speech as a whole according to an embodiment of the present invention.

Referring to FIG. 7, first, the audio analysis module 120 processes noise analysis from a voice signal received from the terminal device 100 (S101).

Then, the audio analysis module 120 detects voice activity information according to noise analysis (S103).

Thereafter, the audio analysis module 120 checks speech data continuity (S105).

Accordingly, the audio analysis module 120 determines whether speech data of the speech signal can be classified according to the comprehensive analysis result (S107), and if the classification is possible, transmits the speech signal to the index module 130 to be used in the local automatic speech recognition process. Enter (S109).

Here, the matching module 140 may set a local EOS time window for the automatic speech recognition process.

In addition, if it is determined in step S107 that it is not possible to classify speech data according to noise detection, the audio analysis module 120 may perform noise analysis on the speech signal of the next time frame.

In the local automatic speech recognition process, the index module 130 performs speech data pre-categorization (S111). The pre-classification may include classification processing of matching speech data to preset classification values.

In addition, the matching module 140 determines whether there is a disfluency section in the speech data from the analysis result of the audio analysis module 120 (S113), and determines whether the content validity of the speech data exists ( S115), when there is no influency and validity, the index module 130 performs a data analysis-based matching process of the local database 110 (S117).

Here, the matching process may be based on the history information cached in the local database 110 or may be processed by the TTS and STT engines mounted in the local database 110, and the matching module 140 is By checking the response result, it is possible to check whether there is a match (S119).

On the other hand, if there is an inflexible section in the speech data in step S113, the content validity of the speech data does not exist in step S115, or if matching is not confirmed in step S119, the matching module 140 presets the EOS time window in advance. It extends for the set time period (S125).

Thereafter, the matching module 140 checks whether the time window has expired (S127), calls EOS at the expiration point to determine the end of the sentence according to the matching confirmation (S121), and when output data according to the end of the sentence exists Provided to the terminal device 100, the appropriate data processing corresponding to the sentence response is performed (S123).

Meanwhile, FIG. 8 is a more detailed flowchart illustrating a speech data classification process according to an embodiment of the present invention.

Referring to FIG. 8, the audio analysis module 120 first reads data from a voice buffer (S201).

In addition, the audio analysis module 120 detects voice activity information for each data frame (S203), determines the activity information for each frame (S205), and determines continuous activity accordingly (S207), and continuous activity information By determining the existence of the continuous feature corresponding to the speech pattern from (S209), it is possible to determine the possibility of classification as speech data (S211).

According to the classification possibility, the audio analysis module 120 classifies as speech data if it is possible to classify (S213), and performs data ignoring processing if it is impossible to classify it (S215).

According to the processing of the audio analysis module 120, it is possible to minimize the effect of EOS calls due to incorrect audio data before entering the local automatic speech recognition process for voice data or transferring it to the automatic speech recognition system 200. have. To this end, the audio analysis module 120 can check the voice activity information of the data read from the voice buffer, detects the voice activity based on a fixed time frame, and detects continuous activity for setting the expected value of the next frame configuration. You can judge whether or not.

For example, the audio analysis module 120 may predict the possibility of the existence of a voice in the next frame according to the detection of voice activity in the current frame according to a continuous waveform, and in the case of speech data, a silent section tends to exist between the voice data. Therefore, according to the time frame pattern analysis of the corresponding voice activity, whether the voice data is noise or includes speech data can be detected.

More specifically, the audio analysis module 120 detects a silence pattern and burst speech data corresponding to the speech prediction data from the speech signal, and calculates the speech prediction data according to a silence-to-sound level ratio calculation of the speech prediction data. It can be classified as the speech data.

In addition, the audio analysis module 120 determines a noise level according to continuous detection of the continuous level of the speech signal, and according to the ratio of the non-silent section and the non-noise section to the voice signal sample, the content validity of the speech data (Valuable ) May be determined, and the matching module 140 may determine whether to perform data analysis-based matching according to the content validity analysis result.

Meanwhile, FIG. 9 is a flowchart illustrating a speech recognition process based on the presence or absence of non-fluency according to an embodiment of the present invention.

9, when voice data is received by the audio analysis module 120 (S301) and speech data is classified (S303), the matching module 140 sets a speech end (EOS) timer window, and accordingly The EOS timer is started (S305).

Then, the matching module 140 acquires text information corresponding to the speech data through the cache data of the local database 110 or the STT/TTS engine (S307).

Here, the matching module 140 determines the presence of disfluency of speech data according to the analysis of the audio analysis module 120 (S309), and if it exists, extends the EOS timer (S311), and The importance weight of the corresponding word is increased (S313).

On the other hand, when influency is not confirmed, the matching module 140 performs response matching analysis based on the importance weight of the text information through the index module 130 (S315), and determines whether unnecessary words are predicted ( S317), if it is not an unnecessary word, it is determined whether the EOS timer has ended before the additional audio (S319), an output corresponding to the response result is returned, and the command data processing corresponding thereto is processed by the terminal device 100 ( S321).

If unnecessary words are predicted in step S317 or the EOS timer has not expired in step S319, the audio analysis module 120 receives and processes the next voice data again.

In addition, the matching module 140 may store and manage a list of non-fluency words and predicted keyword weight information for obtaining the disfluency information in the local database 110.

Accordingly, the matching module 140 sets the speech end timer window variable value corresponding to the non-fluency information based on pre-stored information, and the degree of variability according to the non-fluency information is based on the predicted keyword weight information. It can be set to be determined.

10 is a ladder diagram for explaining a difference in waiting time reduction between a speech recognition process and an existing process according to an embodiment of the present invention.

Referring to FIG. 10A, it can be seen that the terminal device of the prior art transmits the end speech data according to the end of speech, and the speech response time is delayed according to the EOS detection of the ASR system.

On the other hand, referring to FIG. 10(B), the terminal device 100 according to an embodiment of the present invention enables forced EOS processing by the local recognition system even when the same speech end point arrives. Processing of the final recognition result by varying the time window is performed early, so that a speech response can be provided to the talker faster than in the prior art.

Meanwhile, the above-described method according to various embodiments of the present invention may be implemented as a program and provided to each server or devices while being stored in various non-transitory computer readable media. Accordingly, the user terminal 100 can access the server or device and download the program.

The non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, not a medium that stores data for a short moment, such as a register, cache, or memory. Specifically, the above-described various applications or programs may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, and ROM.

In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

Claims (8)

In the automatic speech recognition device,
Receives a talker's voice signal from the user terminal, obtains speech data by analyzing the voice signal, and provides response information corresponding to the speech data by local voice automatic recognition while the speech data for the duration of the speech end timer window is maintained. do or,
And a local speech recognition system configured to transmit speech data that does not match in local speech recognition to a remote automatic speech recognition system to provide response information corresponding to the speech data to the user terminal device;
Automatic speech recognition device. The method of claim 1,
The speech end timer window,
Characterized in that it is varied by a matching module driven by a separate local process that is distinguished from the remote automatic speech recognition system
Automatic speech recognition device. The method of claim 1,
The local speech recognition system,
An audio analysis module for receiving a speech signal of a talker and obtaining speech data including speech data classified as speech speech by a local classification process of the speech signal according to the analysis of the speech signal;
A local index module for receiving voice data from the audio analysis module, searching for audio or text information matching speech data using a local database, and outputting an index result;
When there is audio/text information matched according to the index result of the voice data from the local index module, the matched audio/text is provided to the terminal device as an answer result corresponding to the voice signal, or the matched audio result is the index result. / If there is no text information, a matching module for transmitting the speech data classified as the speech data to the remote automatic speech recognition system; Containing
Automatic speech recognition device. The method of claim 3,
Speech data obtained according to the analysis of the speech signal by the audio analysis module includes speech analysis information,
The speech analysis information includes at least one of validity (Value) information of the speech data detected according to the speech signal analysis, classification information of the speech data (Categrize) information, and non-fluency information of the speech data.
Automatic speech recognition device. According to claim 3
The matching module adaptively adjusts the speech end timer window according to at least one of validity information of the speech data, classification information of the speech data, and disfluency of the speech data detected according to the analysis of the speech signal. Variable to determine the speech endpoint
Automatic speech recognition device. According to claim 1
The local speech recognition system automatically recognizes the speech data while the speech end timer window is sustained through a local speech recognition or an automatic speech recognition system at a remote location and processes the response by the terminal device, but when the speech end timer window expires EOS information is generated and provided to the terminal device to enable local speech termination independent of the remote automatic speech recognition system.
Automatic speech recognition device. The method of claim 3,
The matching module,
Extending the end time of the speech end timer window when there is an influency section in the speech data according to the disfluency information
Automatic speech recognition device. The method of claim 7,
The matching module increases the importance weight for recognizing the next word when there is an influency section in the speech data.
Automatic speech recognition device.

Images