KR20220141554A - Speech end point detection apparatus, program and control method thereof - Google Patents

Abstract

The present invention relates to a technology for accurately recognizing an utterance end point of a speaker in speech recognition. More specifically, the present invention relates to the technology that detects a phrase included in an audio signal inputted in time series based on a speech recognition model, enables a silent syllable frame to be detected from among a plurality of frames included in the audio signal based on the phrase detection result, enables a silence syllable count value for which is the number of consecutive silence syllable frames to be calculated, enables to be determined as an end point when the silence syllable count value exceeds a first value, and enables an initialization part to initialize the speech recognition model when the silence syllable count value exceeds a second value.

Description

Voice endpoint detection device, program, and control method thereof

The present invention relates to an apparatus for recognizing an endpoint of a voice, a program, and a control method thereof, and more particularly, to an endpoint detection device for more accurately recognizing an endpoint of a long voice input using an end-to-end voice recognition technology; It relates to a program and a method for controlling the same.

Speech recognition technology is a technology in which a computer interprets a speech language spoken by humans for communication and converts it into text data, and is rapidly developing according to the needs of users who want convenience.

It can receive a voice signal, which is an analog signal, through a microphone, and convert the voice part into computer-readable text using an Automatic Speech Recognition (ASR) model. The speech recognition model may be an artificial intelligence model. The artificial intelligence model models the characteristics of human biological nerve cells by mathematical expressions, and uses an algorithm that mimics the ability of learning that humans have.

The AI model has the ability to generalize based on the learned results to generate relatively correct outputs for input patterns that have not been used for learning. Due to the advantages of such generalization ability, neural networks are widely used in the field of speech recognition technology.

The detection of the end point of the voice refers to a step of detecting the end point of the voice utterance as a pre-stage of voice recognition and operating the recognizer only as much as the user has uttered. Separately from the speech recognition model for recognizing speech, additional resources for endpoint detection are allocated, and a separate additional process for inferring endpoints is made.

This endpoint detection method has a problem in that accuracy is inevitably reduced in a noisy environment because it uses the energetic characteristics of voice.

Accordingly, there is a need for research on a technology for inferring a voice endpoint based on an end-to-end voice recognition technology using energy features and linguistic context for accurate endpoint detection even in a noisy situation.

SUMMARY OF THE INVENTION An object of the present invention is to provide an apparatus, method, and program for detecting an end point of a voice utterance with high accuracy by using both the energy feature of the voice and the linguistic feature.

Another object to be solved by the present invention is to provide an apparatus, method, and program for detecting an endpoint of a voice utterance without additional resource utilization.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the description below. will be able

According to one aspect of the present invention in order to solve the above or other problems, in the apparatus for detecting an endpoint of a voice comprising a memory for storing instructions and a processor configured to execute the stored instructions, the processor is inputted audio in time series detecting a phrase included in the signal; detecting, by the processor, a silent frame from among a plurality of frames included in the audio signal, based on a result of the phrase detection; calculating, by the processor, a silence count value that is the number of consecutive silence frames; determining, by the processor, as an endpoint when the silence count value exceeds a first value; and initializing, by the processor, the voice recognition model when the silence count value exceeds a second value.

The method may further include, by the processor, initializing the calculated silence count value when the phrase is detected.

In the detecting of the silent frame, the processor may detect a frame that does not correspond to the detected phrase among a plurality of frames included in the audio signal as a silent frame.

The speech recognition model may be a speech recognition model based on a recurrent neural network (RNN) in which past recognition results are reflected in current speech recognition.

When the silence count value exceeds a second value, the processor may initialize the recurrent neural network.

The initialization of the voice recognition model may include restoring at least one parameter constituting the voice recognition model to an initial state.

According to another aspect of the present invention to solve the above or other problems, a memory for storing instructions; and a processor configured to execute the stored instructions, wherein the processor is configured to: detect a phrase included in an audio signal inputted in time series based on a speech recognition model, and based on a result of the phrase detection, the audio signal A silence frame is detected from among a plurality of frames included in the Provided is an apparatus for detecting an endpoint of a voice that initializes the voice recognition model when a value of 2 is exceeded.

The processor may initialize the calculated silence count value when the phrase is detected.

When detecting the silence frame, the processor may detect a frame that does not correspond to the detected phrase among a plurality of frames included in the audio signal as a silence frame.

The speech recognition model may be a speech recognition model based on a recurrent neural network (RNN) in which past recognition results are reflected in current speech recognition.

The processor may initialize the recurrent neural network when the silence count value exceeds a second value.

The initialization of the voice recognition model may include restoring at least one parameter constituting the voice recognition model to an initial state.

The effects of the endpoint detection apparatus, program, and control method according to the present invention will be described as follows.

According to at least one of the embodiments of the present invention, there is an advantage in that an end point can be detected with higher accuracy than a conventional end point detection apparatus.

In addition, according to at least one of the embodiments of the present invention, there is an advantage that efficient resource utilization is possible because a separate resource for endpoint detection is not required.

Further scope of applicability of the present invention will become apparent from the following detailed description. However, it should be understood that the detailed description and specific embodiments such as preferred embodiments of the present invention are given by way of illustration only, since various changes and modifications within the spirit and scope of the present invention may be clearly understood by those skilled in the art.

1 is a diagram illustrating a block diagram of an endpoint detection apparatus 100 according to an embodiment of the present invention.
2 is a diagram schematically illustrating the configuration of an RNN.
3 is a conceptual diagram for detecting a silent frame according to an embodiment of the present invention.
4 is a diagram illustrating an example of an end-to-end speech recognition model 101' according to an embodiment of the present invention.
5 is a diagram illustrating a control flowchart of the endpoint detection apparatus 100 according to an embodiment of the present invention.
6 is a diagram illustrating a control flowchart of the endpoint detection apparatus 100 according to another embodiment of the present invention.
7 illustrates a control method for determining whether a phrase is detected by considering a plurality of frames together according to an embodiment of the present invention.
8 is a conceptual diagram illustrating a control method for determining whether a phrase is detected by considering a plurality of frames together according to an embodiment of the present invention.
9 is a diagram illustrating a configuration of an endpoint detecting apparatus 100 according to an exemplary embodiment.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "part" for components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention , should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as first, second, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The singular expression includes the plural expression unless the context clearly dictates otherwise.

In the present application, terms such as “comprises” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

1 is a diagram illustrating a block diagram of an endpoint detection apparatus 100 according to an embodiment of the present invention.

According to an embodiment of the present invention, the endpoint detection device 100 is a mobile phone, a cellular phone, a smart phone, a personal computer, a laptop, a notebook computer, a netbook or tablet, a personal digital assistant (PDA), a digital camera, Game console, MP3 player, personal multimedia player (PMP), electronic book (E-Book), navigation, disk player, set-top box, home appliance, communication device, display device, or other electronic device It may be built into or interact with the device. In addition, the endpoint detection device 100 may be embedded in or interact with a smart home appliance, an intelligent vehicle, an autonomous driving device, a smart home environment, a smart building environment, a smart office environment, a smart electronic security system, or the like. In addition, the endpoint detection apparatus 100 may be included in a wearable device worn on the user's body to operate or to interact with it. The wearable device may have the form of, for example, a ring, a watch, glasses, a bracelet, a belt, a band, a necklace, an earring, a helmet, or clothes.

Specifically, the endpoint detection apparatus 100 may be configured to include a voice recognition model 101 , a silent frame detection unit 102 , a count unit 103 , an initialization unit 104 , and an endpoint determination unit 105 . The components shown in FIG. 1 are not essential for implementing the endpoint detection apparatus 100, so the endpoint detection apparatus 100 described herein may include more or fewer components than those listed above. can have

The voice recognition model 101 recognizes a voice included in an audio signal inputted in time series. In addition, the speech recognition model 101 may detect a phrase included in the recognized speech. The speech recognition model 101 may be a speech recognition model based on a recurrent neural network (RNN) in which past recognition results are reflected in current speech recognition.

RNN is a type of artificial neural network having a cyclic structure, and is a model suitable for processing data that is sequentially configured such as voice or text. Hereinafter, the concept of a recurrent artificial neural network will be described with reference to FIG. 2 .

The silent frame detection unit 102 is configured to detect a silent frame among frames included in an audio signal inputted in time series. The silent frame is a frame corresponding to silence and may mean a frame that is irrelevant to a recognized voice. The detection of the silent frame will be described in more detail below with reference to FIG. 4 .

The counting unit 103 is configured to count the number of consecutive silent frames detected by the silent frame detection unit 102 and calculate it as a silence count value.

The initialization unit 104 is configured to initialize the voice recognition model 101 .

The end point determining unit 105 is configured to determine the end point of the utterance based on the silence count value calculated by the counting unit 103 .

2 is a diagram schematically illustrating the configuration of an RNN.

Looking at the basic configuration of the RNN with reference to FIG. 2 , x _t is an input value in time step t, h _t is a 'hidden state' in time step t, and the 'hidden state' value in the previous time step and It is calculated by the input value of the current time step. And y _t is the output value at time step t. That is, according to the artificial neural network as shown in FIG. 2 , past data may affect current output.

For example, when the character string 'apple' is recognized, the character string 'is' is recognized in consideration of past data of 'apple' recognized earlier. When it is unclear whether 'silver' and 'is', it can be recognized as 'is' by considering 'apple', which is the past data.

3 is a conceptual diagram for detecting a silent frame according to an embodiment of the present invention.

Referring to FIG. 3 , the audio signal 300 including the voice of 'apple' is divided in units of frames. The voice recognition model 101 may recognize the audio signal 300 to detect a phrase 310 (a character string) of 'apple'.

The silent frame detection unit 102 according to an embodiment of the present invention proposes to determine a silent frame and a non-silent frame based on the detected phrase 310 . That is, the silent frame detection unit 102 determines that a frame corresponding to the detected phrase 310 is a silent frame 301-1, and a frame that does not correspond to the detected phrase 310 is a silent frame 301-2. ) and can be detected.

The reason why the silent frame detection unit 102 distinguishes the silent frame 301-2 and the non-silent frame 301-1 based on the detected phrase 310 is to more accurately detect an end point in a noisy environment. . It is possible to minimize the error of erroneously judging that no voice is input, but the utterance is not finished due to the surrounding noise environment.

Also in the drawing shown in FIG. 3 , it can be confirmed that the audio waveform is also detected in the silent frame 301 - 2 before the voice 'apple' is detected. These audio waveforms are most likely due to the surrounding noise environment. If it is simply determined that the audio waveform is not silence, the accuracy of determining the end point is inevitably lowered.

The speech recognition model 101 according to an embodiment of the present invention may be an end-to-end Automatic Speech Recognition model.

In general, conventional speech recognition systems include an AM (Acoustic Model) that extracts acoustic features and predicts sub-word units such as phonemes, a PM (Pronunciation Model) that maps phoneme sequences to words, and a method that assigns probabilities to word sequences. Includes LM (Language Model). And in the conventional speech recognition system, AM, PM, and LM are generally independently learned from different data sets. However, in recent years, an end-to-end speech recognition model that combines the components of AM, PM, and LM into a single neural network has been developed.

End-to-end speech recognition refers to a speech recognition method that recognizes a string or word sequence from an input speech through a single integrated neural network. In the conventional conventional speech recognition system described above, a pronunciation dictionary needs to be separately prepared for training, and the acoustic model is trained through several steps to make the correct phoneme of the frame. In addition, weighted finite-state transducers (WFST) are used to combine the acoustic model, the pronunciation dictionary, and the separately trained language model to find the most probable path and finally recognize the speech.

This process is cumbersome and requires prior knowledge of speech recognition to understand the role of each. With the recent development of deep neural network technology, an end-to-end speech recognition model with a simple structure that solves these shortcomings has been proposed.

As a representative end-to-end speech recognition model, there is a connectionist temporal classification (CTC) method. This method uses a recursive neural network to infer strings directly from speech features. Similar to HMM, the character posterior probability is estimated for every frame, and the estimated character string operates to have an optimal path.

As a model different from the CTC method, a speech recognition 'Attention Based Model' based on a sequence-to-sequence (seq2seq) model was proposed. This model consists of an encoder and a decoder that are recursive neural networks, and the encoder calculates an output for every frame from the input speech features. The decoder calculates which frame to pay attention to the encoder output, and uses the encoder value as an input according to the attention level to estimate the final string.

According to the end-to-end speech recognition model, there is an advantage in that the speech recognition process can be simplified in that a separate pronunciation dictionary (pronunciation lexicon) for mapping from phoneme units to words is not required. Hereinafter, the 'Attention Based Model' among the above-described models will be described in more detail with reference to FIG. 4 .

4 is a diagram illustrating an example of an end-to-end speech recognition model 101' according to an embodiment of the present invention. 4 shows an 'Attention Based Model' as an example of the end-to-end speech recognition model, but it is not limited thereto, and it is obvious that various end-to-end speech recognition models can be applied to the embodiment of the present invention.

As shown in FIG. 4 , the end-to-end speech recognition model 101 ′ may include an encoder 401 , an attention unit 402 , and a decoder 403 . In addition, at least one of the encoder 401 and the decoder 403 may be implemented as an RNN.

The encoder 401 receives the user voice x and maps the acoustic features of x to a higher order feature representation h. When the high-dimensional acoustic feature h is transmitted to the attention unit 402, the attention unit 402 determines which part of the acoustic feature x should be considered importantly in order to predict the output y, and an attention context c is transmitted to the decoder 403 . When the attention turntext c is transmitted to the decoder 403, the decoder 403 receives the attention context c and y _i-1 corresponding to the embedding of the previous prediction, generates a probability distribution P, and outputs y as a character (column) predict _i .

According to the end-to-end speech recognition model 101' as described above, a model may be implemented in which a user's voice is an input value and a character (column) corresponding to the user's voice is an output value. And, according to the size of the input data and training of the artificial neural network on the input data, a character (column) corresponding to the user's voice more accurately may be identified.

The configuration shown in FIG. 4 is merely exemplary, and it goes without saying that various types of end-to-end speech recognition models may be applied within the scope that can achieve the object of the present invention.

A method of controlling an endpoint detecting apparatus according to an embodiment of the present invention will be described with reference to the flowcharts of FIGS. 5 and 6 .

5 is a diagram illustrating a control flowchart of the endpoint detection apparatus 100 according to an embodiment of the present invention.

In step S501, the endpoint detection apparatus 100 divides the input audio signal based on a predetermined criterion. In particular, in an embodiment of the present invention, the predetermined criterion may be in units of frames. Hereinafter, a case of dividing the frame unit by frame will be described as an example, but embodiments of the present invention are not necessarily limited thereto. The endpoint detection apparatus 100 inputs the first frame to the voice recognition model 101 ( S502 ).

The voice recognition model 101 analyzes the input frame (S503). In step S504, the endpoint detection apparatus 100 proceeds to step S505 when a silent frame is detected as a result of the analysis in step S503, and proceeds to step S508 when a phrase (or phoneme) is detected.

In step S505, the counting unit 103 may increase the silence count by a unit value (eg, '1') (S505). Conversely, when a phrase (or phoneme) is detected, the count unit 103 determines that the frame is a silent frame and initializes the silence count value ( S508 ).

That is, according to steps S504, S505, and S508, the count unit 103 increments the silence count value whenever a silent frame is detected, and when a silent frame is detected, initializes the silence count value in step S508 to ensure continuous silence frames number of can be calculated.

Referring back to the flowchart of FIG. 5 , the endpoint determining unit 105 determines whether the silence count value exceeds a first value ( S506 ), and when it exceeds the first value, the input frame is used as the endpoint of the utterance. It can be determined (S507).

If it does not exceed the first value, it may proceed to step S510 to receive the next frame.

When a phrase or phoneme is detected in step S504, the counting unit 103 initializes the silence count value (S508) as described above. In addition, the endpoint detection apparatus 100 may output the detected phrase or phoneme to the user ( S509 ). Subsequently, the process proceeds to step S510, and the next frame is input to the voice recognition model 101 (S510), and the above-described steps S503 to S510 may be repeatedly performed.

6 is a diagram illustrating a control flowchart of the endpoint detection apparatus 100 according to another embodiment of the present invention.

In step S601 , the endpoint detecting apparatus 100 divides the input audio signal based on a predetermined criterion. As in the embodiment of FIG. 5 , in the embodiment of FIG. 6 , the case of division in frame units will be described as an example.

The endpoint detection apparatus 100 inputs the first frame to the voice recognition model 101 ( S602 ).

The voice recognition model 101 analyzes the input frame (S603). In step S604 , the endpoint detection apparatus 100 proceeds to step S605 when a silent frame is detected as a result of the analysis in step S603 , and proceeds to step S611 when a phrase (or phoneme) is detected.

In step S605 , the count unit 103 may increase the silence count value by a unit value (eg, '1'). Conversely, when a phrase (or phoneme) is detected, the count unit 103 determines that the frame is a silent frame and initializes the silence count value (S611 and S612).

That is, according to steps S604, S605 and S612, the count unit 103 increments the silence count value whenever a silent frame is detected, and initializes the silence count value in step S611 when a silent frame is detected (phrase detection). The number of consecutive silent frames may be calculated.

In step S606, the endpoint determination unit 105 determines whether the silence count value exceeds a second value.

If the silence count value does not exceed the second value in step S606, the process may proceed directly to step S610. In step S610, the endpoint detection apparatus 100 inputs the next frame to the voice recognition model 101, and returns to step S603.

If the silence count value exceeds the second value in step S606 , the initialization unit 104 initializes the voice recognition model 101 ( S607 ) and proceeds to step S608 .

The initialization of the voice recognition model 101 may mean returning to a state in which the first audio signal is input without considering the previously input audio signal or the recognition result. The initialization of the voice recognition model 101 may include restoring at least one parameter constituting the voice recognition model 101 to an initial state.

Following step S607, the endpoint determining unit 105 determines whether the silence count value exceeds a first value (S608), and if it exceeds the first value, the process proceeds to step S609. In step S609, the end point determination unit 105 may determine the corresponding frame as the end point and end it.

If the silence count value does not exceed the first value in step S608, the process proceeds to step S610. As described above, the endpoint detection apparatus 100 inputs the next frame to the voice recognition model 101 in step S610 and returns to step S603.

When the phrase or phoneme is detected in step S604 , the end point detection apparatus 100 proceeds to step S611 . In step S611, the endpoint detection apparatus 100 outputs a phrase or phoneme detection result. Subsequently, the count unit 103 initializes the silence count ( S612 ). Then, it is possible to return to step S603 through step S610 of inputting the next frame.

Although the control sequences of FIGS. 5 and 6 have been described separately from each other, it will be apparent that the two methods may be combined.

Meanwhile, in the embodiment described above together with FIGS. 5 and 6 , there may exist a case in which it is not possible to determine whether a phrase or phoneme is detected using only a single currently input frame. That is, it means a case in which it is possible to determine whether an accurate word or phoneme is detected only by checking the subsequent input frames. Accordingly, the determination of whether to detect a phrase according to an embodiment of the present invention may be performed in consideration of at least one subsequent frame. This embodiment will be described in more detail with reference to FIG. 7 .

7 illustrates a control method for determining whether to detect a phrase by considering a plurality of frames together according to an embodiment of the present invention.

8 is a conceptual diagram illustrating a control method for determining whether a phrase is detected by considering a plurality of frames together according to an embodiment of the present invention.

Hereinafter, it will be described with reference to FIGS. 7 and 8 together.

In step S701, the endpoint detecting apparatus 100 divides the input audio signal based on a predetermined criterion. As in the embodiments of FIGS. 5 and 7 , in the embodiment of FIG. 7 , the case of division in frame units will be described as an example.

The endpoint detection apparatus 100 inputs the first frame to the voice recognition model 101 ( S702 ).

The voice recognition model 101 analyzes the input frame (S703). In step S704, the endpoint detecting apparatus 100 proceeds to step S709 if the phrase (or phoneme) is not detected in step S703, and proceeds to step S705 if detected.

When the phrase is detected in step S704 , the end point detection apparatus 100 calculates a frame difference between the detected phrase and the immediately preceding detection phrase ( S705 ).

Referring to FIG. 8 , a first phrase 801 of 'delicious' is detected, and a second phrase 802 of 'next' is detected based on the currently input frame. That is, the second phrase 802 may be a phrase detected based on the currently input frame, and the first phrase 801 may be a phrase detected immediately before.

In the first example of calculating the frame difference, the endpoint detection apparatus 100 determines the frame 811 immediately before the second phrase 802 and the frame 810 immediately after the first phrase 801, and By determining the number of frames existing between the frame 810 immediately after the phrase 801 and the frame 811 immediately before the second phrase 802, the frame difference 800 in step S704 can be calculated (S705). .

In the second example of calculating the frame difference, the endpoint detection apparatus 100 includes a first silent frame 801 ′ corresponding to the first phrase 801 and a second silent frame 801 ′ corresponding to the second phrase 802 . Frame 802' can be distinguished. Then, the frame difference 800 of step S704 is calculated through a method of counting the number of silent frames 812 existing between the first silent frame 801' and the second silent frame 802' ( S705) will be possible.

Next, the endpoint determination unit 105 determines whether the difference calculated in step S705 exceeds a first value (S706).

If the difference calculated in step S705 exceeds the first value, the endpoint determination unit 105 may proceed to step S707 to determine the endpoint as an endpoint, and end.

If the difference calculated in step S705 does not exceed the first value, the endpoint detecting apparatus 100 may proceed to step S708 to input the next frame to the voice input model 101 .

If it is determined in step S704 that the phrase is not detected, the process proceeds to step S709.

In step S709, the counting unit 103 increases the silence count by a unit value (eg, '1').

In step S710 , the initialization unit 104 determines whether the silence count value exceeds a second value. If the silence count value exceeds the second value in step S710, the process proceeds to step S711. If not, the process proceeds to step S708 and repeats the process for the next frame (S703 to S711).

In step S711 , the initialization unit 104 initializes the voice recognition model 101 .

The initialization of the voice recognition model 101 may include restoring at least one parameter constituting the voice recognition model 101 to an initial state, as in the above-described embodiment.

9 is a diagram illustrating a configuration of an endpoint detecting apparatus 100 according to an exemplary embodiment.

Referring to FIG. 9 , the endpoint detection apparatus 100 includes a memory 192 and a processor 191 . Memory 192 stores one or more instructions executable by processor 191 . Processor 191 executes one or more instructions stored in memory 192 . The processor 191 may execute one or more operations described above with respect to FIGS. 1-8 by executing instructions.

The memory 192 may include software in which the end-to-end speech recognition model 101' is implemented. In addition, the processor 191 may execute software stored in the memory 192 and input an audio signal to the end-to-end speech recognition model to identify a character string for the spoken voice included in the audio signal.

On the other hand, the end-to-end speech recognition model 101' may be implemented as software and stored in the memory 192, and the end-to-end speech recognition model may perform the algorithm of the end-to-end speech recognition model 101'. It may be implemented as a dedicated chip and included in the processor 191 .

Also, the configuration of the present invention described above with reference to FIG. 1 may be a configuration implemented by an instruction executed by the processor 191 .

As described above, the embodiments of the endpoint detection device and the control method using the same have been described according to the present invention, but this will be described as at least one embodiment, and the technical spirit of the present invention and its configuration and operation are not limited thereby. , the scope of the technical idea of the present invention is not limited / limited by the drawings or the description with reference to the drawings. In addition, the concepts and embodiments of the present invention presented in the present invention can be used by those of ordinary skill in the art as a basis for modifying or designing other structures in order to perform the same purpose of the present invention. , an equivalent structure modified or changed by a person of ordinary skill in the art to which the present invention belongs is bound by the technical scope of the present invention described in the claims, and does not depart from the spirit or scope of the invention described in the claims. Various changes, substitutions and changes are possible within the limits.

Claims (21)

In the voice endpoint detection apparatus comprising a memory for storing instructions and a processor configured to execute the stored instructions,
detecting, by the processor, a phrase included in an audio signal inputted in time series;
detecting, by the processor, a silent frame from among a plurality of frames included in the audio signal, based on a result of the phrase detection;
calculating, by the processor, a silence count value that is the number of consecutive silence frames;
determining, by the processor, as an endpoint when the silence count value exceeds a first value; and
When the silence count value exceeds a second value, the processor initializes the speech recognition model,
A control method of a voice endpoint detection device.
The method of claim 1,
Further comprising the step of initializing, by the processor, the calculated silence count value when the phrase is detected,
A control method of a voice endpoint detection device.
The method of claim 1,
The step of detecting the silent frame comprises:
wherein the processor detects, as a silent frame, a frame that does not correspond to the detected phrase by the silence frame detection unit among a plurality of frames included in the audio signal;
A control method of a voice endpoint detection device.
The method of claim 1,
The speech recognition model is
A speech recognition model based on a recurrent neural network (RNN) in which past recognition results are reflected in current speech recognition,
A control method of a voice endpoint detection device.
5. The method of claim 4,
The initialization of the voice recognition model is,
Including, by the processor, restoring at least one parameter constituting the speech recognition model to an initial state,
A control method of a voice endpoint detection device.
a memory storing instructions; and
a processor configured to execute the stored instructions, the processor comprising:
Detecting a phrase included in an audio signal inputted in time series based on a speech recognition model,
Detecting a silent frame from among a plurality of frames included in the audio signal based on the phrase detection result,
Calculate the silence count value, which is the number of consecutive silence frames,
If the silence count value exceeds the first value, it is determined as an end point,
initializing the speech recognition model when the silence count value exceeds a second value,
Voice endpoint detection device.
The method of claim 6, wherein the processor comprises:
Initializing the calculated silence count value when the phrase is detected,
Voice endpoint detection device.
7. The method of claim 6, wherein the processor is configured to detect the silent frame,
Detecting a frame that does not correspond to the detected phrase among a plurality of frames included in the audio signal as a silent frame,
Voice endpoint detection device.
The method of claim 6, wherein the speech recognition model,
A speech recognition model based on a recurrent neural network (RNN) in which past recognition results are reflected in current speech recognition,
Voice endpoint detection device.
10. The method of claim 9,
The initialization of the voice recognition model is,
Comprising restoring at least one parameter constituting the speech recognition model to an initial state,
Voice endpoint detection device.
In the voice endpoint detection apparatus comprising a memory for storing instructions and a processor configured to execute the stored instructions,
detecting, by the processor, a phrase included in a time-series input audio signal based on a speech recognition model;
when the phrase is detected, calculating, by the processor, a frame difference from the phrase detected immediately before; and
If the frame difference exceeds a first value, the processor determines as an endpoint,
A control method of a voice endpoint detection device.
12. The method of claim 11,
Calculating the difference is
determining, by the processor, a frame immediately preceding the detected phrase;
determining, by the processor, a frame immediately after the phrase detected immediately before; and
calculating, by the processor, a difference between a frame immediately before the detected phrase and a frame immediately after the immediately detected phrase;
A control method of a voice endpoint detection device.
12. The method of claim 11,
Calculating the difference is
determining, by the processor, a first silent frame corresponding to the detected phrase;
determining, by the processor, a second silent frame corresponding to the phrase detected immediately before; and
calculating, by the processor, a difference by the number of silent frames existing between the first and second silent frames;
A control method of a voice endpoint detection device.
12. The method of claim 11,
If the phrase is not detected, the processor further comprising the step of increasing a silence count value that is the number of consecutive silence frames,
A control method of a voice endpoint detection device.
15. The method of claim 14,
if the phrase is not detected, determining, by the processor, whether the silence count value exceeds a second value; and
If the second value is exceeded, the processor further comprising the step of initializing the speech recognition model,
A control method of a voice endpoint detection device.
a memory storing instructions; and
a processor configured to execute the stored instructions, the processor comprising:
Detecting a phrase included in an audio signal inputted in time series based on a speech recognition model,
When the phrase is detected, the difference between the phrase and the frame detected just before is calculated,
If the frame difference exceeds the first value, it is determined as an endpoint,
Voice endpoint detection device.
17. The method of claim 16,
The processor in calculating the difference,
Determining the frame immediately preceding the detected phrase,
determining a frame immediately after the phrase detected immediately before,
calculating a difference between a frame immediately before the detected phrase and a frame immediately after the immediately detected phrase;
Voice endpoint detection device.
17. The method of claim 16,
The processor in calculating the difference,
determining a first silent frame corresponding to the detected phrase;
determining a second silent frame corresponding to the phrase detected immediately before,
calculating the difference by the number of silent frames existing between the first and second silent frames,
Voice endpoint detection device.
17. The method of claim 16,
The processor is
If the phrase is not detected, increasing the silence count value, which is the number of consecutive silence frames,
Voice endpoint detection device.
20. The method of claim 19,
The processor is
if the phrase is not detected, determine whether the silence count value exceeds a second value;
When the second value is exceeded, the voice recognition model is initialized,
Voice endpoint detection device.
A computer program stored on a medium in combination with hardware to execute the method of any one of claims 1 to 5.

Images