KR20230120790A - Speech Recognition Healthcare Service Using Variable Language Model - Google Patents

Abstract

A voice recognition device according to an embodiment of the present specification performs voice recognition using the language model stored in the storage unit for a user's voice input through a voice input unit, but if at least one language disorder occurs in the input voice If there is, the speech recognition is performed using the tuning language model by classifying the language disorder type and selecting a tuning language model according to the classified language disorder type from the storage unit. Accordingly, the voice recognition rate can be improved even when the input speech includes a language impediment component.

Description

Speech Recognition Healthcare Service Using Variable Language Model

The present specification relates to a voice recognition apparatus and method.

When a conversation is conducted through a voice recognition device in which voice, sound, and vocalization peculiar to the disabled are mixed, it may be difficult for non-loving people to understand, resulting in difficulties in smooth communication through voice. In particular, even if the pronunciation is the same, it is often recognized as a word with a completely different meaning or OOV (Out of Vocabulary) when transmitted to the other party through the machine.

In particular, various language disorders such as Parkinson's disease, stroke, and aphasia exist, and it is very difficult to recognize the voice of a person with such a language disorder through a conventional voice recognition system.

The present specification is intended to solve the above problems, and provides a speech recognition apparatus and method capable of improving speech recognition performance by using a plurality of language models reflecting characteristics according to the type of speech impairment.

The technical problems to be achieved by the present invention are not limited to the above-mentioned technical problems, and other technical problems not mentioned are clear to those skilled in the art from the detailed description of the invention below. will be understandable.

A voice recognition device according to an embodiment of the present specification includes a voice input unit; a storage unit storing at least one language model for performing speech recognition; a voice recognition unit performing voice recognition on the user's voice input through the voice input unit using the language model stored in the storage unit; and when it is determined that at least one language disorder exists in the voice inputted through the voice input unit, the type of language disorder is classified, a tuning language model according to the classified language disorder type is selected from the storage unit, and the selected tuning language model is selected. and a processor controlling the voice recognition unit to perform the voice recognition using a language model.

When the processor detects a predetermined speech pattern with respect to the input voice, it is determined that the speech disorder exists, and the predetermined speech pattern is among stuttering, repetitive reading, inaccurate pronunciation, long pause, and lack of volume. may contain at least one.

The storage unit may include at least two or more of a first language model corresponding to a stuttering pattern, a second language model corresponding to a repetitive reading pattern, a third language model corresponding to an inaccurate pronunciation pattern, and a speech pattern determined to be the language disorder. A fourth language model corresponding to the combined pattern may be included.

The processor may include: a learning data acquiring unit acquiring a speech pattern corresponding to the language disorder as learning data; It may further include a model learning unit for learning a language model corresponding to a specific language disorder through a supervised learning process with respect to the learning data.

When it is determined that at least one language disorder exists in the voice input through the voice input unit, the processor responds to a general language model and a specific language disorder among language models stored in the storage unit in the process of performing the voice recognition. It is possible to control to output a voice recognition result by applying each of the tuning language models.

A voice recognition method according to another embodiment of the present specification includes receiving a user's voice through a voice input unit; and performing voice recognition on the user's voice input through the voice input unit using a language model stored in a storage unit, wherein it is determined that at least one language disorder exists in the voice input through the voice input unit. if it is, classifying the language disorder type and selecting a tuning language model according to the classified language disorder type from the storage unit; and controlling the voice recognition unit to perform the voice recognition using the selected tuning language model.

The voice recognition method further includes determining that the speech disorder exists when a predetermined speech pattern is detected for the input voice, wherein the predetermined speech pattern includes stuttering, repetitive reading, and pronunciation. It may include at least one of inaccuracies, long pauses, and lack of volume.

The storage unit may include at least two or more of a first language model corresponding to a stuttering pattern, a second language model corresponding to a repetitive reading pattern, a third language model corresponding to an inaccurate pronunciation pattern, and a speech pattern determined to be the language disorder. A fourth language model corresponding to the combined pattern may be included.

The voice recognition method may include acquiring learning data for acquiring a speech pattern corresponding to the language disorder as learning data; The method may further include learning a language model corresponding to a specific language disorder through a supervised learning process with respect to the learning data.

When it is determined that there is at least one language disorder in the voice inputted through the voice input unit, a general language model and a language model corresponding to a specific language disorder among the language models stored in the storage unit are each in the process of performing the voice recognition. It may further include; controlling to output a voice recognition result by applying.

Another embodiment of the present specification includes a computer program implemented to execute the above-described voice recognition method and stored in a computer-readable recording medium.

According to an embodiment of the present specification, speech recognition performance may be improved by using a plurality of language models reflecting characteristics according to the type of language impairment.

The effects obtainable in the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below. .

The accompanying drawings, which are included as part of the detailed description to aid understanding of the present specification, provide examples of the present specification and describe technical features of the present specification together with the detailed description.
1 is a diagram for explaining the configuration of a voice recognition apparatus according to an embodiment of the present specification.
2 is a diagram for specifically explaining a voice processing module according to an embodiment of the present specification.
3 is a flowchart of a voice recognition method according to an embodiment of the present specification.

Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings, but the same or similar components are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used together in consideration of ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves. In addition, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiment disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, the technical idea disclosed in this specification is not limited by the accompanying drawings, and all changes included in the spirit and technical scope of the present invention , it should be understood to include equivalents or substitutes.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as "comprise" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

1 is a diagram for explaining the configuration of a voice recognition apparatus according to an embodiment of the present specification.

The voice recognition apparatus 100 may include an electronic device including an AI module capable of performing AI processing or a server including the AI module. The AI processing may include all operations related to device control.

The voice recognition device 100 includes mobile phones, smart phones, desktop computers, laptops, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), tablet PCs, wearable devices, set-top boxes (STBs), and DMB receivers. , it can be implemented as a fixed device such as a robot or a movable device. If the above example is only an example of a user terminal, the voice recognition device 100 may be a server device connected to a network.

The voice recognition device 100 may include training data 110, a learning unit 120, a neural network model 130, a communication unit 140, an input unit 150, an output unit 160, and a processor 170. there is. A voice processor that processes the input voice may be included in the processor 170 or provided as an independent module.

The training data 110 is data for training the artificial neural network, and is information about objects, events, concepts or rules, their relationships and properties, organized of goal-oriented systematic use acquired through experience or education. ) and the like. The training data 110 may include the meaning of a training data acquisition unit.

According to an embodiment of the present specification, the training data 110 may include a voice signal input to the voice recognition apparatus 100. Alternatively, the training data 110 may mean only a specific voice signal including a language disorder component among voice signals input to the voice recognition apparatus 100 . Alternatively, as a result of performing voice recognition on the input voice signal, the processor 170 may distinguish a voice signal having a voice recognition rate equal to or less than a predetermined recognition rate, and obtain a voice signal having a low voice recognition rate as the training data 110 .

Alternatively, when a voice corresponding to a predetermined speech pattern is input, the processor 170 may use it as training data separately from the process of performing voice recognition. Here, the predetermined speech pattern may include a voice signal corresponding to at least one speech pattern that can be classified into a speech disorder type according to an embodiment of the present specification. In order to determine whether or not the voice corresponds to the predetermined speech pattern, the processor 170 converts the voice signal into text through voice recognition, and then analyzes the converted text so that the input voice corresponds to the speech pattern of speech impairment. It may be determined whether or not the signal is a voice signal.

According to an embodiment, the predetermined speech pattern for speech impairment may refer to a speech pattern in a state in which smooth speech recognition is impossible due to stuttering, repetitive reading, inaccurate pronunciation, long pauses, insufficient volume, and the like. On the other hand, the aforementioned speech patterns such as stuttering, repetitive reading, inaccurate pronunciation, long pauses, and lack of volume are examples for explaining the technical idea of the present specification, and the predetermined speech pattern for speech impairment is not limited to the above-described examples and is a new one. By registering the type of language disorder found, it is possible to build a speech recognition language model optimized for various symptoms of language disorder.

The learning unit 120 may learn criteria regarding which training data to use and how to classify and recognize data using the training data in order to determine data classification/recognition. The learning unit 120 may learn the neural network model by obtaining learning data to be used for learning and applying the obtained learning data to the neural network model.

The learning unit 120 may be manufactured in the form of at least one hardware chip and mounted in the voice recognition device 100 . For example, the learning unit 120 may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or manufactured as a part of a general-purpose processor (CPU) or a graphics-only processor (GPU) for the voice recognition device 100. may be mounted. Also, the learning unit 120 may be implemented as a software module. When implemented as a software module (or a program module including instructions), the software module may be stored in a computer-readable, non-transitory computer readable recording medium (non-transitory computer readable media). In this case, at least one software module may be provided by an Operating System (OS) or an application.

When the training data 110 is determined, the learning unit 120 according to an embodiment of the present specification performs a pre-processing process on the training data 110 and uses the pre-processed data to learn the artificial neural network.

The learning unit 120 may further include a data acquisition unit (not shown) and a model learning unit (not shown).

The learning data acquisition unit may acquire learning data required for a neural network model for classifying and recognizing data. For example, the data acquisition unit 23 may acquire sample data to be input to a neural network model as training data. According to an embodiment of the present specification, the sample data for learning are all voice signals attracted to the voice recognition apparatus 100 or, when a predetermined speech pattern is registered, a voice signal corresponding to the predetermined speech pattern. can be extracted and defined as learning data.

The model learning unit may learn to have a criterion for determining how to classify predetermined data by using the acquired training data. At this time, the model learning unit may learn the neural network model through supervised learning using at least a part of the learning data as a criterion. Alternatively, the model learning unit may learn the neural network model through unsupervised learning in which a decision criterion is discovered by self-learning using training data without guidance. In addition, the model learning unit may train the neural network model through reinforcement learning using feedback about whether the result of the situation judgment according to learning is correct. In addition, the model learning unit may train the neural network model using a learning algorithm including error back-propagation or gradient decent.

When the neural network model is learned, the model learning unit may store the learned neural network model in memory. The model learning unit may store the learned neural network model in a memory of a server connected to the voice recognition device 100 through a wired or wireless network.

The learning unit 120 may further include a learning data pre-processing unit (not shown) and a learning data selection unit (not shown) in order to improve the analysis result of the recognition model or save resources or time required for generating the recognition model. may be

The learning data pre-processing unit may pre-process the acquired data so that the acquired data can be used for learning for situation determination. For example, the learning data pre-processing unit may process the acquired data into a preset format so that the model learning unit can use voice data input to the voice recognition device 100 and text data corresponding to a voice recognition result as training data. can

In addition, the learning data selector may select data necessary for learning from among the learning data acquired by the learning data acquisition unit or the learning data preprocessed by the preprocessor. According to one embodiment of the present specification, a specific speech pattern having a language disorder component includes a speech pattern showing a single language disorder symptom, a speech pattern including two symptoms of a language disorder, and a speech pattern including two or more symptoms of a language disorder. etc. may be included. In the process of selecting training data, the processor may learn a single language disorder language model using the above-described single language symptom speech pattern data as training data. In addition, the processor may learn a complex language disorder language model using data of a plurality of speech patterns of speech symptoms as learning data. For example, the processor may use data of a stuttering speech pattern and a repeated reading speech pattern as training data for tuning a language model.

In addition, the learning unit 120 may further include a model evaluation unit (not shown) to improve the analysis result of the neural network model.

The model evaluation unit inputs evaluation data to the neural network model, and when an analysis result output from the evaluation data does not satisfy a predetermined criterion, it may cause the model learning unit 120 to learn again. In this case, the evaluation data may be predefined data for evaluating the recognition model. For example, the model evaluator may evaluate that the predetermined criterion is not satisfied when the number or ratio of the evaluation data for which the analysis result is inaccurate among the analysis results of the learned recognition model for the evaluation data exceeds a preset threshold. there is.

The communication unit 140 may transmit the AI processing result by the processor 170 to an external electronic device. Here, the external electronic device may include an external server, a user terminal, and the like.

Communication technologies used by the communication unit 140 include Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Long Term Evolution (LTE), 5G, Wireless LAN (WLAN), Wireless-Fidelity (Wi-Fi), There are Bluetooth?, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), ZigBee, Near Field Communication (NFC), and the like.

The input unit 140 may acquire various types of data. The input unit 120 may include a camera for inputting a video signal, a microphone for receiving an audio signal, and a user input unit for receiving information from a user. Here, a camera or microphone may be treated as a sensor, and signals obtained from the camera or microphone may be referred to as sensing data or sensor information.

The input unit 150 may obtain learning data for model learning and input data to be used when obtaining an output using the learning model. The input unit 150 may obtain raw input data, and in this case, the processor 170 may extract input features as preprocessing of the input data.

The output unit 160 may generate an output related to sight, hearing, or touch. The output unit 160 may include a display unit that outputs visual information, a speaker that outputs auditory information, and a haptic module that outputs tactile information.

The processor 170 may store the neural network model (NN Model, 130) in a memory (not shown) when the artificial neural network learning is completed in the learning unit 120.

The processor 170 may learn the neural network using a program stored in a memory (not shown). In particular, when the speech input to the voice recognition apparatus 100 includes a language impairment factor, the processor 170 uses a neural network based on the training data including the language disability factor to increase the voice recognition rate despite the language disability factor. can learn Here, the neural network may be designed to simulate the structure of the human brain on a computer, and may include a plurality of network nodes having weights that simulate neurons of the human neural network. A plurality of network modes may transmit and receive data according to a connection relationship, respectively, so as to simulate synaptic activity of neurons that transmit and receive signals through synapses. Here, the neural network may include a deep learning model developed from a neural network model. In the deep learning model, a plurality of network nodes may exchange data according to a convolution connection relationship while being located in different layers. Examples of neural network models are deep neural networks (DNN), convolutional deep neural networks (CNN), recurrent Boltzmann machines (RNNs), restricted Boltzmann machines (RBMs), deep trust It includes various deep learning techniques such as deep belief networks (DBN) and deep Q-networks, and can be applied to fields such as computer vision, voice recognition, natural language processing, and voice/signal processing.

Meanwhile, the processor performing the functions described above may be a general-purpose processor (eg, CPU), or may be an AI-only processor (eg, GPU) for artificial intelligence learning.

Meanwhile, the voice recognition apparatus 100 may further include a memory (not shown). The memory may store various programs and data required for speech recognition operation. The memory may be implemented as non-volatile memory, volatile memory, flash-memory, hard disk drive (HDD) or solid state drive (SDD). The memory is accessed by the processor 170, and data can be read/written/modified/deleted/updated by the processor 170. In addition, the memory is a neural network model (e.g., a language model corresponding to a specific language disorder, a complex language corresponding to a plurality of symptoms of a language disorder) generated through a learning algorithm for data classification / recognition according to an embodiment of the present invention. model) can be saved.

In the present specification, one or more of various devices to which a voice recognition device can be utilized, for example, a surveillance camera, an autonomous vehicle, a user terminal, and a server, an artificial intelligence module, a robot, and an augmented reality (AR) ) device, a virtual reality (VT) device, and a device related to 5G service.

Meanwhile, referring to FIG. 2 , the voice recognition apparatus 100 according to an embodiment of the present specification may include a voice processing module (hereinafter described as a voice processing module provided inside the processor 170). The voice processing module 170 includes an Auto Speech Recognition (ASR) module 171, an Artificial Intelligent Agent (172), a Natural Language Understanding (NLU) module 173, and text-to-speech conversion. (Text-to-Speech, TTS) module 174 may be included.

The ASR module 171 may convert the received user voice input into text data. The ASR module 171 may include a front-end speech pre-processor. A front-end speech preprocessor extracts representative features from speech input. For example, a front-end speech preprocessor performs a Fourier transform on a speech input to extract spectral features characterizing the speech input as a sequence of representative multi-dimensional vectors. Additionally, the ASR module 171 includes one or more speech recognition models (eg, acoustic models and/or language models) and may include one or more speech recognition engines. Examples of speech recognition models may include Hidden Markov Models, Gaussian-Mixture Models, Deep Neural Network Models, n-gram language models, and other statistical models. . Examples of speech recognition engines may include dynamic time warping based engines and weighted finite state transformer (WFST) based engines. One or more speech recognition models and one or more speech recognition engines may generate intermediate recognition results (e.g., phonemes, phoneme strings, and subordinate words), and ultimately text recognition results (e.g., words, word strings, or tokens). sequence) can be used to process the extracted representative features of the front-end speech preprocessor.

When ASR module 171 generates a recognition result comprising a text string (eg, words, or sequences of words, or sequences of tokens), the recognition results are passed to natural language processing module 173 for intent inference. can be conveyed In some examples, ASR module 171 generates multiple candidate textual representations of speech input. Each candidate text representation is a sequence of words or tokens corresponding to speech input.

The NLU module 173 may determine user intention by performing syntactic analysis or semantic analysis. The grammatical analysis may divide grammatical units (eg, words, phrases, morphemes, etc.) and determine which grammatical elements the divided units have. The semantic analysis may be performed using semantic matching, rule matching, formula matching, and the like. Accordingly, the NUL module 173 may obtain a domain, an intention, or a parameter required to express the intention.

The NLU module 173 may determine the user's intention and parameters using a mapping rule divided into a domain, an intention, and parameters necessary to determine the intention. For example, one domain (eg, weather information) may include a plurality of intents (eg, hot, cold, emotional expression for weather, etc.), and one intent may include a plurality of parameters (eg, For example, time, number of repetitions, alarm sound, etc.) may be included. A plurality of rules may include, for example, one or more mandatory element parameters. The matching rules may be stored in a Natural Language Understanding Database.

The NLU module 173 identifies the meaning of words extracted from user input using linguistic features (eg, grammatical elements) such as morphemes and phrases, and matches the meaning of the identified words to domains and intents. to determine the user's intent. For example, the NLU module 173 may determine the user intent by calculating how many words extracted from the user input are included in each domain and intent. According to an embodiment, the NLU module 173 may determine a parameter of the user input using a word based on determining the intention. According to an embodiment, the NLU module 173 may determine the user's intention by using a natural language recognition database in which linguistic features for identifying the intention of the user input are stored. Also, according to an embodiment, the NLU module 173 may determine the user's intention using a personal language model (PLM). For example, the NLU module 173 may determine the user's intention using personalized information (eg, language use characteristics, language disorder, etc.). The personalized language model may be stored, for example, in a natural language recognition database. According to an embodiment, the ASR module 173 as well as the NLU module 173 may recognize a user's voice by referring to a personalized language model stored in a natural language recognition database.

The NLU module 173 may further include a natural language generation module (not shown). The natural language generating module may change designated information into a text form. The information changed to the text form may be in the form of natural language speech. The designated information may include, for example, additional input information, information guiding completion of an operation corresponding to a user input, or information guiding additional user input. The information changed in text form may be transmitted to a client device and displayed on a display, or may be transmitted to a TTS module and changed into a voice form.

Here, the NLU module 173 may use a general language model in inferring the user's intention. The general language model may be a language model used by the voice recognition device 100 when a speaker inputs a voice without any language disorder. However, according to one embodiment of the present specification, when the user's intention is not normally inferred using the general language model, in particular, when the input voice includes a language disorder element and uses the general language model, normal voice recognition is impossible. If it is determined that it is, a tuning language model in which characteristic elements of language disorder are tuned in a general language model may be used.

According to an embodiment, as a result of performing voice recognition using the general language model in the NLU module 173, the processor may determine that the input voice contains a language impediment component when it is impossible to extract the user's intention. there is.

In addition, according to an embodiment, the processor performs the grammatical analysis in the NLU module 173 on the result of converting the input voice into text in the ASR module 171, and determines what grammatical elements the classified morphemes have. If not, it may be determined that the input voice includes a language disorder element.

The method of determining that the input voice includes a language impediment component is not limited to the above example and may be implemented through various methods.

The voice synthesis module (TTS module, 174) may change text-type information into voice-type information. The TTS module 174 may receive text-type information from the natural language generation module of the NLU module 173, convert the text-type information into voice-type information, and output the voice-type information through a speaker.

Speech synthesis module 174 synthesizes speech output based on the provided text. For example, the result generated by the speech recognition module (ASR) 171 is in the form of a text string. Speech synthesis module 174 converts the text string into audible speech output. Speech synthesis module 174 uses any suitable speech synthesis technique to generate speech output from text, including concatenative synthesis, unit selection synthesis, diphone synthesis, domain- but is not limited to specific synthesis, formant synthesis, articulatory synthesis, hidden Markov model (HMM) based synthesis, and sinewave synthesis.

In some examples, speech synthesis module 174 is configured to synthesize individual words based on phoneme strings corresponding to the words. For example, a phoneme string is associated with a word in a generated text string. Phoneme strings are stored in metadata associated with words. Speech synthesis module 174 is configured to directly process phoneme strings in the metadata to synthesize words in speech form.

According to an embodiment of the present specification, as a result of performing speech synthesis through the TTS 174 on the speech processing results of the ASR module 171 and the NLU module 173, the output of the TTS 174 has independent linguistic meaning. For example, the user's intention was to input the voice "hot", but due to the user's language impairment, TTS 174 results such as "hotter", "more hotter", "more hotter", etc. is output, it is determined that the input voice includes a language disorder component, and the input voice may be stored as learning data for language model tuning later. That is, in an embodiment of the present specification, it is not determined whether or not the input voice includes a language disorder only through the ASR module 171 and the NLU module 173, but the voice processing result of the ASR module 171 and the NLU module 173. Although it was not determined whether a language disorder element was included in <RTI ID=0.0>,</RTI> it is possible to finally determine whether or not a language disorder is included through the output result of the TTS (174).

Meanwhile, the chatbot conversation processing apparatus according to an embodiment of the present invention may further include an artificial intelligence agent (AI agent) 172. The intelligent agent 172 may be designed to perform at least some of the functions performed by the aforementioned ASR module 171, NLU module 173, and/or TTS module 173. In addition, the intelligent agent module 172 may contribute to performing independent functions of each of the ASR module 171, the NLU module 173, and/or the TTS module 174.

The intelligent agent module 173 may perform the aforementioned functions through deep learning (deep learning). The deep learning includes deep neural networks (DNN), convolutional deep neural networks (CNN), recurrent Boltzmann Machine (RNN), Restricted Boltzmann Machine (RBM), and deep trust neural networks. Various deep learning techniques such as deep belief networks (DBN) and deep Q-networks can be applied to fields such as computer vision, voice recognition, natural language processing, and voice/signal processing.

The intelligent agent module 172 can perform various natural language processing processes including machine translation, emotion analysis, and information retrieval by using a deep artificial neural network structure in the field of natural language processing. .

An embodiment of the present specification may determine whether a speech input through the intelligent agent module 172 includes a language impediment component. The intelligent agent module 172 may be used to learn a personalized language model by collecting various personalized information. According to an embodiment, it is possible to determine major types of language disorder mainly expressed by the user of the voice recognition apparatus 100, and to perform tuning on a language model corresponding to the type of language disorder. The intelligent agent module 172 may utilize personalized information, for example, maps, SMS, News, Music, Stock, Weather, and wikipedia information, to learn the personalized language model.

The intelligent agent 172 is expressed as a separate block to be distinguished from the ASR module 171, NLU module 173, and TTS module 174 for convenience of description, but the intelligent agent 172 is each of the modules ( 171,173,174) may perform at least some or all of the functions.

3 is a flowchart of a voice recognition method according to an embodiment of the present specification. The voice recognition method may be implemented through a processor or a voice processor of FIG. 1 .

Referring to FIG. 3 , the processor 170 may receive a voice input through an input unit (S300). The processor 170 may determine whether the input voice includes a language disorder component (S310). The processor 170 determines whether to use a language model that is generally used for voice recognition operation or a tuning language model specialized for speech including a language disorder. It is necessary to determine whether it is included or not.

As described above, the processor 170 may determine whether or not a speech impediment component is included in the input speech based on the ASR result, the NLU result, and/or the TTS result of the input speech. According to an embodiment, the processor 170 may determine that the language disorder exists when a predetermined speech pattern is detected with respect to the input voice. Here, the predetermined speech pattern for determining the language disorder component may include at least one of stuttering, repetitive reading, inaccurate pronunciation, long pauses, and lack of volume.

When it is determined that the input voice includes a language disorder component, the processor 170 may distinguish the type of the language disorder. The type of language disorder may be classified into items such as stuttering, repetitive reading, inaccurate pronunciation, long pauses, lack of volume, and the like, and the processor 170 determines whether the input voice is included in at least one of the above-exemplified language disorder types. Determine (S320). When the processor 170 determines that the input voice is not included in at least one of the above exemplified language disorder types, the processor 170 registers the input voice as a new language disorder type and creates a tuning language model for the new language disorder type. It can be transmitted to the learning data acquisition unit for. According to an example, the processor 170 may control speech recognition to be performed by applying a general language model when at least one of the above-exemplified speech impairment types is not included.

When the type of language disorder is classified, the processor 170 may select a tuning language model corresponding to the classified language disorder type from the storage unit (S330).

The storage unit may include a first language model corresponding to a stuttering pattern, a second language model corresponding to a repetitive reading pattern, and a third language model corresponding to an inaccurate pronunciation pattern. According to an embodiment, the storage unit may include a fourth language model corresponding to a pattern in which at least two or more of the speech patterns determined to be a language disorder are combined. For example, the fourth language model includes a tuning model specialized for a stuttering pattern and a repetitive reading pattern, a tuning model specialized for a stuttering pattern and a long pause pattern, a stuttering pattern, a long pause pattern, and a lack of volume pattern at the same time. It may include language models optimized for each language disorder symptom, such as a tuning model specialized for speech. That is, the fourth language model can be applied in the process of voice recognition (determining user's intention) to an input voice in which two symptoms of language disorder or three symptoms of language disorder appear in combination. According to an example, the processor 170 may perform voice processing by selecting one of the fourth language models from the storage unit when a stuttering pattern and a repetitive reading pattern are simultaneously identified in the input voice. .

The processor 170 may perform a voice recognition operation by applying the tuning language model selected from the storage unit (S340). Meanwhile, according to an embodiment of the present specification, an acoustic model may be applied in the same manner as a general voice recognition operation.

On the other hand, according to an embodiment, when the language disorder element is included, the processor 170 may perform voice recognition together by applying a general language model as well as using a tuning language model, and provide two results together. You may.

According to an embodiment, the processor 170 may acquire a speech pattern corresponding to a language disorder as learning data, and continuously learn a language model corresponding to a specific language disorder through a supervised learning process with respect to the learning data.

The above-described present invention can be implemented as computer readable code on a medium on which a program is recorded. The computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. , and also includes those implemented in the form of a carrier wave (eg, transmission over the Internet). Accordingly, the above detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (10)

voice input unit;
a storage unit storing at least one language model for performing speech recognition;
a voice recognition unit performing voice recognition on the user's voice input through the voice input unit using the language model stored in the storage unit; and
When it is determined that at least one language disorder exists in the voice input through the voice input unit, a language disorder type is classified, a tuning language model according to the classified language disorder type is selected from the storage unit, and the selected tuning language is selected. a processor controlling the voice recognition unit to perform the voice recognition using a model;
Voice recognition device comprising a. According to claim 1,
the processor,
When a predetermined speech pattern is detected for the input voice, it is determined that the language disorder exists;
The speech recognition device of claim 1 , wherein the predetermined speech pattern includes at least one of stuttering, repetitive reading, inaccurate pronunciation, long pauses, and lack of volume. According to claim 2,
the storage unit,
A pattern combining at least two of a first language model corresponding to a stuttering pattern, a second language model corresponding to a repetitive reading pattern, a third language model corresponding to an inaccurate pronunciation pattern, and a speech pattern determined to be the language disorder A voice recognition device comprising a corresponding fourth language model. According to claim 1,
the processor,
a learning data acquiring unit acquiring a speech pattern corresponding to the language disorder as learning data; and
a model learning unit for learning a language model corresponding to a specific language disorder through a supervised learning process with respect to the learning data;
Voice recognition device further comprising a. According to claim 1,
the processor,
When it is determined that there is at least one language disorder in the voice inputted through the voice input unit, the general language model and the tuning language corresponding to the specific language disorder among the language models stored in the storage unit in the process of performing the voice recognition. A voice recognition device characterized by controlling to output a voice recognition result by applying each model. receiving a user's voice through a voice input unit;
Performing voice recognition using a language model stored in a storage unit for the user's voice input through the voice input unit;
if it is determined that at least one language disorder exists in the voice inputted through the voice input unit, classifying a language disorder type and selecting a tuning language model according to the classified language disorder type from the storage unit;
controlling the voice recognition unit to perform the voice recognition using the selected tuning language model;
Voice recognition method comprising a. According to claim 6,
Further comprising: determining that the language disorder exists when a predetermined speech pattern is detected with respect to the input voice;
The speech recognition method of claim 1 , wherein the predetermined speech pattern includes at least one of stuttering, repetitive reading, inaccurate pronunciation, long pauses, and lack of volume. According to claim 6,
the storage unit,
A pattern combining at least two of a first language model corresponding to a stuttering pattern, a second language model corresponding to a repetitive reading pattern, a third language model corresponding to an inaccurate pronunciation pattern, and a speech pattern determined to be the language disorder A speech recognition method comprising a corresponding fourth language model. According to claim 6,
Acquiring learning data for obtaining a speech pattern corresponding to the language disorder as learning data;
learning a language model corresponding to a specific language disorder through a supervised learning process with respect to the learning data;
Voice recognition method further comprising a. According to claim 6,
When it is determined that there is at least one language disorder in the voice inputted through the voice input unit, a general language model and a language model corresponding to a specific language disorder among the language models stored in the storage unit are each in the process of performing the voice recognition. controlling to output a voice recognition result by applying;
Voice recognition method further comprising a.

Images