KR20240036280A - Method for voice memo service and apparatus therefor - Google Patents

Abstract

The present invention relates to a method and device for a voice memo service, and more specifically, to obtaining information about intent and information about named entity for one or more first text sentences; obtaining summary text for the one or more first text sentences based on the information about the intent and the information about the entity name; determining a priority for the one or more first text sentences based on the information about the intent and the information about the entity name; and providing the summary text to a terminal device according to the determined priority, and a device therefor. According to the present invention, it is possible to respond quickly and effectively by providing a summary of voice memos according to priority.

Description

Method for voice memo service and device therefor {METHOD FOR VOICE MEMO SERVICE AND APPARATUS THEREFOR}

The present invention relates to a voice memo service, and more specifically, to a method and device for providing summary text for a voice memo based on voice recognition.

The voice memo service, in which a user leaves a memo by voice to the other party and the system stores it and then passes it on to the other party, has been widely used. The voice memo service receives and stores user voice data from various devices such as mobile phones, telephones, and voice recorders, and provides the stored voice data as is when requested by the other party, or converts the voice data into text data using voice recognition technology. Includes services that

In existing voice memo services, the voice recognition results are simply provided as text, so the other party can understand the content after reading the entire text. For people who receive many voice memos, reading these texts and understanding the content takes a very long time. and requires effort. In particular, small business owners receive voice memos related to many inquiries and requests, but when small business owners receive the results of these voice memo services in text form, there is a problem in that it is difficult to check the contents individually, accurately analyze the user's intent, and respond quickly. Therefore, a method for effectively understanding voice memos is required.

In addition, as new industries emerge every day, new related vocabulary and sentences are added, so it may be difficult to guarantee voice recognition performance for new vocabulary and sentences no matter how much data is trained in advance. Because users also leave voice notes by using vocabulary and sentences in various modified forms rather than using them in a standardized form, there is a problem in securing voice recognition performance for all users' voices or utterances. Therefore, there is a need for a method to improve voice recognition performance for voice memos in response to changing industries and user tastes.

Registered Patent No. 10-2147619

The purpose of the present invention is to provide a method and device for effectively providing summary text for a voice memo.

Specifically, the purpose of the present invention is to classify the user's intention based on the voice recognition result in the voice memo service, extract entity names, generate summary text for the voice memo, and provide the content of the voice memo according to the priority of the content. The purpose is to provide a method and a device for effectively identifying

Another object of the present invention relates to a method and device for improving voice recognition performance for voice memos.

Specifically, another object of the present invention is to provide a method and device for improving speech recognition performance by generating sentence patterns and new sentences based on text sentences with low speech recognition reliability and using them for automatic transcription and speech recognition model learning. There is something to do.

Another object of the present invention is to provide a method and device for improving customer response efficiency of small business owners using voice memo services.

Specifically, another purpose of the present invention is to classify the user's intention based on the voice recognition results of the voice memo left by the customer in a call to the small business owner in order to provide a convenient call-based free speech voice memo service to small business owners and enter an entity name (keyword) The purpose is to provide a method and device for small business owners to efficiently respond to customers by extracting the memo contents, briefly summarizing them, and printing them on the screen according to the priority of the contents.

The technical problems to be solved by the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned are clearly apparent to those skilled in the art from the contents described in this specification. It will be understandable.

In a first aspect of the invention, a method is provided for processing voice data in a server device, the method comprising: obtaining information about intent and information about named entities for one or more first text sentences; thing; obtaining summary text for the one or more first text sentences based on the information about the intent and the information about the entity name; determining a priority for the one or more first text sentences based on the information about the intent and the information about the entity name; And it may include providing the summary text to the terminal device according to the determined priority.

In a second aspect of the invention, there is provided a processor; and a memory, wherein the memory includes instructions configured to implement specific operations that, when performed by the processor, cause the device to process voice data, the specific operations comprising: one or more first text sentences; obtaining information about intent and information about named entities; obtaining summary text for the one or more first text sentences based on the information about the intent and the information about the entity name; determining a priority for the one or more first text sentences based on the information about the intent and the information about the entity name; And it may include providing the summary text to the terminal device according to the determined priority.

In a third aspect of the invention, there is provided a computer-readable storage medium storing instructions configured to, when executed by a processor, cause a device including the processor to implement specific operations for processing voice data, the specific operations comprising: : Obtaining information about intent and information about named entities for one or more first text sentences; obtaining summary text for the one or more first text sentences based on the information about the intent and the information about the entity name; determining a priority for the one or more first text sentences based on the information about the intent and the information about the entity name; And it may include providing the summary text to the terminal device according to the determined priority.

Preferably, obtaining the summary text may include: obtaining the summary text by combining information about the intent and information about the entity name.

Preferably, determining the priority includes: obtaining a second priority score of the information about the intent and a third priority score of the information about the entity name in a priority list; And it may include determining the priority based on the sum of the second priority score and the third priority score by applying weights to each of them.

Preferably, the method or specific operation further comprises: obtaining information regarding the industry classification for the one or more first text sentences, wherein determining the priority comprises: determining information regarding the industry classification in the priority list; obtaining a first rank score of information, a second rank score of information about the intent, and a third rank score of the information about the entity name; And it may include determining the priority based on a value obtained by applying a weight to each of the first priority score, the second priority score, and the third priority score.

Preferably, the method or specific operation includes: obtaining the one or more first text sentences based on the user's speech data using a first speech recognition model; Based on the speech recognition reliability for the one or more first text sentences not satisfying at least one first criterion, a sentence pattern and at least one entity name based on the information about the intent and the information about the entity name Obtaining candidate words and generating one or more second text sentences based on the at least one entity name candidate word according to the obtained sentence pattern; and based on the speech recognition reliability of the one or more second text sentences satisfying at least one second criterion, a second text sentence having the highest speech recognition reliability among the one or more second text sentences is transmitted to the user's voice. It may further include determining based on the voice recognition results for the data.

Preferably, the method or specific operation further comprises: learning a second speech recognition model for each of the one or more second text sentences using each of the one or more second text sentences, wherein the one or more second Speech recognition reliability for each text sentence may be obtained based on the second speech recognition model.

Preferably, the voice recognition reliability includes reliability for sentences and reliability for words, and if the speech recognition reliability for the one or more first text sentences does not satisfy the at least one first criterion, the one or more It may include that the reliability of a sentence of the first text sentence is less than a first threshold or that the reliability of the words of the one or more first text sentences is less than a second threshold.

Preferably, the speech recognition reliability includes reliability for sentences and reliability for words, and the speech recognition reliability for the one or more second text sentences satisfies at least one second criterion. It may include that the reliability of the sentences of the sentence is greater than or equal to a third threshold and that the reliability of the words of the at least one second text sentence is greater than or equal to a fourth threshold.

Preferably, the method or specific operation: selects a second text sentence having the highest speech recognition reliability, based on the speech recognition reliability for the one or more second text sentences satisfying the at least one second criterion. further comprising learning the first speech recognition model using, wherein, based on the speech recognition reliability for the one or more second text sentences not satisfying the at least one second criterion, the highest speech recognition reliability Learning of the first speech recognition model using the second text sentence having can be omitted.

Preferably, the method or specific operation: converts the one or more first text sentences into the user's voice data based on the speech recognition reliability for the one or more first text sentences satisfying the at least one first criterion. It may further include determining based on the voice recognition result.

According to the present invention, it is possible to effectively provide summary text for voice memos.

Specifically, according to the present invention, the voice memo service classifies the user's intention based on the voice recognition result, extracts the entity name, generates a summary text for the voice memo, and provides the content of the voice memo according to the priority of the content. allows you to understand effectively.

Additionally, according to the present invention, voice recognition performance for voice memos can be improved.

Specifically, according to the present invention, speech recognition performance can be improved by generating sentence patterns and new sentences based on text sentences with low speech recognition reliability and using them for automatic transcription and speech recognition model learning.

Additionally, according to the present invention, the customer response efficiency of small business owners using voice memo services can be improved.

Specifically, according to the present invention, in order to provide a convenient call-based free speech voice memo service to small business owners, the user's intention is classified based on the voice recognition results from the voice memo left by the customer over the phone to the small business owner, and the entity name is extracted and memoized. It briefly summarizes the content and displays it on the screen according to the priority of the content, allowing small business owners to respond to customers efficiently.

The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the contents described in this specification. There will be.

The accompanying drawings are included as part of the detailed description to aid understanding of the present invention and explain embodiments and technical features of the present invention together with the detailed description.
Figure 1 illustrates a functional block diagram of a voice memo service system according to the proposed method of the present invention.
Figure 2 illustrates a flowchart of the proposed method of the present invention.
Figure 3 illustrates obtaining summary text according to the proposed method of the present invention.
Figure 4 illustrates voice memo output according to the proposed method of the present invention.
Figure 5 illustrates a method of building an intent classification model according to the proposed method of the present invention.
Figure 6 illustrates the intent classification results according to the proposed method of the present invention.
Figure 7 illustrates a priority list according to the proposed method of the present invention.
Figure 8 illustrates priority determination according to the proposed method of the present invention.
Figure 9 illustrates a flowchart of the proposed method of the present invention.
Figure 10 illustrates generating a sentence pattern and a new sentence according to the proposed method of the present invention.
Figure 11 illustrates creating a small speech recognition model and generating a new sentence according to the proposed method of the present invention.
Figure 12 illustrates a device to which the proposed method of the present invention can be applied.

The present invention can be modified in various ways and can have various embodiments. Hereinafter, specific embodiments will be described in detail based on the accompanying drawings.

The following examples are provided to provide a comprehensive understanding of the methods, devices and/or systems described herein. However, this is only an example and the present invention is not limited thereto.

In describing the embodiments of the present invention, if it is determined that a detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. In addition, the terms described below are terms defined in consideration of functions in the present invention, and may vary depending on the intention or custom of the user or operator. Therefore, the definition should be made based on the contents throughout this specification. The terminology used in the detailed description is merely for describing embodiments of the present invention and should in no way be limiting. Unless explicitly stated otherwise, singular forms include plural meanings. In this description, expressions such as “including” or “including” are intended to indicate certain features, numbers, steps, operations, elements, parts or combinations thereof, and one or more than those described. It should not be construed to exclude the existence or possibility of any other characteristic, number, step, operation, element, or part or combination thereof.

In addition, terms such as first, second, etc. may be used to describe various components, but the components are not limited by the terms, and the terms are used for the purpose of distinguishing one component from another component. It is used only as

Artificial Intelligence (AI)

Artificial intelligence refers to implementing artificial intelligence in computer devices based on mathematical models, and machine learning is a field of artificial intelligence that allows computer devices to solve specific problems through learning without explicit programming. It refers to doing something.

An artificial neural network is a model used in machine learning and refers to a model composed of artificial neurons that form a network through the combination of synapses. Artificial neural networks can be abbreviated as neural networks. An artificial neural network may include an input layer, an output layer, and optionally one or more hidden layers. Each layer contains one or more neurons, and neurons between different layers can be connected through synapses. Neurons in the input layer may have feature vectors used for learning as input signals, and neurons in the output layer output the function values of the activation function for the input signals input through the synapse, the weight of the synapse, and the bias. can do. Neurons in the hidden layer may have values calculated based on the weights of the synapse and the neuron signal of the input layer or previous hidden layer as input signals, and provide the signal to the next hidden layer or output layer.

Model parameters refer to parameters determined through learning and may include synaptic weights and neuron biases. Hyper parameters refer to parameters that must be set before learning in a machine learning algorithm and include learning rate, number of iterations, mini-batch size, initialization function, etc. The learning goal of an artificial neural network can be seen as determining model parameters that minimize the loss function. The loss function can be used as an indicator to determine optimal model parameters in the learning process of an artificial neural network.

Machine learning can be classified into supervised learning, unsupervised learning, and reinforcement learning depending on the learning method. Supervised learning refers to a method of training an artificial neural network given a label for the training data. A label is the correct answer (or result value). Unsupervised learning can refer to a method of training an artificial neural network in a state where no labels for training data are given. Reinforcement learning can refer to a learning method in which an agent defined within an environment learns to select an action or action sequence that maximizes the cumulative reward in each state.

Among artificial neural networks, machine learning implemented as a deep neural network including multiple hidden layers can be referred to as deep learning, and deep learning is a field of machine learning.

As used herein, the term “learning” or “machine learning” may include machine learning or deep learning, as well as learning based on other artificial intelligence algorithms, rule-based learning, and “AI” or “learning”. “Model” or “machine learning model” may include artificial neural network models learned based on machine learning or deep learning, as well as artificial intelligence models learned based on other artificial intelligence algorithms, and rule-based learning models.

Proposed method of the present invention

In the present invention, the user receiving the voice memo (e.g., customer) processes voice memos left by various users (e.g., customers) to classify user intent, summarize the voice memos, and provide summary content and/or voice memos according to priority. We propose a method to enable business operators to effectively identify and respond to voice memos. For example, the proposed method of the present invention can be used in services such as contact centers and call assistants based on artificial intelligence (AI), and can be used by business operators such as small business owners to avoid direct phone calls to various customers. Even when a large amount of voice memos are received, the proposed method of the present invention automatically summarizes the voice memos and provides the summarized contents to the business operator according to priority, allowing the business operator to respond efficiently without missing a single customer. It allows you to

In addition, the present invention proposes a method for generating a sentence pattern using intent classification and summary results for a voice memo, generating a sentence according to the sentence pattern, and improving speech recognition performance based on the generated sentence. Specifically, in the present invention, intent classification and entity name recognition are performed on the voice recognition results for voice memos, and sentences extracted according to sentence patterns are used using the intent classification and entity name recognition results to provide transcription data and voice for speech recognition. We propose a method to improve automatic transcription and speech recognition performance by using it as learning data for a recognition model. For example, the proposed method of the present invention can be used in services such as artificial intelligence-based contact centers, call assistants, etc., as well as services provided based on voice recognition, and small business industries are continuously created and added and user utterances are diverse. This makes it possible to easily secure voice recognition performance even in situations where voice recognition of free speech is difficult depending on the industry.

Speech recognition can be performed using a machine learning model learned based on artificial intelligence (e.g., deep neural network). In this specification, the machine learning model for speech recognition is referred to as a speech recognition model. The user's voice data can be converted into text data or sentences using a voice recognition model. For example, a speech recognition model is an acoustic model (AM) that is learned by statistically modeling the acoustic characteristics of speech data, and/or a language model (Language Model) that is learned by reflecting the characteristics of language expressions used in a specific field. Model, LM) may be included. Speech recognition may be referred to by other terms such as Speech-To-Text (STT) or Automatic Speech Recognition (ASR).

Figure 1 illustrates a functional block diagram of a voice memo service system 100 according to the proposed method of the present invention. For example, the voice memo service system 100 may be implemented in a device 1200 that provides services such as an AI contact center, AI call assistant, etc. The example in FIG. 1 does not limit the present invention and may be modified by deleting or modifying some blocks or adding new blocks.

The voice memo service system 100 processes the voice memo through voice recognition 102 and performs intent classification 110 on the voice recognition result (text) sentence to distinguish with what intention the user uttered the voice memo. Intent classification 110 may be performed using, for example, artificial neural networks (e.g., Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), etc.), language models (e.g., Bidirectional Encoder Representations from Transformers (BERT), A Lite BERT (ALBERT), etc. It can be performed using an intent classification model based on (etc.) etc.

In addition, the voice memo service system 100 performs named entity recognition 112 on the voice-recognized (text) sentence to extract sentence keywords and generates a summary text through sentence summary 114. Extract. Entity names can be used interchangeably with various terms such as keywords, key words, and main vocabulary. Entity name recognition 112 can be constructed by designing entities, building each thesaurus based on the knowledge model 120, and learning by tagging entity names in speech sentences. Entity name recognition (112) is, for example, a model using Hidden Markov Model (HMM), Support Vector Machine (SVM), Bi-directional Long Short Term Memory (Bi-LSTM) and Conditional Random Fields (CRF), and language model ( Yes, it can be performed based on KoBERT (Korean BERT), BERT, ALBERT, etc.). Sentence summary 114 is performed, for example, by combining intent classification results and entity name recognition results, or by using text summarization or generation techniques based on attention (or attention mechanism). It can be done.

The voice memo service system 100 determines the priority of the results generated or extracted in this way (116) and manages the voice memo output (118). For example, the priority determination 116 obtains the industry priority score, intent priority score, and entity name priority score from the priority list 700, as described with reference to FIGS. 7 and 8, and Based on this, the final priority score can be calculated and then the priority can be determined. For example, the voice memo output 118 may be managed in the form of a list, with a field indicating priority 410, a field indicating whether processing has been completed 420, and a field indicating work to be processed 430. , and may include at least one of the fields 440 indicating a voice memo. The memo service system 100 delivers voice memo output according to priority to the terminal, synchronizes it with the terminal, and outputs it on the terminal screen.

The voice memo service system 100 builds a knowledge model (120) and can be used to build a thesaurus for intent classification (110) and entity name recognition (112) and generate new candidate sentences (108) (e.g., Figure 5 and the related explanation of Figure 9). For example, the voice memo service system 100 may build a knowledge model (120) by developing an ontology-based open small business model.

In addition, the voice memo service system 100 verifies voice recognition reliability (104) to select voice/text sentences for automatic learning of the voice recognition model, creates a new sentence (108), and creates a machine learning model for the new sentence. By learning (106) and verifying the reliability of the new sentences (108), valid voice/text sentences can be collected (e.g., see FIG. 9 and related description). For example, the voice memo service system 100 may extract a candidate word group for creating a new sentence 108 based on the relevant industry, corresponding intention, and entity name from the knowledge model 120. The voice memo service system 100 can automatically learn a voice recognition model based on the verified and collected sentence/text data, thereby improving voice recognition performance and saving costs for transcription.

Figure 2 illustrates a flowchart of providing a voice memo service according to the proposed method of the present invention. The method illustrated in FIG. 2 may be performed in device 1200. The example of FIG. 2 is only an example and may be modified and implemented so that some components of FIG. 2 are deleted or modified, or new components not illustrated in FIG. 2 are added.

Referring to FIG. 2, the device 1200 can obtain information about intent and information about the entity name for one or more text sentences (S202). As an example, the device 1200 may perform voice recognition 102 on the user's utterance or voice data (e.g., voice memo) using a voice recognition model to obtain one or more text sentences. For convenience, the device 1200 ) may refer to the text sentence obtained through voice recognition 102 as a first text sentence. Additionally, for convenience of explanation, information about the industry, information about the intent, and information about the entity name may be abbreviated as industry information, intent information, and entity name information, respectively.

For example, in S202, the device 1200 may obtain intent information through intent classification 110 and obtain entity name information through entity name recognition 112 for the first text sentence. As a more specific example, assuming that the device 1200 acquires or receives a plurality of first text sentences, the device 1200 determines the intent classification 110 and the entity for each first text sentence among the plurality of first text sentences. By performing name recognition 112, intent information and entity name information can be obtained. Additionally, the device 1200 may obtain industry information through intent classification 110 for the first text sentence.

The device 1200 may obtain a summary text for one or more first text sentences based on the information about the intent and the information about the entity name obtained in S202 (S204). For example, the device 1200 combines the intent classification 110 result (or intent information) and the entity name recognition 112 result (or entity name information) obtained for the first text sentence in S202, or The summary text can be obtained by performing sentence summary 114, such as text summary or generating based text summary. As a more specific example, assuming that the device 1200 acquires or receives a plurality of first text sentences in S202, the device 1200 obtains intent information and entity information for each first text sentence among the plurality of first texts. By performing sentence summary 114 using name information, a summary text for the corresponding first text sentence can be obtained. Additionally, the device 1200 may additionally use industry information in addition to intent information and entity name information to obtain summary text.

Additionally, the device 1200 may determine the priority for one or more first text sentences based on the information about intent and the information about the entity name obtained in S202 (S206) (e.g., FIGS. 7 and 8, (see explanation related to Equation 1). For example, the device 1200 may determine the priority for one or more first text sentences through priority determination 116 based on intent information and entity name information. As a more specific example, assuming that the device 1200 acquires or receives a plurality of first text sentences in S202, the device 1200 obtains intent information and entity information for each first text sentence among the plurality of first texts. By performing priority determination 116 using name information, a priority score can be obtained and the priority for each first text sentence can be determined according to the priority score.

Then, the device 1200 may provide the summary text obtained in S204 to the terminal device according to the priority determined in S206 (S208). For example, the device 1200 synchronizes with the terminal device through the voice memo output management 118 so that the first text output according to priority is output on the screen of the terminal device in the same manner as the one managed by the device 1200. make it possible

Figure 3 illustrates obtaining summary text according to the proposed method of the present invention. The example in FIG. 3 is only an example to aid understanding of the present invention and does not limit the present invention.

The device 1200 acquires or receives one or more first text data 310 through voice recognition 102 and performs intent classification 110 on the one or more first text data 310 to provide information about intent. By acquiring 320 and performing entity name recognition 112, information 330 about the entity name can be obtained (e.g., see S202 in FIG. 2). In the example of FIG. 3, intent information 320 may include phone purchase, contact request, seat reservation, packaging order, and price inquiry, and entity name information 330 may include [KT] (Company), [iPhone Thirteen]. (Phone_Model), [Purchase](Action), [15th of July](Phone_No), [June 15th](date), [Window seat](Location), [Four people](Person_No), [Reservation] It can include (Order_Reservation), [Jjambbong](Item), [One piece](Item_No), [Jajangmyeon](Item), [Package order](Action), and [20,000 won](Price_Inquiry).

In addition, in order to improve the readability of the pure Korean speech recognition result 310, the device 1200 converts Korean characters into numbers, English letters, and special symbols using, for example, a text inverse conversion normalization method and produces the final speech recognition result 340. It can be obtained. In the example of FIG. 3, KT -> KT, iPhone -> Iphone, thirteen -> 13, 10 days, 20 days, 5 days a week -> 010-252-5870, June -> June, 15 days -> 15 days, four people -> 4 people, 1 -> 1, 20,000 won -> 20,000 won, and a question mark (?) can be added. Before outputting the final recognition result (340), a supplementary candidate sentence is generated by referring to the intent classification result (320) and the entity name recognition result (330), and the speech recognition result sentence (310) is compared with the supplementary candidate sentence to provide more information. The final recognition result (340) is provided as a highly reliable sentence. The sentence summary result 350 may also be corrected according to this result. This method can also be used in the proposed method of the present invention described with reference to FIGS. 9 and 10. For example, the sentence summary result 350 can be provided by combining the intent classification result 320 and the entity name recognition result 330 for reservations, orders, contact requests, etc. with clear user requirements, and in other cases. Attention (or attention mechanism)-based text summarization or generation techniques can be used.

The device 1200 provides a sentence summary for one or more first text sentences (or first text sentences converted to numbers, English letters, and special symbols) based on the information about the intention 320 and the information 330 about the entity name. 114) can be performed to obtain the summary text 350 (eg, see S204 in FIG. 2). In the example of Figure 3, the first text sentence (310) “I would like to purchase a Katie Joe iPhone 13, please call GongilgongiOiopalchigong” for the summary text (350) “[Phone purchase] Iphone13 [Please contact me] 010- You can obtain “252-5870”.

Figure 4 illustrates voice memo output according to the proposed method of the present invention. The example in FIG. 4 is merely an illustration to aid understanding of the present invention and does not limit the present invention.

Referring to FIG. 4, the device 1200 selects one or more first text sentences 310 (or numbers) through priority determination 116 based on information about intent 320 and information 330 about entity names. , the priority 410 for the converted first text sentence 340) can be determined (e.g., refer to S206 of FIG. 2, FIGS. 7 and 8, and the related description of Equation 1). The device 1200 synchronizes with the terminal device through voice memo output management 118 and outputs the voice memo output on the screen of the terminal device. For example, the voice memo output 118 may be managed in the form of a list, with a field indicating priority 410, a field indicating whether processing has been completed 420, and a field indicating work to be processed 430. , and may include at least one of the fields 440 indicating a voice memo.

For example, the field 430 indicating a task to be processed may include a summary text 350, and the field 440 indicating a voice memo may include the final voice recognition processed first text sentence (or number, English letter). , may include a first text sentence converted to a special symbol (340). The field 420 indicating whether processing is complete may be set to indicate processing completion when the terminal user processes the task or through input from the terminal user. The voice memo output may be sorted based on the field 410 indicating priority, and if the field 420 indicating whether processing is complete is set to indicate processing completion, the device 1200 determines the priority of the task. You can adjust it (lowest) or delete the entire relevant field (410, 420, 430, 440) from the list.

If the user inputs the phone number that needs to be contacted within the voice memo output, a call can be automatically connected. Simple processing matters, such as requests for parking lot guidance, can be configured so that the user can set a processing method in advance through a pre-provided administrator menu, etc. and process it immediately according to the set method.

Figure 5 illustrates a method of building an intent classification model according to the proposed method of the present invention. The example of FIG. 5 is only an example and may be modified and implemented so that some components of FIG. 5 are deleted or modified, or new components not illustrated in FIG. 5 are added.

As described with reference to FIG. 1, intent classification 110 may be performed using an intent classification model based on an artificial neural network (e.g., CNN, RNN, etc.) or a language model (e.g., BERT, ALBERT, etc.). The device 1200 collects sentences using the transcription results of the voice memo and the voice recognition results to use them as a corpus for intent classification 110 (S502). The device 1200 may initially automatically classify (S504), divide into categories, and manually label the industry/intention for each category (S508). The device 1200 can continuously collect sentences labeled with industry and intent (S510) and use them to perform classification learning (S512). The device 1200 reflects the results of classification learning (S512) to classify industry/intention again (S504), automatically labels appropriately classified sentences among unlabeled sentences (S506), and manually labels those that require adjustment (S508). )can do. As this process continues, increasingly segmented and sophisticated intent classification (110) becomes possible.

Figure 6 illustrates the results of intent classification 110 according to the proposed method of the present invention. The example in FIG. 6 is only an example to aid understanding of the present invention and does not limit the present invention.

Referring to FIG. 6, the device 1200 divides the collected sentences into categories (612, 614, 616, 618) according to industry and/or intent, and divides the collected sentences into categories (612, 614, 616, 618) by industry and/or intent. You can manage the collected sentences (622, 624, 626, 628) by labeling. The device 1200 initially automatically classifies and manages only the categories (612, 614, 616, 618) without industry or intent results, and then automatically or manually labels the industry and intent by category (612, 614, 616, 618). can do. For sentences that are not properly classified, it is possible to move between categories (612, 614, 616, 618), and the final saved sentences (622, 624, 626, 628) are automatically labeled with the current classification. The device 1200 may separately provide a management screen labeling each sentence 622, 624, 626, and 628.

Figure 7 illustrates a priority list 700 according to the proposed method of the present invention. The example in FIG. 7 is only an example to aid understanding of the present invention and does not limit the present invention. For example, in the example of Figure 7, 19 industries, 37 intention classifications, and 13 entity name classifications are illustrated, but different numbers and types of industries, intentions, and entity names are included or some industries, intentions, and entity names are excluded. It can be modified and implemented to do so. The priority list 700 may be managed in the form of a list or table and may also be referred to as a priority score list or table.

Referring to FIG. 7, the priority list 700 includes a first list 710 for industry priority (or priority score), a second list 720 for intention priority (or priority score), and an entity It may include at least one of the third list 730 for name priority (or priority score). Additionally or independently, weights may be set for the first list 710, the second list 720, and the third list 730, respectively. In the example of FIG. 7, the weights are set to 0.01 for the first list 710, 1 for the second list 720, and 0.1 for the third list 730, but the present invention is limited to these weights. This is not true and can be set to various values. For convenience of explanation, the industry priority score may be referred to as a first priority score, the intent priority score may be referred to as a second priority score, and the entity name priority score may be referred to as a third priority score.

Device 1200 provides lists 700, 710, 720, and 730 to the user through a user interface and allows the user to add or change industry, intent, entity name, and adjust priority scores and/or weights. The lists 700, 710, 720, and 730 can be continuously expanded and supplemented as services expand, user usage increases, and speech patterns increase. Additionally, the weight can be set independently for industry, intent, and entity name, and the priority score and weight can be used to determine priority 116 (e.g., see S206 in FIG. 2).

Figure 8 illustrates priority determination 116 according to the proposed method of the present invention. The example in FIG. 8 is only an example to aid understanding of the present invention and does not limit the present invention. For example, the order of industry classification (S804), intent classification (S806), and entity name recognition (S808) can be modified in various ways.

Referring to FIG. 8, the device 1200 obtains industry classification results (or information about industry classification) for the voice recognition results (or first text sentences) (S802) for three voice memos and determines industry priority ( Alternatively, an industry priority score may be obtained by referring to the first list 710 (S804). The device 1200 obtains an intent classification result (or information about intent) for the voice recognition result (or first text sentence) S802 and creates a second list 720 for intent priority (or priority score). An intent priority score can be obtained by referring to (S806), an entity name recognition result (or information about the entity name) is obtained, and a third list 730 for entity name priority (or priority score) is obtained. An entity name priority score can be obtained (S808) by reference (e.g., see S202 in FIG. 2).

As described with reference to FIG. 3, the device 1200 obtains the final voice recognition result by converting numbers, English letters, and special symbols (S810), and obtains a summary result (or summary text) through sentence summary 114. (S812) can be done (e.g., see S204 in FIG. 2). The device 1200 may calculate the final priority score (S814) and determine the priority (S816) based on the industry priority score, intent priority score, and entity name priority score. As an example, the priority may be determined in descending order of priority score (S816). As another example, the priority may be determined in the order of the priority score (S816).

For example, the device 1200 may obtain the final priority score (S814) based on Equation 1. Specifically, the device 1200 is based on Equation 1: the industry priority score (or first priority score) obtained in S804, the intent priority score (or second priority score) obtained in S806, and the The weights (e.g., 0.01, 1, 0.1) set in the respective lists 710, 720, and 730 are applied to the entity name priority score (or third priority score), and the summed value is determined as the final priority score ( S814), the priority for the voice-recognized text sentence (or first text sentence) (S802) may be determined (S816) based on the final priority.

[Equation 1]

Final Priority = (Industry Priority Score)*(Industry Weight) + (Intent Priority Score)*(Intent Weight) + (Entity Name Priority Score)*(Entity Name Weight)

When a plurality of entity names appear in one text sentence (or first text sentence) (S802), the device 1200 may determine the final priority by applying the (entity name) weight only to the entity name with the highest priority. there is. The reason for doing this is to increase the ranking of sentences with important keywords such as date and time. It is also possible to increase the weight of the actual word corresponding to each entity name within the sentence. For example, in the example of Figure 7, the entity names Date or Time have a priority of 2, but if the actual word related to them is “today” or “now,” the priority can be increased by adjusting the priority to “1”. there is.

Figure 9 illustrates a flowchart of automatically generating training data for learning a voice recognition model according to the proposed method of the present invention. The method illustrated in FIG. 9 may be performed in device 1200. The example of FIG. 9 is only an example and may be modified and implemented so that some components of FIG. 9 are deleted or modified, or new components not illustrated in FIG. 9 are added.

The method illustrated in Figure 9 generates a new sentence using a sentence pattern for a text sentence with low voice recognition reliability and uses this to backtrack the actual utterance content of the user's voice data (e.g., voice memo) to improve voice recognition performance. and automatically derive learning data for learning the voice recognition model.

According to the proposed method of the present invention, if the voice recognition reliability for a word/sentence in a text sentence obtained by performing voice recognition 102 on user voice data (e.g., voice memo) exceeds a certain threshold, it is considered valid. Text sentences are collected as training data and used to train a speech recognition model. If the voice recognition reliability does not exceed a certain threshold, a sentence pattern is created according to the industry/intention classification result and the entity name recognition result, and all possible candidate words are combined to create a new sentence according to the sentence pattern. Create a small speech recognition model for speech recognition and perform speech recognition again on the corresponding utterance to verify the reliability value for the word/sentence. The step of verifying the reliability value for the word/sentence is executed for each speech recognition model for new sentences and among the new sentences. The sentence with the highest reliability is selected as the sentence for final transcription. For example, reliability measurement uses methods such as Bayes Risk. The sentences selected in this way can also be used in the final voice recognition results for voice memos. This is because voice memos may provide results with some delay rather than in real time.

Since candidate words are selected from a continuously updated knowledge model, they can effectively respond to newly released menus or new words, and can also be used to learn voice recognition models.

Referring to FIG. 9, the device 1200 may obtain one or more text sentences (or first text sentences) based on the user's voice data (e.g., voice memo) using the first voice recognition model (S902) ). Additionally, the device 1200 may verify the voice recognition reliability for each first text sentence by performing reliability measurement on a word and/or sentence basis for one or more first text sentences (S904). For example, reliability measurement may be performed based on methods such as Bayes Risk.

The device 1200 may determine whether the voice recognition reliability for one or more first text sentences satisfies at least one first criterion (S906). For example, the at least one first criterion may include the speech recognition reliability for a sentence and the speech recognition reliability for a word, and if the speech recognition reliability for a word/sentence is greater than a certain threshold, the at least one first criterion may be selected. It can be determined that it satisfies, and if it is below a certain threshold, it can be determined that it does not satisfy at least one first criterion (or, conversely, if it is below a certain threshold, it can be determined that it satisfies at least one first standard, and if it is below a certain threshold, it can be determined that it satisfies at least one first standard). If so, it is also possible to determine that at least one first criterion is not satisfied). As a more specific example, failing to meet at least one first criterion may include speech recognition confidence for a sentence being less than a first threshold or speech recognition confidence for a word being less than a second threshold, and at least one Satisfying the first criterion may include that the speech recognition reliability for the sentence is greater than or equal to the first threshold and the speech recognition reliability for the word is greater than or equal to the second threshold.

If the voice recognition reliability for one or more first text sentences does not satisfy at least one first criterion (S906), the device 1200 provides information about the intention (or intention) obtained for the one or more first text sentences. information) and information about the entity name (or entity name information) (e.g., see S202 in FIG. 2) (S908), obtain at least one entity name candidate word from the knowledge model 912 (S910), and at least One or more text sentences can be generated according to a new sentence pattern based on one entity name candidate word (S912). In this specification, a text sentence generated according to a new sentence pattern based on at least one entity name candidate word may be referred to as a second text sentence.

The device 1200 may train a small speech recognition model based on one or more second text sentences (S914). For convenience, in this specification, the speech recognition model used for speech recognition in S902 is referred to as a first speech recognition model, and a small speech recognition model learned based on one or more second text sentences is referred to as a second speech recognition model. refers to For example, the second speech recognition model uses the acoustic model (AM) of the first speech recognition model, but creates a new language model (LM) and uses the second text sentence to create a new language model (LM). It may be a trained model. As a more specific example, a second speech recognition model may be generated for each second text sentence, and each second text sentence may be used to train the corresponding second speech recognition model (language model). Alternatively, the second speech recognition model may be generated for one or more second text sentences, and the second speech recognition model (language model) may be trained using the one or more second text sentences.

The device 1200 may verify the voice recognition reliability for each second text sentence by performing a reliability measurement on a word and/or sentence basis for one or more second text sentences using the learned second speech recognition model. (S916). For example, reliability measurements can be performed based on methods such as Bayes Risk, etc.

The device 1200 may determine whether the voice recognition reliability for one or more second text sentences satisfies at least one second criterion (S918). For example, the at least one second criterion may include the speech recognition reliability for a sentence and the speech recognition reliability for a word, and if the speech recognition reliability for a word/sentence is above a certain threshold, the at least one second criterion may be selected. It can be determined that it satisfies, and if it is below a certain threshold, it can be determined that at least one second standard is not satisfied (or, conversely, if it is below a certain threshold, it can be determined that it satisfies at least one second standard, and if it is below a certain threshold, it can be determined that it satisfies at least one second standard). If so, it is also possible to determine that at least one second criterion is not satisfied). As a more specific example, failing to meet at least one second criterion may include speech recognition confidence for a sentence being less than a third threshold or speech recognition confidence for a word being less than a fourth threshold, and at least one Satisfying the second criterion may include that the speech recognition reliability for the sentence is greater than or equal to the third threshold and the speech recognition reliability for the word is greater than or equal to the fourth threshold.

When the speech recognition reliability for one or more second texts satisfies at least one second criterion, the device 1200 selects the text sentence with the highest speech recognition reliability among the one or more second text sentences as a first speech recognition model. (S920), and the first voice recognition model can be trained based on the second text sentence with the highest voice recognition reliability (S922). In addition, when the speech recognition reliability for one or more second texts satisfies at least one second criterion, the second text sentence with the highest speech recognition reliability among the one or more second text sentences is the final text for the user's speech data. It can be determined or used as a result of voice recognition. If the speech recognition reliability for one or more second texts does not satisfy at least one second criterion, collecting one or more second text sentences as training data and training the first speech recognition model based on them may be omitted. There is (S924).

In addition, for example, in the process of verifying using a small speech recognition model (or a second speech recognition model), a newly generated sentence (or a second speech recognition reliability) that is not higher than the existing speech recognition reliability (i.e., the speech recognition reliability in S904) text sentences) are not reflected in the final speech recognition result, and are not collected as training data for the first speech recognition model. Likewise, even if the existing speech recognition reliability (i.e., the speech recognition reliability in S904) does not satisfy at least one first criterion (S906), one or more first text sentences are collected as training data for the first speech recognition model. and do not use

Meanwhile, when applying the proposed method of the present invention, if the user wants to confirm the accuracy of word recognition, the word can be expressed as a link on the user terminal screen and the user can directly edit the word. In this case, it can be used more effectively and usefully for collecting new words such as new menus. Words that provide links are selected based on a combination of intent, entity name, and reliability.

Figure 10 illustrates generating a sentence pattern and a new sentence according to the proposed method of the present invention. The example in FIG. 10 is only an example to aid understanding of the present invention and does not limit the present invention.

Referring to FIG. 10, the device 1200 performs entity name recognition 112 on one or more first text data 310 obtained from the user's voice data (e.g., voice memo), and as a result, Month, Date, and Time , Assume that information 330 about the entity name, including Location, Person_No, and Reservation, has been obtained. The device 1200 may generate a new sentence pattern 1010 based on the information 330 about the entity name.

In addition, the device 1200 selects entity name candidate words (1022, 1024, 1026, 1028, 1030) and combining the entity name candidate words (1022, 1024, 1026, 1028, 1030) according to the new sentence pattern (1010) to generate a new sentence (or one or more second text sentences). The device 1200 can use these combined new sentences to learn a small speech recognition model (or a second speech recognition model) (for each new sentence or for new sentences).

In order to learn a speech recognition model (e.g., an acoustic model) using an actual utterance, it is necessary to verify whether the utterance is valid. According to the proposed method of the present invention, the sentence most similar to the user's voice data can be found from a combination of new sentence patterns. The device 1200 creates a pattern file that creates a sentence composed of keywords and a category file that categorizes words that can be included in each item of the previously created pattern, which can be used to create sentence patterns and new sentences. there is.

According to the proposed method of the present invention, new sentences can be created and training data can be provided, which is significantly advantageous for the rapid expansion of industries and speech patterns in services such as AI contact centers and AI call assistants, thereby enabling improvement in voice recognition performance. I will do it. Additionally, the newly created sentence can also be used as the final speech recognition result, further improving the performance of the speech recognition model.

Figure 11 illustrates creating a small speech recognition model and generating a new sentence according to the proposed method of the present invention. Specifically, the device 1200 creates a small speech recognition model (or second speech recognition model) based on the speech recognition result and reliability, verifies the reliability, and selects the final text sentence (e.g., satisfies at least one second criterion and is the most A second text sentence with high voice recognition reliability can be selected (e.g., see S908 to S918 of FIG. 9). The example in FIG. 11 is only an example to aid understanding of the present invention and does not limit the present invention.

Referring to FIG. 11, it is assumed that the user voice data includes “Today at 2 o’clock, 4 servings of rose tteokbokki are packaged” and the first text sentence for this is voice recognized as “Today at 2 o’clock, 4 servings of spicy tteokbokki are packaged” (1110). do. In other words, unlike the user voice data, we assume that “Rosé Tteokbokki” was incorrectly recognized as “Spicy Tteokbokki.” Additionally, it is assumed that the device 1200 measures the voice recognition reliability for a word/sentence and determines the sentence reliability to be 0.838, and the word reliability of the incorrectly recognized word “spicy tteokbokki” to 0.3.

In this case, if the device 1200 sets the sentence reliability threshold (or first threshold) for the first criterion to 0.9 and the word reliability threshold (or second threshold) to 0.5, 1 The voice recognition reliability of the text sentence does not satisfy at least one first criterion, and the device 1200 checks the industry/intention classification results and entity name recognition results to generate sentences for learning the second voice recognition model. And you can create new sentence patterns and new sentences accordingly. In the example of Figure 11, a sentence pattern is created using the entity names <Date>, <Time>, <Item>, <Item_No>, and <Action>, and the entity name candidate words “Bomb Rice Cake” and “Roje Rice Cake” are obtained. Then, combine the entity name candidate words according to the sentence pattern to create a new sentence (or second text sentence) “Today at 2 o’clock, bomb tteokbokki is packaged for 4 people” (1120), “Today at 2 o’clock, rose tteokbokki is packaged for 4 people” (1130) can be created.

In this example, the device 1200 says “Today at 2 o’clock, spicy tteokbokki for 4 servings” (1110), “Today at 2 o’clock, bomb tteokbokki for 4 servings” (1120), “Today at 2 o’clock, rose tteokbokki for 4 servings” Reliability verification for words/sentences is performed again by creating a small speech recognition model (or second speech recognition model) based on the three sentences (1130). As described previously, the small speech recognition model (or A second speech recognition model) can be created for each of the three sentences and learned using each sentence, or one small speech recognition model (or a second speech recognition model) can be created for each of the three sentences. It can also be created by learning using three sentences: “Today at 2 o’clock, bomb tteokbokki is packaged for 4 people” (1120) is eliminated because it has the lowest word/sentence reliability, while the most reliable sentence is “Today at 2 o’clock, rose tteokbokki is packaged for 4 people.” “Iyo” (1130) is selected as the sentence and/or final speech recognition result for learning the final transcription and speech recognition model (or first speech recognition model). The device 1200 verifies reliability using the second speech recognition model. The sentence 1130 can be determined as the final voice recognition result, collected as training data for the first voice recognition model, and used for training the first voice recognition model.

As described above, according to the proposed method of the present invention, when a customer leaves a voice memo in free speech to a small business owner through a voice memo service such as an AI call assistant, the voice memo is summarized and provided to the small business owner in order of priority. Customer response efficiency can be improved by allowing small business owners to easily determine whether a voice memo requires a quick response or a voice memo that can be responded to slowly.

In addition, according to the proposed method of the present invention, in a small business voice memo service where the industry is continuously updated and user utterances dynamically change accordingly, the intent and summary results classified by the system using the voice recognition results are used to automatically By extracting sentence patterns and using them for automatic transcription and speech recognition model learning, speech recognition performance can be improved and high-quality and smooth service can be provided. According to the proposed method of the present invention, in order to ensure the quality of automatic transcription, the validity of newly created patterns or sentences is immediately verified and the verified sentence corpus is used to learn a speech recognition model, thereby improving and maintaining the recognition rate for new sentences. It can be. Additionally, it provides highly readable text by applying text normalization techniques to convert numbers, English letters, and special symbols.

According to the proposed method of the present invention, when a customer calls a small business owner and is not immediately connected, he or she can comfortably leave a verbal voice memo, and the small business owner can efficiently respond to customers based on the summarized content according to the proposed method of the present invention. can do. Depending on the summarized results, simple inquiries or requests such as parking lot information or menu information can be automatically responded to using a pre-created process. Urgent requests, such as phone contact requests and delivery confirmation, are given higher priority so they can be provided to small business owners and processed quickly.

In addition, according to the proposed method of the present invention, the voice recognition results provided in pure Korean due to the influence of large-scale learning data are converted into numbers, English letters, and special symbols with which the user is familiar to make them readable, thereby ensuring the readability of the voice memo left by the customer. As a result, the convenience of small business owners can also be improved.

In addition, according to the proposed method of the present invention, it is possible to secure voice recognition quality by extracting sentence patterns from voice recognition results using classified intent and summary results and using them for automatic transcription and voice recognition model learning.

Device to which the proposed method of the present invention can be applied

Figure 12 illustrates a device 1200 to which the proposed method of the present invention can be applied.

Referring to FIG. 12, device 1200 may be configured to implement the proposed method of the present invention. As an example, the device 1200 may be configured to process voice data (e.g., voice memo, etc.), and for this purpose, sentence summary 114 is performed on one or more first text data according to the method proposed by the present invention to create a summary text. It may be configured to obtain and provide summary text to the terminal device according to a priority determined based on at least one of industry, user intent, and entity name. In addition or instead of this, the device 1200 generates a sentence pattern and a new sentence according to at least one of industry, user intention, and entity name according to the proposed method of the present invention, and uses the generated sentence to create a second voice recognition model. It can be configured to improve voice recognition performance by learning, obtaining voice recognition results, and learning the first voice recognition model. For example, device 1200 may include a network device, a server device, or a terminal device.

For example, devices 1200 to which the proposed method of the present invention can be applied include network devices such as repeaters, hubs, bridges, switches, routers, gateways, computer devices such as desktop computers, workstations, etc., and mobile terminals such as smartphones. , portable devices such as laptop computers, home appliances such as digital TVs, and means of transportation such as cars. As another example, the device 1200 to which the present invention can be applied may be included as part of an Application Specific Integrated Circuit (ASIC) implemented in the form of a System On Chip (SoC).

The memory 1204 may store programs and/or instructions for processing and controlling the processor 1202, and may store data and information used in the present invention, control information, data and information required for processing data and information according to the present invention. Temporary data generated during information processing can be stored. The memory 1204 includes Read Only Memory (ROM), Random Access Memory (RAM), Erasable Programmable Read Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), flash memory, and Static RAM (SRAM). , can be implemented as a storage device such as a hard disk drive (HDD), solid state drive (SSD), etc.

Processor 1202 controls the operation of each module within device 1200. In particular, the processor 1202 can perform various control functions to perform the proposed method of the present invention. The processor 1202 may also be called a controller, microcontroller, microprocessor, microcomputer, etc. The proposed method of the present invention may be implemented by hardware, firmware, software, or a combination thereof. When implementing the present invention using hardware, an application specific integrated circuit (ASIC) or a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), or an FPGA (FPGA) configured to perform the present invention is used. A field programmable gate array) may be provided in the processor 1202. Meanwhile, when the proposed method of the present invention is implemented using firmware or software, the firmware or software includes instructions related to modules, procedures, or functions that perform the functions or operations necessary to implement the proposed method of the present invention. It may include, and the instructions are stored in the memory 1204 or in a computer-readable recording medium (not shown) separate from the memory 1204, and when executed by the processor 1202, the device 1200 uses the proposed method of the present invention. It can be configured to implement.

Additionally, device 1200 may include a network interface module (NIM) 1206. The network interface module 1206 is operatively connected to the processor 1202, and the processor 1202 controls the network interface module 1206 to provide information and/or data, signals, and messages through a wireless/wired network. It can transmit or receive wireless/wired signals carrying lights. The network interface module 1206 supports various communication standards, such as IEEE 802 series, 3GPP LTE(-A), 3GPP 5G, etc., and can transmit and receive control information and/or data signals according to the corresponding communication standards. Network interface module 1206 may be implemented outside of device 1200 as needed.

The embodiments described above combine the components and features of the present invention in a predetermined form. Each component or feature should be considered optional unless explicitly stated otherwise. Each component or feature may be implemented in a form that is not combined with other components or features. Additionally, it is also possible to configure an embodiment of the present invention by combining some components and/or features. The order of operations described in embodiments of the present invention may be changed. Some features or features of one embodiment may be included in other embodiments or may be replaced with corresponding features or features of other embodiments. It is obvious that claims that do not have an explicit reference relationship in the patent claims can be combined to form an embodiment or included as a new claim through amendment after filing.

The present invention can be applied to various devices such as network devices, server devices, terminal devices, etc. configured to provide voice-based services, including artificial intelligence contact centers, artificial intelligence call services, etc.

100: Voice memo service system 102: Voice recognition
104: Verification of voice recognition reliability
106: Learning a speech recognition model for new sentences
108: New sentence creation/verification 110: Intent classification
112: Entity name recognition 114: Sentence summary
116: Prioritization 118: Voice memo output management
120: Building a knowledge model
310: One or more first text data
320: Information about intent 330: Information about entity name
340: Final voice recognition result 350: Summary text
410: Field indicating priority 420: Field indicating whether processing is complete
430: Field indicating work to be done 440: Field indicating voice memo
612, 614, 616, 618: Categories based on industry and/or intent.
622, 624, 626, 628: Sentences classified by category
700: Priority list
710: First list for industry priorities (or priority scores)
720: Second list for intent priorities (or priority scores)
730: Third list for entity name priority (or priority score)
1010: New sentence pattern
1022, 1024, 1026, 1028, 1030: Entity name candidate words
1110: first text sentence 1120, 1130: second text sentence
1200: Device 1202: Processor
1204: memory 1206: network interface module

Claims (14)

A method of processing voice data in a server device, comprising:
obtaining information about intent and information about named entities for one or more first text sentences;
obtaining summary text for the one or more first text sentences based on the information about the intent and the information about the entity name;
determining a priority for the one or more first text sentences based on the information about the intent and the information about the entity name; and
A method comprising providing the summary text to a terminal device according to the determined priority. In claim 1,
Obtaining the summary text includes:
Method comprising combining information about the intent and information about the entity name to obtain the summary text. In claim 1,
These priorities determine:
obtaining a second priority score of the information about the intent and a third priority score of the information about the entity name in a priority list; and
A method comprising determining the priority based on a sum of the second priority score and the third priority score by applying weights to each of them. In claim 1,
Further comprising obtaining information about industry classification for the one or more first text sentences,
These priorities determine:
Obtaining a first priority score of the information about the industry classification, a second priority score of the information about the intention, and a third priority score of the information about the entity name in the priority list; and
A method comprising determining the priority based on a value obtained by applying a weight to each of the first priority score, the second priority score, and the third priority score. In claim 1,
obtaining the one or more first text sentences based on the user's speech data using a first speech recognition model;
Based on the speech recognition reliability for the one or more first text sentences not satisfying at least one first criterion, a sentence pattern and at least one entity name based on the information about the intent and the information about the entity name Obtaining candidate words and generating one or more second text sentences based on the at least one entity name candidate word according to the obtained sentence pattern; and
Based on the speech recognition reliability of the one or more second text sentences satisfying at least one second criterion, the second text sentence with the highest speech recognition reliability among the one or more second text sentences is selected as the user's speech data. A method further comprising determining with a speech recognition result. In claim 5,
It further includes learning a second speech recognition model for each of the one or more second text sentences using each of the one or more second text sentences,
The method wherein speech recognition reliability for each of the one or more second text sentences is obtained based on the second speech recognition model. In claim 5,
The voice recognition reliability includes reliability of sentences and reliability of words,
The speech recognition reliability of the one or more first text sentences does not satisfy the at least one first criterion. This means that the speech recognition reliability of the one or more first text sentences is less than a first threshold or the one or more first text sentences do not satisfy the first criterion. A method, wherein the confidence level for a word in a text sentence is less than a second threshold. In claim 5,
The voice recognition reliability includes reliability of sentences and reliability of words,
Speech recognition reliability of the one or more second text sentences satisfies at least one second criterion, meaning that the reliability of the sentences of the one or more second text sentences is greater than or equal to a third threshold and that the speech recognition reliability of the one or more second text sentences is greater than or equal to a third threshold. wherein the confidence level for is greater than or equal to a fourth threshold. In claim 5,
Based on the speech recognition reliability of the one or more second text sentences satisfying the at least one second criterion, learning the first speech recognition model using the second text sentence with the highest speech recognition reliability. Including more,
Based on the speech recognition reliability of the one or more second text sentences not satisfying the at least one second criterion, training of the first speech recognition model using the second text sentence with the highest speech recognition reliability is omitted. In claim 5,
It further includes determining the one or more first text sentences as a speech recognition result for the user's speech data, based on the speech recognition reliability of the one or more first text sentences satisfying the at least one first criterion. How to. processor; A device comprising: and a memory,
The memory includes instructions configured to implement specific operations that, when performed by the processor, cause the device to process voice data, the specific operations being:
obtaining information about intent and information about named entities for one or more first text sentences;
obtaining summary text for the one or more first text sentences based on the information about the intent and the information about the entity name;
determining a priority for the one or more first text sentences based on the information about the intent and the information about the entity name; and
A device comprising providing the summary text to a terminal device according to the determined priority. In claim 11,
The specific operations above are:
obtaining the one or more first text sentences based on the user's speech data using a first speech recognition model;
Based on the speech recognition reliability for the one or more first text sentences not satisfying at least one first criterion, a sentence pattern and at least one entity name based on the information about the intent and the information about the entity name Obtaining candidate words and generating one or more second text sentences based on the at least one entity name candidate word according to the obtained sentence pattern; and
Based on the speech recognition reliability of the one or more second text sentences satisfying at least one second criterion, the second text sentence with the highest speech recognition reliability among the one or more second text sentences is selected as the user's speech data. The device further includes determining based on voice recognition results. A computer-readable storage medium storing instructions configured to, when executed by a processor, cause a device including the processor to implement specific operations for processing voice data, the specific operations comprising:
obtaining information about intent and information about named entities for one or more first text sentences;
obtaining summary text for the one or more first text sentences based on the information about the intent and the information about the entity name;
determining a priority for the one or more first text sentences based on the information about the intent and the information about the entity name; and
A computer-readable storage medium comprising providing the summary text to a terminal device according to the determined priority. In claim 13,
The specific operations above are:
obtaining the one or more first text sentences based on the user's speech data using a first speech recognition model;
Based on the speech recognition reliability for the one or more first text sentences not satisfying at least one first criterion, a sentence pattern and at least one entity name based on the information about the intent and the information about the entity name Obtaining candidate words and generating one or more second text sentences based on the at least one entity name candidate word according to the obtained sentence pattern; and
Based on the speech recognition reliability of the one or more second text sentences satisfying at least one second criterion, the second text sentence with the highest speech recognition reliability among the one or more second text sentences is selected as the user's speech data. A computer-readable storage medium further comprising determining based on a voice recognition result for .

Images