KR102335048B1 - User device, method and server for providing voice recognition service - Google Patents

Abstract

A user terminal providing a voice recognition service includes a query word extractor that extracts a query word and a service domain from text for a voice command, and a query word expander that generates at least one extended query word by expanding the query word based on the extracted service domain , determine at least one database for candidate search corresponding to the extracted service domain, search for at least one candidate for each of the at least one extended query word from the determined at least one database, and the at least one candidate A candidate extension unit generating at least one extension candidate by expanding and a service providing unit providing a service for the voice command based on the selected final candidate.

Description

User terminal, method, and server for providing voice recognition service

The present invention relates to a user terminal, method and server for providing a voice recognition service.

An intelligent personal assistant is a software agent that processes tasks requested by users and provides specialized services to users. The intelligent personal assistant collects and provides customized information to the user based on artificial intelligence (AI) engine and voice recognition, and performs various functions such as scheduling, sending e-mail, and restaurant reservation according to the user's voice command. Improves convenience.

These intelligent personal assistants are mainly provided in the form of customized personal services on smartphones.

When a user utters a query to receive a service desired by the user, the intelligent personal assistant performs voice recognition on the uttered query, analyzes the recognized query, and provides a service related to the query.

However, the intelligent personal assistant may misrecognize the user's voice depending on the surrounding noise and speech type during voice recognition. That is, there is a problem of providing a service completely different from the user's intention by misrecognizing the user's voice.

Korean Patent Publication No. 2016-0071111 (published on June 21, 2016)

The present invention is to solve the above problem, by extracting a service domain from a user's voice command and performing voice recognition by expanding a query word and a candidate based on the extracted service domain, thereby misrecognizing a voice command uttered by the user. Even so, it is an object to provide a user terminal, a method, and a server providing a voice recognition service for providing a voice recognition service that provides a service intended by a user.

However, the technical task to be achieved by the present embodiment is not limited to the technical task as described above, and other technical tasks may exist.

As a technical means for achieving the above-described technical problem, according to an embodiment of the present invention, a query word extractor for extracting a query word and a service domain from a text for a voice command, and the query word is expanded based on the extracted service domain a query term extension unit generating at least one extended query term by A candidate extension unit generating at least one extension candidate by searching for one candidate and expanding the at least one candidate, calculating a similarity between the at least one extension query word, and the at least one extension candidate, and the calculated similarity It is possible to provide a user terminal including a final candidate selector that selects a final candidate based on , and a service provider that provides a service for the voice command based on the selected final candidate.

According to another embodiment of the present invention, extracting a query word and a service domain from text for a voice command, generating at least one extended query word by expanding the query word based on the extracted service domain, the extracted determining at least one database for candidate search corresponding to the service domain; searching for at least one candidate for each of the at least one extended query word from the determined at least one database; expanding the at least one candidate generating at least one extension candidate by performing It is possible to provide a method for providing a voice recognition service including providing a service for the voice command based on the voice command.

According to another embodiment of the present invention, a voice command receiving unit that receives a voice command from a user terminal and converts the voice command into text through voice recognition for the received voice command, a query word and a service from the converted text a query word extracting unit for extracting a domain, a query term extension unit generating at least one extended query word by expanding the query word based on the extracted service domain, and at least one database for candidate search corresponding to the extracted service domain. a candidate extension unit that determines at least one extension candidate, searches for at least one candidate for each of the at least one extension query term from the determined at least one database, and expands the at least one candidate to generate at least one extension candidate, the at least one a final candidate selector that calculates a similarity between the extended query word of and the at least one extended candidate, selects a final candidate based on the calculated similarity, and executes the service for the voice command based on the selected final candidate It is possible to provide a server including a service control command transmission unit for transmitting a service control command to the user terminal.

According to any one of the above-described problem solving means of the present invention, even if the user's voice command is misrecognized, a user providing a voice recognition service that allows the user to provide a desired service through a query word and a candidate extended based on the service domain It is possible to provide a terminal, a method, and a server for providing a voice recognition service.

1 is a block diagram of a system for providing a voice recognition service according to an embodiment of the present invention.
2 is a block diagram of a user terminal according to an embodiment of the present invention.
3A to 3C are exemplary diagrams for explaining a process of providing a service related to a first service domain according to an embodiment of the present invention.
4A to 4C are exemplary diagrams for explaining a process of providing a service related to a second service domain according to an embodiment of the present invention.
5A to 5C are exemplary diagrams for explaining a process of providing a service related to a third service domain according to an embodiment of the present invention.
6 is a flowchart of a method for providing a voice recognition service in a user terminal according to an embodiment of the present invention.
7 is a block diagram of a voice recognition service providing server according to an embodiment of the present invention.
8 is a flowchart of a method for providing a voice recognition service in a voice recognition service providing server according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. . Also, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

In this specification, a "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. In addition, one unit may be implemented using two or more hardware, and two or more units may be implemented by one hardware. Meanwhile, '~ unit' is not limited to software or hardware, and '~ unit' may be configured to be in an addressable storage medium or to reproduce one or more processors. Accordingly, as an example, '~' indicates components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. The functions provided in the components and '~ units' may be combined into a smaller number of components and '~ units' or further separated into additional components and '~ units'. In addition, components and '~ units' may be implemented to regenerate one or more CPUs in a device.

1 is a block diagram of a system for providing a voice recognition service according to an embodiment of the present invention. Referring to FIG. 1 , a system for providing a voice recognition service 1 may include a user terminal 110 and a server for providing a voice recognition service 120 .

The user terminal 110 and the voice recognition service providing server 120 exemplarily show components that can be controlled by the voice recognition service providing system 1, and the voice recognition service providing system 1 of FIG. ) is only one embodiment of the present invention, so the present invention is not limitedly interpreted through FIG. 1, and may be configured differently from FIG. 1 according to various embodiments of the present invention.

Each component of the voice recognition service providing system 1 of FIG. 1 is generally connected through a network. For example, as shown in FIG. 1 , the user terminal 110 may be simultaneously connected to the voice recognition service providing server 120 at a time interval.

A network refers to a connection structure in which information can be exchanged between each node, such as terminals and servers, and includes a local area network (LAN), a wide area network (WAN), and the Internet (WWW: World). Wide Web), wired and wireless data communication networks, telephone networks, wired and wireless television networks, and the like. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasound Communication, Visible Light Communication (VLC), LiFi, and the like are included, but are not limited thereto.

The user terminal 110 may include all terminals providing a voice recognition service including, for example, a voice recognition speaker, a smart phone, a vehicle speaker, a tablet PC, and the like.

The user terminal 110 may receive a voice command and convert the voice command into text through voice recognition for the received voice command. For example, the user terminal 110 may convert the received voice command into text through STT (Speak to Text).

The user terminal 110 may extract a query word and a service domain from the text for the voice command. A query language is a command related to a service that a user wants to execute, and may include an entity that allows the user to obtain an intent of a service that the user wants to execute and parameters for executing a requested function. A service domain is an area of a specific service and may be defined by properties of information utilized in a specific service. That is, the service domain may be used to identify a subject related to a voice command input from the user. For example, when the user terminal 110 receives a voice command "Play IU's song", the user terminal 110 extracts the service domain from the text for the voice command as 'music', and The query word can be extracted as 'IU-music playback'.

Here, the service domain includes a first service domain corresponding to a call service, a second service domain corresponding to a text message service, a third service domain corresponding to an instant message service, and a fourth service domain corresponding to a content search service. can do. The service domain may be variously configured in addition to the first to fourth service domains, but is not limited thereto.

The user terminal 110 extracts the service domain related text related to the service domain from the text, and based on the extracted service domain related text, any one of the first service domain, the second service domain, the third service domain, and the fourth service domain can be extracted. Here, the service domain related text may be extracted by comparing it with keywords of the service domain stored in the database. A detailed description thereof will be provided later.

The user terminal 110 may generate at least one extended query by expanding the query based on the extracted service domain. Here, the user terminal 110 may expand the query word in a different way for each type of the extracted domain.

The user terminal 110 determines at least one database for candidate search corresponding to the extracted service domain, searches for at least one candidate for each of at least one extended query word from the determined at least one database, and at least one At least one extension candidate may be generated by expanding the candidate. Here, the candidate may be a keyword that is the same as or similar to the extracted query word. Even when a query word is misrecognized by searching for a candidate, a service can be provided according to a user's intention by increasing the accuracy of voice recognition by searching for a candidate similar to the query word.

The user terminal 110 may calculate a similarity between at least one extended query word and at least one extended candidate, and select a final candidate based on the calculated similarity.

To this end, the user terminal 110 may convert the at least one extension query term and the at least one extension candidate into at least one transformation format for comparison between the at least one extension query term and the at least one extension candidate.

Here, the at least one conversion format may include a keystroke, a phonetic symbol, a character string, and the like. Thereafter, the user terminal 110 calculates a similarity of each of the at least one extended query term and each of at least one extension candidate corresponding to each of the at least one extended query term, and based on the similarity of each of the at least one extension candidate, a predetermined number of Similar candidates can be selected. In this case, the user terminal 110 may select a final candidate from among a predetermined number of similar candidates based on different criteria for each type of the extracted domain.

The user terminal 110 may provide a service for a voice command based on the selected final candidate.

According to another embodiment of the present invention, the user terminal 110 transmits a voice command to the voice recognition service providing server 120 , and thus may be performed by the voice recognition service providing server 120 . Hereinafter, a corresponding embodiment will be described.

The voice recognition service providing server 120 may receive a voice command from the user terminal 110 and convert the voice command into text through voice recognition for the received voice command.

The voice recognition service providing server 120 may extract a query word and a service domain from the converted text. Here, the service domain includes a first service domain corresponding to a call service, a second service domain corresponding to a text message service, a third service domain corresponding to an instant message service, and a fourth service domain corresponding to a content search service. can do.

For example, the voice recognition service providing server 120 extracts a service domain related text related to a service domain from the text, and based on the extracted service domain related text, a first service domain, a second service domain, a third service domain and Any one of the fourth service domains may be extracted.

The voice recognition service providing server 120 may generate at least one extended query by expanding the query based on the extracted service domain. Here, the voice recognition service providing server 120 may expand the query word in a different way for each type of the extracted domain.

The voice recognition service providing server 120 determines at least one database for candidate search corresponding to the extracted service domain, and searches for at least one candidate for each of at least one extended query word from the determined at least one database, At least one extension candidate may be generated by extending the at least one candidate.

The voice recognition service providing server 120 may calculate a similarity between at least one extended query word and at least one extended candidate, and select a final candidate based on the calculated similarity.

To this end, the voice recognition service providing server 120 may convert the at least one extended query term and the at least one extended candidate into at least one conversion format for comparison between the at least one extended query term and the at least one extended candidate. Here, the at least one conversion format may include a keystroke, a phonetic symbol, a character string, and the like.

Thereafter, the voice recognition service providing server 120 calculates a similarity of each of the at least one extended query term and each of at least one extension candidate corresponding to each of the at least one extended query term, and based on the similarity of each of the at least one extension candidate, A predetermined number of similar candidates may be selected. In this case, the voice recognition service providing server 120 may select a final candidate from among a predetermined number of similar candidates based on different criteria for each type of the extracted domain.

The voice recognition service providing server 120 may transmit a service control command for executing a service for a voice command to the user terminal 110 based on the selected final candidate.

Hereinafter, the operation of each component of the voice recognition service providing system of FIG. 1 will be described in more detail.

2 is a block diagram of a user terminal according to an embodiment of the present invention. 1 and 2 , the user terminal 110 includes a voice recognition unit 210 , a query word extraction unit 220 , a query term expansion unit 230 , a candidate expansion unit 240 , and a final candidate selection unit 250 . , a service providing unit 260 and a database 270 may be included.

The voice recognition unit 210 may receive a voice command. The voice command may include, for example, a name of a recipient of a text message or a call, a name of a friend included in a friend list of an instant message, a name of a play list for searching content, and the like.

For example, the voice recognition unit 210 may receive a voice command related to a call service, such as “Call Daniel,” from the user. As another example, the voice recognition unit 210 may receive a voice command related to a text message, such as “send a text to Daniel” from the user. As another example, the voice recognition unit 210 may receive a voice command related to the instant message, such as “Read the instant message from Daniel” from the user. As another example, the voice recognition unit 210 may receive a voice command related to the content search service, such as “play a lie song” from the user.

The voice recognition unit 210 may convert the voice command into text through voice recognition for the received voice command. For example, the voice recognition unit 210 may convert the received voice command into text through Speak to Text (STT).

The query word extractor 220 may extract a query word from the text for the voice command. For example, the query word extraction unit 220 may extract a recipient name, a messenger friend name, a playlist name, and the like as a query word from the text.

The query word extractor 220 may extract a service domain from the text for the voice command. For example, the query word extractor 220 may extract a service domain related text related to a service domain from the text for the voice command, and extract any one service domain based on the extracted service domain related text. Here, the service domain includes a first service domain corresponding to a call service (eg, making a call), a second service domain corresponding to a text message service (eg, sending a text message, reading a text), and an instant message service ( For example, it may include a third service domain corresponding to sending an instant message and reading an instant message) and a fourth service domain corresponding to a content search service (eg, playing a play list).

For example, the query word extractor 220 may extract any one of the first to fourth service domains using a domain classifier. Here, the query word extractor 220 may use a domain-specific semantic vector instead of a character-based semantic vector used in the past to obtain a domain-specific word semantic vector while maintaining a fast execution speed. Domain-specific semantic vectors are for expressing the meaning of patented words for the problem that cannot be expressed with general word semantic vectors, such as, for example, GloVe (Global Vectors) semantic vectors. can Here, GloVe is a model for a distributed word expression, and the model may be an unsupervised learning algorithm for obtaining a vector expression for a word, and the model may be obtained by mapping a word into a meaningful space, and between words Distance is related to semantic similarity.

By using the domain-specific semantic vector, the performance of sentence unit accuracy is improved, and the domain-specific semantic vector can be effective in fine-tuning the semantic vector of a word in each problem. The query word extractor 220 may extract the service domain in various ways other than the method using the domain classifier based on the domain-specific semantic vector, but is not limited thereto.

As another example, the query word extractor 220 may extract any one of the first to fourth service domains by comparing the extracted service domain related text with the database 270 in which keywords of each service domain are stored.

The database 270 may include first to fourth databases. Keywords related to the first service domain may be stored in the first database, such as 'call, phone call, outgoing, receiving, calling, receiving, calling, connection'. The second database may store keywords related to the second service domain, such as 'text, sending, sending, receiving, sender, receiver, response, message, send, read'. The third database may store keywords related to the third service domain, such as 'friend, acquaintance, message, chat, chat, group chat, send, read'. The 4th database is connected with the 4th service domain such as 'song, music, movie, drama, mid, yield, play, play, fast, slow, next song (song), previous song (song), pause, stop' Related keywords may be stored.

For example, the query word extractor 220 may extract the first service domain by extracting the service domain related text of [Call, call me] from the voice command "Call Daniel". As another example, the query word extracting unit 220 may extract the second service domain by extracting the service domain related text of [text, send me] from the voice command of "send a text to Daniel". As another example, the query word extraction unit 220 may extract the third service domain by extracting the service domain related text of [instant message, read me] from the voice command "Read the instant message from Daniel". have. As another example, the query word extractor 220 may extract the service domain related text of [sing, play] from the voice command of "play a lie song" to extract the fourth service domain.

The query term extension unit 230 may generate at least one extended query term by expanding the query term based on the extracted service domain. Here, the query word extension unit 230 may expand the query word in a different way for each type of the extracted domain.

For example, when the extracted service domain corresponds to the first service domain or the second service domain, the query word extension unit 230 extends the query word except for combinations and suffixes according to fields of the address book of the user terminal 110 . and the like to generate at least one extended query word. For example, the query word extension unit 230 may generate an extended query word such as 'Daniel by proxy' by combining the query word 'Daniel, proxy' according to the address book field.

As another example, the query term extension unit 230 may generate at least one extended query term by performing extension excluding the suffix on the query term based on the third service domain. For example, the query word extension unit 230 may generate the extended query word 'Daniel' by expanding the query word 'Daniel' by excluding the suffix.

As another example, the query term extension unit 230 may generate at least one extended query term by performing date format normalization on the query term based on the fourth service domain. For example, when the query word includes a date format of '2019/12/31', the query word extension unit 230 performs normalization of the date format to generate the extended query word as 'December 31, 2019'. can

The candidate extension unit 240 determines at least one database 270 for candidate search corresponding to the extracted service domain, and at least one candidate for each of the at least one extended query word from the determined at least one database 270 . At least one extension candidate may be generated by searching for , and expanding the at least one candidate. Here, the candidate may be composed of a key and a value corresponding to an ID for a service. Here, a key indicates a parameter name related to a specific text, a value indicates a value extracted in response to an ID for a service, and the key and value are, for example, key1 and value. 1 (value1) may be matched and stored.

The database 270 may further include fifth to seventh databases. The fifth database may store a friend name, a friend phone number, etc. registered by the user through the installed address book app of the user terminal 110 . The sixth database may store a friend name or friend ID registered by the user through the instant message service, conversation details, and the like. The seventh database may store a plurality of content names for providing a content search service, metadata related to the plurality of content names, and the like.

For example, when a candidate for each extended query word is searched for from the fifth database, the candidate extension unit 240 expands the searched candidate by excluding a suffix and a combination according to a field of the address book to obtain at least one extension candidate. can create For example, when 'Daniel, Assistant' is registered as a friend name in the address book of the user terminal 110, the candidate extension unit 240 searches for 'Daniel, Assistant' as a candidate from the fifth database based on the extended query word. and, based on the searched candidate, an extension candidate such as 'Daniel Deputy' may be generated.

As another example, when candidates for each extended query word are searched for from the sixth database, the candidate extension unit 240 separates English and Korean for the searched candidates, and performs expansion to extract names except for surnames. At least one extension candidate may be generated. For example, when 'Daniel' is registered as a friend name of the instant message service of the user terminal 110, the candidate extension unit 240 selects 'Daniel' as a candidate from the sixth database based on the extended query word. (daniel)' is searched, and extension candidates such as 'Daniel', 'Daniel', and 'daniel' may be generated based on the searched candidate.

As another example, when candidates for each extended query word are searched for from the seventh database, the candidate extension unit 240 performs date format normalization on the searched candidates or generates at least one possible candidate combination. of extension candidates can be created. For example, the third party (3 ^rd -party) if the content through the application program such as search, use, purchase, held, candidate extended portion 240 of the user terminal 110 based on the expanded query phrase candidates from the database 7 '2019/12/31' may be searched as '2019/12/31', and an extension candidate may be generated such as 'December 31, 2019' based on the found candidate.

Table 1 below shows extended query words and extended candidates generated for the query word 'Daniel, Deputy'. It is assumed that the service domain is the first service domain.

For example, when the query word is "Daniel, proxy", the query word extension unit 230 may generate extended query words such as 'Daniel by proxy' and 'Daniel' through extension excluding the suffix. Candidate extension unit 240 ) may extract 'Daniel, sir' and 'Daniel Kang' as candidates for the extended query term 'Daniel Assistant' from the fifth database In this case, the candidate extension unit 240 is one of the candidates. Extension candidates such as 'Daniel Daniel', 'Daniel Daniel', and 'Daniel' may be created by expanding 'Daniel, Assistant Manager.' Also, the candidate extension unit 240 may include one of the candidates, 'Kang Daniel,' (daniel)' can be extended to create extension candidates such as 'Daniel Kang', 'Daniel', and 'daniel'.

Also, the candidate extension unit 240 may extract 'Daniel Kang' as a candidate for the extended query word 'Daniel' from the sixth database. In this case, the candidate extension unit 240 may expand the candidate 'Daniel Kang' to generate extension candidates 'Daniel Kang', 'Daniel', and 'daniel'.

The final candidate selector 250 may calculate a similarity between at least one extended query word and at least one extended candidate, and select a final candidate based on the calculated similarity.

The final candidate selection unit 250 may include a format conversion unit 251 and a similar candidate selection unit 252 .

The form converter 251 may convert the at least one extension query word and the at least one extension candidate into at least one transformation form for comparison between the at least one extension query word and the at least one extension candidate. Here, the at least one conversion format may include a keystroke, a phonetic symbol, a character string, and the like. A keystroke refers to a typing query expressed in alphabetical terms, a phonetic symbol refers to a pronunciation expressed using a predefined symbol, and a character string refers to using an input string as it is.

For example, when the conversion format is set to a keystroke, the format conversion unit 251 may convert the extended query word 'Daniel' to 'eksldpf', and may also convert the extension candidate into the same format. Here, in the case of an extended query word composed of English, keystroke conversion may be performed after conversion into Korean pronunciation. For example, the format conversion unit 251 may convert the extended query word 'daniel' into a Korean pronunciation like 'Daniel' and then convert it into 'eksldpf'.

As another example, when the conversion format is set to a phonetic symbol, the format conversion unit 251 may convert the extended query word 'Daniel' to the same format as 'da ni el', and the extension candidates may also be converted into the same format. .

As another example, when the conversion format is set to a character string, the format conversion unit 251 may use the extended query word 'Daniel' as it is, like 'Daniel'.

The similarity candidate selector 252 may calculate the similarity of each of the at least one extended query term and each of the at least one extended candidate corresponding to each of the at least one extended query term.

Table 2 below shows the similarity between the extended query word and the extended candidate in the query word 'Daniel, Deputy'.

Referring to Table 2, the similar candidate selection unit 252 calculates similarity by comparing the extended query word 'Daniel's proxy' with the extended candidate, respectively, and compares the extended query word 'Daniel' with the extended candidate for the same. The similarity can be calculated.

For example, in Table 2, the similarity between the extended query term 'Daniel Daniel' and the extended candidate 'Daniel Daniel' is 95%, and the similarity between the extended query term 'Daniel' and the extended candidate 'Kang Daniel' is 80. % is shown.

As described above, according to the present invention, the probability of a user's voice command being misrecognized can be significantly reduced by expanding not only the query word but also the candidate and calculating the similarity between the extended query word and the extended candidate to perform the voice recognition service.

In this case, the similarity candidate selector 252 may calculate each degree of similarity based on an error model according to an edit distance (ED). Editing distance means the number of edits required to make the keyword A into the keyword B. In this case, the edit distance is 'one time'. The error score means the probability that a typo is entered between the regular hit and the typo. For example, in the case of 'The Hankyoreh' and 'The Hankyoreh', it may mean the probability that 'ㅔ' is entered but 'ㅖ' is entered. .

The similarity candidate selector 252 may select a predetermined number of similarity candidates based on the similarity of each of the at least one extension candidate. For example, when there are N extended query words, the similar candidate selector 252 selects a maximum of K similar candidates for each extended query (where K may be the maximum among the number of extended candidates for each extended query). can

For example, the similarity candidate selector 252 may select a maximum of K extended candidates having a predetermined similarity or higher as similarity candidates. For example, in Table 2, 'Daniel's representative' (similarity: 95%), 'Daniel's representative' (similarity: 100%) and 'daniel' (similarity: 90%) with a degree of similarity of 90% or higher are similar candidates. can be selected as

The final candidate selector 250 may select a final candidate from among a predetermined number of similar candidates based on different criteria for each type of the extracted domain.

The final candidate selection unit 250 is, for example, similar to 'Daniel's assistant' (similarity: 95%), 'Daniel's assistant' (similarity: 100%), and 'daniel' (similarity: 90%) as described above. In the case of candidates, one of them may be selected as the final candidate.

When the similarity of one similar candidate is uniquely higher than that of another similar candidate (the difference in similarity exceeds a preset range), the final candidate selector 250 may select the corresponding similar candidate as the final candidate. For example, when the similarity of the first similarity candidate is 90% and the similarity of the second similarity candidate and the third similarity candidate are 65% and 45%, respectively, the final candidate selection unit 250 selects the first similarity candidate as the final candidate. can be selected as

As another example, when the difference in similarity is within a preset range, the final candidate selector 250 may select a similar candidate by further considering the characteristics of the service domain.

For example, when the query term is related to the first service domain, the final candidate selection unit 250 may calculate a probability that the user 100 will transmit based on the recent call history, and further consider this to select the final candidate. .

In addition, when the query word relates to the third service domain, the final candidate selection unit 250 may be configured based on the number of chats with the candidate corresponding to the extension candidate, the number of group chat rooms with the candidate, the recent chat history with the candidate, etc. A final candidate may be selected by further considering the calculated intimacy score.

Also, the final candidate selector 250 may select the final candidate by applying different weights to the similarity and service domain characteristics based on the difference in the similarity between the similar candidates. For example, in selecting a final candidate, the final candidate selector 250 may apply a first weight to the similarity and apply a second weight to the characteristic of the domain. In this case, the first weight may be proportional to the difference in the similarity, and the second weight may be inversely proportional to the difference in the similarity.

The service providing unit 260 may provide a service for a voice command based on the selected final candidate. In this case, the service providing unit 260 may provide a service for the voice command using a service ID corresponding to a value of the final candidate. A process of providing a service for a voice command will be described in detail with reference to FIGS. 3A to 6C .

3A to 3C are exemplary diagrams for explaining a process of providing a service related to a first service domain according to an embodiment of the present invention.

Referring to FIG. 3A , the voice recognition unit 210 performs voice recognition on the voice command “Hey Kakao, call Daniel,” 300 uttered from the user 100, and converts the voice command into text. can be converted

The query word extractor 220 may extract a query word and a service domain from the text for the converted voice command. For example, the query word extraction unit 220 extracts the query word 'Daniel' corresponding to the recipient's name from the text, searches the first database, and extracts [Call, Call] as text related to the first service domain, , it is possible to extract the first service domain corresponding to the call service through this.

Referring to FIG. 3B , the query term extension unit 230 generates an extended query term such as 'Daniel' by expanding the query term 'Daniel' based on the extracted first service domain, and the candidate extension unit 240 provides a user Through the address book 310 installed in the terminal 110, the candidate 'Daniel, deputy' 311 for the extended query word 'Daniel' is searched for and 'Daniel, deputy' 311 is expanded, 'Daniel deputy', An extended candidate is generated as 'Daniel', and the final candidate selection unit 250 compares the similarity between the extended query word 'Daniel' and 'Daniel's representative' and 'Daniel', and 'Daniel, substitute' 311 as a final candidate. can be selected.

Referring to FIG. 3C , the service providing unit 260 may provide a call service so that the user 100 can make a voice call with 'Daniel, Assistant' 320 .

4A to 4C are exemplary diagrams for explaining a process of providing a service related to a second service domain according to an embodiment of the present invention.

Referring to FIG. 4A , the voice recognition unit 210 performs voice recognition for the voice command "Hey Kakao, send a text message to Daniel to see you at work tomorrow at 10:00" (400) uttered by the user 100, can be converted to

The query word extractor 220 may extract a query word and a service domain from the text for the converted voice command. For example, the query word extraction unit 220 extracts the query word 'Daniel' corresponding to the recipient's name from the text, searches the second database to extract [text, send me] as the text related to the service domain, and Through this, the second service domain corresponding to the text message service may be extracted.

Referring to FIG. 4B , the query term extension unit 230 generates an extended query term such as 'Daniel' by expanding the query term 'Daniel' based on the extracted second service domain, and the candidate extension unit 240 provides a user Through the address book 410 installed in the terminal 110, by searching for a candidate 'Daniel, deputy' 411 for the extended query word 'Daniel' and expanding 'Daniel, deputy' 411, 'Daniel deputy', An extended candidate is generated as 'Daniel', and the final candidate selection unit 250 compares the similarity between the extended query word 'Daniel' and 'Daniel's representative' and 'Daniel' to obtain 'Daniel, the representative' (411). can be selected.

Referring to FIG. 4C , the service provider 260 may transmit a text message saying “I will see you at work tomorrow at 6 o’clock” 421 to 'Daniel, the assistant' 411 using the text message application 420 .

5A to 5C are exemplary diagrams for explaining a process of providing a service related to a third service domain according to an embodiment of the present invention.

Referring to FIG. 5A , the voice recognition unit 210 may perform voice recognition on the voice command "Hey Kakao, read Kakao Talk from Daniel" 500 uttered from the user 100 and convert it into text. have.

The query word extractor 220 may extract a query word and a service domain from the text for the converted voice command. For example, the query word extraction unit 220 extracts the query word 'Daniel' corresponding to the messenger friend name from the text, searches the third database 273, and searches the third service domain related text [Kakao Talk, Read Me ], and through this, the third service domain corresponding to the instant message service can be extracted.

Referring to FIG. 5B , the query term extension unit 230 generates an extended query term such as 'Daniel' by expanding the query term 'Daniel' based on the extracted third service domain, and the candidate extension unit 240 provides a user Through the friend list 510 of the instant messenger installed in the terminal 110, the candidate 'Kang Daniel' 511 for the extended query word 'Daniel' is searched for and 'Kang Daniel' 511 is expanded, so that 'Kang Daniel '(511) and 'Daniel' generate an extension candidate, and the final candidate selector 250 compares the similarity between the extended query word 'Daniel' and 'Daniel Kang' (511) and 'Daniel' to select the final candidate. You can choose 'Kang Daniel' (511).

Referring to FIG. 5C , the service providing unit 260 sends an instant message “See you tomorrow at Gangnam Station at 6 o’clock” 521 from 'Kang Daniel' 511 through the instant message application 520 to the user terminal 110. It can output audio through its own built-in speaker. At this time, when the user terminal 110 is a voice recognition speaker, "See you tomorrow at Gangnam Station at 6 o'clock" 530 may be output through the voice recognition speaker.

According to another embodiment of the present invention, the user terminal 110 may receive a service command through a camera (not shown) provided in the user terminal 110 . For example, the user terminal 110 may further include a photographing unit (not shown) for obtaining an image including a service command.

For example, the user terminal 110 is an image taken through the camera of the user terminal 110, a service command (text ) can be extracted.

In this case, the user terminal 110 extracts a query word and a service domain from the converted text, and expands the extracted query word based on the service domain (eg, a service domain related to a Wi-Fi connection service, a map service, an order service, etc.), You can create extended query words.

Also, the user terminal 110 may generate at least one extension candidate by searching for a candidate related to the service domain based on the extension query and expanding the searched candidate. In this case, the user terminal 110 selects any one of the generated at least one extension candidate as a final candidate, thereby providing a service (eg, a Wi-Fi connection service, a map service, an ordering service, etc.) related to the selected final candidate. can do.

Although the embodiment has not been described in detail, all of the contents described in the embodiment using the above-described voice command are applied.

6 is a flowchart of a method for providing a voice recognition service in a user terminal according to an embodiment of the present invention. The method for providing a voice recognition service in the user terminal 110 shown in FIG. 6 includes steps processed in time series by the voice recognition service providing system 1 according to the embodiment shown in FIGS. 1 to 5C . . Therefore, even if omitted below, it is also applied to the method of providing a voice recognition service in the user terminal 110 according to the embodiment shown in FIGS. 1 to 5C .

In step S600, the user terminal 110 may extract a query word and a service domain from the text for the voice command.

In step S610, the user terminal 110 may generate at least one extended query by expanding the query based on the extracted service domain.

In step S620, the user terminal 110 may determine at least one database for a candidate search corresponding to the extracted service domain.

In operation S630, the user terminal 110 may search for at least one candidate for each of the at least one extended query word from the determined at least one database.

In operation S640, the user terminal 110 may generate at least one extension candidate by extending the at least one candidate.

In step S650, the user terminal 110 may calculate a similarity between at least one extension query word and at least one extension candidate.

In step S660, the user terminal 110 may select a final candidate based on the calculated similarity.

In step S670, the user terminal 110 may provide a service for a voice command based on the selected final candidate.

In the above description, steps S600 to S670 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, and the order between the steps may be switched.

7 is a block diagram of a voice recognition service providing server according to an embodiment of the present invention. The voice recognition service proposed by the present invention may be provided by the voice recognition service providing server 120 .

Referring to FIG. 7 , the voice recognition service providing server 120 includes a voice command receiver 710 , a query word extractor 720 , a query word expander 730 , a candidate expander 740 , and a final candidate selector 750 . and a service control command transmission unit 760 .

The voice command receiver 710 may receive a voice command from the user terminal 110 and convert the voice command into text through voice recognition for the received voice command.

The query word extracting unit 720 , the query word expansion unit 730 , the candidate expansion unit 740 , and the final candidate selection unit 750 included in the voice recognition service providing server 120 of FIG. 7 is the user terminal described with reference to FIG. 2 . Since it performs the same functions as the query word extractor 220 , the query word expander 230 , the candidate expander 240 , and the final candidate selector 250 of ( 110 ), the following description will be omitted.

The service control command transmission unit 760 may transmit a service control command for executing a service for a voice command to the user terminal 110 based on the selected final candidate.

8 is a flowchart of a method for providing a voice recognition service in a voice recognition service providing server according to an embodiment of the present invention. The method for providing a voice recognition service in the voice recognition service providing server 120 shown in FIG. 8 is a step of time-series processing by the voice recognition service providing system 1 according to the embodiment shown in FIGS. 1 to 7 . include those Accordingly, even if the description is omitted below, it is also applied to the method of providing the voice recognition service in the voice recognition providing server 120 according to the embodiment shown in FIGS. 1 to 7 .

In step S800 , the voice recognition service providing server 120 may receive a voice command from the user terminal 110 .

In step S810, the voice recognition service providing server 120 may convert the voice command into text through voice recognition for the received voice command.

In step S820, the voice recognition service providing server 120 may extract a query word and a service domain from the converted text.

In operation S830, the voice recognition service providing server 120 may generate at least one extended query by expanding the query based on the extracted service domain.

In step S840, the voice recognition service providing server 120 may determine at least one database for candidate search corresponding to the extracted service domain.

In step S850, the voice recognition service providing server 120 may search for at least one candidate for each of the at least one extended query word from the determined at least one database.

In step S860, the voice recognition service providing server 120 may generate at least one extension candidate by extending the at least one candidate.

In operation S870, the voice recognition service providing server 120 may calculate a similarity between at least one extended query word and at least one extended candidate.

In step S880, the voice recognition service providing server 120 may select a final candidate based on the calculated similarity.

In step S890 , the voice recognition service providing server 120 may transmit a service control command for executing a service for a voice command to the user terminal 110 based on the selected final candidate.

In the above description, steps S800 to S890 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, and the order between the steps may be switched.

Embodiments of the present invention may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Computer-readable media may also include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. The above description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

Although the methods and systems of the present invention have been described with reference to specific embodiments, some or all of their components or operations may be implemented using a computer system having a general purpose hardware architecture.

The above description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

110: user terminal
120: voice recognition providing server
210: voice recognition unit
220: query word extraction unit
230: query extension
240: candidate extension
250: final candidate selection unit
251: format conversion unit
252: similar candidate selection unit
260: service provider
270: database
710: voice command receiver
720: query word extraction unit
730: Query word extension
740: candidate extension
750: final candidate selection unit
751: format conversion unit
752: similar candidate selection unit
760: service control command transmission unit
770: database

Claims (19)

In a user terminal providing a voice recognition service,
a query word extraction unit for extracting a query word and a service domain from the text for the voice command;
a query word extension unit for generating at least one extended query word by expanding the query word based on the extracted service domain;
determining at least one database for candidate search corresponding to the extracted service domain, searching for at least one candidate for each of the at least one extended query word from the determined at least one database, and selecting the at least one candidate a candidate extension unit for generating at least one extension candidate by expanding;
a final candidate selector calculating a similarity between the at least one extended query word and the at least one extended candidate, and selecting a final candidate based on the calculated similarity; and
Comprising a service provider providing a service for the voice command based on the selected final candidate,
and the final candidate selector includes a similarity candidate selector that calculates a similarity of each of the at least one extended query term and each of the at least one extended candidate corresponding to each of the at least one extended query term.
The method of claim 1,
A voice recognition unit that receives the voice command and converts the voice command into the text through voice recognition for the received voice command
Which will further include, the user terminal.
The method of claim 1,
The service domain includes at least one of a first service domain corresponding to a call service, a second service domain corresponding to a text message service, a third service domain corresponding to an instant message service, and a fourth service domain corresponding to a content search service. That comprising, the user terminal.
4. The method of claim 3,
The query word extracting unit extracts service domain related text related to the service domain from the text, and based on the extracted service domain related text, the first service domain, the second service domain, the third service domain, and the fourth service To extract any one of the domain, the user terminal.
4. The method of claim 3,
The query word extension unit expands the query word in a different way for each type of the extracted domain.
The method of claim 1,
The final candidate selection unit includes a format conversion unit that converts the at least one extended query term and the at least one extension candidate into at least one transformation format for comparison between the at least one extended query term and the at least one extension candidate In, user terminal.
7. The method of claim 6,
The at least one conversion format includes any one of a keystroke, a phonetic symbol, and a character string, the user terminal.
The method of claim 1,
and the similarity candidate selection unit selects a predetermined number of similarity candidates based on the similarity of each of the at least one extension candidate.
9. The method of claim 8,
wherein the final candidate selector selects the final candidate from among the extracted predetermined number of similar candidates based on different criteria for each type of the extracted domain.
The method of claim 1,
Further comprising a photographing unit for obtaining an image including a service command,
The query word extraction unit is to extract the query word and the service domain from the service command included in the image, the user terminal.
A method of providing a voice recognition service, comprising:
extracting a query word and a service domain from the text for the voice command;
generating at least one extended query term by expanding the query term based on the extracted service domain;
determining at least one database for candidate search corresponding to the extracted service domain;
searching for at least one candidate for each of the at least one extended query term from the determined at least one database;
generating at least one extension candidate by expanding the at least one candidate;
calculating a similarity between the at least one extension query word and the at least one extension candidate;
selecting a final candidate based on the calculated similarity; and
providing a service for the voice command based on the selected final candidate;
The step of selecting the final candidate is
and calculating a similarity of each of the at least one extended query term and each of the at least one extended candidate corresponding to each of the at least one extended query term.
In the server providing a voice recognition service,
a voice command receiver for receiving a voice command from the user terminal and converting the voice command into text through voice recognition for the received voice command;
a query word extraction unit for extracting a query word and a service domain from the converted text;
a query word extension unit for generating at least one extended query word by expanding the query word based on the extracted service domain;
determining at least one database for candidate search corresponding to the extracted service domain, searching for at least one candidate for each of the at least one extended query word from the determined at least one database, and selecting the at least one candidate a candidate extension unit for generating at least one extension candidate by expanding;
a final candidate selector calculating a similarity between the at least one extended query word and the at least one extended candidate, and selecting a final candidate based on the calculated similarity; and
Comprising a service control command transmission unit for transmitting a service control command for executing the service for the voice command to the user terminal based on the selected final candidate,
and the final candidate selector includes a similarity candidate selector that calculates a similarity of each of the at least one extended query term and each of the at least one extended candidate corresponding to each of the at least one extended query term.
13. The method of claim 12,
The service domain includes at least one of a first service domain corresponding to a call service, a second service domain corresponding to a text message service, a third service domain corresponding to an instant message service, and a fourth service domain corresponding to a content search service. which includes, the server.
14. The method of claim 13,
The query word extracting unit extracts service domain related text related to the service domain from the text, and based on the extracted service domain related text, the first service domain, the second service domain, the third service domain, and the fourth service The server, which extracts any one of the domains.
15. The method of claim 14,
The server, wherein the query word extension unit expands the query word in a different way for each type of the extracted domain.
13. The method of claim 12,
The final candidate selection unit includes a format conversion unit that converts the at least one extended query term and the at least one extension candidate into at least one transformation format for comparison between the at least one extended query term and the at least one extension candidate In, server.
17. The method of claim 16,
The server, wherein the at least one conversion format includes any one of a keystroke, a phonetic symbol, and a character string.
15. The method of claim 14,
and the similarity candidate selection unit selects a predetermined number of similarity candidates based on the similarity of each of the at least one extension candidate.
19. The method of claim 18,
The server, wherein the final candidate selection unit selects the final candidate from the extracted predetermined number of similar candidates based on different criteria for each type of the extracted domain.

Images