KR102411619B1 - Electronic apparatus and the controlling method thereof - Google Patents

Abstract

An electronic device is disclosed. The electronic device detects a storage unit that stores domain information categorized for each conversation topic, a speaker unit that outputs a system response corresponding to the user's uttered voice, and a domain corresponding to the user's uttered voice, between the user uttered voice and the detected domain. and a processor for generating a system response by determining a domain to process a user's uttered voice from among the detected domain and the previous domain based on confidence. Accordingly, it is possible to recognize various user utterances and generate a system response based on functions of a plurality of terminal devices.

Description

ELECTRONIC APPARATUS AND THE CONTROLLING METHOD THEREOF

The present invention relates to an electronic device and a method for controlling the same, and more particularly, to an electronic device for generating a system response corresponding to a user uttered voice, and a method for controlling the same.

With the development of electronic technology, various types of electronic products are being developed and distributed. In particular, various display devices such as TV, mobile phone, PC, notebook PC, PDA, etc. are widely used in most general homes.

As the use of display devices increases, user needs for more diverse functions have also increased. Accordingly, each manufacturer's efforts to meet the user's needs have also increased, and products with new functions that were not previously available are appearing one after another.

In particular, not only the display device but also various home appliances such as refrigerators, air conditioners, and electric lamps are used in general households, and a home network system for controlling these various home appliances by connecting them through a network has been commercialized.

On the other hand, the user can directly manipulate the home network system to achieve a desired purpose, but when the home network system supports a voice recognition function, the user can utter a voice command to achieve the desired purpose.

However, the current system for recognizing a user's uttered voice and generating a corresponding response is used to simply control a function provided by one terminal device, and cannot generate a response by combining and comparing the functions of several terminal devices. Alternatively, when a user's spoken voice outside of a dialog programmed in the system is recognized, there is a problem in that it cannot be processed.

In addition, there is a problem in that if the user proceeds with a conversation on another domain in a situation where the user and the system exchange a conversation for one specific domain, the system cannot normally process it.

Accordingly, there has been an increase in demand for recognizing various user voices, generating responses corresponding thereto, and generating responses based on functions of a plurality of terminal devices.

SUMMARY OF THE INVENTION The present invention has been devised to solve the above problems, and an object of the present invention is to provide an electronic device for generating a system response by determining a domain to process a user's uttered voice, and a method for controlling the same.

In order to achieve this object, an electronic device according to an embodiment of the present invention includes a storage unit for storing domain information categorized for each conversation topic, a speaker unit for outputting a system response corresponding to the user's uttered voice, and the user's uttered voice. A processor configured to detect a corresponding domain and generate the system response by determining a domain to process the user uttered voice from among the detected domain and previous domains based on confidence between the user uttered voice and the detected domain includes

Here, the storage unit categorizes and stores conversation topics corresponding to each of the domains for each context, and the processor is configured to store, when the previous domain is determined as a domain to process the user's spoken voice, corresponding to the user's spoken voice. The system response may be generated by determining a context, and determining a context to process the user's spoken voice from among the determined context and previous contexts based on the reliability between the user's spoken voice and the determined context.

In addition, when the determined context is determined as a context for processing the user's spoken voice, the processor stores information related to the previous context in the storage unit, and when the spoken voice processing in the determined context is terminated, A new spoken voice may be processed based on the stored information related to the previous context.

In addition, when the detected domain is determined as a domain to process the user's speech voice, the processor stores information related to the previous domain in the storage unit, and when the speech speech processing in the detected domain ends, A new spoken voice may be processed based on the stored information related to the previous domain.

In addition, the processor is configured to: Based on a confidence score based on whether at least one speech element constituting the user's speech voice is identical to at least one speech element belonging to the detected domain, between the user's speech voice and the detected domain reliability can be judged.

Meanwhile, the electronic device according to an embodiment of the present invention further includes a communication unit configured to communicate with at least one external device, wherein the processor is configured to provide a system response corresponding to the spoken voice to the at least one within the determined domain. When the function control of the external device is generated based on a context in which the function control of the external device is requested, the system response for controlling the function of at least one external device may be generated based on the information on the function of the external device.

In addition, the storage unit further stores information on functions of the external device, the communication unit receives function information on at least one external device added in a preset network, and the processor, Information stored in the storage unit may be updated based on function information about one external device.

In addition, the processor generates the system response by determining a domain to process the user uttered voice based on the utterance history information, and the utterance history information includes a previously received user uttered voice and the previously received user uttered voice. It may include at least one of information related to the domain in which the voice has been processed and a system response corresponding to the previously received user uttered voice.

Also, the domain information may include at least one of control information for performing a task corresponding to the conversation topic and a conversation pattern for each conversation topic.

Meanwhile, the electronic device according to an embodiment of the present invention may further include a microphone unit for receiving the user's uttered voice.

Meanwhile, in the control method of an electronic device including a storage unit for storing domain information categorized for each conversation topic according to an embodiment of the present invention, the step of detecting a domain corresponding to a user uttered voice and the and generating a system response by determining a domain to process the user's uttered voice from among the detected domain and previous domains based on confidence between the detected domains.

Here, the storage unit categorizes and stores the conversation topics corresponding to each of the domains for each context, and the generating of the system response includes, The system response may be generated by determining a context corresponding to the spoken voice, and determining a context to process the user's spoken voice among the determined context and previous contexts based on the reliability between the user's spoken voice and the determined context have.

Also, in the generating of the system response, when the determined context is determined as a context to process the user uttered voice, information related to the previous context is stored in the storage unit, and the utterance in the determined context When the voice processing is finished, the newly uttered voice may be processed based on the stored information related to the previous context.

In addition, the generating of the system response may include, when the detected domain is determined as a domain to process the user's uttered voice, information related to the previous domain is stored in the storage unit, and the uttered voice from the detected domain is stored in the storage unit. When the processing is completed, the newly uttered voice may be processed based on the stored information related to the previous domain.

In addition, the generating of the system response may include: based on a reliability score according to whether at least one speech element constituting the user's speech voice and at least one speech element belonging to the detected domain is identical to the user's speech voice and Reliability between the detected domains may be determined.

In addition, the generating of the system response may include, when a system response corresponding to the spoken voice is generated based on a context in which function control of the at least one external device is required within the determined domain, the function of the external device is determined. The system response for controlling the function of the at least one external device may be generated based on the related information.

Meanwhile, the method of controlling an electronic device according to an embodiment of the present invention further includes the step of receiving function information on at least one external device added in a preset network and updating the previously stored information on the function of the external device. may include

In addition, the method of controlling an electronic device according to an embodiment of the present invention further includes generating the system response by determining a domain to process the user's uttered voice based on utterance history information, wherein the utterance history information includes: It may include at least one of a previously received user uttered voice, information related to a domain in which the previously received user uttered voice has been processed, and a system response corresponding to the previously received user uttered voice.

Also, the domain information may include at least one of control information for performing a task corresponding to the conversation topic and a conversation pattern for each conversation topic.

In addition, the electronic device according to an embodiment of the present invention may further include the step of receiving the user's spoken voice.

In addition, in a storage medium storing a program for generating a system response according to an embodiment of the present invention, the program includes the steps of: detecting a domain corresponding to a user's uttered voice and reliability between the user's uttered voice and the detected domain and generating the system response by determining a domain to process the user's uttered voice from among the detected domain and the previous domain based on (confidence).

According to various embodiments of the present disclosure as described above, it is possible to recognize various user voices and generate a system response based on functions of a plurality of terminal devices.

1 is a block diagram illustrating the configuration of an electronic device according to an embodiment of the present invention.
2 is a diagram illustrating a process of processing a user uttered voice according to an embodiment of the present invention.
3 is a flowchart illustrating a process of processing a user uttered voice according to an embodiment of the present invention.
4 is a diagram illustrating a processing process when a domain for processing a user's uttered voice is changed according to an embodiment of the present invention.
5 is a domain related to a processing process when a domain for processing a user's uttered voice is not changed according to an embodiment of the present invention.
6 is a diagram illustrating a processing process in the DM module of the present invention.
7 is a diagram for explaining a process performed in a system including the electronic device 100 and a database including information about an external device according to an embodiment of the present invention.
8 is a block diagram illustrating a configuration of an electronic device according to another embodiment of the present invention.
9 is a block diagram illustrating a configuration of an electronic device according to another embodiment of the present invention.
FIG. 10 is a block diagram illustrating a specific configuration of the electronic device shown in FIG. 1 .
11 is a diagram of a software module stored in a storage unit according to an embodiment of the present invention.
12 is a flowchart illustrating a method of controlling an electronic device according to an embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings. In the description of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. And, the terms to be described later are terms defined in consideration of functions in the present invention, which may vary depending on the intention or relationship of a user or an operator. Therefore, the definition should be made based on the content throughout this specification.

1 is a block diagram illustrating the configuration of an electronic device according to an embodiment of the present invention.

Referring to FIG. 1 , the electronic device 100 includes a communication unit 110 and a processor 120 . Here, the electronic device 100 includes a device capable of recognizing a user's spoken voice and processing a system response corresponding to the user's spoken voice, for example, a TV, an electronic blackboard, an electronic table, an LFD ( Large Format Display), a smart phone, a tablet, a desktop PC, a notebook computer, a home network system server, etc. may be implemented in various types of electronic devices. Of course, the electronic device 100 may be implemented in the form of a system on chip (SOC) for recognizing the user's spoken voice and processing a system response corresponding to the user's spoken voice.

The storage 110 may store domain information categorized for each conversation topic. Here, the storage unit 110 is a storage medium in which various programs necessary for operating the electronic device 100 are stored, and may be implemented as a memory, a hard disk drive (HDD), or the like. For example, the storage 110 may include a ROM for storing a program for performing an operation of the processor 130 , a RAM for temporarily storing data according to an operation of the controller 150 , and the like. In addition, an electrically erasable and programmable ROM (EEROM) for storing various types of reference data may be further provided.

In particular, the storage 110 may store domain information categorized for each conversation topic, and the domain information categorized for each conversation topic means information about a group divided according to a topic to which a user's utterance voice belongs. Here, the domain information may include at least one of control information for performing a task corresponding to a conversation topic and a conversation pattern for each conversation topic.

Specifically, the storage 110 may store a conversation pattern for each service domain, and for this, the storage 110 may include a corpus database for each service domain. Here, the service domain may be classified according to a subject to which the user's spoken voice belongs, as described above.

For example, the storage 110 may include a first corpus database for a broadcast service domain and a second corpus database for a weather service domain.

In this case, the first corpus database may store various conversation patterns that may occur in the broadcast service domain. For example, as an answer to "When does a program start?", "Which program do you want to know the start time of?" is stored, and as an answer to "When does ○○○ (broadcast program name) start?" ○○○'s start time is ..." can be saved.

Also, the second corpus database may store a conversation pattern that may occur within the weather service domain. For example, as an answer to "How's the weather in ○○ (region name)?", "Can you tell me the temperature?" is saved, and as an answer to "How is the temperature in Seoul?" The temperature is ..." can be stored as an answer.

In addition, the storage 110 may match and store the control command according to the user's utterance intention. For example, when the user's utterance intention is to change the channel, a control command for changing the channel of the display device (not shown) is matched and stored, and when the user's utterance intention is reserved recording, a specific control command is displayed on the display device (not shown). A control command for executing a reserved recording function for a program may be matched and stored.

In addition, when the user's speech intention is temperature control, a control command for adjusting the temperature of the air conditioner (not shown) is matched and stored, and when the user's speech intention is music playback, the sound output device (not shown) is played It can be stored by matching the control commands for As such, the storage 110 may match and store control commands for controlling various external devices according to the user's utterance intention.

The speaker unit 120 may output a system response corresponding to the user's uttered voice. Here, the speaker unit 120 may be implemented in the form of a speaker that outputs the system response in the form of a voice, but is implemented as an output port such as a jack for connecting an external speaker in order to output the system response in the form of a voice through the external speaker it might be

The processor 130 detects a domain corresponding to the user uttered voice, and determines a domain to process the user uttered voice from among the detected domain and the previous domain based on the confidence between the user uttered voice and the detected domain to respond to the system can create

Specifically, when a user uttered voice is input, the processor 130 may analyze the user uttered voice to determine which conversation topic the user uttered voice corresponds to to detect a domain corresponding to the user uttered voice.

Here, the processor 130 may convert the user's uttered voice into text using an Automatic Speech Recognition (ASR) module to generate text corresponding to the user's uttered voice. The ASR module is a module for converting a voice signal into text, and may convert the voice signal into text using various ASR algorithms disclosed in the prior art.

For example, the processor 130 determines the voice section by detecting the beginning and the end of the voice uttered by the user in the received voice signal. Specifically, the processor 130 may calculate the energy of the received voice signal, classify the energy level of the voice signal according to the calculated energy, and detect the voice section through dynamic programming. Then, the processor 130 generates phoneme data by detecting a phoneme, which is a minimum unit of speech, based on an acoustic model within the detected speech section, and a Hidden Markov Model (HMM) probability model based on the generated phoneme data. can be applied to convert the user's spoken voice into text.

In addition, the processor 130 analyzes various types of speech, such as part of speech, named entity extraction, information extraction, semantic analytic, etc. so that the processor 130 can understand the text corresponding to the user's spoken voice by using the Spoken Language Understanding (SLU) module. can be performed.

Thereafter, the processor 130 may detect a domain corresponding to the user's spoken voice by detecting a corpus database in which a dialog pattern matching the converted text of the user's spoken voice exists.

For example, when the text "When does the program start?" is received, the processor 130 determines that the user's spoken voice corresponds to the broadcast service domain, and the text "How is the weather in ○○ (region name)?" When received, it may be determined that the user's uttered voice corresponds to the weather service domain.

In addition, the processor 130 may analyze the reliability between the user's uttered voice and the detected domain.

Specifically, the processor 130 determines the reliability between the user's spoken voice and the detected domain based on a reliability score according to whether at least one speech element constituting the user's speech voice and at least one speech element belonging to the detected domain are identical. can judge

For example, the processor 130 may extract a dialogue act, a main action, and a component slot from the user's spoken voice. Here, the dialogue act, the driving, and the components are included in the speech element.

In addition, the processor 130 may extract dialogue acts and driving from the user's uttered voice using Maximum Entropy Classifier (MaxEnt), and extract components using Conditional Random Field (CRF). However, the present invention is not limited thereto, and dialogue acts, driving, and components may be extracted through various known methods.

Here, the dialogue act is a classification criterion related to the form of a sentence, and indicates whether the corresponding sentence is a Statement, a Request, a Why question (WH-Question), or a Yes-No question (YN-Question). Driving is semantic information in which a corresponding utterance indicates a desired action through a conversation in a specific domain. For example, in the broadcast service domain, driving may include TV on/off, program search, program time search, program reservation, and the like. A component is information added to specify the meaning of an action intended in a specific domain, that is, individual information about a specific domain appearing in the speech. For example, in the broadcast service domain, components may include a genre, a program name, a start time, a channel name, an actor name, and the like.

In addition, the processor 130 may calculate a reliability score according to whether at least one of dialogue acts, driving, and components extracted from the user's speech voice is identical to at least one speech component belonging to the detected domain, and the processor 130 . may determine the reliability between the user's uttered voice and the detected domain based on the calculated reliability score.

Also, the processor 130 may determine a domain to process the user's uttered voice from among the detected domain and the previous domain based on the reliability between the determined user uttered voice and the detected domain.

For example, if it is assumed that the previously received user uttered voice relates to the weather and thus the previous domain relates to the weather, the processor 130 determines the broadcasting domain when the currently received user uttered voice is “When does the program start?” can be detected, and the confidence score between the currently received user-spoken voice “when does the program start?” and the domain about the weather is 10, and the confidence score between the user-spoken voice “when does the program start?” and the domain about the broadcast is 80 In the case of , the processor 130 may determine that the domain in which the user uttered voice “when does the program start?” is processed is a broadcasting domain, not a previous domain (weather related domain).

Also, for example, if it is assumed that the previously received user's uttered voice is "close the window when it rains" and the domain that has processed this is determined to be a weather-related domain, the processor 130 determines that the currently received user uttered voice is In the case of "Turn on the TV when it rains", the weather domain and the broadcasting domain can be detected, and the reliability score between the user's voice saying "Turn on the TV when it rains" and the weather domain is 70, and "if it rains, When the reliability score between the user uttered voice “Turn on the TV” and the domain related to broadcasting is 50, the processor 130 determines that the domain to process the user uttered voice “Turn on the TV when it rains” is the same as the previous domain, the domain related to the weather. it can be decided that

That is, even if the domain for processing the previous user's uttered voice has already been determined, the processor 130 may determine each domain for processing the newly received user's uttered voice whenever a new user's uttered voice is received.

In addition, the processor 130 determines a domain in which the currently received user uttered voice is to be processed because the confidence score between the currently received user uttered voice and the previous domain and the confidence score between the domain corresponding to the currently received user uttered voice are similar to each other. If this is impossible, a message asking the user for which domain the currently received user uttered voice is used may be generated and outputted through the speaker unit 120 .

Meanwhile, the processor 130 may determine the utterance intention included in the user's uttered voice by using the extracted dialogue acts, driving, and components.

For example, when a user's uttered voice "When do you do ○○○ (broadcast program name)?" is received, the processor 130 searches a corpus database in which a conversation pattern matching the user uttered voice exists, and "○ When do you do ○○?" may be detected as being included in the broadcast service domain.

Thereafter, the processor 130 determines that the sentence form of the corresponding voice is "interrogative" through the dialogue act, and determines that it wants to know the "program start time" for "○○○" through driving and key elements. As a result, the processor 130 may determine that the utterance intention included in the user's voice is "inquiring" for "program start time" for "○○○".

And, considering that the user's spoken voice corresponds to the broadcast service domain and the utterance intention is to "inquire" the "program start time" for "○○○", finally the user's spoken voice is transmitted in the broadcast service domain. For "○○○", it can be judged as "inquiring" about "program start time".

In particular, the processor 130 may respectively determine a domain to process a user's spoken voice whenever a user's spoken voice is received, and also determines the user's utterance intention in each domain based on a context included in each of the determined domains. can do.

In the existing speech recognition-based processing system, when a domain for processing a user's spoken voice is determined by receiving a user's spoken voice, the user's speech intention can be determined based only on the context within the determined domain. The processor 130 can determine each domain to process the user's uttered voice whenever a user's uttered voice is received, and can determine the user's uttered intention in each domain based on the context belonging to each of the determined domains. It can process various user uttered voices.

Meanwhile, when a domain for processing the user's spoken voice is determined, the processor 130 may generate a system response corresponding to the user's spoken voice.

For example, if it is determined that the processor 130 "inquires" the "program start time" for "○○○" in the broadcast service domain, "the start time of the inquired ○○○ is .. .” is extracted from the corpus database of the broadcast service domain. In this case, the processor 130 searches for the broadcast start time for "○○○" using EPG (Electronic Program Guide) information pre-stored in the storage 110, and "The start time of the inquired ○○○ is It is 7 o'clock on Saturday".

In addition, the processor 130 may generate a control command for performing a function corresponding to the user's utterance intention as a system response.

For example, it may be assumed that a user uttered voice “Reserve ○○○ (broadcast program name)” is received. In this case, the controller 330 may search the corpus database in which a conversation pattern matching the user's voice exists, and determine that the user's voice "Reserve ○○○" is included in the broadcast service domain.

In addition, the processor 130 may determine that the corresponding voice is in the form of a sentence related to "request" through dialogue act, and may determine that "program reservation" for "○○○" is desired through driving and key elements. As a result, the processor 130 may determine that the utterance intention included in the user's voice is "requesting" a "program reservation" for "○○○".

Thereafter, the processor 130 detects, from the storage 110 , a control command corresponding to an utterance intention of “requesting a “program reservation” for “○○○”, and displays “○○” on the display device (not shown). It is possible to create a control command for performing the reserved recording function of "○". In this case, the processor 130 extracts a response message “recording of the requested program is reserved” from the corpus database of the broadcast service domain for the utterance intention of “requesting” “program reservation” for “○○○” Therefore, it can be output together through the speaker unit 120 .

Meanwhile, the storage 110 categorizes and stores the conversation topics corresponding to each domain for each context, and the processor 130 determines that the previous domain is a domain to process the user's spoken voice, the context corresponding to the user's spoken voice. may be determined, and a context to process the user's spoken voice may be generated from among the determined contexts and previous contexts based on the reliability between the user's spoken voice and the determined context.

Here, the category for each context of the conversation topic corresponding to each domain stored in the storage 110 may correspond to the corpus database for the above-described domain. That is, the context related to the conversation topic corresponding to each domain includes various conversation patterns that can occur within each domain, and not only these conversation patterns, but also words such as specific terms, proper nouns, and place names used in specific conversation topics. It can be included by categorizing it according to the topic of conversation.

Then, when the previous domain is determined as a domain to process the currently received user uttered voice, the processor 130 determines a context corresponding to the user uttered voice, and the reliability between the currently received user uttered voice and the context corresponding to the user uttered voice and comparing the reliability between the currently received user's uttered voice and the previous context to determine a context in which to process the currently received user's uttered voice to generate a system response.

For example, if the previously received user's spoken voice is "When watching TV in a large room, turn off the lights in other rooms and make the big room cool and dark", the processor 130 processes the previously received user's spoken voice. The domain is determined as a domain related to the display device, and the context is determined to be related to lighting and temperature when watching TV in a large room. Thereafter, when the received user's uttered voice is "How many channels do movies come out of?", the processor 130 may determine a domain for processing the user's uttered voice as a domain related to the display device, like the previous domain, and "movies." It may be determined that the context corresponding to "what number of channels appears?" relates to channel information. In addition, the processor 130 has a reliability score of 80 between a user uttered voice saying “how many times is a movie on a channel?” and a context related to channel information, and a user uttered voice saying “how many times is a movie on a channel?” and TV When the reliability score between the context regarding the light and the temperature during viewing is 40, the context in which the user uttered voice “what is the number of channels for the movie?” may be determined as a context regarding channel information.

That is, even if the domain for processing the previous user's uttered voice is already determined, the processor 130 may determine each domain for processing the newly received user's uttered voice each time a new user's uttered voice is received, and also newly Even if the domain for processing the received user's uttered voice is the same as the previous domain, it is possible to determine a context for processing the newly received user's uttered voice from among a plurality of contexts included in the domain.

And, if the determined context is the same as the previous context, the processor 130 processes the newly received user uttered voice in the previous context, and if the determined context is different from the previous context, processes the newly received user uttered voice within the determined context. do.

On the other hand, when the determined context is determined as a context for processing the user's spoken voice, the processor 130 stores information related to the previous context in the storage 110 , and when processing of the spoken voice in the determined context is finished, the stored A newly uttered voice may be processed based on information related to the previous context.

For example, when a user uttered voice is received, "When watching TV in a large room, turn off other rooms and make the large room cool and dark", the processor 130 sets the display device as a domain to process the received user uttered voice. domain, and the context is determined to be about light and temperature when watching TV in a large room, and in response to the user's voice, a system response such as "Shall we turn off the lights in the living room and hallway together?" can be generated.

Thereafter, when a user's spoken voice asking "what channel is a movie on?" is received, the processor 130 determines a domain for processing the user's uttered voice as a domain related to the display device as in the previous domain. It is determined that the context corresponding to "how many times?" relates to channel information. Accordingly, the processor 130 stores information related to the context related to lighting and temperature when watching TV in the storage 110 , and processes a user uttered voice received in the context related to channel information. Here, the information related to the context related to the light and temperature when watching TV includes the user uttered voice received within the context of the light and temperature when watching TV, data used to process the received user uttered voice, and the received user uttered voice. It may include information about the correspondingly generated system response.

On the other hand, when the processing of the user's spoken voice received in the context related to channel information is finished, the processor 130 reads out information related to the context related to the light and temperature when watching TV stored in the storage 110 again, and based on this Thus, a newly received user uttered voice can be processed.

For example, when the processing of the user's spoken voice received in the context related to channel information is finished, the processor 130 reads information related to the context related to lighting and temperature when watching TV stored in the storage 110 again. , "I will continue the conversation that was previously in progress." It is possible to generate a voice message such as, and output through the speaker unit (120).

Then, when the processor 130 outputs the voice message of “Shall we turn off the lights in the living room and hallway together?” again through the speaker unit 120 based on the context related to the light and temperature when watching TV, the user asks “ A new utterance voice of “turn off the living room and turn on the hallway” is received, and accordingly, the processor 130 processes the user uttered voice “turn off the living room and turn on the hallway” based on the context regarding lights and temperature when watching TV can do.

The above-described example describes a case where the previous domain is determined as a domain to process a user's uttered voice, and when a domain detected corresponding to a user's uttered voice other than the previous domain is determined as a domain for processing the user's uttered voice It will also be explained.

When the domain detected in response to the user's uttered voice is determined as a domain to process the user's uttered voice, the processor 130 stores information related to the previous domain in the storage unit 110, and processing of the uttered voice in the detected domain is performed. Upon completion, the newly uttered voice may be processed based on the stored information related to the previous domain.

For example, when a user's voice saying "When watching TV in a large room, turn off the lights in the other room and make the large room cool and dark" is received, the processor 130 determines as a domain related to the display device, and relates to the display device. The domain will process the user uttered voice. In this case, the processor 130 may generate a system response such as "Shall we turn off the lights in the living room and hallway together?"

Thereafter, when a user uttered voice “Register a wedding schedule for next weekend” is received from the user, the processor 130 sets the domain corresponding to the user uttered voice “True, register a wedding schedule for next weekend” into the schedule. It is possible to detect as a domain related to the user's speech and schedule a domain to process a user's voice saying "Really, register a wedding schedule next weekend" based on the reliability of each of the domain related to the user's voice and the schedule and the domain related to the display device. It can be determined by the domain of

In this case, the processor 130 stores information related to the previous domain, that is, information related to the domain related to the display device, in the storage 110 , and a user utterance such as “Saturday 12:00” received from the user in the domain related to the schedule. By processing the voice to generate a system response such as “schedule registration is complete” and outputting it through the speaker unit 120 , when the speech processing in the domain related to the schedule ends, it is again displayed on the display device stored in the storage unit 110 . While reading information related to the related domain, a voice message such as “I will continue the conversation that was previously in progress” may be generated and outputted through the speaker unit 120 .

Then, when the processor 130 outputs the voice message of “Shall we also turn off the lights in the living room and hallway together?” that was previously conducted on the basis of the domain related to the display device through the speaker unit 120 again, the user tells the user “Turn off the living room and turn off the hallway lights.” A new uttered voice “Turn on” is received, and accordingly, the processor 130 may process a user uttered voice “Turn off the living room and turn on the hallway” based on the domain related to the display device.

2 is a diagram illustrating a process of processing a user uttered voice according to an embodiment of the present invention.

Referring to FIG. 2 , when a user's spoken voice is input, the processor 130 converts a voice signal into text by referring to the language model 211 using an Automatic Speech 210 (ASR) module, and SLU (Spoken Language Understanding). ) 220 module may be used to perform various analyzes so that the processor 130 can understand the text input by referring to the SLU Model 221 .

In addition, the processor 130 may generate various natural language conversations while analyzing the user's uttered voice and the user's utterance intention using the DM (Dialogue Manager) 230 module. Specifically, the processor 130 detects a domain corresponding to the user uttered voice using the DM 230 module, and divides the user uttered voice among the detected domain and the previous domain based on the reliability between the user uttered voice and the detected domain. You can decide which domain to process.

In addition, the processor 130 determines a context corresponding to the user uttered voice using the DM 230 module, and processes the user uttered voice among the context determined based on the reliability between the user uttered voice and the determined context and the previous context. You can decide which context to use. In particular, the processor 130 may store information related to the previous domain or information related to the previous context in the Context Stack 231 , and store data related to conversation content between the user and the electronic device 100 in the Context History 232 . can Here, the context stack 231 means a storage space for storing information related to a previous context when switching to another conversation topic during a conversation with the user, and the context history 232 is related to the conversation between the user and the electronic device A storage space for storing data.

In addition, the processor 130 may monitor a conversation between the electronic device 100 and the user by using the Context Manager & Task Delegating 240 module, and may support information available in the conversation with the user.

In addition, the processor 130 processes at least one of the Action Plannig Agent 250 , the Family Member Agent 260 , and the Health Agent 270 using the Context Manager & Task Delegating 240 module to process the user's spoken voice. can make it

On the other hand, the processor 130 uses the Action Plannig Agent 250 to control the at least one external device based on the user uttered voice analyzed in the DM 230 module, the user uttered intention, the domain and context to process the user uttered voice, etc. function can be controlled.

Also, the processor 130 may determine which external device to use to perform a function corresponding to the user's uttered voice based on the function and state of the external device using the Action Plannig Agent 250 . Here, the processor 130 may determine the function and state of the external device based on the Action Ontology 251 and the Things Graph DB 252 using the Action Plannig Agent 250 .

Meanwhile, the processor 130 may convert the generated system response into text so that the user can understand it using the Spoken Language Understanding (SLU) 280 module.

In addition, the processor 130 may convert the text into a voice signal using a Text to Speech (TTS) 290 module. Accordingly, the processor 130 may convert the generated system response into a voice signal and output it through the speaker unit 120 .

Meanwhile, FIG. 3 is a flowchart illustrating a process of processing a user uttered voice according to an embodiment of the present invention.

Referring to FIG. 3 , when a user's spoken voice is input, the processor 130 may convert a voice signal into text using an Automatic Speech Recognition (ASR) module (S310), and detect a domain corresponding to the user's uttered voice. (S321), it is possible to analyze the reliability between the user's uttered voice and the detected domain and the previous domain (S322). In addition, the processor 130 may analyze the user's utterance intention (S320). Since the detailed description of detecting a domain corresponding to the user's spoken voice, analyzing the reliability between the user's uttered voice and the detected domain and the previous domain, and analyzing the user's uttered intention has been previously described, a detailed description thereof will be omitted.

Thereafter, the processor 130 may determine whether the domain to process the user's uttered voice is the same as the previous domain based on the reliability between the user uttered voice and the detected domain and the previous domain ( S330 ).

Here, when it is determined that the domain to process the user uttered voice is the same as the previous domain, the processor 130 determines a context corresponding to the user uttered voice, and analyzes a reliability score between the user uttered voice and the determined context and the previous context It can be done (S340).

In addition, the processor 130 may determine whether the context in which the user's spoken voice is to be processed is the same as the previous context or whether processing is possible in the current state based on the confidence score between the user's spoken voice and the determined context and the previous context ( S350).

Here, when it is determined that the context in which the user uttered voice is to be processed is the same as the previous context, the processor 130 may process state management based on the previous context (S360). Then, the processor 130 may complete the processing of the received user uttered voice based on the previous context (S370). Also, the processor 130 may read the context stored before the current user's spoken voice is received from the storage 110 again and process the new user's spoken voice ( S380 ).

Meanwhile, when the domain for processing the user's uttered voice is determined to be a domain corresponding to the detected user's uttered voice instead of the previous domain, the processor 130 stores information related to the previous domain and the previous context (S331), and the detected user's utterance The context state management of the domain corresponding to the voice is performed (S332) to process the user's uttered voice.

In addition, even when it is determined that the domain for processing the user's uttered voice is the same as the previous domain, the processor 130 proceeds to the previous Information related to the current context is stored (S351), and a new Ccontext state management is performed in the previous domain (S352) to process the user's uttered voice.

Meanwhile, FIG. 4 is a diagram illustrating a processing process when a domain for processing a user's uttered voice is changed according to an embodiment of the present invention.

For example, when a user's voice saying "When watching TV in a big room, turn off the lights in another room and make this room cool and dark" is received from the user (411), the processor 130 "lights another room when watching TV in a big room" A system such as “Should I turn off the lights in the living room and hallway as well?” A response may be generated and output (421).

Thereafter, when a user's uttered voice is received from the user, "Yes, register the wedding schedule for next weekend" (412), the processor 130 responds to the user's uttered voice 'True, register the wedding schedule for the next weekend'. The domain related to the schedule is detected and the intention of the user's speech is analyzed (S431), and the reliability between the user's speech voice and the domain related to the detected schedule is analyzed (S432), and according to the analyzed reliability result, "True, wedding schedule for next weekend It may be determined whether the domain to process the user's utterance "Please register" is the same as the domain related to light and temperature when watching TV before (S433). Here, the processor 130 determines that the domain to process the user's utterance voice "Register the wedding schedule next weekend" is a domain related to a schedule rather than a domain related to lights and temperature when watching TV before, in the previous domain Information related to the ongoing context may be stored in the storage unit 110 (S434), and a context suitable for a domain related to a newly determined schedule may be generated to process the user's uttered voice (S435). As shown in FIG. 4 , the processor 130 may generate and output a system response “What day of the week, what time is it?” (S422), and in response to this, a user uttered voice saying “Saturday, 12:00” from the user When this is received, it is still the same context within the domain regarding the schedule, so it is possible to generate and output 423 a system response "Do you have any additional information?" while maintaining it. In addition, when a user uttered voice is received from the user, "Kim Dae-kyung's wedding, the location is Sinchon ○○○ University Alumni Hall" (414), the processor 130 is still the same context in the domain regarding the schedule, so it is maintained A system response “schedule registration is complete” may be generated and output (424), and a user uttered voice saying “OK” may be received from the user (415). Accordingly, the processor 130 may determine that the currently ongoing task has been completed (S436).

Thereafter, the processor 130 reads a domain and a related context related to light and temperature from the storage 110 when watching TV in progress ( S437 ), and a new utterance received from the user based on the previous domain and related context ( S437 ) The voice may be processed (S438).

For example, the processor 130 outputs a system response such as "We will continue the planning that was previously in progress", and when a user's voice saying "Turn off the living room and turn on the hallway" is received from the user, when watching TV, the previous domain Based on the domains of light and temperature, and the relevant context, process the user's spoken voice, and turn off the living room and hallway lights as well? When a user uttered voice "" is received (417), the processor 130 still determines that it is about the domain and related contexts related to light and temperature when watching TV, and maintains it, while maintaining the system such as "How much air conditioning?" You can generate a response and print it out.

Meanwhile, FIG. 5 is a domain for processing when a domain for processing a user's uttered voice is not changed according to an embodiment of the present invention.

For example, when a user uttered voice saying "When watching TV in a large room, turn off the lights in another room and make this room cool and dark" is received from the user (511), the processor 130 is "lighted in another room when watching TV in a large room" Turn it off and make this room cool and dark" as the domain to process the user's voice. When watching TV, it is decided as a domain related to lights and temperature, and a system such as "Should I turn off the lights in the living room and hallway as well?" A response may be generated and output (521).

Thereafter, when a user uttered voice saying “what channel is a movie on?” is received from the user (512), the processor 130 is a channel corresponding to the received user uttered voice “how many times is a movie on a channel?” By detecting a domain related to information and analyzing the user's utterance intention (S531), the reliability of the domain value for the user's uttered voice and the detected broadcast program is analyzed (S532), and according to the analyzed reliability result, "the number of times the channel for the movie is It may be determined whether the domain to process the user's utterance voice "isn't it?" is the same as the domain related to light and temperature when watching TV before (S533). Here, if the processor 130 determines that the domain for processing the user's utterance voice, "What time is the movie on the channel?" is the same as the previous domain, the domain related to lights and temperature when watching TV, light and temperature when watching TV Reliability between contexts regarding channel information detected in response to a user uttered voice saying, “How many channels is a movie on?” in the domain related to It may be determined whether the context for processing the user's utterance voice of "what channel is the movie on?" is the same as the previous context, which is the context regarding lights and temperature when watching TV (S535). Here, the processor 130 may determine the context in which to process the user's uttered voice, "What time is the channel on which the movie comes out?" is a context related to the detected channel information, not the previous context. Context-related information may be stored in the storage unit 110 (S536).

In addition, the processor 130 may process the user's uttered voice by creating a context related to new channel information within the domain related to light and temperature when watching TV (S537).

That is, to the user's voice saying "what channel is the movie on?", the processor 130 may generate and output a system response such as "there are channels 23, 37, 101, 157, and 274" (522). ), when the user's uttered voice "34" is received from the user, the processor 130 may generate and output a system response such as "the channel has been changed to 34" ( 523 ).

Thereafter, the processor 130 may determine that the currently ongoing task has been completed (S538), and the processor 130 reads the context related to the light and temperature when watching TV that was previously in progress from the storage 110 ( S539-1), based on this, a new spoken voice received from the user may be processed (S539-2). For example, the processor 130 outputs a system response such as "I will proceed with the previously ongoing planning" (524), and when a user's voice saying "Turn off the living room and turn on the hallway" is received from the user (514) , it is possible to process the user's uttered voice based on the previous context, the context of light and temperature when watching TV, and generate and output a system response "Do you want to turn off the lights in the living room and hallway together?" (525), and for this, the user When a user's utterance voice is received (515), "Please do it to a medium degree" from The same system response may be generated and output (526).

Meanwhile, FIG. 6 is a diagram illustrating a processing process in the DM module of the present invention.

Referring to FIG. 6 , when a user uttered voice saying “manage energy efficiently” is received, the processor 130 uses the Dialog Manager 10 to refer to Dialog Context, Dialog History, and Question Answering, and a Natural Language Processing (NLP) 640 ) can be controlled to analyze the intention of the user's utterance. For example, the Dialog Manager 10 transmits information about the content of the unclear part in the user's uttered voice saying "manage energy efficiently" to the NLP (640) module (610), and the NLP (640) The module can be used to analyze the exact meaning and intent of the user's spoken voice by applying unnecessary or unused energy management or efficient performance figures.

And, when the Dialog Manager (10) requests unclear content or conditions or information necessary for selection by the Context Manager (650) module (620), using the Context Manager (650) module, "Efficient energy management" It is possible to check the context of various external devices related to the user's uttered voice, "Do it," and analyze the context related to the user's uttered voice to recommend a helpful task (670), or control to suggest a related task (680). ).

In addition, when the processor 130 requests the content necessary for planning with the Planning Manager 660 module ( 630 ), using the Planning Manager ( 660 ) module, the processor 130 responds to the user’s utterance of “manage energy efficiently” by using various external devices. Through the combination, it is possible to control an appropriate system response corresponding to the user's voice, that is, to perform action planning regarding how to combine various external devices to perform a function ( 660 ).

Meanwhile, FIG. 7 is a diagram for explaining a process performed in a system including the electronic device 100 and a database including information about an external device according to an embodiment of the present invention.

Referring to FIG. 7 , the processor 130 is a database (KB) 20 in which the Dialog Manager 10 , the NLP 640 , the Context Manager 650 and the Planning Manager 660 stores information about external devices. When information on functions, performance, characteristics, etc. of the external device is requested from process and the Batch Manager 50 may transmit information about the function, performance, characteristics, etc. of the external device to the Dialog Manager 10 , the NLP 640 , the Context Manager 650 , and the Planning Manager 660 .

Accordingly, the processor 130 uses the Dialog Manager 10 , the NLP 640 , the Context Manager 650 , and the Planning Manager 660 based on the received information about the function, performance, characteristic, etc. of the external device to utter the user's utterance. It can process the voice to generate and output a system response.

8 is a block diagram illustrating a configuration of an electronic device according to another embodiment of the present invention.

Referring to FIG. 8 , the electronic device 100 includes a storage unit 110 , a speaker unit 120 , a processor 130 , and a communication unit 140 , and includes the storage unit 110 , the speaker unit 120 , and the processor. Since 130 has been previously described, a detailed description thereof will be omitted.

The communication unit 140 may communicate with at least one external device, and the processor 130 may be configured based on a context in which a function control of at least one external device is required within a domain in which a system response corresponding to the spoken voice is determined. When generated, a system response for controlling the function of at least one external device may be generated based on the information on the function of the external device.

Specifically, when a user's uttered voice of “lower the house temperature” is received, the processor 130 may determine a temperature-related domain as a domain to process the user's uttered voice, and at least one external device within the temperature-related domain When the function control is generated based on the required context, the processor 130 may search for an external device related to the temperature control function among various external devices disposed in the house. For example, the processor 130 may search for external devices such as an air conditioner, a window, and a light in relation to the temperature control function, turn on the air conditioner to lower the temperature, close the window to increase energy efficiency, and turn off the light A control command to perform a function may be transmitted to each of the air conditioner, the window, and the lamp.

In addition, the processor 130 may determine at least one external device conforming to a system response corresponding to a user uttered voice in consideration of functions, performance, and characteristics of various external devices to control the temperature within the temperature-related domain. have.

Here, the storage 110 may further store information on functions of the external device, and the communication unit 140 may receive function information on at least one external device added in a preset network.

In addition, the processor 130 may update information stored in the storage 110 based on the received function information on at least one external device.

For example, assuming a preset network as an internal network, if a new display device is additionally provided inside the house, the communication unit 140 may receive function information on the new display device added to the internal network. Also, the processor 130 may update information stored in the storage 110 based on the received function information on the new display device. Accordingly, the processor 130 may generate a system response by further considering the function information of the new display device added in processing the user's uttered voice received thereafter.

Meanwhile, the processor 130 generates a system response by determining a domain to process a user's uttered voice based on the uttered history information, and the uttered history information processes a previously received user's uttered voice and a previously received user's uttered voice. It may include at least one of information related to one domain and a system response corresponding to a previously received user uttered voice.

For example, the processor 130 is unable to detect a domain or context corresponding to the user uttered voice or is unable to determine a domain or context in which to process the user uttered voice based on the reliability between the user uttered voice and the detected domain or context. In this case, a domain or context in which the user's uttered voice is to be processed may be determined based on the utterance history information.

In addition, this utterance history information is referred to in generating a previously received user uttered voice, information about a context related to a domain in which the previously received user uttered voice has been processed, and a system response corresponding to the previously received user uttered voice. It may include information about the external device, information about the function of the external device, and the like.

Meanwhile, FIG. 9 is a block diagram illustrating the configuration of an electronic device according to another embodiment of the present invention. Referring to FIG. 9 , the electronic device 100 includes a storage unit 110 , a speaker unit 120 , a processor 130 , and a microphone unit 160 , the storage unit 110 , the speaker unit 120 , and Since the processor 130 has already been described, a detailed description thereof will be omitted.

In addition, the microphone unit 160 may receive a user's spoken voice, and the microphone unit 160 may be provided in the electronic device 100 , but may be separated and exist outside, and is implemented in a detachable form. it could be

Meanwhile, as described above, the electronic device 100 may output a system response corresponding to the user's uttered voice, and may output the system response and provide a feedback effect to the user at the same time. Also, the electronic device 100 may include a display unit (not shown), and may provide a feedback effect through the display unit (not shown).

Specifically, the processor 130 generates and outputs a system response corresponding to the user's uttered voice and at the same time indicates that the operation corresponding to the user's uttered voice is completed through one of an acoustic signal, a message, and a user user interface screen. may be provided to, where the message or the user user interface screen may be output through a display unit (not shown).

For example, when a user uttered voice saying “lower the house temperature” is received, the processor 130 searches for an external device related to a temperature control function among various external devices disposed in the house, and turns on the air conditioner accordingly. Control commands can be sent to each of the air conditioner, window, and light to reduce the temperature, close the window to increase energy efficiency, and turn off the light, while simultaneously "processing your request" or "request" You can give a feedback effect to the user by outputting an acoustic signal such as "I turned on the air conditioner, close the window and turn off the light according to your request" through the speaker unit 120, or "The requested part has been processed" or " As requested, the air conditioner was operated, the window was closed, and the light was turned off” in text form, or a user interface screen or icon, etc. may be

In addition, the processor 130 may output a system response corresponding to the user's uttered voice and additionally recommend or suggest another operation or task related to the output system response to the user.

For example, when a user uttered voice saying “show me the movie I watched yesterday” is received, the processor 130 transmits a control command to display the movie that I saw yesterday on the TV to the TV, and then controls the TV to play the movie In relation to the keyword "play movie" among the commands, another action or task suitable for the current movie playing situation may be additionally recommended or suggested to the user. Specifically, the processor 130 asks "Do you want to lower the light intensity of the lamp?" or "The robot vacuum is currently running, but would you like to stop the robot vacuum as it interferes with watching movies?" or "Should I close the windows so they don't get in the way of watching a movie?" An operation or task related to a control command for playing a movie on the TV, such as the like, may be additionally suggested and recommended to the user.

As such, the processor 130 generates and outputs a system response corresponding to the user's uttered voice and at the same time indicates that the operation corresponding to the user's uttered voice is completed through one of an acoustic signal, a message, and a user interface screen. It can be provided to the user, and another action or task related to the system response corresponding to the user's voice can be additionally recommended or suggested to the user.

FIG. 10 is a block diagram illustrating a specific configuration of the electronic device shown in FIG. 1 .

Referring to FIG. 10 , the electronic device 100 ′ includes a storage unit 110 , a speaker unit 120 , a processor 130 , a communication unit 140 , a display unit 150 , a microphone unit 160 , and a sensor unit ( 170). A detailed description of the portion overlapping with the configuration shown in FIG. 1 among the configurations shown in FIG. 10 will be omitted.

The processor 130 controls the overall operation of the electronic device 100 .

Specifically, the processor 130 includes the RAM 131 , the ROM 132 , the main CPU 133 , the graphic processing unit 134 , the first to n interfaces 135-1 to 135-n, and the bus 136 . include

The RAM 131 , the ROM 132 , the main CPU 133 , the graphic processing unit 134 , and the first to n interfaces 135 - 1 to 135 - n may be connected to each other through the bus 136 .

The first to n-th interfaces 135-1 to 135-n are connected to the various components described above. One of the interfaces may be a network interface connected to an external device through a network.

The main CPU 133 accesses the storage unit 110 and performs booting using the O/S stored in the storage unit 110 . Then, various operations are performed using various programs, contents, data, etc. stored in the storage unit 110 .

The ROM 132 stores an instruction set for system booting and the like. When the turn-on command is input and power is supplied, the main CPU 133 copies the O/S stored in the storage unit 140 to the RAM 131 according to the command stored in the ROM 132, and executes the O/S. Boot the system. When booting is completed, the main CPU 133 copies various application programs stored in the storage unit 140 to the RAM 131 , and executes the application programs copied to the RAM 131 to perform various operations.

The graphic processing unit 134 generates a screen including various objects such as icons, images, and texts by using a calculation unit (not shown) and a rendering unit (not shown). A calculation unit (not shown) calculates attribute values such as coordinate values, shape, size, color, etc. at which each object is to be displayed according to the layout of the screen based on the received control command. The rendering unit (not shown) generates screens of various layouts including objects based on the attribute values calculated by the calculation unit (not shown). In particular, the graphic processing unit 134 may determine a font, size, color, etc. of a character while converting a system response generated in response to a user's spoken voice into a text form. The screen generated by the rendering unit (not shown) may be displayed through the display unit 150 .

Meanwhile, the above-described operation of the processor 130 may be performed by a program stored in the storage 110 .

The storage unit 140 stores various data such as an O/S (Operating System) software module for driving the display device 100 ′ and various multimedia contents.

In particular, the storage unit 110 detects a domain corresponding to a user uttered voice, and determines a domain to process a user uttered voice from among a detected domain and a previous domain based on the reliability between the user uttered voice and the detected domain to respond to the system software module for creating This will be described in detail with reference to FIG. 11 .

Meanwhile, the sensor unit 170 may include various sensors, and may include a touch sensor for recognizing a touch, a motion sensor for detecting a user's movement, and the like. In particular, the sensor unit 170 may include a sensor for discriminating the user's spoken voice and external noise.

11 is a diagram of a software module stored in a storage unit according to an embodiment of the present invention.

Referring to FIG. 11 , the storage unit 110 includes a Dialogue Manager module 111 , a Dialogue Context module 112 , an NLP module 113 , an NLG module 114 , a Discourse Manage module 115 , and a Question Answering module 116 . ), programs such as the Context Manager module 117 and the Action Planner module 118 may be stored.

Meanwhile, the above-described operation of the processor 130 may be performed by a program stored in the storage 110 . Hereinafter, detailed operations of the processor 130 using the program stored in the storage 110 will be described in detail.

The Dialogue Manager module 111 may perform a function of analyzing the user's uttered voice to detect the content and uttering intention of the user's uttered voice, and may perform a function of generating a natural language conversation with the user. In particular, the Dialogue Manager module 111 is a main module of the dialog system, and may perform a function of managing the overall flow of a conversation between the user and the electronic device and overall managing other internal modules.

The Dialogue Context module 112 may perform a function of recording and managing content of conversations exchanged between the user and the electronic device 100 .

In addition, the NLP module 113 may process a natural language and perform a function of detecting a user's utterance intention based thereon.

The NLG module 114 may perform a function of converting a system response generated in response to a user's spoken voice into text.

Also, the Discourse Manage module 115 may perform a function of generating a system response based on the content of the user's uttered voice and utterance intention.

The Question Answering module 116 may perform a function of processing the content of the question received from the user.

The context manager module 117 may perform a function of providing necessary context information while monitoring the content of a conversation between the user and the electronic device 100 or detecting and providing a context corresponding to the user's uttered voice.

The action planner module 118 may perform a function of controlling the at least one external device in consideration of the function and performance of the at least one external device based on the analyzed content of the user's uttered voice and the utterance intention.

Here, the results output from the action planner module 118 may be determined in various types. For example, in general, if the user's spoken voice is processed, "The request has been successfully performed" may be output. can be output as In addition, if the user's utterance voice is processed but different from the user's utterance intention, "It was performed in a different way instead of the user's request" may be output.

In addition, if a part related to this is additionally processed in addition to the part requested through the user's voice, it may output something like "The requested part has been successfully performed. Also, another possible function has been found related to the requested part." have.

In addition, when manual manipulation of the user is required, it can be output as "User's selection is required to process the requested part", and when a request for parameter value is required, "Request input for parameter value" It may be printed as "I will give you". In addition, when two or more processing methods are determined for the user's spoken voice, "the user's selection is required" may be output.

Meanwhile, FIG. 12 is a flowchart illustrating a method for controlling an electronic device according to an embodiment of the present invention.

The control method of the electronic device including a storage unit for storing domain information categorized for each conversation topic shown in FIG. 12 detects a domain corresponding to a user's uttered voice (S1210).

Then, a system response is generated by determining a domain to process the user's spoken voice from among the detected domain and the previous domain based on the reliability between the user's spoken voice and the detected domain ( S1220 ).

Here, the storage unit categorizes and stores the conversation topic corresponding to each domain for each context, and the step of generating a system response includes, when the previous domain is determined as a domain to process the user's uttered voice, the context corresponding to the user's uttered voice. The system response may be generated by determining a context in which to process the user's spoken voice among the determined contexts and previous contexts based on the reliability between the user's spoken voice and the determined context.

In addition, the generating of the system response includes, when the determined context is determined as a context to process the user's spoken voice, stores information related to the previous context in the storage unit, and when the spoken voice processing in the determined context ends, the stored previous A newly uttered voice may be processed based on context-related information.

In addition, the generating of the system response includes, when the detected domain is determined as a domain to process the user's spoken voice, stores information related to the previous domain in the storage unit, and when the spoken voice processing in the detected domain ends, the stored previous A newly uttered voice may be processed based on domain-related information.

In addition, the generating of the system response may include: based on a confidence score according to whether at least one speech element constituting the user's speech voice and at least one speech element belonging to the detected domain are identical between the user's speech voice and the detected domain. reliability can be judged.

In addition, the generating of the system response may include, when a system response corresponding to the spoken voice is generated based on a context in which a function control of at least one external device is required within the determined domain, based on the information on the function of the external device. A system response for controlling a function of the at least one external device may be generated.

In addition, the method of controlling an electronic device according to an embodiment of the present invention further includes the step of receiving function information on at least one external device added in a preset network and updating the information on the function of the pre-stored external device. can do.

In addition, the method of controlling an electronic device according to an embodiment of the present invention further includes generating a system response by determining a domain to process a user's uttered voice based on the utterance history information, wherein the utterance history information has been previously received. It may include at least one of a user uttered voice, information related to a domain in which a previously received user uttered voice has been processed, and a system response corresponding to a previously received user uttered voice.

Also, the domain information may include at least one of control information for performing a task corresponding to a conversation topic and a conversation pattern for each conversation topic.

Also, the method of controlling an electronic device according to an embodiment of the present invention may further include receiving a user's spoken voice.

Meanwhile, a non-transitory computer readable medium in which a program for sequentially performing the control method according to the present invention is stored may be provided.

For example, detecting a domain corresponding to the user's uttered voice and generating a system response by determining a domain to process the user's uttered voice from among a detected domain and a previous domain based on the reliability between the user's uttered voice and the detected domain A non-transitory computer readable medium in which a program for performing a is stored may be provided.

The non-transitory readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, cache, memory, and the like, and can be read by a device. Specifically, the various applications or programs described above may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In addition, although a bus is not illustrated in the above-described block diagram of an electronic device, communication between respective components in the electronic device may be performed through a bus. In addition, each device may further include a processor such as a CPU or a microprocessor that performs the various steps described above.

In addition, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention pertains without departing from the gist of the present invention as claimed in the claims In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: storage unit 120: speaker unit
130: processor

Claims (20)

a storage unit for storing domain information categorized by conversation topic;
microphone unit;
speaker unit; and
detecting a domain corresponding to a user uttered voice received through the microphone unit,
A first confidence between the user uttered voice and the detected domain and a second confidence between the user uttered voice and a previous domain used for processing a previous user uttered voice received through the microphone unit before the user uttered voice determines a domain for processing the user's uttered voice from among the detected domain and the previous domain, generates a system response corresponding to the user's uttered voice based on the determined domain, and outputs the generated system response A processor for controlling the speaker unit so as to include;
The processor is
When it is determined that the detected domain is a domain for processing the user's uttered voice, information related to the previous domain is stored in the storage unit, and when the uttered voice processing in the detected domain is terminated, the stored previous domain and An electronic device characterized in that the newly uttered voice is processed based on related information. According to claim 1,
The storage unit,
categorizing and storing conversation topics corresponding to each of the domains for each context,
The processor is
When the previous domain is determined as the domain to process the user's uttered voice, a context corresponding to the user's uttered voice is determined, and based on the reliability between the user uttered voice and the determined context, among the determined context and the previous context and generating the system response by determining a context in which to process the user's uttered voice. 3. The method of claim 2,
The processor is
When the determined context is determined as a context to process the user's spoken voice, information related to the previous context is stored in the storage unit, and when processing of the spoken voice in the determined context is terminated, the stored previous context related information An electronic device characterized in that the newly uttered voice is processed based on the information. delete According to claim 1,
The processor is
and determining the reliability between the user's uttered voice and the detected domain based on a reliability score according to whether at least one utterance constituting the user's uttered voice is identical to at least one uttered element belonging to the detected domain. electronic device with According to claim 1,
It further includes; a communication unit for performing communication with at least one external device,
The processor is
When a system response corresponding to the spoken voice is generated based on a context in which the function control of the at least one external device is requested within the determined domain, the system response of the at least one external device is generated based on the information on the function of the external device. and generating the system response for controlling a function. 7. The method of claim 6,
The storage unit further stores information on the function of the external device,
The communication unit receives function information on at least one external device added in a preset network,
The processor is
and updating the information stored in the storage unit based on the received function information on the at least one external device. According to claim 1,
The processor is
determining a domain to process the user's uttered voice based on utterance history information to generate the system response,
The utterance history information includes at least one of a previously received user uttered voice, information related to a domain in which the previously received user uttered voice has been processed, and a system response corresponding to the previously received user uttered voice. electronic device with According to claim 1,
The domain information is
The electronic device comprising at least one of control information for performing a task corresponding to the conversation topic and a conversation pattern for each conversation topic. According to claim 1,
The electronic device further comprising; a microphone unit for receiving the user's speech voice. In the control method of an electronic device comprising a storage unit for storing domain information categorized by conversation topic,
receiving a user uttered voice;
detecting a domain corresponding to the user's spoken voice;
the detection based on a first confidence between the user uttered voice and the detected domain and a second confidence between the user uttered voice and a previous domain used before the detected domain to process a previous user uttered voice determining a domain to process the user's uttered voice from among the old domain and the previous domain;
storing information related to the previous domain in the storage unit when the detected domain is selected as a domain to process the user's uttered voice;
generating a system response based on the selected domain;
outputting the system response; and
and processing a new spoken voice based on the stored information related to the previous domain when the spoken voice processing in the selected domain is finished. delete delete delete 12. The method of claim 11,
The step of generating the system response comprises:
and determining the reliability between the user's spoken voice and the detected domain based on a reliability score according to whether at least one speech element constituting the user's speech voice and at least one speech element belonging to the detected domain are identical. A method for controlling an electronic device. 12. The method of claim 11,
The step of generating the system response comprises:
When a system response corresponding to the spoken voice is generated based on a context in which function control of at least one external device is required within the determined domain, the at least one and generating the system response for controlling a function of an external device. 17. The method of claim 16,
The method of controlling an electronic device further comprising: receiving function information on at least one external device added in a preset network and updating the stored information on the function of the external device. 12. The method of claim 11,
Generating the system response by determining a domain to process the user's uttered voice based on the utterance history information;
The utterance history information includes at least one of a previously received user uttered voice, information related to a domain in which the previously received user uttered voice has been processed, and a system response corresponding to the previously received user uttered voice. A method for controlling an electronic device. 12. The method of claim 11,
The domain information is
and at least one of control information for performing a task corresponding to the conversation topic and a conversation pattern for each conversation topic. delete

Images