KR102219800B1 - Voice server, voice recognition server system, and method for operating the same - Google Patents

Abstract

A method of operating a voice server according to an aspect of the present invention includes: receiving voice data from a home appliance, converting the received voice data into text data, and determining a voice command by analyzing text data. , Transmitting a signal corresponding to the determined voice command to a predetermined server, receiving a response signal based on the determined voice command from the predetermined server, and converting the response signal based on the voice command into voice data By including the step of transmitting to the appliance, voice commands can be recognized and processed efficiently.

Description

Voice server, voice recognition server system, and method for operating the same {Voice server, voice recognition server system, and method for operating the same}

The present invention relates to a voice server, a voice recognition server system, and a method of operation thereof, and more particularly, to a voice server capable of recognizing a user's natural language voice command and controlling a home appliance, a voice recognition server system, and an operation method thereof. About.

Home appliances such as air conditioners, washing machines, and vacuum cleaners used in predetermined spaces such as homes and offices each performed their own functions and operations according to user manipulation.

For example, an air conditioner includes a compressor that compresses the refrigerant, a condenser that condenses the compressed refrigerant, an expansion valve that expands the condensed refrigerant, and an evaporator that heats the expanded refrigerant with indoor air to cool and heat the indoor space. As a device, it performs the function of adjusting the room temperature.

In addition, the refrigerator stores food, the washing machine processes laundry, the vacuum cleaner performs a cleaning function, and the cooking appliance cooks food.

In order to operate a home appliance such as an air conditioner, a user can directly manipulate a button provided on the home appliance body or use a remote control device such as a remote control to avoid the inconvenience of having to move to the body and input it each time. .

However, even in the case of using a remote control, since the user has to select and input operation keys for each function, the use is inconvenient, and when the room is dark, there is a problem that a separate lighting for identifying the remote control and operation keys is required. .

Accordingly, research on a method of controlling a home appliance using voice recognition technology is increasing.

In the prior art 1 (Publication Patent Publication No. 10-1999-00069703), a remote control for an air conditioner has a voice input unit and a signal processing unit to generate and transmit an operation signal according to voice recognition.

Prior Art 2 (Public Patent Publication No. 10-2006-0015092) converts an input voice signal into a digital signal and text, then checks the existence of a matching control command in the database, and if there is a matching control command, each of the air conditioners The device is controlled, and if there is no matching control command, a keyword is extracted and each device in the air conditioner is controlled according to the associated control command.

However, there is a limit to the system resources that individual devices such as remote controllers and air conditioners can have. In particular, in order to recognize natural language rather than just a few words, it is difficult to implement with an embedded module installed in individual devices because a high amount of computation is required.

Accordingly, the speech recognition technologies of the prior art 1 and 2 have limitations in recognizing and processing various natural language voice commands of users around the world.

Therefore, there is a need for a method capable of recognizing and processing natural language without limitation of system resources of individual devices, and conveniently controlling a home appliance.

In addition, in order to improve usability and satisfaction, it is necessary to provide a user experience (UX) in consideration of various situations that may occur during a voice recognition process and a home appliance control process.

1. Unexamined Patent Publication No. 10-1999-00069703 (published on September 6, 1999) 2. Unexamined Patent Publication No. 10-2006-0015092 (published on February 18, 2006)

An object of the present invention is to provide a speech recognition method and system capable of efficiently recognizing and processing natural language.

An object of the present invention is to provide a voice recognition server system capable of controlling a home appliance by recognizing a user's voice command.

An object of the present invention is to provide a user experience (UX) in consideration of various situations that may occur during a voice recognition process and a home appliance control process.

In order to achieve the above or other objects, a method of operating a voice server according to an aspect of the present invention includes: receiving voice data from a home appliance, converting the received voice data into text data, and converting the text data. Analyzing and determining a voice command, transmitting a signal corresponding to the determined voice command to a predetermined server, receiving a response signal based on the determined voice command from the predetermined server, and a response based on the voice command By including the step of converting the signal into voice data and transmitting it to the home appliance, voice commands can be efficiently recognized and processed.

In order to achieve the above or other objects, a voice server according to an aspect of the present invention includes an automatic speech recognition server that receives voice data from a home appliance and converts the received voice data into text data. , A natural language processing server that receives text data from an automatic speech recognition server and determines a voice command by analyzing the received text data, and a home appliance by converting a response signal based on the voice command into voice data By including a text-to-speech server that is transmitted to the computer, it is possible to efficiently recognize and process voice commands.

In order to achieve the above or other objects, a method of operating a voice recognition server system according to an aspect of the present invention includes the steps of determining a voice command by analyzing voice data received from a home appliance by a voice server, and the linked service server Receiving a signal corresponding to the voice command determined from, generating a request signal corresponding to the voice command determined by the linked service server, receiving a request signal from the linked service server by the home appliance control server, home appliance control server Transmitting a control signal based on the request signal to the home appliance, and, by including the step of transmitting the voice data including the processing result information based on the voice command to the home appliance by the voice server, various Service can be provided.

In order to achieve the above or other objects, a voice recognition server system according to an aspect of the present invention includes a voice server that receives voice data from a home appliance, analyzes the received voice data, and determines a voice command, and determines from the voice server. A linked service server that receives a signal corresponding to a voice command and generates a request signal corresponding to the determined voice command, and a home appliance control server that transmits a control signal based on the request signal received from the linked service server to the home appliance And, the voice server transmits voice data including processing result information based on the voice command to the home appliance, thereby providing various services according to voice input.

According to at least one of the embodiments of the present invention, since the home appliance operates according to a voice input, the user does not need to operate the remote control, and user convenience can be increased.

In addition, according to at least one of the embodiments of the present invention, it is possible to efficiently recognize and process natural language.

In addition, according to at least one of the embodiments of the present invention, it is possible to provide a user experience (UX) in consideration of various situations that may occur during a voice recognition process and a home appliance control process.

Meanwhile, other various effects will be directly or implicitly disclosed in the detailed description according to the embodiments of the present invention to be described later.

1 is a schematic diagram of a smart home system including a voice recognition server system and a home appliance according to an embodiment of the present invention.
2 is a diagram illustrating various examples of a home appliance.
3A is an example of a voice recognition server system according to an embodiment of the present invention.
3B is an example of a voice recognition server system according to an embodiment of the present invention.
4 to 6 are diagrams illustrating a signal flow of a voice recognition server system according to an embodiment of the present invention.
7 to 9 are diagrams illustrating a signal flow of a voice recognition server system according to an embodiment of the present invention.
10 is a diagram illustrating an example of an internal block diagram of a server according to an embodiment of the present invention.
11 is a diagram illustrating an example of an internal block diagram of a home appliance according to an embodiment of the present invention.
12 is a view showing a part of an internal block diagram of an air conditioner according to an embodiment of the present invention.
13 and 14 are views briefly showing the appearance of an air conditioner according to an embodiment of the present invention.
15 is a flowchart illustrating a method of operating a home appliance according to an embodiment of the present invention.
16 is a flowchart illustrating a method of operating a smart home system including a voice recognition server system and a home appliance according to an embodiment of the present invention.
17 to 21 are views referenced for descriptions of various operations of a voice recognition server system and a home appliance according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, it goes without saying that the present invention is not limited to these embodiments and may be modified in various forms.

In the drawings, in order to clearly and briefly describe the present invention, the illustration of parts not related to the description is omitted, and the same reference numerals are used for identical or extremely similar parts throughout the specification.

Meanwhile, the suffixes "module" and "unit" for the constituent elements used in the following description are given in consideration of only ease of writing in the present specification, and do not impart a particularly important meaning or role by themselves. Therefore, the "module" and "unit" may be used interchangeably with each other.

1 is a diagram schematically illustrating a smart home system including a voice recognition server system and a home appliance according to an embodiment of the present invention, and FIG. 2 is a diagram illustrating various examples of a home appliance.

Referring to FIG. 1, a smart home system 10 according to an embodiment of the present invention includes a home appliance 200 capable of communicating with other devices or accessing a network by having a communication module (not shown). It may be configured to include a voice recognition server system 100 including a plurality of servers for voice recognition and home appliance control.

Referring to FIG. 2, the home appliance 200 may include an air conditioner 200a having a communication module, a robot cleaner 200b, a refrigerator 200c, a washing machine 200d, and a cooking appliance 200e. have.

Further, the smart home system 10 according to an embodiment of the present invention may include a portable terminal (not shown) such as a smart phone or a tablet PC.

The home appliance 200 may have a communication module therein to communicate with electronic devices inside/outside the smart home system 10.

The smart home system 10 according to an embodiment of the present invention may further include an access point (AP) device 300, and the home appliance 200 is wireless Internet through the access point device 300. You can connect to a network and communicate with other devices.

The access point device 300 may allocate a wireless channel according to a predetermined communication method to electronic devices in the smart home system 10 and perform wireless data communication through the channel.

Here, the predetermined communication method may be a Wi-Fi communication method. Correspondingly, the communication module provided in the home appliance 200 may be a Wi-Fi communication module, but the present invention is not limited to the communication method.

Alternatively, the home appliance 200 may include other types of communication modules or may include a plurality of communication modules. For example, the home appliance 200 may include an NFC module, a zigbee communication module, a Bluetooth™ communication module, and the like.

The home appliance 200 can be connected to a server included in the voice recognition server system 100 or an external predetermined server, a user's portable terminal through a Wi-Fi communication module, etc. Function can be supported.

The user can check information about the home appliance 200 in the smart home system 10 or control the home appliance 200 through the portable terminal.

Meanwhile, even when a user wants to control the home appliance 200 or check predetermined information in the home, it may be inconvenient to have to use a portable terminal.

For example, it is more efficient to have a means to control the home appliance 200 in a different way if the user does not know the current location of the portable terminal or is in a different location.

The home appliance 200 according to an embodiment of the present invention may receive a user's voice input, and the voice recognition server system 100 may control the home appliance 200 by recognizing and analyzing the user's voice input. have.

Accordingly, the user can control the home appliance 200 without operating the mobile terminal or the remote control device.

Meanwhile, at least some of the servers included in the voice recognition server system 100 may be servers operated by a manufacturing company or a sales company of a home appliance, or a server operated by a company entrusted with a service by a manufacturing company or a sales company. .

3A is an example of a voice recognition server system according to an embodiment of the present invention.

3A, a voice recognition server system according to an embodiment of the present invention includes a voice server 110 that receives voice data from a home appliance 200a and analyzes the received voice data to determine a voice command. Can include.

The voice server 110 may receive voice data from the home appliance 200a, convert the received voice data into text data, and analyze the text data to determine a voice command.

Further, the voice server 110 may transmit a signal corresponding to the determined voice command to a predetermined server.

For example, the voice recognition server system according to an embodiment of the present invention receives a signal corresponding to the determined voice command from the voice server 110, and generates a request signal corresponding to the determined voice command. It may include a connection service server 120 and a home appliance control server 130 for transmitting a control signal based on the request signal received from the connection service server 120 to the home appliance (200a).

The home appliance 200a may receive a voice command input uttered by a user and transmit voice data based on the received voice command input to the voice server 110.

The voice server 110 is an automatic speech recognition (ASR) server 111 that receives voice data from the home appliance 200a and converts the received voice data into text data. Natural language processing for receiving the text data from the voice recognition server 111, analyzing the received text data to determine a voice command, and transmitting a response signal based on the determined voice command to the home appliance 200a ( Natural Language Processing: Receives a signal including text corresponding to the response signal from the NLP server 112 and the home appliance 200a, converts the text included in the received signal into voice data, and It may include a text-to-speech (TTS) server 113 that is transmitted to the appliance 200a.

The automatic voice recognition server 111 may perform voice recognition on voice data received from the home appliance 200a, generate text data, and transmit the text data to the natural language processing server 112.

The natural language processing server 112 may determine a voice command by analyzing text data received from the automatic speech recognition server 111 according to a natural language processing algorithm.

The natural language processing server 112 may process natural language, which is a language commonly used by humans, according to a natural language processing algorithm, and analyze a user's intent. The natural language processing server 112 may perform natural language processing on text data received from the automatic speech recognition server 111 to determine a voice command conforming to the user's intention.

Accordingly, the natural language processing server 112 may determine a voice command conforming to the user's intention even if the user inputs a voice command in a common language.

The natural language processing server 112 may transmit a signal corresponding to the natural language processing result, that is, a signal corresponding to the determined voice command, to the linked service server 120.

The linked service server 120 may receive a signal corresponding to the determined voice command from the natural language processing server 112.

If the determined voice command is related to the home appliance 200a, the linked service server 120 may communicate with the home appliance control server 130 to perform a corresponding operation.

Alternatively, if the determined voice command is not related to the home appliance 200a, the linked service server 120 may communicate with an external external service 121 to perform a corresponding operation.

For example, if the determined voice command is a command for requesting information such as weather, stock, and news, the linked service server 120 may request and receive the information from a server providing a service corresponding to the requested information. I can.

In addition, the linked service server 120 may transmit the received information to the voice server 110, and the natural language processing server 112 may transmit the received information to the home appliance 200a.

If the determined voice command is related to the home appliance 200a, the linked service server 120 may generate a request signal corresponding to the determined voice command and transmit it to the home appliance control server 130.

The home appliance control server 130 may transmit a control signal based on a request signal received from the linked service server 120 to the home appliance 200a.

For example, when a request for changing the set temperature of the air conditioner 200a is received, the home appliance control server 130 may transmit a control signal for changing the set temperature to the air conditioner 200a.

Meanwhile, the home appliance 200a may perform a corresponding operation according to a control signal received from the home appliance control server 130.

In addition, the home appliance 200a may transmit a signal indicating that the operation has been performed after performing the requested operation to the home appliance control server 130.

In addition, the home appliance control server 130 may receive a response signal for the control signal from the home appliance 200a, and transmit processing result information corresponding to the response signal to the linked service server 120 have.

The voice server 110 may transmit a response signal including the processing result information to the home appliance 200a.

Further, the voice server 110 receives a signal including an output phrase text corresponding to the processing result information from the home appliance 200a, converts the received output phrase text into voice data, and It can be transmitted to (200a).

In this case, a response signal based on the determined voice command transmitted from the natural language processing server 112 to the home appliance 200a may include the processing result information.

Meanwhile, the home appliance 200a may receive a response signal based on the determined voice command from the natural language processing server 112. Here, the response signal may include text data of a response corresponding to the determined voice command.

For example, when the user inputs a voice command for changing the set temperature of the air conditioner, the response signal may include text data indicating that the set temperature has been changed.

Meanwhile, the home appliance 200a may transmit a signal including text corresponding to the received response signal to the text-to-speech server 113. Here, the signal including the text corresponding to the response signal may include the output phrase text corresponding to the processing result information.

Meanwhile, the text-to-speech server 113 may convert text included in the received signal into voice data and transmit it to the home appliance 200a. Here, the converted voice data may include a sound source file.

The home appliance 200a may output a voice guidance message based on voice data received through a speaker.

On the other hand, the linked service server 120 requests the state information of the home appliance 200a to the home appliance control server 130 based on a signal corresponding to the determined voice command, and the home appliance control server 130 ) May transmit the status information of the home appliance 200a to the linked service server 120. When the state information of the home appliance 200a is not secured, the home appliance control server 130 may request and receive the state information from the home appliance 200a.

Meanwhile, the linked service server 120 responds to the determined voice command with the home appliance control server 130 when it is possible to support the determined voice command based on the status information of the home appliance 200a. You can send a request signal to say.

Alternatively, the linked service server 120 is a function that is not supported in the current state by the natural language processing server 112 when it is impossible to support the determined voice command based on the state information of the home appliance 200a. It is possible to transmit a signal indicating that it is.

Even in this case, the home appliance 200a may request and receive voice data from the text-to-speech server 113 and output a voice guidance message indicating that a function is not supported in the current state.

Depending on the embodiment, the voice server 110 may determine whether or not the determined voice command is supported. For example, the natural language processing server 112 analyzing the intention of the user's voice command may determine whether the determined voice command is supported.

In this case, when the determined voice command includes a command that is not supported, the response signal based on the determined voice command transmitted by the natural language processing server 112 is the determined voice command is the home appliance ( It may be a signal indicating that the function is not supported by 200a).

The voice server 110 and the voice recognition server system 100 including the same according to an embodiment of the present invention may organically connect and use servers that play various roles for natural language voice processing.

The home appliance 200 may perform operations up to reception and preprocessing of voice commands and transmission to a server, and the voice server 110 may perform natural language processing such as voice/text conversion, intention analysis, and command identification.

Since the voice server 110 performs natural language processing, it is possible to reduce the burden of CPU, memory, etc. of the embedded module inside the home appliance.

Meanwhile, the linked service server 120 may communicate with the external service and home appliance control server 130 to perform an operation based on a user's voice command.

Meanwhile, the home appliance 200 may receive voice data including a sound source file from the voice server 110 and output a voice guidance message as an audio, thereby responding to a user's voice input through auditory feedback.

The home appliance 200 may receive a voice file from the voice server 110 via streaming, and play and output a voice guide message to a user. Accordingly, the home appliance 200 does not need to store various sound source files.

On the other hand, through the linked service server 120, it is possible to link with various external services without colliding with other servers. In addition, it is possible to increase the success rate of intention analysis by reflecting external information when analyzing intention through an external service linking server.

The voice recognition server system 100 according to an embodiment of the present invention secures compatibility and connectivity through a plurality of servers, and the final control command utilizes the home appliance control server 130 to control the voice recognition process and home appliance. It is possible to prevent a collision between home appliance control using Wi-Fi communication through the server 130 and a collision between home appliance control through a portable terminal and home appliance control through voice input through the home appliance 200.

The voice recognition server system 100 according to an embodiment of the present invention can reduce the load on a specific server depending on a server through an organic connection between servers, and the roles of each server are different. If a problem occurs in the server, it is possible to easily respond by linking with other servers that play the same role.

In addition, since a plurality of servers can be independently updated at any time, it is advantageous to improve performance.

3B is an example of a voice recognition server system according to an embodiment of the present invention.

The voice recognition server system illustrated in FIG. 3B improves the voice control response time by improving the process in which the voice recognition server system illustrated in FIG. 3A transmits voice data for outputting a voice guidance message to the home appliance 200a. .

Accordingly, the speech recognition server systems illustrated in FIGS. 3A and 3B may perform substantially the same operations other than the above differences, and the same parts will be briefly described below.

3B, the voice recognition server system according to an embodiment of the present invention includes a voice server 110 that receives voice data from a home appliance 200a and analyzes the received voice data to determine a voice command. Can include.

In addition, the voice recognition server system according to an embodiment of the present invention receives a signal corresponding to the determined voice command from the voice server 110, and generates a request signal corresponding to the determined voice command. It may further include a home appliance control server 130 for transmitting a control signal based on the request signal received from the service server 120 and the link service server 120 to the home appliance (200a).

The voice recognition server system illustrated in FIG. 3B transmits voice data including processing result information based on the voice command to the home appliance 200a by the voice server 110 without a request from the home appliance 200a. can do.

The voice server 110 is an automatic voice recognition server 111 for receiving voice data from the home appliance 200a and converting the received voice data into text data, from the automatic voice recognition server 111 A natural language processing server 112 that receives the text data and determines a voice command by analyzing the received text data, and converts a response signal based on the voice command into voice data and transmits it to the home appliance 200a It may include a text-to-speech server 113.

Also in this embodiment, the home appliance control server 130 receives a response signal to the control signal from the home appliance 200a, and sends processing result information corresponding to the response signal to the linked service server 120 Can send.

The linked service server 120 may transmit the processing result information to the voice server 110, more specifically, the natural language processing server 112.

In this case, the voice data transmitted from the text-to-speech server 113 to the home appliance 200a may include the processing result information.

In addition, the linked service server 120 requests status information of the home appliance 200a from the home appliance control server based on a signal corresponding to the determined voice command, and the home appliance control server The status information of (200a) may be transmitted to the linked service server 120.

In addition, the linked service server 120 responds to the determined voice command with the home appliance control server 130 when it is possible to support the determined voice command based on the status information of the home appliance 200a. You can send a request signal to say.

Alternatively, the linked service server 120 is a function that is not supported in the current state by the voice server 110 when it is impossible to support the determined voice command based on the state information of the home appliance 200a. You can send an informing signal.

For example, the linked service server 120 may transmit a signal indicating that a function is not supported in the current state to the natural language processing server 112.

In addition, the natural language processing server 112 transmits a signal to the text-to-speech server 113 notifying that a function is not supported in the current state, and the text-to-speech server 113 generates corresponding voice data to It can be transmitted to the appliance (200a).

The home appliance 200a may receive voice data from the text-to-speech server 113 and output a voice guidance message indicating that a function is not supported in the current state.

Depending on the embodiment, the voice server 110 may determine whether or not the determined voice command is supported. For example, the natural language processing server 112 analyzing the intention of the user's voice command may determine whether the determined voice command is supported.

In this case, when the determined voice command includes a command that is not supported, the response signal based on the determined voice command transmitted by the natural language processing server 112 is the determined voice command is the home appliance ( It may be a signal indicating that the function is not supported by 200a).

After the home appliance 200a operates, when the text-to-speech server 113 for voice guidance is the last to request a guidance comment for a corresponding movement, a time difference may occur between the movement and the guidance comment.

However, according to an embodiment of the present invention, when the intention analysis is completed and an operation request is transmitted from the natural language processing server 112, information may be simultaneously provided to the text-to-speech server 113.

In addition, the text-to-speech server 113 may provide an announcement to the home appliance 200a based on a time point when the home appliance control server 130 gives a control command to the home appliance 200a.

Accordingly, an announcement may be ignited at the same time as or immediately after the operation of the home appliance 200a.

According to the present embodiment, by directly connecting the natural language processing server 112 and the text-to-speech server 113, it is possible to minimize a time difference between a control command and an announcement through the home appliance control server 130.

Meanwhile, in FIGS. 3A and 3B, the air conditioner 200a is illustrated as the home appliance 200, but the present invention is not limited thereto. For example, in addition to the air conditioner 200a, the home appliance 200 may include a robot cleaner 200b, a refrigerator 200c, a washing machine 200d, and a cooking appliance 200e.

Meanwhile, according to one side of the present invention, unlike FIGS. 3A and 3B, the automatic speech recognition server 111 for speech recognition and processing, the natural language processing server 112, and the text-to-speech server 113 are integrated into one It can be configured as a server.

In addition, depending on the embodiment, the link service server 120 and the home appliance control server 130 may be configured as one integrated server.

According to the present invention, since the home appliance operates in response to a voice input, the user does not need to operate a remote control device such as a remote control or a portable terminal, and user convenience can be increased.

In addition, as described with reference to FIGS. 3A and 3B, the present invention recognizes a user's natural language voice command using a plurality of servers, and performs a corresponding control operation, thereby reducing system resources of the home appliance and each server. Natural language can be recognized and processed efficiently without limitation.

4 to 6 are diagrams illustrating the signal flow of the voice recognition server system according to an embodiment of the present invention, and are diagrams illustrating the signal flow of the voice recognition server system illustrated in FIG. 3A.

4 illustrates a signal flow in a typical situation in which a home appliance operates according to a user's voice command.

Hereinafter, the air conditioner 200a is exemplified as the home appliance 200, but the present invention is not limited thereto.

4, the home appliance 200, for example, the air conditioner 200a according to an embodiment of the present invention may receive a user's voice command (S410), and the received user's voice command May be transmitted to the voice server 110 (S420).

For example, when the user receives a command for changing the temperature setting such as "LG Whissen, set the temperature to 18 degrees", the air conditioner 200a transmits the received voice command to a digital voice in a predetermined format such as a wave file. It can be converted into data and transmitted to the automatic speech recognition server 111.

On the other hand, the air conditioner 200a may receive a wake-up signal including a pager and then transmit a voice command inputted while waiting for a command input to the voice server 110.

Alternatively, the air conditioner 200a may receive a continuous voice input including a pager and a voice command. In this case, the air conditioner 200a may recognize a call word and transmit a voice command to the voice server 110.

On the other hand, when communicating between devices inside the system 10, the signals to be transmitted and received may further include identification information of each device, a session value for a task to be performed, in addition to the main data transmitted and received. have.

The automatic speech recognition server 111 may recognize the received voice data and convert it into text data (S431), and transmit the text data to the natural language processing server 112 (S433).

The natural language processing server 112 may analyze and determine the intention of the user's voice command by performing natural language processing on the received text data (S441).

For example, the natural language processing server 112 may determine a voice command intended by the user to change the set temperature of the air conditioner 200a to 18 degrees in "LG Whissen, set the temperature to 18 degrees".

The natural language processing server 112 may transmit a signal corresponding to the determined voice command to the linked service server 120 (S443).

When the linked service server 120 requests the current state information of the air conditioner 200a from the home appliance control server 130 (S451), the home appliance control server 130 inquires the current state information of the air conditioner 200a. Thus (S453), the current state information of the air conditioner 200a can be transmitted to the linked service server 120 (S455).

If the home appliance control server 130 does not have the current state information of the air conditioner 200a, the home appliance control server 130 may request and receive the current state information from the air conditioner 200a.

Meanwhile, the linked service server 120 receiving the status information may transmit a request signal to change the set temperature of the air conditioner 200a to 18 degrees to the home appliance control server 130 according to the determined voice command ( S461).

The home appliance control server 130 may control the air conditioner 200a by generating a control signal based on the request signal and transmitting it to the air conditioner 200a (S463).

In addition, the home appliance control server 130 receives a response signal to the control signal from the home appliance after the control of the air conditioner 200a, and sends processing result information notifying that the operation to the request has been successful, the linked service server ( 120) can be transmitted (S465).

The linked service server 120 can transmit a signal corresponding to the processing result information such as "set the air conditioner temperature to 18 degrees" to the natural language processing server 112 (S470), and the natural language processing server 112 A response signal based on the user's voice command may be transmitted to the home appliance 200a (S481). Here, the response signal based on the determined user's voice command may include the processing result information.

On the other hand, the home appliance (200a) transmits a signal including text (output phrase) corresponding to the response signal (S483), and the text-to-speech server 113 receiving it voices the text included in the received signal. It can be converted into data (S485) and transmitted to the home appliance (200a) (S487).

The home appliance 200a may output a voice guidance message such as "Set the air conditioner temperature to 18 degrees" based on the received voice data (S490).

5 illustrates a signal flow when requesting an unsupported function of a corresponding home appliance.

Referring to FIG. 5, the air conditioner 200a according to an embodiment of the present invention can receive a user's voice command for a laundry function not supported by the air conditioner 200a, such as "LG Whissen, start laundry". It can be done (S510).

The air conditioner 200a may transmit the received user's voice command to the voice server 110 (S520). The air conditioner 200a may convert the received voice command into digital voice data in a predetermined format, such as a wave file, and transmit the converted voice command to the automatic voice recognition server 111.

The automatic speech recognition server 111 may recognize the received voice data and convert it into text data (S531), and transmit the text data to the natural language processing server 112 (S533).

The natural language processing server 112 may analyze and determine the intention of the user's voice command by performing natural language processing on the received text data (S541).

In addition, the natural language processing server 112 may determine whether or not the determined voice command is supported. In this embodiment, the natural language processing server 112 may determine that the user has requested a laundry function that is not supported by the air conditioner 200a.

Thereafter, the natural language processing server 112 may transmit a response signal notifying that the determined voice command is a function not supported by the home appliance to the air conditioner 200a (S543).

On the other hand, the home appliance (200a) transmits a signal including text (output phrase) corresponding to the response signal (S551), and the text-to-speech server 113 receiving it voices the text included in the received signal. It can be converted into data (S553) and transmitted to the home appliance (200a) (S555).

The home appliance 200a may output a voice guidance message such as "This function is not supported" based on the received voice data (S560).

6 illustrates a signal flow when an unsupported function is requested in a current operation mode of a corresponding home appliance.

6, the air conditioner 200a according to an embodiment of the present invention receives a user's voice command instructing the operation of a specific mode of the air conditioner 200a, such as "LG Whissen, start cool power". Can be (S610).

The air conditioner 200a may transmit the received user's voice command to the voice server 110 (S620). The air conditioner 200a may convert the received voice command into digital voice data in a predetermined format, such as a wave file, and transmit the converted voice command to the automatic voice recognition server 111.

The automatic speech recognition server 111 may recognize the received voice data and convert it into text data (S631), and transmit the text data to the natural language processing server 112 (S633).

The natural language processing server 112 may analyze and determine the intention of the user's voice command by performing natural language processing on the received text data (S641).

For example, the natural language processing server 112 may determine a voice command intended for the user to drive the air conditioner 200a in the cool power mode in "LG Whissen, start cool power".

The natural language processing server 112 may transmit a signal corresponding to the determined voice command to the linked service server 120 (S643).

When the linked service server 120 requests the current state information of the air conditioner 200a from the home appliance control server 130 (S651), the home appliance control server 130 queries the current state information of the air conditioner 200a Thus (S653), the current state information of the air conditioner 200a can be transmitted to the linked service server 120 (S655).

If the home appliance control server 130 does not have the current state information of the air conditioner 200a, the home appliance control server 130 may request and receive the current state information from the air conditioner 200a.

Meanwhile, the linked service server 120 receiving the status information may determine whether or not the determined voice command can be supported based on the current status information of the air conditioner 200a (S657). For example, when the air conditioner 200a is currently operating in the dehumidification mode and the cool power mode is supported only in the cooling mode, the natural language processing server 112 requests a cool power mode that is not supported in the current state. It can be determined that it was done.

Thereafter, the linked service server 120 may transmit a response signal indicating that the determined voice command is a function that the home appliance 200a does not support in the current state to the natural language processing server 112 (S660).

In addition, the natural language processing server 112 may transmit a response signal indicating that the determined voice command is a function not supported in the current state to the home appliance 200a (S671).

On the other hand, the home appliance (200a) transmits a signal including text (output phrase) corresponding to the response signal (S673), and the text-to-speech server 113 receiving it voices the text included in the received signal. It can be converted into data (S675) and transmitted to the home appliance (200a) (S677).

The home appliance 200a may output a voice guidance message such as "Cool power is supported only in the cooling mode" based on the received voice data (S680).

7 to 9 are diagrams illustrating the signal flow of the voice recognition server system according to an embodiment of the present invention, and are diagrams illustrating the signal flow of the voice recognition server system illustrated in FIG. 3B.

7 illustrates a signal flow in a typical situation in which a home appliance operates according to a user's voice command.

Referring to FIG. 7, the home appliance 200, for example, the air conditioner 200a, according to an embodiment of the present invention may receive a user's voice command (S710), and the received user's voice command May be transmitted to the voice server 110 (S720).

The automatic speech recognition server 111 may recognize the received voice data and convert it into text data (S731), and may transmit the text data to the natural language processing server 112 (S733).

The natural language processing server 112 may analyze and determine the intention of the user's voice command by performing natural language processing on the received text data (S741).

For example, the natural language processing server 112 may determine a voice command intended by the user to change the set temperature of the air conditioner 200a to 18 degrees in "LG Whissen, set the temperature to 18 degrees".

The natural language processing server 112 may transmit a signal corresponding to the determined voice command to the linked service server 120 (S743).

When the linked service server 120 requests the current state information of the air conditioner 200a to the home appliance control server 130 (S751), the home appliance control server 130 queries the current state information of the air conditioner 200a. Thus (S753), the current state information of the air conditioner 200a can be transmitted to the linked service server 120 (S755).

If the home appliance control server 130 does not have the current state information of the air conditioner 200a, the home appliance control server 130 may request and receive the current state information from the air conditioner 200a.

Meanwhile, the linked service server 120 receiving the status information may transmit a request signal to change the set temperature of the air conditioner 200a to 18 degrees to the home appliance control server 130 according to the determined voice command ( S761).

The home appliance control server 130 may control the air conditioner 200a by generating and transmitting a control signal to the air conditioner 200a based on the request signal (S763).

In addition, the home appliance control server 130 receives a response signal to the control signal from the home appliance after the control of the air conditioner 200a, and sends processing result information notifying that the operation to the request has been successful, the linked service server ( 120) can be transmitted (S765).

The linked service server 120 may transmit, to the natural language processing server 112, a signal corresponding to processing result information such as "set the air conditioner temperature to 18 degrees" (S770).

Meanwhile, the natural language processing server 112 may transmit a response signal based on the determined user's voice command to the text-to-speech server 113 (S781). Here, the response signal based on the determined user's voice command may include the processing result information.

Meanwhile, the text-to-speech server 113 may convert a response signal based on the voice command into voice data (S783) and transmit it to the home appliance 200a (S785).

The home appliance 200a may output a voice guidance message such as "Set the air conditioner temperature to 18 degrees" based on the received voice data (S790).

8 illustrates a signal flow when requesting an unsupported function of a corresponding home appliance.

Referring to FIG. 8, the air conditioner 200a according to an embodiment of the present invention may receive a user's voice command for a laundry function not supported by the air conditioner 200a, such as "LG Whissen, start laundry". Can be (S810).

The air conditioner 200a may transmit the received user's voice command to the voice server 110 (S820). The air conditioner 200a may convert the received voice command into digital voice data in a predetermined format, such as a wave file, and transmit the converted voice command to the automatic voice recognition server 111.

The automatic speech recognition server 111 may recognize the received voice data and convert it into text data (S831), and transmit the text data to the natural language processing server 112 (S833).

The natural language processing server 112 may analyze and determine the intention of the user's voice command by performing natural language processing on the received text data (S841).

In addition, the natural language processing server 112 may determine whether or not the determined voice command is supported. In this embodiment, the natural language processing server 112 may determine that the user has requested a laundry function that is not supported by the air conditioner 200a.

Thereafter, the natural language processing server 112 may transmit a response signal indicating that the determined voice command is a function not supported by the home appliance to the text-to-speech server 113 (S843).

The text-to-speech server 113 may convert the response signal based on the voice command into voice data (S845) and transmit the converted signal to the home appliance 200a (S847).

The home appliance 200a may output a voice guidance message such as "This function is not supported" based on the received voice data (S850).

9 illustrates a signal flow when an unsupported function is requested in a current operation mode of a corresponding home appliance.

9, the air conditioner 200a according to an embodiment of the present invention receives a user's voice command instructing the operation of a specific mode of the air conditioner 200a, such as "LG Whissen, start cool power". Can be (S910).

The air conditioner 200a may transmit the received user's voice command to the voice server 110 (S920). The air conditioner 200a may convert the received voice command into digital voice data in a predetermined format, such as a wave file, and transmit the converted voice command to the automatic voice recognition server 111.

The automatic speech recognition server 111 may recognize the received voice data and convert it into text data (S931), and transmit the text data to the natural language processing server 112 (S933).

The natural language processing server 112 may analyze and determine the intention of the user's voice command by performing natural language processing on the received text data (S941).

For example, the natural language processing server 112 may determine a voice command intended for the user to drive the air conditioner 200a in the cool power mode in "LG Whissen, start cool power".

The natural language processing server 112 may transmit a signal corresponding to the determined voice command to the linked service server 120 (S943).

When the connection service server 120 requests the current state information of the air conditioner 200a to the home appliance control server 130 (S951), the home appliance control server 130 inquires the current state information of the air conditioner 200a Thus (S953), the current state information of the air conditioner 200a can be transmitted to the linked service server 120 (S955).

If the home appliance control server 130 does not have the current state information of the air conditioner 200a, the home appliance control server 130 may request and receive the current state information from the air conditioner 200a.

Meanwhile, the linked service server 120 receiving the status information may determine whether or not the determined voice command is supported based on the current status information of the air conditioner 200a (S957). For example, when the air conditioner 200a is currently operating in the dehumidification mode and the cool power mode is supported only in the cooling mode, the natural language processing server 112 requests a cool power mode that is not supported in the current state. It can be determined that it was done.

Thereafter, the linked service server 120 may transmit a response signal indicating that the determined voice command is a function that the home appliance 200a does not support in the current state to the natural language processing server 112 (S960).

In addition, the natural language processing server 112 may transmit a response signal indicating that the determined voice command is a function not supported in the current state to the text-to-speech server 113 (S971).

Meanwhile, the text-to-speech server 113 may convert a response signal based on the voice command into voice data (S973) and transmit it to the home appliance 200a (S975).

The home appliance 200a may output a voice guidance message such as "Cool power is supported only in the cooling mode" based on the received voice data (S980).

According to the present invention, it is possible to implement a user experience that provides a suitable voice guidance for a case in which a user's command is not understood or a function that cannot be provided.

It is possible to minimize user discomfort by providing a guide comment appropriate for each situation and time point that occurs during command processing.

10 is a diagram showing an example of an internal block diagram of a server according to an embodiment of the present invention, wherein the server includes an automatic speech recognition server 111, a natural language processing server 112, and a text-to-speech server 113 , A connection service server 120, it may be a home appliance control server 130.

Referring to FIG. 10, the server may include a communication module 1020, a storage unit 1030, and a processor 1010.

The processor 1010 may control the overall operation of the server.

The communication module 1020 may receive various data such as status information, operation information, operation information, voice data, text data, etc. from a home appliance such as a portable terminal or an air conditioner, or another server.

In addition, the communication module 1020 may transmit data corresponding to various types of received information as it is or after predetermined processing to a home appliance such as a portable terminal or an air conditioner, or to another server.

To this end, the communication module 1020 may include one or more communication modules such as an Internet module and a mobile communication module.

The storage unit 1030 may store received information and include data for generating result information corresponding thereto.

The storage unit 1030 may store data for operation of a corresponding server. For example, in the storage unit 1030, in the case of the automatic speech recognition server 111, an automatic speech recognition algorithm, etc. are stored, and in the case of the home appliance control server 130, product information, status information, and control of the home appliance Data for can be stored.

Meanwhile, the server may be a server operated by a home appliance manufacturer such as an air conditioner or a server operated by a service provider, or may be a kind of cloud server.

11 is a diagram illustrating an example of an internal block diagram of a home appliance according to an embodiment of the present invention.

Referring to FIG. 11, a home appliance 200 according to an embodiment of the present invention includes an audio input unit 220 receiving a user's voice command, a memory 250 storing various data, and wireless communication with other electronic devices. A communication unit 270 for performing an operation, a driving unit 280 for performing an operation implemented in each home appliance, an output unit 290 for displaying predetermined information as an image or outputting audio, and a control unit 240 for controlling overall operations. Can include.

The audio input unit 220 may receive an external audio signal and a user voice command. To this end, the audio input unit 220 may include one or more microphones (MIC). In addition, in order to more accurately receive a user's voice command, the audio input unit 220 may include a plurality of microphones 221 and 222. The plurality of microphones 221 and 222 may be disposed to be spaced apart from each other, and may obtain an external audio signal and process it as an electrical signal.

In FIG. 11 and the like, an example in which the audio input unit 220 includes two microphones, the first microphone 221 and the second microphone 222 is illustrated, but the present invention is not limited thereto.

The audio input unit 220 may include a processing unit that converts analog sound into digital data or may be connected to the processing unit to convert a user input voice command into data so that the controller 240 or a predetermined server can recognize it.

Meanwhile, the audio input unit 220 may use various noise removal algorithms for removing noise generated in a process of receiving a user's voice command.

In addition, the audio input unit 220 may include components for audio signal processing, such as a filter that removes noise from an audio signal received from each of the microphones 221 and 222 and an amplifier that amplifies and outputs a signal output from the filter. .

The memory 250 records various types of information necessary for the operation of the home appliance, and may include a volatile or nonvolatile recording medium. A recording medium stores data that can be read by a micro processor, and is a hard disk drive (HDD), solid state disk (SSD), silicon disk drive (SDD), ROM, RAM, CD-ROM, magnetic It may include a tape, a floppy disk, an optical data storage device, and the like.

Meanwhile, data for voice recognition may be stored in the memory 250, and the controller 240 may process a user's voice input signal received through the audio input unit 220 and perform a voice recognition process.

Meanwhile, simple speech recognition may be performed by the home appliance 200, and high-level speech recognition such as natural language processing may be performed by the speech recognition server system 100.

For example, when a wake up voice signal including a preset call word is received, the home appliance 200 may be switched to a state for receiving a voice command. In this case, the home appliance 200 may perform only a voice recognition process up to whether or not a caller has a voice input, and voice recognition for a user's voice input may be performed through the voice recognition server system 100.

Since the home appliance has a limit in system resources, complex natural language recognition and processing may be performed through the voice recognition server system 100.

Limited data may be stored in the memory 250. For example, the memory 250 may store data for recognizing a wake-up voice signal including a preset call word. In this case, the controller 240 may recognize a wake-up voice signal including a preset call word from a user's voice input signal received through the audio input unit 220.

Meanwhile, the pager may be set by a manufacturer, and a different pager may be set for each home appliance. For example, in the case of an air conditioner, "LG Whissen" and in the case of a refrigerator may be set to "LG Dios" as a call word.

In addition, the caller can be set by a user.

The controller 240 may control to transmit a user's voice command input after recognition of a wake-up voice signal to the voice recognition server system 100 through the communication unit 270.

The communication unit 270 may include one or more communication modules to perform wireless communication with other electronic devices to exchange various signals. For example, the communication unit 270 may communicate with electronic devices inside/outside the mart home system 10.

In addition, the communication unit 270 may communicate with the access point device 300 and connect to a wireless Internet network through the access point device 300 to communicate with other devices.

In addition, the controller 240 may transmit state information of the home appliance 200 and a user's voice command to the voice recognition server system 100 through the communication unit 270.

Meanwhile, when a control signal is received through the communication unit 270, the control unit 240 may control the home appliance 200 to operate according to the received control signal.

The output unit 290 may include a display 292 that displays information corresponding to a user's command input, a processing result corresponding to the user's command input, an operation mode, an operation state, an error state, etc. as an image.

Depending on the embodiment, the display 292 may form a layer structure with a touch pad to form a touch screen. In this case, the display 292 may be used as an input device capable of inputting information by a user's touch in addition to an output device.

Also, the output unit 290 may further include an audio output unit 291 outputting an audio signal. The audio output unit 291 provides a warning sound, an operation mode, a notification message such as an operation state, an error state, etc., information corresponding to a user's command input, a processing result corresponding to the user's command input, etc. It can be output as audio. The audio output unit 291 may convert an electrical signal from the control unit 240 into an audio signal and output it. To this end, a speaker or the like may be provided.

The driving unit 280 performs an operation implemented in the home appliance, and may be configured differently for each home appliance.

For example, when the home appliance is a refrigerator, the driving unit 280 may include a refrigerating compartment driving unit that operates a refrigerating compartment fan for supplying cooled air to the refrigerating compartment, and a freezing compartment driving unit that operates a freezer compartment fan for supplying cooled air to the freezer compartment. And a compressor driving unit for operating a compressor for compressing a refrigerant.

As another example, when the home appliance is a washing machine, the driving unit 280 may include a driving unit for driving a drum or a tub.

In another example, when the home appliance is an air conditioner, the driving unit 280 includes a compressor driving unit for driving a compressor in the outdoor unit, an outdoor unit fan driving unit for operating an outdoor unit fan for heat exchange, and an indoor unit for operating an indoor unit fan for heat exchange. It may include a fan driving unit and the like.

As another example, when the home appliance is a cooking appliance, the driving unit 280 may include a microwave driving unit that outputs microwaves into the cavity.

As another example, when the home appliance is a vacuum cleaner, the driving unit 280 may include a fan motor driving unit for inhaling air.

Meanwhile, the home appliance 200 may further include a manipulation unit 230 for user input and a camera 210 capable of photographing a predetermined range around the home appliance 200.

The manipulation unit 230 may include a plurality of manipulation buttons and may transmit a signal corresponding to an input button to the controller 240.

The camera 210 photographs the surroundings of the home appliance 200, the external environment, and the like, and several cameras may be installed for each part for shooting efficiency.

For example, the camera 210 includes at least one optical lens and a plurality of photodiodes (for example, pixels) formed by light passing through the optical lens. CMOS image sensor), and a digital signal processor (DSP) that composes an image based on signals output from photodiodes. The digital signal processor is capable of generating not only still images but also moving images composed of frames composed of still images.

Meanwhile, an image captured by the camera 210 may be stored in the memory 250.

12 is a view showing a part of an internal block diagram of an air conditioner according to an embodiment of the present invention, and FIGS. 13 and 14 are views briefly showing the appearance of an air conditioner according to an embodiment of the present invention to be.

In more detail, FIG. 12 is a vane driver included in the driving unit 280a of the air conditioner 200a when the home appliance illustrated in FIG. 11 is the air conditioner 200a illustrated in FIGS. 13 and 14 The 1210 and the fan drive 1220 are illustrated.

13 and 14 are views briefly showing the appearance of an indoor unit of an air conditioner according to an embodiment of the present invention.

11 to 14, the air conditioner 200a according to an embodiment of the present invention includes microphones 221 and 222 for receiving a user's voice command, and a communication unit 270 for wirelessly communicating with other electronic devices. , A control unit 240 for controlling an overall operation, and a vane driving unit 1210 for driving a vane according to the control of the control unit 240.

In addition, the air conditioner 200a according to an embodiment of the present invention may further include a fan driving unit 1220 that drives a fan under the control of the controller 240.

The microphones 221 and 222 may receive external audio signals and user voice commands. In addition, the microphones 221 and 222 may acquire external audio signals and user voice commands and process them as electrical signals.

The air conditioner 200a of the present embodiment is a suction body 202 having an air inlet (not shown), a heat exchanger (not shown) through which air sucked through the air inlet passes, and air is sucked through the air inlet to pass through the heat exchanger. A fan (not shown) that blows air after the air is blown, and discharge ports 207R, 207U, and 207D may be formed so that air is discharged back into the room.

In addition, at the discharge ports 207R, 207U, 207D, wind direction control means such as louvers (not shown) and/or vanes 261R, 261L, 261U, 261D that open and close the discharge ports 190 and adjust the wind direction of the discharged air Is disposed, and the angle and direction of the wind direction adjusting means may be changed by driving a motor or the like under the control of the controller 240. Accordingly, the wind direction of the air discharged through the discharge ports 207R, 207U, and 207D can be adjusted. In addition, by controlling the operation of the wind direction adjusting means, it is possible to adjust the blowing direction and the blowing range of the airflow.

For example, it is possible to fix the vanes 261R, 261L, 261U, 261D to face in any one direction so that the air blows in any one direction, or an operation in which the direction of the vanes is continuously changed within a set range ( Hereinafter, "swing operation") may be performed so that the blowing direction of the set range is continuously changed within the set range. In addition, by adjusting the angle range in which the swing motion of the vanes 261R, 261L, 261U, and 261D is performed, the range in which the blowing is performed may be narrowed or widened.

Meanwhile, a fan is installed inside the air conditioner to control the flow of indoor air sucked through the inlet and discharged to the indoor space through the discharge ports 207R, 207U, 207D, and the fan is installed in the fan driving unit 1220. Rotation is controlled by this, and the operation of the fan driving unit 1220 is controlled by the control unit 240.

Accordingly, the controller 240 may control the vane driving unit 1210 and the fan driving unit 1220 to control the direction of the air flow (airflow) discharged from the air conditioner 200a. The controller 240 may control the amount and speed of the airflow by controlling the speed of the fan motor, and control the direction of the airflow by controlling wind direction adjustment means such as louvers and vanes.

Meanwhile, the air conditioner may include a plurality of vanes 261R, 261L, 261U, and 261D. In addition, at least some of the plurality of vanes 261R, 261L, 261U, and 261D may operate in different ways.

For example, the upper vane 261U is disposed to be elevating on the upper part of the fan, so that it can be elevated and driven during operation. To this end, the air conditioner 200a may include a lifting mechanism 1211 for lifting the upper vane 261U.

In addition, the upper vane 261U may rise and function as an upper discharge body that discharges at least part of the air blown from the fan into the room through the air discharge port 207U.

The suction body 202 may be installed above the base 201. The suction body 202 may form a rear exterior of the air conditioner together with the base 201. The suction body 202 may form an upper exterior of the rear surface of the air conditioner. The suction body 202 may constitute the rear case of the air conditioner together with the base 201, the suction body 202 may be a rear upper case of the air conditioner, and the base 201 is It may be a rear lower case.

A control unit 240 for controlling various electronic components of the air conditioner may be installed in the base 201. Here, various electric components of the air conditioner may include a fan or the like. A base cover 225 disposed in front of the base 201 may be installed on the base 201.

The air conditioner may further include a purification unit (not shown) that purifies the air. The purification unit may be disposed on the suction body 202. The purification unit may be disposed before the heat exchanger in the air flow direction. The purification unit may be disposed between the suction body 202 and the heat exchanger, and may be disposed behind the suction body 202. The purification unit may include a filter that filters foreign substances in the air. The purification unit may include an ion generator that generates ions in the air. The purification unit may include an electric precipitator that discharges and collects foreign substances in the air. The purification unit may include at least one of a filter, an ion generator, and an electric precipitator.

The heat exchanger may be located in front of the air intake. The heat exchanger may exchange heat with the refrigerant air sucked through the air inlet. The heat exchanger may be installed to be located between the air intake and the fan.

When the compressor installed in the outdoor unit is driven in the air conditioner, a low-temperature, low-pressure refrigerant can pass through the heat exchanger, and the heat exchanger can cool the air that has passed through the air inlet. When the compressor is turned on, the heat exchanger can cool the air by exchanging heat with the refrigerant and the air conditioner can function as a cooler.

The fan can be located in front of the heat exchanger. The fan may be configured as a centrifugal fan that sucks air from the rear and blows air in at least one of up, down, left and right. The fan may be composed of a sirocco fan. The fan may be composed of a sirocco fan that blows air in four directions, up, down, left and right.

The fan may include a plurality of blowing fans. The plurality of blowing fans may be sequentially disposed in the vertical direction. Each of the plurality of blowing fans may be composed of a sirocco fan. Upper vane (261U)

An upper front air discharge port 207U may be formed in the front of the upper vane 261U. An air inlet (not shown) through which air blown upward from the fan is introduced may be formed on a lower surface of the upper vane 261U. The upper vane 261U may guide the air blown upward from the internal fan in the forward direction.

The upper vane 261U may be disposed to be elevated on the upper side of the fan. When the upper vane 261U rises, the upper front air discharge port 207U is raised to discharge air, and the upper front air discharge port 207U may be concealed when descending.

The air conditioner may further include an elevation guide 208 for guiding the elevation of the upper vane 261U. The lifting guide 208 may be installed to be located above the fan and the heat exchanger. When the upper vane (261U) rises, the upper front air discharge port (207U) is located above the lifting guide 208 to discharge air, and the upper front air discharge port (207U) is located inside the lifting guide 208 when descending. Can be hidden.

The lifting mechanism 1211 may include a drive source such as a motor, and a power transmission member that lifts and lowers the upper vane 261U when the drive source is driven. The power transmission member may include a pinion installed on the driving source and a rack engaged with the pinion.

The lifting mechanism 1211 may rise to different heights depending on the driving mode. The controller 240 may control the lifting mechanism 1211 according to the user's selection of the driving mode.

For example, the control unit 240 may raise the upper vane 261U to a maximum opening height at which the upper front air discharge port 207U is opened to the maximum, or only a part of the upper front air discharge port 207U is opened. The upper vane 261U can be raised to the height.

A side air discharge port 207R through which air blown from the fan in the lateral direction passes may be formed on the side of the air conditioner 200a.

In addition, side vanes 261R and 261L capable of opening and closing the left air outlet and the right air outlet 207R may be disposed on the side of the air conditioner to adjust the wind direction of air blown from the fan sideways. The side vanes 261R and 261L may include a left vane 261L and a right vane 261R.

The side vanes 261R and 261L may be rotatably installed. The air conditioner may include side driving mechanisms 1212 and 1213 for rotating the side vanes 261R and 261L. The side drive mechanisms 1212 and 1213 may include a left vane drive mechanism 1212 and a right vane drive mechanism 1213.

The side driving mechanisms 1212 and 1213 may include a driving source such as a motor. The side drive mechanisms 1212 and 1213 may include at least one power transmission member, such as a link or gear, which is connected to the drive source and rotates the side vanes 261R and 261L when the drive source is driven.

The side driving mechanisms 1212 and 1213 may be controlled by the controller 240 in an open mode during cooling operation. Further, the side drive mechanisms 1212 and 1213 can rotate the left vane 261L and the right vane 261R at a predetermined opening angle under the control of the control unit 240.

Accordingly, the blowing direction may be controlled, or the blowing direction may be continuously changed within the set range by allowing the swing operation to be performed so that the blowing of the set range is performed. In addition, by adjusting the angle range in which the swing motion is performed, the range in which the blowing is performed may be narrowed or widened.

The air conditioner may further include a lower vane 261D located at the lower side and having a lower front air discharge port 207D formed thereon.

The lower vane 261D may discharge air blown downward from the fan in a forward direction. The lower vane 261D may have a flow path through which air passes. A lower front air discharge port 207D for discharging air to the outside may be formed on the front of the lower vane 261D.

The air conditioner may further include an opening/closing mechanism 1214 for opening and closing the lower front air outlet 207D. The opening/closing mechanism 1214 may include a moving kit (not shown) that moves the lower vane 261D to open and close the lower front air outlet 207D. The moving kit may include a driving source such as a motor, a moving kit carrier on which the moving kit is mounted, and at least one power transmission member for advancing and retreating the moving kit carrier when the driving source is driven.

The lower vane 261D may advance to shield the lower front air discharge port 207D, and may be retracted to open the lower front air discharge port 207D.

The opening/closing mechanism 1214 may be controlled in an open mode in which the lower front air outlet 207D is opened when operating in a predetermined mode.

The air conditioner may further include a blower cover 282 disposed in front of the fan, and the blower cover 282 may form an upper front exterior of the air conditioner. The blower cover 282 may be disposed to be positioned above the base cover 225, and together with the base cover 225 may form a front-side exterior of the air conditioner. The blower cover 282 may constitute a front cover of the air conditioner together with the base cover 225. The blower cover 282 may be a front upper cover of the air conditioner, and the base cover 225 may be a front lower cover of the air conditioner.

The air conditioner may be equipped with a display 292 that displays various types of information. In addition, the air conditioner may further include an audio output unit 291 for outputting an audio signal, and a speaker of the audio output unit 291 may be disposed in front of the air conditioner or at an appropriate position.

In addition, the air conditioner may include an operation unit 230 including a plurality of buttons for inputting various commands, and a camera 210 capable of photographing a predetermined range around the home appliance 200.

Meanwhile, according to one side of the present invention, the front panel 1300 is disposed on the front of the blower cover 282 or the base cover 225, and the display 292 and the audio output unit 291 are provided on the front panel 1300. Operation buttons of the speaker and the operation unit 230 may be disposed.

The air conditioner can be operated according to a plurality of operation modes. The control unit 240 may control the vane driving unit 1210 and the fan driving unit 1220 according to the selected driving mode.

For example, in a predetermined driving mode, the controller 240 may control the plurality of vanes 261R, 261L, 261U, and 261D to open all discharge ports.

Alternatively, in a predetermined driving mode, the control unit 240 may control a specific vane to be closed without opening the discharge port. For example, in a predetermined driving mode, the controller 240 may control the lower vane 261D to be closed.

In addition, the controller 240 may adjust the elevation height of the upper vane 261U and the opening angles of the left/right vanes 261R and 261L according to the driving mode.

In addition, the control unit 240 may adjust the air volume by controlling the rotational speed of the fan according to the operation mode.

Meanwhile, the appearance and structure of the air conditioner illustrated in FIGS. 13 and 14 are exemplary, and the present invention is not limited thereto. For example, the location, number, and structure of the microphones 221 and 222, the camera 210, the audio output unit 291, and the vanes 261R, 261L, 261U, and 261D may vary according to design specifications.

15 is a flowchart illustrating a method of operating a home appliance according to an embodiment of the present invention.

Referring to FIG. 15, the home appliance 200 may activate a voice recognition function according to a user input (S1510). The controller 240 of the home appliance 200 may activate the microphones 221 and 222 according to a user input.

Alternatively, the home appliance 200 may automatically activate the voice recognition function according to a setting for activation of the voice recognition function.

For example, when the power is turned on, the home appliance 200 may automatically activate the microphones 221 and 222 and activate a voice recognition function.

When a voice input is received from a user while the microphones 221 and 222 of the home appliance are activated, the microphones 221 and 222 may receive the input voice and transmit it to the controller 240.

Meanwhile, the home appliance 200 may enter a wake-up signal standby mode for inputting a wake-up voice signal including a preset call word (S1515).

When a wake-up voice signal including a preset call word is received through the microphones 221 and 222 (S1520), the home appliance 200 is switched to a command standby mode for receiving a voice command. Can be (S1525).

If the home appliance 200 always waits for natural language commands, the power consumed by the voice recognition function, CPU share, and the server load of the voice recognition server system are burdened.

Accordingly, the home appliance 200 may transmit only the voice signal that is received in the command waiting state to the server.

In addition, the home appliance 200 may give a condition to a time for waiting for a command, and when a command is input within the time, a sound is received and transmitted to the server until the command is completed.

According to one aspect of the present invention, the home appliance 200 performs only a voice recognition process up to whether or not a caller is voice input, and voice recognition for a user's voice input may be performed through the voice recognition server system 100. .

Meanwhile, the pager may be set by a manufacturer, and a different pager may be set for each home appliance. For example, in the case of an air conditioner, "LG Whissen" and in the case of a refrigerator may be set to "LG Dios" as a call word. In addition, the caller can be set by a user.

Meanwhile, the control unit 240 may control to transmit a user's voice command input after recognition of a wake up voice signal to the voice recognition server system 100 through the communication unit 270. (S1530).

On the other hand, waiting for a user's voice command indefinitely wastes system resources and does not correspond to the user's intention not to input a command, so in the command waiting mode, it may be set to process only the voice command input during a predetermined elapsed time (S1527 ).

In this case, the controller 240 may control to transmit voice data including a voice command received within the preset elapsed time to the voice server 110 (S1530).

On the other hand, if the voice command is not received within the preset elapsed time (S1527), the controller 240 may control to switch back to the wakeup signal standby mode.

According to an embodiment, when the voice command is received within the preset elapsed time (S1527), in order to inform the user that the home appliance 200 has received the command, the controller 240 performs a feedback operation corresponding to the command reception. It can be controlled to perform (S1529).

In this case, under the control of the controller 240, the driving unit 280 may perform a preset operation.

For example, when the home appliance 200 is the air conditioner 200a, the driving unit 280 may include a vane driving unit 1210, a fan driving unit 1220, and the like.

As the voice command is received within the preset elapsed time, the vane driving unit 1210 drives one or more vanes 261R, 261L, 261U, 261D to perform at least one of lifting, moving, rotating, and swinging operations. can do.

For example, the elevating mechanism 1211 can elevate the upper vane 261U. In a state in which the upper vane 261U is closed without opening the discharge port 207U, the lifting mechanism 1211 can raise the upper vane 261U to a predetermined height.

Further, the left vane driving mechanism 1212 and the right vane driving mechanism 1213 can rotate the left vane 261L and the right vane 261R at a predetermined angle under the control of the control unit 240.

In addition, the left vane driving mechanism 1212 and the right vane driving mechanism 1213 can be controlled to perform a swing operation in which the left vane 261L and the right vane 261R repeat rotation within a predetermined range.

Further, the opening/closing mechanism 1214 can move the lower vane 261D forward in the inward direction to open the lower front air discharge port 207D.

According to an embodiment of the present invention, if a voice command is received within a preset elapsed time, the vanes 261R, 261L, 261U, 261D can be moved to intuitively notify the user of the command reception, thereby increasing the user's reliability. .

In addition, when the user inputs a voice command to the air conditioner 200a, since most cases are intended to operate the air conditioner 200a, preemptively move the vanes 261R, 261L, 261U, 261D, and then By operating in response to the voice command determined by the user, it is possible to respond faster to the user's voice command.

Meanwhile, the fan driving unit 1220 may drive the fan as the voice command is received within the preset elapsed time.

For example, the fan driving unit 1220 may rotate a fan that has been stopped. Alternatively, the fan driving unit 1220 may lower or stop the rotation speed of the rotating fan.

Meanwhile, the voice server 110 and the voice recognition server system 100 including the same can recognize and process voice data including voice commands received from the home appliance 200 as described with reference to FIGS. 1 to 9. I can.

Accordingly, when the communication unit 270 receives a control signal based on the voice command from the home appliance control server 130 (S1540), the control unit 240 controls the home appliance 200 to operate in response to the received control signal. It can be done (S1545).

In addition, when the communication unit 270 receives a response signal based on the voice command from the voice server 110 (S1550), the control unit 240 outputs a voice guidance message corresponding to the received response signal. 291) can be controlled (S1570).

Depending on the embodiment, when the response signal does not include voice data (S1560), the control unit 240 requests voice data from the voice server 110 (S1565), and the request from the voice server 110 It can be controlled to receive voice data.

On the other hand, according to the voice recognition result, a signal corresponding to the voice recognition failure may be received from the voice server 110 through the communication unit 270. In this case, the control unit 240 may control the audio output unit 291 to output a voice guidance message requesting re-input of the voice command, and control to switch to the command standby mode again.

In addition, according to the determination of whether or not the determined voice command can be supported, a signal indicating that the voice command is not supported from the voice server 110 may be received from the voice server 110 through the communication unit 270 (S1535). In this case, the controller 240 may control the audio output unit 291 to output a voice guidance message indicating that the voice command is related to a function not supported (S1537).

In this case, the controller 240 may control to switch to the wakeup signal standby mode.

Alternatively, the controller 240 may control to switch to the command standby mode again.

According to the present invention, by providing a voice guidance message suitable for each situation, there is an advantage in that it is possible to accurately grasp which problem is not normally controlled at any stage during a voice control command. Also, the usability is improved due to the high degree of freedom of commands to control the product.

16 is a flowchart illustrating a method of operating a smart home system including a voice recognition server system and a home appliance according to an embodiment of the present invention.

Referring to FIG. 16, the home appliance 200 activates the voice recognition function (S1611), and enters a wake-up signal standby mode for input of a wake-up voice signal including a preset call word. Can be (S1612).

When a preset call word such as "LG Whissen", "LG Dios", etc. is input through the microphones 221 and 222 (S1613), the home appliance 200 may switch to a command standby mode for receiving a voice command. (S1614).

In the command standby mode, it may be set to process only voice commands input during a predetermined elapsed time (S1615). For example, an elapsed time of 6 seconds may be set, and in this case, the controller 240 may control to transmit voice data including a voice command received within the 6 seconds to the voice server 110. Yes (S1621).

On the other hand, if the voice command is not received within the preset elapsed time (S1621), the controller 240 may control to switch back to the wakeup signal standby mode.

According to one aspect of the present invention, the home appliance 200 performs only a voice recognition process up to whether or not a caller is voice input, and voice recognition for a user's voice input may be performed through the voice recognition server system 100. .

When voice data including a voice command is received from the home appliance 200 (S1621), the voice server 110 may analyze the user's intention by performing natural language processing and determine the voice command (S1622).

If the voice command is not normally analyzed (S1623), the voice server 110 notifies the home appliance 200 of the failure to analyze the voice command, and the home appliance 200 utters a voice guidance message including a user's recommand request. It can be done (S1662).

In this case, when the voice command analysis fails, the recommand request and the recommand analysis process may be set to be performed only a predetermined number of times (S1661).

For example, if the normal analysis fails three or more times, without requesting a recommand, the controller 240 may control to switch to the wakeup signal standby mode again.

On the other hand, if the voice command is normally analyzed (S1623), the voice server 110 may transmit to the signal-linked service server 120 corresponding to the determined voice command.

Meanwhile, the linked service server 120 may determine whether or not the determined voice command is supported.

For example, the linked service server 120 may determine whether the determined voice command is a function supported by the corresponding home appliance 200 (S1631).

In addition, the linked service server 120 may determine whether the determined voice command is a function supported by the current state of the home appliance 200 or a function provided as a voice command (S1632).

If the determined voice command is not a function that can be supported, as described with reference to FIGS. 1 to 9, the linked service server 120 passes through the voice server 110 to the home appliance 200. It is possible to transmit a response signal indicating that the function is not supported.

Accordingly, the home appliance 200 may output voice guidance messages such as'This function does not support' and'This function does not support voice' (S1671 and S1672).

In addition, the control unit 240 may control to switch to the wakeup signal standby mode.

If the determined voice command is related to the home appliance 200, the linked service server 120 may communicate with the home appliance control server 130 to perform a corresponding operation.

Alternatively, if the determined voice command is not related to the home appliance 200, the linked service server 120 may communicate with an external external service to perform a corresponding operation.

If the determined voice command is to control the home appliance 200, the linked service server 120 may generate a request signal corresponding to the determined voice command and transmit it to the home appliance control server 130. .

The home appliance control server 130 may receive the request signal from the link service server 120 and transmit a control signal based on the request signal to the home appliance 200.

For example, when a request for changing the set temperature of the air conditioner 200a is received, the home appliance control server 130 may transmit a control signal for changing the set temperature to the air conditioner 200a.

Meanwhile, the home appliance 200 may perform a corresponding operation according to the control signal received from the home appliance control server 130 (S1640).

In addition, the home appliance 200 may transmit a signal indicating that the operation has been performed after performing the requested operation to the home appliance control server 130.

In addition, the home appliance control server 130 may receive a response signal to the control signal from the home appliance 200 and transmit processing result information notifying that the operation for the request was successful to the linked service server 120. have.

The linked service server 120 can transmit a signal corresponding to the processing result information such as "set the air conditioner temperature to 18 degrees" to the natural language processing server 112, and the natural language processing server 112 A response signal based on the command may be transmitted to the home appliance 200.

On the other hand, the home appliance 200 may utter a voice guidance message including an announcement such as "set the air conditioner temperature to 18 degrees" based on the voice data received from the voice server 110 (S1650).

17 to 21 are views referenced for descriptions of various operations of a voice recognition server system and a home appliance according to an embodiment of the present invention.

When a voice command is received within a preset elapsed time, the home appliance 200 according to an embodiment of the present invention may perform a feedback operation corresponding to the command reception to notify the user that the command has been received.

17 and 18 illustrate feedback operations performed when the home appliance 200 is the air conditioner 200a.

Referring to FIG. 17, when a voice command is received within a set elapsed time through the microphones 221 and 222 of the air conditioner 200a, the vane driving unit 1210 can raise the upper vane 261U to a predetermined height. .

Also, the vane driving unit 1210 may rotate the left vane 261L and the right vane 261R at a predetermined angle.

In addition, the vane driving unit 1210 may move the lower vane 261D forward in the inward direction to open the lower front air discharge port 207D.

When the air conditioner 200a is not operating in a specific operation mode and the air outlets are closed, when a voice command is received, the vane driving unit 1210 controls the plurality of vanes 261R according to the control of the controller 240. , 261L, 261U, 261D) all can be driven.

Alternatively, the vane driver 1210 may drive some of the plurality of vanes 261R, 261L, 261U, and 261D.

Referring to FIG. 18, when a voice command is received within a set elapsed time through the microphones 221 and 222, the vane driving unit 1210 operates the left vane 261L and the right vane 261R under the control of the control unit 240. It can be rotated at a predetermined angle.

Also, the vane driving unit 1210 may control the left vane 261L and the right vane 261R to perform a swing operation in which rotation is repeated within a predetermined range.

According to an embodiment of the present invention, if a voice command is received within a preset elapsed time, the vanes 261R, 261L, 261U, 261D can be moved to intuitively notify the user of the command reception, thereby increasing the user's reliability. .

In addition, when the user inputs a voice command to the air conditioner 200a, since most cases are intended to operate the air conditioner 200a, preemptively move the vanes 261R, 261L, 261U, 261D, and then By operating in response to the voice command determined by the user, it is possible to respond faster to the user's voice command.

19 to 21 illustrate various examples in which an air conditioner performs a corresponding operation in response to a user's natural language voice command input.

Referring to FIG. 19, a user may input a voice command 1910 in a natural language such as "Remove odor". The air conditioner 200a receiving the user's voice command 1910 through the microphones 221 and 222 may transmit the voice command 1910 to the voice recognition server system 100.

The voice recognition server system 100 may analyze the received voice command 1910 to determine the intention of a user who desires the air cleaning function. Accordingly, the voice recognition server system 100 may transmit a control signal for driving the air cleaning function to the air conditioner 200a and a response signal corresponding to a corresponding operation.

The air conditioner 200a may drive the air cleaning function and output a voice guidance message 1920 such as "drive the air cleaning function".

Referring to FIG. 20, a user may input a voice command 2010 in a natural language such as "Turn on the power saving function". The air conditioner 200a receiving the user's voice command 2010 through the microphones 221 and 222 may transmit the voice command 2010 to the voice recognition server system 100.

The voice recognition server system 100 may analyze the received voice command 2010 to determine the intention of a user who desires a power saving function. Accordingly, the voice recognition server system 100 may transmit a control signal for driving the power saving function to the air conditioner 200a and a response signal corresponding to a corresponding operation.

The air conditioner 200a may drive the power saving function and output a voice guidance message 2020 such as "Starting the power saving function".

Meanwhile, if the determined voice command is not related to the home appliance 200a, the voice recognition server system 100 according to an embodiment of the present invention may communicate with the external service 121 to perform a corresponding operation. .

Referring to FIG. 21, a user may input a voice command 2110 in a natural language such as "how is the weather today". The air conditioner 200a receiving the user's voice command 2110 through the microphones 221 and 222 may transmit the voice command 2110 to the voice recognition server system 100.

The voice recognition server system 100 may analyze the received voice command 2110 to determine the intention of a user who desires weather information. Accordingly, the voice recognition server system 100 may request and receive weather information by accessing an external service providing weather information or an Internet network.

In addition, the voice recognition server system 100 may transmit data based on the received weather information to the home appliance 200a.

The air conditioner 200a may output a voice guide message 2120 including weather information such as "Seoul is raining from afternoon" based on the received data.

The voice server, the voice recognition server system, and the home appliance according to the present invention are not limited to the configuration and method of the embodiments described as described above, but the embodiments are All or some may be selectively combined and configured.

Meanwhile, a method of operating a voice server, a voice recognition server system, and a home appliance according to an embodiment of the present invention can be implemented as code that can be read by a processor on a recording medium that can be read by a processor. The processor-readable recording medium includes all types of recording devices that store data that can be read by the processor. Examples of recording media that can be read by the processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., and also include those implemented in the form of carrier waves such as transmission through the Internet. . Further, the processor-readable recording medium is distributed over a computer system connected through a network, so that the processor-readable code can be stored and executed in a distributed manner.

In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

Speech Recognition Server System: 100
Voice Server: 110
ASR Server: 111
NLP Server: 112
TTS Server: 113
Linked service server: 120
Appliance Control Server: 130
Home Appliance: 200
Air conditioner: 200a

Claims (11)

Converting, by the home appliance, to a command waiting mode for receiving a voice command when a wake up signal including a preset call word is received in the wake-up signal waiting mode;
When the home appliance receives the voice command from the user in the command standby mode, performing a feedback operation corresponding to the reception of the voice command, and transmitting data including the voice command to a voice server;
Determining, by the voice server, whether the voice command received from the home appliance includes an unsupportable command;
If the voice command includes the unsupported command, transmitting, by the voice server, a signal indicating that the voice command is a command for a function not supported by the home appliance to the home appliance;
When the voice command does not include the unsupportable command, transmitting, by the voice server, a signal corresponding to the voice command to a linked service server;
Transmitting, by the linked service server, a request signal corresponding to the voice command to the home appliance control server based on the signal received from the voice server;
Transmitting, by the home appliance control server, a control signal corresponding to the voice command to the home appliance based on a request signal received from the link service server; And
And transmitting, by the voice server, a voice message for a result of processing the voice command to the home appliance,
The voice message is output as it is from the home appliance,
When the home appliance is an air conditioner having one or more fans and one or more vanes, the feedback operation is performed by the one or more vanes performing at least one of lifting, moving, rotating, and swinging, and the voice server or And opening an air outlet of the air conditioner before a signal is received from the home appliance control server. The method of claim 1,
Transmitting the voice message to the home appliance,
Receiving, by the home appliance control server, a response signal to the control signal from the home appliance;
Transmitting, by the home appliance control server, a signal for a processing result of the voice command corresponding to the response signal to the voice server through the linked service server; And
And generating, by the voice server, the voice message corresponding to a signal for a result of processing the voice command, and transmitting the voice message to the home appliance. The method of claim 1,
Transmitting the voice message to the home appliance,
When the voice command does not include the unsupportable command, generating, by the voice server, the voice message for a result of processing the voice command; And
At a time when the home appliance control server transmits the control signal to the home appliance, the voice server transmitting the voice message to the home appliance. The method of claim 3,
Transmitting the voice message to the home appliance,
The method of operating a system, wherein the voice server transmits the voice message to the home appliance in a streaming manner. The method of claim 4,
Receiving, by the link service server, status information of the home appliance from the home appliance control server;
Determining, by the linked service server, whether the home appliance can perform an operation according to the voice command based on the state information; And
When the home appliance is unable to perform the operation according to the voice command, the linked service server transmitting the support impossible signal to the home appliance through the voice server; operation of the system further comprising: Way. A home appliance for transmitting data including a voice command to a voice server;
The voice server for receiving data including the voice command from the home appliance and determining whether the voice command includes a command that cannot be supported;
A linked service server for receiving a signal corresponding to the voice command from the voice server, and generating a request signal corresponding to the voice command based on the signal received from the voice server; And
A home appliance control server receiving the request signal from the link service server and transmitting a control signal corresponding to the voice command to the home appliance based on the request signal received from the link service server,
The home appliance,
When a wake-up signal including a preset caller is received in the wake-up signal standby mode, switching to a command standby mode for receiving the voice command,
When the voice command is received from the user in the command standby mode, a feedback operation corresponding to the reception of the voice command is performed,
The voice server,
When the voice command includes the unsupported command, a signal informing that the voice command is a command for a function not supported by the home appliance is transmitted to the home appliance,
When the voice command does not include the unsupportable command, transmits a signal corresponding to the voice command to the linked service server,
It characterized in that transmitting a voice message for the processing result of the voice command to the home appliance,
The voice message is output as it is from the home appliance,
When the home appliance is an air conditioner having one or more fans and one or more vanes, the feedback operation is performed by the one or more vanes performing at least one of lifting, moving, rotating, and swinging, and the voice server or And opening the air outlet of the air conditioner before a signal is received from the home appliance control server. The method of claim 6,
The voice server,
An automatic speech recognition server for receiving data including the voice command from the home appliance and converting the data including the voice command into text data;
A natural language processing server that receives the text data from the automatic speech recognition server and processes the speech command based on the received text data; And,
And a text-to-speech server that receives a signal from at least one of the home appliance and the natural language processing server, generates a voice message corresponding to the received signal, and transmits it to the home appliance. The method of claim 7,
The home appliance control server,
Receiving a response signal to the control signal from the home appliance,
A signal for the processing result of the voice command corresponding to the response signal is transmitted to the voice server through the linked service server,
The voice server,
And generating the voice message corresponding to a signal for a result of processing the voice command, and transmitting the voice message to the home appliance. The method of claim 7,
The voice server,
When the voice command does not include the non-supportable command, generating the voice message for the processing result of the voice command,
At a time when the home appliance control server transmits the control signal to the home appliance, the voice message is transmitted to the home appliance. The method of claim 9,
The voice server,
And transmitting the voice message to the home appliance in a streaming method. The method of claim 10,
The linked service server,
Receiving state information of the home appliance from the home appliance control server,
Based on the state information, it is determined whether the home appliance can perform an operation according to the voice command,
When the home appliance is unable to perform an operation according to the voice command, the system transmits the support disable signal to the home appliance through the voice server.

Images