KR20180110973A - Home appliance and method for operating the same - Google Patents

Abstract

According to one aspect of the present invention, a method for operating home appliance comprises the steps of: activating a voice recognition function; entering a wake-up signal standby mode for inputting a wake-up voice signal including a preset call word; receiving the wake-up voice signal through a microphone (mic); entering into a command standby mode for inputting a voice command of a user for a preset elapsed time; receiving a voice command through the mic; and transmitting voice data including the voice command, received within the preset elapsed time, to a voice server. Therefore, the voice command may be efficiently recognized and processed.

Description

Home appliance and method of operating same

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to a home appliance and an operation method thereof, and more particularly, to a home appliance and an operation method thereof that can be controlled according to a user's natural language voice command.

Home appliances such as an air conditioner, a washing machine, a vacuum cleaner, and the like used in a predetermined space such as a home or an office have performed unique functions and operations according to user's operations, respectively.

For example, the air conditioner includes a compressor for compressing the refrigerant, a condenser for condensing the compressed refrigerant, an expansion valve for expanding the condensed refrigerant, and an evaporator for exchanging the expanded refrigerant with the room air, The device performs the function of adjusting the room temperature.

The refrigerator stores food, the washing machine processes the laundry, the cleaner performs the cleaning function, and the cooking device performs the cooking function.

In order to operate the home appliances such as the air conditioner, the user can use a remote control device such as a remote controller to avoid the inconvenience of manually operating buttons or the like provided on the home appliance main body or moving to the main body every time .

However, even when a remote controller is used, the user has to select and input an operation key for each function, which is inconvenient to use, and when the room is dark, separate illumination for identifying the remote controller and the operation key is required .

Therefore, there is an increasing research on a method of controlling a home appliance using speech recognition technology.

Prior Art 1 (Laid-Open Patent Publication No. 10-1999-00069703) discloses a remote controller for an air conditioner having an audio input unit and a signal processing unit to generate and transmit an operation signal according to voice recognition.

In the conventional art 2 (Japanese Unexamined Patent Publication (Kokai) No. 10-2006-0015092), an input voice signal is converted into a digital signal and text, and it is checked whether there is a control command in the database. And if there is no matching control command, extracts the keyword and controls each device in the air conditioner according to the associated control command.

However, there is a limit to the system resources that individual devices such as remote controllers and air conditioners can have. Especially, it is difficult to realize embedded modules that are installed in individual devices because it requires a high computation amount to recognize natural words rather than only a few words.

Therefore, the speech recognition technology of the related art 1 and the conventional art 2 has a limitation in recognizing and processing various natural language voice commands of users all over the world.

Therefore, there is a need for a method that can recognize and process natural language without restriction of the system resources of the individual devices, and can conveniently control the home appliance.

Also, in order to improve usability and satisfaction, it is necessary to provide a user experience (UX) considering various situations that may occur during a speech recognition process and a home appliance control process.

1. Published Patent Application No. 10-1999-00069703 (published September 6, 1999) 2. Open Patent Publication No. 10-2006-0015092 (published on February 18, 2006)

An object of the present invention is to provide a speech recognition method and system capable of efficiently recognizing and processing natural language.

An object of the present invention is to provide a home appliance controlled by a voice command of a user and an operation method thereof.

It is an object of the present invention to provide a user experience (UX) considering various situations that may occur during a speech recognition process and a home appliance control process.

According to an aspect of the present invention, there is provided a method of operating a home appliance, the method comprising: activating a voice recognition function; generating a wake-up voice signal including a pre- Up signal standby mode, receiving a wake-up voice signal through a microphone (MIC), entering a command standby mode for inputting a user's voice command for a preset elapsed time, And transmitting the voice data including the voice command received within a predetermined elapsed time to the voice server so that the voice command can be recognized and processed efficiently.

According to an aspect of the present invention, there is provided a home appliance, comprising: a microphone; A communication unit for wirelessly communicating with other electronic devices and a wake-up signal standby mode for inputting a wake-up voice signal including a predetermined call word when the voice recognition function is activated, When a wake up voice signal is received, enters a command standby mode for inputting a voice command of a user for a preset elapsed time. When a voice command is received within a preset elapsed time through a microphone, And a control unit for controlling the voice data to be transmitted to the voice server, so that voice commands can be efficiently recognized and processed.

According to at least one of the embodiments of the present invention, since the home appliance operates according to the voice input, the user does not need to operate the remote controller, and the user convenience can be increased.

Further, according to at least one of the embodiments of the present invention, the natural language can be efficiently recognized and processed.

In addition, according to at least one embodiment of the present invention, a user experience (UX) considering various situations that may occur during a speech recognition process and a home appliance control process can be provided.

Meanwhile, various other effects will be directly or implicitly disclosed in the detailed description according to the embodiment of the present invention to be described later.

1 is a view illustrating a smart home system including a voice recognition server system and a home appliance according to an embodiment of the present invention.
2 is a diagram illustrating various examples of home appliances.
3A is an example of a speech recognition server system according to an embodiment of the present invention.
3B is an example of a speech recognition server system according to an embodiment of the present invention.
4 to 6 are diagrams illustrating a signal flow of a speech recognition server system according to an embodiment of the present invention.
7 to 9 are diagrams illustrating a signal flow of a speech recognition server system according to an embodiment of the present invention.
FIG. 10 is a diagram showing an example of an internal block diagram of a server according to an embodiment of the present invention.
11 is a block diagram illustrating an example of an internal block diagram of a home appliance according to an embodiment of the present invention.
12 is a view showing a part of an internal block diagram of an air conditioner according to an embodiment of the present invention.
13 and 14 are views showing an outline of an air conditioner according to an embodiment of the present invention.
15 is a flowchart illustrating an operation method of a home appliance according to an embodiment of the present invention.
16 is a flowchart illustrating an operation method of a smart home system including a speech recognition server system and a home appliance according to an embodiment of the present invention.
17 to 21 are views referencing descriptions of various operations of a speech recognition server system and a home appliance according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, it is needless to say that the present invention is not limited to these embodiments and can be modified into various forms.

In the drawings, the same reference numerals are used for the same or similar parts throughout the specification.

The suffix "module" and " part "for components used in the following description are given merely for convenience of description and do not give special significance or role in themselves. Accordingly, the terms "module" and "part" may be used interchangeably.

FIG. 1 is a schematic view of a smart home system including a speech recognition server system and a home appliance according to an embodiment of the present invention. FIG. 2 is a view for explaining various examples of a home appliance.

Referring to FIG. 1, a smart home system 10 according to an embodiment of the present invention includes a home appliance 200 having a communication module (not shown) and capable of communicating with other devices or connecting to a network, And a voice recognition server system 100 including a plurality of servers for voice recognition and home appliance control.

2, the home appliance 200 may include an air conditioner 200a having a communication module, a robot cleaner 200b, a refrigerator 200c, a washing machine 200d, and a cooking device 200e. have.

In addition, the smart home system 10 according to an embodiment of the present invention may include a mobile terminal (not shown) such as a smart phone, a tablet PC, and the like.

The home appliance 200 may include a communication module to communicate with electronic devices inside / outside the smart home system 10.

The smart home system 10 according to an embodiment of the present invention may further include an access point (AP) apparatus 300. The home appliance 200 may be connected to an access point 300 via a wireless Internet It can connect to the network and communicate with other devices.

The access point 300 can allocate a wireless channel according to a predetermined communication method to the electronic devices in the smart home system 10 and perform wireless data communication through the corresponding channel.

Here, the predetermined communication method may be a Wi-Fi communication method. Correspondingly, the communication module included in the home appliance 200 may be a Wi-Fi communication module, but the present invention is not limited to the communication method.

Alternatively, the home appliance 200 may have another type of communication module or a plurality of communication modules. For example, the home appliance 200 may include an NFC module, a zigbee communication module, a Bluetooth communication module, and the like.

The home appliance 200 can be connected to a server included in the voice recognition server system 100 or a predetermined external server or a user's portable terminal through a wi-fi communication module or the like, Function can be supported.

The user can check information on the home appliance 200 in the smart home system 10 or control the home appliance 200 through the portable terminal.

On the other hand, it may be inconvenient for a user to use the portable terminal even if the user wants to control the home appliance 200 or confirm certain information in the home.

For example, it is more efficient to have a means to control the home appliance 200 in other ways when the user does not know the current location of the mobile terminal or is in another location.

The home appliance 200 according to an exemplary embodiment of the present invention can receive a user's voice input and the voice recognition server system 100 can recognize and analyze the user's voice input to control the home appliance 200 have.

Accordingly, the user can control the home appliance 200 without operating the portable terminal or the remote control device.

At least some of the servers included in the voice recognition server system 100 may be a server operated by a manufacturer of a home appliance, a sales company, or a server operated by a manufacturer or a company entrusted with a service by a sales company .

3A is an example of a speech recognition server system according to an embodiment of the present invention.

3A, a speech recognition server system according to an embodiment of the present invention includes a voice server 110 that receives voice data from a home appliance 200a and analyzes voice data received to determine a voice command .

The voice server 110 receives voice data from the home appliance 200a, converts the received voice data into text data, and analyzes the text data to determine a voice command.

Further, the voice server 110 can transmit a signal corresponding to the voice command determined to the predetermined server.

For example, the speech recognition server system according to an embodiment of the present invention receives a signal corresponding to the identified voice command from the voice server 110, and generates a request signal corresponding to the determined voice command And a home appliance control server 130 that transmits a control signal based on a request signal received from the connection service server 120 and the connection service server 120 to the home appliance 200a.

The home appliance 200a may receive voice command input by the user and transmit voice data based on the received voice command input to the voice server 110. [

The voice server 110 includes an Automatic Speech Recognition (ASR) server 111 for receiving voice data from the home appliance 200a and converting the received voice data into text data, Processing for receiving the text data from the speech recognition server 111, analyzing the received text data to determine a voice command, and transmitting a response signal based on the determined voice command to the home appliance 200a A local language processing (NLP) server 112, and a home appliance 200a for receiving a signal including a text corresponding to the response signal, converting the text included in the received signal into voice data, And a Text to Speech (TTS) server 113 for transmitting to the appliance 200a.

The automatic speech recognition server 111 may perform speech recognition on the speech data received from the home appliance 200a to generate text data and transmit the text data to the natural language processing server 112. [

The natural language processing server 112 can analyze the text data received from the automatic speech recognition server 111 according to a natural language processing algorithm to discriminate a voice command.

The natural language processing server 112 can process a natural language that is a language that a person routinely uses according to a natural language processing algorithm, and can analyze an intent of a user. The natural language processing server 112 can perform natural language processing on the text data received from the automatic speech recognition server 111 to discriminate a voice command that matches the user's intention.

Accordingly, the natural language processing server 112 can discriminate a voice command that matches the user's intention even if the user inputs a voice command in a daily use language.

The natural language processing server 112 can transmit a signal corresponding to the natural language processing result, that is, a signal corresponding to the discriminated voice command, to the linked service server 120.

The connection service server 120 can receive a signal corresponding to the determined voice command from the natural language processing server 112. [

The connection service server 120 can perform a corresponding operation by communicating with the home appliance control server 130 if the voice command is related to the home appliance 200a.

Alternatively, if the determined voice command is not related to the home appliance 200a, the connection service server 120 can perform a corresponding operation by communicating with the external service 121.

For example, if the determined voice command is a command for requesting information such as weather, stock, news, etc., the connection service server 120 requests and receives the information from the server providing the service corresponding to the requested information .

The connection service server 120 may transmit the received information to the voice server 110 and the natural language processing server 112 may transmit the received information to the home appliance 200a.

If the determined voice command is related to the home appliance 200a, the connection service server 120 may generate a request signal corresponding to the determined voice command and transmit the request signal to the home appliance control server 130. [

The home appliance control server 130 may transmit a control signal based on a request signal received from the associated service server 120 to the home appliance 200a.

For example, when a request to change the set temperature of the air conditioner 200a is received, the home appliance control server 130 may transmit a control signal for changing the set temperature to the air conditioner 200a.

Meanwhile, the home appliance 200a may perform a corresponding operation according to a control signal received from the home appliance control server 130. [

Also, the home appliance 200a may transmit a signal to the home appliance control server 130 indicating that the home appliance 200a has performed the requested operation.

The home appliance control server 130 may receive a response signal to the control signal from the home appliance 200a and may transmit processing result information corresponding to the response signal to the connection service server 120 have.

The voice server 110 may transmit a response signal including the processing result information to the home appliance 200a.

Also, the voice server 110 receives a signal including an output text text corresponding to the processing result information from the home appliance 200a, converts the received output text text into voice data, To the mobile station 200a.

In this case, the response signal based on the voice command determined by the natural language processing server 112 to be transmitted to the home appliance 200a may include the processing result information.

On the other hand, the home appliance 200a can receive a response signal based on the determined voice command from the natural language processing server 112. [ Here, the response signal may include text data of a response corresponding to the identified voice command.

For example, when the user inputs a voice command to change the set temperature of the air conditioner, the response signal may include text data indicating that the set temperature has been changed.

On the other hand, the home appliance 200a can transmit a signal including the text corresponding to the received response signal to the text-to-speech conversion server 113. [ Here, the signal including the text corresponding to the response signal may include an output text corresponding to the processing result information.

On the other hand, the text-to-speech conversion server 113 converts the text included in the received signal into voice data and transmits the voice data to the home appliance 200a. The converted voice data may include a sound source file.

The home appliance 200a can output a voice guidance message based on voice data received through the speaker.

Meanwhile, the connection service server 120 requests the home appliance control server 130 from the state information of the home appliance 200a based on the signal corresponding to the determined voice command, and the home appliance control server 130 May transmit state information of the home appliance 200a to the linked service server 120. [ The home appliance control server 130 can request and receive status information from the home appliance 200a when the home appliance 200a is not in a state where the home appliance 200a has the state information.

If the determined voice command can be supported based on the state information of the home appliance 200a, the connection service server 120 responds to the voice command determined by the home appliance control server 130 The request signal can be transmitted.

Alternatively, when it is impossible to support the determined voice command on the basis of the state information of the home appliance 200a, the connection service server 120 notifies the natural language processing server 112 of a function that is not supported in the current state A signal indicating that the signal is transmitted can be transmitted.

Also in this case, the home appliance 200a may request and receive the voice data to the text-to-speech conversion server 113 and output a voice guidance message indicating that the voice data is not supported in the current state.

According to the embodiment, the voice server 110 can determine whether or not the identified voice command can be supported. For example, the natural language processing server 112 that has analyzed the intention of the user's voice command can determine the supportability of the determined voice command.

In this case, when the determined voice command includes an instruction that can not be supported, a response signal based on the determined voice command transmitted from the natural language processing server 112 is transmitted to the home appliance 200a may be a signal that the function is not supported.

The voice server 110 and the voice recognition server system 100 including the voice server 110 according to an exemplary embodiment of the present invention can organically connect servers having various roles for natural voice processing.

The home appliance 200 performs operations from reception and preprocessing of voice commands to server transmission and the voice server 110 can perform a natural language process such as voice / text conversion, intent analysis, and command identification.

By performing the natural language processing by the voice server 110, burden on the CPU, memory, etc. of the embedded module in the home appliance can be reduced.

Meanwhile, the connection service server 120 may communicate with the external service and home appliance control server 130 to perform an operation based on the voice command of the user.

On the other hand, the home appliance 200 can receive voice data including a sound source file from the voice server 110 and output a voice guidance message to the voice to reply to the voice input of the user through auditory feedback.

The home appliance 200 receives the voice file streaming from the voice server 110 and reproduces and outputs the voice guidance message to the user. Accordingly, the home appliance 200 need not store various sound source files.

Meanwhile, it is possible to link with various external services without colliding with other servers through the connection service server 120. In addition, through the external service interworking server, it is possible to increase the success rate of the intention analysis by reflecting external information in the intention analysis.

The voice recognition server system 100 according to an embodiment of the present invention assures compatibility and connectivity through a plurality of servers and the final control command uses the home appliance control server 130 to perform a voice recognition process and a home appliance control It is possible to prevent a collision between the home appliance control using the Wi-Fi communication through the server 130 and the home appliance control through the mobile terminal and the home appliance control by the voice input through the home appliance 200. [

 The speech recognition server system 100 according to an embodiment of the present invention can reduce a load on a specific server depending on a server through an organic connection between the servers, If there is a problem with the server, it can be easily responded to by linking with other servers having the same role.

In addition, a plurality of servers can be independently updated from time to time, which is advantageous for performance improvement.

3B is an example of a speech recognition server system according to an embodiment of the present invention.

The voice recognition server system illustrated in FIG. 3B improves the voice control response time by improving the process of transmitting voice data for voice announcement message output to the home appliance 200a by the voice recognition server system illustrated in FIG. 3A .

Therefore, the speech recognition server system illustrated in FIGS. 3A and 3B can perform substantially the same operations other than the above-described differences, and the same portions will be briefly described below.

3B, a speech recognition server system according to an embodiment of the present invention includes a voice server 110 that receives voice data from a home appliance 200a and analyzes voice data received to determine a voice command .

In addition, the voice recognition server system according to an embodiment of the present invention may include a voice recognition server system 110 for receiving a signal corresponding to the voice command discriminated from the voice server 110 and generating a request signal corresponding to the discriminated voice command And a home appliance control server 130 for transmitting a control signal based on a request signal received from the service server 120 and the connection service server 120 to the home appliance 200a.

The voice recognition server system illustrated in FIG. 3B can transmit the voice data including the processing result information based on the voice command to the home appliance 200a without the request of the home appliance 200a can do.

The voice server 110 includes an automatic voice recognition server 111 for receiving voice data from the home appliance 200a and converting the received voice data into text data, A natural language processing server (112) for receiving the text data and analyzing the received text data to determine a voice command; and a conversion unit for converting a response signal based on the voice command into voice data and transmitting the voice data to the home appliance And a text-to-speech conversion server 113 for converting text to speech.

Also in this embodiment, the home appliance control server 130 receives a response signal to the control signal from the home appliance 200a, and transmits processing result information corresponding to the response signal to the connection service server 120 Can be transmitted.

The linking service server 120 can transmit the processing result information to the voice server 110, more specifically, the natural language processing server 112.

In this case, the voice data transmitted from the text-to-speech conversion server 113 to the home appliance 200a may include the processing result information.

In addition, the connection service server 120 requests the home appliance control server from the state information of the home appliance 200a based on a signal corresponding to the determined voice command, and the home appliance control server transmits, To the cooperative service server 120, the status information of the service server 200a.

If the determined voice command can be supported based on the state information of the home appliance 200a, the connection service server 120 responds to the voice command determined by the home appliance control server 130 The request signal can be transmitted.

Alternatively, if the determined voice command can not be supported on the basis of the state information of the home appliance 200a, the connection service server 120 determines that the voice server 110 is a function that is not supported in the current state An alert can send a signal.

For example, the connection service server 120 may transmit a signal to the natural language processing server 112 indicating that the function is an unsupported function in the current state.

In addition, the natural language processing server 112 transmits a signal indicating the unsupported function in the current state to the text-to-speech conversion server 113, and the text-to-speech conversion server 113 generates corresponding voice data, It can be transmitted to the appliance 200a.

The home appliance 200a receives the voice data from the text-to-speech conversion server 113 and outputs a voice guidance message indicating that the voice data is not supported in the current state.

According to the embodiment, the voice server 110 can determine whether or not the identified voice command can be supported. For example, the natural language processing server 112 that has analyzed the intention of the user's voice command can determine the supportability of the determined voice command.

In this case, when the determined voice command includes an instruction that can not be supported, a response signal based on the determined voice command transmitted from the natural language processing server 112 is transmitted to the home appliance 200a may be a signal that the function is not supported.

When the home appliance 200a requests the announcement about the corresponding operation to the text-to-speech server 113 for voice guidance for the last time after the operation of the home appliance 200a, a time difference may occur between the operation and the announcement.

However, according to an embodiment of the present invention, when the intention analysis is completed and the operation request is transmitted from the natural language processing server 112, the text voice conversion server 113 can provide information at the same time.

In addition, the text-to-speech conversion server 113 can provide the announcement to the home appliance 200a on the basis of the time when the home appliance control server 130 issues a control command to the home appliance 200a.

Accordingly, the announcement can be made simultaneously or immediately with the operation of the home appliance 200a.

According to the present embodiment, by directly connecting the natural language processing server 112 and the text-to-speech conversion server 113, the time difference between the control command and the announcement through the home appliance control server 130 can be minimized.

3A and 3B, the air conditioner 200a is exemplified as the home appliance 200, but the present invention is not limited thereto. For example, the home appliance 200 may include a robot cleaner 200b, a refrigerator 200c, a washing machine 200d, a cooking device 200e, and the like in addition to the air conditioner 200a.

3A and 3B, an automatic speech recognition server 111, a natural language processing server 112, and a text-to-speech conversion server 113 for speech recognition and processing are integrated Server.

Also, according to an embodiment, the connection service server 120 and the home appliance control server 130 may be configured as one integrated server.

According to the present invention, since the home appliance operates according to the voice input, the user does not need to operate a remote control device such as a remote control device, a portable terminal, etc., thereby improving user convenience.

As described with reference to FIGS. 3A and 3B, the present invention recognizes a user's natural language voice command using a plurality of servers, and performs a corresponding control operation so that the system resources of the home appliance, Natural language can be recognized and processed efficiently without limitation.

FIGS. 4 to 6 are diagrams illustrating a signal flow of a speech recognition server system according to an embodiment of the present invention, which illustrates a signal flow of a speech recognition server system illustrated in FIG. 3A.

4 illustrates signal flow in a typical situation in which the home appliance operates in response to a user's voice command.

Hereinafter, the air conditioner 200a will be described as an example of the home appliance 200, but the present invention is not limited thereto.

4, the home appliance 200, for example, the air conditioner 200a according to an exemplary embodiment of the present invention can receive a voice command of a user (S410) To the voice server 110 (S420).

For example, when the user receives an instruction to change the temperature setting such as "LG Whishen, temperature 18 degrees ", the air conditioner 200a converts the received voice command into a digital voice of a predetermined format such as a wave file Data can be converted into data and transmitted to the automatic voice recognition server 111.

On the other hand, the air conditioner 200a may receive a wakeup signal including a caller, and then send a voice command to the voice server 110 while waiting for a command input.

Alternatively, the air conditioner 200a may receive continuous speech input including a caller and a voice command. In this case, the air conditioner 200a can recognize the caller and send a voice command to the voice server 110. [

Meanwhile, when communicating with each other in the system 10, signals transmitted and received may further include identification information of each device, a session value for a task to be performed, and the like in addition to main data to be exchanged have.

The automatic speech recognition server 111 recognizes the received speech data and converts the received speech data into text data in step S431 and transmits the text data to the natural language processing server 112 in step S433.

The natural language processing server 112 may perform natural language processing on the received text data to analyze and determine the intention of the user's voice command (S441).

For example, the natural language processing server 112 can determine a voice command that the user intends to change the set temperature of the air conditioner 200a to 18 degrees at "LG Whischen, temperature 18 degrees Celsius ".

The natural language processing server 112 may transmit a signal corresponding to the determined voice command to the link service server 120 (S443).

When the connection service server 120 requests the current state information of the air conditioner 200a to the home appliance control server 130 in step S451, the home appliance control server 130 inquires about the current state information of the air conditioner 200a (S453), and the current state information of the air conditioner 200a can be transmitted to the linking service server 120 (S455).

If the home appliance control server 130 does not have the current state information of the air conditioner 200a, the home appliance control server 130 can request and receive current state information from the air conditioner 200a.

On the other hand, upon receiving the status information, the connection service server 120 may transmit a request signal to the home appliance control server 130 to change the set temperature of the corresponding air conditioner 200a by 18 degrees according to the determined voice command ( S461).

The home appliance control server 130 generates a control signal based on the request signal and transmits the control signal to the air conditioner 200a to control the air conditioner 200a (S463).

After the control of the air conditioner 200a, the home appliance control server 130 receives a response signal for the control signal from the home appliance, and transmits processing result information indicating that the operation for the request is successful to the connection service server 120 (S465).

The connection service server 120 can transmit a signal corresponding to the processing result information to the natural language processing server 112 (S470), such as "set the air condition temperature to 18 degrees" A response signal based on the voice command of the user can be transmitted to the home appliance 200a (S481). Here, the response signal based on the voice command of the discriminating user may include the processing result information.

On the other hand, the home appliance 200a transmits a signal including a text (output phrase) corresponding to the response signal (S483), and the text-to-speech conversion server 113 receives the text included in the received signal (S485), and transmits the data to the home appliance 200a (S487).

The home appliance 200a can output a voice guidance message such as "Set the air conditioner temperature to 18 degrees" based on the received voice data (S490).

FIG. 5 illustrates signal flow when requesting a non-serving function of the home appliance.

Referring to FIG. 5, the air conditioner 200a according to the embodiment of the present invention receives a user's voice command for a laundry function not supported by the air conditioner 200a, such as "LG Whishen, (S510).

The air conditioner 200a may transmit the voice command of the received user to the voice server 110 (S520). The air conditioner 200a may convert the received voice command into digital voice data of a predetermined format such as a wave file and transmit the converted digital voice data to the automatic voice recognition server 111. [

The automatic speech recognition server 111 recognizes the received speech data and converts the received speech data into text data in step S531 and transmits the text data to the natural language processing server 112 in step S533.

The natural language processing server 112 may perform natural language processing on the received text data to analyze and determine the intention of the user's voice command (S541).

In addition, the natural language processing server 112 can determine whether or not the identified voice command can be supported. In this embodiment, the natural language processing server 112 can determine that the user has requested the laundry function not supported by the air conditioner 200a.

Thereafter, the natural language processing server 112 may transmit a response signal to the air conditioner 200a indicating that the voice command is a function not supported by the home appliance (S543).

On the other hand, the home appliance 200a transmits a signal including a text (output phrase) corresponding to the response signal (S551), and the text-to-speech conversion server 113 receives the text included in the received signal (S553), and transmits the data to the home appliance 200a (S555).

The home appliance 200a can output a voice guidance message such as "This is a function that is not supported" based on the received voice data (S560).

6 illustrates signal flow when requesting a non-serving function in the current operating mode of the home appliance.

Referring to FIG. 6, the air conditioner 200a according to an embodiment of the present invention receives a voice command of a user instructing a specific mode operation of the air conditioner 200a, such as "LG Whishen, (S610).

The air conditioner 200a can transmit the voice command of the received user to the voice server 110 (S620). The air conditioner 200a may convert the received voice command into digital voice data of a predetermined format such as a wave file and transmit the converted digital voice data to the automatic voice recognition server 111. [

The automatic speech recognition server 111 recognizes the received speech data and converts the received speech data into text data (S631), and transmits the text data to the natural language processing server 112 (S633).

The natural language processing server 112 may perform natural language processing on the received text data to analyze and determine the intention of the user's voice command (S641).

For example, the natural language processing server 112 can determine the voice command of the intention of the user to operate the air conditioner 200a in the cool power mode at "LG Whishen, CoolPower Startup ".

The natural language processing server 112 may transmit the signal corresponding to the identified voice command to the link service server 120 (S643).

When the connection service server 120 requests current state information of the air conditioner 200a to the home appliance control server 130 in step S651, the home appliance control server 130 inquires current state information of the air conditioner 200a (S653), and the current state information of the air conditioner 200a can be transmitted to the linking service server 120 (S655).

If the home appliance control server 130 does not have the current state information of the air conditioner 200a, the home appliance control server 130 can request and receive current state information from the air conditioner 200a.

Meanwhile, the connection service server 120 receiving the status information can determine whether or not the voice command can be supported based on the current status information of the air conditioner 200a (S657). For example, when the current air conditioner 200a is operating in the dehumidification mode and the cool power mode is supported only in the cooling mode, the natural language processing server 112 requests the cool power mode, which is not supported in the current state, Can be determined.

Thereafter, the connection service server 120 may transmit a response signal to the natural language processing server 112 indicating that the voice command is a function that the home appliance 200a does not support in the present state (S660).

In addition, the natural language processing server 112 may transmit a response signal indicating that the voice command is a function not supported by the home appliance 200a in the current state (S671).

On the other hand, the home appliance 200a transmits a signal including a text (output phrase) corresponding to the response signal (S673), and the text-to-speech conversion server 113 receives the text included in the received signal (S675), and transmits the data to the home appliance 200a (S677).

The home appliance 200a can output a voice guidance message such as "Cool Power is supported only in the cooling mode" based on the received voice data (S680).

7 to 9 are diagrams illustrating a signal flow of a speech recognition server system according to an embodiment of the present invention, and are a diagram illustrating a signal flow of a speech recognition server system illustrated in FIG. 3B.

7 illustrates a signal flow in a typical situation in which the home appliance operates according to a user's voice command.

7, the home appliance 200, for example, the air conditioner 200a according to an exemplary embodiment of the present invention can receive a voice command of a user (S710) To the voice server 110 (S720).

The automatic speech recognition server 111 recognizes the received speech data and converts the received speech data into text data in step S731 and transmits the text data to the natural language processing server 112 in step S733.

The natural language processing server 112 may perform natural language processing on the received text data to analyze and determine the intention of the user's voice command (S741).

For example, the natural language processing server 112 can determine a voice command that the user intends to change the set temperature of the air conditioner 200a to 18 degrees at "LG Whischen, temperature 18 degrees Celsius ".

The natural language processing server 112 may transmit the signal corresponding to the determined voice command to the link service server 120 (S743).

When the connection service server 120 requests the current state information of the air conditioner 200a to the home appliance control server 130 in step S751, the home appliance control server 130 inquires current state information of the air conditioner 200a (S753), and the current state information of the air conditioner 200a can be transmitted to the linking service server 120 (S755).

If the home appliance control server 130 does not have the current state information of the air conditioner 200a, the home appliance control server 130 can request and receive current state information from the air conditioner 200a.

On the other hand, upon receiving the status information, the connection service server 120 may transmit a request signal to the home appliance control server 130 to change the set temperature of the corresponding air conditioner 200a by 18 degrees according to the determined voice command ( S761).

The home appliance control server 130 generates a control signal based on the request signal and transmits the control signal to the air conditioner 200a to control the air conditioner 200a (S763).

After the control of the air conditioner 200a, the home appliance control server 130 receives a response signal for the control signal from the home appliance, and transmits processing result information indicating that the operation for the request is successful to the connection service server 120 (S765).

The linking service server 120 can transmit a signal corresponding to the processing result information to the natural language processing server 112 such as "set the air conditioning temperature to 18 degrees" (S770).

On the other hand, the natural language processing server 112 can transmit a response signal based on the voice command of the discriminating user to the text / voice conversion server 113 (S781). Here, the response signal based on the voice command of the discriminating user may include the processing result information.

On the other hand, the text-to-speech conversion server 113 converts the response signal based on the voice command into voice data (S783) and transmits the voice data to the home appliance 200a (S785).

The home appliance 200a can output a voice guidance message such as "Set the air conditioner temperature to 18 degrees" based on the received voice data (S790).

FIG. 8 illustrates a signal flow when requesting a non-supported function of the home appliance.

Referring to FIG. 8, the air conditioner 200a according to an embodiment of the present invention receives a user's voice command for a laundry function not supported by the air conditioner 200a, such as "LG Whishen, (S810).

The air conditioner 200a may transmit the voice command of the received user to the voice server 110 (S820). The air conditioner 200a may convert the received voice command into digital voice data of a predetermined format such as a wave file and transmit the converted digital voice data to the automatic voice recognition server 111. [

The automatic speech recognition server 111 recognizes the received speech data and converts the received speech data into text data (S831), and transmits the text data to the natural language processing server 112 (S833).

The natural language processing server 112 can perform natural language processing on the received text data to analyze and determine the intention of the user's voice command (S841).

In addition, the natural language processing server 112 can determine whether or not the identified voice command can be supported. In this embodiment, the natural language processing server 112 can determine that the user has requested the laundry function not supported by the air conditioner 200a.

Thereafter, the natural language processing server 112 may transmit a response signal to the text / voice conversion server 113 indicating that the voice command is a function not supported by the home appliance (S843).

The text-to-speech conversion server 113 converts the response signal based on the voice command into voice data (S845) and transmits the voice data to the home appliance 200a (S847).

The home appliance 200a can output a voice guidance message such as "This is a function that is not supported" based on the received voice data (S850).

FIG. 9 illustrates signal flow when requesting a non-serving function in the current operating mode of the home appliance.

Referring to FIG. 9, the air conditioner 200a according to the embodiment of the present invention receives a voice command of a user instructing a specific mode operation of the air conditioner 200a, such as "LG Whishen, (S910).

The air conditioner 200a can transmit the voice command of the received user to the voice server 110 (S920). The air conditioner 200a may convert the received voice command into digital voice data of a predetermined format such as a wave file and transmit the converted digital voice data to the automatic voice recognition server 111. [

The automatic speech recognition server 111 recognizes the received speech data and converts the received speech data into text data (S931), and transmits the text data to the natural language processing server 112 (S933).

The natural language processing server 112 may perform natural language processing on the received text data to analyze and determine the intention of the user's voice command (S941).

For example, the natural language processing server 112 can determine the voice command of the intention of the user to operate the air conditioner 200a in the cool power mode at "LG Whishen, CoolPower Startup ".

The natural language processing server 112 can transmit a signal corresponding to the determined voice command to the link service server 120 (S943).

When the connection service server 120 requests the current state information of the air conditioner 200a to the home appliance control server 130 in step S951, the home appliance control server 130 inquires current state information of the air conditioner 200a (S953), the current state information of the air conditioner 200a can be transmitted to the linking service server 120 (S955).

If the home appliance control server 130 does not have the current state information of the air conditioner 200a, the home appliance control server 130 can request and receive current state information from the air conditioner 200a.

On the other hand, the connection service server 120 receiving the status information can determine whether the voice command can be supported based on the current status information of the air conditioner 200a (S957). For example, when the current air conditioner 200a is operating in the dehumidification mode and the cool power mode is supported only in the cooling mode, the natural language processing server 112 requests the cool power mode, which is not supported in the current state, Can be determined.

Thereafter, the connection service server 120 may transmit a response signal to the natural language processing server 112 indicating that the voice command is a function that the home appliance 200a does not support in the present state (S960).

In step S971, the natural language processing server 112 may transmit a response signal indicating that the voice command is a function not supported by the text-to-speech conversion server 113 in the current state.

On the other hand, the text-to-speech conversion server 113 converts the response signal based on the voice command into voice data (S973) and transmits the voice data to the home appliance 200a (S975).

The home appliance 200a can output a voice guidance message such as "Cool Power is supported only in cooling mode" based on the received voice data (S980).

According to the present invention, it is possible to implement a user experience that provides appropriate voice guidance for cases in which a command of a user is not understood or a function that can not be provided.

By providing an announcement tailored to each situation and time occurring during command processing, the inconvenience of the user can be minimized.

The server includes an automatic speech recognition server 111, a natural language processing server 112, a text-to-speech conversion server 113, A connection service server 120, and a home appliance control server 130. [

Referring to FIG. 10, the server may include a communication module 1020, a storage unit 1030, and a processor 1010.

The processor 1010 can control the overall operation of the server.

The communication module 1020 can receive various data such as status information, operation information, operation information, voice data, and character data from a home appliance such as a portable terminal, an air conditioner, or another server.

The communication module 1020 can transmit the data corresponding to the received various information to a home appliance such as a portable terminal, an air conditioner, or another server, as it is or after a predetermined process.

To this end, the communication module 1020 may include one or more communication modules such as an Internet module, a mobile communication module, and the like.

The storage unit 1030 may store the received information and may have data for generating the corresponding result information.

The storage unit 1030 may store data for operation of the server. For example, in the case of the automatic speech recognition server 111, an automatic speech recognition algorithm and the like are stored in the storage unit 1030. In the case of the home appliance control server 130, the product information of the home appliance, Can be stored.

Meanwhile, the server may be a server operated by a home appliance manufacturer such as an air conditioner or a server operated by a service provider, or may be a kind of a cloud server.

11 is a block diagram illustrating an example of an internal block diagram of a home appliance according to an embodiment of the present invention.

11, a home appliance 200 according to an exemplary embodiment of the present invention includes an audio input unit 220 for receiving voice commands of a user, a memory 250 for storing various data, A control unit 240 for controlling the overall operation of the mobile terminal 200. The control unit 240 controls the entire operation of the home appliances, .

The audio input unit 220 can receive external audio signals and user voice commands. To this end, the audio input unit 220 may include one or more microphones (MICs). In addition, the audio input unit 220 may include a plurality of microphones 221 and 222 in order to more accurately receive a user's voice command. The plurality of microphones 221 and 222 can be disposed at different positions and can acquire an external audio signal and process it as an electrical signal.

11 and the like, the audio input unit 220 includes two microphones including a first microphone 221 and a second microphone 222, but the present invention is not limited thereto.

The audio input unit 220 may include a processor for converting analog sound into digital data, or may be connected to a processing unit to convert user input voice commands into data to be recognized by the controller 240 or a predetermined server.

Meanwhile, the audio input unit 220 may use various noise reduction algorithms for eliminating noise generated in the process of receiving a voice command of a user.

The audio input unit 220 may include a filter for removing noise from an audio signal received from each of the microphones 221 and 222 and an amplifier for amplifying and outputting a signal output from the filter, .

The memory 250 records various kinds of information necessary for the operation of the home appliance, and may include a volatile or nonvolatile recording medium. The storage medium stores data that can be read by a microprocessor, and includes a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD- Tape, floppy disk, optical data storage, and the like.

Meanwhile, the memory 250 may store data for voice recognition, and the control unit 240 may process the voice input signal of the user received through the audio input unit 220 and perform a voice recognition process.

Meanwhile, a simple speech recognition may be performed by the home appliance 200, and a high-level speech recognition such as natural language processing may be performed in the speech recognition server system 100.

For example, in the case where a wake up voice signal including a preset caller is received, the home appliance 200 can be switched to a state for receiving a voice command. In this case, the home appliance 200 performs only the voice recognition process up to the voice input of the caller, and voice recognition for the subsequent voice input of the user can be performed through the voice recognition server system 100.

Home appliances Because there is a limit to these system resources, complex natural language recognition and processing can be performed through the speech recognition server system 100.

Limited data may be stored in the memory 250. For example, in the memory 250, data for recognizing a wake up voice signal including a predetermined caller may be stored. In this case, the controller 240 can recognize a wake-up voice signal including a preset caller from the voice input signal received through the audio input unit 220.

On the other hand, the caller can be set by a manufacturer, and another caller can be set for each home appliance. For example, "LG whisen " in the case of an air conditioner and" LG dios "in the case of a refrigerator may be set as an invocation word.

The caller can be changed by the user.

The control unit 240 can control the voice recognition server system 100 to transmit the voice command of the user inputted after recognizing the wake up voice signal to the voice recognition server system 100 through the communication unit 270. [

The communication unit 270 includes one or more communication modules and can perform wireless communication with other electronic devices to exchange various signals. For example, the communication unit 270 can communicate with electronic devices inside / outside the mall home system 10. [

The communication unit 270 may communicate with the access point 300 and may communicate with other devices by accessing the wireless Internet network through the access point 300. [

The control unit 240 can also transmit the state information of the home appliance 200, voice commands of the user, and the like to the voice recognition server system 100 or the like through the communication unit 270.

Meanwhile, when a control signal is received through the communication unit 270, the control unit 240 may control the home appliance 200 to operate according to the received control signal.

The output unit 290 may include a display 292 for displaying information corresponding to a command input by a user, a processing result corresponding to a command input by the user, an operation mode, an operation state, an error state,

According to an embodiment, the display 292 may be a touch screen having a mutual layer structure with a touch pad. In this case, the display 292 may be used as an input device capable of inputting information by a user's touch in addition to the output device.

The output unit 290 may further include an audio output unit 291 for outputting an audio signal. The audio output unit 291 outputs a notification message such as a warning sound, an operation mode, an operation state, an error state, etc., information corresponding to a command input by the user, a processing result corresponding to a command input by the user, Audio can be output. The audio output unit 291 can convert an electric signal from the control unit 240 into an audio signal and output it. For this purpose, a speaker or the like may be provided.

The driving unit 280 performs an operation implemented in the home appliance, and may be configured differently for each home appliance.

For example, when the home appliance is a refrigerator, the driving unit 280 includes a refrigerator compartment driving unit for operating a refrigerator compartment fan for supplying cooled air to the refrigerator compartment, a freezer compartment driving unit for operating a freezer compartment fan for supplying cooled air to the freezer compartment, A compressor driving unit for operating a compressor for compressing the refrigerant, and the like.

As another example, when the home appliance is a washing machine, the driving unit 280 may include a driving unit for driving the drum or the tub.

As another example, when the home appliance is an air conditioner, the driving unit 280 includes a compressor driving unit for driving the compressor in the outdoor unit, an outdoor fan driving unit for operating the outdoor fan for heat exchange, A fan drive unit, and the like.

As another example, when the home appliance is a cooker, the driving unit 280 may include a microwave driving unit that outputs microwaves into the cavity.

As another example, when the home appliance is a cleaner, the driving unit 280 may include a fan motor driving unit for air suction.

The home appliance 200 may further include an operation unit 230 for user input and a camera 210 for photographing a predetermined range around the home appliance 200.

The operation unit 230 may include a plurality of operation buttons, and may transmit a signal corresponding to the input button to the control unit 240.

The camera 210 photographs the surroundings of the home appliance 200, the external environment, and the like.

For example, the camera 210 may include at least one optical lens and an image sensor (e.g., an optical sensor) configured to include a plurality of photodiodes (e.g., pixels) that are imaged by light passing through the optical lens. A CMOS image sensor, and a digital signal processor (DSP) that forms an image based on signals output from the photodiodes. The digital signal processor is capable of generating moving images composed of still frames as well as still images.

Meanwhile, the image captured by the camera 210 may be stored in the memory 250.

FIG. 12 is a view showing a part of an internal block diagram of an air conditioner according to an embodiment of the present invention, and FIGS. 13 and 14 are views schematically showing an appearance of an air conditioner according to an embodiment of the present invention to be.

12 is a view showing a state in which the home appliance illustrated in Fig. 11 is the air conditioner 200a illustrated in Figs. 13 and 14, and the vane driver 200a included in the driving unit 280a of the air conditioner 200a, (1210) and a fan driving unit (1220).

FIG. 13 and FIG. 14 are views briefly showing an appearance of an indoor unit of an air conditioner according to an embodiment of the present invention.

11 to 14, an air conditioner 200a according to an embodiment of the present invention includes microphones 221 and 222 that receive voice commands of a user, a communication unit 270 that wirelessly communicates with other electronic devices, A control unit 240 for controlling the overall operation of the vane, and a vane driver 1210 for driving the vane under the control of the control unit 240.

In addition, the air conditioner 200a according to an embodiment of the present invention may further include a fan driving unit 1220 for driving the fan under the control of the controller 240. [

The microphones 221 and 222 can receive external audio signals and user voice commands. Also, the microphones 221 and 222 can acquire external audio signals and user voice commands and process them as electrical signals.

The air conditioner 200a of the present embodiment includes a suction body 202 having an air inlet (not shown), a heat exchanger (not shown) through which the air sucked by the air inlet is passed, air sucked into the air inlet, And a discharge port 207R, 207U, 207D formed to discharge air back into the room.

The air outlets 207R, 207U, and 207D are provided with wind direction adjusting means such as louvers (not shown) and / or vanes 261R, 261L, 261U, and 261D that open / close the air outlets 190 to adjust the air- And the angle and direction of the wind direction adjusting means can be switched by driving the motor or the like under the control of the control unit 240. [ Accordingly, the airflow direction of the air discharged through the discharge ports 207R, 207U, and 207D can be adjusted. Further, by controlling the operation of the wind direction adjusting means, it is possible to adjust the blowing direction of the airflow, the blowing range, and the like.

For example, the vanes 261R, 261L, 261U, and 261D may be fixed in either direction so as to blow air in either direction, or an operation in which the direction of the vane continues to change within a set range (Hereinafter referred to as "swinging operation") is performed so that the blowing operation can be performed over a predetermined range in which the blowing direction is continuously changed within the set range. Further, by adjusting the angular range in which the swing motion of the vanes 261R, 261L, 261U, and 261D is performed, the range in which the blowing is performed can be further narrowed or widened.

Meanwhile, in the air conditioner, there is provided a fan for controlling the flow of indoor air sucked into the indoor space through the discharge ports 207R, 207U, and 207D to the indoor space. The fan is connected to the fan driving unit 1220 And the operation of the fan driving unit 1220 is controlled by the controller 240. [

Accordingly, the control unit 240 controls the vane driving unit 1210 and the fan driving unit 1220 to control the direction of the airflow (airflow) discharged from the air conditioner 200a. The controller 240 controls the speed of the fan motor to control the amount and speed of the airflow, and controls the direction of the airflow by controlling the louver, the vane, and the like.

On the other hand, the air conditioner may include a plurality of vanes 261R, 261L, 261U, and 261D. Also, at least some of the plurality of vanes 261R, 261L, 261U, 261D may operate in different ways.

For example, the upper vane 261U can be raised and lowered on the upper portion of the fan, and can be raised and lowered during operation. To this end, the air conditioner 200a may include an elevating mechanism 1211 for elevating and lowering the upper vane 261U.

The upper vane 261U may function as an upper discharge body for discharging at least a part of the air blown from the fan through the air outlet 207U to the room.

The suction body 202 may be installed above the base 201. The suction body 202 can form the outer surface of the rear side of the air conditioner together with the base 201. [ The suction body 202 can form the upper surface of the rear surface of the air conditioner. The suction body 202 may constitute a rear case of the air conditioner together with the base 201. The suction body 202 may be a rear upper case of the air conditioner, It may be a rear lower case.

A control unit 240 for controlling various electrical components of the air conditioner may be installed in the base 201. Here, various electric components of the air conditioner may include fans and the like. The base 201 may be provided with a base cover 225 disposed in front of the base 201.

The air conditioner may further include a purifying unit (not shown) for purifying the air. The purifying unit may be disposed in the suction body 202. The purifying unit may be disposed before the heat exchanger in the air flow direction. The purifying unit can be disposed between the suction body 202 and the heat exchanger, and is disposed behind the suction body 202. The purifying unit may include a filter for filtering foreign matter in the air. The purifying unit may include an ion generator for generating ions in the air. The purifying unit may include an electrostatic precipitator for discharging foreign matter in the air to collect the dust. The purification unit may include at least one of a filter, an ion generator, and an electrostatic precipitator.

The heat exchanger may be located in front of the air inlet. The heat exchanger can heat-exchange the air sucked into the air inlet with the refrigerant. The heat exchanger may be installed between the air inlet and the fan.

When the compressor installed in the outdoor unit is driven, the air conditioner can cool the low-temperature and low-pressure refrigerant through the heat exchanger, and the heat exchanger can cool the air passing through the air inlet. When the compressor is on, the heat exchanger can cool the air by exchanging heat between the refrigerant and the air, and the air conditioner can function as a cooler.

The fan may be located in front of the heat exchanger. The fan may be constituted by a centrifugal fan that sucks air in the rear and blows air in at least one of the upper, lower, left, and right directions. The fan can consist of a sirocco fan. The fan may be a sirocco fan that blows air in four directions: up, down, left, and right.

The fan may include a plurality of blowing fans. The plurality of blowing fans can be sequentially arranged in the vertical direction. Each of the plurality of blowing fans may be constituted by a sirocco fan. The upper vane 261U,

An upper front air outlet 207U may be formed on the front surface of the upper vane 261U. An air inlet (not shown) may be formed on the lower surface of the upper vane 261U to allow the air blown from the fan upward. The upper vane 261U can guide the air blown upward from the inner fan in the forward direction.

The upper vane 261U may be arranged to be raised and lowered above the fan. When the upper vane 261U rises, the upper front air outlet 207U can be raised to discharge air, and the upper front air outlet 207U can be concealed when the vane 261U is lowered.

The air conditioner may further include an elevation guide 208 for guiding the elevation of the upper vane 261U. The elevation guide 208 may be installed above the fan and heat exchanger. The upper vane 261U is positioned on the upper side of the elevation guide 208 to discharge air when the upper vane 261U rises and the upper front air discharge port 207U is positioned on the inner side of the elevation guide 208 And can be concealed.

The elevating mechanism 1211 may include a driving source such as a motor and a power transmitting member for raising and lowering the upper vane 261U when the driving source is driven. The power transmitting member may include a pinion installed on the drive source and a rack engaged with the pinion.

The elevating mechanism 1211 can be raised to a different height depending on the operation mode. The control unit 240 may control the elevating mechanism 1211 according to the selection of the operation mode of the user.

For example, the control unit 240 may raise the upper vane 261U to the maximum opening height at which the upper front air outlet 207U is maximally opened, or only a part of the upper front air outlet 207U may be opened So that the upper vane 261U can be raised.

A side air outlet 207R through which the air blown in the lateral direction from the fan passes may be formed on the side surface of the air conditioner 200a.

Side vanes 261R and 261L can be disposed on the side surface of the air conditioner, which can control the direction of the air blown from the fan to the side and open and close the left air outlet and the right air outlet 207R. The side vanes 261R and 261L may include a left vane 261L and a right vane 261R.

The side vanes 261R and 261L may be rotatably installed. The air conditioner may include side driving mechanisms 1212 and 1213 for rotating the side vanes 261R and 261L. The side drive mechanisms 1212 and 1213 may include a left vane drive mechanism 1212 and a right vane drive mechanism 1213. [

The side driving mechanisms 1212 and 1213 may include a driving source such as a motor. The side drive mechanisms 1212 and 1213 may include at least one power transmitting member such as a link or a gear that is connected to the drive source and rotates the side vanes 261R and 261L when the drive source is driven.

The side drive mechanisms 1212 and 1213 can be controlled by the control unit 240 in the open mode during the cooling operation. The side drive mechanisms 1212 and 1213 can rotate the left vane 261L and the right vane 261R at a predetermined opening angle under the control of the control unit 240. [

Thus, the blowing direction can be controlled, and blowing can be performed in a range set in a manner that continuously changes the blowing direction within the set range by causing the swinging operation to be performed. In addition, by adjusting the angle range in which the swing motion is performed, the range in which the blowing is performed can be further narrowed or broadened.

The air conditioner may further include a lower vane 261D having a lower front air outlet 207D formed therein.

The lower vane 261D can discharge the air blown downward from the fan in the forward direction. The lower vane 261D may have a passage through which air flows. A lower front air discharge port 207D for discharging air to the outside may be formed on the front surface of the lower vane 261D.

The air conditioner may further include an opening / closing mechanism 1214 for opening / closing the lower front air outlet 207D. The opening and closing mechanism 1214 may include a moving kit (not shown) for moving the lower vane 261D to open and close the lower front air outlet 207D. The moving kit may include a driving source such as a motor, a moving kit carrier on which the moving kit is placed, and at least one power transmitting member for moving the moving kit carrier when the driving source is driven.

The lower vane 261D is advanced to cover the lower front air outlet 207D and can be retracted to open the lower front air outlet 207D.

The opening / closing mechanism 1214 can be controlled to an open mode in which the lower front air outlet 207D is opened during operation in a predetermined mode.

The air conditioner may further include a blower cover 282 disposed in front of the fan, and the blower cover 282 may form an upper front surface of the air conditioner. The blower cover 282 can be disposed on the upper side of the base cover 225 and can form an outer appearance of the front side of the air conditioner together with the base cover 225. The blower cover 282 can constitute the front cover of the air conditioner together with the base cover 225. The blower cover 282 may be a front upper cover of the air conditioner, and the base cover 225 may be a front lower cover of the air conditioner.

The air conditioner may be provided with a display 292 for displaying various information. The air conditioner may further include an audio output unit 291 for outputting an audio signal, and a speaker of the audio output unit 291 may be disposed on a front surface of the air conditioner or at an appropriate position.

  In addition, the air conditioner may include an operation unit 230 having a plurality of buttons for inputting various commands, and a camera 210 capable of photographing a predetermined range around the home appliance 200.

According to one aspect of the present invention, a front panel 1300 is disposed on the front surface of the blower cover 282 or the base cover 225, and a display 292, an audio output unit 291, Speakers, and operation buttons of the operation unit 230 can be disposed.

 The air conditioner can be operated according to a plurality of operation modes. The control unit 240 can control the vane driver 1210, the fan driver 1220, and the like according to the selected operation mode.

For example, in the predetermined operation mode, the control unit 240 can control the plurality of vanes 261R, 261L, 261U, and 261D to open all the discharge ports.

Or in the predetermined operation mode, the control unit 240 can control the specific vane to be closed without opening the discharge port. For example, in the predetermined operation mode, the controller 240 can control the lower vane 261D to be closed.

Also, the control unit 240 can adjust the opening height of the upper vane 261U and the opening angles of the left and right vanes 261R and 261L according to the operation mode.

In addition, the control unit 240 can adjust the air volume by controlling the rotation speed of the fan according to the operation mode.

On the other hand, the appearance and structure of the air conditioner illustrated in Figs. 13 and 14 are illustrative, and the present invention is not limited thereto. For example, the positions, numbers, structures, and the like of the microphones 221 and 222, the camera 210, the audio output unit 291, and the vanes 261R, 261L, 261U, and 261D may vary according to design specifications.

15 is a flowchart illustrating an operation method of a home appliance according to an embodiment of the present invention.

Referring to FIG. 15, the home appliance 200 can activate the voice recognition function by user input (S1510). The control unit 240 of the home appliance 200 can activate the microphones 221 and 222 according to user input.

Alternatively, the home appliance 200 may automatically activate the voice recognition function according to the setting for activating the voice recognition function.

For example, when the power is turned on, the home appliance 200 can automatically activate the microphones 221 and 222 and activate the voice recognition function.

When a voice input is received from the user while the microphones 221 and 222 of the home appliance are activated, the microphones 221 and 222 receive the input voice and can transmit the input voice to the control unit 240.

Meanwhile, the home appliance 200 may enter a wake up signal standby mode for inputting a wake up voice signal including a preset caller (S1515).

If a wake-up voice signal including a predetermined caller is received via the microphones 221 and 222 (S1520), the home appliance 200 switches to a command standby mode for receiving a voice command (S1525).

If the home appliance 200 always waits for a natural language command, the power consumed by the voice recognition function, the CPU occupation rate, and the load on the server of the voice recognition server system are burdensome.

Therefore, the home appliance 200 can transmit only the voice signal coming in the standby state of the command to the server.

In addition, the home appliance 200 provides a condition at a time of waiting for an instruction, and when receiving an instruction within a time, the home appliance 200 can receive a sound until the completion of the instruction and transmit the sound to the server.

According to one aspect of the present invention, the home appliance 200 performs only a voice recognition process up to whether a caller voice is input, and voice recognition for a subsequent user voice input can be performed through the voice recognition server system 100 .

On the other hand, the caller can be set by a manufacturer, and another caller can be set for each home appliance. For example, "LG whisen " in the case of an air conditioner and" LG dios "in the case of a refrigerator may be set as an invocation word. The caller can be changed by the user.

Meanwhile, the control unit 240 may control the voice recognition server system 100 to transmit the voice command of the user inputted after recognizing the wake-up voice signal through the communication unit 270 (S1530).

On the other hand, waiting indefinitely for a user's voice command is wasteful of system resources and does not match the intention of a user who does not input a command. Therefore, in the command standby mode, only voice commands input during a predetermined elapsed time can be set to be processed ).

In this case, the control unit 240 may control the voice server 110 to transmit the voice data including the voice command received within the predetermined elapsed time (S1530).

On the other hand, if the voice command is not received within the preset elapsed time (S1527), the controller 240 can control to switch to the wake-up signal standby mode again.

According to an embodiment of the present invention, when the voice command is received within the preset elapsed time (S1527), the control unit 240 notifies the user that the home appliance 200 has received the command, (S1529).

In this case, according to the control of the controller 240, the driver 280 may perform a preset operation.

For example, when the home appliance 200 is the air conditioner 200a, the driving unit 280 may include a vane driving unit 1210, a fan driving unit 1220, and the like.

As the voice command is received within the preset elapsed time, the vane driver 1210 drives one or more vanes 261R, 261L, 261U, 261D to perform at least one of up, down, swivel, can do.

For example, the elevating mechanism 1211 can elevate the upper vane 261U. In a state in which the upper vane 261U is closed without opening the discharge port 207U, the elevating mechanism 1211 can raise the upper vane 261U to a predetermined height.

The left vane drive mechanism 1212 and the right vane drive mechanism 1213 can rotate the left vane 261L and the right vane 261R by a predetermined angle under the control of the control unit 240. [

The left vane drive mechanism 1212 and the right vane drive mechanism 1213 can also be controlled to perform a swing operation in which the left vane 261L and the right vane 261R repeatedly rotate within a predetermined range.

Further, the opening / closing mechanism 1214 can advance the lower vane 261D inwards to open the lower front air outlet 207D.

According to an embodiment of the present invention, when a voice command is received within a preset elapsed time, the vanes 261R, 261L, 261U, and 261D are moved to intuitively inform the user of the reception of the command, .

When the user inputs voice commands to the air conditioner 200a, since most of the cases are intended to operate the air conditioner 200a, the vanes 261R, 261L, 261U, and 261D are moved preliminarily, The user can respond to the voice command more quickly.

On the other hand, the fan driving unit 1220 can drive the fan as the voice command is received within the predetermined elapsed time.

For example, the fan driving unit 1220 can rotate the stopped fan. Or the fan driving unit 1220 can lower or stop the rotational speed of the rotating fan.

Meanwhile, the voice server 110 and the voice recognition server system 100 including the voice server 110 recognize and process voice data including voice commands received from the home appliance 200, as described with reference to FIGS. 1 to 9 .

Accordingly, when the communication unit 270 receives the control signal based on the voice command from the home appliance control server 130 (S1540), the control unit 240 controls the home appliance 200 to operate in accordance with the received control signal (S1545).

When the communication unit 270 receives a response signal based on the voice command from the voice server 110 in step S1550, the control unit 240 controls the audio output unit to output the voice guidance message corresponding to the received response signal 291 (S1570).

The controller 240 requests the voice server 110 for voice data in step S1565 and transmits the voice data requested by the voice server 110 to the voice server 110 in step S1560. It is possible to control to receive voice data.

On the other hand, in accordance with the result of speech recognition, a signal corresponding to the speech recognition failure can be received from the speech server 110 via the communication unit 270. In this case, the control unit 240 may control the audio output unit 291 to output the voice guidance message requesting re-input of the voice command, and control to switch to the command standby mode again.

In step S1535, a signal indicating that the voice command is not supported is received from the voice server 110 via the communication unit 270 according to the discrimination of supportability of the discriminated voice command. In this case, the control unit 240 may control the audio output unit 291 to output a voice guidance message indicating that the voice command is related to the unsupported function (S1537).

In this case, the control unit 240 can control to switch to the wake-up signal standby mode.

Alternatively, the control unit 240 may control to switch to the command standby mode again.

According to the present invention, by providing a voice guidance message for each situation, it is possible to accurately grasp at which stage a voice control command is not normally controlled. In addition, the degree of freedom of commands for controlling the product is high, and usability is improved.

16 is a flowchart illustrating an operation method of a smart home system including a speech recognition server system and a home appliance according to an embodiment of the present invention.

16, the home appliance 200 activates the voice recognition function (S1611) to enter a wake-up signal standby mode for inputting a wake-up voice signal including a preset caller (S1612).

 When the preset callers such as "LG Whishen "," LG Dios ", etc. are input through the microphones 221 and 222 (S1613), the home appliance 200 can be switched to a command standby mode for receiving voice commands (S1614).

In the command standby mode, it is possible to set to process only voice commands input during a predetermined elapsed time (S1615). For example, an elapsed time of 6 seconds may be set. In this case, the control unit 240 may control to transmit the voice data including the voice command received within the 6 seconds to the voice server 110 (S1621).

On the other hand, if the voice command is not received within the preset elapsed time (S1621), the control unit 240 can control to switch to the wake-up signal standby mode again.

According to one aspect of the present invention, the home appliance 200 performs only a voice recognition process up to whether a caller voice is input, and voice recognition for a subsequent user voice input can be performed through the voice recognition server system 100 .

The voice server 110 may analyze the intention of the user and discriminate the voice command by performing the natural language processing (S1621) when the voice data including the voice command is received from the home appliance 200 (S1621).

If the voice command is not normally analyzed (S1623), the voice server 110 informs the home appliance 200 of the voice command analysis failure, and the home appliance 200 calls the voice guidance message including the user's re- (S1662).

In this case, the re-order request and the re-order analysis process may be set to be performed only a predetermined number of times (S1661).

For example, if the normal analysis fails three times or more, the control unit 240 can again control to switch to the wake-up signal standby mode without requesting re-ordering.

On the other hand, if the voice command is normally analyzed (S1623), the voice server 110 can transmit the voice command to the signaling service server 120 corresponding to the voice command.

Meanwhile, the connection service server 120 can determine whether the voice command can be supported or not.

For example, the connection service server 120 may determine whether the determined voice command is a function supported by the home appliance 200 (S1631).

In step S1632, the connection service server 120 may determine whether the voice command is a function supported by the current state of the home appliance 200, a function provided by a voice command, or the like.

1 to 9, if the determined voice command is not a function that can be supported by the voice command, the connection service server 120 transmits the voice command to the home appliance 200 via the voice server 110 It is possible to transmit a response signal indicating that it is not a function that can be supported.

Accordingly, the home appliance 200 can output a voice guidance message such as 'a function that is not supported' or 'a function that does not support voice' (S1671, S1672).

Also, the control unit 240 can control to switch to the wake-up signal standby mode.

The connection service server 120 can perform a corresponding operation by communicating with the home appliance control server 130 if the voice command is related to the home appliance 200. [

Alternatively, if the determined voice command is not related to the home appliance 200, the connection service server 120 may perform a corresponding operation by communicating with an external service.

If the determined voice command controls the home appliance 200, the connection service server 120 may generate a request signal corresponding to the determined voice command and transmit the request signal to the home appliance control server 130 .

The home appliance control server 130 may receive the request signal from the associated service server 120 and transmit a control signal based on the request signal to the home appliance 200. [

For example, when a request to change the set temperature of the air conditioner 200a is received, the home appliance control server 130 may transmit a control signal for changing the set temperature to the air conditioner 200a.

Meanwhile, the home appliance 200 may perform a corresponding operation according to the control signal received from the home appliance control server 130 (S1640).

In addition, the home appliance 200 may transmit a signal to the home appliance control server 130 indicating that the home appliance 200 has performed the requested operation.

Also, the home appliance control server 130 may receive a response signal to the control signal from the home appliance 200, and may transmit processing result information indicating that the operation for the request is successful to the linkage service server 120 have.

The connection service server 120 can transmit a signal corresponding to the processing result information to the natural language processing server 112 such as "set the air condition temperature to 18 degrees ", and the natural language processing server 112 can transmit the user's voice A response signal based on the command can be transmitted to the home appliance 200.

On the other hand, the home appliance 200 may generate a voice guidance message including an announcement such as "set the air conditioner temperature at 18 degrees" based on the voice data received from the voice server 110 (S1650).

17 to 21 are views referencing descriptions of various operations of a speech recognition server system and a home appliance according to an embodiment of the present invention.

The home appliance 200 according to an exemplary embodiment of the present invention may perform a feedback operation corresponding to reception of a command in order to notify a user that a voice command has been received within a predetermined elapsed time.

17 and 18 illustrate feedback operations performed when the home appliance 200 is the air conditioner 200a.

17, when a voice command is received within the set elapsed time through the microphones 221 and 222 of the air conditioner 200a, the vane driver 1210 may raise the upper vane 261U to a predetermined height .

In addition, the vane driver 1210 can rotate the left vane 261L and the right vane 261R at a predetermined angle.

In addition, the vane driver 1210 may move the lower vane 261D inwards to open the lower front air outlet 207D.

When the voice command is received when the air conditioner 200a is not operating in the specific operation mode and the air outlets are closed, the vane driver 1210 controls the vanes 261R , 261L, 261U, and 261D can be driven.

Alternatively, the vane driver 1210 may drive some of the plurality of vanes 261R, 261L, 261U, and 261D.

18, when a voice command is received within the set elapsed time through the microphones 221 and 222, the vane driver 1210 controls the left vane 261L and the right vane 261R under the control of the controller 240 And can be rotated at a predetermined angle.

In addition, the vane driver 1210 can control the swing operation in which the left vane 261L and the right vane 261R repeatedly rotate within a predetermined range.

According to an embodiment of the present invention, when a voice command is received within a preset elapsed time, the vanes 261R, 261L, 261U, and 261D are moved to intuitively inform the user of the reception of the command, .

When the user inputs voice commands to the air conditioner 200a, since most of the cases are intended to operate the air conditioner 200a, the vanes 261R, 261L, 261U, and 261D are moved preliminarily, The user can respond to the voice command more quickly.

19 to 21 show various examples in which the air conditioner performs a corresponding operation in response to a user's natural language voice command input.

Referring to Fig. 19, the user can input a voice command 1910 in a natural language such as "Deodorize ". The air conditioner 200a receiving the voice command 1910 of the user through the microphones 221 and 222 can transmit the voice command 1910 to the voice recognition server system 100. [

The speech recognition server system 100 may analyze the received voice command 1910 to determine the intent of the user for the air cleaning function. Accordingly, the speech recognition server system 100 can transmit a control signal to the air conditioner 200a to operate the air cleaning function and a response signal corresponding to the operation.

The air conditioner 200a drives the air cleaning function and can output a voice guidance message 1920 such as "drive the air cleaning function ".

Referring to FIG. 20, the user can input a voice command 2010 in a natural language such as "turn on the power save function ". The air conditioner 200a receiving the user voice command 2010 through the microphones 221 and 222 can deliver the voice command 2010 to the voice recognition server system 100. [

The voice recognition server system 100 may analyze the received voice command 2010 to determine the intention of the user who desires the power save function. Accordingly, the speech recognition server system 100 can transmit a control signal for driving the power saving function and a response signal corresponding to the operation to the air conditioner 200a.

The air conditioner 200a drives the power saving function and can output a voice guidance message 2020 such as "start power saving function ".

Meanwhile, the voice recognition server system 100 according to the embodiment of the present invention may perform a corresponding operation by communicating with the external service 121, if the voice command is not related to the home appliance 200a .

Referring to FIG. 21, the user may enter a voice command 2110 in a natural language such as "Today's weather." The air conditioner 200a receiving the voice command 2110 of the user through the microphones 221 and 222 can deliver the voice command 2110 to the voice recognition server system 100. [

The speech recognition server system 100 may analyze the received voice command 2110 to determine a user's intention of the weather information. Accordingly, the speech recognition server system 100 can access an external service or an Internet network that provides weather information, and can request and receive weather information.

In addition, the voice recognition server system 100 can transmit data based on the received weather information to the home appliance 200a.

The air conditioner 200a can output a voice guidance message 2120 including weather information such as "Seoul is raining from afternoon" based on the received data.

The voice server, the voice recognition server system, and the home appliance according to the present invention are not limited to the configuration and method of the embodiments described above, but the embodiments may be modified in various ways, All or some of them may be selectively combined.

Meanwhile, the voice server, the voice recognition server system, and the operation method of the home appliance according to the embodiment of the present invention can be implemented as a processor readable code on a recording medium readable by a processor. The processor-readable recording medium includes all kinds of recording apparatuses in which data that can be read by the processor is stored. Examples of the recording medium that can be read by the processor include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet . In addition, the processor-readable recording medium may be distributed over network-connected computer systems so that code readable by the processor in a distributed fashion can be stored and executed.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It should be understood that various modifications may be made by those skilled in the art without departing from the spirit and scope of the present invention.

Speech Recognition Server System: 100
Voice server: 110
ASR server: 111
NLP server: 112
TTS server: 113
Linked service server: 120
Home appliance control server: 130
Home Appliance: 200
Air conditioner: 200a

Claims (20)

Activating a voice recognition function;
Entering a wake up signal standby mode for input of a wake up voice signal comprising a preset call word;
Receiving the wake-up voice signal through a microphone (MIC);
Entering a command standby mode for inputting a voice command of a user for a preset elapsed time;
Receiving a voice command through the microphone; And
And transmitting voice data including a voice command received within the predetermined elapsed time to a voice server. The method according to claim 1,
And performing a feedback operation corresponding to the reception of the voice command when the voice command is received within the preset elapsed time. 3. The method of claim 2,
If the home appliance is an air conditioner,
Wherein the step of performing the feedback operation corresponding to the reception of the command comprises:
And driving one or more vanes. The method of claim 3,
Wherein the at least one vane performs at least one of elevating, moving, rotating, and swinging operations. The method according to claim 1,
And switching to the wake-up signal standby mode if the voice command is not received within the preset elapsed time. The method according to claim 1,
Receiving a control signal based on the voice command from the home appliance control server, and operating in response to the received control signal. The method according to claim 6,
Receiving a response signal based on the voice command from the voice server, and outputting a voice guidance message corresponding to the received response signal. 8. The method of claim 7,
Requesting voice data to the voice server when the response signal does not include voice data;
Further comprising: receiving voice data requested from the voice server;
Wherein the voice guidance message is based on the received voice data. The method according to claim 1,
Outputting a voice guidance message requesting re-input of a voice command upon receiving a signal corresponding to a voice recognition failure from the voice server;
And switching to the command standby mode. The method according to claim 1,
And outputting a voice guidance message informing that the voice command is related to the unsupported function when receiving a signal from the voice server indicating that the voice command is related to an unsupported function . Microphone (MIC);
A communication unit for wirelessly communicating with another electronic device; And
When the voice recognition function is activated, enters a wake up signal standby mode for inputting a wake up voice signal including a preset caller, and receives the wake up voice signal through the microphone When a voice command is received within the preset elapsed time through the microphone, the voice data including the voice command is transmitted to the voice server The home appliance comprising: a home appliance; 12. The method of claim 11,
And a driving unit for performing a feedback operation corresponding to the reception of the voice command when the voice command is received within the preset elapsed time. 13. The method of claim 12,
And the driving unit includes a vane driver for driving the at least one vane when the voice command is received within the preset elapsed time. 14. The method of claim 13,
Wherein the vane drive unit controls the at least one vane to perform at least one of an operation of lifting, moving, rotating, and swinging the at least one vane. 12. The method of claim 11,
Wherein the control unit controls to switch to the wake-up signal standby mode if the voice command is not received within the preset elapsed time. 12. The method of claim 11,
Wherein when the communication unit receives a control signal based on the voice command from the home appliance control server, the control unit controls the home appliance to operate in accordance with the received control signal. 17. The method of claim 16,
And an audio output unit for outputting a voice guidance message corresponding to the received response signal when the communication unit receives a response signal based on the voice command from the voice server. 18. The method of claim 17,
Wherein the control unit controls the communication unit to request voice data to the voice server when the response signal does not include voice data,
Wherein the communication unit receives the voice data requested from the voice server,
Wherein the voice guidance message is based on the received voice data. 12. The method of claim 11,
And an audio output unit outputting a voice guidance message requesting re-input of a voice command when the communication unit receives a signal corresponding to a voice recognition failure from the voice server,
Wherein the control unit controls to switch to the command standby mode after outputting the voice guidance message. 12. The method of claim 11,
And an audio output unit for outputting a voice guidance message informing that the voice command is related to the unsupported function when the communication unit receives a signal from the voice server indicating that the voice command is not supported Home Appliance.

Images