KR102080514B1 - Mobile terminal, home appliance, and method for operating the same - Google Patents

Abstract

The present invention relates to a mobile terminal, a home appliance and a method for operating the same. According to an embodiment of the present invention, a method of operating a mobile terminal includes: displaying a predetermined screen on a display, entering an acoustic communication mode, and information related to a screen according to the acoustic communication mode. And outputting a sound including a to the home appliance. Accordingly, it is possible to transmit information between the home appliance and the mobile terminal by acoustic communication.

Description

Mobile terminal, home appliance, and method for operating the same

The present invention relates to a mobile terminal, a home appliance and a method for operating the same, and more particularly, to a mobile terminal, a home appliance and a method for operating the mobile terminal capable of transmitting information between the home appliance and the mobile terminal by acoustic communication. .

Of the home appliances placed in the building, the refrigerator stores food for the user, the laundry treatment device processes the laundry, the air conditioner adjusts the room temperature, and the cooking device performs functions such as cooking food. .

On the other hand, as various communication methods have been developed, various methods for increasing user's usability through communication with a home appliance have been studied.

An object of the present invention is to provide a mobile terminal, a home appliance and a method of operating the same, which can transmit information between the home appliance and the mobile terminal by means of acoustic communication.

A method of operating a mobile terminal according to an exemplary embodiment of the present invention for achieving the above object includes displaying a predetermined screen on a display, entering an acoustic communication mode, and displaying information related to the screen according to the acoustic communication mode. And outputting the included sound to the home appliance.

In addition, the operation method of the home appliance according to an embodiment of the present invention for achieving the above object, receiving a sound including information related to the screen displayed on the mobile terminal according to the acoustic communication mode, and the screen from the sound And extracting information related to and executing the corresponding operation based on the extracted information.

In addition, an operation method of a mobile terminal according to an embodiment of the present invention for achieving the above object is a step of displaying a predetermined screen, entering an acoustic communication mode, and receiving sound including information from a home appliance. And extracting information from the sound and displaying the extracted information.

In addition, the operation method of the home appliance according to an embodiment of the present invention for achieving the above object, when in operation, storing information related to the home appliance and outputting sound including stored information according to an acoustic communication mode It includes the steps.

In addition, a mobile terminal according to an embodiment of the present invention for achieving the above object, a display for displaying a predetermined screen, an audio output module for outputting sound to the outside, a microphone for receiving sound from the outside, and an acoustic communication mode In accordance with this, an acoustic communication unit for outputting sound including information related to a screen to an audio output module, or extracting predetermined information from a sound received from a microphone, and a control unit for controlling to display information extracted from the acoustic communication unit on a display do.

In addition, the home appliance according to an embodiment of the present invention for achieving the above object, a driving unit, a sound output module for outputting sound to the outside, a microphone for receiving sound from the outside, and when the home appliance is operating, the home A memory for storing information related to the appliance, an audio communication unit for outputting sound including information related to the home appliance to the sound output module, or extracting predetermined information from the sound received from the microphone according to the sound communication mode, and the sound communication unit It includes a control unit for controlling the driving unit, according to the information extracted from.

According to an embodiment of the present invention, the mobile terminal may output sound including information related to a displayed screen by sound communication. In addition, the home appliance may receive sound and extract information in the received sound. Accordingly, it is possible to simply transmit information desired by the user to the home appliance, thereby increasing user convenience.

On the other hand, according to an embodiment of the present invention, the home appliance is simply by receiving a sound including information related to the screen displayed on the mobile terminal by sound communication, and executing a corresponding operation according to the received information, , The user can operate as desired.

Meanwhile, according to another embodiment of the present invention, the mobile terminal can receive a sound including information related to a displayed screen from a home appliance by acoustic communication. Accordingly, it is possible to simply receive information desired by the user from the home appliance, so that the user's usability can be increased.

Meanwhile, according to another embodiment of the present invention, the home appliance may store information related to the home appliance when in operation, and output sound including stored information by sound communication, thereby simplifying information desired by the user. It can be transmitted.

1 is an example of a configuration diagram of a communication system according to an embodiment of the present invention.
2A to 2E are diagrams for explaining various examples of acoustic communication between a mobile terminal and a home appliance.
3A is an example of a simplified interior block diagram of the home appliance of FIG. 1.
3B is another example of a simplified interior block diagram of the home appliance of FIG. 1.
4A to 4D are diagrams referenced to describe the acoustic communication of FIG. 1.
FIG. 5 is an internal block diagram of the server of FIG. 1.
6 is an internal block diagram of the mobile terminal of FIG. 1.
7 is a flowchart illustrating an operation method of a mobile terminal according to an embodiment of the present invention.
8 is a flowchart illustrating a method of operating a home appliance according to an embodiment of the present invention.
9 to 13E are views referred to for explanation of the operation method of FIGS. 7 or 8.
14 is a flowchart illustrating a method of operating a mobile terminal according to another embodiment of the present invention.
15 is a flowchart illustrating a method of operating a home appliance according to another embodiment of the present invention.
16A to 22C are views referred to for explanation of the operation method of FIGS. 14 or 15.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

The suffixes "modules" and "parts" for components used in the following description are given simply by considering the ease of writing the present specification, and do not give meanings or roles particularly important in themselves. Therefore, the "module" and the "unit" may be used interchangeably.

1 is an example of a configuration diagram of a communication system according to an embodiment of the present invention.

Referring to the drawings, the communication system 10 according to an embodiment of the present invention may include a home appliance 200, a mobile terminal 600, and a server 500.

In particular, in relation to the embodiment of the present invention, the home appliance 200 and the mobile terminal 600 may perform acoustic communication (Acoustic communication or sound communication).

The acoustic communication according to the embodiment of the present invention means that a sound in an audible frequency band is output, but predetermined information data is added to the sound. In order to add such information data, signal modulation is required, and in the embodiment of the present invention, it is assumed that modulation is performed based on a modulated complex lapped transform (MCLT).

Specifically, the mobile terminal 600 may output a sound including information related to a screen displayed on the mobile terminal 600 through the audio output module 653 by an acoustic communication method. Then, the home appliance 200 may receive the corresponding sound through the microphone 223, extract information in the sound, and receive the corresponding information. Accordingly, it is possible to conveniently transmit the related information to the home appliance 200, so that the user's usability can be increased.

For example, when the screen displayed on the mobile terminal 600 is a power information screen related to the home appliance, the mobile terminal 600 may output a sound including power information related to the home appliance.

As another example, when the screen displayed on the mobile terminal 600 is a recipe screen related to a home appliance, the mobile terminal 600 may output a sound including recipe information.

As another example, when the screen displayed on the mobile terminal 600 is a driving course screen or a driving course list screen related to the home appliance, the mobile terminal 600 outputs sound including driving course information related to the home appliance. can do.

As another example, when the screen displayed on the mobile terminal 600 is a firmware screen related to the home appliance, the mobile terminal 600 may output sound including firmware information related to the home appliance.

As another example, when the screen displayed on the mobile terminal 600 is an access point device list screen, the mobile terminal 600 may output sound including network information about the access point device.

Meanwhile, the mobile terminal 600 may receive a sound including information previously stored in the home appliance 200 from the home appliance 200 through a microphone 223 by an acoustic communication method. Then, by extracting the corresponding information from the received sound, it is possible to conveniently receive the relevant information from the home appliance 200, so that user convenience can be increased.

To this end, the home appliance 200 may, at any time, store information related to the home appliance. For example, the operation status information, operation history information, product information, failure information, power consumption information, stored article information, or recipe information of the home appliance may be stored in the memory 260.

Here, the operation state information may include driving period information, current driving time information, remaining driving time information, current driving type information, and the like when currently driving.

The motion history information may include motion history information for a specific period of time in the home appliance.

For example, door opening and closing information of the refrigerator 200a, driving of the washing machine 200b, frequently used course driving information, driving of the washing machine 200b, water usage information, cooling frequency information of the air conditioner 200c, cooling Temperature setting information, internal cleaning frequency information, when driving the cooking appliance 200d, frequently used course driving information, frequently used cooking time driving information, cleaning frequency information of the cleaner 200e, cleaning time zone information, cleaning course information, etc. It may include.

Meanwhile, the product information may be a concept including product type information (refrigerator, washing machine, etc.) of the home appliance, product manufacturing date information, product serial number, and firmware type / version information installed in the product.

On the other hand, the failure information, when the home appliance is broken, at the time of the failure, may include information about the part, which is not operated, the time of the failure, the time of the failure, and the driving information during operation.

The power consumption information may include power consumption information, cost information according to the power consumption, and the like.

The stored article information may include food type information stored in the refrigerator 200a, food location information stored in the refrigerator 200a, food shelf life information, food arrival date information, and the like when the home appliance is the refrigerator 200a. You can. When the home appliance is the refrigerator 200a, the recipe information may be recipe information that can be cooked in correspondence with the food type stored in the refrigerator. Alternatively, it may be recipe information that can be cooked in the cooking appliance 200d.

Meanwhile, the mobile terminal 600 may display information received from the home appliance 200 on the mobile terminal 600. Thereby, the user can easily grasp the received information.

Meanwhile, the mobile terminal 600 may transmit information received from the home appliance 200 to the server 500 and receive a transmission result. Accordingly, it is possible to simply perform management of the home appliance 200 using the mobile terminal 600.

For example, if the information received from the home appliance 200 is fault information, the mobile terminal 600 may transmit fault information to the server 500 and receive fault result information from the server 500. .

As another example, when the information received from the home appliance 200 is product information, the mobile terminal 600 may transmit product information to the server 500 and receive product result information from the server 500.

As another example, when the information received from the home appliance 200 is power consumption information, the mobile terminal 600 transmits power consumption information to the server 500 and saves power operation information or power information from the server 500. Can receive. Here, the power information may include power price information, on-peak time power information, off-peak time power information, and real-time power information.

According to such an acoustic communication method, the mobile terminal 600 and the home appliance 200 need only be provided with an audio output module and a microphone, respectively, so information can be easily exchanged. In particular, since the home appliance 200 does not need to have a separate wireless communication module, such as WiFi, manufacturing cost can be reduced.

Meanwhile, the home appliance 200 is an electronic device for a user, for example, a refrigerator 200a, a washing machine 200b, an air conditioner 200c, a cooking appliance 200d, a cleaner 200e, etc. This can be illustrated. In addition, it may be a concept including TV and the like.

Meanwhile, a mobile terminal 600, a mobile phone, a smart phone, a laptop, and a tablet PC are possible.

2A to 2E are diagrams for explaining various examples of acoustic communication between a mobile terminal and a home appliance.

FIG. 2A illustrates that acoustic communication is performed between the mobile terminal 600 and the refrigerator 200a, and FIG. 2B illustrates that acoustic communication is performed between the mobile terminal 600 and the washing machine 200b, 2C illustrates that acoustic communication is performed between the mobile terminal 600 and the air conditioner 200c, and FIG. 2D illustrates that acoustic communication is performed between the mobile terminal 600 and the cooking appliance 200d. Illustratively, FIG. 2E illustrates that acoustic communication is performed between the mobile terminal 600 and the cleaner 200e.

According to FIGS. 2A to 2E, acoustic communication by sound output from a mobile terminal, communication by sound output from a home appliance, or a combination thereof is possible, respectively.

3A is an example of a simplified interior block diagram of the home appliance of FIG. 1.

Referring to the drawings, the home appliance 200 includes an input unit 220 for user input, a display unit 230 for displaying an operation state of the home appliance, a communication unit 222 for communication with other external devices, and a home appliance It may include a driving unit 245 for driving, and a control unit 270 for internal control.

For example, when the home appliance is a refrigerator, the driving unit 245 may include a refrigerator compartment driving unit and a freezing compartment driving unit.

As another example, when the home appliance is a washing machine, the driving unit 245 may be a driving unit that drives a drum or tub.

In another example, when the home appliance is an air conditioner, the driving unit 245 may include a compressor driving unit in the outdoor unit, an outdoor unit fan driving unit, and an indoor unit fan driving unit.

In another example, when the home appliance is a cooking appliance, the driving unit 245 may include a microwave driving unit that outputs microwaves into the cavity.

In another example, when the home appliance is a vacuum cleaner, the driving unit 245 may include a fan motor driving unit.

Meanwhile, the communication unit 222 may include an acoustic communication unit 217 capable of acoustic communication. The acoustic communication unit 217 will be described later with reference to FIGS. 4A to 4D.

3B is another example of a simplified interior block diagram of the home appliance of FIG. 1.

Referring to the drawings, the internal block diagram of the home appliance of FIG. 3B is similar to the internal block diagram of the home appliance of FIG. 3A, but differs in that the display microcomputer 221 is further provided.

When the display unit 230 is provided in the home appliance, a display microcomputer 221 may be provided for controlling a separate display unit 230. At this time, the control unit 270 may be provided on a circuit board separate from the display microcomputer 221 as the main microcomputer.

On the other hand, as in the embodiment of the present invention, when the home appliance includes an acoustic communication unit 217 capable of acoustic communication, for the acoustic communication, the communication unit 222, particularly the acoustic communication unit 217, the display microcomputer 221 It is possible to be provided on the same circuit board. Accordingly, as shown in FIG. 3B, the control of the audio communication unit 217 is primarily performed by the display microcomputer 221, and thereafter, the control unit 270, that is, the main microcomputer 270.

4A to 4D are diagrams referenced to describe the acoustic communication of FIG. 1.

First, FIG. 4A illustrates that the mobile terminal 600 outputs sound by sound communication, and the home appliance 200 receives the sound.

To this end, the mobile terminal 600 includes an audio communication unit 617 and an audio output module 653, and the home appliance 200 may include a microphone 223 and an audio communication unit 217.

The acoustic communication unit 617 of the mobile terminal 600 receives the digital signal to be output from the control unit 680, adds predetermined data to the audio signal to be output, and modulates it. Sound is output. Then, the sound output module 653 outputs the modulated sound through an acoustic channel in an audible frequency band.

The microphone 223 of the home appliance 200 receives a sound output from the mobile terminal 600. Then, the received sound signal is transmitted to the acoustic communication unit 217. The acoustic communication unit 217 extracts data from the sound signal.

Thereby, it is possible to exchange simple information data between the mobile terminal 600 and the home appliance 200.

Meanwhile, in contrast to FIG. 4A, the home appliance 200 outputs sound by sound communication, and the mobile terminal 600 can also receive the sound, and it is also possible that two-way sound communication is performed.

Accordingly, the mobile terminal 600 may further include a microphone 623 in addition to the audio communication unit 617 and the audio output module 653, and the home appliance 200 may include a microphone 223 and an audio communication unit ( In addition to 217), an audio output module 253 may be further provided.

4B illustrates an internal block diagram of each of the acoustic communication unit 617 in the mobile terminal 600 of FIG. 4A and the acoustic communication unit 217 in the home appliance 200.

In order to output sound, the acoustic communication unit 617 in the mobile terminal 600 may include a frequency conversion unit 410, a data insertion unit 415, an inverse conversion unit 420, and a multiplexing unit 425. .

The frequency converter 410 may receive an audio signal to be output in a time domain and perform frequency conversion. In this case, the frequency conversion may be performed on a frame-by-frame basis, and specifically, it may be a frequency conversion based on a modulated complex lapped transform (MCLT).

The MCLT method has an advantage of reducing blocking artifacts at a frame boundary according to overlapping characteristics for each frame.

The data insertion unit 415 adds or inserts predetermined data to the frequency-converted audio signal. That is, data is added to the audio signal by a modulation method. Specifically, the phase coefficient among frequency-converted coefficients (magnitude, phase, etc.) may be varied to add data. For example, the variable phase coefficient may have a value of 0 degrees or 180 degrees. Thereby, the added data can be distinguished.

The inverse transform unit 420 inverse transforms an audio signal in which information data is inserted. When the MCLT conversion is performed by the frequency converter 410, an inverse MCLT conversion is performed to output a time-based sound signal.

Meanwhile, the frequency converter 410, the data inserter 415, and the inverse converter 420 may perform the corresponding operation in units of frames.

The multiplexer 425 multiplexes and outputs the inverse-converted audio signal. That is, multiple frames are multiplexed to output one continuous sound.

Then, the sound output module 253 outputs one sound multiplexed to the outside. To this end, the sound output module 253 may include at least one speaker.

The home appliance 200 receives sound through the microphone 425 and converts it into an electrical signal. In addition, the acoustic communication unit 217 in the home appliance 200 may include a synchronization unit 430, a frequency conversion unit 435, and a data extraction unit 440 for data extraction from the received sound.

The synchronization unit 430 synchronizes the received audio signal. That is, multiplexed audio signals can be synchronized and separated in units of frames.

The frequency converter 435 may receive an audio signal to be output in a time domain and perform frequency conversion. In this case, the frequency conversion may be performed on a frame-by-frame basis, and specifically, it may be a frequency conversion based on a modulated complex lapped transform (MCLT).

The data extraction unit 440 extracts the added data from the frequency-converted audio signal path. As described above, data is added by varying the phase coefficient among the coefficients for each frequency, so that data can be extracted from the phase coefficient. In particular, when the phase coefficient has a value of 0 degrees or 180 degrees, data can be extracted based on this.

The extracted data may be transmitted to the control unit 270 of the home appliance 200.

According to the MCLT-based acoustic communication method, data can be added while maintaining a sound similar to the audio signal originally intended to be output. And, according to the overlapping characteristics of each frame, there is an advantage in that blocking artifacts at a frame boundary can be reduced.

4C is a view for explaining the operation of the acoustic communication unit 617 of FIG. 4B together with an audio waveform.

Referring to the drawing, an audio signal 400 to be output is illustrated, and the audio signal 400 to be output may be divided into a plurality of frames FR1 to FR4 based on time. In the drawing, in the first frame FR1 section, the first audio signal 400a is in the second frame FR2 section, the second audio signal 400b is in the third frame FR3 section, and the third The audio signal 400c is illustrated in the fourth frame FR4, and the fourth audio signal 400d is illustrated.

The frequency converter 410 frequency-converts audio data for each frame. Specifically, frequency conversion using the MCLT method is performed.

Then, the data insertion unit 415 adds predetermined data to the frequency-converted audio signal for each frame. Specifically, the phase coefficient among the coefficients (magnitude, phase, etc.) for each frequency that is frequency-converted is varied to add data.

The inverse transform unit 420 inversely transforms an audio signal in which information data is inserted for each frame. Accordingly, a time-based sound signal is output.

In the drawing, the first to fourth sound signals 401a to 401d, which are similar to the first to fourth audio signals 400a to 400d, are illustrated for each frame. Looking at the drawing, it can be seen that the first to fourth sound signals 401a to 401d are similar to the audio signals originally intended to be output, respectively.

Meanwhile, the multiplexing unit 425 may output a single sound signal by multiplexing the sound signal to which data is added for each frame.

Meanwhile, in FIGS. 4A to 4C, it is described that a sound including predetermined information is output from the mobile terminal 600, and the home appliance 200 receives the sound and extracts the predetermined information, and vice versa It is also possible.

That is, the sound including the predetermined information is output from the home appliance 200, and the mobile terminal 600 receives the corresponding sound to extract the predetermined information.

In this case, the acoustic communication unit 217 in the home appliance 200 may include a frequency conversion unit 410, a data insertion unit 415, an inverse conversion unit 420, and a multiplexing unit 425, and a mobile terminal. The acoustic communication unit 617 in the 600 may include a synchronization unit 430, a frequency conversion unit 435, and a data extraction unit 440.

4D illustrates that the sound generating unit 618 is provided in the mobile terminal 600 and the sound detecting unit 218 is provided in the home appliance 200.

The audio generating unit 618 in the mobile terminal 600 of FIG. 4D is similar to the acoustic communication unit 617 in the mobile terminal 600 of FIG. 4B, but generates audio data by itself to generate input audio. There is a difference in output in addition to data. That is, it is possible to generate audio data without receiving audio data from the control unit 680 or the like. To this end, in the acoustic communication unit 617, a frequency conversion unit 410, a data insertion unit 415, an inverse conversion unit 420, a multiplexing unit 425, in addition to a separate audio generating unit (not shown) to be provided You can.

For example, when the sound generating unit 618 composes four consecutive sounds using 36 sounds, approximately 168,000 sounds may be generated. And data can be added to it.

Next, the acoustic detection unit 218 in the home appliance 200 is the same as the acoustic communication unit 217 in the home appliance 200 of FIG. 4B. That is, for extracting data from the received sound, a synchronization unit 430, a frequency conversion unit 435, and a data extraction unit 440 may be provided.

FIG. 5 is an internal block diagram of the server of FIG. 1.

Referring to the drawings, the server 500 may include a communication module 530, a storage unit 540, and a processor 520.

The communication module 530 may receive information related to the home appliance from the mobile terminal 600. Then, the communication module 530 may transmit result information corresponding to the received home appliance related information to the mobile terminal 600.

To this end, the communication module 530 may include an Internet module or a mobile communication module.

The storage unit 540 may store received home appliance-related information and may include data for generating result information corresponding to the home appliance-related information.

For example, when the information related to the home appliance is failure information of the home appliance, the storage unit 540 may store data for diagnosing a failure of the home appliance.

Meanwhile, as another example, when the information related to the home appliance is product information of the home appliance, the storage unit 540 may store received product information for product registration of the home appliance.

Meanwhile, as another example, when the home appliance-related information is power consumption information of the home appliance, the storage unit 540 may store power price information, on-peak time power information, off-peak time power information, real-time power information, and the like. have.

The processor 520 may control the overall operation of the server 500.

On the other hand, when receiving the home appliance-related information from the mobile terminal 600, the processor 520 may control to generate result information corresponding to the home appliance-related information. Then, the generated result information can be controlled to be transmitted to the mobile terminal 600.

For example, when the information related to the home appliance is fault information of the home appliance, the processor 520 may generate fault diagnosis result information of the home appliance using the fault diagnosis data stored in the storage unit 540. .

Meanwhile, as another example, when the home appliance-related information is product information of the home appliance, the processor 520 may generate product registration result information of the home appliance.

Meanwhile, as another example, when the home appliance-related information is power consumption information of the home appliance, the processor 520 stores power price information, on-peak time power information, and off-peak time power information stored in the storage unit 540, Real-time power information and the like can be controlled to transmit power information to the mobile terminal 600.

Meanwhile, the server 500 may be a server operated by the home appliance 200 manufacturer or a server operated by an application store operator related to the home appliance 200.

6 is an internal block diagram of the mobile terminal of FIG. 1.

Referring to FIG. 6, the mobile terminal 600 includes a wireless communication unit 610, an audio / video (A / V) input unit 620, a user input unit 630, a sensing unit 640, and an output unit 650, It may include a memory 660, an interface unit 670, a control unit 680, and a power supply unit 690.

Meanwhile, the wireless communication unit 610 may include a broadcast reception module 611, a mobile communication module 613, a wireless Internet module 615, an acoustic communication unit 617, and a GPS module 619.

The broadcast receiving module 611 may receive at least one of a broadcast signal and broadcast-related information from an external broadcast management server through a broadcast channel. At this time, the broadcast channel may include a satellite channel, a terrestrial channel, and the like.

The broadcast signal and / or broadcast-related information received through the broadcast receiving module 611 may be stored in the memory 660.

The mobile communication module 613 transmits and receives a radio signal to and from at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include a voice call signal, a video call signal, or various types of data according to transmission / reception of text / multimedia messages.

The wireless Internet module 615 refers to a module for wireless Internet access, and the wireless Internet module 615 may be built in or external to the mobile terminal 600. For example, the wireless Internet module 615 may perform WiFi-based wireless communication or WiFi Direct-based wireless communication.

The acoustic communication unit 617 may perform acoustic communication. The audio communication unit 617 may output sound by adding predetermined information data to audio data to be output in the audio communication mode. In addition, in the acoustic communication mode, the acoustic communication unit 617 may extract predetermined information data from sounds received from the outside. For detailed description of this, refer to FIGS. 4A to 4D.

In addition, Bluetooth, Bluetooth (Radio Frequency Identification), infrared communication (IrDA, infrared data association), UWB (Ultra Wideband), ZigBee, etc. may be used as the short-range communication technology.

The Global Position System (GPS) module 619 may receive location information from a plurality of GPS satellites.

The A / V (Audio / Video) input unit 620 is for inputting an audio signal or a video signal, which may include a camera 621 and a microphone 623.

The user input unit 630 generates key input data input by the user to control the operation of the terminal. To this end, the user input unit 630 may be configured with a key pad, a dome switch, a touch pad (static pressure / blackout), and the like. In particular, when the touch pad forms a mutual layer structure with the display 651, it may be referred to as a touch screen.

The sensing unit 640 detects the current state of the mobile terminal 600, such as an open / closed state of the mobile terminal 600, a location of the mobile terminal 600, or a user contact, to control the operation of the mobile terminal 600. Sensing signals can be generated.

The sensing unit 640 may include a sensing sensor 641, a pressure sensor 643, and a motion sensor 645. The motion sensor 645 may detect the movement or position of the mobile terminal 600 using an acceleration sensor, a gyro sensor, a gravity sensor, and the like. In particular, the gyro sensor is a sensor that measures angular velocity, and can detect a direction (angle) that has been turned relative to a reference direction.

The output unit 650 may include a display 651, an audio output module 653, an alarm unit 655, and a haptic module 657.

The display 651 displays and outputs information processed by the mobile terminal 600.

On the other hand, as described above, when the display 651 and the touch pad are configured as a touch screen by forming a mutual layer structure, the display 651 may be used as an input device capable of inputting information by a user's touch in addition to the output device. You can.

The sound output module 653 outputs audio data received from the wireless communication unit 610 or stored in the memory 660. The sound output module 653 may include a speaker, a buzzer, and the like.

The alarm unit 655 outputs a signal for notifying the occurrence of the event of the mobile terminal 600. For example, a signal can be output in the form of vibration. .

The haptic module 657 generates various tactile effects that the user can feel. A typical example of the tactile effect generated by the haptic module 657 is a vibration effect.

The memory 660 may store a program for processing and control of the control unit 680, and provide a function for temporarily storing input or output data (for example, a phone book, a message, a still image, a video, etc.). You can also do

The interface unit 670 serves as an interface with all external devices connected to the mobile terminal 600. The interface unit 670 may receive data from such an external device or receive power and transmit it to each component inside the mobile terminal 600, and allow data inside the mobile terminal 600 to be transmitted to the external device. .

The control unit 680 typically controls the operation of each part to control the overall operation of the mobile terminal 600. For example, it may perform related control and processing for voice calls, data communication, video calls, and the like. Also, the control unit 680 may include a multimedia playback module 681 for multimedia playback. The multimedia playback module 681 may be configured with hardware in the control unit 680 or may be configured with software separately from the control unit 680.

The power supply unit 690 receives external power and internal power under the control of the control unit 680 and supplies power required for the operation of each component.

On the other hand, the block diagram of the mobile terminal 600 shown in Figure 6 is a block diagram for an embodiment of the present invention. Each component of the block diagram may be integrated, added, or omitted depending on the specification of the mobile terminal 600 that is actually implemented. That is, two or more components may be combined into one component, or one component may be divided into two or more components as necessary. In addition, the function performed in each block is for explaining an embodiment of the present invention, the specific operation or device does not limit the scope of the present invention.

7 is a flowchart illustrating an operation method of a mobile terminal according to an embodiment of the present invention, and FIG. 8 is a flowchart illustrating an operation method of a home appliance according to an embodiment of the present invention, and FIGS. 9 to 13E are It is a figure referred to in the description of the operating method of FIG. 7 or 8.

First, referring to FIG. 7, the mobile terminal 600 displays a predetermined screen on the display (S710). Meanwhile, step 910 of FIG. 9 (S910) corresponds to step 710 of FIG. 7 (S710).

In particular, the control unit 680 of the mobile terminal controls to display the corresponding screen in response to a user input.

For example, when there is an input for viewing power information by a user input, a power information screen may be displayed. The power information screen may be a screen displayed after an application related to the home appliance is executed, but is not limited thereto, and is a screen corresponding to a case where a user directly accesses a power supply company server and views power information. It may be.

10A illustrates that the power information screen 610 is displayed on the mobile terminal 600. In the figure, it is illustrated that the power information screen 610 includes peak time information and price information at the peak time. In addition, the power information screen 610 may further include off-peak time power information, real-time power information, and the like.

Next, the mobile terminal 600 determines whether it is in the acoustic communication mode (S720). Then, if applicable, a sound including information related to a screen displayed on the mobile terminal 600 is output (S730). Meanwhile, step 920 (S920) of FIG. 9 corresponds to step 720 (S720) of FIG. 7, and step 820 (S820) of FIG. 8, and step 930 (S930) of FIG. 9 is Corresponds to step 730 (S730) and step 830 (S830) of FIG. 8.

10B illustrates a case where a specific key 601 in the mobile terminal 600 is selected. Thereby, the mobile terminal 600 can enter the acoustic communication mode. On the other hand, unlike the drawing, when the acoustic communication application in the mobile terminal 600 is selected, it is also possible to enter the acoustic communication mode.

When entering the acoustic communication mode, the mobile terminal 600 may output sound in which information data is added to the audio signal, as described in FIGS. 4A to 4D. In particular, a sound to which information related to a screen displayed on the mobile terminal 600 is added may be output.

10C illustrates that power information Spow for the washing machine 200b is transmitted from the mobile terminal 600 to the washing machine 200b by the acoustic communication mode. Accordingly, the washing machine 200b can easily and easily acquire power information through the mobile terminal 600.

Next, the home appliance 200 extracts information related to a screen displayed on the mobile terminal 600 from the received sound (S835). Then, according to the information related to the screen displayed on the extracted mobile terminal 600, the corresponding operation is executed (S840). Meanwhile, steps 935 (S935) and 940 (S940) of FIG. 9 correspond to steps 835 (S835) and 840 (S840) of FIG. 8, respectively.

When entering the acoustic communication mode, the home appliance 200 may extract information related to the screen from the received sound, as described in FIGS. 4A to 4D. Then, the control unit 270 of the home appliance 200 may control to execute the corresponding operation according to information related to the screen.

FIG. 10D illustrates that the washing machine 200b performs power saving driving based on the power information Spow received in FIG. 10C.

For example, avoiding the period from 13:00 to 15:00, which is the peak time, the washing machine 200b is driven, or a part of the washing cycle, rinsing stroke, etc., which consumes less power in the entire washing section, is partially in peak time. It is possible to drive the washing machine 200b so that it overlaps and the power consumption, stroke stroke, or drying stroke never overlaps with the peak time.

Meanwhile, in FIGS. 10A to 10D, when the power information screen is displayed on the mobile terminal 600, when the mobile terminal 600 is in an acoustic communication mode, power information Spow is transmitted to the washing machine 200b, and the washing machine It is illustrated that the power saving operation is performed in response to the received power information Spow.

On the other hand, unlike Figures 10a to 10d, embodiments of the present invention, various modifications are possible.

11A to 11E below, when a recipe-related screen is displayed on the mobile terminal 600, when the mobile terminal 600 is in an acoustic communication mode, sound including recipe information Sreci is output, and a cooking appliance It is exemplified to extract the recipe information Sreci from the sound from which the 200d is received, and perform the corresponding operation in correspondence with the extracted recipe information Sreci.

11A illustrates that an application related to the cooking appliance 200d is executed, and the application screen 1110 related to the cooking appliance 200d is displayed on the mobile terminal 600.

11B illustrates that a recipe-related screen 1120 among screens of an application related to the cooking appliance 200d is displayed on the mobile terminal 600. The drawing illustrates a recipe screen related to the 'apple tart'.

Next, FIG. 11C illustrates a case in which a specific key 601 in the mobile terminal 600 is selected while the recipe-related screen 1120 is displayed on the mobile terminal 600. Thereby, the mobile terminal 600 can enter the acoustic communication mode. On the other hand, unlike the drawing, when the acoustic communication application in the mobile terminal 600 is selected, it is also possible to enter the acoustic communication mode.

Accordingly, as illustrated in FIG. 11D, the mobile terminal 600 may output a sound including recipe information Sreci related to an 'apple tart' by an acoustic communication mode. Here, the recipe information may include cooking time information, cooking course information, recipe type information, and the like.

11E illustrates that the cooking apparatus 200d performs a cooking operation related to an apple tart based on recipe information Sreci related to the 'apple tart' extracted from the received sound. Thereby, the user can perform cooking for a desired food ('apple tart') simply without pressing a cooking start button immediately or without an operation of a cooking start button without additional setting.

12A to 12E below, when the laundry course screen is displayed on the mobile terminal 600, when the mobile terminal 600 is in an acoustic communication mode, a sound including laundry course information Scos is output, and the washing machine It is illustrated that the laundry course information Scos is extracted from the sound at which the 200b is received, and corresponding driving is performed in response to the extracted laundry course information Scos.

12A illustrates that an application related to the washing machine 200b is executed, and the application screen 1210 related to the washing machine 200b is displayed on the mobile terminal 600.

In the drawing, although the application screen 1210 related to the washing machine 200b includes firmware items and laundry course items 1215, various modifications are possible.

When the laundry course item 1215 is selected, the laundry course screen 1220 may be displayed on the mobile terminal 600 among the screens of the application related to the washing machine 200b, as shown in FIG. 12B. In the drawing, a washing course screen 1220 related to the 'course a' composed of a washing stroke 60 minutes, a rinsing stroke 20 minutes, and a dehydration stroke 10 minutes is illustrated.

Next, FIG. 12C illustrates a case in which a specific key 601 in the mobile terminal 600 is selected while the laundry course screen 1220 is displayed on the mobile terminal 600. Thereby, the mobile terminal 600 can enter the acoustic communication mode. On the other hand, unlike the drawing, when the acoustic communication application in the mobile terminal 600 is selected, it is also possible to enter the acoustic communication mode.

Accordingly, as illustrated in FIG. 12D, the mobile terminal 600 may output a sound including laundry course information Scos related to 'course a' by an acoustic communication mode. Here, the laundry course information (Scos) related to the 'course a' may include time information for a washing process 60 minutes, a rinsing process 20 minutes, a dehydration process 10 minutes, and the like.

12E illustrates that the washing machine 200b performs the laundry course operation related to the 'course a' based on the laundry course information (Scos) related to the 'course a' extracted from the received sound. Thereby, the user can perform the washing for the desired washing course (course a ') simply without pressing a start button immediately without a separate setting or without operating the start button.

13A to 13E below, when a remote control screen is displayed on the mobile terminal 600, particularly a remote control screen for the air conditioner 200c, when the mobile terminal 600 is in an acoustic communication mode, air conditioner remote control information It is illustrated that (Son) is output, and the air conditioner remote control information (Son) is extracted from the sound from which the air conditioner 200c is received, and corresponding operation is performed in response to the extracted air conditioner remote control information (Son).

13A illustrates that the remote control application for the home appliance 200 is executed, and the remote control screen 1210 for the home appliance 200 is displayed on the mobile terminal 600.

In this case, the remote control screen may include a plurality of home appliance items so that a plurality of home appliances can be remotely controlled. In the drawing, as an example, the robot cleaner item 1312 and the air conditioner item 1315 are illustrated, but various examples are possible.

If an air conditioner item 1315 is selected, a remote control screen 1320 for the air conditioner 200c may be displayed on the mobile terminal 600, as shown in FIG. 13B. In the drawing, a remote control screen 1320 for the air conditioner 200c including a power item 1325 and a temperature adjustment item is illustrated, but other wind direction adjustment items, dehumidifying operation items, and operation time adjustment items are also displayed. Can be.

Meanwhile, FIG. 13B illustrates that the power item 1325 is selected or focused.

Next, in FIG. 13C, the remote control screen 1320 for the air conditioner 200c is displayed on the mobile terminal 600, and among them, the power item 1325 is selected or focused, and the mobile terminal 600 is specified. The case where the key 601 is selected is illustrated. Thereby, the mobile terminal 600 can enter the acoustic communication mode. On the other hand, unlike the drawing, when the acoustic communication application in the mobile terminal 600 is selected, it is also possible to enter the acoustic communication mode.

Accordingly, as illustrated in FIG. 13C, the mobile terminal 600 may output a sound including information (Son) related to power-on by an acoustic communication mode.

13D illustrates that the air conditioner 200c is powered on based on information (Son) related to power on extracted from the received sound. Thereby, the air conditioner 200c can be remotely controlled through the received sound without a separate wireless communication module. This remote control is applicable to all home appliances other than the air conditioner 200c.

Meanwhile, it is also possible to perform a firmware upgrade for the home appliance.

Meanwhile, although not illustrated in FIGS. 10A to 13D, when the screen displayed on the mobile terminal 600 is an access point device list screen, the mobile terminal 600 is a network for an access point device according to an acoustic communication mode. A sound including information can be output.

Accordingly, when the home appliance 200 is provided with a communication module such as WiFi, the home appliance 200 simply wirelessly connects to an adjacent access point (AP) device using the received network information. It is possible to increase the user's ease of use.

14 is a flowchart illustrating a method of operating a mobile terminal according to another embodiment of the present invention, and FIG. 15 is a flowchart illustrating a method of operating a home appliance according to another embodiment of the present invention, FIGS. 16A to 22C It is a figure referred to in the description of the operation method of FIG. 14 or 15.

First, referring to FIG. 15, the home appliance 200 determines whether the power of the home appliance is on (S1500). Then, if applicable, the home appliance-related information is stored in the memory (S1501).

When the power of the home appliance 200 is turned on, the control unit 200 of the home appliance includes related information of the home appliance, for example, operation status information, operation history information, product information, failure information, power consumption information, Stored article information or recipe information may be stored. In particular, it can be stored in the memory 260 in the home appliance 200.

For example, when the washing machine 200b is in the washing operation, the washing machine driving time information, washing machine driving course information, and washing machine failure information may be stored in the memory 260 in the washing machine 200b.

Meanwhile, when the home appliance 200 operates, information related to the home appliance may be periodically stored in the memory 260.

Alternatively, as another example, the home appliance-related information may be stored in the memory 260 at a first cycle interval, and then thereafter, at a second cycle interval faster than the first cycle. ).

Next, referring to FIG. 14, the mobile terminal 600 displays a predetermined screen according to a user input (S1420). Here, various examples, such as a web screen and an application screen, are possible according to a user input.

16A illustrates that the monitoring screen 1620 is displayed on the mobile terminal 600. The monitoring screen 1620 may be a screen displayed when a monitoring item is selected among execution screens of a home appliance-related application.

Next, the mobile terminal 600 determines whether the home appliance is an acoustic communication mode (S1440). Then, if applicable, a sound including predetermined information is received from the home appliance (S1445). Then, the mobile terminal 600 extracts predetermined information from the received sound, and in particular, extracts information related to a screen displayed on the mobile terminal 600 (S1447). Then, the mobile terminal 600 displays the extracted information (S1450).

Here, step 1540 (S1540) of FIG. 15 corresponds to step 1440 (S1440) of FIG. 14, and step 1545 (S1545) of FIG. 15 corresponds to step 1445 (S1445) of FIG. 14.
Meanwhile, step 1440 (S1440) of FIG. 14 may be omitted. That is, after the screen display step S1420, if the sound reception step S1445 is performed, it is also possible to automatically enter the acoustic communication mode.

16A illustrates that the washing machine 200b is operating and the monitoring screen 1620 is displayed on the mobile terminal 600. The washing machine 200b may periodically store operation state information in a memory during operation.

Meanwhile, as illustrated in FIG. 16A, when a specific key 201b in the washing machine 200b is selected, the washing machine 200b may enter an acoustic communication mode.

Accordingly, as shown in FIG. 16B, the washing machine 200b may output sound including predetermined information through an acoustic communication mode. At this time, the information included in the sound may include product information, operation status information, and the like of the washing machine 200b. In the drawing, for convenience, it is exemplified that the sound including the operation state information Sstp is output.

The mobile terminal 600 may receive sound from the washing machine 200b and extract predetermined information from the received sound. In particular, operation state information related to the monitoring screen 1620 displayed on the mobile terminal 600 may be extracted. Accordingly, the mobile terminal 600 can easily obtain information related to the displayed screen from the washing machine 200b.

Meanwhile, the mobile terminal 600 can display information received from the home appliance.

16C illustrates that the operation state information Sstp of the washing machine 200b is displayed on the mobile terminal 600.

In the drawing, the information 1630 indicating the current dehydration, the remaining time information 1635, and the total progress time information 1637 are illustrated on the mobile terminal 600, but various modifications are possible.

17A to 17D below, when the diagnostic screen is displayed on the mobile terminal 600, when the washing machine 200b enters the acoustic communication mode, a sound including failure information Sdia of the washing machine 200b is generated. The output, and the mobile terminal 600 extracts the failure information (Sdia) of the washing machine 200b from the sound received, and using the extracted failure information (Sdia), connected to the server 500, the failure diagnosis result information Illustrates receiving.

17A illustrates that the diagnostic screen 1720 is displayed on the mobile terminal 600.

Meanwhile, as illustrated in FIG. 17A, when a specific key 201b in the washing machine 200b is selected, the washing machine 200b may enter an acoustic communication mode.

Accordingly, as shown in FIG. 17B, the washing machine 200b may output sound including predetermined information through an acoustic communication mode. At this time, the information included in the sound may include product information of the washing machine 200b, operation state information, and fault diagnosis result information. In the drawing, for convenience, it is exemplified that a sound including the fault diagnosis result information Sres is output.

The mobile terminal 600 may receive sound from the washing machine 200b and extract predetermined information from the received sound. In particular, failure diagnosis result information Sres related to the diagnosis screen 1720 displayed on the mobile terminal 600 may be extracted. Accordingly, the mobile terminal 600 can easily obtain information related to the displayed screen from the washing machine 200b.

17C illustrates that the failure result information 1730 displayed on the display is based on the result of the failure diagnosis result Sres of the extracted washing machine.

In the drawing, as the washing machine failure result information 1730, the phrase 'water outlet is frozen' is exemplified. As a result, the user can easily identify the cause of the washing machine failure, and the user's usability can be increased.

Meanwhile, the mobile terminal 600 may further display the AS application item 1735 in addition to the washing machine failure result information 1730.

When the AS application item 1735 is selected by the user input in FIG. 17C, as in FIG. 17D, the AS application screen 1740 may be displayed. As a result, the user can immediately perform an AS application, etc., in addition to the failure result information, so that the user's usability can be increased.

18A to 18C below, when the monitoring screen is displayed on the mobile terminal 600, when the refrigerator 200a enters the acoustic communication mode, sound including operation state information Sst of the refrigerator 200a It is illustrated that the output, the mobile terminal 600 extracts the operation status information (Sst) of the refrigerator 200a from the sound received, and displays the extracted operation status information (Sst).

18A illustrates that the monitoring screen 1820 related to the refrigerator 200a is displayed.

Meanwhile, as illustrated in FIG. 18A, when a specific key 201a in the refrigerator 200a is selected, the refrigerator 200a may enter an acoustic communication mode.

Accordingly, as shown in FIG. 18B, the refrigerator 200a may output sound including predetermined information through an acoustic communication mode. At this time, the information included in the sound may include product information, operation status information, and the like of the refrigerator 200a. In the drawing, for convenience, it is exemplified that the sound including the operation state information Sst is output.

The mobile terminal 600 may receive sound from the refrigerator 200a and extract predetermined information from the received sound. In particular, the operation state information Sst related to the monitoring screen 1820 displayed on the mobile terminal 600 may be extracted. Accordingly, the mobile terminal 600 can easily obtain information related to the displayed screen from the refrigerator 200a.

18C illustrates an operation status information screen 1830 of the refrigerator 200a based on the extracted operation status information Sst. Here, the operation state information screen 1830 of the refrigerator 200a is illustrated as including freezer temperature information, refrigerator temperature information, limited express refrigeration operation information, smart power saving operation information, door opening / closing frequency information, and power usage information. Deformation is possible. Through the operation state information screen 1830 of the refrigerator 200a, the user can easily grasp the operation state of the refrigerator 200a.

19A to 19C below, when the magic space screen related to food management is displayed on the mobile terminal 600, when the refrigerator 200a enters the acoustic communication mode, food storage information stored in the refrigerator 200a The sound including (Sfood) is output, and the mobile terminal 600 extracts the food storage information (Sfood) stored in the refrigerator 200a from the received sound, and displays the extracted food storage information (Sfood). do.

19A illustrates that the magic space screen 1920 related to the refrigerator 200a is displayed.

Meanwhile, as illustrated in FIG. 19A, when a specific key 201a in the refrigerator 200a is selected, the refrigerator 200a may enter an acoustic communication mode.

Accordingly, as illustrated in FIG. 19B, the refrigerator 200a may output sound including predetermined information through an acoustic communication mode. At this time, the information included in the sound may include product information of the refrigerator 200a, operation status information, food storage information stored in the refrigerator 200a (Sfood), and the like. In the drawing, for convenience, it is illustrated that sound including food storage information (Sfood) stored in the refrigerator 200a is output.

The mobile terminal 600 may receive sound from the refrigerator 200a and extract predetermined information from the received sound. In particular, the food storage information (Sfood) stored in the refrigerator 200a associated with the magic space screen 1920 displayed on the mobile terminal 600 may be extracted. Accordingly, the mobile terminal 600 can easily obtain information related to the displayed screen from the refrigerator 200a.

19C illustrates the magic space screen 1930 of the refrigerator 200a based on the extracted food storage information (Sfood). Here, the magic space screen 1930 is a screen modeling a case in which the door of the refrigerator is opened, and may be displayed using information on the type of food stored in the refrigerator, storage location information, and the like. In addition, it is possible to further display food expiration date information, food arrival date information, and the like. Through the magic space screen 1930 representing food storage information (Sfood) stored in the refrigerator 200a, the user can easily grasp food information stored in the refrigerator 200a.

20A to 20C below, when a recipe-related screen is displayed on the mobile terminal 600, when the refrigerator 200a enters an acoustic communication mode, recipe information based on food storage stored in the refrigerator 200a (Sreci) ) Is output, the mobile terminal 600 extracts the recipe information (Sreci) from the received sound, and transmits the extracted recipe information (Sreci) to the server 500, from the server 500, It is exemplified to receive the content (Svod) related to the recipe.

20A illustrates that a recipe-related screen 2020 associated with the refrigerator 200a is displayed.

Meanwhile, as illustrated in FIG. 20A, when a specific key 201a in the refrigerator 200a is selected, the refrigerator 200a may enter an acoustic communication mode.

Accordingly, as shown in FIG. 20, the refrigerator 200a may output sound including predetermined information through an acoustic communication mode. In this case, the information included in the sound may include product information of the refrigerator 200a, operation status information, recipe information (Sreci), and the like. In the drawing, for convenience, it is illustrated that a sound including recipe information (Sreci) is output.

Here, the recipe information Sreci may be recipe information generated by the refrigerator 200a itself based on the food information stored in the refrigerator.

The mobile terminal 600 may receive sound from the refrigerator 200a and extract predetermined information from the received sound. In particular, recipe information Sreci related to the recipe-related screen 2020 displayed on the mobile terminal 600 may be extracted. Accordingly, the mobile terminal 600 can easily obtain information related to the displayed screen from the refrigerator 200a.

Then, the mobile terminal 600 may transmit the extracted recipe information Sreci to the server 500 and receive content Svod related to the recipe from the server 500, as shown in FIG. 20C. Here, the received content (Svod) may be a video.

The mobile terminal 600 may reproduce and display the received content Svod. Accordingly, when the user prepares food based on the food information stored in the refrigerator 200a, the user can conveniently watch the content for the recipe, and in the kitchen, can directly refer to the recipe while cooking. Therefore, user convenience can be increased.

21A to 21C below, the sound including the operation state information Sst of the cleaner 200e when the cleaner 200e enters the acoustic communication mode while the monitoring screen is displayed on the mobile terminal 600 It is illustrated that the output, the mobile terminal 600 extracts the operation state information (Sst) of the cleaner 200e from the sound received, and displays the extracted operation state information (Sst).

21A illustrates that the monitoring screen 2120 related to the cleaner 200e is displayed on the mobile terminal 600.

Meanwhile, as illustrated in FIG. 21A, when a specific key 201e in the cleaner 200e is selected, the cleaner 200e may enter an acoustic communication mode.

Accordingly, as shown in FIG. 21B, the cleaner 200e may output sound including predetermined information through an acoustic communication mode. At this time, the information included in the sound may include product information of the cleaner 200e, operation state information, and fault diagnosis result information. In the drawing, for convenience, it is exemplified that the sound including the operation state information Sst is output.

The mobile terminal 600 may receive sound from the cleaner 200e and extract predetermined information from the received sound. In particular, it is possible to extract the operation status information Sst associated with the monitoring screen 2120 displayed on the mobile terminal 600. Accordingly, the mobile terminal 600 can easily obtain information related to the displayed screen from the cleaner 200e.

21C illustrates an operation status information screen 2130 of the cleaner 200e based on the extracted operation status information Sst. Here, although the operation state information screen 2130 of the cleaner 200e includes examples of the moving path information of the cleaner, a patrol stop item, a charging item, a cleaning item, and a patrol item of the cleaner, various modifications are possible. Through the operation state information screen 2130 of the cleaner 200e, the user can easily grasp the operation state of the cleaner 200e.

22A to 22C below, when the monitoring screen is displayed on the mobile terminal 600, when the air conditioner 200c enters the acoustic communication mode, sound including operation state information Sst of the air conditioner 200c It is illustrated that the output, the mobile terminal 600 extracts the operation status information (Sst) of the air conditioner (200c) from the sound received, and displays the extracted operation status information (Sst).

22A illustrates that the air conditioner 200c receives the remote control signal Sre from the remote control device 210c for remotely controlling the air conditioner 200c.
To this end, the air conditioner 200c may include a remote control signal receiver (not shown). The remote control signal receiving unit (not shown) may be provided in the communication unit 222 of FIG. 3A or 3B.
The air conditioner 200c, which has received the remote control signal Sre, may perform a corresponding operation. For example, a power-on operation by a power-on signal, a temperature setting by a temperature setting signal, and the like can be performed. Accordingly, the air conditioner 200c may perform various operations such as indoor cooling and indoor dehumidification.

Meanwhile, as illustrated in FIG. 22A, when a specific key 201c in the remote control device 210c for remotely controlling the air conditioner 200c is selected, the air conditioner 200c may enter the acoustic communication mode.

Accordingly, as shown in FIG. 22B, the air conditioner 200c may output sound including predetermined information through an acoustic communication mode. In this case, the information included in the sound may include product information of the air conditioner 200c, operation status information, and fault diagnosis result information. In the drawing, for convenience, it is exemplified that the sound including the operation state information Sst is output.

The mobile terminal 600 may receive sound from the air conditioner 200c and extract predetermined information from the received sound. At this time, if the monitoring screen 2220 is displayed on the mobile terminal 600, the operation status information Sst related to the monitoring screen 2220 displayed on the mobile terminal 600 may be extracted from various received information. have. Accordingly, the mobile terminal 600 can easily obtain information related to the displayed screen from the air conditioner 200c.

22C illustrates the operation status information screen 2230 of the air conditioner 200c based on the extracted operation status information Sst. Here, the operation state information screen 2230 of the air conditioner 200c is illustrated as including indoor temperature information, target temperature information, driving time information, smart power saving driving information, robot cleaning frequency information, and power usage information, but various modifications This is possible. Through the operation state information screen 2230 of the air conditioner 200c, the user can easily grasp the operation state of the air conditioner 200c.
In particular, it is possible to simply check the operation state of the air conditioner 200c remotely controlled by the remote control device 210c through the mobile terminal 600.
On the other hand, the control of the air conditioner 200c by the remote control device 210c, and the remote control result through the mobile terminal 600 of FIGS. 22A to 22C, in addition to the air conditioner, is applicable to other home appliances.

The mobile terminal, the home appliance, and the operation method according to the present invention are not limited to the configuration and method of the above-described embodiments, and the above embodiments are all of the embodiments so that various modifications can be made. Alternatively, a part may be selectively combined and configured.

On the other hand, the operation method of the mobile terminal or the home appliance of the present invention can be implemented as a code readable by the processor on a recording medium readable by a processor provided in the mobile terminal or the home appliance. The processor-readable recording medium includes all types of recording devices in which data that can be read by the processor are stored. Examples of the recording medium that can be read by the processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., and include those implemented in the form of carrier waves such as transmission through the Internet. . In addition, the processor-readable recording medium can be distributed over network coupled computer systems so that the processor-readable code is stored and executed in a distributed fashion.

In addition, although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. In addition, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical idea or prospect of the present invention.

Claims (26)

Displaying a predetermined screen on the display;
Entering an acoustic communication mode; And
And outputting a sound including information related to the screen to a home appliance according to the acoustic communication mode.
When the screen displayed on the display is an access point device list screen, the sound output step outputs sound including network information about the access point device,
When receiving a second sound including product information from the home appliance, extracting product information from the second sound, and transmitting the product information to a server. According to claim 1,
The sound output step,
Frequency converting an audio signal to be output;
Inserting information data related to the screen into the frequency-converted audio signal;
Inversely transforming the audio signal in which the information data is inserted; And
And outputting the sound based on the inverse-converted audio signal. According to claim 2,
Multiplexing the inverse transformed audio signal; further comprising,
The sound output step,
And outputting the sound based on the multiplexed audio signal. According to claim 2,
The frequency conversion step,
A method of operating a mobile terminal characterized in that frequency conversion of the audio signal is based on a modulated complex lapped transform (MCLT). According to claim 2,
The frequency conversion step,
Inserting information data related to the screen,
A method of operating a mobile terminal, characterized in that it is performed by a phase variable for the frequency-converted audio signal. According to claim 1,
When the screen is any one of the power information screen related to the home appliance, a recipe related screen, a driving course screen related to the home appliance, a firmware screen related to the appliance, a remote control screen, or the access point device list screen,
The information related to the screen may include one of power information related to the home appliance, recipe information, driving course information related to the home appliance, firmware information related to the home appliance, remote control information, or network information about the access point device. Method of operating a mobile terminal comprising a. In the operating method of the home appliance including a wireless communication module,
Receiving a sound including information related to a screen displayed on the mobile terminal according to the acoustic communication mode;
Extracting information related to the screen from the sound; And
Includes; based on the extracted information, executing the corresponding operation;
When receiving a sound including network information for an access point device from the mobile terminal, the network information for the access point device is extracted from the sound,
A method of operating a home appliance using the network information, wirelessly accessing the access point device through the wireless communication module. The method of claim 7,
The extraction step,
Synchronizing the received sound;
Frequency converting the synchronized sound signal;
And extracting information related to the screen from the frequency-converted sound signal. The method of claim 8,
The frequency conversion step,
A method of operating a home appliance, characterized in that frequency conversion of the sound signal based on a modulated complex lapped transform (MCLT). The method of claim 7,
The execution step,
When the information related to the screen is power information related to the home appliance, power saving operation for the home appliance is performed based on the power information related to the home appliance,
When the information related to the screen is driving course information related to the home appliance, the home appliance is driven based on the driving course information,
If the information related to the screen is firmware information related to the home appliance, based on the firmware information, the firmware of the home appliance is installed,
When the information related to the screen is remote control information related to the home appliance, a method of operating the home appliance, wherein the home appliance is operated based on the remote control information. delete delete delete delete delete delete In the operating method of the home appliance including a wireless communication module,
When in operation, storing information related to the home appliance;
Entering an acoustic communication mode; And
And outputting a sound including the stored information according to the acoustic communication mode.
When receiving a sound including network information for an access point device from a mobile terminal, the network information for the access point device is extracted from the sound,
A method of operating a home appliance using the network information, wirelessly accessing the access point device through the wireless communication module. The method of claim 17,
The storage information,
A method of operating a home appliance, comprising operation status information, operation history information, product information, failure information, power consumption information, stored article information, or recipe information associated with the home appliance. The method of claim 17,
Receiving a remote control signal from the remote control device; And
Further comprising; performing an operation corresponding to the remote control signal;
The sound output step,
According to the acoustic communication mode, the operation method of the home appliance, characterized in that for outputting the sound including the stored operation status information according to the operation performed. A display displaying a predetermined screen;
A sound output module that outputs sound to the outside;
A microphone that receives sound from the outside;
An acoustic communication unit that outputs a sound including information related to the screen to the sound output module or extracts predetermined information from the sound received from the microphone according to the sound communication mode; And
Includes a control unit for controlling to display the information extracted from the acoustic communication unit on the display,
The control unit,
When the screen displayed on the display is an access point device list screen, the sound output step controls to output a sound including network information about the access point device to a home appliance,
When receiving a second sound including product information from the home appliance, the mobile terminal characterized in that it extracts product information from the second sound, and controls to transmit the product information to a server. The method of claim 20,
The acoustic communication unit,
A frequency converter for frequency converting the audio signal to be output;
A data inserting unit inserting information data related to the screen into the frequency-converted audio signal;
An inverse transform unit to inversely transform the audio signal into which the information data is inserted; And
And a multiplexer configured to multiplex the inverse transformed audio signal. The method of claim 20,
The microphone,
A mobile terminal receiving a sound including operation status information of the home appliance from the home appliance operated by a remote control device. Wireless communication module;
Drive unit;
A sound output module that outputs sound to the outside;
A microphone that receives sound from the outside;
A memory for storing information related to the home appliance when the home appliance is in operation;
An acoustic communication unit for outputting sound including information related to the home appliance to the sound output module or extracting predetermined information from the sound received from the microphone according to the sound communication mode; And
Includes a control unit for controlling the driving unit, according to the information extracted from the acoustic communication unit,
The control unit,
When receiving a sound including network information for an access point device from a mobile terminal, the network information for the access point device is extracted from the sound,
And using the network information, wirelessly connects to the access point device through the wireless communication module. The method of claim 23,
The acoustic communication unit,
A synchronization unit synchronizing the received sound;
A frequency converter for frequency converting the synchronized sound signal;
And a data extraction unit for extracting information related to a screen from the frequency-converted sound signal. The method of claim 23,
The storage information,
A home appliance comprising operation status information, operation history information, product information, failure information, power consumption information, stored article information, or recipe information related to the home appliance. The method of claim 23,
Further comprising a remote control signal receiving unit for receiving a remote control signal from the remote control device,
The control unit,
Home appliance characterized in that it controls to perform an operation corresponding to the remote control signal, and outputs a sound including stored operation status information according to the performance of the operation according to the sound communication mode during the sound output step. .

Images