KR102192387B1 - Mobile terminal and method for operating the same - Google Patents

Abstract

The present invention relates to a mobile terminal and a method of operation thereof. A method of operating a mobile terminal according to an embodiment of the present invention includes receiving a sound including product information of a home appliance, extracting product information of the home appliance from the sound, and transmitting the extracted product information to a server. It includes the step of. Accordingly, product registration of the home appliance can be performed easily.

Description

Mobile terminal and method for operating the same

The present invention relates to a mobile terminal and a method of operation thereof, and more particularly, to a mobile terminal capable of easily registering a product of a home appliance, and an operation method thereof.

Among the home appliances placed in the building, the refrigerator stores food for the user, the laundry treatment machine handles laundry, the air conditioner adjusts the room temperature, and the cooking appliance cooks food. .

On the other hand, as various communication methods are developed, various methods for increasing user convenience for a home appliance through communication are being studied.

An object of the present invention is to provide a mobile terminal capable of easily registering a product of a home appliance and a method of operation thereof.

In order to achieve the above object, a method of operating a mobile terminal according to an embodiment of the present invention includes receiving a sound including product information of a home appliance, extracting product information of a home appliance from the sound, and the extracted And transmitting product information to the server.

In addition, a mobile terminal according to an embodiment of the present invention for achieving the above object includes a display, a communication module for exchanging data with a server, a microphone for receiving sound including product information of a home appliance, and a home from sound. It includes an acoustic communication unit for extracting product information of the appliance, and a control unit for controlling to transmit the extracted product information to the server.

According to an embodiment of the present invention, the mobile terminal receives a sound including product information of the home appliance from the home appliance, extracts product information from the received sound, and sends the extracted product information to the server by an acoustic communication method. By sending to, it becomes possible to easily perform product registration of the home appliance. Accordingly, user convenience may be increased.

Meanwhile, the mobile terminal receives the product registration result information from the server and displays it, so that the user can easily recognize the completion of product registration.

In particular, when a sound is received while the product registration execution screen is displayed, the mobile terminal may extract product information from among various pieces of information added to the sound. Accordingly, product registration can be easily performed.

Meanwhile, in order to transmit product information to the server, by displaying a list of adjacent access point devices and selecting any one of the access point devices from the list, transmission to the server can be performed easily and quickly.

Meanwhile, according to an embodiment of the present invention, when the home appliance stores product information and enters the acoustic communication mode, a separate communication module is provided by outputting a sound including product information according to the acoustic communication mode. Without it, product information can be output to the outside.

1 is an example of a configuration diagram of a communication system according to an embodiment of the present invention.
2A to 2E are diagrams for explaining various examples of acoustic communication between a mobile terminal and a home appliance.
3A is an example of a simplified internal block diagram of the home appliance of FIG. 1.
3B is another example of a simplified internal block diagram of the home appliance of FIG. 1.
4A to 4D are views referenced for describing the acoustic communication of FIG. 1.
5 is an internal block diagram of the server of FIG. 1.
6 is an internal block diagram of the mobile terminal of FIG. 1.
7 is a flowchart illustrating a method of operating a mobile terminal according to an embodiment of the present invention.
8 is a flow chart illustrating a method of operating a home appliance according to an embodiment of the present invention.
9 to 13D are views referenced for explanation of the operation method of FIG. 7 or 8.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

The suffixes "module" and "unit" for the constituent elements used in the following description are given in consideration of only the ease of writing in the present specification, and do not impart a particularly important meaning or role by themselves. Therefore, the "module" and "unit" may be used interchangeably with each other.

1 is an example of a configuration diagram of a communication system according to an embodiment of the present invention.

Referring to the drawings, the communication system 10 according to an embodiment of the present invention may include a home appliance 200, a mobile terminal 600, and a server 500.

In particular, in relation to the embodiment of the present invention, the home appliance 200 and the mobile terminal 600 may perform acoustic communication or sound communication.

Acoustic communication according to an embodiment of the present invention means that a sound in an audible frequency band is output, but predetermined information data is added to the sound. To add such information data, signal modulation is required, and in the embodiment of the present invention, modulation is performed based on a modulated complex lapped transform (MCLT).

Specifically, the home appliance 200 may output a sound including product information of the home appliance through the sound output module 253 by an acoustic communication method. In addition, the mobile terminal 600 may receive the sound through the microphone 623 and extract product information of the home appliance in the sound.

In addition, the mobile terminal 600 may transmit product information of the home appliance to the server 500 for product registration. Accordingly, product registration of the home appliance can be performed easily. Accordingly, user convenience can be increased.

Meanwhile, the mobile terminal 600 receives product registration result information from the server 500 and displays it, so that a user can easily recognize that product registration is complete.

Meanwhile, the product information may be a concept including product type information (refrigerator, washing machine, etc.), product manufacturing date information, product serial number, type/version information of firmware installed on the product, and the like of the home appliance.

According to this acoustic communication method, since the mobile terminal 600 and the home appliance 200 only need to have an audio output module and a microphone, respectively, information can be easily exchanged. In particular, since the home appliance 200 does not need to have a separate wireless communication module such as WiFi, manufacturing cost can be reduced.

Meanwhile, the home appliance 200 is an electric device for a user, for example, a refrigerator 200a, a washing machine 200b, an air conditioner 200c, a cooking appliance 200d, a vacuum cleaner 200e, etc. This can be illustrated. In addition, it may be a concept including a TV or the like.

On the other hand, a mobile terminal 600, a mobile phone, a smart phone, a notebook, a tablet PC, etc. are possible.

2A to 2E are diagrams illustrating various examples of acoustic communication between a mobile terminal and a home appliance.

2A illustrates that acoustic communication is performed between the mobile terminal 600 and the refrigerator 200a, and FIG. 2B illustrates that the acoustic communication is performed between the mobile terminal 600 and the washing machine 200b, FIG. 2C illustrates that acoustic communication is performed between the mobile terminal 600 and the air conditioner 200c, and FIG. 2D illustrates that the acoustic communication is performed between the mobile terminal 600 and the cooking appliance 200d. For example, FIG. 2E illustrates that acoustic communication is performed between the mobile terminal 600 and the cleaner 200e.

According to FIGS. 2A to 2E, acoustic communication by sound output from a mobile terminal, communication by sound output from a home appliance, or a combination thereof are possible, respectively.

3A is an example of a simplified internal block diagram of the home appliance of FIG. 1.

Referring to the drawings, the home appliance 200 includes an input unit 220 for user input, a display unit 230 for displaying an operation state of the home appliance, a communication unit 222 for communication with other external devices, and a home appliance. It may include a driving unit 245 for driving and a control unit 270 for internal control.

For example, when the home appliance is a refrigerator, the driving unit 245 may include a refrigerator compartment driving unit and a freezing compartment driving unit.

As another example, when the home appliance is a washing machine, the driving unit 245 may be a driving unit that drives a drum or a tub.

As another example, when the home appliance is an air conditioner, the driving unit 245 may include a compressor driving unit in the outdoor unit, an outdoor unit fan driving unit, and an indoor unit fan driving unit.

As another example, when the home appliance is a cooking appliance, the driving unit 245 may include a microwave driving unit that outputs microwaves into the cavity.

As another example, when the home appliance is a vacuum cleaner, the driving unit 245 may include a fan motor driving unit.

Meanwhile, the communication unit 222 may include an acoustic communication unit 217 capable of acoustic communication. The acoustic communication unit 217 will be described later with reference to FIGS. 4A to 4D.

3B is another example of a simplified internal block diagram of the home appliance of FIG. 1.

Referring to the drawings, the internal block diagram of the home appliance of FIG. 3B is similar to the internal block diagram of the home appliance of FIG. 3A, but the difference is that the display microcomputer 221 is further provided.

When the display unit 230 is provided in the home appliance, a display microcomputer 221 may be provided to control the separate display unit 230. In this case, the control unit 270 may be provided on a circuit board separate from the display micom 221 as a main micom.

On the other hand, as in the embodiment of the present invention, when the home appliance includes an acoustic communication unit 217 capable of acoustic communication, for acoustic communication, the communication unit 222, in particular, the acoustic communication unit 217 is a display microcomputer 221 It is possible to be provided on the same circuit board as ). Accordingly, as shown in FIG. 3B, control of the sound communication unit 217 is primarily performed by the display micom 221, and thereafter, the control unit 270, that is, the main micom 270 may be in charge.

4A to 4D are views referenced for describing the acoustic communication of FIG. 1.

First, FIG. 4A illustrates that the mobile terminal 600 outputs sound through acoustic communication and the home appliance 200 receives the sound.

To this end, the mobile terminal 600 may include an acoustic communication unit 617 and an audio output module 653, and the home appliance 200 may include a microphone 223 and an acoustic communication unit 217.

The sound communication unit 617 of the mobile terminal 600 receives the digital sound to be output from the control unit 680, adds predetermined data to the digital sound to be output, and modulates Sound is output. Then, the sound output module 653 outputs the modulated sound through an audible frequency band, an acoustic channel.

The microphone 223 of the home appliance 200 receives a sound output from the mobile terminal 600. Then, the received sound signal is transmitted to the sound communication unit 217. The acoustic communication unit 217 extracts data from the sound signal.

This enables simple information data exchange between the mobile terminal 600 and the home appliance 200.

Meanwhile, opposite to FIG. 4A, the home appliance 200 may output sound through acoustic communication, the mobile terminal 600 may receive the sound, and interactive acoustic communication may be performed.

Accordingly, the mobile terminal 600 may further include a microphone 623 in addition to the sound communication unit 617 and the sound output module 653, and the home appliance 200 includes the microphone 223 and the sound communication unit ( In addition to the 217), a sound output module 253 may be further provided.

FIG. 4B illustrates an internal block diagram of an acoustic communication unit 617 in the mobile terminal 600 of FIG. 4A and an acoustic communication unit 217 in the home appliance 200, respectively.

In order to output sound, the acoustic communication unit 617 in the mobile terminal 600 may include a frequency conversion unit 410, a data insertion unit 415, an inverse conversion unit 420, and a multiplexing unit 425. .

The frequency converter 410 may receive an audio signal to be outputted in a time domain and perform frequency conversion. The frequency transformation at this time may be performed on a frame basis, and specifically, may be a frequency transformation based on a modulated complex lapped transform (MCLT).

The MCLT method has the advantage of reducing blocking artifacts at frame boundaries according to the overlapping characteristics for each frame.

The data insertion unit 415 adds or inserts predetermined data (data) to the frequency-converted audio signal. That is, data is added to the audio signal by the modulation method. Specifically, data may be added by varying a phase coefficient among coefficients (magnitude, phase, etc.) for each frequency that has been converted into a frequency. For example, the variable phase coefficient may have a value of 0 degrees or 180 degrees. Thereby, the added data can be distinguished.

The inverse transform unit 420 performs inverse transform on the audio signal into which the information data is inserted. When the frequency converter 410 performs MCLT transformation, inverse MCLT transformation is performed to output a time-based sound signal.

Meanwhile, the frequency conversion unit 410, the data insertion unit 415, and the inverse conversion unit 420 may perform a corresponding operation in units of frames.

The multiplexer 425 multiplexes and outputs the inversely transformed audio signal. That is, a plurality of frames are multiplexed to output one continuous sound.

Then, the sound output module 253 outputs one multiplexed sound to the outside. To this end, the sound output module 253 may include at least one speaker.

The home appliance 200 receives sound through the microphone 425 and converts it into an electrical signal. In addition, the acoustic communication unit 217 in the home appliance 200 may include a synchronization unit 430, a frequency conversion unit 435, and a data extraction unit 440 to extract data from received sound.

The synchronization unit 430 synchronizes the received audio signal. That is, the multiplexed audio signals can be synchronized and separated in units of frames.

The frequency converter 435 may receive an audio signal to be output in a time domain and perform frequency conversion. The frequency transformation at this time may be performed on a frame basis, and specifically, may be a frequency transformation based on a modulated complex lapped transform (MCLT).

The data extraction unit 440 extracts added data from the frequency-converted audio signal. As described above, since data is added by varying a phase coefficient among coefficients for each frequency, data can be extracted from the phase coefficient. In particular, when the phase coefficient has a value of 0 degrees or 180 degrees, data may be extracted based on this.

The extracted data may be transmitted to the controller 270 of the home appliance 200.

According to this MCLT-based acoustic communication method, data can be added while maintaining a sound similar to an audio signal originally intended to be output. And, according to the overlapping characteristics of each frame, there is an advantage that it is possible to reduce the blocking artifact (blocking artifact) at the frame boundary.

4C is a diagram illustrating the operation of the sound communication unit 617 of FIG. 4B together with audio waveforms.

Referring to the drawing, an audio signal 400 to be output is illustrated, and the audio signal 400 to be output may be divided into a plurality of frames FR1 to FR4 based on time. In the figure, in a first frame (FR1) section, a first audio signal (400a), a second frame (FR2) section, a second audio signal (400b), a third frame (FR3) section, a third The audio signal 400c is exemplified in the fourth frame FR4 section, the fourth audio signal 400d.

The frequency converter 410 frequency-converts the audio data for each frame. Specifically, frequency conversion is performed according to the MCLT method.

Then, the data insertion unit 415 adds predetermined data (data) to the frequency-converted audio signal for each frame. Specifically, data is added by varying a phase coefficient among coefficients (magnitude, phase, etc.) for each frequency that has been converted into frequency.

The inverse transform unit 420 performs inverse transform on an audio signal into which information data is inserted, for each frame. Accordingly, a time-based sound signal is output.

In the drawings, for each frame, first to fourth sound signals 401a to 401d similar to the first to fourth audio signals 400a to 400d are illustrated. Looking at the drawings, it can be seen that the first to fourth sound signals 401a to 401d are each similar to an audio signal originally intended to be output.

Meanwhile, the multiplexer 425 may output one sound signal by multiplexing the sound signal to which data is added for each frame.

Meanwhile, FIGS. 4A to 4C describe that a sound including predetermined information is output from the mobile terminal 600, and the home appliance 200 receives the sound and extracts predetermined information. It is also possible.

That is, a sound including predetermined information may be output from the home appliance 200, and the mobile terminal 600 may receive the corresponding sound and extract predetermined information.

In this case, the acoustic communication unit 217 in the home appliance 200 may include a frequency conversion unit 410, a data insertion unit 415, an inverse conversion unit 420, and a multiplexing unit 425, and the mobile terminal The acoustic communication unit 617 in 600 may include a synchronization unit 430, a frequency conversion unit 435, and a data extraction unit 440.

4D illustrates that the sound generator 618 is provided in the mobile terminal 600 and the sound detector 218 is provided in the home appliance 200.

The sound generator 618 in the mobile terminal 600 of FIG. 4D is similar to the sound communication unit 617 in the mobile terminal 600 of FIG. 4B, but generates audio data by itself to generate the input data. There is a difference in outputting in addition to the data. That is, it is possible to generate audio data by itself without receiving the audio data from the control unit 680 or the like. To this end, in addition to the frequency conversion unit 410, the data insertion unit 415, the inverse conversion unit 420, and the multiplexing unit 425 in the acoustic communication unit 617, a separate audio generation unit (not shown) is provided. I can.

For example, when the sound generator 618 configures 4 consecutive sounds using 36 sounds, it may generate approximately 168,000 sounds. And, to this, data can be added.

Next, the sound detection unit 218 in the home appliance 200 is the same as the sound communication unit 217 in the home appliance 200 of FIG. 4B. That is, in order to extract data from the received sound, a synchronization unit 430, a frequency conversion unit 435, and a data extraction unit 440 may be provided.

5 is an internal block diagram of the server of FIG. 1.

Referring to the drawings, the server 500 may include a communication module 530, a storage unit 540, and a processor 520.

The communication module 530 may receive home appliance related information from the mobile terminal 600. In addition, the communication module 530 may transmit result information corresponding to the received home appliance related information to the mobile terminal 600.

To this end, the communication module 530 may include an Internet module or a mobile communication module.

The storage unit 540 may store received home appliance-related information and may include data for generating result information corresponding to the home appliance-related information.

For example, when the home appliance-related information is product information of the home appliance, the storage unit 540 may store received product information for product registration of the home appliance.

The processor 520 may control the overall operation of the server 500.

Meanwhile, when receiving home appliance related information from the mobile terminal 600, the processor 520 may control to generate result information corresponding to the home appliance related information. In addition, the generated result information may be controlled to be transmitted to the mobile terminal 600.

For example, when the home appliance related information is product information of the home appliance, the processor 520 may generate product registration result information of the home appliance.

Meanwhile, the server 500 may be a server operated by a manufacturer of the home appliance 200 or a server operated by an application store operator related to the home appliance 200.

6 is an internal block diagram of the mobile terminal of FIG. 1.

Referring to FIG. 6, the mobile terminal 600 includes a wireless communication unit 610, an audio/video (A/V) input unit 620, a user input unit 630, a sensing unit 640, an output unit 650, and A memory 660, an interface unit 670, a control unit 680, and a power supply unit 690 may be included.

Meanwhile, the wireless communication unit 610 may include a broadcast receiving module 611, a mobile communication module 613, a wireless Internet module 615, an acoustic communication unit 617, and a GPS module 619.

The broadcast receiving module 611 may receive at least one of a broadcast signal and broadcast related information from an external broadcast management server through a broadcast channel. In this case, the broadcast channel may include a satellite channel, a terrestrial channel, and the like.

The broadcast signal and/or broadcast related information received through the broadcast receiving module 611 may be stored in the memory 660.

The mobile communication module 613 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include a voice call signal, a video call signal, or various types of data according to transmission/reception of text/multimedia messages.

The wireless Internet module 615 refers to a module for wireless Internet access, and the wireless Internet module 615 may be built-in or external to the mobile terminal 600. For example, the wireless Internet module 615 may perform WiFi-based wireless communication or WiFi Direct-based wireless communication.

The acoustic communication unit 617 may perform acoustic communication. The acoustic communication unit 617 may output sound by adding predetermined information data to audio data to be output in the acoustic communication mode. In addition, the acoustic communication unit 617 may extract predetermined information data from sound received from the outside in the acoustic communication mode. For a detailed description of this, refer to FIGS. 4A to 4D.

In addition, as a short-range communication technology, Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, and the like may be used.

The GPS (Global Position System) module 619 may receive location information from a plurality of GPS satellites.

The A/V (Audio/Video) input unit 620 is for inputting an audio signal or a video signal, and may include a camera 621 and a microphone 623.

The user input unit 630 generates key input data that the user inputs to control the operation of the terminal. To this end, the user input unit 630 may include a key pad, a dome switch, and a touch pad (positive pressure/power failure). In particular, when the touch pad forms a layered structure with the display 651, this may be referred to as a touch screen.

The sensing unit 640 detects the current state of the mobile terminal 600, such as an open/closed state of the mobile terminal 600, a location of the mobile terminal 600, and whether a user is in contact, and controls the operation of the mobile terminal 600. It can generate a sensing signal.

The sensing unit 640 may include a detection sensor 641, a pressure sensor 643, a motion sensor 645, and the like. The motion sensor 645 may detect the motion or position of the mobile terminal 600 using an acceleration sensor, a gyro sensor, a gravity sensor, or the like. In particular, the gyro sensor is a sensor that measures an angular velocity, and can detect a direction (angle) that has turned relative to a reference direction.

The output unit 650 may include a display 651, an audio output module 653, an alarm unit 655, and a haptic module 657.

The display 651 displays and outputs information processed by the mobile terminal 600.

Meanwhile, as described above, when the display 651 and the touch pad form a mutual layer structure to form a touch screen, the display 651 may be used as an input device capable of inputting information by a user's touch in addition to an output device. I can.

The sound output module 653 outputs audio data received from the wireless communication unit 610 or stored in the memory 660. The sound output module 653 may include a speaker, a buzzer, and the like.

The alarm unit 655 outputs a signal for notifying the occurrence of an event of the mobile terminal 600. For example, a signal can be output in the form of vibration. .

The haptic module 657 generates various tactile effects that a user can feel. A typical example of the tactile effect generated by the haptic module 657 is a vibration effect.

The memory 660 may store a program for processing and control of the controller 680, and provides a function for temporary storage of input or output data (eg, phonebook, message, still image, video, etc.). You can also do it.

The interface unit 670 serves as an interface with all external devices connected to the mobile terminal 600. The interface unit 670 may receive data from such an external device or receive power and transmit it to each component inside the mobile terminal 600, and transmit the data inside the mobile terminal 600 to an external device. .

The controller 680 typically controls the operation of each unit to control the overall operation of the mobile terminal 600. For example, it is possible to perform related control and processing for voice calls, data communication, and video calls. In addition, the controller 680 may include a multimedia playback module 681 for playing multimedia. The multimedia playback module 681 may be configured as hardware in the control unit 680 or may be configured as software separately from the control unit 680.

The power supply unit 690 receives external power and internal power under the control of the control unit 680 and supplies power necessary for the operation of each component.

Meanwhile, the block diagram of the mobile terminal 600 shown in FIG. 6 is a block diagram for an embodiment of the present invention. Each component of the block diagram may be integrated, added, or omitted according to the specifications of the mobile terminal 600 that is actually implemented. That is, if necessary, two or more components may be combined into a single component, or one component may be subdivided into two or more components and configured. In addition, the functions performed by each block are for explaining the embodiments of the present invention, and specific operations or devices thereof do not limit the scope of the present invention.

7 is a flow chart showing a method of operating a mobile terminal according to an embodiment of the present invention, Figure 8 is a flow chart showing a method of operating a home appliance according to an embodiment of the present invention, and FIGS. 9 to 13D are It is a diagram referred to for explanation of the operation method of FIG. 7 or 8.

First, referring to FIG. 7, the mobile terminal 600 determines whether there is an application execution input related to the home appliance (S705). And, if applicable, the application is executed (S710). Meanwhile, step 905 (S905) of FIG. 9 corresponds to step 705 (S705) of FIG. 7, and step 910 (S910) of FIG. 9 corresponds to step 710 (S710) of FIG. 7.

10A illustrates that a home appliance-related application execution screen 1010 is displayed on the mobile terminal 600.

In response to a user input, when there is an input for executing a home appliance-related application in the mobile terminal 600, the control unit 680 of the mobile terminal 600 controls to run the pre-installed home appliance-related application.

On the other hand, when the home appliance-related application is not installed, the mobile terminal 600 may access the server 500, download the home appliance-related application, and install it in the mobile terminal 600.

In particular, the application execution screen 1010 of FIG. 10A illustrates that a user log-in item (ID, password), a member sign-up item for g-member sign-up, and the like are included in the case of not signing up as a member.

When a login or the like is performed after the home appliance-related application is executed in the mobile terminal 600, various operations related to the home appliance may be controlled through the application.

Next, when a product registration item in the executed application is selected according to a user input, the mobile terminal 600 selects the item (S715), and displays a product registration execution screen (S720). Meanwhile, step 915 (S915) of FIG. 9 corresponds to step 715 (S715) of FIG. 7, and step 920 (S920) of FIG. 9 corresponds to step 720 (S720) of FIG. 7.

Next, FIG. 10B illustrates that the view more item 1025 is selected from the home appliance-related application screen 1020. Accordingly, as shown in FIG. 10C, a product registration start screen 1030 may be displayed. In addition, FIG. 10C illustrates that the product registration item 1035 is selected in the product registration start screen 1030. Accordingly, as shown in FIG. 10D, a product registration execution screen 1040 may be displayed on the mobile terminal 600.

Meanwhile, the home appliance-related application screen 1020 of FIG. 10B may provide various functions to a user. The application screen 1020 of FIG. 10B is a smart access item capable of remote control and monitoring, a smart recipe item capable of providing recipe-related information, and a shopping list related to a refrigerator or cooking appliance. Including a shopping list item that can provide a shopping list item, a smart diagonosis item that can provide services related to fault diagnosis of a home appliance, and a more item for viewing other additional carriers. can do.

Meanwhile, with reference to FIG. 8, the home appliance 600 stores product information of the home appliance in the memory 260 (S830 ). Meanwhile, step S930 of FIG. 9 corresponds to step S830 of FIG. 8.

When manufacturing the home appliance 200, product information of the home appliance may be stored in the memory 260.

Meanwhile, the washing machine product information may be a concept including product type information indicating that the product type is a washing machine, product manufacturing date information, product serial number, and type/version information of firmware installed on the product.

Referring to FIG. 8, the home appliance 200 determines whether it is in the acoustic communication mode (S840). Further, in the case of the acoustic communication mode, a sound to which product information is added to an audio signal to be output is output. Accordingly, the mobile terminal 600 receives a sound including product information, which is information related to product registration, from the home appliance (S745). Then, product information is extracted from the received sound (S747).

Meanwhile, step 940 (S940) of FIG. 9 corresponds to step 740 (S740) of FIG. 7 and step 840 (S840) of FIG. 8, and step 945 (S945) of FIG. It corresponds to step 745 (S745) and step 845 (S845) of FIG. 8, and step 947 (S947) of FIG. 9 corresponds to step S747 of FIG. 7.

Meanwhile, the acoustic communication mode may be entered when a specific key operates in the home appliance 200 or when the power of the home appliance is first turned on. According to this acoustic communication mode, the acoustic communication unit 217 in the home appliance 200 includes an internal frequency conversion unit 410, a data insertion unit 415, an inverse conversion unit 420, and a multiplexing unit 425. Then, a sound to which product information is added can be output.

10E illustrates a case in which a specific key 201a in the refrigerator 200a operates while the product registration execution screen 1040 is displayed on the mobile terminal 600. Thereby, the refrigerator 200a can enter the acoustic communication mode.

Accordingly, as shown in FIG. 10E, the refrigerator 200a may output a sound including product information Spr in an acoustic communication mode. In this case, the information included in the sound may include product information, operation state information, and the like of the refrigerator 200a. In the drawings, for convenience, it is illustrated that sound including product information Spr is output.

The mobile terminal 600 may receive sound from the refrigerator 200a and extract predetermined information from the received sound. In particular, product information Spr related to the product registration execution screen 1040 displayed on the mobile terminal 600 may be extracted. Accordingly, the mobile terminal 600 can simply obtain information related to the displayed screen from the refrigerator 200a.

Meanwhile, when product information of the home appliance is extracted from the received sound, the control unit 680 of the mobile terminal 600 may control to display an object indicating whether product registration is performed according to product information reception.

10F illustrates that the mobile terminal 600 notifies the received product information, and displays a screen 1050 indicating whether to perform product registration according to the product information. In this case, when the user selects the product registration item 1055, the control unit 680 of the mobile terminal 600 may control to transmit product information to the server 500.

Next, the mobile terminal 600, in particular, the communication module 613 or 615 in the mobile terminal 600 transmits the received product information to the server (S760). Then, the mobile terminal 600 receives, through the communication module 613 or 615, product registration result information, which is result information, corresponding to the product information received from the server (S765). Then, product registration result information is displayed on the display (S770). Meanwhile, step 960 (S960) of FIG. 9 corresponds to step 760 (S760) of FIG. 7, and step 965 (S965) of FIG. 9 corresponds to step 765 (S765) of FIG. Step 970 (S970) of FIG. 7 corresponds to step 770 (S770) of FIG. 7.

10G illustrates that the mobile terminal 600 transmits product information (Spr) received from the home appliance 200 to the server 500, and the server 500 is the result of product registration to the mobile terminal 600 It exemplifies transmission of information Sres to the mobile terminal 600.

According to the transmission of product information Spr from the mobile terminal 600, the communication module 530 of the server 500 receives the home appliance product information Spr from the mobile terminal 600.

The processor 520 of the server 500 controls to store the home appliance product information Spr in the storage unit 540, and may control to perform product registration based on the product information Spr.

When registering a product, product type information, product manufacturing date information, product serial number, and type/version information of the firmware installed on the product can be used.

On the other hand, in addition, when product registration, when user information (user's name, ID, password, address, age, family relationship, etc.) is required, the processor 520 of the server 500, when registering the product with the mobile terminal 600 It is possible to request transmission of necessary additional information, for example, user information.

In this case, the mobile terminal 600 may display a message indicating that there is a request to transmit user information from the server 600 on the screen. In addition, when the user approves the transmission of user information, the control unit 680 of the mobile terminal 600 may control the transmission to the server 600 by using the user information previously stored in the mobile terminal 600. .

Alternatively, when user information is not stored in the mobile terminal 600, the controller 680 of the mobile terminal 600 may control to display a screen for inputting user information.

The server 500 may complete product registration with the server 500 by using the input product information and other additional information.

Further, the processor 520 of the server 500 may control to generate product registration result information of the home appliance and transmit the product registration result information to the mobile terminal 600. Accordingly, the communication module 530 of the server 500 may transmit product registration result information to the mobile terminal 600.

The communication module of the mobile terminal 600, for example, the mobile communication module 613 or the wireless Internet module 615 may receive product registration result information from the communication module 530 of the server 500. Then, the received product registration result information is transmitted to the control unit 680.

Meanwhile, the control unit 680 of the mobile terminal 600 may generate and display a product registration result screen using the received product registration result information.

10H illustrates that the received product registration completion result screen 1060 is displayed on the mobile terminal 600. The mobile terminal receives product registration result information from the server and displays it, so that the user can easily recognize the completion of product registration.

Meanwhile, the product registration completion result screen 1060 of FIG. 10H may include a home screen movement screen 1065. When the moving item 1067 is selected from the home screen moving screen 1065, as shown in FIG. 10I, a smart access item capable of remote control and monitoring, a smart recipe that can provide recipe-related information, etc. recipe) item, a shopping list item that can provide a shopping list in relation to a refrigerator or cooking appliance, a smart diagonosis item that can provide a service related to failure diagnosis of a home appliance, and other additions An application screen 1020 including a more item for viewing a carrier may be displayed. As a result, the user can use various services.

Alternatively, when the moving item 1067 is selected from the home screen moving screen 1065 of FIG. 10H, a login screen 1010 among the home appliance related application screens may be displayed as shown in FIG. 10A.

Meanwhile, wireless data communication between the mobile terminal 600 and the server 500 may be performed through an adjacent access point (AP) device.

11A illustrates that the mobile terminal 600 transmits product information Spr to the AP device 400, and the AP device 400 transmits product information Spr to the server 500 through a network.

Next, FIG. 11B shows that the server 500 transmits product registration result information (Sres) to the AP device 400 through the network, and the AP device 400 transmits product registration result information (Sres) to the mobile terminal 600. To illustrate the transmission.

In this way, when wireless data communication between the mobile terminal 600 and the server 500 is performed through the adjacent access point (AP) device 400, the mobile terminal 600 provides product information Before transmitting the data to the server 500, it is also possible to display the AP device list screen 1110 as shown in FIG. 11C. Specifically, between FIGS. 10F and 10H, an AP device list screen 1110 as shown in FIG. 11C may be displayed.

11C illustrates an AP device list screen 1110 including a plurality of AP device items (AP1.AP2, AP3...). The AP device list screen 1110 may include names, network state information, encryption status information, and the like for a plurality of AP device items.

When the first AP device item 1115 is selected by user selection, the password input item 1115 may be displayed as shown in FIG. 11D. Meanwhile, unlike FIG. 11D, the password input item 1115 may be displayed as a pop-up window in FIG. 11C.

Accordingly, product information of the home appliance can be transmitted to the server 500 through the AP device 400 desired by the user.

Meanwhile, the AP device list screen 1110 of FIG. 11C may be displayed when wireless network identifier information (SSID) is received from the AP device through the mobile terminal 600 before product registration is performed. In addition, when any one AP device is selected from the AP device list by user selection, wireless access to the AP device is possible. That is, even during product registration, it is possible to access the server 500 using a pre-selected AP device without displaying the AP device list screen 1110.

12A is a diagram illustrating that the mobile terminal 600 receives product information from various home appliances.

For example, when a specific key 201b of the washing machine 200b is operated, a sound including washing machine product information is output according to an acoustic communication mode, and the mobile terminal 600 receives the corresponding sound and receives the received sound. Washing machine product information can be extracted from sound.

As another example, when a specific key 201c of the remote control device 210c for remote control of the air conditioner 200c is operated, a sound including air conditioner product information is output, and a mobile terminal ( 600) may receive the corresponding sound and extract air conditioner product information from the received sound.

As another example, when a specific key 201d of the cooking appliance 200d operates, a sound including cooking appliance product information is output according to an acoustic communication mode, and the mobile terminal 600 receives the corresponding sound and , Cooking appliance product information may be extracted from the received sound.

As another example, when a specific key 201e of the robot cleaner 200e is operated, a sound including robot cleaner product information is output according to the acoustic communication mode, and the mobile terminal 600 receives the corresponding sound and , The robot cleaner product information may be extracted from the received sound.

12B is an example of transmitting product information (Spr) received by the mobile terminal 600 to the server 500, and the server 500 transmitting product registration completion result information (Sres) to the mobile terminal 600 do.

Accordingly, the server 500 may simply complete product registration by using the received product information Spr. As a result, it is possible to easily perform product registration for various home appliances through acoustic communication.

In addition, the mobile terminal 600 may display a received product registration completion result screen 1360, as shown in FIG. 12C. On the other hand, the product registration completion result screen 1360 may include a home screen movement screen, similar to FIG. 10H. As shown in FIG. 10H, when a movement item 1067 is selected among the home screen movement screen 1065 , As shown in FIG. 10A, a login screen 1010 among the application screens related to the home appliance may be displayed.

Meanwhile, when the home appliance 200 includes a WiFi module, it may register network information on the access point device and access the access point device by using the registered network information. However, the display or operation unit provided by the home appliance 200 is inconvenient to display and input network information. In order to alleviate such inconvenience, it is possible to output network information on the access point device by using acoustic communication.

Next, FIGS. 13A to 13D illustrate that, for the home appliance 200 having a WiFi module, the mobile terminal 600 outputs a sound including network information about a previously stored access point device.

First, FIG. 13A illustrates that an access point device list screen 1300 of the mobile terminal 600 is displayed. The access point device list screen 1300 may include pre-stored access point device items or currently accessible access point device items.

Among them, when a specific access point device item 1305 is selected by user selection, an object 1315 indicating outputting network information on the selected access point device in an acoustic communication mode may be displayed as shown in FIG. 13B. have. On the other hand, entry into the acoustic communication mode can be performed by selecting a specific key or a specific item.

In addition, as shown in FIG. 13C, the mobile terminal 600 may output a sound including network information (Snet) for the selected access point device through the sound output module 653.

Accordingly, the home appliance 200 may receive sound through the microphone 223 and extract network information (Snet) for a specific access point device from the received sound.

In addition, the home appliance 200 can access the corresponding access point device 400 using the extracted network information (Snet) for the specific access point device, as shown in FIG. 13D, and furthermore, the server 500 You will be able to connect.

The mobile terminal and its operation method according to the present invention are not limited to the configuration and method of the embodiments described as described above, but the embodiments are all or part of each embodiment so that various modifications can be made. It may be configured in combination selectively.

Meanwhile, the method of operating a mobile terminal or a home appliance according to the present invention can be implemented as a code that can be read by a processor on a recording medium that can be read by a processor provided in the mobile terminal or home appliance. The processor-readable recording medium includes all types of recording devices that store data that can be read by the processor. Examples of recording media that can be read by the processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., and also include those implemented in the form of carrier waves such as transmission through the Internet. . Further, the processor-readable recording medium is distributed over a computer system connected through a network, so that the processor-readable code can be stored and executed in a distributed manner.

In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. In addition, various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

Claims (16)

Displaying an access point device list screen;
When a predetermined access point device item in the access point device list is selected, outputting a sound including network information on the selected access point device item;
Receiving a sound including product information of the home appliance;
Extracting product information of the home appliance from the sound; And
And transmitting the extracted product information to a server. The method of claim 1,
Receiving product registration result information from the server; And
Displaying the product registration result information; and further comprising. The method of claim 1,
Displaying a product registration related screen; further includes,
The extraction step,
And extracting product information of the home appliance related to the displayed product registration-related screen from the sound. The method of claim 1,
When product information of the home appliance is extracted, displaying an object indicating whether product registration is performed according to the product information reception. The method of claim 1,
The extraction step,
Synchronizing the received sound;
Frequency converting the synchronized sound signal;
And extracting product information of the home appliance from the frequency-converted sound signal. The method of claim 5,
The frequency conversion step,
And frequency transforming the sound signal based on a modulated complex lapped transform (MCLT). delete The method of claim 1,
When a predetermined access point device item in the access point device list is selected, displaying an object indicating outputting a sound including network information for the selected access point device item; movement further comprising: How to operate the terminal. display;
A communication module for exchanging data with a server;
A microphone for receiving a sound including product information of the home appliance;
An acoustic communication unit for extracting product information of the home appliance from the sound;
A control unit for controlling to transmit the extracted product information to a server;
Includes; a sound output module,
The display,
Displays the access point device list screen,
The control unit, when a predetermined access point device item in the access point device list is selected, controls the sound output module to output a sound including network information on the selected access point device item . The method of claim 9,
The control unit,
When receiving product registration result information from the server through the communication module, controlling to display the product registration result information on the display. The method of claim 9,
The acoustic communication unit,
And when the sound is received while the product registration related screen is displayed on the display, product information of the home appliance related to the displayed product registration related screen is extracted from the sound. The method of claim 9,
The control unit,
When product information of the home appliance is extracted, controlling to display an object indicating whether product registration is performed according to the product information reception. The method of claim 9,
The display,
To transmit product information to the server, display a list of adjacent access point devices,
The control unit,
And controlling to transmit the product information to the server using the selected access point device when any one access point device from the list is selected. The method of claim 11,
The acoustic communication unit,
A synchronization unit synchronizing the received sound;
A frequency converter configured to frequency convert the synchronized sound signal;
And a data extracting unit for extracting information related to the screen from the frequency-converted sound signal. delete The method of claim 9,
The control unit,
When a predetermined access point device item in the access point device list is selected, controlling the display to display an object indicating outputting a sound including network information on the selected access point device item .

Images