KR20130038757A - Mobile terminal and operation method thereof - Google Patents

Abstract

PURPOSE: A portable terminal and an operation method thereof are provided to include a user interface which controls and monitors electronic devices in a home network system. CONSTITUTION: A sensing unit(140) generates directivity information by sensing a direction which is indicated by a main body. A memory(160) stores mapping information for mapping the electronic devices in virtual space corresponding to a home network system. A control unit(180) detects the one or more electronic devices mapped in the virtual space by using the directivity information and the mapping information. A display unit(151) displays the state information of the detected one or more electronic devices in the virtual space. [Reference numerals] (110) Wireless communication unit; (111) Broadcasting reception module; (112) Mobile communication module; (113) Wireless Internet module; (114) Local area communication module; (115) Location information module; (120) A/V input unit; (121) Camera; (122) Microphone; (130) User input unit; (140) Sensing unit; (141) Proximity sensor; (150) Output unit; (151) Display unit; (152) Sound output module; (153) Alarm unit; (154) Haptic module; (160) Memory; (170) Interface unit; (180) Control unit; (181) Multimedia module; (190) Power supplying unit;

Description

Mobile Terminal and How It Works {MOBILE TERMINAL AND OPERATION METHOD THEREOF}

The present invention relates to a mobile terminal, and more particularly, to a mobile terminal capable of controlling electronic devices in a home network system.

A terminal can be divided into a mobile or portable terminal and a stationary terminal depending on whether the terminal is movable or not. The mobile terminal may be further classified into a handheld terminal and a vehicle mount terminal according to whether a user can directly carry it.

As the functions of such terminals are diversified, they are implemented in the form of a multimedia player having complex functions such as photographing and photographing of movies, reproduction of music and video files, games, and broadcasting reception . Further, in order to support and enhance the function of the terminal, it may be considered to improve the structural and software parts of the terminal.

Thanks to this improvement, the function of the terminal can be applied to the management of the home network system. The home network system is a collection of technologies for maximizing the convenience of life as a system integrating various electronic devices using a home network utilizing various wired and wireless technologies. For example, home networks include wired technologies such as Ethernet, PLC, IEEE 1394, home PNA, wireless technologies such as IEEE 802.11 WLAN, IEEE 802.15 WPAN, UWB, UPnP, HAVI, JINI, Home Network Control Protocol (HNCP), etc. Middleware and the like can be utilized.

An object of the present invention is to provide a mobile terminal having a user interface for monitoring and controlling electronic devices of a home network system.

One embodiment of the present invention relates to a mobile terminal for controlling electronic devices in a home network system. The mobile terminal may include a main body, a sensing unit configured to sense direction indicated by the main body to generate direction information, and a storage unit which stores mapping information for mapping the electronic devices to a virtual space corresponding to the home network system. A control unit for detecting at least one of the electronic devices mapped to the virtual space using the orientation information and the mapping information, and a display unit displaying state information of the detected at least one electronic device in the virtual space. do.

The control unit may generate the mapping information based on location information of the electronic devices transmitted from the relay device of the home network system.

The control unit may further include a registration mode for mapping the electronic device to the virtual space when an encrypted registration message is received from the electronic device of the home network system. You can run The controller may decrypt the encrypted registration message using an encryption key provided from the electronic device in the registration mode. The controller may map the electronic device to the virtual space when the encrypted registration message is decrypted.

In an embodiment, the mobile terminal may further include a communication unit configured to receive tag information from the electronic device through adjacent magnetic field communication (NFC). The tag information may include the encryption key and detailed information of the electronic device.

The control unit may execute a control mode for controlling the at least one selected electronic device when the selection of at least one of the electronic devices corresponding to the state information is detected. .

The display unit may display a control menu corresponding to the at least one selected electronic device in the control mode. The control menu may be provided from the at least one selected electronic device.

In at least one example embodiment, the detected at least one electronic device may be disposed in a space formed in a direction indicated by the orientation information at a position of the main body.

The control unit may map the electronic devices to the virtual space based on a touch input applied to the display unit.

In an embodiment, the image corresponding to the virtual space may be displayed as a two-dimensional plan view or a three-dimensional stereogram.

The display unit may display a menu for generating and editing an image corresponding to the virtual space. The sensing unit may be a gyroscope sensor.

One embodiment of the present invention relates to a method of operating a mobile terminal in a home network system. The method of operating the mobile terminal may include generating direction information by sensing a direction indicated by a main body, and loading mapping information for mapping electronic devices of the home network system to a virtual space corresponding to the home network system. Detecting at least one of electronic devices mapped to the virtual space using the orientation information and the mapping information, and displaying state information of the detected at least one electronic device in the virtual space. It includes.

The method of operating the mobile terminal may further include executing a registration mode for mapping the electronic device to the virtual space when an encrypted registration message is received from the electronic device of the home network system. can do.

The method of operating the mobile terminal may include mapping the electronic device to the virtual space when decrypting the encrypted registration message using an encryption key provided from the electronic device in the registration mode. It may further include.

The method of operating the mobile terminal may include executing a control mode for controlling the at least one selected electronic device when a selection of at least one of the electronic devices corresponding to the state information is detected. It may further include.

According to the mobile terminal according to the present invention, some of the electronic devices mapped to the virtual space are selected based on the orientation direction of the terminal body and the state information is displayed, so that the user indicates a simple gesture of pointing the electronic device to be controlled to the mobile terminal. Only the electronic device can be monitored. As a result, a more manageable and intuitive user interface may be provided in managing electronic devices connected to a home network using a mobile terminal.

1 is a block diagram illustrating a mobile terminal according to the present invention.
2A and 2B are perspective views showing an appearance of a mobile terminal according to the present invention.
3 is a conceptual diagram illustrating a home network system according to an exemplary embodiment.
4 is a flowchart illustrating an embodiment of a method of registering an electronic device using a mobile terminal in a home network system.
5 is a flowchart illustrating an embodiment of a method of detecting an electronic device using a mobile terminal in a home network system.
6 is a flowchart illustrating an embodiment of a control method of an electronic device using a mobile terminal in a home network system.
7A and 7B are conceptual views illustrating an operation example of a mobile terminal associated with registration of an electronic device according to an embodiment of the present disclosure.
8 is a conceptual diagram illustrating an operation example of a mobile terminal related to image generation corresponding to a virtual space according to an embodiment of the present invention.
9A and 9B are conceptual views illustrating an operation example of a mobile terminal related to detection of an electronic device according to an embodiment of the present disclosure.
10 is a conceptual diagram illustrating an operation example of a mobile terminal associated with controlling an electronic device according to an embodiment of the present disclosure.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings so that those skilled in the art may easily implement the technical idea of the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

The mobile terminal described herein may include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a navigator, have. However, it should be understood that the configuration according to the embodiments described herein may be applied to a fixed terminal such as a digital TV, a desktop computer, and the like, unless it is applicable only to a mobile terminal. It will be easy to see.

1 is a block diagram illustrating a mobile terminal 100 according to the present invention. 1, a mobile terminal 100 includes a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, A controller 160, an interface unit 170, a controller 180, and a power supply unit 190. The components shown in Fig. 1 are not essential, so that a mobile terminal having more or fewer components can be implemented.

Hereinafter, the components 110 to 190 of the mobile terminal 100 will be described in order.

The wireless communication unit 110 may include one or more modules for enabling wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and the network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short distance communication module 114, and a location information module 115.

The broadcast receiving module 111 receives broadcast signals and broadcast related information from an external broadcast management server through a broadcast channel. Here, the broadcast-related information means information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast-related information can also be provided through a mobile communication network. In this case, the broadcast-related information may be received by the mobile communication module 112. The broadcast signal and the broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. Such wireless signals may include various types of data depending on a voice call signal, a video call signal, a text message, or a multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and may be embedded in the mobile terminal 100 or externally. Wireless Internet technologies may include Wireless LAN (Wi-Fi), Wireless Broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), and the like.

The short range communication module 114 refers to a module for short range communication. As a short range communication technology, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and the like can be used.

The location information module 115 is a module for acquiring a location of the mobile terminal 100, and a representative example thereof is a GPS (Global Position System) module.

1, an A / V (Audio / Video) input unit 120 is for inputting an audio signal and a video signal, and may include a camera 121, a microphone 122, and the like. The camera 121 processes image frames such as still images and moving images obtained by the image sensor in the video communication mode or the photographing mode. The image frame processed by the camera 121 can be displayed on the display unit 151. [ The image frame may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [ More than two cameras 121 may be provided depending on the use environment.

The microphone 122 processes the sound signal input from the outside in the communication mode, the recording mode, the voice selection mode, and the like as electrical voice data. The voice data processed by the microphone 122 in the communication mode can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 and output. The microphone 122 may be implemented with various noise reduction algorithms for eliminating noise generated when an external sound signal is input.

The user input unit 130 generates input data for controlling the operation of the mobile terminal 100 by a user. The user input unit 130 may be composed of a key pad, a dome switch, a touch pad (static and electrostatic), a jog wheel, a jog switch, and the like.

The sensing unit 140 senses the current state of the mobile terminal 100 such as presence or absence of a user, the open / close state of the mobile terminal 100, position, orientation, acceleration, deceleration, And generates a sensing signal. For example, when the mobile terminal 100 is in the form of a slide phone, the sensing unit 140 may detect whether the slide phone is opened or closed. The sensing unit 140 may sense whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like.

The sensing unit 140 may include a proximity sensor 141. The sensing unit 140 may include a touch sensor (not shown) for sensing a touch operation with respect to the display unit 151.

The touch sensor may have the form of a touch film, a touch sheet, a touch pad, or the like. The touch sensor may be configured to convert a pressure applied to a specific portion of the display portion 151 or a change in capacitance generated at a specific portion of the display portion 151 into an electrical input signal. The touch sensor may be configured to detect a touch pressure as well as a touched position and area.

When the touch sensor and the display unit 151 have a mutual layer structure, the display unit 151 can be used as an input device in addition to the output device. The display unit 151 may be referred to as a 'touch screen'.

If there is a touch input via the touch screen, signals corresponding thereto are sent to a touch controller (not shown). The touch controller processes the signals transmitted from the touch sensor, and then transmits data corresponding to the processed signals to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched.

When the touch screen is capacitive, the touch screen may be configured to detect the proximity of the sensing object by the change of the electric field according to the proximity of the sensing object. Such a touch screen may be classified as proximity sensor 141. [

The proximity sensor 141 refers to a sensor that detects the presence or absence of an object to be sensed without mechanical contact using an electromagnetic force or infrared rays. The proximity sensor 141 has a longer life than the contact type sensor and its utilization is also high. Examples of the proximity sensor 141 include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor.

Hereinafter, for convenience of explanation, a proximity action is referred to as " proximity touch " while an object to be sensed does not touch the touch screen, and an action of touching the sensing object on the touch screen is called & touch ".

The proximity sensor 141 detects the presence or absence of a proximity touch and a proximity touch pattern (for example, a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, The information corresponding to the presence / absence of proximity touch and the proximity touch pattern can be output to the touch screen.

The output unit 150 generates an output related to visual, auditory, tactile, and the like. The output unit 150 may include a display unit 151, an audio output module 152, an alarm unit 153, and a haptic module 154.

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, when the mobile terminal 100 operates in the call mode, the display unit 151 displays a UI (User Interface) or a GUI (Graphic User Interface) related to the call. When the mobile terminal 100 operates in the video communication mode or the photographing mode, the display unit 151 displays the photographed image, the received image, the UI, the GUI, and the like.

The display unit 151 may be a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT-LCD), an organic light emitting diode (OLED), a flexible display display, a 3D display, and an e-ink display.

At least one display (or display element) included in the display unit 151 may be configured to be transparent or light transmissive so that the display can see the outside through it. This may be referred to as a transparent display. A representative example of such a transparent display is TOLED (Transparant OLED). The rear structure of the display unit 151 may also be of a light transmission type. With this structure, the user can see an object located behind the terminal body through the area occupied by the display unit 151 in the terminal body.

There may be two or more display units 151 according to the embodiment of the mobile terminal 100. [ For example, in the mobile terminal 100, a plurality of display portions may be spaced apart from each other or positioned integrally with one another, and may be located on different surfaces.

The sound output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception, a call mode or a recording mode, a voice selection mode, a broadcast reception mode, and the like. The sound output module 152 may also output a sound signal related to a function (eg, a call signal reception sound, a message reception sound, etc.) performed in the mobile terminal 100. The sound output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events generated in the mobile terminal 100 include call signal reception, message reception, key signal input, touch input, and the like. The alarm unit 153 may output a signal for notifying occurrence of an event by vibration, in addition to a video signal or an audio signal. Since the video signal or the audio signal may also be output through the display unit 151 or the audio output module 152, the display unit 151 and the audio output module 152 may be classified as part of the alarm unit 153.

The haptic module 154 generates various tactile effects that the user can feel. Vibration is a representative example of the haptic effect generated by the haptic module 154. The intensity, pattern, and the like of the vibration generated by the haptic module 154 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

In addition to vibration, the haptic module 154 may be configured to provide a pin array that vertically moves with respect to the contact skin surface, a jetting force or suction force of air through the jetting or suction port, grazing to the skin surface, contact of the electrode, electrostatic force, and the like. Various tactile effects can be generated, such as effects due to endothermic or reproducing a sense of cold using a heat generating element.

The haptic module 154 may not only deliver a haptic effect through direct contact, but may also be configured to allow the user to feel the haptic effect through a muscle sense such as a finger or an arm. At least two haptic modules 154 may be provided according to the configuration of the mobile terminal 100.

The memory 160 may store a program for the operation of the controller 180 and temporarily store input and output data (e.g., a phone book, a message, a still image, a moving picture, etc.). The memory 160 may store data related to vibrations and sounds of various patterns that are output upon touch input on the touch screen.

The memory 160 may be a flash memory, a hard disk, a multimedia card micro type, a card type memory (e.g., SD or XD memory), a random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM) And may include at least one storage medium. The mobile terminal 100 may operate in association with a web storage that performs storage functions of the memory 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device, receives power, transfers the power to each component inside the mobile terminal 100, or transmits data inside the mobile terminal 100 to an external device. For example, the interface unit 170 may include a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, Input / output ports, video I / O ports, earphone ports, and the like.

The identification module is a chip for storing various information for authenticating the usage right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM), a general user authentication module Universal Subscriber Identity Module (USIM)). A device equipped with an identification module (hereinafter referred to as "identification device") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

The interface unit 170 may be a path through which the power from the cradle is supplied to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle or various command signals input from the cradle by the user, (Not shown). The various command signals or power supplied from the cradle may also act as a signal for recognizing that the mobile terminal 100 is correctly mounted in the cradle.

The controller 180 controls the overall operation of the mobile terminal 100. For example, control and processing related to voice communication, data communication, video communication, and the like. The control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [ The control unit 180 may perform pattern selection processing for selecting handwriting input and drawing input on the touch screen as characters and images, respectively.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 to supply power for operation of each component.

The various embodiments described herein may be implemented in a recording medium readable by a computer or similar device using software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be applied to various types of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays Microprocessors, microprocessors, and other electronic units for performing other functions, as will be appreciated by those skilled in the art. In some cases, the embodiments described herein may be implemented by the controller 180 itself.

According to the software implementation, embodiments such as the procedures and functions described herein may be implemented as separate software modules. Each of the software modules may perform one or more of the functions and operations described herein. Software code may be implemented in a software application written in a suitable programming language. Such software code may be stored in the memory 160 and executed by the controller 180.

Hereinafter, a method of processing user input to the mobile terminal 100 will be described.

The user input unit 130 is operated to receive a command for controlling the operation of the mobile terminal 100, and may include a plurality of operation units. The operating units may also be referred to as manipulating portions and may be employed in any manner as long as they are operated in a tactile manner by the user's tactile sense.

Various kinds of time information can be displayed on the display unit 151. [ Such visual information can be displayed in the form of letters, numbers, symbols, graphics, icons, and the like, and can be formed as a three-dimensional stereoscopic image. At least one of a character, a number, a symbol, a graphic, and an icon may be displayed in a predetermined arrangement for inputting time information, thereby being implemented as a keypad. Such a keypad may be referred to as a so-called " soft key ".

The display unit 151 may operate as an entire area or may be divided into a plurality of areas and operated. In the latter case, the plurality of areas can be configured to operate in association with one another. For example, an output window and an input window may be displayed on the upper and lower portions of the display unit 151, respectively. The output window and the input window are areas allocated for outputting or inputting information, respectively. In the input window, a soft key with a number for inputting a telephone number may be output. When the soft key is touched, the number corresponding to the touched soft key is displayed in the output window. When the operating unit is operated, a call connection to the telephone number displayed in the output window may be attempted, or the text displayed in the output window may be entered into the application.

The display unit 151 or the touch pad may be configured to detect touch scrolling. The user can move the cursor or the pointer positioned on the displayed object, for example, the icon, on the display unit 151 by scrolling the display unit 151 or the touch pad. Further, when the finger is moved on the display unit 151 or the touch pad, the path along which the finger moves may be visually displayed on the display unit 151. [ This will be useful for editing the image displayed on the display unit 151. [

One function of the mobile terminal 100 may be executed in response to a case where the display unit 151 and the touch pad are touched together within a predetermined time range. In the case of being touched together, there may be a case where the user clamps the main body of the mobile terminal 100 using the thumb and index finger. At this time, one function of the mobile terminal 100 to be executed may be activation or deactivation for the display unit 151 or the touch pad, for example.

2A and 2B are perspective views showing an appearance of a mobile terminal 100 according to the present invention. FIG. 2A shows a front side and one side of the mobile terminal 100, and FIG. 2B shows a rear side and the other side of the mobile terminal 100. FIG.

Referring to FIG. 2A, the mobile terminal 100 includes a bar-shaped terminal body. However, the mobile terminal 100 is not limited thereto, and may be realized in various forms such as a slide type, a folder type, a swing type, and a swivel type in which two or more bodies are movably coupled to each other.

The terminal body includes a case (a casing, a housing, a cover, and the like) that forms an appearance. In the embodiment, the case may be divided into a front case 101 and a rear case 102. A variety of electronic components are embedded in the space formed between the front case 101 and the rear case 102. At least one intermediate case may be additionally disposed between the front case 101 and the rear case 102. [

The cases may be formed by injection molding of a synthetic resin or may be formed to have a metal material such as stainless steel (STS), titanium (Ti) or the like.

The display unit 151, the sound output unit 152, the camera 121, the user input unit 130 (see FIG. 1), the microphone 122, the interface 170, can do.

The display unit 151 occupies a main portion of the front case 101. [ The audio output unit 152 and the camera 121 are located in a region adjacent to one end of the display unit 151 and the first user input unit 131 and the microphone 122 are located in a region adjacent to the other end. The second user input unit 132 and the interface 170 may be located on the sides of the front case 101 and the rear case 102. [

The user input unit 130 is operated to receive a command for controlling the operation of the mobile terminal 100. [ The user input unit 130 may include a plurality of operation units 131 and 132.

The first or second operating units 131 and 132 can receive various commands. For example, the first operation unit 131 can receive commands such as start, end, scroll, and the like. The second operation unit 132 can receive commands such as adjusting the size of the sound output from the sound output unit 152, switching to the touch selection mode of the display unit 151, and the like.

Referring to FIG. 2B, a rear camera 121 'may be additionally mounted on the rear surface of the terminal body, that is, the rear case 102. The rear camera 121 'has a photographing direction opposite to that of the front camera 121 (see FIG. 2A), and may be configured to have pixels different from those of the front camera 121.

For example, the front camera 121 may be configured to have a low pixel, and the rear camera 121 'may be configured to have a high pixel. Accordingly, when the front camera 121 is used during a video call, the size of the transmission data may be reduced when the user's face is photographed and transmitted to the counterpart in real time. On the other hand, the rear camera 121 'can be used for the purpose of storing a high-quality image.

Meanwhile, the cameras 121 and 121 'may be installed in the terminal body so as to be rotated or popped up.

The flash 123 and the mirror 124 may be further positioned adjacent to the rear camera 121 '. The flash 123 emits light toward the subject when the user photographs the subject with the rear camera 121 '. The mirror 124 illuminates the user's face when the user photographs himself (self-photographing) using the rear camera 121 '.

A rear sound output unit 152 'may be additionally disposed on the rear surface of the terminal body. The rear sound output unit 152 ′ may perform a stereo function together with the front sound output unit 152 (see FIG. 2A), and may perform a speakerphone function during a call.

In addition to the antenna for communication, an antenna 116 for receiving broadcast signals may be additionally provided on the side surface of the terminal body. The antenna 116 constituting a part of the broadcast receiving module 111 (see FIG. 1) can be installed to be able to be drawn out from the terminal body.

A power supply unit 190 for supplying power to the mobile terminal 100 is mounted on the terminal body. The power supply unit 190 may be built in the main body of the terminal or may be detachable from the outside of the main body of the terminal.

The rear case 102 may further include a touch pad 135 for sensing a touch. The touch pad 135 may be of a light transmission type like the display unit 151 (see FIG. 2A). In addition, a rear display unit for outputting visual information may be additionally mounted on the touch pad 135. In this case, information output from both the front display unit 151 and the rear display unit may be controlled by the touch pad 135.

The touch pad 135 operates in correlation with the display unit 151. [ The touch pad 135 may be located parallel to the rear of the display unit 151. The touch pad 135 may have a size equal to or smaller than that of the display unit 151.

In an embodiment of the present invention, the mobile terminal 100 may be a component of a home network system. Hereinafter, the functions of the mobile terminal 100 performed in the home network system will be described in detail.

3 is a conceptual diagram illustrating a home network system 1000 according to an exemplary embodiment. Referring to FIG. 3, the home network system 1000 may include a mobile terminal 1100, a relay device 1200, and a plurality of electronic devices 1310-1360. The home network system 1000 may be built using network protocols and middleware such as Universal Plug and Play (UPnP) and Digital Living Network Alliance (DLNA).

The mobile terminal 1100 may communicate with the electronic devices 1310-1360 through the relay device 1200. In more detail, the mobile terminal 1100 may monitor the states of the electronic devices 1310-1360 by accessing the electronic devices 1310-1360 by the relay operation of the relay device 1200. The mobile terminal 1100 may control the electronic devices 1310 to 1360 based on the monitoring result. In other words, in the home network system 1000 of the present invention, the mobile terminal 1100 may function as a primary controller.

However, the mobile terminal 1100 does not necessarily need to access the electronic devices 1310-1360 through the relay device 1200, but may directly connect to the electronic devices 1310-1360. This means that one-to-many bidirectional communication is possible between the mobile terminal 1100 and the electronic devices 1310-1360.

The mobile terminal 1100 may be implemented as a near field communication (NFC) reader. In this case, the mobile terminal 1100 may receive corresponding tag information from the electronic devices 1310 to 1360 through NFC.

Here, NFC refers to a short-range wireless communication technology that transmits data between mobile terminals at low power at a certain distance, for example, 10 cm to several hundred m. NFC is a modification of RFID that uses frequencies in the 13.56 MHz band and transmits data at a rate of 424 Kbits / s (424 Kbps). Near field communication is specified in accordance with ISO / IEC, ECMA, and ETSI international standards and is compatible with Bluetooth, Wi-Fi / 802.11, and UWB.

The NFC can be divided into a passive communication mode and an active communication mode. In the passive communication mode, an initiator forms an RF field and a target responds by applying a load modulation scheme to the command of the transmitter. On the other hand, in the active communication mode, both the transmitter and the target communicate by forming an RF field.

The relay device 1200 may be implemented as a Wi-Fi router, a home network gateway, or the like. The relay device 1200 relays bidirectional communication between the mobile terminal 1100 and the electronic devices 1310-1360. For example, the relay device 1200 may receive state information of the electronic devices 1310-1360 from the electronic devices 1310-1360 and transmit the state information to the mobile terminal 1100.

In addition, the relay device 1200 may provide location information of the electronic devices 1310 to 1360 to the mobile terminal 1100 using a well-known location determination algorithm such as UbiSense and Ekahau.

The electronic devices 1310-1360 include a refrigerator 1310, an electronic oven 1320, a TV 1330, an audio device 1340, a notebook computer 1350, a washing machine 1360, an air conditioner, a printer, a lighting device, and a security. Device or the like.

4 is a flowchart illustrating an embodiment of a method of registering an electronic device 1300 using a mobile terminal 1100 in a home network system 1000. Referring to FIG. 4, a process in which the electronic device 1300 is registered with the mobile terminal 1100 is illustrated so that the mobile terminal 1100 can smoothly monitor and control the electronic device 1300.

First, when the electronic device 1300 is connected to a home network, for example, when the electronic device 1300 is recognized by the relay device 1200, the electronic device 1300 may encrypt an encrypted registration message. ) Is transmitted to the relay device 1200 (step S110). Then, the relay device 1200 transmits the encrypted registration message to the mobile terminal 1100 (step S120).

Next, the mobile terminal 1100 executes a registration mode in response to receiving the encrypted registration message (step S130). Here, the registration mode refers to a mode for mapping the electronic device 1300 that has requested a registration to a virtual space corresponding to the real space where the home network system is implemented.

When the registration mode is executed, a short range communication module (see FIG. 1) of the mobile terminal 1100, for example, an NFC module, may be activated to receive data necessary for registration of the electronic device 1300.

Thereafter, the mobile terminal 1100 requests tag information to the electronic device 1300 using the activated NFC module (step S140). In response to the tag information request, the electronic device 1300 transmits the tag information to the mobile terminal 1100 (step S150). Here, the tag information may include an encryption key for decrypting the encrypted registration message and detailed information of the electronic device 1300. As such, the mobile terminal 1100 may receive data necessary for registration from the electronic device 1300 through NFC communication.

When the registration message encrypted using the encryption key provided from the electronic device 1300 in the registration mode is decrypted, the mobile terminal 1100 registers the electronic device 1300 to generate mapping information (step S160). The generated map dep information may be stored in a memory of the mobile terminal 1100. Here, the registration of the electronic device 1300 means that the electronic device 1300 is mapped to the virtual space based on the location information of the electronic device 1300.

Location information of the electronic device 1300 required to generate the mapping information may be provided from the relay device 1200 as described above, or may be provided by the user. Generation of mapping information according to a user input will be described in detail with reference to FIGS. 7A to 7C and 8.

As described above, according to an embodiment of the present disclosure, when the encrypted registration message is decrypted using an encryption key provided from the electronic device 1300 requesting registration, registration of the electronic device 1300 may be allowed. . Therefore, it is possible to prevent hacking, interference, and the like by external devices not identified in the home network system.

5 is a flowchart illustrating an embodiment of a method of detecting an electronic device using a mobile terminal in a home network system. Referring to FIG. 5, a process of detecting and monitoring at least one of electronic devices already registered, that is, electronic devices mapped to a virtual space, is illustrated.

First, the sensing unit senses the direction indicated by the terminal body to generate the orientation information (step S210). To this end, the sensing unit may include, for example, a gyroscope sensor. That is, the orientation information may be an output value of a gyroscope sensor.

Thereafter, the controller loads the mapping information stored in the storage unit (eg, memory) (step S220). The controller detects at least one of the electronic devices mapped to the virtual space using the orientation information and the mapping information (step S230). For example, an electronic device arranged in a space formed in the direction indicated by the orientation information at the position of the terminal body may be detected.

Next, the display unit displays the icon and status information of the detected electronic device (step S240). At this time, the communication unit may receive the state information from the corresponding electronic device in real time.

As described above, according to an exemplary embodiment of the present disclosure, some of the electronic devices mapped to the virtual space are selected based on the directing direction of the terminal body and the state information is displayed, so that the user controls the electronic device to be controlled. The electronic device can be monitored by a simple gesture that is indicated by. As a result, a more manageable and intuitive user interface may be provided in managing electronic devices connected to a home network using a mobile terminal.

6 is a flowchart illustrating an embodiment of a control method of an electronic device 1300 using a mobile terminal 1100 in a home network system 1000. Referring to FIG. 6, first, the mobile terminal 1100 detects a selection of at least one of the electronic devices 1300 detected by the method as described with reference to FIG. 5 (S310). In this case, the mobile terminal 1100 executes a control mode for controlling the selected electronic device (step S320).

In the control mode, the mobile terminal 1100 may display a control menu of the selected electronic device. To this end, the mobile terminal 1100 requests a control menu from the electronic device 1300 via the relay device 1200 (steps S330 and S340). In response to the control menu request, the electronic device 1300 transmits the control menu to the mobile terminal 1100 via the relay device 1200 (steps S350 and S360).

When the user inputs a control command using the control menu provided from the electronic device 1300, the mobile terminal 1100 transmits the control command to the electronic device 1300 via the relay device 1200 (step S370, S380). Then, the electronic device 1300 performs a control operation in response to the control command (step S390).

Although the drawing shows that the request and the transmission of the control menu and the transmission of the control command are performed by the relay operation of the relay device 1200, the present invention is not limited thereto. It may be performed by direct communication between the 1100 and the electronic device 1300.

7A and 7B are conceptual views illustrating an operation example of a mobile terminal 1100 associated with registration of an electronic device according to an embodiment of the present disclosure. In relation to an operation example described herein, the mobile terminal 1100 may include a communication unit, a controller, a display unit 1151, and the like.

Referring to FIG. 7A, an encrypted registration message is transmitted from the electronic device 1300 to the mobile terminal 1100 via the relay device 1200 for registration of the electronic device 1300.

When the communication unit receives the encrypted registration message of the electronic device 1300, the controller executes a registration mode. In addition, the display 1151 may display information for notifying that the registration mode has been executed, information for informing a job required for registration, and the like. For example, the display 1151 may display a guide phrase such as 'request password'.

Referring to FIG. 7B, in the registration mode, the communication unit may receive tag information from the electronic device 1300 through NFC communication. The controller may decrypt the encrypted registration message using the encryption key included in the tag information.

When the encrypted registration mesh is decrypted, the display 1151 may display detailed information of the corresponding electronic device 1300 included in the tag information. In addition, the display 1151 may display an image corresponding to the virtual space to which the electronic device 1300 is mapped in the form of a two-dimensional plan view or a three-dimensional stereogram.

According to an embodiment of the present disclosure, when the electronic device 1300 is mapped to a virtual space, a mapping position may be specified in a user input. In more detail, the controller may generate mapping information on the electronic device 1300 by mapping the electronic device 1300 to a location corresponding to a touch point in a virtual space based on a touch input applied to the display 1151. Can be.

When the electronic device 1300 is mapped to the virtual space, as illustrated, the display 1151 may display an icon of the electronic device 1300 at a mapping position on the virtual space. In addition, the display 1151 may display a name, detailed description, location information, and the like of the registered electronic device 1300. Here, the position information may be represented as various coordinates such as three-dimensional rectangular coordinates and dimensional polar coordinates. In addition, the display 1151 may display a menu EDIT for correcting such information.

8 is a conceptual diagram illustrating an operation example of a mobile terminal 1100 related to image generation corresponding to a virtual space according to an embodiment of the present invention. Referring to FIG. 8, the display 1151 may display a menu 1152 for generating and editing an image corresponding to a virtual space to which electronic devices are mapped. The user may draw, for example, a floor plan of his / her home on which the home network system is implemented as an image corresponding to the virtual space using the menu 1152.

9A and 9B are conceptual views illustrating an example of an operation of the mobile terminal 1100 related to detection of an electronic device according to an embodiment of the present disclosure.

9A and 9B, in the operation of the mobile terminal 1100 in the home network system 1000, the display 1151 may display electronic devices detected based on a direction in which the terminal body is directed.

When the terminal body points in the first direction, the controller may include electronic devices disposed in a space formed in the first direction from the position of the terminal body, for example, the TV 1330 and the audio device (as shown in FIG. 9A). 1340 may be detected. The display 1151 may display icons, location information, and status information (for example, a power on or off state) of the TV 1330 and the audio device 1340.

On the other hand, when the terminal body points in the second direction, the control unit detects the electronic device disposed in the space formed in the second direction from the position of the terminal body, for example, the electronic oven 1320, as shown in FIG. 9B. can do. The display 1151 may display an icon, location information, status information, and the like of the electronic oven 1320.

10 is a conceptual diagram illustrating an operation example of a mobile terminal 1100 related to control of an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 10, when a selection of at least one of the detected electronic devices is detected while the display unit 1151 displays icons, location information, status information, etc. of the detected electronic devices, A control mode for controlling the selected electronic device can be executed. In this case, the display 1151 may display a control menu for the selected electronic device.

For example, as shown in the drawing, when a touch on an icon of a sound device is sensed among a TV and a sound device, the controller executes a control mode for controlling the sound device, and the display 1151 is provided from the sound device. Control menu can be displayed. In addition, the display 1151 may display more detailed state information related to the sound device.

According to the embodiment disclosed herein, the above-described method can be implemented by a code that can be read by a processor on a medium on which the program is recorded. Examples of the medium that can be read by the processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., and may be implemented in the form of a carrier wave (e.g., transmission over the Internet) .

In the mobile terminal disclosed in this specification, the configuration and method of the above-described embodiments are not limitedly applied, but all or some of the embodiments may be selectively combined so that various modifications can be made.

Claims (18)

In a mobile terminal for controlling electronic devices in a home network system:
main body;
A detector configured to detect the direction indicated by the main body and generate direction information;
A storage unit which stores mapping information for mapping the electronic devices to a virtual space corresponding to the home network system;
A controller configured to detect at least one of electronic devices mapped to the virtual space using the orientation information and the mapping information; And
And a display unit for displaying the detected state information of the at least one electronic device in the virtual space. The method of claim 1,
The control unit,
And generating the mapping information based on the location information of the electronic devices transmitted from the relay device of the home network system. The method of claim 1,
The control unit,
When an encrypted registration message is received from the electronic device of the home network system, the mobile terminal is characterized by executing a registration mode for mapping the electronic device to the virtual space. . The method of claim 3, wherein
The control unit,
And decrypting the encrypted registration message using an encryption key provided from the electronic device in the registration mode. The method of claim 4, wherein
The control unit,
And when the encrypted registration message is decrypted, mapping the electronic device to the virtual space. The method of claim 5, wherein
Further comprising a communication unit for receiving the tag information from the electronic device through the adjacent magnetic field communication (NFC),
The tag information includes the encryption key and detailed information of the electronic device. The method of claim 1,
The control unit,
And detecting a selection of at least one of the electronic devices corresponding to the status information, executing a control mode for controlling the at least one selected electronic device. The method of claim 7, wherein
The display unit includes:
And displaying a control menu corresponding to the selected at least one electronic device in the control mode. The method of claim 8,
The control menu is provided from the at least one selected electronic device. The method of claim 1,
And the detected at least one electronic device is disposed in a space formed in a direction indicated by the orientation information at a position of the main body. The method of claim 1,
The control unit,
And the electronic devices are mapped to the virtual space based on a touch input applied to the display unit. The method of claim 1,
The image corresponding to the virtual space is displayed as a two-dimensional plan or three-dimensional stereogram. 13. The method of claim 12,
The display unit includes:
And a menu for creating and editing an image corresponding to the virtual space. The method of claim 1,
The detecting unit is a gyroscope sensor. In the method of operating a mobile terminal in a home network system:
Generating orientation information by sensing a direction indicated by the main body;
Loading mapping information for mapping electronic devices of the home network system to a virtual space corresponding to the home network system;
Detecting at least one of electronic devices mapped to the virtual space using the orientation information and the mapping information; And
Displaying the detected state information of the at least one electronic device in the virtual space. The method of claim 15,
And when the encrypted registration message is received from the electronic device of the home network system, executing a registration mode for mapping the electronic device to the virtual space. 17. The method of claim 16,
And when the encrypted registration message is decrypted using the encryption key provided from the electronic device in the registration mode, mapping the electronic device to the virtual space. The method of claim 15,
If a selection of at least one of the electronic devices corresponding to the state information is detected, executing a control mode for controlling the at least one selected electronic device.

Images