KR101925809B1 - Method and apparatus for controlling of home network system using mobile terminal - Google Patents

Abstract

A method for controlling at least one electronic device in a home network system in a home network system configured using Universal Plug and Play (UPnP) middleware, the method comprising: Receiving a list of electronic devices connected to the home network from the gateway, the method comprising the steps of: receiving a device list of electronic devices connected to the home network from the gateway; Displaying the electronic devices based on the received device list on the touch screen of the mobile terminal in the form of an icon when the icon for the first electronic device is selected based on the touch input among the displayed icons; The state information of the electronic device to the application Requesting the first electronic device via the gateway to request a first electronic device to receive the first electronic device from the first electronic device; Displaying a first user interface including a control button on the touch screen, the at least one control button being displayed under the status information of the first electronic device; Transferring the control command to the first electronic device via the gateway when a control command for controlling the first electronic device is input; and transmitting the control command from the first electronic device, Receiving a control result for responding and displaying the result on the touch screen, The step of displaying the electronic devices based on the device list on the touch screen of the mobile terminal in the form of an icon comprises the steps of: determining whether a second electronic device, which is not controlled by the application, exists in the electronic devices; Receiving, via the gateway, a sub-application including a second user interface for displaying status information of the second electronic device and controlling operation of the second electronic device that is not controlled by the application; And displaying an icon for the second electronic device included in the received sub-application on the touch screen, wherein the first user interface can be received from the first electronic device or the gateway.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a method and apparatus for controlling a home network system using a mobile terminal,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to home appliance control of a home network system, and more particularly, to a method and apparatus for observing the state of devices in a home network system, and managing and controlling operations using the mobile terminal.

The home network is a next generation IT (Information Technology) technology, which is currently attracting the most attention. It is a collection of technologies for maximizing the convenience of living by combining with the Internet based on control, management, integration and interworking of home information appliances. The home network is divided into subnetwork technology for physical data transmission, middleware technology for interworking with higher application, and information appliances technology applied to each home appliance according to its technical layer. Currently, it is expanding into ubiquitous computing in combination with broadband communication, wireless Internet, and sensor technology.

In this home network system, a home network middleware corresponding to an interface between a lower network layer and an application layer for controlling and managing information appliances is currently being studied. However, UPnP (Universal Plug and Play), Jini, OSGi (Open Service Gateway Initiative), and Home Audio / Video Interoperability (HAVi). However, none of them has a technical advantage and coexists with each other.

The home network system, which has been installed in the home centered on the apartment, is connected to wired / wireless devices around a single home gateway and provides various services. However, except for services using an external internet network, It is not largely deviated, and practicality is lacking due to lack of application.

It is an object of the present invention to provide a method and apparatus for observing, managing, and controlling the state of devices of a home network system from outside conveniently using a mobile terminal.

According to an aspect of the present invention, there is provided a method of controlling at least one electronic device in a home network system in a home network system configured using UPnP (Universal Plug and Play) middleware, The method comprising the steps of: executing an application for controlling electronic devices connected to a home network; connecting to a gateway of the home network based on the executed application; The method of claim 1, further comprising the steps of: receiving a device list; displaying the electronic devices based on the received device list on a touch screen of the mobile terminal in an icon form; When the icon for the first electronic device is selected, Requesting the first electronic device via the gateway based on the application status information based on the status information of the first electronic device received from the first electronic device and the type of the first electronic device Displaying on a touch screen a first user interface including at least one control button for controlling another operation, the at least one control button being displayed under the status information of the first electronic device; Transmitting a control command to the first electronic device via the gateway when a control command for controlling the first electronic device is input through a first user interface displayed; Receiving a control result from the electronic device in response to the control command, Wherein the step of displaying the electronic devices based on the received device list on the touch screen of the mobile terminal in the form of an icon comprises the steps of: Determining whether the second electronic device is present, if the second electronic device is not controlled by the application, if the second electronic device is present; And displaying the icon for the second electronic device included in the received sub-application on the touch screen, wherein the first user interface comprises: Or may be received from the gateway.

According to another aspect of the present invention, there is provided a mobile terminal for controlling at least one electronic device existing in the home network system in a home network system configured using UPnP (Universal Plug and Play) middleware, A touch screen for displaying the device list, and an application for controlling the electronic devices connected to the home network, and for executing the application, The gateway device is connected to a gateway of the home network, receives a device list of electronic devices connected to the home network from the gateway, and displays the electronic devices based on the received device list in a form of an icon Mark on Requesting the first electronic device via the gateway based on the application, if the icon for the first electronic device is selected based on the touch input among the displayed icons, A first user interface including at least one control button for controlling different operations according to the status information of the first electronic device received from the electronic device and the type of the first electronic device is displayed on the touch screen And transmits the control command to the first electronic device via the gateway when a control command for controlling the first electronic device is input through the displayed first user interface, 1 receives a control response from the electronic device in response to the control command, Wherein the control unit determines whether there is a second one of the electronic devices that was not controlled by the application and if the second electronic device that was not controlled by the application is present Receiving, via the gateway, a sub-application including a second user interface for displaying status information of the second electronic device and controlling operation thereof; and receiving an icon for the second electronic device included in the received sub- Wherein the at least one control button is displayed under the status information of the first electronic device and the first user interface can be received from the first electronic device or the gateway.

The present invention allows a user to observe, manage, and control the status of devices in a system of a home network using an application of a mobile communication terminal outside the home. That is, it is possible to access the home network through the mobile communication terminal outside the home, to grasp the current state of home appliances and control devices in the home, and to control home appliances or control devices connected to the home network through an easy operation through an application in the mobile communication terminal Or the like.

1 is a diagram illustrating a configuration of a home network system and a mobile terminal connected thereto according to an embodiment of the present invention;
2 is a diagram illustrating a configuration of a mobile terminal according to an embodiment of the present invention;
3 is a flowchart illustrating an operation of controlling a home appliance of a home network system according to an embodiment of the present invention.
4 is a view illustrating an example of a display of an application selection screen for controlling a home appliance of a home network system in an operation of controlling a home appliance of a home network system according to an embodiment of the present invention;
5 is a view showing an example of display of a terminal when an application for controlling a home appliance of a home network system is selected on the screen of FIG.
6 is a diagram showing a display example of a terminal when a TV is selected on the screen of Fig. 5
7 is a view showing a display example of a terminal when a security camera is selected on the screen of Fig. 5
8 is a view showing a display example of a terminal when selecting illumination on the screen of Fig. 5
9 is a view showing an example of display of a terminal when a printer is selected on the screen of Fig. 5

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an apparatus and an operation method of the present invention will be described in detail with reference to the accompanying drawings. It will be appreciated that those skilled in the art will readily observe that certain changes in form and detail may be made therein without departing from the spirit and scope of the present invention as defined by the appended claims. To those of ordinary skill in the art. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

1 is a block diagram of a home network system and a mobile terminal connected thereto according to an embodiment of the present invention. 1, the home network system according to an exemplary embodiment of the present invention includes a home network gateway 103, which operates as a server and is connectable to an external communication network 102, and a home appliance 104, which is a client device connected thereto, . The home appliances 104 are connected to a home network such as a printer, a TV, a refrigerator, an air conditioner, a security camera, a facsimile, a lighting, a DVD, a door, a telephone, and a washing machine. Meanwhile, the home network gateway 103 may be implemented in the form of an electronic frame or a personal computer (PC). The home network gateway 103 may operate as the home appliance 104 to transmit status information to an external mobile terminal 101, .

The home appliances 104 may provide their status information to the external communication network 102 via the gateway 103. [ In the present invention, an external mobile terminal (101) connected to an external communication network receives the status information and transmits a command for controlling the home appliances (104) through the gateway (103) to control the home appliances do. Here, the external communication network 102 may be a general mobile communication network, a 3G network, or a wireless Internet network.

In the home network system configured as described above, each home network middleware is supported, so that a home application for the home service can communicate without considering the details of the devices supporting the home network middleware In the case of individual devices, if only the home network middleware is supported, it becomes possible to connect with all applications for supporting services.

2 is a diagram illustrating a configuration of a mobile terminal according to an embodiment of the present invention. 2, the mobile terminal 101 includes a communication unit 202, an input unit 204, a display unit 203, a storage unit 205, and a control unit 201.

The control unit 201 performs general communication functions such as communication and data transmission and reception, and controls other components. Also, the control unit 350 of the present invention accesses a home network, receives a list of devices connected to the home network from a gateway of the home network, displays devices connected to the received home network through the display unit 203, Requesting the selected device via the gateway for status information of the selected device when one of the selected devices is selected by the user, receiving status information from the selected device, and controlling the received status information and the selected device Is displayed on the display unit (203).

The display unit 203 displays various display contents and messages under the control of the control unit 201. [ The display unit 203 may be a liquid crystal display (LCD), a thin film transistor (TFT), or an organic electroluminescence (EL) display.

The input unit 204 includes a plurality of numeric keys and function keys and outputs key input data corresponding to a key pressed by the user to the control unit 201. [ If the display unit 203 is configured as a touch screen, the display unit 203 can operate as the input unit 204 at the same time.

The storage unit 205 comprises a ROM (Read Only Memory) and a RAM (Random Access Memory) for storing a plurality of programs and data necessary for the operation of the mobile terminal. Also, the storage unit 205 of the present invention includes home network application 206 data in a mobile terminal. The application data 206 includes information on a sub-application including information necessary for executing a home network application and a user interface for displaying state information and controlling operation of each of the home appliances 104 connected to the home network.

The mobile station typically performs wireless communication with the base station, and the communication unit 202 transmits and receives RF signals to and from the base station through the antenna. The communication unit 202 also communicates with the home network system gateway to control home appliances of the home network system.

3 is a flowchart illustrating an operation of controlling a home appliance of a home network system according to an embodiment of the present invention. 3 illustrates a process in which the mobile communication terminal 101 enters the home network and grasps the state information of the home appliances connected to the home network and controls the operation.

3, in step 301, the home appliances 104 connect to the home network of the gateway 103 and connect them. In step 301, UPnP, which is the middleware of the home network, may be used. In this process, the gateway 103 finds the home appliance 104 on the network, addresses it, discovers the device, identifies the device, describes the event / And performing operations of Control / Presentation.

The UPnP, which is middleware illustrated in each process, is an IP (Internet Protocol) -based network, so IP is required for each device connected to the network. Therefore, IP allocation, that is, address allocation is performed first. When each device receives an IP address through address assignment, it is then necessary to recognize the devices to be controlled. To this end, the controller (gateway) recognizes the device using a protocol called SSDP (Simple Service Discovery Protocol). At this point, the controller searches for the devices of interest and the controller (home appliance) responds to it. When the controller is connected to the network, the controller automatically informs the other devices of the connection, and periodically informs whether the connection is continued.

When you recognize a device, you need to know what services it will perform for each device. When the controller finds a controller in the controller for this purpose, the controller sends a URL (Uniform Resource Locator) containing the specification of the device, and the controller obtains the device specification in XML from the eXtensible Markup Language (XML) document. This document contains manufacturer information, product information (model, serial number, ...), service list, and so on.

Then, the controller obtains the device specification from the controller, analyzes the service of the device, and then sends an appropriate command to the controller to control the controller. The protocol used at this time is extensible Markup Language (XML) / Simple Object Access Protocol (SOAP). This is called control.

In the home network, the state of the device frequently changes according to the surrounding environment. UPnP defines events because sometimes these changes can be important information to the user. The controller is paying attention to the change of the state of the controller, and the controller transmits an event message to the controller when its state changes. Events are a pair of (name, value), and the protocol used in the event is GENA (Generic Event Notification Architecture) in XML format. This is called an event.

Then, the controller can read the hypertext markup language (HTML) page of the controller. This HTML page shows the user interface related to the use of the equipment, which controls the equipment or shows the status. This is called a presentation.

In this way, a user in the home network system can perform an organic connection through networking of individual devices in the home in the home using UPnP, and various services can be provided accordingly.

As described above, the home network configuration of the present invention is described based on UPnP middleware. However, the home network of the present invention uses various home network middleware such as Jini, Open Service Gateway Initiative (OSGi), Home Audio / Video Interoperability (HAVi) Can be implemented.

Returning to FIG. 3, after the home network connection is performed in step 301 through the above-described procedure, if an external mobile terminal 101 executes an application for accessing the home network in step 302, Transmits a connection request to the gateway 103 of the home network. The access request may include authentication information (e.g., user authentication card information, user ID, password, etc.) of the user of the mobile terminal 101 and information of the mobile terminal 101.

4 is a diagram illustrating an example of a display of an application selection screen for controlling a home appliance of a home network system in a terminal in an operation of controlling the home appliance of the home network system according to an embodiment of the present invention. In step 302, the user can select and execute an application for accessing a home network displayed in an icon form on the screen as shown in FIG.

In step 304, the gateway performs an authentication process using authentication information for the user of the mobile terminal 101. In step 306, the gateway 103 provides list information of the home appliances 104 connected to the home network to the mobile terminal 101. In step 306, The list information of the home appliances 104 may include basic information such as the names and types of the home appliances 104. [

Next, the mobile terminal 101 displays a device list connected to the home network using the device list information received in step 307. 5 is a diagram illustrating a display example of a terminal when an application for controlling a home appliance of the home network system is selected on the screen of FIG. In step 307, the user can know the list of the devices 104 connected to the home network and displayed on the screen as shown in FIG. 5, and can select the specific device 104.

Meanwhile, the list of the home appliances 104 displayed in the application may be automatically added or deleted according to the home appliances 104 currently connected to the home network, and the icon shape may be set by the user. The user interface for displaying the status information of the home appliance 104 and the type and type of the control button can be received via the gateway 103 or directly from the home appliance 104 via the gateway 103 . Accordingly, when a new home appliance 104 connected to the home network exists (when the home appliance is not stored in the home network application of the storage unit) when the application is executed, the user displays the state information of the home appliance 104 and performs an operation A sub-application dedicated to the home appliance 104 including a user interface for controlling the home appliances 104 can be downloaded. These sub-applications can also be downloaded via the Internet.

If the user selects the specific device 104 in step 308, the gateway 103 transmits a signal requesting the current status information of the selected device 104 to the gateway 103 in step 309. The state information of the apparatus includes at least one of a power on / off state of the apparatus, a current operating state, a part or a material state. In step 310, the gateway 103 transmits a signal (UPnP action) for requesting status information to the selected device, and receives status information from the device 104 in step 311. In step 312, the gateway 103 transmits the received status information to the mobile terminal 101 via the external communication network. In step 313, the mobile terminal 101 transmits status information on the device selected by the user using the received status information And a user interface for performing a control operation such as a control button. The mobile terminal 101 transmits a control command to the gateway 103 in step 315 and the gateway 103 transmits the control command to the gateway 103 in step 316. In step 316, (UPnP action) to the device 104 so that the device 104 performs an operation according to the control command. In step 317, the home appliance 104 transmits the operation result to the gateway 103 in response to the control command. In step 313, the gateway 103 transmits a control command response to the mobile terminal 101.

6, 7, 8, and 9 show examples of screens in which the user selects one of the devices connected to the home network in step 313. Fig. 6 is a diagram showing a display example of a terminal when a TV is selected on the screen of Fig. 5. Fig. As shown in FIG. 6, the user can easily know status information such as whether the power of the TV is currently turned on, the selected channel, the usage time, the volume, etc. through the status information window, / Off (on / off), use lock, reserved recording, and so on.

7 is a diagram showing an example of display of a terminal when a security camera is selected on the screen of FIG. As shown in FIG. 7, the user can know the current shooting screen of the streamed security camera through the status information window, turn on / off the power through the control button located below the status information window, Can be performed remotely.

Fig. 8 is a diagram showing a display example of a terminal when lighting is selected on the screen of Fig. 5. Fig. As shown in FIG. 8, the user can know the state of the lighting of the current house through the status information window, and can directly touch the status information window, turn on / off each illumination through different operation buttons or user interface, Can be set.

9 is a diagram showing a display example of a terminal when a printer is selected on the screen of Fig. As shown in Fig. 9, the user can grasp the power on / off state of the printer, the output state, the state of the cartridge or the paper, and can control power on / off or output.

In addition, although not shown in the drawings, even in the case of a refrigerator, it is possible to grasp the temperature and the set temperature of the present refrigerator, the amount of food stored, the presence of ice, etc. with respect to the reference amount of food through the status display window, You can adjust the set temperature or power on / off through user interface such as touch screen.

Also, in the case of a washing machine, state information such as the current state of the washing machine, the amount of laundry, the start / end time and the time taken can be displayed through a status display window. Can be controlled.

In the case of an air conditioner or boiler, status information such as current temperature, set temperature, reservation setting, etc. can be displayed through the status display window and the set temperature or reservation setting can be performed through a user interface such as a control button or touch screen have.

As described above, the operation and configuration of a method and apparatus for controlling a home network system using a mobile terminal according to an embodiment of the present invention can be performed. While the present invention has been described with reference to specific embodiments, Various modifications may be made without departing from the scope of the present invention.

Claims (11)

A method for controlling at least one electronic device in a home network system in a home network system configured using UPnP (Universal Plug and Play) middleware,
Executing an application for controlling electronic devices connected to a home network;
Accessing a gateway of the home network based on the executed application;
Receiving a device list for electronic devices connected to the home network from the gateway;
Displaying the electronic devices based on the received device list on a touch screen of the mobile terminal in an icon form;
Requesting the first electronic device through the gateway based on the application status information of the first electronic device when an icon for the first electronic device is selected based on a touch input among the displayed icons;
A first user interface including at least one control button for controlling different operations according to the status information of the first electronic device received from the first electronic device and the type of the first electronic device, Wherein the at least one control button is displayed under the status information of the first electronic device,
Transmitting a control command to the first electronic device through the gateway when a control command for controlling the first electronic device is input through the first user interface displayed;
Receiving a control result in response to the control command from the first electronic device that has received the control command, and displaying the control result on the touch screen;
And displaying the electronic devices based on the received device list on the touch screen of the mobile terminal in the icon form,
Determining whether a second electronic device that is not controlled by the application exists among the electronic devices;
Receiving, via the gateway, a sub-application including a second user interface for displaying status information of the second electronic device and controlling operation thereof if the second electronic device is not controlled by the application;
And displaying an icon for the second electronic device included in the received sub-application on the touch screen,
Wherein the first user interface is received from the first electronic device or the gateway.
The method according to claim 1,
Further comprising the steps of: monitoring the status information and operation of each of the electronic devices using the application after accessing the home network using the application.
delete The method according to claim 1,
Wherein the status information of the first electronic device comprises:
Different from each other depending on the type of the first electronic device, including at least one of a power on / off state, a current operating state, a part or a material state.
The method according to claim 1,
And displaying the status information of the first electronic device and the first user interface for controlling the first electronic device on the touch screen,
Displaying status information of the first electronic device on a status information window;
And displaying the first user interface under the status information window including the at least one control button for controlling the first electronic device.
1. A home network system configured using Universal Plug and Play (UPnP) middleware, the home network system comprising: at least one electronic device in the home network system;
A communication unit for receiving a device list of electronic devices connected to the home network from a gateway of the home network;
A touch screen for displaying the device list,
An application for controlling electronic devices connected to the home network, accessing a gateway of the home network based on the executed application, receiving a device list of electronic devices connected to the home network from the gateway, Displays the electronic devices based on the received device list on the touch screen of the mobile terminal in the form of an icon, and when an icon for the first electronic device is selected based on the touch input among the displayed icons, Request status information of the device to the first electronic device via the gateway based on the application and determine the status of the first electronic device based on the status information of the first electronic device received from the first electronic device and the type of the first electronic device At least one < RTI ID = 0.0 > A first user interface including a first user interface and a second user interface on the touch screen, and when a control command for controlling the first electronic device is input through the first user interface displayed, And a control unit for transmitting the control command to the first electronic device and receiving a control result from the first electronic device that has received the control command in response to the control command and displaying the control result on the touch screen,
Wherein the controller determines whether a second of the electronic devices is not controlled by the application and if the second electronic device that was not controlled by the application is present, Receiving, via the gateway, a sub-application including a second user interface for control of display and operation, displaying an icon for the second electronic device contained in the received sub-application on the touch screen,
Wherein the at least one control button is displayed under the status information of the first electronic device,
Wherein the first user interface is received from the first electronic device or the gateway.
The method according to claim 6,
Further comprising a storage unit for storing the application for controlling the state of the electronic devices and for controlling each of the electronic devices.
8. The method of claim 7,
Wherein the control unit controls the observation and operation of the status information of each of the electronic devices after the connection to the home network.
delete 7. The method of claim 6, wherein the status information of the first electronic device comprises:
Wherein the mobile terminal comprises at least one of a power on / off state of the first electronic device, a current operating state, a part or a material state.
The touch screen of claim 6,
Displaying the status information of the first electronic device in a status information window and displaying the first user interface including the at least one control button for controlling the first electronic device under the status information window, .

Images