KR102219982B1 - Gateway and method for controlling thereof - Google Patents

Abstract

The present invention relates to a home network, and more particularly, to a meta-information management structure for connecting and controlling various devices using a gateway, a gateway device using the same, and a control method thereof. A gateway according to an embodiment of the present invention includes: a communication unit for exchanging signals with at least one external device by wire/wireless; Memory; And establishing a data path through the at least one external device and the communication unit, obtaining information on each of the at least one external device through the data path, and converting the obtained information into a tree of a structure preset in the memory. It may include a control unit to store. At this time, the tree is a top-level item and includes at least one item of a device, a device function, a location, a user, a service, a device adapter, and a gateway, and is managed in the middleware platform layer and is referenced in the service layer, and the control unit includes the at least one The operation of the at least one external device may be managed according to any one of a plurality of states differently defined according to an operation state of the external device of and a user's location.

Description

Gateway and its control method {GATEWAY AND METHOD FOR CONTROLLING THEREOF}

The present invention relates to a home network, and more particularly, to an information management structure for connecting and controlling various devices using a gateway, a gateway device using the same, and a control method thereof.

With the recent spread of smart-related technologies, the construction of smart home technology that supports linking functions between home appliances is drawing attention in the home. In order to build a smart home, the introduction of a home gateway that supports various wired/wireless connectivity may be considered to enable interoperability and management of other devices.

However, in the case of using a gateway, it is desirable to standardize the platform in order to control various home appliances from various manufacturers and perform a linkage function through the control. This is because when the gateway's Metadata Management structure is standardized, the linkage operation between various devices of different manufacturers is guaranteed, and service development is also facilitated.

The present invention is to provide a gateway for providing a more convenient home network service.

In particular, the present invention is to provide a meta-information management structure of a gateway for connecting and controlling various external devices.

In addition, the present invention is to provide a method of performing a service requiring participation and control of a plurality of devices in execution by using a meta information management structure.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present invention belongs from the following description. I will be able to.

A gateway according to an embodiment of the present invention for realizing the above problems includes: a communication unit for exchanging signals with at least one external device by wire/wireless; Memory; And establishing a data path through the at least one external device and the communication unit, obtaining information on each of the at least one external device through the data path, and converting the obtained information into a tree of a structure preset in the memory. It may include a control unit to store. At this time, the tree is a top-level item and includes at least one item of a device, a device function, a location, a user, a service, a device adapter, and a gateway, and is managed in the middleware platform layer and is referenced in the service layer, and the control unit includes the at least one The operation of the at least one external device may be managed according to any one of a plurality of states differently defined according to an operation state of the external device of and a user's location.

A method for controlling a gateway according to an embodiment of the present invention for realizing the above-described problem includes: establishing a data path with at least one external device through a communication unit; Obtaining information on each of the at least one external device through the data path; Storing the obtained information as a tree of a predetermined structure in a memory; And controlling a service accompanying the operation of the at least one external device by referring to the tree. Here, the tree includes at least one of a device, a device function, a location, a user, a service, a device adapter, and a gateway as a top-level item, and can be managed in the middleware platform layer and referenced in the service layer, and the controlling step May include managing the operation of the at least one external device according to one of a plurality of states differently defined according to an operation state of the at least one external device and a location of a user.

According to the present invention, more diverse and reliable services can be provided in a home network environment.

In particular, it is efficient because the gateway organizes and manages various meta information about external devices in a tree format.

In addition, control of a service in which a plurality of devices participate may be performed more efficiently.

The effects that can be obtained in the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those of ordinary skill in the art from the following description. will be.

1 shows an example of a hierarchical structure applied to a gateway according to an embodiment of the present invention.
2 shows an example of a top-level classification form of a meta information management tree according to an embodiment of the present invention.
3A to 3C each show an example of a sub-configuration of a device item of a meta information management tree according to an embodiment of the present invention.
4 shows an example of an operation process performed when an external device is connected to a gateway according to an embodiment of the present invention.
5A and 5B show examples of sub-structures of a device function item of a meta information management tree according to an embodiment of the present invention.
6A and 6B show an example of a sub-structure of a location item of a meta information management tree according to an embodiment of the present invention.
7 to 10 illustrate examples of sub-configurations of a user item, a service item, a device adapter item, and a customer premises device of a meta information management tree according to an embodiment of the present invention.
11 shows an example of a form in which items of a meta information management tree are referred to each other according to an embodiment of the present invention.
12 shows an example of an operation performed when services of two different users collide in a gateway according to an embodiment of the present invention.
13 shows an example of a remote door control using a plurality of gateways according to an embodiment of the present invention.
14 illustrates an example of a process in which one of the devices participating in service provision is changed when external devices are connected through a router, and FIG. 15 is a P2P method.
16 shows an example of a process in which a device participating in a mirroring service is changed according to an embodiment of the present invention.
17 illustrates another example of a process in which a device participating in a mirroring service is changed according to an embodiment of the present invention.
18 shows an example of a transition method and transition conditions between four states according to an embodiment of the present invention.
19 shows an example of interference caused by wireless signals between homes.
20 shows an example of a block diagram of a gateway applicable to embodiments of the present invention.

Hereinafter, a gateway related to the present invention will be described in more detail with reference to the drawings. The suffixes "module" and "unit" for components used in the following description are given or used interchangeably in consideration of only the ease of preparation of the specification, and do not have meanings or roles that are distinguished from each other by themselves. Further, the gateway referred to in the present invention may be referred to as a smart home gateway or a customer premises equipment (CPE). Of course, the gateway mentioned in the present invention is preferably installed indoors, but this is exemplary and is not limited to the installation location. In addition, in describing the embodiments disclosed in the present specification, when it is determined that a detailed description of related known technologies may obscure the subject matter of the embodiments disclosed in the present specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are for easy understanding of the embodiments disclosed in the present specification, and the technical idea disclosed in the present specification is not limited by the accompanying drawings, and all modifications included in the spirit and scope of the present invention It should be understood to include equivalents or substitutes.

In a general home network, devices are directly connected to perform a linkage function by exchanging signals, but in the present invention, each device is connected through a gateway to ensure connectivity and linkage function performance between various devices. It is suggested to be controlled through. In addition, the gateway proposes to manage meta information (executable functions, locations, states, etc.) of connected devices in common through tree-type meta information management (ie, meta data management tree). Here, the meta-information management tree may mean a kind of database and functions to define information to provide a standardized/generalized interface between a service and a middleware platform. Through this, a service developer can improve a service development environment by acquiring the structure and contents of information necessary for service execution based on the meta information management tree according to the present invention.

Hereinafter, a configuration of a gateway including a meta information management tree according to the present invention will be described with reference to FIG. 1.

1 shows an example of a hierarchical structure applied to a gateway according to an embodiment of the present invention.

Referring to FIG. 1, a service is provided in an application layer as a top layer, an interface and process and a meta information management tree are provided as a middleware platform layer in the middle, and a plurality of device adapters are provided as a driver layer in a lower layer.

Since a service has a function similar to a general application layer that controls application programs, redundant descriptions will be omitted for the sake of brevity of the specification.

The interface and process of the middleware platform layer are middleware for facilitating communication between a service and a device adapter connected to an external device, and may operate with reference to a meta information management tree.

In addition, the device adapter may functionally include a driver and a semantic conversion unit. The driver provides connectivity with various different devices, and the semantic conversion unit converts event-related signals from connected devices into signals that can be identified by the gateway internal application. You can perform the function to convert.

On the other hand, in the block of devices to be connected under the driver layer (not shown), the definition of meta information is required to provide information required by the middleware platform and service. This definition is the same as or at least the same as the meta information management tree structure described above. It is desirable that a part is stored and managed in the device as a list/database in the form of a modified structure.

Hereinafter, a detailed classification of the meta information management tree according to the present invention will be described with reference to FIG. 2.

2 shows an example of a top-level classification form of a meta information management tree according to an embodiment of the present invention.

Referring to FIG. 2, the top classification of the meta-information management tree includes device, device function, location, user, service, device adapter, and connectivity ( connectivity) and customer premises equipment (CPE, that is, gateway). Here, each item may be classified into a group 210 for interoperability between a service and a middleware platform and a group 220 for analysis/implementation between each service according to its purpose. Here, a device may be replaced with a device set, and a device function may be replaced with a semantic.

Hereinafter, each item belonging to the highest tree will be described in more detail with reference to FIGS. 3 to 10.

3A to 3C each show an example of a sub-configuration of a device item of a meta information management tree according to an embodiment of the present invention.

Referring to FIG. 3A, a device deals with a classification system for a device model actually connected to a gateway. Referring to Figure 3, the device can be divided into brown appliances, white appliances, lighting, CPE, etc., and the lowest attribute is Owner, Priority, Supported device function or Supported semantic ), cluster, etc. may be included.

The owner defines the ownership of the product, and the priority defines the ranking of primary/secondary, etc. for a product family (eg, TV). In addition, the support function definition is a definition of the function supported by each product group and indicates whether only the required items or optional specifications are supported. Among the functions supported by each product group, which function is required/optional is preferably determined in the form of a function profile in advance, and the supported function definition item at the end of the tree for each device can refer to this function profile. .

An example of such a functional profile is shown in Table 1 below.

Funtion Set Get Event Support start O M stop O M target temperature O O M wind speed O O M wind swing O O M operation fail O M filter change alert O M operation mode O O O special function O O O target humidity O O O smart energy operation O O O current temperature O O operation status changed O O smart meter O O O O motion detector O O O O Etc .

Referring to Table 1, it may be indicated whether a parameter that can be set for each function of a refrigerator, whether a value can be obtained, or whether it is required support. For example, in FIG. 3, that the attribute of the supported device function is required is that the M item is supported in the support of Table 1, and the optional specification is that the O item is supported in the support. Therefore, it is desirable to define support for each device in advance as shown in Table 1. Of course, unlike Table 1, instead of item division of M/O, item division of S (supported)/N (Not supported) may be used.

This is because it is not easy for service developers to research and develop all function support items for each device manufacturer, so it is easy to define the required/optional specifications. Of course, the middleware platform is preferably capable of discovering an option item between a service and a driver of the device adapter, that is, performing function discovery.

Meanwhile, a cluster can be applied when a device is classified into a cluster in advance for a situation in which several devices operate together, such as a light fixture.

Alternatively, as shown in FIG. 3B, a unique name of the device, a vendor, an internal resource used (for example, a tuner in the case of a TV), a location, and the like may be additionally included. In addition, user priority and service priority may be set respectively.

Meanwhile, in the case of FIG. 3B, a gateway (CPE) may be included in a sub-item of the stereo device item, which will be described in more detail later with reference to FIG. 9C.

In addition, as shown in FIG. 3C, user priority and service priority may be set for each utilized internal resource item. In this case, more detailed resource management is performed. It is possible.

Next, an example of a process of performing the above-described discovery of whether the required/optional is required will be described with reference to FIG. 4.

4 shows an example of an operation process performed when an external device is connected to a gateway according to an embodiment of the present invention.

Referring to FIG. 4, first, as 1) an external device is connected to a gateway, information related to a corresponding device may be stored in a meta information management tree. At this time, whether or not a supported device function is required/optional for the corresponding device may be set together. 2) The service reads connected device information from the meta-information management tree and performs function discovery on the item if there is an optional function. 3) To this end, the service transmits a function discovery request message to the device adapter, and the device adapter provides service through a function discovery response message for optional items of the connected device based on the received message. You can reply with At this time, if the optional item of the connected device is also defined in the device in a form similar to Table 1, it may be used.

5A and 5B show examples of sub-structures of device function items of a meta information management tree according to an embodiment of the present invention, respectively.

Referring to FIG. 5A, a device function may be in the form of listing functions of devices. By the combination of these device functions, the entire function of each device can be expressed, and whether the required/optional specifications of each device function can be defined similarly to Table 1. The device function may include items such as on, off, display, input, broadcasting, web browser, TV program, and energy measurement as sub-items. Here, as for the input item, sub-items may be configured for each input means (eg, basic key, mouse, keyboard, voice recognition, etc.). The device function may be linked with an open API (Open Application Programmer Interface) provided by the gateway, and in FIG. 5A, a basic key input type of input is further illustrated as an example. Meanwhile, to express this structure, a language/method for describing a general tree structure such as XML, oBIX schema, SOAP, etc. can be used.

Alternatively, as shown in FIG. 5B, the sub-items of the device function item may be configured to specify the device and the function together. For example, TV on/off and TV key input may each be configured as one sub-item. In addition, each sub-item may be specified as to whether it is required/optional for each input means.

6A and 6B show an example of a sub-structure of a location item of a meta information management tree according to an embodiment of the present invention.

In order to provide a smart home service, it is desirable to provide location information of each device, and for this purpose, a classification as shown in FIG. 6A may be considered. Specifically, there may be classifications by location, indoor/outdoor, floor, and room from top to bottom, and each room has a name, common/personal space, owner (or user), etc. Can have properties for

Alternatively, as shown in FIG. 6B, an installed device attribute may be added to the attribute of each lowest position item (for example, an indoor room or an outdoor area).

Such a classification system can be particularly useful when collectively controlling devices in a specific location/space.

7 to 10 show examples of sub-configurations of a user item, a service item, a device adapter item, and a customer premises device of a meta information management tree according to an embodiment of the present invention.

7 to 10, when user, service, device adapter, and CPE information are provided, resources are shared when independent services operating on the gateway are simultaneously executed. Problems that may be caused by can be resolved through priorities, owners, notifications, and collision avoidance procedures.

In the case of a user item, each attribute value may be arranged in parallel under the user item as shown in FIG. 7A, and as shown in FIG. 7B, the family and guest are first classified, and the name, current location, and available device for each member , Usable service, ID, etc. attributes may be assigned.

In the case of a service item, as shown in FIG. 8A, request meta information, request device, category, service name, version information, etc. may be configured as attribute values of each service. Alternatively, as shown in FIG. 8B, a required function/Required semantic item defining what function is required for service execution in the attribute value of each service or a required resource item for checking the occurrence of a collision with another service, A Resource Access Priority item that defines the priority of resource use for each service may be additionally included.

In the case of a device adapter, as shown in (a) of FIG. 9A, it may have properties such as a type, a supported device, and a firmware/platform version.

In the case of the gateway, as shown in (b) of FIG. 9A, a type attribute indicating whether or not the standard is compatible/non-compliant and a connected devices attribute indicating a device connected thereto may be configured.

On the other hand, when a device (e.g., stereo headphones) connected to a non-compliant CPE participates in the configuration of a smart home network, the non-compliant CPE is recognized as a sub-device of the standard CPE and A home network can be configured. In other words, a non-compliant CPE provider can perform the same role as a proxy by supplying a device adapter to the CPE. At this time, necessary meta information is reflected in the entire meta information management tree, Interconnectivity for connected devices may be provided. At this time, if all rights are granted to non-compliant CPEs, the discrimination against compatible devices may disappear. Therefore, limited meta-information is limited to the non-compliant devices (lower connected devices of Non-conpliant CPE). Can be provided.

In addition, the incompatible CPE item may be located in a lower item of the device item as shown in FIG. 9B, and the stereo headphones may be included in the tree as a sub device set item of the corresponding CPE. Accordingly, the stereo headphone item can be managed by assigning attribute values such as a user, priority, and support function, like other devices.

Alternatively, as shown in FIG. 9C, stereo headphones may be registered as a sub-item of the device item, and information of incompatible CPE may be configured as a property of the stereo headphone item.

2 and 10, the connectivity item is located at the top of the tree, but this item does not necessarily need to be located at the top level, and may be located in a subtree of other items.

11 shows an example of a form in which items of a meta information management tree are referred to each other according to an embodiment of the present invention.

Referring to FIG. 11, two or more top-level items in a tree may refer to each other. For example, when a device and a function item are referenced together, meta information called supported semantics for each device may be configured. As another example, when a function item and a service item are referred to together, meta information called a requested semantic may be configured.

Hereinafter, with reference to FIGS. 12 and 13, specific functions that can be performed by the gateway using the meta information management tree described above will be described.

12 shows an example of an operation performed when services of two different users collide in a gateway according to an embodiment of the present invention.

In FIG. 12, it is assumed that the terminal 1110 of user A, the terminal 1140 of user B, and the TV 1130 are connected to the gateway 1120 according to the present invention after registration. That is, it is assumed that information of each device has already been registered in the meta information management tree of the gateway 1120 through the above-described procedure such as function discovery. For connection and message exchange between devices, the TV 1130 may directly communicate with the gateway 1120 through wired/wireless. In the case of the terminals 1110 and 1140, they may be directly connected to a gateway in a local, or may be connected to a gateway through the Internet in a remote situation. In addition, it is assumed that the priority of user A is higher than that of user B.

Referring to FIG. 12, first 1) User A sends a reservation recording for the TV 1130 (for example, channel 200 at 11 o'clock on May 28, 2013, 1 hour program) to the gateway 1120 through the terminal 1110. Can be requested. 2) Accordingly, the gateway 1120 may set a reservation recording function on the TV 1130 by referring to a device, a device function, a location, a user, and a service item in the meta information management tree. For example, the gateway 1120 may inquire the priority of user A in the user item and check the location of the TV in the location item. In addition, the gateway may check whether the TV 1130 supports the reservation recording function in the device function item and whether the TV 1130 supports recording, and obtains information related to the reservation recording from the service.

3) On the other hand, user B may also request a reservation recording for the TV 1130 through the terminal 1140 (eg, channel 210 at 10 o'clock on May 28, 2013, a 2-hour program) from the gateway 1120. 4) When the reservation recording requested by User B overlaps the reservation recording requested by User A and the channel is different, the gateway can transmit the corresponding contents to the terminal 1140 of user B by referring to the information registered in the meta information management tree. have. 5) As user A's priority is set high, user B requests user A whether or not it is possible to change the scheduled recording, and 6) user A requests user B's request based on the information in the meta information management tree. It may be received through the gateway 1120. 7) Accordingly, User A can deliver decisions such as acceptance/rejection to User B, and 8) User B can receive decisions such as acceptance/rejection from User A. 9) If user A accepts, the reservation recording request desired by user B is transmitted to the gateway 1120 again, and 10) gateway 1120 can transmit information on the reservation recording request of user B to the TV 1130. have.

Of course, even without steps 9) and 10), depending on settings, reservation recording may be set according to the information requested in step 3) by user B only with user A's acceptance.

Next, a case in which a service using a plurality of gateways is performed will be described with reference to FIG. 13.

13 shows an example of a remote door control using a plurality of gateways according to an embodiment of the present invention.

In FIG. 13, two gateways are used, one of which 1221 is provided in the user's home and the other 1223 is provided in the user's office, and is assumed to be connected to each other through the Internet. In addition, it is assumed that the entrance door lock system 1210 is connected to the home gateway 1221 and includes a doorbell, a door locking device, and a camera. In addition, it is assumed that the user's terminal 1230 is directly connected to the office gateway 1223 through wired/wireless.

Referring to FIG. 13, first 1) As a visitor presses the doorbell at the entrance of the house, the gateway 1221 provides device, location, and service information for the door lock system 1210 stored in the meta information management tree as a service layer (application). Is delivered. 2) Accordingly, the gateway 1221 can operate the camera using the device and location information. 3) In addition, the gateway 1221 inquires the user's location information using the user and location information. 4) As a result, if the location of the user is confirmed as the office, the gateway 1221 may perform service interworking with the gateway 1223 provided in the office using the customer's CPE item and service information. 5) The gateway 1223 provided in the office calls the user's terminal 1230 using the information of the device item, and 6) transmits and receives data between the camera 1210 and the terminal 1230 using the information of the device adapter item. You can determine the best video codec by checking the speed. 7) Thereafter, using the device function information, the home gateway 1221 may transmit the screen and audio of the entrance camera to the office gateway 1223 so that the user and the visitor can make a video call. 8) Here, when the user inputs a door open command through the terminal, the home gateway 1221 may release the door lock and stop the camera using the device and device function information.

Hereinafter, a service control method according to the present invention will be described when a plurality of devices are involved in executing one service based on the above-described gateway.

In particular, in the present embodiment, a service control method is provided when a replacement of some devices is required in a situation in which a plurality of devices are interlocked to operate to provide one service.

Here, service control is performed through a gateway, but direct data exchange between devices participating in the service provision may be performed through a router, or through a peer-to-peer (P2P) method without passing through a router. It can also be done. Regardless of the name, the router is not limited to any method or type as long as it can provide a path for data exchange between two or more devices in a wired or wireless manner.

14 illustrates an example of a process in which one of the devices participating in service provision is changed when external devices are connected through a router, and FIG. 15 is a P2P method.

In FIGS. 14 and 15, it is assumed that the device 2 is replaced by the device 3 while one service is being executed through the device 1 and the device 2. Here, it is assumed that the subject of the service (ie, a master device that provides content of a service or performs control during execution) is device 1.

First, referring to FIG. 14, device 1 and device 2 are each connected to both a router and a gateway, and when a service is started according to a preset procedure (standard service session start), device 1 and device 2 Service-related data is exchanged through a router.

As a specific service is executed, Device 1 and Device 2 report the information (ie meta information) to the gateway (①, ②). At this time, the ② process can be performed selectively. That is, all devices associated with the service may report to the gateway, or the master device (ie, device 1) may report to the gateway together with information of other devices (ie, device 2).

The meta data reported here are Session state, Connectivity name, Connectivity service category, Channel, Service contents, and Device type. ), device address, device role, etc. Among these, the device type, device address, and device role may be information already held in the gateway. In this case, the corresponding information may be managed by the gateway in the form of the meta information management tree described above.

After that, device 3 is turned on, but it can report a request for a specific service to the gateway (③). The meta information reported here may include connectivity capability, connectivity service capability, device type, device address, device role, etc., and such information It can also be managed in the gateway in the form of a meta information management tree.

The gateway can select a service item that can be converted to device 3 among the currently running services based on the meta information reported in the process ③ (④).

Meta information on the selected service can be reported to device 3 (⑤). Meta information reported here may include a connection name, a connection service category, a channel, a device type, a device address, a device role, and the like.

Using the received metadata, device 3 can determine whether to participate in the service (⑥). For example, it is possible to visually output metadata through a display means provided in the device 3, and thereby help a user to make a decision.

As the user determines the service participation of the device 3 (ie, the device 3 instead of the device 2), a service change request message may be transmitted from the device 3 to the gateway (⑦). In this case, the service change request message may include meta information such as a connection name, a connection service category, a channel, a device type, a device address, and a device role.

Upon receiving the change request message, the gateway may transmit a control command to device 1 and device 2 connected to the corresponding service item to change the service configuration (⑧, ⑨). In this case, the control command may include a connection name, a connection service category, and a target device address.

Thereafter, the service session between the device 1 and the device 2 is stopped, and the device 3 may participate in the service execution on behalf of the device 2.

Meanwhile, in the case of FIG. 15, since the process of FIG. 14 is similar to the process of FIG. 14, except that service-related data is directly exchanged between devices without going through a router, a redundant description will be omitted for simplicity of the specification.

Specific examples of each meta-information in the above-described processes are as follows.

-Connectivity name: WiFi, WiFi Direct, TDLS (Tunneled Direct Link Setup), Bluetooth, Bluetooth Low Energy (LE), etc.

-Connectivity service category: Miracast, DLNA, A2DP (Advance Audio Distribution Profile), etc.

-Service contents: video, audio, image, screen, etc.

-Device type: TV, mobile terminal, network attached storage (NAS), etc.

-Device role: source/sink, master/slave, digital media server (DMS), digital media player (DMP), digital media DMR (digital media renderer), digital media controller (DMC), etc.

-Session state: Ongoing, expiration, etc.

Hereinafter, a more specific example will be described of the above-described device change procedure with reference to FIGS. 16 and 17.

16 shows an example of a process in which a device participating in a mirroring service is changed according to an embodiment of the present invention.

In FIG. 16, it is assumed that an image displayed through a terminal (e.g., a smartphone) is mirrored through a living room TV, and mirroring continues through the bedroom TV as the user moves to the bedroom with the terminal. do. Here, mirroring refers to a service that outputs a screen displayed on a terminal as it is through an external display, and of course, it can be performed through various protocols (eg, Miracast, DLNA, etc.). In addition, it is assumed that the living room TV, the bedroom TV, and the terminal are already registered in the gateway and are being connected or can be connected at any time.

Referring to FIG. 16, first, a mirroring service is started so that the screen of the terminal may be mirrored to the living room TV (①).

Accordingly, the terminal may report meta information related to the mirroring service (eg, session state, connection name, connection service category, service content, channel, device type, device address, device role, etc.) to the gateway (②). .

While mirroring is in progress, the user can move to the bedroom with the terminal (③).

As the user who has moved to the bedroom turns on the bedroom TV (④), the bedroom TV can report the power-on status to the gateway (⑤).

When there is no terminal of another user connected to the gateway, it can be determined that only the user exists in the house, and when the bedroom TV is turned on, it can be determined that the user has moved to the bedroom. Meanwhile, the gateway acquires the device function of the bedroom TV at the initial connection and manages it in the form of a meta information management tree. Accordingly, the gateway can extract a service that can be switched from a living room TV to a bedroom TV and transmit meta information related thereto to the bedroom TV (⑥).

The bedroom TV may output a user interface for receiving confirmation from the user whether to change the output device of the mirroring service being executed from the living room TV to the bedroom TV to the user by using the received meta information (⑦).

Here, when the user selects to change to the bedroom TV, the bedroom TV may transmit a mirroring scene change request message to the gateway (⑧).

The gateway may transmit a mirroring scene change command message to the terminal according to the request message (⑨). In this case, the mirroring change command message may include meta information such as the address, type, and capability of the bedroom TV to indicate that the role of the living room TV is replaced by the bedroom TV.

The terminal receiving the mirroring change command message can change the output device of the mirroring service to the bedroom TV (⑩). Of course, in this process, a user interface for receiving user confirmation may be provided through the display of the terminal.

When the conversion from the living room TV to the bedroom TV is completed, the terminal can report the completion status of the mirroring conversion to the gateway (⑪).

As the conversion completion is reported, the gateway can instruct the living room TV to power off (⑫). In this case, the living room TV may be immediately turned off, or information on a turn-off notice may be output on the screen through a predetermined user interface. In this case, if there is no user input for a certain period of time, the living room TV may be turned off.

17 illustrates another example of a process in which a device participating in a mirroring service is changed according to an embodiment of the present invention.

In FIG. 17, while content stored in a network attached storage (NAS) is transmitted to a living room TV through a wireless router (AP) and played (i.e., a share scene), the bedroom TV as a user moves to the bedroom. Assume a situation in which playback of the corresponding content continues. In addition, it is assumed that the user is wearing Bluetooth headphones and can be paired with both a living room TV and a bedroom TV.

Referring to FIG. 17, a Bluetooth earphone is connected to a living room TV and sound from the living room TV is output through the Bluetooth earphone, and as specific content stored in the NAS is transmitted to the living room TV through the AP, the corresponding content is output through the living room TV. Can be (①).

The living room TV accordingly includes meta information related to content playback (e.g., session state, connection name, connection service category, service content, channel, device type, device address, device role, etc.) and device connection state information (i.e., Bluetooth The device type, connection name, connection service category, device address, device role, etc. for headphones can be reported to the gateway (②).

During content playback, the user can move to the bedroom while wearing the Bluetooth headphones (③).

As the user who has moved to the bedroom turns on the bedroom TV (④), the bedroom TV can report the power-on to the gateway (⑤).

Accordingly, the gateway may determine that the user has moved to the bedroom. Meanwhile, the gateway acquires the device function of the bedroom TV at the initial connection and manages it in the form of a meta information management tree. Accordingly, the gateway can extract a service that can be switched from a living room TV to a bedroom TV and transmit meta information related thereto to the bedroom TV (⑥).

The bedroom TV may output a user interface for receiving confirmation from the user whether to change the output device of the content sharing service being executed from the living room TV to the bedroom TV to the user by using the received meta information (⑦).

Here, when the user selects to change to the bedroom TV, the bedroom TV may transmit a share scene change request message to the gateway (⑧).

The gateway may transmit a share scene change command message to the NAS and the bedroom TV according to the request message (⑨). In this case, the share change command message transmitted to the NAS may include meta information such as the address, type, and capability of the bedroom TV. In addition, the sharing change command message transmitted to the bedroom TV may include connection information for Bluetooth headphones.

Upon receiving the mirroring change command message, the NAS can change the output device of the shared service to the bedroom TV (⑩). In addition, the bedroom TV can be paired with Bluetooth headphones so that sound is output through the Bluetooth headphones (⑪). At this time, the Bluetooth headphones may be disconnected from the living room TV.

When the conversion from the living room TV to the bedroom TV is completed, the bedroom TV can report the completion status of sharing conversion to the gateway (⑫).

As the conversion completion is reported, the gateway can instruct the living room TV to power off (⑬). As described above, the living room TV may be immediately turned off, or information on a turn-off notice may be output on the screen through a predetermined user interface. In this case, if there is no user input for a certain period of time, the living room TV may be turned off.

According to the above-described FIG. 16, even if the gateway does not directly support Bluetooth, since the pairing can be controlled through another device, there is an advantage in that it is possible to collectively change a service for a plurality of devices participating in the Haeda service.

In addition, in the above-described embodiments, the movement of the user was determined by whether or not there was a device that was newly powered on at a location different from the location of the existing device, but this is an example and the location of the user such as a motion sensor or a camera If possible, the method through any device is not limited.

Operation state definition

Meanwhile, according to an embodiment of the present invention, it is proposed to define a state according to the operation status of devices connected to the gateway and whether the user is local/remote, so that the gateway or the server performs management according to the status. Through this, response to an emergency situation, energy saving, signal interference reduction, and convenience can be provided.

The states according to the present embodiment may include an active state, a park state, a sleep state, and an emergency state. Hereinafter, each state will be described in detail.

First, the active state will be described.

The active state means a state in which the device is operating, and the active state may be divided into a local active and a remote active.

It is possible to know the active state by a situation in which the gateway or the server sends a command to the device or receives an event for an operation from the device and the operation time of the device expected by the command/event issuance.

The local and remote classification criteria may be classified according to whether a delivery path of commands and events is in a local area or a remote area. Additionally, whether the user's master user device has been found locally and whether the user is in the house may be additionally determined by context aware/motion sensing.

In terms of operation, the local active state allows the user to use all functions supported by the device. In the case of the remote active state, the functions of the device may be limited. Limited functions can be pre-selected by the device manufacturer. For example, functions that may be dangerous when applying remotely, such as starting cooking of an oven, can be set in advance so that they are only available locally.

Next, the parked state will be described.

The park state may mean a state in which there is no change in operation of devices other than devices related to security and safety or devices that are always turned on, and a state in which there is no schedule for operation is certain to continue.

The method of determining such a state may be a situation in which it is determined that the user is not in the house, or when a command to enter the state is given by the user regardless of the operating state of the device. The determination of the user's location is as described above in the active state.

In terms of operation, in the park state, devices and services related to security/safety are activated by a command of a gateway or a server, and power of devices other than those that are always on are turned off. In this case, the energy saving function may be activated even in a device that is always turned on. In the case of transitioning from this state to another state, it is preferable that metadata on the operation state before entering the present state is stored in advance for retention.

Next, the sleep state will be described.

The sleep state may mean a state in which it is uncertain whether there is a user in the house when a device other than a device related to security and safety or a device that is always turned on does not operate continuously for a predetermined time or longer.

In terms of operation, in the sleep state, sleep notifications are transmitted to devices other than devices related to security/safety by a command of a gateway or a server. When the sleep notification is delivered, the device performing the periodic operation may change the period to be longer. In addition, a device that is always turned on upon receiving a sleep notification may be activated if there is an energy saving function.

Next, the emergency state will be described.

The emergency state may mean a state in which an event is detected by a device or service related to security or safety.

In terms of operation, each device receiving an emergency notification according to a command from a gateway or a server may stop an operation being performed and perform a preset emergency operation. Through this, users can be informed of an immediate situation and a warning message can be sent to intruders or neighbors.

The relationship between the above four states is shown in FIG. 18.

In Fig. 18, the transition method and condition between the states are indicated in the direction of the arrow. The circular area in the state diagram categorizes services running in the house and indicates in which state each service category operates. Of course, these services can change depending on the devices installed in the house. In addition, security/safety and always-on devices/services are shown to always operate in all states. On the other hand, the sleep and park states contribute to energy saving and interference reduction, and the emergency mode service operates in an emergency state and can perform a predetermined service with a higher priority than all other operations.

The above-described state management method can be applied to control interference problems caused by wireless signals between homes.

19 shows an example of interference caused by wireless signals between homes.

As shown in FIG. 19, when devices complying with various protocols such as WiFi, Zigbee, KNX, and eQ3 are installed in a home for smart home service, interference by wireless signals may be increased not only within homes but also between homes. Accordingly, it is possible to prevent wireless signals from being transmitted in unnecessary situations through a sleep state or a park state. Of course, it is possible to adjust the period of a heartbeat detection for awakening from this state to an active state, or a Simple Service Discovery Protocol (SSDP) advertisement for device discovery. Of course, FIG. 19 is an example, and it goes without saying that the wireless signal propagation coverage may vary according to the connectivity standard.

Meanwhile, the aforementioned Home State Management function may be implemented in a system service of a gateway. The home state management service can check the status of in-house devices through a meta-information management function, control other services in the system service layer and services in the application layer, or provide feedback.

Hereinafter, a mechanical configuration of a gateway capable of performing the above-described functions will be described with reference to FIG. 20.

20 shows an example of a block diagram of a gateway applicable to embodiments of the present invention.

The gateway 100 may include a wireless communication unit 110, a user input unit 120, an output unit 150, an interface unit 160, a memory 170, a control unit 180, a power supply unit 190, and the like. have. The components shown in FIG. 17 are not essential in implementing the gateway, and thus the gateway described in the present specification may have more or fewer components than the components listed above.

More specifically, among the components, the wireless communication unit 110 may include a gateway 100 and an external device, a gateway 100 and another gateway 100, or a gateway 100 and another gateway 100, or an external server. ) May include one or more modules to enable wireless communication between the located networks.

The wireless communication unit 110 may include at least one of a broadcast reception module 111, a mobile communication module 112, a wireless Internet module 113, a short-range communication module 114, and a location information module 115. .

The user input unit 120 may include a touch key, a mechanical key, or the like for receiving a control command from a user.

The output unit 150 is for generating an output related to visual, auditory or tactile sense, and may include at least one of a display unit 151, an audio output unit 152, and a light output unit 153. The display unit 151 may implement a touch screen by forming a layer structure or integrally with the touch sensor. Such a touch screen may function as a user input unit 123 that provides an input interface between the gateway 100 and a user, and may provide an output interface between the gateway 100 and a user.

The interface unit 160 serves as a path to various types of external devices connected to the gateway 100. The interface unit 160 connects a wired/wireless headset port, an external charger port, a wired/wireless data port, a memory card port, and a device equipped with an identification module. It may include at least one of a port, an audio input/output (I/O) port, an input/output (video I/O) port, and an earphone port. In response to the connection of the external device to the interface unit 160, the gateway 100 may perform appropriate control related to the connected external device.

The memory 170 may store a plurality of application programs (application programs or applications) driven by the gateway 100, data for operation of the gateway 100, commands, and the above-described meta information management tree. . At least some of these application programs may be downloaded from an external server through wireless communication. In addition, at least some other of these application programs, the gateway 100 from the time of shipment for basic functions of the gateway 100 (for example, providing wired/wireless connectivity with external devices, detection and registration of nearby connectable devices). May exist in the phase. Meanwhile, the application program may be stored in the memory 170, installed on the gateway 100, and driven by the controller 180 to perform an operation (or function) of the gateway.

In addition to the operation related to the application program, the control unit 180 generally controls the overall operation of the gateway 100. The controller 180 processes signals input or output through the above-described elements, data, and items of the meta information management tree, or drives an application program stored in the memory 170 to provide appropriate information or functions to the user or You can handle it.

In addition, in order to drive the application program stored in the memory 170, the controller 180 may control at least some of the components described with reference to FIG. 20. Furthermore, in order to drive the application program, the controller 180 may operate by combining at least two or more of the components included in the gateway 100 with each other.

Specifically, the control unit 180 establishes a data path with the external device through the wireless communication unit 110 or the interface unit 160, and when the external device is connected through this, reads information of the external device according to a predetermined registration procedure. It can be registered in the meta information management tree of the memory 170. Thereafter, when a service request message is transmitted from an external device, information necessary for the execution of the service is obtained from the meta information management tree, and necessary procedures (e.g., the above-described conflict resolution procedure, interworking with other gateways according to user location detection) Etc.) can be performed.

In addition, the controller 180 receives meta information on the running service from external devices, determines whether or not the running service can be switched to the turned on external device when another external device is turned on, and Information can be transferred to an external device that is turned on. Thereafter, according to a user's command input, a device running a specific service can be controlled to be changed. As described above, whether or not the device is newly turned on may be replaced by other information capable of determining whether or not the user moves the location.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power to each of the components included in the gateway 100. The power supply unit 190 may also include a battery.

Meanwhile, when the wireless communication unit 110 and the interface unit 160 are used for a common purpose of providing a data path with an external device, they may be collectively referred to as a “communication unit”.

At least some of the respective components may operate in cooperation with each other to implement the operation, control, or control method of the gateway according to various embodiments described below. In addition, the operation, control, or control method of the gateway may be implemented on the gateway by driving at least one application program stored in the memory 170.

It is obvious to those skilled in the art that the present invention can be embodied in other specific forms without departing from the spirit and essential features of the present invention.

Further, the present invention described above can be implemented as a computer-readable code on a medium on which a program is recorded. The computer-readable medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include ROM, RAM, CD-ROM, magnetic tapes, floppy disks, optical data storage devices, and the like, and are implemented in the form of carrier waves (for example, transmission over the Internet). Also includes. In addition, the computer may include the control unit 180 of the terminal.

Therefore, the detailed description above should not be construed as restrictive in all respects and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (15)

A communication unit for exchanging signals with at least one external device by wire/wireless;
Memory; And
Establishing a data path through the at least one external device and the communication unit, obtaining information on each of the at least one external device through the data path, and storing the obtained information as a tree of a preset structure in the memory Including a control unit,
The tree includes at least one of a device, a device function, a location, a user, a service, a device adapter, and a gateway as a top-level item, and is managed at the middleware platform layer and referenced at the service layer,
The control unit manages the operation of the at least one external device according to any one of a plurality of states differently defined according to an operation state of the at least one external device and a location of a user,
The control unit,
Transition to one of the plurality of states according to at least one of a user's command through the gateway, the presence or absence of an operation schedule of the at least one external device, a possibility of a change in operation of the at least one external device, and occurrence of a specific event,
The control unit,
When it is certain that there is no change in operation of devices other than safety-related devices or devices that are always turned on, and there is no schedule for operation, and when it is determined that the user is not in the house, or when there is a user's command, the above Entering the park state among multiple states,
The control unit,
Activate devices and services related to security or safety in the park state, and turn off power to devices other than those that are always on,
The gateway, characterized in that storing metadata on an operation state before entering the park state in the memory.
delete The method of claim 1,
The device item,
Includes at least one of device owner, priority, support function definition, and cluster classification as the lowest level item,
The above support function definition,
A gateway defined in a predetermined form in each of the at least one external device.
The method of claim 3,
The control unit,
When the at least one external device is connected, the gateway performs function discovery as to whether an optional function exists among supported functions according to a preset profile for each device. The method of claim 3,
The control unit,
When two or more service requests received from the at least one external device collide with each other, the gateway performs a conflict resolution procedure by referring to the priority item in the tree. The method of claim 1,
The tree is,
A gateway comprising a first group of items for providing interoperability between a service layer and a middleware platform and a second group of items for analysis and implementation between services. The method of claim 6,
The first group of items includes at least one of the device, the device function, and the location,
The second group of items includes at least one of the user, the service, the device adapter, and the gateway. The method of claim 1,
The control unit,
A gateway that refers to the tree in controlling a service accompanying an operation of the at least one external device. In the gateway control method,
Establishing a data path with at least one external device through a communication unit;
Obtaining information on each of the at least one external device through the data path;
Storing the obtained information as a tree of a predetermined structure in a memory; And
Including the step of controlling a service accompanying the operation of the at least one external device with reference to the tree,
The tree is,
The top-level item includes at least one of device, device function, location, user, service, device adapter, and gateway, and is managed at the middleware platform layer and referenced at the service layer,
The controlling step,
And managing the operation of the at least one external device according to any one of a plurality of states differently defined according to an operation state of the at least one external device and a location of a user,
The controlling step,
When the external device corresponds to a non-compliant CPE, the non-compliant CPE is recognized as a subordinate device of the standard CPE, and limited meta information of a device connected to the non-compliant CPE is obtained through the non-compliant CPE, and the incompatible CPE Control method of a gateway, characterized in that obtaining limited rights to the device connected to the CPE.
The method of claim 9,
The managing step,
Transitioning to one of the plurality of states according to at least one of a user's command through the gateway, the presence or absence of an operation schedule of the at least one external device, a possibility of a change in operation of the at least one external device, and occurrence of a specific event Including, the control method of the gateway. The method of claim 10,
The device item,
Includes at least one of device owner, priority, support function definition, and cluster classification as the lowest level item,
The above support function definition,
A method of controlling a gateway defined in advance in a predetermined form in each of the at least one external device. The method of claim 11,
The obtaining step,
And performing function discovery as to whether an optional function exists in each of the at least one external device among support functions according to a preset profile for each external device. The method of claim 11,
The controlling step,
When two or more service requests received from the at least one external device collide with each other, performing a conflict resolution procedure by referring to the priority item in the tree. The method of claim 11,
The tree is,
A method of controlling a gateway comprising a first item group for providing interoperability between a service layer and a middleware platform, and a second item group for analysis and implementation between services. The method of claim 14,
The first group of items includes at least one of the device, the device function, and the location,
The second item group includes at least one of the user, the service, the device adapter, and the gateway.

Images