KR102233602B1 - Method and device for setting and recommending device action using user context - Google Patents

Abstract

A method of automating a procedure in which a smart home system recognizes a user context and sets or recommends an operation of each device in a smart home system based on the recognized user context. It provides a method for a user to newly add or define a mode service or to manage or modify the operation of a device included in the mode service. It also provides a method of recommending functions that can be provided through each device when defining rules.

Description

Device operation setting and recommendation method and device using user context {METHOD AND DEVICE FOR SETTING AND RECOMMENDING DEVICE ACTION USING USER CONTEXT}

In the smart home service, various devices such as home appliances, sensors, and servers are linked to each other to perform an operation desired by the user. Home appliances, sensors, and servers included in the smart home system may transmit and receive information to and from each other through a gateway device. On the other hand, as the number of devices interlocked in the smart home system increases, the complexity of setting and controlling for using the smart home service also increases.

The smart home service may include a mode service in which an action of devices corresponding to the specific mode is collectively executed when a user selects a specific mode provided by a manufacturer in advance. By using the mode service, a user can control multiple devices to run at once with a single input.

An embodiment of the present invention provides a method for a smart home system to recognize a user context and to automate a procedure for setting or recommending an operation of each device in a smart home system based on the recognized user context.

An embodiment of the present invention provides a method for a user to newly add or define a mode service (rule), or to manage or modify an operation of a device included in the mode service. In addition, when defining rules, it recommends functions that can be provided through each device, or provides a function to automatically generate rules by detecting a user's device usage pattern.

An embodiment of the present invention allows a smart home service to be provided without relying on a user's execution input.

A method of controlling another device by a gateway device according to an embodiment of the present invention is, when receiving user operation information from a first device group, device state information received from the second device group is received. Recording during a recording period; Extracting common device state information based on the recorded device state information and at least one previously recorded device state information; And when receiving device state information corresponding to the extracted common device state information from the second device group, requesting the first device group to execute a user operation corresponding to the user operation information. It characterized in that it comprises a.

A gateway device for controlling another device according to an embodiment of the present invention includes: a communication unit for transmitting/receiving information with another device; A storage unit for recording or storing the information; And when user operation information is received from the first device group, device state information received from the second device group is recorded during a recording period, and the recorded device state information and previously recorded device state information are recorded. If common device state information is extracted based on at least one device state information, and device state information corresponding to the extracted common device state information is received from the second device group, the first device group And a control unit for controlling to request the first device group to execute a user operation corresponding to the operation information.

Since the pattern of the user context when each device of the smart home system is executed is learned, it is possible to provide convenience and optimized services to the user by executing the device without a user action for executing the device. In addition, users can conveniently and efficiently manage rules through automatic creation and recommendation of rules.

1 shows the configuration of a smart home system according to an embodiment of the present invention.
2 shows the configuration of a smart home system according to another embodiment of the present invention.
3 shows a process of generating a rule by receiving a condition and an action according to an embodiment of the present invention.
4 illustrates a process of editing a rule and performing an operation according to application of the rule, according to an embodiment of the present invention.
5 illustrates a process of generating a rule based on a pattern of a user's device usage and context and applying the generated rule according to an embodiment of the present invention.
6 shows an operation section according to an embodiment of the present invention.
7 shows a process of extracting an event from an operation section according to an embodiment of the present invention.
8 shows names and icons according to an embodiment of the present invention.
9 shows a configuration of a gateway device according to an embodiment of the present invention.
10 is a flowchart illustrating a rule generation and execution of the generated rule according to an embodiment of the present invention.
11 is a flowchart illustrating a rule generation and execution of the generated rule according to another embodiment of the present invention.
12 is a flowchart of a gateway controlling another device according to an embodiment of the present invention.
13 is a flowchart illustrating a device operation using a context according to another embodiment of the present invention.

Terms or words used in the specification and claims described below should not be construed as being limited to a conventional or dictionary meaning, but should be construed as meanings and concepts consistent with the technical idea of the present invention. In the following description, only parts necessary to understand the operation according to each embodiment of the present invention will be described, and descriptions of other parts may be simplified or omitted so as not to distract the gist of the present invention. The features of the present invention are not limited to the examples disclosed in this specification, and may be interpreted as including changes in the form of each configuration or even additional functions. In the drawings, the size of some elements may be enlarged for illustration and are not drawn in proportion to the size.

In the specification and claims, "comprising" does not mean excluding other elements or actions. Singular nouns may include plural nouns unless specifically stated otherwise. For example, "user action" or "state" may mean a plurality of user actions or a plurality of states. The suffix "unit" for a component is given or used interchangeably in consideration of only the ease of writing the specification, and does not itself have a distinct meaning or role from each other. In the specification and claims, "first, second, third" are used to distinguish between similar elements, and are not necessarily used to describe sequentially or chronologically. In the present specification and claims, "first, second, third" are not used in a mutually exclusive sense. For example, devices included in the "first device group" may be included in the "third device group".

In the present specification and claims, a rule refers to a relationship between a condition and an action triggered by the condition, and is determined in consideration of the history of context information. do. The rule can be used in the same meaning as a mode service. In the present specification and claims, a condition may mean a specific state or state change of the device(s) included in the smart home system. In the present specification and claims, an action may mean a user operation for the device(s) included in the smart home system.

Rules can be expressed as shown in Table 1. As shown in Table 1, conditions can include the state of one or more devices, and actions can include actions on one or more devices.

Rule Condition Action name Device 1 Device 2 … Device 1 Device 2 … icon state state … action action …

Rules may have a unique name and icon, defined by the user or recommended by the smart home system. When a rule includes conditions and actions, each condition and action may also have a unique name and icon. The name of the rule may correspond to, for example, Market, Rainy, Away, or Sleep. The name of the condition could be, for example, NFC, door, Date & Time, Weather, Temperature, Humidity, or Light in living room. I can. The name of the operation may correspond to, for example, one of light, security, and air conditioner. According to the device type, a device may be classified into, for example, a home appliance/lighting device and other devices. The rule may include information on the device type together.

The following sentences express rules including conditions (if clause) and actions (then clause) in sentences.

If the current time is between 10:00 and 21:00 and it rains, turn on the lights in the living room. If Tom goes out and there is no motion in Tom's room for 5 minutes, turn off the lights in Tom's room. If Jane is over 30 degrees, turn on the air conditioner. If going out after 9 o'clock is detected, the camera and sensor are activated. If the washing machine is finished washing, it is notified using pop-up and LED lights. If it detects the proximity of the NFC tag Alice possesses, it turns on the computer in Alice's room and runs the word program. If yesterday's electricity usage is more than twice the electricity usage two days ago, a warning message is displayed on Margie's terminal.

When the gateway detects a condition and an operation, it may mean detection of a sensor or an input unit included in the gateway, or it may mean that the gateway receives sensing information or input information of another device.

Looking specifically at the condition, the condition may indicate external environment information measured by a sensor of the smart home system or a state or state change of the device(s) included in the smart home system. In the present specification and claims, "condition" may be used in the same sense as "user situation" or "device state". Table 2 shows examples of conditions.

External condition Home appliance/lighting device Other devices Temperature weather Electricity price time Refrigerator air conditioner washer dryer Oven TV LED switch door lock Motion detector IP camera Thermostat warmth Sunny Expensive dawn On On On On On On On On Door open On On On usually blur usually morning Off Off Off Off Off Off Off Off Door locked Off Off Off Cold ratio wrapping lunch Refrigerator open High temperature Washing completed Drying complete Cooked Channel 1 Bright Motion detection Motion detection High temperature Eye afternoon Refrigerating room closed Temperature Channel 2 usually Sound detection Temperature dinner Freezer open Low temperature gloominess Low temperature night Freezer closed

In the external conditions of Table 2, the time may correspond to one of the conditions of dawn, morning, lunch, afternoon, evening, and night. In the home appliance/illumination device of Table 2, the TV may correspond to one or more of on, off, channel 1 display, and channel 2 display. In the other devices in Table 2, the thermostat may correspond to one of high temperature, medium temperature, and low temperature. Information on conditions detected by each device may be delivered to the gateway device.

Looking at the action in detail, the action represents the operation of the device(s) included in the smart home system. In the present specification and claims, "action" may be used with the same meaning as "user operation". Table 3 shows an example of the operation.

Home appliance/lighting device Other devices Refrigerator air conditioner washer dryer Oven TV LED switch door lock Motion detector IP camera Thermostat Refrigerator open Power on Power on Power on Power on Power on ON ON Door open Motion detection Motion detection Power on Refrigerating room closed Power off Power off Power off Power off Power off Off Off Door locked Motionlessness Sound detection Power off Freezer open Temperature increase brightly Video delivery Temperature increase Freezer closed Temperature down darkly Temperature down Temperature down Temperature increase

In a home appliance/lighting device, the LED can detect an operation corresponding to one of an ON operation, an OFF operation, a brightening operation, and a darkening operation. In other devices, a motion detector may detect a motion corresponding to one of motion detection and no motion detection. Information on the motion detected by each device may be delivered to the gateway device. Table 3 is only an example of an operation, and the user can edit the operation to determine whether to repeat the operation, the number of repetitions, and the repetition interval time. Information on the motion detected by each device may be delivered to the gateway device.

Hereinafter, various embodiments of the present invention will be described with reference to the accompanying drawings.

1 shows the configuration of a smart home system according to an embodiment of the present invention.

According to an embodiment of the present invention, a smart home system includes a gateway 110, a user equipment 120, an associated device 130, a smart appliance 140, or a remote server ( 150) may be included.

The gateway 110 may be referred to as a gateway device, and may communicate with the user terminal 120, the associated device 130, the smart home appliance 140, or the remote server 150. The gateway 110 is the user terminal 120, the associated device 130, the smart home appliance 140 or data received from the remote server 150 or the user terminal 120, the associated device 130, the smart home appliance 140 Alternatively, data required to control the remote server 150 may be stored. The gateway 110 may include one or more displays (touch screens) capable of displaying a user interface (UI). According to an embodiment of the present invention, the gateway 110 may perform all or part of the functions of the user terminal 120, the associated device 130, or the smart home appliance 140. In other words, a smart phone, a wearable device, a tablet pc, a vacuum cleaner, a refrigerator, a TV, a thermostat, etc. may serve as the gateway 110.

The user terminal 120 is an electronic device configured to execute an application driven by an operating system or display content. The user terminal 120 includes a display (touch screen) capable of displaying a UI, a communication unit communicating with an external device, a storage unit storing data, an input unit receiving user input, or a control unit controlling each operation. I can. The user terminal 120 includes, for example, a tablet personal computer (PC), a portable multimedia player (PMP), a personal digital assistant (PDA), and a smart phone. Phone), mobile phones, digital picture frames, wearable devices, and the like. The wearable device is an electronic device that can be worn over the body, and may be in the form of, for example, glasses, watches, bracelets, clothing, sports equipment, medical equipment, and the like. The wearable device may be attached to the skin or implanted into the human body.

The associated device 130 may include PV, ESS, door lock, IP camera, door cam, and the like. The associated device 130 may include a sensor that detects geomagnetic, atmospheric pressure, a user's motion, a user's proximity, a user's location, a window, temperature, humidity, illuminance, gravity, and the like.

The smart home appliance 140 may include a television, a refrigerator, a washing machine, an air conditioner, an oven, a dishwasher, or a robot cleaner.

In FIG. 1, the user terminal 120, the associated device 130, and the smart home appliance 140 may be collectively referred to as a "device (device)". In other words, in the present specification and claims, a device may be defined as a device through which a user can monitor the state of the device or control the operation of the device through a gateway (GW), a terminal, or a server. Meanwhile, the user terminal 120, the associated device 130, and the smart home appliance 140 may correspond to a device even if not illustrated.

The remote server 150 may perform device operation control through remote access. Although the present specification describes device operation control by a gateway or a device, the remote server 150 may also perform device operation control. The remote server 150 controls a display (touch screen) that can display a UI, a communication unit that communicates with an external device, a storage unit that stores data, an input unit that receives user input, and controls the operation of other units (units). It may be provided with a control unit.

2 shows the configuration of a smart home system according to another embodiment of the present invention.

The smart home platform 210 may include a rule database 211. The smart home platform 210 includes a context checker module 212 that executes an appropriate rule by comparing the status information of the device monitored by the smart home platform 210 with the user context information defined in the Rule DB. can do. The smart home platform 210 may correspond to the gateway 110 of FIG. 1. The smart home platform 210 may be connected to other devices 220 and 230, a manufacturer server 240, or a social network 250. The manufacturer server 240 or the social network 250 may include the remote server 150 of FIG. 1.

3 shows a process of generating a rule by receiving a condition and an action according to an embodiment of the present invention.

When the rule generation procedure starts, the gateway of the smart home system determines whether the user selects a condition provided as a default (310). If the user does not select a condition provided by default, the gateway creates a new condition (315). The gateway may receive information necessary to create a condition from a user. The process of receiving the necessary information includes, for example, a process of receiving a selection of a person to be sensed (320), a process of receiving a selection of a sensing device (325), a process of receiving a selection of a sensing item (330), a process 335 of receiving the creation or addition of a name or icon of a condition triggering sensing is input. The process of receiving a selection for a person to be sensed may include a process of receiving a selection for an NFC tag corresponding to a person to be sensed. According to an embodiment of the present invention, when a condition provided by default is selected, the condition input may be completed only by an input for selecting a person to be sensed. When the condition input is completed, it is determined whether there is an input for adding a new condition (340).

If there is no input for adding a new condition, an action corresponding to the condition selected or created by the user is input from the user. The process of receiving an operation corresponding to the condition may include a process 345 of selecting a device that performs an operation corresponding to the condition and a process 350 of selecting an operation of the selected device. If it is determined that there is no input for adding a new action after completing the action input in step 355, a rule is generated by matching the condition and action input by the user, and the generated rule is stored (360). The process of storing the generated rule may include a process of receiving and storing the name and icon of the rule from the user.

4 illustrates a process of editing a rule and performing an operation according to application of the rule, according to an embodiment of the present invention.

In a state in which the smart home service is activated, the gateway GW may determine whether the device is added or linked to the smart home system (410). The gateway may record a rule list in which the device and the rule are associated in the form of a mapping table. The mapping table may further include information on a device type. If the gateway determines that the device has been added in step 410, the gateway may query whether the mapping table contains a rule including information on the device type or the device to be mapped to the added device. (420). The mapping table may be stored in a gateway, another device, or a server. In response to a query, if it is determined that a device or device type is mapped between the added or linked device and the device included in a specific rule in the rule list, the gateway may display a recommendation for the specific rule to the user. There is (430). When the user selects the displayed specific rule, the selected rule is registered as a new rule corresponding to the added device.

In a state in which the rule is registered, the gateway GW may determine whether a user context included in the registered rule occurs (440). Determining that the user situation has occurred may correspond to detecting at least one or more of the states illustrated in Table 2 or detecting a state change to at least one or more of the states illustrated in Table 2. When the gateway determines that a user situation has occurred, it can query the information included in the rule list stored in the rule DB. For example, the Context Checker module 212 may compare device status information with condition information defined in the Rule DB 211. The condition information may include a specific time or date defined by the user.

When it is determined that the user situation has occurred, the gateway may control an operation corresponding to the user situation to be performed by the device(s) in the smart home system (460).

The user can inquire the rule list stored in the rule DB. While the rule list is displayed, the user can activate or deactivate (on/off) each rule. Even if there is no user's deactivation input, a rule that has a problem in interworking with the device when inquiring the rule list may become inactive. When searching the rule list, whether to activate or deactivate (on/off) can be displayed separately. The distinctive display may correspond to displaying different colors, for example. When searching the rule list, the user can add, edit or delete condition (user context), action (user operation), name or icon for each rule.

The user may create a new rule by inputting conditions and actions through a gateway or a tool included in a device connected to the gateway. Alternatively, the gateway may analyze the relationship between the user's device usage and the pattern of the context, and extract conditions and actions through the analysis result to automatically generate a rule.

5 illustrates a process of generating a rule based on a pattern of a user's device usage and context and applying the generated rule according to an embodiment of the present invention.

In FIG. 5, an event is defined to mean a user situation detected by the device and a user action on the device. The smart home system may include an Event History Repository 505 for storing events. Information on the user situation and user actions stored in the event history storage 505 may be automatically discarded after a predetermined time after storage.

The smart home system may store information monitored by the smart home system in the event history storage 505 during a certain time period before and after the time when the motion of the device by the user is detected (510). There is (515). The operation of the device may correspond to, for example, a movement of a user opening a front door and entering from outside. A certain period of time before and after the sensed time point may correspond to, for example, two minutes from the time when the motion is sensed. The information monitored by the smart home system is information obtained from a sensor/device included in the smart home system. The information monitored by the smart home system may include, for example, a user operation of turning on a light, a user operation of turning on a TV, and a user entering the kitchen.

The smart home system periodically retrieves sensor/device information (event information) stored in the event history storage 505 in order to generate a rule by a user input or whenever an event is detected (520), and the searched event information It may be determined whether there is an event that is repeated for each time interval based on the present invention (525). If a recurring event exists, the user situation and user action included in the event may be extracted, and the extracted user situation and user action may be stored in the event history storage 505 (530 ). By the storage, a rule generation procedure including a specific user action triggered by a specific user situation is completed (535).

According to another embodiment of the present invention, when there is a condition(s) that are repeated more than a certain number of times for a specific operation(s) in the process of storing repetitive event information, the repeated condition(s) ) And a rule including specific action(s) can be created and recommended to a user.

When the smart home system detects a change in the sensor/device state of the device (540), before detecting the change in the sensor/device state, the smart home system may search for sensor/device information (event information) stored in the event history storage 505 (545). ). The smart home system determines whether the user situation included in the event information acquired by the search matches the detected state change (550), and if the user situation matches the detected state change, the operation corresponding to the user situation is performed. Execute (555).

6 shows an operation section according to an embodiment of the present invention.

Figures (a) and (b) represent an action period when k7 (condition) is detected and an event (user situation and user action) detected during the action period, respectively. In the present specification and claims, the operation period is used in the same meaning as the recording period. As can be seen from FIG. 6, the operation section corresponding to k7 (condition) starts before the detection time of k7 and ends after the detection time of k7. Alternatively, it can be implemented as an operation section starting or ending at a time when a condition is detected. The operation period is a preset time period and may correspond to one of 30 seconds, 2 minutes, 30 minutes, 2 hours, and 48 hours, for example. The operation section (recording section) is the time it takes for the user to move from the entrance of the house to the farthest room before and after a specific action (e.g., opening the door of the house from the outside). ) Can be determined in consideration of the time it normally takes to perform.

In Fig. 6, k2, k3, k7 denote changes in the state of the device (user situation), a, b, c, d, ... Means user actions on the device. In FIG. 6, the event E1 detected during the operation period in (a) = {a, b, c, d, m, n, u, w, k2, k3, k7}. In FIG. 6, the event E2 detected during the operation period in (b) = {a, b, c, d, e, m, r, t, y, k3, k7}. E1 and E2 are displayed without considering each user situation and the time point at which the user action occurs. Alternatively, a user situation and user action may be specified in consideration of the order of occurrence.

7 shows a process of extracting an event from an operation period according to an embodiment of the present invention.

When a change in the state of the device (for example, a sensor) is detected (710), the smart home system stores the information monitored by the smart home system in the event history storage 720 for a certain period of time before and after the time when the state change is detected. I can. In FIG. 5, the storage is based on detection of a user's motion, while FIG. 7 shows storage based on the detection of a state change. The state change of the device may correspond to, for example, changing the current time from dawn (05:00 to 08:00) to morning (08:00 to 12:00). The specific time corresponding to the dawn or morning can be arbitrarily changed by the user. The information monitored by the smart home system may include, for example, detecting a user's motion in a bedroom, detecting a user's motion in a living room, and a user's motion of turning on a light in the living room.

The smart home system periodically retrieves event information stored in the event history storage 720 in order to generate a rule by a user input or whenever an event is detected. The smart home system may determine whether there is an event of a pattern that is repeated for each operation section based on the searched event information (730).

A process of determining whether an event of a pattern repeated for each operation section exists based on the searched event information will be described with reference to FIG. 6. As shown in FIG. 6, past events can be divided into respective motion sections in the time dimension.

When explaining the process of first extracting the user motion for each motion section, the user motion S1 in (a) = {a, b, c, d, m, n, u, w}, and the user motion in (b) It can be assumed that S2 = {a, b, c, d, e, m, r, t, y}. In this case, it may be determined that the intersection of S1 and S2 S={a, b, c, d, m} is the extracted user action. The extracted user operation may be referred to as a common user operation.

When explaining the process of extracting the user situation (device state) for each operation section, the user situation K1 in (a) = {k2, k3, K7}, and the user situation K2 in (b) = {k3, k2} This can be assumed. In this case, the intersection of K1 and K2 K = {k2, k3} represents the extracted user situation. The extracted user context may be referred to as a common user context or a common device state.

In FIG. 6, only comparing two motion sections is shown, but the present invention is not limited thereto, and a common event (user motion and user situation) may be extracted by comparing more than two motion sections.

The smart home system determines whether there is an event of a repetitive pattern (730). If the smart home system determines that an event of a repetitive pattern exists, it may extract a user situation and user action included in the event, and store the extracted user situation and user action in the event history storage 720 (740).

In the above embodiments, a condition and an action are distinguished according to whether the device detects a user context or a user operation for the device. The criteria for distinguishing between conditions and actions are not limited thereto. For example, when a specific user situation or specific user action is detected, the other user situation or user action detected before detecting a specific user situation or specific user action is a condition, and after detecting a specific user situation or specific user action Another user situation or user action detected in may be defined as an action.

8 shows names and icons according to an embodiment of the present invention.

The user can inquire a rule list through a device or gateway of the smart home system. In this case, when the rule includes a name or icon, the name or icon may be displayed on the device or gateway. An example of a rule name 810 and an icon 820 is shown in FIG. 8. Meanwhile, conditions and actions included in the rule may have names and icons, respectively. An example of an operation name 830, an operation icon 840, a condition name 850, and a condition icon 860 is shown in FIG. 8.

9 shows a configuration of a gateway device according to an embodiment of the present invention.

Referring to FIG. 9, a gateway device (gateway) may include configurations of a sensing unit 910, an input unit 920, a storage unit 930, a communication unit 940, and a display unit 950.

The sensing unit 910 may include one or more sensors to detect a user situation or user motion. For example, geomagnetic sensor, temperature sensor, barometric pressure sensor, proximity sensor, illuminance sensor, GPS (Global Positioning System), acceleration sensor, angular velocity sensor, velocity sensor, gravity sensor, tilt sensor, gyro sensor, gas sensor, camera or microphone. And the like may be included.

The input unit 920 transmits a signal input by a user to the control unit 960 in connection with function control of the gateway and setting of various functions. In addition, the input unit 920 may be combined with the display unit 950 to form a touch screen or a general keypad. In addition, the input unit 920 may be composed of a function key, a soft key, or the like.

The storage unit 930 may record or store monitoring information collected by the gateway or information input by a user. The monitoring information may include information on a user situation sensed by devices (including a gateway) of the smart home system or information on user actions on the devices. According to another embodiment of the present invention, the storage unit 930 may be implemented in a form included in a separate server.

The communication unit 940 may communicate with other devices or servers to transmit and receive monitoring information or information necessary for controlling other devices. The display unit 950 may display information such as a rule list or a recommendation rule through a user interface.

The controller 960 controls the overall state and operation of components constituting the gateway. The control unit 960 controls to store the monitoring information transmitted through the detection unit 910 or the communication unit 940 in the storage unit 930. The control unit 960 controls the display unit 950 to display information, and controls other units to perform a function corresponding to the user input received through the input unit 920, according to various embodiments described in the present invention. Let the process run.

In FIG. 9, the sensing unit 910, the input unit 920, the storage unit 930, the communication unit 940, the display unit 950, and the control unit 960 are configured as separate blocks, and each block performs a different function. Although described as above, this is only for technical convenience, and each function is not necessarily classified as such.

10 is a flowchart illustrating a rule generation and execution of the generated rule according to an embodiment of the present invention.

The gateway device of the smart home system may detect a customized user motion for a first device group assigned by the user, among devices included in the smart home system (1005). The device group may also include a case of having only one device. The customized user action may include one or more actions illustrated in Table 3. The customized user operation may correspond to, for example, an operation of turning on a TV and turning on an air conditioner. The recording period is a time period in which monitoring information is recorded for automatic rule generation. The gateway device may record state changes of devices included in the smart home system during the recording period (1010).

In order to automatically generate a rule, the gateway device may periodically retrieve monitoring information (status change) recorded so far when it detects a user input or detects an event (1015). Periodically searching for monitoring information may be performed every 12 hours, for example. Event detection may correspond to, for example, no motion being detected in the house for more than 30 minutes.

When there is a state change having a repeating pattern with respect to the searched monitoring information (state change), the gateway device determines the corresponding state change as a candidate state change and displays the candidate state change so that the user can view it (1020). ). The candidate state change may include, for example, the arrival of 10:00 am on Saturday.

The common state change is a state change that triggers a customized user action after registration of a rule. When there is a user input for setting the candidate state change as the common state change, the gateway device sets the candidate state change as the common state change (1025). According to another embodiment, even when there is no user input for a predetermined period of time, the candidate state change may be set as the common state change. Alternatively, if there is no user input for a predetermined period of time, the display of the candidate state change may be terminated without setting the common state change.

The gateway device maps and stores the set common state change and customized user operation (1030). Through the above process, a rule is created.

After the rule is generated, the gateway device may determine whether a state change matching the set common state change is detected (1035). If the gateway device determines that a state change matching the set common state change is detected, it may request the first device group to perform the customized user operation without a user input (1040).

Through the process of FIG. 10, the gateway device collects situation information when an operation routinely input by a user is performed, and in the case of the same situation as the collected situation information, the operation performed by the user input is performed by the user. Make it run without input. For example, a gateway device having learned a pattern in which a user turns on a TV and an air conditioner every 10 AM on a Saturday may automatically turn on the TV and an air conditioner even if the user does not input every 10 AM on a Saturday.

11 is a flowchart illustrating a rule generation and execution of the generated rule according to another embodiment of the present invention.

The gateway device of the smart home system may detect a user's motion with respect to the devices included in the smart home system or the states of the devices (1105). The user action may include one or more actions illustrated in Table 3. The state may include one or more states illustrated in Table 2. The gateway device may record the sensed user motion or state (1110). Here, recording the status may correspond to recording the status of a predetermined device(s), not recording the status of all devices of the smart home system.

In order to automatically generate a rule, the gateway device may periodically retrieve a user's motion and state recorded so far when it detects a user input or whenever an event is detected (1115).

The gateway device may extract a user action for the first device group and a state of the first device group from among the searched user actions and states (1120). The first device group refers to one or more devices to which a recording section is assigned by a user among devices included in the smart home system. The recording period is a time period allocated for each first device group to group user actions and states. The recording period may start at or before the first device group senses the user's motion or state, and end at or after the first device group senses the user's motion or state.

The gateway device may group user actions and states for each of the extracted user actions or states for each recording period (1125). When there is a grouped user operation or a grouped state having a repeating pattern, the gateway device may determine the corresponding user operation or state as a common user operation and a common state (1130). Having the repeating pattern may be defined as being repeated a predetermined number of times (eg, 5 times). The predetermined number of times may be differently defined according to grouped user actions or grouped states. According to another embodiment, as in step 1020 of FIG. 10, in order to determine the common user motion and the common state, the candidate user motion and the candidate state may be displayed to receive a user's selection input.

The gateway device maps and stores the determined common user operation and common state (1135). Through the above process, a rule is created.

After the rule is generated, the gateway device may determine whether a current state of a predetermined device(s) among devices included in the smart home system matches the stored common state (1140). The predetermined device(s) may be the same as or included in the device(s) for which a user motion or state is detected in step 1105. If the gateway device determines that the current state of the predetermined device(s) matches the stored common state, the gateway device may request the first device group to perform the common user operation without a user input (1145).

Through the same process of FIG. 11, the gateway device collects context information (conditions) when a user routinely repeats an operation, and when the same situation as the collected context information is detected, the repeated operation is performed without user input. To be performed.

12 is a flowchart of a gateway controlling another device according to an embodiment of the present invention.

When the gateway device of the smart home system receives user operation information from the first device group, it records device state information received from the second device group during a recording period (1205). . Each of the first device group and the second device group may include at least one device. At least one device included in the first device group and at least one device included in the second device group may be the same.

The user motion information may include a user's motion with respect to the first device group. The user motion information may include one or more user motions customized by the user. The customized user operation may correspond to, for example, an operation of pressing a washing mode button set by the user in a washing machine.

The device status information may include status information detected by the second device group or status information of at least one device included in the second device group. The device status information may include time information or access information of a specific person. The time information may include a specific date, day of the week, time, or a combination thereof. The specific person's access access information may include information on an access notification of an NFC tag possessed by the specific person. The device state information may include state changes of one or more devices included in the first device group. A change in state of the device means sensing that the device (eg, a thermostat) has changed from one state (eg, high temperature) to another (eg, normal temperature). That is, when the device state information includes a state change of the device, the device state information may include both a past state and a current state of the device.

The recording period may be determined based on the first device group and the user motion information. For example, when the first device group is a refrigerator and the user operation information is temperature rise, the recording period is 3 minutes, whereas when the first device group is a refrigerator and the user operation information is temperature decrease, the recording period is determined to be 4 minutes. I can. For another example, when the first device group is a TV and an air conditioner, the recording period is 30 minutes regardless of user operation information, whereas when the first device group is TV, the recording period may be determined to be 10 minutes regardless of user operation information. have. The above example is only an example and can be implemented in other forms.

The recording period may start at or before the user motion information is received from the first device group, and may end at or after the user motion information is received from the first device group. For example, when the user motion information is received (detected) at 10:00, the recording period is any one of 09:50~10:00, 09:55~10:05, or 10:00~10:10 It may correspond to.

The gateway device extracts common device state information based on the recorded device state information and at least one previously recorded device state information (1210). The extraction may be performed periodically or when a predetermined event occurs. The predetermined event may include detecting a user input or receiving specific information.

The gateway device may determine device status information recorded more than a predetermined number of times (eg, 5 times) for a predetermined time (eg, 7 days) as the common device status information. The predetermined time may correspond to three times or more times of the recording period. In another form, when the display unit displays the device status information recorded more than a predetermined number of times (for example, 7 times), and the input unit detects a predetermined user input (for example, a selection input for the displayed device status information) The displayed device state information may be determined as common device state information. The device status information recorded more than the predetermined number of times may be displayed in the form of a name or an icon.

The gateway device may match and store the extracted common device state information with the user operation information (1215). The gateway device may display a user interface capable of editing the stored common device state information and user operation information when it detects an input to request editing of information. When a device selection input through a user interface is detected, stored common device state information and user operation information corresponding to the selected device may be displayed in the form of a name or an icon. The user can select information to be edited from the displayed common device status information and user operation information.

When the gateway device receives device state information corresponding to the extracted common device state information from the second device group, the first device group sends the first device group to execute a user operation corresponding to the user operation information. Can be requested (1220). Accordingly, the first device group executes the requested user operation.

13 is a flowchart illustrating a device operation using a context according to another embodiment of the present invention.

For device operation using context, the gateway divides the past operation into each operation interval (recording interval) in the time dimension (1310), extracts an action for each operation interval (1320), and Conditions can be extracted (1330).

110: gateway 120: user terminal
130: related device 140: smart home appliance
150: remote server

Claims (22)

In a method for a gateway device to control another device,
Recording device state information received from the second device group during a recording period when user operation information is received from the first device group;
Extracting common device state information based on the recorded device state information and at least one previously recorded device state information; And
Upon receiving device state information corresponding to the extracted common device state information from the second device group, requesting the first device group to execute a user operation corresponding to the user operation information from the first device group. Including,
The device status information is information detected by a sensor, and the common device status information is common device status information of devices constituting the second device group. The method of claim 1, wherein the recording section,
The device control method, characterized in that it is determined based on the first device group and the user motion information. The method of claim 1, wherein the recording section,
A device control method, comprising: starting at or before receiving the user motion information from the first device group, and ending at or after receiving the user motion information from the first device group. The method of claim 1,
Matching and storing the extracted common device state information with the user motion information; And
And editing the stored common device state information and user operation information according to a user input. The method of claim 4, wherein editing the stored common device state information and user operation information comprises:
And displaying stored common device state information and user motion information corresponding to the selected device in the form of a name or an icon when detecting a device selection input by a user. The method of claim 1, wherein the extracting the common device state information comprises:
And determining device status information recorded at least a predetermined number of times during a predetermined time as common device status information. The method of claim 1, wherein the extracting the common device state information comprises:
Displaying device status information recorded more than a predetermined number of times;
And determining the displayed device state information as common device state information upon detecting a predetermined user input. The method of claim 1, wherein the device status information,
Device control method comprising time information or access notification information of a specific person. The method of claim 1, wherein the device status information,
A device control method comprising a state change of one or more devices included in the first device group. The method of claim 1, wherein the user motion information,
Device control method comprising one or more user actions customized by a user. The method of claim 1, wherein the extracting the common device state information comprises:
And extracting the common device state information periodically or when a predetermined event occurs. In the gateway device for controlling other devices,
A communication unit for transmitting and receiving information with another device;
A storage unit for recording or storing the information; And
When user operation information is received from the first device group, device state information received from the second device group is recorded during a recording period, and the recorded device state information and at least the previously recorded device state information are recorded. If common device state information is extracted based on one device state information, and device state information corresponding to the extracted common device state information is received from the second device group, the first device group performs the user operation. And a control unit for controlling to request the first device group to execute a user operation corresponding to the information,
The device status information is information detected by a sensor, and the common device status information is common device status information of devices constituting the second device group. The method of claim 12, wherein the recording section,
The gateway device, characterized in that it is determined based on the first device group and the user operation information. The method of claim 12, wherein the recording section,
A gateway device, characterized in that: starting at or before receiving the user motion information from the first device group, and ending at or after receiving the user motion information from the first device group. The method of claim 12,
The gateway device further includes an input unit for detecting an input by a user and a display unit for displaying information,
The control unit may match and store the extracted common device state information from the storage unit with the user operation information, and when the input unit detects an input to request editing of information, the stored common device state information and user operation The gateway device, characterized in that for controlling to display a user interface in which information can be edited. The method of claim 15,
The control unit, when the input unit detects a device selection input, controls the display unit to display stored common device state information and user operation information corresponding to the selected device in the form of a name or an icon. . The method of claim 12, wherein the control unit,
The gateway device, characterized in that the device status information recorded more than a predetermined number of times during a predetermined time is determined as common device status information. The method of claim 12,
The gateway device further includes an input unit for detecting an input by a user and a display unit for displaying information,
The control unit controls the display unit to display device state information recorded more than a predetermined number of times, and to determine the displayed device state information as common device state information when a predetermined user input is detected by the input unit. Gateway device. The method of claim 12, wherein the device status information,
A gateway device comprising time information or access notification information of a specific person. The method of claim 12, wherein the device status information,
A gateway device comprising a state change of one or more devices included in the first device group. The method of claim 12, wherein the user motion information,
A gateway device comprising one or more user actions customized by a user. The method of claim 12, wherein the control unit,
And extracting the common device state information periodically or when a predetermined event occurs.

Images