CN111433755A - Home network, electronic device, processing device, and display method - Google Patents

Abstract

Provided are a home network, an electronic device, a processing device, and a display method. A home network according to an embodiment includes an information processing unit and a data holding unit. The information processing unit determines a device to be operated based on the input information, and outputs a control signal for controlling the operation of the device. The data holding unit holds a condition for controlling the operation of the device by the information processing unit and information for specifying the device based on the input information.

Description

Home network, electronic device, processing device, and display method

Technical Field

The invention relates to a home network, an electronic device, a processing apparatus, and a display method.

Background

In recent years, sensors, switches, or controllers (control terminals) that can be used for realizing home security, home automation, or home energy management are sold from various manufacturers (manufacturers). The respective sensors, switches, and control terminals are often designed according to standardized specifications such as (radio waves in) the 2.4GHz band conforming to ieee802.15.4, the 900MHz band conforming to the 802.15.4e, or the 2.4GHz band conforming to the 802.15.11.

However, the sensor, the switch, or the control terminal is not necessarily designed according to the specification of the manufacturer (manufacturer). Further, even a structure designed according to specifications is functionally independent, and there are various cases such as a system in which one sensor has a plurality of functions and a control method is unique to a manufacturer.

Prior art documents

Patent document

Patent document 1: japanese laid-open patent publication No. 2007-174208

Patent document 2: japanese patent laid-open publication No. 2009 and 290758

Patent document 3: japanese laid-open patent publication No. 2009-75735

Disclosure of Invention

Summary of The Invention

Problems to be solved by the invention

In response to a request for uniformly controlling the sensors, switches, control terminals, and the like that have been distributed in the market, intra-domain communication units (home gateways, hereinafter, referred to as HGWs) that communicate with the sensors, switches, or control terminals and can control the sensors, switches, or control terminals individually have been put into practical use. In the HGW, it is required that as many sensors, switches, or control terminals as possible can be supported and controlled regardless of a manufacturer (manufacturer) or a specification to be complied with. Further, it is also one of the market demands that replacement (replacement) of already installed sensors, switches, or control terminals is not necessary.

There are also many demands for controlling the above-described sensors, switches, or control terminals by sound. In addition, in the case of controlling the above-described sensors, switches, or control terminals by sound, it is required that the sensors, switches, or control terminals can be controlled without approaching them beyond necessity even when there is a distance from each sensor, switch, or control terminal. Also, it is required that the respective sensors, switches, or control terminals can be controlled even in a state where they are separated (or invisible). Further, in the case where each sensor, switch, or control terminal is controlled by voice, there is a demand for controlling them individually by an interactive expression.

In the case where a device is loaded into a home network using an HGW without replacing an already installed sensor, switch, or control terminal, the job or procedure required to be loaded does not burden the installer (user). If the operation or procedure for installing the above-described sensors, switches, or control terminals into the home network is complicated, the number of sensors, switches, or control terminals to be actually added is limited, and the installation thereof may be suspended by the number smaller than the required number of sensors, switches, or control terminals to be originally added. In this case, as a result, functions of home security, home automation, or home energy management that can be achieved through the home network are limited.

The invention aims to provide a home network, an electronic machine, a processing device and a display method.

Means for solving the problems

A home network according to an embodiment includes an information processing unit [ HGW600] and a data holding unit [700 ]. The information processing unit [ HGW600] specifies a device to be operated based on input information, and outputs a control signal for controlling the operation of the device. A data holding unit (700) holds conditions for controlling the operation of the device by the information processing unit and information for specifying the device based on the input information.

Drawings

Fig. 1 is a schematic diagram showing an example of an environment in which the home network according to the embodiment can be applied.

Fig. 2(a) to (c) are schematic diagrams showing an example of an environment in which the home network according to the embodiment can be applied.

Fig. 3 is a schematic diagram showing an example of initial setting screen display of the home network according to the embodiment.

Fig. 4 is a schematic diagram showing an example of a setting screen display of the home network according to the embodiment.

Fig. 5 is a schematic diagram showing an example of display of a setting screen of a home network according to the embodiment.

Fig. 6 is a schematic diagram showing an example of display of a setting screen of a home network according to the embodiment.

Fig. 7 is a schematic diagram showing an example of display of a setting screen of a home network according to the embodiment.

Fig. 8 is a schematic diagram showing an example of display of a setting screen of a home network according to the embodiment.

Fig. 9 is a schematic diagram showing an example of display of a setting screen of a home network according to the embodiment.

Fig. 10 is a schematic diagram showing an example of zone (zone) location information (data table) to be referred to by the home network according to the embodiment.

Fig. 11 is a schematic diagram showing an example of display of a setting screen of a home network according to the embodiment.

Fig. 12 is a schematic diagram showing an example of display of a setting screen of a home network according to the embodiment.

Fig. 13A is a schematic diagram showing an example of a procedure for setting a device to be controlled to which the home network of the embodiment is applied.

Fig. 13B is a schematic diagram showing an example of a procedure for setting a device to be controlled to which the home network of the embodiment is applied.

Fig. 13C is a schematic diagram showing an example of a procedure for setting a device to be controlled to which the home network of the embodiment is applied.

Fig. 13D is a schematic diagram showing an example of a procedure for setting a device to be controlled to which the home network of the embodiment is applied.

Fig. 13E is a schematic diagram showing an example of a procedure for setting a device to be controlled to which the home network of the embodiment is applied.

Fig. 13F is a schematic diagram showing an example of a procedure for setting a device to be controlled to which the home network of the embodiment is applied.

Fig. 13G is a schematic diagram showing an example of a procedure for setting a device to be controlled to which the home network of the embodiment is applied.

Fig. 13H is a schematic diagram showing an example of a procedure for setting a device to be controlled to which the home network of the embodiment is applied.

Fig. 13I is a schematic diagram showing an example of a procedure for setting a device to be controlled to which the home network of the embodiment is applied.

Fig. 14 is a schematic diagram showing an example of intra-zone information (data table) to be referred to by the home network according to the embodiment.

Fig. 15 is a schematic diagram showing an example of sensor characteristic information (data table) to be referred to by the home network according to the embodiment.

Fig. 15A is a schematic diagram showing an example of sensor characteristic information (data table) to be referred to by the home network according to the embodiment.

Fig. 15B is a schematic diagram showing an example of sensor characteristic information (data table) referred to by the home network according to the embodiment.

Fig. 15C is a schematic diagram showing an example of sensor characteristic information (data table) referred to by the home network according to the embodiment.

Fig. 15D is a schematic diagram showing an example of sensor characteristic information (data table) referred to by the home network according to the embodiment.

Fig. 16 is a schematic diagram showing an example of the intention interpretation rule (data table) referred to by the home network according to the embodiment.

Fig. 16A is a schematic diagram showing an example of intent interpretation rules (data table) referred to by the home network according to the embodiment.

Fig. 16B is a schematic diagram showing an example of the intention interpretation rule (data table) referred to by the home network according to the embodiment.

Fig. 17 is a schematic diagram showing an example of sensor control (data table) referred to by the home network according to the embodiment.

Fig. 17A is a schematic diagram showing an example of sensor control (data table) referred to by the home network according to the embodiment.

Fig. 18 is a schematic diagram showing an example of a Home Gateway (HGW) that can be used in the home network according to the embodiment.

Fig. 19A is a schematic diagram showing an example of a procedure for setting a device to be controlled to which the home network of the embodiment is applied.

Fig. 19B is a schematic diagram showing an example of a procedure for setting a device to be controlled to which the home network of the embodiment is applied.

Fig. 19C is a schematic diagram showing an example of a procedure for setting a device to be controlled to which the home network of the embodiment is applied.

Fig. 20A is a schematic diagram showing an example of a procedure for installing a device to be controlled to which the home network of the embodiment is applied.

Fig. 20B is a schematic diagram showing an example of a procedure for setting a device to be controlled to which the home network of the embodiment is applied.

Fig. 20C is a schematic diagram showing an example of a procedure for setting a device to be controlled to which the home network of the embodiment is applied.

Detailed Description

Hereinafter, an embodiment will be described with reference to the drawings.

First, as shown in fig. 18, an example of a home gateway (hereinafter, referred to as HGW) that can be used in a home network according to the embodiment will be described.

In fig. 18, a server 1000 can be connected to the HGW600 via the internet (network) 300. The network 300 is connected with a voice processing unit 400, an input device (smartphone (portable terminal device)) 500, a zone management unit 700, a data table 800, and the like. The zone management unit 700 and the data table 800 hold various data described with reference to fig. 14, 15A, 15B, 15C, 15D, 16A, 16B, 17, and 17A. At least 1 of the audio processing unit 400, the zone management unit 700, and the data table 800 may be integrated with the HGW 600.

The HGW600 includes a control data management unit 601, a system controller 602, a device manager 603, a network interface (hereinafter referred to as a network I/F)605, a recording manager 607, a camera 611, a microphone 613, a speaker 615, and the like. Also, the HGW600 includes a sensor control table 609.

The control data management unit 601 includes an application manager (hereinafter abbreviated as APP-Mg), an event manager (hereinafter abbreviated as EVT-Mg), and a configuration manager (hereinafter abbreviated as CONFIG-Mg). The APP-Mg manages multiple applications for controlling various actions of the HGW 600. EVT-Mg management is used for event applications and the like for controlling various operations caused by occurrence of various events described later. The CONF-Mg management identifies various functions related to the HGW600 and functions within the HGW600, and performs structural applications such as operation order and operation restriction.

The system controller 602 can centrally control the modules within the HGW 600. The system controller 602 controls each module in the HGW600 in sequence (in the embodiment, the operations of 1 or more sensors, switches, or control terminals are associated (coordinated) in time series or at a predetermined timing with respect to 1 instruction). When associating (coordinating) operations, there may be 2 or more operations that are divided according to a predetermined rule.

The EVT-Mg enables control of the camera 611, the microphone 613, the speaker 615, the recording manager 607, and the like. The EVT-Mg determines detection data from external sensors and data from the camera 611, the microphone 613, and the speaker 615, which are taken in from the network I/F605, and can control the next action or behavior (behavior). The CONFG — Mg can perform initial setting of each functional block inside the HGW600, restriction of functions, expansion of functions, priority, operation time, and the like.

The device manager 603 can authenticate another device operating in association with the HGW600 and register the device in a memory (storage device). Therefore, the device manager 603 can manage the other sensors 101, 102, 103, … connected via the network I/F605. The device manager 603 can also register and identify identification data of the server 1000 connected via the internet 300. The device manager 603 can also register and recognize identification data such as the smartphone GUI-1 (screen display 501) connected via the internet 300.

The sensor control table 609 stores names (Device IDs/names) of sensors, switches, and control terminals 101, 102, 103, and …, position information of the sensors, switches, and control terminals, and data for controlling the sensors. Further, the names and position information of the sensors, switches, and control terminals can be displayed on the GUI-1 of the smartphone 500, so that the user (installer/user) can confirm the types, installation positions, functions, and the like of the sensors, switches, and control terminals.

The network I/F605 is connected to, for example, sensors or switches 101, 102, 103, and … with a sensor function located at arbitrary positions in a home via short-range wireless. In fig. 18, the sensor or the switches 101, 102, 103, and … having a sensing function are configured as an active (active) sensor having 2 or more functions. In fig. 18, the plurality of sensors 102, 103, and … are also representatively illustrated by 1 sensor 101.

The sensor 101 includes a control data management unit 112 and a network I/F115. The sensor 101 includes a function 1 (first detection function) 116 and a function 2 (second detection function) 117 as sensing elements. However, the type of sensor is not limited to this type, and various types can be used. Of course, in the case where other sensors, switches or control terminals 101, 102, 103, … are switches, the switching on and off is controlled in accordance with the input signal with respect to the machine to which it is primarily connected. Note that, as the switch, for example, there is a case where the switch has a timer function and is turned off after X (X is a positive integer) minutes have elapsed from the time when the switch is turned on. The switch may be combined with a temperature sensor or an infrared sensor, and may be turned on when the temperature reaches a set temperature or when an object emitting a specific temperature (heat) is detected.

The control data management unit 112 includes an application manager (APP-Mg), an event manager (EVT-Mg), and a configuration manager (CONFIG-Mg). The CONFIG-Mg manages various applications for controlling the operation of the sensor 101 and the system as a whole in cooperation with the sensor 101 (associated with the sensor 101). EVT — Mg manages event applications for executing the next actions of sensor 101 based on detection data from function 1 (first detection function) 116 and function 2 (second detection function) 117. The function 1 (first detection function) 116 and the function 2 (second detection function) 117 have various elements depending on the purpose of detection. As various elements, for example, a camera or a microphone may be used as in the HGW 600. The various elements include a heat sensor, a temperature sensor, a humidity sensor, an illumination sensor, a pressure sensor, a switch, and the like. The sensor 101 may include 1 or more detection elements depending on the purpose of use.

The sensors 101, 102, 103, and … are sensors for checking opening and closing of a door or a window, sensors for checking generation of sound larger than a certain level, sensors for checking movement of a person (movement detection/infrared ray), sensors for checking abnormality (impact) of a door or a window, sensors for taking an image (camera), sensors for detecting temperature, sensors for detecting brightness, and the like, and are disposed in various places in a home. The switch detection sensor or the Open detection sensor is preferably a two-circuit type in which a first circuit that outputs when the switch is Open (Open) and a second circuit that outputs when the switch is closed (Close) are integrally provided so that a failure of the sensor itself can be detected. For example, as a sensor for detecting opening and closing of the curtain, a sensor that opens when the curtain is opened (open) and a sensor that opens when the curtain is closed (closed) are combined. Thus, even when one sensor fails, the HGW600 can easily determine the abnormality.

In the above system, when a detection signal is output from any one or more of the sensor 611 (camera), the sensor 613 (microphone), and the sensors 101, 102, and … located at arbitrary positions in the HGW600, the control data management unit 601 recognizes that an event has occurred. Then, the control data management section 601 controls the camera (sensor) 611 via the recording manager 607. Thus, the camera 611 transmits the monitoring data captured before the event occurrence time (for example, 10 minutes ago) to the storage medium via the recording manager 607 and the control data management unit 601, and continues to transmit the monitoring data captured at a certain time (for example, 3 minutes, 5 minutes, 10 minutes, 20 minutes, 30 minutes, or the like) later. In the present system, event-related data (which may also be referred to as event attribute data) at the time of detection of the event is also transmitted to the storage medium 1010 (storage means/memory) together with the monitoring data.

The event-related data may include, for example, any 1 or more of the occurrence time of the event, the kind of sensor (sensor ID) that detected the event, the position data of the sensor, the recording start time, the recording end time, and the like.

In fig. 18, the storage medium 1010 is, for example, a memory in the server 1000, but is not necessarily a storage medium in the server 1000. The storage location of the monitoring data may be a storage medium in the HGW600 or a storage medium connected via the network I/F605. The storage medium 1010 includes a data area 1011 and a management area 1021. The monitoring data 1012 is stored in the data area 1011, and the event-related data 1022 is stored in the management area 1021.

The monitoring data 1012 may include not only video data but also measurement data from a sensor. For example, the temperature change, humidity change, pressure change, and the like of a specific portion. Management data for reproducing the monitoring data is described in the management area 1021. The management data also contains previous event related data. The management data includes event-related data and a record address of monitoring data corresponding to the event-related data. When a plurality of events occur, there are a plurality of event-related data and a plurality of monitoring data corresponding to the plurality of event-related data.

The event-related data includes the category of the event (which may also be referred to as a sensor). Further, although monitoring data (for example, monitoring video) is recorded based on an event, the event-related data includes the recording start time and the recording end time thereof.

However, in order to cause the HGW600 to function as described above, it is necessary to register various devices to be controlled such as various sensors, switches, and control terminals in a home (hereinafter, referred to as target devices, but may be referred to as equipment or simply as equipment) in the equipment management unit. This registration mechanism will be explained below.

< correlation of applicable space with HGW >

Fig. 1 shows a space (front open layout) to which a home network and a setting method (setting support method) according to the embodiment can be applied. In this space, various target devices capable of performing signal transmission and reception with the home network are arranged.

The target device is assumed to be a sensor or a camera for capturing an image, but is not limited thereto, and may be a device providing various other functions for detecting that the surrounding environment has reached a certain condition.

The object device is connected to the home network through the intra-domain communication unit. The target device provides the operation state and/or arbitrary data to the home network through communication or a signal of a specific format in accordance with a protocol determined for each device.

The target devices receive control information conforming to a protocol determined for each target device from the home network via the HGW600, or control data of a specific format prepared by the target devices, which can be provided by various manufacturers, via the HGW 600. And operates based on the received control information or control data. Note that since the sensor 101, which is a specific form of the control data control target device, and the operation of the entire system in cooperation with the sensor 101 (associated with the sensor 101), can be obtained, for example, via the server 1000 or the data table 800 connected to the HGW600 (for the form of data, see, for example, fig. 15, 15A to 15D, 16A, 16B, 17, and 17A). The acquired control data of a specific format may be data that can be recognized from each other and updated from the HGW600 (to the target device) as needed in the sensors 101 and the entire system cooperating with the sensors 101 (associated with the sensors 101).

In the case where the target device is a sensor or a camera, the detection or output condition may be changed from the outside by the HGW 600. In the case where the target device is, for example, a sensor, the HGW600 can change the threshold (detection determination) level when detecting and notifying the detection target. When the target device is, for example, a sensor, the detection timing (detection interval) or the non-detection time may be set. In addition, in the case where the target device is, for example, a camera, the non-shooting period (off time) may be set or the set non-shooting period may be arbitrarily changed. Of course, the user may directly instruct the camera to perform shooting at shooting timing. In this case, the user can set, for example, still image shooting, continuous shooting (moving image) between fixed periods, intermittent (intermittent) shooting at regular intervals between fixed periods, or the like. In still image shooting, shooting may be performed a predetermined number of times when the user instructs shooting.

The HGW600 also connects the home network with an external network (e.g., the internet or the cloud). The HGW600 may also be configured to supply control information or control data to the target device, the control information or control data being determined based on the operation state and/or any data received from the target device, for example, control information or control data of a specific format described later using fig. 15, 15A to 15D, 16A, 16B, 17, and 17A. The specific form of control data includes, for example, a voice instruction (expression itself based on voice) associated with the control information, a determination condition to be coordinated according to the voice instruction, and the like.

The HGW600 can receive a user instruction (operation input) from a terminal device held by a user, for example, a smartphone or a tablet (PC), in an arbitrary communication format (protocol) supported by the HGW 600. The HGW600 can provide an output signal in a message or a predetermined format to a smartphone or a tablet (PC) held by a user, using an arbitrary communication format (protocol) supported by the smartphone or tablet, as necessary.

Fig. 1 shows a space comprising at least 1 floor (building/floor) and at least 1 divided area. The space is a limited space formed by a predetermined range in a relatively small-sized building and around the building, for example, a target area of a network mainly using wireless communication. The building is preferably, for example, a general house, a small-scale commercial facility (a small store), or a small-scale unit. The building may be, for example, a hospital, a child-oriented facility, or an elderly-oriented facility, etc.

As a network (home network) using wireless communication, a network designed in accordance with a standardized standard in international organizations such as ieee (institute of Electrical and Electronics engineers) is preferable. The network is preferably, for example, a network (Bluethooth (registered trademark)) that uses a radio Wave in a 2.4GHz band in conformity with ieee802.15.1, a network (zigbee (registered trademark)) that uses a radio Wave in a 2.4GHz band in conformity with 802.15.4, a network (Z-Wave (registered trademark)) that uses a radio Wave in a 900MHz band in conformity with 802.15.4e, or the like. Examples of usable networks include a network (Wi-Fi (registered trademark)) that uses radio waves of 2.4 or 2.5GHz band in compliance with IEEE802.11, a network (Wi-MAX (registered trademark)) that uses radio waves of 2.5GHz band in compliance with 802.15.16e, and a network (Wi-sun (smart Utility networks)) that uses radio waves of 2.4GHz band in compliance with 802.15.4 g.

Hereinafter, a space to which the home network and the setting support method shown in fig. 1 can be applied will be described as a limited space configured by a predetermined range around a general building (house). It should be noted that the limited space around the building is sometimes referred to as the inside of the land or simply the land.

< specific example of the space to which the present embodiment is applied >

In fig. 1, a building 100 includes a first floor 10 located at a reference level in a floor (limited space) 1, for example, at a height (also referred to as a height) substantially equal to or slightly higher than the ground, a second floor 20 located at a height higher than the first floor 10, a third floor 30 located at a height higher than the second floor 20, and the like. In the three-story building 100 shown in fig. 1, a portion having a height higher by a certain height than the third story 30 may be classified as the fourth story 40. The fourth layer 40 in fig. 1 is sometimes referred to as the uppermost layer. Of course, the number of floors (stories) is arbitrary, and in the case of only the first floor 10, the second floor 20 becomes the uppermost floor. Also, the uppermost layer is sometimes referred to as an eave. The eaves (uppermost layer) need not have a structure distinguished from the eaves, and may be formed flat, for example.

In the building 100 shown in fig. 1, various devices that operate by electric power supplied from a commercial power source and/or a battery (including a fuel cell) are disposed in each of the first section 11 to the fourth section 14 of the first floor 10, the first section 21 to the fourth section 24 of the second floor 20, and the first section 31 to the third section 33 of the third floor 30. As devices, there are known, for example, a television broadcast receiving device (also referred to as a TV device), a lighting device, a door locking device, a window opening/closing device, a curtain opening/closing device, an air conditioner, an elevator, a personal computer (also referred to as a PC), a printer, a microwave oven, a toaster, an electromagnetic cooker (ih (induction heating) stove), a vacuum cleaner, a washing machine, a fan, an electric cooker, a refrigerator, a telephone, a charging device, a socket or a switch for supplying electric power thereto, a sensor for realizing a predetermined function, and the like. In recent years, power generation devices, smart meters (electricity meters), heat reuse (hot-spot cogeneration/tri-cogeneration) devices, power storage devices, monitoring cameras, and the like have been handled as devices. Although most of the sensors are configured to detect that a required detection output (detection result) has reached a condition to be output, the condition to be output can be changed from the outside in practical use. The plurality of sensors have a function of setting a plurality of conditions to be detected and output and/or a function of setting a plurality of detection outputs under different conditions. It is preferable that the HGW600 be capable of autonomous movement within the same layer or within each of the sections described below.

The first layer 10 includes at least a first segment 11, a second segment 12, a third segment 13, and a fourth segment 14, which are each divided into predetermined areas. However, the number of segments is arbitrary, and one layer may be only one segment. The first section 11 is, for example, a section that is located at the same height as the floor or the like and is divided into a predetermined size (area), and is, for example, a garage (also referred to as a garage), a warehouse, or a storage space. The second section 12 is a section adjacent to the first section 11 and located at a height higher than the first section 11 by a predetermined height, and is a space having no wall surface in a part thereof, such as an entrance and a corridor. The third section 13 is a space at least a part of which is connected to the second section 12 through the same plane and is located at a height substantially equal to that of the second section 11, and is, for example, a restaurant (also referred to as a first living room, a single living room, or a single room inside). The fourth zone 14 is a zone in which the second zone 12 and the third zone 13 are connected and at least a part of which is located at the same height as the second zone 12, and is provided as, for example, a stairwell (also referred to as a first-floor stair, a lower stair, a downstair stair, or the like).

The second floor 20 includes at least a first section 21, a second section 22, and a third section 23 each divided into a predetermined area, and a portion of the first section (garage) 11 in the first floor 10 may be referred to as a fourth section 24, and the number of sections is arbitrary, and one floor may be only one section, the first section 21 is a stairwell connected to the fourth section 14 of the first floor 10, and includes a first stairwell 15 connecting a first building (the first floor 10) and a second building (the second floor 20) so that an occupant can move, the second section 22 is a space at least a portion of which is connected to the first section 21 on the same plane and is located at a height substantially equal to that of the first section 21, and is, for example, a bedroom (also referred to as a room on the second living room side or a garage side, or a room of ○○ (for example, a name of a parent family), and the third section 23 is a room or a room having a structure such that at least a portion of which is connected to the first section 21 and is located at the same height as that of the first section 21, and the third section 23 is referred to as a room (also referred to as a room having a drying structure of a hanging table).

The third layer 30 includes at least a first segment 31 and a second segment 32 each divided into a predetermined area. The number of segments is arbitrary, and one layer may be only one segment. The first section 31 is a stairwell connected to the first section 21 of the second floor 20, and includes a second stair 25 (also referred to as a second-floor stair, an upper stair, a lower stair, or the like) that connects the second floor (the second floor 20) and the third floor (the third floor 30) so that occupants can move. The second section 32 is a space at least a part of which is connected to the first section 31 through the same plane and is located at a height substantially equal to that of the first section 31, and is, for example, a roof (also referred to as a terrace, a third floor, an eave, or the like). It should be noted that the space which is substantially the third section 23 of the second layer 20 and is located above the section may be distinguished from the third section 23 of the second layer 20 as, for example, a vent (third section of the third layer 30) 33. That is, as will be described later, the third section 23 of the second floor 20 in which the device to be controlled is located can be used (i.e., the determination of the controlled object) by using 1 determination method out of a plurality of expressions, as a name used for determining a sun screen (and/or an open/close sensor for a high window as a detection object), for example. For example, with respect to the screen (and/or the sun screen for high windows) in the ventilation area of the third section 23 of the second floor 20, the same control target can be controlled both in the case of a state of being viewed from the upper part (third floor) of the first section (staircase) 31 of the third floor 30, which is referred to as a "screen for third floor", and in the case of a state of being viewed from the second floor, which is referred to as a "sun screen for high windows".

The fourth floor 40 includes eaves 41 that cover the first section (second stairwell) 31 of the third floor 30 and the third section 23 (vent (third section 33 of the third floor 30)) of the second floor 20. The eaves 41 preferably have, for example, a solar power generation device (solar power generation panel) 42.

In the plot 1, it is preferable to provide a detection mechanism capable of detecting an object moving back and forth within the plot 1, such as a cat or a dog, in a portion (non-building space) other than the portion where the building (house) 100 is located. The detection means is preferably configured to be able to detect and record an object even at night, such as a camera unit, an illumination unit, or a camera (infrared camera) capable of taking an image by infrared rays.

In the building 100 shown in fig. 1, the HGW600 is located, for example, at any position of the second section 12 to the fourth section 14 of the first floor 10 and/or at any position of the first section 21 to the fourth section 23 of the second floor 20. HGW600 is preferably capable of autonomous movement within the same layer, which may be considered to be in substantially the same plane, or within various segments described below. In the same layer or in each zone, there is a case where the unevenness in the height direction of, for example, about several cm (centimeters) is included. The HGW600 may have a mechanism suitable for the movement in the height direction, such as an endless track type driving mechanism (crawler) or a lifting mechanism, in order to perform the movement in the height direction in substantially the same layer or each zone. The lifting mechanism may be provided independently of a pushing (moving) mechanism for moving the plane. The HGW600 may also be a structure capable of recovering, for example, dust or dirt in an area where it moves. When the driving mechanism for movement is in the form of a roller, a brush or the like capable of moving dust or dirt, for example, may be integrally provided.

The HGW600 is capable of receiving detection outputs from respective (any one of) sensors located in respective sections of the building 100. The HGW600 also outputs (supplies) a control signal conforming to a protocol previously determined for each control target device, control data conforming to a specific signal format, or the like to an arbitrary control target device. For example, when 2 or more HGWs 600 are present in two or the same layer of the first layer 10, the second layer 20, or the third layer 30, any HGW600 becomes a master. In this case, the remaining HGWs 600 become slave machines. When another communication unit provided independently of the HGW600 is already present, all the HGWs 600 may be slave devices. On the other hand, when other communication units already existing can function as slave devices to the HGWs 600, any HGW600 may be a master device and the other communication units may be slave devices. The HGW600 can also communicate with a smartphone or a television device registered in the device management unit via a network. When a specific application or program is started in the smartphone or the television device, the smartphone or the television device enters a mutual communication mode with the HGW 600.

< example of producing a spatial layout diagram to which the present embodiment is applied >

Fig. 2 illustrates in plan the various levels of the building 100 shown in fig. 1. In fig. 2, the fourth floor (uppermost part) of the building 100 shown in fig. 1 is not shown.

Fig. 2(a) shows the first section 11 to the fourth section 14 of the first layer 10, fig. 2(b) shows the first section 21 to the third section 23 of the second layer 20, and fig. 2(c) shows the first section 31 and the second section 32 of the third layer 30.

Hereinafter, an example of the procedure will be described with respect to a case where a user wants to add a lighting device that can be turned on/off at an arbitrary timing from an arbitrary position (room, section, or place of use) in the first section (storage/garage) 11 of the first floor (first floor) 10. Note that a socket (Electrical outlet) is already provided in the vicinity of a position where a lighting device of the first section 11 of the first layer 10 is to be added.

First, a mapping ("layout") mode within the sector is initiated.

In the initial setting screen display shown in fig. 3, for example, when the user clicks a [ layout creation ] button (display) 511 displayed in a display area 501 of a smartphone (input terminal apparatus) 500, a "layout creation" mode of an example of the display shown in fig. 4 is activated. The click may be referred to as a touch or a tap, but is not limited to an operation in which the user touches a button (display) on the screen display with a finger and then moves the finger away from the button for a certain period of time. However, it is preferable to distinguish an operation in which the user separates the finger from the button after touching with the finger for a certain time (so-called long press). In addition to the [ layout drawing ] button (display) 511, the display area 501 preferably displays, for example, a [ device registration ] button (display) 512 for activating a "device registration" mode, a "return/cancel ] button (display) 513 for displaying a previous screen display, and the like.

In the "map creation" mode shown in fig. 4, when the user clicks the "on" or "start/start" button 521 displayed in the display area 501, the GPS (global Positioning system)/G L ONASS (global navigation Satellite system) signal reception mode of the smartphone 500 is activated, in the "map creation" mode, the smartphone 500 specifies the position in the space where the user is present based on the received GPS/G L ONASS signal, and holds the position as reference position information, the reference position information includes a latitude (L), a longitude (L) and an Altitude (alert), and, as shown in fig. 5, the current position 541 of the user corresponding to the reference position information is virtually displayed at an arbitrary position in the display area 501.

In the "map creation" mode, a virtual map (map) is created in a predetermined area in the memory (RAM)502 based on the position (destination) of the user in the object space (in this case, in the first section 11) specified based on the successively (repeatedly) received GPS/G L ONASS signals and the reference position information (previously specified).

The smartphone 500 can communicate with the HGW600 described later. Therefore, data such as a layout created by the smartphone 500 is temporarily stored in the memory in the smartphone 500, but data to be stored is stored in the memory, the device management unit, and the like in the HGW600 via the network.

In this case, the application of the smartphone 500 can present to the user on the screen a movement to a position where the GPS/G L ONASS signal can be received and can specify a reference position, and when the application moves to a GPS/G L ONASS signal reception unstable space after receiving the GPS/G L ONASS signal, the output of the gyroscope (direction sensor) of the smartphone 500, the output of the acceleration sensor, and the movement time are used to calculate the size of the space where the user moves, and virtually create a layout diagram.

In addition, in the middle of the mapping, there is a case where the smartphone 500 has moved to the GPS/G L ONASS signal reception unstable area after receiving the GPS/G L ONASS signal, in such a case, for example, with respect to a space where layout information is disclosed such as a stadium or the like installed together with a living room, the application of the smartphone 500 can acquire the layout information via the internet, can use the layout information in combination, and can then increase the detection accuracy of the size (layout) of the calculated space.

Specifically, as shown in fig. 5, the smartphone 500 can display the current position 541 of the user and the message 531 in the display area 501. The message 531 is a message such as "please move to the boundary of a wall, a window, or a room (section) in the front direction", that is, a message urging the user to move in the object space.

In fig. 5, the current position 541 of the user is a position where the user has clicked the [ open ] or [ start/start ] button 521 in the "map creation" mode shown in fig. 4, and becomes a reference position later.

The reference position is also held when information of a map (virtual map) created in the memory 502 is updated.

The current position 541 ' of the user is created based on the difference between the GPS/G L ONASS signal of the current position and the GPS/G L ONASS signal of the reference position, which are received successively, and the current position 541 ' is displayed in the display area 501 as a result of updating the information of the map (virtual map) created in the memory 502, and the current position 541 ' of the user is continuously displayed in the display area 501 as the trajectory along which the user moves.

As the user continues to move within the first section 11, at the boundary of the wall or window or room, the user changes the direction of movement. Therefore, the smartphone 500 detects the reference position or its vicinity at a predetermined timing when the change in the movement direction of the tenth time is detected in a state where the "map creation" mode is activated. At this time, the information of the layout drawing being created in the memory 502 is information that can be approximated to a closed space, that is, a rectangle, a triangle, or a polygon. Therefore, the trajectory 541 'of the user's movement displayed in the display area 501 can be displayed as a plan view showing a certain section as shown in fig. 6. In the application/program used in the "map creation" mode, the track 541' displayed in the display area 501 is converted (or approximated) to be displayed linearly as illustrated as a rectangle 542 in fig. 8 or 9.

In the "map creation" mode, when it is estimated that the trajectory 541' of the user movement, i.e., the information of the map is in a shape similar to a room, a message 532 and a button 522 are displayed as shown in fig. 7. The message 532 is, for example, "do the shape of the section be registered as a layout? "etc., and the button 522 is a button that receives an instruction input from the user, such as [ register ] or [ yes ] or [ redo ] or [ cancel ], and the like.

When the instruction input (click/touch on the screen) from the user is an instruction input to confirm (register as a map) the created map, such as [ register ] or [ yes ], the display area 501 is changed to the display as shown in fig. 8. That is, display "is the next room (section) set? A message 533 to prompt confirmation of the next action of the user, and a button display 523 to receive an instruction input from the user, such as [ continue ] or [ yes ] or [ register and end ].

Here, when the instruction input (click) from the user is [ continue ] or [ yes ], a message for prompting the user to move to the next room (section) is displayed in the display area 501, and thereafter, the layout in the memory 502 is updated successively based on the position of the user's destination of movement and the reference position specified by the GPS/G L ONASS signal received successively.

In fig. 8, in a case where the instruction input (click) from the user is [ register and end ], as shown in fig. 9, "will register the device in the room (section)? A message 534 urging registration of the apparatus such as "and a button 524 such as [ register ] or [ yes ] or [ register later ] are displayed in the display area 501. When the instruction input (click) by the user is [ register later ], the "map" completed at the current time is stored in a predetermined area of the memory 502, and the "map creation" mode is ended.

In this case, as the information of the "map", as shown in fig. 10, the latitude (L satude), the longitude (L ongitude), and the Altitude (Altitude) are stored for each block ([ zone ]). in the case where the shape of the block is the rectangle 11 shown in fig. 9, the latitude and the longitude are respectively specified and stored by dd ° (degrees), mm' (minutes), and ss ″ (seconds) for each corner 11-1, 11-2, 11-3, and 11-4, respectively, it is preferable that the block 11 shown in fig. 9 is a rectangle and includes 4 corners, and therefore, it is preferable that the position of each of the latitude and longitude of the zone (block) shown in fig. 10 is displayed by adding a label specifying each corner together with a label indicating the latitude or longitude.

The information of the "layout map" is usually assigned temporary segment names of A (area)11-1, A (area)11-2, and … in the order in which the layout map is created, for example. The temporary segment name is then utilized in assigning names to the rooms.

< example of registration method for devices such as sensor and controlled object device >

As described above, there are TV devices, lighting devices, door locking devices, window opening/closing devices, curtain opening/closing devices, air conditioners, elevators, PCs (personal computers), printers, microwave ovens, bread toasters, electromagnetic cookers, dust collectors, washing machines, fans, electric rice cookers, refrigerators, telephones, charging devices, sockets, switches, sensors, power generation devices, power meters, heat reuse devices, power storage devices, monitoring cameras, and the like.

Next, "registration of a device" is explained.

In the case where the [ registration of device ] button (display) 512 is clicked on the initial setting screen shown in fig. 3, or in the case where the "registration of device" mode is selected in the above-described "map creation" mode (in the "map creation" mode, the click (input) of the button 524 from the user is [ registration ] or [ yes ]), for example, the "registration of device" mode shown in fig. 11 is started.

Fig. 11 shows an example in which, for example, button displays 551([ new registration ]), 552([ registration change ]), and 553([ registration deletion ]) prompting the user to select (input) any one of [ new registration ], [ registration change ], and [ registration deletion ] are displayed in the display area 501 as a method (category) of "registration of device".

For example, when registration of a new (or additional) device is instructed by the [ new (additional) registration ] button 551, the "registration of device" mode for guiding the user is activated by the display as shown in fig. 13A to 13I/fig. 19A to 19C/fig. 20A to 20C. The application/program used in the "registration of device" mode can automatically determine the specifications of a plurality of wireless communications and automatically determine the communication mode.

Thus, the user can easily register the device without being aware of "how the specification of wireless communication is adopted by the new or additional device (sensor, control target device)". In the detection of "how the standard of wireless communication the additional device belongs to", it is preferable that a display such as "detection communication method" is displayed in the display area 501 at a stage before the screen of each "registration of device" mode is displayed.

Here, before "registration of the device", it is preferable to cause the user to register "name of room" by displaying a display 560 of "name bar of registered room" or the like which urges registration of the name (name) of the room (section) as shown in fig. 12.

In registering "name of room", information such as "layout diagram" shown in fig. 10 is used. For example, when the user enters a room in which a name is to be registered and clicks on "new registration" 551, if the name of the room is not set, the temporary segment name of the current room (for example, any one of a (area)11-1, a (area)11-2, and … shown in fig. 10) is displayed. When the user inputs a room name by voice or via a character input screen and clicks a decision button, the inputted room name is displayed next to the temporary segment name (e.g., a (area) 11-1).

Note that registration of the names of rooms may be started from a message such as "register the name bar of a room" when registration of a room is made after the display shown in fig. 8 or 9 is made.

Fig. 13A to 13I show an example of registration procedures in the "registration of device" mode when the network specification of wireless communication used by the device to be registered (communication specification suitable for the device) is, for example, a structure of 900MHz band [ type a < Z-Wave > ] conforming to ieee802.15.4e.

Fig. 13A is an example of display at the time of activation of the "registration of device" mode. Fig. 13A shows the following: on the "layout (plane)" registered in the "layout creation" mode explained using fig. 4 to 8, the state of the current position of the user is planarly displayed. Then, a message 561 such as "please move to the vicinity of the device to be registered" is displayed in the display area 501, and the user is prompted to specify the device to be registered. The activation of the device varies depending on the shape of the device and the state in which the device is fixed, but conforms to standard specifications such as turning on an activation switch (embedded reset button) provided at a predetermined position of the device.

For example, when the time is set in a remote controller, a clock, or the like, the reset button may be turned on (input mode) when the reset button is pressed by the tip of a pen/ball-point pen, or turned off (non-input mode) when the reset button is pressed again.

When the user moves to the vicinity of a device to be registered (described by taking the device 571 as an example) and the user recognizes the device 571 and activates the device 571, the display shown in fig. 13B is obtained. That is, the "layout (plane)" 542 is updated to a state in which the position of the device 571 is displayed together with the position 543 of the user.

At this time, as shown in fig. 13C, a display area 501 is displayed in the form of a list 572, including an Image (Image) showing a schematic shape of the device 571, a standard function-by-function logo name such as a switch (On/offset), a sensor (Motion sensor), a sensor (Open/close sensor), and a socket (electrically connectable/rechargeable outlet), a product specification, and/or an individual name (model number) prepared by a supplier.

The list 572 is obtained as follows, for example. That is, when the device 571 is started, the HGW600 recognizes the device 571, and transmits information of, for example, the manufacturer of the device 571 to the cloud server via the network. Then, the smartphone 500 is transferred with the data of the list 572 from the cloud server, and therefore the HGW600 transmits the received data of the list 572 to the smartphone 500. Thereby, the smartphone 500 can display the previous list 572.

The cloud server stores feature tables as shown in fig. 15, 15A, 15B, 15C, and 15D, which will be described later. The feature table may be downloaded to HGW600 and stored in a memory within HGW 600. However, in this case, there is a case where the feature table is updated, and therefore the HGW600 may be fetched from the cloud server when receiving the notification of the presence of the update or periodically.

In the feature tables shown in fig. 15, 15A, 15B, 15C, and 15D, for example, column C102 includes information on the manufacturer of the device, column C103 includes information on the product name of the device, column C104 includes information on the identification name of the device, column C105 includes information on the function of the device, and column C110 includes a display example (Image data) of the device. The information is constructed by each manufacturer uploading to the cloud server.

In fig. 13C, the display position, display order, superimposition, transmission/non-transmission when superimposed, and other display-related rules of the list 572, the "layout (plane)" display 542, the message 573, and the button 525 are arbitrary. It is preferable that the display size is controlled in accordance with the size of the display area 501, and the display amount of each screen display is controlled, and all screen displays are displayed by scrolling. For example, the setting of the smartphone 500 may be applied regardless of the display rule adopted by the application/program used in the "registration of device" mode.

However, the device of [ type a ] is not configured to autonomously activate the device in the "registration of device" mode, and therefore requires the user to activate each device for registration. In this case, the user needs to move to the vicinity of the target device and activate the device, and thus each device can be easily identified. This device of [ type a ] has a feature that only 1 device that is activated is displayed in the "map (plane)" display 542.

Although there are other types ("type B", "type C") in which a plurality of devices are simultaneously activated, unlike the device of "type a", other steps for determining a device to be registered are required when using the device of the type. This apparatus is explained later with reference to fig. 19A to 19C/fig. 20A to 20C.

Next, for example, "register the device? A message 573 of "or the like, and a button 525 for receiving an instruction input such as [ register ] or [ yes ] or [ redo ] or [ abort ] are displayed on the display area 501.

In the screen display of fig. 13C, when the user makes an instruction input (click) of [ register ] or [ yes ], the screen display is switched to the screen display shown in fig. 13D. Namely, the following can be shown: a message 574 such as "set a name bar (please set the name of the device)" to the device, an input field 575 of the name, a button 525 for receiving an instruction input such as [ register ] or [ yes ] or [ redo ] or [ abandon ], and the like are displayed on the display field 501.

In fig. 13D, the "layout" display 542, the message 574, the input region 575, and/or the button 525 have any display-related rule (actual display form) such as the display position, the display order, the superimposition, and the transmission/non-transmission at the time of superimposition. It is preferable that the size of each display is controlled in accordance with the size of the display area 501, and the display amount of each screen display is controlled, and all screen displays are displayed by scrolling. However, the display sizes of the respective display portions may be reduced to display all the display contents in the display area 501.

As shown in fig. 13E, a message 575 such as "set nickname" may be displayed, and a nickname may be input into the input area 581.

In addition, as shown in fig. 13F, in the case where the name (name) of the device input by the user already exists, it is displayed that, for example, "the device of the same name has already been registered. A message 576 such as "enter other name bar" urging the user to avoid registration of duplicate names (names), which are displayed in the input area 581.

As described above, in fig. 13D, although fig. 10 shows an example in the case where the instruction input (click) from the user is [ register ] or [ yes ], as the information of the position of the Device in the "layout diagram", the latitude (L atitude), the longitude (L ongitude), and the Altitude (Altitude) are registered in a predetermined area in the memory 502 together with the Zone (section) name (Zone) and the registered Device name (Device).

When the registration of the name (name) of the device by the user is completed from the screen display shown in fig. 13D, as shown in fig. 13E, a message 575 for urging the user to perform an operation, such as "set a nickname (nickname) to N (N is a positive integer)", and at least 1 input region 581, … can be displayed in the display region 501. The input areas 581 and … may be in the form of displaying the next input area when the input of 1 nickname is completed, or in the form of displaying N input areas in advance as shown in fig. 13E. The display of the input regions 581 and … is preferably skipped by a user operation, and may be added (or edited) at a predetermined timing using, for example, a "name edit" mode. However, in the case where the input areas 581, … already have a nickname of the device input by the user, it is preferable to display, for example, "the device of the same name is already registered" as shown in fig. 13F. A message 576 to prompt the user to avoid duplicate nickname registrations, such as entering another nickname bar. In this case, it is preferable to display the unique characteristics of the device as shown in fig. 13H by referring to the product name unique to the device held in the column C103 or the identification name (product specification) held in the column C104 of the characteristic table shown in fig. 15.

Note that, in order to describe the nickname, the name of the device (of the device) whose registration is completed is referred to as "the spotlight of the garage". Further, as nicknames, there are various revisions (changing expressions) such as "illumination of a garage", "light-gathering of a garage (ガレージ for japanese)," light-gathering of a garage "," light outside the first floor (an example of an expression not necessarily specified as "garage"), "garage, illumination (sentence break exist)", "spotlight, garage (sentence break exist)", and the like. The names and nicknames of the registered devices are registered in column C107 (audio instruction column) in fig. 15 and 15B in association with the control information shown in column C106. Thus, even if the device having completed device registration described above uses a plurality of names (names) by combining a name (name) given as a name (name) and a nickname, the device can be determined to be the same device.

However, in the following case, the HGW600 can transmit feedback (challenge) for specifying the device to a terminal device held by the user. When the HGW600 includes an audio output mechanism, the feedback may be directly output as audio by the audio output mechanism. There is a condition that the HGW600 feeds back to the user (reason for the feedback). For example, there may be a case where devices with similar names (names) exist in addition to the above, a case where a device with the same name exists in a different field and the field cannot be distinguished, or a case where the user's instruction (speech/voice) is unclear and the device and/or the field cannot be secured.

When the user has completed inputting at least 1 nickname or skipped inputting the nickname in the input areas 581 and …, a message 577 urging the user to perform an action, such as "decide a task bar (define action/function)", and at least 1 input area 591 are displayed in the display area 501 as shown in fig. 13G.

The devices registered as "spot lights for garages" are the following: in this embodiment, by being notified of the presence of an indication (input) of a sound (voice) via the HGW600, it is possible to turn on/off the lighting device connected via the socket that has been set. That is, by being notified of a sound instruction such as "the garage spotlight is on" or "the garage spotlight is on" via the HGW600, the above-described device (in this example, the device to which the garage spotlight is connected) is controlled so that the connected lighting device is turned on (lit). Further, by being notified of the presence of an instruction of a sound such as "the spotlight is off in the garage" or "the spotlight is off in the garage" via the HGW600, the device is controlled so that the connected lighting device is turned off (extinguished).

That is, the task of the device registered as the "spotlight of the garage" is "on" and "off" of the lighting device. The device (lamp) registered as "living room lighting" is intended to perform lighting control on a control unit (controller) in which the lamp (device) is incorporated in advance. From such a background, actually, a task is specified (determined) based on column C113 (audio instruction) and column C117 (control information) of the intention explanation (data table) shown in fig. 16, 16A, and 16B, with respect to information (required operation of the device) obtained from any one of the name of the device specified by the user by voice and "on" or "on" and "off" or "off". Based on the task of determination (determination), the control mode is read from column C118 in fig. 16A and 16B, and notified to the device to be controlled via HGW 600.

Even if the apparatus of [ type a ] cannot directly communicate with the HGW600, the signal from the HGW600 to the apparatus and the signal from the apparatus to the HGW600 can be popped up (relayed) by using another apparatus of [ type a ] as a relay apparatus. This makes it possible to suppress the communication capabilities of the devices of [ type a ] and to construct a network with low power consumption.

Note that, in some cases, a sound instruction such as "on" or "off" does not include a direct instruction. For example, in the sound indication of turning on/off the lamp, there is a (apparent) dissimilar sound indication such as "Good night"/"Good dream" (or "come a dream") or "come out from home". However, the control of the sound is linked in such a way that whichever sound indication can be interpreted as the turning off ("turning off") of the light. In addition, for the sound indication such as "good night (Goodnight)"/"good dream (Have a nice dream)", there are various substitution expressions such as "sleeping good" (Sleepwell) "and" Sweet dreams (Sweet dreams) ". Therefore, the various voice instructions are associated with the operation mode of the device so that an appropriate operation mode can be determined from (the expression of) the various voice instructions even when the actual operation mode of the device shown in fig. 16A is determined based on the association between the voice instructions and the rules shown in fig. 15B to 15D, and the details will be described later.

Similarly, for example, regarding the fan (ventilation fan), the control of the sound is associated so that the sound instruction such as "sound-proof (sound is generated indoors)" or "listening bar (quiet point)", "movie start now (quiet point)" can be interpreted as the turning off ("stop (off)") of the fan (ventilation fan). however, regarding the fan, as shown in column C107 of fig. 15B, the speed control (speed change) can be performed.

However, there are cases where the task of the device, i.e., the operation/function of the device, is uniquely determined based on the product specification of the device held in the feature table shown in fig. 15 and 15A to 15D, as described above, in the case where the product specification of the device is, for example, a switch capable of turning on/off only the power supply, as shown in fig. 13H, the name of the device that can be obtained based on the product specification of the device (the content of the feature table) may be displayed in a region 578 that has been used as an input region up to this point (a region that is not displayed for other display), as shown in fig. 13H, for example, △△ (standard function-by-function name) where "○○ (the name that has been input) is ◇◇ (the name of a room), here, a message 579 such as" on/off the spotlight where the device is a garage "may be displayed, as shown in fig. 15C, a message 579 such as" on/off the spotlight 'of the device' may be prepared for a "lighting and a sound reproduction by a" sound feedback mechanism "may be a sound reproduction by a multi-stage sound feedback (a sound reproduction control mode such as a" light and a sound reproduction "may be provided, or a sound reproduction mechanism" may be provided, a sound reproduction (a sound reproduction mode "may be provided by a sound reproduction device (a sound reproduction mode" playback mechanism "w" (a sound reproduction mode "may be provided, a sound reproduction mode, or a sound reproduction mode" may be provided such as a sound reproduction mode "may.

When the target device is, for example, a sensor (temperature sensor) and the task (operation/function) thereof is a temperature check of a room, temperature data of the temperature at the time when the query (or the transmission instruction from the user) is received is usually output. However, in the case where a change in a specific pattern is detected, for example, the humidity exceeds 77% and the temperature exceeds 31 ℃, it is also possible to autonomously notify that a change in temperature has occurred. For example, as shown in fig. 13I, it is possible to set a function (task) of the display device in the region 578 used as the input region up to this point, and display a message 579 such as "when a certain temperature is exceeded, the detected temperature is notified" in the remaining portion (region not displayed for other display) of the display region 501.

Fig. 10 shows an example of sector (Zone) position information (data table), the data table shown in fig. 10 is located in, for example, an external server, and holds data of latitude (L satude), longitude (L engitude), and Altitude (Altitude) for each sector (Zone), the data of latitude (L satude), longitude (L engitude), and Altitude (Altitude) held in the data table shown in fig. 10 can be acquired by the movement of the user as described with reference to fig. 3 to 7, but in this case, for example, the HGW600 may be acquired by the movement of the user itself, and in this case, the trajectory of the movement of the HGW600 is stored in the sector position information (data table) at a predetermined timing, and it is needless to say that, when the position information of each sector (Zone) acquired by the HGW600 moving itself exists, the "map creation" mode in the sector described with reference to fig. 3 to 8 may be omitted.

Fig. 14 shows an example of in-Zone (location) information (data table), the in-Zone location information data table shown in fig. 14 holds, for example, the latitude (L, atitude), the longitude (L, ongitude), and the Altitude (Altitude) for devices located in the storage medium 1010 of the server 1000 and located in arbitrary zones (zones) A, B, …, and it should be noted that, in fig. 14, for the zones (zones) A, B, C, …, a correspondence relationship with the temporary Zone names assigned in fig. 10, that is, a temporary Zone name corresponding to the Zone (Zone) in which the current user is located is shown in a predetermined order, that is, for example, a case where the Zone (Zone)11 corresponds to the Zone (Zone) a (area)11-1, the Zone (Zone)12 corresponds to the Zone (Zone) a (area)11-4, the Zone (Zone)13 corresponds to the Zone (Zone) a (area)11-3, and the Zone (Zone) 14) corresponds to the Zone (area) a (area) 2 a (area).

As described above with reference to fig. 9, 11, 13A, 13B, and 13C, the data of the latitude (L attiude), the longitude (L engitude), and the Altitude (Altitude) held in the data table shown in fig. 14 is stored in association with the device at the time of registration by the user performing the registration operation for the device to be registered.

The position information (latitude, longitude, and altitude) held in the data table shown in fig. 14 is stored for each device every time the user registers the device. The location information shown in fig. 14 may be used to analogize (estimate) the zone when the device is registered next, and the location information shown in fig. 14 may be used to display the closest zone of the analogized (estimated) zones as a default zone in the display area 501 of the input terminal device (smartphone) 500 via the GUI-1.

Fig. 15 and 15A to 15D show an example of device (sensor) characteristic information (data table). The device (sensor) characteristic information data table shown in fig. 15 and fig. 15A to 15D is located in the storage medium 1010 of the server 1000, for example, and includes, for example, column C100 to column C110.

In fig. 15, a column C100 indicates a serial number (management number) for identifying an arbitrary number of devices used in the embodiment. The column C101 holds information on the specification of wireless communication corresponding to an arbitrary device. Column C102 holds information, for example, of the manufacturer of the device. Column C103 holds, for example, the product name inherent to the device. Column C104 holds, for example, an identification name (product specification) which is information capable of uniquely identifying the function of the device. The reference numerals shown in the column C104 preferably indicate functions unique to the devices, such as "switch", "sensor", and "fan". Column C105 holds, for example, the functions of the devices and information related to the control methods. The function (name) of the device indicated in the column C105 is the same as the identification name indicated in the column C104, and the function (name) may directly indicate the function specific to the device and the operation thereof. Therefore, in the "registration of devices" described with reference to fig. 13D, 13E, 13F, and 13G, it is preferable that functions (names) are displayed as default on the display area 501 of the smartphone 500 by the GUI-1. Column C106 holds information related to the control method for implementing the function of the device. The information held in the column C106 corresponds one-to-one to the functions of the devices shown in the column C105.

Column C107 holds the relationship between the names or nicknames of the devices and the operation of each device when instructed by voice. That is, based on the information registered in the column C107 and associated with the rereading (changing language) held in the column C107, when the apparatus having completed the registration instructs an operation in various changing languages, the operation can be specified. For example, the case where a voice indication of "good night"/"good dream" or "come out from home" is recognized as a voice indication for determining "off" is as described above.

The column C108 holds rules for determining the operation of various devices held in the column C107 and for the operation of the determination result. The rule held in the column C108 is composed of, for example, "< IF (IF …/at …) > - < THEN (proceed-) >" sentence structure, and is characterized by, for example, a representative such as "Turn on Dad button light (Turn on the lamp of Dad Bedroom)", "Turn on garage light (Turn on the lamp of garage/Turn on the lamp of garage)", "Turn off pass light (the lamp of opening the passage)" or "Set/wave temperature up passage room for 70F (the temperature in the bathroom is (heated) to (about) 21 ℃ before bathing)," and the action of the device with which the association is established in advance is realized (< THEN >) based on the output from the device (< IF >). In this case, the apparatus to which the association is established in advance may be the own apparatus or an apparatus according to the determined correspondence relationship. For example, when the temperature detected by a temperature sensor (device) of a certain room is a temperature at which the air conditioner of the room should be operated, the HGW600 outputs an instruction (to the air conditioner) to operate the air conditioner of the room (by a cooling operation or a heating operation) so as to reach a predetermined temperature. For example, in the case of receiving a notification of "detectedpermentation of the room ABCover DE ℃ (the temperature of the room ABC exceeds DE ℃)" (< IF >), the HGW600 preferably outputs "settemetration of the ABC room for FG ℃ with the air conditioner (turning on the air conditioner, setting the temperature of the room ABC to FG ℃)" (< THEN >).

Note that, regarding the rule constituted by the sentence structure of "< IF (IF …/in …) > < THEN (proceeding- >) >," a plurality of < THEN > can be set for 1 < IF >, as already described.

For example, as shown in row C108 of fig. 15, 15C, and 15D, the lamp in the living room may be turned on when a sound such as "on" indicates the presence, or when the brightness of the section (room) is darker than a certain illuminance, for example. Of course, it is also possible to have the light turned on with the presence of a sound indication such as "on". In the case of a lamp whose brightness can be adjusted, the brightness at the start of lighting may be fed back to the user, or the brightness at the start of lighting may be set by the user.

When the light in the living room is turned off in response to the voice instruction "go out from home", the light may be turned off after a certain time (for example, 1 minute or N (N is a positive integer)) has elapsed in consideration of the moving time until the user goes out from the room (section). On the other hand, when the lamp in the living room is turned off in response to the voice indication such as "good night"/"good dream", it is necessary to consider a case where the user stays (remains in the section) as it is. In this case, it is preferable to turn off the lamp at a point when it is detected that the user goes out of the zone based on, for example, the surrounding sensor output (not to turn off during the stay of the user in the zone). Alternatively, control may be performed in a stepwise manner such that the brightness of the lamp is reduced every time a certain time (for example, about 15 seconds or 1 minute) elapses.

For example, as the sound instruction for the fan (ventilation fan), as shown in fig. 15B, 15C, and 15D, it is necessary to consider a case where an instruction of not explicitly showing on and off such as "sound-proof" is given, in this case, it is determined that the fan is off because it is expected that a large sound can be emitted in this section, for example, it is possible to switch the speed of the fan to an operation of 1-stage or 2-stage low speed in the sound instruction of "listening to music bar" or "movie bar from now on", on the other hand, in the sound instruction of "ventilation", it is possible to switch the speed of the fan to an operation of 1-stage or 2-stage low speed in the case where the fan is turned on already, and it is preferable to operate the fan at a low speed (L ow) in the state where the fan is turned off in accordance with the sound instruction of "sound-proof", for example, in the case where the temperature sensor located in the same section detects that the temperature exceeds a predetermined temperature.

The column C109 holds information on an interface for controlling the operation of the target device based on the contents held in the columns C106 to C108, for example, information on an interface to be actually controlled, such as on (or off) of a control switch or a change in (set) temperature (a change in the click voltage of the controller).

Column C110 holds: in the "registration of devices" described with reference to fig. 13D, 13E, 13F, and 13G, data of an image of a target device displayed to be visually presented to the user is preferable.

Fig. 16, 16A, and 16B show an example of the intention interpretation (data table). The intention interpretation data table shown in fig. 16, 16A, and 16B is located, for example, at an external server, and includes, for example, column C111-column C118.

The column C111 holds a management number (Device ID)/serial number (Item No.) of each Device unique to the home network. The management number (Device ID)/serial number (Item No.) indicates a correspondence relationship with an arbitrary Device held in the sensor characteristic information (data table) shown in fig. 15. The management number (Device ID) is associated with the serial number (Item No.) held in the column C100 in fig. 15, and is used for managing each Device by using the HGW600 in the home network described below in fig. 16, 16A, and 16B.

Column C112 holds the trade names in the characteristic information of the devices held by column C103 in fig. 15. This ensures the correspondence between the management number and the device whose registration is completed.

The column C113 holds information on various readouts (changing expressions) for identifying the voice instructions (the phrases constituting the voice instructions) indicated by the column C107, among the feature information of the sensors held in fig. 15 and 15B. That is, the column C117 holds information on the control method for realizing the function of the device held in the column C106 among the feature information shown in fig. 15 and 15B, and information in which the association of the above-described voice instruction or phrase is associated with information input by the user (for example, by voice). Thus, when a voice instruction in various languages (expressions) is present for a device to which a name shown in the column C116 is assigned, the corresponding device can be identified, and the control information (what operation is requested) held in the column C117 can be identified.

Column C114 holds information of each section held in fig. 14. The information of each section held in the column C114 corresponds to the space shown in fig. 1, that is, the section of the building 100.

The column C115 holds a segment name corresponding to information of each segment held in the column C114, and the segment name may be arbitrarily set by a user, and for example, the name (name) of a room set in fig. 12 is held, and it should be noted that the segment name is also located as a nickname corresponding to an arbitrary room (segment). for example, in fig. 1, the same segment (room) can be recognized even in the case where the segment 23 corresponding to the living room (L iving) is called by various revisions (change expressions) such as "living room (living room)", "room where people are located", "room with a television on the second floor", and "room with a pet game", and similarly, various modifications such as "garage" (japanese translation) (ガレージ for corresponding japanese language), "outside" and the like can be preferably applied to the segment 11 corresponding to a slightly larger space (parking space).

Column C116 holds a product name and/or an identification name (product specification) unique to the device.

Column C117 holds control information (the same as in fig. 15 and 15A) relating the voice commands (the phrases constituting the voice commands) held in column C107 in fig. 15.

Column C118 holds information on the actual control method (control mode) for realizing the function of the device held in column C106 in fig. 15. That is, the column C118 shows a task of determining (judging) a control mode for actually controlling the target device based on the column C113 (sound instruction) and the column C117 (control information) according to the intention interpretation (data table) shown in fig. 16 and fig. 16A and 16B. The control mode shows the operation of the device recognized by the above-described voice indication, and shows a case where the timing (mode) of turning off differs depending on the respective voice indications such as "turn off", "come out relatively far", "late"/"good dream", for example, when the lighting (lamp) is turned off.

For example, in the case where the sound indication is "off", it is preferable to immediately turn off the lamp (lighting) in accordance with the case where the control amount is 0x 00. In contrast, in the case where the voice is indicated as "come out from back", although the control amount is 0x00, in the former stage, it is preferable that the lamp is turned off when the door sensor detects that, for example, the door is temporarily opened and then turned off. In addition, when the sound instruction is "good night"/"good dream", it is preferable to check, for example, that a turn-off timer is set or adjustment of brightness is instructed (not turned off). Note that, when the off timer is set or adjustment of the brightness is instructed, the control should be shifted to this control. In addition, for the audio instruction such as "turn off", for example, when the air conditioner adjusts the temperature of the zone, for example, when the television apparatus is operated, it is preferable to obtain the approval of which apparatus is the target from the user (user) who has instructed (issued) the audio instruction. In this case, it is preferable to determine 1 device and to inquire about the performance of the user using a performance such as "do the television device turned off" to which the next control action is agreed. On the other hand, in the case of the voice instruction of the target device such as "turn off the air conditioner", since it is sometimes complicated to confirm (acknowledge) that the voice of "turn off the air conditioner" is outputted, the HGW600 can output the control signal to the target device without sound (without voice output). In this case, for example, it is preferable to display a reply of the cause of reception confirmation such as "turn off the air conditioner" (change to the mobile terminal apparatus 500) to the GUI-1 of the mobile terminal apparatus 500 held by the user. It is preferable that the output of the silent (no sound output) control signal is selectable by the user through, for example, a setting screen displayed on the GUI-1.

When the voice instruction is "exit from home", it is determined that the occupant (user) has exited outside the section (building) by detecting, for example, a door sensor that the door has been closed after being temporarily opened, and the HGW600 outputs a control signal to turn off the lamp after, for example, 3 seconds. The user may be detected by using, for example, the position information of the portable terminal device (smartphone) 500 held by the user when the user goes out of the building. In this case, the HGW600 outputs a control signal such that the lamp (switch) is turned off when the position information is detected to be outside the building. Of course, in order to detect that the user has gone outside the building, the output of the door sensor (detection of opening/closing) and the position information of the smartphone may be used in combination. Further, the HGW600 may output a control signal so that a lamp (switch) is turned off by detecting the movement of the user by combining various outputs from other sensors, for example, the entrance of a person into a garage and the exit of a vehicle following the entrance from the garage.

Further, it is preferable that the HGW600 detects the opening/closing of the door sensor, checks whether or not a person such as a family member including the user remains in the building using an output of another sensor, and invalidates the turning off of the lamp (switch) when the person (including the user) remains in the building. Whether or not a person remains in the building may be checked using the position information of the smartphone held by each person including the user.

In addition, in the case where a pet is left in the building, the HGW600 controls each device according to information preset by the user. In this case, the HGW600 outputs various control signals to a plurality of devices which can cooperate with a user's instruction, such as a state in which the air conditioner is kept operating even though the lamp (switch) is off, or a state in which the motion sensor is turned on when the motion sensor is off. However, in the case where only a pet remains in a person not remaining in the building, it is preferable to feedback (transmit) a message for confirming an operation such as whether or not the air conditioner is kept in an operating state, or whether or not the air conditioner is automatically operated based on a detected temperature when the air conditioner is turned off, to a smartphone (portable terminal device) held by the user, and thereby obtain approval (approval).

Note that, the learning function may refer to a control pattern executed and stored in the past, and automatically output a control signal as, for example, [ an outgoing pattern (door closed (lock check)) ]. In this case, a plurality of control modes that are distinguished based on the control mode, such as "outgoing mode (night)", "outgoing mode (daytime)", and the like, may be prepared. In addition, as for the control mode such as "out mode (daytime)", a plurality of modes can be prepared such as "out mode (daytime-summer)", and "out mode (daytime-winter)".

The control mode referred to by the learning function may be held in the server 1000 or may be held inside the HGW600, for example. The learning function may be prepared in advance in the control data management unit 601, for example, stored in the storage medium 1010, expanded into a memory (work memory) in the control data management unit 601 when there is a read request from the control data management unit 601, and executed by the CPU in the control data management unit 601.

In the case where the sound indication is "good night"/"good dream", it is checked whether, for example, a turn-off timer is set or whether adjustment of brightness is instructed (performed without turning off) or the like. For example, in the case where a turn-off timer is set, the lamp (switch) is turned off at the time when the end of the count of the turn-off timer (time managed by the timer) is detected. For example, when adjustment of the brightness is instructed, the level set for the lamp (switch) is Read (Read), and the brightness is set to the level. For example, the brightness may be adjusted by changing the power supplied to the lamp by setting the value (duty) of the pwm (pulse Width modulation) of the inverter.

As shown in fig. 18, the process of recognizing the voice instruction and determining the intention thereof may be shared by the voice processing unit 400 provided independently of the HGW600, or may be processed by the control data management unit 601 in the HGW 600.

Fig. 17 and 17A show an example of sensor control information (data table). The sensor control information data table shown in fig. 17 and 17A is located in an external server, for example, and includes, for example, column C121 to column C129.

Column C121 holds management numbers of the devices unique to the home network. The management number indicates a correspondence relationship with an arbitrary device held in the sensor characteristic information data table shown in fig. 15 and 15A to 15D. The management number is associated with the sensor characteristic information held in fig. 15 and 15A to 15D, and the same number as the management number shown in fig. 16, 16A, and 16B is used.

Column C122 holds information on the specification of wireless communication corresponding to an arbitrary device. Column C123 holds the product name unique to the apparatus. The column C124 holds an identification name (product specification) which is information that can uniquely identify the function of the device. The information held in the column C123 and the column C124 is the same information as the column C102 and the column C103 in fig. 15, respectively, and thus, it is possible to ensure correspondence with the device (sensor) characteristic information shown in fig. 15 and fig. 15A to 15D.

The column C125 holds the functions of the devices and information related to the control method. Column C126 holds information relating to a control method for realizing the functions of the apparatus. The information held in the column C125 and the column C126 is the same information as the column C106 and the column C107 in fig. 15, 15A, and 15B, respectively, and thus, the correspondence with the sensor characteristic information (data table) shown in fig. 15 and 15A to 15D can be secured.

Column C127 holds information related to various rereads (changing expressions) when the name and/or nickname (of the device) of the registered device is/are indicated by voice. That is, when a registered device is designated by a name given as various names (names) based on the information related to rereading (changing) registered in the column C127 and held in the column C127, the corresponding device can be specified.

Column C128 holds the segment names of the respective segments held by column C115 in fig. 16.

Column C129 holds an identification name (product specification) which is information that can uniquely ensure the function of the device.

Control of various devices in the home network via the HGW600 will be described below based on the information described in fig. 14, 15A to 15D, 16A, 16B, 17, and 17A.

By combining the information held in fig. 14, 15A to 15D, 16A, 16B, 17, and 17A, it is possible to relay the HGW600 in a home network including the HGW600 and control a detection mechanism (sensor) or a control target device, which is an arbitrary device, by an audio instruction (each phrase constituting the audio instruction).

Here, by specifying names given as various names (names) of arbitrary devices held in the intention interpretation data table from phrases included in the voice instruction, it is possible to specify the corresponding devices (control targets). With regard to names, in a manner that can be uniquely identified even when, for example, "spot light" is referred to as "lamp" or "spot light", and when, for example, "garage" (corresponding to japanese automobile) is referred to as "garage" (corresponding to japanese ガレージ) or "outside (entrance)", various modifications (changes of expressions) can be made with regard to the control object (device) and the phrase, respectively. Note that the name of the device (sensor) already used is associated with a uniquely identifiable name or information for changing the name so as to be able to correspond to various kinds of rereading (changing the expression), and is held in a data table. The name or information for reading the name is preferably prepared in plural. The data table holds control information necessary for controlling the device by voice so that a phrase indicating an intention to be controlled can be associated with information for controlling the device in accordance with the phrase.

That is, by identifying a device using a name or information that can uniquely identify the device, a phrase indicating an intention to be controlled can be extracted from the above-described data table. In this way, the information (name) of the recognized device is linked to a phrase (voice command) indicating an intention to be controlled, and control information (control) for controlling the device to be controlled by the HGW600 is generated using the "< IF > - < THEN >" sentence structure format. The generated control information can be registered as, for example, a dictionary in association with various pieces of rereading (changing) information. The dictionary can of course be updated at any time.

HGW600 also determines whether it can be defined as < IF > or < THEN > based on the characteristics of the input information (information of the identified device). In the case of < IF >, it is possible to acquire information indicating what kind of condition (information for controlling the controlled object) and what kind of information (which is necessary) should be acquired from the device for determining the condition from the data table. For example, when the indoor temperature of the section 23 (see fig. 1) where the pet stays reaches a certain temperature, the < THEN > may be set in advance so as to allow the air conditioning operation for cooling the section. In this case, it is also possible to monitor which device (sensor) has reached a certain temperature (set as < IF >). Further, the temperature rise of the sector 23, which is < IF >, may be detected based on the outputs of 2 or more sensors (temperature sensor, solar radiation sensor, or the like). Of course, when the temperature of the block 23 is lower than a certain temperature, the block 23 may be heated by air conditioning (the air conditioning may be operated in the heating mode). In this case, as the control command for the air conditioner, a control command different from the control command for "cooling (cooling on)" and the control command for "heating (heating on)" may be used as the control command for the air conditioner, as in the setting of the automatic mode (on the air conditioner side). The temperature of the zone 23 can be set in the range of 20 to 29 ℃ by simultaneously setting the upper limit temperature and the lower limit temperature with respect to the center temperature, for example, "temperature setting, zone 23, 25 ℃ +4 ℃/-5 °".

When the user sets a rule relating to the operation of the device (sensor), it is preferable to automatically generate a selection list to facilitate registration. When the device (sensor) has a plurality of functions, it is preferable to set the combination of < IF > and < THEN > or perform an explanation of the intention for control, in accordance with the name of each function. That is, in the case where there are 2 or more < THEN > with respect to 1 < IF >, < IF > is defined for each < THEN >. In addition, when the number of functions of the device (sensor) is 1, and when there are 2 or more < THEN > with respect to 1 < IF >, the < IF > is defined for each < THEN >.

When the HGW600 initially registers the device (sensor), when the location where the device is set is paired (registered) with the host machine (HGW600), the position information output from the device is sequentially stored, and the device can be learned while being updated. The position information includes the longitude, latitude, and altitude detected by an input device, for example, a smartphone (which may be a portable terminal device such as a tablet PC) held by the user at the time of registration. In the registration (pairing) of the next device (sensor), it is preferable that the information of the nearest Zone (Zone) is displayed in advance on the display unit of the input device based on the longitude, latitude, and altitude detected by the input device at that time. In the registration of the device, it is preferable that data (feature information) indicating the feature of the device, information (control information) for controlling the device, and a nickname (user setting) of the device be associated with each other and displayed on the display unit 501 of the portable terminal device 500 as initial values.

In addition, by combining the above-described transfer of information to and from each device (sensor) by the HGW600, that is, by combining various information supplied from the sensor (device) to the HGW600, various rules (cooperative operations) can be executed in the space shown in fig. 1.

For example, a rule of "door closed (lock check)" may be considered. The rule is initiated (executed) before going out or before going to bed. In the case where the voice instruction issued by the user is "come out back", the HGW600 determines that the "close door (go out)" rule should be started, and executes the rule. If the sound instruction from the user is "good night"/"good dream", the HGW600 determines that the "close door (bedtime)" rule should be started, and executes the rule. The "door closing (out)" rule is also referred to as a rule G. Similarly, the rule "close door (sleep)" is also referred to as a rule S.

In the case where the rule G (the "door closing (out)" rule) is activated, the current position of the user is determined based on the output of an arbitrary sensor (movement detection sensor). In the case of determining the current position of the user, position information provided from a portable terminal device (smartphone) held by the user, for example, may be utilized. In the case where the current position of the user is determined by the movement detection sensor, position information from the smartphone may also be used in combination.

Under rule G, the output of a sensor (opening/closing sensor) provided in the window is referred to in parallel with or at an arbitrary timing with the determination of the current position of the user. As shown in fig. 1, when the target space includes a door that can be used for entrance and exit between open spaces 32 connected to the outside, the output of the opening/closing sensor provided in the door is also referred to.

When an output from an arbitrary sensor, for example, a sensor that detects opening and closing of a window is "Open", a message such as "○○ window is not closed and confirmation is output by the HGW600 located at or near a position where a sound output can be delivered (heard) to a user, and when the HGW600 located at or near a position where a sound output can be delivered (heard) to a user does not have a sound output mechanism, it is preferable that a display unit 501 of a portable terminal device (smartphone) 500 held by the user displays a warning such that" the window of section HI is not closed and confirmation is requested "in the display unit.

In rule G, in the case where the door, which is available for the access to the window or the open space, can be confirmed, for example, the lamp (lighting) of the section (room) in which the user does not reside is turned off. When it is detected that a family or a pet other than the user remains in any room (section), the user can wait for an instruction from the user by feedback regarding the room to be turned off as described above. This can avoid the situation where the lighting (lamp) is turned off uniformly, for example, when someone who is not outside stays at home. On the other hand, when only the pet is left, it is possible to perform control such that the lighting is turned off to ensure rest (sleep) of the pet. In this case, the air conditioner (cooling) may be turned on or the heater (heating) may be turned on based on the weather of the day and the conditions of the section. Further, in the case where the locking mechanism of the door or the window can be locked by remote control from the HGW600 or the smartphone 500, the locking mechanism may be locked at a predetermined timing after the state in which the respective doors or windows are closed can be checked.

In the rule G, when it can be checked that no one is left in the building due to the user going out, the position of the user may be detected based on the position information from the smartphone 500 held by the user, and the lights of the zones after the user has passed may be sequentially turned off.

Hereinafter, when it is detected that the user opens the door of the entrance and closes the door after a predetermined period of time has elapsed, all the lighting in the space (building) is turned off. In the case where the door closing is detected, or in the case where the door locking is not completed for a certain period of time, a warning urging the door locking may be presented to the user. Further, in the case where the locking mechanism of the door can be locked by remote control from the HGW600 or the smartphone 500, the locking may be performed at a predetermined timing after the user is detected to be left outside the space (building). When an image of the land 1, which is a place outside the space (building), can be acquired (the monitoring camera is prepared on the land 1), the "out mode" or the like may be activated when the user can be specified to go out from the land 1 to the outside by image processing and image recognition of the image of the camera. Further, the user may go out without holding the smartphone. In this case, the door can be locked at a predetermined timing after the movement detection sensor detects that the vehicle (or the bicycle or the motorcycle) has exited from the garage (the space 11 in fig. 1), for example. For example, when the user can be specified to go out of the right of way 1 by image processing and image recognition of the image of the camera that acquires the image of the right of way 1, the door may be locked.

However, even when the user goes out without holding the smartphone, the user can be detected to go out of the venue 1 by image processing and image recognition of the image of the camera that acquires the image of the venue 1 as described above. However, when the user goes home and enters the land 1 (without the smartphone), it is conceivable that the user cannot determine whether the user is a visitor or a user. Therefore, when it is detected that someone enters the site 1 and the output of a sensor (opening/closing sensor) provided at the door of the entrance (section 12) changes later, the HGW600 can "who came? "such warning is notified to other HGWs within the building. In this case, when the object detected in site 1 is a suspicious person, it is preferable to stop the sound output in order to avoid an intentional stop of the operation of the HGW600 (suspicious person). When the camera (sensor) 611 is activated to acquire an image of the inside of the land 1, it is preferable to perform imaging in an infrared imaging mode or the like without turning on a flash even at night, for example. Note that, when it is detected that someone enters the site 1 and the output of a sensor (opening/closing sensor) provided at the door of the entrance (section 12) does not change later, it can be recognized that a suspicious person has entered the site. In this case, it is preferable that the invasion (into the plot) of a pet such as a cat in the neighborhood, for example, be clearly distinguishable from the invasion of the suspicious individual. Therefore, in order to identify the pet by image processing and image recognition of the camera image of the venue 1, it is preferable to connect to a database in which the total number of image data and the type of image data that can be referred to from the database are large.

It is needless to say that various control modes referred to by the learning function described above can be applied to the rule G.

In the case where the rule S ("door closing (sleeping)" rule) is activated, the current position of the user is determined based on the output of an arbitrary sensor (movement detection sensor). In the case of determining the current position of the user, position information provided from a portable terminal device (smartphone) held by the user, for example, may be utilized. In the case where the current position of the user is determined by the movement detection sensor, position information from the smartphone may also be used in combination.

In the rule S, outputs of a sensor (opening/closing sensor) provided in a window and a sensor (opening/closing sensor) provided in a door of an entrance (section 12 in fig. 1) are referred to in parallel with or at an arbitrary timing when the current position of the user is determined. As shown in fig. 1, when the target space includes a door that can be used for entrance and exit between open spaces 32 connected to the outside, the output of the opening/closing sensor provided in the door is also referred to.

When an output from an arbitrary sensor, for example, a sensor that detects opening and closing of a window is "Open", a message such as "○○ window is not closed and confirmation is output by the HGW600 located at or near a position where a sound output can be delivered (heard) to a user, and when the HGW600 located at or near a position where a sound output can be delivered (heard) to a user does not have a sound output mechanism, it is preferable that a display unit 501 of a portable terminal device (smartphone) 500 held by the user displays a warning such as" the window of section HI is not closed and confirmation is requested ".

However, it is also conceivable that a window used for ventilation purposes, such as a window of a section 33 that is a high window in the living room (section) 23 of the second floor of the building of fig. 1 (see fig. 1), need not be completely closed. In such a case, the HGW600 can wait for an instruction from the user by feedback. When it is expected that heavy rain (rain, cats and dogs (are run)) or strong wind arrives, for example, by weather forecast, a message such as "close the bar with the high window today" may be presented to the user (smartphone).

In rule S, in the case where it is confirmed that the window or door is closed, for example, the lights (lighting) of the section (room) in which the case where the user does not reside is detected are turned off. In the case where the locking mechanism of the door or window can be locked by remote control from the HGW600 or the smartphone 500, the locking mechanism may be locked at a predetermined timing after the detection that each is in the closed state is possible. If a situation in which other family members than the user live in the building can be confirmed based on, for example, the location information of the respective smartphone, the smartphone held by the user (the user's own smartphone) can instruct (transmit) HGW600 to transmit a message to another family member, such as "sleep first".

It is to be noted that, of course, various control modes referred to by the above-described learning function may be applied to the rule S.

Note that, regarding the association of the voice instruction and the rule and the registration (recording) of the actual operation pattern of the device described with reference to fig. 15, 15A to 15D, 16A, 16B, 17, and 17A, each time a new device (sensor) is added by executing the process of "registration of a device" described with reference to fig. 13D, 13E, and 13G, the update is performed automatically. That is, when a device is added, the characteristics of the added device are read from the characteristic information shown in fig. 15 and 15A to 15D. From the read-out characteristics, the operation (rule) of the added device is determined based on the intention explanations (rules) shown in fig. 16, 16A, and 16B. That is, the operation of the device corresponding to various voice instructions is specified. Therefore, the position of the device as shown in fig. 17 and 17A is associated with the actual operation of the device. In other words, in the embodiment, when a sensor (device) is added, a rule (rule) is automatically created based on the characteristics (properties) of the added sensor, and the rule can be added to the data table.

Fig. 19A to 19C show an example of a registration procedure in the "registration of device" mode when the network specification of wireless communication used by the device to be registered (communication specification suitable for the device) conforms to ieee802.15.4, for example, and uses [ type B < zigbee > ] of radio waves in the 2.4GHz band.

Fig. 19A is an example of display when registration of the device is started in the "registration of device" mode, and a state in which the current position of the user is planarly displayed by a symbol 1541 is displayed in a "layout (plane)" 1551 registered in the "layout creation" mode described with reference to fig. 4 to 8. It is preferable that a message 1535 such as "please move to the vicinity of the device to be registered" be displayed in the display area 501, and the user be prompted to specify the device to be registered. In the case of the device type B, if there are a plurality of devices that can be registered, the devices that can be registered are displayed in the "layout diagram" 1551. That is, when the "registration of device" mode is started, as shown in fig. 19A, the schematic positions of devices 1561, 1562, …, 1565 which are located in the vicinity and are to be registered are displayed in a "layout diagram" 1551. Note that the devices 1564 and 1565 to be registered shown in fig. 19A are devices in the sections (1 and 11) other than the section 11, which cannot be seen (cannot be seen) by the user who wants to register the devices. Although not shown in fig. 19A, the range displayed in the display area 501 is scrolled to display the approximate position of the device to be registered, which is located in the second layer 20 shown in fig. 2(b) or the third layer 30 shown in fig. 2(c), for example.

When the user moves to the vicinity of the device to be registered (described by taking the device 1561 as an example), and the user identifies the device 1561 and connects the device 1561 as the device to be registered (the user establishes communication with the terminal device [ smartphone ]500 held by the user), as shown in fig. 19B, a "layout diagram" 1551 is updated to a state in which the position of the device 1561 is displayed together with the position 1543 of the user. In this case, as shown in fig. 19C, it is preferable that a standard function-specific logo name, product specification, and/or individual name (model) prepared by a supplier, such as an Image (Image) and/or a switch (On/off switch), a sensor (Motionsensor), a sensor (Open/close sensor), and a socket (Electrical outlet/receptor outlet) indicating the general shape of the device 1561, be read from the profile shown in fig. 15, and displayed in the display area 501 in the form of, for example, a list 1570. In fig. 19C, the display-related rules such as the display position, display order, superimposition, transmission/non-transmission when superimposed, and the like of the list 1570, the "layout" 1551, the message 573, and/or the button 525 are arbitrary. It is preferable that the display size is controlled in accordance with the size of the display area 501, and the display amount of each screen display is controlled, and all screen displays are displayed by scrolling.

Next, as shown in fig. 19C, for example, "register the device? A message 1571 of "etc., and a button 525 for receiving an instruction input such as [ register ] or [ yes ] or [ redo ] or [ abort ], etc.

In the case where the instruction input (click) from the user to the screen display shown in fig. 19C is [ registration ] or [ yes ], the "registration of device" process of the registration based on the device of type a, which has been described with reference to fig. 13D, 13E, and 13G, is sequentially executed. Further, in the case where the name (name) of the device input by the user during the "registration of the device" already exists, it is preferable to display, for example, "the device of the same name is already registered" as in the case described with reference to fig. 13F. A message 576 to prompt the user to avoid registration of duplicate names (names) such as enter another name bar.

Even if the device of [ type B ] cannot directly communicate with the HGW600, the device of [ type B ] can use another device of [ type B ] as a relay device to eject (relay transfer) a signal from the HGW600 to the device and a signal from the device to the HGW 600. This makes it possible to suppress the communication capabilities of the devices of [ type B ], and to construct a network with low power consumption.

Fig. 20A to 20C show an example of registration procedures in the "registration of device" mode when the network specification of wireless communication (communication specification suitable for the device) used by the device to be registered conforms to ieee802.15.1, for example, and uses radio waves of 2.4GHz band [ type C < Bluethooth > ].

Fig. 20A is an example of display when registration of the device is started in the "registration of device" mode, and a state in which the current position of the user is planarly displayed by a symbol 2541 is displayed in a "layout (plane)" 2551 registered in the "layout creation" mode described with reference to fig. 4 to 8. In the case of the device type C, if there are a plurality of devices that can be registered, the identifiers of the devices that can be registered, for example, the mac (media Access control) addresses and product codes, are displayed in a list in the "layout" 2551.

That is, when the "registration of device" mode is started, as shown in fig. 20A, the identifiers of devices to be registered that are located in the vicinity are displayed in the "layout diagram" 2551 in a state where the positions are not specified. The identifiers of the devices to be registered shown in fig. 20A are also displayed in a list with respect to the devices to be registered located in the second tier 20 shown in fig. 2(b) or the third tier 30 shown in fig. 2(c), for example. Therefore, it is necessary to display a message 2536 such as "please select a device to be registered" in the display area 501, and prompt the user to specify the device to be registered.

In the registration of the device of type C, first, the user identifies a device to be registered (taking the device 2561 as an example for explanation), and the device 2561 is connected as a registration target device (communication with the terminal device [ smartphone ]500 held by the user is established).

When the user establishes communication with the terminal apparatus with respect to the apparatus to be registered, as shown in fig. 20B, the "layout diagram" 2551 is updated to a state in which the position of the apparatus 2561 is displayed together with the position 2541 of the user.

In this case, as shown in fig. 20C, it is preferable that a standard function-specific logo name, product specification, and/or individual name (model) prepared by a supplier, such as an Image (Image) and/or a switch (On/off switch), a sensor (Motion sensor), a sensor (Open/close sensor), and a socket (Electrical outlet/receive outlet) indicating a schematic shape of the device 2561, be read from the feature table shown in fig. 15, and be displayed in the display area 501, for example, in the form of a list 2570. In fig. 13C, the display-related rules such as the display positions, display orders, overlapping, and transmission/non-transmission in the overlapping of the list 2570, the "layout diagram" 2551, the message 573, and the button 525 are arbitrary. It is preferable that the display size is controlled in accordance with the size of the display area 501, and the display amount of each screen display is controlled, and all screen displays are displayed by scrolling. Next, as shown in fig. 20C, for example, "register the device? A message 2571 of "etc., a button 525 for receiving an instruction input such as [ register ] or [ yes ] or [ redo ] or [ abort ], and the like.

In the case where the instruction input (click) from the user to the screen display shown in fig. 20C is [ registration ] or [ yes ], the "registration of device" process of the registration based on the device of type a, which has been described with reference to fig. 13D, 13E, and 13G, is sequentially executed. Further, in the case where the name (name) of the device input by the user during the "registration of the device" already exists, it is preferable to display, for example, "the device of the same name is already registered, as in the case described with reference to fig. 13F. Enter another name bar "or the like to urge the user to avoid instances of duplicate names (names) 576.

Although the embodiments of the present invention have been described, these embodiments are presented as examples and do not limit the scope of the invention. These new embodiments can be implemented in various other ways, and various omissions, substitutions, and changes can be made without departing from the spirit of the invention. The above-described embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalent scope thereof.

Description of the reference numerals

1 … floor (limited space), 11-14, 21-24, 31-34 … sections (rooms), 100 … object space (building), 101, 102, …, 105 … devices (sensors), 300 … network, 400 … sound processing unit, 500 … smart phone (input device), 501 … display unit, 600 … Home Gateway (HGW).

Claims (11)

1. A home network is provided with:

an information processing unit that determines a device to be operated based on input information, and outputs a control signal for controlling the operation of the device; and

and a data holding unit that holds a condition for controlling the operation of the device by the information processing unit and information for specifying the device based on the input information.

2. The home network of claim 1, wherein,

the information processing unit is capable of autonomous movement.

3. The home network according to claim 1 or 2,

the information processing unit obtains, by voice, a condition for controlling an operation of the device and information for specifying the device based on the input information.

4. The home network according to any one of claims 1 to 3,

the information processing unit determines "if …" or "…" and "performs to" according to conditions for controlling the operation of the device.

5. The home network according to any one of claims 1 to 4,

the information processing unit adds, to the data holding unit, a condition for controlling an operation of the device and information for specifying the device based on the input information when the device is not registered.

6. An electronic apparatus includes:

a first processing unit that determines a device to be operated based on input information; and

and a second processing unit that outputs a control signal for controlling the operation of the device.

7. A processing apparatus, comprising:

a home gateway connected to a home network;

a system controller provided in the home gateway and configured to output command signals to a plurality of control target devices; and

a data holding unit that holds source data for constructing the command signals to the plurality of control target devices,

the source data includes at least segment identification data for dividing a room in a home, position data indicating a position in the segment, and device identification data of a control target device placed at the position.

8. The processing apparatus according to claim 7,

the home gateway is also provided with a microphone,

the system controller determines the source data based on the voice phrase input from the microphone, making the instruction signal determined.

9. The processing apparatus according to claim 8,

the source data includes a control frame corresponding to a voice phrase input from the microphone, a control phrase thereof, the section identification data, location data, and the device identification data registered.

10. A display method displays acquired position information based on movement trajectory data, and displays the position information and device information in the same display interface so that a device operating according to a predetermined communication method can be registered in the display interface.

11. The display method according to claim 10,

the movement trajectory data is obtained by displaying the movement of the device.

Images