WO2022176946A1

WO2022176946A1 - Control system, and control method

Info

Publication number: WO2022176946A1
Application number: PCT/JP2022/006382
Authority: WO
Inventors: 彩衣吉川; 恵里眞田; 洋子藤原
Original assignee: パナソニックＩｐマネジメント株式会社
Priority date: 2021-02-22
Filing date: 2022-02-17
Publication date: 2022-08-25
Also published as: JP2022128255A

Abstract

A control system (10) comprising a setting unit (25) that sets content of scene control with devices (30) installed in a facility (90) as targets, and a control unit (26) that executes only a portion of, or all of, the content of the scene control set by the setting unit (25) in accordance with the variety of an information terminal that receives an execution instruction for the scene control or the current position of the information terminal.

Description

Control system and control method

The present invention relates to control systems and control methods.

Various technologies have been proposed for controlling devices. Patent Literature 1 discloses a home appliance control system in which a sound collector connectable to a network collects voice information and controls the home appliance via the network based on the collected voice information.

WO2014/171144

The present invention provides a control system and a control method that can adaptively change the content of device control.

A control system according to an aspect of the present invention includes a setting unit that sets details of scene control for equipment installed in a facility, and a type of information terminal that receives an instruction to execute the scene control or a current state of the information terminal. and a control unit that executes only part or all of the content of the scene control set by the setting unit according to the position.

A control method according to an aspect of the present invention includes a setting step of setting details of scene control for equipment installed in a facility; and a control step of executing only part or all of the content of the scene control set in the setting step according to the position.

A program according to one aspect of the present invention is a program for causing a computer to execute the control method.

A control system and a control method according to one aspect of the present invention can adaptively change the details of device control.

FIG. 1 is a block diagram showing the functional configuration of the control system according to the embodiment. FIG. 2 is a diagram showing an example of setting information for executing scene control. FIG. 3 is a flowchart of an operation for setting the content of scene control according to Operation Example 1. As shown in FIG. FIG. 4 is a diagram showing an example of a display screen of a device to be controlled. FIG. 5 is a flowchart of local scene control according to Operation Example 1. FIG. FIG. 6 is a sequence diagram of audio scene control according to Operation Example 1. FIG. FIG. 7 is a diagram showing an example of how the content of scene control is narrowed down when the information terminal that has received the execution instruction is an audio UI device. FIG. 8 is a sequence diagram of remote scene control according to Operation Example 1. FIG. FIG. 9 is a diagram showing an example of how the content of scene control is narrowed down when the information terminal that has received the execution instruction is a mobile terminal. FIG. 10 is a diagram showing an example of a notification screen. FIG. 11 is a sequence diagram of an operation for setting the content of scene control according to Operation Example 2. As shown in FIG. FIG. 12 is a diagram showing an example of setting information for audio scene control. FIG. 13 is a diagram showing an example of setting information for remote scene control. 14 is a sequence diagram of audio scene control according to Operation Example 2. FIG. 15 is a sequence diagram of remote scene control according to Operation Example 2. FIG. FIG. 16 is a first sequence diagram of scene control according to Operation Example 3. FIG. 17 is a second sequence diagram of scene control according to Operation Example 3. FIG. 18 is a first sequence diagram of scene control according to Operation Example 4. FIG. 19 is a second sequence diagram of scene control according to Operation Example 4. FIG. FIG. 20 is a diagram showing an example of a display screen for obtaining consent to restrict functions in audio scene control.

Hereinafter, embodiments will be specifically described with reference to the drawings. It should be noted that the embodiments described below are all comprehensive or specific examples. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are examples and are not intended to limit the present invention. Further, among the constituent elements in the following embodiments, constituent elements not described in independent claims will be described as optional constituent elements.

It should be noted that each figure is a schematic diagram and is not necessarily strictly illustrated. Moreover, in each figure, the same code|symbol is attached|subjected with respect to substantially the same structure, and the overlapping description may be abbreviate|omitted or simplified.

(Embodiment)
[Constitution]
First, the configuration of the control system according to the embodiment will be described. FIG. 1 is a block diagram showing the functional configuration of the control system according to the embodiment.

The control system 10 shown in FIG. 1 is a device control system capable of controlling a plurality of devices 30 installed in a facility 90. The facility 90 is, for example, a residence such as an apartment complex or a detached house, but may be a facility other than a residence such as an office.

As shown in FIG. 1, the control system 10 includes a control terminal 20, a plurality of devices 30, a wireless communication device 35, a voice UI device 40, a voice recognition server 50, a voice control server 60, and a mobile phone. A terminal 70 and a remote control server 80 are provided. The audio UI device 40 , the control terminal 20 , and the plurality of devices 30 are installed inside the facility 90 . It should be noted that the mobile terminal 70 may be positioned inside the facility 90 or may be positioned outside the facility 90 .

First, the control terminal 20 will be explained. The control terminal 20 is, for example, a HEMS (Home Energy Management System) controller having an energy management function. The control terminal 20 is installed in the facility 90 and manages the amount of electricity used (in other words, power consumption) of the plurality of devices 30 installed in the facility 90 . The control terminal 20 also controls a plurality of devices 30 installed within the facility 90 (or within the site of the facility 90). The control terminal 20 is not limited to the HEMS controller, and may be another home controller that does not have an energy management function, or a gateway device. The control terminal 20 is an example of an information terminal.

Specifically, the control terminal 20 includes a display unit 21, a communication unit 22, an information processing unit 23, and a storage unit 24.

The display unit 21 is a display device having an image display function and a function of accepting manual input from the user. The display unit 21 is implemented by a touch panel and a display panel such as a liquid crystal panel or an organic EL (Electro Luminescence) panel. The touch panel is, for example, a capacitive touch panel, but may be a resistive touch panel.

The communication unit 22 is a communication circuit for the control terminal 20 to communicate with other devices via the wide area communication network 100 . The communication unit 22 is, for example, a wireless communication circuit that performs wireless communication, but may be a wired communication circuit that performs wired communication. A communication standard for communication performed by the communication unit 22 is not particularly limited.

The information processing section 23 performs information processing related to control of the device 30 . The information processing section 23 is realized by, for example, a microcomputer, but may be realized by a processor or a dedicated circuit. The information processing section 23 has a setting section 25, a control section 26, and a display control section 27 as functional components. Functions of the setting unit 25 , the control unit 26 , and the display control unit 27 are realized by, for example, executing a computer program stored in the storage unit 24 by a microcomputer or the like that constitutes the information processing unit 23 . Details of the functions of the setting unit 25, the control unit 26, and the display control unit 27 will be described later.

The storage unit 24 is a storage device that stores computer programs executed by the information processing unit 23, setting information for scene control (described later), and the like. The storage unit 24 is implemented by, for example, a semiconductor memory.

Next, the multiple devices 30 will be described. The plurality of devices 30 are devices to be controlled in the control system 10 installed in the facility 90 . The multiple devices 30 are controlled by control signals transmitted by the control unit 26 of the control terminal 20 . The plurality of devices 30 include an air conditioner 31, a lighting device 32, an electric lock 33, an electric blind 34, and the like.

The air conditioner 31 is a general household air conditioner. The air conditioner 31 is an air conditioner capable of adjusting the temperature of the air sent from the air conditioner 31 by having a heat exchanger (not shown) or the like. That is, the air conditioner 31 has a temperature adjustment function (blowing function and cooling/heating function). The air conditioner 31 is not limited to a general household air conditioner, and may be an industrial air conditioner.

The lighting device 32 illuminates the interior (room) of the facility 90 . The lighting device 32 is, for example, a ceiling light, but the specific aspect of the lighting device 32 is not particularly limited. The lighting device 32 may be a downlight, pendant light, spotlight, bracket light, or the like. Also, the lighting device 32 may be a device that illuminates the outside (outdoor) of the facility 90 .

The electric lock 33 is a security device that controls the unlocking and locking of the doors (or windows, etc.) of the facility 90 . The electric lock 33 has an RFID reader that acquires key information from, for example, a card key. Also, the electric lock 33 may include a biosensor that acquires biometric information such as a fingerprint as key information. In the following embodiments, the electric lock 33 is described as an electric lock provided on the front door.

The electric blinds 34 are placed inside the windows of the facility 90 and adjust the amount of outside light that enters the facility 90 . Also, the electric blind 34 may be used as a partition inside the facility 90 . The electric blind 34 has a plurality of slats, and the angles of the slats (hereinafter also simply referred to as the angle of the electric blind 34, etc.) can be adjusted.

Next, the wireless communication device 35 will be explained. The wireless communication device 35 is a device that performs short-range wireless communication, and is a wireless LAN (Local Area Network) router installed in the facility 90 . The wireless communication device 35 is connected to the control terminal 20 and the audio UI device 40 for communication. Communication between the control terminal 20 and the voice UI device 40 through the wide area communication network 100 is performed via the wireless communication device 35 .

Next, the voice UI device 40 will be explained. The voice UI device 40 is an information terminal into which user's uttered voice is input, and transmits a voice signal of the input uttered voice to the voice recognition server 50 via the wide area communication network 100 . The audio UI device 40 is, for example, a smart speaker installed at the facility 90 . The voice UI device 40 specifically has a microphone 41 and a speaker 42 .

The microphone 41 acquires the voice uttered by the user (speech voice). The microphone 41 is specifically a condenser microphone, a dynamic microphone, a MEMS (Micro Electro Mechanical Systems) microphone, or the like.

The speaker 42 outputs sound (mechanical sound) as a response to the speech sound acquired by the microphone 41 . This allows the user to interactively control the device 30 .

Next, the speech recognition server 50 will be explained. The speech recognition server 50 is a computer located outside the facility 90 . Specifically, the voice recognition server 50 is a cloud server that receives voice signals transmitted by the voice UI device 40 and performs voice recognition processing on the received voice signals. A business that provides a speech recognition service uses the speech recognition server 50 to provide the speech recognition service. The voice recognition server 50 , for example, converts a voice signal transmitted by the voice UI device 40 into text information and transmits the text information to the voice control server 60 .

Next, the voice control server 60 will be explained. The voice control server 60 is a computer located outside the facility 90 . The voice control server 60 is specifically a cloud server that generates a control command based on text information transmitted from the voice recognition server 50 and transmits the generated control command to the control terminal 20 . The voice control server 60 includes a first communication section 61 , a first information processing section 62 and a first storage section 63 .

The first communication unit 61 is a communication module (communication circuit) for the voice control server 60 to communicate with other devices via the wide area communication network 100 . The communication performed by the first communication unit 61 is, for example, wired communication, but may be wireless communication. The communication standard used for communication is also not particularly limited.

The first information processing unit 62 performs information processing for controlling the device 30 based on voice input received by the voice UI device 40 . The first information processing section 62 is implemented by, for example, a microcomputer, but may be implemented by a processor. The first information processing section 62 has a first setting section 64 and a first control section 65 as functional components. The functions of the first setting unit 64 and the first control unit 65 are realized by, for example, executing a computer program stored in the first storage unit 63 by a microcomputer or the like that constitutes the first information processing unit 62. be. Details of the functions of the first setting unit 64 and the first control unit 65 will be described later.

The first storage unit 63 is a storage device that stores computer programs and the like executed by the first information processing unit 62 . The first storage unit 63 is implemented by, for example, an HDD (Hard Disc Drive).

Next, the portable terminal 70 will be explained. The mobile terminal 70 is a mobile information terminal for manual input (that is, operation) for the user to remotely control the device 30, and is specifically a smart phone, a tablet terminal, or the like. The mobile terminal 70 has a display section 71 , a position acquisition section 72 and a wireless communication section 73 .

The display unit 71 is a display device having an image display function and a function of accepting manual input from the user. The display unit 71 is realized by a touch panel and a display panel such as a liquid crystal panel or an organic EL panel. The touch panel is, for example, a capacitive touch panel, but may be a resistive touch panel.

The location acquisition unit 72 acquires (in other words, calculates) the current location of the mobile terminal 70 and outputs current location information indicating the acquired current location. The position acquisition unit 72 is implemented by, for example, a positioning module compatible with a satellite positioning system such as GPS (Global Positioning System) or Quasi-Zenith Satellite System.

The wireless communication unit 73 is a communication circuit for communication connection between the mobile terminal 70 and the wireless communication device 35 . The wireless communication unit 73 is specifically a short-range wireless communication circuit compatible with wireless LAN.

Next, the remote control server 80 will be explained. Remote control server 80 is a computer located outside facility 90 . Specifically, the remote control server 80 is a cloud server that generates a control command based on information transmitted from the mobile terminal 70 and transmits the generated control command to the control terminal 20 . The remote control server 80 includes a second communication section 81 , a second information processing section 82 and a second storage section 83 .

The second communication unit 81 is a communication module (communication circuit) for the remote control server 80 to communicate with other devices via the wide area communication network 100 . The communication performed by the second communication unit 81 is, for example, wired communication, but may be wireless communication. The communication standard used for communication is also not particularly limited.

The second information processing unit 82 performs information processing for controlling the device 30 based on the user's manual input received by the mobile terminal 70 . The second information processing section 82 is implemented by, for example, a microcomputer, but may be implemented by a processor. The second information processing section 82 has a second setting section 84 and a second control section 85 as functional components. The functions of the second setting unit 84 and the second control unit 85 are realized by, for example, executing a computer program stored in the second storage unit 83 by a microcomputer constituting the second information processing unit 82. be. Details of the functions of the second setting unit 84 and the second control unit 85 will be described later.

The second storage unit 83 is a storage device in which computer programs and the like executed by the second information processing unit 82 are stored. The second storage unit 83 is implemented by, for example, an HDD.

[Outline of device control]
A user of the control system 10 can control the equipment 30 by making manual inputs to the control terminal 20 . When the display unit 21 receives the manual input, the control unit 26 of the control terminal 20 transmits a control signal corresponding to the manual input to the device 30 via the local communication network. Thereby, the device 30 is controlled. The local communication network is, for example, a communication network conforming to ECHONET Lite (registered trademark). In the following, such control of the device 30 based on manual input to the control terminal 20 is also described as local control.

Also, the user can control the device 30 by inputting voice to the voice UI device 40 . When voice input is received by the microphone 41 , the voice UI device 40 transmits a voice signal corresponding to the voice input to the voice recognition server 50 via the wide area communication network 100 . The voice recognition server 50 converts the voice signal into text information and transmits the text information to the voice control server 60 via the wide area communication network 100 . The voice control server 60 generates a control command according to the text information and transmits the generated control command to the control terminal 20 via the wide area communication network 100 . When the communication unit 22 receives the control command, the control unit 26 of the control terminal 20 transmits a control signal corresponding to the control command to the device 30 via the local communication network. Thereby, the device 30 is controlled. Hereinafter, such control of the device 30 based on voice input to the voice UI device 40 is also described as voice control.

Also, the user can control the device 30 by performing manual input to the mobile terminal 70 . When the manual input is accepted by the display unit 71 , the mobile terminal 70 transmits control request information according to the manual input to the remote control server 80 via the wide area communication network 100 . The remote control server 80 generates a control command according to the control request information, and transmits the generated control command to the control terminal 20 via the wide area communication network 100 . When the communication unit 22 receives the control command, the control unit 26 of the control terminal 20 transmits a control signal corresponding to the control command to the device 30 via the local communication network. Thereby, the device 30 is controlled. Hereinafter, such control of the device 30 based on manual input to the mobile terminal 70 is also referred to as remote control.

[Scene control]
A user of the control system 10 can also perform scene controls (sometimes called routine actions, etc.). Scene control is control that collectively operates one or a plurality of devices 30 arranged in facility 90 in order to bring the interior of facility 90 closer to a predetermined indoor environment. FIG. 2 is a diagram showing an example of setting information for executing scene control.

In the scene control setting information, the control details of the device 30 are associated with each life scene (scene name) such as going home, getting up, going out, and going to bed. For example, when the user performs a manual input to the display unit 21 of the control terminal 20 to instruct the execution of the scene control of "coming home", the control unit 26 of the control terminal 20 changes the setting information stored in the storage unit 24 to The device 30 is controlled by referring to it. As a result, the air conditioner 31 and the lighting device 32 are turned on with pre-registered settings (brightness, set temperature, etc.), the electric lock 33 is locked, and the slats of the electric blind 34 are opened to the set angle. The user can set what kind of device 30 performs what kind of operation when the execution of the scene control of "coming home" is instructed. The information transmission path at this time is the same as that of the local control, and the scene control of the device 30 based on manual input to the control terminal 20 is also described as local scene control.

In addition, the user can also execute the "coming home" scene control by speaking to the voice UI device 40, "make it a coming home scene". The information transmission path at this time is the same as the voice control described above, and scene control based on voice input to the voice UI device 40 is also described as voice scene control.

　There are restrictions on the content of the audio scene control. For example, the angle of the electric blind 34 can be designated in local scene control, but the angle of the electric blind 34 cannot be designated in audio scene control, and only opening and closing is possible. In other words, in audio scene control, the setting information of local scene control cannot be used as it is.

Therefore, in general, the setting information for executing the audio scene control is stored in advance in the first storage unit 63 of the audio control server 60 separately from the setting information (FIG. 2) for executing the local scene control. remembered. When setting (storing in the first storage unit 63 ) setting information for executing audio scene control, the user performs manual input to the portable terminal 70 .

In addition, the user can also execute scene control of "coming home" by performing manual input to the mobile terminal 70. The information transmission path at this time is the same as that of the remote control, and scene control based on manual input to the mobile terminal 70 is also described as remote scene control.

Since the remote scene control assumes control of the device 30 from outside the facility 90, there are restrictions on the content of the control. For example, in remote scene control, the electric lock 33 is excluded from scene control from the viewpoint of security. Further, in remote scene control, detailed operations of the device 30 may be excluded. That is, in remote scene control, the setting information for local scene control and the setting information for audio scene control cannot be used as they are.

Therefore, in general, the setting information for executing remote scene control is stored in the remote control server separately from the setting information for executing local scene control and the setting information for executing audio scene control. It is stored in advance in the second storage unit 83 of 80 . When setting (storing in the second storage unit 83 ) setting information for remote scene control, the user performs manual input to the portable terminal 70 .

Thus, in general, the user needs to individually set the control terminal 20, the voice control server 60, and the remote control server 80 for the same "coming home" scene control. On the other hand, in the control system 10, the user's burden can be reduced by sharing setting information for scene control.

[Operation example 1]
An operation example 1 in which setting information for scene control is shared will be described below. First, an operation for setting the content of scene control according to operation example 1 will be described. FIG. 3 is a flowchart of an operation for setting the content of scene control according to Operation Example 1. As shown in FIG.

The display control unit 27 of the control terminal 20 displays a scene control setting screen on the display unit 21 (S11). Note that the setting screen may include a display screen showing the device 30 to be controlled by each of the local scene control, the audio scene control, and the remote scene control. FIG. 4 is a diagram showing an example of a display screen of a device to be controlled. In the example of FIG. 4, the device 30 to be controlled is displayed as an icon. Such a display screen is an example of an image showing the device 30 to be controlled and the device 30 excluded from the control target when only part of the content of the scene control is executed.

While the setting screen is displayed, the user performs manual input intended to set the scene control on the display unit 21 of the control terminal 20 . The display unit 21 accepts such manual input (S12). The setting unit 25 sets the content of scene control according to the manual input accepted in step S12 (S13). Specifically, the setting unit 25 generates setting information as shown in FIG. 2 according to the manual input accepted in step S12, and stores the generated setting information in the storage unit 24.

Next, the local scene control according to Operation Example 1 will be described. FIG. 5 is a flowchart of local scene control according to Operation Example 1. FIG.

After the setting information is stored in the storage unit 24 of the control terminal 20 as described above, the user performs manual input intended to execute scene control on the display unit 21 . The display unit 21 accepts such a manual input (an example of a scene control execution instruction) (S14). By referring to the setting information stored in the storage unit 24, the control unit 26 specifies the details of the scene control indicated by the manual input accepted in step S14 (S15), and stores all of the specified details of the scene control. Execute (S16). Specifically, the control unit 26 executes all scene control by transmitting a control signal to the device 30 via the local communication network.

Next, audio scene control according to Operation Example 1 will be described. FIG. 6 is a sequence diagram of audio scene control according to Operation Example 1. FIG.

After the setting information is stored in the storage unit 24 of the control terminal 20 as described above, when the user utters a voice intended to execute scene control (for example, a scene of returning home), the microphone of the voice UI device 40 41 acquires this speech sound. In other words, the microphone 41 receives an audio input for scene control (an example of an instruction to execute scene control) (S21). The voice UI device 40 transmits the acquired voice signal of the uttered voice to the voice recognition server 50 (S22).

Upon receiving the voice signal, the voice recognition server 50 performs voice recognition processing on the received voice signal (that is, spoken voice) (S23). Specifically, the voice recognition server 50 converts the received voice signal into text information and transmits the text information to the voice control server 60 (S24).

The first communication unit 61 of the voice control server 60 receives text information from the voice recognition server 50. Note that the voice recognition server 50 may convert the text information into command information (information similar to a control command described later), and the voice control server 60 may acquire the command information from the voice recognition server 50 .

The first control unit 65 identifies the scene intended by the user based on the received text information (S25), and the information terminal that received the scene information indicating the identified scene and the execution instruction is the audio UI device 40. A control command including terminal information indicating that there is a terminal is transmitted to the control terminal 20 (S26). The first communication unit 61 is used for transmitting the control command. The terminal information is information indicating the type of information terminal that has received the execution instruction.

The communication unit 22 of the control terminal 20 receives control commands (scene information and terminal information) from the audio control server 60 . By referring to the setting information stored in the storage unit 24, the control unit 26 identifies the content of the scene control indicated by the scene information received in step S26 (S27). Further, the control unit 26 narrows down the content of the specified scene control according to the terminal information received in step S26 (S28). FIG. 7 is a diagram showing an example of how the content of scene control is narrowed down when the information terminal that has received the execution instruction is the audio UI device 40 . In the example of FIG. 7, the details of the control of the air conditioner 31, the lighting device 32, and the electric lock 33 are maintained, but the details of the control of the electric blind 34 are narrowed down to basic operations.

Then, the control unit 26 executes the content of the narrowed down scene control (that is, part of the content of the scene control) (S29). Specifically, the control unit 26 executes a part of scene control by transmitting a control signal to the device 30 via the local communication network.

Next, remote scene control according to Operation Example 1 will be described. FIG. 8 is a sequence diagram of remote scene control according to Operation Example 1. FIG.

After the setting information is stored in the storage unit 24 of the control terminal 20 as described above, the user performs manual input intended to set the scene control on the display unit 71 of the mobile terminal 70 . The display unit 71 accepts such a manual input (an example of a scene control execution instruction) (S31). The portable terminal 70 transmits to the remote control server 80 control request information requesting execution of scene control indicated by the manual input accepted in step S31 (S32).

The second communication unit 81 of the remote control server 80 receives the control request information. The second control unit 85 specifies the scene intended by the user based on the received control request information (S33), and the information terminal that received the scene information indicating the specified scene and the execution instruction is the portable terminal 70. A control command including terminal information indicating that there is a terminal is transmitted to the control terminal 20 (S34). The second communication unit 81 is used for transmitting the control command.

The communication unit 22 of the control terminal 20 receives control commands (scene information and terminal information) from the remote control server 80 . By referring to the setting information stored in the storage unit 24, the control unit 26 specifies the content of the scene control indicated by the scene information received in step S34 (S35). Further, the control unit 26 narrows down the contents of the specified scene control according to the terminal information received in step S34 (S36). FIG. 9 is a diagram showing an example of how the content of scene control is narrowed down when the information terminal that has received the execution instruction is the mobile terminal 70 . In the example of FIG. 9, the details of the control of the air conditioner 31, the lighting device 32, and the electric blinds 34 are all narrowed down to basic operations. Also, the control of the electric lock 33 is excluded.

Then, the control unit 26 notifies the content of the narrowed down scene control (that is, that only part of the content of the scene control is executed) (S37). Specifically, the control unit 26 transmits notification information to the remote control server 80 , and the remote control server 80 transmits notification information to the mobile terminal 70 . As a result, a notification screen as shown in FIG. 10 is displayed on the display unit 71 of the mobile terminal 70 (S38). FIG. 10 is a diagram showing an example of a notification screen.

When the user performs an approval operation to approve execution of scene control while the notification screen of FIG. 10 is displayed, the display unit 71 receives manual input intended to approve execution of scene control (S39). The mobile terminal 70 transmits the approval information to the remote control server 80, and the remote control server 80 transmits the approval information to the control terminal 20 (S40).

The communication unit 22 of the control terminal 20 receives the approval information. Triggered by the reception of the approval information by the communication unit 22, the control unit 26 executes the content of the narrowed-down scene control (that is, part of the content of the scene control) (S41). Specifically, the control unit 26 executes a part of scene control by transmitting a control signal to the device 30 via the local communication network. The control unit 26 does not execute scene control when the approval information is not received within a certain period of time after the notification information is transmitted.

Note that the processing (notification and approval) of steps S36 to S39 may be omitted. Further, the control unit 26 may notify that only part of the scene control has been executed after executing part of the scene control.

As described above, in operation example 1, the setting information is stored only in the control terminal 20, and the control unit 26 of the control terminal 20 selects the scene indicated by the setting information according to the type of information terminal that has received the execution instruction. Determine whether to execute all or part of the contents of the control. The user of the control system 10 only needs to set the contents of scene control in the control terminal 20 , and has the advantage of not having to set the scene control in the voice control server 60 and the remote control server 80 .

In operation example 1, the manual input intended to set the content of the scene control was received by the control terminal 20, but may be received by the mobile terminal 70. In this case, the setting information is transmitted to the control terminal 20 via the voice control server 60 or remote control server 80 and stored in the storage unit 24 .

[Operation example 2]
An operation example 2 in which setting information for scene control is shared will be described below. First, the operation of setting the content of scene control according to Operation Example 2 will be described. FIG. 11 is a sequence diagram of an operation for setting the content of scene control according to Operation Example 2. As shown in FIG.

The display control unit 27 of the control terminal 20 displays a scene control setting screen on the display unit 21 (S51). While the setting screen is displayed, the user performs manual input (an example of the first manual input) intended to set the scene control on the display unit 21 of the control terminal 20 . The display unit 21 accepts such manual input (S52). The setting unit 25 sets the content of scene control according to the manual input accepted in step S52 (S53). Specifically, the setting unit 25 generates setting information as shown in FIG. 2 according to the manual input accepted in step S12, and stores the generated setting information in the storage unit 24. The processing of steps S51 to S53 is the same as the processing of steps S11 to S13.

After step S53, the setting unit 25 transmits the setting information generated in step S53 to each of the voice control server 60 and the remote control server 80 (S54). As described above, the voice control server 60 is used when the voice UI device 40 accepts voice input, and the remote control server 80 operates when the mobile terminal 70 accepts manual input. It is a server device used for

The first communication unit 61 of the voice control server 60 receives the setting information. The first setting unit 64 narrows down the received setting information (S55). Specifically, the first setting unit 64 extracts, from the received setting information, only the part to be executed by the audio scene control. The first setting unit 64 stores the extracted setting information in the first storage unit 63 as setting information for audio scene control (S56). FIG. 12 is a diagram showing an example of setting information for audio scene control, the contents of which are narrowed down from the original setting information (FIG. 2).

Also, the second communication unit 81 of the remote control server 80 receives the setting information. The second setting unit 84 narrows down the received setting information (S57). Specifically, the second setting unit 84 extracts only the portion to be executed by remote scene control from the received setting information. The second setting unit 84 stores the extracted setting information in the second storage unit 83 as setting information for remote scene control (S58). FIG. 13 is a diagram showing an example of setting information for remote scene control, the contents of which are narrowed down from the original setting information (FIG. 2).

As described above, in Operation Example 2, setting information with different contents is stored in each of the control terminal 20, the voice control server 60, and the remote control server 80. The distribution of the setting information (the process of steps S54 to S56) is automatically performed when the setting information is stored in the storage unit 24 of the control terminal 20. FIG.

　The distribution of the setting information may be performed by the user's manual input. The manual input in this case is the second manual input with the intention of consenting to transmitting the setting information to the voice control server 60 and the remote control server 80 (sharing the setting information). The second manual input is a simple one-touch operation on a predetermined icon.

Next, local scene control, audio scene control, and remote scene control according to Operation Example 2 will be described. Since the local scene control according to Operation Example 2 is the same as the local scene control according to Operation Example 1, description thereof is omitted.

Next, audio scene control according to Operation Example 2 will be described. 14 is a sequence diagram of audio scene control according to Operation Example 2. FIG.

After the setting information is stored in the first storage unit 63 of the voice control server 60 as described above, the microphone 41 receives voice input intended to execute scene control (S61). The voice UI device 40 transmits the acquired voice signal of the uttered voice to the voice recognition server 50 (S62). The voice recognition server 50 performs voice recognition processing on the received voice signal (S63), and transmits text information to the voice control server 60 (S64). The processing of steps S61 to S64 is the same as that of steps S21 to S24.

The first communication unit 61 of the voice control server 60 receives text information from the voice recognition server 50. The first control unit 65 identifies the content of scene control determined by the received text information by referring to the setting information stored in the first storage unit 63 (S65). Then, the first control unit 65 transmits to the control terminal 20 a control command indicating the content of the specified scene control (specific control content of the target device 30) (S66). The first communication unit 61 is used for transmitting the control command.

The communication unit 22 of the control terminal 20 receives control instructions from the voice control server 60 . The control unit 26 executes the content of the scene control indicated by the received control command (that is, part of the original content of the scene control) (S67). Specifically, the control unit 26 executes scene control by transmitting a control signal to the device 30 via the local communication network.

Next, remote scene control according to Operation Example 2 will be described. 15 is a sequence diagram of remote scene control according to Operation Example 2. FIG.

After the setting information is stored in the second storage unit 83 of the remote control server 80 as described above, the display unit 71 of the mobile terminal 70 accepts manual input intended to execute scene control (S71). The portable terminal 70 transmits to the remote control server 80 control request information requesting execution of scene control indicated by the manual input accepted in step S71 (S72). The processing of steps S71 to S72 is the same as that of steps S31 to S32.

The second communication unit 81 of the remote control server 80 receives the control request information. The second control unit 85 identifies the content of scene control determined by the received control request information by referring to the setting information stored in the second storage unit 83 (S73).

Also, the second control unit 85 notifies the content of the specified scene control (that is, that only part of the content of the original scene control is executed) (S74). Specifically, the second control unit 85 transmits notification information to the mobile terminal 70 . As a result, a notification screen as shown in FIG. 10 is displayed on the display unit 71 of the mobile terminal 70 (S75).

When the user performs an approval operation to approve execution of scene control while the notification screen of FIG. 10 is displayed, the display unit 71 receives manual input intended to approve execution of scene control (S76). The mobile terminal 70 transmits the approval information to the remote control server 80 (S77), and the second communication section 81 of the remote control server 80 receives the approval information.

When the approval information is received, the second control unit 85 transmits to the control terminal 20 a control command indicating the content of the scene control specified in step S73 (specific control content of the target device 30). (S78). The second communication unit 81 is used for transmitting the control command. The second control unit 85 does not transmit the control command when the approval information is not received within a certain period of time after transmitting the notification information.

The communication unit 22 of the control terminal 20 receives control instructions from the remote control server 80 . The control unit 26 executes the content of the scene control indicated by the received control command (that is, part of the original content of the scene control) (S79). Specifically, the control unit 26 executes scene control by transmitting a control signal to the device 30 via the local communication network.

Note that the processing (notification and approval) of steps S74 to S77 may be omitted. Also, the second control unit 85 may notify that only part of the content of the scene control has been executed after transmitting the control command.

As described above, in the operation example 2, the setting information stored in the control terminal 20 is distributed to the voice control server 60 and the remote control server 80, and the control terminal 20, the voice control server 60 and the remote control server Setting information is shared among the servers 80 . If the user of the control system 10 sets the content of scene control only on the control terminal 20, the setting information is shared automatically or by simple manual input. In other words, there is an advantage that the user does not need to set the scene control in the voice control server 60 and the remote control server 80 .

In operation example 2 above, scene control settings were made in the control terminal 20, and the setting information was distributed from the control terminal 20 to the voice control server 60 and the remote control server 80. Scene control may be set in the voice control server 60 , and the setting information may be distributed from the voice control server 60 to the control terminal 20 and the remote control server 80 . Further, scene control settings may be made to the remote control server 80 via the mobile terminal 70 , and the setting information may be distributed from the remote control server 80 to the control terminal 20 and the voice control server 60 .

[Operation Example 3]
In the operation examples 1 and 2, examples have been described in which only part or all of the content of the scene control is executed according to the type of the information terminal that receives the execution instruction of the scene control. Depending on the current position of the information terminal (more specifically, mobile terminal 70) that receives the control execution instruction, only a part or all of the contents of the scene control may be executed. 16 and 17 are sequence diagrams of scene control according to Operation Example 3. FIG. 16 is a sequence diagram when the mobile terminal 70 is located inside the facility 90, and FIG. 17 is a sequence diagram when the mobile terminal 70 is located outside the facility 90. FIG.

First, the operation of the control system 10 when the mobile terminal 70 is located inside the facility 90 will be described with reference to FIG.

After the setting information is stored in the storage unit 24 of the control terminal 20 as in operation example 1, the display unit 71 of the mobile terminal 70 receives manual input intended to execute scene control (S81). The mobile terminal 70 transmits to the remote control server 80 control request information requesting execution of scene control indicated by the manual input accepted in step S81 (S82). Here, the control request information includes location information indicating the current location of the mobile terminal 70 . The location information is output by the location acquisition unit 72 included in the mobile terminal 70 .

The second communication unit 81 of the remote control server 80 receives the control request information. The second control unit 85 identifies the scene intended by the user based on the received control request information (S83). Also, the second control unit 85 determines that the mobile terminal 70 is located inside the facility 90 based on the location information included in the control request information (S84a). Note that the determination criteria (coordinates of the facility 90) at this time are stored in the second storage unit 83 of the remote control server 80 in advance.

Next, the second control unit 85 transmits to the control terminal 20 a control command including scene information indicating the specified scene and determination result information indicating that the mobile terminal 70 is located inside the facility 90 (S85a). . The second communication unit 81 is used for transmitting the control command.

The communication unit 22 of the control terminal 20 receives control commands (scene information and determination result information) from the remote control server 80 . Since the received determination result information indicates that the mobile terminal 70 is located within the facility 90, the control unit 26 executes all the contents of the scene control indicated by the scene information received in step S85a ( S86a).

Next, the operation of the control system 10 when the mobile terminal 70 is located outside the facility 90 will be described with reference to FIG. The processing of steps S81 to S83 is the same as in FIG.

After step S83, the second control unit 85 determines that the mobile terminal 70 is located outside the facility 90 based on the location information included in the control request information (S84b).

Next, the second control unit 85 transmits to the control terminal 20 a control command including scene information indicating the identified scene and determination result information indicating that the portable terminal 70 is located outside the facility 90 (S85b). . The second communication unit 81 is used for transmitting the control command.

The communication unit 22 of the control terminal 20 receives control commands (scene information and determination result information) from the remote control server 80 . Since the received determination result information indicates that the portable terminal 70 is located outside the facility 90, the control unit 26 narrows down the content of the scene control (for example, narrows down as shown in FIG. 9), and in step S85b Part of the content of the scene control indicated by the received scene information is executed (S86b).

In this way, the control system 10 can execute only part or all of the contents of the scene control according to the current position of the mobile terminal 70 that receives the instruction to execute the scene control. In operation examples 1 and 2, the execution instruction to the mobile terminal 70 is regarded as a control instruction for the device 30 from outside the facility 90, and only part of the content of the scene control is executed. In contrast, according to Operation Example 3, scene control is adaptively executed according to the current position of the mobile terminal 70 .

In Operation Example 3, the remote control server 80 determines whether or not the mobile terminal 70 is located within the facility 90 , but the control terminal 20 may determine whether or not the mobile terminal 70 is located within the facility 90 . Also, between steps S84b and S85b in FIG. 17, processing (notification and approval) similar to steps S74 to S76 may be performed. Further, in the operation example 3 as well, the control unit 26 may notify that only part of the content of the scene control has been executed after executing a part of the content of the scene control.

[Operation example 4]
In the operation example 3, the location information output by the location acquisition unit 72 is used to determine whether or not the mobile terminal 70 is located within the facility 90, but the above determination may be made by other methods. can. 18 and 19 are sequence diagrams of scene control according to the operation example 4. FIG. 18 is a sequence diagram when the mobile terminal 70 is located inside the facility 90, and FIG. 19 is a sequence diagram when the mobile terminal 70 is located outside the facility 90. FIG.

After the setting information is stored in the storage unit 24 of the control terminal 20 as in operation example 1, the display unit 71 of the mobile terminal 70 receives manual input intended to execute scene control (S91). The portable terminal 70 transmits to the remote control server 80 control request information requesting execution of scene control indicated by the manual input accepted in step S91 (S92). Here, the control request information does not include location information indicating the current location of the mobile terminal 70 .

The second communication unit 81 of the remote control server 80 receives the control request information. The second control unit 85 identifies the scene intended by the user based on the received control request information (S93).

Next, the second control unit 85 transmits a control command including scene information indicating the specified scene to the control terminal 20 (S94). The second communication unit 81 is used for transmitting the control command.

The communication unit 22 of the control terminal 20 receives control instructions (scene information) from the remote control server 80 . The control unit 26 determines whether or not the mobile terminal 70 is connected to the wireless communication device 35 for communication. When the mobile terminal 70 is connected to the wireless communication device 35 for communication, it is considered that the mobile terminal 70 is located near the facility 90 (inside the facility 90). Therefore, when determining that the mobile terminal 70 is connected to the wireless communication device 35 (S95a), the control section 26 executes all scene control contents indicated by the scene information received in step S94 (S96a). .

Next, the operation of the control system 10 when the mobile terminal 70 is located outside the facility 90 will be described with reference to FIG. The processing of steps S91 to S94 is the same as in FIG.

After step S94, the control unit 26 determines whether or not the mobile terminal 70 is connected to the wireless communication device 35 for communication. When the mobile terminal 70 is not connected to the wireless communication device 35 for communication, it is considered that the mobile terminal 70 is located far from the facility 90 (outside the facility 90). Therefore, when determining that the mobile terminal 70 is not connected to the wireless communication device 35 (S95b), the control unit 26 narrows down the details of the scene control (for example, narrowing down as shown in FIG. 9), A part of the contents of the scene control indicated by the scene information obtained is executed (S96b).

In this manner, the control system 10 determines whether or not the mobile terminal 70 that receives the execution instruction for scene control is connected to the wireless communication device 35 installed in the facility 90 . It can be considered whether or not Thereby, the control system 10 can adaptively execute scene control according to the current position of the mobile terminal 70 .

Note that, in Operation Example 4, the same processing (notification and approval) as in steps S37 to S40 may be performed between steps S95b and S96b in FIG. Also in the operation example 4, the control unit 26 may notify that only a part of the content of the scene control has been executed after executing a part of the content of the scene control.

[Modification]
When the user starts using the voice control server 60 , the user needs to perform user registration by manually inputting to the portable terminal 70 and consent to the terms of use of the voice control server 60 . When agreeing to the terms of use, the user may be asked to agree that the audio scene control has limited functionality. FIG. 20 is a diagram showing an example of a display screen for obtaining consent to restrict functions in audio scene control.

Also, when starting to use the remote control server 80, it may be necessary to perform user registration by manually inputting to the mobile terminal 70 and consent to the terms of use of the remote control server 80. Again, when agreeing to the terms of use, the user may agree that remote scene control is limited in functionality.

[Effects, etc.]
As described above, the control system 10 includes the setting unit 25 for setting the content of scene control for the equipment 30 installed in the facility 90, and the type of information terminal or information terminal that receives the instruction to execute the scene control. A control unit 26 that executes only part or all of the content of scene control set by the setting unit 25 according to the current position.

Such a control system 10 can adaptively change the content of scene control according to the type of information terminal that accepts a scene control execution instruction or the current position of the information terminal. That is, the control system 10 can adaptively change the content of control of the device 30 .

Also, in operation examples 1 and 2, the control unit 26 executes only part or all of the content of the scene control set by the setting unit 25 according to the type of information terminal. When the information terminal is the first information terminal, the entire contents of the scene control are executed, and when the information terminal is the second information terminal, only part of the contents of the scene control are executed.

Such a control system 10 can adaptively change the content of scene control according to the type of information terminal that receives the instruction to execute scene control.

Also, for example, the first information terminal is an information terminal (eg, control terminal 20) installed in facility 90, and the second information terminal is a portable information terminal (eg, mobile terminal 70).

Such a control system 10 adaptively changes the content of scene control according to whether the information terminal that receives the instruction to execute scene control is an information terminal installed in the facility 90 or a mobile terminal. be able to.

Further, for example, the first information terminal is an information terminal (for example, the mobile terminal 70) that accepts a user's manual input as an execution instruction, and the second information terminal is an information terminal that accepts a user's voice input as an execution instruction. (For example, the audio UI device 40).

Such a control system 10 adaptively changes the content of scene control according to whether the information terminal that receives the instruction to execute scene control is an information terminal that receives manual input or an information terminal that receives voice input. can do.

Further, in operation examples 3 and 4, the control unit 26 executes only part or all of the content of the scene control set by the setting unit 25 according to the current position of the information terminal (for example, the mobile terminal 70). , when the information terminal is located inside the facility 90, the entire contents of the scene control are executed, and when the information terminal is located outside the facility 90, only a part of the contents of the scene control are executed.

Also, for example, when executing only part of the content of scene control, the control unit 26 (or the second control unit 85) notifies that only part of the content of scene control will be executed before execution.

Such a control system 10 can notify that only part of the content of scene control is executed.

Also, for example, the control system 10 further includes a display control unit 27 that displays an image showing the device 30 to be controlled when only part of the content of scene control is executed.

Such a control system 10 can display an image showing the equipment 30 to be controlled.

Further, for example, the setting unit 25 generates setting information indicating details of scene control based on the user's first manual input to the first information terminal (for example, the control terminal 20), and sends the generated setting information to the first It is transmitted to the server device (eg, voice control server 60 or remote control server 80) used when the second information terminal (eg, voice UI device 40 or portable terminal 70) receives the execution instruction.

Such a control system 10 can share setting information between the first information terminal and the server device.

Further, for example, the setting unit 25 generates setting information indicating details of scene control based on the user's first manual input to the first information terminal, and converts the generated setting information to the user's second manual input. and send it to the server device.

Such a control system 10 can share setting information between the first information terminal and the server device based on manual input by the user.

Also, for example, the setting unit 25 and the control unit 26 are provided by the control terminal 20 installed in the facility 90 .

Such a control system 10 can be realized as a control terminal 20 installed in the facility 90.

Further, the control method executed by a computer such as the control system 10 includes a setting step of setting the content of scene control for the equipment 30 installed in the facility 90, and the type or and a control step of executing only part or all of the content of the scene control set in the setting step according to the current position of the information terminal.

Such a control method can adaptively change the content of scene control according to the type of information terminal that receives the execution instruction of scene control or the current position of the information terminal. That is, the control method can adaptively change the content of control of the device 30 .

(Other embodiments)
Although the embodiments have been described above, the present invention is not limited to the above embodiments.

For example, in the above embodiments, the control system was realized by a plurality of devices, but it may be realized by a single device. For example, the control system may be implemented as a single device corresponding to the voice control server or the remote control server in the control terminal of the above embodiments. When the control system is implemented by multiple devices, the components included in the control system may be distributed among the multiple devices in any way.

Also, for example, the communication method between devices in the above embodiment is not particularly limited. Further, a relay device (not shown) may intervene in communication between devices. Further, the information transmission paths described in the above embodiments are not limited to the transmission paths shown in the sequence diagrams.

Further, in the above embodiment, the processing executed by a specific processing unit may be executed by another processing unit. For example, the processing executed by the setting unit of the control terminal may be executed by the first setting unit of the voice control server or the second setting unit of the remote control server. Further, the processing executed by the control unit of the control terminal may be executed by the first control unit of the voice control server or the second control unit of the remote control server. In addition, the order of multiple processes may be changed, and multiple processes may be executed in parallel.

Also, in the above embodiments, each component may be realized by executing a software program suitable for each component. Each component may be realized by reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory by a program execution unit such as a CPU or processor.

Also, each component may be realized by hardware. For example, each component may be a circuit (or integrated circuit). These circuits may form one circuit as a whole, or may be separate circuits. These circuits may be general-purpose circuits or dedicated circuits.

Also, general or specific aspects of the present invention may be implemented in a system, apparatus, method, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM. Also, any combination of systems, devices, methods, integrated circuits, computer programs and recording media may be implemented.

For example, the present invention may be realized as a control terminal or server device (voice control server or remote control server) according to the above embodiments, or as a control system corresponding to the control terminal or server device. good too. Further, the present invention may be implemented as a control method executed by a computer such as a control system, or may be implemented as a program for causing a computer to execute such a control method. The present invention may be implemented as a computer-readable non-temporary recording medium in which such a program is recorded.

In addition, forms obtained by applying various modifications to each embodiment that a person skilled in the art can think of, or realized by arbitrarily combining the constituent elements and functions of each embodiment without departing from the spirit of the present invention. Also included in the present invention.

REFERENCE SIGNS LIST 10 control system 20 control terminal 25 setting unit 26 control unit 27 display control unit 30 device 60 voice control server (server device)
80 Remote control server (server device)
90 facilities

Claims

a setting unit for setting the content of scene control for equipment installed in the facility;
a control unit that executes only part or all of the content of the scene control set by the setting unit according to the type of the information terminal that receives the instruction to execute the scene control or the current position of the information terminal; Equipped with a control system.
The control unit
executing only a part or all of the scene control contents set by the setting unit according to the type of the information terminal;
if the information terminal is the first information terminal, execute all of the contents of the scene control;
2. The control system according to claim 1, wherein when the information terminal is a second information terminal, only part of the content of the scene control is executed.
The first information terminal is an information terminal installed in the facility,
The control system according to claim 2, wherein the second information terminal is a portable information terminal.
The first information terminal is an information terminal that receives a user's manual input as the execution instruction,
The control system according to claim 2, wherein the second information terminal is an information terminal that receives a user's voice input as the execution instruction.
The control unit
executing only part or all of the scene control content set by the setting unit according to the current position of the information terminal;
when the information terminal is located within the facility, executing all of the contents of the scene control;
2. The control system according to claim 1, wherein only part of the content of the scene control is executed when the information terminal is located outside the facility.
6. The control unit according to any one of claims 1 to 5, wherein when only a part of the content of the scene control is to be executed, the control unit notifies that only a part of the content of the scene control is to be executed before execution. Control system as described.
7. The control system according to any one of claims 1 to 6, further comprising a display control unit that displays an image showing a device to be controlled when only part of the content of the scene control is executed.
The setting unit generates setting information indicating details of the scene control based on a user's first manual input to the first information terminal, and transmits the generated setting information to the second information terminal according to the execution instruction. 5. The control system according to any one of claims 2 to 4, wherein the control system transmits to a server device used upon acceptance.
The setting unit generates setting information indicating details of the scene control based on a user's first manual input to the first information terminal, and sets the generated setting information based on the user's second manual input. The control system according to claim 8, which transmits to the server device.
The control system according to any one of claims 1 to 9, wherein the setting unit and the control unit are provided by a control terminal installed in the facility.
a setting step of setting the content of scene control for equipment installed in the facility;
a control step of executing only part or all of the content of the scene control set in the setting step according to the type of the information terminal that receives the execution instruction of the scene control or the current position of the information terminal; Including control method.
A program for causing a computer to execute the control method according to claim 11.