CN111416757A - Remote operation terminal, program, remote control device, and remote operation system - Google Patents

Abstract

Provided are a remote control terminal, a program, a remote control device, and a remote control system, which can improve the operability of the remote control terminal that enables the operation of a plurality of target devices with 1 remote control terminal. The remote control terminal of the embodiment has a plurality of operation units and a display control unit. The plurality of operation units are virtual operation units associated with the devices registered as the remote operation targets, and receive operation inputs to the devices associated with the operation units, respectively. The display control unit changes the display mode of the operation unit on the display unit according to the position of the device.

Description

Remote operation terminal, program, remote control device, and remote operation system

Technical Field

Embodiments of the present invention relate to a remote control terminal, a program, a remote control device, and a remote control system.

Background

Conventionally, in order to improve convenience for users who operate home appliances and the like, remote operation terminals (so-called remote control devices) have been developed which can operate a plurality of appliances with 1 remote operation terminal. However, in the conventional remote operation terminal, there are cases where it is not possible to operate a device to be operated (hereinafter referred to as "target device") with good operability, such as when an unexpected device is operated or when special interaction is required between the remote operation terminal and the device to be operated.

Patent document 1: japanese laid-open patent publication No. 10-336769

Patent document 2: japanese laid-open patent application No. 2001-142825

Disclosure of Invention

The present invention addresses the problem of providing a remote control terminal, a program, a remote control device, and a remote control system that can improve the operability of a remote control terminal that enables the operation of a plurality of target devices with 1 remote control terminal.

The remote control terminal of the embodiment has a plurality of operation units and a display control unit. The plurality of operation units are virtual operation units associated with the devices registered as the remote operation targets, and receive operation inputs to the devices associated with the operation units, respectively. The display control unit changes the display mode of the operation unit on the display unit according to the position of the device.

Drawings

Fig. 1 is a diagram showing a specific example of a remote operation system using a remote operation terminal according to a first embodiment.

Fig. 2 is a block diagram showing a specific example of the functional configuration of the remote operation terminal in the first embodiment.

Fig. 3 is a diagram showing a specific example of a device selection screen displayed on the remote operation terminal according to the first embodiment.

Fig. 4 is a diagram showing a specific example of the device selection screen displayed on the remote operation terminal according to the first embodiment.

Fig. 5 is a diagram showing a specific example of the device selection screen displayed on the remote operation terminal according to the first embodiment.

Fig. 6 is a diagram showing a specific example of an operation screen displayed on the remote operation terminal according to the first embodiment.

Fig. 7 is a flowchart showing a flow of processing in which the remote operation terminal changes the display mode of the virtual remote controller of each home appliance 1 according to the distance from the registered device in the first embodiment.

Fig. 8 is a block diagram showing a specific example of the functional configuration of the remote operation terminal in the second embodiment.

Fig. 9 is a flowchart showing a flow of processing in which the remote operation terminal changes the display mode of the virtual remote controller for each device group in accordance with the distance from the registered device in the second embodiment.

Fig. 10 is a diagram showing a specific example of an operation screen of a virtual remote controller of a device group displayed on the remote operation terminal according to the second embodiment.

Fig. 11 is a block diagram showing a specific example of the functional configuration of the remote operation terminal in the third embodiment.

Fig. 12 is a block diagram showing a specific example of the functional configuration of the remote control device in the third embodiment.

Fig. 13 is a flowchart showing the flow of processing in which the remote control apparatus transmits a notification in accordance with the position of the remote operation terminal in the third embodiment.

Fig. 14 is a block diagram showing a specific example of the functional configuration of the remote control device in the fourth embodiment.

Fig. 15 is a flowchart showing the flow of processing in which the remote control apparatus transmits a notification in accordance with the position of the remote operation terminal in the fourth embodiment.

Fig. 16 is a block diagram showing a specific example of a functional configuration of the remote operation terminal in the fifth embodiment.

Fig. 17 is a diagram showing a first specific example of a display mode of the device selection screen based on the priority.

Fig. 18 is a diagram showing a second specific example of a display mode of the device selection screen based on the priority.

Fig. 19 is a diagram showing a third specific example of the display form of the device selection screen based on the priority.

Description of the reference numerals

100 … remote operating system; 1. 1-1 to 1-5 … household electrical appliances; 2. 2a, 2b … remote operation terminal; 201 … a first communication part; 202 … a second communication part; 203 … display part; 204 … storage section; 205 … input; 211. 211a … device registration unit; 212. 212a … virtual remote control unit; 213 … a position information transmitting unit; 214 … notification receiving unit; 3. 3c … remote control device; 31 … a communication part; 32. 32c … storage section; 33 … equipment control part; 34. 34c … to the transmitting part.

Detailed Description

Hereinafter, a remote control terminal, a program, a remote control device, and a remote control system according to the embodiments will be described with reference to the drawings.

(first embodiment)

Fig. 1 is a diagram showing a specific example of a remote operation system using a remote operation terminal 2 according to a first embodiment, a remote operation system 100 includes a plurality of home appliances 1 to be remotely operated, a remote operation terminal 2 that receives AN operation input to the home appliance 1 to be operated, and a remote control device 3 that instructs AN operation corresponding to the operation input to the home appliance 1 in accordance with the operation input to the remote operation terminal 2, in fig. 1, home appliances 1-1 (refrigerator), home appliances 1-2 (lighting device), home appliances 1-3 (air conditioner), home appliances 1-4 (television), home appliances 1-5 (washing machine) are shown as AN example of the home appliance 1 to be operated, the home appliances 1 and the remote operation terminal 2 are communicably connected to the remote control device 3 via, for example, a wireless router RT and a Network NW, the Network NW may be a WAN (Wide Area Network) such as the internet or a dedicated line, and may be L AN (L AN, local Area Network).

Here, the home appliance 1 has 2 wireless communication functions. The 1 is a function of transmitting and receiving radio waves for radio communication with the radio router RT. With this function, the home appliance 1 can perform bidirectional communication with the remote control apparatus 3 via the wireless router RT. The other 1 is a function of transmitting radio waves for distance measurement. With this function, the remote operation terminal 2 that has received the radio wave for distance measurement can measure the distance to each home appliance 1. When the remote operation terminal 2 can receive the radio wave for radio communication, the radio wave for radio communication may be used as a radio wave for distance measurement.

Fig. 2 is a block diagram showing a Specific example of a functional configuration of the remote operation terminal 2 according to the first embodiment, the remote operation terminal 2 according to the first embodiment includes a CPU (Central Processing Unit), a memory, an auxiliary storage Device, and the like connected by a bus, and executes a program, the remote operation terminal 2 functions as a Device including a first communication Unit 201, a second communication Unit 202, a display Unit 203, a storage Unit 204, an input Unit 205, a Device registration Unit 211, and a virtual remote control Unit 212 by executing the program, all or a part of each function of the remote operation terminal 2 may be realized by hardware such as an ASIC (Application Specific Integrated Circuit) or a P L D (Programmable L analog Device) or an FPGA (Field Programmable gate array), a computer readable recording medium, such as a computer readable recording medium, a ROM (compact disk), a CD-readable recording medium, and a transmission medium.

The first communication unit 201 is a wireless communication interface for communicably connecting the remote operation terminal 2 and the wireless router RT. The first communication unit 201 is communicably connected to the wireless router RT, and the remote operation terminal 2 can communicate with the remote control device 3 via the wireless router RT and the network NW.

The second communication unit 202 (an example of a distance measuring unit) is a communication interface that receives radio waves for distance measurement transmitted from each home appliance 1. The second communication unit 202 may have a function of receiving radio waves for distance measurement and a function of measuring the intensity of the received radio waves. Therefore, the second communication unit 202 may or may not have a function of transmitting radio waves. For example, the second communication unit 202 measures the distance between each home appliance 1 and the remote operation terminal 2 by measuring the intensity of the distance measurement radio wave transmitted from each home appliance 1.

The display unit 203 is configured to include a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, an organic E L (Electro-luminescence) display, and the like, the display unit 203 has a function of displaying various information related to the remote operation terminal 2, and for example, a registration screen, an operation screen, and the like of the home appliance 1 are displayed on the display unit 203, the registration screen is a screen displayed when a desired home appliance 1 to be operated is registered with respect to the remote operation terminal 2, and the operation screen is a screen displayed when an operation input is received with respect to the registered home appliance 1 (hereinafter, referred to as "registered device").

The storage unit 204 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 204 stores various setting information related to the operation of the present apparatus. For example, the storage unit 204 stores setting information such as registration information of the home appliance 1 and information related to an operation screen of each registered device (hereinafter referred to as "screen information").

The input unit 205 is configured using an input device such as a keyboard or a touch panel. The input unit 205 has a function of receiving input of various operations to the remote operation terminal 2 and outputting the input information to a function unit corresponding to the operation. Specifically, when an operation input to the registration screen is accepted, the input unit 205 outputs the input information to the device registration unit 211. On the other hand, when receiving an operation input on the operation screen, the input unit 205 outputs the input information to the virtual remote control unit 212.

The device registration unit 211 has a function of registering the home device 1 as an operation target with respect to the remote operation terminal 2. Specifically, the device registration section 211 displays a registration screen on the display section 203, and updates the registration information based on the input information for the registration screen. The update of the registration information may be addition of the home appliance 1 specified by the input information, or deletion of the specified home appliance 1.

The virtual remote control unit 212 has a function of generating and controlling a virtual remote controller for each registered device. The virtual remote controller is a virtual remote control device generated by software, and is an example of a virtual operation unit in which correspondence is established in a one-to-one manner for each registered device. Further, the generation of the virtual remote controller means: an operation unit functioning as a remote controller for each registered device is generated on the memory by executing the application program. The display of the virtual remote controller is an example of an operation screen for receiving an operation input to the registration device. Fig. 2 shows an example in which the virtual remote control unit 212 generates N (N is an integer equal to or greater than 1) virtual remote controllers #1 to # N.

Specifically, the virtual remote control unit 212 generates virtual remote controllers for the respective registered devices in advance before receiving an operation input to the home appliance 1, and changes the display modes of the generated virtual remote controllers according to the position of the device (remote operation terminal 2). Information necessary for generating the virtual remote controller is stored in advance in the storage unit 204 as setting information such as registration information and screen information. The virtual remote control unit 212 is an example of each of the "control unit" and the "display control unit".

For example, the virtual remote control unit 212 changes the display mode of the virtual remote controller of each registered device according to the distance between the remote operation terminal 2 and the registered device. For example, the virtual remote control unit 212 may be configured to select the home appliance 1 closest to the home device (remote operation terminal 2) from among a plurality of registered appliances, and preferentially display the virtual remote controller of the selected home appliance 1 on the display unit 203. For example, the virtual remote control unit 212 may switch the display of the display unit 203 to the display of the virtual remote controller of the selected home appliance 1, or may display the virtual remote controller of the selected home appliance 1 to be larger than the other virtual remote controllers.

In this case, the virtual remote control unit 212 needs to acquire information (hereinafter referred to as "distance information") indicating the distance (or a correlation value thereof) between the device (remote operation terminal 2) and each registered device, but the acquisition of the distance information may be realized by any method. For example, the virtual remote control unit 212 may be configured to acquire distance information from another device, or may have a function of generating distance information. In general, it is known that the intensity of a radio wave observed at a receiving side is correlated with the distance between the transmitting side and the receiving side of the radio wave. The reason why the home appliance 1 in the present embodiment has a function of transmitting a radio wave for distance measurement is to measure the distance between each registered device and the remote operation terminal 2 based on the correlation.

In this case, the virtual remote control unit 212 can generate distance information by measuring (estimating) the distance between the self-apparatus and each of the registered devices based on the reception intensity of the radio wave for distance measurement observed in the self-apparatus. In this case, the reception intensity of the radio wave for distance measurement observed in the present apparatus can be measured by the second communication unit 202. The second communication unit 202 can measure the received radio wave intensity for each of the home appliances 1 that have transmitted the radio wave for distance measurement.

The virtual remote control unit 212 has a function of acquiring information of operation input to the virtual remote controller being displayed from the input unit 205 and transmitting the input information to the remote control apparatus 3 via the first communication unit 201. The remote control apparatus 3 analyzes the input information and generates control information for instructing an operation corresponding to the content of the operation input to each registered device. Further, the generated control information is transmitted from the remote control apparatus 3 to each registered device via the network NW, thereby realizing remote operation of the home appliance 1 using the remote operation terminal 2. The virtual remote control unit 212 may update the display of the virtual remote controller according to the content of the input information.

Fig. 3, 4, and 5 are diagrams showing specific examples of the device selection screen displayed on the remote operation terminal 2 according to the first embodiment. The device selection screen is a screen for accepting an input of an operation of a virtual remote controller for selecting a display object. For example, the device selection screen is generated by the virtual remote control unit 212 and displayed on the display unit 203. Fig. 3 shows an example of a device selection screen in a case where even 1 home appliance 1 as an operation target has not been registered by the remote operation terminal 2. In this case, only the button B1 for instructing addition of a new operation object is displayed on the device selection screen G1. Further, a registration screen for adding a new home appliance 1 is displayed in response to an input of an operation of the selection button B1.

Fig. 4 shows an example of a device selection screen in a case where a refrigerator, an air conditioner, and a washing machine are registered as the home appliance 1 to be operated. In this case, icons C11 to C13 for selecting each registered device and a button B1 similar to that in fig. 3 are displayed on the device selection screen G1. Further, in response to an input of an operation to select any one of the icons C11 to C13, an operation screen of a virtual remote controller of the registered device corresponding to the selected icon is displayed.

Fig. 5 is an example of a device selection screen in a case where a plurality of home devices 1 of the same type are registered as operation targets. In this case, the device selection screen G1 displays the icons C21 to C23 of the respective registered devices and the button B1, as in fig. 4. When the same type of home electric appliances 1 are registered as the operation target (air conditioners in this case), it is difficult to distinguish which icon corresponds to which registered appliance if the same icon image is used for each type of appliance. Therefore, in such a case, the identification information of each registered device can be displayed at the same time as the icon is displayed. Fig. 5 shows an example of a device selection screen that can identify the same type of air conditioners from the display of a common icon and the display of the lower 4 bits ("xxxx", "yyyyyy", "zzzz" in the drawing) of a MAC (Media Access Control) address unique to each air conditioner. The identification information of each registered device may be, for example, an installation location of an air conditioner (living room, bedroom, etc.), an arbitrary number assigned at the time of registration, or a character string input by the user.

Fig. 6 is a diagram showing a specific example of an operation screen displayed on the remote operation terminal 2 according to the first embodiment. Fig. 6 shows an example of an operation screen of the air conditioner. For example, the air conditioner operation screen G2 includes a display area C31 for displaying identification information of an operation target, a display area C32 for an operation mode of the air conditioner, a display area C33 for indoor temperature and humidity, a display area C34 for a clock, a display area C35 for operation buttons for setting temperature and setting temperature, a display area C36 for operation buttons for starting or stopping the air conditioner, a display area C37 for various states, a menu button B2 for calling various functions other than the above, and the like. For example, the operation menu displayed by selecting the menu button B2 may include a function of displaying information specific to the device (for example, error information, device identification information, and the like), or may include a function common to the device such as customer support.

Fig. 7 is a flowchart showing a flow of processing in which the remote operation terminal 2 changes the display mode of the virtual remote controller of each home appliance 1 according to the distance from the registered device in the first embodiment. Here, it is assumed that all the home appliances 1 to be operated are registered in the remote operation terminal 2 and no new operation target is added. First, the virtual remote control unit 212 determines whether or not the user is operating the remote operation terminal 2 (step S101). Here, the situation in which the remote operation terminal 2 is operating is a situation in which the user is performing input of information with respect to the registration screen or the operation screen. In this case, for example, the virtual remote control unit 212 may determine that the operation is being performed when the latest input information is output from the input unit 205 within a predetermined time, and may determine that the operation is not being performed when the input information is not output from the input unit 205 for a predetermined time or more.

When it is determined that the remote operation terminal 2 is in operation (yes in step S101), the virtual remote control unit 212 returns the process to step S101 and repeats the determination as to whether or not the own apparatus is in operation. On the other hand, when it is determined that the remote operation terminal 2 is not being operated (no at step S101), the virtual remote control unit 212 acquires the distance information from the second communication unit 202 (step S102), and selects the home appliance 1 closest to the home appliance based on the acquired distance information (step S103). Further, the virtual remote control unit 212 displays the operation screen on the display unit 203 for the virtual remote controller of the selected home appliance 1 (step S104).

With the remote operation terminal 2 according to the first embodiment configured as described above, by preferentially displaying the operation screen of the virtual remote controller of the home appliance 1 closest to the registered device, it is possible to improve the operability of the remote operation terminal that enables the operation of a plurality of target devices with 1 remote operation terminal. Specifically, the user who wants to select the operation target can cause a desired operation screen to be displayed only by approaching the home appliance 1 as the operation target while holding the remote operation terminal 2, without inputting the selection operation of the operation target to the remote operation terminal 2.

(modification example)

The method of measuring the distance between each home appliance 1 and the remote operation terminal 2 is not limited to the method of observing the reception intensity of the radio wave for distance measurement transmitted from each home appliance 1 on the remote operation terminal 2 side. For example, each of the home appliances 1 may output the same sound (the same volume and tone) at different timings, and the home appliance 1 that outputs the sound with the smallest attenuation of the sound observed at the remote operation terminal 2 may be determined as the home appliance 1 closest to the remote operation terminal 2. For example, the remote operation terminal 2 may be configured to generate distance information based on an indoor map created by the sweeping robot, or may be configured to generate distance information based on image data obtained by imaging the inside of the room. The remote operation terminal 2 may be configured to generate the distance information based on indoor model data in which a plurality of image data are 3D modeled. The image data may be acquired in advance, or when the remote control terminal 2 includes an imaging unit such as a camera, the image data may be captured by the remote control terminal 2 every time the distance is measured. The remote operation terminal 2 may generate distance information using highly accurate GPS position information.

The virtual remote control unit 212 may be configured to acquire the distance information and change the display mode of the operation screen when a specific operation (first specific operation) is performed by the user. For example, the first specific operation may be selection of information being displayed (for example, pressing a menu button such as "search remote controller"), an operation of shaking the remote operation terminal 2, or a specific input operation (for example, an operation of sliding the screen down from above).

The remote operation terminal 2 may include a unit (hereinafter, referred to as "mode switching unit") capable of switching an operation mode in which: an operation mode (automatic selection mode) in which the display mode of the operation screen is automatically changed according to the distance between the remote operation terminal 2 and the home appliance 1, and an operation mode (manual selection mode) in which the display object of the operation screen is changed according to the designation by the user. The "manual selection mode" is an operation mode in which the display mode of the operation screen is fixed regardless of the distance between the home appliance 1 and the remote operation terminal 2 (that is, regardless of the position of the remote operation terminal 2) unless there is a user specification (user operation). The mode switching unit may switch the operation mode in accordance with a specific operation (second specific operation) performed by the user, or may switch the operation mode in accordance with an instruction input by the user. The second specific operation may be selection of information being displayed (for example, pressing of a menu button such as "automatic selection mode" or "manual selection mode"), an operation of shaking the remote operation terminal 2, or a specific input operation (for example, an operation of sliding the screen down from above).

Further, even during the period of the "manual selection mode", the remote operation terminal 2 may be configured to acquire the distance information and change the display mode of the operation screen when a specific operation (the first specific operation) is performed by the user.

At least one of the device registration unit 211 and the virtual remote control unit 212 included in the remote operation terminal 2 may be included in the remote control apparatus 3. In this case, the remote control device 3 may be configured to acquire the distance information from the remote operation terminal 2 and display the operation screen of the virtual remote controller selected based on the acquired distance information on the remote operation terminal 2. In this case, for example, the remote operation terminal 2 may be configured to include a display control unit that controls the display mode of the display unit 203 in place of the device registration unit 211 and the virtual remote control unit 212, transmit the distance information to the remote control device 3, and display the operation screen of the selected virtual remote controller based on the screen information acquired from the remote control device 3. In this case, the input unit 205 of the remote operation terminal 2 may be configured to transmit the input information of the user to the remote control device 3.

(second embodiment)

Fig. 8 is a block diagram showing a specific example of the functional configuration of the remote operation terminal 2a in the second embodiment. The remote operation terminal 2a according to the second embodiment is different from the remote operation terminal 2 according to the first embodiment in that an apparatus registration unit 211a is provided instead of the apparatus registration unit 211, and a virtual remote control unit 212a is provided instead of the virtual remote control unit 212. The other functional units are the same as those of the remote operation terminal 2 in the first embodiment. Therefore, in fig. 8, the same reference numerals as those in fig. 2 are given to the same functional portions as those in the first embodiment, and the description thereof will be omitted.

The device registration unit 211a has a function of classifying registered devices into 1 or more device groups, in addition to the same function as the device registration unit 211 in the first embodiment. For example, the device registration unit 211a may display a registration screen of a device group on the display unit 203, and classify the registered devices based on input information for the registration screen, or may classify the registered devices based on arbitrary attribute information of the registered devices. The device registration unit 211a updates the registration information by adding group information indicating the correspondence relationship between the registered devices and the device groups. The update of the registration information may be performed by adding the specified home device 1 to the specified device group, or by deleting the specified home device 1 from the specified device group. Further, the update of the registration information may be performed by adding or deleting the device group.

Specifically, the virtual remote control unit 212a has a function of collectively inputting the same operation to virtual remotes of home electric appliances 1 classified into a registered device group for each of the registered device groups, and for example, fig. 8 shows an example in which the virtual remote control unit 212a classifies registered devices into M (M is an integer of 1 or more) device groups #1 to # M, and fig. 8 shows an example in which L (L is an integer of 1 or more and N or less) virtual remotes #1 to # L are classified into the device group # 1.

Specifically, the virtual remote control unit 212a generates a virtual remote controller for each of the registered devices based on the setting information such as the registration information and the screen information, and displays a screen (hereinafter referred to as "collective operation screen") for accepting an input of an operation to be input to the generated virtual remote controller on the display unit 203. Here, it is assumed that the screen information in the second embodiment also includes screen information for displaying the collective operation screen for each device group.

For example, when the home appliances 1 belonging to the same device group are of the same model, the same operation screen as the virtual remote controller of the home appliance 1 belonging to the device group can be used for the collective operation screen of the device group. When the home appliances 1 of different models are mixed in the same appliance group, the collective operation screen of the appliance group may have an input interface (for example, power on or off) through which a common operation can be specified with respect to the virtual remote controllers of the home appliances 1 belonging to the appliance group.

The virtual remote control unit 212a in the second embodiment has a function of changing the display mode of the collective operation screen for each device group in accordance with the distance between the remote operation terminal 2a and the registered device. For example, the virtual remote control unit 212a may be configured to select a home appliance 1 closest to the remote operation terminal 2a from a plurality of registered appliances, and to display a collective operation screen of an appliance group to which the selected home appliance 1 belongs, in priority to a collective operation screen of another appliance group. Specifically, the virtual remote control unit 212a may be configured to switch the display of the display unit 203 to the display of the collective operation screen of the device group to which the selected home appliance 1 belongs. For example, the virtual remote control unit 212a may be configured to display a collective operation screen of a device group to which the selected home appliance 1 belongs so as to be larger than collective operation screens of other device groups.

Fig. 9 is a flowchart showing a flow of processing in which the remote operation terminal 2a changes the display form of the collective operation screen for each device group in accordance with the distance from the registered device in the second embodiment. In fig. 9, the same processes as those in the first embodiment are denoted by the same reference numerals as those in fig. 7, and the description of the same processes will be omitted. Here, the virtual remote control unit 212a selects a device group including the home device 1 selected in step S103 (step S201). Further, the virtual remote control unit 212a causes the display unit 203 to display the collective operation screen of the selected device group (step S202).

Fig. 10 is a diagram showing a specific example of a collective operation screen of a device group displayed on the remote operation terminal 2a according to the second embodiment. For example, fig. 10 shows an example of a collective operation screen of a device group "living room" in which home electric appliances 1 arranged in the living room are grouped. For example, on the collective operation screen G3 of the device group "living room", there are displayed: icons C41 to C43 of virtual remote controllers for individually selecting the home electric appliances 1 included in the device group; and collective operation buttons B31 and B32 for instructing collective operations with respect to the virtual remote controllers of the home electric appliances 1 included in the device group. The remote operation terminal 2a can switch the display of the operation screen G3 to the display of the virtual remote controller of the home appliance 1 corresponding to the selected icon in response to an input of an operation to select any one of the icons C41 to C43.

The remote operation terminal 2a according to the second embodiment configured as described above is configured to preferentially display the collective operation screen of the device group including the home appliance 1 closest to the registered devices over the collective operation screen of the other device group, thereby improving operability in selecting the operation target device from the plurality of registered devices. Specifically, the user who intends to select the operation target can cause a desired collective operation screen to be displayed and collectively input the operation of the virtual remote controller for the home appliances 1 included in the device group by simply holding the remote operation terminal 2a without inputting the selection operation of the operation target to the remote operation terminal 2a and by any home appliance 1 included in the device group that is close to the target.

Further, depending on the home electric appliance 1, there is a case where the remote operation terminal 2 according to the first embodiment cannot realize such remote operation of the home electric appliance 1 because it has a communication function but does not have a function of transmitting a radio wave for distance measurement. However, since the remote operation terminal 2a according to the second embodiment enables the plurality of home appliances 1 to be operated in units of appliance groups, it is not necessary to provide all the home appliances 1 with a function of transmitting radio waves for distance measurement. For example, if the home appliances 1 close to each other are registered in the same device group, if at least 1 home appliance 1 capable of transmitting the radio wave for distance measurement exists in the same device group, and the collective operation screen of the device group is preferentially displayed, it is possible to collectively operate a plurality of home appliances 1 close to each other, including the home appliance 1 incapable of transmitting the radio wave for distance measurement.

Further, according to the remote operation terminal 2a of the second embodiment, since the collective operation screen in which the home electric appliances 1 sorted according to the attribute (classification criterion of the appliance group) of the remote operation terminal 2a are the objects of the remote operation is displayed, selection of the operation object becomes easy, and the operability of the remote operation can be improved.

(modification example)

In the second embodiment, the case where the device group is registered mainly by the input operation of the user has been described, but the remote operation terminal 2a may be configured to classify the home appliance 1 into the device group corresponding to the distance from the home appliance itself using the generated distance information. According to the remote operation terminal 2a having such a configuration, the user does not need to register the device group, and thus the labor and the load can be reduced. In this case, the remote operation terminal 2a may be configured to generate distance information at a timing when the user registers the device group, and display a list of home appliances 1 close to the home operation terminal.

For example, only 1 device group (for example, an air conditioner group or a television set) may be displayed, and the displayed device group may be changed in accordance with an input operation by a user. In this case, the operation screen of the virtual remote controller of the home appliance 1 closest to the remote operation terminal 2a among the virtual remote controllers of the displayed device group may be preferentially displayed.

The virtual remote control unit 212a may be configured to acquire the distance information and change the display form of the collective operation screen when the user performs the first specific operation, as in the first embodiment.

(third embodiment)

Fig. 11 is a block diagram showing a specific example of the functional configuration of the remote operation terminal 2b in the third embodiment. The remote control terminal 2b according to the third embodiment is different from the remote control terminal 2 according to the first embodiment in that it further includes a position information transmitting unit 213 and a notification receiving unit 214. The other functional units are the same as the remote operation terminal 2 in the first embodiment. Therefore, in fig. 11, the same reference numerals as those in fig. 2 are given to the same functional portions as those in the first embodiment, and the description thereof will be omitted.

The position information transmitting unit 213 has a function of acquiring position information of its own device from a position information providing device such as a GPS (Global Positioning System) or the like, and a function of transmitting the acquired position information to the remote control device 3.

The notification receiving unit 214 has a function of receiving a notification from the remote control apparatus 3 and reporting the received notification. For example, as an example of a method of reporting a notification, the notification receiving unit 214 may display the content of the received notification on the display unit 203.

Fig. 12 is a block diagram showing a specific example of the functional configuration of the remote control device 3 in the third embodiment, the remote control device 3 in the third embodiment includes a CPU, a memory, an auxiliary storage device, or the like connected by a bus, and executes a program, the remote control device 3 functions as a device including a communication unit 31, a storage unit 32, a device control unit 33, and a notification transmission unit 34 by executing the program, all or a part of the functions of the remote control device 3 may be realized by hardware such as ASIC, P L D, or FPGA, the program may be recorded in a computer-readable recording medium, the computer-readable recording medium may be, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage device such as a hard disk incorporated in a computer system, and the program may be transmitted via an electric communication line.

The communication unit 31 is a communication interface for communicably connecting the remote control device 3 to each of the home electric appliances 1 and the remote operation terminal 2b via the network NW.

The storage unit 32 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 32 stores the registration position information of the remote operation terminal 2b in advance. The registered position information is information indicating a position of a range associated with the remote operation terminal 2b in advance.

The device control unit 33 has a function of controlling the operation of the designated home appliance 1 in accordance with an operation input to the home appliance 1 from the remote operation terminal 2 b. Specifically, the device control unit 33 acquires information (hereinafter referred to as "operation information") indicating an operation input on the operation screen of the home appliance 1 or the device group from the remote operation terminal 2b via the communication unit 31. The device control unit 33 generates control information for causing the home appliance 1 as the operation target to execute the operation indicated by the acquired operation information, and transmits the generated control information to the home appliance 1 as the operation target.

The notification transmission unit 34 has a function of transmitting a notification according to the position of the remote operation terminal 2 b. Specifically, the notification transmission unit 34 acquires the position information from the remote operation terminal 2b via the communication unit 31, and acquires the registered position information from the storage unit 32. The notification transmission unit 34 determines a notification condition based on the position of the remote operation terminal 2b based on the acquired position information and the registered position information, and transmits a notification when the notification condition is satisfied.

Fig. 13 is a flowchart showing the flow of processing in which the remote control apparatus 3 transmits a notification in accordance with the position of the remote operation terminal 2b in the third embodiment. First, the notification transmission unit 34 determines whether or not the position information is received from the remote operation terminal 2b (step S301). When it is determined that the position information has been received (yes at step S301), the notification transmission unit 34 determines whether or not the remote operation terminal 2b is located within a predetermined range registered in advance, based on the received position information and the registered position information stored in the storage unit 32 (step S302).

When the notification transmission unit 34 determines that the remote operation terminal 2b is not located within the predetermined range registered in advance (no at step S302), the notification transmission unit 34 determines whether or not the home appliance 1 in operation is present in the registered devices of the remote operation terminal 2b (step S303). Here, it is assumed that the remote control device 3 performs status management of each registered device by the device control unit 33. Specifically, the state information indicating the state of each registered device is recorded in the storage unit 32, and the state information is updated by the device control unit 33 in accordance with a change in the state of each registered device.

When it is determined that the home appliance 1 that is operating is present in the registered device of the remote operation terminal 2b (yes in step S303), the notification transmission unit 34 notifies the remote operation terminal 2b that the home appliance 1 that is operating is present in the registered device (step S304), and returns the process to step S301.

On the other hand, when it is determined in step S301 that the location information has not been received (step S301 — no), when it is determined in step S302 that the remote operation terminal 2b is located within the predetermined range registered in advance (step S302-yes), or when it is determined in step S303 that the home appliance 1 in operation is not present in the registered device of the remote operation terminal 2b (step S303-no), the notification transmission unit 34 does not perform the notification and returns the process to step S301.

In the remote operation system 100 according to the third embodiment configured as described above, notification regarding the status of the registered device is performed in accordance with the position of the remote operation terminal 2 b. By performing such notification, the user of the remote operation terminal 2b can perform a more appropriate operation on the home appliance 1.

For example, if the user registers the location information of his/her own home as the registered location information, even if the user forgets to turn off the power supply of the air conditioner and goes out, the remote control device 3 detects such a situation and notifies the remote operation terminal 2b carried by the user. Therefore, the user can notice the forgetting to turn off the power supply around the home, and can perform an appropriate operation of turning off the power supply more quickly.

(modification example)

In the third embodiment, an example in which the notification receiving unit 214 outputs the notification received from the remote control apparatus 3 to the display unit 203 has been described, but the notification receiving unit 214 may be configured to notify the virtual remote control unit 212 of the notification. On the other hand, the virtual remote control unit 212 may be configured to preferentially display the operation screen of the home appliance 1 to be notified on the display unit 203 in response to reception of the notification. With such a configuration, the virtual remote control unit 212 can change the display mode of the operation screen of each home appliance 1 according to the distance between the predetermined specific position and the position of the remote operation terminal 2b, and can improve the operability in selecting the operation target device.

(fourth embodiment)

Fig. 14 is a block diagram showing a specific example of the functional configuration of the remote control device 3c according to the fourth embodiment. The remote control apparatus 3c according to the fourth embodiment is different from the remote control apparatus 3 according to the third embodiment in that the storage unit 32c is provided instead of the storage unit 32, and the notification transmission unit 34c is provided instead of the notification transmission unit 34. The other functional units are the same as the remote control device 3 in the third embodiment. Therefore, in fig. 14, the same reference numerals as those in fig. 12 are given to the same functional portions as those in the third embodiment, and the description thereof will be omitted.

The storage unit 32c is different from the remote control device 3 according to the third embodiment in that information related to sharing of setting information of the remote operation terminal 2b (hereinafter referred to as "shared setting information") is stored in advance in addition to the registered position information. For example, the shared setting information may include settings related to the sharing of the notification. The sharing of the notification here means that the notification for a certain user is also notified to the shared user. The setting relating to the sharing of the notification may be set for each of the notified contents or may be set for each of the users.

The notification transmission unit 34c has a function similar to that of the notification transmission unit 34 in the third embodiment, and also has a shared function of transmitting a notification based on the position of the remote operation terminal 2b to another communication device associated with the remote operation terminal 2 b.

Fig. 15 is a flowchart showing the flow of processing in which the remote control apparatus 3c transmits a notification in accordance with the position of the remote operation terminal 2b in the fourth embodiment. In fig. 15, the same processes as those in the third embodiment are denoted by the same reference numerals as those in fig. 13, and thus descriptions of the same processes are omitted. Here, the notification transmission unit 34c determines whether or not the shared setting of the notification is set for the remote operation terminal 2 that is the notification target in step S304, based on the shared setting information (step S401).

When the remote operation terminal 2 to which the notification is to be sent determines that the shared setting of the notification is set (yes in step S401), the notification transmission unit 34c transmits the notification to the remote operation terminal 2 to which the notification is to be sent to the other communication device associated with the remote operation terminal 2 (step S402), and returns the process to step S301.

On the other hand, when it is determined in step S401 that the notification target remote operation terminal 2 has not set the common setting for notification (no in step S401), the notification transmission unit 34c returns the process to step S301 without performing notification.

In the remote operation system 100 according to the fourth embodiment configured as described above, in addition to the notification relating to the state of the registered device in accordance with the position of the remote operation terminal 2b, when the common setting of the notification is set for the remote operation terminal 2b, the notification relating to the state of the registered device can be performed also for the communication device to be shared. By sharing such a notification, the user of the remote operation terminal 2b can perform a more appropriate operation with respect to the home appliance 1.

For example, if the user registers the location information of his own home as the registered location information in advance, even if the user forgets to turn off the power supply of the air conditioner and goes out, the remote control device 3c detects this situation and notifies the remote operation terminal 2b carried by the user, and also notifies another communication device associated with the remote operation terminal 2 b. Therefore, even if the user does not notice the notification of the remote control apparatus 3c, the possibility that the user of another communication apparatus notices the notification can be increased, and the possibility of appropriate measures can be increased. For example, it is possible to increase the possibility that the user of the other communication device contacts the user of the remote operation terminal 2b to notify that the power supply of the air conditioner has been forgotten to be turned off, and the user of the remote operation terminal 2b that has received the contact returns to his/her home to perform the operation of turning off the power supply of the air conditioner.

(modification example)

Since the result of the activity of a certain user is shared by other users according to the sharing of the notification in the present embodiment, it can be considered as one of the methods for realizing the nursing function. For example, when the device control unit 33 records information indicating the control history of the home appliance 1 in the storage unit 32c, the notification transmission unit 34c may detect that a certain user does not open or close the door of the refrigerator for 3 days, and notify others who share the setting of the condition.

The remote control device 3c may have a function of sharing the notification based on the position of the remote operation terminal 2b among predetermined users and sharing the operation screen among predetermined users. In this case, for example, the remote control device 3c may store screen information of registered devices of the remote operation terminals 2b in advance, or may acquire screen information of registered devices from the remote operation terminals 2b in advance, and transmit screen information of the home appliance 1 to be notified when notification is performed based on the position of the remote operation terminal 2 b. Thus, the remote operation terminal 2b that has received the notification can display the operation screen of the home appliance 1 notified using the received screen information. With this configuration, the remote control terminal 2b on the side of the notification can change the display mode of the operation screen in the other remote control terminal 2b according to the position of the device. On the other hand, the remote control terminal 2b on the side receiving the notification can change the display mode of the operation screen in the device according to the position of the other remote control terminal 2 b.

The remote control terminal 2b may be configured to change a display form of an operation screen of a device registered in advance as an operation target of another remote control terminal 2b in the own apparatus according to a position of the own apparatus. In this case, for example, the remote operation terminal 2b may store registration information and screen information of the registered device of another remote operation terminal 2b in advance, measure the distance between the local apparatus and the registered device of the local apparatus by the same method as the method for measuring the distance between the local apparatus and the registered device of the local apparatus, and preferentially display an operation screen satisfying a predetermined condition in the operation screen of the registered device of the other remote operation terminal 2b on the display unit 203 of the local apparatus based on the measured distance.

In each of the above embodiments, the description has been given of the case where the virtual remote control unit 212 selects the operation screen of the virtual remote controller of the registered device or the device group displayed on the display unit 203 based on the distance between the present apparatus and the registered device, but in the case where the virtual remote control unit 212 is generated by executing software, the virtual remote control unit 212 may be configured to select the operation screen displayed at the time of its own generation in the same manner as the above-described method. For example, when the remote operation terminal is configured using a smartphone, it cannot be guaranteed that the application program for realizing the virtual remote control unit 212 is always started in the remote operation terminal 2. It is instead believed that the likelihood of not starting is higher. Therefore, by measuring the distance to the home appliance 1 at the timing of starting the application program and displaying the operation screen of the home appliance 1 in the vicinity, the convenience and operability of the user can be further improved.

(fifth embodiment)

Fig. 16 is a block diagram showing a specific example of the functional configuration of the remote operation terminal 2d in the fifth embodiment. The remote control terminal 2d in the fifth embodiment is different from the remote control terminal 2 in the first embodiment in that a virtual remote control unit 212d is provided instead of the virtual remote control unit 212. The other functional units are the same as the remote operation terminal 2 in the first embodiment. Therefore, in fig. 16, the same reference numerals as those in fig. 2 are given to the same functional portions as those in the first embodiment, and the description thereof will be omitted.

The virtual remote control unit 212d, like the virtual remote control unit 212 of the first embodiment, can automatically determine whether or not a predetermined condition relating to a registered device is satisfied without being operated by a user, and display an operation screen of the registered device satisfying the predetermined condition on the display unit 203 so as to be easier for the user to operate than other operation screens. Here, the virtual remote control unit 212d of the second embodiment determines the priority of display related to the operation screen for each registered device, and changes the display mode of the operation screen based on the determined priority, while the virtual remote control unit 212 of the first embodiment changes the display mode of the operation screen for each registered device according to the position of the remote operation terminal 2. The operation screen mentioned here includes a screen of a virtual remote controller indicating each registered device as in the first embodiment, and a screen of a registered device (device selection screen) for selecting a remote operation target from among the registered devices. The display mode of the operation screen mentioned here includes the display mode of each registered device in the device selection screen, in addition to the display mode of each operation screen of the virtual remote controller of each registered device.

Note that "automatic" as referred to herein means that the process of changing the display form of the operation screen is not actively executed by the virtual remote control unit 212d in response to an execution instruction of the process by the user. In this sense, typically, the case where the display mode change process is executed when the user does not operate the remote operation terminal 2d corresponds to "automatic". Note that "automatic" mentioned here does not mean that it is always necessary to be automatic, but means that the process of changing the display mode is executed at least within a range that does not hinder the use of the remote operation terminal 2d by the user. In this sense, even when the user is operating the remote operation terminal 2d at a timing, the execution of the process of changing the display form at such a timing is included in the scope of "automatic" without interfering with the operation performed by the user.

The virtual remote control unit 212d displays the operation screen of each registered device on the display unit 203 in a display mode corresponding to the determined priority. Specifically, the virtual remote control unit 212d determines which of the operation screens of the respective registered devices is preferentially displayed on the display unit 203 based on the priority determined for each registered device. The virtual remote control unit 212d then displays the display of each registered device on the device selection screen on the display unit 203 in the size, shape, character, color, and order corresponding to the priority determined for each registered device. The virtual remote control unit 212d may display the display of each registered device on the device selection screen at a display position corresponding to the priority determined for each registered device. In the following, some specific examples will be described for such a priority determination method.

[ first determining method ]

The first determination method is a method of determining the priority of each registered device based on the action pattern of the user. Specifically, the virtual remote control unit 212d estimates an action pattern of the user based on the usage history of each registered device, and determines the priority of each registered device based on the estimated action pattern. The action pattern mentioned here refers to a tendency regarding a time zone, a place, or the like in which the user performs an operation of registering the device. This action pattern can be estimated by recording information indicating the location or time at which the user performs the operation of the registered device using the remote operation terminal 2d in association with the use history of the registered device in advance. Alternatively, a function of acquiring an action plan of the user may be provided, and an action pattern related to an operation of a registered device using the remote operation terminal 2d may be estimated based on the action plan acquired by the function.

For example, the operation frequency or the number of operations of the registration device by the user can be cited as an example of the action pattern. In this case, the virtual remote control unit 212d estimates the operation frequency or the operation frequency of the registered device by the user based on the usage history of each registered device, and determines a priority corresponding to the estimated operation frequency or operation frequency for each registered device. For example, the virtual remote control unit 212d may be configured to determine a higher priority as the operation frequency is higher or the number of operations is larger for the registered device.

Further, for example, as an example of the action pattern, a time zone in which the user operates the registration device can be cited. The time period may also be a specific time of day. In this case, the virtual remote control unit 212d estimates a time period in which the possibility of the user's operation is considered to be high for each registered device based on the use history of each registered device, and determines the priority of each registered device so as to have a high priority in the estimated time period. For example, when a usage history of an operation screen of the air conditioner in the bedroom before and after 23 hours per day is obtained, it is estimated that the user operates the air conditioner in the bedroom before and after 23 hours. In this case, the virtual remote control unit 212d can determine a high priority for the air conditioning of the bedroom before and after 23 hours per day.

In the above case, the period during which the display mode of the operation screen is changed based on the priority (i.e., the period from the start to the end of the change of the display mode based on the priority) can be arbitrarily set. The period may be changed according to the estimated accuracy of the action pattern. For example, in the case of the bedroom air conditioner described above, if a use history is obtained in which the user operates the bedroom air conditioner with almost no deviation from 23, which is the time period estimated as the action pattern, the time period may be changed to a shorter time width. On the other hand, when a usage history that the air conditioner in the bedroom is operated at 22 hours or operated at 23 hours and 30 minutes is obtained with respect to the time period estimated as the action pattern, the period may be changed to a longer time width.

Further, the action pattern corresponding to the day of the week may be estimated by recording the use history of the registered device in advance together with information indicating the day of the week. For example, when it is estimated that the behavior pattern is often operated on the bedroom air-conditioning on days 23 on monday through friday every week and is not particularly specified in a time zone with a high operation frequency on saturday and sunday, the virtual remote control unit 212d may be configured to increase the priority on the days 23 only on monday through friday every week for the bedroom air-conditioning.

Further, information indicating the operation history of each operation screen is recorded in the storage unit 204 by the virtual remote control unit 212d, for example, in accordance with the operation of each operation screen. In this case, the remote operation terminal 2d may be configured to directly acquire information indicating the use history of each registered device from each registered device, or may be configured to indirectly acquire information from another device such as the remote control apparatus 3.

Further, for example, when it is estimated that the action mode is one in which the power of the television is turned on at 9 days of each week, the virtual remote control unit 212d may determine a high priority for the television before and after 9 days of each week. Then, for example, an action pattern is assumed in which the user ends use of each registered device in the house around 8 hours per day and resumes use around 18 hours. This behavior pattern can be considered as a behavior pattern in which the user goes out before and after 8 hours and goes home before and after 18 hours per day. Therefore, when such an action pattern is estimated, the virtual remote control unit 212d can determine a high priority for the device with a high possibility of being used immediately after returning home, of the registered devices, before and after 18 hours per day.

Further, in the case where the use of each registered device is restarted before and after half 18 hours, it is assumed that an action pattern in which the door of the refrigerator is opened with a high probability after the restart is estimated. This action pattern can be considered as an action pattern in which the user goes home after shopping. Therefore, when such an action pattern is estimated, the virtual remote control unit 212d can determine a high priority for the refrigerator before and after 18 hours and half.

[ second determination method ]

The second determination method is a method of determining the priority of display when the operation screen of the virtual remote controller of each registered device is displayed on the display unit 203 based on the user's place. Specifically, the virtual remote control unit 212d estimates the location of the user based on the positional information of the own device (i.e., the remote operation terminal 2d), and determines the priority of each registered device based on the estimated location of the user.

More specifically, when the position information indicating the current position of the device coincides with the position information of any one of predetermined places in the real space (for example, each room in the user's house), the virtual remote control unit 212d estimates the place corresponding to the coincident position information as the user's place. In this case, it is assumed that information indicating the correspondence between a predetermined location and its position information is registered in advance in the remote operation terminal 2 d.

For example, if "kitchen" is registered as a predetermined location and "a" is registered as location information corresponding to the kitchen, it is assumed that "a" is acquired as location information indicating the current location of the remote operation terminal 2 d. In this case, the virtual remote control unit 212d can estimate that the user is located in the "kitchen" on the assumption that the user holds the remote operation terminal 2 d.

As the position information, satellite positioning information that can be acquired from a positioning system such as a GPS can be used. In this case, even when the position information indicating the current position of the remote operation terminal 2d does not completely match the position information indicating each place (for example, kitchen) registered in advance, when the two pieces of position information indicate positions within a predetermined range, it can be determined that both pieces of position information are present in the same place.

Further, as the position information, instead of absolute position information such as satellite positioning information, relative position information based on a predetermined device installed indoors may be used. For example, assume that a refrigerator capable of performing short-range wireless communication with other devices existing within a predetermined range is provided in a kitchen. In this case, the location information corresponding to "kitchen" can be set to "(can communicate with) the refrigerator (short-range wireless communication)". In this way, the virtual remote control unit 212d can estimate that the user is located in the kitchen based on the fact that the short-range wireless communication between the remote operation terminal 2d and the refrigerator is established.

The virtual remote control unit 212d may estimate the user's place based on the correspondence between the predetermined place and the motion sensor provided at the place and the detection result of the motion sensor. For example, when the human body sensor a corresponds to "kitchen" as the installation location thereof, the virtual remote control unit 212d can estimate that the user is located in the kitchen based on the fact that the human body sensor a senses a person.

The virtual remote control unit 212d may estimate the user's place based on the correspondence between the predetermined place and the registered device provided at the place and the state of the registered device. For example, when a living room television is on, it is highly likely that the user is located in the living room. Therefore, the virtual remote control unit 212d can be assumed to be positioned in the living room when the television set provided in the living room is in the power on state.

The remote operation terminal 2d may be configured to directly acquire information indicating a detection result of the motion sensor or information indicating a state of the registration device from the motion sensor or the registration device, or may be indirectly acquired from another device such as the remote control apparatus 3.

The human body sensor may be of a type that detects a person by using electromagnetic waves, sound waves, or the like, a type that detects a person from an image, a type that detects a person based on fluctuations of radio waves, or the like, or any other type. The body sensor may also be equipped with a registration device.

The virtual remote control unit 212d determines the priority of display when the operation screen of the virtual remote controller for each registered device is displayed on the display unit 203 based on the location of the user estimated in this manner. In this case, the virtual remote control unit 212d may determine a higher priority for the registered device close to the user than for the other registered devices, or conversely, may determine a lower priority for the registered device close to the user than for the other registered devices. Such a method of determining the priority may be used in a differentiated manner according to the content of the operation performed on the registration device that is the subject of the determination of the priority.

For example, regarding an operation performed on the assumption that the user is located at the same place as the device to be operated, such as the viewing setting of a television, the priority may be set high when the operation target is close to the user's place. In contrast, the degree to which the user's location contributes to the determination of the priority can be made relatively small for operations that can be performed without the user being located at the same location as the device to be operated, such as the recording reservation setting of the television program recorder. Further, for example, there is also a registration device such as a washing machine that requires a direct operation (for example, an operation of putting laundry into the washing machine) for the device in order to achieve the purpose of use. In such a case, it is considered that there is a low possibility that the user is located near the registration device of the operation target but intentionally performs an operation of the registration device (for example, an operation start operation of the washing machine) using the operation screen of the remote operation terminal 2 d. Therefore, with regard to such a registration apparatus, a low priority can be set when the registration apparatus as the operation target is close to the place of the user.

[ third determination method ]

The third determination method is a method of determining the priority of display when the operation screen of the virtual remote controller of each registered device is displayed on the display unit 203 based on the presence or absence of communication connection between the own apparatus (remote operation terminal 2d) and each registered device.

For example, the virtual remote control unit 212d determines the priority of each registered device based on the presence or absence of communication connection between the own apparatus and each registered device. In this case, the virtual remote control unit 212d determines a higher priority for the devices which are connected to the own apparatus by communication, among the registered devices, than for the devices which are not connected to the own apparatus by communication. This is because: with regard to a registered device that is not communicatively connected to the device (or has a communication function set to off), it is considered that the possibility of operation using the operation screen is low, and if a high priority is determined for such a registered device, there is a possibility that the convenience of remote operation is reduced. For example, the following can be suppressed: when a user directly operates a washing machine which is not in communication connection with the washing machine, and wants to operate a registered device located in another place, an operation screen of the washing machine which is not desired by the user is preferentially displayed.

In this case, the virtual remote control unit 212d may be configured to determine the priority determined for the registered device based on information received from the registered device connected to the own apparatus by communication. For example, when the user knows that recording of a program scheduled in a television having a program recording function has ended, it is considered that there is a high possibility that the television is operated to view the recorded program. Therefore, in the case where such a situation is detected, a high priority can be decided for the television set. In this case, when the end of recording of the program is notified from the registration device connected to the apparatus itself by communication and the television having the program recording function is notified, the virtual remote control unit 212d can determine the high priority for the television. According to such a method of determining the priority, the virtual remote control unit 212d can determine the priority of each registered device so that the operation screen can be displayed in association with each situation occurring in each registered device.

The remote operation terminal 2d according to the fifth embodiment having the above-described configuration can determine the priority of display when the operation screen of the virtual remote controller of each registered device is displayed on the display unit 203 based on the action pattern of the user, the location of the user, or the presence or absence of communication connection between the apparatus and the registered device. Thus, the remote operation terminal 2d can preferentially display the operation screen of the registered device with a higher possibility of use, and can provide an appropriate operation screen for the user who wants to perform the operation of each registered device using the remote operation terminal 2d without the operation of the user.

The virtual remote control unit 212d may be configured to change the display mode of another operation screen related to the registered device (hereinafter referred to as "related screen") based on the priority determined for each registered device. For example, the virtual remote control unit 212d may be configured to preferentially display a management screen of the food material stored in the refrigerator when a high priority is determined for the refrigerator.

The virtual remote control unit 212d may be configured to change the display mode of each registered device on the device selection screen based on the priority determined for each registered device. In this case, the virtual remote control unit 212d may be configured to determine which of the operation screen and the related screen is to be displayed, based on the above-described action pattern, time period, and the like. The priority of each registered device can be determined by appropriately combining the above determination criteria. For example, the virtual remote control unit 212d may determine a higher priority for a device corresponding to the action pattern of the user among the registered devices selected according to the user's place. For example, the virtual remote control unit 212d may determine a higher priority for a registered device that is in communication connection with the apparatus, among registered devices selected according to the user's place.

In the following, a specific example of a display mode of the device selection screen is shown as another specific example of a display mode of the operation screen based on the priority.

Fig. 17 is a diagram showing a first specific example of a display mode of the device selection screen based on the priority. Specifically, the device selection screen G5 shown in fig. 17 is an example in which the priority of each registered device is expressed by the size of the icons C51 to C55 indicating each registered device. Hereinafter, the registered device indicated by each icon Ci (i corresponds to the number of the reference numeral) is referred to as a "device name Ci". The example of fig. 17 shows: the priority of the refrigerator C51 is higher than that of the washing machine C52, and the priority of the washing machine C52 is higher than that of the air conditioner C53, the fan C54, and the television C55. In the device selection screen, it is preferable that an icon of a registered device to be displayed (hereinafter referred to as a "display target device") is displayed on one screen of the display unit 203. Therefore, the virtual remote control unit 212d adjusts the size of the icon of the display target device displayed on the device selection screen so as to have a size corresponding to the priority and to be visually confirmed by the user in the device selection screen at the same time. By doing so, the user can select a desired device without performing an operation of scrolling or switching the screen. The display target devices may be part or all of the registered devices as long as they are registered devices having a priority equal to or higher than a predetermined priority.

Fig. 18 is a diagram showing a second specific example of a display mode of the device selection screen based on the priority. Specifically, the selection screen G6 shown in fig. 18 shows an example of a device selection screen in which icons of respective registered devices are displayed in groups corresponding to the priorities of the respective registered devices. Fig. 18 is an example in which icons of registered devices having a high priority are classified into a high priority tab T61, and icons of registered devices having a low priority are classified into a low priority tab T62. The high priority tab T61 and the low priority tab T62 can be switched in display by a selection operation. In this case, the virtual remote control unit 212d classifies each registered device into each tag corresponding to the priority, and displays the device selection screen in a state where the high priority tag T61 is selected. By classifying the registered devices into a plurality of groups corresponding to the priority levels and preferentially displaying icons of the registered devices belonging to the group into which the device having the high priority level is classified, the virtual remote control unit 212d can easily identify and select the target home device 1 from among the registered devices having the high priority level.

Fig. 19 is a diagram showing a third specific example of the display form of the device selection screen based on the priority. Specifically, the device selection screen G7 shown in fig. 19 is an example of a device selection screen in which a first region G71 in which icons of registered devices having a high priority are displayed and a second region G72 in which icons of all the registered devices including the registered devices having a high priority are displayed are provided in the screen. According to such a display mode, the user can easily identify and select the registered device having a high priority, and can select another registered device without performing an operation of scrolling or switching the screen.

In either of fig. 18 and 19, as in the example of fig. 17, the size of the icon in each area may be adjusted so as to have a size corresponding to the priority and to be simultaneously visually confirmed by the user in the device selection screen.

In the example of fig. 19, the virtual remote control unit 212d may fix the display positions of the icons in the second region G72 to a predetermined arrangement. Also, the configuration can be changed by an operation of the user. Thus, the user can easily select a registered device having a high priority, and can easily select a registered device other than the high priority by the fixed arrangement.

As described above, although some specific examples have been described for the display form of the operation screen (including the device selection screen) based on the priority, the display form of the operation screen is not limited to these forms. The display form of the operation screen may be represented by any other form as long as the priority of each registered device can be recognized according to the order of display, display position, color, size, characters, and the like.

The virtual remote control unit 212d may be configured to allow the user to switch the display mode of the operation screen to a display mode based on the priority of the registered device or to a display mode other than the display mode by an input operation. For example, the switching may be realized by an input interface such as a button, or may be realized by switching a screen by a tag or the like that displays the target device in a different manner.

The virtual remote control unit 212d may be configured to switch the display mode of the operation screen to a display mode based on the priority of the registered device or to a display mode other than the display mode according to the state of each registered device. For example, the virtual remote control unit 212d may be configured to switch the display mode when the recording end of the program is notified from the television. Thus, the user can recognize the state change of each registered device without performing an operation of switching the display mode of the operation screen and can quickly respond to the state change.

Further, the virtual remote control unit 212d may be configured to automatically display an operation screen of a registration device that is highly likely to be used by the user after that time on the display unit 203 regardless of the user's operation when no operation input is made by the user to the device itself for a predetermined time. With this configuration, when the user who has once finished operating the remote operation terminal 2d wants to operate the remote operation terminal again, the screen related to the registered device that is highly likely to be used can be displayed. Therefore, the convenience of the remote operation terminal 2d can be improved.

Further, the virtual remote control unit 212d may be configured to automatically display the operation screen of the registered device, which is highly likely to be used by the user after that time, on the display unit 203 regardless of the user's operation when the application displaying the operation screen is switched from the standby state to the execution state. For example, in a portable terminal such as a smartphone, an application displayed on a screen may be controlled to be in an execution state, and an application not displayed on the screen may be controlled to be in a standby state. According to such a configuration, when the application program of the display device selection screen, the operation screen, or the related screen is switched to the execution state, the operation screen of the registration device with a high possibility of use can be immediately displayed. Therefore, the convenience of the remote operation terminal 2d can be improved.

According to at least one embodiment described above, operability in selecting an operation target device from a plurality of devices can be improved. For example, by having: a device registration unit that registers a home appliance to be remotely operated; an operation screen on which a virtual operation unit corresponding to the registered home appliance is established and which is a plurality of virtual remote controllers that accept operation inputs to the home appliances respectively associated with the virtual operation unit; and a display control unit that changes the mode of displaying the operation screen on the display unit according to the position of the device. And, for example, by having: a display control unit for controlling a display unit capable of displaying arbitrary information; and a plurality of operation units that can be displayed on the display unit and that can accept operation input relating to the device that has been registered as the remote operation target, wherein the display control unit automatically determines whether or not a predetermined condition relating to the device is satisfied without user operation, and displays the operation unit of the registered device that satisfies the predetermined condition on the display unit in such a manner that the operation unit is easier for the user to operate than the other operation units, thereby achieving the above-described effects.

While several embodiments of the present invention have been described above, the above embodiments are merely presented as examples, and are not intended to limit the scope of the invention. The above embodiments can be implemented in various other ways, and various omissions, substitutions, and changes can be made without departing from the spirit of the invention. The above-described embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalent scope thereof.

Claims (25)

1. A remote operation terminal is provided with:

a plurality of operation units which are virtual operation units associated with the devices registered as remote operation targets and receive operation inputs for the devices associated with the operation units, respectively; and

and a display control unit that changes the display mode of the operation unit on the display unit according to the position of the device.

2. The remote operation terminal according to claim 1,

further comprises a distance measuring unit for measuring the distance between the registered device and the apparatus,

the display control unit changes a mode of display of the operation unit on the display unit according to a distance between the device and the equipment.

3. The remote operation terminal according to claim 1,

the display control unit changes the mode of display of the operation unit on the display unit according to a distance between a predetermined specific position and the position of the device.

4. The remote operation terminal according to claim 1,

the display control unit changes a display mode of the operation unit on the display unit according to a distance between a predetermined specific position and a position of another remote operation terminal.

5. The remote operation terminal according to any one of claims 1 to 4,

further comprising a device classification unit for classifying the operation unit corresponding to the registered device into a predetermined device group,

the display control unit displays a collective operation screen that receives an operation input for performing a collective operation on 1 or more devices belonging to the device group.

6. The remote operation terminal according to any one of claims 1 to 5,

the display control unit changes a display mode of the operation unit or the collective display screen when a specific operation input is received.

7. The remote operation terminal according to any one of claims 1 to 6,

the above-mentioned operation section is generated by executing an application program,

the display control unit changes a display mode at the time of generation of the operation unit according to a distance between the device and the apparatus.

8. The remote operation terminal according to any one of claims 1 to 7,

the display control unit changes a display mode of an operation unit of a device registered in advance as an operation target of another remote operation terminal in the own device according to a position of the own device.

9. The remote operation terminal according to any one of claims 1 to 8,

the display control unit changes a display mode of an operation unit of a device registered in advance as an operation target of another remote operation terminal in the device according to a position of the other remote operation terminal.

10. A program for causing a computer to function as a remote operation terminal, wherein,

the remote operation terminal includes:

a plurality of operation units which are virtual operation units associated with the devices registered as remote operation targets and receive operation inputs for the devices associated with the operation units, respectively; and

and a display control unit that changes the display mode of the operation unit on the display unit according to the position of the device.

11. A remote control device capable of communicating with a remote operation terminal, wherein,

the remote control terminal includes a control unit for displaying a plurality of operation units on a display unit of the remote control terminal, wherein the operation units are virtual operation units that are associated with devices registered as remote operation targets and receive operation inputs for the devices associated with the operation units,

the control unit changes a display mode of the operation unit on the display unit according to a position of the remote operation terminal.

12. A program for use in a remote control device capable of communicating with a remote operation terminal, wherein,

the program is for causing a computer to function as a remote control device,

the remote control device includes a control unit that causes a plurality of operation units to be displayed on a display unit of the remote operation terminal, the operation units being virtual operation units that are associated with devices registered as remote operation targets and that receive operation inputs for the devices associated with the operation units,

the control unit changes a display mode of the operation unit on the display unit according to a position of the remote operation terminal.

13. A remote operation system is provided with:

a device registered as a remote operation object;

a remote operation terminal for receiving operation input of the device; and

a remote control device for controlling the operation of the equipment according to the operation input,

at least one of the remote operation terminal and the remote control device includes:

a plurality of operation units which are virtual operation units corresponding to the devices and receive operation inputs to the devices corresponding to the operation units, respectively; and

and a control unit that changes a display mode of the operation unit on a display unit according to a position of the remote operation terminal.

14. A remote operation terminal capable of remotely operating a predetermined device, wherein,

the disclosed device is provided with:

a display control unit that controls a display unit capable of displaying arbitrary information; and

a plurality of operation units capable of displaying on the display unit and accepting an operation input relating to the device registered as a remote operation target,

the display control unit automatically determines whether or not a predetermined condition relating to the device is satisfied without an operation by a user, and causes the display unit to display an operation unit of a registered device satisfying the predetermined condition so as to be easier to operate by the user than other operation units.

15. The remote operation terminal according to claim 14,

the display control unit determines a priority for each of the devices with respect to display of the plurality of operation units, and changes a mode of display of the plurality of operation units based on the determined priority.

16. The remote operation terminal according to claim 15,

the display control unit identifies the user's place based on the correspondence between a predetermined place and its position information and the position information of the own device, and determines the priority based on the identified user's place.

17. The remote operation terminal according to claim 16,

the display control unit estimates an action pattern of the user based on a history of use of the device, and determines the priority based on the estimated action pattern of the user.

18. The remote operation terminal according to claim 17,

the display control unit determines the priority based on whether or not the device is in communication connection with the apparatus.

19. The remote operation terminal according to any one of claims 15 to 18,

the display control unit causes the display unit to display the operation unit in a size, shape, or color according to the priority of each corresponding device.

20. The remote operation terminal according to any one of claims 15 to 19,

the display control unit causes the plurality of operation units to be displayed at display positions corresponding to the priorities of the corresponding devices.

21. The remote operation terminal according to any one of claims 15 to 20,

the plurality of operation units are displayed on a device selection screen for accepting an input of an operation for selecting a remote operation object from the device,

the display control unit adjusts the size of the display of the operation unit on the device selection screen so that the operation unit of a device having a priority higher than a predetermined priority can be visually confirmed by a user in the device selection screen at the same time.

22. The remote operation terminal according to any one of claims 15 to 21,

the display control unit classifies the devices into a plurality of groups according to the priority, and preferentially displays the operation units of the devices belonging to the group into which the device having the higher priority is classified.

23. The remote operation terminal according to any one of claims 15 to 22,

when the operation input by the user is not performed for a predetermined time, the display control unit causes the display unit to display the operation unit, which is highly likely to be used by the user after that time, based on the priority.

24. The remote operation terminal according to any one of claims 15 to 23,

when the application program displaying the operation unit is switched from a standby state to an execution state, the display control unit causes the operation unit, which is highly likely to be used by the user at that time or later, to be displayed on the display unit based on the priority.

25. A program for causing a computer to function as a remote operation terminal, wherein,

the remote operation terminal can remotely operate a predetermined device,

the remote operation terminal includes:

a display control unit that controls a display unit capable of displaying arbitrary information; and

a plurality of operation units capable of displaying on the display unit and accepting an operation input relating to the device registered as a remote operation target,

the display control unit automatically determines whether or not a predetermined condition relating to the device is satisfied without an operation by a user, and causes the display unit to display an operation unit of a registered device satisfying the predetermined condition so as to be easier to operate by the user than other operation units.

Images