WO2018087805A1

WO2018087805A1 - Remote control system, remote control method, and mobile information terminal

Info

Publication number: WO2018087805A1
Application number: PCT/JP2016/083060
Authority: WO
Inventors: 吉澤　和彦; 清水　宏; 橋本　康宣; 具徳野村; 奥　万寿男
Original assignee: マクセル株式会社
Priority date: 2016-11-08
Filing date: 2016-11-08
Publication date: 2018-05-17

Abstract

Provided is a technology that can realize responsiveness and ease of use or secure security and safety for remote operation in accordance with a device and an operation thereof for a remote control system. This remote control system comprises a mobile information terminal 2 of a user, at least one device 3, which is a device to be controlled, and a server 1, the foregoing being connected to a communication network 5. The server 1 supplies a remote control screen of the device 3 to the mobile information terminal 2. On the basis of an operation performed by a user on the remote control screen, the mobile information terminal 2 generates a first type command and transmits same to the server 1 in the case of an operation corresponding to instruction of a first operation of a first device, and generates a second type command and transmits same to a second device via one-to-one communication in the case of an operation corresponding to instruction of a second operation of a second device. The server 1 receives the first type command, and after checking, transmits same to the first device. The first device receives the first type command and executes the first operation. The second device receives the second type command and executes the second operation.

Description

Remote control system, remote control method, and portable information terminal

The present invention relates to an electronic device and an information processing service, and relates to a technique for remotely controlling the device.

At the user's home, etc., a television receiver (hereinafter sometimes referred to as “TV”), a set-top box, a content recorder (hereinafter sometimes referred to as “recorder”), a DVD player, an audio device, a game device, lighting Various devices such as devices and air conditioning devices are installed. These devices may be connected to a LAN to form a home network or the like. These devices and their operations can be operated from an operation input panel, a dedicated remote control device, or the like.

Regarding remote operation of a plurality of devices at a user's home or the like, since the operation and management of the plurality of remote control devices are complicated, a technique for integrating and managing them in one remote control device has also been proposed. For example, a technique has been proposed in which a user's portable information terminal has a function as an integrated remote control device, and the device is remotely operated from the portable information terminal.

JP-A-2014-45232 (Patent Document 1) can be cited as a prior art example related to remote control of equipment. Patent Document 1 describes the following as a remote control system and the like. The system includes a terminal device, a control device (for example, a server of an external network), and a controlled device (for example, a television). The terminal device receives and displays operation screen information (for example, a program guide) from the control device, and receives an operation input to the controlled device based on the information. The terminal device transmits operation information (for example, channel selection) to the control device based on the operation input. The control device converts the operation information into control information for controlling the controlled device, and transmits the control information to the controlled device. The controlled device has an API (Application Programming ため Interface) for controlling the controlled device based on the control information. The API is an interface for transferring control information from the outside to internal processing.

JP 2014-45232 A

As in the remote operation system of Patent Document 1, in the conventional remote control system, it is possible to remotely operate a controlled apparatus from a user's portable information terminal by remote control communication via a server or the like of an external network. The external network is the Internet or the like, and TCP / IP communication or the like is performed. Therefore, a large response delay that cannot be accepted by the user may occur during the remote control communication. When the response to the user input operation is slow, it is not convenient for the user.

Also, it is necessary to consider security and safety in remote control of equipment. It is necessary to prevent unauthorized remote operations. Examples of countermeasures include authentication such as device authentication and user authentication via a server. For example, when performing a remote operation such as power-on of a device, the user is concerned not only with safety but also with respect to power consumption and electricity charges. If remote control can be recorded and monitored and related services can be provided with remote control via a server, it is beneficial to the user.

Also, with regard to remote control of a plurality of user devices, the conventional technique that uniformly applies a communication method via a server of an external network may not be effective as a whole. Depending on a specific device and its operation, the user may want to give priority to the responsiveness and usability of remote operation, or may want to give priority to ensuring security and safety. A uniform communication method cannot satisfy both of these effects.

The object of the present invention relates to the technology of a remote control system that enables remote operation of a device from a user's portable information terminal, depending on the device and its operation, and the response and usability of remote operation, or security and safety. It is to provide a technology that can realize such as securing.

A representative embodiment of the present invention is a remote control system, a remote control method, and a portable information terminal, and is characterized by having the following configuration.

A remote control system according to an embodiment includes a user's portable information terminal connected to a communication network, a device that is one or more controlled devices of the user, and a server device. Providing a remote control screen for remote operation of the operation of the device to the portable information terminal, the portable information terminal being based on the input operation of the user on the remote control screen, among the devices In the case of selection of the first object corresponding to the instruction of the first operation of the first device, the first type command is generated, the first type command is transmitted to the server device, and the second operation of the second device is performed. In the case of selection of the second object corresponding to the instruction, the second type command is generated, the second type command is transmitted to the second device by one-to-one communication, and the server device receives the portable information Terminal The first type command is received, and after confirmation, the first type command is transmitted to the first device. The first device receives the first type command from the server device, and The first operation corresponding to the type 1 command is executed, and the second device receives the type 2 command from the portable information terminal in the one-to-one communication, and the type 2 command corresponds to the type 2 command. Two operations are executed.

According to a typical embodiment of the present invention, regarding the technology of a remote control system that enables remote operation of a device from a user's portable information terminal, depending on the device and its operation, Usability, security and safety can be ensured.

It is a figure which shows the structure of the remote control system of Embodiment 1 of this invention. 1 is a diagram illustrating a detailed configuration example of a remote control system according to a first embodiment. In Embodiment 1, it is explanatory drawing regarding the proper use of two types of API commands and communication methods of remote control. In Embodiment 1, it is a figure which shows the structural example of a server. 6 is a diagram illustrating a configuration example of a television as the device 3 in Embodiment 1. FIG. In Embodiment 1, it is a figure which shows the structural example of a recorder as the apparatus 3. FIG. In Embodiment 1, it is a figure which shows the structural example of the portable information terminal 2. FIG. FIG. 3 is a diagram showing a first communication sequence in the remote control system of the first embodiment. It is a figure which shows a 2nd communication sequence in the remote control system of Embodiment 1. FIG. FIG. 6 is a diagram illustrating an example of a menu screen in the first embodiment. In Embodiment 1, it is a figure which shows the example of all the apparatus list screens. In Embodiment 1, it is a figure which shows the example of a vicinity apparatus list screen. In Embodiment 1, it is a figure which shows the 1st example of the group of an apparatus list screen. In Embodiment 1, it is a figure which shows the 2nd example of the group of an apparatus list screen. FIG. 6 is a diagram illustrating an example of a remote control screen in the first embodiment. FIG. 10 is a diagram illustrating an example of a device update notification screen in the first embodiment. In Embodiment 1, it is a figure which shows the example of a setting of the user terminal information among registration information. In Embodiment 1, it is a figure which shows the example of a setting of the apparatus information among registration information. In Embodiment 1, it is a figure which shows the example of apparatus product information. In Embodiment 1, it is a figure which shows the example of log | history information. It is a figure which shows the structure of the remote control system of the modification of Embodiment 1. FIG. It is a figure which shows the structure of the remote control system of Embodiment 2 of this invention. In Embodiment 2, it is a figure which shows the example of the management information of related service "eco-service". In Embodiment 2, it is explanatory drawing regarding the related service "recording reservation assistance service". It is a figure which shows the structure of the remote control system of Embodiment 3 of this invention. In Embodiment 3, it is a figure which shows a communication sequence.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that components having the same function are denoted by the same reference symbols throughout the drawings for describing the embodiment, and the repetitive description thereof will be omitted.

(Embodiment 1)
A remote control system, a remote control method, and a portable information terminal according to the first embodiment of the present invention will be described with reference to FIGS. The remote control system according to the first embodiment is a system that performs remote control via a provider's server when the user's device is remotely operated from the user's portable information terminal. The remote control method of the first embodiment is a method having steps executed in the remote control system of the first embodiment. The portable information terminal according to the first embodiment is a portable information terminal constituting the remote control system according to the first embodiment.

[Remote control system (1)]
FIG. 1 shows the configuration of the remote control system of the first embodiment. The remote control system according to the first embodiment includes an operator server 1, a user portable information terminal 2, a user device 3 and a LAN 4, which are connected via a communication network 5. A device 3 is connected to the LAN 4. The communication network 5 is an external network for the user's LAN 4 and includes the Internet as a WAN, a mobile communication network, and the like. In the remote control system, the user's device 3 and the user's portable information terminal 2 are connected to the server 1 via the communication network 5. Further, the device 3 and the portable information terminal 2 are P2P connected.

The provider server 1 is configured on the communication network 5 as, for example, a cloud server using cloud computing. The server 1 is a remote control management device having a function of performing remote control and information management thereof. The server 1 provides the remote control service 10 to the user's portable information terminal 2 and the device 3. There are a plurality of users and portable information terminals 2 that use the remote control service 10. The server 1 may be composed of a plurality of servers, and may have a DB server or the like.

The remote control service 10 is realized by the hardware and software of the server 1. The remote control service 10 performs processing related to remote control related to remote operation of the device 3 from the user's portable information terminal 2. The remote control service 10 performs processing by communicating with the remote control middleware 30 of the device 3. In addition, the remote control service 10 performs processing by communicating with the remote control application 20 of the portable information terminal 2. The remote control service 10 provides the remote control application 20 with remote control screen data 10G and the like. The remote control screen data 10G is data including an API command. The manager of the business operator manages the server 1 and sets the remote control screen data 10G, registration information 50, and the like. The API command is a command corresponding to the API of the device 3.

The server 1 holds registration information 50 related to control of the remote control service 10 and information management. The registration information 50 includes information on the user, the portable information terminal 2, the device 3, the LAN 4, and the like. The information on the device 3 includes API command information. When the user makes the device 3 a candidate for remote control, the user can register information regarding the device 3 in the remote control service 10. When using the portable information terminal 2 as a remote control operation terminal, the user can register the information of the portable information terminal 2. Each information is registered in the registration information 50 based on the input operation of the administrator or the user. The remote control service 10 performs processing while referring to the registration information 50. The manufacturer (manufacturer) of the device 3 discloses information including the API of the device 3. The API is interface information that enables the operation of the device 3 to be controlled from the outside. The administrator registers information including the API of the device 3 in the registration information 50.

The LAN 4 is a LAN constituting a home network at the user's home, for example. The user may be a single user or a plurality of users such as family members or employees for the LAN 4. The user possesses and uses at least one portable information terminal 2. One user may possess and use a plurality of portable information terminals 2.

The device 3 is a controlled device to be remotely controlled. As the device 3, various electronic devices such as a television can be applied. The device 3 is installed, for example, in the user's home. The device 3 includes a remote control middleware 30 in addition to the function of the device body. The remote control middleware 30 is realized by the hardware and software of the device 3. The remote control middleware 30 performs processing on the device 3 side related to remote control. The remote control middleware 30 performs processing by communicating with the remote control service 10 of the server 1. The remote control middleware 30 performs processing by communicating with the remote control application 20 of the portable information terminal 2.

The portable information terminal 2 is a remote control operation terminal for remotely operating the device 3. The portable information terminal 2 can be applied to various electronic devices such as general smartphones and tablets. The position of the portable information terminal 2 varies with the movement of the user carrying the portable information terminal 2. The user and the portable information terminal 2 exist at a position near the user's LAN 4 or at a position outside the LAN 4.

The mobile information terminal 2 includes a remote control application 20 in addition to the function of the terminal body. The remote control application 20 is realized by hardware and software of the portable information terminal 2. The remote control application 20 can be downloaded from the operator's server 1. The remote control application 20 is a client application for the remote control service 10 and performs processing by communicating with the remote control service 10 by a client server method. The remote control application 20 displays a remote control screen 21 on the screen of the touch panel of the portable information terminal 2 based on the remote control screen data 10G obtained from the remote control service 10.

The remote control screen 21 is a screen having a graphical user interface (GUI) for remote operation and remote control of the device 3. The remote control screen 21 receives a touch input operation by the user. On the remote control screen 21, a GUI object for remote operation of the operation of the device 3 is arranged. In the remote control screen data 10G, information such as an API command is described in association with the object. The user performs a touch input operation for instructing the operation of the device 3 on the object on the remote control screen 21. The remote control application 20 generates a first type API command or a second type API command as an API command associated with the selected object.

The API command has a first type API command and a second type API command as types. The remote control service 10 manages these two types of API commands and uses them according to the device 3 and its operation. The proper use can be set in the registration information 50 of the server 1 in advance. The API command in the portable information terminal 2 may be obtained from the server 1 as information attached to the remote control application 20 in advance, or may be obtained later as information attached to the remote control screen data 10G.

The remote control application 20 transmits the generated API command to the remote control service 10 of the server 1 by the first communication method when the generated API command is the first type API command. When the generated API command is the second type API command, the remote control application 20 transmits the command to the remote control middleware 30 of the device 3 by the second communication method.

TCP / IP communication or the like is possible between the devices of the portable information terminal 2, the server 1, and the device 3 via the communication network 5. The portable information terminal 2 and the device 3 can access the remote control service 10 of the server 1 based on TCP / IP communication. In the first communication method, TCP / IP communication via the server 1 is used as non-P2P communication. In addition, between the portable information terminal 2 and the device 3, P2P communication is possible without going through the server 1 when each has a P2P communication function. In the second communication method, P2P communication that does not pass through the server 1 is used.

[Remote control system (2)]
FIG. 2 shows a detailed configuration example of the remote control system according to the first embodiment, a user usage example, and the like. In the remote control system of FIG. 2, the server 1, the portable information terminal 2, and the device 4 of the LAN 4 are connected via the Internet 6 and the mobile communication network 7. The server 1 is connected to the Internet 6 including an IP network. A mobile communication network 7 is connected to the Internet 6. TCP / IP communication or the like is performed on the Internet 6 and the mobile communication network 7. The mobile communication network 7 includes a mobile base station 71 and performs wireless communication with the portable information terminal 2. The mobile information terminal 2 can perform mobile communication on the mobile communication network 7 through a wireless connection with the mobile base station 71 and can connect to the Internet 6 through mobile communication. Between the mobile base station 71 and the Internet 6, there may be a network operation device of a network operator of the mobile communication network 7.

FIG. 2 shows one user U1 as an example of the user. Further, the LAN 4 associated with the user includes a first LAN 41 and a second LAN 42 to which the user U1 has authority. The first LAN 41 is a LAN at the home of the user U1. The second LAN 42 is the LAN of the user U1 office. The first LAN 41 includes a communication device such as a gateway 81, and the second LAN 42 includes a communication device such as a gateway 82. The

gateways

81 and 82 are connected to the Internet 6 and are communication devices such as routers and wireless LAN access points. Between the

gateways

81 and 82 and the Internet 6, there may be a proxy server, a firewall server, a network operator's network operation device, or the like.

FIG. 2 shows one portable information terminal 2 possessed by the user U1, and its terminal ID is “M1”. The portable information terminal 2 is a smartphone, for example. The state of the portable information terminal 2 when the user U1 is in the vicinity of the first LAN 41 at home is the portable information terminal 2A, and the position is the position L1. The state of the portable information terminal 2 when the user U1 is outside the first LAN 41 at home is the portable information terminal 2B, and the position is the position L2. The position can be generally expressed by GPS position coordinates or the like. The user U1 can also move in the vicinity of the second LAN 42 in the office, and the position in this case is set as a position L3.

In the first LAN 41, a television 31a, a recorder 32a, a lighting device 33a, an air conditioning device 34a, and the like are connected to the gateway 81 as a plurality of devices 3. There may be a plurality of various devices 3. In the second LAN 42, a television 31b, a recorder 32b, a lighting device 33b, an air conditioning device 34b, and the like are connected to the gateway 82 as the plurality of devices 3. The device 3 of the first LAN 41 and the device 3 of the second LAN 42 may be different. The connection method between each device 3 and the gateway 81 may be different, or a plurality of gateways 81 may exist.

The portable information terminal 2A can be connected to the first LAN 41, and can be connected to the gateway 81, for example, by a wireless LAN. Similarly, the portable information terminal 2 B can be connected to the second LAN 42, for example, can be connected to the gateway 82 by a wireless LAN. Not only wireless LAN connection but wired LAN connection may be used.

The television 31a is a television receiver and has a function of receiving broadcast waves and reproducing content such as broadcast programs. The recorder 32a is a content recorder, for example, a recorder using a recording medium such as a DVD or a Blu-ray disc, and has functions for recording and reproducing content. The recorder 32a is connected to the television 31a, for example. The lighting device 33a is installed in each room or the like, and can be turned on / off, adjusted in brightness, and the like. The air conditioner 34a is installed in each room or the like, and can be turned on / off, adjusted in temperature, and the like.

The device 3 of the first LAN 41 can be connected to the Internet 6 via the gateway 81. The device 3 of the second LAN 42 can be connected to the Internet 6 via the gateway 82. The portable information terminal 2 A can be connected to the Internet 6 via the gateway 81.

Each device 3 of the portable information terminal 2 and the first LAN 41 has a P2P communication function. The portable information terminal 2A is capable of P2P communication based on the P2P connection with each device 3 of the first LAN 41. The LAN 4 may include a device 3 having a P2P communication function and a device 3 having no P2P communication function depending on the device 3. In the first embodiment, Bluetooth (registered trademark) is used as a standard of P2P communication and a communication interface.

The device 3 registered in the remote control service 10 is not limited to the device 3 etc. connected to the same LAN 4, and physical and logical restrictions are small. For example, in FIG. 2, not only the device 3 of the first LAN 41 can be registered as a target of remote operation from the user's portable information terminal 2, but also a device 3 of a different second LAN 42 can be registered as a target.

[Remote control]
The outline of the remote control in the remote control system of the first embodiment is as follows. The remote control service 10 in FIG. 1 has, as functions, management of remote control of a group of devices 3 for each user, provision of a remote control screen for each device 3, remote control using two types of methods, and the like. The remote control service 10 enables a remote operation in an integrated manner from the portable information terminal 2 regarding the remote operation of the plurality of devices 3 on the user's LAN 4. The remote control service 10 performs remote control and information management for that purpose. The remote control service 10 provides the mobile information terminal 2 with a remote control screen for each device 3 that is a target of remote operation.

Remote control using both types is as follows. The remote control service 10 uses the first communication method and the second communication method in combination, and performs different control depending on the device 3 and the operation. The first communication method is a method of performing non-P2P communication via the server 1, for example, TCP / IP communication, using the first type API command. The second communication method is a method of performing P2P communication directly without using the server 1 by using the second type API command. When the device 3 to be remotely operated or its operation is the specific first device or the first operation, the first communication method is used, and when the device is the specific second device or the second operation. The second communication method is used.

Suppose that the mobile information terminal 2A of the user U1 is at a position L1 near the first LAN 41 at home in FIG. On the remote control screen 21 of the target device 3 in the portable information terminal 2A, an object associated with a desired operation instruction is selected, and an API command associated with the object is generated.

In the case of the first communication method, it is as follows. A first type API command is generated in response to an input operation for the first operation of the first device (for example, the television 31a). In the case of the first type API command, the remote control application 20 of the portable information terminal 2 sends the first type API command C11 to the server by TCP / IP communication using the IP communication unit 206 of FIG. 1 to the remote control service 10. Specifically, an IP packet including the first type API command is transmitted to the server 1 via the gateway 81 and the Internet 6. The remote control service 10 of the server 1 receives the first type API command C11 through the IP communication unit 106 in FIG. The server 1 performs processing such as confirmation and history recording on the first type API command C11, and thereafter, the first type API command C12 having the same content is transmitted by TCP / IP communication using the IP communication unit 106. Transmit to the target device 3. Specifically, an IP packet including the first type API command C12 destined for the IP address of the target device 3 is transmitted to the remote control middleware 30 of the target device 3.

The remote control middleware 30 of the device 3 (for example, the television 31a) receives and acquires the first type API command C12 from the server 1 through the IP communication unit 316 in FIG. 5 and passes it to the CPU 311. The device 3 executes an operation according to the first type API command, and transmits response information to the command to the server 1 through the IP communication unit 316 by TCP / IP communication. The remote control service 10 of the server 1 receives response information from the device 3 through the IP communication unit 106 and passes it to the CPU 101. The remote control service 10 performs processing such as confirmation and history recording on the response information, and then transmits the response information having the same content to the remote control application 20 of the portable information terminal 2 by TCP / IP communication through the IP communication unit 106. Send to. The remote control application 20 of the portable information terminal 2 receives the response information from the server 1 through the IP communication unit 206.

The first type API command C11 corresponds to operation input information corresponding to the portable information terminal 2 side API. The first type API command C12 corresponds to operation control information corresponding to the device 3 side API.

In the case of the second communication method, it is as follows. A second type API command is generated in response to an input operation for the second operation of the second device (eg, the recorder 32a). In the case of the second type API command, a P2P connection is established between the portable information terminal 2 and the target device 3 based on the control of the remote control service 10 of the server 1. In this state, the remote control application 20 of the portable information terminal 2 transmits the second type API command C2 to the remote control middleware 30 of the target device 3 by P2P communication using the P2P communication unit 208 of FIG. Specifically, a Bluetooth packet in which a second type API command or the like is described is transmitted. The remote control middleware 30 of the device 3 receives the second type API command C2 through the P2P communication unit 418 in FIG. 6 and passes it to the CPU 411. The device 3 performs an operation according to the command, and transmits response information to the command to the portable information terminal 2 by P2P communication through the P2P communication unit 418. The portable information terminal 2 receives the response information through the P2P communication unit 208.

Note that the device 3 is not limited to response information to the API command, and may transmit control information, status information, and the like to the mobile information terminal 2 or the server 1.

As described above, the remote control system uses two types of communication depending on the target device 3 and the operation. In the first communication method, the first type API command is transferred via the server 1. The remote control service 10 confirms the first type API command and records history information. In addition, the remote control service 10 can perform control to block or restrict the first type API command to the device 3 as necessary. Thereby, security and safety can be secured. In the first communication method, the remote control service 10 can be applied based on the grasp of the first type API command, and can provide related services described later. In the second communication method, the second type API command is directly transmitted from the portable information terminal 2 to the device 3 without going through the server 1. Therefore, since there is no delay in the Internet 6 or the like, good responsiveness such as a high response speed can be obtained. You can use them according to the effect you want to prioritize.

[Separate use of two types of API commands and communication methods]
The two types of API commands and communication methods corresponding to the device 3 in remote control will be described below. Basically, the server 1 can arbitrarily set the two types of usage based on the operation of the administrator or the user. In the device information 52 and the like of the registration information 50 to be described later, the setting information for proper use is included. In the registration information 50, for each of the user's portable information terminal 2, the LAN 4 device 3, and its operation, it is set whether to use the first or second type API command communication method. Basically, it is set by the administrator, but may be variable depending on user settings. In that case, a setting screen is provided to the portable information terminal 2 to enable user settings regarding the proper use of the two types of methods.

The basic idea is as follows. In the operation of the device 3, when priority is given to ensuring security and safety, application control, use of related services, and the like, the first communication method using the first type API command is used. When priority is given to responsiveness and usability regarding the operation of the device 3, the second communication method using the second type API command is used. In order to use the second communication method, it is necessary that the device 3 and the portable information terminal 2 have a P2P communication function.

∙ Regarding remote control, you can enjoy the advantages of each method by using the two methods according to the device 3 and the operation. Compared to the prior art that uniformly applies the same method, an overall efficient system can be realized.

The server 1 controls the remote control communication based on the setting information regarding the above-mentioned usage. Based on the setting information, at least one of two types of API commands is described in the remote control screen data 10G in association with the GUI object. Since the communication method and the like are automatically controlled by the server 1, the user does not need to be aware of the communication method used. It should be noted that information such as a communication method being used by remote control may be displayed on the screen of the portable information terminal 2. For example, information such as “P2P communication in progress” and “Server communication in progress” may be displayed. In this case, the user can recognize the communication method at the time of remote operation.

The main patterns of usage are as follows. A description will be given of the positional relationship of the user's portable information terminal 2 with respect to the device 4 of the LAN 4 and the presence or absence of the P2P communication function of the device 3.

(1) When the user's portable information terminal 2 is located in the vicinity of the device 4 of the LAN 4 (first LAN 41 or second LAN 42), if the portable information terminal 2 and the device 3 have the P2P communication function, the first Either the communication method or the second communication method can be used. In particular, the first communication method is used for operations that prioritize safety and the like, and the second communication method is used for operations that prioritize responsiveness and the like.

(2) When the user's portable information terminal 2 does not have the P2P communication function and the device 3 has the P2P communication function, the first communication method is used. The first communication method can be used even when the user's portable information terminal 2 is located some distance away from the LAN 4 device 3.

(3) When the user's portable information terminal 2 has the P2P communication function and the device 3 does not have the P2P communication function, the first communication method is used. The first communication method can be used even when the user's portable information terminal 2 is located some distance away from the LAN 4 device 3.

(4) When the user's portable information terminal 2 is located at a certain distance from the device 4 of the LAN 4 and the portable information terminal 2 and the device 3 have a P2P communication function, the following is performed. Although depending on the details of the P2P communication function, when the portable information terminal 2 and the device 3 are separated by a predetermined distance or more, the P2P communication becomes impossible. In this state, the first communication method is used. For example, when the home device 3 is remotely operated from the portable information terminal 2B outside the first LAN 41 at home, the first communication method is used.

FIG. 3 shows an explanatory diagram regarding the proper use of two types of methods in remote control. FIG. 3 shows three examples of proper use according to the relationship between the user's portable information terminal 2 and the device 3. In the example on the left side, the mobile information terminal 2 (terminal ID = M1) of the user U1 and the target device 3 (device ID = X1) are shown. The portable information terminal 2 has a P2P function, and the device 3 does not have a P2P function. Therefore, only the first communication method can be used. The operation of the device 3 is roughly classified into two types, a first operation group and a second operation group. For example, it may be set to use the first communication method for all operations of the device 3. Or the setting which uses a 1st communication system regarding a specific 1st operation group may be sufficient.

In the example on the right side, the mobile information terminal 2 (terminal ID = M2) of the user U2 and the target device 3 (device ID = X2) are shown. The portable information terminal 2 does not have a P2P function, and the device 3 has a P2P function. Therefore, only the first communication method can be used. Similarly to the example on the left side, for example, the setting using the first communication method may be used for all operations of the device 3, or the setting using the first communication method may be used for a specific first operation group.

In the center example, the mobile information terminal 2 (terminal ID = M1) of the user U1 and the target device 3 (device ID = X2) are shown. The portable information terminal 2 and the device 3 have a P2P function. Therefore, both the first communication method and the second communication method can be selectively used. In this case, as shown in the figure, first, as shown in the drawing, both systems may be set to be used together, such as the first communication system for the first operation group and the second communication system for the second operation group. Or it is good also as a setting which uses a 1st communication system regarding all the operation | movement or a 1st operation group as a setting which uses only a 1st communication system. Or it is good also as a setting which uses a 2nd communication system regarding all the operation | movement or a 2nd operation group as a setting which uses only a 2nd communication system.

Examples of proper use according to the operation of the device 3 are as follows. The recorder 32a in FIG. 2 includes operations that can be remotely operated, such as power-on, setting, recording reservation, playback, and pause. As a setting example, the first type API command is used for the first operation group such as power-on, setting, and recording reservation, and the second type API command is used for the second operation group such as reproduction and pause.

[P2P communication, Bluetooth]
Supplementary explanation will be given for the base technologies such as P2P communication and Bluetooth. P2P (Peer to Peer) is one of the communication methods, and is a method in which one-to-one communication is directly performed between devices that are each established peer, unlike the client-server method. In this specification, P2P communication is also referred to as one-to-one communication. In the first embodiment, P2P communication is performed in pairs in which the portable information terminal 2 and the device 3 are peer devices. In P2P communication, compared with non-P2P communication such as TCP / IP communication via an external communication network and a server, a response delay is small, so that a good response can be expected.

In Embodiment 1, P2P communication by Bluetooth is used as the second communication method. When the portable information terminal 2 is in the vicinity of the LAN 4, P2P communication by Bluetooth is possible between the portable information terminal 2 and the device 3. As a modification, P2P communication may be realized using a standard other than Bluetooth. As will be described later, P2P communication is not limited to Bluetooth or the like, and can also be realized as P2P communication on an IP network. In the first embodiment, such P2P communication on the IP network is not used.

* Bluetooth is as follows. Bluetooth (IEEE802.15.1) is one of short-range wireless communication methods, and is a method of performing wireless communication using radio waves between compatible devices arranged at a short distance. When establishing a P2P connection in Bluetooth, a Bluetooth pairing process is performed. When there is a corresponding device that is a candidate for a connection partner within a predetermined short-distance range, list information of candidate devices is obtained for each device. The same PIN (Personal Identity Identity Number) corresponding to the authentication key is input to the two devices. The PIN may be a fixed value. After confirming the PIN between the two devices, Bluetooth pairing is completed. Once the pairing is performed, a connection is automatically established by omitting some procedures when performing communication again.

[server]
FIG. 4 shows a configuration example of the server 1. The server 1 includes a CPU 101, a ROM 102, a RAM 103, an internal storage 104, an external storage interface unit 105, an IP communication unit 106, an input / output interface unit 109, and the like, which are connected to each other via an internal bus. An external storage 110 is connected to the external storage interface unit 105. The internal storage 104 is configured by a nonvolatile memory or the like. The internal storage 104 holds data and information such as a remote control service program 10P and remote control screen data 10G.

The external storage 110 stores a DB (database) used by the remote control service 10 or the like. The CPU 101 accesses the DB of the external storage 110 via the external storage interface unit 105. The DB stores data and information such as registration information 50, device product information 55, and device list information 56, and is read out to the internal storage 104 as necessary. The registration information 50 includes user terminal information 51, device information 52, and the like.

The server 1 realizes the remote control service 10 by executing processing according to the remote control service program 10P based on the CPU 101, the ROM 102, the RAM 103, and the like. Specifically, the CPU 101 expands the remote control service program 10P of the internal storage 104 in the RAM 102 and executes processing according to the program. Thereby, the entity of the remote control service 10 is configured in accordance with the access from each portable information terminal 2.

Various services provided by the server 1 are given unique URIs (Universal Resource Identifiers) including the remote control service 10. The user's portable information terminal 2 transmits a request including the URI of the service to be used to the server 1. The IP communication unit 106 of the server 1 receives the URI and transmits it to the CPU 101. The CPU 101 determines the URI and starts providing the corresponding service.

The remote control service 10 performs client-server communication with the remote control application 20 based on TCP / IP communication on the Internet 6, and sends remote control screen data corresponding to the remote control target device 3 to the remote control application 20. 10G etc. are provided.

The IP communication unit 106 includes an interface device that performs TCP / IP communication with the Internet 6 that is an IP network. The IP communication unit 106 receives the first type API command C11 from the above-described portable information terminal 2, and transmits the first type API command C12 and the like to the device 3. The input / output interface unit 109 is connected to an input device and an output device, and accepts an input operation from an administrator. A configuration example of the registration information 50 will be described later.

[Equipment-TV]
FIG. 5 shows a configuration example of a television 31 a as an example of the device 3. The television 31b has the same configuration. The television 31a includes a CPU 311, a ROM 312, a RAM 313, a reception channel selection unit 301, a signal separation unit 302, a video processing unit 303, a graphics unit 304, a display unit 305, an audio processing unit 306, an audio output unit 307, a digital input unit 308, and IP communication. Unit 316, P2P communication unit 318, operation input unit 319, internal storage 310, and the like, which are connected to each other via an internal bus. The internal storage 310 stores a remote control middleware program 30P, device setting information 351, and the like.

The television 31a implements the remote control middleware 30 based on the CPU 311, the ROM 312, the RAM 313, and the like. Specifically, the CPU 311 expands the remote control middleware program 30P of the internal storage 310 in the RAM 313 and executes processing according to the program. Thereby, the entity of the remote control middleware 30 is configured.

The IP communication unit 316 is a part that includes an interface device that is connected to the gateway 81 and performs TCP / IP communication or the like via the Internet 6. The IP communication unit 316 receives the first type API command C12 and the like from the remote control service 10 of the server 1 based on TCP / IP communication.

The P2P communication unit 318 is a part including an interface device that is P2P connected to the portable information terminal 2 and performs P2P communication. In Embodiment 1, the P2P communication unit 318 is a Bluetooth communication unit. The P2P communication unit 318 establishes a P2P connection with the portable information terminal 2 and receives the second type API command C2 and the like from the portable information terminal 2 based on the P2P communication.

The CPU 311 of the device 3 controls each unit such as the reception channel selection unit 301 based on the first type API command or the second type API command as an API command, and performs an operation according to the API command. The CPU 311 controls the video data and audio data obtained from the broadcast wave or the video data and audio data obtained from the digital data signal to be output to the display unit 305 and the audio output unit 307.

The reception channel selection unit 301 receives a broadcast wave, selects a specific channel designated by the CPU 311, and sends a digital content stream to the signal separation unit 302. The digital content stream is MPEG TS (Transport Stream) data in which image data, audio data, PSI (Program Specific Information) data, and the like are multiplexed. The signal separation unit 302 separates video ES (Elementary Stream) data and audio ES data from the TS data, and sends the video ES data to the video processing unit 303 and the audio ES data to the audio processing unit 306. The video ES data and audio ES data are encoded to compress the information amount of the original data. The video processing unit 303 performs decompression decoding processing from the video ES data, acquires video data, and sends the video data to the graphics unit 304. The sound processing unit 306 performs decompression decoding processing from the sound ES data, acquires sound data, and sends the sound data to the sound output unit 307.

The graphics unit 304 manages video to be displayed in units of layers based on the video data. For example, an OSD (On Screen Display) layer is superimposed on the video data and sent to the display unit 305. The display unit 305 is a part including, for example, a display panel and a display drive circuit thereof, and displays a video on the screen of the display panel based on the video data. The audio output unit 307 includes a speaker or the like, and outputs audio based on audio data. The digital input unit 308 includes, for example, an HDMI terminal as an input terminal, and inputs a digital data signal in which video data and audio data are multiplexed. The digital input unit 308 separates the video data and audio data from the digital data signal, and sends the video data to the graphics unit 304 and the audio data to the audio output unit 307. The operation input unit 319 includes an operation input panel, a remote control receiving unit, and the like, and accepts user input operations.

The device setting information 351 includes information such as the device ID and installation position of the device 3, information such as permission regarding the user who uses the device 3 and the portable information terminal 2, user setting information of the device 3, and the like. Yes. For example, when a user installs a certain device 3 on the LAN 4 and uses it for the first time, the user registers device setting information of the device 3 in the device 3. Registration of the device setting information can be performed on a screen such as a user setting by the function of the device 3. Alternatively, the device setting information may be registered on the setting screen of the portable information terminal 2 through the function of the remote control service 10 of the server 1 that cooperates with the device 3. In that case, the server 1 holds device setting information for each user and each device 3 in, for example, the DB of the external storage 110. The information such as permission is information of the user and portable information terminal 2 that are permitted to use the device 3, and may be set with permission such as permission / non-permission for each operation and content. .

[Equipment-Recorder]
FIG. 6 shows a configuration example of a recorder 32 a as an example of the device 3. The recorder 32b has the same configuration. The recorder 32a includes a CPU 411, ROM 412, RAM 413, reception channel selection unit 401, signal separation unit 402, video processing unit 403, graphics unit 404, audio processing unit 406, digital output unit 408, IP communication unit 416, P2P communication unit 418, operation. An input unit 419, an internal storage 410, a content storage unit 421, a recording medium unit 422 and the like are connected to each other via an internal bus. The internal storage 410 stores a remote control middleware program 30P, device setting information 351, and the like. The functions of each unit from the CPU 411 to the operation input unit 419 of the recorder 32a, the internal storage 410, and the reception channel selection unit 401 to the audio processing unit 406 are the same as the functions of the corresponding units of the television 31a, and the description thereof is omitted.

Similar to the IP communication unit 316, the IP communication unit 416 is a part including an interface device connected to the gateway 81 and performing TCP / IP communication or the like via the Internet 6. Similarly to the P2P communication unit 318, the P2P communication unit 418 is a part including an interface device that is P2P connected to the portable information terminal 2 and performs P2P communication, and is a Bluetooth communication unit.

The content storage 421 includes, for example, a storage medium such as an HDD or a flash memory and a drive circuit thereof, and can store content data. The CPU 411 controls storage of content data in the content storage 421 and reading of content data from the content storage 421. The recording medium unit 422 is a part including a recording medium such as a DVD or a Blu-ray disc, a drive circuit thereof, and the like, and can record data on the recording medium and read data from the recording medium. The CPU 411 controls recording on the recording medium.

The content storage 421 receives the content data signal from the reception channel selection unit 401 and stores it in the storage medium. The recording medium unit 422 receives the content data signal from the reception channel selection unit 401 and stores it in the recording medium. When storing content data, the TS data may be left as is, or the TS data may be re-encoded to reduce the amount of data, or may be converted into a format conforming to a standard such as a DVD. Good.

The content data stored in the content storage 421 or the recording medium unit 422 can be read and reproduced. The reproduction data read from the content storage 421 or the recording medium unit 422 is input to the signal separation unit 402 and separated into video data and audio data. The video data is input to the digital output unit 408 through the video processing unit 403 and the graphics unit 404. The audio data is input to the digital output unit 408 via the audio processing unit 406. The digital output unit 408 includes an output terminal corresponding to, for example, HDMI, multiplexes the video signal and the audio signal, and outputs the multiplexed signal as a digital data signal from the output terminal.

[Portable information terminal]
FIG. 7 shows a configuration example of the portable information terminal 2. The portable information terminal 2 includes a CPU 201, a ROM 202, a RAM 203, an internal storage 204, an IP communication unit 206, a mobile communication unit 207, a P2P communication unit 208, an operation input unit 209, a sensor unit 211, a camera unit 212, an audio device 213, and a graphics unit. 214, a display device 215, and the like, which are connected to each other via an internal bus.

The internal storage 204 is configured with a nonvolatile memory or the like. The internal storage 204 holds data and information such as a remote control application program 20P and remote control screen data 10G. The remote control application 20 stores the remote control screen data 10G received from the remote control service 10 of the server 1 in the internal storage 204.

The portable information terminal 2 implements the remote control application 20 by executing processing according to the remote control application program 20P based on the CPU 201, the ROM 202, the RAM 203, and the like. Specifically, the CPU 201 expands the remote control application program 20P in the internal storage 204 in the RAM 202 and executes processing according to the program. Thereby, the entity of the remote control application 20 is configured.

The IP communication unit 206 is a part including an interface device that performs TCP / IP communication with the Internet 6 that is an IP network. The IP communication unit 206 includes an interface device corresponding to the LAN 4, for example, a wireless LAN. The IP communication unit 206 transmits the first type API command C11 and the like to the remote control service 10 of the server 1 via the gateway 81 and the Internet 6.

The P2P communication unit 208 is a part including an interface device that is P2P connected to the device 3 and performs P2P communication, and is a Bluetooth communication unit. When performing P2P communication, the P2P communication unit 208 establishes a P2P connection with the device 3, and transmits a second type API command C 2 based on an input operation on the remote control screen 21 to the device 3.

The mobile communication unit 207 includes an interface device that performs voice communication, data communication, and the like in the mobile communication network 7. The mobile communication unit 207 transmits the first type API command C11 and the like to the server 1 based on the wireless connection with the mobile base station 71.

A portable information terminal 2 such as a smartphone generally includes both an IP communication unit 206 and a mobile communication unit 207. The portable information terminal 2 can perform IP communication using the IP communication unit 206 in a situation where IP communication is possible. The portable information terminal 2 can perform data communication using the mobile communication unit 207 when IP communication using the IP communication unit 206 is impossible.

The remote control application 20 generates a first type API command or a second type API command as an API command associated with the object in accordance with the operation of the object on the remote control screen 21. In the case of the first type API command, TCP / IP communication with the server 1 is performed using the IP communication unit 206. In the case of the second type API command, P2P communication with the device 3 is performed using the P2P communication unit 208.

The operation input unit 209 is one of input means and includes hardware buttons and the like. The sensor unit 211 includes an acceleration sensor, a gyro sensor, a geomagnetic sensor, a GPS receiver, a proximity sensor, an illuminance sensor, and the like, detects the state of the portable information terminal 2, such as the orientation, angular velocity, and position coordinates, and outputs it to the CPU 201 and the like. . The CPU 201 controls the portable information terminal 2 using the detection information. The camera unit 212 includes a camera and can capture images of the outside world (moving images and still images) based on user input operations. Video data shot by the camera unit 212 is stored in the internal storage 204 or the like. The portable information terminal 2 may transfer the video data or the like as content data to the device 3 of the LAN 4 and store it, or may transfer and store it to a server (or the server 1) on the Internet 6. Good. The content data may be shared with other users. The audio device 213 includes a microphone, a speaker, and the like, and can perform audio recording and audio output.

The graphics unit 214 generates graphics data to be displayed on the screen of the display device 215 based on an instruction from the OS, an application, and the like based on the control of the CPU 201 and the input video data, and supplies the graphics data to the display device 215. The display device 215 includes, for example, a display panel with a built-in touch sensor 216 and its drive circuit. The display device 215 displays video on the screen based on the graphics data from the graphics unit 214. The display device 215 is one of input means, and detects a user's touch input operation (for example, tap, swipe, etc.) on the screen through the touch sensor 216. The touch sensor 216 is, for example, an electrostatic capacitance type, detects a change in capacitance according to the proximity or contact of a finger or the like as an electrical signal, and outputs it as a touch detection signal to the CPU 201 or the like. On the screen of the display device 215, a menu screen, a remote control screen 21, and the like are displayed by the remote control application 20.

The remote control screen 21 is generated as a part of the remote control application 20. The remote control application 20 generates the remote control screen 21 through the graphics unit 214 or the like based on the remote control screen data 10G obtained from the remote control service 10.

[Sequence (1)]
FIG. 8 shows a first sequence between apparatuses in the remote control system of the first embodiment. Assuming the configuration of FIG. 2 and the like, communication between the remote control service 10 of the server 1, the remote control application 20 of the portable information terminal 2, and the remote control middleware 30 of the device 3 is shown. FIG. 8 has steps S1 to S13. Hereinafter, it demonstrates in order of a step.

8 includes, in particular, an automatic device authentication and device registration process 800 performed between the remote control middleware 30 of the device 3 of the first LAN 41 and the remote control service 10 of the server 1. In the process 800, the power source of the device 3 is turned on in S1. For example, the user installs a new device 3 to connect to the gateway 81 of the first LAN 41 and turns on the power. In S 2, the device 3 connects to the first LAN 41 through connection with the gateway 81. The device 3 transmits a device authentication request to the remote control service 10 of the server 1 for device authentication. Information such as a device authentication request is transmitted toward the URI of the remote control service 10. In S3, when the remote control service 10 of the server 1 receives the device authentication request in S2, it performs device authentication with the device 3. In the first embodiment, publicly known HTTP digest authentication is applied as the device authentication, but the present invention is not limited to this and can be applied. The device authentication confirms that the device 3 is valid from the perspective of the server 1 and that the server 1 is valid from the perspective of the device 3. This device authentication is performed as automatic device authentication in consideration of security and the like related to the remote control service 10.

In S4, the remote control service 10 of the server 1 transmits a device authentication response indicating success or failure as a result of device authentication to the device 3. The remote control service 10 registers the information in the registration information 50 with the device 3 that has been successfully authenticated as a target and candidate for remote control. By this device registration, information on the device 3 is automatically registered in the remote control service 10. Therefore, there is little time and effort for manual registration of device information by the user.

The remote control service 10 of the server 1 grasps information such as the IP address of the device 3 through the gateway 81 in the device authentication process 800 with the device 3 and registers it in the registration information 50. The local IP address of the first LAN 41 is assigned to the device 3 by a DHCP server provided in the gateway 81. The DHCP server of the gateway 81 holds information on the correspondence between the global IP address on the Internet 6 and the local IP address of the device 3 on the first LAN 41.

The remote control service 10 may periodically and repeatedly execute the process 800 including the device authentication. For example, the remote control service 10 performs device authentication again in order to confirm the state of the device 3 after a predetermined time elapses after the first device authentication. In addition, when the DHCP server of the gateway 81 reassigns the IP address of the device 3, it is necessary to perform device authentication again to grasp the information of the device 3. For example, when the manufacturer updates the firmware of the device 3 and the ID or password of the device 3 is changed, it is necessary to perform device authentication again to grasp the information of the device 3.

The process 800 including the device authentication is automatically executed between the remote control middleware 30 of the device 3 and the remote control service 10 of the server 1, and the user and the portable information terminal 2 are not involved. The user's portable information terminal 2 inquires of the remote control service 10 for information as necessary. On the other hand, the remote control service 10 replies with information on the range of the device 3 that is a candidate for remote control regarding the portable information terminal 2 of the user based on the grasped device information. As a result, the user and the portable information terminal 2 can know the range of the devices 3 that are candidates for remote control, and can select the target device 3 from the range.

In the process 800, information on the device 3 is registered based on automatic device authentication. However, the device information can be manually registered by the user. In S 9 below, the user can manually register and change settings of the information of the device 3 on the device registration screen of the setting screens provided by the server 1.

In S5, the user activates the portable information terminal 2 and the remote control application 20. In S 6, the remote control application 20 displays a login screen related to the remote control service 10 on the screen of the display device 215 based on the URI of the remote control service 10. The user inputs user authentication input information such as a user ID and a password on the login screen. The remote control application 20 attempts to log in by transmitting user authentication input information to the server 1. In S7, the remote control service 10 of the server 1 compares the user authentication input information from the portable information terminal 2 with the user ID and password of the user list registered in the user terminal information 51, and performs user authentication. . If the user ID and password match, the result of user authentication is successful. If it does not match, it will fail and you will need to try again. The remote control service 10 transmits response information representing the result of user authentication to the portable information terminal 2.

In S8, if the response of S7 is successful, the remote control application 20 of the portable information terminal 2 logs in and displays the menu screen of the remote control service 10 (FIG. 10 described later). The menu screen includes information on the remote control service 10, items that allow selection of various functions, and the like. The menu screen includes items such as settings, device list, and remote control screen display. The steps after S9 are selected according to the user's needs and can be omitted.

In S9, the user selects a setting item when registering user terminal information or device information on the menu screen. The remote control application 20 displays a setting screen. The user registers information on the user, the portable information terminal 2, the device 3, and the like on the setting screen, or changes the registered content. The remote control service 10 registers the information in the registration information 50 (user terminal information 51, device information 52) or updates the registration contents. As a setting example, it is set so that a remote operation related to a certain device 3 and its operation is permitted from a portable information terminal 2 of a certain user. As the information is registered, the device list information 56 and the like are also updated.

Also, the user can make user settings related to the use of the functions of the remote control service 10 on the setting screen. As an example of the user setting, it is possible to set which function is enabled / disabled among a plurality of functions provided by the remote control service 10.

In S10, when displaying the device list on the menu screen, the user selects a device list item. The device list includes an all device list and a nearby device list. Here, it is assumed that all device list items are selected and the all device list is displayed. The remote control application 20 displays a selected type of device list screen (FIG. 11 described later) based on communication with the remote control service 10. The remote control service 10 holds the latest device list information 56 based on the registration information 50. The device list information 56 includes information on the registered user device 3. The remote control service 10 confirms the configuration of the user's LAN 4 device 3 through device authentication and updates the device list information 56 to include the latest device 3 as needed. The remote control service 10 provides the remote control application 20 with information on all the devices 3 of the user from the device list information 56 in the case of displaying the all device list screen. The remote control application 20 holds the latest device list information of the user. The remote control application 20 displays the all device list screen using the information.

In S11, when the user wants to remotely operate the device 3, the user selects the target device 3 from the device list screen. The remote control application 20 displays a remote control screen 21 (FIG. 15 described later) of the selected device 3 based on the remote control screen data 10G. At this time, the remote control application 20 may acquire the latest remote control screen data 10 G from the remote control service 10. If the remote control screen 21 of the device 3 has been displayed once, the information is held in the remote control application 20, and the remote control screen 21 of the device 3 can be selected and displayed again from the menu screen. It is.

In S12, the user can remotely operate the device 3 in response to an input operation on the remote control screen 21 of the device 3. In S12, a remote control process at that time is performed. As described above, when the target is the first device and the first operation, communication of the first communication method is performed. That is, the operation of the device 3 is remotely controlled by TCP / IP communication via the server 1 using the first type API commands C11 and C12. When the target is the second device and the second operation, communication of the second communication method is performed. That is, using the second type API command C2, the operation of the device 3 is remotely controlled by P2P communication without going through the server 1.

In S13, when ending the remote operation of the device 3, the user performs an end operation from the remote control screen 21. In response to the end operation, the remote control application 20 ends the display of the remote control screen 21 and returns to the menu screen.

[Sequence (2)]
FIG. 9 shows a second sequence of the remote control system. The sequence in FIG. 9 includes steps S21 to S43. Hereinafter, it demonstrates in order of a step. The sequence of FIG. 9 specifically includes a process for automatically providing a neighbor list. The nearby device list is a list of available candidate devices 3 existing in the vicinity of the user's portable information terminal 2. Further, the sequence of FIG. 9 is a process of automatically establishing a P2P connection between the portable information terminal 2 of the user U1 and the device 3 of the first LAN 41 based on the control of the remote control service 10 to perform P2P communication. Including. As a premise of FIG. 9, the process up to S 8 in FIG. 8 is completed, and the menu screen is displayed on the portable information terminal 2.

In S21, the user selects a nearby device list item on the menu screen. The remote control application 20 transmits the location information of its own mobile information terminal 2 and the nearby device list request to the remote control service 10 of the server 1. The position information is obtained from a GPS receiver. In addition, the form which transmits a near device list request | requirement automatically based on a user setting may be sufficient. Further, the location information may be transmitted at the time of login in S6.

The remote control service 10 responds to the nearby device list request by configuring the nearby device list information based on the location information of the portable information terminal 2, the registration information 50, and the device list information 56. The remote control service 10 also refers to the installation position information of the device 3 included in the registration information 50. The remote control service 10 searches the device 3 existing in the vicinity with respect to the position of the portable information terminal 2 from the entire device list of the user, and configures the nearby device list information from the search result. The remote control service 10 transmits response information including the neighboring device list information to the remote control application 20. The remote control application 20 displays a nearby device list screen (FIG. 12 described later) using the nearby device list information.

In S22, the user selects the device 3 to be remotely operated on the nearby device list screen. The remote control application 20 transmits device selection information representing the selected device 3 to the remote control service 10. The remote control service 10 receives the device selection information.

In S23, the remote control service 10 performs remote control determination including confirmation of the validity of the device selection in S22. In the confirmation of the legitimacy of this device selection, it is confirmed whether the remote operation with the combination of the user's portable information terminal 2 and the selected device 3 is valid in consideration of the authority of the user of the registration information 50, etc. Is done. If it is valid, the selected device 3 is determined as the target device 3. The remote control service 10 transmits response information indicating whether or not the device can be selected to S22 to the remote control application 20.

Further, in the remote control determination in S23, based on the registration information 50 and the status of each device, the remote control screen data 10G used for remote control between the portable information terminal 2 and the target device 3 and the communication method are determined. Check. Basically, as described above, since the proper use such as which communication method to use is set in advance according to the operation of the device 3, the remote control screen data 10G according to the setting is used. . When both the portable information terminal 2 and the target device 3 have a Bluetooth function, P2P communication is possible.

In S24, the remote control service 10 transmits the remote control screen data 10G for the target device 3 to the remote control application 20 based on the determination in S23. If the data has already been transmitted, transmission is not necessary. In S25, the remote control application 20 displays the remote control screen 21 on the screen of the display device 215 based on the remote control screen data 10G acquired from the server 1.

Next to S25, a process 900 for establishing a P2P connection is performed. This process 900 is performed when the P2P communication of the second communication method may be used between the target portable information terminal 2 and the device 3 and the P2P connection is not established. Process 900 includes S26-S28. In this example, the process 900 is a process of establishing Bluetooth pairing (shown as “BT_pairing”). This process 900 is not performed directly between the portable information terminal 2 and the device 3 but is performed via the remote control service 10 of the server 1. This example shows a case where P2P communication is performed in the Bluetooth security mode.

In this example, the remote control screen is displayed in S25 prior to the P2P connection establishment process 900. However, the present invention is not limited to this, and the remote control screen may be displayed after the process 900. The second type API command can be used after the P2P connection is established.

In S26, the remote control service 10 transmits a Bluetooth PIN (shown as “BT_PIN”) to each of the device 3 and the portable information terminal 2. Both “BT_PIN” are the same number. Note that “BT_PIN” may be transmitted to the portable information terminal 2 together with the response information of S23.

In S27, upon receiving “BT_PIN”, the portable information terminal 2 transmits a Bluetooth pairing request (shown as “BT_Pairing request”) to the target device 3. The device 3 receives “BT_PIN” from the server 1 and receives “BT_pairing request” from the portable information terminal 2.

In S28, the portable information terminal 2 and the device 3 mutually perform Bluetooth authentication processing (shown as “BT_authentication processing”) based on the received information. Due to the success of this “BT_authentication process”, Bluetooth pairing, that is, P2P connection is established between the portable information terminal 2 and the device 3. If the result of the “BT_authentication process” is successful, the result is stored inside the portable information terminal 2 and the device 3, and the result is referred to in the subsequent P2P connection establishment process 900. The processing can be omitted.

After establishing the P2P connection, the user performs a touch input operation for selecting an object corresponding to the operation instruction of the device 3 on the remote control screen 21 in S30. The remote control application 20 generates an API command associated with the selected object. When the generated API command is the first type API command, the remote control application 20 executes the non-P2P communication process 901 that is the first communication method, and when the generated API command is the second type API command, the remote control application 20 executes the second type. A P2P communication process 902 which is a communication method is executed.

In process 901, in S31, the remote control application 20 transmits information including the first type API command C11 to the remote control service 10 of the server 1 by wireless communication and TCP / IP communication. In S32, the remote control service 10 of the server 1 confirms the validity of the received first type API command C11 based on the registration information 50, and also records history information. Thereafter, in S33, the remote control service 10 transmits information including the first type API command C12 to the target device 3. In S34, the target device 3 performs an operation according to the received first type API command C12. For example, when the first type API command C12 is a “setting” API command, an operation of changing the internal setting of the device 3 to a designated state is performed. In S35, the device 3 transmits response information to the first type API command C12 to the remote control service 10 of the server 1. In S36, the remote control service 10 of the server 1 confirms the validity of the response information from the device 3 based on the registration information 50, and records the history information. Thereafter, in S37, the remote control service 10 transmits the confirmed response information to the portable information terminal 2.

The remote control service 10 refers to the contents of the first type API command and the response information when performing the first type API command and response information confirmation (S32, S36). You may go.

In process 902, in S41, the remote control application 20 transmits the second type API command C2 to the target device 3 by P2P communication. In S42, the device 3 performs an operation according to the received second type API command C2. In S43, the device 3 transmits response information to the second type API command C2 to the portable information terminal 2 by P2P communication.

As described above, since the necessary P2P connection is automatically established according to the selection of the device 3, the user's effort is reduced. The nearby device list is not limited to the method obtained using the GPS position information. For example, a method using a signal such as a beacon transmitted from the device 3, a method using a signal such as Bluetooth, wireless LAN, or NFC (Near Field Communication) may be used.

[Screen-Menu screen]
FIG. 10 shows a configuration example of the menu screen. The housing of the portable information terminal 2 has a screen 220, a home button 221 and the like. A screen 220 is a screen of the display device 215. A menu screen or the like is displayed on this screen 220. On the menu screen, objects such as buttons for receiving touch input operations are arranged. In the example of FIG. 10, a plurality of items 230 that enable selection of functions to be used are arranged as objects. Items 230 include “setting (user terminal)”, “setting (device)”, “all device list”, “neighboring device list”, “remote control screen display”, and the like.

“Setting (user terminal)” item and “setting (device)” item correspond to S9 described above. The “all device list” item and the “neighboring device list” item correspond to S10. The “remote control screen display” item corresponds to S11. In response to an item selection operation, a transition is made to a screen corresponding to the function of the item. When the home button 221 is pressed, the screen returns to the OS home screen of the portable information terminal 2. On the home screen, all applications including the remote control application 20 are displayed in the state of icons, pages, and the like. The remote control application 20 can be selected and activated from among them.

[Screen-List of all devices]
FIG. 11 shows a configuration example of the all device list screen. In this example, an example is shown in which information of all devices 3 of the first LAN 41 in the portable information terminal 2 of the user U1 in FIG. 2 is displayed. On the all device list screen, information on all the devices 3 of the LAN 4 can be displayed on a user basis, and information on a group to be described later can also be displayed. In this all device list screen, a plurality of device items 241 are arranged in order as each row of the list as an object. Each device item 241 includes a selection column and a device information column. When the device item 241 is selected by a user operation, a check mark is added to the selection column. Information such as the name and ID of the device 3 is displayed in the device information column. In this example, the device ID is displayed in parentheses for easy understanding, but may be omitted. The device ID is also shown in parentheses on the device 3 in FIG. For example, a certain TV 31a has a device ID “A1”, and a certain recorder 32a has a device ID “B1”. There may be a plurality of televisions 31a in the first LAN 41. In this case, the device IDs of the plurality of televisions 31a are {A1, A2,.

In the all device list, the device item 241 on the first line indicates the first television of the plurality of televisions 31a, the device ID is A1, and the device name is “Living TV”. The second line shows the second television, the device ID is A2, and the device name is “bedroom television”. The third line shows the first recorder among the plurality of recorders 32a, and the device ID is B1 and the device name is “Living recorder”. The same applies to the other devices 3. For example, “Living recorder (B1)” in the device item 241 in the third row is selected by the user.

At the bottom of the all device list screen, buttons 242 such as “OK”, “Cancel”, “Menu”, “Previous”, and “Next” are arranged as objects for other operations. “Determination” and “Cancel” buttons are used to determine or cancel device selection. The “menu” button is used when returning to the menu screen. The “Previous” and “Next” buttons are used to change the display page state to the previous page or the next page when all the device list information cannot be displayed on the screen. The display state can also be changed by a touch input operation such as a user swipe.

When the “OK” button is pressed in the device selection state of FIG. 11, the recorder 32 a corresponding to the selected “Living recorder (B1)” is determined as the target device 3, and the remote control screen of that device 3 is displayed. Transition. A plurality of devices 3 can be selected simultaneously. In that case, the remote control screen of one device 3 among the selected devices 3 is displayed as a representative, and the remote control screen can be arbitrarily transitioned from the remote control screen to the remote control screen of another selected device 3.

The user can easily know the device 3 that can be remotely operated by the all device list, can easily select the target device 3, and is easy to use. In the device list, information on how to use the first communication method and the second communication method may be displayed for each device 3.

[Screen-List of nearby devices]
FIG. 12 shows a configuration example of the neighboring device list screen. In the neighboring device list screen, similarly to the object, there is a device item 241 in each row in a list format. In this example, information of four devices 3 is displayed as the plurality of devices 3 of the first LAN 41 in the vicinity of the mobile information terminal 2A of the user U1. For example, “Living TV (A1)”, “Living Recorder (B1)”, “Living Lighting (C1)”, and “Living Air Conditioner (D1)” in a room. For example, “Living recorder (B1)” is selected by the user.

In this example, the recorder 32a corresponding to “Living recorder (B1)” has “paid viewing service” as an external service. In the device item 241, a mark (for example, a star) indicating that there is an external service is displayed. In addition, this screen has an external service item 243. In the external service item 243, information on an external service associated with the device 3 of the device item 241, for example, “paid viewing service” is displayed, and a selection operation is possible. In the external service item 243, link information to an external service is described. When the external service item 243 is selected, the screen can be changed to a screen for using the external service. An external service item may be included in another screen such as a remote control screen.

This external service “paid viewing service” is provided by an external server of an operator different from the operator of the server 1. For example, a provider related to broadcasting, communication, or a manufacturer provides a “paid viewing service” from an external server on the Internet 6.

Since the nearby device list allows the user to easily know the remotely operable device 3 in the vicinity of the user and easily select the target device 3, it is easy to use. Also, the external service can be easily recognized and used from the external service item in the screen, and it is easy to use.

[Screen-Device List (Group)]
FIG. 13 shows a display example in units of groups as another example of the device list screen. As one of the functions of the remote control service 10, a plurality of devices 3 can be arbitrarily set as a group by the user. The user can select a group setting item among the setting items from the menu screen, and can set the device 3 to be grouped, the group name, and the like on the setting screen. The registration information 50 stores group setting information.

As an example of group setting on the device list screen of FIG. 13, all devices 3 of the first LAN 41 corresponding to the home of FIG. 2 are set to the first group, and all devices 3 of the second LAN 42 corresponding to the office are set to the second group. . The ID of the first group is “G1”, and the ID of the second group is “G2”. The group name of the first group “G1” is set as “Home group”. The group name of the second group “G2” is set as “Office group”. The first group “G1” includes, for example, “Living TV (A1)”, “Living Recorder (B1)”, and the like.

When a group is set, a group item 244 is displayed on the device list screen. The group item 244 includes a selection field and a group information field. In the group item 244, the device items 241 of the devices 3 constituting the group are hierarchically related. In addition, according to the group item selection operation, it is possible to switch between a state in which the device item 241 is expanded and displayed and a state in which the device item 241 is not displayed. In addition, group display and group non-display can be switched by a user operation. FIG. 11 shows the case of non-group display, and FIG. 13 shows the case of group display. With the group setting function, the user can set a desired plurality of devices 3 as a group, and the management and remote operation of the plurality of devices 3 become easier.

FIG. 14 shows another group setting example on the device list screen. In this example, it has a group “Home Eco List (G3)”. This group “G3” is composed of eco-related devices 3 such as the lighting device 33a and the air-conditioning device 34a, etc., which are concerned about power consumption and electricity charges, among the devices 3 of the first LAN 41 at home. This group “G3” has, for example, “Living lighting (C1)”, “Bedroom lighting (C2)”, “Living air conditioner (D1)”, “Bedroom air conditioner (D2)”, “Smart meter (E1)”, etc. . The device list screen also includes a group item of group “G3”. On the device list screen, the device item 241 of the single device 3 and the device item 241 of the group item 244 may be displayed in duplicate.

Also, this group “G3” has “eco service” as a related service, and a group item 244 is marked with a mark (for example, a star) indicating that there is a related service. In addition, a related service item 245 is included in the screen. In the related service item 245, information on a related service associated with the device 3 or the group, for example, “eco service” is displayed, and a selection operation is possible. In the related service item 245, link information to the related service is described. When the related service item 245 is selected, the screen can be changed to a screen for using the related service. Related service items may be included in other screens such as a remote control screen.

This related service is provided by the server 1 of the business operator. The remote control service 10 and related services cooperate. This “eco-service”, for example, for the device 3 of the group “G3”, uses information acquired from the device 3 to monitor the power consumption, electricity bill, etc. of the device 3 or from a smart meter or the like. Information is collected and aggregated. The “eco service” processes the aggregated information, creates a monthly graph or the like, and provides a screen of the graph or the like. The “eco service” presents the user with easy-to-understand information such as the power consumption and the electricity charges of the plurality of devices 3 in the group “G3”, and recommends usage methods that lead to power saving. Thereby, it is easy for the user to grasp the power consumption, the electricity charge, etc. of the device 3, and effective power saving can be achieved in the whole home. As will be described later, the content of the related service is not limited to the above example.

The group is not limited to the above example but can be set freely. For example, as a group for each device type, a television group, a recorder group, or the like is possible. For example, a group of a plurality of televisions 31a {A1, A2,..., Am} on the first LAN 41 is possible. A group for each area or room can be set. For example, a group of devices 3 {TV “A1”, recorder “B1”, lighting device “C1”, air-conditioning device “D1”, etc.} in “Room 1 (Living)” at home is possible. In addition, groups corresponding to user attributes and authorities such as father, mother, and child are possible. For example, a group of devices 3 that the user U1 having the user attribute “Father” has authority to use is possible.

Further, the server 1 may automatically form a group by using information included in the registration information 50 and the device product information 55. The remote control service 10 registers the group after confirming the group configuration with the user on the screen. Note that the group can be set not only in units of the devices 3 but also in units of operations, users, portable information terminals 2 and the like. In addition, use of the first type API command and the second type API command can be set for each group.

[Screen-Remote control screen]
FIG. 15 shows a display example of the remote control screen 21. In this example, the remote control screen 21 when the target device 3 is the recorder 32a is shown. The remote control screen of FIG. 15 includes a device information column 250, a first type command column 251, a second type command column 252, a content information column 253, and a button column 254. In the device information column 250, device information indicating which device 3 is the remote control screen is displayed.

In the first type command column 251, an object 255 such as a button associated with the first type API command is arranged. In this example, buttons such as “setting”, “recording reservation”, “reproduction list (recorded content)”, “device search”, “paid viewing (use of paid viewing service)”, “HELP”, and the like are provided. The “paid viewing” button corresponds to the external service “paid viewing service” in FIG. 12, and is an object linked to the external service.

In response to the operation of the object 255 in the first type command column 251, non-P2P communication with the device 3 is performed via the server 1 using the first type API command in the first communication method. At that time, although not shown, there may be a transition from the remote control screen to a remote control screen of a lower hierarchy corresponding to the selected object 255. The remote control screen data 10G also describes the transition control information. For example, when the “recording reservation” button is selected, a transition is made to the “recording reservation” remote control screen. The screen includes, for example, an object of a broadcast program guide and an object capable of inputting recording reservation information such as content and date / time. Similarly, the input information on the screen is transmitted to the remote control service 10 as the first type API command and the accompanying parameter information. It is also possible to return from the lower layer screen to the upper layer screen.

In the second type command column 252, an object 256 such as a button associated with the second type API command is arranged. In this example, the second type command field 252 includes an object 256 for controlling recording and playback of content as an object 256 corresponding to the operation instruction selected on the first type command field 251 and the lower layer screen. Has been placed. In this example, as the playback control object 256, buttons for fast reverse playback, normal playback / pause and fast forward playback are arranged in order from the left in the upper row, and reverse skip, stop, The order skip button is arranged. In accordance with the operation of the object 256 in the second type command column 252, P2P communication with the device 3 is performed using the second type API command without using the server 1 in the second communication method. Therefore, the responsiveness of the reproduction control is good.

In the content information column 253, for example, information on the content to be recorded or reproduced by the recorder 32a, for example, the title of the broadcast program is displayed. For example, when the “play list” button is selected, the content is selected by the user from the recorded content list displayed on the “play list” screen in the lower hierarchy. Returning to the remote control screen of FIG. 15, the information (for example, “broadcast program A”) is displayed in the content information field 253.

In the button column 254, objects such as buttons such as “Menu”, “Previous”, and “Next” are arranged. The “menu” button is used when returning to the menu screen. The “Previous” and “Next” buttons are used to switch the display state in the remote control screen 21 in units of pages or the like. The display state can also be changed by a predetermined touch input operation. The remote control screen of one device 3 may be composed of a plurality of remote control screens.

In the remote control application 20 of the mobile information terminal 2, when the remote control screens of a plurality of devices 3 are being executed simultaneously, one selected remote control screen is displayed on the screen of the mobile information terminal 2. Other remote control screens can be selected and displayed by a predetermined button, touch input operation, or operation of a remote control screen display item on the aforementioned menu screen.

In this example, in the remote control screen, objects are intensively arranged in areas distinguished as the first type command column 251 and the second type command column 252 according to the type of API command. Thereby, a user's convenience can be made favorable. The present invention is not limited to this, and various objects may be mixedly arranged in the remote control screen without providing the first type command column 251 and the second type command column 252 separately. The first type command column 251 and the second type command column 252, the object 255, and the object 256 may be displayed in different information and colors so that the user can know the type of API command.

The design of the image and arrangement of objects in the remote control screen data 10G is defined by the remote control service 10 of the server 1 and the manager of the business operator. This design may be different depending on the portable information terminal 2 and the remote control application 20. Based on the remote control screen data 10G, the remote control application 20 may generate a remote control screen 21 that has been changed to a unique image or arrangement according to user settings or the like.

[Screen-Device update notification screen]
The remote control service 10 has a device update notification function. If the user is set to use this function, the remote control service 10 displays a device update notification screen on the user's portable information terminal 2. When the user's new device 3 is connected to the user's LAN 4, the remote control service 10 acquires information on the device 3 through automatic device authentication as in the process 800 of FIG. At that time, if the remote control service 10 can acquire information such as API commands and functions from the device 3, the remote control service 10 may acquire the information and store it in the device product information 55. If information such as the API command of the device 3 is registered in the device product information 55 in advance, the information may be used.

The remote control service 10 transmits a device update notification to the remote control application 20 in order to confirm with the user whether or not to register the new device 3 as a remote control target or candidate. The remote control application 20 displays a device update notification screen based on the device update notification. The device update notification screen is a screen that informs the user that the device 3 is updated, confirms whether or not to add the remote control screen of the device 3, and prompts the user to download update information for the addition. The update information is the remote control screen data 10G including the API command of the device 3 or the update information thereof, in other words, the update information of the remote control application 20 for that purpose.

FIG. 16 shows an example of a device update notification screen. This screen has a notification column 261, a device column 262, and a button column 263. In the notification column 261, a notification message such as "A new device has been found. Do you want to register this device in the remote control service?" In the device column 262, various types of information such as a device name (eg, “health care meter”), presence / absence of a P2P function, and the like are displayed as information on the device 3 to be confirmed. The button field 263 has a button that can select whether or not to register the device 3. When the “Register” button is selected, the device 3 is registered as a candidate in the remote control service 10, and the process proceeds to an update information download process. If the “Do not register” button is selected, the process ends without registering the device 3.

Further, when the firmware of the device 3 registered in the remote control service 10 is updated and the API command to be used is updated, the device update notification can be performed in the same manner. The remote control service 10 confirms the state of the device 3 in response to communication with the registered device 3, and similarly performs device update notification when there is an update. On the device update notification screen, the user can select whether to continuously register the device 3 or cancel the registration. In addition, the user can select a setting item on the menu screen at any time, display information on the registered device 3 on the setting screen, select the device 3 to be deregistered from the information, and cancel the registration. .

Furthermore, in the device update notification function, the remote control service 10 may confirm whether the user's portable information terminal 2 has the authority to use the new device 3 based on the registration information 50. If the authority for the device 3 is already set by the user, it can be confirmed. When the authority regarding the device 3 is not yet set, the user may be prompted to set the authority together with the device update notification. The remote control service 10 confirms whether or not to register the device 3 when the user has authority to the new device 3.

As described above, the device 3 registered in the remote control service 10 can be arbitrarily added or released based on user confirmation. Flexible remote control is possible for the registered device 3. This is not limited to the screen examples described above, and the design and the like are not limited.

[Registration information (1)-User terminal information]
FIG. 17 shows a configuration example of the user terminal information 51 regarding the user and the portable information terminal 2 in the registration information 50 of the server 1. The table of user terminal information 51 includes a user ID, a password, a terminal ID, a terminal IP address, LAN information, attributes, and the like as columns. “User ID” and “Password” are information used in user authentication. The “terminal ID” and “terminal IP address” are the ID and IP address of the portable information terminal 2 used by the user. “LAN information” is information related to the LAN 4 for which the user has authority, and includes, for example, the IP address of the gateway 81. “Attribute” is user attribute information, such as “father”, “mother”, “child”, etc. for a plurality of related users.

[Registration information (2)-Device information]
FIG. 18 shows a configuration example of the device information 52 related to the device 3 among the registration information 50 of the server 1. The table of device information 52 includes a user ID, device ID, device IP address, device type, P2P function, operation, API command, API command type, and the like as columns. “User ID” is information corresponding to “User ID” of the user terminal information 51. “Device ID” is the ID of the device 3 used by the user. “Device IP address” is the IP address of the device 3. The “device type” is a type of the device 3 and indicates a rough classification such as a television or a recorder. “P2P function” indicates whether or not the device 3 has a P2P function such as a Bluetooth function. “Operation” indicates an operation that can be performed by the device 3. “API command” indicates an API command associated with the operation. “API command type” indicates a first type API command or a second type API command as a type set in the API command.

The setting example is as follows. In the first to fifth lines, corresponding to the configuration shown in FIG. 2, the device 3 used by the user U1 has the television 31a whose device ID is “A1”, and has the P2P function. The operation of the device 3 includes “power on”, “power off”, “setting”, “channel selection”, “volume adjustment”, and the like, and has an API command associated with each operation. For example, operations such as “power on”, “power off”, and “setting” are settings that use the first type API command. Operations such as “channel selection” are settings that use the second type API command.

Similarly, in the eleventh line, the second television 31a whose device ID is “A2” is included, the P2P function is set, and the first type API command is used for a predetermined first operation group. is there. The twelfth line is a setting for using the second type API command for the predetermined second operation group. The first operation group or the like can be set separately by a user or a business operator. Similarly, the 21st line has the third television 31a whose device ID is “A3”, the device 3 does not have the P2P function, and the first type API command is assigned to the predetermined first operation group. It is a setting to use.

Similarly, in the 31st to 34th lines, the device 3 used by the user U1 has a recorder 32a whose device ID is “B1”, and has a P2P function. The operation of the device 3 includes “power on”, “setting”, “recording reservation”, “playback”, and the like, and has an API command associated with each operation. For example, operations such as “power on”, “setting”, and “recording reservation” are settings using the first type API command. An operation such as “play” is a setting for using the second type API command.

Similarly, the 41st line has the lighting device 33a whose device ID is “C1”, has a P2P function, and “power on” is a setting to use the first type API command. Similarly, from the 51st line onward, the apparatus 3 used by the user U2 has the television 31a whose apparatus ID is “A1”, which is the same as that on the 1st line, and an API command related to the operation of the apparatus 3 is set. ing. The same is set for other users. As in the above example, the type of API command to be used can be set according to the usage relationship of the user, the portable information terminal 2, the device 3, and the operation.

[User permission settings]
The user authority can also be set in the registration information 50, and a setting example is as follows. Using the setting function, authority such as permission / non-permission of remote operation can be set in units of users, portable information terminals 2, devices 3, groups, and the like. Authority can also be set in the operation of the device 3, that is, in units of API commands. Also, the authority can be set in the unit of the content that is the target of the operation of the device 3. It is also possible to use a different communication method depending on the authority.

For example, the user U1 “father” uses the portable information terminal 2 whose terminal ID is “M1”, the user U2 “mother” uses the portable information terminal 2 whose terminal ID is “M2”, and the user U3 “child” "Uses the portable information terminal 2 whose terminal ID is" M3 ". The device ID of the device 3 that permits the remote operation from the portable information terminal 2 “M1” of the user U1 is {A1, A2, B1, B2, C1, C2, D1, D2, etc.} described above. Assume that the device ID of the device 3 that permits remote operation from the mobile information terminal 2 “M2” of the user U2 is {A2, B2, C2, D2, etc.}. The device ID of the device 3 that permits remote operation from the mobile information terminal 2 “M3” of the user U3 is {A3, B3, C3, D3, etc.}.

[Product information]
FIG. 19 shows a maker device list table as a configuration example of the device product information 55 of the server 1. This table includes a manufacturer, a device type, a model name, an API command (link), and the like as columns. “Manufacturer” indicates the ID of the manufacturer of the device 3. “Device type” indicates a rough type of the device 3. “Model name” indicates identification information of the device 3 in the manufacturer. “API command (link)” indicates link information related to an API command disclosed by the device 3, and is linked to another table. In another table, API commands related to each operation of the device 3 are described. When a new device 3 is released from the manufacturer, the manager of the business operator registers information including the API command of the device 3 in the device product information 55.

[History information]
FIG. 20 shows a configuration example of history information recorded and grasped by the remote control service 10 of the server 1. The remote control service 10 grasps the communication using the first type API command, and records history information indicating the status and results of access to the plurality of devices 3 from the plurality of portable information terminals 2 in the DB. The remote control service 10 can also provide related services to be described later using the history information. The history information table in FIG. 20 includes, as columns, date and time, user ID, terminal ID, device ID, operation, API command, API command type, and the like. “Date and time” is the date and time when the history information was created when the remote operation access occurred. “User ID” and “terminal ID” indicate the user who is the subject of the remote operation and the portable information terminal 2. “Device ID” indicates the device 3 to be remotely operated. “Operation”, “API command”, and “API command type” indicate the operation, API command, and type of the target of remote operation.

[Effects]
As described above, according to the first embodiment, regarding the remote control system that enables remote operation of the device 3 from the user's portable information terminal 2, depending on the device 3 and its operation, Usability, security and safety can be ensured. When the user is at home or the like and wants to remotely operate the device 3 of the first LAN 41 from the portable information terminal 2, a first communication method that passes through the server 1 and a second communication method that performs P2P communication without passing through the server 1 Can be used properly.

According to the first embodiment, based on the setting and control of the remote control service 10, different communication methods can be properly used according to the operation of the device 3, and responsiveness and safety, which are advantages according to the type. Etc. are obtained. It is possible to selectively use P2P communication for the operation (for example, content reproduction) of the device 3 that emphasizes responsiveness, and non-P2P communication for the operation (for example, power-on) of the device 3 for which safety is important. In the first communication method, the first type API command can be confirmed, recorded, restricted, etc. via the server 1, so that safety and the like can be ensured. The remote operation of a large number of devices 3 can also be operated safely. In the first communication method, a related service can be provided by the server 1, and the provision of an external service by an external server can be grasped.

In the second communication method, since P2P communication can be performed directly without going through the Internet 6, the response speed can be increased. Communication using the first type API command includes TCP / IP communication via the external Internet 6 to the LAN 4. Therefore, delays and variations may occur during the communication. For this reason, for example, with respect to an operation for which priority is given to responsiveness such as “playback” of the recorder 32a, it is possible to ensure good responsiveness by using the second communication method.

In the conventional system, since the remote operation screen is realized by the processing of the Web browser, it takes a relatively long time to display the screen. On the other hand, in the portable information terminal 2, remote control is realized by the remote control application 20 that cooperates with the remote control service 10, and the display of the remote control screen 21 is relatively fast. Therefore, it is possible to realize better responsiveness and usability than the conventional technology realized by a Web browser.

Also, in the conventional system, when the user himself selects a device to be remotely operated from a large number of devices, the operation is troublesome, it is difficult to judge, and the usability is not good. According to the first embodiment, it is easy to select the device 3 from the device list or the like and it is easy to use.

In addition, in the conventional system, it is necessary to perform authentication such as user authentication and device authentication serially between devices for security. For this reason, the authentication takes a relatively long time, and the usability may not be good. On the other hand, according to the first embodiment, it is possible to reduce the manual setting operation by the user, such as automatically performing device authentication when the device 3 is connected, and it is convenient.

[Modification]
The following is given as a modification of the remote control system of the first embodiment. In a modification, the portable information terminal 2 transmits control information corresponding to an object selection operation on the remote control screen 21 to the server 1. The remote control service 10 of the server 1 converts the control information received from the portable information terminal 2 and obtains a first type API command as a conversion result. The remote control service 10 transmits the first type API command to the target device 3.

[Modification (1)-Second communication method: P2P communication over IP]
FIG. 21 shows a configuration of a remote control system of a modification. In a modified example, P2P communication of the second communication method can be performed as P2P communication on the IP network via the Internet 6 without passing through the server 1. As one of the above cases, P2P communication is possible when the user's portable information terminal 2 is located at a certain distance from the LAN 4 device 3 and the portable information terminal 2 and device 3 have the P2P communication function. Can be. For example, the portable information terminal 2B outside the home in FIG. 2 may be able to perform P2P communication on the IP network with the device 3 via the mobile communication network 7 and the Internet 6. The portable information terminal 2B is not connected to the gateway 81 of the first LAN 41 by a wireless LAN or directly connected to the device 3 by P2P.

In such a case, according to the operation of the device 3, P2P communication on the IP may be used as the second communication method. In that case, based on the control of the server 1, a P2P connection over IP is established with the device 3 from the portable information terminal 2 via the mobile communication network 7 and the Internet 6. Then, the second type API command is transmitted from the portable information terminal 2 to the device 3 by P2P communication over IP without going through the server 1.

However, since the P2P communication on the IP is P2P communication via the Internet 6, there is a possibility that good responsiveness may not be ensured as compared with the P2P communication of the first embodiment not via the Internet 6. The P2P communication control over IP may be used with the user's consent to such restrictions. The user can remotely operate the device 3 such as the first LAN 41 from the portable information terminal 2B outside the home, for example, after approving the restrictions. Further, the user can operate the device 3 of the second LAN 42 from the portable information terminal 2A inside the home, for example.

The details of the P2P communication on the IP are as follows. The mobile communication unit 207 of the portable information terminal 2B can transmit both the first type API command and the second type API command as API commands. In this modification, communication of the second type API command is performed as P2P communication over IP using the mobile communication unit 207 instead of P2P communication using the P2P communication unit 208.

The remote control service 10 of the server 1 grasps the IP address and the like of the device 3 through the gateway 81 in advance in the device authentication process with the device 3 and stores it inside. When establishing a P2P connection over IP with the target device 3, the CPU 201 of the portable information terminal 2 B inquires the remote control service 10 of the server 1 about the IP address of the target device 3 through the mobile communication unit 207. In response to the inquiry from the portable information terminal 2B, the remote control service 10 teaches the IP address of the target device 3 and assists the establishment of the P2P connection on the IP. The CPU 201 and the mobile communication unit 207 of the portable information terminal 2B establish a P2P connection over IP with the target device 3 based on information such as an IP address obtained from the remote control service 10. Then, the portable information terminal 2B transmits the second type API command with the IP address as the destination. The mobile communication network 7 and the Internet 6 route the packet including the second type API command and transmit it to the destination device 3.

[Variation (2)-Device Function Server Merge]
As a modification, the remote control service 10 of the server 1 or the related service may have a part or all of the functions and data of the device 3 or may be duplicated in both. For example, functions such as “recording reservation” and “play list” of the recorder 32a and data thereof are not provided to the recorder 32a but to the related service of the server 1. For example, the recording reservation function of the recorder 32a may be implemented and provided as a recording reservation service by the server 1. In this case, the implementation of the function in the device 3 can be omitted, and the function can be merged with the server 1. For example, when the user's portable information terminal 2 remotely operates a function of the device 3, for example, a recording reservation operation, the corresponding first type API command is transmitted to the server 1. The server 1 executes a recording reservation operation according to the command and transmits a response to the portable information terminal 2.

In the case of duplexing, for example, both the function and data of the device 3 are duplicated in both the device 3 and the related service of the server 1. In this case, even when a failure occurs in the communication network 5, the function can be used through either the device 3 or the server 1. In this case, either the server 1 or the device 3 may be selected on the remote control screen 21.

[Variation (3)-Multiple device linkage control]
As a modification, it has a multiple device interlocking control function. This function enables remote control of the devices 3 registered in the remote control service 10 as well as remote control of a single device 3 and interlocking a plurality of devices 3 together. Using a setting function, a desired plurality of devices 3 that the user wants to link and their operations can be set in units such as a simultaneous execution group that is a specific group.

For example, as a plurality of devices 3 installed in the same room, there are a television 31a, a lighting device 33a, an air conditioner 34a, and the like. In that case, the user sets those devices 3 as a group and sets a predetermined operation of the devices 3 in the group, for example, power-on. The remote control service 10 sets the operation of the interlock control group to use the first type API command. The remote control screen data 10G describes items for the interlock control (for example, “Living power on”). The user can select the item of the interlock control on the remote control screen 21. Communication with the server 1 using the first type API command is performed according to the selection of the item. The remote control service 10 of the server 1 controls the linked operation of the selected group. At that time, a plurality of first type API commands are sequentially transmitted in series within a short time from the server 1 to a plurality of devices 3 in the target group. Accordingly, the plurality of devices 3 in the group perform operations such as power-on almost simultaneously in a short time. Thereby, the user can operate the plurality of devices 3 together with a simple operation, which is easy to use.

Further, in the above-described interlock control, a specific operation of another device 3 or use of a specific content may be interlocked according to a specific operation on a certain device 3 or use of a specific content. For example, in conjunction with the playback operation of the content on the television 31a in a room, the state of the lighting device 33a in the room, such as power on / off and the amount of illumination, is controlled. Further, when the specific content is a movie, the illumination amount of the illumination device 33a is adjusted so that the environment becomes dark as an environment suitable for the reproduction.

Furthermore, a remote control learning function may be provided based on the history information of the remote control service 10. With the learning function, the user's preference is learned regarding the remote operation access to the device 3 from the user's portable information terminal 2. As the learning, for example, the frequency, the bias, and the like are grasped as to which operation of which device 3 is remotely operated from which user's portable information terminal 2. Note that a technique such as machine learning may be applied to the learning function. As remote control according to the learning result, for example, for the operation of the device 3 that is frequently accessed, the first type API command is switched to the use of the second type API command, and for the operation of the device 3 that is not accessed much, For example, switching from using the second type API command to using the first type API command. Thereby, the efficient remote control according to a user preference is realizable.

[Variation (4)-Remote control according to communication status, mask display control]
As a modification, it has a remote control function according to the communication status. The server 1 grasps the communication status of the portable information terminal 2 and the device 3 connected to the communication network 5. The communication status includes the load status of the device 3 and the usage status of two types of communication methods. Further, the server 1 considers the positional relationship of the user's portable information terminal 2 with respect to the device 3 of the LAN 4, the function of each device, authority, and the like. The server 1 determines which method to use according to the communication status or the like when two types of communication methods can be used for remote control communication between the portable information terminal 2 and the device 3.

In addition, the server 1 may control switching from one method to the other method according to the communication status or the like. For example, the second type API command is basically set for the operation of the device 3 for which responsiveness is required. In that case, there may be a case where the response of P2P communication is not good. For example, this is the case where the portable information terminal 2 is separated from the device 3 to some extent. When the server 1 grasps such a situation, the server 1 temporarily switches from P2P communication to communication using the first type API command. At that time, the server 1 may display a message to switch the method on the remote control screen. Similarly, switching from the first communication method to the second communication method is also possible. Further, when two types of methods can be used, the method may be switched by selecting a method by a user operation on the screen of the portable information terminal 2.

In addition, the server 1 has a function of controlling a response to the user's portable information terminal 2 in accordance with a communication situation or the like. For example, when the device 3 is in an operation error state, a failure state, a congestion state of the LAN 4, etc., there are cases where normal operation or response cannot be performed to a remote operation using an API command. When the server 1 receives the first type API command from the user's mobile information terminal 2, the server 1 sends the status to the user's mobile information terminal 2 based on the status of the device 3 or the LAN 4 and the remote operation disabled state. The response information to the effect is notified. In addition, when the server 1 grasps such a situation in advance before receiving the first type API command from the user's portable information terminal 2, the server 1 informs the user's portable information terminal 2 in advance of the situation or You may notify that a remote operation is impossible. Further, when the server 1 is in a situation where the responsiveness of the P2P communication is not good, the server 1 may display the state of the situation on the remote control screen.

Further, the server 1 may perform mask display on the remote control screen so that the operation of a certain device 3 is temporarily in a remote operation disabled state depending on the situation. In the remote control screen data 10G, a mask display object is described so that a certain operation is temporarily unavailable. Thereby, on the remote control screen, the object corresponding to the operation is in the mask display state, and the user cannot perform the selection operation. Therefore, unnecessary operations and communication can be reduced. As the mask display, the object may be displayed in gray or shaded, or may not be selected by not displaying the object.

(Embodiment 2)
A remote control system and the like according to the second embodiment of the present invention will be described with reference to FIGS. The basic configuration of the second embodiment is the same as the configuration of the first embodiment, and hereinafter, the components of the second embodiment that are different from the first embodiment will be described. In the second embodiment, in addition to the remote control service 10, there is a related service for performing applied control. The server 1 provides related services to the user based on remote control by the remote control service 10. The related service may be implemented as a part of the remote control service 10.

[Remote control system]
FIG. 22 shows the configuration of the remote control system of the second embodiment. The remote control system of the second embodiment includes a server 1, a portable information terminal 2, a LAN 4 device 3, and an external server 9, which are connected via a communication network 5.

The server 1 provides a remote control service 10 and a related service 15. The remote control service 10 and the related service 15 operate in cooperation. The plurality of services are realized based on the hardware and software of the server 1 and can be executed simultaneously. In the second embodiment, the internal storage 104 in FIG. 4 of the server 1 stores a related service program (not shown) in addition to the remote control service program 10P. The CPU 101 or the like of the server 1 configures the entity of the related service 15 based on processing according to the related service program. In the second embodiment, the “eco service” described above is included as an example of the related service 15, but the present invention is not limited to this. The related service 15 is not limited to one, and a plurality of types may be provided.

The external server 9 is an external service providing apparatus, which is a server apparatus provided by an external business operator different from the business operator of the server 1, and may be provided by the manufacturer of the device 3, for example. The external server 9 provides an external service 90 based on software program processing. There may be a plurality of types of external services 90 of the external server 9. The external service 90 is a service associated with a certain device 3, and is, for example, a “paid viewing service” associated with the television 31a or the recorder 32a.

The portable information terminal 2 can use the remote control service 10 and the related service 15 of the server 1 through the remote control application 20 and the remote control screen 21. In Embodiment 2, in addition to the remote control application program 20P, the portable information terminal 2 stores an application program for using an external service and related services, which is not shown, in addition to the remote control application program 20P. The portable information terminal 2 implements an application for using an external service or a related service based on the application program. In this example, the “eco service” application is provided as the related service application, and the “paid viewing service” application is provided as the external service application. Those apps operate in cooperation with the remote control app 20. Those apps may be configured as a part of the remote control app 20. Those application programs can be downloaded from the server 1 or the external server 9 as necessary.

The related service application, the external service application, and the remote control application 20 can be linked in a screen such as the remote control screen 21 or the device list screen of the portable information terminal 2. For example, as shown in FIG. 12, FIG. 14, and FIG. 15, the device list screen and the remote control screen 21 have link information to external services and related services as objects.

For example, when the object 255 which is the “pay viewing” button is selected on the remote control screen 21 of FIG. 15, the remote control application 20 cooperates with the “paid viewing service” application, and the control is switched. The “paid viewing service” application communicates with the external service 90 of the external server 9 based on the URI, and displays the “paid viewing service” screen.

For example, when “eco service” of the related service item 245 is selected on the device list screen of FIG. 14, the remote control application 20 cooperates with the “eco service” application, and the control is switched. Based on the URI, the “eco service” application communicates with the related service 15 of the server 1 and displays the “eco service” screen.

In this way, related services and external services related to the device 3 and the group can be easily used from each screen of the portable information terminal 2, and the convenience of the user is high. The server 1 grasps the remote operation using the first type API command by the remote control service 10 and provides the related service 15 based on the grasp. The related service 15 can provide application control to the basic control of the remote control service 10 and various services useful for the user.

[External service]
An example of using the external service 90 is as follows. When the use of the external service is selected on the screen of the portable information terminal 2, the external service application accesses the external service 90 of the external server 9 through the communication network 5. The external service 90 of the external server 9 provides external service screen data and the like for access. The external service application displays an external service screen. The user inputs an instruction or the like on the external service screen. For example, playback of paid viewing content is designated. The external service application transmits request information such as the designation. The external service 90 transmits response information and the like with respect to the request information, and performs accounting processing and the like. For example, data of designated pay viewing content is transmitted. The external service application receives the data and reproduces the paid viewing content.

[Related Services]
An example of using the related service 15 is as follows. When the use of the related service is selected on the screen of the portable information terminal 2, the related service application accesses the related service 15 of the server 1 through the communication network 5. The related service 15 of the server 1 provides related service screen data and the like for access. The related service application displays a related service screen. The user inputs an instruction or the like on the related service screen. For example, an electric power consumption graph display in the function of “eco service” is designated. The related service application transmits request information such as the designation. The related service 15 transmits response information and the like with respect to the request information. For example, graph data of the power consumption for a predetermined period of the devices 3 of the specified group is transmitted. The related service application receives the data and displays a power consumption graph on the screen.

[Related service management information]
FIG. 23 shows a configuration example of management information of “eco service” as management information of the related service 15 of the server 1. This table has a group ID, a group name, a device ID, power consumption, an electricity bill, a graph link, a history link, and the like as columns. “Group ID” and “Group Name” indicate, for example, the above-described group “G3” and “Home Eco List”. “Device ID” indicates the devices 3 constituting the group. “Power consumption” indicates power consumption ascertained for the device 3, for example, watts per month (W). “Electricity rate” indicates an electricity rate ascertained for the device 3, for example, a yen per month. The “power consumption” and the “electricity charge” may be grasped in cooperation with the device 3 such as a smart meter. “Graph link” indicates a link to a graph of “power consumption” and “electricity charge” of the device 3. “History link” indicates a link to usage history information of the device 3. Not only for each device 3, but also a graph can be created and presented with statistics for each device type and group.

[Effects]
As described above, according to the second embodiment, the related service 15 associated with the remote control service 10 can provide high added value to the user.

[Modification]
The following is given as a modification of the remote control system of the second embodiment. In the following modification, an example of services and functions that can be added as the related service 15 is shown.

[Variation (1)-Recording reservation support service, duplicate use arbitration function]
FIG. 24 is an explanatory diagram regarding a “recording reservation support service” which is an example of the related service 15 in the modification. A support function linked to the function of the device 3 (example: recorder 32a) (example: recording reservation function) is provided as the related service 15 “recording reservation support service” of the server 1. The function of “recording reservation support service” includes a function of arbitrating and avoiding overlapping use and access related to recording reservations to a plurality of devices 3 from a plurality of users and a plurality of portable information terminals 2.

As an example, it has a plurality of users and a plurality of portable information terminals 2 using a plurality of devices 3 including a plurality of recorders 32a in the first LAN 41 of FIG. For example, the first recorder “B1” is provided as the first device, and the second recorder “B2” is provided as the second device. For example, the first portable information terminal “M1” of the first user (U1, “father”), the second portable information terminal “M2” of the second user (U2, “mother”), the third user (U3, “child”) )) Third portable information terminal “M3”, and the like. Operations such as “recording reservation” and “playback” of the recorder 32a are remotely operated from the remote control screen 21 of the portable information terminal 2 of each user. Examples of target content include a first broadcast program (content c1) and a second broadcast program (content c2) that are broadcast programs of the television 31a.

In the example of FIG. 24, the first portable information terminal of the first user is the first type API command for the first operation group of the first device and the second device, and the second type for the second operation group. Remote operation is performed using an API command. The second user's second portable information terminal uses the first type API command for the first operation group of the second device and the third device, and the second type API command for the second operation group, Perform remote operation.

For example, the first user performs an operation for recording reservation of the first broadcast program on the remote control screen 21 of the second recorder in the first portable information terminal. The recording reservation is included in the first operation group and is associated with the first type API command. The first portable information terminal transmits a first type API command indicating the recording reservation to the remote control service 10, and the remote control service 10 transmits the first type API command to the second recorder. The second recorder makes a recording reservation for the first broadcast program in accordance with the first type API command. Next, it is assumed that the second user has performed a recording reservation operation for the first broadcast program on the remote control screen 21 of the second recorder in the second portable information terminal. At this time, the first user has already reserved recording of the first broadcast program with the second recorder. That is, the recording reservation for the first broadcast program of the second recorder overlaps. The second portable information terminal transmits a first type API command for recording reservation to the remote control service 10.

Here, the related service 15 “recording reservation support service” of the server 1 refers to the first type API command received and grasped through the remote control service 10 and performs the overlapping use arbitration control regarding the recording reservation as follows. .

First, in the case of a technology that does not have a duplicate use arbitration function, it is as follows. A first type API command from the second portable information terminal is transmitted to the second recorder. The second recorder attempts to make a recording reservation for the first broadcast program according to the command, but the first portable information terminal of the first user has already made a recording reservation for the first broadcast program. For this reason, the second recorder sets a response as a result of the command as an error.

(1) As a first function of the duplicate use arbitration function, when performing control to avoid the simultaneous use and simultaneous access to the operation of the device 3 and the content, it is as follows. Based on the reference to the first type API command, the related service 15 grasps the usage status of the plurality of portable information terminals 2 of the plurality of users, the recording reservation operation of the plurality of devices 3, and the plurality of contents. The related service 15 performs a duplicate use arbitration determination process based on the usage status of the first type API command for recording reservation from the second portable information terminal of the second user. Based on the determination, the related service 15 indicates that the recording reservation of the first broadcast program on the target second recorder has already been completed from the first portable information terminal of the first user, and the usage is duplicated. The response is sent to the second portable information terminal of the two users as a response. The related service 15 omits sending the first type API command to the second recorder. As described above, in the first function, when the use is duplicated, access on the side to be accessed later is blocked. Thereby, it is possible to avoid duplication of recording reservations.

(2) As a second function, when the use is overlapped, the control is performed as a shared recording reservation between the first user and the second user. In the “recording reservation support service”, user setting for this shared recording reservation can be made in advance. Whether or not to make a shared recording reservation can be set based on the relationship and authority between users. For example, it is assumed that the setting is to make a shared recording reservation between father and mother users.

The related service 15 performs a duplicated use arbitration determination process on the first type API command for recording reservation from the second portable information terminal of the second user based on the usage status and user setting information. If the related service 15 determines that the shared recording reservation is to be performed based on the determination, the related service 15 executes the shared recording reservation process. The related service 15 notifies the second user's second portable information terminal of a usage status and a response to make a shared recording reservation. Accordingly, the second user can use the first broadcast program that has been reserved for recording by the first recorder by the first user. The related service 15 transmits the first type API command indicating the shared recording reservation to the second recorder. In accordance with the command, the second recorder controls the first broadcast program reserved for recording from the first portable information terminal of the first user so that it can be shared and used from the second portable information terminal of the second user. Update information.

As a modification, the remote control middleware 30 of the device 3 may have a function of avoiding duplication and collision of remote operation access. For example, the first device receives the first type API command from the first portable information terminal and executes the first operation. Assume that a similar first type API command is received from the second portable information terminal during the execution. In this case, the first device transmits a response indicating that the use of the first operation overlaps with the first portable information terminal.

[Variation (2)-Non-overlapping device range information providing function]
The related service 15 in the modification has a non-overlapping device range information providing function. The remote control service 10 grasps the connection, access status, and usage status of the portable information terminal 2 and the device 3 to the communication network 5 and holds them as usage status information. The related service 15 provides non-overlapping device range information (in other words, non-overlapping device list) based on the usage status information when the user tries to remotely operate any device 3 from the portable information terminal 2. This information is information representing the range of the device 3 that can be accessed from the portable information terminal 2 at that time without overlapping with other portable information terminals 2 and can be remotely operated. The user can check the information on the screen, and can easily know and select the device 3 that can be remotely operated from his / her portable information terminal 2 without duplication. Thereby, it is possible to avoid a collision of access in remote operation.

For example, in FIG. 24, when trying to use the recording reservation of any recorder from the second portable information terminal of the second user, the recording reservation of the second recorder has already been made from the first portable information terminal of the first user. Assume that it is in use. In this case, the second portable information terminal is provided with non-overlapping device range information indicating that the first recorder and the third recorder can be used.

[Variation (3)-Priority order control function]
Further, the related service 15 may have a priority control function. Priorities for a plurality of users, a plurality of portable information terminals 2 and a plurality of devices 3 can be set in advance by user settings. Based on this setting information, use overlap and collision during remote control are avoided. For example, in FIG. 24, the first user's first portable information terminal, the second user's second portable information terminal, and the third user's third portable information terminal are set in the order of

priority

1, 2, 3, and so on. Is done. For example, assume that the first portable information terminal and the second portable information terminal try to remotely operate the first operation of the same first device almost simultaneously. In that case, the related service 15 transmits the first type API command from the first portable information terminal to the first device so that the remote operation from the first portable information terminal is preferentially executed based on the priority order. To do. The related service 15 notifies the second portable information terminal of a response to the effect of overlapping use.

Furthermore, the related service 15 may have a standby control function. In the case of duplication of usage during remote control, the mobile information terminal 2 that has been accessed later is notified of a response so as to temporarily wait, and the remote operation by the previously accessed mobile information terminal 2 has been completed. At the time, the remote operation from the later portable information terminal 2 is started.

[Variation (4)-Usage authority control function]
In the modification, the remote control service 10 and the related service 15 have a use authority control function. As described above, the usage authority can be set for each user, portable information terminal 2, device 3, and content in the user settings. As an example of authority setting, in FIG. 24, the second portable information terminal of the user U2 can use all the operations of the second device and the third device. The third mobile information terminal of the user U3 can use only the second operation group of the third device. The following controls are possible for the usage authority control function.

(1) On the remote control screen 21, an executable object is displayed for an operation with a remote operation authority, and an unexecutable object is displayed for an operation without a remote operation authority. For example, on the remote control screen 21 of the third device in the third portable information terminal of the user U3, only the second action group is displayed as an executable object and can be selected and executed.

(2) The remote control screen 21 is the same as the normal display. In the authority setting state, for example, it is assumed that the operation of the first operation group of the third device is to be remotely operated from the third portable information terminal. A first type API command corresponding to the operation is transmitted to the remote control service 10. The related service 15 refers to the first type API command and confirms the authority. When the related service 15 confirms that there is no authority for the operation, the related service 15 transmits a response indicating that there is no authority.

(3) An example of controlling the authority related to the content is as follows. It is assumed that content playback on the recorder 32a can be remotely operated. User authority can be set for each content to be played. For example, in FIG. 24, the user U2 has the authority to make recording reservations and play back all content. The user U3 has no authority to reserve recording or play back the contents c2, c3 and the like.

When the related service 15 uses the recorder 32a from the user's portable information terminal 2, the related service 15 grasps the use status from the input information on the menu screen and confirms the user's authority. The related service 15 controls through the remote control screen 21 so as to permit the reproduction of the content for which the user has authority, and to prohibit the reproduction of the content for which the user has no authority.

The related service 15 grasps, for example, a playback list and playback content related to playback of content on the recorder 32a. Each reproduction operation is processed using a second type API command. The reproduction list is a list of recorded contents that are permitted to be reproduced by the recorder 32a in accordance with the authority of the user. The playback content is content that is being played back at that time. The related service 15 provides the playlist to the user's portable information terminal 2. The user can reproduce content by selecting from the reproduction list. Thereby, reproduction | regeneration of the content with the apparatus 3 from the portable information terminal 2 of the user who is not authorized can be prevented.

It should be noted that the functions and information management as described above become more difficult as the number of users, portable information terminals 2, devices 3, contents, etc. increases when the device 3 is realized as a conventional technique. On the other hand, the remote control service 10 and the related service 15 can realize the functions and information management as described above with relative ease, and can be operated safely and with high added value for the user.

(Embodiment 3)
A remote control system according to the third embodiment of the present invention will be described with reference to FIGS. The basic configuration of the third embodiment is the same as the configuration of the first embodiment, and hereinafter, the components of the third embodiment that are different from the first embodiment will be described. In the third embodiment, P2P communication is performed on the LAN 4 IP network as the second communication method.

[Remote control system]
FIG. 25 shows the configuration of the remote control system of the third embodiment. In the third embodiment, in the LAN 4, for example, the first LAN 41, the portable information terminal 2A and the device 3 such as the television 31a do not have a function of physically realizing P2P communication such as a Bluetooth communication function. In the third embodiment, the P2P communication corresponding to the second communication method is realized as P2P communication on the LAN 4 IP network. The portable information terminal 2A and the target device 3 each have a function of performing P2P communication on the LAN 4 IP network. In FIG. 25, this P2P communication and P2P connection are shown as “P2P on IP in LAN”. P2P communication over IP in the LAN 4 is realized as P2P communication in the session layer and application layer above the physical layer, MAC layer, and IP layer of the network.

The portable information terminal 2A is wirelessly connected to the gateway 81 and can perform IP communication in the first LAN 41. Each device 3 is connected to the gateway 81 and can perform IP communication in the first LAN 41. The portable information terminal 2 A can perform IP communication with each device 3 via the gateway 81, and can perform P2P communication over IP in the first LAN 41.

As in the first embodiment, as the remote control, the non-P2P communication via the server 1 is performed using the first communication method using the first type API command for the specific operation of the specific device 3. . In the third embodiment, for a specific operation of another specific device 3, the second communication method using the second type API command is used, and P2P communication over IP in the LAN 4 is performed without going through the server 1. Done.

[machine]
In the third embodiment, the device 3 does not have a P2P function like the Bluetooth communication function, but has a function of performing P2P communication over IP in the LAN 4. 5 does not include the P2P communication unit 318. Instead, the IP communication unit 316 has a function of performing P2P communication over IP in the LAN 4. The CPU 311 controls P2P communication over IP in the LAN 4. The IP communication unit 316 establishes a P2P connection over IP in the LAN 4 based on the IP address of the device 3 and the IP address of the partner with the partner mobile information terminal 2, and then the partner mobile information terminal 2 performs P2P communication for exchanging second type API commands and the like. The same applies to the device 3 such as the recorder 32a.

[Communication sequence]
FIG. 26 shows a communication sequence in the third embodiment. FIG. 26 includes steps S51 to S62. In FIG. 26, it is assumed that the menu screen display step S8 of FIG. 8 is the same as the premise, and the device list display step S51 will be described. FIG. 26 includes a process 260 for establishing a P2P connection over IP in the LAN.

In S51, the user selects the device 3 to be remotely operated from the device list displayed on the screen of the portable information terminal 2. The remote control application 20 transmits device selection information to the remote control service 10. In S 52, the remote control service 10 uses the device selection information from the portable information terminal 2 to refer to the registered information 50 for the device 3 selected by the user and the information of the user who has logged in from the portable information terminal 2. Confirm based on. Then, the remote control service 10 notifies the target device 3 of the information of the user and the portable information terminal 2 {information such as user ID, terminal ID, IP address}.

In S53, the remote control middleware 30 of the device 3 confirms the information of the user and the portable information terminal 2 notified from the remote control service 10 of the server 1, and transmits response information to the remote control service 10. The response information is information such as whether to permit remote operation. In S 54, the remote control service 10 transmits the IP address of the target device 3 together with the response information for the portable information terminal 2 based on the response information from the device 3. The response information is information such as whether or not the device selection in S51 is permitted. The IP address of the target device 3 is an IP address registered in advance or an IP address that can be acquired from the gateway 81.

In S55, the remote control application 20 of the portable information terminal 2 transmits a P2P communication request to the device 3 to the IP address of the device 3 notified from the remote control service 10 in S54. The transmission of this request is IP communication. This request is a request for establishing a P2P connection over IP in the LAN 4. In step S56, in response to the received P2P communication request, the device 3 receives information on the counterpart user and the portable information terminal 2 from the remote control service 10 in advance in step S52. Check if it matches. If they match, the device 3 transmits response information indicating that the P2P communication is permitted to the other mobile information terminal 2. The process 260 establishes a P2P connection over IP in the LAN 4. The subsequent steps after S61 are basically the same as described above.

In S61, the remote control screen 21 of the target device 3 is displayed, and the user selects an object corresponding to a desired action. A first type API command or a second type API is generated according to the selected object, and a remote control process according to the API command is performed. In the case of the first type API command, the process 261 of the first communication method is performed. In the case of the second type API command, a process 262 by P2P communication over IP in the LAN 4 is performed as the second communication method.

[Effects]
As described above, in the third embodiment, an effect similar to that of the first embodiment can be obtained with respect to the second communication method. In the third embodiment, the implementation of the Bluetooth communication function of the device 3 can be omitted. Therefore, the system can be realized at a lower cost. When the portable information terminal 2A and the device 3 are physically connected to the same LAN 4 as in the first LAN 41 of FIG. 2, the IP address of the portable information terminal 2 and the IP address of the device 3 are respectively DHCP of the gateway 81. A local IP address is assigned by the server function. A second type API command issued from the portable information terminal 2 (specifically, a packet containing it) has the IP address of the device 3 as a local IP address as a destination. Therefore, the second type API command is delivered from the gateway 81 directly to the target device 3 without being flowed to the Internet 6 by the routing function of the gateway 81. As described above, regarding the P2P communication, even if the IP network of LAN4 is used, the second type API command can be transmitted by the physically shortest path, so that a good response can be realized.

As mentioned above, although this invention was concretely demonstrated based on embodiment, this invention is not limited to embodiment mentioned above, A various change is possible in the range which does not deviate from the summary. It is possible to add, delete, separate, merge, replace, and combine components of the embodiment. The numerical values in the specific examples of the embodiments are examples. In the drawing, some lines are shown as lines connecting components. A part or all of the functions and the like of the embodiments may be realized by hardware such as an integrated circuit, or may be realized by software program processing. Software constituting functions of each device may be stored in the device in advance at the time of product shipment, or may be acquired from an external device via communication after the product is shipped.

DESCRIPTION OF SYMBOLS 1 ... Server, 2 ... Portable information terminal, 3 ... Equipment, 4 ... LAN, 5 ... Communication network, 10 ... Remote control service, 10G ... Remote control screen data, 20 ... Remote control application, 21 ... Remote control screen, 30 ... Remote control middleware, 50 ... registration information, C11, C12 ... first type API command, C2 ... second type API command.

Claims

A user's personal digital assistant connected to a communication network, a device that is one or more controlled devices of the user, and a server device,
The server device provides the portable information terminal with a remote control screen for remote operation of the device;
In the case of selecting the first object corresponding to the instruction of the first operation of the first device among the devices based on the user's input operation on the remote control screen, the portable information terminal is the first Generating a seed command, transmitting the first type command to the server device, and in the case of selecting a second object corresponding to a second operation instruction of the second device, generating a second type command, Send the second type command to the second device by one-to-one communication,
The server device receives the first type command from the portable information terminal, and after confirmation, transmits the first type command to the first device,
The first device receives the first type command from the server device, executes the first operation according to the first type command,
The second device receives the second type command from the portable information terminal by the one-to-one communication, and executes the second operation according to the second type command.
Remote control system.
The remote control system according to claim 1, wherein
The one-to-one communication uses a short-range wireless communication method including Bluetooth,
Remote control system.
The remote control system according to claim 1, wherein
As the device, there are a plurality of devices connected to the LAN,
Of the plurality of devices, the first type command is used for remote control of the first device, and the second type command is used for remote control of the second device.
Remote control system.
The remote control system according to claim 1, wherein
As the operation of the device, it has a plurality of operations,
Of the plurality of operations, the first type command is used for remote control of the first operation, and the second type command is used for remote control of the second operation.
Remote control system.
The remote control system according to claim 1, wherein
As the portable information terminal of the user, it has a plurality of portable information terminals of a plurality of users,
Among the plurality of portable information terminals of the plurality of users, the first type command is used in the case of remote control from the first portable information terminal of the first user, and from the second portable information terminal of the second user. Use the second type command for remote control,
Remote control system.
The remote control system according to claim 1, wherein
The server device, based on an operation of an administrator or the user, corresponds to the user's portable information terminal, the device, the operation, and a group of those units, the first type command or the second Whether to use any of the seed commands is set and stored in the registration information, and the remote control is performed according to the registration information.
Remote control system.
The remote control system according to claim 1, wherein
When the server device is capable of both communication using the first type command and the one-to-one communication using the second type command between the portable information terminal and the device, Preferential use of the one-to-one communication;
Remote control system.
The remote control system according to claim 1, wherein
When the server device is capable of both communication using the first type command and the one-to-one communication using the second type command between the portable information terminal and the device, The communication of the one selected according to the judgment of the situation including the communication situation between the portable information terminal and the device is used.
Remote control system.
The remote control system according to claim 1, wherein
The server device confirms the validity of the operation of the device from the portable information terminal of the user during communication using the first type command, records the history information, and then records the history information. Sending the first type command to the device;
Remote control system.
The remote control system according to claim 1, wherein
The server device provides the portable information terminal with device list information regarding the device that is a candidate for the remote operation from the portable information terminal of the user,
The portable information terminal displays the device list information, transmits information on the device selected based on the operation of the user to the server device,
The server device provides the remote control screen of the selected device and establishes a connection for the one-to-one communication between the portable information terminal and the device when the one-to-one communication is possible. To control the
Remote control system.
The remote control system according to claim 1, wherein
The server device has a remote control service realized by software program processing and a related service, and the related service is transferred to the mobile device based on grasping communication using the first type command by the remote control service. Provided to the information terminal and the device,
The portable information terminal can use the related service based on an operation of the user with respect to an object on a screen.
Remote control system.
The remote control system according to claim 1, wherein
The remote control screen includes a first column in which the first object associated with the first type command is arranged, and a second column in which the second object associated with the second type command is arranged. Including two columns,
Remote control system.
The remote control system according to claim 1, wherein
The device is connected to a LAN;
When the portable information terminal is at a position connected to an external communication network for the LAN, P1P communication over IP via the external communication network is performed as the one-to-one communication.
Remote control system.
The remote control system according to claim 1, wherein
The device is connected to a LAN;
The portable information terminal is connectable to the LAN;
When the portable information terminal is at a position connected to the LAN, P1P communication over IP in the LAN is performed as the one-to-one communication.
Remote control system.
The remote control system according to claim 1, wherein
The server device performs device authentication between the server device and the device when the device is connected to the communication network, and registers the device that has been successfully authenticated as a candidate for remote control. And setting the first type command or the second type command according to the operation of the device,
Remote control system.
The remote control system according to claim 1, wherein
The server device mask-displays the second object on the remote control screen when the one-to-one communication using the second type command is unavailable or when a response delay occurs.
Remote control system.
The remote control system according to claim 1, wherein
The server device provides a remote control service realized by software program processing;
The remote control service performs processing related to remote control including provision of the remote control screen,
The portable information terminal has a remote control application realized by software program processing,
The remote control application communicates with and cooperates with the remote control service and performs processing related to the remote control,
The device has remote control middleware realized by software program processing,
The remote control middleware communicates with the remote control service and the remote control application to perform processing related to the remote control in cooperation with each other.
Remote control system.
A remote control method in a remote control system, comprising: a user's portable information terminal connected to a communication network; a device that is one or more controlled devices of the user; and a server device,
As steps executed in the remote control system,
The server device providing the portable information terminal with a remote control screen for remote operation of the device;
In the case where the portable information terminal selects the first object corresponding to the instruction of the first operation of the first device among the devices based on the input operation of the user on the remote control screen, the first information Generating a seed command, transmitting the first type command to the server device, and in the case of selecting a second object corresponding to a second operation instruction of the second device, generating a second type command, Transmitting the second type command to the second device by one-to-one communication;
The server device receives the first type command from the portable information terminal, and after confirmation, transmits the first type command to the first device;
The first device receiving the first type command from the server device and executing the first operation according to the first type command;
The second device receiving the second type command from the portable information terminal in the one-to-one communication and executing the second operation according to the second type command;
A remote control method.
A portable information terminal in a remote control system, comprising: a user portable information terminal connected to a communication network; a device that is one or more controlled devices of the user; and a server device,
A remote control screen for remote operation of the operation of the device is provided from the server device,
In the case of selection of the first object corresponding to the instruction of the first operation of the first device among the devices based on the input operation of the user on the remote control screen, the first type command is generated, In the case of selecting the second object corresponding to the second operation instruction of the second device by transmitting the first type command to the server device, the second type command is generated, and the second type command is Transmitting to the second device by one-to-one communication;
Mobile information terminal.