JP2008181319A - Apparatus linkage operation system, linkage operation execution method, linkage-operable apparatus and pointing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve accuracy of user identification to perform pairing between apparatuses with intuitive operability when performing linkage operation between the plurality of apparatuses. <P>SOLUTION: In this apparatus linkage operation system, a pointing device 30 common to the plurality of apparatuses is provided with a sensor 33 such as a fingerprint sensor. One apparatus of a first apparatus 10 and a second apparatus 20 notifies the other apparatus about the effect that the one apparatus receives an instruction of the linkage operation and personal identification information detected by the sensor 33 when receiving the instruction of the linkage operation from the pointing device 30. The other apparatus of the first apparatus 10 and the second apparatus 20 performs the linkage operation with the one apparatus subject to reception of the notification of the effect of the reception of the instruction of the linkage operation from the one apparatus, and accordance between the personal identification information notified from the one apparatus and the personal identification information received from the pointing device 30. At least one apparatus can perform user authentication based on the personal identification information. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a device-linked operation system that performs a linked operation between a first device and a second device using a common pointing device for the first device and the second device, and a method and device related thereto. .

  Currently, in offices of companies and public institutions, a plurality of PCs (Personal Computers) and various peripheral devices are connected to each other by a communication network such as a LAN (Local Area Network), and the LAN is further connected to the Internet.

  2. Description of the Related Art In recent years, home networking for exchanging commands and data between PCs and home appliances or between home appliances has been studied not only in PCs but also in homes due to the development of digital home appliances. Already, audio / visual (AV) -related digital home appliances (TVs, PCs, audio systems, HDD recorders, etc.) are linked to each other using technologies such as DLNA (Digital Living Network Alliance) and Bluetooth (registered trademark). It is possible to do. For example, digital contents such as music, photos, and videos can be seamlessly shared between home appliances, PCs, and mobile devices. Specifically, the music stored in the PC can be created and listened with an audio system, the videos stored in the PC can be enjoyed on a large-screen TV, and the video content of the HDD recorder can be played on a PC in another room. It is possible to perform operations such as reproduction.

In Patent Document 1, in a network control system in which a plurality of devices connected via a network operate in cooperation, a user intuitively designates a device to be operated in cooperation without using an operation screen. Techniques that enable this are disclosed.
Japanese Patent Laid-Open No. 2005-204099

  The technology using the above Bluetooth (registered trademark) technology lacks intuitive operability because it does not directly indicate an operation target when performing an operation linked between a plurality of devices. Insufficient flexibility in connection (pairing) between devices.

  The technology using the DLNA requires operation on the screen by the controller, and lacks intuitive operability and flexibility of pairing between devices.

  In addition, in order to perform a coordinated operation between multiple devices, instead of instructing coordination between devices on a single screen, Each device needs to recognize that individual operations on the device belong to the same linkage operation.

  The conventional technology described in Patent Document 1 is superior in terms of intuitive operation since the device is directly identified using an infrared remote controller, but for example, everyone in the family has a common link to the device. There is a problem that privacy within the family is not maintained because the operation can be performed. For example, it may be possible to set an ID or password for each individual in the family. However, this requires a complicated operation such as character input, which impairs good operability.

   Accordingly, the applicant of the present application is a patent application (Japanese Patent Application No. 2006-206656) filed on July 28, 2006, for example, as a device linkage operation system for linking operation of a plurality of home appliances in a home network. We proposed a device-linked operation system that is highly intuitive and can protect personal privacy.

  However, in this prior art, when personal identification information such as a fingerprint is acquired in a fixed manner by each device that performs the linkage operation, it is necessary to provide a sensor for personal identification information for each device. For example, when a fingerprint is detected by a separate sensor of each device, even if the same finger of the same user is applied to the same type of fingerprint sensor, an error may occur in the reading result due to the individual difference of the sensor. As a result, when user authentication is performed for each device, the overall probability of an error increases. Even when the detection results of the sensors in a plurality of devices are collated, an error may occur in the reading results due to individual differences of the sensors. In particular, the type of sensor of each device may be different, especially when the manufacturer of the device is different, even more so.

  The present invention has been made in such a background. The purpose of the present invention is to provide intuitive operability and improve the accuracy of user identification when performing a linkage operation between a plurality of devices. It is an object to provide a device link operation system and a link operation execution method capable of performing pairing, and a device capable of link operation and a pointing device used therefor.

  A device linkage operation system according to the present invention includes a first and second devices that can perform a linkage operation, and a pointing device that is shared by the first and second devices and that can detect personal identification information. Each of the first and second devices includes means for receiving personal identification information of a user detected by the pointing device, and registration for registering link operation information indicating a counterpart device and the type of link operation thereof Means and data communication means for performing data communication with another device, wherein either one of the first and second devices is connected to the other device by the data communication means. The personal identification information received from the device, the fact that the link operation instruction has been received, and the device identification information of its own device are notified. The device identified by the device identification information via the data communication means is registered in accordance with the result of collating the personal identification information received from the communication device with the personal identification information notified from the first device. It is characterized by performing a linked operation.

  In the present invention, a function for detecting personal identification information is attached to the pointing device, and the personal identification information obtained using the same pointing device and the same sensor is used when giving an instruction to each device.

  It is also possible to register personal identification information in each device in advance and perform user authentication by comparing the personal identification information detected by the pointing device with the registered personal identification information when an instruction is given to the device.

  The type of linkage operation between devices may be determined in advance by registering the types of linkage operations that can be executed with other devices in each device, and specifying the linkage operation according to the counterpart device. You may make it identify | isolate by notifying linkage operation information showing the kind of linkage operation between apparatuses each time.

  It is also possible to perform the linkage operation between the first and second devices via the third device.

  Other configurations, operations, and effects of the present invention will be described in detail in the following embodiments.

  According to the present invention, when a linkage operation is performed between a plurality of devices, the pointing device can be used to directly instruct each device, so that the operation is highly intuitive.

  Further, by providing the pointing device with a function of detecting personal identification information, it is not necessary to provide a sensor for each device. In addition, by using the same sensor when instructing each device, the accuracy of user identification can be improved and the pairing between devices can be easily performed.

  If user authentication is performed to confirm that the operator is a registered user for the device when operating each device, privacy and security can be protected when the device is operated.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the drawings.

<First Embodiment>
FIG. 1 is a block diagram showing a schematic configuration of a network system according to the first embodiment of the present invention. As shown in this figure, a plurality of devices 10 (here, device A and device B) are connected by a home network 100 so that they can communicate with each other. A wireless pointing device 30 is shared by the plurality of devices 10.

  The home network 100 as data communication means includes a wired or wireless LAN (Local Area Network), a power line, Bluetooth (registered trademark), and the like and an interface associated therewith.

  The device 10 mainly assumes various home appliances, but may include a personal computer (PC), a portable information terminal, and the like. Various home appliances can include arbitrary devices that can be connected to the network, such as televisions, video recorders, audio devices, intercoms, surveillance cameras, digital cameras, washing machines, refrigerators, and water heaters.

  Each device 10 includes a control unit 11 that performs overall control and necessary processing, a network connection unit 14 for connecting to other devices belonging to the home network 100, and a memory 12 that preferably stores various data in a nonvolatile manner. , A short-range communication unit 13 that performs short-range communication with the pointing device 30 and a device-specific function unit 15 for realizing a function for each device. Although not shown, the control unit 11 is a program storage device such as a microprocessor, a ROM (Read Only Memory) and / or a HDD (Hard Disc Drive), a RAM (Random Access Memory) that provides a temporary storage area, a work area, and the like. ) And is realized by program execution processing.

  The short-range communication that can be used in this embodiment includes electromagnetic coupling, electrostatic coupling, radio waves such as RFID (Radio Frequency IDentification), IrDA (Infrared Data Association), and laser. Examples of such light are as follows.

  The pointing device 30 is a device that communicates with only one device at a time, and gives an instruction to each device 10 or inputs data. Specifically, it includes a short-range communication unit 32 corresponding to the short-range communication unit 13 of the device, a sensor 33 for detecting personal identification information, and a control unit 31 for controlling them. As shown in FIG. 2, it may include a pen-type pointing device 30a, a mouse-type pointing device 30b, a remote controller (hereinafter abbreviated as a remote controller) 30c, and the like. In this example, a fingerprint sensor is assumed as the sensor 33. However, the sensor is not limited to the fingerprint sensor, and may be any sensor capable of detecting personal identification information, such as another biological information sensor, an RFID reader / writer (only a reader may be used).

  In the case of the pen-type pointing device 30a, a laser pointer may be provided at the tip of the pen, and the laser output may be modulated to transmit data such as detected personal identification information. The laser pointer is also useful for the user to reliably recognize the operation target device by spot irradiation with visible light. In the case of detecting the movement of the mouse with infrared using the mouse-type pointing device 30b, an infrared light receiving unit is attached to the mouse pad connected to the device through an interface such as USB, and the device is transmitted to the device through the mouse pad in the form of infrared data. Personal identification information may be transmitted. The interface of the remote controller 30c is not limited to IrDA of a normal remote controller, and any of the above-described short-distance communications can be used.

  Further, instead of the sensor 33 in the pointing device such as the mouse 30b or the remote controller 30c, a short-range communication unit (corresponding to the short-range communication unit 13 in the device) corresponding to the short-range communication unit 32 of the pen 30a is provided. The personal identification information may be transmitted from the pointing device (for example, the pen 30a) having the function to the device via the other pointing device. In this case, the other pointing device may be specific to a specific device.

  FIG. 3 shows a sequence diagram for explaining an operation example of the network system of FIG. Now, a case where the user performs a specific linkage operation between the device A and the device B is taken as an example.

  First, the user performs an operation (link target operation) that is a target of a predetermined link operation on the device A using the pointing device (S11). Since this is a user's pointing device operation on the device A, a broken line arrow is shown from the pointing device to the device A in the figure. At this time, a connection is established with the device A (S12). Personal identification information (for example, a user's fingerprint) is acquired from the sensor 33 and transmitted to the device A via the short-range communication unit 32 (S13). The “association target operation” varies depending on the type of linkage operation, but is, for example, an operation in which a user performs a copy in a copy and paste process described later. In an example of a washing machine and a television set, which will be described later, this is an operation for notifying the end of washing.

  It is assumed that personal identification information of the user is registered in advance in each device. Therefore, the device A checks whether the operator is an appropriate user by checking the registered personal identification information 121 (FIG. 1) with the detected personal identification information (detected personal identification information). User authentication is performed (S14). If the result of user authentication is not OK, an error message or the like is output. If the result of the user authentication is OK, the device A executes the pre-processing for the linkage operation (pre-linkage processing) (S15). The pre-linkage process is a process that is a precondition for performing a predetermined link operation between devices. For example, in the case of copy and paste processing, the character string designated by the copy operation is acquired from the application and stored in a buffer area in the RAM. The device A also notifies other devices by the broadcast (or multicast) that the link operation instruction has been received before or after the link pre-processing (S16). Here, when devices other than the devices A and B exist in the home network 100, the “other devices” include those devices. The contents of this broadcast include personal identification information and link partner information (in this example, device identification information indicating device A). “Device identification information” is information for identifying each device, and is, for example, a device name or an IP address added to each device in the home network. Upon receiving this broadcast, device B (and another device, if present) temporarily stores the received device identification information as a link partner in a predetermined recording area in memory 12 (S17). The presence / absence of the association partner functions as flag information indicating whether or not the association operation is on standby. Of course, independent flag information may be stored in the memory 12 separately.

  When checking whether the users of the devices A and B match, the detected personal identification information is also stored as user identification information in S17. On the basis of the device identification information received from the device A and the instruction of the link operation performed by the device, the device B refers to the link operation / device identification information 122 (FIG. 1) stored in advance in the memory 12. Thus, it is possible to recognize what kind of operation the linkage operation with the device A is.

  Thereafter, the user performs an operation to be linked using the pointing device to the device B (S18). This linkage target operation is, for example, an operation for pasting in the above copy and paste example. As a result, a connection is established between the pointing device 30 and the device B (S19). At this time, the personal identification information is transmitted to the device B via the short-range communication unit 32 (S20). The personal identification information used at this time may be a method of using the personal identification information acquired from the sensor 33 in S13 and stored in the pointing device, or a method of detecting the personal identification information again by the sensor 33. In the former case, it is possible to save time and effort required for user authentication again. According to the latter, it is possible to prevent an operator different from the operator of the device A from operating the device B to perform a linked operation.

  The device B performs user authentication by collating the registered personal identification information with the detected personal identification information (S21). When checking whether the users of the devices A and B match, the user is further checked for identity at this point.

  If the result of user authentication is not OK, an error message or the like is output. If the result of the user authentication is OK, the device B executes the link operation with the device A designated as the link partner (S22, S23). In the copy and paste example, the character string stored in the buffer is transferred from the device A to the device B and passed to the application of the device B. This application performs a process of pasting the character string at a position designated by the pointing device.

  Note that if the operation instructed by the pointing device in the device A is an operation that is not the target of the linkage operation, steps S12 to S14 need not be executed. Further, when a connectionless transfer protocol is used in near field communication, there is no “connection establishment” step of S12 and S19.

  Further, in the process of FIG. 3, when the device A that first receives the operation to be linked in a certain linkage operation broadcasts to another device, the device B stores the linked partner information. Instead, device B does not respond at the time of broadcast from device A, and device B broadcasts to device A the next time it receives an operation to be linked, and based on this, device A detects from device B The personal identification information may be acquired, and the device A may perform the linkage process with the device B.

  FIG. 4 is a sequence diagram for explaining another operation example of the network system of FIG.

  In the operation example of FIG. 3, the personal identification information of the user is registered in advance in each device, and the personal identification information detected when the device is used (at least when the linkage operation is executed) is collated with the registered personal identification information. Thus, user authentication was performed to confirm whether the operator is a proper user. On the other hand, in the operation example of FIG. 4, whether the operator of the device A and the operator of the device B are the same, and whether each operator is an appropriate user for each device. It doesn't matter.

  First, the user performs an operation (link target operation) that is a target of a predetermined link operation on the device A using the pointing device (S31). At this time, a connection is established with the device A (S32). Personal identification information is acquired from the sensor 33 and transmitted to the device A via the short-range communication unit 32 (S33). Therefore, the device A executes pre-processing of the linkage operation (pre-linkage processing) without performing user authentication (S34). The device A also notifies the other device that it has received the instruction for the link operation together with the detected personal identification information by broadcast (or multicast) before or after the link pre-processing (S35). Upon receiving this broadcast, the device B (and another device, if present) temporarily stores the received detected personal identification information in a predetermined recording area in the memory 12 (S36). In addition, the linked partner information is temporarily stored (S37).

  After that, the user performs an association target operation on the device B using the pointing device (S38). As a result, a connection is established between the pointing device 30 and the device B (S39). At this time, personal identification information is newly acquired from the sensor 33 of the pointing device 30, and is transmitted to the device B via the short-range communication unit 32 (S40).

  The device B checks the identity of the user by collating the detected personal identification information stored in S36 with the detected personal identification information acquired from the pointing device 30 (S41). When personal identification information of each user is registered in the device in advance, in addition to this identity check, user authentication may be performed by comparing the detected personal identification information with the registered personal identification information. .

  If the result of the user identity check is not OK, an error message or the like is output. If the result of the user identity check is OK, the device B executes the link operation with the device A designated as the link partner (S42, S43).

In the above operation example, particularly in the operation example of FIG. 4, personal identification information is transferred between devices. When it is considered that it is not appropriate to flow such personal identification information over the network even though it is on the home network 100, as shown in FIG. 5, a device equipped with a random number generator 34 as a pointing device 30a. May be used. The control unit 31 processes the personal identification information detected by the sensor 33 using the random number generated by the random number generation unit 34. Thereby, the session of each cooperation operation | movement can be identified mutually. For example, when the same user uses a plurality of pointing devices at the same time, it is possible to correctly perform a linking operation between devices using the same pointing device. In addition, when the personal identification information is changed from raw data to another data, it is possible to avoid flowing the raw personal identification information on the network. As processing methods using random numbers for personal identification information, the following may be considered.
(1) A random number is added to personal identification information.
(2) The personal identification information is converted according to a predetermined rule (for example, hash).
(3) Encrypt with a random number key. In this case, decryption is not necessary when only identity checking is performed without user authentication. When user authentication is required, each device receives the same random number key from the pointing device, decrypts it locally, and collates it with the registered personal identification information.

  As described above, when the detected personal identification information is newly acquired when the detected personal identification information is notified to the device B (S20, S40), the random number generation unit 34 uses the same random number. The generated random number is held until the linkage operation is completed.

  Further, in the processing of FIG. 4, when the device A that first receives the operation to be linked in a certain linkage operation broadcasts to another device, the device B stores the detected personal identification information and the linked partner information. I made it. Instead, device B does not respond at the time of broadcast from device A, and device B broadcasts to device A the next time it receives an operation to be linked, and based on this, device A detects from device B The personal identification information and the link partner information may be acquired to check the user identity, and the device A may perform the link process with the device B.

<Second Embodiment>
Next, a second embodiment of the present invention will be described. FIG. 6 is a block diagram showing a schematic configuration of a network system according to the second embodiment. Elements that are the same as those shown in FIG. 1 are given the same reference numerals, and redundant descriptions are omitted.

  In the above-described embodiment, each device performs fingerprint authentication independently, determines whether or not the user is an authorized user, and the linkage operation is also performed by the devices communicating with each other. Although configured as described above, a home gateway capable of controlling each device may perform some of these functions. That is, after fingerprint authentication is performed by each device, the linkage operation may be executed under the control of the home gateway 20.

  Therefore, in this embodiment, the home network 100 includes the home gateway 20 as the device C. The home gateway 20 is connected to the Internet 50, and performs control between the Internet communication protocol and the communication protocol in the home network 100, a network connection unit 24 for communication with the device 10, and preferably Has a non-volatile memory 22. The control unit 21 is a part that controls the entire home gateway 20, and includes a program storage device such as a microprocessor, a ROM and / or an HDD, a RAM that provides a temporary storage area and a work area, etc., although not shown. This is realized by program execution processing. The memory 22 sequentially stores personal identification information 221 of a user who uses the home network 100, device linkage operation, and device identification information 222.

  FIG. 7 is a sequence diagram for explaining an operation example of the network system of FIG. Also in this case, a case where the user performs a linking operation between the device A and the device B is taken as an example.

  First, the user performs an operation (link target operation) that is a target of a predetermined link operation with respect to the device A by using a pointing device (S51). At this time, a connection is established with the device A (S52). Personal identification information is acquired from the sensor 33 and transmitted to the device A via the short-range communication unit 32 (S53). Therefore, the device A performs user authentication by comparing the registered personal identification information 221 (FIG. 6) with the detected personal identification information (S54). If the result of user authentication is not OK, an error message or the like is output. If the result of the user authentication is OK, the device A executes a pre-process (pre-link process) of the link operation (S55). The pre-linkage process is a process that is a premise for performing a link operation between devices, and includes at least a process of waiting for pairing with a partner device. The device A also notifies the home gateway, which is the device C, that other device has received the instruction for the link operation before or after the link pre-processing (S56). The contents of this notification include personal identification information and linked partner information (in this example, device identification information indicating device A). In response to this notification, the device C temporarily stores the received information in a predetermined recording area of the memory 22 (S57). In addition, the device C stores the device identification information of the device A as a link partner (S58). The device C refers to the link operation / device identification information 122 (FIG. 6) stored in advance in the memory 12 based on the device identification information received from the device A and the link operation instruction received by the device C. Thus, it is possible to recognize what kind of operation is associated with the next instructed device.

  Thereafter, the user performs an operation to be linked using the pointing device with respect to the device B (S59). As a result, a connection is established between the pointing device 30 and the device B (S60). At this time, the personal identification information is transmitted to the device B via the short-range communication unit 32 (S61). As described above, the personal identification information used at this time is acquired from the sensor 33 in S53 and stored in the pointing device, and the personal identification information is detected again by the sensor 33. There can be.

  The device B performs user authentication by comparing the registered personal identification information with the detected personal identification information (S62). If the result of user authentication is not OK, an error message or the like is output. If the user authentication result is OK, the device B performs a predetermined pre-linkage process (S63). The device B transmits the detected personal identification information received in S61 and the device identification information of the device B to the device C (S64).

  The device C checks the identity of the user by collating the detected personal identification information received in S64 with the detected personal identification information stored in S57 (S65). If the identity of the user is not confirmed, predetermined error processing is performed. If the identity is confirmed, a linkage operation execution instruction is output to each of the devices A and B (S66, S67). In response to this instruction, device A and device B perform the associated operation (S68, S69).

  FIG. 8 is a sequence diagram for explaining another operation example of the network system of FIG.

  In the operation example of FIG. 7, personal identification information of the user is registered in advance in each device, and the personal identification information detected when the device is used (at least when the linkage operation is executed) is collated with the registered personal identification information. Thus, user authentication was performed to confirm whether the operator is a proper user. On the other hand, in the operation example of FIG. 8, whether or not each operator is an appropriate user for each device only by confirming that the operator of device A and the operator of device B are the same. It doesn't matter.

  First, the user performs an operation (link target operation) that is a target of a predetermined link operation on the device A using the pointing device (S71). At this time, a connection is established with the device A (S72). The detected personal identification information is acquired from the sensor 33 and transmitted to the device A via the short-range communication unit 32 (S73). Therefore, the device A executes the pre-linkage process without performing user authentication (S74). The device A also notifies the other device that it has received the link operation instruction before or after the link pre-processing, together with the detected personal identification information by broadcast (or multicast) (S75). Upon receiving this notification, the device C temporarily stores the received detected personal identification information in a predetermined recording area in the memory 22 (S76). In addition, the linked partner information is temporarily stored (S77).

  Thereafter, the user performs an operation to be linked using the pointing device to the device B (S78). As a result, a connection is established between the pointing device 30 and the device B (S79). At this time, personal identification information is newly acquired from the sensor 33 of the pointing device 30, and is transmitted to the device B via the short-range communication unit 32 (S80).

  Device B performs the necessary pre-association processing (S81), and transmits the detected personal identification information received in S80 and the device identification information of device B to device C (S82).

  The device C checks the identity of the user by collating the detected personal identification information received in S82 with the detected personal identification information stored in S76 (S83). If the identity of the user is not confirmed, predetermined error processing is performed. When the identity is confirmed, a linkage operation execution instruction is output to each of the devices A and B (S84, S85). In response to this instruction, device A and device B perform the associated operation (S86, S87). Similarly to the above, when personal identification information of each user is registered in the device in advance, in addition to this identity check, user authentication is performed by comparing the detected personal identification information with the registered personal identification information. It may be.

  FIG. 9 shows a device 10 a including a touch panel 16 and a display unit 17 as the device specific function unit 15. The system configuration may be either the configuration shown in FIG. 1 or FIG. An example of performing a copy and paste operation as an example of a linkage operation between the two devices using the device 10a as the device A and another device of the same type as the device B will be described with reference to FIG.

  FIG. 10A shows a state of the display screen when the user performs a copy operation on an arbitrary character string on the display screen with the pointing device 30a on the touch panel of the device A. With this operation, the copied character string is stored in the memory of the device A. In addition, the device B or the device C stores the link operation information indicating that the instruction of the link operation is performed or the type of the link operation, and the device identification information of the device A. Next, when the user performs a paste operation on the device B, it is recognized that the user has already performed a copy operation corresponding to the paste operation in the device A. As a result, the copied character string data is transferred from the device A to the device B, and the paste operation is executed in the device B. Corresponding linkage operations are executed on condition that the user matches between device A and device B.

  Further, by using the pointing device 30a that uses random numbers as described in FIG. 5, even if the same user is used, if the pointing devices used for the device A and the device B are different, the linkage operation is performed. Absent. Furthermore, by making the random numbers the same only within the same session, it is possible to prevent the linkage operation from being performed even with the same pointing device if the sessions are different. Here, the session corresponds to a period from when the first operation for the linking operation is performed on the device A to when the second operation for the linking operation is performed on the device B. . However, as described above, when a predetermined time elapses after the first operation on the device A is performed and before the second operation on the other device is performed, it is determined that a timeout has occurred and the session is performed. It is good also as an end. In this case, the device waiting for the second operation deletes or invalidates the flag information.

  11, 12, and 13 are diagrams for explaining another specific example of the linkage operation.

  In any case, a description will be given of a linkage operation in which devices such as devices in the home network 100 establish and execute a secure connection. Moreover, although the example using the remote control 30c mentioned above is shown as the pointing device 30, it is not restricted to this, The arbitrary pointing devices mentioned above can be used.

  Fig.11 (a) has shown the cooperation operation | movement between the washing machine 10b and the television apparatus (TV) 10c.

  For example, the cooperative operation between the washing machine 10b and the television apparatus 10c is notified from the washing machine 10b to the television apparatus 10c at the end of the washing process by the washing machine 10b, and when the television apparatus receives the notification, for example, The operation is to display a message such as “Laundry is complete”. In this case, the pre-processing of the linkage operation is a process of waiting for a washing end notification reservation process in the washing machine and a pairing with the destination device of the notification destination. Usually, the washing machine is also provided with a function for generating an end alarm sound, but there may be a case where the user cannot hear the alarm sound of the washing machine well in a living room or the like. This linkage operation is useful in such a case. In addition to the display on the screen, a predetermined alarm sound or sound on the television device may be output. The user can receive a laundry end notification at a desired location by designating a device at the location where he / she wants to receive the notification.

  Although not shown, a linking operation between the washing machine 10b and the radio device or the audio device may be used.

  FIG. 11B shows a linking operation between the monitoring camera 10d and the television device 10c. As this linkage operation, for example, a real-time captured image of the monitoring camera 10d as the device A is transferred to, for example, the television device 10c as the device B in the living room or own room, and displayed on the screen. . Thereby, the picked-up image of the monitoring camera 10d can be monitored by the television apparatus in an arbitrary room. The preprocessing in this case is processing for waiting for pairing with the counterpart device to which the monitoring data is transferred in the monitoring camera 10d. The end of the transfer is not particularly limited, but is based on an end instruction to the monitoring camera 10d by a remote controller, an automatic transfer end after a predetermined time, or the like.

  FIG. 12A shows a linking operation between the facsimile machine (FAX) 10g and the television apparatus 10c. As this linkage operation, for example, the received image data of FAX 10g as the device A is transferred to the television device designated as the device B and displayed on the screen. Accordingly, it is possible to perform FAX reception and confirmation of received images with a television apparatus in an arbitrary room. The preprocessing in this case is processing for waiting for pairing with the transfer destination of received data in the FAX 10g. It should be noted that printing output in the FAX 10g can be performed separately.

  FIG. 12B shows a linking operation between the interphone 10h and the telephone 10i. As this linking operation, for example, when a visitor accesses the slave unit (not shown) of the interphone 10h that is the device A, the interphone 10h and the telephone 10i that is the device B are connected via the home network 100. Thus, voice data is exchanged between the two. The pre-processing in this case is processing for waiting for pairing with the device B that is designated to exchange audio data later in the intercom.

  Here, an interphone with a monitor screen is taken as an example. Therefore, when a TV device having a videophone function is designated as the device B instead of a telephone, an image of the monitor screen is displayed on the screen of the TV device and the interphone The telephone device can make a call with the other handset.

  FIG. 13 shows a linking operation between the digital camera 10e and the television device 10c or the printer 10f. As this linkage operation, for example, a captured image of the digital camera 10e, which is the device A, is displayed on the screen of the television device 10c designated as the device B, or is displayed on the printer 10f also designated as the device B. It can be printed out. The preprocessing in this case is a process of waiting for pairing with the counterpart device to which the digital camera 10e transfers the shooting data.

  An operation example in the case where the linkage operation includes a plurality of operations that can be selected by the user will be described using an example of an intercom. The linkage operation is selected from the device A via the home gateway, for example, using the user's mobile phone terminal.

  Note that the user of the home network 100 and the owner of the mobile phone terminal, for example, accesses the homepage of the device manufacturer site from the mobile phone terminal via the Internet 50, performs a predetermined operation, and performs the device control software. The wear can be downloaded to the mobile phone terminal. Downloading and updating may be executed automatically.

  Specifically, when the user accesses the interphone 10 h with a pointing device, the information is sent to the mobile phone terminal via the home gateway 20. The mobile phone terminal launches a device control application and displays a link operation selection screen (not shown). On this screen, a plurality of linkage operation options, for example, “transfer to this mobile phone”, “transfer by voice mail”, and “pointing transfer destination” are presented. When the user selects any of these options, processing corresponding to the selection is executed.

  For example, when “transfer to this mobile phone” is selected, processing is performed so that a call can be made between the intercom 10 h and the mobile phone terminal. When “forward by voice mail” is selected, the voice input from the intercom 10h is processed so as to be transmitted as voice mail to the mobile phone terminal. In this case, the interphone 10h or the like may be provided with a guidance function for a visitor to input voice mail to the interphone 10h. When “pointing to transfer destination” is selected and then pointing to the fixed telephone 10i using the sensor 33 as a transfer destination, for example, the fixed telephone 10i is set as a response device for the interphone 10h.

  As described above, according to the present embodiment, when performing a linkage operation between a plurality of devices, it is possible to directly instruct each device with a pointing device. Excellent in properties. Further, by providing the pointing device with a sensor for detecting personal identification information, it is not necessary to provide a sensor for each device individually. In addition, by using the same sensor when instructing each device, the accuracy of user identification can be improved and the pairing between devices can be easily performed. Further, when performing user authentication for confirming that the operator is a registered user for the device when operating each device, privacy and security can be protected when the device is operated.

  In the above example, the remote controller 30c is used as the pointing device. However, the pointing device is not limited to the remote controller 30c, and may be a pen-type device, a mouse-type device, or any other pointing device. . In any of the pointing devices, the user can intuitively specify the device by causing the pointing device to approach or point to the device to be operated.

  On the basis of the device identification information received from the device A and the instruction of the link operation performed by the device, the device B refers to the link operation / device identification information 122 (FIG. 1) stored in advance in the memory 12. Thus, the operation of recognizing the link operation with the device A is recognized. However, if the device A notifies the other device of the identification information of the link operation during broadcasting, the link operation is performed. It is not necessary to store the device identification information 122 in each device in advance. Even in the case where such notification is made, the link operation / device identification information 122 may be stored in advance in each device for confirmation.

  The pen 30a, which is a pen-type pointing device shown in FIG. 2 (a), shows an example in which the sensor 33 is provided on the head of the pen. However, the position of the sensor 33 is not limited to this. For example, FIG. 14 shows an example in which the sensor 33 is arranged as a pen-type pointing device at a position where the tip of a finger (for example, an index finger) contacts when the user holds the pen. By doing so, it is also possible to perform the instruction for the linkage operation and the acquisition of the personal identification information in parallel using the pen.

  2B shows the wireless mouse 30b including the sensor 33. As shown in FIGS. 15A and 15B, an example in which the wired mouse 30e and the pen 30a are used in combination is shown. . The mouse 30e and the pen 30a have means for performing data communication (short-range communication) in a non-contact manner. Further, in the example shown in the figure, a hole 35 that can be held by the mouse 30e by inserting the pen 30a is provided in a part of the body of the mouse 30e, and the pen 30a is inserted from the pen 30a through the mouse 30e. Information is sent to the device. Although not specifically shown, the mouse 30e is provided with the short-range communication unit 13 in the device 10 and an interface for transmitting information received from the pen 30a to the device. The remote control 30c may be a mobile phone terminal having a remote control function.

  The preferred embodiments of the present invention have been described above, but various modifications and changes other than those mentioned above can be made.

  For example, the identity of the user instructed to each of the first and second devices is checked by confirming the coincidence of the personal identification information obtained from the pointing device, but only one user is checked. In the case of registration, only user authentication in each device is required, and there is no need to separately check the identity of both devices.

1 is a block diagram showing a schematic configuration of a network system according to a first embodiment of the present invention. It is a figure which shows the some example of the pointing device to which this invention is applied. It is a sequence diagram for demonstrating the operation example of the network system of FIG. FIG. 6 is a sequence diagram for explaining another example of operation of the network system of FIG. 1. It is a block diagram which shows schematic structure of the network system using the pointing device provided with the random number generator. It is a block diagram which shows schematic structure of the network system concerning the 2nd Embodiment of this invention. It is a sequence diagram for demonstrating the operation example of the network system of FIG. FIG. 7 is a sequence diagram for explaining another operation example of the network system of FIG. 6. It is a figure which shows the apparatus provided with the touchscreen and the display part as an apparatus intrinsic | native function part in the apparatus in embodiment of this invention. It is a figure which shows the copy and paste operation | movement as an example of the cooperation operation | movement between the apparatuses in embodiment of this invention. It is a figure for demonstrating the other specific example of the cooperation operation | movement between the apparatuses in embodiment of this invention. It is a figure for demonstrating the other specific example of the cooperation operation | movement between the apparatuses in embodiment of this invention. It is a figure for demonstrating another specific example of the cooperation operation | movement between the apparatuses in embodiment of this invention. It is a figure which shows the modification of the pen-type pointing device in embodiment of this invention. It is a figure which shows the example used combining the wired mouse | mouth and pen in embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10,10a ... Apparatus, 10b ... Washing machine, 10c ... Television apparatus, 10d ... Surveillance camera, 10e ... Digital camera, 10f ... Printer, 10h ... Interphone, 10i ... Fixed telephone, 11 ... Control part, 12 ... Memory, 13 ... Short-range communication unit, 14 ... network connection unit, 15 ... device specific function unit, 16 ... touch panel, 17 ... display unit, 20 ... home gateway, 21 ... control unit, 22 ... memory, 24 ... network connection unit, 30 ... pointing Device 30a ... Pen 30b ... Mouse 30c ... Remote control 30e ... Mouse 31 ... Control unit 32 ... Short-range communication unit 33 ... Sensor 34 ... Random number generator 50 ... Internet 100 ... Home network 121 ... personal identification information, 122 ... linkage operation / device identification information

Claims (29)

A first device and a second device capable of cooperating with each other;
A pointing device shared with the first and second devices and capable of detecting personal identification information;
Each of the first and second devices is
Means for receiving personal identification information of the user detected by the pointing device;
Registration means for registering linkage operation information indicating the type of counterpart device and its linkage operation;
A data communication means for performing data communication with other devices,
One of the first device and the second device receives the personal identification information received from the pointing device from the pointing device by the data communication means, the fact that it has received a link operation instruction, and its own device identification. The other device that received the notification receives the personal identification information received from the pointing device, and the data communication is performed according to a result of collating the personal identification information received from the one device with the personal identification information notified from the one device. A device linkage operation system that performs the registered linkage operation with the device indicated by the device identification information via a means. A first device and a second device capable of cooperating with each other;
A pointing device shared with the first and second devices and capable of detecting personal identification information;
Each of the first and second devices is
Means for receiving personal identification information of the user detected by the pointing device;
A data communication means for performing data communication with other devices,
One of the first device and the second device is connected to the other device by the data communication means, the personal identification information received from the pointing device, the link operation information indicating the type of link operation, and its own device The other device that has received the notification is notified via the data communication means according to the result of collating the personal identification information from the pointing device with the personal identification information notified from the one device. A device linkage operation system that performs the linkage operation indicated by the linkage operation information with the device indicated by the device identification information.   The at least one of the first and second devices collates the personal identification information received from the pointing device with the personal identification information registered in advance, and uses the match as a condition for executing the linkage operation. The device linkage operation system described. A first device and a second device capable of cooperating with each other;
A pointing device shared with the first and second devices and capable of detecting personal identification information;
Each of the first and second devices is
Means for receiving personal identification information of the user detected by the pointing device;
Registration means for registering personal identification information and registering linked operation information indicating the type of linked device and its linked operation;
Collating means for collating newly detected personal identification information with the registered personal identification information;
A data communication means for performing data communication with other devices,
Either one of the first device and the second device notifies the other device of the fact that it has received the instruction of the linked operation received from the pointing device and its own device identification information by the data communication means. The other device that has received this notification is connected to the device indicated by the device identification information via the data communication means according to the result of collation of the personal identification information against the personal identification information received from the pointing device. A device linkage operation system characterized by performing a linkage operation. A first device and a second device capable of cooperating with each other;
A pointing device shared with the first and second devices and capable of detecting personal identification information;
Each of the first and second devices is
Means for receiving personal identification information of the user detected by the pointing device;
A registration means for registering personal identification information;
Collating means for collating newly detected personal identification information with the registered personal identification information;
A data communication means for performing data communication with other devices,
One of the first and second devices is linked to indicate that the data communication means has received a link operation instruction received from the pointing device to the other device and indicates the type of linkage operation. Information and its own device identification information, and the other device that has received this notification receives the notification through the data communication means according to the result of collation of the personal identification information with respect to the personal identification information received from the pointing device. A device linkage operation system that performs the linkage operation indicated by the linkage operation information with the device indicated by the device identification information. A first device and a second device capable of cooperating with each other;
A third device connected to the first and second devices by the data communication means;
A pointing device shared with the first and second devices and capable of detecting personal identification information;
Each of the first and second devices is
Means for receiving personal identification information of the user detected by the pointing device;
Means for transmitting to the third device that the detected personal identification information and the instruction of the linkage operation have been received;
Data communication means for data communication with other devices;
In response to an instruction from the third device, a means for performing a linkage operation with another device via the data communication means,
The third device is:
Registration means for registering linkage operation information indicating the type of linkage operation between devices;
Means for determining whether or not the personal identification information matches when receiving the personal identification information and the instruction of the link operation from the first and second devices, respectively;
A device linkage operation system comprising: means for instructing the first and second devices to execute the registered linkage operation according to the determination result. A first device and a second device capable of cooperating with each other;
A third device connected to the first and second devices by the data communication means;
A pointing device shared with the first and second devices and capable of detecting personal identification information;
Each of the first and second devices is
Means for receiving personal identification information of the user detected by the pointing device;
Means for transmitting to the third device linkage operation information indicating that the detected personal identification information and link operation instruction have been received and the type of link operation between the devices;
Data communication means for data communication with other devices;
In response to an instruction from the third device, a means for performing a linkage operation with another device via the data communication means,
The third device is:
Means for determining whether or not the personal identification information matches when receiving the personal identification information and the instruction of the link operation from the first and second devices, respectively;
A device linkage operation system comprising: means for instructing the first and second devices to execute the linkage operation indicated by the linkage operation information according to the determination result.   The at least one of the first and second devices collates the personal identification information received from the pointing device with the personal identification information registered in advance, and uses the match as a condition for executing the linkage operation. The device linkage operation system described. A first device and a second device capable of cooperating with each other;
A third device connected to the first and second devices by the data communication means;
A pointing device shared with the first and second devices and capable of detecting personal identification information;
Each of the first and second devices is
Means for receiving personal identification information of the user detected by the pointing device;
Registration means for registering personal identification information and registering linked operation information indicating the type of linked device and its linked operation;
Collating means for collating newly detected personal identification information with the registered personal identification information;
Means for transmitting to the third device that the detected personal identification information and the instruction of the linkage operation have been received as a result of the verification;
Data communication means for data communication with other devices;
In response to an instruction from the third device, a means for performing a linkage operation with another device via the data communication means,
The third device is:
Registration means for registering linkage operation information indicating the type of linkage operation between devices;
Means for determining whether or not the personal identification information matches when receiving the personal identification information and the instruction of the link operation from the first and second devices, respectively;
A device linkage operation system comprising: means for instructing the first and second devices to execute the registered linkage operation according to the determination result. A first device and a second device capable of cooperating with each other;
A third device connected to the first and second devices by the data communication means;
A pointing device shared with the first and second devices and capable of detecting personal identification information;
Each of the first and second devices is
Means for receiving personal identification information of the user detected by the pointing device;
A registration means for registering personal identification information;
Collating means for collating newly detected personal identification information with the registered personal identification information;
Means for transmitting, to the third device, linkage operation information indicating that the detected personal identification information and linkage operation instruction have been received as a result of the verification and indicating the type of linkage operation;
Data communication means for data communication with other devices;
In response to an instruction from the third device, a means for performing a linkage operation with another device via the data communication means,
The third device is:
Registration means for registering linkage operation information indicating the type of linkage operation between devices;
Means for determining whether or not the personal identification information matches when receiving the personal identification information and the instruction of the link operation from the first and second devices, respectively;
A device linkage operation system comprising: means for instructing the first and second devices to execute the linkage operation indicated by the linkage operation information according to the determination result. A linked operation execution method for performing a linked operation between the first and second devices using a common pointing device for the first and second devices,
When one of the first device and the second device receives the instruction of the linked operation determined in advance by the pointing device, the fact that the linked operation instruction has been received and its own device identification information The step of notifying
When the other device receives an instruction for linkage operation together with the predetermined personal identification information from the pointing device, whether or not the personal identification information and the personal identification information notified from one device match. A determining step;
On the condition that this personal identification information matches, the other device performs the predetermined linkage operation with one device indicated by the device identification information;
A linked operation execution method comprising: A linked operation execution method for performing a linked operation between the first and second devices using a common pointing device for the first and second devices,
When one of the first device and the second device receives a link operation instruction from the pointing device, it indicates that the link operation instruction has been received, link operation information indicating the type of link operation, and its own Notifying device identification information to other devices;
When the other device receives the link operation instruction indicated by the link operation information together with the personal identification information from the pointing device, does this personal identification information match the personal identification information notified from the one device? Determining whether or not,
On the condition that this personal identification information matches, the other device performs the linkage operation indicated by the linkage operation information with one device indicated by the device identification information;
A linked operation execution method comprising:   The at least one of the first and second devices collates the personal identification information received from the pointing device with the personal identification information registered in advance, and uses the match as a condition for executing the linkage operation. The linked operation execution method described. A linked operation execution method for performing processing linked between the first and second devices using a common pointing device for the first and second devices,
When one of the first device and the second device receives an instruction of a linked operation determined in advance by the pointing device, performs user authentication based on the personal identification information of the user received from the pointing device; ,
One device notifying the other device of the fact that the user authentication result is OK and receiving the instruction of the link operation and its own device identification information;
A step of performing user authentication based on the personal identification information of the user received from the pointing device when the other device receives the predetermined linkage operation instruction from the pointing device;
The other device performs the predetermined linkage operation with one device on the condition that the result of the user authentication is OK;
A linked operation execution method comprising: A linked operation execution method for performing a linked operation between the first and second devices using a common pointing device for the first and second devices,
When any one of the first and second devices receives an instruction for a linkage operation from the pointing device, the user authentication is performed based on the personal identification information of the user received from the pointing device;
One device notifying the other device of the link operation information indicating the type of the link operation and its own device identification information on the condition that the result of the user authentication is OK;
When the other device is instructed by the pointing device to perform the linkage operation indicated by the linkage operation information, performing user authentication based on the personal identification information received from the pointing device;
The other device performs the linkage operation indicated by the linkage operation information with the one device indicated by the device identification information on the condition that the result of the user authentication is OK;
A linked operation execution method comprising: A linked operation execution method for performing processing linked between the first and second devices using a common pointing device for the first and second devices,
A step of notifying the third device that the first device has received the user's personal identification information and the link operation instruction received from the pointing device when the pointing device receives a link operation instruction determined in advance by the pointing device; When,
When the second device receives the predetermined link operation instruction from the pointing device, the second device notifies the third device that the user's personal identification information received from the pointing device and the link operation instruction have been received. Steps,
A step in which a third device collates personal identification information received from the first and second devices;
A step in which the third device instructs the first and second devices to execute the linkage operation according to the result of the verification;
A step in which the first and second devices execute the predetermined linkage operation in response to an instruction from the third device;
A linked operation execution method comprising: A linked operation execution method for performing processing linked between the first and second devices using a common pointing device for the first and second devices,
When the first device receives a link operation instruction from the pointing device, the personal identification information of the user received from the pointing device, the link operation information indicating that the link operation instruction has been received, A step of notifying the device of
When the second device receives a link operation instruction from the pointing device, the user identification information received from the pointing device, the link operation information indicating that the link operation is received, and the link operation information indicating the type of link operation are A step of notifying the device of
A step in which a third device collates personal identification information received from the first and second devices;
In response to the result of this collation, the third device instructs the first and second devices to execute the linkage operation indicated by the linkage operation information;
The first and second devices executing the linkage operation in response to an instruction from the third device;
A linked operation execution method comprising:   The at least one of the first device and the second device collates the personal identification information received from the pointing device with the personal identification information registered in advance, and uses the match as a condition for executing the linkage operation. The linked operation execution method described. A linked operation execution method for performing processing linked between the first and second devices using a common pointing device for the first and second devices,
A step of performing user authentication based on the personal identification information of the user received from the pointing device when the first device receives an instruction of a predetermined linked operation by the pointing device;
A step of notifying the third device of the fact that the first device has received the instruction of the link operation and its own device identification information on condition that the result of the user authentication is OK;
A step of performing user authentication based on the personal identification information of the user received from the pointing device when the second device receives the predetermined linkage operation instruction from the pointing device;
A step of notifying the third device of the fact that the second device has received the instruction of the link operation and its own device identification information on condition that the result of the user authentication is OK;
A step in which the third device instructs the first and second devices to execute the linkage operation;
A step in which the first and second devices execute the predetermined linkage operation in response to an instruction from the third device;
A linked operation execution method comprising: A linked operation execution method for performing processing linked between the first and second devices using a common pointing device for the first and second devices,
A step of performing user authentication based on the personal identification information of the user received from the pointing device when the first device receives the instruction of the linkage operation from the pointing device;
On the condition that the result of the user authentication is OK, the first device receives the link operation instruction, the link operation information indicating the type of link operation, and its own device identification information in the third device. A notification step;
A step of performing user authentication based on the personal identification information of the user received from the pointing device when the second device receives the instruction of the linkage operation from the pointing device;
On the condition that the result of the user authentication is OK, the second device receives the link operation instruction, the link operation information indicating the type of link operation, and its own device identification information to the third device. A notification step;
A step in which the third device instructs the first and second devices to execute the linkage operation indicated by the linkage operation information;
The first and second devices executing the linkage operation in response to an instruction from the third device;
A linked operation execution method comprising:   21. The linked operation execution method according to claim 11, wherein the pointing device transmits the detected personal identification information processed by using a random number to the first and second devices.   21. The personal identification information of a user received by the second device from a pointing device is that in which the pointing device holds the personal identification information transmitted to the first device. The linked operation execution method described.   21. The linked operation execution method according to claim 11, wherein the personal identification information of the user received from the pointing device by the second device is newly acquired by the pointing device. A device capable of linking operation with other devices,
Means for receiving personal identification information of a user detected by a pointing device shared by a plurality of devices;
A registration means for registering personal identification information and registering linked operation information indicating the counterpart device and its linked operation;
Collating means for collating newly detected personal identification information with the registered personal identification information;
A data communication means for notifying that the instruction of the linkage operation has been performed and performing data communication with other devices;
Based on the collation result of the personal identification information and the registered link operation information, a unit that performs link operation with another device via the data communication unit;
A device capable of linking operation. A device capable of linking operation with other devices,
Means for receiving personal identification information of a user detected by a pointing device shared by a plurality of devices;
A registration means for registering personal identification information;
Collation means for collating the personal identification information received from the pointing device with the registered personal identification information;
Data communication means for notifying other devices of link operation information indicating the type of link operation instructed, and performing data communication with other devices,
Based on the collation operation information notified from the result of collation of the personal identification information and the other device, means for performing a linking operation with the other device via the data communication unit;
A device capable of linking operation. A device capable of linking operation with other devices,
Means for receiving personal identification information of a user detected by a pointing device shared by a plurality of devices;
A registration means for registering link device operation information indicating the counterpart device and its link operation;
A data communication means for notifying the other device of the fact that the instruction of the linkage operation has been performed and the personal identification information, and for performing data communication with the other device;
Means for determining whether or not the personal identification information notified from the other device matches the personal identification information received from the pointing device;
Means for performing the registered linkage operation with another device via the data communication means on condition that the personal identification information matches;
A device capable of linking operation. A device capable of linking operation with other devices,
Means for receiving personal identification information of a user detected by a pointing device shared by a plurality of devices;
A data communication means for notifying other devices of the link operation information indicating the link operation and personal identification information to the effect that the link operation has been instructed, and data communication with other devices;
Means for determining whether or not the personal identification information notified from the other device matches the personal identification information received from the pointing device;
Means for performing a linkage operation indicated by the notified linkage operation information with another device via the data communication means, on condition that the personal identification information matches;
A device capable of linking operation. A sensor for detecting the user's personal identification information;
Near field communication means for transmitting the detected personal identification information to an external device;
Means for outputting instruction information to the outside,
A pointing device that can be shared by a plurality of devices.   The pointing device according to claim 28, wherein the pointing device is one of a pen, a mouse, and a remote controller.

Images