JP2011010222A - Remote control system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a remote control system capable of simultaneously controlling control units to be operated from a plurality of remote control terminals by two-way communication.SOLUTION: In a portable phone 3, an operation request is notified to a main control unit 300 at time T1. The main control unit 300 acquires a Bluetooth dongle ID and application ID at time T2, and converts an operation request message in which the identification ID, the application ID, and an operation code are described to a Bluetooth SPP packet for transmission to a computer terminal 1 at time T3. A communication management section 101 receives an SPP packet in the computer terminal 1 for transfer to a main control unit 100 at time T5. The main control unit 100 extracts the identification ID, the application ID, an operation code, and the like for transfer to an operation conversion unit 103 at time T6. The operation conversion unit 103 reproduces a keyboard operation based on the operation code for transfer to the display update section 104 at time T7. The display update section 104 updates the display contents based on the keyboard operation.

Description

  The present invention relates to a remote control system that receives remote control signals transmitted from a plurality of wireless remote control terminals and processes them simultaneously or in parallel.

  If a single computer can be remotely operated by a plurality of users at the same time using a remote control terminal, a plurality of users can simultaneously use one or a plurality of applications running on the computer. For example, when a car racing game program is executed on a computer connected to a large screen monitor, a wireless remote control terminal is assigned to each of a plurality of racing vehicles, and each user uses a wireless remote control terminal to play a racing vehicle. Can be operated freely, it is possible to develop a game full of realism by competing with many users.

  At this time, if a wireless remote control terminal is used, each user can perform an operation at a position away from the computer, so that simultaneous operation by a large number of users is facilitated. However, the conventional infrared remote control that is widely used for remote controls for mobile phones and home appliances is not suitable for a plurality of simultaneous use because pulse signals transmitted from each remote control interfere with each other. On the other hand, a radio remote controller can be used simultaneously, but since the penetration rate is low, a dedicated remote controller must be prepared separately.

  On the other hand, Bluetooth has spread as a short-range wireless communication system, and Bluetooth has been implemented in many mobile phones. Therefore, if the wireless remote control is a Bluetooth system, many users can use their own mobile phone as a remote control terminal.

  In Patent Document 1, a playback device transmits resume information to a remote control device side using a Bluetooth serial port profile (SPP), and the remote control device transmits the resume information to the playback device at the next playback. A technique is disclosed that makes it unnecessary to manage resume information for each user in a playback device by jumping and playing back.

  In Patent Document 2, when a wristwatch type terminal device transmits a camera shutter pressing command to a mobile phone device using a Bluetooth hands-free profile (HSF), the camera of the mobile phone device executes imaging, and after imaging is completed. When the wristwatch terminal device requests the mobile phone device to acquire a captured image, the mobile phone device uses the Bluetooth image profile (BIP) to transfer image data to the wristwatch terminal and display it on the screen of the wristwatch terminal. Technology is disclosed.

  In Patent Document 3, the information terminal controls the imaging operation of the imaging terminal using a Bluetooth serial port profile (SPP), and before the imaging terminal transfers image information with a large amount of data to the information terminal, the captured image The related information is automatically transferred to the information terminal, and when the information terminal requests the imaging terminal to acquire a captured image after the imaging is completed, the imaging terminal can acquire the Bluetooth image profile (BIP) or the Bluetooth object exchange profile (OPP). A technique is disclosed in which captured image data is transferred to an information terminal using an image and displayed on the screen of the information terminal.

  In Patent Document 4, when the video display device stores in advance a remote control transmission code of a plurality of remote controllers and a divided screen area in association with each other and receives the remote control transmission code together with the key operation key code from each remote control device, the screen is displayed. A technique is disclosed in which a user divides a plurality of screens and assigns a divided screen area to each user as an operation target.

JP 2009-43309 A JP 2009-118474 A JP 2007-19665 A JP 2009-27485 A

  In Patent Document 1, one-to-one pairing can be realized between the remote control device and the playback device, but the playback device cannot be controlled simultaneously from a plurality of remote control devices. In this method, Bluetooth SPP is used, and resume information can be transmitted and received bidirectionally between the remote control device and the playback device. However, since the amount of data transmitted once by SPP is limited to 1024 bytes, the capacity is large. Not suitable for data transfer.

  In Patent Document 2, one-to-one pairing can be realized between a wristwatch-type terminal device and a camera-equipped mobile phone device, but a camera-equipped mobile phone device cannot be controlled simultaneously from a plurality of wristwatch-type terminal devices. . In addition, it is necessary to disconnect the control HFP session before transferring image data from the mobile phone device to the wristwatch type terminal device by BIP, and a control command from the wristwatch type terminal device cannot be accepted at the time of transfer.

  In Patent Literature 3, an information terminal transmits a control command to an imaging terminal and can simultaneously receive related information of a captured image from the imaging terminal by SPP. However, as in the case of Patent Document 2, the imaging terminals cannot be controlled simultaneously from a plurality of information terminals. Further, since SPP, BIP, and OPP cannot be used together, as in the case of Patent Document 2, there still remains a technical problem that a control command from an information terminal cannot be received when captured image data is transferred.

  In Patent Document 4, the screen area of the video display device is divided for each of a plurality of remote controllers so that a plurality of users (viewers) can share the video display device at the same time. Since it has to be used, the circuit and the control mechanism are complicated, and as a result, there is a problem that the remote controller and the receiver are large and expensive. In addition, bidirectional communication was not possible with this method.

  An object of the present invention is to solve the above-described problems of the prior art and to provide a remote control system capable of simultaneously or in parallel controlling controlled devices to be operated from a plurality of remote control terminals by bidirectional communication.

  In order to achieve the above object, the present invention provides a remote control system in which a controlled device is remotely operated by a plurality of remote control terminals, wherein the controlled device includes a plurality of Bluetooth dongles, and further takes the following measures. There is a feature in the point.

  (1) Each remote control terminal searches for a free Bluetooth dongle, means for generating a registration request including the identification ID of the Bluetooth dongle obtained by the search and the terminal ID of the remote control terminal, and according to the remote operation Means for generating an operation code; means for generating an operation request including the operation code; means for converting each request into a Bluetooth SPP packet; and means for transmitting the SPP packet to the empty Bluetooth dongle. In response to the registration request, the control terminal manages the association between the Bluetooth dongle and the remote control terminal and the correspondence between the operation code of the remote control terminal and the key code of the controlled terminal for each Bluetooth dongle. Means for converting the registered operation code into a key code based on the Bluetooth dongle that received the operation request, and predetermined control based on the key code Characterized in that and means for line.

  (2) The registration request and the operation request include an application ID that identifies an application that can be executed by the controlled device, and the conversion means performs the operation code of the remote control terminal and the key of the controlled terminal for each Bluetooth dongle and application ID pair. The association with the code is managed, and the operation code registered in the operation request is converted into a key code based on the Bluetooth dongle and the application ID that have received the operation request.

  (3) The controlled terminal includes means for generating a feedback command for the operation request, and the feedback command is converted into a Bluetooth SPP packet in the Bluetooth dongle that has received the operation request and is returned to the remote control terminal. And

  (4) The order of packets is registered in each Bluetooth SPP packet, and the controlled device rearranges each received Bluetooth SPP packet based on the order.

  (5) The remote control terminal is one of a mobile phone, a smartphone, a PDA, and a pocket computer that supports the SPP by mounting a Bluetooth module.

  (6) The means for generating an operation code corresponding to the remote operation in the remote control terminal is any one of a direction key, a cross key, a numeric keypad, a touch panel, voice recognition, camera motion detection, a trackball, and an acceleration sensor. To do.

  According to the present invention, the following effects are achieved.

  (1) Since the remote operation signal transmitted from the remote control terminal to the controlled device is converted into a Bluetooth SPP packet, one controlled device can be remotely operated simultaneously and wirelessly by a plurality of remote control terminals.

  (2) Since the remote operation signal includes an application ID for identifying an application, even when a plurality of applications are simultaneously executed in the controlled device, the remote operation by each remote control operation can be identified for each application.

  (3) Since the remote control terminal and the controlled device can communicate bidirectionally, the remote control terminal can receive a feedback command from the controlled device.

  (4) Since the order of packets is registered in each Bluetooth SPP packet, the receiving side can accurately combine the SPP packets.

  (5) Since a Bluetooth module is mounted as a remote control terminal and a mobile phone, smartphone, PDA, or pocket computer that supports SPP can be used as one of the profiles, there is no need to prepare a dedicated remote control terminal. .

  (6) Direction keys, cross keys, numeric keys, touch panel, voice recognition, camera motion detection, trackball, and acceleration sensor provided in the remote control terminal can be used as input operation means.

It is the block diagram which showed the structure of the remote control system to which this invention is applied. It is the figure which showed the concept which allocates operation of a remote control terminal to keyboard operation. It is the functional block diagram which showed the structure of the computer terminal and a mobile telephone. It is the figure which showed the registration content of the tied information storage part. It is the figure which showed an example of the conversion table. It is a sequence flow of a registration procedure. It is the figure which showed the format of the registration request message. It is the figure which showed the registration content of the tied information storage part after registration completion. It is the figure which showed the format of the registration response message. It is the sequence flow which showed the operation | movement of one Embodiment of this invention. It is the figure which showed the format of the operation request message. It is the flowchart which showed the transmission / reception procedure of the Bluetooth SPP packet.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a remote control system to which the present invention is applied, which executes a predetermined control in response to a remote operation signal and outputs a video signal reflecting the control contents. The apparatus 1 is mainly composed of a large monitor apparatus 2 that reproduces the video signal and a plurality of remote control terminals 3 that transmit a radio signal for remote operation to the controlled apparatus 1.

  The controlled device 1 is a game terminal, a digital signage terminal or a computer terminal on which a plurality of Bluetooth dongles or Bluetooth modules (hereinafter collectively referred to as Bluetooth dongles BTD) are mounted, and the remote control terminal 3 is also a Bluetooth dongle. Alternatively, it is a mobile terminal such as a mobile phone, a smartphone, a PDA, or a pocket computer on which a Bluetooth module (hereinafter collectively referred to as Bluetooth module BTM) is mounted. In the present embodiment, a mobile phone is assumed as the remote control terminal 3, a computer terminal is assumed as the controlled device 1, and each mobile phone 3 and the computer terminal 1 perform bidirectional packet communication by Bluetooth SPP.

  Referring also to FIG. 2, in this embodiment, each key area of the keyboard 4 connected to the computer terminal 1 is assigned to each mobile phone 3, and when the terminal user operates the mobile phone 3, this is the operation of the keyboard 4. Recognized by the computer terminal 1. The computer terminal 1 can execute a plurality of applications AP1 and AP2 at the same time, and converts each operation of the mobile phone 3 into a unique keyboard operation for each application and executes each application.

  In such a configuration, in this embodiment, when each mobile phone 3 transmits an operation code representing the contents of key operations or the like to the computer terminal 1 by Bluetooth SPP, the computer terminal 1 converts the received operation code into a keyboard operation. Then, control corresponding to the key code is executed. At this time, the computer terminal 1 returns a feedback command to each mobile phone 3 by Bluetooth SPP as necessary. Each mobile phone 3 interprets the received feedback command and switches the display on the terminal display or outputs force feedback.

  FIG. 3 is a functional block diagram showing the configuration of the main parts of the computer terminal 1 and each mobile phone 3.

  In Bluetooth, one of the devices that communicate with each other is a master and the other is a slave. According to Bluetooth specifications, up to seven slaves can be connected to one master, and any device can be a master and a slave. However, the device that first requests connection is the master, and the device that responds to this connection request is the slave. In the present embodiment, since various request messages are transmitted to the computer terminal 1 when the mobile phone 3 is operated, the mobile phone 3 becomes a master and the computer terminal 1 becomes a slave. For this reason, in this embodiment, since the number of remote control terminals that can accept each Bluetooth dongle BTD mounted on the computer terminal 1 is limited to “1”, the computer terminal 1 has the mobile phone 3 to be functioned as a remote control terminal. A number of Bluetooth stacks corresponding to the number are installed, and a plurality of corresponding Bluetooth dongle BTDs are installed.

  In FIG. 3, each mobile phone 3 is implemented with a general-purpose OS such as BREW (registered trademark: Binary Runtime Environment for Wireless) as an OS executed by the main control unit 300, and in a communication layer controlled by the communication management unit 303. The Bluetooth (registered trademark) transmission module 301 and reception module 302 operate. The identification ID storage unit 304 stores an identification ID of an empty Bluetooth dongle BTD notified from the computer terminal 1. The application ID storage unit 305 stores an application ID unique to each application described in the application list notified from the computer terminal 1.

The input unit 31 includes an operation code detected by an input device such as a direction key, a cross key, a numeric keypad, a touch panel, voice recognition, camera motion detection, a trackball or an acceleration sensor, and an identification ID and application of the selected Bluetooth dongle BTD. Convert ID to input information. The display unit 32 includes a terminal display and its control circuit, and controls display on the terminal display in response to output information. The vibration unit 33 generates vibration in response to the output information of the force feedback request.
The sound unit 34 generates sound effects in response to the output information of the force feedback request.

  The computer terminal 1 is implemented with a general-purpose OS such as Windows (registered trademark) or Linux (registered trademark) as an OS executed by the main control unit 100, and a plurality of communication management units 101 are each equipped with a Bluetooth dongle BTD. ing. In the communication layer, the transmission module 105 and the reception module 106 of each Bluetooth dongle BTD operate to exchange packets with the mobile phone 3 by Bluetooth SPP.

  In the linking information storage unit 102, as shown in FIG. 4, for example, the identification ID of the Bluetooth dongle BTD, the MAC address of the mounted Bluetooth dongle BTD, the serial port number, and the user terminal ID are linked to each other. Yes. The operation conversion unit 103 includes a conversion table 103a, and converts the operation code received from the mobile phone 3 into a key code based on the conversion table 103a. The display update unit 104 updates the display content of the monitor device 2 by executing a process in response to the key code.

  FIG. 5 is a diagram showing an example of the conversion table 103a, in which the Bluetooth dongle BTD identification ID, application ID, operation code, and corresponding keyboard operation are associated with each other.

  For example, since the identification ID “1” is associated with the application ID “APP1”, the operation code “LEFT”, and the keyboard operation “KEY_LEFT”, the mobile phone 3 associated with the Bluetooth dongle with the identification ID “1”. When the “LEFT” key is operated in an application whose application ID is “APP1”, this is regarded as an operation of “KEY_LEFT” on the keyboard and processed.

  FIG. 6 is a sequence flow showing a procedure for registering the mobile phone 3 as a remote control terminal in the computer terminal 1 prior to performing the remote operation.

  When the mobile phone 3 transmits a search request, the computer terminal 1 refers to the association information storage unit 102 to search for an empty Bluetooth dongle MTD, and searches for the address information (MAC address in this embodiment). Reply as a response. Here, as shown in FIG. 4, since all the Bluetooth dongles MTD are in an empty state, any Bluetooth dongle MTD is arbitrarily selected and its MAC address is returned.

  Upon receiving this search response, the mobile phone 3 generates a registration request message and transmits it to the computer terminal 1. FIG. 7 is a diagram showing the format of the registration request message. In this embodiment, three Value fields (0), (1), (2) are secured in the Data field of the Bluetooth SPP packet. Information indicating that the message is a registration request is registered in the (0) field, the MAC address of the Bluetooth dongle is registered in the Value (1) field, and the user terminal ID is registered in the Value (2) field. It is registered.

  When receiving the registration request message, the computer terminal 1 registers the user terminal ID in the corresponding record in the association information storage unit 102. FIG. 8 is a diagram showing the registration contents of the association information storage unit 102 after the registration is completed. In the record with the identification ID “1”, the Bluetooth dongle MTD MAC address [00: 21: 4F. The terminal ID [090-5555 ...] of the remote control terminal is linked.

  When the computer terminal 1 completes registration in the association information storage unit 102, it generates a registration response message and sends it back to the mobile phone 3. FIG. 9 is a diagram showing the format of the registration response message. Information indicating that the message is a registration response is registered in the Value (0) field, and the string is stored in the Value (1) field. In the attached information storage unit 102, an identification ID “1” for specifying the Bluetooth dongle is registered. In the Value (2) field, a list of application IDs of executable applications is registered.

  Upon receiving this registration response message, the mobile phone 3 extracts the identification ID and registers it in the identification ID storage unit 304, further extracts the application ID and registers it in the application ID storage unit 305, and performs a series of registration procedures. Complete.

  When canceling registration, a message requesting cancellation of pairing registration is transmitted from the mobile phone 3 to the computer terminal 1. The format of this deregistration request message is the same as that of the registration request message described with reference to FIG. 7, but the deregistration Flag (0xFE) is registered in the Value (0) field, and the Value (1) field is registered. The MAC address of the Bluetooth dongle MTD to be canceled is registered, and the user terminal ID is registered in the Value (2) field.

  Upon receiving this registration cancellation request message, the computer terminal 1 notifies the association information storage unit 102 of this message. In response to this notification, the association information storage unit 102 selects the Bluetooth dongle corresponding to the registered contents of the Value (1) field and Value (2) field, and clears the user terminal ID.

  Next, an embodiment in which a terminal user operates his / her mobile terminal 3 to operate an application of the computer terminal 1 will be described in detail along the sequence flow of FIG. 10.

  In the mobile phone 3, when a Bluetooth dongle and an application are selected by the terminal user at time T1, and when an input event such as a key operation is detected, this is notified to the main control unit 300 as an operation request. The main control unit 300 acquires the identification ID of the Bluetooth dongle from the identification ID storage unit 304 at time T2, and further acquires the application ID from the application ID storage unit 304. At time T3, the identification ID, application ID, and operation code are acquired. Are generated and transferred to the communication management unit 303.

  In this operation request message, as shown in FIG. 11, five Value fields (0), (1), (2), (3), (4) are secured in the Data field, and Value ( The ID is registered in the (0) field, the application ID is registered in the Value (1) field, the operation code is registered in the Value (2) field, the packet number is registered in the Value (3) field, Various data are registered in the Value (4) field. The communication management unit 303 converts the transferred operation request message into a Bluetooth SPP packet and transmits it from the transmission module 301 to the computer terminal 1 at time T4.

  In the computer terminal 1, the Bluetooth SPP packet is received by the communication manager 101 and transferred to the main controller 100 at time T5. The main control unit 100 extracts the transferred identification ID, application ID, operation code, and the like, and transfers them to the operation conversion unit 103 at time T6. The operation conversion unit 103 reproduces the keyboard operation based on the transferred operation code, and transfers the key code to the display update unit 104 at time T7. The display update unit 104 updates the display content based on the transferred key code.

  At this time, if it is necessary to respond to the mobile phone 3 with text output, image output, audio output, force feedback, or the like, the corresponding feedback command is transferred from the main control unit 100 to the communication management unit 101 at time T8. The communication management unit 101 converts the transferred feedback command into a Bluetooth SPP packet, and transmits it from the transmission module to the mobile phone 3 at time T9.

  In the mobile phone 3, the Bluetooth SPP packet is received by the communication management unit 303 and transferred to the main control unit 300 at time T10. The main control unit 300 executes control such as display update, audio output, or vibration operation at time T11.

  FIG. 12 is a flowchart showing a procedure for transmitting each of the message packets by Bluetooth SPP. Here, the case of transmitting from the cellular phone 3 [FIG. (A)] to the computer terminal 1 [(b)] is shown. An example will be described.

  In the mobile phone 3, the data to be transmitted to the computer terminal 1 is divided into data pieces in units of 1022 bytes, converted into Bluetooth SPP packets, and enqueued in a buffer. In step S1, when it is detected that a Bluetooth SPP packet for transmission is enqueued in the buffer, in step S2, a transmission Callback routine is executed in an interrupt process, and the packets are dequeued one by one from the buffer and transmitted. It is transmitted to the computer terminal 1 by the module 301.

  In the computer terminal 1, when the Bluetooth SPP packet is received by the reception module 102 in step S11, in step S12, a reception Callback routine is executed by an interrupt process, and the reception packet is enqueued in the buffer. In step S13, an Ack packet is returned to the normally received Bluetooth SPP packet. In step S14, it is determined whether or not a transmission completion notification is registered in the received packet. If not registered, the process returns to step S11 to prepare for reception of the next packet.

  In the mobile phone 3, when the Ack packet returned from the computer terminal 1 is received in step S3, the process returns to step S1, and the above transmission process is repeated for all the enqueued Bluetooth SPP packets. “0xFFFF” is registered in Value (3) of the packet, and is transmitted as a transmission completion notification.

  In the computer terminal 1, when a transmission completion notification is detected in step S14, the process proceeds to step S15, where the data pieces and the packet order are extracted from the received packets, and the data pieces are combined by combining the data pieces according to the packet order. Restore. In step S16, the restored data is transferred to the display update unit 104 and processed.

  DESCRIPTION OF SYMBOLS 1 ... Computer terminal, 2 ... Monitor apparatus, 3 ... Mobile phone, 31 ... Input part, 32 ... Display part, 33 ... Vibrating part, 34 ... Acoustic part, 100 ... Main control part, 101 ... Communication management part, 102 ... String Additional information storage unit 103 ... Operation conversion unit 103a ... Conversion table 104 ... Display update unit 105 ... Transmission module 106 ... Reception module 300 ... Main control unit 301 ... Transmission module 302 ... Reception module 303 ... Communication management unit 304... Identification ID storage unit 305... Application ID storage unit

Claims (6)

In a remote control system for remotely controlling a controlled device with a plurality of remote control terminals,
The controlled device implements a plurality of Bluetooth dongles,
Means for searching for a free Bluetooth dongle for each remote control terminal;
Means for generating a registration request including the identification ID of the Bluetooth dongle obtained by the search and the terminal ID of the remote control terminal;
Means for generating an operation code according to remote operation;
Means for generating an operation request including the operation code;
Means for converting each request into a Bluetooth SPP packet;
Means for transmitting the SPP packet to the empty Bluetooth dongle,
The controlled terminal is
In response to the registration request, means for associating the Bluetooth dongle and the remote control terminal,
Manages the correspondence between the operation code of the remote control terminal and the key code of the controlled terminal for each Bluetooth dongle, and converts the operation code registered in the operation request into a key code based on the Bluetooth dongle that received the operation request Means to
A remote control system comprising: means for executing predetermined control based on the key code. The registration request and the operation request include an application ID that identifies an application that can be executed by the controlled device.
The conversion means manages the correspondence between the operation code of the remote control terminal and the key code of the controlled terminal for each pair of the Bluetooth dongle and the application ID, and receives the operation request registered in the operation request. The remote control system according to claim 1, wherein the remote control system converts the key code based on the Bluetooth dongle and the application ID. The controlled terminal comprises means for generating a feedback command for the operation request;
3. The remote control system according to claim 1, wherein the feedback command is converted into a Bluetooth SPP packet in a Bluetooth dongle that has received the operation request and is returned to the remote control terminal.   The order of packets is registered in each Bluetooth SPP packet, and the controlled device rearranges each received Bluetooth SPP packet based on the order. Remote control system.   5. The remote control system according to claim 1, wherein the remote control terminal is one of a mobile phone, a smartphone, a PDA, and a pocket computer that mounts a Bluetooth module and supports SPP.   The means for generating an operation code corresponding to a remote operation in the remote control terminal is any one of a direction key, a cross key, a numeric keypad, a touch panel, voice recognition, camera motion detection, a trackball, and an acceleration sensor. Item 6. The remote control system according to any one of Items 1 to 5.