JP2014219765A - Operation device, and information processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display device with an operability enhanced touch panel.SOLUTION: An operation device comprises: a touch panel 101; a coordinate information generating unit 116 that has a plurality of coordinate generating modes including a first coordinate generating mode; an accepting unit 116B that accepts particular operation to switch from a coordinate generating mode to other coordinate generating mode among the plurality of coordinate generating modes; and a communication interface 110 that outputs coordinate information generated by the coordinate information generating unit 116. The coordinate information generating unit 116 generates two-dimensional coordinates on a plane of a touch position on a surface of the touch panel 101 and height information (Z) in a perpendicular direction to the surface of the touch panel 101 according to a coordinate generating mode.

Description

  The present invention relates to an operation device having a touch panel and an information processing system using the same.

  Recently, there is a technique for reflecting an operation on a touch panel on a display screen using the touch panel as an operation target.

  In Patent Document 1, when a finger is made to face a display screen on which an image representing an operation key is displayed, a cursor corresponding to the distance between the display screen and the finger is displayed at a position facing the finger on the display screen. Technology is disclosed.

JP 2010-61224 A

  However, in the above technique, when an object displayed on the display screen is selected and moved, the fingertip must be moved while always maintaining a certain height range from the display screen. Thus, skill is required to move the fingertip while constantly maintaining a certain height range from the display screen, and it cannot be said that the operability is good.

  SUMMARY An advantage of some aspects of the invention is that it provides an operation device with good operability and an information processing system using the same.

  In order to achieve the above object, an operating device according to an aspect of the present invention has a touch panel surface, detects a coordinate position on a plane touched by the touch panel surface, and the touched coordinate position. A coordinate information generation unit that generates coordinate information that includes a plurality of coordinate generation modes including a first coordinate generation mode that uses planar coordinate information that represents the coordinate position of the fingertip on the touch panel surface; A receiving unit that receives a specific operation for switching from one coordinate generation mode to another coordinate generation mode among the plurality of coordinate generation modes, and an output unit that outputs the coordinate information generated by the coordinate information generation unit. And the coordinate information generator is perpendicular to the two-dimensional coordinates of the touch position on the touch panel surface and the touch panel surface. A direction of the height (Z), is generated according to the coordinate generating mode.

  According to this configuration, with the fingertip placed on the touch panel surface, two-dimensional coordinates on the plane of the touch position on the touch panel surface and height information (Z) in the direction perpendicular to the touch panel surface are set in the coordinate generation mode. Accordingly, the user does not have to move the fingertip while always maintaining a certain height range from the display screen.

  Therefore, the user can operate the operating device without stress without being confused by the distance between the display screen and the finger. As a result, an operating device with good operability can be provided.

  In addition, an operating device according to another aspect of the present invention has a touch panel surface, and has a function of detecting a touch position of the fingertip on the touch panel surface and a height of the fingertip existing above the touch panel surface with respect to the touch panel surface. Detected when there is a fingertip over the touch panel surface, a touch panel having a function of detecting the touch panel, a 3D operation mode for generating the three-dimensional position information of the fingertip over the touch panel surface as three-dimensional coordinate information Coordinate information generation having a plurality of coordinate generation modes including a first coordinate generation mode that uses plane coordinate information representing the coordinate position of the fingertip on the touch panel surface without using the three-dimensional position information. Unit, an output unit that outputs the coordinate information generated by the coordinate information generation unit, and the 2D operation mode A reception unit that receives a specific operation for switching from one coordinate generation mode to another coordinate generation mode, wherein the coordinate information generation unit is (1) in the 3D operation mode, the fingertip is Generating the detected position of the fingertip as a three-dimensional coordinate when it exists in a three-dimensional space within a certain height from the touch panel surface, including when touching the touch panel surface; (2) the 2D operation In the mode, two-dimensional coordinates on the plane where the touch panel is touched and height information (Z) in a direction perpendicular to the touch panel surface are generated according to the coordinate generation mode.

  According to this configuration, when the fingertip is present in a three-dimensional space within a certain height range from the touch panel surface, the fingertip detection position is generated as a three-dimensional coordinate.

  In other words, the 3D operation mode means that when the fingertip is positioned within a certain height range from the touch panel surface, the plane position information indicating the two-dimensional coordinates on the touch panel surface of the fingertip and the height from the fingertip and the touch panel surface are calculated. In this mode, three-dimensional position information including the height information to be expressed is output to an external processing device. In this mode, the external processing device displays a cursor corresponding to the distance between the fingertip and the touch panel surface at the coordinate position represented by the plane position information.

  In the 2D operation mode, the operation device having such a 3D operation mode generates two-dimensional coordinates on the plane touched on the touch panel and height information (Z) in a direction perpendicular to the touch panel surface. Generate according to the mode.

  As a result, it is possible to provide an operation device that combines the 2D operation mode described above and the existing 3D operation mode, so that the user can select the 3D operation mode and the 2D operation depending on the case of using the operation device. Since it can be separated from the mode, it is excellent in convenience.

  In addition, the coordinate information generation unit has at least a first coordinate generation mode and a second coordinate generation mode, and the reception unit switches the first coordinate generation mode to the second coordinate generation mode. 1 coordinate operation is received, and the coordinate information generation unit gives a positive value as the height information Z in the first coordinate generation mode, and gives a zero value as the height information Z in the second coordinate generation mode. May be.

  According to this configuration, since a positive value is given as the height information Z in the first coordinate generation mode and a zero value is given as the height information Z in the second coordinate generation mode, a touch panel capable of detecting the plane coordinate position is used. Thus, both solid coordinates and plane coordinates can be obtained.

  The coordinate information generation unit has at least a first coordinate generation mode, a second coordinate generation mode, and a third coordinate generation mode, and the reception unit changes from the first coordinate generation mode to the third coordinate generation mode. A second specific operation for switching to the second coordinate generation mode, and a third specific operation for switching from the third coordinate generation mode to the second coordinate generation mode. A positive value greater than a certain value is given as the height information Z, a zero value is given as the height information Z in the second coordinate generation mode, and less than a certain value as the height information Z in the third coordinate generation mode May be given a positive value.

  According to this configuration, in the 2D operation mode, a positive value of a certain value or more is given as the height information Z in the first coordinate generation mode, and a zero value is given as the height information Z in the second coordinate generation mode. In the three-coordinate generation mode, a positive value less than a certain value is given as the height information Z.

  According to this configuration, since the third coordinate generation mode provides height information having a value different from that of the first coordinate generation mode and the second coordinate generation mode, the third coordinate generation mode may be provided between the first coordinate generation mode and the second coordinate generation mode. It can be an intermediate mode.

  Therefore, the third coordinate generation mode, which is an intermediate mode, is sandwiched between the first coordinate generation mode and the second coordinate generation mode as compared with the case where the first coordinate generation mode is suddenly shifted to the second coordinate generation mode. Therefore, the user can obtain a fine operation feeling.

  The coordinate information generation unit generates a change amount information indicating a change amount of the plane position coordinate when the plane position coordinate of the touch position of the fingertip with respect to the touch panel changes within a preset setting time. You may change to the operation mode.

  According to this configuration, when the plane position coordinate of the touch position of the fingertip on the touch panel changes within the set time, the mode is changed to the mouse operation mode for generating change amount information indicating the change amount of the plane position coordinate.

  In the above-described operation device, only the two-dimensional position information on the touch panel surface and the height information of the fingertip above the touch panel surface were output, but according to this configuration, in the situation where an extremely accurate pointing operation is required, It changes to the mouse operation mode only by changing the touch position of the fingertip within the time. Thereby, it is possible to provide the user with the operability of the accurate pointing operation.

  The coordinate information generation unit may generate the coordinates of each touch position when detecting that the touch panel is touched by at least two fingertips.

  According to this configuration, when the touch panel is touched by at least two fingertips, coordinate information of each touch position is generated, so that a multi-touch gesture involving an operation with at least two fingers is performed on the external processing device side. Can be realized. Therefore, for example, the multi-finger gesture such as pinch-in and pinch-out can be realized on the processing device side.

  Further, the coordinate information generation unit detects that the touch panel is touched with at least one fingertip from the standby state in a standby state that is not in any coordinate generation mode after the operation device is activated. In some cases, the mode may transition to the first coordinate generation mode.

  According to this configuration, when it is detected that the touch panel is touched with at least one fingertip from the standby state, the mode is shifted to the first coordinate generation mode. Therefore, if the touch panel is touched with one finger in the standby state, The touch of the fingertip on the touch panel surface is equivalent to the state where the fingertip is present above the touch panel surface. Thereby, even if it is a case where the touchscreen which can output only two-dimensional information is used, the three-dimensional position information containing height information can be output by changing to 1st coordinate production | generation mode.

  The coordinate information generation unit may transition to the second coordinate generation mode when the reception unit receives a first specific operation in the first coordinate generation mode.

  According to this configuration, when the first specific operation is received in the first coordinate generation mode, the mode transitions to the second coordinate generation mode.

  According to this configuration, when the first specific operation is accepted, since it is possible to transition from a state where the fingertip is located above the touch panel surface to a state where the fingertip is touching the touch panel surface, only two-dimensional information can be output. Even when a touch panel is used, the above two states can be realized.

  The coordinate information generation unit transitions to the third coordinate generation mode when the reception unit receives a second specific operation in the first coordinate generation mode, and receives the reception in the third coordinate generation mode. When the unit accepts the third specific operation, the mode may transit to the second coordinate generation mode.

  According to this configuration, when the second specific operation is received in the first coordinate generation mode, the state transitions to the third coordinate generation mode, and when the third specific operation is received in the third coordinate generation mode, the second Transition to coordinate generation mode.

  Thus, the mode of the controller device can be changed to the third coordinate generation mode, which is an intermediate mode between the first coordinate generation mode and the second coordinate generation mode, by the second specific operation. In the third mode, the intermediate mode can be changed to the second coordinate generation mode by the third specific operation. Therefore, it is possible to realize an operation for finely giving height information to the operating device.

  Further, in any of the first coordinate generation mode, the second coordinate generation mode, and the third coordinate generation mode, when the touch is not continuously detected for a certain period of time, the standby state It may be.

  According to this configuration, in any of the first coordinate generation mode to the third coordinate generation mode, when the touch is not continuously detected for a certain period of time on the touch panel, a standby state is entered. -No special operation is required to enter a standby state from any of the third coordinate generation modes. Therefore, a natural operation feeling can be realized.

  The coordinate information generation unit further includes a first operation key that can make the transition from the first coordinate generation mode to the second coordinate generation mode. The coordinate information generation unit is configured such that the reception unit performs the first specific operation. When the operation of the first operation key is accepted, the mode is shifted to the second coordinate generation mode. When the operation of the first operation key is not accepted, the mode is shifted to the first coordinate generation mode. May be.

  According to this configuration, the operation of the first operation key makes a transition to the second coordinate generation mode in which the fingertip is positioned so as to be positioned on the touch panel surface. Transition to the first coordinate generation mode that is treated as if it exists.

  Thus, since the operation of the first operation key corresponds to the transition of the coordinate generation mode, a natural operation feeling can be realized.

  Furthermore, in the first coordinate generation mode, the coordinate information generation unit detects the second touch when the touch panel detects a continuous touch for a certain time at the same position of the touch panel as the first specific operation. When the mode is changed to the coordinate generation mode and the touch is released, the mode may be changed to the first coordinate generation mode.

  According to this configuration, the first coordinate generation mode or the second coordinate generation mode is set depending on the presence or absence of a continuous touch for a certain period of time at the same position on the touch panel. Thereby, since the coordinate generation mode is changed by an operation accompanied by the intention of holding the touch position on the touch panel surface, a natural operation feeling can be realized.

  In addition, the first specific operation is a series of operations in which the user touches the plane again on the touch panel after releasing the touch within a predetermined time after continuing the operation of stopping the user's finger at the same position on the touch panel. Yes, the coordinate information generation unit transitions to the second coordinate generation mode when the reception unit receives the first specific operation in the first coordinate generation mode, and when the touch is released, You may transfer to the first coordinate generation mode.

  According to this configuration, the first coordinate generation mode is continued while the fingertip is stationary at the same position on the touch panel and during the subsequent touch release within a certain time, and the second coordinate generation mode is performed by the subsequent retouch. Transition to.

  Therefore, a virtual touch operation is realized in which height information is output even if the touch panel is touched for a certain period of time after the fingertip is stopped at the same position on the touch panel. Since the transition to an actual touch equivalent state where no height information is output, a more natural operation feeling can be realized.

  In addition, the first specific operation is a touch operation on the plane by a finger different from the finger making a touch that causes the transition to the first coordinate generation mode, and the coordinate information generation unit includes The plane coordinate information is generated based on the plane coordinates indicating the touch position of the finger making the touch that has caused the transition to the second coordinate generation mode, and the cause of the transition to the second coordinate generation mode When the touch of the fingertip is released, the mode transitions to the first coordinate generation mode, and when the touch of the fingertip that causes the transition to the second coordinate generation mode is released for a predetermined time or longer, the standby state is entered. You may make a transition.

  According to this configuration, the first specific operation is a touch operation on the plane with a finger different from the finger that is touching which has caused the transition to the first coordinate generation mode. While the fingertip that caused the transition to is continuously touched, it is possible to transit to the second coordinate generation mode by a touch operation of a finger different from the fingertip.

  As a result, while keeping the fingertip that caused the first coordinate generation mode stationary, the coordinates of the moving object are tracked, and the coordinate generation mode is changed to the touch equivalent state with respect to the coordinates of the object. It is effective in such cases.

  Moreover, since it is possible to continuously return to the first coordinate generation mode by releasing the touch of the finger that caused the second coordinate generation mode, depending on the fingertip that caused the transition to the first coordinate generation mode. It is even more effective when tracking objects continuously.

  In the first coordinate generation mode, when the touch panel detects a touch for a certain period of time at the same position of the touch panel in the first coordinate generation mode, the coordinate information generation unit performs the third operation. When transitioning to the coordinate generation mode and the touch position is changed, or when the touch is released, the mode may be shifted to the first coordinate generation mode.

  According to this configuration, the second specific operation is a continuous touch for a certain time at the same position of the touch panel. Therefore, since the first coordinate generation mode and the third coordinate generation mode can be moved back and forth by an operation with the intention to continue the touch position, a natural operation feeling can be realized.

  Further, the coordinate information generation unit transitions to the third coordinate generation mode as the second specific operation in the first coordinate generation mode as the change of the touch position on the touch panel is delayed, and the touch position As the change in is made faster, the first coordinate generation mode may be entered.

  According to this configuration, the first coordinate generation mode and the third coordinate generation mode can be moved back and forth depending on the manner in which the intention of whether the moving speed of the touch position is fast or slow is easily reflected. Can be realized.

  Furthermore, the coordinate information generation unit includes a second operation key that can make the transition from the third coordinate generation mode to the second coordinate generation mode, and the coordinate information generation unit is configured such that the reception unit performs the third specific operation. When the operation of the second operation key is accepted, the mode shifts to the second coordinate generation mode. After that, when the operation of the second operation key is not accepted, the third coordinate generation is performed. You may transition to mode.

  According to this configuration, the operation of the second operation key makes a transition to the second coordinate generation mode in which the fingertip is positioned on the touch panel surface, and the operation of the second operation key is canceled, Transition to the third coordinate generation mode, which is an intermediate mode between the second coordinate mode and the second coordinate mode.

  As described above, since the operation of the second operation key corresponds to the transition of the coordinate generation mode, a natural operation feeling can be realized.

  Furthermore, in the third coordinate generation mode, the coordinate information generation unit detects the second touch when the touch panel detects a continuous touch for a certain time at the same position of the touch panel as the third specific operation. When the transition to the coordinate generation mode is made and then the touch is released, the transition to the third coordinate generation mode may be made.

  According to this configuration, the third coordinate generation mode or the second coordinate generation mode is set depending on the presence or absence of a continuous touch for a certain period of time at the same position on the touch panel. Thereby, since the coordinate generation mode is changed by an operation accompanied by the intention of holding the touch position on the touch panel surface, a natural operation feeling can be realized.

  Further, the third specific operation is a series of operations for touching the plane again on the touch panel after the touch is released within a predetermined time, and the coordinate information generation unit is configured to operate the reception unit in the third coordinate generation mode. When the third specific operation is accepted, the mode may transition to the second coordinate generation mode, and then, when the touch is released, the mode may transition to the third coordinate generation mode.

  According to this configuration, the third coordinate generation mode is continued while the fingertip is stationary at the same position on the touch panel and during the subsequent touch release within a certain time, and the second coordinate generation mode is performed by the subsequent re-touch. Transition to.

  Therefore, a virtual touch operation is realized in which height information is output even if the touch panel is touched for a certain period of time after the fingertip is stopped at the same position on the touch panel. Since the transition to an actual touch equivalent state where no height information is output, a more natural operation feeling can be realized.

  In addition, the third specific operation is a touch operation on the plane with a finger different from the finger making a touch that causes the transition to the first coordinate generation mode and the third coordinate generation mode. The coordinate information generation unit generates the plane coordinate information based on plane coordinates indicating a touch position of a finger making a touch that has caused the transition to the third coordinate generation mode, and the third coordinates When the touch of the fingertip that is the factor of the generation mode is released, the mode is shifted to the first coordinate generation mode, and when the touch of the fingertip that is the factor of the third coordinate generation mode is released for a predetermined time or more, the standby state is entered. You may make a transition.

  According to this configuration, the third specific operation is a touch operation on a plane with a finger different from the finger making a touch that causes the transition to the third coordinate generation mode.

  Therefore, the fingertip that caused the transition to the third coordinate generation mode can be changed to the first coordinate generation mode by a touch operation of a finger other than the fingertip while continuously touching.

  As a result, while keeping the fingertip that caused the third coordinate generation mode stationary, the coordinates of the moving object are tracked, and the coordinate generation mode is set to a virtual touch equivalent state with respect to the coordinates of the object. It is effective when you want to change.

  In addition, the fingertip that caused the transition to the third coordinate mode can be continuously returned to the first coordinate mode by releasing the touch of the finger that caused the transition to the third coordinate generation mode. This is even more effective when the object is continuously tracked.

  An information processing system according to another aspect of the present invention provides the operation device according to any one of claims 1 to 20, a display device, and the coordinate information output from the output unit as the display device. A processing device for converting the coordinate information into the coordinate information and displaying a cursor image at the coordinate position represented by the converted coordinate information, and a communication unit for communicating at least the coordinate information between the operation device and the processing device; When the coordinate information and the height information are received from the output unit, the processing device moves the first cursor when the height information receives a positive coordinate value greater than or equal to a set value. A second cursor is displayed when the height information receives a zero value, and a third cursor is displayed when the height information receives a positive coordinate value less than a set value.

  According to this configuration, the first cursor is displayed when the height information receives a positive coordinate value greater than or equal to the set value, and the second cursor is displayed when the height information receives a zero value. The third cursor is displayed when a positive coordinate value whose information is less than the set value is received.

  Thereby, it can be easily distinguished whether the operating device is in the first coordinate generation mode, the second coordinate generation mode, or the third coordinate generation mode. In addition, the user himself can intuitively recognize that the height coordinate Z changes according to his / her operation, even though the operation is on the same touch surface, by the difference in cursor display. .

  Each of the first cursor image, the second cursor image, and the third cursor image may be different in at least one of shape and color.

  According to this configuration, since each of the first cursor image, the second cursor image, and the third cursor image is different in at least one of shape and color, the respective cursor images can be more easily distinguished.

  Further, the processing device may cause the display unit to display a mouse cursor image as the cursor image when the coordinate information generation unit is executing the mouse operation mode.

  According to this configuration, since the mouse cursor image is displayed in the mouse operation mode, it can be seen at a glance that it is the mouse operation mode.

  In the processing device, the coordinate information generation unit may change a shape and a color of the cursor image displayed on the display device between the 3D operation mode and the 2D operation mode. Also good.

  According to this configuration, since the shape and color of the cursor image displayed on the display device are different between the 3D operation mode and the 2D operation mode, it is easy to distinguish between the 3D operation mode and the 2D operation mode. Get on.

  Further, the processing device holds operation mode information indicating any of the 2D operation mode, the 3D operation mode, and the mouse operation mode, and the processing device holds the operation mode information via the communication unit. The operation mode information is provided to the operation device, and the coordinate information generation unit executes any of the 2D operation mode, the 3D operation mode, and the mouse operation mode according to the received operation mode information. Also good.

  According to this configuration, the operation mode information indicating any one of the 2D operation mode, the 3D operation mode, and the mouse operation mode is held in the processing device, and the operation device is based on the operation mode information sent from the processing device. Mode.

  Thereby, for example, the operation device can be in an operation mode according to a program executed by the processing device, which is convenient.

  The processing device is preinstalled with a plurality of programs for accepting operations by the operating device, and the processing device is executed or is being executed during execution of each program. The operation mode information representing any one of the 3D operation mode, the 2D operation mode, and the mouse operation mode may be transmitted to the coordinate information generation unit according to an individual program.

  According to this configuration, the operation mode information is transmitted to the operation device in accordance with the execution state of the program being executed by the processing device or the individual program being executed. Therefore, the following effects can be expected.

  For example, since the 3D operation mode is suitable for a specific game application, the processing device can output 3D operation mode information to the operation device to shift to the 3D operation mode.

  Alternatively, since the mouse operation mode is suitable for a WEB browser that often handles fine screen objects, the processing device can output the mouse operation mode information to the operation device to shift to the mouse operation mode.

  Alternatively, since the 2D operation mode is suitable when the software keyboard is executed, the processing device can output 2D operation mode information to the operation device and shift to the 2D operation mode.

  The processing apparatus may further include a selection key for selecting the operation mode of the 3D operation mode, the 2D operation mode, or the mouse operation mode in the coordinate information generation unit.

  According to this configuration, the operation mode can be switched according to the operator's intention to use because it is possible to select the operation mode of the 3D operation mode, the 2D operation mode, and the mouse operation mode by operating the selection key. It is convenient because it can.

  In addition, the coordinate information generation unit may execute any one of the 3D operation mode, the 2D operation mode, and the mouse operation mode based on posture information representing the posture of the operation device.

  According to this configuration, the 3D operation mode, the 2D operation mode, and the mouse operation mode are set according to the posture information indicating the posture of the controller device, which is convenient.

  Furthermore, an acceleration sensor that detects the attitude of the operating device and outputs the attitude information indicating the detected attitude may be provided.

  According to this configuration, since the posture information is obtained by the acceleration sensor, the posture in any direction in the three-dimensional space can be known, and the dynamic change in posture can also be known from the change in acceleration.

  The operating device may further include a vibration unit.

  According to this configuration, since the operating device includes the vibration unit, it is possible to convey a predetermined matter to the user by tactile sense by vibrating at a predetermined timing.

  The operating device may further include a light emitting unit.

  According to this configuration, since the operation device includes the light emitting unit, it is possible to visually convey a predetermined matter to the user by emitting light at a predetermined timing.

  According to the present invention, the operating device can be operated without stress without being confused by the distance between the touch panel and the finger.

1 is a system configuration diagram illustrating an example of an information processing system according to a first embodiment of the present invention. It is the block diagram which showed an example of the functional block of the operating device which concerns on Embodiment 1 of invention. It is the figure which showed an example of the structure of the specific operation table. It is the block diagram which showed an example of the function structure of the processing apparatus. It is the figure which showed an example of the structure of the corresponding | compatible mode table. It is the block diagram which showed an example of the function structure of the display apparatus. It is the flowchart shown about an example of the basic operation | movement of the coordinate information generation part of an operating device. It is the flowchart shown about the coordinate generation mode transition process in a coordinate information generation part. It is the flowchart shown about an example of the basic operation | movement of a processing apparatus. It is the figure which showed an example of the cursor image displayed on a display apparatus. It is the figure which showed an example of the mouse cursor image. It is the flowchart which showed an example of the basic operation | movement of the coordinate information generation part of the operating device which concerns on Embodiment 2 of this invention. It is the figure which showed an example of the cursor image displayed on a display apparatus in Embodiment 2 of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. It should be noted that each of the embodiments described below shows a comprehensive or specific example. The constituent elements, the arrangement positions of the constituent elements, and the connection forms shown in the following embodiments are merely examples, and are not intended to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements.

(Embodiment 1)
<Configuration of information processing system>
FIG. 1 is a system configuration diagram showing an example of an information processing system according to Embodiment 1 of the present invention. The information processing system illustrated in FIG. 1 includes an operation device 1, a display device 2, and a processing device 3. In this information processing system, when the processing device 3 executes an application program preinstalled in the processing device 3, the user operates the touch panel 101 on the operation device 1 as if on the display screen 20 of the display device 2. It is a system that makes you think you are doing it.

  In other words, this information processing system is a system that provides the user with the display screen 20 of the display device 2 as a virtual touch panel.

  In addition to the touch panel 101, the operation device 1 includes various operation keys 11A to 11C described later. For example, the controller device 1 performs processing according to Android (registered trademark) which is one of the platforms 1.

  The display device 2 has the display screen 20 as described above. The processing device 3 performs wireless communication with the controller device 1 according to, for example, Bluetooth (registered trademark), which is one of the short-range wireless communication standards, and transmits and receives predetermined information.

  The processing device 3 is connected to the display device 2 via the cable C, and provides the display device 2 with predetermined image information. Note that the processing device 3 and the display device 2 may be able to perform wireless communication. In the first embodiment, the controller device 1 and the processing device 3 are provided separately. However, the present invention is not limited to this example, and the single device having the functions of the controller device 1 and the processor device 3 may be used. Good.

<Configuration of operation device 1>
FIG. 2 is a block diagram showing an example of functional blocks of the operating device according to Embodiment 1 of the present invention.

  As shown in FIG. 2, the operating device 1 includes an SOC (System-On-a-Chip) 300, a touch panel 101, a first operation key 11A, a second operation key 11B, a third operation key 11C, Audio control circuit 106 that controls microphone MI and speaker SP, wireless communication interface 111, antenna 112, external power connector 113, power control circuit 114, rechargeable battery 115, acceleration sensor S, and vibration unit 117 And a light emitting unit 118.

  In the SOC 300, the following components are connected to the internal bus 103. The SOC 300 includes a touch panel interface 102, an operation key interface 104, an audio interface 105, a memory 107, a CPU (Central Processing Unit; display control unit) 108, a clock output unit 109, and a communication interface (communication unit, output unit). ) 110 and a coordinate information generation unit 116 having a specific operation table 116A.

  The touch panel interface 102 is an interface for connecting the touch panel 101 to the SOC 300.

  The operation key interface 104 is an interface for connecting the operation keys 11A to 11C to the SOC 300. The operation key interface 104 receives information indicating that the operation keys 11 </ b> A to 11 </ b> C have been operated (for example, information indicating a signal ON period) and outputs the information to the CPU 108. The audio interface 105 is an interface for connecting the audio control circuit 106 to the SOC 300.

  The memory 107 is a storage medium that stores various control programs necessary for the operation device 1 to operate. In the memory 107, as one of the control programs, a program constituting the above-described Android (registered trademark) is installed in advance.

  The CPU 108 functions as a control unit by being operated by a control program stored in the memory 107, for example, Android (registered trademark). The clock output unit 109 is a unit that outputs a clock signal for the CPU 108 to operate.

  The communication interface 110 is an interface for connecting the wireless communication interface 111 to the SOC 300.

  The coordinate information generation unit 116 has a first coordinate generation mode, a second coordinate generation mode, and a third coordinate generation mode. Details of these coordinate generation modes will be described later. In addition, the coordinate information generation unit 116 includes a specific operation table 116A and a reception unit 116B.

  FIG. 3 is a diagram showing an example of the configuration of the specific operation table 116A. The specific operation table 116A stores a specific operation 1160 and an operation method 1161 corresponding to the specific operation 1160.

  In the first embodiment, as the specific operation 1160, a first specific operation 1160A, a second specific operation 1160B, and a third specific operation 1160C are stored.

  Corresponding to the first specific operation 1160A, (1) “pressing operation key 11A”, (2) “continuous touch operation on touch panel 101”, (3) “user's operation on touch panel 101” After continuing the operation of stopping the finger at the same position, after a touch release within a certain time, a series of operations for touching the touch panel 101 again, and (4) “cause of transition to the first coordinate generation mode. “Touch operation with a finger different from touch” is stored.

  Corresponding to the second specific operation 1160B, (5) “continuous touch operation at the same position on the touch panel 101” and (6) “operation for slowing the change of the touch position on the touch panel 101” are performed. It is remembered.

  Further, corresponding to the third specific operation 1160C, (7) “pressing operation key 11B”, (8) “continuous touch operation at the same position on the touch panel 101”, (9) “in touch panel 101” “A series of operations for touching again after touch release within a certain period of time” and (10) “a touch operation with a finger other than the touching finger that caused the transition to the third coordinate generation mode” are stored Has been.

  The accepting unit 116B accepts a first specifying operation, a second specifying operation, and a third specifying operation, which will be described later. The touch panel 101 has a touch panel surface and detects a planar coordinate position touched on the touch panel surface.

  The operation key 11A is an operation key for performing a first specific operation described later. The second key 11B is an operation key for performing a second specific operation described later. The third operation key 11C is a key for performing a selection operation described later. In the present embodiment, a dedicated function is assigned to each of the first key 11A to the third key 11C. However, the present invention is not limited to this example, and each of the first key 11A to the third key. A function corresponding to a control program executed by the CPU 108 may be assigned to the CPU 108.

  The audio control circuit 106 is a circuit for performing a call with a partner (for example, a mobile phone) through the microphone MI and the speaker SP. That is, the controller device 1 also functions as a mobile phone.

  The wireless communication interface 111 is an interface for performing wireless communication with the controller device 1 through the antenna 112.

  The external power connector 113 receives power supply from a charger (not shown). The power supply control circuit 114 controls charging from the charger to the rechargeable battery 115.

  The acceleration sensor S is composed of, for example, a triaxial acceleration sensor, and outputs posture information representing the posture of the controller device 1 to the CPU 108.

  The vibration unit 117 is configured by, for example, a motor or a piezoelectric element, and vibrates at a predetermined timing. The light emitting unit 118 is configured by, for example, a light emitting diode, and emits light at a predetermined timing. An example of the predetermined timing will be described later.

<Configuration of Processing Device 3>
An image provided by the processing device 3 is displayed on the display device 2. Therefore, the processing device 3 has the following configuration. FIG. 4 is a block diagram illustrating an example of a functional configuration of the processing device 3.

  The processing device 3 includes a wireless information transmission / reception unit (communication unit) 31, a coordinate information conversion unit 32, a CPU 33, an interface 34, and a memory 35.

  The wireless information transmission / reception unit 31 is a unit that transmits and receives various types of information to and from the controller device 1 wirelessly. The type of information will be described later.

  The coordinate information conversion unit 32 is a unit that converts the coordinate information generated by the operation device 1 into coordinate information in the display device 2. For this purpose, the coordinate information conversion unit 32 stores the association between the coordinate information by the controller device 1 and the coordinate information on the display device 2. The CPU 33 controls the processing device 3 according to a control program stored in the memory 35, for example, Android (registered trademark).

  The interface 34 is an interface for connecting the display device 2 to the processing device 3. In addition to various control programs necessary for the processing device 3 to operate, the memory 35 includes a corresponding mode table 35A, application programs 35B to 35D, 2D operation mode information 35E, 3D operation mode information 35F, and a mouse. Operation mode information 35G. Details of the 2D operation mode information 35E, the 3D operation mode information 35F, and the mouse operation mode information 35G will be described later.

  FIG. 5 is a diagram showing an example of the configuration of the corresponding mode table. The corresponding mode table 35A stores operation mode information 351 corresponding to the application program 350. In this example, 3D operation mode information 351A is stored corresponding to the application program 350A, 2D operation mode information 351B is stored corresponding to the application program 350B, and mouse operation mode corresponding to the application program 350C is stored. Information 351C is stored. Details of the 3D operation mode information 351A, the 2D operation mode information 351B, and the mouse operation mode information 351C will be described later.

<Configuration of Display Device 2>
FIG. 6 is a block diagram illustrating an example of a functional configuration of the display device 2. The display device 2 includes a display screen 20, a display control unit 21, an information reception unit 22, and an interface 23. The display screen 20 is configured by a liquid crystal display, for example. The display control unit 21 controls display of images on the display screen 20. For example, the display control unit 21 displays an image on the display screen 20 by controlling the orientation of the liquid crystal molecules. The information receiving unit 22 receives various information transmitted from the processing device 3 through the interface 23. The type of information will be described later.

<Operation of operating device 1>
FIG. 7 is a flowchart illustrating an example of a basic operation of the coordinate information generation unit 116 of the controller device 1. First, when the power source of the controller device 1 is turned on, the coordinate information generation unit 116 enters a standby state (step S10).

  When the touch panel 101 detects a touch operation on the touch panel surface (YES in step S11), the coordinate information generation unit 116 determines that the coordinate information generation unit 116 has the first coordinate generation mode and the second coordinate generation. The following processing is performed depending on whether the mode or the third coordinate generation mode is selected.

  That is, when the coordinate information generation unit 116 is in the first coordinate generation mode (YES in step S12), the two-dimensional coordinate information of the touched position and the height information greater than or equal to Z1 (where Z1 is a positive numerical value) Is output to the processing device 3 (step S13).

  When the coordinate information generation unit 116 is in the second coordinate generation mode (YES in step S14), the coordinate information generation unit 116 generates two-dimensional coordinate information of the touched position and zero-value height information, and Output (step S15).

  Further, when the coordinate information generation unit 116 is in the third coordinate generation mode (NO in step S14), the coordinate information generation unit 116 generates two-dimensional coordinate information of the touched position and positive height information less than Z1, The data is output to the processing device 3 (step S16).

  Then, after the first touch operation is detected in step S11, when the touch position changes, that is, when the plane coordinate information on the touch panel surface changes within a predetermined time (YES in step S17), the mouse operation mode is set. (Step S18).

  In the mouse operation mode, the coordinate information generation unit 116 generates change amount information indicating the change amount of the plane position coordinates, and outputs the information to the processing device 3.

  Thereafter, when the touch operation is not continuously detected for a certain time (YES in step S19), the coordinate information generating unit 116 returns to the standby state (step S10).

  Next, the coordinate generation mode transition process in the coordinate information generation unit 116 will be described. FIG. 8 is a flowchart showing the coordinate generation mode transition process in the coordinate information generation unit 116.

  In the standby state (step S100), the coordinate information generation unit 116 enters the first coordinate generation mode when the touch panel surface of the touch panel 101 is operated with one finger (YES in step S101) (step S102). On the other hand, when the touch panel surface is operated with two fingers in the standby state (NO in step S101, YES in step S103), the coordinate information generation unit 116 generates the respective plane coordinate information of the touched position. Is output to the processing device 4 (step S104).

  Thereafter, when the receiving unit 116B receives the first specific operation 1160A (see FIG. 3) (YES in step S105), the coordinate information generating unit 116 transitions to the second coordinate generation mode (step 106).

  At this time, when the first specific operation 1160A is “a touch operation with a finger different from the touch that caused the transition to the first coordinate generation mode”, the plane coordinates indicating the touch position with the other finger The plane coordinate information is generated based on the above and output to the processing device 3. At this time, the plane coordinate information indicating the touch position is converted into coordinate information in the display device 2 by the processing device 3, and the second cursor image CU2 (see FIG. 10) is displayed at the coordinate position represented by the converted coordinate information. Display.

  On the other hand, when the receiving unit 116B receives the second specific operation 1180B (see FIG. 3) (YES in step S109), the process proceeds to the third coordinate generation mode (step S110).

  After entering the second coordinate generation mode (step S106), the coordinate information generation unit 116 transitions to the first coordinate generation mode when the transition condition to the first coordinate generation mode is satisfied (YES in step S107). (Step S102).

  Here, as the “transition condition from the second coordinate generation mode to the first coordinate generation mode”, for example, “stopping the pressing operation of the first operation key 11A”, “continuous for a certain time at the same position of the touch panel 101” After the touch, the touch has been released "," A series of operations for touching the touch panel 101 again after releasing the touch within a certain time after continuing the operation of keeping the user's finger stationary at the same position on the touch panel 101 " And “cancellation of the fingertip that causes the transition to the second coordinate generation mode”.

  In the second coordinate generation mode, if the transition condition to the first coordinate generation mode is not satisfied (NO in step S107), the coordinate information is determined if the transition condition to the standby state is satisfied (YES in step S108). The generation unit 116 returns to the standby state (step S100).

  Here, as “the transition condition to the standby state”, for example, “the touch panel 101 is not continuously touched for a certain period of time” and “the fingertip that causes the transition to the second coordinate generation mode” "Cancellation of touch for a certain period of time".

  After the transition to the third coordinate generation mode (step S110), the coordinate information generation unit 116 transitions to the second coordinate generation mode when the reception unit 116B receives the third specific operation 1160C (YES in step S111). (Step S114).

  At this time, when the third specific operation 1160C is “a touch operation with a finger different from the finger that has caused the touch that caused the transition to the third coordinate generation mode”, the touch with another finger is performed. Based on the plane coordinates indicating the position, plane coordinate information is generated and output to the processing device 3. At this time, the plane coordinate information indicating the touch position is converted into coordinate information in the display device 2 by the processing device 3, and the third cursor image CU3 (see FIG. 10) is displayed at the coordinate position represented by the converted coordinate information. Display.

  On the other hand, after the transition to the third coordinate generation mode (step S110) after the transition to the third coordinate generation mode (step S110), the coordinate information generation unit 116 determines that the first coordinate generation mode is satisfied. (Step S102).

  Here, as the “transition condition from the third coordinate generation mode to the first coordinate generation mode”, for example, “when the touch position changes after detecting a continuous touch at the same position of the touch panel 101 for a certain period of time, Alternatively, “when the touch is released”, “release of the fingertip that causes the third coordinate generation mode”, and “an operation that accelerates the change of the touch position on the touch panel 101”.

  Then, after the transition to the third coordinate generation mode (step S110), the coordinate information generation unit 116 returns to the standby state when the transition condition to the standby state is satisfied (YES in step S113) (step S100).

  Here, as the “transition condition from the third coordinate generation mode to the standby state”, for example, “a touch is not continuously detected for a certain period of time” and “a factor of the third coordinate generation mode” "Cancellation of a fingertip touch for a certain period of time".

  In addition, the coordinate information generation unit 116 transitions to the third coordinate generation mode when the transition condition to the third coordinate generation mode is satisfied (YES in step S115) after entering the second coordinate generation mode (step S114). (Step S110).

  Here, as the “transition condition from the second coordinate generation mode to the third coordinate generation mode”, for example, “stopping the pressing operation of the second operation key 11B”, “continuous for a certain time at the same position of the touch panel 101” For example, “the touch is released after the touch is detected” and “the touch is released again after the touch is released within a predetermined time and the touch is released”.

  Then, after the transition to the second coordinate generation mode (step S114), the coordinate information generation unit 116 returns to the standby state when the transition condition to the standby state is satisfied (YES in step S116) (step S100).

  Here, “the transition condition from the second coordinate generation mode to the standby state” includes “the touch panel 101 is not continuously touched for a certain period of time” and “the cause of the transition to the second coordinate generation mode” "Cancellation of a fingertip touch for a certain period of time".

<Operation of Processing Device 3>
FIG. 9 is a flowchart illustrating an example of a basic operation of the processing device 3. FIG. 10 is a diagram illustrating an example of a cursor image displayed on the display device 2. FIG. 11 is a diagram illustrating an example of a mouse cursor image.

  As shown in the flowchart of FIG. 7, since the controller device 1 outputs two-dimensional coordinate information and height information (Z) to the processing device 3, the CPU 33 of the processing device 3 outputs them. When received (YES in step S200), the following operation is performed.

  That is, when the CPU 33 receives height information equal to or greater than a certain value Z1 (where Z1 is a positive value) (YES in step S201), the CPU 33 converts the received two-dimensional coordinate information into coordinate information in the display device (step S202). ), The converted coordinate information and the first cursor image information are output to the display device 2, and the first cursor image CU1 (see FIG. 10) is displayed at the coordinate position represented by the converted coordinate information (step S203). ).

  When the CPU 33 receives zero-value height information (YES in step S204), the CPU 33 converts the received two-dimensional coordinate information into coordinate information in the display device (step S205), and converts the converted coordinate information and the second cursor. The image information is output to the display device 2, and the second cursor image CU2 (see FIG. 10) is displayed at the coordinate position represented by the converted coordinate information (step S206).

  Further, when the CPU 33 receives height information that is greater than 0 and less than a certain value Z1 (where Z1 is a positive value) (YES in step S207), the CPU 33 converts the received two-dimensional coordinate information into coordinate information in the display device. (Step S208), the converted coordinate information and the third cursor image information are output to the display device 2, and the third cursor image CU3 (see FIG. 10) is displayed at the coordinate position represented by the converted coordinate information. (Step S209).

  When the controller device 1 transitions to the mouse operation mode (step S210), the change amount information is output from the controller device 1 to the processing device 3, so that the processing device 3 receives the change information and receives the display device 2. The mouse cursor image information is output to display the mouse cursor image CU4 (see FIG. 11), and the mouse cursor image CU4 is moved on the display screen 20 based on the change amount information (step S211).

  Note that the CPU 33 of the processing device 1 changes the shapes and colors of the first cursor image CU1, the second cursor image CU2, and the third cursor image CU3. The first cursor image CU1 shown in FIG. 10 (a) is represented by a circle, and the second cursor image CU2 shown in FIG. 10 (b) is represented by a hatched square, as shown in FIG. 10 (c). The third cursor image CU3 is represented by a white square.

<Effect>
As described above, according to the first embodiment, the planar two-dimensional coordinates of the touch position on the touch panel surface of the touch panel 101 and the height information (Z) in the direction perpendicular to the touch panel surface are coordinated. Since it produces | generates according to production | generation modes, a user does not need to move, always holding a fixed height from a touch-panel surface.

  Therefore, the user can operate the operating device 1 without stress without being confused by the distance between the touch panel 101 and the finger. As a result, it is possible to provide the operating device 1 with good operability.

  In addition, since a plurality of types of cursor images CU1 to CU3 corresponding to the coordinate generation mode of the controller device 1 are displayed on the display screen 20 of the display device 2, it is possible to see at a glance which coordinate generation mode the controller device 1 is in. I understand.

  Further, when the finger is moved on the touch panel 101, the mouse cursor image displayed on the display screen 20 is also moved, so that the user can operate the touch panel 101 as if it were performed on the display screen 20 of the display device 2. It can be reminiscent and is very convenient.

(Embodiment 2)
Next, a second embodiment of the present invention will be described. Note that the basic configurations of the controller device 1, the display device 2, and the processing device 3 are the same as those described above. However, the function of the touch panel 101 (see FIG. 2) is different.

  In the second embodiment, the touch panel 101 has a function of detecting the touch position of the fingertip on the touch panel surface and a function of detecting the height of the fingertip existing above the touch panel surface with respect to the touch panel surface.

  That is, the touch panel 101 outputs the three-dimensional position information of the fingertip to the coordinate information generation unit 116 when the fingertip is located in a three-dimensional space within a certain height above the touch panel surface.

  The coordinate information generation unit 116 has a 3D operation mode and a 2D operation mode in addition to the first coordinate generation mode, the second coordinate generation mode, and the third coordinate generation mode described above.

  Furthermore, a function for the coordinate information generation unit 116 to select either the 3D operation mode or the 2D operation mode is assigned to the third operation key 17C.

  Further, the CPU 33 of the processing device 3 outputs any of the 2D operation mode information 35E, 3D operation mode information 35F, and the mouse operation mode 35G to the operation device 1.

<Operation of operating device 1>
FIG. 12 is a flowchart showing an example of the basic operation of the coordinate information generation unit 116 of the controller device 1 according to Embodiment 2 of the present invention. FIG. 13 is a diagram showing an example of a cursor image displayed on the display device 2 in the second embodiment of the present invention.

  When the coordinate information generation unit 116 receives the 3D operation mode information 35F (see FIG. 4) from the processing device 3 (YES in step S300), the coordinate information generation unit 116 enters the 3D operation mode (step S301). Here, the processing device 3 outputs the 3D operation mode information 35F, for example, when the 3D operation mode is selected by operating the third operation key 11C (see FIG. 2) and when the processing device 4 is the application. When the program 350A (see FIG. 5) is being executed,

  In the 3D operation mode, the coordinate information generation unit 116 generates solid coordinate information representing the three-dimensional position information of the fingertip in the three-dimensional space within a certain height above the touch panel, and outputs it to the processing device 4.

  The processing device 4 recognizes the XY coordinate position information of the fingertip above the touch panel 101 using the three-dimensional coordinate information, and converts it into coordinate information on the display device 2. Further, the processing device 4 recognizes the height information from the touch panel 101 of the fingertip using the three-dimensional coordinate position.

  The processing device 4 outputs the converted coordinate information to the display device 2 together with the cursor image corresponding to the height information. Thereby, the display device 2 displays a cursor image corresponding to the height information at the coordinate position corresponding to the touch position on the touch panel surface.

  Note that the cursor image CU5 displayed in the 3D operation mode preferably has a different color and shape from the cursor images CU1 to CU3 in the 2D operation mode, as shown in FIG.

  Further, when the 2D operation mode information 35E (see FIG. 4) is received from the processing device 3, the coordinate information generation unit 116 enters the 2D operation mode (step S303). Here, the processing device 3 outputs the 2D operation mode information 35E, for example, when the 2D operation mode is selected by operating the third operation key 11C (see FIG. 2) and when the processing device 4 is the application. When the program 350B (see FIG. 5) is being executed,

  When the coordinate information generation unit 116 enters the 2D operation mode, the coordinate information generation unit 116 does not use the three-dimensional position information of the fingertip output from the touchscreen 101, and represents the plane coordinate information that represents the coordinate position of the fingertip on the touchscreen surface of the touchscreen 101. Is used to perform the same processing as in the first embodiment described above.

  Furthermore, when receiving the mouse operation mode information 35G (see FIG. 4) from the processing device 3 (YES in step S304), the coordinate information generation unit 116 enters the mouse operation mode (step S305).

<Effect>
As described above, according to the second embodiment, even when the touch panel 101 capable of detecting three-dimensional position information is used, when the coordinate information generation unit 116 is in the 2D operation mode, the coordinate information generation unit 116 The same operation as in the first embodiment is performed.

  Therefore, even if a touch panel capable of detecting three-dimensional position information is used, the same effects as those of the first embodiment can be obtained.

(Other examples)
Note that the processing device 3 outputs any of the 3D operation mode information 351A, 2D operation mode information 351B, and mouse operation mode information 351C to the operation device 1 according to the execution state of any of the application programs 350A to 350C. May be.

  Further, the coordinate information generation unit 116 may be in any one of the 3D operation mode, the 2D operation mode, and the mouse operation mode based on the posture information output from the acceleration sensor S. Examples of the posture detected by the acceleration sensor S include a gripping posture, a stationary posture, a gripping posture in the horizontal direction, and a gripping posture in the vertical direction.

  Further, the controller device 1 may operate the vibration unit 117 (see FIG. 2) or the light emitting unit 118 (see FIG. 2) at a predetermined timing. In addition, as the operation timing, for example, the timing when the mode is switched to the second coordinate generation mode, the timing during the second coordinate generation mode, the timing during the touch on the touch surface in the 3D operation mode, and the touch on the touch surface during the mouse operation mode And the timing at which the operation mode is switched, the timing at which communication between the controller device 1 and the processing device 3 is established, and the like.

  While the information processing system and the operation device according to the embodiment of the present invention have been described above, the present invention is not limited to this embodiment.

  For example, the above-described device may be specifically configured as a computer system including a microprocessor, ROM, RAM, hard disk drive, display unit, keyboard, mouse, and the like. A computer program is stored in the RAM or hard disk drive. Each device achieves its functions by the microprocessor operating according to the computer program. Here, the computer program is configured by combining a plurality of instruction codes indicating instructions for the computer in order to achieve a predetermined function.

  Furthermore, some or all of the constituent elements constituting each of the above-described devices may be configured by one system LSI (Large Scale Integration). The system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip, and specifically, a computer system including a microprocessor, ROM, RAM, and the like. . A computer program is stored in the RAM. The system LSI achieves its functions by the microprocessor operating according to the computer program.

  Furthermore, some or all of the constituent elements constituting each of the above-described devices may be configured from an IC card that can be attached to and detached from each device or a single module. The IC card or module is a computer system that includes a microprocessor, ROM, RAM, and the like. The IC card or the module may include the super multifunctional LSI described above. The IC card or the module achieves its function by the microprocessor operating according to the computer program. This IC card or this module may have tamper resistance.

  Further, the present invention may be the method described above. Further, the present invention may be a computer program that realizes these methods by a computer, or may be a digital signal composed of the above computer program.

  Furthermore, the present invention provides a non-transitory recording medium that can read the computer program or the digital signal, for example, a flexible disk, a hard disk, a CD-ROM, an MO, a DVD, a DVD-ROM, a DVD-RAM, a BD ( It may be recorded on a Blu-ray (registered trademark) Disc), a semiconductor memory or the like. The digital signal may be recorded on these non-temporary recording media.

  In the present invention, the computer program or the digital signal may be transmitted via an electric communication line, a wireless or wired communication line, a network represented by the Internet, a data broadcast, or the like.

  The present invention may be a computer system including a microprocessor and a memory, wherein the memory stores the computer program, and the microprocessor operates according to the computer program.

  Further, by recording the program or the digital signal on the non-temporary recording medium and transferring it, or transferring the program or the digital signal via the network or the like, another independent computer It may be implemented by the system.

  Furthermore, the above embodiment and the above modification examples may be combined.

  The present invention can be applied to, for example, an information processing system including a smartphone and a processing device that performs processing based on a platform between the smartphone.

DESCRIPTION OF SYMBOLS 1 Operation apparatus 2 Display apparatus 10 Liquid crystal display 11A 1st operation key 11B 2nd operation key 11C 3rd operation key 20 Display screen 3 Processing apparatus 31 Radio | wireless information transmission / reception part 32 Coordinate information conversion part 35 Memory 35A Corresponding mode table 35B-35D Application Program 35E 2D operation mode information 35F 3D operation mode information 35G Mouse operation mode information 101 Touch panel 108 CPU
DESCRIPTION OF SYMBOLS 110 Communication interface 116 Coordinate information production | generation part 116A Specific operation table 116B Reception part 117 Vibrating part 118 Light emission part 1160A 1st specific operation 1160B 2nd specific operation 1160C 3rd specific operation CU1 1st cursor image CU2 2nd cursor image CU3 3rd cursor Image CU4 Mouse cursor image CU5 Cursor image in 3D operation mode S Acceleration sensor

Claims (31)

A touch panel having a touch panel surface and detecting a coordinate position on a plane touched by the touch panel surface;
A coordinate information generation unit that generates coordinate information representing the touched coordinate position, and includes a plurality of coordinate generation modes including a first coordinate generation mode that uses planar coordinate information representing a coordinate position of a fingertip on the touch panel surface. A coordinate information generator having
A receiving unit that receives a specific operation for switching from one coordinate generation mode to another coordinate generation mode among the plurality of coordinate generation modes;
An output unit that outputs the coordinate information generated by the coordinate information generation unit,
The coordinate information generation unit generates two-dimensional coordinates on the plane of the touch position on the touch panel surface and height information (Z) in a direction perpendicular to the touch panel surface according to the coordinate generation mode. apparatus. A touch panel having a touch panel surface and a function of detecting the touch position of the fingertip on the touch panel surface and a function of detecting the height of the fingertip existing above the touch panel surface with respect to the touch panel surface;
Without using the 3D operation mode for generating the 3D position information of the fingertip above the touch panel surface as 3D coordinate information and the 3D position information detected when the fingertip exists above the touch panel surface. A coordinate information generation unit having a 2D operation mode having a plurality of coordinate generation modes including a first coordinate generation mode using plane coordinate information representing the coordinate position of the fingertip on the touch panel surface;
An output unit for outputting the coordinate information generated by the coordinate information generation unit;
A reception unit that receives a specific operation for switching from one coordinate generation mode to another in the case of the 2D operation mode;
The coordinate information generation unit
(1) In the 3D operation mode, the fingertip is detected when the fingertip is present in a three-dimensional space within a certain height from the touch panel surface, including when the touch panel surface is touched. Generate the position as 3D coordinates,
(2) In the 2D operation mode, two-dimensional coordinates on the plane touched by the touch panel and height information (Z) in a direction perpendicular to the touch panel surface are generated according to the coordinate generation mode. Operation device. The coordinate information generation unit has at least a first coordinate generation mode and a second coordinate generation mode,
The receiving unit receives a first specific operation for switching from the first coordinate generation mode to the second coordinate generation mode;
The coordinate information generation unit
In the first coordinate generation mode, a positive value is given as the height information Z,
The operating device according to claim 1, wherein a zero value is given as the height information Z in the second coordinate generation mode. The coordinate information generation unit has at least a first coordinate generation mode, a second coordinate generation mode, and a third coordinate generation mode,
The reception unit performs a second specifying operation for switching from the first coordinate generation mode to the third coordinate generation mode, and a third specifying operation for switching from the third coordinate generation mode to the second coordinate generation mode. Accept,
The coordinate information generation unit
In the first coordinate generation mode, as the height information Z, a positive value greater than a certain value is given,
In the second coordinate generation mode, a zero value is given as the height information Z,
The operating device according to claim 1 or 2, wherein a positive value less than a certain value is given as the height information Z in the third coordinate generation mode. The coordinate information generation unit generates a change amount information indicating a change amount of the plane position coordinates when the plane position coordinates of the touch position of the fingertip on the touch panel change within a preset setting time. The operating device according to any one of claims 1 to 4. The coordinate information generation unit
The operation device according to any one of claims 1 to 5, wherein when detecting that the touch panel is touched by at least two fingertips, coordinates of the touch position are generated. The coordinate information generation unit
At least after the operating device is activated, it is in a standby state that is not in any coordinate generation mode,
The operating device according to any one of claims 1 to 6, wherein when the touch panel is detected to be touched with at least one fingertip from the standby state, the mode is shifted to the first coordinate generation mode. The coordinate information generation unit
The operating device according to claim 4, wherein when the accepting unit accepts a first specific operation in the first coordinate generation mode, the operation unit transits to the second coordinate generation mode. The coordinate information generation unit
When the accepting unit accepts a second specific operation in the first coordinate generation mode, the mode transitions to the third coordinate generation mode, and the accepting unit accepts the third specific operation in the third coordinate generation mode. The operating device according to claim 4, wherein, in the event of a failure, a transition is made to the second coordinate generation mode. In any one of the first coordinate generation mode, the second coordinate generation mode, and the third coordinate generation mode, when the touch is not continuously detected for a certain period of time, the standby state is set. The operating device according to any one of claims 4, 8, and 9. further,
A first operation key capable of transitioning the coordinate information generation unit from the first coordinate generation mode to the second coordinate generation mode;
The coordinate information generation unit transitions to the second coordinate generation mode when the reception unit receives an operation of the first operation key as the first specific operation, and the coordinate information generation unit The operation device according to any one of claims 3, 4, and 8, wherein when the operation is not accepted, the mode is shifted to the first coordinate generation mode. further,
The coordinate information generation unit
In the first coordinate generation mode, as the first specific operation, when the touch panel detects a continuous touch for a certain time at the same position of the touch panel, the touch panel transits to the second coordinate generation mode, and the touch The operation device according to any one of claims 3, 4, and 8, wherein when the is released, the mode is shifted to the first coordinate generation mode. The first specific operation is a series of operations of touching the plane again on the touch panel after releasing the touch within a certain time after continuing the operation of stopping the user's finger at the same position on the touch panel.
The coordinate information generation unit transitions to the second coordinate generation mode when the reception unit receives the first specific operation in the first coordinate generation mode, and when the touch is released, The operating device according to claim 3, wherein the operation device transitions to a one-coordinate generation mode. The first specific operation is a touch operation on the plane by a finger different from the finger performing the touch that has caused the transition to the first coordinate generation mode,
The coordinate information generation unit
Generating the plane coordinate information based on the plane coordinates indicating the touch position of the finger making the touch that caused the transition to the second coordinate generation mode;
Transition to the first coordinate generation mode by releasing the touch of the fingertip that causes the transition to the second coordinate generation mode,
The operating device according to any one of claims 3, 4, and 8, wherein a transition to the standby state is made by releasing the touch of the fingertip that is a factor of the transition to the second coordinate generation mode for a predetermined time or longer. The coordinate information generation unit
In the first coordinate generation mode, when the touch panel detects a touch for a certain period of time at the same position of the touch panel as the second specific operation, the mode transitions to the third coordinate generation mode,
The operating device according to claim 4 or 9, wherein when the touch position is changed or when the touch is released, the mode is shifted to the first coordinate generation mode. In the first coordinate generation mode, the coordinate information generation unit transitions to the third coordinate generation mode as the second specific operation slows down the change in the touch position on the touch panel, and the change in the touch position. The operating device according to claim 4, wherein the operation mode is shifted to the first coordinate generation mode as the speed is increased. further,
A second operation key capable of transitioning the coordinate information generation unit from the third coordinate generation mode to the second coordinate generation mode;
The coordinate information generation unit
When the accepting unit accepts that the second operation key has been operated as the third specific operation, the mode shifts to the second coordinate generation mode, and then no operation of the second operation key is accepted. The operation device according to claim 4 or 9, wherein the operation mode is changed to the third coordinate generation mode. In the third coordinate generation mode, the coordinate information generation unit further detects the second coordinate when the touch panel detects a continuous touch for a certain time at the same position of the touch panel as the third specific operation. The operating device according to claim 4, wherein the operation device transitions to a generation mode, and then transitions to the third coordinate generation mode when the touch is released. The third specific operation is a series of operations for touching the plane again on the touch panel after the touch is released within a predetermined time,
The coordinate information generation unit transitions to the second coordinate generation mode when the reception unit receives the third specific operation in the third coordinate generation mode, and then when the touch is released, The operating device according to claim 4 or 9, wherein the operation mode transits to the third coordinate generation mode. The third specific operation is a touch operation on the plane by a finger different from the finger performing the touch that has caused the transition to the first coordinate generation mode and the third coordinate generation mode,
The coordinate information generation unit
Generating the plane coordinate information based on the plane coordinates indicating the touch position of the finger making the touch that caused the transition to the third coordinate generation mode;
Transition to the first coordinate generation mode by releasing the touch of the fingertip that is a factor of the third coordinate generation mode,
The operating device according to claim 4, wherein the operation device transitions to a standby state by releasing the touch of the fingertip that is a factor of the third coordinate generation mode for a predetermined time or longer. The operating device according to any one of claims 1 to 20,
A display device;
A processing device that converts the coordinate information output from the output unit into coordinate information in the display device, and displays a cursor image at a coordinate position represented by the converted coordinate information;
A communication unit for communicating at least the coordinate information between the operating device and the processing device,
When the processing device receives the coordinate information and the height information from the output unit,
When the height information receives a positive coordinate value greater than a set value, the first cursor is displayed,
When the height information receives a zero value, a second cursor is displayed,
An information processing system for displaying a third cursor when the height information receives a positive coordinate value less than a set value. The information processing system according to claim 21, wherein each of the first cursor image, the second cursor image, and the third cursor image is different in at least one of a shape and a color. The processor is
The information processing system according to claim 21 or 22, wherein a mouse cursor image is displayed as the cursor image on the display unit when the coordinate information generation unit is executing the mouse operation mode. The processing device causes the coordinate information generation unit to change the shape and color of the cursor image displayed on the display device between the 3D operation mode and the 2D operation mode. The information processing system of any one of -23. The processing apparatus holds operation mode information indicating any of the 2D operation mode, the 3D operation mode, and the mouse operation mode,
Providing the operation device with the operation mode information held by the processing device via the communication unit;
The coordinate information generation unit executes any one of the 2D operation mode, the 3D operation mode, and the mouse operation mode according to the operation mode information that has been provided. Information processing system. In the processing device, a plurality of programs for receiving an operation by the operation device is installed in advance,
During the execution of each of the programs, the processing device performs one of the 3D operation mode, the 2D operation mode, and the mouse operation mode according to the execution state of the program or an individual program being executed. The information processing system according to claim 25, wherein the operation mode information to be transmitted is transmitted to the coordinate information generation unit. The processor is
27. The coordinate information generation unit includes a selection key for selecting which of the 3D operation mode, the 2D operation mode, and the mouse operation mode is selected. Information processing system. The coordinate information generation unit executes any one of the 3D operation mode, the 2D operation mode, and the mouse operation mode based on posture information representing the posture of the operation device. The information processing system according to claim 1. further,
The information processing system according to claim 28, further comprising an acceleration sensor that detects an attitude of the operation device and outputs the attitude information representing the detected attitude. The information processing system according to any one of claims 21 to 29, wherein the operation device further includes a vibration unit. The information processing system according to any one of claims 21 to 30, wherein the operation device further includes a light emitting unit.

Images