JP2012088919A - Input operation device and input operation system for touch panel - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a new input operation device and a new input operation system capable of assisting an input operation in a static capacitance type touch panel.SOLUTION: An input operation device system 1 includes: an electrode panel 12 in which a plurality of electrode units including a plurality of electrode plates and switches provided between the plurality of electrode plates, for switching between connection and disconnection with each other in the electrode plates, are arranged in matrix from, the electrode panel disposed in contact with or adjacent to a touch panel 104; and a control controller 50 connected to the electrode panel 12, for controlling driving of the switches in the electrode units provided in the electrode panel 12. Driving control of the switches changes coupling capacitance of the electrode plates to change static capacitance at corresponding positions on the touch panel 104, thereby allowing action similar to a touch operation to be obtained.

Description

  The present invention relates to an input operation device and an input operation system that are used in devices equipped with a capacitive touch panel and perform an input operation of the touch panel.

  Conventionally, touch panels are used in various devices. Here, the touch panel refers to a device in which a device for detecting a touch operation by a user and a display device are integrally configured.

  For example, touch panels are widely used such as mobile phones, electronic book terminals, game terminals, digital cameras, car navigation systems, multifunction peripherals, ticket machines such as tickets, ATMs (automatic teller machines).

  The touch panel has various advantages. For example, a physical button is not required, and the operation target on the display screen matches the input target, so that an intuitive operation is possible. There are advantages such as being advantageous in terms of flexibility and design freedom, and being advantageous in terms of maintenance because dust or the like does not enter the gap as in the case of keyboards and switches.

In particular, in a capacitive touch panel, it is possible to detect two or more touches at the same time and perform operations such as image enlargement or reduction, for example.
A capacitive touch panel is disclosed in, for example, Japanese Patent Application Laid-Open No. 2009-258935.

JP 2009-258935 A

  However, there are disadvantages in the touch panel as described above. For example, since an operation feeling (click feeling) like a keyboard or a mouse cannot be obtained, a touch operation may be uncomfortable for a person who is used to operating the keyboard or mouse. In addition, it is unsuitable for character input or the like due to the fact that an operation feeling (click feeling) like a keyboard or a mouse cannot be obtained. For example, it is generally difficult for a touch panel to input characters at an input speed as when a keyboard is used.

  The present invention has been made in view of these problems, and an object thereof is to provide a new input operation device and an input operation system that can assist an input operation on a capacitive touch panel.

The present invention made in order to solve the above problems is an input operation device that performs an input operation on a capacitive touch panel, and includes an electrode panel and a controller.
The electrode panel is configured by arranging a plurality of electrode units having a plurality of electrode plates and a switch provided between the electrode plates to switch connection and non-connection between the electrode plates in a matrix. A touch panel is disposed in contact with or adjacent to the electrode panel.

The control controller is connected to the electrode panel and controls driving of a switch in an electrode unit included in the electrode panel.
According to such an input operation device of the present invention, by turning on a switch of an electrode unit at a desired position by a controller, a plurality of electrode plates corresponding to the switch are electrically connected to each other, The coupling capacity changes. At this time, in the touch panel that is in contact with or adjacent to the electrode panel, the capacitance changes in a portion of the electrode unit of the electrode panel that faces the electrode unit whose coupling capacitance has changed. If it does so, in the apparatus provided with the touch panel or the touch panel, the position where the electrostatic capacitance on the touch panel changed will be detected. This detection principle is not different from the detection principle in which, for example, the touched location is detected by a change in capacitance generated on the touch panel when the touch panel is touched with a finger.

  That is, according to the input operation device of the present invention, by controlling the drive of the electrode unit of the electrode panel (more specifically, the drive of the switch of the electrode unit), the same input as when operating the touch panel with a finger Operation can be realized.

  Further, the input operation device of the present invention performs image recognition on an imaging unit that captures the display screen of the touch panel and an image captured by the imaging unit, and detects an operation location on the display screen of the touch panel based on the recognized image. And a microcomputer that identifies an electrode unit that exists at a position opposite to the operation location. The microcomputer outputs information representing the identified electrode unit to the control controller, and the control controller outputs the information from the microcomputer. The driving of the switch in the electrode unit indicated by the information may be controlled.

  According to such a configuration, even if the display mode on the touch panel is different for each device equipped with the touch panel, the display mode on the touch panel can be recognized by image recognition. Therefore, the control of the electrode unit in the electrode panel can be realized. Therefore, versatility can be improved.

  In the present invention, one or a plurality of icons are displayed on the display screen of the touch panel, and the microcomputer includes a memory that stores in advance information on icons that can be displayed on the display screen of the touch panel. The type of the icon displayed on the display screen of the touch panel and recognized by the image recognition may be specified based on the icon information stored in advance in the memory.

  According to such a configuration, the type of icon displayed on the display screen of the touch panel can be recognized more reliably and easily, and the accuracy with which a desired icon is operated on a device equipped with the touch panel is improved. Can be increased.

  Further, the present invention may be an input operation system for performing an input operation on a capacitive touch panel, and the input operation system is provided between a plurality of electrode plates and the electrode plates. A plurality of electrode units having a switch for switching between connection and non-connection, and an electrode panel configured such that a touch panel is arranged in contact with or adjacent to each other, and connected to the electrode panel, A controller for controlling the drive of the switch in the electrode unit included in the electrode panel, an imaging unit for imaging the display screen of the touch panel, and image recognition for the image captured by the imaging unit, and the display screen of the touch panel based on the recognized image Detects the operation location on the top and exists at a position facing the operation location A microcomputer that identifies the electrode unit; and an output device that displays an image captured by the imaging unit. The microcomputer outputs information representing the identified electrode unit to the control controller, and the control controller is output from the microcomputer. It may be configured to control driving of the switch in the electrode unit indicated by the information.

  In the input operation system of the present invention, one or a plurality of icons are displayed on the display screen of the touch panel, and the microcomputer stores in advance information on icons that can be displayed on the display screen of the touch panel. The type of icon that is displayed on the display screen of the touch panel and recognized by image recognition may be specified based on the icon information stored in advance in the memory.

  The input operation system according to the present invention further includes an input device for the user to input information, and the microcomputer is information input by the user via the input device, and is desired in the capacitive touch panel. Based on the information representing the input operation, an electrode unit to be driven among the electrode units included in the electrode panel may be identified, and information representing the identified electrode unit may be output to the control controller.

  According to such an input operation system, an effect similar to that of the input operation device described above can be obtained.

It is a lineblock diagram of input operation system 1 of this embodiment. 2 is a diagram illustrating an example of a mobile terminal 100. FIG. It is explanatory drawing of the electrode pattern of a projection type capacitive touch panel. It is explanatory drawing of the electrode pattern and laminated structure of a projection type capacitive touch panel. It is explanatory drawing of the detection principle of a projection capacitive touch panel. It is a block diagram of the electrode panel 12 of the input operation system 1 of this embodiment. It is explanatory drawing of the operation principle by the input operation system 1 of this embodiment. It is a flowchart showing the process performed in the input operation system 1 of this embodiment. It is drawing explaining the touch panel operation by the input system 1 of this embodiment. It is an enlarged view explaining the touch panel operation by the input system 1 of this embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a configuration diagram of an input operation system 1 according to the present embodiment.
The input operation system 1 according to the present embodiment is a device that operates (inputs) the touch panel 104 provided in the mobile terminal 100 (see FIG. 2 for details).

First, the portable terminal 100 to be operated will be described.
FIG. 2 is a diagram illustrating an example of the mobile terminal 100.
The mobile terminal 100 includes a touch panel display device (hereinafter simply referred to as a touch panel) 104 in a terminal main body 102.

  On the touch panel 104, for example, icon groups 110 and 120 including icons indicating applications are displayed. For example, when the icon 111 of the icon group 110 is touched by the user, the application represented by the icon 111 is activated on the mobile terminal 100.

  More specifically, when a touch operation is performed on the touch panel 104 by the user, a location touched by the user on the touch panel 104 is detected, and the icon 111 displayed at the detected location is recognized, and the icon is displayed. An application corresponding to 111 is started.

  The icon group 110 may include, for example, icons related to basic applications or standard applications. Examples of the basic application or standard application include a web browser and mail software. And the arrangement position (display position) of the icon which concerns on such a basic application or a standard application on the touch panel 104 may be defined as a common thing in each of the some portable terminal 100. FIG.

  In the icon group 120, the number and type of icons, the arrangement position (display position), and the like can be changed by adding an application (adding by downloading an application) by the user (in other words, the number of icons for each portable terminal 100, An icon such as “the type or the display position is various” may be included.

  The touch panel 104 is a touch panel that detects a change in electrical characteristics of a touched portion when a touch operation is performed by a user. Specifically, it is a capacitive touch panel, and more specifically, a projected capacitive touch panel.

3 to 5 are diagrams for explaining a projected capacitive touch panel.
As shown in FIG. 3, the projected capacitive touch panel has a predetermined pitch in the X-axis direction (shown as a vertical direction in FIG. 3. Further, in the present embodiment, for example, the short side direction of the touch panel 104). And the Y-axis direction (shown as the horizontal direction of the drawing in FIG. 3. Further, in this embodiment, for example, the long side of the touch panel 104. And a plurality of electrodes 2Y arranged to extend at a predetermined pitch.

In FIG. 3, a plurality of electrodes 1X in the X-axis direction and a plurality of electrodes 2Y in the Y-axis direction are shown separately.
In the actual product, the plurality of electrodes 1X in the X-axis direction and the plurality of electrodes 2Y in the Y-axis direction shown in FIG. 3 are arranged so as to intersect with each other as shown in FIG. To do.

  As shown in FIGS. 3 and 4, the plurality of electrodes 1 </ b> X includes an electrode portion 1 a and a connection portion 1 b that connects the electrode portions 1 a. Each of the plurality of electrodes 1X has a configuration in which a plurality of electrode portions 1a and connection portions 1b are alternately arranged in the X-axis direction.

  The plurality of electrodes 2Y include an electrode portion 2a and a connection portion 2b that connects the electrode portions 2a. Each of the plurality of electrodes 2Y has a configuration in which a plurality of electrode portions 2a and connection portions 2b are alternately arranged in the Y-axis direction.

The electrode 1X and the electrode 2Y are each made of a transparent conductive material such as ITO (Indium Tin Oxide).
Next, as shown in FIG. 5A in FIG. 5, the plurality of electrodes 2Y are arranged on the substrate 200 and covered with an insulating film 202 formed in an upper layer thereof. The insulating film 202 may be made of glass, for example.

  The plurality of electrodes 1X are disposed on the insulating film 202 and covered with a protective film 204 formed in an upper layer thereof. The protective film 204 may be made of, for example, PET (polyethylene terephthalate) resin.

  Then, as shown in FIG. 5B, when the user touches the touch panel 104 with the finger 210, a capacitance indicated by C <b> 1 and C <b> 2 is newly formed between the finger 210 and the electrode 1 </ b> X. Capacitors indicated by C3 and C4 are newly formed between the electrode 2Y and the electrode 2Y. In addition, C1-C4 shown in FIG. 5 does not indicate that a capacitor is actually connected. The same applies to FIG. 7 described later.

  In this way, when the user touches the touch panel 104 with the finger 210, the coupling capacitance between the electrodes 1X in the same row and the coupling capacitance between the electrodes 2Y in the same row change. In the touch panel 104 (in the portable terminal 100), a voltage change caused by a change in the coupling capacitance between the electrode 1X and the electrode 2Y formed on the touch panel 104 is detected, and the touched position (the position in the X-axis direction and the position) The position in the Y-axis direction) is detected.

Specifically, among the columns of the electrodes 1X and 2Y, the intersection of the column where the voltage change is detected can be detected as the touched position.
In addition, a capacitor is formed by the electrode 1X, a capacitor is similarly formed by the electrode 2Y, and an oscillation circuit is configured by these capacitors, and a change in the oscillation frequency of the oscillation circuit due to a change in the capacitance C is detected. Thus, the touched position may be detected. Specifically, for the electrode 1X and the electrode 2Y, an oscillation circuit is configured for each column, and among the columns of the electrodes 1X and 2Y, the column where the oscillation frequency is changed is detected, and the intersection is touched. Can be detected as

Returning to FIG. 1, the input operation system 1 of this embodiment will be described.
The input operation system 1 includes a mounting table 2 on which the mobile terminal 100 is mounted, a control device 3, an input device 4, and an output device 5.

The input device 4 is specifically a keyboard (not shown) and a mouse (not shown). The output device 5 is specifically a display device (not shown).
The mounting table 2 has a mounting surface 10 on which the mobile terminal 100 is mounted. The mounting surface 10 is surrounded by a jaw portion 11 protruding from the mounting surface 10 at the end of the mounting table 2. Such a jaw portion 11 urges the mobile terminal 100 to be placed in an appropriate position.

The portable terminal 100 is placed on the placement surface 10 of the placement table 2 such that the surface of the touch panel 104 of the portable terminal 100 faces the placement surface 10.
The mounting table 2 includes an electrode panel 12 for operating the touch panel 104, an object detection sensor 14 for detecting that the mobile terminal 100 is mounted, and a CCD (Charge Coupled Device) camera 16. Yes.

The electrode panel 12 is embedded in the mounting surface 10 of the mounting table 2. Details of the electrode panel 12 will be described later.
The object detection sensor 14 is provided so as to be embedded in the jaw portion 11, and is connected to the microcomputer 20 via the input port 26 of the microcomputer 20. The object detection sensor 14 includes an LED light source (not shown) and a phototransistor (not shown).

  When an object approaches the object detection sensor 14, light from the LED light source in the object detection sensor 14 is reflected by the object and enters the base portion of the phototransistor. The phototransistor generates a current in response to incident light. The current generated in the phototransistor is amplified and output as a signal. This signal is input to the microcomputer 20 via the input port 26. The microcomputer 20 confirms that the portable terminal 100 is placed on the placement surface 10 based on a signal output from the object detection sensor 14 when the portable terminal 100 is placed on the placement surface 10 of the placement table 2. To detect.

  The CCD camera 16 includes a lens unit 16a and an image sensor 16b. An image (light) is formed by the lens unit 16a and irradiated to a light receiving unit (not shown) of the image sensor 16b. In the light receiving portion of the image sensor 16b, photodiodes (not shown) as light receiving elements are arranged in units of millions to tens of millions. Each photodiode generates and stores electric charges according to the intensity of incident light. The image sensor 16b is provided with a transfer unit (not shown), and the charges stored in the respective light receiving elements are sent out by the transfer unit in the order in which the light receiving elements are arranged and output to the outside. More specifically, an analog signal corresponding to the amount of charge is output, the analog signal is converted into a digital signal by an A / D conversion circuit (not shown), and the digital signal is sent to an image processing unit 40 described later. Is output.

  The CCD camera 16 is disposed toward the mounting surface 10 at the bottom 2t, which is the portion opposite to the mounting surface 10 in the mounting table 2. More specifically, it is disposed toward the electrode panel 12 on the mounting surface 10. The CCD camera 16 is provided so as to be able to image a region covering the electrode panel 12. The electrode panel 12 is composed of transparent electrodes and is transparent as a whole. The electrode panel 12 is mounted on the mounting surface 10 with the transparent electrode panel 12 sandwiched from the bottom 2t where the CCD camera 16 is provided. The placed mobile terminal 100 and the touch panel 104 of the mobile terminal 100 can be seen. In the present embodiment, the CCD camera 16 is provided so as to capture at least the entire touch panel 104 of the mobile terminal 100 placed on the placement surface 10.

The control device 3 includes a microcomputer 20, an image processing unit 40, a control controller 50, and a network connection circuit 60.
The microcomputer 20 is a known device including a CPU 22, a memory 24, an input port 26, and an output port 28.

  The CPU 22 is a computing device that executes various programs stored in the memory 24 to process information. The memory 24 includes a ROM and a RAM (not shown), and stores programs executed by the CPU 22, calculation results of the CPU 22, and the like. The input port 26 is an interface for connecting to an external device and inputting data from the external device. The output port 28 is an interface for connecting to an external device and outputting data to the external device.

The CPU 22, the memory 24, the input port 26, and the output port 28 are connected to each other by a data bus 30, an address bus 32, and a control bus 34.
The data bus 30 is a signal line for transmitting information to the CPU 22. Here, the information includes numerical information for the CPU 22 to execute a calculation, information indicating a calculation result of the CPU 22, and the like. Further, a signal input from an external device via the input port 26 and a signal output to an external device via the output port 28 are also included in the information here.

The address bus 32 is a signal line for designating (exchanging) a storage location for storing data and instructions (storage location in the memory 24).
The control bus 34 is a signal line that transmits information for designating the operation of reading or writing data. In the signal on the control bus 34, for example, the Read signal for designating “read” for the CPU 22 to receive data from the memory 24 or the input port 26, and the CPU 22 for storing the data in the memory 24 or the output port 28 “ There is a Write signal designating “write”.

  The image processing unit 40 receives a digital signal representing an image captured by the CCD camera 16 from the CCD camera 16 (more specifically, from the image sensor 16b). Then, based on the digital signal, that is, a digital signal representing the amount of received light (charge amount) of each light receiving element of the image sensor 16b, density analysis and color analysis (color separation) are performed according to the amount of charge, and the digital representing the color image. Form data.

The image processing unit 40 is connected to the input port 26 and the output port 28 of the microcomputer 20.
Digital data representing a color image generated in the image processing unit 40 is output to the output device 5 (display device) under the control of the microcomputer 20, and the color image is displayed on the output device 5. Specifically, a color image representing the touch panel 104 of the mobile terminal 100 is displayed on the output device 5. The digital data representing the color image is also input to the microcomputer 20 via the input port 26 of the microcomputer 20.

  The controller 50 is connected to the electrode panel 12 provided on the placement surface 10. The controller 50 is also connected to the output port 28 of the microcomputer 20, and controls the driving of the electrode panel 12 (energization of the electrode panel 12) under the control of the microcomputer 20.

  The network connection circuit 60 is a circuit for connecting the control device 3 to the Internet network 90. The network connection circuit 60 is connected to the microcomputer 20 via the input port 26 and the output port 28 of the microcomputer 20. Under the control of the microcomputer 20, data transmission / reception with the Internet network 90 is managed.

FIG. 6 is a diagram showing the configuration of the electrode panel 12.
The electrode panel 12 has a configuration in which electrode units 17 having two electrode plates 13 and a semiconductor switch 15 provided between the two electrode plates 13 are arranged in a matrix. The electrode plate 13 may be formed of ITO, which is a transparent conductive material similar to the electrode 1X and the electrode 2Y, for example.

  The electrode unit 17 may be provided in units of several hundred to several thousand, for example, and the greater the number, the higher the resolution of the operation position on the touch panel 104. In consideration of replacing the operation with the finger 210 (see FIG. 5), the number of electrode units 17 may be about several hundred.

  The semiconductor switch 15 in each electrode unit 17 is connected to the controller 50 via the signal line 19. The semiconductor switch 15 in the electrode unit 17 is on / off controlled by a signal from the controller 50. The ON state is an energized state (connected state), and the OFF state is a non-energized state (non-connected state).

  The position of the electrode unit 17 can be specified by the X coordinate value and the Y coordinate value in the electrode panel 12, respectively. That is, an X coordinate value and a Y coordinate value on the electrode panel 12 are given for each electrode unit 17. Here, the X coordinate value and the Y coordinate value given to each electrode unit 17 have a width according to the size of the electrode unit 17. For example, with respect to a predetermined electrode unit 17, coordinate values are given for each electrode unit 17 such as an X coordinate value of 10 (mm) to 12 (mm) and a Y coordinate value of 20 (mm) to 22 (mm). Such coordinate values are stored in advance in the memory 24 of the microcomputer 20.

  The controller 50 receives from the microcomputer 20 the data of the X coordinate value and the Y coordinate value of the electrode unit 17 to be controlled on / off, and the electrode unit 17 corresponding to the received X coordinate value and Y coordinate value. On / off control is executed.

FIG. 7 is a diagram for explaining the operation principle of the touch panel 104 by the input operation system 1.
As described above, in the touch panel 104, the plurality of electrodes 2Y are formed on the substrate 200 and the upper layer thereof is covered with the insulating film 202, and the plurality of electrodes 1X are formed on the insulating film 202, and the upper layer is a protective film. The structure covered with 204 is provided.

  7A shows a state in which the portable terminal 100 is placed on the placement surface 10 (see FIG. 1) and the touch panel 104 of the portable terminal 100 and the electrode panel 12 of the placement surface 10 face each other. Further, the semiconductor switch 15 in the electrode unit 17 is in an off state (non-connected state). One electrode plate 13 (the electrode plate 13 on the left side of the drawing) of the two electrode plates 13 and the electrode 1X closer to the electrode plate 13 (the drawing of the two electrodes 1X shown in the drawing) A C5 capacitor is formed between the left electrode 1X) and a C6 capacitor is formed between the other electrode plate 13 and the electrode 1X. Similarly, one of the two electrode plates 13 (the electrode plate 13 on the left side of the drawing) and the electrode 2Y closer to the electrode plate 13 (of the two electrodes 2Y shown in the drawing) A capacitance of C7 is formed between the electrode 2Y) on the left side of the drawing and a capacitance of C8 is formed between the other electrode plate 13 and the electrode 2Y.

  7B in FIG. 7 shows a state where the semiconductor switch 15 in the electrode unit 17 is turned on (connected state). In this case, the two electrode plates 13 are electrically connected, and in the formation of the capacitance, the two electrode plates 13 can be regarded as an electrode plate 13a in which the two electrode plates 13 are integrated.

  In FIG. 7B, the capacitances C10 and C11 are formed between the electrode 1Y and the two electrode plates 13 (virtual electrode plate 13a as an integral unit) that can be regarded as an integral unit in terms of the capacitance forming function. This represents a state in which C12 and C13 are formed between the two electrode plates 13 (virtual electrode plate 13a as a unit) and the electrode 2Y. In this case, the coupling capacity of the adjacent electrode 1X and the coupling capacity of the adjacent electrode 2Y are changed as compared with FIG. 7A.

  As described above, the semiconductor switch 15 in the electrode unit 17 is controlled to be turned on / off, whereby the area of the electrode that forms a capacitance with the electrode 1X and the electrode 2Y (specifically, the electrode plate 13) The area, that is, the functional area for forming the capacitance) changes. Due to this change, a voltage change occurs in the electrode 1X and the electrode 2Y. This voltage change is the same voltage change as the voltage change detected when the touch panel 104 is touched with the finger 210 (see FIG. 5) (the voltage change caused by the change in capacitance when touched), It can be detected by the same principle as the detection principle of the touch operation.

  Therefore, by turning on / off the semiconductor switch 15 in the electrode unit 17, more specifically, by turning the semiconductor switch 15 from the off state (non-connected state) to the on state (connected state), the finger 210 The touch panel 104 can be operated in the same manner as when the touch panel 104 is touched and the touch panel 104 is operated.

  FIG. 8 is a flowchart illustrating an example of processing executed by the microcomputer 20 of the control device 3 (more specifically, executed by the CPU 22). This process is a process for operating the touch panel 104 in the portable terminal 100 by the input operation system 1, and is periodically executed.

  In this process, first, in S100, whether or not the portable terminal 100 is detected (in other words, whether or not the portable terminal 100 is placed on the placement surface 10 of the placement table 2) is determined from the object detection sensor 14. It is determined based on the signal.

If it is determined that the mobile terminal 100 is not detected (S100: NO), the process is temporarily terminated.
On the other hand, when it is detected that the mobile terminal 100 is placed on the placement surface 10 based on the signal from the object detection sensor 14, the process proceeds to S110.

In S <b> 110, control is performed to cause the CCD camera 16 and the image processing unit 40 to perform imaging processing of the touch panel 104 of the mobile terminal 100.
More specifically, the image processing unit 40 is caused to analyze data output from the CCD camera 16 (digital data representing the amount of charge stored in each light receiving element and representing a captured image). As described above, the image processing unit 40 generates data representing the color image of the touch panel 104 based on the data output from the CCD camera 16.

  Next, the process proceeds to S120, and a process of outputting data representing the color image generated in the image processing unit 40 to the output device 5 is executed. Specifically, an output command for data representing a color image is transmitted to the image processing unit 40, and data representing the color image generated by the image processing unit 40 is output to the output device 5. Let In the present embodiment, as described above, the output device 5 is a display device, and a color image representing the touch panel 104 is displayed on the display device (output device 5).

Here, the microcomputer 20 also executes a process of receiving data representing a color image generated by the image processing unit 40 via the input port 26.
Next, in S130, image recognition is executed based on the data representing the color image received from the image processing unit 40.

  Specifically, the image of the touch panel 104 represented by the data received from the image processing unit 40 is analyzed, and a process of extracting an icon that appears in the image (included in the image) is executed. The icons here include, for example, icons representing applications as indicated by icon groups 110 and 120 in FIG. In addition, icons for each button representing the keyboard as shown in FIGS.

  In S130, the design of each icon is recognized and the position of each icon is specified. Specifically, in the present embodiment, a reference mark (see FIG. 9, reference mark P) serving as a reference may be formed on the surface of the electrode panel 12 on the side facing the touch panel 104. Then, the reference mark may be captured when the touch panel 104 is imaged, and the position of the icon may be specified based on the reference mark. That is, the position of the icon image included in the image may be calculated using the position of the reference mark included in the captured image as the origin.

Next, it progresses to S140 and a matching process is performed about the icon extracted by S130. Specifically, a process for specifying the type of the extracted icon is performed.
Here, in the mobile terminal 100, the icon design is unique to each icon. In the microcomputer 20, the icon design information and the application type or button type (button type representing the keyboard) information corresponding to the icon are associated with each icon and stored in advance in the memory 24. The

  Then, the microcomputer 20 performs a process (matching process) for determining whether or not the icon recognized in the process of S130 is included in the icon information stored in advance in the memory 24 based on the icon design, for example. If it is included, for the icon that is determined to be included and recognized in step S130, the application type or button type that the icon represents is read from the memory 24 and temporarily stored in a predetermined work area. Remember.

  Next, it progresses to S150 and it is determined whether an automatic operation process is performed. Here, the automatic operation process is a process for operating the touch panel 104, and is a process for realizing an operation based on a preset program, not an operation based on an input from the user.

  For example, in the control apparatus 3, when the portable terminal 100 is mounted on the mounting base 2, it can be set so that a predetermined touch panel operation is automatically executed. For example, when the mobile terminal 100 is placed on the placing table 2, a process such as automatically starting web browsing software (application), a process such as starting mail software (application), and the like It can be set to automatically execute a process such as inputting a predetermined character string after starting a predetermined application.

In other words, in S150, it is determined whether or not such automatic processing has been set.
If it is determined in S150 that automatic operation processing is to be executed (automatic operation is set) (S150: YES), the process proceeds to S180, and processing for recognizing the operation content of the automatic operation is executed. Here, the operation content is recognized based on data representing the operation content stored (set) in advance.

  Next, proceeding to S190, based on the operation content recognized in the process of S180, an icon representing the application or button to be activated is extracted from the information of the icon specified in the processes of S130 and S140.

  In other words, an icon representing an application to be started or an icon representing a button to be operated among the icons displayed on the touch panel 104 is specified. Note that the display position of the icon is specified in S130 as described above. Then, based on the display position of the icon, the position of the electrode panel 12 facing the icon is calculated, and the address (X coordinate value and Y coordinate) of the electrode unit 17 at the position facing the icon on the electrode panel 12 is calculated. Value).

  Next, in S200, information indicating the address (X coordinate value and Y coordinate value) of the electrode unit 17 calculated in S190 is output to the controller 50, and the electrode unit corresponding to the address is output to the controller 50. 17 is executed (control for changing the state of the semiconductor switch 15 from the OFF state to the ON state).

  At this time, in the touch panel 104, the electrostatic capacitance of the part facing the turned-on electrode unit 17 changes. Thereby, the same operation as that of the touch operation with the finger 210 (see FIG. 5) is realized.

  In this case, the touch panel 104 recognizes that the operation of the icon displayed at the position facing the electrode unit 17 has been performed, and the mobile terminal 100 performs processing corresponding to the operation of the icon (application activation or button operation). Process).

  After S200, the process proceeds to S210, and it is determined whether or not to end the processing flow. Specifically, the determination is made based on whether or not an instruction to end the process is input from the user. In addition, the determination is made based on whether or not a predetermined time has passed without any operation from the user. When a process end command is input from the user (S210: YES), or when a predetermined time has passed without any operation from the user (S210: YES), the process flow ends. When the process end command is not input from the user (S210: NO) and when the predetermined time has not elapsed (S210: NO), the determination of S210 is repeated again (in other words, waiting).

  Next, in S150, when it is determined that the automatic operation process is not executed (automatic operation is not set) (S150: NO), the process proceeds to S160, and an operation instruction on the touch panel 104 is input via the input device 4. It is determined whether or not it has been input.

  If it is determined that an operation instruction on the touch panel 104 has been input (S160: YES), the process proceeds to S180, and processing for recognizing the content of the operation instruction is executed. Here, the content of the operation instruction is recognized based on data input from the user via the input device 4. For example, if text is input via the input device 4, the input text is recognized. When the mouse as the input device 4 is operated, the position of the pointer and the click position related to the mouse operation are detected. The position of the pointer or the click position can be recognized by being replaced with a position on the touch panel 104.

  Specifically, the user operates a pointer displayed on the screen of the display device on the screen of the display device while referring to the image of the touch panel 104 displayed on the display device as the output device 5. Various mouse operations (for example, click operations) can be performed. The microcomputer 20 of the control device 3 detects such a mouse operation on the screen of the display device (on the image on the touch panel 104) and recognizes the mouse operation as an operation on the touch panel 104.

The subsequent processing after S190 is as described above.
On the other hand, if it is determined in S160 that there is no input of an operation instruction from the user (S160: NO), the process proceeds to S170.

In S170, it is determined whether or not a predetermined time has elapsed. If it is determined that the predetermined time has not elapsed (S170: NO), the process returns to S160.
On the other hand, if it determines with predetermined time having passed in S170 (S170: YES), the said process will be complete | finished.

  In addition, although it has already been described that the screen of the touch panel 104 is displayed on the output device 5, more specifically, in the present embodiment, during the operation of the touch panel 104 by the input operation system 1, the image of the touch panel 104 is displayed. The image is picked up by the CCD camera 16 in real time and displayed on the output device 5. Therefore, the user can check the screen of the touch panel 104 of the mobile terminal 100 in real time by referring to the output device 5.

9 and 10 are diagrams for explaining a touch panel operation by the input operation system 1 (also see FIG. 1) of the present embodiment.
FIG. 9 is a diagram schematically showing a state in which the mobile terminal 100 is placed on the placement surface 10 so that the electrode panel 12 of the placement surface 10 and the touch panel 104 of the mobile terminal 100 face each other. . The mounting surface 10, the electrode panel 12, and the electrode unit 17 included in the electrode panel 12 are indicated by a two-dot chain line. Note that only a part of the electrode unit 17 is illustrated.

The mobile terminal 100 is indicated by a solid line. The mobile terminal 100 is shown in a state where a keyboard screen is displayed on the touch panel 104.
The input operation system 1 (see also FIG. 1) of the present invention recognizes the screen of the touch panel 104 and operates the electrode unit 17 so that a desired button among the buttons on the keyboard displayed on the touch panel 104 is operated. Control the drive.

FIG. 10 is an enlarged view of a region R in FIG.
In FIG. 10, keyboard buttons displayed on the touch panel 104 are denoted by reference numerals 131 to 136.

Further, the electrode units 17 adjacent to the buttons 131 to 136 are indicated by reference numerals 17a to 17p.
The microcomputer 20 of the control device 3 recognizes an image of the screen of the touch panel 104, and uses the coordinate values of the positions of the icons (buttons 131 to 136) on the touch panel 104 as reference marks P (see FIG. 9) formed on the electrode panel 12. ) Based on

  For example, the coordinates of the button 131 are calculated as (x1, y1), the coordinates of the button 132 are calculated as (x2, y1), the coordinates of the button 133 are calculated as (x3, y1), and the coordinates of the button 134 are ( x1, y2), the coordinates of the button 135 are calculated as (x2, y2), and the coordinates of the button 136 are calculated as (x3, y2).

  When the microcomputer 20 of the control device 3 operates the button 131, the microcomputer 20 specifies the electrode unit 17f that is the electrode unit 17 arranged at the position of the coordinates (x1, y1), and displays information indicating the electrode unit 17f. It outputs to the controller 50 (refer FIG. 1). Then, the drive of the electrode unit 17f is controlled by the controller 50, and a change in capacitance and a voltage change due to the change in capacitance occur in the button 131 portion of the touch panel 104. In other words, a pseudo touch operation of the button 131 is realized by driving control of the electrode unit 17f.

  According to the input operation system 1 of this embodiment, the mobile terminal 100 is placed on the mounting table 2 so that the touch panel 104 of the mobile terminal 100 faces the electrode panel 12 formed on the mounting surface 10 of the mounting table 2. By installing, it is possible to realize an input operation on the touch panel 104 via the input device 4 while referring to the screen of the touch panel 104 displayed on the output device 5.

  For this reason, in the input on the touch panel 104, the convenience of the keyboard or mouse as the input device 4 can also be enjoyed. And since the screen of the touch panel 104 can be displayed more largely via the display apparatus as the output device 5, visibility can be improved.

In this embodiment, the semiconductor switch 15 corresponds to the switch of the present invention, and the CCD camera 16 and the image processing unit 40 correspond to the imaging unit of the present invention.
As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, A various aspect can be taken within the technical scope of this invention.

  For example, in the above-described embodiment, the input operation system 1 may be configured as an input operation device including the mounting table 2, the control device 3, the input device 4, and the output device 5. . Further, the input operation device may be configured to include at least the electrode panel 12 and the control controller.

  In the above embodiment, the microcomputer 20 receives data representing a color image from the image processing unit 40 and performs image recognition based on the data. However, the image recognition is executed by the image processing unit 40. You may be comprised so that. In this case, the image recognition result in the image processing unit 40 may be sent from the image processing unit 40 to the microcomputer 20.

  In the above embodiment, the position of the icon on the touch panel 104 included in the image captured by the CCD camera 16 and the image processing unit 40 is calculated using the reference mark formed on the electrode panel 12 as a reference. However, the method for calculating the icon position is not limited to this method. For example, the mounting table 2 is fixed so that the mounting table 2 cannot move, and a reference coordinate system is stored in advance with the fixed position as a reference, and a captured image is based on the coordinate system. The position of the icon included in the icon may be calculated.

DESCRIPTION OF SYMBOLS 1 ... Input operation system, 2 ... Mounting table, 3 ... Control apparatus, 4 ... Input device, 5 ... Output device, 10 ... Mounting surface, 11 ... Jaw part, 12 ... Electrode panel, 13 ... Electrode plate, 14 ... Object Detection sensor, 15 ... Semiconductor switch, 16 ... CCD camera, 16a ... Lens unit, 16b ... Image sensor, 17 ... Electrode unit, 19 ... Signal line, 20 ... Microcomputer, 24 ... Memory, 26 ... Input port, 28 ... Output port 30 ... Data bus, 32 ... Address bus, 34 ... Control bus, 40 ... Image processing unit,
DESCRIPTION OF SYMBOLS 50 ... Controller, 60 ... Network connection circuit, 90 ... Internet network, 100 ... Portable terminal, 102 ... Terminal main body, 104 ... Touch panel, 202 ... Insulating film, 204 ... Protective film

Claims (6)

An input operation device for performing an input operation on a capacitive touch panel,
A plurality of electrode units having a plurality of electrode plates and a switch provided between the electrode plates to switch connection and non-connection between the electrode plates are arranged in a matrix, and the touch panel is in contact with or adjacent to the touch panel An electrode panel configured to be disposed as
A controller that is connected to the electrode panel and controls driving of a switch in the electrode unit included in the electrode panel;
An input operation device comprising: An imaging unit for imaging the display screen of the touch panel;
A microcomputer that performs image recognition on an image captured by the imaging unit, detects an operation location on the display screen of the touch panel based on the recognized image, and identifies the electrode unit existing at a position opposite to the operation location And comprising
The microcomputer outputs information indicating the identified electrode unit to the controller.
The input controller according to claim 1, wherein the controller is configured to control driving of a switch in an electrode unit indicated by information output from the microcomputer. One or more icons are displayed on the display screen of the touch panel,
The microcomputer is
A memory for storing in advance information on icons that can be displayed on the display screen of the touch panel;
3. The input according to claim 2, wherein the type of the icon displayed on the display screen of the touch panel and recognized by the image recognition is specified based on icon information stored in advance in the memory. Operating device. An input operation system for performing an input operation on a capacitive touch panel,
A plurality of electrode units having a plurality of electrode plates and a switch provided between the electrode plates to switch connection and non-connection between the electrode plates are arranged in a matrix, and the touch panel is in contact with or adjacent to the touch panel An electrode panel configured to be disposed as
A controller that is connected to the electrode panel and controls driving of a switch in the electrode unit included in the electrode panel;
An imaging unit for imaging the display screen of the touch panel;
A microcomputer that performs image recognition on an image captured by the imaging unit, detects an operation location on the display screen of the touch panel based on the recognized image, and identifies the electrode unit existing at a position opposite to the operation location When,
An output device for displaying an image captured by the imaging unit,
The microcomputer outputs information indicating the identified electrode unit to the controller.
The input controller system, wherein the controller is configured to control driving of a switch in an electrode unit indicated by information output from the microcomputer. One or more icons are displayed on the display screen of the touch panel,
The microcomputer is
A memory for storing in advance information on icons that can be displayed on the display screen of the touch panel;
5. The input according to claim 4, wherein the type of the icon displayed on the display screen of the touch panel and recognized by the image recognition is specified based on icon information stored in advance in the memory. Operation system. An input device is provided for the user to input information,
The microcomputer is
Based on information input by the user via the input device and representing a desired input operation on the capacitive touch panel, an electrode unit to be driven is identified among the electrode units included in the electrode panel The input operation system according to claim 4 or 5, wherein information indicating the specified electrode unit is output to the control controller.

Images