JP2012038289A - Touch panel device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a touch panel device with which the position of operation articles can be easily grasped by an operator having inconvenience at seeing and the power saving can be achieved.SOLUTION: When the operation position is in the vicinity of a GUI part (YES in S4) and the pressure degree is below the predetermined threshold value (YES in S5), a main control device estimates that an operator did not recognize the position of the GUI part and makes a stationary wave pattern (S11B), so that a stationary wave in which the bulge exists in the center of the GUI part and sections exit in the border of the GUI part is generated on the surface substrate.

Description

  The present invention relates to a touch panel device that displays an operation component on an image display unit and receives an operation input input from an operation input panel.

  An input device using a touch panel (hereinafter referred to as a touch panel device) is widely used for a display screen of a terminal device such as a mobile phone or a music player. A general touch panel device is used by superimposing an operation input panel on a display screen such as a liquid crystal screen as an image display unit. In such a touch panel device, an operation component such as a button is displayed on the display screen, and a position on the operation input panel corresponding to the display position of the operation component is touched with a finger as an operation body to operate the operation component. Examples of operation parts such as buttons include GUI (Graphical User Interface) parts. Unlike touch panels with mechanical buttons, sliders, dials, and other interface parts, these touch panel devices are highly convenient because the layout of the control parts can be freely changed by software. Is expected to increase.

  Patent Documents 1 and 2 describe a touch panel device that provides an operator with a feeling of stroke and a feeling of clicking when an operator operates an operation component. Specifically, the operation input panel is displaced according to a signal waveform having a predetermined amplitude from the time when the operation input panel is pressed until the press is confirmed, thereby providing the operator with a feeling of stroke or clicking. ing.

  However, in the touch panel devices disclosed in Patent Documents 1 and 2, although the operator can provide an operational feeling when operating the operation component, vibration is generated from the time when the operation input panel is pressed. The operation parts could not be recognized when the surface of the operation input panel was lightly touched. For this reason, there is a problem that it is difficult for an visually handicapped operator to grasp the position of the operation component.

  In Japanese Patent Application No. 2009-060259 (hereinafter referred to as a prior application), the present applicant has proposed a touch panel device that allows the operator to grasp the positional relationship of the operation components simply by touching the surface of the operation input panel. Specifically, a standing wave (standing wave) is generated in the operation input panel so that a portion corresponding to the position of the operation component on the operation input panel becomes an antinode. Thereby, when the part corresponding to the operation component of the operation input panel is touched, it is possible to feel the vibration at the fingertip and recognize that the operation component exists at that position. Thus, since the position of the operation component can be grasped only by tactile sense, even the visually impaired operator can easily grasp the position of the operation component.

  However, the touch panel device of the prior application always generates a standing wave on the operation input panel, so that there is a problem that power for generating the standing wave is always required and much power consumption is required. It was. In particular, when the above-mentioned touch panel device is adopted as a battery-driven device such as a portable device, there is a problem that the power of the battery is lost early and the usage time of the device is shortened.

  The present invention has been made in view of the above problems, and the object thereof is to make it possible for a visually-impaired operator to easily grasp the position of an operation component and to realize power saving. It is providing the touch panel device which can do.

In order to achieve the above object, an invention according to claim 1 is directed to an operation input panel that receives an operation input when an operating body is in contact with its surface, and an image display that is provided opposite to the operation input panel and displays an image. And an operation component generation unit that generates an operation component to be displayed as an image on the image display unit, and a position on the operation input panel corresponding to the position of the operation component displayed on the image display unit. In the touch panel device in which the operation body is brought into contact and the operation part is operated by the operation body, the operation part is located at a position of the operation input panel corresponding to the position of the operation part displayed on the image display unit. Vibration generating means for generating vibration that can be recognized tactilely in the operation input panel; estimation means for estimating a recognition state of the operator with respect to the position of the operation component; Based on the recognition state estimation means has estimated the operator, is characterized in that a control means for controlling said vibration generating means.
According to a second aspect of the present invention, in the touch panel device according to the first aspect, the control means drives the vibration generating means when the estimation means estimates that the operator recognizes the operation component. It is characterized by controlling to stop.
According to a third aspect of the present invention, in the touch panel device according to the first or second aspect, the control unit determines the position of the operation component when the estimation unit estimates that the operator recognizes the operation component a little. The operation input panel corresponding to the position of the operation component when the estimation means estimates that the vibration amplitude generated at the position of the operation input panel corresponding to the operator is not recognized at all by the operator The vibration generating means is controlled so as to be smaller than the amplitude of the vibration generated at this point.
According to a fourth aspect of the present invention, in the touch panel device according to any one of the first to third aspects, the estimation means performs the operation on the position of the operation component based on a contact state of the operation body with the operation input panel. The recognition state of the person is estimated.
The invention according to claim 5 is the touch panel device according to claim 4, wherein the degree of pressing of the operation body on the operation input panel surface is used as the contact state of the operation body with the operation input panel. It is what.
According to a sixth aspect of the present invention, in the touch panel device according to the fourth aspect, the moving speed of the operating body is used as the contact state of the operating body with the operation input panel.
According to a seventh aspect of the present invention, in the touch panel device according to the fourth aspect, the movement acceleration of the operating body is used as a contact state of the operating body with the operation input panel.
The invention according to claim 8 is the touch panel device according to any one of claims 1 to 7, further comprising imaging means for imaging the face of the operator when the operator is operating the apparatus. The recognition state of the operator with respect to the position of the operation component is estimated based on the face image captured by the imaging unit.

  According to the present invention, if the vibration generating means is controlled as follows based on the estimating means for estimating the operator's recognition state with respect to the position of the operating component, the operation can be easily performed for the visually handicapped operator. The position of the component can be grasped, and power saving can be realized. That is, when the estimation unit estimates that the operator correctly recognizes the position of the operation component, the drive of the vibration generation unit is stopped and no vibration is generated. When it is estimated that the position is not recognized correctly, the vibration generating means is driven to control to generate vibration at the position of the operation input panel corresponding to the position of the operation component. As a result, when an operator who cannot correctly recognize the position of the operation component displayed on the image display unit, such as a visually impaired person, vibration is generated at the position of the operation input panel corresponding to the position of the operation component. If the operator touches the operation input panel, the position of the operation component can be grasped by vibration. Therefore, the position of the operation component can be easily grasped by the visually impaired operator. In addition, for a person who can correctly recognize the position of the operation component displayed on the image display unit, the operation component is generated without causing the operation component to be recognized tactilely by generating vibration as described above. It can be operated correctly. Therefore, the drive of the vibration generating means is stopped. As a result, power consumption can be suppressed and power saving can be realized.

The figure which shows the cross-section of the touchscreen apparatus of this embodiment. The figure which shows the positional relationship of the GUI component displayed through the surface substrate of the touch panel apparatus, and the peak value of the amplitude of the standing wave produced | generated on a surface substrate. The figure which shows an example of the display of the liquid crystal panel of the touch panel device. The block diagram which shows the one part circuit structure contained in the main controller of the touch panel device. The block diagram which shows the circuit structure of the state determination circuit shown by FIG. The figure explaining the difference in an operator's finger | toe movement by the recognition state of GUI components. The figure shown about an example of the change of the operation position when a finger | toe is moved, tracing the surface of a surface board. The flowchart which shows the procedure of the production | generation process of the drive pattern performed by the main controller of the touch panel device. The figure which shows the cross-section of the touchscreen apparatus of a modification.

  Hereinafter, embodiments to which the touch panel device of the present invention is applied will be described. FIG. 1 is a diagram illustrating a cross-sectional structure of a touch panel device 100 according to the present embodiment.

  The touch panel device 100 of this embodiment includes a substrate 1, a liquid crystal panel 2, an operation input panel 17, piezoelectric drive devices 5A and 5B, a contact sensor processing circuit 6, an image display circuit 7, a drive control circuit 8, a main control device 9, and A memory 10 is provided. The operation input panel 17 includes the contact sensor 3 and the surface substrate 4.

  The substrate 1 is a substrate on which the liquid crystal panel 2, the operation input panel 17, and the piezoelectric driving devices 5A and 5B are mounted. For example, a printed circuit board (PCB: Printed Circuit Board) composed of a glass epoxy substrate or a glass composite substrate. ) Can be used.

  The liquid crystal panel 2 serving as an image display unit is a display unit that is mounted on the substrate 1 and displays GUI components generated by the image display circuit 7. The display unit is not limited to the liquid crystal panel 2 as long as it can display GUI parts. For example, an organic EL (Electro-Luminescence) panel may be used instead of the liquid crystal panel 2.

  The contact sensor 3 is a coordinate detection device that is mounted on the liquid crystal panel 2 and detects the coordinates of the position pressed by the operator. In the embodiment, an embodiment using a resistive film type sensor as the contact sensor 3 will be described. However, the contact sensor 3 is not limited to a resistive film type sensor, and may be, for example, a pressure-sensitive sensor or a capacitance type sensor. Well-known sensors such as the above-mentioned sensors and acoustic wave matching type sensors can be used.

  The resistive film type sensor has an electrode sheet in which a plurality of transparent electrodes are arranged to face each other in a matrix at regular intervals. When the operator presses the surface of the surface substrate 4 with a fingertip that is an operating body, the opposing electrodes come into contact with each other, and the resistance values in the X and Y directions of the electrode sheet change depending on the contact position. Due to the change in the resistance value, the voltage value corresponding to the X direction and the Y direction output from the resistive film type sensor changes. By this change in voltage value, the coordinates of the pressed operation position are specified.

  The front substrate 4 is a transparent substrate serving as an operation input unit of the touch panel device 100. When the operator presses the surface of the front substrate 4 as described above, the coordinates of the operation position are detected by the contact sensor 3. For this reason, the surface of the surface substrate 4 becomes a coordinate input surface for operating the touch panel device 100 of the embodiment. As the surface substrate 4, for example, an acrylic or polycarbonate resin substrate or a glass substrate can be used.

  The piezoelectric drive devices 5A and 5B as vibration generating means are package type drive elements in which a plurality of laminated thin plate-like piezoelectric elements are sandwiched between electrode plates and housed in a resin casing. As shown in FIG. 1, the piezoelectric driving devices 5A and 5B are sandwiched between the substrate 1 and the surface substrate 4 so as to be paired on both sides of the liquid crystal panel 2 and the contact sensor 3 one by one. Arranged. The piezoelectric driving devices 5A and 5B are actually elongate driving elements having substantially the same length as the sides of the substrate 1 and the front substrate 4 passing through the paper surface. When a voltage is applied, the piezoelectric driving devices 5 </ b> A and 5 </ b> B are distorted and displaced in the stacking direction, and generate vibration on the surface substrate 4.

  The contact sensor processing circuit 6 is a circuit that performs signal processing on a voltage value that represents the coordinates of the operation position output from the contact sensor 3. By this signal processing, the voltage value representing the coordinates of the operation position is amplified, noise-removed, and digitally converted, and output as a digital voltage value. Since signal processing is performed in this way, the operation position can be accurately detected even if the voltage value output from the contact sensor 3 is very small. In addition, when the resistive film type contact sensor 3 in which a plurality of transparent electrodes are arranged in a matrix as described above is used, the contact area can be obtained from the number of electrodes that are brought into contact by pressing. Further, the operation position (contact position) can be accurately derived by calculating the center position of the contact surface. Furthermore, the contact sensor processing circuit 6 can also detect the time change amount of the operation position and the contact area. Further, the contact sensor 3 and the contact sensor processing circuit 6 also have a function as a pressing degree detection unit that detects the pressing degree of the operation input made on the coordinate input surface.

  Further, the contact sensor processing circuit 6 includes a time (hereinafter referred to as movement time) required until the operation position (contact position) changes, and an operation position (contact position) after the operator contacts the surface substrate 4. The time (movement time) when is changed is measured. Specifically, when the operation position (contact position) is derived, a timer is started. When the operation position (contact position) changes, the time is stored as a movement time in a memory buffer 16 (see FIG. 5) described later. The movement time can be calculated by subtracting the movement time of the previous operation position from the movement time of the development operation position.

  The contact sensor processing circuit 6 performs the above-described signal processing on the voltage value representing the coordinates of the operation position, and obtains input coordinate information representing the operation position (contact position) and area information representing the contact area, which is an index value of the pressing degree. Output. The input coordinate information represents, for example, the coordinates of the center position of the area where the input operation has been performed, and the area information indicates the size of the contact area. When the degree of pressing by the operator increases, the contact area with which the electrode sheets of the contact sensor 3 arranged in contact with each other increases, so that the resistance value between the electrode sheets decreases and represents the area information output from the contact sensor processing circuit 6. The voltage value becomes low. On the other hand, when the degree of pressing is low, the contact area is reduced, and the voltage value representing the area information is high. The change in the area information can be used as information indicating the change in the degree of pressing of the surface substrate 4 by the operator. The input coordinate information and the area information are input to the main control device 9 as information representing the contents of the operation input by the operator.

  The image display circuit 7 as an operation component generation unit is a circuit for driving each pixel of the liquid crystal panel 2 to display a desired image. For example, when the liquid crystal panel is driven in an active matrix, the image display circuit 7 includes a TFT (Thin Film Transistor) for driving each pixel based on image data and display coordinate data input from the main control device 9. To drive. The image display circuit 7 converts the image data read from the memory 10 by the main controller 9 into an analog voltage signal and outputs the analog voltage signal to drive the liquid crystal panel 2. Thereby, on the liquid crystal panel 2, an image (image pattern) corresponding to the image data related to the GUI component or the image data other than the GUI component is displayed at the display position corresponding to the display coordinate data. The display coordinate data is data for specifying the position representing the image data on the coordinates, and is stored in the memory 10 in association with the image data.

  The drive control circuit 8 is a circuit that outputs a drive voltage (drive signal) for driving the piezoelectric drive devices 5A and 5B. For example, a function generator can be used. The drive control circuit 8 modulates and amplifies the voltage waveform of the drive voltage in accordance with the drive pattern input from the main controller 9, and drives the piezoelectric drive devices 5A and 5B. Since the drive pattern is determined by the frequency and amplitude of the voltage waveform, the frequency and amplitude of the voltage waveform are set by the frequency data and amplitude data that the main controller 9 reads from the memory 10.

  The main control device 9 as a control means is configured by, for example, a CPU (Central Processing Unit) and the like, and is a processing device that controls the touch panel device 100 according to the embodiment. In the embodiment, when the main control device 9 executes a program stored in the memory 10 to provide a predetermined service to the operator, the input coordinate information and / or the area input from the contact sensor processing circuit 6 Based on the information and data representing the type of GUI component displayed on the liquid crystal panel 2, the operation content of the operator is determined. Then, a process and an image pattern corresponding to the determined operation content are displayed. For the display of the image pattern, image data for generating an image pattern corresponding to the determined operation content is read from the memory 10 and displayed on the liquid crystal panel 2 via the image display circuit 7. For example, when the touch panel device 100 according to the embodiment is used as an ATM, the main control device 9 provides a GUI part that represents a button for inputting a password or an amount on the liquid crystal panel 2 in accordance with an operation input by the operator. The cash payment process and the transfer process are executed according to the operation input of the operator. Further, as will be described later, main controller 9 estimates the recognition state of the operator's GUI component based on the input coordinate information and the area information.

  Further, the main control device 9 as a control means reads out image data from the memory 10 and displays the image data in the memory 10 when displaying the image on the liquid crystal panel 2 in order to provide the predetermined service as described above. The associated amplitude data, frequency data, and phase difference data are read, and the piezoelectric drive devices 5A and 5B are driven via the drive control circuit 8 to execute processing for vibrating the surface substrate 4. The details of the process for vibrating the surface substrate 4 will be described later.

The memory 10 is a memory that stores various data such as a program necessary for driving the touch panel device 100 according to the embodiment. In the embodiment, a program for providing a predetermined service, image data, display coordinate data, and the like. In addition, amplitude data, frequency data, and phase difference data are stored.
The image data is data representing an image of a GUI component to be displayed on the liquid crystal panel 2 and other images. The display coordinate data is data for specifying on the coordinates a position where an image based on each image data is displayed.
The amplitude data, frequency data, and phase difference data are data representing drive patterns for driving the piezoelectric drive devices 5A and 5B via the drive control circuit 8.
The amplitude data, frequency data, and phase difference data representing the drive pattern are stored in the memory 10 in association with image data and display coordinate data.

Details of the process for vibrating the surface substrate 4 will be described.
The touch panel device 100 according to the embodiment drives the piezoelectric driving devices 5A and 5B, has a belly located at the center of the GUI part which is a button or the like displayed on the liquid crystal panel 2, and is connected to the boundary of the GUI part. A standing wave is generated on the surface substrate 4 such that is located.

  FIG. 2 is a diagram illustrating a positional relationship between a GUI component displayed through the surface substrate 4 of the touch panel device 100 according to the embodiment and the peak value of the amplitude of the standing wave generated on the surface substrate 4. Here, the standing wave 233 generated at a location corresponding to the GUI component is generated by the piezoelectric driving device 5A disposed at one end of the surface substrate 4 and the piezoelectric driving device 5B disposed at the other end of the surface substrate 4. It is formed by exciting at the natural frequency of the surface substrate 4. The positions of the antinodes and nodes of the standing wave are determined by the frequency and phase difference of the vibration wave generated by each of the piezoelectric driving devices 5A and 5B. By adjusting these, the position of the standing wave can be set at any position on the surface substrate. Abdomen and nodes can be generated.

  FIG. 3 is a diagram illustrating an example of display on the liquid crystal panel 2 of the touch panel device 100 according to the embodiment. The main control device 9 accesses the memory 10 using an image pattern ID, which will be described later, and reads image data corresponding to the image data ID associated with the image pattern ID and display coordinate data. When the main controller 9 inputs image data and display coordinate data to the image display circuit 7, the image display circuit 7 uses the image data and display coordinate data to display an image (image pattern) at a desired position in the liquid crystal panel 2. Is generated. Thus, for example, 14 GUI buttons 21 and a GUI input confirmation display screen 22 are displayed on the liquid crystal panel 2 as GUI components.

  FIG. 4 is a block diagram showing a partial circuit configuration included in the main control device 9 of the touch panel device 100 of the present embodiment. As shown in FIG. 4, the main controller 9 includes a state determination circuit 11, an operational feeling generation circuit 12, and a standing wave generation circuit 13. Note that, as described above, the main control device 9 is a processing device that controls various processes of the touch panel device 100, and therefore the circuit shown here is a part of the main control device 9. Specifically, the circuit of the main controller 9 shown in FIG. 4 includes input coordinate information, which is information representing the operation position (contact position), and information representing the degree of pressing, among the information input from the contact sensor processing circuit 6. Only the circuit for determining the drive pattern based on the area information, the movement time, etc. is extracted.

  The state determination circuit 11 serving as an estimation unit is a circuit that determines the GUI component recognition state of the operator from input coordinate information, area information, movement time (movement time), and the like among information input from the contact sensor processing circuit 6. . The operational feeling generation circuit 12 and the standing wave generation circuit 13 are circuits that generate drive patterns for driving the piezoelectric drive devices 5A and 5B. The operation feeling generation circuit 12 generates a drive pattern (operation completion drive pattern) for giving a tactile sensation to notify the completion of the operation when the operator touches the GUI button or slide bar displayed on the liquid crystal panel 2 to perform the operation. Circuit. On the other hand, the standing wave generating circuit 13 is a circuit that generates a driving pattern (standing wave driving pattern) for giving a tactile sensation as if touching an actual button. Specifically, a standing wave is generated on the surface substrate 4 such that a belly is located at the center of the GUI part, such as a button displayed on the liquid crystal panel 2, and a node is located at the boundary of the GUI part. It is a circuit that generates a drive pattern.

  FIG. 5 is a block diagram showing a circuit configuration of the state determination circuit 11 shown in FIG. As shown in FIG. 5, the state determination circuit 11 includes a contact state determination circuit 14 and a recognition state determination circuit 15. The contact state determination circuit 14 is a circuit that determines whether or not the GUI component has been operated based on input coordinate information and area information obtained by the operator from the contact sensor processing circuit 6. The memory buffer 16 stores input coordinate information (operation position) obtained from the contact sensor processing circuit 6, movement time (movement time), and the like. The recognition state determination circuit 15 is a circuit that determines the recognition state of the operator based on the operation state obtained from the contact state determination circuit 14 and information stored in the memory buffer 16.

In the memory buffer 16, as shown in Table 1 below, an ID, a movement time (movement time), and input coordinate information (operation position) are stored in association with each other. The ID is a number for associating the movement time and the operation position.

Next, the determination of the operator's GUI component recognition state in the state determination circuit 11 shown in FIG. 5 will be described.
FIG. 6 is a diagram for explaining a difference in the movement of the operator's finger depending on the recognition state of the GUI component.
When the operator recognizes the GUI component, the operator presses the location of the operation target GUI component indicated by the black circle in the drawing with a predetermined pressing force. On the other hand, an operator who has not recognized the GUI component confirms the position of the GUI component and presses the GUI component while recognizing the vibration generated at the location corresponding to the GUI component with his / her finger. The operator who recognizes the part of the GUI part to some extent can trace the surface substrate 4 with his finger and confirm the position of the GUI part with his finger, as shown by the dotted line in FIG. The GUI part to be operated is pushed at a certain moving speed. On the other hand, an operator who has not recognized any GUI part slowly moves his / her fingertip to meander as shown by the solid line in FIG. 6 and confirms the position of the surrounding GUI part with the finger, The position is grasped and the GUI part to be operated is pushed.

  In this way, for an operator who can correctly recognize the position of the GUI component, the button can be operated correctly without generating a vibration that allows the GUI component to be tactilely recognized. It is not necessary to generate Since the standing wave for the operator who knows the position of the GUI component to some extent is an auxiliary one for grasping the position of the GUI component, it is sufficient that the amplitude of the standing wave is small. On the other hand, since the standing wave for the operator who does not recognize the GUI component at all is for tactile recognition of the GUI component, it is necessary to increase the amplitude of the standing wave.

Table 2 is a table showing the determination in each circuit in the state determination circuit 11 and the determination result of the state determination circuit 11.

  First, when the area information indicating the degree of pressing and the input coordinate information indicating the operation position (contact position) are input from the contact sensor processing circuit 6 to the contact state determination circuit 14, the contact state determination circuit 14 inputs the input coordinate information. To determine whether the operation position (contact position) is the position of the GUI component. Then, when the operation position (contact position) is the position of the GUI part, it is determined whether or not the pressing degree is equal to or greater than a preset threshold value from the area information. At this time, if the degree of pressing is equal to or greater than the threshold value, it is determined that the GUI component has been operated. In this case, since it is presumed that the operator correctly recognizes the position of the GUI component, the state determination circuit 11 determines that “there is no hesitation” as the recognition state. If the contact state determination circuit 14 determines that “there is no hesitation”, the data stored in the memory buffer 16 is deleted.

  On the other hand, an operator who cannot correctly recognize the position of the GUI component tries to recognize the position of the GUI component while tracing the front substrate 4, and thus the pressing degree is less than the threshold value. Therefore, when the operation position (contact position) is less than the threshold value in the vicinity of the GUI part, it is determined that the position of the GUI part is not correctly recognized and “there is a problem”. As described above, when the contact state determination circuit 14 determines that “there is a hesitation”, the recognition state determination circuit 15 determines the degree of the operator's hesitation.

  The recognition state determination circuit 15 performs determination based on the moving speed of the finger when the operator moves while tracing the front substrate 4.

数１におけるｎは、ＩＤ番号であり、Ｔは、移動時刻である。上記数１に示すように、本実施形態の移動速度Ｖは、接触位置間の移動速度の平均値を用いた。 The moving speed V is calculated from the following equation.

In Expression 1, n is an ID number, and T is a movement time. As shown in the above equation 1, the average value of the moving speed between the contact positions is used as the moving speed V of the present embodiment.

FIG. 7 is a diagram showing an example of changes in input coordinate information (operation position) when a finger moves while tracing the surface of the front substrate 4. Table 3 below is stored in the memory buffer 16 at that time. It shows about the data. For example, as shown in FIG. 7, when it is determined that the degree of pressing at the contact position a is less than the threshold, the contact sensor processing circuit 6 starts a timer and starts measuring the movement time. Further, the movement time and the operation position (x2, y2) are stored in the memory buffer 16. Next, when the operation position is switched from a to b, and the degree of pressing at the position of b is less than the threshold value, the movement time T1 and the operation position (x3, y2) at that time are stored. The recognition state determination circuit 15 reads the operation positions of ID = 0 and 1 and the movement time stored in the memory buffer 16 and calculates the movement speed V. When the moving speed V is equal to or higher than the threshold value, the recognition state determination circuit 15 estimates that the position of the GUI component is recognized to some extent, and determines that “there is little hesitation”. On the other hand, when the moving speed V is equal to or higher than the threshold, it is estimated that the position of the GUI component is not recognized at all, and it is determined that “there is a problem”. When the operation position changes from the position b to the position c, the recognition state determination circuit 15 reads the operation position of ID = 0 to 2 and the movement time stored in the memory buffer 16, and moves the movement speed. V is calculated, and the operator's recognition state is estimated again from the calculated moving speed V.

  In the above, when the operation position is changed once, the movement time and the operation position are stored in the memory buffer 16, but since the interval between the transparent electrodes is very narrow, the resolution of the movement time may not be in time. . Therefore, the movement time and the operation position when the operation position is changed a plurality of times may be stored in the memory buffer 16. For example, when the operation position is changed three times and stored in the memory buffer, the operation position at the position d shown in FIG. 7 and the movement time at that time are stored in the memory buffer as data of ID = 1. The moving speed V is calculated from the stored position data “a” and position data “d”.

The calculation formula for the movement speed V is not the average value of the movement speed between the operation positions as shown in Equation 1, but the operation position at which the measurement of the movement time is started as shown in Equation 2 (a in FIG. 7). ) To the current operation position and the movement time may be calculated. In this case, the memory buffer 16 may store the operation position (x2, y2) at the position a, the current movement time, and the operation position.

Further, as shown in Equation 3, the moving speed V may be a moving speed from the previous operation position to the current operation position. In this case, the memory buffer 16 may store the previous operation position and movement time, and the current operation position and movement time.

  In addition, when the operator who recognizes the position of the GUI component to some extent moves the fingertip to the vicinity of the GUI component to be operated, the fingertip quickly moves so that the fingertip reaches the vicinity of the GUI component to be operated, and the GUI Move the fingertip slowly to determine the part position. Therefore, as shown in Equation 3, the movement speed V is set to the movement speed from the previous operation position to the current operation position, and when the movement is to determine the GUI part position, “there is a hesitation”. The amplitude of the standing wave increases. This facilitates the determination of the GUI part position of the operator who recognizes the position of the GUI part to some extent.

  Further, the determination by the recognition state determination circuit 15 may be determined not based on the moving speed of the fingertip but based on the moving distance or acceleration of the fingertip when a predetermined time has elapsed.

  In the case of the movement distance, when it is determined that the pressure value near the GUI component (operation position a in FIG. 7) is less than the threshold value, the contact sensor processing circuit 6 starts a timer and starts measuring time. Further, the input coordinate information (operation position) at that time is written in the memory buffer 16. When the time reaches a predetermined value, the contact sensor processing circuit 6 resets the timer and writes the operation position at that time in the memory buffer 16. When the current operation position is written in the memory buffer 16, the recognition state determination circuit 15 reads the operation position at the start of the timer start and the current operation position from the memory buffer 16, calculates the movement distance, and calculates The operator's GUI recognition state is estimated from the movement distance.

  When the movement distance is long, it is considered that the movement to the next GUI component is performed at a stroke, and therefore, it is considered that the position of the GUI component is recognized to some extent. On the other hand, when the movement distance is short, it is considered that the position of the GUI component is grasped while slowly checking the vibration with the fingertip and the GUI component is not recognized at all. Therefore, when the movement distance is less than the threshold, the recognition state determination circuit 15 determines that “there is a problem”, increases the amplitude of the standing wave, and when the movement distance is equal to or greater than the threshold, the recognition state determination circuit 15 , "Determine less" and reduce the amplitude of the standing wave.

  Also, when determining by the movement distance, when the fingertip is moved closer to the operation target GUI component, the movement distance of the fingertip becomes longer, and the fingertip approaches the operation target GUI component to determine the GUI component position. The moving distance is shortened. Therefore, also in this case, when the GUI component position is determined, it is determined that “there is a problem”, the amplitude of the standing wave is increased, and the GUI component position is easily determined.

  Further, when the determination is made based on the acceleration of the fingertip, the moving speed from the previous operation position to the current operation position is calculated using the equation shown in Equation 3 above. Then, the movement speed and the movement time are stored in the memory buffer 16. When the operation position is changed, the movement speed and the movement time are similarly stored in the memory buffer 16. The recognition state determination circuit 15 reads the movement speed and movement time calculated last time and the movement speed and movement time calculated this time from the memory buffer 16 and calculates acceleration. When the acceleration is a positive value, it is considered that the GUI component that is the operation target has started to move at a stretch, and therefore it is considered that “there is little hesitation”. On the other hand, when the fingertip approaches the GUI component to be operated, the acceleration becomes negative and it is considered that the fingertip moves slowly so as to determine the GUI component position. Therefore, in this case, it is determined that “there is a problem” and the amplitude of the standing wave is increased.

  In the present embodiment, the recognition state of the GUI part of the operator is determined in three stages, but is not limited to three stages.

FIG. 8 is a flowchart showing a procedure of drive pattern generation processing executed by the main control device 9 of the touch panel device 100 of the present embodiment.
When the touch panel device 100 is activated, the main control device 9 starts drive pattern generation processing (Start). First, main controller 9 detects a contact state with the coordinate input surface (the surface of front substrate 4) (S1), and reads a voltage value representing area information input from contact sensor processing circuit 6 (S2). Next, main controller 9 calculates the degree of pressing based on the value read in step S2 (S3). Next, the main control device 9 calculates the input coordinate information (operation position) by calculating the center position of the contact surface from the voltage value representing the area information input from the contact sensor processing circuit 6, and the operation position is the GUI. It is determined whether or not it is near the part (S4). When the operation position is a position close to the GUI component (YES in S4), it is determined whether or not the pressing degree calculated in step S3 is less than or equal to a predetermined threshold (S5).

  If the determination result in step S5 is less than the threshold value (YES in S5), main controller 9 determines whether data equal to or greater than ID = 1 is stored in the memory buffer (S6). When data of ID = 1 or more is stored in the memory buffer, data for determining the state change degree is read from the memory buffer 16 (S7). Next, the main control device 9 calculates the state change degree based on the data read in step S7 (S8). Specifically, when the moving speed V is used as the state change degree, the moving time and the operation position (contact position) are read from the memory buffer 16 to calculate the moving speed V. When acceleration is used as the degree of state change, the movement time and movement speed are read from the memory buffer 16 and the acceleration is calculated.

  Next, main controller 9 determines whether the value calculated in step S8 is less than or equal to a predetermined threshold value (S9). If the value calculated in step S8 is less than the predetermined threshold value (YES in S9), it is determined that the recognition state A is in doubt (S10B), and a standing wave pattern with a large amplitude is generated (S11B).

  On the other hand, when the value calculated in step S8 is equal to or greater than the predetermined threshold (No in S9), it is determined that the recognition state B is less confusing (S10C), and a standing wave pattern with a small amplitude is generated (S11C). .

  On the other hand, when the determination result in step S5 is equal to or greater than the threshold (NO in S5), it is determined that the recognition state X (no doubt) (S10A). In this case, since the operation of the GUI component is confirmed, an operation completion drive pattern is generated instead of the standing wave pattern (S11A).

  Next, the main control device 9 drives the piezoelectric drive devices 5A and 5B based on the pattern generated in step S11 (S12).

  Steps S2 to S5 are parts processed by the contact state determination circuit 14 in FIG. Steps S6 to S9 are parts processed by the recognition state determination circuit 15 in FIG. Step S11A is a part processed by the operational feeling generation circuit 12 in FIG. 4, and Steps S11B and 11C are parts processed by the standing wave generation circuit 13.

  As can be seen from the flow shown in FIG. 8, when the operator recognizes the GUI component, the contact state is detected when the pressing degree is equal to or greater than the threshold value and the operation is confirmed. Therefore, when the operator recognizes the GUI component, only the operation completion drive pattern that gives the operator the feeling of operating the GUI component is generated, and the standing wave pattern is not generated. Therefore, power consumption can be suppressed as compared with a device that always generates a standing wave pattern.

  On the other hand, when the degree of pressing when the contact state is detected is less than the threshold value, the operator does not recognize the position of the GUI component, and traces the panel surface in order to recognize the position of the GUI component. Since it is in a state, a standing wave pattern can be generated and the operator can recognize the position of the GUI component by vibration. As a result, even a visually impaired operator can operate the touch panel device 100.

  Also, as shown in step S4 of FIG. 8, when the operation position comes near the GUI component, the state change is calculated and a standing wave is generated. Thus, when the operator is tracing the position far from the GUI component with his / her finger, the standing wave is not generated and the power consumption can be suppressed.

  Further, when the touch panel device 100 is activated (Start), when the touch panel is touched, a notification that the position of the GUI component is to be notified by vibration may be given by voice from a speaker (not shown). In addition, when the degree of pressing is less than the threshold and the operation position (contact position) is on a GUI component, information on the GUI component may be notified by voice. For example, when the GUI component corresponds to a number, the number information is notified by voice. As a result, an operator with visual impairment can recognize the GUI component well.

Next, a modification of this embodiment will be described.
FIG. 9 is a diagram illustrating a cross-sectional structure of a modified touch panel device 100A.
The touch panel device 100 </ b> A of this modification is provided with a camera 18 and an image arithmetic processing circuit 19 that performs image processing on an image captured by the camera 18, and an operator obtained from the image arithmetic processing circuit 19 through the camera 18. The recognition state of the GUI part of the operator is estimated using a facial expression, a line of sight, and the like.

  If the operator does not correctly recognize the GUI component, a specific expression such as wrinkles between the eyebrows or narrowing of the eyes appears. In this way, a facial part (such as around the eyes) where a specific expression appears when the GUI part is not correctly recognized is used as a feature point, and is digitized to determine whether the GUI part is correctly recognized. . Specifically, a normal face before the touch panel operation is imaged by the camera 18, and characteristic parts that change when the GUI part cannot be recognized, such as a part between eyebrows and an eye part, are stored. Next, the face when the operator is operating the touch panel device 100A is captured by the camera 18, and compared with the normal time, the recognition state of the GUI component is estimated. When there is almost no change between the feature point portion of the face at the time of operation and the normal feature portion before the operation, it is determined that the GUI component is recognized, and the standing wave pattern is not generated. On the other hand, if there is a change in the characteristic part at the time of operation and the characteristic part in the normal state before the operation, it is estimated that the GUI component cannot be recognized, and a standing wave is generated. A normal face before the operation may be learned, and the recognition state may be grasped by comparing the face during the operation with the normal face before the operation. In this case, if there is almost no change between the normal face before the operation and the face at the time of the operation, it is estimated that the GUI component is recognized, and if the face is changed, the GUI component is recognized. It is estimated that

  Further, the GUI component recognition state may be estimated by combining the captured image of the camera 18 and information such as the moving speed of the fingertip. In this case, even if the face at the time of operation changes from the normal face before the operation, such as wrinkles between the eyebrows at the time of operation, if the position of the GUI component is pressed with a predetermined pressing force It is assumed that the GUI component is recognized, and no standing wave is generated. On the other hand, when the face at the time of operation has changed from the normal face before the operation and the operation panel is traced with a finger, it is estimated that the GUI component is not recognized, and the standing wave Is generated.

  As described above, according to the touch panel device 100 of the present embodiment, the operation input panel 17 that receives an operation input when the operating body is in contact with its surface, and the image display unit that is provided to face the operation input panel 17 and displays an image. The liquid crystal panel 2 is an image display circuit 7 that is an operation component generation unit that generates a GUI component that is an operation component to be displayed as an image on the liquid crystal panel 2. In addition, a piezoelectric drive device serving as a vibration generating unit that generates vibrations on the operation input panel 17 so that the GUI parts can be tactilely recognized at locations on the operation input panel 17 corresponding to the positions of the GUI parts displayed on the liquid crystal panel 2. 5A, 5B, a state determination circuit 11 as estimation means for estimating the operator's recognition state with respect to the position of the GUI component, and the piezoelectric drive devices 5A, 5B based on the operator's recognition state estimated by the state determination circuit 11 And a main control device 9 which is a control means for controlling. When the state determination circuit 11 estimates that the operator recognizes the GUI component, the state determination circuit 11 controls to stop driving the piezoelectric drive devices 5A and 5B. As a result, a standing wave is not generated for the operator who recognizes the GUI component, so that power consumption can be suppressed. Further, even when no standing wave is generated, an operator who recognizes the GUI component can operate the touch panel device 100 without hesitation. On the other hand, since a standing wave is generated for an operator who does not recognize the GUI component, the position of the GUI component can be recognized by touch, and an operator who does not recognize the GUI component can also recognize the touch panel device. 100 operations can be performed.

  Further, the state determination circuit 11 operates the amplitude of the vibration generated at the position of the operation input panel 17 when the state determination circuit 11 estimates that the operator recognizes the GUI part a little (there is little hesitation). The piezoelectric driving devices 5A and 5B are controlled so as to be smaller than the amplitude of the vibration generated at the position of the operation input panel 17 when it is estimated that the user does not recognize the operation component at all (there is a doubt). As a result, the vibration is an auxiliary role for recognizing the position of the GUI component for the operator who recognizes the GUI component a little, so that the position of the GUI component can be sufficiently recognized even if the vibration is weak. . As a result, the operator can operate the GUI component satisfactorily and power consumption can be suppressed. Further, when the operator does not recognize the GUI component at all, the amplitude is large, so that the position of the GUI component can be recognized well in a tactile sense. As a result, an operator who does not recognize any component can also operate the touch panel device 100 satisfactorily.

  Further, the state determination circuit 11 estimates the operator's recognition state with respect to the position of the GUI component based on the degree of pressing of the finger on the operation input panel surface as the contact state of the fingertip with the operation input panel 17. The operator who recognizes the GUI part presses the position of the GUI part with a predetermined pressing force, whereas the person who does not recognize the GUI part touches the operation input panel with his / her finger. In order to recognize the position of the GUI component, the pressing force to the operation input panel is weak. Thus, the degree of pressing of the finger on the operation input panel surface is different between the operator who recognizes the GUI component and the operator who does not recognize the position of the GUI component. Therefore, it is possible to estimate whether the GUI component is recognized based on the degree of pressing.

  Further, the moving speed of the finger may be used as the contact state of the fingertip with the operation input panel 17. Since the operator who recognizes the GUI component does not need to confirm the position of the GUI component by tactile sense, the operator can quickly move the finger to the position of the GUI component to be operated, so that the moving speed is fast. On the other hand, an operator who has not recognized the GUI component moves the finger to the position of the GUI component to be operated while confirming the position of the GUI component with a tactile sense, so the moving speed is slow. Therefore, it is possible to estimate whether or not the GUI component is recognized from the moving speed of the finger. Further, by using the moving speed, it is possible to generate vibration so that the GUI part to be operated can be recognized by tactile sensation when the moving speed decreases by approaching the GUI part to be operated.

  Furthermore, finger movement acceleration may be used as the contact state of the finger with the operation input panel. Since the person who recognizes the GUI component moves the finger to the GUI component to be operated at once, the movement acceleration of the finger is large. On the other hand, an operator who has not recognized the GUI component moves the finger to the position of the GUI component to be operated while confirming the position of the GUI component with a tactile sense, and thus the movement acceleration of the finger is small. Therefore, it is possible to estimate whether the GUI component is recognized from the movement acceleration of the finger. Further, by using the movement acceleration, it is possible to generate a vibration so that the GUI part to be operated can be recognized by tactile sensation when the movement acceleration becomes negative by approaching the operation target GUI part.

  In addition, the camera 18 is provided as an imaging unit that captures an operator's face when the operator is operating the apparatus, and the state determination circuit 11 determines the position of the GUI component based on the face image captured by the camera 18. The recognition state of the operator may be estimated. An operator who does not recognize the GUI component correctly wrinkles between the eyebrows, narrows his eyes, or makes a characteristic expression. Further, an operator who is visually impaired does not have his / her eyes on the operation input panel 17. Thus, since the operator who does not recognize the GUI component correctly has a characteristic expression or a characteristic point of view, the operator's face when the operator imaged by the camera 18 is operating the apparatus. Thus, it can be estimated whether or not the GUI component is recognized.

2: Liquid crystal panel 3: Contact sensor 4: Surface substrate 5A, 5B: Piezoelectric drive device 6: Contact sensor processing circuit 7: Image display circuit 8: Drive control circuit 9: Main control device 11: State determination circuit 12: Operation feeling generation Circuit 13: Standing wave generation circuit 14: Contact state determination circuit 15: Recognition state determination circuit 16: Memory buffer 17: Operation input panel 18: Camera 19: Image arithmetic processing circuit 21: GUI button 21
100: Touch panel device 233: Standing wave

JP 2005-149405 A JP 2005-149197 A

Claims (8)

An operation input panel that receives an operation input by contacting an operating body with its surface;
An image display unit provided opposite to the operation input panel and displaying an image;
An operation component generation unit that generates an operation component to be displayed as an image on the image display unit,
In the touch panel device in which the operation body is brought into contact with a position on the operation input panel corresponding to the position of the operation component displayed on the image display unit, and the operation component is operated by the operation body.
Vibration generating means for causing the operation input panel to generate vibration such that the operation component can be tactilely recognized at a position of the operation input panel corresponding to the position of the operation component displayed on the image display unit;
Estimating means for estimating the recognition state of the operator with respect to the position of the operation component;
A touch panel device comprising: control means for controlling the vibration generating means based on an operator's recognition state estimated by the estimating means. The touch panel device according to claim 1,
The touch panel device according to claim 1, wherein the control unit controls the driving of the vibration generating unit to stop when the estimation unit estimates that the operator recognizes the operation component. The touch panel device according to claim 1 or 2,
The control means is configured to estimate the amplitude of vibration generated at the position of the operation input panel corresponding to the position of the operation part when the estimation means estimates that the operator recognizes the operation part a little. The vibration control means controls the vibration generation means to be smaller than the amplitude of vibration generated at the position of the operation input panel corresponding to the position of the operation component when it is estimated that the operator does not recognize the operation component at all. A touch panel device characterized by that. The touch panel device according to any one of claims 1 to 3,
The said estimation means estimates the said operator's recognition state with respect to the position of the said operation component based on the contact state with the said operation input panel of the said operation body, The touch panel apparatus characterized by the above-mentioned. The touch panel device according to claim 4,
The touch panel device according to claim 1, wherein a degree of pressing of the operation body against the operation input panel surface is used as a contact state of the operation body with the operation input panel. The touch panel device according to claim 4,
A touch panel device using a moving speed of the operating body as a contact state of the operating body with the operation input panel. The touch panel device according to claim 4,
A touch panel device, wherein a moving acceleration of the operating body is used as a contact state of the operating body with the operation input panel. The touch panel device according to claim 1,
Comprising imaging means for imaging the face of the operator when the operator is operating the device;
The touch panel device is characterized in that the estimation means estimates a recognition state of the operator with respect to a position of the operation component based on a face image captured by the imaging means.

Images