JP2006085687A - Input device, computer device, information processing method and information processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an input device appropriately determining a contact state of an object to the input device, and also to provide a computer device, an information processing method and an information processing program. <P>SOLUTION: This input device includes: a display device 5 displaying an image of the input device 20; a contact position detecting part 21 for detecting a position of an object brought into contact with a contact detecting surface provided along a display surface of the display device 5; a contact strength detecting part 21 for detecting strength when the object contacts the contact detecting surface; and a determining part 23 which extracts a feature quantity related to the strength detected by the contact strength detecting part 21, compares the feature quantity with a preset threshold, and thereby determines a contact state of the object. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an input device, a computer device, an information processing method, and an information processing program for inputting information to, for example, a computer.

  Conventionally, as an interface of a terminal device such as a computer, a keyboard and a mouse are generally used as an input device, and a CRT (cathode-ray tube) or an LCD (liquid crystal display) is provided as a display device.

  A so-called touch panel in which a display device and an input device are stacked is also used for an interface of various terminal devices and a tablet-type portable small computer, and its use is expanding.

Furthermore, a technique related to a touch panel that can easily input characters even when the surface area of a device that can be used for character input, such as a mobile phone or a PDA (Personal Digital Assistant), is small is disclosed (for example, , See Patent Document 1).
JP 2003-196007 A (paragraph number “0037”, FIG. 2)

  However, when an object such as a fingertip or a pen-shaped pointing stick is brought into contact with a conventional touch panel, it is difficult to determine whether the touch panel is on the touch panel or the touch panel is pressed. Input may occur.

  An object of the present invention made to solve such a technical problem is to provide an input device, a computer device, an information processing method, and an information processing program that can appropriately determine the contact state of an object. .

  A feature of the embodiment of the present invention is that, in the input device, a display device that displays an image for recognizing the input position, and a position of an object that contacts a contact detection surface provided along the display surface of the display device A feature detection unit that extracts a feature amount related to the intensity detected by the contact strength detection unit, a contact strength detection unit that detects strength when the object contacts the contact detection surface, and a preset feature amount. A determination unit that determines a contact state of the object on the contact detection surface by comparing the threshold value.

  Further, according to the embodiment of the present invention, the input device includes a display device that displays an image for recognizing the input position, and a position of an object that crosses a contact detection area provided along the display surface of the display device. A contact position detection unit that detects the intensity of the object, a contact intensity detection unit that detects an intensity when the object crosses the contact detection region, and a feature amount related to the intensity detected by the contact intensity detection unit. And a determination unit that determines a contact state of an object in contact with the display surface by comparing the set threshold value.

  Further, according to the embodiment of the present invention, in the computer device, a display device that displays an image for recognizing an input position, and an object that contacts a contact detection surface provided along the display surface of the display device. A feature value relating to the intensity detected by the contact detection unit that detects the position, a contact strength detection unit that detects the strength when the object contacts the contact detection surface, and the strength detected by the contact strength detection unit is extracted in advance. A determination unit that determines the contact state of the object on the contact detection surface by comparing the set threshold value, and an input that is input via the contact detection surface based on the contact state of the object detected by the determination unit And a processing unit that executes processing according to the result.

  Further, according to the embodiment of the present invention, in the information processing method, the step of displaying an image for recognizing the input position on the display device, and the position of the object in contact with the contact detection surface stacked on the display device are provided. By comparing the step of detecting, the step of detecting the strength when the object is in contact, the step of extracting the feature amount related to the strength, and the feature amount extracted by the extracting step with a preset threshold value, Determining the contact state of the object on the contact detection surface.

  Further, according to the embodiment of the present invention, in the information processing program, an input device that displays an image for recognizing an input position on the display device is provided on a contact detection surface provided along the display surface of the display device. Procedures for detecting the position of an object in contact, procedures for detecting the strength when the object contacts the contact detection surface, procedures for extracting feature quantities related to the strength, and feature quantities extracted by the extraction procedure and presetting A procedure for determining the contact state of the object on the contact detection surface by comparing the threshold value.

  According to the present invention, it is possible to provide an input device, a computer device, an information processing method, and an information processing program that can appropriately determine the contact state of an object.

  Next, embodiments of the present invention will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. However, it should be noted that the drawings are schematic and ratios of dimensions and the like are different from actual ones. Accordingly, specific dimensions and the like should be determined in consideration of the following description. Moreover, it is a matter of course that portions having different dimensional relationships and ratios are included between the drawings.

[First embodiment]
An input device according to an embodiment of the present invention is a kind of input / output device of a terminal device such as a computer.

  As shown in FIG. 1, a notebook computer 1 having an input device according to the present embodiment includes a computer main body 30 having a calculation unit such as a central processing unit and an input unit 3 that is a user interface for the computer main body 30. A lower housing 2A provided and an upper housing 2B provided with a display unit 4 including a display device 29 having an LCD structure are provided.

  The computer main body 30 processes information input via the input unit 3 of the lower casing 2A in the central processing unit, and displays the result on the display unit 4 of the upper casing 2B.

  The input unit 3 provided in the lower housing 2A detects a contact between a display device 5 configured by an LCD and an object (such as a fingertip or a tip of a pen-shaped pointer) on the display surface of the display device 5. And an image for allowing the display device 5 to recognize the input position (for example, each key of the keyboard, various input operation keys, a mouse having a right button, a left button, a scroll wheel, and a body, etc. An image representing the input device operated by the user is displayed. FIG. 1 shows a state in which an image representing a keyboard (virtual keyboard 5a) and an image representing a mouse (virtual mouse 5b) are displayed.

(Configuration of input device)
As shown in FIG. 2, the input unit 3 includes a backlight device 6 having a light emitting region, a display device 5 stacked on the light emitting region of the backlight device 6, and a display screen of the display device 5. The touch panel 10 is provided.

  The backlight device 6 may be configured by combining a fluorescent tube and a light guide plate, which are generally employed in display devices such as notebook personal computers in recent years, and a white LED (light emitting diode) that has recently been put into practical use. ) May be arranged in a distributed manner on a plane.

  The configuration of the display device 5 and the backlight device 6 may be the same as that of a display device employed in a conventional notebook computer display or an external LCD display for a desktop computer. 5, if a self-luminous pixel that has been put into practical use is adopted, the display device 5 does not need to have transparency, and the backlight device 6 can be omitted.

  The display device 5 includes a plurality of pixels 5c arranged in a grid in the x direction and the y direction to form a display screen. The display device 5 is driven by a display driver 22 (FIG. 6), which will be described later. An image of the input device is displayed.

  The touch panel 10 is provided on the most surface layer portion of the input unit 3 and is provided so as to be exposed from the housing as an input operation surface of the lower housing 2A. The touch panel 10 detects a contact of an object (for example, a fingertip or a tip of a pen-shaped pointing rod) on the contact detection surface 10a.

  In the present embodiment, a resistive film type touch panel 10 is used. The resistive film type touch panel includes an analog type and a digital type. There are a plurality of types of analog type touch panels, such as a 4- to 8-wire type. Basically, parallel electrodes are provided, and the coordinates are extracted by detecting the potential of the contacted part. By configuring and laminating this independently for each of the X direction and the Y direction, the X and Y coordinates of the contacted part can be detected. However, in the analog method, it is difficult to detect multi-point simultaneous pressing, and it is not suitable for detecting a contact area. In this embodiment, a digital method is used to detect the contact area in addition to the contact position. Note that, in any case, the display device 5 can be visually recognized from the surface layer side because the contact detection surface 10a has light transmittance.

  As shown in FIGS. 3 and 4, in the touch panel 10, a base 11 having X electrodes 12 formed of a plurality of (n) transparent electrodes in a strip shape arranged at regular intervals in the X direction, and the Y direction. And a base 13 having a plurality of (m) transparent electrodes Y arranged as a plurality of (m) transparent electrodes arranged at regular intervals are laminated so that the electrode formation surfaces face each other. In this laminated structure, the X electrode 12 and the Y electrode 14 are laminated so as to be orthogonal to each other. As a result, n × m contact detectors 10b where the X electrode 12 and the Y electrode 14 intersect are formed in a matrix.

  In the base 11, convex dot spacers 15 made of an insulating material are provided at regular intervals between the X electrodes 12 on the surface on which the X electrodes 12 are provided. The height of the dot spacer 15 is formed to be greater than the total thickness of the X electrode 12 and the Y electrode 14, and the tip of the dot spacer 15 contacts the surface 13 B of the base 13 exposed between the Y electrodes 14. Touch. As a result, as shown in FIG. 5, in the configuration in which the bases 11 and 13 are stacked, the dot spacer 15 is sandwiched between the bases 11 and 13 and the X electrode 12 and the Y electrode 14 are not in contact with each other. That is, the X electrode 12 and the Y electrode 14 are held apart by the dot spacer 15. In this state where the X electrode 12 and the Y electrode 14 are held apart from each other, the X electrode 12 and the Y electrode 14 can be brought into contact with each other by bending the base 13.

  That is, the touch panel 10 is configured such that the surface 13B opposite to the surface on which the Y electrode 14 of the base 13 is provided is exposed to the outside of the casing as an input operation surface, and this surface is exposed to a fingertip or a pen-like shape. By pressing with the tip of the indicator bar or the like, the base 13 can be bent and the Y electrode 14 can be brought into contact with the X electrode 12.

  In this case, when the input operation surface of the touch panel 10 is pressed with a fingertip or the tip of a pen-shaped indicator bar or the like, if the pressure is equal to or lower than a predetermined pressure, the base 13 is not sufficiently bent, and the Y electrode 14 and the X electrode 12 Does not touch. When the pressing force exceeds a certain pressure, the base 13 is sufficiently bent so that the Y electrode 14 and the X electrode 12 come into contact with each other. Thereby, only when the base 13 is pressed with a pressing force larger than a certain pressing force, the Y electrode 14 and the X electrode 12 come into contact and are electrically connected.

  In the notebook computer 1 of the present embodiment, the contact position between the Y electrode 14 and the X electrode 12 in the input unit 3 having such a configuration is detected by a contact detection device 21 (see FIG. 6).

  That is, the notebook computer 1 includes the input unit 3 shown in FIG. 1 and a contact detection device 21 that detects a contact position between the X electrode 12 and the Y electrode 14 of the touch panel 10 provided in the input unit 3. An input device 20 is provided in the lower housing 2A.

  As shown in FIGS. 2 and 6, the input information processing device 20 includes an input unit 3, a contact detection device 21 that detects a contact position of a fingertip or the like on the touch panel 10 of the input unit 3, and a contact detection device 21. A device control IC 23 that converts the detected contact position information into a digital signal and performs various processes to be described later and I / O control related to communication with the computer main body 30, and a memory that stores various processing programs and data in the input device 20 24, and a speaker driver 25 and a speaker 26 that perform various notifications by voice and warnings by beep sounds.

  The contact detection device 21 sequentially applies a voltage to each X electrode 12 (FIG. 3) and outputs the voltage applied to the X electrode 12 while measuring the voltage of each Y electrode 14 (FIG. 3). The Y electrode 14 is specified.

  That is, as shown in FIG. 6, the touch panel 10 includes a voltage application unit 11 a including a power supply unit and a switch unit that selectively applies a reference voltage of the power supply unit to each X electrode 12. . The voltage application unit 11 a sequentially selects the X electrode 12 to which the reference voltage is applied based on the applied electrode selection signal supplied from the contact detection device 21, and applies the reference voltage to the X electrode 12. It is made like that.

  In the touch panel 10, a voltage measurement unit 10 b that selectively measures the voltage value of the Y electrode 14 specified by the measurement electrode selection signal from the contact detection device 21 is provided. The voltage measuring unit 11b selectively measures the voltage value appearing on each Y electrode 14 based on the measurement electrode selection signal supplied from the contact detection device 21 and sends the measurement result to the contact detection device 21. .

  As a result, when the touch panel 10 is pressed with a fingertip or the tip of a pen-shaped indicator bar, the X electrode 12 and the Y electrode 14 come into contact with each other at the pressed position, and the reference voltage applied to the X electrode 12 is It is measured through the Y electrode 14 that is in contact at this pressed position. Therefore, in the contact detection device 21, when the reference voltage is measured as the output voltage of the Y electrode 14, the Y electrode 14 and the X electrode 12 to which the reference voltage is applied can be specified. The contact detection device 21 can specify the contact detection unit 10b pressed by the fingertip or the tip of the pen-shaped indicator bar by the combination of the X electrode 12 and the Y electrode 14.

  The contact detection device 21 repeatedly detects the contact state between the X electrode 12 and the Y electrode 14 at a sufficiently high speed, thereby detecting multi-point simultaneous pressing with the accuracy of the arrangement interval of the X electrodes 12 and the arrangement interval of the Y electrodes 14. can do.

  For example, when the fingertip presses the touch panel 10 strongly, the contact area increases, and the plurality of contact detection units 10b are pressed simultaneously. In this case as well, the contact detection device 21 is connected to each X electrode 12. By repeatedly applying a reference voltage to the Y electrode sequentially at a high speed and repeatedly measuring a voltage at each Y electrode 14 at a high speed, it is possible to detect the contact detection unit 10b being simultaneously pressed. The contact detection device 21 can detect the contact area based on the number of contact detection units 10b detected in this way.

  Here, the display driver 22 is one of images (an image for recognizing an input position) representing an input device as a user interface in a computer such as a button, an icon, a keyboard, a numeric keypad, and a mouse in accordance with a command from the device control IC 23. Display above. The light emitted from the backlight device 6 is transmitted from the back side to the front side of the LCD, so that the image of the input device displayed on the display device 5 can be viewed from the front side.

  The device control IC 23 is displayed at the touched position based on the display position of each key in the image such as a keyboard displayed on the display device 5, the contact position and the contact area detected by the contact detection device 21. Determine the key image. Then, the computer main body 30 is notified of the information of the determined key.

  Based on the key information supplied from the device control IC 23, the computer main body 30 executes processing when the key is operated.

  That is, as shown in FIG. 7, a motherboard 30 a that is a computer main body 30 provided in the lower housing 2 </ b> A has a north bridge 31 and a south bridge 32 connected by a high-speed dedicated bus B <b> 1. A CPU (Central Processing Unit) 33 is connected via a system bus B2, a main memory 34 is connected via a memory bus B3, and a graphics circuit 35 is connected via an AGP (Accelerated Graphics Port) bus B4. Yes.

  The graphics circuit 35 outputs a digital image signal to the display driver 28 provided in the display unit 4 of the upper housing 2B. The display driver 28 displays an image on the display screen (LCD) of the display device 29 by driving the display device 29 based on the digital video signal.

  The south bridge 32 is connected to a PCI device 37 via a PCI (Peripheral Component Interconnect) bus B5, and to a USB device 38 via a USB (Universal Serial Bus) B6. The south bridge 32 can connect various devices connectable to the PCI bus B5 via the PCI device 37, and connect various devices connectable to the SUB bus B6 via the USB bus B6. be able to.

  A hard disk device (HDD) 41 is connected to the south bridge 32 via an ATA (AT Attachment) bus B 7 and an IDE (Integrated Drive Electronics) interface 39. Furthermore, a removable media device (magnetic disk device) 44, a serial / parallel port 45, and a keyboard / mouse port 46 are connected to the south bridge 32 via an LPC (Low Pin Count) bus B8. The keyboard / mouse port 46 passes a signal representing the operation result of the keyboard and mouse output from the input device 20 to the south bridge 32. As a result, the operation results of the keyboard and mouse are transferred from the south bridge 32 to the CPU 33 via the north bridge 31, and the CPU 33 executes processing corresponding to the operation results.

  Further, an audio signal output circuit 47 is connected to the south bridge 32 via a dedicated bus, and the audio signal is output from the audio signal output circuit 47 to the speaker 48 built in the computer main body 30, thereby the speaker 48. Can output various sounds.

  The CPU 33 displays various images on the display device 5 (see FIG. 6) provided in the display unit 4 of the upper housing 2B by executing various processing programs stored in the hard disk of the hard disk device 41 or the main memory 34. The voice is output from the speaker 48 built in the computer main body 30 of the lower casing 2A, and the operation results of the keyboard and mouse output from the input device 20 provided in the input unit 3 of the lower casing 2A are represented. Processing according to the signal is executed. Specifically, the CPU 33 controls the graphics circuit 35 according to the operation result, thereby causing the graphics circuit 35 to output a digital video signal to the display device 5 and causing the display device 5 to respond to the operation result. Display an image. Further, the CPU 33 controls the audio signal output circuit 47 according to the operation result, so that the audio signal is output from the audio signal output circuit 47 to the speaker 48 and the sound corresponding to the operation result is output from the speaker 48.

  Next, as shown in FIG. 6, a process for detecting a contact state when an object such as a fingertip or a tip of a pen-shaped pointing stick comes into contact with the contact detection surface 10 a of the touch panel 10 in the input device 20 will be described.

  The contact detection device 21 (contact position detection unit) detects the position of an object that contacts the contact detection surface 10 a of the touch panel 10 stacked on the display device 5 at predetermined processing time intervals and outputs the detected position to the device control IC 23.

  Further, the contact detection device 21 (contact strength detection unit) detects the strength when the object contacts the contact detection surface 10a. This intensity may be expressed by two or three or more discrete values, or may be expressed by a continuous value. And the contact detection apparatus 21 outputs intensity | strength information to CPU6 for every predetermined | prescribed processing time interval.

  The intensity is detected, for example, by using the contact area of the object with respect to the contact detection surface 10a or the change over time of the contact area. 8 and 9 show examples of the transition of the detected contact area. 8 and 9, the axes are made dimensionless and units and scales are omitted, but true values may be used during mounting.

  Here, the transition of the contact area is calculated, for example, by periodically acquiring the contact detection unit 10b in which contact on the touch panel 10 is detected at a predetermined scan frequency. This means that the higher the frequency, the greater the number of signal groups acquired in a given time, and the time resolution will of course improve, but the response speed on the device side and the performance of the processing circuit are also required to be increased accordingly. Therefore, an appropriate scan frequency is set.

  FIG. 8 is an example of the transition of the contact area A in a state where an object is merely in contact with the contact detection surface 10a. When the user is not conscious of keystrokes and simply puts his / her finger on the contact detection surface 10a, in this way, there is often a relatively gentle area transition.

  On the other hand, FIG. 9 is an example of a transition of the contact area A in a state where a key is pressed on the touch panel 10 with an image of an input device such as a keyboard displayed on the display device 5 of the input unit 3 as a target. The contact area is characterized by increasing sharply from zero or near zero and decreasing sharply immediately thereafter.

  As described above, if the case of FIG. 8 is close to the state of “place” the object, it can be explained that the case of FIG. 9 is close to the operation of “hit”.

  Further, the strength may be detected by using the contact pressure of the object with respect to the contact detection surface 10a or the change over time of the contact pressure. In this case, a sensor that converts pressure into an electrical signal may be provided as the contact detection surface 10a.

  10 to 12 show a touch panel 210 as an example of a sensor that converts pressure into an electrical signal.

  As shown in FIGS. 10 and 11, in the touch panel 210, a base 211 having X electrodes 212 made up of a plurality of (n) transparent electrodes in a strip shape arranged at regular intervals in the X direction, and the Y direction. And a base 213 having Y electrodes 214 made up of a plurality of (m) transparent electrodes in the form of strips arranged at regular intervals are laminated so that the electrode formation surfaces face each other. In this stacked structure, the X electrode 212 and the Y electrode 214 are stacked so as to be orthogonal to each other. As a result, n × m contact detectors 210b to 210d where the X electrode 212 and the Y electrode 214 intersect are formed in a matrix.

  In the base 211, convex dot spacers 215 made of an insulating material are provided at regular intervals between the X electrodes 212 on the surface on which the X electrodes 212 are provided. The height of the dot spacer 215 is formed to be greater than the total thickness of the X electrode 212 and the Y electrode 214, and the tip of the dot spacer 215 contacts the surface of the base 213 exposed between the Y electrodes 214. .

  Further, a high dot spacer 215a (indicated as “H” in FIG. 11) and a low dot spacer 215b (indicated as “L” in FIG. 11) are provided as the dot spacers 215. As shown in FIG. 10, in the dot spacer 215, four high dot spacers 215a constitute one set, and four low dot spacers 215b constitute one set. Further, as shown in FIG. 11, a set of these four high dot spacers 215a and a set of four low dot spacers 215b are arranged on a staggered lattice. Note that the number of dot spacers 215a constituting the set of high dot spacers 215a and the number of dot spacers 215b constituting the set of low dot spacers 215b can be arbitrarily set.

  Accordingly, as shown in FIG. 12, the dot spacer 215 is sandwiched between the bases 211 and 213, and the X electrode 212 and the Y electrode 214 are not in contact with each other. That is, the X electrode 212 and the Y electrode 214 are held apart by the dot spacer 215, and the contact detection units 210b to 210e are held electrically closed.

  In such a state where the X electrode 212 and the Y electrode 214 are held apart from each other, the X electrode 212 and the Y electrode 214 can be opened by bending the base 213.

  In other words, the touch panel 210 is configured so that the surface 213A opposite to the surface on which the Y electrode 214 of the base 213 is provided is exposed to the outside of the housing as an input operation surface, and this surface is pressed with a fingertip or the like. By doing so, the base 213 can be bent and the Y electrode 214 can be brought into contact with the X electrode 212.

  In this case, when the input operation surface of the touch panel 210 is pressed with a fingertip or the like, if the pressure is equal to or lower than the first pressing force, the base 213 is not sufficiently bent, and the Y electrode 214 and the X electrode 212 do not contact each other.

  Then, when the pressing force exceeds the first pressing force, the base 213 is sufficiently bent, so that low dot spacers 215b are adjacent to each other at four locations around the plurality of contact detection units (the Y electrode 214 and the X electrode 212). The contact detection unit 210b adjacent to each other without being sandwiched is opened. However, among the plurality of contact detection units, the contact detection units 210c and 210d adjacent to the higher two or more surrounding dot spacers 215a remain closed.

  Further, when the pressing force exceeds the second pressing force that is larger than the first pressing force, the base 213 is further bent, and the contact that is adjacent to the low dot spacers 215b at the two surrounding locations among the plurality of contact detection units. The detection unit 210c is electrically opened. However, the contact detection unit 210d in which the high dot spacers 215a are adjacent to the surrounding four locations among the plurality of contact detection units is still closed.

  Further, when the pressing force exceeds a third pressing force that is larger than the second pressing force, the base 213 is further bent, so that the high dot spacers 215a are adjacent to each other at four locations among the plurality of contact detection units. The detection unit 210d is also opened.

  Since these three types of contact detectors 210b to 210d are present in the area of the area that is pressed by the fingertip or the like, they can detect the contact position and operate as a sensor that converts the pressing force into a three-stage electrical signal. To do.

  In the notebook computer 1 of the present embodiment having such a touch panel 210, the contact detection device 21 detects which of the plurality of contact detection units is open.

  The contact detection device 21 detects, for example, a position where the contact detection unit located at the center of a group of adjacent open contact detection units is a pressed position of the contact detection surface 10a.

  For example, the contact detection device 21 defines ranks 1 to 3 for each of the contact detection units 210b to 210d, and the highest rank among a group of adjacent open contact detection units is set as a pressing force. Output. The detection method of the contact area and pressure distribution of the contact detector 221 is as follows.

  FIG. 13 is a layout diagram of the low dot spacer 215b and the high dot spacer 215a. In the arrangement of the low dot spacer 215b and the high dot spacer 215a shown in FIG. 11, the contact detection unit 210 is surrounded by four dot spacers. Among these, the number of high dot spacers 215a to be fastened is represented by a numeral and shown in the corresponding part of the contact detection unit 210 (210a to 210d) in FIG.

  The ellipse shown in FIG. 14 is the area of the part touched by the finger. This is called the outer circle.

  When the contact pressure in the contact area at this time (ie, the pressing force per unit area) is only large enough to contact the contact detection unit indicated by the numeral “0”, the contact detection device 21 is shown in FIG. Only the contact detection unit (contact detection unit 210b shown in FIG. 11) indicated by the numeral “0” located inside the ellipse shown in FIG.

  However, if a stronger pressing force is applied than in the case of FIG. 14 even when the contact area is the same, the contact detection device 21 is another ellipse (referred to as a middle circle) inside the outer circle in FIG. The contact detection unit (contact detection unit 210c shown in FIG. 11) indicated by the numeral “2” existing inside is also detected as a contact.

  When the pressing force is strong, the outer circle area actually increases as described in the operation principle of this embodiment. However, for the sake of explanation, it is assumed that the size of the outer circle is constant.

  Actually, neither the outer circle nor the middle circle is classified into an elliptical shape as shown in FIG. 15, and actually, as shown in FIG. In many cases, there are contact detection portions that are detected as contacts, and conversely there are “2” and “0” contact detection portions that are not detected as contacts even inside the middle circle. These exceptions are indicated by italic numbers in FIG. However, the contact detection device 21 determines the size, position, and shape of the outer circle and the middle circle as well as the inner circle, which will be described later, as the boundary line so that such a mixed error is minimized. At this time, the contact detection device 21 does not give the boundary line a degree of freedom with a very complicated shape, but by appropriately setting the radius of curvature, the boundary line shape is moderately smooth and yet has few errors. To decide. The radius of curvature is appropriately set based on a machine learning algorithm based on experience or experiment. The objective function at this time is the keystroke identification error rate based on the area surrounded by the outer circle, the middle circle, and the inner circle at the time of keystroke and the rate of change with time, and the minimum radius of curvature is determined so that this is minimized.

  The boundary line determination method described here is also applied to the above-described and later-described FIG. 14, FIG. 15, FIG. 17, and FIG. 18, but for the sake of simplicity, the description and the contact detection unit will be described hereinafter. Illustrations of crossing borders and crossings are omitted.

  Now, following the description of FIG. 15, the description shifts to the description of FIG. FIG. 17 shows a case where a stronger pressing force is applied than in FIG. Here, a region indicated by an inner circle appears inside the middle circle. In this area, all of the contact detection units (contact detection units 210b, 210c, and 210d shown in FIG. 11) indicated by numerals “0”, “2”, and “4” are determined to be in contact.

  Subsequently, in FIG. 18, the area indicated by the middle circle and the inner circle is enlarged. This indicates that a stronger pressing force is applied than in FIG.

  As described above, as shown in FIG. 14, FIG. 15, FIG. 17, and FIG. 18, by detecting the time transition of the area of each circle and the time transition of the area ratio of each circle, a finger, etc. It is possible to detect whether the contact is simply a contact state or intended for keystroke with little misdiagnosis. For example, the graph of the pressure transition when the strength is detected from the contact pressure of the object with respect to the contact detection surface 10a or the time change of the contact pressure using the sensor for converting the pressure into an electric signal as described above is shown in FIG. In both graphs of FIG. 9, if the vertical axis is replaced with the contact pressure, the same tendency can be seen between “put” and “hit”.

  The device control IC 23 (determination unit) shown in FIG. 6 receives the intensity detected by the contact detection device 21, extracts a feature quantity related to the intensity, and is preset with the feature quantity or a value calculated from the feature quantity. The contact state of the object is determined by comparing the threshold value. Examples of the contact state of the object include three patterns of “non-contact”, “contact”, and “keystroke”. “Non-contact” is a state where an object is not in contact with the image displayed on the display device 5, and “Contact” is a state where an object is placed on the image displayed on the display device 5. "Is a state where an object is hitting the image displayed on the display device 5. A method for determining such a contact state will be described in detail later with reference to FIGS.

  The threshold value for determining the contact state can be adjusted. For example, as shown in FIG. 19, the device control IC 23 of the input information processing apparatus 20 displays a “weak” button 20 b, a “strong” button 20 c, and a level meter 20 a indicating the magnitude of the threshold on the display device 5. It is assumed that the level meter 20a sets threshold values for the “contact” state and the “keystroke” state. In the case where the user does not recognize the keystroke in many cases even though the user presses the image while paying attention to the keystroke, the display area of the “weak” button 20b is depressed. The device control IC 23 determines whether or not the “weak” button 20 b is pressed based on the display position of the “weak” button 20 b and the contact position detected by the contact detection device 21. In this case, the display driver 22 is controlled to move the display position of the level meter 20a displayed on the display device 5 to the left side, thereby lowering the threshold value. It should be noted that the image is not actually “pressed” (pressed down), but only pressure is applied to the surface of the screen, but here, for convenience of explanation, the user is conscious of keystrokes on the screen. Contact is defined as “pressing”. Alternatively, the level meter 20a may be directly changed by dragging the slider display portion 20d displayed on the level meter 20a.

  In addition, the device control IC 23 (notification unit) receives, for example, from the contact detection device 21 on the mother board 30a (FIG. 7) of the computer main body 30 that performs processing based on the operation result of the input device such as a keyboard or a mouse. The information indicating the contact position of the object and the contact state detected based on the position is notified. For example, the key position in the “keystroke” state and the key position in the “contact” state are notified to the terminal device.

  The device control IC 23 (display control unit) shown in FIG. 6 changes the display form of the image displayed on the input device 5 according to the contact state of the object on the contact detection surface 10a. As described above, the contact state includes, for example, three patterns of “non-contact”, “contact”, and “keystroke”. The device control IC 23 changes and displays the brightness, color, shape, contour pattern and thickness of the image, blinking / lighting, blinking interval, etc., according to such a contact state.

  For example, it is assumed that a virtual keyboard is displayed on the display device 5 and the user performs an input operation. As shown in FIG. 20, when the user places a finger at a so-called home position that is a preparation position for input operation, “S”, “D”, “F”, “J”, “K”, “L” The key is a state in which the finger is “contacting”, and the device control IC 23 lights an image representing these keys in yellow, for example, in accordance with the “contacting” state. The other keys are in a “non-contact” state, and the device control IC 23 lights images representing these keys in blue, for example. As shown in FIG. 21, when the “O” key is pressed, the “O” key is in a state where the finger is “keying”, and the device control IC 23 displays an image representing this key, for example. The keys “S”, “D”, “F”, “J” are in a state where the finger is “touching”, and the device control IC 23 turns the image representing these keys yellow. Light up.

  At this time, if it is not necessary to identify all of “non-contact”, “contact”, and “keystroke”, the user may be able to select a contact state for changing the display form.

  The device control IC 23 (sound generator) shown in FIG. 6 is predetermined according to the contact state determined by the relative relationship between the position of the object detected by the contact detection device 21 and the position of the image of the input device such as a keyboard or a mouse. The recognition sound is determined, and the speaker driver 25 is controlled based on this determination, so that the recognition sound is output from the speaker 26. For example, it is assumed that a virtual keyboard is displayed on the display device 5 and the user taps a key. At this time, the device control IC 23 calculates a relative position between the key input position detected by the contact detection device 21 and the center of the key displayed on the display device 5. A method for calculating the relative position will be described in detail later with reference to FIGS.

  When the device control IC 23 determines that the relative distance between the position of the key input and the center of the displayed key is larger than a predetermined value in the “hit” state, the device control IC 23 controls the speaker driver 25 to perform predetermined recognition. Sound is output from the speaker 26. This recognition sound may be expressed in a form (different timbre, time, pattern, etc.) different from the recognition sound emitted in the normal “striking” state.

  For example, it is assumed that a virtual keyboard is displayed on the display device 5 and the user performs an input operation. The user registers the finger position of the so-called home position in advance. If the device control IC 23 determines that the finger is placed on a key other than the home position (“contact” state) when the user places the finger, the finger is placed on the home position (“contact”). It is also possible to emit a recognition sound in a different form (different timbre, time, pattern, etc.) from the case where it is determined as the “state”.

  The light emitting device 27 (light emitting unit) is provided on the surface of the input device 20 and emits light according to the contact state according to the determination of the device control IC 23. For example, when it is determined that the user has placed a finger at the home position, the device control IC 23 causes the light emitting device 24 to emit light.

  The memory 24 stores the history of the contact position and the contact strength of the object over the past certain time. The memory 24 may be constituted by a RAM (Random Access Memory), a non-volatile memory such as a flash memory, a magnetic disk such as a hard disk or a flexible disk, an optical disk such as a compact disk, an IC chip, a cassette tape, or the like. Good.

  Next, holding of various programs will be described. The input device 20 according to the present embodiment stores in the memory 24 an information processing program for causing the contact detection device 21 and the device control IC 23 to execute contact position detection processing, contact strength detection processing, determination processing, and the like. The memory 24 is constituted by a RAM or the like, and the input device 20 is provided with an information reading device (not shown) for storing a program in the memory 24. As this information reading device, for example, a device that reads information from a recording medium such as a magnetic disk such as a flexible disk, an optical disk, an IC chip, or a cassette tape, or a device that downloads these programs from a network may be used. it can. When a recording medium is used, the program can be easily stored, transported, sold, and the like. Note that a storage device such as a hard disk device may be used instead of the memory 24.

(Information processing)
Next, the information processing method according to the first embodiment will be described with reference to FIGS. Note that the information processing method described below is realized by the device control IC 23 or the like executing an information processing program stored in the memory 24 or the like. That is, each step in the information processing method corresponds to each procedure that can be executed by the information processing program.

  Here, a virtual keyboard is assumed as an image of the input device displayed on the display device 5 of the input unit 3, and the user performs an input operation by touching a key of the virtual keyboard with a finger.

  First, the flow of a basic information processing method will be described with reference to FIG.

  In step S <b> 101 of FIG. 22, the input device 20 displays an image (virtual keyboard) of the input device on the display device 5. In step S102, a detection region on the touch detection surface 10a of the touch panel 10 is acquired. In step S103, it is determined whether or not there is a region where the finger touches the touch detection surface 10a. If there is no contact area, the process returns to step S102, and if there is a contact area, the process proceeds to step S104.

  In step S104, the input device 20 detects the position of the finger in contact with the contact detection surface 10a. In step S105, the input device 20 detects the strength when the finger is in contact.

Next, in step S106, the input device 20 extracts a feature amount related to the detected intensity, compares the feature amount or a value calculated from the feature amount with a preset threshold value, The contact state of the tip of the pen-shaped indicator bar is determined. For example, as described above, the contact state includes a “non-contact”, “contact”, and “keystroke” state. For example, as shown in FIG. 9, it is detected that the contact area A is zero or very small until just before, and rises steeply, and this is determined to be a “keystroke” state. Specifically, in FIGS. 8 and 9, a contact area is extracted as a feature quantity, and an area velocity or area acceleration, that is, an amount represented by ΔA / Δt or Δ2A / Δt ² is calculated from the contact area. If this numerical value is larger than a preset threshold value, it is recognized as a “keystroke” state.

Note that the threshold of ΔA / Δt or Δ2A / Δt ² when recognizing “keystroke” or “contact” varies depending on the individual or the application being used. It is possible that it will gradually change during this period. By learning and calibrating this at an appropriate timing, the identification accuracy is improved as compared with a predetermined fixed value.

  Next, in step S107, it is determined whether or not it is in the “keystroke” state. If it is not in the “keystroke” state, the process returns to step S102 to acquire a detection area. On the other hand, if it is in the “keystroke” state, the process proceeds to step S108, and the input device 20 notifies the computer main body 30 that the key has been “keyed”. Even in the “keystroke” state, in order to determine the next contact state, the process returns to step S102, and the process of acquiring the detection area is performed in parallel.

  Next, in step S109, the input device 20 changes the display form of the key image of the virtual keyboard to the display form in the “keystroke” state. Specifically, the brightness, color, shape, outline pattern and thickness, blinking / lighting, blinking interval, etc. of the key that has been “keyed” are changed to the “keystroke” state display form and displayed. Further, the input device 20 determines whether or not a predetermined time has elapsed since the key display in the “keystroke” state has been performed, and continues the display if it has not elapsed. When the time has elapsed, the display mode of the key image is returned to the normal display. Here, the determination may be made based on whether or not the number of blinks has reached a predetermined number, rather than the passage of a predetermined time.

  Next, in step S110, the input device 20 generates a recognition sound. This process will be described later in detail with reference to FIG.

  And input device 20 repeats processing of Steps S101-110 until input operation is completed (Step S111).

  Here, another example of the contact state determination method in step S106 will be described with reference to FIG.

First, in step S1061, multivariate (feature) is extracted. For example, from the graph shown in FIG. 9, the maximum area A _max at the time of contact, the transient area S _{A obtained} by integrating the contact area A, the maximum arrival time T _p required to reach the maximum area, from the start to the end of keystroke extracting as a feature amount keystroke total time T _e required for. Based on these feature amounts, the rising slope k = A _max / T _{p and the} like are calculated.

As these qualitative physical properties, the maximum area A _max is thicker finger keying strong enough large, transient area S _a is keying strong, large enough when the Uchikata a firm, up to reach time T _p is soft fingers, large enough keystrokes is slowly stronger, typing total time T _e is, slow typing speed, as the soft finger large, the rising slope k = a _max / T _p, stiff finger, typing There is a tendency that the higher the speed and the stronger the keystroke, the larger.

  These feature quantities are learned for each user, for example, by averaging from a plurality of keystrokes, and used for keystroke determination. As a method for collecting the keystroke data, it is possible to accumulate only the keystroke data by excluding the keystroke once determined to be keyed by the user, so it is possible to accumulate the keystroke data by analyzing the accumulated data. An identification threshold can be determined.

  This may be measured in common for all keys, but there may be cases where the accuracy of judgment can be further improved by dividing each finger, key, or key group into several groups.

  At this time, a threshold value is determined for each variable independently, and a logical / conditional branch such as whether any one or more predetermined variables exceeds the threshold value may be used as a basis for the determination. There is also a method for identifying keys by multivariate analysis.

  As an example of multivariate analysis, a plurality of keystrokes are recorded, a Mahalanobis space, for example, is learned from a predetermined number of multivariate sets, and then the input keystrokes are calculated with the Mahalanobis distance in this Mahalanobis space ( "See the Mahalanobis-Taguchi System, ISBN0-07-136263-0, McGraw-Hill" etc.) An algorithm that judges that "the key is keyed" as the Mahalanobis distance becomes smaller is one way to improve judgment accuracy It is.

  Specifically, in step S1062 of FIG. 23, for the target multivariate data set, an average value and a standard deviation are calculated for each variable, and the original data is z-converted (standardized) using the average value and the standard deviation. , Also called standardization). Next, a correlation coefficient between variables is calculated to obtain a correlation matrix. By the way, this learning is performed once when the initial predetermined keystroke data is collected, and it may not be changed again. However, if the keystroke habit of the user has changed, or the mechanical and electrical characteristics of the device side If the judgment accuracy drops for some reason, such as a change over time or a temperature change, the judgment accuracy can be recovered by relearning. In addition, when a plurality of users log in, etc., the determination accuracy may be recovered for each user.

  Next, in step S1063, the Mahalanobis distance of the keystroke data to be determined is calculated using the average value and standard deviation obtained for each variable and a set of correlation matrices.

  In step S1064, the contact state is determined by recognizing the multivariate (feature value). For example, when the Mahalanobis distance is smaller than a predetermined threshold, it is determined that the contact state of the object is the “keystroke” state.

As described above, if an algorithm that determines that the probability of “keystroke” is higher as the Mahalanobis distance is smaller is employed, the determination accuracy is significantly improved as compared to the case where the feature amount is used as it is. This is because according to the Mahalanobis distance, recognition considering the correlation between learned variables, that is, pattern recognition is performed, so even if only the peak value A _max is close to the average value of the keystroke data, the time T _p to reach it is For a contact pattern different from keystroke, such as when it is long, there is a feature that it shows a high discrimination ability.

  Although the keystroke identification algorithm using the Mahalanobis distance has been described in the present embodiment, it goes without saying that other multivariate analysis algorithms may be adopted to make multivariate determinations.

  Next, a process for changing the display of keys in the “non-contact” state and the “contact” state as well as the “keystroke” state will be described with reference to FIG.

  The processing in steps S201 to S202 in FIG. 24 is the same as the processing in steps S101 to S102 in FIG.

  Next, in step S203, the input device 20 determines whether or not there is a region where the object is in contact with the contact detection surface 10a. If there is no contact area, the process proceeds to step S212, and if there is a contact area, the process proceeds to step S204. In step S212, the input device 20 determines that the key of the virtual keyboard is in the “non-contact” state, and changes the display form of the key (displays “standby”). Specifically, the lightness, color, shape, outline pattern and thickness, blinking / lighting, blinking interval, etc. of the key in the “non-contact” state are changed from those in the “contact” or “keystroke” state. To display. And it returns to step S202 and acquires a detection area.

  Next, the processing in steps S204 to 206 is the same as the processing in steps S104 to S106, and thus description thereof is omitted here.

  Next, in step S207, if it is not the “keystroke” state, the process proceeds to step S213. In step S213, the input device 20 determines that the key of the virtual keyboard is in the “contact” state, and changes the display form (displays “contact”). Specifically, the lightness, color, shape, outline pattern and thickness, blinking / lighting, blinking interval, etc. of the key in the “contact” state are changed from those in the “non-contact” or “key-pressed” state. To display. And it returns to step S202 and acquires a detection area. On the other hand, if it is in the “keystroke” state, the process proceeds to step S208, and in order to determine the next contact state, the process returns to step S202, and a detection area is acquired.

  Since the processing of steps S208 to 211 is the same as the processing of steps S108 to 111, description thereof is omitted here.

  Next, in step S110 in FIG. 22, when the “keystroke” state is recognized, a recognition sound (warning sound) is generated when the keystroke position and the key position of the image of the input device (keyboard, etc.) are misaligned. The generated process will be described with reference to FIG.

  First, in step S301, the input device 20 uses the keystroke reference coordinates calculated from the coordinate group of the touch detection unit 10b of the keyed finger (for example, the contact estimated by approximating the coordinate group determined to be in the “keystroke” state). Get the center of gravity of the region.

  Next, in step S302, the input device 20 compares the keystroke reference coordinates with the reference coordinates (for example, center coordinates) of the key on the virtual keyboard. Then, a deviation between the two (hereinafter referred to as a “keystroke deviation vector”), that is, a keystroke reference coordinate and a direction and a length in the x and y planes having the reference coordinate of the key as a start point and an end point are calculated.

  Next, in step S303, the input device 20 determines in which area of the displayed key the coordinates with which the finger touches are present. This area may be divided into left and right parts, or may be divided into five areas 51 to 55 as shown in FIGS. The user may select the type of area division. Note that an area 55 in FIGS. 26 and 27 is an area recognized as having been correctly keyed.

  Next, in step S304, the input device 20 determines a recognition sound according to the area. In FIGS. 26 and 27, different recognition sounds, for example, recognition sounds having different timbre, time, and pattern are determined for the areas 51 to 55, respectively.

  Further, the input device 20 may change the recognition sound according to the length of the keystroke deviation vector, or may change the recognition sound according to the direction of the keystroke deviation vector. For example, the longer the key-off vector, the higher the tone, and the interval or tone color of the recognition sound may be changed according to the direction of the key-off vector.

  In addition, when a finger touches across two areas, an intermediate recognition sound of recognition sounds corresponding to the two areas may be adopted, and generation of an intermediate timbre takes place in the contact area of the two areas. Accordingly, the sound may be proportionally distributed, or the recognition sound of the region having a large contact area among the two regions may be employed. Further, at this time, two sounds may be generated to form a chord.

  Next, in step S305, the input device 20 generates the recognition sound determined in step S304 at a predetermined volume. Then, the input device 20 determines whether or not a predetermined time has elapsed since the generation of the recognition sound. If the predetermined time has not elapsed, the input device 20 continues to generate the recognition sound. When it has passed, the recognition sound is stopped.

  Although it has been described that different recognition sounds are determined for the regions 1 to 5 in step S304, the region 55 and the regions 51 to 54 may be different recognition sounds. For example, when the area 55 is pressed, it is recognized that the key has been correctly input, and a pattern sound different from the areas 51 to 54 or silence may be used.

  The shape and size of the area 55 may be appropriately adjusted by the user, for example, in the form of a percentage, a ratio, or a ratio with respect to the entire key top, and the hit rate of the area 55, the x and y component distribution of the keystroke deviation vector, etc. It may be recorded and automatically adjusted from the data.

  In addition, the recognition sound may be changed by distinguishing only the inside or the outside of the region 55 without distinguishing the regions 51 to 54.

  Further, the adjustment of the area 55 may be performed independently for each key, or all the areas may be collectively or divided into a plurality of groups. For example, only the main keys of the keyboard, for example, frequently used keys and kana keys (50-note keys) are collected to accumulate the keystroke deviation vector, and the shape and size change parameters are also changed simultaneously with the main key group in common You may do it.

(Function and effect)
According to the input device 20, the information processing method, and the information processing program according to the first embodiment, the device includes the contact detection device 21 (contact position detection unit and contact strength detection unit) and the device control IC 23 (determination unit). Thus, using the feature amount related to the contact strength, the determination of the contact state as to whether the user's finger placed on the contact detection surface 10a of the touch panel 10 is intended for keystroke or just touching is performed. It can be carried out.

  In the first embodiment, the contact strength can be detected using the contact area. When the strength is detected using the contact area, the contact state can be determined with higher accuracy than a method that relies on the pressure strength of the keystroke as in a conventional pressure sensor type touch panel.

  In addition, in the case of a conventional infrared type or image sensor type touch panel, it is difficult to determine “keystroke” and “contact” because only the contact area and shape are detected. According to the input device 20 according to the first embodiment, the contact state of an object can be determined easily and with high accuracy.

  Also, when detecting the strength using contact pressure, for example, the tip of a pen-shaped indicator bar is relatively hard unlike a fingertip, and the change in the contact area due to pressing is small and difficult to detect. By evaluating the rate of change of the contact area in the time axis direction, the contact state of the object can be determined easily and with high accuracy.

  Further, in the past, it was difficult to recognize keystrokes of a plurality of keys in a very short period of time. However, according to the input device 20 according to the first embodiment, a plurality of touching fingertips are used. Even in such a case, it is possible to accurately determine “a fingertip that is keyed” and “a fingertip that is just touched”. Therefore, when key input for multiple keys is performed in a very short time, with partial time overlap and parallel input, such as high-speed input by a key input expert. In addition, the contact state can be accurately grasped.

  According to the input device 20 according to the first embodiment, the device control IC 23 (determination unit) compares the feature amount related to contact strength or a value calculated from the feature amount with a preset threshold value, thereby The contact state can be determined. For this reason, the user himself / herself adjusts the threshold value in accordance with the characteristics and habits of keystrokes, so that, for example, when the same machine is used by multiple users, it is possible to make an optimal decision for each person and improve the identification accuracy. it can. In addition, even when the keystroke strength changes while using for a while, the user can freely adjust to maintain a comfortable use environment. Further, by storing a threshold value for each login user, this threshold value can be used as an initial value for each login user.

  According to the device control IC 23 (display control unit) and the display device 5 of the input device 20 according to the first embodiment, the display form of an image representing the input device can be changed according to the contact state. For this reason, for example, when a keyboard is displayed as an image representing the input device, the user can easily recognize the “non-contact”, “contact”, and “keystroke” states for the fingertip key, and the user device It is also very effective in supporting learning. In addition, by displaying the keys in the “contact” state in different display forms, it is possible to recognize whether both hands are correctly positioned at a so-called home position and to correctly set a habit.

  Moreover, when changing the brightness of a key according to a contact state, utilization of the input device 20 becomes easy even in a dark place. In addition, the colorful and dynamic display effect of the usage state also has secondary effects that satisfy the user's “joy to use”, “playfulness”, “ownership” and “satisfaction”.

  According to the device control IC 23 (sound generation unit) and the speaker 26 of the input device 20 according to the first embodiment, predetermined recognition according to the contact state is made based on the relative relationship between the contact position of the object and the position of the image of the input device. Can emit sound. For this reason, since the user can recognize the frequency of mistyping and the amount of deviation from the center, the user can practice to correct the mistake himself, which is very effective in learning.

  According to the device control IC 23 (notification unit) of the input device 20 according to the first embodiment, a contact state can be notified to a device that performs processing based on an output signal from the input device. For this reason, for example, when recognizing that a finger is placed at a home position designated by the user in advance, it is possible to notify the connected terminal device to that effect.

  According to the light emitting device 27 of the input device 20 according to the first embodiment, light can be emitted according to the contact state of the object on the contact detection surface 10a (FIG. 2). For this reason, for example, when recognizing that a finger is placed at a home position designated by the user in advance, the user can visually recognize that.

[Second Embodiment]
In the second embodiment, an image representing an input device displayed on the display device 5 is an image of a keyboard, and an input device that distinguishes and detects whether a key on the keyboard is pressed with the left hand or the right hand is described. To do. Note that in this embodiment, the same parts as those of the input device 20 according to the first embodiment are denoted by the same reference numerals, and redundant description is omitted.

  As shown in FIG. 28, the input device 60 according to the second embodiment includes a resistance detection surface 65 provided on the most surface layer portion, a touch panel 10 having a contact detection surface 10a on the surface, a display device 5, The backlight device 6 is stacked. Since the touch panel 10, the display device 5, and the backlight device 6 are the same as those in the first embodiment, description thereof is omitted here.

On the surface of the resistance detection surface 65, as shown in FIG. 29, a key detection element 66 arranged corresponding to the key position of the virtual keyboard, a left detection electrode 68a for detecting the palm of the left hand, and the palm of the right hand A right-side detection electrode 68b for detecting the pattern is printed and wired with a pattern surface made of a transparent conductive film. This transparent conductive film has the same structure as that generally used in various displays such as an LCD. A wiring extends from the resistance detection surface 65 to the resistance detection device 71 for each key detection element 66, and a detection signal is output. Figure 30 is an enlarged view of a transparent conductive film, when contacted by the object in the key-detecting element 31, wiring X ₁ to X _n, by weak current flows through the wiring Y ₁ to Y _m, the resistance detector 71 Detects that an object has touched.

When the touch panel 10 is an electric resistance type, such electrical wiring (wiring X _{1 to} X _n , wiring Y _{1 to} Y _m ) is drawn to the outside for the detection element itself for detecting contact / keystroke. This may be used. Moreover, when using what identifies a keystroke optically as the touch panel 10, you may laminate | stack the resistance detection surface 65 on the contact detection layer.

Further, as shown in FIG. 31, when the object on the left side detection electrode 68a is in contact, a weak current flows from the left side detection electrodes 68a to the left side contact P _L. Similarly, when the object on the right side detection electrodes 68b are in contact, a weak current flows from the right side detection electrodes 68b to the right contacts P _R.

  As shown in FIG. 32, the input device 60 according to the second embodiment includes a resistance detection surface 65, a resistance detection device 71, a touch panel 10, a contact detection device 21, a device control IC 23, a memory 24, A display driver 22 and a display device 5 are provided. Since the contact detection device 21 is the same as that of the first embodiment, description thereof is omitted here.

The resistance detection device 71 (left / right detection unit) detects whether a key of the keyboard is pressed with the left hand or the right hand. At the end of the resistance detection surface 65, wirings (wirings X _{1 to} X _n and wirings Y _{1 to} Y _m ) extending from the key detection element 66 and contacts P _L extending from the left detection electrode 68a and the right detection electrode 68b are provided. , P _R are provided. For example, if a key detecting element 66 is determined to be in contact, the resistance detector 71, the left hand palm contact P _L, in the way of fingers small electric resistance of the right hand palm contact P _R in contact with the key It is determined that

  The device control IC 23 (determination unit) receives the intensity detected by the contact detection device 21, and determines the contact state of the object by comparing this intensity with a preset threshold value.

  Further, the device control IC 23 (communication unit) notifies the mother board 30a (FIG. 7) of the computer main body 30 of the determined contact state and information on whether the contact is made with the left hand or the right hand.

  Further, the display driver 22 (display control unit) changes the display form of the image of the input device according to the contact state, and further changes the display form of the image depending on whether the contact is made with the left hand or the right hand. Change and display. For example, in FIG. 20, the “S”, “D”, and “F” keys touched with the left hand and the “J”, “K”, and “L” keys touched with the right hand have different display forms.

  Further, the input device 60 according to the second embodiment may include the speaker driver 25 and the speaker 26 as in the case of the input device 20 according to the first embodiment. The device control IC 23 controls the speaker driver 25 (sound generator) to output a predetermined recognition sound corresponding to the contact state according to the relative relationship between the position of the object and the position of the image of the input device, and further, the left hand Different recognition sounds are output when touching with the right hand and when touching with the right hand.

  The light emitting device 27 (light emitting unit) shown in FIGS. 1 and 2 emits light according to the contact state, and further, different modes (for example, different colors, different when touched with the left hand and when touched with the right hand). Flashes at the flashing time). The input device 60 may include two light emitting devices on the left and right surfaces, and the light emitting device on the side in contact with the object may emit light.

  Further, the memory 24 stores the contact position of the object, the contact strength, and the history of contact with the left or right hand over a certain period of time in the past.

  The holding of various programs such as the input information processing program is the same as that in the first embodiment, and thus the description thereof is omitted here.

(Input information processing)
Next, an information processing method according to the second embodiment will be described with reference to FIG. Note that the information processing method described below is realized by the device control IC 23 or the like executing an information processing program stored in the memory 24 or the like. That is, each step in the information processing method corresponds to each procedure that can be executed by the information processing program.

  Here, a virtual keyboard is assumed as an image of the input device, and the user performs an input operation by touching a key of the virtual keyboard with a finger.

  First, steps S401 to S403 are the same as the processing of steps S101 to S103 in FIG.

Next, in step S <b> 404, the input device 60 distinguishes and detects whether a keyboard key is pressed with the left hand or the right hand. For example, it is determined from the detection signal from the resistance detection surface 65 whether the contact area is the left palm detection area or the right palm detection area. Specifically, if there key detection element 66 is determined to be in contact, the resistance detector 71, the left contact P _L, in contact with the key in the way of fingers small electric resistance of the right contact P _R It is determined that

  In step S405, the input device 60 detects the position of the finger in contact with the contact detection surface 10a of the touch panel 10, and in step S406, detects the strength when the finger contacts.

  Steps S407 to 408 are the same as the processes of steps S106 to S107 in FIG.

  Next, in step S409, the input device 60 notifies the mother board 30a of the computer main body 30 that the key has been “keyed”. At this time, it is also notified of whether or not the keystroke is performed with the right hand or the left hand.

  Next, in step S410, the input device 60 changes the display form of the key image of the virtual keyboard according to the contact state. Specifically, the brightness, color, shape, outline pattern and thickness, flashing / lighting, flashing interval, etc. of the key that has been “keyed” are changed and displayed. At this time, different display modes are used depending on whether the key is hit with the right hand or the left hand.

  Steps S411 to 412 are the same as the processing of steps S110 to 111 in FIG.

(Function and effect)
According to the input device 60, the information processing method, and the information processing program according to the second embodiment, whether the keyboard key is pressed with the left hand or the right hand by providing the resistance detection device 71 (left / right detection unit). Can be detected separately. For this reason, for example, in the case of a general QWERTY type keyboard pattern, the left hand key is “F”, “D”, “S”, “A” key, and the right hand key is “J”, “K”, “L”, “;” It is possible to determine that both hands are at the home position when touching and, when touching the home position with the opposite left and right fingers, the home position is not recognized and errors can be avoided. In addition, it is good also as a structure which can change freely the kind and number of keys used as the reference | standard of determination.

  In addition, for example, in the case of a key that is easy to touch with either finger, such as a key near the center of the keyboard, it is possible to determine whether or not it is touching with the correct assignment hand, and the total of such determinations can be calculated. By saving and showing to the user, the user can be made to recognize the correct guideline for correcting the right and left fingers.

[Other embodiments]
Although the present invention has been described according to the above-described embodiments, it should not be understood that the descriptions and drawings constituting a part of this disclosure limit the present invention. From this disclosure, various alternative embodiments, examples and operational techniques will be apparent to those skilled in the art.

  For example, in the above-described first and second embodiments, the input unit 3 is configured integrally with the computer main body 30, but instead of this, as a so-called external device, for example, a USB (Universal Serial Bus) It may be configured to be externally attached to the computer body using an existing connection standard such as.

  In this case, for example, as shown in FIG. 34, the input device 20 configured externally to the computer main body has an image of the input device (here, the virtual keyboard 5a and the virtual keyboard) on the display device (here, the LCD) 5. The mouse 5b) is displayed. The input device 20 is externally attached to the computer main body by a USB (universal serial bus) cable 7 or the like. The operation result of the keyboard or the like output from the input device 20 is transmitted to the computer main body and processed in the computer main body. The processing result is displayed on a display device connected to the computer main body.

  34, in the same manner as the input device 20 shown in FIG. 1, the process of displaying the image of the input device such as the virtual keyboard 5a on the display device 5 is shown in FIGS. The display processing of the image of the input device such as a keyboard image based on the operation result of the virtual keyboard 5a and the virtual mouse 5b is executed on the input device 20 side, but may be executed on the computer main body side.

  That is, FIG. 35, which shows parts corresponding to those in FIG. As shown in FIG. 35, the input device 141 receives a digital video signal for displaying an image representing a keyboard or the like from the graphics circuit 35 of the computer main body 130 by the display driver 22. An image of an input device such as the virtual keyboard 5a is displayed.

  The keystroke / contact position detection device 142 detects the contact position or contact state of an object detected on the contact detection surface 10a of the touch panel 10 by the method described above with reference to FIGS. The keyboard operation result or mouse operation result is output to the keyboard / mouse port 46 of the computer main body 130 via the keyboard connection cable (PS / 2 cable) or the mouse connection cable (PS / 2 cable).

  The computer main body 130 executes processing based on these operation results, using the keystroke results and the like input via the keyboard / mouse port 46 as keyboard operation results and mouse operation results. In this processing, the computer main body 130 outputs a digital video signal corresponding to the operation result from the graphics circuit 35 to the display driver 28 of the display unit 150. As a result, an image corresponding to the operation result is displayed on the display device 29. Further, in the computer main body 130, a keystroke result or the like input via the keyboard / mouse port 46 is processed based on a keyboard operation result and a mouse operation result, and a digital video signal corresponding to the operation result is converted into a graphics circuit. 35 to the display driver 22 of the input device 141. Accordingly, for example, the display processing shown in FIGS. 20 and 21 in which the color of the keyboard image is changed according to the operation result can be executed on the display device 5.

  In this case, the computer main body 130 operates as a display control unit, a contact strength detection unit, and a determination unit.

  As indicated by broken lines in FIG. 35, the operation results of the keyboard and mouse are output to the USB device 38 of the computer main body 130 via the USB connection cables 7a and 7b instead of the keyboard connection cable and the mouse connection cable. It may be.

  As another example in which the input device 141 is externally connected to the computer main body 130, as shown in FIG. 36, in the input device 141, the operation result of the touch panel 10 is detected by the touch panel control / processing device 143. The operation result is output to the serial / parallel port 45 of the computer main body 130 via the serial connection cable 9.

  The computer main body 130 recognizes the input device 141 as a touch panel by a touch panel driver installed in advance, and executes processing based on the operation result. As described above, the computer main body 130 can perform processing in which the input device 141 is recognized as a touch panel.

  In this case, the computer main body 130 operates as a display control unit, a contact strength detection unit, and a determination unit.

  In FIG. 36, instead of the serial connection cable 9, the operation result of the touch panel may be output to the USB device 38 of the computer main body 130 using the USB connection cable 7.

  Further, in the first and second embodiments, the touch panel 10 is provided only in the input unit 3, but in addition to this, the touch panel 10 may be provided in the display unit.

  For example, the touch panel 10 may be provided in the upper housing 2B in addition to the lower housing 2A, as shown in FIG. As shown in FIG. 37, the contact detection result of the touch panel 10 provided in the upper housing 2 </ b> B is supplied to the touch panel control / processing device 143. The touch panel control / processing device 143 outputs the detection result as a touch panel operation result to the serial / parallel port 45 of the computer main body 130 via the serial connection cable 9.

  In the computer main body 130, the touch panel driver installed in advance recognizes the touch panel of the upper housing 2 </ b> B and executes processing based on the operation result.

  Further, the computer main body 130 outputs a digital video signal to the display driver 28 of the upper housing 2B via the graphics circuit 35. As a result, various images are displayed on the display device 29 of the upper housing 2B. A signal line is connected between the computer main body 30 and the upper housing 2B via, for example, a hinge portion 19 shown in FIG.

  In the lower housing 2A, a keystroke / contact position detection device 142 is provided. The keystroke / contact position detection device 142 detects the contact position or contact state of an object detected on the contact detection surface 10a of the touch panel 10 by the method described above with reference to FIGS. Accordingly, the operation result of the keyboard or the operation result of the mouse is output to the keyboard / mouse port 46 of the computer main body 130 via the keyboard connection cable (PS / 2 cable) or the mouse connection cable (PS / 2 cable).

  In the computer main body 130, the keystroke result input via the keyboard / mouse port 46 is processed based on the keyboard operation result and the mouse operation result, and the digital video signal corresponding to the operation result is sent from the graphics circuit 35. The data is output to the display driver 22 of the input device 141. Accordingly, for example, the display processing shown in FIGS. 20 and 21 in which the color of the keyboard image is changed according to the operation result can be executed on the display device 5.

  In this case, the computer main body 130 operates as a display control unit, a contact strength detection unit, and a determination unit.

  As shown by broken lines in FIG. 37, the operation results of the keyboard and mouse are output to the serial / parallel port 45 of the computer main body 130 via the serial connection cable 9a instead of the keyboard connection cable and mouse connection cable. It may be.

  In FIG. 37, the lower casing 2A is also provided with a touch panel control / processing device 143 similar to that of the upper casing 2B in place of the keystroke / contact position detection device 142, and is operated by the installed touch panel driver in the computer main body 130. You may make it recognize a result and perform the process according to this.

  In the first and second embodiments, the resistance film type touch panel 10 is used. However, instead of this, for example, an optical type may be used. As an example of the configuration based on the optical method, an infrared scanning sensor array can be cited as shown in FIG. Light scans from the light emitting X-axis sensor array 151e to the light receiving X-axis sensor array 151c, and from the light emitting Y-axis sensor array 151d to the light receiving Y-axis sensor array 151b. A spatial layer in which the optical paths 151f of the light intersect in a matrix form becomes a contact detection area in place of the touch panel 10. When an attempt is made to press the display surface of the display device 5 with a fingertip or the like, the fingertip crosses the contact detection area before contacting the display surface. When an object such as a fingertip crosses the contact detection area, the light path 151f is blocked, and therefore the light reception result cannot be obtained in the light receiving X-axis sensor array 151c and the light receiving Y-axis sensor array 151b. In the detection device 21, these positions can be detected by the X coordinate and the Y coordinate. Then, the contact detection device 21 detects the strength of the object crossing the contact detection area (that is, the strength when contacting the display surface of the display device 5) and the feature amount based on the strength, and determines the contact state. can do. For example, when a fingertip having a constant cross-sectional area crosses the contact detection area, a plurality of infrared rays are blocked by the fingertip, and the increase rate per unit time of the number of blocked infrared rays is determined by the fingertip. It changes according to the speed to cross. And if it is going to press a display surface strongly, it can be judged whether it pressed strongly based on the increase rate of the number of the infrared rays interrupted because the speed which a fingertip crosses a contact detection area becomes high.

  In the first and second embodiments, a notebook computer is exemplified as the terminal device. However, the present invention is not limited to this, and it is needless to say that an electronic notebook, a PDA (Personal Digital Assistant), a mobile phone, or the like may be used. is there.

  In FIG. 22, it has been described that the contact position is detected (step S104) and then the contact strength is detected (step S105). However, this order may be reversed. Similarly, although it has been described that the processing is performed in the order of notification of the keystroke state (step S108), keystroke display (step S109), and generation of a recognition sound (step S110), this order may be changed. The same applies to FIGS. 24 and 33.

  In FIG. 33, it has been described that the palm position is determined (step S404) and then the contact position is detected (step S405). However, this order may be reversed.

1 is a perspective view of a notebook computer according to a first embodiment. It is a perspective view which shows the input part of the notebook computer of FIG. It is a perspective view which shows the touchscreen of the notebook computer of FIG. It is a top view which shows the touchscreen of the notebook computer of FIG. It is sectional drawing which shows the touchscreen of the notebook computer of FIG. It is a block diagram which shows the structure of the input device of the notebook computer of FIG. It is a block diagram which shows the notebook computer of FIG. It is a graph which shows the contact state in the touchscreen of the notebook computer of FIG. It is a graph which shows the contact state in the touchscreen of the notebook computer of FIG. It is a perspective view which shows the touchscreen which converts a pressure into an electric signal. It is a top view which shows the touchscreen which converts a pressure into an electric signal. It is sectional drawing which shows the touchscreen which converts a pressure into an electrical signal. It is a basic diagram which shows the example of arrangement | positioning of the contact detection part of a touch panel. It is a basic diagram which shows the contact detection part by which a contact is detected when it presses with a weak pressing force. It is a basic diagram which shows the contact detection part by which a contact is detected when it presses with a moderate pressing force. It is a basic diagram which shows the contact detection part by which a contact is detected when it presses with a moderate pressing force. It is a basic diagram which shows the contact detection part by which a contact is detected when it presses with a strong pressing force. It is a basic diagram which shows the contact detection part by which a contact is detected when it presses with the strongest pressing force. It is a perspective view which shows the lower housing | casing of the notebook computer of FIG. It is a top view which shows the display mode in the input device of the notebook computer of FIG. It is a top view which shows the display mode in the input device of the notebook computer of FIG. 3 is a flowchart illustrating an information processing method in the input device of the notebook computer of FIG. 1. It is a flowchart which shows the detail of step S106 of FIG. 3 is a flowchart illustrating an information processing method in the input device of the notebook computer of FIG. 1. 3 is a flowchart illustrating an information processing method in the input device of the notebook computer of FIG. 1. It is a figure which shows the key area | region in the input device of the notebook computer of FIG. It is a figure which shows the key area | region in the input device of the notebook computer of FIG. It is a perspective view of the display layer of the input device which concerns on 2nd Embodiment. It is a top view which shows the detail of the resistance detection surface of the input device which concerns on 2nd Embodiment. It is a top view which shows the detail of the resistance detection surface of the input device which concerns on 2nd Embodiment. It is a basic diagram which shows the detail of the resistance detection surface of the input device which concerns on 2nd Embodiment. It is a block diagram which shows the input device which concerns on 2nd Embodiment. It is a flowchart which shows the information processing method which concerns on 2nd Embodiment. It is a perspective view which shows the input device which concerns on other embodiment. It is a block diagram which shows the input device which concerns on other embodiment. It is a block diagram which shows the input device which concerns on other embodiment. It is a block diagram which shows the input device which concerns on other embodiment. It is a perspective view which shows the touchscreen which concerns on other embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Notebook-type computer 2A ... Upper housing | casing 2B ... Lower housing | casing 3, 140 ... Input part 4 ... Display part 5, 29 ... Display apparatus 5a ... Virtual keyboard 5b ... Virtual mouse 6 ... Backlight apparatus 10, 210 ... Touch panel 10a ... contact detection surfaces 11, 13, 211, 213 ... base 15, 215 ... dot spacers 20, 60, 141 ... input device 21 ... contact detection devices 22, 28 ... display driver 23 ... device control IC
30 ... Computer body 33 ... CPU

Claims (20)

A display device for displaying an image for recognizing the input position;
A contact position detector that detects the position of an object that contacts a contact detection surface provided along the display surface of the display device;
A contact strength detector that detects strength when the object contacts the contact detection surface;
A determination unit that extracts a feature amount related to the intensity detected by the contact intensity detection unit and compares the feature amount with a preset threshold value to determine a contact state of the object on the contact detection surface; An input device comprising:   The input device according to claim 1, wherein the determination unit extracts a change in a contact area of the object as the feature amount.   The input device according to claim 1, wherein the determination unit extracts a change in contact pressure of the object as the feature amount.   The input device according to claim 1, further comprising: a display control unit that changes a display form of an image for recognizing the input position in accordance with the contact state determined by the determination unit.   Sound generation that emits a predetermined recognition sound according to the contact state determined by the determination unit based on the relative relationship between the position of the object detected by the contact position detection unit and the display position of the image for recognizing the input position The input device according to claim 1, further comprising means.   The input device according to claim 1, further comprising: a notification unit that notifies a device that performs processing based on an output signal from the input device of the contact state determined by the determination unit.   The input device according to claim 1, further comprising a light emitting unit that emits light according to the contact state determined by the determining unit. The image for recognizing the input position is a keyboard image,
The input device according to claim 1, further comprising a left / right detection unit that detects whether a key of the keyboard is pressed with a left hand or a right hand. The contact strength detector is
A first base and a second base, which are provided along the display surface and in which electrodes are respectively stacked on opposing surfaces;
The input device according to claim 1, further comprising: dot spacers having different heights provided between the first and second bases. A display device for displaying an image for recognizing the input position;
A contact position detector that detects the position of an object that crosses a contact detection area provided along the display surface of the display device;
A contact strength detection unit that detects strength when the object crosses the contact detection region;
A determination unit that extracts a feature amount related to the intensity detected by the contact intensity detection unit and compares the feature amount with a preset threshold value to determine a contact state of the object that contacts the display surface. An input device comprising: A display device for displaying an image for recognizing the input position;
A contact position detector that detects the position of an object that contacts a contact detection surface provided along the display surface of the display device;
A contact strength detector that detects strength when the object contacts the contact detection surface;
A determination unit that extracts a feature amount related to the intensity detected by the contact intensity detection unit and compares the feature amount with a preset threshold value to determine a contact state of the object on the contact detection surface;
A computer device comprising: a processing unit that executes processing according to an input result input via the contact detection surface based on a contact state of the object detected by the determination unit.   The computer apparatus according to claim 11, further comprising a display control unit that changes a display form of an image for recognizing the input position according to the contact state determined by the determination unit. Displaying an image for recognizing the input position on a display device;
Detecting a contact position of an object in contact with a contact detection surface laminated on the display device;
Detecting the intensity when the object contacts;
Extracting a feature quantity related to the intensity;
An information processing method comprising: determining a contact state of the object on the contact detection surface by comparing the feature amount extracted in the extracting step with a preset threshold value.   The information processing method according to claim 13, wherein in the step of extracting the feature amount, a change in a contact area of the object is extracted as the feature amount.   The information processing method according to claim 13, wherein in the step of extracting the feature amount, a change in contact pressure of the object is extracted as the feature amount.   The information processing method according to claim 13, further comprising a step of changing a display form of an image for recognizing the input position in accordance with the contact state determined by the determining step. In the input device that displays the image for recognizing the input position on the display device,
Detecting a contact position of an object that contacts a contact detection surface provided along a display surface of the display device;
Detecting the strength when the object contacts the contact detection surface;
A procedure for extracting a feature quantity related to the intensity;
An information processing program for executing a procedure for determining a contact state of the object on the contact detection surface by comparing a feature amount extracted by the extracting procedure with a preset threshold value.   The information processing program according to claim 17, wherein the feature amount extracting step extracts a change in a contact area of the object as the feature amount.   The information processing program according to claim 17, wherein the procedure of extracting the feature amount extracts a change in the contact pressure of the object as the feature amount. The information processing program according to claim 17, further comprising a procedure for changing a display form of an image for recognizing the input position in accordance with the contact state determined by the determining procedure.

Images