JP6315443B2 - Input device, input detection method for multi-touch operation, and input detection program - Google Patents

Description

  The present invention relates to an input device having a multi-touch input function that can be touch-operated with a plurality of fingers, and more particularly, to an input detection method for a multi-touch operation having a function of predicting touch positions after the second point.

  Currently, various electronic devices such as a personal computer, a smartphone, a tablet terminal, and an in-vehicle device are equipped with a touch panel, and various instructions are input by a touch operation (contact or approach of a finger or the like) on the touch panel. The touch operation is widely used by ordinary people who do not have IT skills because the operation buttons displayed on the display can be intuitively recognized. Touch operations include, for example, a tap operation or a double tap operation that presses an arbitrary position, a long tap operation that keeps pressing for a certain period of time, a slide operation or a flick operation that moves a finger in the pressed state, or two fingers are moved closer or away. There are various operation methods such as pinch operation, and smartphones and in-vehicle devices utilize these operation methods to improve operability and convenience, enabling various operation inputs such as menu display and various settings. Yes.

  For example, Patent Document 1 discloses a technique related to an optical touch panel that is intended to shorten the time of touch input on the optical touch panel and to obtain a high resolution coordinate value. This optical touch panel has an effect of enabling a high resolution capable of identifying the position information of the light shielding object with a dimension equal to or smaller than the arrangement interval of the optical elements and enabling touch input.

  Further, Patent Document 2 discloses a technique related to a light-transmissive touch input device that improves the efficiency of light blocking location scanning by utilizing the fact that there can be only one target touch input on the entire display screen. Yes. By increasing the number of times of light scanning in the target area without reducing the scanning time required to discriminate from ambient light per single light emitting / receiving element, it is possible to improve both measurement accuracy and detect moving objects early. Is.

JP 2000-20227 A JP 60-156132 A

  FIG. 10 is a schematic diagram illustrating a scanning method for specifying a touch position. In the touch panel, scanning for specifying a touch position is performed sequentially. In general, as shown in FIG. 10A, first, scanning in the X-axis direction is sequentially performed from the upper side to the lower side. When scanning in the X-axis direction is completed, scanning in the Y-axis direction is sequentially performed from the left side to the right side as shown in FIG. In such a sequential scan, it is necessary to scan the entire touch panel, and it takes a certain time to specify the touch position of the user.

  In recent years, the time required for one scan tends to increase as the screen becomes larger and finer. Further, in multi-touch input with a plurality of fingers, scanning is repeated a plurality of times, so that the scanning time becomes long, and there is a concern that response characteristics with respect to a touch operation are deteriorated. For example, an in-vehicle device may use a pinch operation using multi-touch input when scaling a road map. In such a scene, since it takes time to read map data corresponding to the scale from the memory, it is particularly desired to shorten the scan time.

  The present invention solves such a conventional problem, and provides an input device that can shorten the scan time by predicting the second and subsequent touch positions and controlling the detection range. For the purpose.

An input device according to the present invention includes a detection unit that detects an approach or contact of an object on a touch panel, a first contact detection unit that detects coordinates of a first contact with the touch panel, and the first contact. Determining means for determining the contact mode, a predicting unit for predicting a second contact range based on the coordinates detected by the first contact detecting unit and the contact mode determined by the determining unit, and the predicting unit Control means for controlling the detection range of the detection means based on the prediction result.

  Preferably, the predicting unit predicts the second contact range according to the type of the hand or finger determined by the determining unit. The prediction means predicts a second contact range according to the type of the hand or finger determined by the determination means and the positional relationship of the first contact on the touch panel.

  Preferably, the control means limits a detection range by the detection means to the range of the second contact. The prediction means predicts a second contact range based on a contact range prediction table prepared in advance, and the contact range prediction table defines a relationship between the type of finger and hand and the second contact range. .

  Preferably, the determination unit determines a contact mode of the object based on a change in capacitance of the detection unit. The discrimination means discriminates the contact mode of the object based on the output of the proximity sensor that detects the approach of the object provided around the touch panel. The discrimination means discriminates the right hand or the left hand based on the output of the proximity sensor. The discriminating unit discriminates a contact mode based on image information captured by an imaging camera that captures the periphery of the touch panel.

  An input detection method for a multi-touch operation according to the present invention is an input detection method for a multi-touch operation in an input device including a touch panel, and a detection step for detecting an approach or contact of an object on the touch panel; Discrimination based on the first contact detection step for detecting the coordinates of the first contact, the discrimination step for discriminating the contact mode of the first contact, the coordinates detected by the first contact detection step, and the discrimination step A prediction step of predicting a range of the second contact based on the contact mode performed, and a control step of controlling the detection range of the detection step based on the prediction result of the prediction step.

  A multi-touch operation detection program according to the present invention is a multi-touch operation input detection program executed by an input device including a touch panel, the detection step of detecting the approach or contact of an object on the touch panel, and the touch panel A first contact detecting step for detecting the coordinates of the first contact, a determining step for determining the contact mode of the first contact, the coordinates detected by the first contact detecting step, and the determining step. A prediction step of predicting a second contact range based on the determined contact mode; and a control step of controlling the detection range of the detection step based on a prediction result of the prediction step.

  According to the present invention, since the contact range related to the touch operation after the second point is predicted and the detection range for detecting the touch position is limited, the scan time can be shortened, and the response characteristic to the touch operation is improved. To do.

It is a block diagram which shows the structure of the vehicle-mounted apparatus which concerns on the Example of this invention. It is a figure explaining the detection method of the touch input by a touch panel. It is the schematic which shows arrangement | positioning of the proximity sensor which concerns on a present Example. It is the schematic which shows arrangement | positioning of the imaging camera which concerns on a present Example. It is a block diagram which shows the functional structure of the input detection program with respect to multi-touch operation. It is a contact range prediction table used for predicting the second contact range. It is a flowchart explaining the input detection operation | movement with respect to multi-touch operation by a present Example. It is a figure explaining the prediction method of the contact range of the 2nd point which concerns on a present Example. It is a figure which illustrates the other prediction method which estimates the contact range of the 2nd point. It is the schematic explaining the scanning method which pinpoints a touch position.

  Embodiments of the present invention will be described in detail with reference to the drawings. The input device of the present invention includes a touch panel as an input interface, and may be, for example, a notebook PC, a tablet PC, a personal computer, an in-vehicle device, a game machine, a facility guide display device, or other display device capable of touch input. it can. As a preferable mode, in the case of touch input, it corresponds to multi-touch input that can be touch-operated with a plurality of fingers, and has a function that can detect a touching mode at the time of touch operation or an approach mode immediately before the touch operation. For example, after the touch input of the first point is performed, the contact range related to the touch position of the second point can be predicted based on the contact mode.

  Thereby, when the input device detects the second touch position, the input range can be limited, and the response characteristic to the touch operation is improved. For example, in the case of an in-vehicle device, the scan time can be shortened with respect to a pinch operation using a road map for enlargement, and the in-vehicle device can immediately start reading a road map according to the scale.

  Next, an in-vehicle device will be described with reference to the drawings as an example of an input device according to an embodiment of the present invention. FIG. 1 is a block diagram illustrating a configuration example of an in-vehicle device. The in-vehicle device 10 displays a road map around the vehicle position and guides a guidance route to the destination, and audio data and video data obtained from a CD, DVD, Blu-ray disc, hard disk device, etc. Multimedia playback unit 40 for playback, input unit 60 for receiving instructions from the user such as a touch panel and switches, proximity sensor 80 capable of detecting the approach of an object around the touch panel, and an image of the periphery of the touch panel attached to the interior of the vehicle Imaging camera 100, a display control unit 120 that displays a display image including an input interface such as a navigation image and a media playback image on a display, and a voice output that outputs route guidance, road traffic information, and a warning to the destination by voice Unit 140 and programs and data A storage unit 160 for 憶 configured to include a control unit 180 for controlling each unit by executing a program.

  The input unit 60 includes a touch panel superimposed on a display as an input interface. The touch panel can detect, as XY coordinate data, a position where an object such as a finger contacts or a position away from the position, as is well known. In a preferred embodiment, a capacitive touch panel is used, and an object that touches and approaches can be detected according to a change in capacitance.

  FIG. 2 is a diagram for explaining a method for detecting a touch position on the touch panel and its contact state. For example, when the user performs a pinch operation, as shown in FIG. 2A, a hand is brought close to the touch panel 62 and an index finger is brought into contact therewith. The touch panel 62 can detect the capacitance generated between the touching index finger and another finger approaching. In addition, the control unit 180 creates a capacitance intensity distribution corresponding to each position on the panel based on the output from the touch panel.

  FIG. 2B is a distribution diagram schematically showing a capacitance intensity distribution corresponding to FIG. FIG. 2B shows an example in which the strength of the capacitance detected by the control unit 180 is expressed in three levels, and reflects the contact mode such as the shape and position of the hand or finger that touches or approaches. is there. The dark part in the figure represents the level with the strongest capacitance. For example, the area A indicates the position of the index finger that has touched the touch panel 62, and since a high capacitance is detected, the capacitance at the center is “high level”. On the other hand, the region B corresponds to the position of the thumb close to the touch panel 62, and the capacitance is “medium level”. Further, the region C corresponds to a portion where the middle finger is pinched with the little finger, and is slightly separated from the touch panel 62, so that the capacitance is “low level”. Thus, the capacitance intensity distribution is a function of the position of the hand or finger that touches or approaches, or the distance from the touch panel, and thus can be used for detection of the touch position and discrimination of the touch state. In addition, the control unit 180 monitors the time-series change in the intensity distribution of the capacitance immediately before the touch operation, that is, until the touch is input, within a certain period, so that the approach direction of the hand or finger, for example, Information such as whether a hand or a finger is approached from the left side of the display or from the right side can be acquired.

  FIG. 3 is a schematic diagram illustrating an example of the arrangement of proximity sensors. As shown in FIG. 3, the proximity sensors 80 are arranged on the left and right sides of the display 122, and indicate the approach direction in which an object such as a hand or a finger approaches, such as the driver's seat side, the passenger seat side, or the upper side or the lower side. Can be detected. For example, an infrared sensor can be used as the proximity sensor 80, and the infrared sensor includes a light emitting element that emits infrared light and a light receiving sensor that receives reflected light of the irradiated object. The detection result of the proximity sensor 80 is provided to the control unit 180. Information on the approaching direction of the object detected by the proximity sensor 80 is not essential, but can be used to determine the contact mode as needed, such as when the touch mode is not sufficiently detected or determined by the touch panel.

  FIG. 4 is a schematic diagram showing the arrangement of the imaging cameras. The imaging camera 100 is for imaging a hand or a finger approaching the display 122, and is attached to a predetermined position in the vehicle. The imaging camera 100 is installed in the vicinity of a room mirror or room light in a vehicle, for example, and can capture a hand or finger approaching the display 122. In addition to this, the imaging camera 100 may be integrally attached to the outer edge portion of the display 122 or may be attached to a dashboard, a seat, or a dashboard provided with instruments. The imaging camera 100 is not limited to this, and may be another 3D sensing device that can specify the shape and position of a finger three-dimensionally. Imaging data captured by the imaging camera 100 is provided to the control unit 180. Information on the shape and position of the finger acquired by the imaging camera 100 is not essential, but should be used for determining the contact mode as necessary, such as when the detection or determination of the contact mode by the touch panel or proximity sensor is not sufficient. Can do.

  The display control unit 120 can display a display image including an input interface such as a navigation image and a media reproduction image on the display. For example, when the navigation function is executed, a road map indicating a guidance route or the like is displayed, and a navigation screen that can accept user instructions such as enlargement / reduction and movement by a touch operation is displayed. In addition to this, the display control unit 120 displays a plurality of operation buttons that can be input by the user on the menu screen and various setting screens.

  In addition to programs essential for system operation, the storage unit 160 can store road map data necessary for navigation, music data necessary for media reproduction, and the like. The storage unit 160 also stores pattern information for determining the finger contact mode based on the intensity distribution of capacitance.

  In a preferred embodiment, the control unit 180 includes a microcontroller or a microprocessor that includes a ROM, a RAM, and the like, and the ROM or the RAM can store various programs for controlling the operation of each unit of the in-vehicle device. . Further, the present embodiment includes an input detection program for shortening the scan time for multi-touch input. The input detection program predicts the contact range of the second point according to the contact mode of the contact of the first point, and limits the detection range for detecting the contact of the second point.

  Next, FIG. 5 shows a functional configuration of the input detection program for reducing the scan time for the multi-touch operation according to the present embodiment. The input detection program 200 for the multi-touch operation includes a screen display unit 202 that displays a user input screen corresponding to the multi-touch input on the display, a contact detection unit 204 that detects a touch on the touch panel, and a contact detection unit 204. When contact of the point is detected, the contact point determination unit 206 that determines the contact state of the hand or finger that touches the display, and the second point based on the detected contact state of the first point A contact range prediction unit 208 that predicts a contact range related to the touch position, and a detection range control unit 210 that controls a detection range by the contact detection unit 204 based on the predicted contact range.

  The screen display unit 202 displays a user input screen for receiving a user operation instruction on a navigation screen, a multimedia playback screen, or the like. For example, each function is associated with the user input screen, and an operation button for selecting a destination is displayed, or a road map that can be enlarged / reduced is displayed.

  The contact detection unit 204 detects a touch position of a touched finger or the like based on a signal output from the touch panel. The contact detection unit 204 usually scans the entire touch panel as a detection range, and detects XY coordinate data of the touch position touched by the finger.

  When there is a first contact, the contact state determination unit 206 determines the contact state of the hand or finger approaching the display. When the first contact is detected by the contact detection unit 204, the contact mode determination unit 206 creates a distribution diagram showing the capacitance intensity distribution as shown in FIG. 2B based on the output of the touch panel. Then, by collating with the pattern information stored in the storage unit 160, it is possible to specify which finger is the contact of the right hand or the left hand. The contact state determination unit 206 can also use the distribution map immediately before the first point of contact as necessary, and can specify the approach direction of the hand or finger from the temporal change of the distribution map. . In addition to this, the contact mode determination unit 206 can use information obtained by the proximity sensor 80, the imaging camera 100, or the like as an auxiliary, and can more accurately determine the contact mode due to the first contact. .

  The contact range predicting unit 208 predicts a contact range where the second contact is made based on the contact mode determined by the contact mode determining unit 206. Preferably, the contact range prediction unit 208 follows the contact range prediction table stored in the storage unit 160 and the like, based on the type of the first hand or finger touched and the coordinates of the touch position, the second touch position Predict the contact range related to

  FIG. 6 is a contact range prediction table used for predicting the contact range of the second point. The contact range prediction table is based on the first contact range based on the first touch position and the first touch position on the touch panel for the type of hand or finger of the contact mode to be identified. A second contact range is defined.

  For example, when the first touch is performed with the “index finger” of the “right hand”, according to the contact range prediction table, the “left side” of the touch position becomes the first contact range. If the first touch position is “right side” on the touch panel, the “left side” of the touch position becomes the second contact range. The contact range prediction unit 208 can predict an area where the first contact range and the second contact range specified by the contact range prediction table overlap as the second contact range.

  The detection range control unit 210 can control the detection range in which scanning in the X and Y directions is performed by the contact detection unit 204 based on the prediction result by the contact range prediction unit 208. For example, the detection range control unit 210 can reduce the scan time required to detect the second touch position by limiting the detection range to the predicted contact range.

  FIG. 7 is a flowchart for explaining the operation of the input detection program for the multi-touch operation of this embodiment. The input detection program 200 is activated when the screen display unit 202 displays a user input screen corresponding to multi-touch input. First, the screen display unit 202 displays a user input screen corresponding to multi-touch input such as a navigation screen showing a road map (S100). In order to enlarge the road map, the user touches the touch panel with a finger to perform a pinch-in operation. The contact detection unit 204 scans the entire touch panel to detect the touched user's finger, and detects the first touch position as XY coordinate data (S102). Then, coordinates (X1, Y1) indicating the first touch position are output to the contact mode determination unit 206, the contact mode determination unit 208, and the like.

  When the first contact is made, the contact mode determination unit 206 is based on the output of the touch panel, and as shown in FIG. 2B, a distribution diagram showing the intensity distribution of the capacitance corresponding to each position of the touch panel. Is created (S104). When the distribution map is created, the contact mode determination unit 206 reads pattern information for determining the contact mode from the storage unit 160, performs pattern matching with the generated distribution map, and touches the first point. The type of input hand or finger is discriminated (S106). Thus, the finger related to the first contact is specified.

  When the first touch position is specified by the contact detection unit 204 and the type of the finger is specified by the contact mode determination unit 206, the contact range prediction unit 208 determines that the 2nd touch position is 2 in accordance with the contact range prediction table shown in FIG. The contact range of the point is predicted (S108).

  FIG. 8 is a diagram illustrating a method for predicting the contact range of the second point. As shown in FIG. 8, on the touch panel 62, when the first point is touched with the index finger of the right hand, the contact range predicting means 208 follows the contact range prediction table and the coordinates of the first touch position according to the type of the finger. A certain range on the “left side” of (X1, Y1) is defined as a first contact area E1. On the other hand, since the first point is on the “upper side” on the touch panel, the contact range predicting unit 208 determines a certain range of “lower side” of the touch position (X1, Y1) of the first point as the second contact area E2. And Then, the contact range predicting unit 208 predicts an area where the first contact area E1 and the second contact area E2 overlap as the second contact range E (indicated by hatching in the figure).

  Next, the detection range control means 210 restricts the detection range for detecting the second touch position based on the predicted contact range E (S110), and the contact detection means 204 is only within the restricted detection range. The second touch position is detected by scanning in the X and Y directions (S112). As described above, when a touch operation by contact of two points is detected, a process of enlarging the road map is executed according to the pinch operation (S114).

  According to the present embodiment, when the second touch position is detected with respect to the multi-touch operation, the detection range can be limited, and response characteristics to the touch operation can be improved. Thereby, the in-vehicle device can shorten the scan time for the pinch operation for enlarging / reducing the road map, and can immediately read the road map corresponding to the scale.

  In the above embodiment, the contact range predicting means predicts an area where the first contact area (E1 in FIG. 8) and the second contact area (E2 in FIG. 8) overlap as the contact range. One or both of the contact area and the second contact area may be predicted as the contact range. The shape and size of the first contact area and the second contact area can be set as appropriate. Further, the detection range control unit 208 corresponds to the pinch operation, and as shown in FIG. 9, an ellipse having a long axis as a straight line connecting the first touch position and the predicted center point of the contact range E is provided. The detection range E ′ may be limited to the shape.

  In the above-described embodiment, the contact state determination unit 206 determines the contact state of the finger or the hand according to the capacitance distribution diagram of the touch panel, but in addition to this, the contact state is determined using the detection result of the proximity sensor 80. It is also possible to determine. As shown in FIG. 3, when the proximity sensor 80 is installed around the display 122, it can be determined whether the finger or the hand approaches from the left or right direction of the display 122. For example, when a finger or hand is approached from the left side of the display 122, it is determined that the right hand is approached. Conversely, when a finger or hand is approached from the right side of the display, it is determined that the left hand is approached. When it is difficult to discriminate between the left and right hands from the touch panel alone, the discrimination accuracy can be increased by using the discrimination result based on the proximity sensor.

  Further, it is possible to determine the contact mode using the imaging result of the imaging camera 100. Fingers and hands are extracted by performing image processing on the imaging data of the imaging camera 100, and using a known pattern matching method or the like, discrimination of the direction in which the fingers or hands approach, discrimination of right and left hands, discrimination of thumbs and index fingers It can be performed. The imaging result of the imaging camera 100 can be used in combination with the detection result of the touch panel and / or the proximity sensor, or the contact mode may be determined only from the imaging result of the imaging camera 100.

  Although the preferred embodiments of the present invention have been described in detail, the present invention is not limited to specific embodiments, and various modifications and changes can be made within the scope of the gist of the invention described in the claims. Is possible.

10: On-vehicle device 20: Navigation unit 40: Multimedia playback unit 60: Input unit 80: Proximity sensor 100: Imaging camera 120: Display control unit 140: Audio output unit 160: Storage unit 180: Control unit 200: Input detection program 202 : Screen display means 204: Contact detection means 206: Contact mode determination means 208: Contact range prediction means 210: Detection range control means

Claims (10)

An input device for an in-vehicle device,
Detecting means for detecting an approach or contact of an object on a touch panel of a display device capable of touch input ;
First contact detection means for detecting coordinates of the first contact with the touch panel;
Determining means for determining a contact mode of the first contact;
Predicting means for predicting a second contact range based on the coordinates detected by the first contact detecting means and the contact mode determined by the determining means;
Proximity sensors provided on the left and right sides of the driver's seat and passenger's seat side of the display device,
Control means for controlling the detection range of the detection means based on the prediction result of the prediction means,
Said predicting means, left hand or right hand, and the type of the thumb or other than the thumb finger, and the reference to the first contact range that originates the coordinates of the contact of the first point, the coordinates of the contact of the first point With reference to a contact range prediction table that predetermines the relationship with the second contact range, the contact mode of the first contact determined by the determination unit and the first detected by the first contact detection unit identifying a second range of the contact based on the coordinates of the contact,
The input unit is configured to determine a right hand or a left hand by determining a direction approaching from a driver seat side or a passenger seat side based on an output of the proximity sensor . The input device according to claim 1, wherein the control unit limits a detection range by the detection unit to the range of the second contact. The input device according to claim 1, wherein the determination unit determines a contact mode of an object based on a change in capacitance of the detection unit. It said discriminating means discriminates the contact mode based on the image information captured by the imaging camera for imaging the surrounding panel input device according to 3 any one claims 1. An input detection method for multi-touch operation in an input device for an in- vehicle device provided with a touch panel of a display device capable of touch input ,
A detection step for detecting an approach or contact of an object on the touch panel;
A first contact detecting step of detecting coordinates of the first contact with the touch panel;
A determining step of determining a contact mode of the first contact;
A prediction step of predicting a second contact range based on the coordinates detected by the first contact detection step and the contact mode determined by the determination step;
A control step for controlling a detection range of the detection step based on a prediction result of the prediction step;
The predicting step includes a first contact range based on a left hand or right hand, a finger type other than the thumb or thumb, and a first contact coordinate, and a first contact coordinate. A contact range prediction table that predetermines a relationship with the second contact range, the first contact state determined by the determination step and the first contact detection step detected by the first contact detection step. Identify a second contact area based on the coordinates of the contact ;
The determining step determines a right hand or a left hand by determining a direction approaching from the driver seat side or the passenger seat side based on outputs of proximity sensors provided on both the left and right sides of the driver seat side and the passenger seat side of the display device. to, input method of detecting a multi-touch operation. The multi-touch operation input detection method according to claim 5 , wherein the control step limits a detection range of the detection step to the range of the second contact. The multi-touch operation input detection method according to claim 5 , wherein the determination step determines a contact mode of an object based on a change in capacitance in the detection step. An input detection program for multi-touch operation executed by an input device for an in- vehicle device having a touch panel of a display device capable of touch input ,
A detection step for detecting an approach or contact of an object on the touch panel;
A first contact detecting step of detecting coordinates of the first contact with the touch panel;
A determining step of determining a contact mode of the first contact;
A prediction step of predicting a second contact range based on the coordinates detected by the first contact detection step and the contact mode determined by the determination step;
A control step for controlling a detection range of the detection step based on a prediction result of the prediction step;
The predicting step includes a first contact range based on a left hand or right hand, a finger type other than the thumb or thumb, and a first contact coordinate, and a first contact coordinate. A contact range prediction table that predetermines a relationship with the second contact range, the first contact state determined by the determination step and the first contact detection step detected by the first contact detection step. Identify a second contact area based on the coordinates of the contact ;
The determining step determines a right hand or a left hand by determining a direction approaching from the driver seat side or the passenger seat side based on outputs of proximity sensors provided on both the left and right sides of the driver seat side and the passenger seat side of the display device. to, multi-touch operation input detection program. The multi-touch operation input detection program according to claim 8 , wherein the control step limits a detection range of the detection step to the range of the second contact. The multi-touch operation input detection program according to claim 8 , wherein the determination step determines a contact mode of an object based on a capacitance change in the detection step.

Images