TWI490735B - Multiple fingers touch sensing method using matching algorithm - Google Patents

Description

Multi-finger touch sensing method using comparison algorithm

The present invention relates to touch sensing, and more specifically to touch sensing methods using alignment algorithms, which are particularly suitable for sensing simultaneous multi-touch.

Fig. 1 is a schematic view showing a general touch sensing device 1. The touch sensing device 1 has a touch panel 10 including a sensing array having rails (eg, 12 and 13) staggered in rows and columns. The rail row 13 is detected via the touch sensor 14, and the rail row 12 is detected via the touch sensor 16. The touch sensors 14 and 16 send a sensing signal indicating the detection result to the touch controller 18 to determine the position and intensity of one or more touches that occur on the touch panel 10. Such touch sensing device 1 and its operation are well known in the art.

For example, when two fingers touch the positions 15 and 17 of the touch panel 10 at the same time, the waveforms of the sensing signals on the X-axis and the Y-axis are respectively shown below and to the left of FIG. 1 . As shown, in the dimension of the Y-axis, since the two positions 15 and 17 are substantially in the same column, the waveform has only one peak. In the X-axis dimension, since the two touch positions 15 and 17 are in different rows, the waveform has two peaks.

The position of the touch can be known by observing the waveform of the sensing signal. Ideally, when two touches occur in different rows or columns, the waveform will show a "crest-valley-peak" curve. That is, there are two maxima and one minima in the waveform curve appearing between the two. Figure 2 shows the various waveforms of the sensed signals when two touches occur. Chart (a) shows the ideal situation. As shown, the waveform has a curve that appears as "peak-valley-peak". The two peaks represent the coordinates of the two touch locations in this dimension, while the valleys represent the space between the two touch locations. However, in practice, the waveform may not be so perfect. Figure (b) shows the condition that the sensing signal is disturbed by noise. As visible. Because of the noise, the trough disappeared. In this case, it is difficult to distinguish between two touch positions. Graph (c) shows a situation in which the two touch positions are too close so that the two peaks overlap each other. In this case, it is also difficult to distinguish between two touch positions.

In the prior art, the position is calculated by the following center point formula for finding the touch position:

Wherein X _touch coordinate of the touch position, X [i] is the i th line: signal amplitude (e.g., the i-th row) coordinates, W [i] for the i-th line, i of a touch region starting from The starting point ends at the end of the touch area. This method is susceptible to noise interference, that is, the calculated position may drift due to noise.

Therefore, it is desirable to have a method for more effectively and accurately determining the touch position occurring on a touch panel or the like, even when the touch positions are close to each other, and are not disturbed by noise.

An object of the present invention is to provide a touch sensing method that can accurately and accurately detect a touch position of a touch sensor such as a touch panel.

According to one aspect of the invention, a touch sensing method for detecting a touch position of a touch sensor includes providing a reference touch waveform representing an ideal touch; capturing the sensing from the touch sensor a signal; by moving the reference touch waveform to compare the waveform with the sensed signal to find the best result; and corresponding to the best result, confirming the touch position from the reference touch waveform .

According to another aspect of the present invention, a touch sensing method for detecting a touch position of a touch sensor includes providing a reference touch waveform representing an ideal touch; capturing a sense from the touch sensor Measuring a signal; synthesizing a plurality of reference touch waveform curves to form an ideal combined waveform curve; comparing the ideal combined waveform curve with the sense by changing a center point of the reference touch waveform curves constituting the ideal combined waveform curve The signal is measured to find the best result; and the best result is obtained, and the touch position is confirmed from the ideal combined waveform.

The comparison can be performed by calculating the reference touch waveform curve or the correlation between the ideal combined waveform curve composed of a plurality of reference touch waveform curves and the sensed signal. Alternatively, the comparison can be performed by calculating the reference touch waveform curve or the difference between the ideal combined waveform curve composed of a plurality of reference touch waveform curves and the sensed signal.

In order to make the above-mentioned contents of the present invention more comprehensible, the preferred embodiments are described below, and the detailed description is as follows:

Figure 3 shows the ideal waveform of two touches and the waveform of the sensing signal with two touches. In this example, two touches occur on different coordinates of the X-axis dimension. For simplicity and clarity, the Y-axis dimension will not be discussed. However, it is well known to those skilled in the art that the Y-axis dimension is similar to the X-axis dimension. In Figure 3, waveform 32 represents the first touch and waveform 34 represents the second touch. These touches can be done with fingers or other objects. Both touches occur simultaneously on different X-axis coordinates. As shown, the center point of the first waveform 32 is at position u, and the center point of the second waveform 34 is at position v. Waveform curves 32 and 34 are ideal waveform curves. In practice, a sense signal having a waveform 38 is detected, which is ideally the sum of the first and second touch waveforms 32 and 34.

In the present invention, a reference touch waveform is stored in advance. The reference touch waveform curve is an ideal touch waveform curve, such as a first touch waveform curve 32 or a second touch waveform curve 34. For example, when u=3, it indicates that the center point of the reference touch waveform is at the third line. For two touch conditions, the reference touch waveform is copied and configured at different spaced distances for comparison with the sense signal waveform 38. By doing so, the center points of the two touches can be recognized even when there is noise or when two touches occur in close proximity.

The closest is obtained by gradually moving the two reference touch waveforms 32 and 34, i.e., changing the positions of u and v, and comparing the reference touch waveforms 32, 34 with the sense signal waveform 38. the result of. The set (u, v) that results in the closest result represents the center point position of the two touches.

In order to determine the range of movement of u and the range of movement of v, a threshold value is set as shown in FIG. The level of the threshold is set to divide the touch state from the non-touch state. The threshold intersects the sensing signal waveform at two points, and the X-axis coordinates of the two points are Wa and Wb, respectively. The point Wa is the starting point of the search for the touch position, and the point Wb is the end point of the search for the touch position. That is, the effective range of the search touch position is defined as [Wa, Wb]. In this figure, the point Wm is the midpoint between Wa and Wb. That is, the coordinates of Wm are Wm=(Wa+Wb)/2.

In general, the position of u can be found by searching in the range from Wa to Wm, and the position of v can be found by searching in the range from Wm to Wb. However, in order to ensure that the search results are optimal, the search range of u and v can be slightly expanded. For example, in order to find the position of u, the search range can be expanded to [Wa, Wm+1]; in order to find the position of v, the search range can be expanded to [Wm-1, Wb]. As an example, if the starting point Wa and the ending point Wb determined by the threshold are 9 and 16, respectively, that is, the search range is [9, 16], the search range can be expanded by 2 at the start and end. Therefore, the adjustment of the thick search range becomes [9-2, 16+2] = [7, 18].

Figure 4 shows the different relative positions of the center points of the two reference touch waveforms 32, 34. The ideal combined waveform curve IP( i , j ) consisting of two reference touch waveforms can be expressed as:

IP( i , j )=Su( i )⊕Sv( j ) (2)

Where Su( i ) is the touch waveform curve 32 whose center point is at position i (i.e., i = u); Sv( j ) is the touch waveform curve 34 whose center point is at position j (i.e., j = v). The variables i and j are within the above search range.

In Figure 4, six ideal combined waveforms IP( i , j ) are shown, including IP(Wa, Wm-1)=Su(Wa)⊕Sv(Wm-1), IP(Wa,Wm)= Su(Wa)⊕Sv(Wm), IP(Wa+1,Wm-1)=Su(Wa+1)⊕Sv(Wm-1), IP(Wa+1,Wm)=Su(Wa+1) ⊕Sv(Wm), IP(Wa+2, Wm-1)=Su(Wa+2)⊕Sv(Wm-1), and IP(Wa+2,Wm)=Su(Wa+2)⊕Sv( Wm). Each ideal combination waveform curve IP( i , j ) is compared with the sensing signal waveform curve W(x1, x2) to obtain the best result, where x1 and x2 are the center point positions of the two actual touches.

The ratio of the various ideal combination waveforms IP( i , j ) to the sense signal waveform W(x1, x2) can be implemented using any suitable algorithm, such as correlation operations, SAD (absolute difference sum) or MSE (both variance). When a correlation operation is used, the sum of W(x1, x2)*(Su( i )⊕Sv( j )) is calculated, and the maximum result is the best result. When SAD or MSE is used, the difference between the various ideal combination waveforms IP( i , j ) and the sensing signal waveform W(x1, x2) is calculated. Therefore, the minimum result is the best result.

In the present embodiment, the SAD algorithm is employed. The SAD algorithm can be expressed as:

Where N is the number of points in the SAD map (ie, the number of sets of ( i , j )), that is, the number of combinations of positions u and v.

The values of u and v are gradually changed to synthesize various ideal combined waveforms IP( i , j ). The SAD results between the respective IP( i , j ) and the sensed signal waveform W(x1, x2) are calculated to form a SAD map pair containing the respective SAD results for the different sets (u, v). In this SAD mapping, the minimum result is the best result, and a set (u, v) corresponding to this minimum result represents the estimated touch position.

As mentioned above, other algorithms such as MSE and correlation operations can be employed. The MSE algorithm can be expressed as follows:

The correlation operation can be expressed as follows:

The formula (2) representing the ideal combined waveform curve IP can be called a reference touch waveform curve synthesis formula, and can be further expressed as:

IP( i , j )=Su( i )⊕Sv( j )=IP( i,j )=Su( i )+Sv( j )+ k *Su( i )*Sv( j ) (6)

Where k represents the degree of coupling between the two reference touch waveforms 32 and 34, that is, the degree of overlap of the two reference touch waveforms 32 and 34. When k =0, it means that the two reference touch waveforms are independent and can be directly added together.

Figure 5 is a SAD mapping of various combinations of u and v for two reference touch waveforms 32 and 34. In this SAD mapping, a preliminary point 50 (i.e., a specific set of (u, v)) is selected to perform a search to find the minimum result 55. As shown, the search range of u is determined to be [Wa, Wm+1], and the search range of v is determined to be [Wm-1, Wb]. If the initial point 50 is selected appropriately, the minimum result 55 can be quickly obtained.

In order to find the minimum result from the SAD mapping, various known methods can be utilized. In addition to the full search method, some quick search methods are also applicable, such as the steepest gradient search method, the three-step search method, and the diamond search method.

Figure 6 is a schematic diagram showing the steepest gradient search method used in the present invention. As is known, after selecting a preliminary point 60 (i.e., a set of specific (u, v)), the system checks the eight nearest points around the initial point 60 in the SAD map and the initial point 60 to Find the relative minimum results from these nine points. The search then proceeds further towards the direction of the relatively least result, examining the three points near the relatively least result. The possible conditions are shown at the bottom of the figure.

In order to search for the best point (that is, the best result), in addition to moving one position of u or v each time, you can also move the position by 1/2 unit, such as 1/2 pixel or even 1/4 pixel. . If the position of u or v is moved by 1/2 pixel each time, the resolution of 1/2 pixel can be achieved. If the position of u or v moves by 1/4 pixel each time, the resolution of 1/4 pixel can be achieved.

Figure 7 is a schematic diagram showing the three-step search method used in the present invention. After selecting a first point 70 (ie, a group of specific (u, v)), the first point 70 is examined in the upper, lower, left, right, upper left, lower left, upper right, and lower right directions at predetermined intervals. Eight points and the initial point 70 are used to determine a relatively good point. Then, the other eight points in the eight directions around the relatively preferred point and the relatively preferred points are examined to find the second relatively preferred point. So on and so forth. Finally, you can decide the final best point.

Figure 8 shows four diagrams illustrating an embodiment of a method in accordance with the present invention. The graph (a) shows the waveform of a sensing signal, with the vertical axis being the signal amplitude and the horizontal axis being the position. A threshold is used as described above to define the search range. In this example, we get Wa=9 and Wb=16. To ensure that the best alignment can be done, the range is extended by 2 at the beginning. That is, the adjusted search range is [Wa-2, Wb+2] = [9-2, 16+2] = [7, 18]. Graph (b) is a partial enlargement of graph (a). Figure (c) is a SAD mapping of the SAD algorithm in the present example compared to the sense signal waveform and the two reference touch waveform curves at different positions. By searching for the SAD map using the full search method or the fast search method, the minimum result appears at (u=11, v=14). That is, coordinates 11 and 14 are estimated positions at which two fingers touch. Graph (d) shows two reference touch waveform curves 82, 84, each of which is a Gaussian waveform. Waveform curve 86 represents an ideal combined waveform curve IP (11, 14) obtained by synthesizing two reference touch waveform curves Su(11) 82 and Sv(14) 84. As can be seen, the sensed signal waveform in graph (b) is very similar to the ideal combined waveform curve IP(11, 14) 86.

It should be noted that comparison methods such as correlation calculation, SAD, MSE, etc., are inherently immune to noise. In the example shown in Fig. 8, the signal to noise ratio (SNR) is 10. However, as can be seen, the SAD mapping has the characteristics of noise immunity. The location where the smallest result can be obtained without significant impact by noise.

Figure 9 shows four diagrams illustrating another embodiment of the method in accordance with the present invention. These four charts correspond to the chart of Fig. 8, so the detailed description is omitted here and avoids redundancy. In this example, the two fingers touched each other too close to each other so that the waveforms merged together. As shown in the table (d), the two positions of the minimum result obtained from the SAD mapping are very close, so that the two reference touch waveforms 92 and 94 are very close. As a result, the ideal combined waveform curve 96 obtained by synthesizing the two reference touch waveform curves 92 and 94 does not have the shape of "peak-valley-peak". However, by using the method of the present invention, the correct touch position can still be obtained under such conditions.

The method of the present invention can be summarized as the flowchart shown in FIG. 10 when implemented by a full search. Figure 10 is a flow chart showing the method of the present invention for determining the position of two finger touches using a full search. In step S110, the boundary for performing the comparison operation is determined. As described, a starting point Wa and an ending point Wb can be obtained by setting a threshold for a sensing signal. In addition, a midpoint Wm=(Wa+Wb)/2 can be obtained.

In step S120, the boundary is slightly extended to set the search range. For example, the search range for finding the first finger touch position u can be set to [Wa, Wm+1], and the search range for finding the second finger touch position v can be set to [Wm-1, Wb].

In step S130, a sensing signal waveform curve W(x1, x2) is captured from a touch sensor such as a touch panel, where x1 and x2 indicate positions where two fingers touch. An ideal combined waveform IP( i , j ) is generated by synthesizing two reference touch waveforms Su( i ) and Sv( j ), where in this example Wa≦ i ≦Wm+1 and Wm-1≦ j ≦Wb. By changing the i and j values of the positions u and v, a plurality of ideal combined waveforms IP( i , j ) can be obtained. The paired maps MAP(u,v) (e.g., SAD maps) are obtained by comparing the sensed signal waveforms W(x1, x2) with the IP( i , j ) using any suitable algorithm. In step S140, the best result is searched for in the map MAP(u, v). The specific set of (u, v) corresponding to the best result represents the actual finger touch position.

Figure 11 is a flow chart showing the method of the present invention for determining the position of two finger touches using a quick search. In step S210, a boundary for performing the comparison operation is determined. As described, a starting point Wa and an ending point Wb can be obtained by setting a threshold for a sensing signal. In addition, a midpoint Wm=(Wa+Wb)/2 can be obtained. In step S220, a search initial point (ie, a specific set of (u, v)) is selected, and the search step is selected. If the search step is selected as a pixel, the resolution is one pixel. If the search step is selected as 1/4 pixel, the resolution is 1/4 pixel. In this example, (u = (Wa + Wm) / 2, v = (Wm + Wb) / 2)) is selected as the starting point, and the search step is 1. In step S230, the fast search method such as the steepest gradient search method and the three-step search method is used to map the ratio of the sensing signal waveform curve W(x1, x2) to the ideal combined waveform curve IP( i , j ). Look for the best results from this starting point.

The reference touch waveform can be obtained in various ways. For example, a touch waveform curve may be measured in advance, and the signal level values of the plurality of points of the measured waveform curve are stored in a memory of the product in the form of a lookup table, such as a read-only memory, a flash, or the like. Memory, etc. When starting to use, the signal level values of the multipoints can be read from the lookup table.

The reference touch waveform can be generated by a mathematical formula, such as a Gaussian distribution, which is expressed as:

Where S is the reference touch waveform, i is the position (for example, the ith line), u is the center point of the reference touch waveform S , and σ is the width of the ideal finger or other object. The larger the finger width, the larger the parameter σ.

Otherwise, the reference touch waveform can be obtained or adjusted by dynamically collecting statistics of actual finger touch waveforms.

The method of the present invention is particularly useful for determining the position determination of a multi-touch. However, the present invention is also applicable to determining the position of a single touch. Figure 12 shows the ideal waveform of a touch and the waveform of the sensing signal of a touch. In the figure, the reference touch waveform is represented by reference numeral 82, and the sensing signal waveform is indicated by reference numeral 88. As in the case of two touches, a threshold value of the signal amplitude is given to determine a search range. The two points at which the sensing signal waveform 88 intersects the threshold correspond to a starting point Wa and an ending point Wb. Therefore, the search range is set to [Wa, Wb]. As mentioned above, the search range can be expanded slightly. The moving reference touches the waveform curve Su( i )82, that is, changes the variable i . The sense signal waveform 88 is aligned with the moving reference touch waveform Su( i ) 82 to find the best match. The reference touch waveform Su( i ) 82 can move one pixel, 1/2 pixel or 1/4 pixel at a time. If the fast search method is employed, the midpoint Wm between the starting point Wa and the ending point Wb will generally be an appropriate search starting point.

It can be inferred that the present invention can also be applied to determining the position of three or more touches. In this case, three or more reference touch waveforms should be provided and the best answer should be found in a three-dimensional or higher dimensional mapping (eg, SAD mapping). If an appropriate search range is determined and an appropriate first point is selected, the touch position of the touch can be quickly found by using a quick search method such as the steepest gradient search method described above.

Figure 13 shows the separation and overlap of the two touches. The upper graph in the figure shows that the two touch waveform curves 92 and 94 are separated from each other at time T-n. The two touches move toward each other. At time T, the two touch waveforms 92 and 94 overlap each other, resulting in a combined waveform 98 as shown. As can be seen, the waveform 98 does not have the shape of "peak-valley-peak". Conversely, the waveform 98 is a Gaussian-like waveform. However, the signal width and signal height of waveform 98 is significantly greater than the single touch waveform 92 or 94. Therefore, even when the sensing signal waveform cannot clearly indicate two peaks, it can be judged that the touch event occurring in the specific touch region is a single touch by setting appropriate limits th1 and th2 for the signal width and height. Touch or two touches. When it is determined that there are two touch events, the above-described double touch sensing program is activated. A dual touch sensing procedure can also be initiated when it is known that two separate finger touches have occurred in the previous state.

Figure 14 is a flow chart showing the method for deciding whether to initiate a multi-touch test procedure. In step S310, a touch area is input. In this touch area, the sense signal indicates that at least one touch has occurred. In step S320, it is checked whether any of the following conditions are satisfied: (1) there are two separate finger touches in the previous state (for example, at time Tn); (2) the signal width of the sensing signal is greater than a signal width limit. Th1; and (3) the signal height of the sensing signal is greater than a signal height limit th2. When at least one of the above three conditions is satisfied, the above-described multi-touch detection program is started in step S330 to find the touch position. In step S340, the estimated touch position is output.

Figure 15 shows a method for sensing touches in different areas. The flow chart is displayed on the left side and the waveform chart corresponding to each step of the program is displayed on the right side. In step S410, a sensing signal is input. The waveform of the sensing signal is shown on the right side. In step S420, the touch area is confirmed. In this example, the sense signal indicates that there are two touch areas: touch area 1 and touch area 2. In step S430, the number of touches in each area is confirmed by the program shown in Fig. 14. For example, in this example, it is determined that one finger touches in the touch area 1, and two fingers touches in the touch area 2. In step S440, the touch sensing method of the present invention described above is implemented, and the touch region 1 is compared with the sensing signal waveform curve in the touch region 1 by using a single reference touch waveform curve, and two The reference touch waveform curve is compared with the sensing signal waveform curve in the touch area 2 to the touch area 2 to find the touch position. In step S450, the estimated center point of the touch position of the touch area 1 and the estimated center point of the touch position of the touch area 2 are output.

While the invention has been described above in terms of the preferred embodiments, the invention is not intended to limit the invention, and the invention may be practiced without departing from the spirit and scope of the invention. The scope of protection of the present invention is therefore defined by the scope of the appended claims.

1. . . Touch sensing device

10. . . Sensing panel

12. . . Rail

13. . . Rail

14. . . Touch sensor

15. . . Touch position

16. . . Touch sensor

17. . . Touch position

18. . . Touch controller

32. . . First waveform

34. . . Second waveform

38. . . Sense signal waveform

50. . . First point

55. . . Minimum result point

60. . . First point

70. . . First point

82. . . Reference touch waveform

84. . . Reference touch waveform

86. . . Ideal combination waveform

88. . . Sense signal waveform

92. . . Reference touch waveform

94. . . Reference touch waveform

96. . . Ideal combination waveform

98. . . Combined waveform

Figure 1 is a schematic view showing a general touch sensing device;

Figure 2 shows the various waveforms of the sensed signal when two touches occur;

Figure 3 shows the ideal waveform of two touches and the waveform of the sensing signal with two touches;

Figure 4 shows the different relative positions of the center points of the two reference touch waveforms;

Figure 5 is a diagram showing the SAD (absolute difference sum) mapping of various combinations of u and v of two reference touch waveform curves;

Figure 6 is a schematic view showing the steepest gradient search method used in the present invention;

Figure 7 is a schematic diagram showing the use of a three-step search method for the present invention;

Figure 8 shows four diagrams illustrating an embodiment of a method in accordance with the present invention;

Figure 9 shows four diagrams illustrating another embodiment of the method according to the invention;

Figure 10 is a flow chart showing the method of the present invention for determining the touch position of two fingers by using the full search;

Figure 11 is a flow chart showing the method of the present invention for determining the touch position of two fingers by using a quick search;

Figure 12 shows the ideal waveform of a touch and the waveform of the sensing signal of a touch;

Figure 13 shows the separation and overlap of two touches;

Figure 14 is a flow chart showing a method for deciding whether to initiate a multi-touch test; and

Figure 15 shows a method for sensing touches in different areas.

Claims (9)

A touch sensing method for detecting a multi-touch of a touch sensor, the method comprising: providing a reference touch waveform curve indicating an ideal touch; capturing from the touch sensor a sensing signal; synthesizing a plurality of the reference touch waveform curves to form an ideal combined waveform curve; comparing the ideal combined waveform by changing a center point position of the reference touch waveform curves constituting the ideal combined waveform curve The curve and the sensed signal are used to find the best result; and the touch position is identified from the ideal combined waveform curve corresponding to the best result. For example, the method of claim 1 further includes setting a search range, wherein the search range is defined by setting a signal amplitude threshold for the sensing signal, and when the change constitutes the ideal combination waveform When the reference touches the center point position of the waveform, the position of the center point of the reference touch waveform falls within the search range. The method of claim 2, wherein the comparing step comprises: selecting a search starting point within the search range, the initial point indicating a center point of the reference touch waveforms constituting the ideal combined waveform a specific position; and a position from a center point of the reference touch waveform is started at a state of the initial point, and the ideal combination waveform is compared with the sensing signal by using a fast searching method Out of the best results. The method of claim 3, wherein the fast search method is implemented by a steepest gradient searching method. For example, the method of claim 3, wherein the fast search method is a three-step search method (three-steps searching method) implementation. The method of claim 1, wherein the comparing step for finding the best result is to find the maximum by performing an auto-correlation of the ideal combined waveform and the sensing signal. The result is the best result. The method of claim 1, wherein the comparing step for finding the best result is to find the minimum result by calculating the difference between the ideal combined waveform and the sensed signal. result. For example, the method of claim 7 of the patent scope, wherein the difference is calculated by using the SAD (absolute difference sum) calculation. A touch sensing method for detecting a touch of a touch sensor, the method comprising: providing a reference touch waveform curve indicating an ideal touch; extracting a sense from the touch sensor Detecting a signal; confirming a touch area from the sensing signal, at least one touch occurs in the touch area; confirming how many touches occur in the touch area; and determining that there is only one touch in the touch area Comparing the reference waveform curve with the sensing signal to find an optimal result by moving the reference touch waveform curve, and synthesizing a plurality of the reference touches when determining that there are multiple touches in the touch region Touching the waveform curve to form an ideal combined waveform curve, and by comparing the center point positions of the reference touch waveform curves constituting the ideal combined waveform curve, comparing the ideal combined waveform curve with the sensing signal to find the most a good result; and identifying one or more touch locations from the reference touch waveform or the ideal combined waveform corresponding to the best result.