KR102601375B1 - Method and apparatus for detecting the convenience of touch depending on the position on the touch panel - Google Patents

Abstract

The present invention relates to a method and device for detecting the convenience of touch according to the location of a touch panel.
In the present invention, the x-axis value of the accelerometer measured for i seconds (i is an arbitrary positive real number) is used to infer whether the user operates the touch panel with the left hand, right hand, or both hands, and the obtained data set is A touch method and device that visually provides touch after clustering into three are disclosed.
According to the method and device for inferring smartphone user characteristics according to the present invention, it is possible to predict the characteristics of a user using a smartphone by complexly utilizing sensor values of the touch panel, gyroscope, and accelerometer.

Description

Method and device for detecting convenience of touch according to touch panel location {METHOD AND APPARATUS FOR DETECTING THE CONVENIENCE OF TOUCH DEPENDING ON THE POSITION ON THE TOUCH PANEL}

The present invention relates to a method and device for detecting the convenience of touch according to the location of the touch panel. More specifically, it uses an acceleration sensor and a gyroscope to determine the characteristics of the user's touch on the screen and provides a convenient touch using this. It relates to a method and device for detecting the convenience of touch according to the location of a touch panel.

The screen size of electronic devices is increasing day by day according to the needs of various users and services. As screen sizes increase, many users are having difficulty manipulating all areas of the screen with one hand. To alleviate these difficulties, some electronic device manufacturers provide a one-hand mode, but the horizontal and vertical ratios are fixed, and there is the inconvenience of having to manually adjust the appropriate screen size. In addition, most applications and services have inflexible designs that simply fill up a large screen and fixed button placement, which has limitations in not being able to reflect different user characteristics in the User Interface/Experience.

For this reason, in the case of electronic devices equipped with large screens, placement work is necessary, such as placing frequently used applications in areas that are easy to touch and placing less frequently used applications in areas that are difficult to touch. For application developers, it is also a good idea to program information that needs to be touched frequently to be placed in an area that is easy to touch. In order to provide a more convenient usage environment for electronic device users, in the case of electronic devices with large screen touch panels, when the user touches with one or two hands, the screen area that is easily touched and the screen area that is difficult to touch are accurately identified in each case. It is necessary to understand

Korean Patent Publication No. 10-2012-0064756 (published on June 20, 2012)

The present invention is intended to solve the above-mentioned need, and is a touch panel location that determines whether each area of the touch panel is an area that is easy to touch or is an inconvenient area when used with one hand or two hands and provides good visibility. The purpose is to provide a method and device for detecting the convenience of touch according to.

The object of the present invention is a method for detecting the convenience of touch according to the position of the touch panel in an electronic device equipped with a touch panel. The first step is to collect the touch position, the x-axis value of the accelerometer, and the y-axis value of the gyroscope for a certain period of time. Wow, - the data collected in the first step includes time, whether the touch panel was touched at that time, touch location, accelerometer x-axis value, and gyroscope y-axis value - for ±n ms based on each time the touch occurred (n is a positive integer) The acceleration x value obtained by subtracting the minimum value from the maximum value of the A second step of processing the data set containing the data, and whether the user operates the touch panel with the left hand, right hand, or both hands using the x-axis value of the accelerometer measured for i seconds (i is an arbitrary positive real number). A third step of inferring whether the data set obtained in the second step is clustered into three, and a fourth step of mapping the three clusters to the screen area of the touch panel and displaying them. This can be achieved by detecting the convenience of touch.

In the third step, if the average value of the If the average value is less than the negative value of the first threshold, it is inferred that the user is operating the touch panel with the right hand, and if the average value is greater than the negative value of (3) the first threshold and less than the first threshold, It is desirable to further include step 3-1 of reasoning using both hands, and clustering is preferably performed through unsupervised classification using the k-means clustering technique.

Another object of the present invention is an electronic device that detects the convenience of a touch according to the position of the touch panel, which consists of a touch panel and a main body that receives touch coordinates from the touch panel, and is provided with an accelerometer and a gyroscope, and A collection unit that collects the Axis values are included - the acceleration x value and the y of the gyroscope obtained by subtracting the minimum value from the maximum value of the A processing unit that processes the data set containing the gyroscope y value obtained by subtracting the minimum value from the maximum value of the axis value, and the user uses the x-axis value of the accelerometer measured for i seconds (i is an arbitrary positive real number). Includes an inference unit that infers whether the hand operating the touch panel is left, right, or both hands, clusters the data set obtained from the processing unit into three, and an expression unit that maps and displays the three clusters to the screen area of the touch panel. This can be achieved by an electronic device that detects the convenience of touch according to the location of the touch panel.

According to the method and device for detecting the convenience of touch according to the position of the touch panel according to the present invention, the characteristics of the user using the smartphone are determined by complexly utilizing the sensor values of the touch panel, gyroscope, and accelerometer. It became predictable.

Representative examples of user information that can be predicted include which hand the user is currently using the smartphone with (left hand, right hand, or both hands), and which areas of the screen the user can comfortably reach, reach, or cannot reach. . Through the method and device for inferring smartphone user characteristics according to the present invention, device manufacturers or application developers can provide customized services to each user by utilizing the user's specialized information.

Based on user-specific information obtained through the technology proposed in the present invention, device manufacturers can include user-customized functions in their devices. In addition, application developers utilize the information obtained through the technology proposed in the present invention to improve the UI (User Interface) and UE (User Experience) by specializing the design of the application and the location of buttons for each user, ultimately improving the user's quality of experience. can be improved.

The present invention does not require any prior information from the user and utilizes sensor values measured as the user uses the device, so it accurately identifies the user's actual behavioral characteristics without any bias. In addition, the technology is easy to apply because it does not require expensive sensors, and because it requires only a very small amount of calculation, it has little effect on the performance of the device to which the technology is applied, and is used in devices with small computing capabilities such as IoT (Internet of Things) devices. It can also be used in . Additionally, the present invention is not limited to smartphones and can be applied to numerous devices that utilize touch panels, so the scope of technological application is wide.

1 is a configuration diagram of a typical electronic device having a touch panel.
Figure 2 is a block diagram of a device that detects the convenience of touch according to the location of the touch panel as an embodiment of the present invention.
3 is an explanatory diagram illustrating an example collection unit according to the present invention.
4 is an explanatory diagram illustrating an example processing unit according to the present invention.
Figure 5 is an explanatory diagram showing the x-, y-, and z-axis configuration of an accelerometer and gyroscope of a typical electronic device.
6A and 6B are graphs representing accelerometer values and gyroscope values measured when a user continuously performs an action of touching different points on the screen.
Figure 7 is a graph showing accelerometer x-axis values measured over time when the user touches the touch panel with the left hand, right hand, or both hands.
FIGS. 8A, 8B, and 8C are graphs showing an example of clustering coordinate values obtained by processing data measured when a user touches a screen divided into a total of 40 areas with the right hand, left hand, or both hands.
Figure 9 is an explanatory diagram explaining 3-means clustering.
Figure 10 is an example screen of the results of an experiment in which analysis was performed after distributing 40 buttons on the screen of a smartphone and touching each button.

The terms used in the present invention are only used to describe specific embodiments and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

In addition, in this specification, “on or above” means located above or below the target portion, and does not necessarily mean located above the direction of gravity. Additionally, when a part of a region, plate, etc. is said to be “on or above” another part, this does not only mean that it is in contact with or at a distance “directly on or above” another part, but also that there is another part in between. Also includes cases where there are.

In addition, in this specification, when a component is referred to as "connected" or "connected" to another component, the component may be directly connected or directly connected to the other component, but specifically Unless there is a contrary description, it should be understood that it may be connected or connected through another component in the middle.

Additionally, in this specification, terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

1 is a configuration diagram of a typical electronic device having a touch panel. The electronic device consists of a main body 150 having a display unit (not shown) and a touch panel 170 provided on the upper part of the display. The main body 150 is equipped with a printed circuit board (not shown), and the printed circuit board includes a plurality of sensors including a gyro sensor 110 and an acceleration sensor 120, a memory element 140, and a computer's central processing unit (CPU). ) is provided with various circuit elements, including the corresponding control chip 130. When a user touches the touch panel, the corresponding touch coordinates are transmitted to the printed circuit board of the main body, and the control chip 130 uses these touch coordinates to control the display unit, etc.

Figure 2 is a block diagram of a device that detects the convenience of touch according to the location of the touch panel as an embodiment of the present invention. The component block shown in FIG. 2 can be implemented in software in at least one circuit element provided in the main body 150 of FIG. 1. This circuit element may be the control chip 130 or a display control-dedicated element (chip, not shown) that controls the display unit, and may be implemented in one or more circuit elements. The device for detecting the convenience of touch according to the position of the touch panel according to the present invention includes a collection unit that collects sensor data, a processing unit that processes the collected sensor data, an inference unit that infers user characteristics using the processed data, and an inference unit. It consists of an expression unit that visually expresses the characteristics of the display unit. The collection unit of the present invention receives touch position values from the touch panel and collects data output from the gyro sensor and accelerometer. Hereinafter, a method of inferring using a device that detects the convenience of touch according to the position of the touch panel according to the present invention will be described.

1. Collection Department

The collection unit continuously collects touch time, presence or absence of touch action, touched position value, and gyroscope and accelerometer measurement values (x, y, z axes). Figure 3 is an exemplary diagram showing an example of operation of the collection unit. As seen in Figure 3, the data input at each time is the touch time, whether the touch is displayed as false or true (if true, a touch occurred), the touch location, and the (x, y, z axes of the gyroscope and accelerometer). ) consists of measured values.

2. Processing department

The processing unit considers the sensor data measured in the adjacent time period (e.g. ±n ms) from the time the touch operation occurred among the series of data input from the collection unit as one event data set, and the x-axis of the accelerometer in each event data set Calculate the differences between the maximum and minimum values based on the y-axis value of the gyroscope. Figure 4 shows an example of the operation of the processing unit. In Figure 4, an event refers to a situation in which a touch occurs. For example, if a touch occurs at time t1, the Find the maximum and minimum values. A dataset for the t1 event is formed by subtracting the minimum value from the maximum value of the accelerometer x-axis value and the minimum value subtracted from the maximum value of the gyroscope y-axis value.

In general, the x, y, and z axis configurations of accelerometers and gyroscopes in electronic devices are shown in Figure 5. When operating an electronic device, the user generally holds the left or right side of the device depending on whether he or she is using the left or right hand, so the axes most affected by this are the x-axis value of the accelerometer and the y-axis of the gyroscope. (Roll) value (marked in red in Figure 5).

The graphs shown in FIGS. 6A and 6B are accelerometer and gyroscope values measured when the user continuously performs an action of touching different points on the screen. In the case of the accelerometer in FIG. 6A, the x-axis value has the largest variation, and in the case of the gyroscope in FIG. 6B, the y-axis value has the largest variation. In other words, the x-axis value of the accelerometer and the y-axis value of the gyroscope are most affected by the action of touching the screen, so the values of these two axes are used.

3. Inference section

The inference unit infers the user's characteristics based on the data processed by the processing unit. In the present invention, an example of predicting two characteristics is shown.

First, it predicts whether the user is currently using the device left-handed, right-handed, or both hands. When inferring the hand performing a touch action, only the x-axis value of the accelerometer is used. The graph presented in Figure 7 shows an example of an accelerometer x-axis measurement value when the user touches the touch panel with the left hand, right hand, or both hands. As shown in the graph of Figure 7, when the touch panel is touched with the left hand, the values are mainly distributed in the positive area, and when the screen is touched with the right hand, the values are mainly distributed in the negative area. On the other hand, when touching the screen with both hands, you can see that the values are distributed in both negative and positive areas with 0 as the periphery. Therefore, it is predicted with which hand the user is using the device based on the ratio of the area where the values are mainly distributed.

Inferences about which hand to use can be made in several ways. We will explain two methods.

(1) Based on time t1, the average value of the values collected during the previous i seconds is (a) left hand if the value is greater than x1, (b) right hand if the value is less than -x1, (c) greater than -x1 but less than x1. If it is a value, it is judged as using both hands (x1 is an arbitrary positive real number)

(2) When the values collected during the previous i seconds based on time t1 are divided into n seconds and the minimum/maximum values in each unit time area are obtained, (a) the absolute value of the sum of the maximum values is the sum of the minimum values Use your left hand if it is p times greater than the absolute value; (b) use your right hand if the absolute value of the sum of the minimum values is p times greater than the absolute value of the sum of the maximum values; (c) use both hands for the rest (p is an arbitrary quantity) mistake)

Second, when the user touches the screen, the Natural zone (an area where touch is natural, can also be expressed as an 'area where touch is comfortable'), and the Reach zone (an area where there is some difficulty in reaching, can also be expressed as an 'area where touch is possible'). can also be expressed), and predicts where the unreachable zone (an area that has great difficulty in reaching, can also be expressed as an 'untouchable area') is. Based on the processed data, coordinate values with the difference between the maximum and minimum values of the accelerometer as the x-axis and the difference between the maximum and minimum values of the gyroscope as the y-axis are configured in a two-dimensional domain and k-means clustering technique is used. By performing unsupervised classification through , it is divided into Natural, Reach, and Unreachable areas. The graphs presented in FIGS. 8A, 8B, and 8C show an example of clustering coordinate values obtained by processing the data measured when the user touches the screen with the right hand, left hand, or both hands by dividing it into a total of 40 areas. These are graphs. Based on these clustering results, the user's Natural, Reachable, and Unreachable areas are inferred.

In the graphs shown in FIGS. 8A, 8B, and 8C, the x-axis represents the difference between the difference value) is displayed.

K-means clustering is a technique of unsupervised classification, and the graphs shown in Figures 8a, 8b, and 8c use 3-means clustering. The Natural area is an area where touch is natural, so there is little shaking of the device when touching. That is why it consists of low values in Figures 8a, 8b, and 8c. That is, in the graphs shown in Figures 8a, 8b, and 8c, the cluster with lower x and y values is classified as Natural, the middle is Reach, and the end is Unreachable, with red dots being Natural, green dots being Reach, and blue dots. expressed as unreachable.

Simply put, 3-means clustering is a technique that randomly divides three areas and adjusts them little by little to minimize the sum of the distances between the center point of each area and the points included in the area. Figure 9 is an explanatory diagram explaining 3-means clustering.

4. Expression part

The expression section is an area that visualizes the results of the inference section for easy understanding. It allows device manufacturers and application developers to more easily identify user characteristics by visually showing how easily the user touched each location on the screen where a touch event occurred.

Figure 10 is an example of the results of an experiment in which 40 buttons were distributed on the smartphone screen and analysis was performed after touching each button. In Figure 10, the Natural area, Reach area, and Unreachable area are shown in blue, green, and yellow, respectively. Figure 10 (a) shows experimental results for the case of operating the touch panel using the right hand, (b) the case of operating the touch panel using the left hand, and (c) the case of operating the touch panel using both hands. This is an example of When collecting touch values, the location value of the touch is also included and collected. Therefore, even if it is not necessarily a button pressing method as in the embodiment described so far, the present invention can be applied by using the accelerometer and gyroscope values along with the position value for the touch action. Touch position values, accelerometer, and gyroscope values are not values that the user must intentionally collect, but are generally values that are continuously collected, so they do not require additional operation of the electronic device. In other words, in smartphones, the accelerometer and gyroscope are sensor values that are generally collected, and the position value of the touch is also a value that is collected each time there is a touch, so in order to apply the present invention, a specific module must be newly operated. No additional operation of the device is required.

When comparing the predicted values for the touch range for each user through the present invention and the direct responses by users, it was shown that user characteristics were predicted with high accuracy, as shown in Table 1. This can be said to show the feasibility and validity of the present invention.

Average match rate of each case Right hand Left hand Both hands 89.6% 91.2% 93.3%

Since smartphones are devices containing touch panels that are most widely used today, the present invention description shows descriptions and examples of the invention based on smartphones. However, the present invention is not limited to smartphones and includes numerous devices using touch panels. It should be understood that it can be applied to electronic devices.

Although preferred embodiments of the present invention have been described and illustrated above using specific terms, such terms are only for clearly describing the present invention, and the embodiments of the present invention and the described terms are in accordance with the technical spirit and scope of the following claims. It is self-evident that various changes and changes can be made without deviating from it. These modified embodiments should not be understood individually from the spirit and scope of the present invention, but should be regarded as falling within the scope of the claims of the present invention.

110: Gyro sensor
120: Acceleration sensor
130: Control chip
140: Memory element
150: body
170: Touch panel

Claims (8)

A method of detecting the convenience of touch according to the location of the touch panel in an electronic device equipped with a touch panel,
A first step of collecting the touch position, the x-axis value of the accelerometer, and the y-axis value of the gyroscope for a certain period of time;
- Data collected in the first step includes time, whether the touch panel was touched at that time, touch location, accelerometer x-axis value, and gyroscope y-axis value -
The maximum value of the acceleration x value and the y-axis value of the gyroscope obtained by subtracting the minimum value from the maximum value of the A second step of processing the data set containing the gyroscope y value obtained by subtracting the minimum value from the value,
Using the x-axis value of the accelerometer measured for i seconds (i is an arbitrary positive real number), it is inferred whether the user operates the touch panel with the left hand, right hand, or both hands, and the data obtained in the second step The third step of clustering three into three and
A method for detecting the convenience of touch according to the location of the touch panel, comprising a fourth step of mapping and displaying the three clusterings to the screen area of the touch panel.
According to paragraph 1,
In the third step, if the average value of the x-axis values of the accelerometer measured for i seconds is greater than the first threshold (1), it is inferred that the user is operating the touch panel with the left hand, and the average value is (2) If the negative value is less than the negative value of the first threshold, it is inferred that the user is operating the touch panel with the right hand, and the average value is (3) greater than the negative value of the first threshold but less than the first threshold. Characterized in that it further includes step 3-1 of reasoning using both hands.
- The first threshold is an arbitrary positive real number -
A method of detecting the convenience of touch according to the location of the touch panel.
According to paragraph 1,
In the third step, the x-axis values of the accelerometer measured for i seconds are divided by n seconds to obtain the minimum and maximum values in each unit time domain, and 1) the absolute value of the sum of the maximum values is p greater than the absolute value of the sum of the minimum values. Step 3-1 of inferring the use of the left hand if p times larger, 2) inferring the use of the right hand if the absolute value of the sum of the minimum values is p times larger than the absolute value of the sum of the maximum values, and 3) inferring the use of both hands in the remaining cases. Characterized by further comprising
- n and p are arbitrary positive real numbers -
A method of detecting the convenience of touch according to the location of the touch panel.
According to any one selected from claims 1 to 3,
The fourth step detects the convenience of touch according to the location of the touch panel, which is characterized by mapping and displaying the screen area of the touch panel into an area that is comfortable to touch, an area that is possible to touch, and an area that is not possible to touch, based on three clustering. How to.
In an electronic device that detects the convenience of touch according to the location of the touch panel, which consists of a touch panel and a main body that receives touch coordinates from the touch panel,
Equipped with an accelerometer and gyroscope,
a collection unit that collects the x-axis value of the accelerometer and the y-axis value of the gyroscope for a certain period of time;
- Data collected by the collection unit includes time, whether the touch panel was touched at the relevant time, accelerometer x-axis value, and gyroscope y-axis value -
The maximum value of the acceleration x value and the y-axis value of the gyroscope obtained by subtracting the minimum value from the maximum value of the A processing unit that processes a data set containing the gyroscope y value obtained by subtracting the minimum value from ,
Using the x-axis value of the accelerometer measured for i seconds (i is an arbitrary positive real number), it is inferred whether the user is operating the touch panel with the left hand, right hand, or both hands, and the data set obtained from the processing unit an inference unit that clusters into three and
An electronic device that senses the convenience of touch according to the location of the touch panel, comprising a representation unit that maps and displays the three clusterings to the screen area of the touch panel.
According to clause 5,
If the average value of the If the average value is less than the negative value of the threshold, it is inferred that the user is operating the touch panel with the right hand, and if the average value is more than the negative value of (3) the first threshold and less than the first threshold, both sides are assumed to be right. Characterized by reasoning using hands
- The first threshold is an arbitrary positive real number -
An electronic device that detects the convenience of touch depending on the location of the touch panel.
According to clause 5,
The inference unit divides the x-axis values of the accelerometer measured for i seconds into n-second units, obtains the minimum and maximum values in each unit time domain, and 1) the absolute value of the sum of the maximum values is p times greater than the absolute value of the sum of the minimum values. In this case, the use of the left hand is inferred, 2) if the absolute value of the sum of the minimum values is p times greater than the absolute value of the sum of the maximum values, the use of the right hand is inferred, and 3) in the remaining cases, the use of both hands is inferred.
- n and p are arbitrary positive real numbers -
An electronic device that detects the convenience of touch depending on the location of the touch panel.
According to any one selected from claims 5 to 7,
The expression unit detects the convenience of touch according to the location of the touch panel, characterized in that it maps and displays the screen area of the touch panel into an area that is comfortable to touch, an area that is possible to touch, and an area that is not possible to touch, based on the three clusterings. Electronics.