WO2023065732A1

WO2023065732A1 - Touch processing method and apparatus based on graphic display system, and electronic device and storage medium

Info

Publication number: WO2023065732A1
Application number: PCT/CN2022/105311
Authority: WO
Inventors: 沙斌
Original assignee: 北京字节跳动网络技术有限公司
Priority date: 2021-10-22
Filing date: 2022-07-12
Publication date: 2023-04-27
Also published as: CN113986107A

Abstract

Disclosed in the present disclosure are a touch processing method and apparatus based on a graphic display system, and an electronic device and a storage medium. The method comprises: in response to detecting a touch operation, generating a touch event, and sending the touch event to a thread of the current UI window; triggering the thread of the current UI window to cache the touch event, and generating a window update request; after a synchronization event is detected, in response to the window update request, acquiring information of a touch point in the touch event, and determining information of a smooth touch point on the basis of the information of the touch point; sending the information of the smooth touch point to a target layout control, and triggering the target layout control to determine, on the basis of the information of the smooth touch point, a dirty region to be subjected to graphic updating; and triggering a rendering thread to render the dirty region, so as to update an interface graphic. By means of the present disclosure, the phenomena of jitter and frame skipping of a graphic interface are avoided, thereby providing a better usage experience for a user.

Description

Touch processing method and device based on graphic display system, electronic equipment, storage medium

Cross References to Related Applications

This disclosure claims the priority of the Chinese patent application with the application number 202111231103.5 and the application name "touch processing method and device based on graphic display system, electronic equipment, and storage medium" submitted to the China Patent Office on October 22, 2021. Its entire content is incorporated herein by reference.

technical field

Embodiments of the present disclosure relate to an interface graphics update technology, and in particular to a touch processing method and device based on a graphics display system, electronic equipment, storage media, computer program products, and computer programs.

Background technique

In some operating systems, the rendering frame rate of the graphics display system does not match the frequency of touch reporting. Taking the Linux operating system as an example, the rendering frame rate of the graphics display system is usually 16ms/60hz. The reporting frequency of a touch device set for an electronic device is usually 10ms/90hz. Therefore, when the touch report event is synchronized to the display system through the input system, there will generally be a delay in the final refresh display on the screen, and, due to the inconsistency between the rendering frame rate and the frequency between the touch report point, it will often cause the jitter of the graphical interface and frame skipping. Specifically, the UI process undergoes multiple cross-line process transfers before receiving the Touch event. Due to the influence of threads and process scheduling, the event receiving time will fluctuate greatly, which is prone to jitter. Within one frame time, the UI process may receive multiple Touch events, and only process the latest report point update to the UI, which will cause point loss and cause frame skipping. This is a very poor user experience for users.

Contents of the invention

In view of this, the embodiments of the present disclosure provide a graphic display system-based touch processing method and device, electronic equipment, storage media, computer program products, and computer programs to at least solve the above technical problems existing in the prior art.

According to a first aspect of an embodiment of the present disclosure, a touch processing method based on a graphic display system is provided, including:

In response to detecting a touch operation, generate a touch event, and send the touch event to the thread of the current user interface (UI, User Interface) window;

triggering the thread of the current UI window to cache the touch event, and generate a window update request;

After detecting the synchronous event, in response to the window update request, obtain the information of the touch point in the cached touch event, and determine the information of the smooth touch point based on the information of the touch point;

Sending the information of the smooth touch point to the target layout control, triggering the target layout control to determine the dirty area to be graphics updated based on the information of the smooth touch point;

Trigger the rendering thread to render the dirty area, and update the interface graphics.

In one embodiment, the determining the information of the smooth touch point based on the information of the touch point includes:

In the touch event cached by the thread of the current UI window, search for a set number of touch points closest to the sampling time, and determine based on the time information of the found touch points and the sampling time The time and location information of the smooth touch point.

In one embodiment, the set number is two;

The determining the time and position information of the smooth touch point includes:

When it is determined that the sampling time is between the times of the two touch points, determine a first offset between the time of the first touch point and the time of the second touch point of the two touch points amount, and a second offset between the sampling time and the time of the second touch point, the first weight is determined based on the first offset and the second offset; based on the The time and position information of the smooth touch point is calculated by the time of the second touch point, the first weight and the first offset.

In one embodiment, the set number is two;

When it is determined that the sampling time is outside the time of the two touch points, determine a first offset between the time of the first touch point and the time of the second touch point of the two touch points amount, and a third offset between the time of the first touch point and the sampling time, the second weight is determined based on the first offset and the third offset; based on the first The time of the touch point, the second weight and the inverse of the first offset are used to calculate the time and position information of the smooth touch point.

In one embodiment, the method also includes:

Setting a scheduling policy for the touch event; wherein the scheduling policy includes: setting a scheduling priority for the touch event, the scheduling priority of the touch event is higher than that of other threads or processes; When scheduling the touch events, the time slice round-robin scheduling is replaced by priority scheduling.

In one embodiment, the method also includes:

When detecting that the application or UI window corresponding to the touch event is switched to the background, lower the priority of the touch event; or

When it is detected that the application or UI window corresponding to the touch event with low priority is switched from the background to the foreground, the priority of the touch event is restored.

According to the second aspect of the embodiments of the present disclosure, there is provided a touch processing device based on a graphics display system, including:

a first detection unit, configured to detect a touch operation;

a generating unit, configured to generate a touch event in response to detecting a touch operation;

A first transmission unit, configured to send the touch event to a thread of the current UI window;

A first trigger unit, configured to trigger the thread of the current UI window to cache the touch event, and generate a window update request;

a second detection unit, configured to detect a synchronization event;

A determining unit, configured to respond to the update window request, acquire information of touch points in the touch event, and determine information of smooth touch points based on the information of the touch points;

The second transmission unit is configured to send the information of the smooth touch point to the target layout control;

A second trigger unit, configured to trigger the target layout control to determine a dirty area to be updated based on the information of the smooth touch point;

The third triggering unit is configured to trigger the rendering thread to render the dirty area and update the interface graphics.

In one embodiment, the determining unit is further configured to:

In one embodiment, the set number is two;

The determining unit is also used for:

In one embodiment, the set number is two;

The determining unit is also used for:

In one embodiment, the device also includes:

A setting unit, configured to set a scheduling policy for the touch event; wherein the scheduling policy includes: setting a scheduling priority for the touch event, and the scheduling priority of the touch event is higher than that of other threads or processes Scheduling priority; when scheduling the touch event, the time slice polling scheduling method is replaced by a priority scheduling method.

In one embodiment, the device also includes:

The second detection unit is configured to lower the priority of the touch event when detecting that the application or window corresponding to the touch event is switched to the background; or

When it is detected that the application or window corresponding to the touch event with low priority is switched from the background to the foreground, the priority of the touch event is restored.

According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including a processor, a memory, and an executable program stored on the memory and capable of being run by the processor. When the processor runs the executable program, Execute the steps of the touch processing method based on the graphic display system.

According to a fourth aspect of the embodiments of the present disclosure, there is provided a storage medium on which an executable program is stored, and when the executable program is executed by a processor, the steps of the touch processing method based on a graphics display system are implemented.

According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the touch processing method based on a graphic display system are implemented.

According to a sixth aspect of the embodiments of the present disclosure, a computer program is provided. When the computer program is executed by a processor, the steps of the touch processing method based on a graphics display system are implemented.

In the embodiment of the present disclosure, when a touch operation is detected, a corresponding touch event is generated for the current display interface according to the current UI window and the corresponding interface display situation, and the generated touch event is sent to the corresponding UI The thread of the window is sent; the embodiment of the present disclosure does not respond to the touch event immediately, but caches the touch event, and after the graphic update synchronization event arrives, it is determined based on the touch point in the cached touch event The smooth touch point determines the dirty area to be updated based on the smooth touch point, triggers a rendering thread to render the dirty area, and updates the interface graphics. The embodiment of the present disclosure performs active smoothing processing on multiple touch points, and can very smoothly update the update of the display interface for high-frequency reporting events, avoiding the jitter and frame skipping phenomenon of the graphical interface, and providing users with Better user experience.

Description of drawings

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings that are required in the description of the embodiments or the prior art. Apparently, the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to these drawings without creative efforts.

FIG. 1 is a schematic flowchart of a touch processing method based on a graphic display system according to an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of a comparison between a Vsync update period and a Touch event reporting period according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a graphical interface update cycle of an embodiment of the present disclosure;

4 is a schematic diagram of the composition and structure of a touch processing device based on a graphic display system according to an embodiment of the present disclosure;

FIG. 5 is a structural diagram of an electronic device according to an embodiment of the present disclosure.

Detailed ways

The essence of the technical solutions of the embodiments of the present disclosure will be explained in detail below in conjunction with the accompanying drawings.

FIG. 1 is a schematic flowchart of a touch processing method based on a graphic display system according to an embodiment of the present disclosure. As shown in FIG. 1 , the touch processing method based on a graphic display system according to an embodiment of the present disclosure includes the following processing steps:

Step 101, in response to detecting a touch operation, generate a touch event, and send the touch event to a thread of the current UI window.

In the embodiment of the present disclosure, when the user performs a touch operation (touch) on the touch screen of the electronic device, the touch device detects the touch operation and generates a touch event. In the embodiments of the present disclosure, a touch event may also be referred to as a Touch event.

Step 102, triggering the thread of the current UI window to buffer the touch event, and generating a window update request.

In the embodiment of the present disclosure, as an implementation manner, the Touch event is reported from the kernel driver to the receiving thread in the user mode at a frequency of 10ms/90hz. After the user mode receiving thread receives the Touch event, it will distribute the event to the window manager of the system. After receiving the Touch event, the window manager makes the following decisions based on different scenarios: intercept the Touch event to perform global gesture actions; or distribute the Touch event to the system UI, perform actions such as dragging the status bar; or distribute the Touch event to the current top layer Window application process processing, such as client click, slide and other common operations. In the embodiment of the present disclosure, the window manager realizes the scheduling and management of the threads of the UI window, and the window manager generates the threads of the UI window.

In the embodiment of the present disclosure, after the UI window of the relevant application process receives the Touch event on the main thread, it caches the Touch event in the buffer queue, and requests to update the window, and the buffer queue corresponds to the window one by one. When the Vsync synchronization event arrives, respond to the window update request.

Step 103, after detecting the synchronous event, responding to the update window request, acquiring the information of the touch point in the touch event, and determining the information of the smooth touch point based on the information of the touch point.

In the embodiment of the present disclosure, the synchronization event is the receipt of a vertical synchronization signal Vsync for the graphical interface, and the Vsync is used for vertical synchronization, and its main function is to prevent tearing of the interface graphics. When Vsync is detected, it responds to the update window request. In the embodiment of the present disclosure, instead of directly using the information of the touch points in the cache queue to update the graphical interface, the touch points corresponding to the last two touch operations are obtained, and the sampling time point of the response is determined, so as to In this way, the smooth touch points are determined, and the dirty area is accurately determined based on the smooth touch points, so as to realize the updated rendering of the interface graphics.

Specifically, in the touch event buffered by the thread of the current UI window, search for a set number of touch points closest to the sampling time, based on the time information of the found touch points, and the sampling Time, determining the time and location information of the smooth touch point. The number of settings here is 2. FIG. 2 is a schematic diagram of the comparison between the Vsync update period and the Touch event reporting period of the embodiment of the present disclosure. As shown in FIG. 2, since the Touch event reporting period is short, for example, 10ms, and the Vsync update period is relatively long, for example, 16ms, in this way, When a synchronization event arrives, it is likely to receive two Touch events. For example, at 32ms, two Touch events at 20ms and 30ms will be received. In view of this situation, the embodiments of the present disclosure need to perform smooth processing on it , in order to better realize the continuity of the graphic interface refresh, make the graphic interface refresh smoother, and there will be no jitter and frame skipping phenomenon.

Specifically, when it is determined that the sampling time is between the times of the two touch points, determine the time between the time of the first touch point and the time of the second touch point among the two touch points A first offset, and a second offset between the sampling time and the time of the second touch point, determining a first weight based on the first offset and the second offset ; Calculate the time and location information of the smooth touch point based on the time of the second touch point, the first weight and the first offset.

The specific calculation method is as follows:

TouchTimeDiff＝LastTouch-FirstTouch

TouchSampleDiff=SampleTime-FirstTouch

Result＝FirstTouch+(LastTouch-FirstTouch)×alpha

Here, SampleTime represents the sampling moment, FirstTouch represents the time stamp of the second touch point, and LastTouch represents the time stamp of the first touch point. Alpha represents the first weight. The significance is that a larger Alpha means that the timestamp Result of the smooth touch point is more inclined to the timestamp of the second touch point, and a smaller Alpha means that the sampling result is more inclined to the first touch point. The timestamp of the point. TouchTimeDiff represents the first offset, and TouchSampleDiff represents the second offset; in the embodiment of the present disclosure, the graphical interface based on the Touch event is updated by smoothing when more than two touch points are in the same Vsync update cycle more stable.

As an implementation manner, when it is determined that the sampling time is outside the time of the two touch points, determine the time difference between the time of the first touch point and the time of the second touch point among the two touch points a first offset between, and a third offset between the time of the first touch point and the sampling time, and determine a second weight based on the first offset and the third offset ; calculating the time information of the smooth touch point based on the time of the first touch point, the second weight and the inverse of the first offset. In the embodiment of the present disclosure, the sampling time is outside the time of the two touch points, which means that the sampling time is later than the touch time of the last touch point.

The specific calculation method is as follows:

TouchTimeDiff＝LastTouch-FirstTouch

TouchSampleDiff=LastTouch-SampleTime

Result＝LastTouch+(FirstTouch-LastTouch)×alpha

Here, SampleTime represents the sampling moment, FirstTouch represents the time stamp of the second touch point, and LastTouch represents the time stamp of the first touch point. Alpha represents the second weight, and its significance is that a larger Alpha means that the timestamp Result of the smooth touch point is more inclined to the sampling moment, and a smaller Alpha means that the sampling result is more inclined to the timestamp of the second touch point. TouchTimeDiff represents the first offset, and TouchSampleDiff represents the second offset; in the embodiment of the present disclosure, the graphical interface based on the Touch event is updated by smoothing when more than two touch points are in the same Vsync update cycle more stable.

In the embodiment of the present disclosure, the sampling time is selected to be 5 ms before the arrival time of the Vsync event. Of course, other times may also be selected, such as 6 ms, 7 ms, or 4 ms before the arrival time of the Vsync event.

In the embodiment of the present disclosure, when there is only one touch event in the cache queue, the dirty area of the graphic control is marked directly with the touch point corresponding to the touch event in the cache queue.

In the embodiment of the present disclosure, the position of the touch point with a later time stamp can be selected as the position information of the smooth touch point, because, for the operator, the touch operation is at the millisecond level, and the position can be considered close. As an implementation manner, the position of the touch point with a later time stamp may also be selected as the position information of the smooth touch point. Certainly, the position information of the smooth touch point may also be determined in a weighted position manner based on the above-mentioned manner of determining the smooth touch point.

Step 104, sending the information of the smooth touch point to the target layout control, triggering the target layout control to determine the dirty area to be updated based on the information of the smooth touch point.

In the embodiment of the present disclosure, after the information related to the smooth touch point is determined, it is sent to a target layout control such as a graphics control, so that the target layout control such as a graphics control determines the dirty area to be updated based on the time information of the smooth touch point. In the embodiment of the present disclosure, the graphical interface is not updated according to the actual touch points, but when the synchronization event arrives, the two buffered actual touch points are smoothed, so that the graphical interface based on the touch operation The update is more uniform, there will be no frame skipping and jittering, making the graphical interface more stable, thereby improving the user experience.

Step 105, trigger a rendering thread to render the dirty area, and update interface graphics.

In the embodiment of the present disclosure, after marking the dirty area, the graphic control sends a rendering event to the rendering thread, updates the rendering tree of the dirty area synchronously, submits the rendering result, and displays the updated graphical interface. In this way, the embodiment of the present disclosure can perfectly and smoothly update high-frequency reporting events to the display system.

Although not explicitly described, the touch processing method disclosed in the embodiments of the present disclosure may be applicable to various operating systems, including but not limited to Linux, Windows, Android, and the like.

In one embodiment of the present disclosure, the disclosed method is applied to a Linux operating system, namely GNU/Linux, such as Ubuntu, Debian, RedHat, etc. The Linux operating system is not limited to running on a physical machine, but can also be an operating system hosted on other operating systems, such as WSL2 in Windows systems.

Fig. 3 is a schematic diagram of the update cycle of the graphical interface of the embodiment of the present disclosure. As shown in Fig. 3 , it can be seen from the comparison of the positions of the Touch reporting points before and after sampling that through the above-mentioned smooth sampling process of the embodiment of the present disclosure, every 10 ms moves 10 pixels The sliding behavior, after smoothing, achieves a graphical interface update of 16ms moving 16 pixels. The graphic interface update frame deviation is 0, which completely eliminates the inter-frame jitter and frame skipping phenomenon, greatly improves the display quality of the graphical interface, and improves the user experience.

In the embodiment of the present disclosure, in order to ensure that the system can respond to the touch event first, a higher priority is set for the touch event, so that the touch event can be scheduled first. Specifically, a scheduling policy is set for the touch event; wherein, the scheduling policy includes: setting a scheduling priority for the touch event, and the scheduling priority of the touch event is higher than that of other threads or processes level; when scheduling the touch events, the time slice round-robin scheduling is replaced by priority scheduling.

Dynamically configure the CPU scheduling policy of the Touch receiving process to improve the throughput of Touch events during the receiving and distribution process, thereby reducing the fluctuation problem caused by cross-process or cross-thread. Specifically, you can directly access the input system process driven by the Touch device. By configuring the FIO scheduling policy, the CPU running priority of the Touch is higher than other processes. The thread priority queue policy is used instead of the time slice polling scheduling policy. When the Touch receives the thread It can always be at the head of the priority queue, and the CPU will not be easily seized by other threads.

In the embodiment of the present disclosure, priority adjustment of touch events is also supported. Specifically, when it is detected that the application or UI window corresponding to the touch event is switched to the background, the priority of the touch event is lowered; or, the application or UI corresponding to the touch event with low priority is detected When the window is switched from the background to the foreground, the priority of the touch event is restored.

In the embodiment of the present disclosure, the CPU scheduling priority of the main thread can be dynamically changed according to different running states of different client application processes. For example, when the process switches to the foreground and starts to respond to UI operations, the CPU scheduling priority of the main thread is increased, and when the process is switched to the background, the CPU scheduling priority of the main thread is lowered to give up the CPU.

The embodiment of the present disclosure performs active smoothing processing on multiple touch points, and can very smoothly update the update of the display interface for high-frequency reporting events, avoiding the jitter and frame skipping phenomenon of the graphical interface, and providing users with Better user experience.

FIG. 4 is a schematic diagram of the composition and structure of a touch processing device based on a graphics display system according to an embodiment of the disclosure. As shown in FIG. 4 , the touch processing device based on a graphics display system according to an embodiment of the disclosure includes:

a first detection unit 40, configured to detect a touch operation;

A generating unit 41, configured to generate a touch event in response to detecting a touch operation;

The first transmission unit 42 is configured to send the touch event to the thread of the current UI window;

The first trigger unit 43 is configured to trigger the thread of the current UI window to cache the touch event, and generate a window update request;

A second detection unit 44, configured to detect a synchronization event;

A determining unit 45, configured to respond to the update window request, acquire information of touch points in the touch event, and determine information of smooth touch points based on the information of touch points;

The second transmission unit 46 is configured to send the information of the smooth touch point to the target layout control;

The second triggering unit 47 is configured to trigger the target layout control to determine the dirty area to be graphically updated based on the information of the smooth touch point;

The third triggering unit 48 is configured to trigger a rendering thread to render the dirty area and update interface graphics.

In one embodiment, the determining unit 45 is further configured to:

In one embodiment, the set number is two;

Correspondingly, the determining unit 45 is further configured to:

In one embodiment, the set number is two;

Correspondingly, the determining unit 45 is further configured to:

In one embodiment, on the basis of the touch processing device based on a graphics display system shown in FIG. 4 , the touch processing device based on a graphics display system in an embodiment of the present disclosure further includes:

A setting unit (not shown in FIG. 4 ), configured to set a scheduling policy for the touch event; wherein, the scheduling policy includes: setting a scheduling priority for the touch event, and the scheduling priority of the touch event is The scheduling priority is higher than that of other threads or processes; when scheduling the touch event, the time slice polling scheduling mode is replaced by priority scheduling mode.

A second detection unit (not shown in FIG. 4 ), configured to lower the priority of the touch event when detecting that the application or window corresponding to the touch event is switched to the background; or

In an exemplary embodiment, the first detection unit 40, the generation unit 41, the first transmission unit 42, the first trigger unit 43, the second detection unit 44, the determination unit 45, the second transmission unit 46, the second trigger unit 47 , the third trigger unit 48, the setting unit and the second detection unit, etc. can be controlled by one or more central processing units (CPU, Central Processing Unit), application-specific integrated circuits (ASIC, Application Specific Integrated Circuit), digital signal processors ( Digital Signal Processor, DSP), programmable logic device (PLD, Programmable Logic Device), complex programmable logic device (CPLD, Complex Programmable Logic Device), field programmable gate array (FPGA, Field-Programmable Gate Array), general processing A controller, a controller, a microcontroller (MCU, Micro Controller Unit), a microprocessor (Microprocessor), or other electronic components are used to execute the steps of the touch processing method based on the graphic display system of the foregoing embodiments.

In the embodiment of the present disclosure, the specific manner of performing operations of each unit in the touch processing device based on the graphic display system shown in FIG. 4 has been described in detail in the embodiment of the method, and will not be elaborated here illustrate.

Below, an electronic device 11 according to an embodiment of the present disclosure will be described with reference to FIG. Assistants, servers, blade servers, mainframes, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, tablet computers, wearable devices, and other similar computing devices.

As shown in FIG. 5 , the electronic device 11 includes one or more processors 111 and a memory 112 .

The processor 111 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 11 to perform desired functions.

Memory 112 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM, Random Access Memory) and/or a cache memory (cache). The non-volatile memory may include, for example, a read-only memory (ROM, Read Only Memory), a hard disk, a flash memory, and the like. One or more computer program instructions can be stored on the computer-readable storage medium, and the processor 111 can execute the program instructions to realize the touch control based on the graphic display system of the various embodiments of the present disclosure described above. processing methods and/or other desired functionality. Various contents such as input signal, signal component, noise component, etc. may also be stored in the computer-readable storage medium.

In one example, the electronic device 11 may further include: an input device 113 and an output device 114 , and these components are interconnected through a bus system and/or other forms of connection mechanisms (not shown in FIG. 5 ).

The input device 113 may include, for example, a keyboard, a mouse, and the like.

The output device 114 can output various information to the outside, including determined distance information, direction information, and the like. The output device 114 may include, for example, a display, a speaker, a printer, a communication network and remote output devices connected thereto, and the like.

Of course, for simplicity, only some components related to the present disclosure in the electronic device 11 are shown in FIG. 5 , and components such as bus, input/output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device 11 may also include any other appropriate components.

The embodiments of the present disclosure also describe a storage medium on which an executable program is stored, and the executable program is executed by a processor to perform the steps of the touch processing method based on a graphics display system in the foregoing embodiments.

In addition to the above-mentioned methods and devices, embodiments of the present disclosure may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the above-mentioned "exemplary method" of this specification. Steps in methods according to various embodiments of the present disclosure described in section.

The computer program product can be written in any combination of one or more programming languages to execute the program codes for performing the operations of the embodiments of the present disclosure, and the programming languages include object-oriented programming languages, such as Java, C++, etc. , also includes conventional procedural programming languages, such as the "C" language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server to execute.

In addition, the embodiments of the present disclosure may also be a computer-readable storage medium, on which computer program instructions are stored, and the computer program instructions, when executed by a processor, cause the processor to perform the above-mentioned "Exemplary Method" section of this specification. Steps in methods according to various embodiments of the present disclosure described in .

The computer readable storage medium may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: electrical connection with one or more conductors, portable disk, hard disk, random access memory (RAM), read only memory (ROM), erasable Type programmable read-only memory (EPROM, Erasable Programmable Read-Only Memory or flash memory), optical fiber, portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or the above any suitable combination.

The embodiments of the present disclosure may also be a computer program, which implements the steps of the graphic display system-based touch processing method of the foregoing embodiments when executed by a processor.

The basic principles of the present disclosure have been described above in conjunction with specific embodiments, but it should be pointed out that the advantages, advantages, effects, etc. mentioned in the present disclosure are only examples rather than limitations, and these advantages, advantages, effects, etc. Various embodiments of the present disclosure must have. In addition, the specific details disclosed above are only for the purpose of illustration and understanding, rather than limitation, and the above details do not limit the present disclosure to be implemented by using the above specific details.

The block diagrams of devices, devices, devices, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, devices, systems may be connected, arranged, configured in any manner. Words such as "including", "comprising", "having" and the like are open-ended words meaning "including but not limited to" and may be used interchangeably therewith. As used herein, the words "or" and "and" refer to the word "and/or" and are used interchangeably therewith, unless the context clearly dictates otherwise. As used herein, the word "such as" refers to and is used interchangeably with the phrase "such as but not limited to".

It should also be pointed out that, in the devices, equipment and methods of the present disclosure, each component or each step can be decomposed and/or reassembled. These decompositions and/or recombinations should be considered equivalents of the present disclosure.

The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit the disclosed embodiments to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

A touch processing method based on a graphic display system, characterized in that the method comprises:

In response to detecting a touch operation, generate a touch event, and send the touch event to a thread of the current UI window;

triggering the thread of the current UI window to cache the touch event, and generate a window update request;

After detecting the synchronous event, responding to the update window request, acquiring the information of the touch point in the touch event, and determining the information of the smooth touch point based on the information of the touch point;

Sending the information of the smooth touch point to the target layout control, triggering the target layout control to determine the dirty area to be graphics updated based on the information of the smooth touch point;

Trigger the rendering thread to render the dirty area, and update the interface graphics.
The method according to claim 1, wherein the determining the information of the smooth touch point based on the information of the touch point comprises:

In the touch event cached by the thread of the current UI window, search for a set number of touch points closest to the sampling time, and determine based on the time information of the found touch points and the sampling time The time and location information of the smooth touch point.
The method according to claim 2, wherein the set number is two;

The determining the time and position information of the smooth touch point includes:

When it is determined that the sampling time is between the times of the two touch points, determine a first offset between the time of the first touch point and the time of the second touch point of the two touch points amount, and a second offset between the sampling time and the time of the second touch point, the first weight is determined based on the first offset and the second offset; based on the The time and position information of the smooth touch point is calculated by the time of the second touch point, the first weight and the first offset.
The method according to claim 2 or 3, wherein the set number is two;

The determining the time and position information of the smooth touch point includes:

When it is determined that the sampling time is outside the time of the two touch points, determine a first offset between the time of the first touch point and the time of the second touch point of the two touch points amount, and a third offset between the time of the first touch point and the sampling time, the second weight is determined based on the first offset and the third offset; based on the The time and position information of the smooth touch point is calculated by the time of the first touch point, the second weight and the inverse of the first offset.
The method according to any one of claims 1 to 4, wherein the method further comprises:

Setting a scheduling strategy for the touch event;

Wherein, the scheduling strategy includes: setting a scheduling priority for the touch event, the scheduling priority of the touch event is higher than that of other threads or processes; when scheduling the touch event, The time-slice round-robin scheduling method is replaced by the priority scheduling method.
The method according to claim 5, wherein the method further comprises:

When detecting that the application or UI window corresponding to the touch event is switched to the background, lower the priority of the touch event; or

When it is detected that the application or UI window corresponding to the touch event with low priority is switched from the background to the foreground, the priority of the touch event is restored.
A touch processing device based on a graphics display system, characterized in that the device includes:

a first detection unit, configured to detect a touch operation;

a generating unit, configured to generate a touch event in response to detecting a touch operation;

A first transmission unit, configured to send the touch event to a thread of the current UI window;

A first trigger unit, configured to trigger the thread of the current UI window to cache the touch event, and generate a window update request;

a second detection unit, configured to detect a synchronization event;

A determining unit, configured to respond to the update window request, obtain information of touch points in the cached touch event, and determine information of smooth touch points based on the information of the touch points;

The second transmission unit is configured to send the information of the smooth touch point to the target layout control;

A second trigger unit, configured to trigger the target layout control to determine a dirty area to be updated based on the information of the smooth touch point;

The third triggering unit is configured to trigger the rendering thread to render the dirty area and update the interface graphics.
The device according to claim 7, wherein the determining unit is further configured to:

In the touch process in the thread cache of the current UI window, search for a set number of touch points closest to the sampling time, and determine based on the time information of the found touch points and the sampling time The time and location information of the smooth touch point.
The device according to claim 8, wherein the set number is two;

The determining unit is also used for:

When it is determined that the sampling time is between the times of the two touch points, determine a first offset between the time of the first touch point and the time of the second touch point of the two touch points amount, and a second offset between the sampling time and the time of the second touch point, the first weight is determined based on the first offset and the second offset; based on the The time and position information of the smooth touch point is calculated by the time of the second touch point, the first weight and the first offset.
The device according to claim 8 or 9, wherein the set number is two;

The determining unit is also used for:

When it is determined that the sampling time is outside the time of the two touch points, determine a first offset between the time of the first touch point and the time of the second touch point of the two touch points amount, and a third offset between the time of the first touch point and the sampling time, the second weight is determined based on the first offset and the third offset; based on the The time and position information of the smooth touch point is calculated by the time of the first touch point, the second weight and the inverse of the first offset.
The device according to any one of claims 7 to 10, wherein the device further comprises:

A setting unit, configured to set a scheduling policy for the touch event; wherein the scheduling policy includes: setting a scheduling priority for the touch event, and the scheduling priority of the touch event is higher than that of other threads or processes Scheduling priority; when scheduling the touch event, the time slice polling scheduling method is replaced by a priority scheduling method.
The device according to claim 11, further comprising:

The second detection unit is configured to lower the priority of the touch event when detecting that the application or window corresponding to the touch event is switched to the background; or

When it is detected that the application or window corresponding to the touch event with low priority is switched from the background to the foreground, the priority of the touch event is restored.
An electronic device, comprising a processor, a memory, and an executable program stored on the memory and capable of being run by the processor, when the processor runs the executable program, it executes any one of claims 1 to 6 Steps of the touch processing method based on the graphic display system described above.
A storage medium, on which an executable program is stored, and when the executable program is executed by a processor, the steps of the touch processing method based on a graphic display system according to any one of claims 1 to 6 are implemented.
A computer program product, characterized in that it includes a computer program, and when the computer program is executed by a processor, the steps of the touch processing method based on a graphics display system according to any one of claims 1 to 6 are implemented.
A computer program, characterized in that, when the computer program is executed by a processor, the steps of the touch processing method based on a graphic display system according to any one of claims 1 to 6 are implemented.