CN116048828A

CN116048828A - Method for processing drawing message, electronic device and storage medium

Info

Publication number: CN116048828A
Application number: CN202210631120.6A
Authority: CN
Inventors: 赵和平; 谭文文
Original assignee: Honor Device Co Ltd
Current assignee: Honor Device Co Ltd
Priority date: 2022-05-26
Filing date: 2022-05-26
Publication date: 2023-05-02
Anticipated expiration: 2042-05-26
Also published as: CN116048828B

Abstract

The application provides a method for processing a drawing message, electronic equipment and a storage medium, wherein the method comprises the following steps: responding to a drawing request instruction of an application program, and requesting a drawing signal; generating a drawing message based on the received drawing signal; identifying whether a message queue of a main thread of the current application program contains at least one perceptible message, wherein the perceptible message is a message which can cause a display interface to generate visual perception change after being processed; if the message queue contains at least one perceptible message, inserting the drawing message into the last perceptible message in the message queue, and carrying out priority processing on the drawing message. According to the method and the device, when the application program generates the drawing message and the perceivable message exists in the message queue of the main thread of the application program, the drawing message is subjected to queue inserting priority processing, normal processing of other perceivable messages is not influenced, and timely refreshing of a display interface is guaranteed.

Description

Method for processing drawing message, electronic device and storage medium

Technical Field

The present disclosure relates to the field of display technologies, and in particular, to a method for processing a drawing message, an electronic device, and a storage medium.

Background

With the configuration of intelligent electronic devices such as smartphones and tablet computers becoming higher, the performance requirements of users on the intelligent electronic devices become higher, especially in scenes with frequent man-machine interaction and obvious visual perception, such as starting application programs, page switching and page sliding of the application programs, the application programs are started quickly, the page switching is quick, the page sliding is smooth, and page contents are refreshed and presented in time, so that better performance experience can be brought to the users. However, in the process of interaction between an application program and a user, the main thread of the application program often needs to process various messages, which results in unnecessary time consumption, and affects timely processing of drawn messages, so that the application is slow to start, the page is slow to switch, and the page is not smooth to slide, thereby affecting the user experience.

Disclosure of Invention

In view of the foregoing, it is necessary to provide a method for processing a drawn message, an electronic device, and a storage medium, so as to solve a technical problem that a main thread of an application program cannot process a drawn message in time due to processing other messages.

In a first aspect, the present application provides a method for processing a drawing message, the method including: responding to a drawing request instruction of an application program, and requesting a drawing signal; generating a drawing message based on the received drawing signal; identifying whether a message queue of a main thread of a current application program contains at least one perceptible message, wherein the perceptible message is a message which can enable a display interface to generate visual perception change after being processed; if the message queue contains at least one perceptible message, inserting the drawing message after the last perceptible message in the message queue, and carrying out priority processing on the drawing message. By the technical scheme, when the application program generates the drawing message and the perceivable message exists in the message queue of the main thread of the application program, the drawing message is dequeued to the perceivable message and then is subjected to priority processing, and meanwhile, the display interface is ensured to be refreshed in time based on the drawing message and the perceivable message.

In one possible implementation, the method further includes: the perceptible message is pre-marked. Through the technical scheme, the perceptible information can be set in advance.

In one possible implementation, the pre-tagged perceptible message includes: and adding a perceptible mark to a preset message through a mark interface, wherein the preset message comprises at least one of a drawing message, an input message, a message for updating interface content and a display screen configuration change message. Through the technical scheme, the marking of the perceivable messages can be realized quickly.

In one possible implementation, the pre-tagged perceptible message includes: obtaining a message to be processed from the message queue, and distributing and processing the message; judging whether the message triggers a drawing request or not; if the message triggers the drawing request, setting a perceivable flag bit for the message; judging whether the message is a main thread message or not; if the message is the main thread message, judging whether a perceivable flag bit of the message is set; if the perceivable flag bit of the message is set, marking the message as a perceivable message, and recording the associated information of the message, wherein the associated information comprises the package name, the version and the message number of the application program generating the message. Through the technical scheme, the message can be automatically marked as the perceptible message when the message triggers the drawing request, so that the perceptible message is marked in a self-learning identification mode.

In one possible implementation manner, the obtaining the message to be processed from the message queue, and distributing and processing the message includes: acquiring a message to be processed from the message queue, and judging whether the message is a main thread message or not; if the message is a main line Cheng Xiaoxi, clearing the perceivable flag bit of the message and distributing and processing the message. Through the technical scheme, the information can be ensured to be correctly identified as the perceivable information in the process of marking the perceivable information by self-learning identification.

In one possible implementation, the pre-tagged perceptible message further includes: after marking the message as a perceptible message, clearing a perceptible flag bit of the message; and recovering the message and releasing the resources occupied by the message when the message does not trigger a drawing request, or the message is not the main thread message, or the perceivable flag bit of the message is not set. By the technical scheme, the next message can be processed in time.

In one possible implementation, the identifying whether the message queue of the current application main thread contains at least one perceptible message includes: identifying whether each message in the message queue is a pre-marked perceptible message; if at least one message in the message queue is a pre-marked perceptible message, determining that the message queue contains at least one perceptible message; or if all the messages in the message queue are not pre-marked perceptible messages, determining that the message queue does not contain perceptible messages. Through the technical scheme, the perceivable messages in the message queue can be rapidly identified.

In one possible implementation, the method further includes: uploading the pre-marked associated information of the perceivable messages to a cloud server; and downloading the associated information of the perceivable messages uploaded by at least one terminal device from the cloud server. Through the technical scheme, the cloud sharing and updating of the perceivable messages can be realized.

In one possible implementation, the method further includes: counting habit information of using an application program by a user at intervals of a preset period; uploading the habit information to the cloud server, and receiving association information of the perceivable messages corresponding to the habit information, wherein the habit information comprises names of a first preset number of application programs with the largest use times in the preset period and a second preset number of scenes with the largest use times of each application program in the first preset number of application programs. Through the technical scheme, the perceivable message can be determined based on big data and user habits, and the perceivable message of the electronic equipment is updated.

In one possible implementation, the method further includes: if the message queue does not contain the perceivable message, inserting the drawing message before the first message in the message queue, and carrying out priority processing on the drawing message. By the technical scheme, the drawing message can be processed in time, and normal refreshing of the display interface is ensured.

In one possible implementation, the method further includes: judging whether the drawing message is expired or not in real time; if the drawing message has expired, identifying whether the message queue contains at least one perceptible message; or if the drawing message does not expire, adding the drawing message to the message queue according to the time sequence, and processing the drawing message according to a default flow. By the technical scheme, when the drawing information is not expired, the drawing information can be executed according to a default flow, and the influence on the normal processing of other information is avoided.

In one possible implementation, the determining in real time whether the draw message has expired includes: judging whether the timing time of the electronic equipment after being started exceeds the expected processing time of the drawing message in real time; if the timing time of the electronic equipment after being started exceeds the expected processing time of the drawing message, determining that the drawing message is expired; or if the timing time of the electronic equipment after being started does not exceed the expected processing time of the drawing message, determining that the drawing message is not expired. By the technical scheme, whether the drawing message expires or not can be accurately determined, and whether the drawing message needs to be subjected to queue-inserting priority processing or not is further determined.

In one possible implementation, the generating a drawing message based on the received drawing signal includes: based on the received draw signal, a barrier message is added to the message queue and the draw message is generated. By the technical scheme, the synchronous message processing can be shielded through the barrier message, so that the drawing message can be preferentially processed as the asynchronous message.

In one possible implementation, the method further includes: if the draw message has expired, identifying whether the message queue contains at least one perceptible message prior to the barrier message; if the message queue contains at least one perceptible message before the barrier message, inserting the drawing message into the message queue after the last perceptible message before the barrier message, and carrying out priority processing on the drawing message; or if the message queue does not contain the perceivable message before the barrier message, inserting the drawing message before the first message in the message queue, and carrying out priority processing on the drawing message. By the technical scheme, the drawing message can be processed in time, and normal refreshing of the display interface is ensured.

In one possible implementation, the method further includes: if the message queue does not contain perceptible messages before the barrier message, judging whether the message queue contains at least one message with delay processing time greater than or equal to a threshold time before the barrier message; if the message queue contains at least one message with delay processing time greater than or equal to the threshold time before the barrier message, inserting the drawing message into the message with the last delay processing time greater than or equal to the threshold time in the message queue, and carrying out priority processing on the drawing message; or if the message queue does not contain the message with the delay processing time greater than or equal to the threshold time, inserting the drawn message before the first message in the message queue, and carrying out priority processing on the drawn message. By the technical scheme, when the message queue contains the long-time unprocessed message, the drawn message is inserted into the long-time unprocessed message, so that the message is prevented from being in a state of being continuously unprocessed for a long time.

In one possible implementation, the method further includes: if the draw message has expired, identifying whether the message queue contains at least one perceptible message prior to the barrier message; if the message queue contains at least one perceptible message before the barrier message, inserting the barrier message into the message queue after the last perceptible message before the barrier message, and carrying out priority processing on the barrier message; or if the message queue does not contain perceptible messages before the barrier message, inserting the barrier message before the first message in the message queue, and carrying out priority processing on the barrier message. By the technical scheme, the barrier message can be subjected to the queue inserting processing, so that the drawn message can be preferentially processed as the asynchronous message, and the normal refreshing of the display interface is ensured.

In a second aspect, the present application provides an electronic device comprising a memory and a processor: wherein the memory is used for storing program instructions; the processor is configured to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, cause the electronic device to execute the above-described rendering message processing method.

In a third aspect, the present application provides a computer storage medium storing program instructions that, when executed on an electronic device, cause the electronic device to perform the above-described method of rendering message processing.

In addition, the technical effects of the second aspect and the third aspect may be referred to in the description related to the method designed by each of the above method sections, which are not repeated herein.

According to the method for processing the drawing message, the electronic equipment and the storage medium, when the application program generates the drawing message and the perceivable message exists in the message queue of the main thread of the application program, the drawing message is inserted into the last perceivable message in the message queue, so that the drawing message can be subjected to queue inserting priority processing, the display interface is ensured to be refreshed in time based on the drawing message, normal processing of other perceivable messages is not influenced, and the display interface is ensured to be refreshed in time based on the perceivable message, so that user experience is improved.

Drawings

FIG. 1A is a diagram of a message queue for an application main thread according to one embodiment of the present application.

FIG. 1B is a diagram of a message queue for another application main thread according to one embodiment of the present application.

FIG. 1C is a diagram of a message queue for another application main thread according to one embodiment of the present application.

FIG. 1D is a diagram of a message queue for another application main thread according to one embodiment of the present application.

Fig. 2A is a schematic diagram of a message queue head insertion barrier message according to an embodiment of the present application.

Fig. 2B is a schematic diagram of a message queue head insertion vsync drawing message according to an embodiment of the present application.

Fig. 3 is a software architecture diagram of an electronic device according to an embodiment of the present application.

Fig. 4 is a flowchart of a method for processing a drawing message according to an embodiment of the present application.

Fig. 5 is a flow chart of a perceived message through self-learning automatic tagging provided in an embodiment of the present application.

Fig. 6 is a schematic diagram of a communication scenario of an electronic device according to an embodiment of the present application.

Fig. 7 is an interaction schematic diagram of an electronic device, a cloud server, and a testing machine according to an embodiment of the present application.

Fig. 8A is a schematic diagram of a message queue of an application main thread according to an embodiment of the present application.

Fig. 8B is a schematic diagram of a message queue insertion drawing message of an application main thread according to an embodiment of the present application.

FIG. 9A is a diagram of a message queue for another application main thread according to one embodiment of the present application.

Fig. 9B is a schematic diagram of a message queue insertion drawing message of another application main thread according to an embodiment of the present application.

Fig. 10 is a flowchart of a method for processing a drawing message according to another embodiment of the present application.

Fig. 11 is a schematic diagram of a message queue of an application main thread according to another embodiment of the present application.

Fig. 12 is a flowchart of a method for processing a drawing message according to another embodiment of the present application.

Fig. 13A is a schematic diagram of a message queue of an application main thread according to another embodiment of the present application.

Fig. 13B is a schematic diagram of a message queue insertion drawing message of an application main thread according to another embodiment of the present application.

Fig. 14A is a schematic diagram of a message queue of another application main thread according to another embodiment of the present application.

Fig. 14B is a schematic diagram of a message queue insertion drawing message of another application main thread according to another embodiment of the present application.

Fig. 15 is a flowchart of a method for processing a drawing message according to another embodiment of the present application.

Fig. 16A is a schematic diagram of a message queue of an application main thread according to another embodiment of the present application.

Fig. 16B is a schematic diagram of a message queue insertion drawing message of an application main thread according to another embodiment of the present application.

FIG. 17A is a diagram of a message queue for another application main thread provided in another embodiment of the present application.

Fig. 17B is a schematic diagram of a message queue insertion drawing message of another application main thread according to another embodiment of the present application.

Fig. 18 is a flowchart of a method for processing a drawing message according to another embodiment of the present application.

Fig. 19 is a flowchart of a method for processing a drawing message according to another embodiment of the present application.

Fig. 20 is a schematic diagram of functional modules of an electronic device according to an embodiment of the present application.

Fig. 21 is a hardware architecture diagram of an electronic device according to an embodiment of the present application.

Detailed Description

For ease of understanding, a description of some of the concepts related to the embodiments of the present application are given by way of example for reference.

It should be noted that the terms "first," "second," and "second" in the embodiments of the present application are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implying a number of technical features being indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. It should be understood that, "/" means or, unless otherwise indicated herein. For example, A/B may represent A or B. The term "and/or" in this application is merely an association relationship describing an association object, and means that three relationships may exist. For example, a and/or B may represent: a exists alone, A and B exist simultaneously, and B exists alone. "at least one" means one or more. "plurality" means two or more than two. For example, at least one of a, b or c may represent: seven cases of a, b, c, a and b, a and c, b and c, a, b and c.

The User Interface (UI) in the embodiment of the present application is a media Interface for interaction and information exchange between an application program or an operating system and a User, and may implement conversion between an internal form of information and an acceptable form of the User. The user interface of the application program is source code written in a specific computer language such as JAVA, extensible markup language (extensible markup language, XML) and the like, and the interface source code is analyzed and rendered on the electronic equipment and finally presented as content which can be identified by a user, such as a control of pictures, words, buttons and the like. Controls, which are basic elements of a user interface, are typically buttons (buttons), gadgets, toolbars, menu bars, text boxes, scroll bars, pictures, and text. The properties and content of the controls in the interface are defined by labels or nodes, such as XML specifies the controls contained in the interface by nodes of < Textview >, < ImgView >, < VideoView >, etc. One node corresponds to a control or attribute in the interface, and the node is rendered into visual content for a user after being analyzed and rendered. In addition, many applications, such as the interface of a hybrid application (hybrid application), typically include web pages. A web page, also referred to as a page, is understood to be a special control embedded in an application interface, which is source code written in a specific computer language, such as hypertext markup language (hyper text markup language, HTML), cascading style sheets (cascading style sheets, CSS), JAVA script (JavaScript, JS), etc., and which can be loaded and displayed as user-identifiable content by a browser or web page display component similar to the browser functionality. The specific content contained in a web page is also defined by tags or nodes in the web page source code, such as HTML defines the elements and attributes of the web page by < p >, < img >, < video >, < canvas >.

A commonly used presentation form of the user interface is a graphical user interface (graphic user interface, GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It may be an interface element such as an icon, window, control, etc. displayed in a display screen of the electronic device.

The following embodiments and features of the embodiments may be combined with each other without conflict.

With the configuration of intelligent electronic devices such as smartphones and tablet computers becoming higher, the performance requirements of users on the intelligent electronic devices become higher, especially in scenes with frequent man-machine interaction and obvious visual perception, such as starting application programs, page switching and page sliding of the application programs, the application programs are started quickly, the page switching is quick, the page sliding is smooth, and page contents are refreshed and presented in time, so that better performance experience can be brought to the users. However, in the process of interaction between the application program and the user, the main thread of the application program needs to process the imperceptible message in addition to the perceptible message, and the time consumption of processing the imperceptible message may affect the timely processing of the perceptible message, so that the application is slow to start, the page is switched slowly, and the page is not smooth in sliding, thereby affecting the user experience.

In order to avoid the influence on the timely processing of the perceivable messages caused by the processing of the imperceptible messages, the embodiment of the application provides a method for processing the drawing messages, so that the situations of slow application starting, slow page switching, unsmooth page sliding and the like are avoided.

In order to better understand the method for processing a drawing message provided in the embodiment of the present application, an application scenario of the method for processing a drawing message of the present application is first described below.

During human interaction, the application main thread needs to process many messages, such as receiving a broadcast, processing vsync (vertical synchronization, vertical sync) rendering messages, starting a service, etc. Wherein it is directly influencing the user experience whether the vsync draw message can be processed to completion within the expected time.

After the electronic equipment draws and renders the data to be displayed, the data to be displayed is stored in a buffer zone (buffer), and when the display screen needs to refresh the display interface, the data of the display interface of the next frame is read from the buffer zone and displayed, wherein the display interface is a user interface displayed by the display screen. In order to avoid tearing of the picture displayed by the display screen, the drawing, rendering and displaying of the display interface usually adopts a double-buffer or triple-buffer mechanism. Taking a double-buffer mechanism as an example, the double-buffer comprises a frame buffer and a background buffer, the display screen only reads data to be displayed from the frame buffer, the electronic equipment stores the data to be displayed obtained by drawing and rendering in the background buffer, and when the display screen needs to refresh the display interface, the frame buffer and the data in the background buffer are exchanged, so that the display screen can read the data to be displayed to refresh the display interface. The application program requests the vsync signal when the display screen starts to display the current frame and needs to refresh the next frame, generates the vsync drawing message after receiving the vsync signal, processes the vsync drawing message, draws and renders the data of the next frame display interface and stores the data in the background buffer, then exchanges the data in the frame buffer and the background buffer, and the display screen reads the data of the next frame display interface from the frame buffer and displays the data, thereby completing the refreshing of the display interface. Based on the dual cache mechanism, applications and rendering services (SurfaceFlinger) may refresh the display interface according to the vsync draw message. For example, if the frame rate of the current electronic device display screen is 60 frames/second, the application program needs to complete rendering and displaying of the display interface within 16.67ms after receiving the vsync drawing message, otherwise frame loss will be caused.

And when the application program needs to draw a display interface, the system intercepts the synchronous message by inserting a barrier (barrier) message so as to lead the subsequent asynchronous message to be processed preferentially. For example, vsync draw messages, input messages, belong to asynchronous messages, and when a barrier message is processed, vsync draw messages may be preferentially processed as asynchronous messages. However, if the message queue has accumulated many messages before the barrier message, the time for executing the messages is long, which may cause that the subsequent vsync drawing message cannot be processed in time, and the display interface cannot be refreshed in time, so that the experience of unsmooth display, frame loss or jamming is brought to the user.

For example, referring to FIG. 1A, messages in the message queue of the current application main thread are msg1-msg4 in sequence. Referring to fig. 1B, when an application program needs to draw a display interface, a vsync signal is requested to a rendering service, then a barrier message barrier is inserted into a message queue, the application program generates vsync drawing messages after receiving the vsync signal, inserts the vsync drawing messages into the message queue in sequence, and when the barrier message barrier is processed, the subsequent vsync drawing messages can be preferentially processed. Referring to FIG. 1C, message msg1 has been processed and message msg5 received before the application inserts the vsync draw message into the message queue. Referring to fig. 1D, after receiving the message msg5, the application inserts the vsync draw message into the message queue, and the application main thread preferentially processes the asynchronous vsync draw message when processing the barrier message, so that if the time spent by the application main thread for processing the messages msg2-msg4 is too long and exceeds 16.67ms, the processing of the vsync draw message is delayed, which further results in frame loss of the display screen.

Referring to fig. 2A, in the prior art, a barrier message may be inserted into the message queue head, so that the barrier message may be preferentially processed, and thus, vsync drawn messages in the message queue may also be preferentially processed. However, if the message before the vsync drawing message includes a perceptible message, where the perceptible message is a message that can make the display interface generate a visual perception change after being processed, for example, the perceptible message msg1 is an input message that the user changes the window size through touch, and the perceptible message msg1 is skipped and is not processed in time, so that the window size change in the display interface is not presented to the user in time, which may cause the display interface to be not corresponding to the user operation, thereby affecting the user experience.

Referring to fig. 2B, in the prior art, after the barrier message is inserted into the message queue, the vsync drawing message may be inserted into the message queue header, so that the vsync drawing message may be preferentially processed. However, the vsync drawn message queue may include a perceptible message, for example, the perceptible message msg1 is an input message that the user changes the window size by touch, and the perceptible message msg1 is skipped and is not processed in time, so that the window size change is not presented to the user in time, thereby affecting the user experience.

Referring to fig. 3, a software architecture diagram of an electronic device according to an embodiment of the present application is shown. The layered architecture divides the software into several layers, each with distinct roles and branches. The layers communicate with each other through a software interface. For example, the Android system is divided into four layers, namely, an application layer 101, a framework layer 102, an Android runtime (Android run) and system library 103, a hardware abstraction layer 104 and a kernel layer 105 from top to bottom.

The application layer may include a series of application packages. For example, the application package may include applications for cameras, gallery, calendar, phone calls, maps, navigation, WLAN, bluetooth, music, video, short messages, device control services, etc.

The framework layer provides an application programming interface (Application Programming Interface, API) and programming framework for application programs of the application layer. The application framework layer includes a number of predefined functions. For example, the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and the like.

Wherein the window manager is used for managing window programs. The window manager can acquire the size of the display screen, judge whether a status bar exists, lock the screen, intercept the screen and the like. The content provider is used to store and retrieve data and make such data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phonebooks, etc. The view system includes visual controls, such as controls to display text, controls to display pictures, and the like. The view system may be used to build applications. The display interface may be composed of one or more views. For example, a display interface including a text message notification icon may include a view displaying text and a view displaying a picture. The telephony manager is for providing communication functions of the electronic device. Such as the management of call status (including on, hung-up, etc.). The resource manager provides various resources for the application program, such as localization strings, icons, pictures, layout files, video files, and the like. The notification manager allows the application to display notification information in a status bar, can be used to communicate notification type messages, can automatically disappear after a short dwell, and does not require user interaction. Such as notification manager is used to inform that the download is complete, message alerts, etc. The notification manager may also be a notification in the form of a chart or scroll bar text that appears on the system top status bar, such as a notification of a background running application, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in a status bar, a prompt tone is emitted, the electronic device vibrates, and an indicator light blinks, etc.

Android run time includes a core library and virtual machines. Android run time is responsible for scheduling and management of the Android system. The core library consists of two parts: one part is a function which needs to be called by java language, and the other part is a core library of android.

The application layer and the framework layer run in virtual machines. The virtual machine executes java files of the application program layer and the framework layer as binary files. The virtual machine is used for executing the functions of object life cycle management, stack management, thread management, security and exception management, garbage collection and the like.

The system library may include a plurality of functional modules. Such as surface manager (surface manager), media library (Media Libraries), three-dimensional graphics processing library (e.g., openGL ES), 2D graphics engine (e.g., SGL), etc.

The surface manager is used for managing the display subsystem and providing fusion of 2D and 3D layers for a plurality of application programs. Media libraries support a variety of commonly used audio, video format playback and recording, still image files, and the like. The media library may support a variety of audio and video encoding formats, such as MPEG4, h.264, MP3, AAC, AMR, JPG, PNG, etc. The three-dimensional graphic processing library is used for realizing three-dimensional graphic drawing, image rendering, synthesis, layer processing and the like. The 2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is a layer between hardware and software. The inner core layer at least comprises a display driver, a camera driver, an audio driver and a sensor driver.

The kernel layer is the core of the operating system of the electronic device, is the first layer of software expansion based on hardware, provides the most basic function of the operating system, is the basis of the operating system, is responsible for managing the processes, the memory, the device driver, the files and the network system of the system, and determines the performance and the stability of the system. For example, the kernel may determine the time an application is operating on a certain portion of hardware.

The kernel layer includes hardware-closely related programs, such as interrupt handlers, device drivers, etc., basic, common, higher frequency of operation modules, such as clock management modules, process scheduling modules, etc., and critical data structures. The kernel layer may be provided in the processor or cured in an internal memory.

Referring to fig. 4, a flowchart of a method for processing a drawing message according to an embodiment of the present application is shown. The method is applied to the electronic equipment, and the method for processing the drawing message comprises the following steps:

s101, marking the perceptible message in advance.

In one embodiment of the present application, pre-tagging the perceptible message includes: and adding a perceptible mark to the preset message through a mark interface, so that the preset message is actively marked as a perceptible message. The active marks comprise a system active mark and an application active mark. The object oriented programming language may perform active tagging of perceptible messages through a tagging interface (marker interfaces), e.g., a tagging interface in JAVA may include a serizable.

In an embodiment of the present application, the preset message includes, but is not limited to, a vsync drawing message, an input message, a message for updating interface content, a display configuration change (configChange) message. Wherein the vsync draw message is a message generated in response to the vsync signal before the application performs interface drawing. The input message is a message generated in response to an input event, wherein the input event is an event generated when a user performs an input operation on the electronic device, for example, the input operation is a touch operation. The message to update the content of the interface is a message generated in response to a change in the content of the display interface, such as a change in text or images of the display interface. The display configuration change message includes, but is not limited to, a size change message, a coordinate change message, a display attribute change message of a window (window) or view (view) of a display interface, where the display configuration change message is a message generated in response to a configuration change of a display, for example, the configuration change of the display includes switching a horizontal display to a vertical display, switching the vertical display to a horizontal display, switching the full screen display to a half screen display, switching the half screen display to the full screen display, and the like.

In an embodiment of the present application, the system may add a perceptible mark to the vsync drawing message, the input message, the message for updating the interface content, and the display configuration change message in advance through the mark interface, so as to actively mark the vsync drawing message, the input message, the message for updating the interface content, and the display configuration change to a perceptible message.

In one embodiment of the present application, the application layer 101 includes a message handling module (Handler) 1011 for performing the retrieval, distribution, processing, and delivery of messages. When an application program sends a message to a main thread through a message processing module, if the message is confirmed to be processed, the content of a display interface can be updated, and a perceptible mark is added to the message through a mark interface, so that the message is actively marked as a perceptible message.

In one embodiment of the present application, for messages that are not actively tagged, one message may be automatically tagged as a perceptible message by self-learning. The refinement process of automatically marking a message as a perceptible message by self-learning is shown in fig. 5, and specifically includes:

s201, obtaining the message to be processed from the message queue.

In an embodiment of the present application, obtaining a message to be processed from a message queue includes: the message to be processed is obtained through the loopofunge function current message queue of the inputread thread of the input system.

S202, judging whether the acquired message is a main thread message. If the acquired message is the main thread message, the flow goes to S203; if the acquired message is not the main thread message, the flow proceeds to S204.

In an embodiment of the present application, when an application program processes a perceptible message, it needs to execute a drawing flow, so as to refresh a display interface, where the drawing flow of the application program is executed only in a main thread, so when it is identified whether an acquired message is a perceptible message, it is first determined whether the message is a loop message in the main thread.

S203, clearing the perceivable flag bit of the message.

In an embodiment of the present application, in order to avoid that the current perceivable flag bit of the message affects the self-learning result, the perceivable flag bit of the message is cleared in advance.

And S204, distributing and processing the message.

S205, it is determined whether the message triggers a drawing (invalidate) request. If the message triggers a drawing request, the flow goes to S206; if the message does not trigger the drawing request, the flow proceeds to S211.

In one embodiment of the present application, invalididate is a function used to invalidate the current window of the display interface and inform the system to redraw the window.

S206, setting a perceivable flag bit for the message.

S207, judging whether the message is a main thread message. If the message is a main thread message, the flow goes to S208; if the message is not a main thread message, the flow proceeds to S211.

S208, judging whether the perceivable flag bit of the message is set. If the perceivable flag bit of the message is set, the flow proceeds to S209; if the perceptible flag bit of the message is not set, the flow proceeds to S211.

S209, marking the message as a perceptible message, and recording the association information of the message.

In an embodiment of the present application, the message is marked as a perceptible message through a marking interface. Related messages of the message include, but are not limited to, a Package name (Package name), version, and message number (ID) of the application that generated the message, where the Package name is a unique identification of the application.

S210, clearing the perceivable flag bit of the message.

S211, recycling the message, releasing the resources occupied by the message, and returning to S201 to process the next message.

The flow of the method for self-learning automatic tagging message shown in fig. 5 can be applied to the electronic device and the cloud server in the architecture shown in fig. 6. Specifically, referring to fig. 6, the electronic device 100 may be communicatively connected to the cloud server 200 through a network, and the cloud server 200 may be communicatively connected to the testing machine 300 through a network, and the testing machine 300 may be a laboratory testing machine.

Referring to fig. 7, in an embodiment of the present application, after the electronic device 100 marks the perceivable message through self-learning identification, the associated information of the marked perceivable message may be uploaded to the cloud server 200, and the cloud server 200 may receive the associated information of the perceivable message uploaded by the electronic devices of multiple users. The electronic device 100 may also download the self-learning data of the perceivable message from the cloud server 200 to obtain the associated information of the perceivable message uploaded by other electronic devices. The self-learning data of the perceptible message by the electronic device 100 may be transferred to another terminal device by cloning, wherein cloning is a terminal device transferring all data to another terminal device.

In an embodiment of the present application, the electronic device 100 may be preset with self-learning data for the perceivable message, and the preset self-learning data is from the tester. When the electronic device 100 is first powered on after leaving the factory or is powered on after restoring the factory setting, the preset self-learning data may be applied to obtain a preset perceivable message, and the updated self-learning data may also be downloaded from the cloud server 200.

In an embodiment of the present application, habit information of a user using an application program may be counted at intervals of a preset period, the habit information of the user using the application program is uploaded to a cloud server, and self-learning data of a perceptible message corresponding to the habit information of the user and issued by the cloud server is received.

In an embodiment of the present application, the habit information of using the application program by the user includes, but is not limited to, names of a first preset number of application programs with the largest number of use times in a preset period, and a second preset number of scenes with the largest number of use times of each application program in the first preset number of application programs. Illustratively, the preset period is one month and the first preset number and the second preset number are both 3. For example, three application names with the largest number of uses in one month may be an instant messaging program, a browser, and a camera, respectively, and three scenes with the largest number of uses of the instant messaging program may be a message interaction, a voice interaction, and a video interaction, respectively.

Specifically, habit information of the user using the application program, such as the number of times each application program is used, the duration of use, the last time of use, the use scenario, etc., is counted by an application program monitoring service (usacestatsmanager). After habit information of using an application program by a user in a preset period is uploaded to a cloud server, the cloud server judges whether the currently uploaded habit information is the same as the habit information uploaded last time. If the cloud server determines that the currently uploaded habit information is different from the habit information uploaded last time, the cloud server sends updated habit information to a testing machine (for example, the testing machine 300 in fig. 7) to trigger the testing machine to run an application program corresponding to the updated habit information, marks a perceptible message of the application program through self-learning identification, and uploads self-learning data of the perceptible message to the cloud server (for example, the cloud server 200 in fig. 7), and the cloud server further downloads the self-learning data of the perceptible message to an electronic device (for example, the electronic device 100 in fig. 7) of a user uploading the habit information, so as to update the perceptible message in the electronic device. Wherein the self-learning data includes associated information of the perceptible message.

For example, the electronic device uploads three application program names with the largest use times in a preset period to the cloud server, if the three application program names with the largest use times in the preset period uploaded by the electronic device are the instant messaging program, the browser and the camera, the cloud server determines that the habit information uploaded by the electronic device currently is different from the habit information uploaded by the electronic device last time, the updated application program name map is sent to the testing machine, the testing machine runs the map application program, the perceptible information of the map application program is marked through self-learning, the self-learning data of the perceptible information is uploaded to the cloud server, and the cloud server further downloads the self-learning data of the perceptible information to the electronic device, so that the perceptible information in the electronic device is updated.

In one embodiment of the present application, automatically marking a message as a perceptible message by self-learning identification further includes: if a message is marked as a perceptible message, the drawing process is not triggered in the process of processing the message, and the message is adjusted to be an imperceptible message. Wherein the message may be adapted to be an imperceptible message by removing a perceptible marking of the message.

S102, responding to a drawing request instruction of an application program, and requesting a drawing signal.

In an embodiment of the present application, a rendering signal may be requested from a rendering service. The hardware abstraction layer 104 includes a rendering service (surfeflinger) 1041, and the rendering signal is a vsync signal. In response to a draw request instruction of an application program, requesting a draw signal from a draw rendering service includes: in response to an operation instruction generated by a display interface change, the application program generates a drawing request instruction, and in response to the drawing request instruction, transmits the drawing request instruction to a drawing rendering service to request a drawing signal from the drawing rendering service.

In an embodiment of the present application, generating, by the application program, a drawing request instruction in response to an operation instruction generated by a display interface change includes: and responding to the display interface change triggered by the touch operation of the user, generating an operation instruction by the application program, and responding to the operation instruction, and generating a drawing request instruction by the application program.

For example, the touch operation of the user may include an operation of clicking an application icon to open an application, an operation of scrolling the application interface by sliding the display screen up and down, and an operation of switching the application interface by sliding the display screen left and right. The touch operation can trigger the display interface to change, the application program responds to the touch operation to generate the operation instruction, and the drawing request instruction is generated according to the operation instruction. When the display interface is changed, the changed display interface needs to be presented to the user through the display screen, that is, the application program needs to perform drawing rendering on the changed display interface, and in order to trigger the drawing rendering flow, the drawing request instruction needs to be generated to request a drawing signal.

In another embodiment of the present application, generating the drawing request instruction by the application program in response to the operation instruction generated by the display interface change includes: and responding to the automatically triggered display interface change, generating an operation instruction by the application program, and responding to the operation instruction, and generating a drawing request instruction by the application program. It will be appreciated that the display interface change may be triggered automatically when the display interface contains automatically switched or scrolled content, such as when a user plays a video or game using an electronic device.

And S103, generating a drawing message based on the received drawing signal.

In one embodiment of the present application, a rendering message is generated after receiving a rendering signal sent by a rendering service. After the drawing rendering service receives the drawing request instruction, initializing a hardware abstraction layer, generating drawing signals by the hardware abstraction layer and returning the drawing signals to the drawing rendering service, sending the drawing signals to an application program by the drawing rendering service, and generating drawing information after receiving the drawing signals sent by the drawing rendering service by the application program. Wherein the drawing message is a vsync drawing message.

S104, identifying whether the message queue of the current application program main thread contains at least one perceptible message. If the message queue of the current application program main thread contains at least one perceptible message, the flow enters S105; if the message queue of the current application main thread does not contain the perceivable message, the flow proceeds to S106.

In one embodiment of the present application, identifying whether the message queue of the current application thread contains at least one perceptible message includes: identifying whether each message in the message queue is a pre-marked perceptible message; if at least one message in the message queue is a pre-marked perceptible message, determining that the message queue contains at least one perceptible message; if all the messages in the message queue are not pre-marked perceptible messages, determining that the message queue does not contain perceptible messages.

S105, inserting the drawing message into the last perceivable message in the message queue, and carrying out priority processing on the drawing message.

Referring to fig. 8A, for example, the message queue of the current application main thread includes a perceptible message msg2 and a perceptible message msg4, where the message msg4 is the last perceptible message. Referring to fig. 8B, after the vsync drawing message is inserted into the message msg4, the vsync drawing message can be preferentially processed after the perceptible message msg4 is processed, so that the normal refreshing of the display interface of the existing perceptible message in the message queue is not affected, the vsync drawing message can be preferentially processed, and the frame loss of the display screen caused by the processing timeout of the vsync drawing message is avoided.

It will be appreciated that if only one perceptible message is included in the message queue of the current application thread, then the vsync draw message is inserted after the perceptible message.

S106, inserting the drawing message before the first message in the message queue, and carrying out priority processing on the drawing message.

Referring to FIG. 9A, for example, the message queue of the current application thread does not contain perceptible messages. Referring to fig. 9B, the vsync draw message may be inserted into the header of the message queue of the application main thread, that is, before the first message msg1, and the wakeup message loop (message loop) immediately performs the first priority processing on the vsync draw message, so as to timely perform drawing rendering and display on the next frame display interface.

In one embodiment of the present application, the kernel layer 105 includes a display driver 1051. The process of processing the drawing message is a process of drawing, rendering and displaying a next frame of display interface, and comprises the following steps: when a vsync signal is received, an application program generates and processes a vsync drawing message, measures the size of each view in a window to be displayed, performs layout and drawing on each view, then transmits each drawn view to a drawing rendering service, synthesizes each drawn view by the drawing rendering service, forms data of a next frame display interface and stores the data in a frame buffer, and a display driver acquires the data of the next frame display interface from the frame buffer, and drives a display screen to display the next frame display interface based on the acquired data, thereby finishing refreshing of the display interface.

Referring to fig. 10, a flowchart of a method for processing a drawing message according to another embodiment of the present application is shown. The method is applied to the electronic equipment, and the method for processing the drawing message comprises the following steps:

s301, the perceptible message is marked in advance.

S302, a drawing signal is requested in response to a drawing request instruction of an application program.

S303, based on the received drawing signal, a drawing message is generated.

S304, judging whether the drawing message is expired in real time. If the drawing message is not expired, the flow proceeds to S308; if the drawing message has expired, the flow advances to S305.

In the further embodiment of the present application, referring to fig. 11, each message in the message queue has an expected processing time Dtime (dispatch event time), which is the time that the message is expected to be processed for distribution, and the reference to the expected processing time is a millisecond (ms) time counted from the time the electronic device is powered on. For example, in fig. 11, the expected processing time of the message msg1 is 100ms, i.e., the expected processing time of the message msg1 is 100ms after the electronic device is turned on, and the expected processing time of the vsync drawing message is 130ms, i.e., the expected processing time of the vsync drawing message is 130ms after the electronic device is turned on.

In the other embodiment of the present application, determining in real time whether the drawing message has expired includes: judging whether the timing time of the electronic equipment after being started exceeds the expected processing time of the drawing message in real time, and if the timing time of the electronic equipment after being started exceeds the expected processing time of the drawing message, determining that the drawing message has expired; and if the timing time after the electronic equipment is started does not exceed the expected processing time of the drawing message, determining that the drawing message is not expired. For example, if the timing time after the electronic device is started is 120ms and the expected processing time of the vsync drawing message is 130ms, the timing time after the electronic device is started does not exceed the expected processing time of the drawing message, and it is determined that the drawing message is not expired. For another example, if the timing time after the electronic device is started is 150ms and the expected processing time of the vsync drawing message is 130ms, the timing time after the electronic device is started exceeds the expected processing time of the drawing message, and it is determined that the drawing message has expired.

S305, identifying whether at least one perceptible message is contained in a message queue of a current application program main thread. If the message queue of the current application program main thread contains at least one perceptible message, the flow goes to S306; if the message queue of the current application main thread does not contain the perceivable message, the flow proceeds to S307.

S306, inserting the drawing message into the last perceivable message in the message queue, and carrying out priority processing on the drawing message.

S307, inserting the drawing message before the first message in the message queue, and carrying out priority processing on the drawing message.

And S308, adding the drawing message to a message queue of the current application program main thread according to the time sequence, and processing the drawing message according to a default flow.

In an embodiment of the present application, the drawn message is added to the end of the message queue of the current application main thread according to the chronological order sent to the message queue, and the message in the message queue is processed according to the default flow.

The specific embodiments of S301-S303 are the same as the specific embodiments of S101-S103, and the specific embodiments of S305-S307 are the same as the specific embodiments of S104-S106, and are not described here again.

In the other embodiment of the present application, in order to ensure that the system of the electronic device operates normally, the message in the message queue of the main thread of the application program has the expected processing time. It will be appreciated that if the expected processing time has expired, the message is still unprocessed, and an exception may occur to the system. For example, if the refresh rate of the display screen is 60Hz, in order to avoid frame loss of the display interface, the time interval between two consecutive drawing messages needs to be less than or equal to 16.67ms. Therefore, if the drawing message does not expire, the message can be processed according to the default flow, and if the drawing message expires, the drawing message needs to be dequeued and processed preferentially in order to avoid frame loss.

Referring to fig. 12, a flowchart of a method for processing a drawing message according to another embodiment of the present application is shown. The method is applied to the electronic equipment, and the method for processing the drawing message comprises the following steps:

s401, marking the perceivable messages in advance.

S402, in response to a drawing request instruction of the application program, a drawing signal is requested.

S403, based on the received drawing signal, adding the barrier message to a message queue of the current application program main thread.

In one embodiment of the present application, barrier messages are inserted to the end of a message queue according to chronological order. For example, referring to fig. 13A, before inserting the barrier message, message msg6 is the last message, after inserting the barrier message into message msg6, message msg7 and message msg8 are messages generated after inserting the barrier message.

In an embodiment of the present application, a barrier message (barrier) may be inserted to intercept a synchronous message, so that a subsequent asynchronous message is executed first, where the vsync drawing message and the input message are asynchronous messages, and when the barrier message is processed, the asynchronous message may be preferentially processed. Therefore, when the application program needs to draw, the subsequent vsync drawing messages can be prioritized by inserting the barrier message.

S404, generating a drawing message.

S405, it is determined in real time whether the drawing message has expired. If the drawing message is not expired, the flow proceeds to S409; if the drawing message has expired, the flow advances to S406.

S406, identifying whether the message queue of the current application main thread contains at least one perceptible message before the barrier message. If the message queue of the current application main thread contains at least one perceivable message before the barrier message, the flow proceeds to S407; if the message queue of the current application main thread does not include the perceivable message before the barrier message, the flow proceeds to S408.

In one embodiment of the present application, identifying whether the message queue of the current application main thread contains at least one perceptible message before the barrier message includes: identifying whether each message of the message queue of the current application main thread preceding the barrier message is a pre-marked perceptible message; if at least one message of the message queue of the current application main thread before the barrier message is a pre-marked perceptible message, determining that the message queue of the current application main thread contains at least one perceptible message before the barrier message; if all the messages in the message queue of the current application main thread before the barrier message are not the pre-marked perceptible messages, determining that the message queue of the current application main thread does not contain perceptible messages before the barrier message.

S407, inserting the drawing message into the message queue after the last perceptible message before the barrier message, and carrying out priority processing on the drawing message.

As shown in fig. 13A, for example, the current application main thread's message queue contains a perceptible message msg2 and a perceptible message msg4 before the barrier message, where message msg4 is the last perceptible message before the barrier message. Referring to fig. 13B, after the vsync drawing message is inserted into the message msg4, the vsync drawing message can be preferentially processed after the perceptible message msg4 is processed, so that normal refreshing of the display interface of the existing perceptible message in the message queue is not affected, and the vsync drawing message can be preferentially processed, so that frame loss of the display screen caused by processing of the vsync drawing message is avoided.

It will be appreciated that if only one perceptible message is included in the message queue of the current application thread before the barrier message, then the vsync draw message is inserted after the perceptible message.

S408, inserting the drawing message before the first message in the message queue, and carrying out priority processing on the drawing message.

Referring to fig. 14A, for example, the message queue of the current application main thread does not include a perceptible message before the barrier message. Referring to fig. 14B, the vsync draw message may be inserted before the first message msg1, and a wakeup message loop (message loop) immediately prioritizes the first bit of the vsync draw message.

S409, inserting the drawing information into the message queue of the current application program main thread according to the time sequence, and processing the drawing information according to the default flow.

In the other embodiment of the present application, due to the existence of the barrier message, the default flow is to intercept the synchronous message and preferentially process the asynchronous message when the barrier message is processed, so that the drawing message can be preferentially processed.

In the further embodiment of the present application, the method further comprises: after the rendering message processing is completed, barrier messages are removed from the message queue. It will be appreciated that if the rendering message processing is complete, there is currently no asynchronous message that requires priority processing by dequeuing, and that barrier messages are removed from the message queue in order to ensure proper processing of synchronous messages.

Referring to fig. 15, a flowchart of a method for processing a drawing message according to another embodiment of the present application is shown. The method is applied to the electronic equipment, and the method for processing the drawing message comprises the following steps:

s501, pre-marking the perceptible message.

S502, in response to a drawing request instruction of an application program, a drawing signal is requested.

And S503, adding the barrier message to a message queue of the current application program main thread based on the received drawing signal.

In one embodiment of the present application, barrier messages are inserted to the end of a message queue according to chronological order.

S504, a drawing message is generated.

S505, judging whether the drawing message is expired in real time. If the drawing message is not expired, the flow proceeds to S509; if the draw message has expired, the flow proceeds to S506.

S506, identifying whether the message queue of the current application main thread contains at least one perceptible message before the barrier message. If the message queue of the current application main thread contains at least one perceivable message before the barrier message, the flow proceeds to S507; if the message queue of the current application main thread does not contain a perceptible message before the barrier message, the flow proceeds to S508.

S507, inserting the barrier message into the message queue after the last perceptible message before the barrier message, and prioritizing the barrier message.

As shown in fig. 16A, for example, the current application main thread's message queue contains a perceptible message msg2 and a perceptible message msg4 before the barrier message, where message msg4 is the last perceptible message before the barrier message. Referring to fig. 16B, after the barrier message is inserted into the message msg4, the barrier message can be preferentially processed after the perceptible message msg4 is processed, and when the barrier message is processed, the vsync drawing message can be preferentially processed as an asynchronous message, so that the normal refreshing of the display interface of the perceptible message in the message queue is not affected, the vsync drawing message can be preferentially processed, and the frame loss of the display screen caused by the processing timeout of the vsync drawing message is avoided.

It will be appreciated that if only one perceptible message is included in the message queue of the current application thread before the barrier message, then the barrier message is inserted after the perceptible message.

S508, inserting the barrier message before the first message in the message queue, and carrying out priority processing on the barrier message.

Referring to FIG. 17A, for example, the message queue of the current application main thread does not contain a perceptible message before the barrier message. Referring to fig. 17B, a barrier message may be inserted before the first message msg1, and a wakeup message loop (message loop) may immediately prioritize the barrier message, and when the barrier message is processed, the vsync draw message may be prioritized as an asynchronous message.

S509, inserting the drawing message into a message queue of the current application program main thread according to the time sequence, and processing the drawing message according to a default flow.

Referring to fig. 18, a flowchart of a method for processing a drawing message according to another embodiment of the present application is shown. The method is applied to the electronic equipment, and the method for processing the drawing message comprises the following steps:

s601, pre-marking the perceptible message.

S602, in response to a drawing request instruction of an application program, a drawing signal is requested.

S603, inserting the barrier message into a message queue of the current application program main thread based on the received drawing signal.

S604, a drawing message is generated.

S605 determines in real time whether the drawing message has expired. If the drawing message is not expired, the flow proceeds to S611; if the draw message has expired, the flow proceeds to S606.

S606, it is identified whether the message queue of the current application main thread contains at least one perceptible message before the barrier message. If the message queue of the current application main thread contains at least one perceivable message before the barrier message, the flow proceeds to S607; if the message queue of the current application main thread does not contain a perceptible message before the barrier message, the flow proceeds to S608.

S607, inserting the drawing message into the message queue after the last perceivable message before the barrier message, and carrying out priority processing on the drawing message.

S608, it is determined whether the message queue of the current application main thread contains at least one message with a delay processing time greater than or equal to a threshold time before the barrier message. If the message queue of the current application main thread contains at least one message with a delay processing time greater than or equal to the threshold time before the barrier message, the flow proceeds to S609; if the message queue of the current application main thread does not include a message with a delay processing time greater than or equal to the threshold time before the barrier message, the flow proceeds to S610.

In an embodiment of the present application, the threshold time is a starvation time, and if the delay processing time of the message exceeds the starvation time, it indicates that the delay processing time of the message is longer, and priority processing is required. The threshold time is expressed as Tn, tn is a preset value, and the cloud server can issue update and can conduct differentiated setting according to different application programs.

In an embodiment of the present application, each message in the message queue has a VInserted field for indicating whether the message is enqueued by a vsync drawn message and a Tvdelay field for indicating a delay time generated after the message is enqueued by the vsync drawn message. If the time consumed for processing the enqueue vsync drawn message is Tv, updating Tvdelay fields of all the enqueued messages, namely TvdelayTv, when the vsync drawn message is processed ₁ +Tv ₂ +…+Tv _n 。

S609, inserting the drawing message into a message queue after the last message with delay processing time greater than or equal to the threshold time before the barrier message, and carrying out priority processing on the drawing message.

In an embodiment of the present application, when generating the vsync drawn message, if there is a perceptible message before the barrier message, inserting the vsync drawn message after the perceptible message, if there is no perceptible message before the barrier message, and if the Tvdelay field of the inserted queue message is greater than or equal to the starvation time, that is, tvdelay is greater than or equal to Tn, inserting the vsync drawn message after the last starvation message.

And S610, inserting the drawing message before the first message in the message queue, and carrying out priority processing on the drawing message.

In an embodiment of the present application, if the Tvdelay field of the enqueued message is less than the starvation time, i.e., tvdelay < Tn, then the vsync draw message is inserted into the head of the message queue.

S611, inserting the drawing message into a message queue of the current application program main thread according to the time sequence, and processing the drawing message according to a default flow.

Referring to fig. 19, a flowchart of a method for processing a drawing message according to another embodiment of the present application is shown. The method is applied to the electronic equipment, and the method for processing the drawing message comprises the following steps:

s701, pre-marking the perceptible message.

S702, responding to a drawing request instruction of an application program, and requesting a drawing signal.

S703, inserting the barrier message into a message queue of the current application program main thread based on the received drawing signal.

S704, a drawing message is generated.

S705, it is determined in real time whether the drawing message has expired. If the drawing message is not expired, the flow proceeds to S511; if the draw message has expired, the flow proceeds to S506.

S706, it is identified whether the message queue of the current application main thread contains at least one perceptible message before the barrier message. If the message queue of the current application main thread contains at least one perceivable message before the barrier message, the flow proceeds to S707; if the message queue of the current application main thread does not contain a perceptible message before the barrier message, the flow proceeds to S708.

S707, inserting the barrier message into the message queue after the last perceivable message before the barrier message, and prioritizing the barrier message.

S708, judging whether the message queue of the current application program main thread contains at least one message with delay processing time greater than or equal to the threshold time before the barrier message. If the message queue of the current application main thread contains at least one message with delay processing time greater than or equal to the threshold time before the barrier message, the flow proceeds to S709; if the message queue of the current application main thread does not include a message with a delay processing time greater than or equal to the threshold time before the barrier message, the flow proceeds to S710.

S709, inserting the barrier message into the message queue, and performing priority processing on the barrier message after the last message with delay processing time greater than or equal to the threshold time before the barrier message.

In an embodiment of the present application, when the vsync drawing message expires, if there is a perceptible message before the barrier message, after the barrier message is inserted into the perceptible message, if there is no perceptible message before the barrier message, if the Tvdelay field of the inserted queue message is greater than or equal to the starvation time, that is, tvdelay is greater than or equal to Tn, the barrier message is inserted into the last starvation message, and the barrier message is preferentially processed.

S710, inserting the barrier message before the first message in the message queue, and carrying out priority processing on the barrier message.

And S711, inserting the drawing message into a message queue of the main thread of the current application program according to the time sequence, and processing the drawing message according to a default flow.

In another embodiment of the present application, starvation message monitoring may also be applied to the method for processing drawn messages in the embodiment shown in fig. 4, after S104, if the message queue of the current application main thread does not include a perceptible message, determining whether the message queue of the current application main thread includes at least one message with a delay processing time greater than or equal to a threshold time, if the message queue of the current application main thread includes at least one message with a delay processing time greater than or equal to the threshold time, inserting the drawn message after the last message with a delay processing time greater than or equal to the threshold time in the message queue; if the message queue of the current application main thread does not contain the message with the delay processing time greater than or equal to the threshold time, the flow proceeds to S106.

In another embodiment of the present application, starvation message monitoring may also be applied to the method for processing drawn messages in the embodiment shown in fig. 10, after S305, if the message queue of the current application main thread does not include a perceptible message, determining whether the message queue of the current application main thread includes at least one message with a delay processing time greater than or equal to a threshold time, if the message queue of the current application main thread includes at least one message with a delay processing time greater than or equal to the threshold time, inserting the drawn message after the last message with a delay processing time greater than or equal to the threshold time in the message queue; if the message queue of the current application main thread does not include the message with the delay processing time greater than or equal to the threshold time, the flow proceeds to S307.

Fig. 20 is a schematic diagram of a functional module of an electronic device according to an embodiment of the present application. The electronic device 100 comprises at least a perceptible message identification module 11, a message managing module 12 and a message processing module 13. The perceivable message recognition module 11 is used for recognizing a User Interface (UI) thread message of an application program, and recognizing a message perceivable by a User, wherein the recognition mode comprises active marking and automatic recognition. The message management module 12 includes vsync priority handling, perceptible asynchronous message priority handling, and message starvation monitoring. The message processing module 13 is configured to process a message according to message monitoring of the message management module 12, and includes: the method can perform the queue inserting processing on the vsync drawn message, avoid that the vsync drawn message is not processed in time because of waiting for the imperceptible message for a long time, and insert the vsync drawn message into the queue head to perform the first queue inserting processing if the perceptible message is not before the barrier message in the message queue. If the barrier message is preceded by a perceptible message, inserting the vsync drawn message after the last perceptible message for priority queue insertion processing, and simultaneously avoiding that other messages cannot be processed for a long time due to queue insertion of the vsync drawn message through message starvation monitoring.

Referring to fig. 21, the electronic device 100 may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an Ultra-mobile personal computer (Ultra-mobile Personal Computer, UMPC), a netbook, a cellular phone, a personal digital assistant (Personal Digital Assistant, PDA), an augmented Reality (Augmented Reality, AR) device, a Virtual Reality (VR) device, an artificial intelligence (Artificial Intelligence, AI) device, a wearable device, a vehicle-mounted device, a smart home device, and/or a smart city device, and the specific type of the electronic device 100 is not particularly limited in the embodiments of the present application.

The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (Universal Serial Bus, USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, keys 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identity module (Subscriber Identification Module, SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

It should be understood that the illustrated structure of the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, electronic device 100 may include more or fewer components than shown, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, such as: the processor 110 may include an application processor (Application Processor, AP), a modem processor, a graphics processor (Graphics Processing Unit, GPU), an image signal processor (Image Signal Processor, ISP), a controller, a video codec, a digital signal processor (Digital Signal Processor, DSP), a baseband processor, and/or a Neural network processor (Neural-network Processing Unit, NPU), etc. Wherein the different processing units may be separate devices or may be integrated in one or more processors.

The controller can generate operation control signals according to the instruction operation codes and the time sequence signals to finish the control of instruction fetching and instruction execution.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to reuse the instruction or data, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby improving the efficiency of the system.

In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an integrated circuit (Inter-integrated Circuit, I2C) interface, an integrated circuit built-in audio (Inter-integrated Circuit Sound, I2S) interface, a pulse code modulation (Pulse Code Modulation, PCM) interface, a universal asynchronous receiver transmitter (universal asynchronous receiver/transmitter, UART) interface, a mobile industry processor interface (Mobile Industry Processor Interface, MIPI), a General-Purpose Input/Output (GPIO) interface, a subscriber identity module (Subscriber Identity Module, SIM) interface, and/or a universal serial bus (Universal Serial Bus, USB) interface, among others.

The I2C interface is a bi-directional synchronous Serial bus, comprising a Serial Data Line (SDA) and a Serial clock Line (Derail Clock Line, SCL). In some embodiments, the processor 110 may contain multiple sets of I2C buses. The processor 110 may be coupled to the touch sensor 180K, charger, flash, camera 193, etc., respectively, through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through an I2C interface, such that the processor 110 communicates with the touch sensor 180K through an I2C bus interface to implement a touch function of the electronic device 100.

The I2S interface may be used for audio communication.

PCM interfaces may also be used for audio communication to sample, quantize and encode analog signals.

The UART interface is a universal serial data bus for asynchronous communications. The bus may be a bi-directional communication bus. It converts the data to be transmitted between serial communication and parallel communication.

The MIPI interface may be used to connect the processor 110 to peripheral devices such as a display 194, a camera 193, and the like. The MIPI interfaces include camera serial interfaces (Camera Serial Interface, CSI), display serial interfaces (Display Serial Interface, DSI), and the like. In some embodiments, processor 110 and display 194 communicate via a DSI interface to implement the display functionality of electronic device 100.

The GPIO interface may be configured by software. The GPIO interface may be configured as a control signal or as a data signal.

The USB interface 130 is an interface conforming to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like.

It should be understood that the interfacing relationship between the modules illustrated in the embodiments of the present invention is only illustrative, and is not meant to limit the structure of the electronic device 100. In other embodiments of the present application, the electronic device 100 may also use different interfacing manners, or a combination of multiple interfacing manners in the foregoing embodiments.

The electronic device 100 implements display functions through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos, and the like. The display 194 includes a display panel. The display panel may employ a liquid crystal display (Liquid Crystal Display, LCD), an Organic Light-Emitting Diode (OLED), an Active-matrix Organic Light-Emitting Diode (AMOLED) or an Active-matrix Organic Light-Emitting Diode (Matrix Organic Light Emitting Diode), a flexible Light-Emitting Diode (Flex), a mini, a Micro-OLED, a quantum dot Light-Emitting Diode (Quantum Dot Light Emitting Diodes, QLED), or the like. In some embodiments, the electronic device 100 may include 1 or N display screens 194, N being a positive integer greater than 1.

The digital signal processor is used for processing digital signals, and can process other digital signals besides digital image signals. For example, when the electronic device 100 selects a frequency bin, the digital signal processor is used to fourier transform the frequency bin energy, or the like.

The internal Memory 121 may include one or more random access memories (Random Access Memory, RAM) and one or more Non-Volatile memories (NVM).

The Random Access Memory may include Static Random-Access Memory (SRAM), dynamic Random-Access Memory (Dynamic Random Access Memory, DRAM), synchronous dynamic Random-Access Memory (Synchronous Dynamic Random Access Memory, SDRAM), double data rate synchronous dynamic Random-Access Memory (Double Data Rate Synchronous Dynamic Random Access Memory, DDR SDRAM, e.g., fifth generation DDR SDRAM is commonly referred to as DDR5 SDRAM), etc.;

the nonvolatile memory may include a disk storage device, a flash memory (flash memory).

The FLASH memory may include NOR FLASH, NAND FLASH, 3d nand FLASH, etc. divided according to an operation principle, may include Single-Level Cell (SLC), multi-Level Cell (MLC), triple-Level Cell (TLC), quad-Level Cell (QLC), etc. divided according to a storage specification, may include universal FLASH memory (Universal Flash Storage, UFS), embedded multimedia memory card (embedded Multi Media Card, eMMC), etc. divided according to a storage specification.

The random access memory may be read directly from and written to by the processor 110, may be used to store executable programs (e.g., machine instructions) for an operating system or other on-the-fly programs, may also be used to store data for users and applications, and the like.

The nonvolatile memory may store executable programs, store data of users and applications, and the like, and may be loaded into the random access memory in advance for the processor 110 to directly read and write.

The external memory interface 120 may be used to connect external non-volatile memory to enable expansion of the memory capabilities of the electronic device 100. The external nonvolatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and video are stored in an external nonvolatile memory.

The internal memory 121 or the external memory interface 120 is used to store one or more computer programs. One or more computer programs are configured to be executed by the processor 110. The one or more computer programs include a plurality of instructions that when executed by the processor 110, implement the method of processing a drawn message executed on the electronic device 100 in the above embodiment to implement the handwriting input display function of the electronic device 100.

The pressure sensor 180A is used to sense a pressure signal, and may convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A may be disposed on the display screen 194. The pressure sensor 180A is of various types, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, and the like. The capacitive pressure sensor may be a capacitive pressure sensor comprising at least two parallel plates with conductive material. The capacitance between the electrodes changes when a force is applied to the pressure sensor 180A. The electronic device 100 determines the strength of the pressure from the change in capacitance. When a touch operation is applied to the display screen 194, the electronic apparatus 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 may also calculate the location of the touch based on the detection signal of the pressure sensor 180A. In some embodiments, touch operations that act on the same touch location, but at different touch operation strengths, may correspond to different operation instructions. For example: and executing an instruction for checking the short message when the touch operation with the touch operation intensity smaller than the first pressure threshold acts on the short message application icon. And executing an instruction for newly creating the short message when the touch operation with the touch operation intensity being greater than or equal to the first pressure threshold acts on the short message application icon.

The touch sensor 180K, also referred to as a "touch device". The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, which is also called a "touch screen". The touch sensor 180K is for detecting a touch operation acting thereon or thereabout. The touch sensor may communicate the detected touch operation to the application processor to determine the touch event type. Visual output related to touch operations may be provided through the display 194. In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 at a different location than the display 194.

The present embodiment also provides a computer storage medium, in which computer instructions are stored, which when executed on the electronic device 100, cause the electronic device 100 to execute the above-mentioned related method steps to implement the method for processing a drawing message in the above-mentioned embodiment.

The present application also provides a computer program product, which when run on a computer, causes the computer to perform the above-mentioned related steps to implement the drawn message processing method in the above-mentioned embodiments.

In addition, embodiments of the present application also provide an apparatus, which may be specifically a chip, a component, or a module, and may include a processor and a memory connected to each other; the memory is used for storing computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory, so that the chip executes the method for processing the drawing message in each method embodiment.

The electronic device, the computer storage medium, the computer program product, or the chip provided in this embodiment are used to execute the corresponding methods provided above, so that the beneficial effects thereof can be referred to the beneficial effects in the corresponding methods provided above, and will not be described herein.

From the foregoing description of the embodiments, it will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of functional modules is illustrated, and in practical application, the above-described functional allocation may be implemented by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to implement all or part of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another apparatus, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and the parts displayed as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed in a plurality of different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated unit may be stored in a readable storage medium if implemented in the form of a software functional unit and sold or used as a stand-alone product. Based on such understanding, the technical solution of the embodiments of the present application may be essentially or a part contributing to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, including several instructions to cause a device (may be a single-chip microcomputer, a chip or the like) or a processor (processor) to perform all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Finally, it should be noted that the above embodiments are merely for illustrating the technical solution of the present application and not for limiting, and although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present application may be modified or substituted without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A method of rendering a message, the method comprising:

responding to a drawing request instruction of an application program, and requesting a drawing signal;

generating a drawing message based on the received drawing signal;

identifying whether a message queue of a main thread of a current application program contains at least one perceptible message, wherein the perceptible message is a message which can enable a display interface to generate visual perception change after being processed;

if the message queue contains at least one perceptible message, inserting the drawing message after the last perceptible message in the message queue, and carrying out priority processing on the drawing message.

2. The drawn message processing method according to claim 1, wherein the method further comprises:

the perceptible message is pre-marked.

3. The drawn message processing method of claim 2, wherein the pre-tagging the perceptible message comprises:

and adding a perceptible mark to a preset message through a mark interface, wherein the preset message comprises at least one of a drawing message, an input message, a message for updating interface content and a display screen configuration change message.

4. The drawn message processing method of claim 2, wherein the pre-tagging the perceptible message comprises:

Obtaining a message to be processed from the message queue, and distributing and processing the message;

judging whether the message triggers a drawing request or not;

if the message triggers the drawing request, setting a perceivable flag bit for the message;

judging whether the message is a main thread message or not;

if the message is the main thread message, judging whether a perceivable flag bit of the message is set;

if the perceivable flag bit of the message is set, marking the message as a perceivable message, and recording the associated information of the message, wherein the associated information comprises the package name, the version and the message number of the application program generating the message.

5. The drawn message processing method of claim 4, wherein the obtaining a message to be processed from the message queue and distributing and processing the message comprises:

acquiring a message to be processed from the message queue, and judging whether the message is a main thread message or not;

if the message is a main line Cheng Xiaoxi, clearing the perceivable flag bit of the message and distributing and processing the message.

6. The drawn message processing method of claim 4, wherein the pre-tagging the perceptible message further comprises:

After marking the message as a perceptible message, clearing a perceptible flag bit of the message;

and recovering the message and releasing the resources occupied by the message when the message does not trigger a drawing request, or the message is not the main thread message, or the perceivable flag bit of the message is not set.

7. The method of any of claims 2 to 4, wherein identifying whether the message queue of the current application main thread contains at least one perceptible message comprises:

identifying whether each message in the message queue is a pre-marked perceptible message;

if at least one message in the message queue is the pre-marked perceptible message, determining that the message queue contains at least one perceptible message; or (b)

And if all the messages in the message queue are not the pre-marked perceptible messages, determining that the message queue does not contain perceptible messages.

8. The drawn message processing method according to any one of claims 2 to 4, characterized in that the method further comprises:

uploading the pre-marked associated information of the perceivable messages to a cloud server;

And downloading the associated information of the perceivable messages uploaded by at least one terminal device from the cloud server.

9. The drawn message processing method of claim 8, wherein the method further comprises:

counting habit information of using an application program by a user at intervals of a preset period;

uploading the habit information to the cloud server, and receiving association information of the perceivable messages corresponding to the habit information, wherein the habit information comprises names of a first preset number of application programs with the largest use times in the preset period and a second preset number of scenes with the largest use times of each application program in the first preset number of application programs.

10. The drawn message processing method according to claim 1, wherein the method further comprises:

if the message queue does not contain the perceivable message, inserting the drawing message before the first message in the message queue, and carrying out priority processing on the drawing message.

11. The drawn message processing method according to claim 1, wherein the method further comprises:

judging whether the drawing message is expired or not in real time;

If the drawing message has expired, identifying whether the message queue contains at least one perceptible message; or (b)

And if the drawing message does not expire, adding the drawing message to the message queue according to the time sequence, and processing the drawing message according to a default flow.

12. The method of drawing message processing according to claim 11, wherein the real-time determination of whether the drawing message has expired comprises:

judging whether the timing time of the electronic equipment after being started exceeds the expected processing time of the drawing message in real time;

if the timing time of the electronic equipment after being started exceeds the expected processing time of the drawing message, determining that the drawing message is expired; or (b)

And if the timing time of the electronic equipment after being started does not exceed the expected processing time of the drawing message, determining that the drawing message is not expired.

13. The method of claim 11, wherein the generating a draw message based on the received draw signal comprises:

based on the received draw signal, a barrier message is added to the message queue and the draw message is generated.

14. The drawn message processing method of claim 13, wherein the method further comprises:

If the draw message has expired, identifying whether the message queue contains at least one perceptible message prior to the barrier message;

if the message queue contains at least one perceptible message before the barrier message, inserting the drawing message into the message queue after the last perceptible message before the barrier message, and carrying out priority processing on the drawing message; or (b)

If the message queue does not contain the perceivable message before the barrier message, inserting the drawing message before the first message in the message queue, and carrying out priority processing on the drawing message.

15. The drawn message processing method of claim 14, wherein the method further comprises:

if the message queue does not contain perceptible messages before the barrier message, judging whether the message queue contains at least one message with delay processing time greater than or equal to a threshold time before the barrier message;

if the message queue contains at least one message with delay processing time greater than or equal to the threshold time before the barrier message, inserting the drawing message into the message with the last delay processing time greater than or equal to the threshold time in the message queue, and carrying out priority processing on the drawing message; or (b)

If the message queue does not contain the message with the delay processing time being more than or equal to the threshold time, inserting the drawn message before the first message in the message queue, and carrying out priority processing on the drawn message.

16. The drawn message processing method of claim 13, wherein the method further comprises:

if the message queue contains at least one perceptible message before the barrier message, inserting the barrier message into the message queue after the last perceptible message before the barrier message, and carrying out priority processing on the barrier message; or (b)

If the message queue does not contain perceptible messages before the barrier message, inserting the barrier message before the first message in the message queue, and carrying out priority processing on the barrier message.

17. An electronic device, the electronic device comprising a memory and a processor:

wherein the memory is used for storing program instructions;

The processor configured to read and execute the program instructions stored in the memory, which when executed by the processor, cause the electronic device to perform the drawn message processing method according to any one of claims 1 to 16.

18. A computer storage medium storing program instructions which, when run on an electronic device, cause the electronic device to perform the method of drawing message processing according to any one of claims 1 to 16.