CN114924837A

CN114924837A - Data processing method, electronic device and readable storage medium

Info

Publication number: CN114924837A
Application number: CN202210546962.1A
Authority: CN
Inventors: 杨永贵
Original assignee: BOE Technology Group Co Ltd
Current assignee: BOE Technology Group Co Ltd
Priority date: 2022-05-18
Filing date: 2022-05-18
Publication date: 2022-08-19
Also published as: WO2023221822A1

Abstract

The present disclosure provides a data processing method, an electronic device, and a readable storage medium. The data processing method comprises the following steps: the basic function packet receives operation input for executing target operation; calling a target function corresponding to the target operation in an application function package through a data interface by a basic function package, wherein the target function is a function in the application function; and generating a processing result corresponding to the target operation through the target function.

Description

Data processing method, electronic device and readable storage medium

Technical Field

The disclosed embodiments relate to the field of computer technologies, and in particular, to a data processing method, an electronic device, and a readable storage medium.

Background

In order to meet different use requirements, the same application program may need to be used on different operating systems, and the implementation manner of the application program is different for different operating systems, which results in targeted development of the application program for different operating systems.

Disclosure of Invention

The embodiment of the present disclosure provides a data processing method, which is applied to an electronic device, where the electronic device includes a processor and a memory, where the memory stores a basic function package and a data interface, and an application program is further installed on the electronic device, where the application program includes an application function package, the application function package is used to provide an application function corresponding to an application function of the application program, and the basic function package is used to provide a basic function for implementing a preset basic function, where at least part of the application function is implemented based on the basic function;

the method comprises the following steps:

the basic function packet receives operation input for executing target operation;

calling a target function corresponding to the target operation in an application function package through a data interface by a basic function package, wherein the target function is a function in the application function;

generating a processing result corresponding to the target operation through the target function;

the basic function package is developed through a first language, the data interface is developed through a second language, the application function package is developed through a third language, and the first language, the second language and the third language are different program languages respectively.

In some embodiments, the application further comprises a first engine sublayer for invoking a function of an operating system on which the application is installed to execute the processing result;

after generating the processing result corresponding to the target operation through the target function, the method further includes:

the application function packet returns the processing result to the basic function packet, and the basic function packet returns the processing result to the first engine sublayer; or

And the application function packet returns the processing result to the first engine sublayer.

In some embodiments, the application program further comprises a second engine sublayer, the second engine sublayer to establish a virtual machine, the virtual machine to generate the processing result according to the objective function.

In some embodiments, the application further comprises an interactive communication sub-layer corresponding to the second language, the interactive communication sub-layer comprising the data interface, the first engine sub-layer and the second engine sub-layer in data communication through the interactive communication sub-layer.

In some embodiments, the first engine sublayer is a Flutter engine sublayer, the second engine sublayer is a Java engine sublayer, and the interactive communication sublayer includes a C-function communication sublayer.

In some embodiments, the objective function comprises a first objective function comprising Java's classes and methods, the C-function communication sublayer comprises a first interface function, and the virtual machine comprises a JVM virtual machine;

before generating the processing result corresponding to the target operation through the target function, the method further includes:

the basic function packet transmits a calling instruction corresponding to the target operation to the Java engine sublayer through the first interface function, wherein the data interface comprises the first interface function, and the data interface is a Dart interface established by the Flutter engine sublayer through an FFI library;

the Java engine sublayer establishes a JVM virtual machine according to the calling instruction;

the JVM virtual machine searches a first target function in the application function package;

the generating, by the objective function, a processing result corresponding to the objective operation includes:

and the JVM virtual machine generates a processing result corresponding to the target operation according to the first target function.

In some embodiments, the objective function further comprises a second objective function, the C-function communication sublayer comprises a second interface function, the second objective function comprises one or more of a C-function library and a drawing library data corresponding to the application function package;

before generating a processing result corresponding to the target operation by the target function, the method further includes:

the Java engine sublayer calls a second target function matched with the first target function through the second interface function, wherein the first target function is realized based on the second target function;

the JVM virtual machine generates a processing result corresponding to the target operation according to the first target function, including:

and obtaining the processing result through the second objective function matched with the first objective function.

In some embodiments, a server is established in the Flutter engine sublayer, a client is established in the Java engine sublayer, and the server and the client perform data communication through the interactive communication sublayer based on a TCP protocol to return the processing result to the Flutter engine sublayer.

In some embodiments, a first callback function is registered in the Flutter engine sublayer, the interactive communication sublayer includes a second callback function established according to the first callback function, and the Java engine sublayer communicates with the Flutter engine sublayer through the second callback function to return the processing result obtained by the second target function to the Flutter engine sublayer.

In some embodiments, the basic function package is in data communication with the interactive communication sublayer through an FFI mechanism, and the application function package is in data communication with the interactive communication sublayer through a JNI mechanism.

In some embodiments, the target operation is a handwriting operation, and the processing result includes a canvas generated according to the handwriting operation and image data of the handwriting operation generated on the canvas.

In some embodiments, after generating the processing result corresponding to the target operation by the target function, the method further includes:

obtaining the type of the target operation, wherein the type of the target operation comprises a first type and a second type, and the display real-time requirement of the first type of target operation is greater than that of the second type of target operation;

executing the processing result in a first mode under the condition that the type of the target operation is the first type, and executing the processing result in a second mode after the target operation is finished;

executing the processing result in a second mode if the type of the target operation is the second type;

the display delay time corresponding to the first mode is less than the display delay time corresponding to the second mode, and the display image quality corresponding to the second mode is less than the display image quality corresponding to the second mode.

In some embodiments, said performing said processing result in the first mode comprises:

sending the canvas and the image data to a graphics display framework;

and rendering the handwriting of the handwriting operation through the graphic display frame.

In some embodiments, the graphics display frame comprises a DRM frame, the rendering the handwriting of the handwriting operation by the graphics display frame comprising:

rendering the canvas and the image data by a LibDRM library;

and sending the rendered canvas and the image data to a buffering frame of the DRM framework.

In some embodiments, said performing said processing result in the second mode comprises:

the first engine sublayer acquires the processing result;

the first engine sublayer renders canvas and image data corresponding to the processing result.

In some embodiments, the application function package is matched to an android system.

In some embodiments, the operating system running the application is the kylin system.

In a second aspect, an embodiment of the present disclosure provides an electronic device, including: a memory, a processor, and a program stored on the memory and executable on the processor; the processor is configured to read a program in the memory to implement the steps in the data processing method according to any one of the first aspect.

In a third aspect, the disclosed embodiments provide a readable storage medium for storing a program, where the program when executed by a processor implements the steps in the data processing method according to any one of the first aspect.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed to be used in the description of the embodiments of the present disclosure will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

FIG. 1 is an architecture diagram of an application in one embodiment of the present disclosure;

FIG. 2 is a flow chart of a data processing method provided by an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of data interaction of an application in an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of yet another interaction of data applied in an embodiment of the present disclosure;

FIG. 5 is a schematic flow chart illustrating interface update in an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of data interaction of image rendering in an embodiment of the present disclosure;

FIG. 7 is a control flow diagram of a DRM device in an embodiment of the disclosure;

FIG. 8 is a flow chart of interface rendering in an embodiment of the present disclosure;

fig. 9 is a schematic structural diagram of an electronic device provided in the present disclosure.

Detailed Description

The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are some, but not all embodiments of the present disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.

The terms "first," "second," and the like in the embodiments of the present disclosure are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Further, as used herein, "and/or" means at least one of the connected objects, e.g., a and/or B and/or C, means 7 cases including a alone, B alone, C alone, and both a and B present, B and C present, both a and C present, and A, B and C present.

The embodiment of the disclosure provides a data processing method.

The technical scheme of the embodiment is applied to electronic equipment, the electronic equipment comprises a processor and a memory, the memory stores a basic function package and a data interface, and the electronic equipment is further provided with an application program, and the application program comprises an application function package.

As shown in FIG. 1, in some of these embodiments, the application program includes an application layer 110 and a framework layer 120.

The application layer 110 includes a first application sublayer 111 developed through a first language, in this embodiment, the first language may be a Dart language (a program development language), the first application sublayer 111 faces to a developer of Flutter (a toolkit for building a user interface), and the developer may write a UI through a Flutter UI (Flutter user interface) to exert its UI rendering advantages.

The application layer 110 further includes a second application sublayer 112 developed by a third language, in this embodiment, the third language may be Java language (a program development language), and the second application sublayer 112 is oriented to Java developers.

The second application sublayer 112 includes an application function package for providing an application function corresponding to an application function of an application program. In some embodiments, the application layer 110 further includes a basic function package for providing basic functions that implement preset basic functions.

The basic function package is a function package for realizing a system function interface, and the system function may be: touch control function, interface function packet of communication function such as bluetooth, etc.

The basic function package comprises an interface function for realizing interaction between the application function package and the bottom layer of the electronic equipment, and the basic function package can further comprise a packaging function for some data structure bodies, for example, touch data sent by the bottom layer is converted into a data structure which can be identified by the application function package.

In this embodiment, the application layer 110 corresponds to specific service requirements of the application program, and the user can input the service requirements provided by the application program based on corresponding operations, so that the application layer 110 can understand that one port is provided, and the user can implement different service requirements by operating inputs for different ports.

In this embodiment, the application program may be applied to different operating systems, and in different operating systems, the service logic is the same, and in implementation, the service logic part in the application layer 110 is implemented by using Java and C functions, so that when the application program is applied to different operating systems, the data of the application layer 110 is the same, and the development cost and workload of the application program can be reduced.

It should be noted that the C function in this embodiment is developed by a C-type language, and the C-type language may specifically include C + +, C #, and the like, which is not further limited herein.

In some embodiments, the application layer 110 is matched with an Android (Android) system, and it can be understood that the Android system is an operating system with a relatively wide application range, so that the application layer 110 is directly developed for the Android system, and the applicability is stronger, further, when an application program needs to be used on other operating systems, the application layer 110 of the application program can be directly transplanted, and thus, a developed and packaged application function package can be directly utilized, so that the application layer 110 does not need to be developed again, the development cost can be reduced, and the development period can be shortened.

In the related art, the application layer data of the android system is developed based on Java language, but the rendering mechanism of the UI component and the running Dalvik virtual machine CPU instruction are register-based, and some operating systems, such as the kylin system, need to be adapted according to the general purpose register, so that the migration difficulty of the application program of the android system in the related art is relatively large.

In some embodiments, the operating system running the application program may be Linux (an operating system), Window (an operating system), MacOs (an operating system), or the like, and in some embodiments, the operating system running the application program may also be Kylin system (Kylin).

The framework layer 120 of the application program is used for realizing the interaction and support of information. In some embodiments, the framework layer 120 includes a first engine sublayer 121, a second engine sublayer 122, and an interactive communication sublayer 123, the interactive communication sublayer 123 includes a data interface to perform data transfer and transmission, and the first engine sublayer 121 and the second engine sublayer 122 perform data communication through the interactive communication sublayer 123.

It should be noted that the first engine sublayer 121 and the interactive communication sublayer 123 may be developed in the C-class language.

Specifically, in some embodiments, the first engine sublayer 121 is a Flutter engine sublayer, the second engine sublayer 122 is a Java engine sublayer, the interactive communication sublayer 123 includes a data interface, and the interactive communication sublayer 123 may include a C function communication sublayer.

As shown in fig. 1, the interactive communication sublayer 123 is used for implementing functions such as process communication, storage sharing, and application process management.

Referring to fig. 1, the Flutter engine sub-layer mainly includes a Dart virtual machine VM and a rendering engine Render, where the Dart virtual machine VM is used for managing and building the Dart virtual machine so as to run the basic function package through the Dart virtual machine. The Flutter engine sublayer can also realize functions such as platform communication, UI layout, resource management and the like. Further, the Flutter engine sublayer is also used for implementing embedded management of the platform, specifically, management of plug-ins, threads, canvas, and the like.

The Flutter engine sublayer is used to invoke the functionality of the operating system of the installed application to perform the processing results. It can be understood that the Flutter engine sublayer can directly interact with the operating system through the corresponding control instruction to call the function of the operating system to execute the corresponding processing result.

For example, if the executed operation is a handwriting operation, the Flutter engine sublayer may interact with the operating system to finally implement the display function by calling display-related hardware (e.g., a video card, a display panel, etc.) through the operating system.

For another example, if the executed operation is to click a bluetooth switch, the application function package sends the instruction to the basic function package, the basic function package further sends the instruction to the Flutter engine sublayer, the Flutter engine sublayer interacts with the operating system, and the bluetooth module is turned on or turned off through the corresponding bottom driver.

The Java engine sublayer is mainly used for realizing virtual machine management (JavaVM Manager) and coordinated management (Java/C interworking) of C functions and Java functions.

The Java engine sublayer is configured to establish a virtual machine, where the established virtual machine may be a JVM virtual machine, and the virtual machine is configured to generate the processing result according to the objective function.

It should be understood that, the Java functions corresponding to the application function package need to run depending on the JVM virtual machine, and in this embodiment, the Java engine sublayer establishes the JVM virtual machine, so as to obtain corresponding processing results based on the target function through the JVM virtual machine.

In some embodiments, the basic Function package performs data communication with the interactive communication sublayer through a Function Interface (FFI) mechanism, and in particular, is implemented through a Dart FFI Interface (Dart FFI Interface). The application function package performs data communication with the interactive communication sublayer through a JNI (Java Native Interface ) mechanism, and specifically, is implemented through a Java JNI Interface (Java JNI Interface).

The Flutter engine sublayer and the Java engine sublayer serve as an intermediate bridge through the C function communication sublayer, information interaction between the Flutter engine sublayer and the Java engine sublayer is achieved, the purpose that original business logic calls Java functions and the C functions to generate data streams is achieved, and the data streams are provided for a data structure which can be identified by the Flutter engine to perform rendering processing.

In some embodiments, the framework layer 120 also includes an Engine Core (Engine Core)124 that implements the basic functionality of the framework layer 120.

In some embodiments, the runtime environment of the application also includes a bottom layer 130, and the bottom layer 130 refers specifically to the bottom layer 130 of the operating system in which the application is installed.

In one embodiment, as shown in fig. 2, the control method of the application program includes the steps of:

step 201: the basis function package receives an operation input to perform a target operation.

In this embodiment, the target operation may be different operations such as a handwriting operation, an enlargement operation, a reduction operation, and the like, and the target operation is realized by different operation inputs, and may be realized by an interactive device such as a mouse, a touch panel, and the like.

Step 202: and the basic function packet calls a target function corresponding to the target operation in the application function packet through a data interface.

In this embodiment, the basic function package refers to a basic function package (SDK) that is edited in advance and packaged, and the SDK may select an existing open source SDK, which has high versatility.

The objective function is one of the application functions included in the application function package.

The basic function package corresponds to a first language, in an exemplary embodiment, the first language is Dart language, and the corresponding basic function package can select flute SDK (an open source basic function package provided by google corporation) developed through Dart language, has high universality, and can reduce development work.

For an application program, executing a certain operation input is realized by a specific function, and therefore, it is necessary to call a target function corresponding to the operation input to execute a target operation.

The application function package includes a plurality of functions therein, and when a target operation is executed, the target operation may be executed by a target function among the plurality of functions included in the application function package.

In this embodiment, the data interface is developed through a second language, the application function package is developed through a third language, and the first language, the second language, and the third language are different programming languages.

In an exemplary embodiment, the second language may be a C-class language, the third language may be a Java language, and the application function package is a Java function package, and more specifically, may be an android function package.

Data between software or software modules developed based on a first language and software or software modules developed based on a third language are not directly recognizable and invokable with each other.

In the related art, the Flutter function can only call the C function through an interface provided by the FFI mechanism, and cannot call the Flutter function through the C function and cannot call the Flutter function through the Java function.

In the technical scheme of this embodiment, the application function packet is realized and called through the basic function packet through the data interface. It can be understood that the data interface developed through the second language is used as an intermediate between the SDK corresponding to the first language and the application function package corresponding to the third language, so that the purpose function in the application function package can be called based on the SDK.

Step 203: generating a processing result corresponding to the target operation through the target function;

after the target function is called, the operation or data processing is executed through the target function, and a corresponding processing result is obtained.

In some embodiments, this step 203 is specifically performed by the second engine sublayer, which in an exemplary embodiment is a Flutter engine sublayer and the second engine sublayer is a Java engine sublayer.

This step 203 may include: and the Java engine sublayer returns the processing result to the Flutter engine sublayer.

In some embodiments, after step 203, the method further comprises:

It can be understood that, in the technical solution of this embodiment, the processing result is obtained by the JVM virtual machine. The Java engine sublayer may directly return the processing result to the Flutter engine sublayer, or may return the processing result to the base function packet first, and then return the processing result to the Flutter engine sublayer by the base function packet.

And after the Flutter engine sublayer obtains the processing result, the Flutter engine sublayer further interacts with the operating system and calls the corresponding function of the operating system.

The first engine sublayer is developed through the second language, and illustratively, when the first language is a Dart language, the first engine sublayer includes a Flutter engine. The first engine sublayer and the second engine sublayer may be developed through a class C language.

It is noted that the first application sublayer is developed in a first language and the first engine sublayer is developed in a second language, the first engine sublayer for interacting with the first application sublayer developed in the first language.

Similarly, a second engine sub-layer is developed in the second language, the second engine sub-layer interacting with a second application sub-layer developed in a third language.

The first engine sublayer further interacts directly or indirectly with the operating system to invoke corresponding functions of the operating system to perform the obtained processing results.

For example, if the target operation is to click an icon of an application program, the execution processing result is to start the corresponding application program, and if the target operation is a writing operation on the handwriting interface, the execution processing result may specifically be to display handwriting.

In some embodiments, the object function includes a first object function, the first object function includes a class and a method of Java, the C-function communication sublayer includes a first interface function, the virtual machine includes a JVM virtual machine, and before step 203, the method further includes:

and the JVM virtual machine searches a first target function in the application function packet.

Correspondingly, step 203 specifically includes:

As shown in fig. 3, in this embodiment, an FFI library is first introduced to establish a Dart interface, and specifically, an interface function of the FFI and the C function may be introduced by establishing an open instruction.

Next, a C function is called through a Flutter engine sublayer, specifically, in an embodiment, a jovm.so shared library can be opened by a dlopen instruction, a JNI _ createjavvm method is executed to establish a JVM virtual machine, an instance of a JNIEnv structure is obtained, and Java classes and methods in an android package are found through FindClass and GetMethodID search instructions of the JNIEnv structure.

Generally, when an application program is not started, a virtual machine corresponding to the application program is not established, where the virtual machine specifically refers to a virtual machine instance, and when an application program is started, the virtual machine corresponding to the application program is established through the above process.

In the running process of the application program, specifically, the virtual machine can be operated in a foreground or a background, and the virtual machine keeps an operating state. And when the application program exits, closing the virtual machine corresponding to the application program.

In some embodiments, the objective function further includes a second objective function, the C-function communication sublayer includes a second interface function, the second objective function includes one or more of a C-function library and a drawing library data corresponding to the application function packet, and before step 203, the method further includes:

and the Java engine sublayer calls a second target function matched with the first target function through the second interface function, wherein the first target function is realized based on the second target function.

Correspondingly, step 203 specifically includes:

Referring to fig. 3, in some embodiments, in the Java function, a shared library of a corresponding C function may be introduced by calling system.

And finally, synchronously returning the result to the Java engine sublayer, and then synchronously returning the result to the Flutter engine sublayer or asynchronously returning the result to the Flutter engine sublayer.

And subsequently, the Flutter engine sublayer renders according to the data of the Java engine sublayer or the interactive communication sublayer to finish displaying a processing result corresponding to the target operation.

It should be understood that, in this embodiment, the second objective function may be understood as a function which is lower than the first objective function.

The operation input is used for drawing a circle correspondingly for an exemplary illustration, and necessary data, such as circle center coordinates and radius lengths, are acquired through the first objective function during implementation. The first objective function sends the circle center coordinates and the radius length to the second objective function, and drawing of a circle can be achieved based on the second objective function on the lower layer.

The drawn circle related data is the processing result corresponding to the operation input, the second engine sublayer returns the processing result to the first engine sublayer, that is, the Java engine sublayer returns the processing result to the Flutter engine sublayer, so that the Flutter engine sublayer further executes the processing result, that is, the drawn circle is displayed.

In this embodiment, the Java engine sublayer may call back the processing result to the Flutter engine sublayer in different ways.

As shown in fig. 4, in some embodiments, a Server (Socket Server) is established in the Flutter engine sublayer, a Client (Socket Client) is established in the Java engine sublayer, and the Server may create a listening Socket using a bind () method and then listen to an incoming connection on the Socket, so that after a connection is established between the Client and the Server, bidirectional real-time data communication can be achieved. In this way, the server and the client can perform data communication through the interactive communication sub-layer based on the TCP protocol to return the processing result to the Flutter engine sub-layer.

In other embodiments, the callback function may be implemented using a function pointer (FPointer) as a parameter. Specifically, a first callback function (register Ca11Back Functions) is registered in the Flutter engine sublayer, the interactive communication sublayer includes a second callback function (Ca11Back Functions) established according to the first callback function, and the Java engine sublayer performs data communication with the Flutter engine sublayer through the second callback function to return a processing result to the Flutter engine sublayer. The interactive communication sublayer includes corresponding C Functions (C Functions), and the Java engine sublayer includes corresponding Java Functions (Java Functions).

In some embodiments, the target operation is a handwriting operation, the processing result includes a generated canvas and image data of handwriting of the handwriting operation generated on the canvas, where the image data refers to single data corresponding to one object or a data set corresponding to multiple objects or multiple layers of images, and the specific purpose to be achieved by executing the processing result is to display the handwriting of the handwriting operation.

In some embodiments, after step 203, the method further comprises:

and the display time delay corresponding to the first mode is less than the display time delay corresponding to the second mode.

Further, in some embodiments, the display quality corresponding to the first mode is smaller than the display quality corresponding to the second mode.

In this embodiment, the target operations may be distinguished, and the display of the processing result may be realized in different manners. It is to be understood that different information may be displayed in different manners, for example, for the display of the thumbnail, the requirement for the real-time performance of the display thereof is relatively low, and therefore, the control operation for displaying the thumbnail is the second type of target operation; and the display real-time performance of the handwriting is higher, so that the display of the handwriting is the first type of target operation.

For the above-described first type of display operation and second type of display operation, different processing modes are further set in the present embodiment to execute the processing result.

The target operation is exemplified as the handwriting operation.

Specifically, in the case where the target operation is a handwriting operation in the first type, during the handwriting, a touch position of the handwriting operation is tracked in real time in the first mode, and then the handwriting is displayed. Therefore, the display of the handwriting has higher real-time performance in the process of the handwriting operation.

After the handwriting operation is finished, the handwriting is redisplayed based on the second mode, the handwriting displayed based on the second mode can be understood as a result obtained after the touch handwriting is optimized, and therefore a better processing result of the image quality can be displayed after the rendering refreshing of the display interface is carried out for one time.

When the target operation is the display canvas thumbnail operation in the second type, the requirement for display real-time performance is low, and therefore, the corresponding processing result can be directly displayed in the second mode with relatively good image quality.

In one embodiment, executing the processing result in the second mode includes:

the first engine sublayer acquires the processing result;

In this embodiment, for data with a low real-time requirement, the canvas and the image data may be returned to the Flutter UI for rendering.

It should be understood that the obtaining of the processing result by the first engine sublayer may be implemented in the two manners described above, specifically, the processing result may be returned to the basic function packet by the application function packet, and then returned to the first engine sublayer by the basic function packet; it is also possible that the application function package returns the processing result directly to the first engine sublayer.

The first engine sublayer further interacts with the operating system and displays the corresponding processing results.

As shown in fig. 5, the Flutter-based display process may be summarized as a flow that, when receiving an operation input from a user, determines whether an image needs to be updated, and in a case that a finer animation is needed, first constructs a control, then performs interface rendering, and after the interface rendering, performs rasterization processing to perform updated display of the interface.

The Flutter interface rendering process is divided into three stages of layout, drawing and synthesis, wherein the layout and the drawing are completed in a Flutter frame, the synthesis is completed in a Flutter engine sub-layer, and finally the rendering result is transmitted to a bottom layer so as to drive the display equipment to realize the display of the content.

However, in the data processing link in the related art, data communication and rendering are required through the Java function, Dart function and C function, and the delay of this data link is high, and generally, the screen display delay is in the order of seconds.

In some embodiments, executing the processing result in the first mode comprises:

sending the canvas and the image data to a graphics display framework;

The canvas and image data may be sent to the graphics display framework in different ways.

In some of these embodiments, the canvas and image data obtained may be directly saved as a file and copied and sent to the graphics display framework. In other embodiments, pointers to the canvas and the image data may be directly pointed to the graphics display frame, where the pointers refer to address information of the canvas and the image data, which may improve data transmission speed.

In this embodiment, a user performs drawing operation on the Flutter UI interface, transmits trajectory information to the Java engine sublayer, performs handwriting synthesis and beautification, and transmits processed data to the C function to form canvas and image data through the skea graphic drawing library. Therefore, the speed of image rendering and drawing can be improved, the drawing delay is maintained at the millisecond level, and the effect of synchronous drawing in vision is realized.

In some embodiments, the graphics display frame comprises a DRM (Direct Rendering Manager) frame, said directing the canvas and the pointer to the image data to the graphics display frame comprising:

rendering the canvas and the image data by a LibDRM library;

In this embodiment, the DRM framework can provide a hardware-oriented buffer, and the user can store the image data information to be displayed based on the DRM framework, and can also monitor the buffered data change, and convert the image data into a data structure driven by the display panel, for example, a data structure usable by the T-con panel.

In this embodiment, the canvas and the image data are rendered by the LibDRM library, data that can be recognized by hardware of the display device can be obtained, the canvas and the image data rendered by the LibDRM library are stored in the buffer frame, and further, handwriting display can be realized by the hardware based on the display device.

The handwriting operation performed by the user is exemplified. As shown in fig. 6 and 7, in implementation, a user performs handwriting operation on an operation interface (Flutter UI) of an application (Applications), handwriting track information is transmitted to the Java module, and handwriting beautification is performed in the Java module. Here, the Java module specifically includes a Java JVM virtual machine and related Java functions (Java Function, abbreviated as Java Fun),

the processed data is transferred to the C function module, and processed through drawing library data such as Skia, and a canvas and a data structure are formed.

For data that does not need to be displayed in real time, the data can be returned to the Flutter UI and sent to a Flutter Engine (Flutter Engine) through the Flutter UI for rendering.

And introducing LibDRM for data needing to be displayed in real time, establishing a DRM channel, connecting the DRM channel to DRM equipment through an OCTL interface, and pointing the canvas and the image data pointer to the buffer frame of the DRM to achieve synchronous drawing.

Referring to fig. 6 and 7, if there is data to be displayed in real time, a DRM Device (DRM Device) is first turned on, and specifically, the DRM Device operating based on a DRM Frame may be turned on through a drmwen or open function, and then, capabilities of the DRM Device are detected, resources (Resource) are retrieved, a connection (Connector) is established, encoding (Encoder) is performed, a Buffer Frame (DRM Frame Buffer or Frame Buffer) is constructed, CRTC is prepared, and an interface is drawn.

A CRTC (Cathode Ray Tube Controller) in the DRM device refers to a data pipe for mixing and transmitting display data.

During implementation, during the process of constructing the Buffer Frame, firstly, a DUMB Buffer (simple Buffer) is created, then, a DRM Frame Buffer is added, a map (mapping) is prepared, and a map operation is performed, so that the construction of the Buffer Frame is completed. And the CRTC reads an image to be displayed from the DRM Frame Buffer and outputs the image to the Encoder according to a corresponding format, thereby realizing the drawing and display of an interface.

As shown in fig. 8, in the interface drawing process of the DRM device, first, a matrix drawing interface is determined, then, the matrix drawing interface of the corresponding device is called, next, a corresponding rasterization algorithm is selected, and then, the algorithm is implemented according to the selected rasterization algorithm, so as to complete drawing of an image. For example, a plurality of horizontal lines can be drawn for a rectangle with a skylight, the drawing of the image is completed by drawing the horizontal lines, corresponding coordinates can be determined for the color of the image, then extended data is written in a memory from the specified coordinates, and finally the data is transmitted to an instruction set AVX.

It is understood that in some embodiments, the image processing, rendering and displaying may be performed based on different graphics display frames, which may be, for example, DirectFB, DRM, etc. DirectFB is a lightweight graphics library providing hardware graphics acceleration, input device processing and abstraction, and is designed for embedded systems to achieve the highest hardware acceleration performance with the minimum resource overhead.

In implementation, DirectFB and DRM can be integrated in the display framework, and for the display system supporting DirectFB and DRM, image rendering can be performed based on DirectFB and DRM at the same time, but in some operating systems, such as the kylin system AArch64 system, the DirectFB is in a non-effective state, and image rendering can be performed through the DRM framework alone.

The technical solution of this embodiment can be summarized as being implemented based on the second mode if real-time display is not required. Specifically, the processing result is returned to the Flutter engine sublayer by the Java engine sublayer, the image is optimized to obtain an image with better image quality, then the Flutter engine sublayer interacts with the operating system, and the graphical display frame is called by the operating system to realize the display of the processing result.

In the process, the Flutter engine sublayer can optimize the image and is beneficial to improving the display effect, but in the process, the data transmission link is long, the magnitude of the display delay is in the second level, and the display delay is obvious to the naked eye.

If the display real-time performance needs to be improved, the Java engine sublayer directly returns the processing result to the graphic display framework, so that the data transmission link is short, the display real-time performance is high, the display delay can reach millisecond level usually, and the real-time display can be achieved on the visual effect of a general user.

In this process, the optimization operation for the image of the handwriting is lacked, and therefore, the image quality of the displayed image of the handwriting is relatively poor.

After the handwriting operation is finished, the display result is optimized again according to the second mode, so that the image with high display quality can be rendered after the image is refreshed once, and the display effect and the display real-time property are considered.

The embodiment of the disclosure also provides an electronic device. Referring to fig. 9, an electronic device may include a processor 901, a memory 902, and a program 9021 stored in the memory 902 and operable on the processor 901.

When executed by the processor 901, the program 9021 may implement any step in the foregoing method embodiments and achieve the same beneficial effects, which are not described herein again.

Those skilled in the art will appreciate that all or part of the steps of the method according to the above embodiments may be implemented by hardware associated with program instructions, and the program may be stored in a readable medium.

An embodiment of the present disclosure further provides a readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program may implement any step in the method embodiment corresponding to fig. 1, and may achieve the same technical effect, and in order to avoid repetition, details are not repeated here.

The storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.

It should be noted that the division of each module is only a logical division, and all or part of the actual implementation may be integrated into one physical entity or may be physically separated. And these modules can be realized in the form of software called by processing element; or can be implemented in the form of hardware; and part of the modules can be realized in the form of calling software by the processing element, and part of the modules can be realized in the form of hardware. For example, the determining module may be a processing element separately set up, or may be integrated into a chip of the apparatus, or may be stored in a memory of the apparatus in the form of program code, and a processing element of the apparatus calls and executes the function of the determining module. The other modules are implemented similarly. In addition, all or part of the modules can be integrated together or can be independently realized. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.

For example, the various modules, units, sub-units or sub-modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), etc. For another example, when some of the above modules are implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor that can call the program code. As another example, these modules may be integrated together, implemented in the form of a system-on-a-chip (SOC).

While the foregoing is directed to the preferred embodiment of the present disclosure, it will be appreciated by those skilled in the art that various changes and modifications may be made therein without departing from the principles of the disclosure, and it is intended that such changes and modifications be considered as within the scope of the disclosure.

Claims

1. A data processing method is applied to electronic equipment, the electronic equipment comprises a processor and a memory, the memory stores a basic function package and a data interface, and an application program is further installed on the electronic equipment and comprises an application function package, the application function package is used for providing an application function corresponding to the application function of the application program, the basic function package is used for providing a basic function for realizing a preset basic function, wherein at least part of the application function is realized based on the basic function;

the method comprises the following steps:

2. The method of claim 1, wherein the application further comprises a first engine sublayer to invoke functionality of an operating system that installs the application to execute the processing result;

the application function packet returns the processing result to the basic function packet, and the basic function packet returns the processing result to the first engine sublayer; or alternatively

3. The method of claim 2, wherein the application further comprises a second engine sub-layer to establish a virtual machine to generate the processing result according to the objective function.

4. The method of claim 3, wherein the application further comprises an interactive communication sub-layer corresponding to the second language, the interactive communication sub-layer comprising the data interface, the first engine sub-layer and the second engine sub-layer in data communication through the interactive communication sub-layer.

5. The method of claim 4, wherein the first engine sublayer is a Flutter engine sublayer, the second engine sublayer is a Java engine sublayer, and the interactive communication sublayer comprises a C-function communication sublayer.

6. The method of claim 5, wherein the objective function comprises a first objective function comprising classes and methods of Java, the C-function communication sublayer comprises first interface functions, and the virtual machine comprises a JVM virtual machine;

the basic function package transmits a calling instruction corresponding to the target operation to the Java engine sublayer through the first interface function, wherein the data interface comprises the first interface function, and the data interface is a Dart interface established by the Flutter engine sublayer through an FFI library;

7. The method of claim 6, wherein the objective function further comprises a second objective function, the C-function communication sublayer comprises a second interface function, the second objective function comprises one or more of a C-function library and a drawing library data corresponding to the application function package;

the Java engine sublayer calls a second objective function matched with the first objective function through the second interface function, wherein the first objective function is realized based on the second objective function;

8. The method according to any one of claims 5 to 7, wherein a server is established in the Flutter engine sublayer, a client is established in the Java engine sublayer, and the server and the client perform data communication through the interactive communication sublayer based on a TCP protocol to return the processing result to the Flutter engine sublayer.

9. The method according to claim 7, wherein a first callback function is registered in the Flutter engine sublayer, the interactive communication sublayer includes a second callback function established according to the first callback function, and the Java engine sublayer performs data communication with the Flutter engine sublayer through the second callback function to return the processing result obtained through the second target function to the Flutter engine sublayer.

10. The method of any of claims 4 to 7, wherein the base function package is in data communication with the interactive communication sub-layer via an FFI mechanism, and the application function package is in data communication with the interactive communication sub-layer via a JNI mechanism.

11. The method of claim 2, wherein the target operation is a handwriting operation, and the processing result comprises a canvas generated according to the handwriting operation and image data of the handwriting generated on the canvas.

12. The method of claim 11, wherein after generating the processing result corresponding to the target operation by the target function, the method further comprises:

acquiring the type of the target operation, wherein the type of the target operation comprises a first type and a second type, and the real-time display requirement of the first type of target operation is greater than that of the second type of target operation;

13. The method of claim 12, wherein said performing the processing result in the first mode comprises:

sending the canvas and the image data to a graphics display framework;

14. The method of claim 13, wherein the graphical display frame comprises a DRM frame, the rendering the handwriting of the handwriting operation through the graphical display frame comprising:

rendering the canvas and the image data by a LibDRM library;

15. The method of any of claims 12 to 14, wherein said performing the processing result in the second mode comprises:

the first engine sublayer acquires the processing result;

16. The method of any of claims 1-7, wherein the application function package is matched to an android system.

17. The method of any one of claims 1 to 7, wherein the operating system running the application is the kylin system.

18. An electronic device, comprising: a memory, a processor, and a program stored on the memory and executable on the processor; the processor, for reading a program in the memory to implement the steps in the data processing method of any one of claims 1 to 17.

19. A readable storage medium storing a program which, when executed by a processor, implements the steps in the data processing method of any one of claims 1 to 17.