CN115185397A

CN115185397A - Handwriting refreshing method and device for ink screen

Info

Publication number: CN115185397A
Application number: CN202210719690.0A
Authority: CN
Inventors: 孙永亮; 梅正怡; 修平
Original assignee: Hisense Mobile Communications Technology Co Ltd
Current assignee: Hisense Mobile Communications Technology Co Ltd
Priority date: 2022-06-23
Filing date: 2022-06-23
Publication date: 2022-10-14

Abstract

The embodiment of the invention relates to the technical field of computers, in particular to a handwriting refreshing method and device for an ink screen. The method comprises the following steps: sending the layer data drawn by the application based on the touch pressing operation to a hardware abstraction layer for synthesis to obtain a first synthetic image; determining a first variation region based on the first and second composite images; if the first change area is located in a preset range near the touch pressing operation, determining that the ink screen is in a handwriting state; after the ink screen is determined to be in a handwriting state, acquiring each composite image from a hardware abstraction layer for preprocessing; and refreshing and displaying the preprocessed composite images in parallel through a water screen. The method does not need to adapt to each application, saves manpower, material resources and time, and is suitable for any application. The refreshing speed of the ink screen in the handwriting scene is improved in a parallel refreshing mode, and the display follow chirality and the fluency are improved. And the writing can be smoothly realized in any application.

Description

Handwriting refreshing method and device for ink screen

Technical Field

The embodiment of the invention relates to the technical field of computers, in particular to a handwriting refreshing method and device for an ink screen, computing equipment and a computer readable storage medium.

Background

The display principle of the ink screen is that hundreds of microcapsules with the same size as the human hair diameter are suspended in the ink screen, and pigment particles with different charges are arranged in the microcapsules. In an initial state, the pigment particles are suspended in the microcapsules, and when an electric field in a certain direction is applied, the corresponding pigment particles are pushed to the top, the microcapsules can display different colors, and the microcapsules with different colors form various characters and patterns. The ink screen has the characteristics of eye protection, power saving and the like, so the ink screen is more and more widely applied to various electronic products, such as an ink screen mobile phone, an ink screen flat plate and the like.

The average time for refreshing an image by the liquid crystal screen is 6-8ms, while the refreshing principle of the ink screen is different from that of display screens such as the liquid crystal screen, the ink screen generally needs to be applied with 8 times of voltage to refresh an image, and the average time can reach 100-120 ms. Because the ink screen is refreshed at a low speed, a user can feel that the user feels stuck when watching videos and performing handwriting by adopting an electronic product with the ink screen, and the user experience is influenced. For example, in a handwriting scene, the user has finished drawing, and the drawing track of the user may not be displayed on the ink screen, so that the handedness and smoothness of the handwriting effect cannot be guaranteed.

Disclosure of Invention

The embodiment of the invention provides an ink screen handwriting refreshing method, which is used for solving the problem of low display speed of handwriting input of an ink screen, improving the display speed, optimizing the display effect and improving the user experience.

In a first aspect, an embodiment of the present invention provides an ink screen handwriting refreshing method, including:

sending the layer data drawn by the application based on the touch pressing operation to a hardware abstraction layer for synthesis to obtain a first synthetic image;

determining a first variation region based on the first and second composite images; if the first change area is located in a preset range near the touch pressing operation, determining that the ink screen is in a handwriting state; the second synthetic image is obtained before the touch pressing operation;

after the ink screen is determined to be in a handwriting state, acquiring each composite image from the hardware abstraction layer for preprocessing; and refreshing and displaying the preprocessed composite images in parallel through the ink screen.

No matter what application is opened by the user, as long as touch pressing operation is detected, the first composite image and the second composite image are compared, and whether the user is in a handwriting scene currently is determined according to a first change area obtained through comparison. No separate adaptation of the application is required. Because each composite image is taken out from the hardware abstraction layer of the pre-installed display system, the composite image can be adapted and compatible with the pre-installed display system only according to the ink screen product without adapting each application, thereby saving manpower, material resources and time and being suitable for any application. The parallel refreshing mode improves the refreshing speed of the ink screen in the handwriting scene, the handwriting track is synchronously displayed in real time in the handwriting process of a user, the condition that the user experience is influenced due to too slow refreshing speed and large display delay is reduced, and the display follow chirality and the fluency are improved. Thus, the writing can be smoothly realized in any application. And each composite image is taken out from the hardware abstraction layer of the pre-installed display system, and each composite image in the hardware abstraction layer of the pre-installed display system is also obtained based on the layer data drawn by the application, so that the handwriting track obtained by performing parallel refreshing on the basis of each composite image is completely consistent with the handwriting track obtained by performing system display refreshing on the basis of the pre-installed display system.

In some embodiments of the present application, after determining that the ink screen is in a handwriting state, the method further includes:

and stopping acquiring the composite image from the hardware abstraction layer to perform system refreshing display of the ink screen.

Therefore, the system of the ink screen does not need to refresh and display, the display speed is improved, the calculated amount is reduced, and the display effect is improved.

In some embodiments of the present application, obtaining each composite image from the hardware abstraction layer for preprocessing includes:

determining a second change area according to the third synthetic image and the fourth synthetic image before the touch-up operation is detected; the third composite image and the fourth composite image are any two adjacent composite images obtained from the hardware abstraction layer;

and refreshing and displaying the preprocessed composite images in parallel through the ink screen, wherein the refreshing and displaying process comprises the following steps:

and refreshing and displaying all the second change areas in parallel through the ink screen.

Only partial refreshing is carried out on each changed area, so that the calculation amount is reduced, and the refreshing speed is further accelerated. The refreshing speed of the ink screen in a handwriting scene is improved in a parallel refreshing mode, the handwriting track is synchronously displayed in real time in the handwriting process of a user, the condition that the user experience is influenced due to too slow refreshing speed and large display delay is reduced, and the display handedness and the smoothness are improved. Thus, the writing can be smoothly realized in any application. And each composite image is taken out from the hardware abstraction layer of the pre-installed display system, and each composite image in the hardware abstraction layer of the pre-installed display system is also obtained based on the layer data drawn by the application, so that the handwriting track obtained by performing parallel refreshing based on each second change area is completely consistent with the handwriting track obtained by performing system display refreshing based on the pre-installed display system.

In some embodiments of the present application, the second variation region is a rectangle that encloses the variation portions of any two adjacent composite images.

The second variation region is reduced as much as possible, and the speed of the partial refresh can be increased.

In some embodiments of the present application, after performing parallel refreshing display on the preprocessed composite images through the ink screen, the method further includes:

and after the touch lifting operation is detected, recovering to acquire the composite image from the hardware abstraction layer to perform system refreshing display of the ink screen, and stopping acquiring the composite image from the hardware abstraction layer to perform preprocessing.

Therefore, the parallel refreshing display method is equivalent to a temporary method, and after a handwriting scene is detected, when a user touches and moves, the parallel refreshing display method can improve the display speed in the handwriting process of the user, and improve the follow chirality and the fluency. And once the user touches and lifts, ending the parallel refreshing display method, continuously displaying the handwriting drawn by the user at one time by using the system refreshing display method, and replacing the handwriting refreshed in parallel.

In some embodiments of the present application, the performing parallel refreshing display on the preprocessed composite images through the ink screen includes:

and performing soft decoding on each preprocessed composite image, and performing parallel refreshing display on the decoded image information through the ink screen.

In some embodiments of the present application, further comprising:

and if the first change area is not located in a preset range near the touch pressing operation, acquiring a composite image from the hardware abstraction layer to perform system display refreshing of the ink screen.

And if the first change area is not located in a preset range near the touch pressing operation, the current ink screen is not in a handwriting state, and therefore system display refreshing of the ink screen is continued.

In a second aspect, an embodiment of the present invention further provides an ink screen handwriting refreshing apparatus, including:

a processing module to:

a pre-processing module to:

after the ink screen is determined to be in a handwriting state, acquiring each composite image from the hardware abstraction layer for preprocessing;

the processing module is further configured to:

and refreshing and displaying the preprocessed composite images in parallel through the ink screen.

In some embodiments of the present application, the preprocessing module is further configured to:

In some embodiments of the present application, the preprocessing module is specifically configured to:

determining a second change area according to the third composite image and the fourth composite image before the touch-up operation is detected; the third composite image and the fourth composite image are any two adjacent composite images obtained from the hardware abstraction layer;

In some embodiments of the present application, the processing module is specifically configured to:

and performing parallel refreshing display on each second change area through the ink screen.

In some embodiments of the present application, the processing module is further configured to:

In a third aspect, an embodiment of the present invention further provides a terminal device, where the terminal device is installed with at least one application; the terminal device includes:

the ink screen is used for sending the detected touch pressing operation to a processor of the terminal equipment and displaying the touch pressing operation under the control of the processor;

a memory for storing a computer program;

and the processor is used for calling the computer program stored in the memory and executing the handwriting refreshing method of the ink screen listed in any mode according to the obtained program.

In some embodiments of the present application, the terminal device is provided with an android system.

In a fourth aspect, the present invention further provides a computer-readable storage medium, where a computer-executable program is stored, where the computer-executable program is configured to enable a computer to execute the handwriting refresh method for an ink screen listed in any one of the above manners.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic diagram of a structure of a terminal device according to an embodiment of the present invention;

fig. 2 is a schematic diagram of a software architecture of a terminal device according to an embodiment of the present invention;

FIG. 3a is a schematic flow chart of a system refresh display in the prior art;

FIG. 3b is a diagram illustrating the effect of refreshing a display in a system according to the prior art;

FIG. 4a is a flowchart illustrating a possible method for refreshing handwriting on an ink screen according to an embodiment of the present invention;

FIG. 4b is a schematic diagram illustrating a possible effect of parallel refresh display according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a possible handwriting refresh method for an ink screen according to an embodiment of the present invention;

FIG. 6a is a schematic diagram of a possible first composite image according to an embodiment of the present invention;

FIG. 6b is a schematic diagram of a possible first variation region provided by the embodiment of the present invention;

FIG. 6c is a schematic diagram of a possible plurality of composite images according to an embodiment of the present invention;

FIG. 6d is a schematic diagram of a possible second variation region provided by the embodiment of the present invention;

FIG. 7 is a diagram illustrating a comparison between a handwriting refresh method and other handwriting refresh methods according to an embodiment of the present invention;

FIG. 8 is a diagram illustrating a possible handwriting refresh method according to an embodiment of the present invention;

fig. 9 is a schematic diagram illustrating a structure of an ink screen handwriting refreshing apparatus according to an embodiment of the present invention.

Detailed Description

To make the objects, embodiments and advantages of the present application clearer, the following description of exemplary embodiments of the present application will clearly and completely describe the exemplary embodiments of the present application with reference to the accompanying drawings in the exemplary embodiments of the present application, and it is to be understood that the described exemplary embodiments are only a part of the embodiments of the present application, and not all of the embodiments.

All other embodiments, which can be derived by a person skilled in the art from the exemplary embodiments described herein without inventive step, are intended to be within the scope of the claims appended hereto. In addition, while the disclosure herein has been presented in terms of one or more exemplary examples, it should be appreciated that aspects of the disclosure may be implemented solely as a complete embodiment.

It should be noted that the brief descriptions of the terms in the present application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of the present application. These terms should be understood in their ordinary and customary meaning unless otherwise indicated.

The terms "first," "second," "third," and the like in the description and claims of this application and in the foregoing drawings are used for distinguishing between similar or analogous objects or entities and are not necessarily intended to limit the order or sequence Unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein.

Furthermore, the terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or device that comprises a list of elements is not necessarily limited to those elements explicitly listed, but may include other elements not expressly listed or inherent to such product or device.

The handwriting refreshing method for the ink screen can be applied to terminal devices such as a mobile phone, a tablet computer, a desktop computer, a laptop computer, a notebook computer, an ultra-mobile personal computer (UMPC), a handheld computer, a netbook, a Personal Digital Assistant (PDA), a wearable terminal device and a virtual reality device, and the method is not limited in any way.

Fig. 1 shows a schematic structural diagram of a terminal device 100.

The following specifically describes an embodiment by taking the terminal device 100 as an example. It should be understood that the terminal device 100 shown in fig. 1 is only an example, and the terminal device 100 may have more or less components than those shown in fig. 1, may combine two or more components, or may have a different configuration of components. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

A block diagram of a hardware configuration of a terminal device 100 according to an exemplary embodiment is exemplarily shown in fig. 1. As shown in fig. 1, the terminal device 100 includes: radio Frequency (RF) circuit 110, memory 120, display unit 130, camera 140, sensor 150, audio circuit 160, wireless Fidelity (Wi-Fi) module 170, processor 180, bluetooth module 181, and power supply 190.

The RF circuit 110 may be used for receiving and transmitting signals during information transmission and reception or during a call, and may receive downlink data of a base station and then send the downlink data to the processor 180 for processing; the uplink data may be transmitted to the base station. Typically, the RF circuitry includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like.

The memory 120 may be used to store software programs and data. The processor 180 performs various functions of the terminal device 100 and data processing by executing software programs or data stored in the memory 120. The memory 120 may include high speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. The memory 120 stores an operating system that enables the terminal device 100 to operate. The memory 120 may store an operating system and various application programs, and may also store program codes for performing the methods described in the embodiments of the present application.

The display unit 130 may be used to receive input numeric or character information and generate signal input related to user settings and function control of the terminal device 100, and particularly, the display unit 130 may include a touch screen 131 disposed on the front surface of the terminal device 100 and may collect touch operations of a user thereon or nearby, such as clicking a button, dragging a scroll box, and the like.

The display unit 130 may also be used to display a Graphical User Interface (GUI) of information input by or provided to the user and various menus of the terminal apparatus 100. Specifically, the display unit 130 may include a display screen 132 disposed on the front surface of the terminal device 100. The display screen 132 may be configured in the form of a liquid crystal display, a light emitting diode, or the like. The display unit 130 may be configured to display a contact session interface or a short message list interface described in the present application.

The touch screen 131 may cover the display screen 132, or the touch screen 131 and the display screen 132 may be integrated to implement the input and output functions of the terminal device 100, and after the integration, the touch screen may be referred to as a touch display screen for short. In the present application, the display unit 130 may display the application programs and the corresponding operation steps.

The camera 140 may be used to capture still images or video. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The light sensing elements convert the light signals into electrical signals which are then passed to the processor 180 for conversion into digital image signals.

The terminal device 100 may further comprise at least one sensor 150, such as an acceleration sensor 151, a distance sensor 152, a fingerprint sensor 153, a temperature sensor 154. The terminal device 100 may also be configured with other sensors such as a gyroscope, barometer, hygrometer, thermometer, infrared sensor, light sensor, motion sensor, and the like.

The audio circuitry 160, speaker 161, microphone 162 may provide an audio interface between a user and the terminal device 100. The audio circuit 160 may transmit the electrical signal converted from the received audio data to the speaker 161, and convert the electrical signal into a sound signal for output by the speaker 161. The terminal device 100 may also be provided with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 162 converts the collected sound signal into an electrical signal, converts the electrical signal into audio data after being received by the audio circuit 160, and outputs the audio data to the RF circuit 110 to be transmitted to, for example, another terminal device, or outputs the audio data to the memory 120 for further processing. In this application, the microphone 162 may capture the voice of the user.

Wi-Fi belongs to a short-distance wireless transmission technology, and the terminal device 100 can help a user to send and receive e-mails, browse webpages, access streaming media and the like through the Wi-Fi module 170, and provides wireless broadband internet access for the user.

The processor 180 is a control center of the terminal device 100, connects various parts of the entire terminal device using various interfaces and lines, performs various functions of the terminal device 100 and processes data by running or executing software programs stored in the memory 120 and calling data stored in the memory 120. In some embodiments, processor 180 may include one or more processing units; the processor 180 may also integrate an application processor, which mainly handles operating systems, user interfaces, applications, etc., and a baseband processor, which mainly handles wireless communications. It will be appreciated that the baseband processor described above may not be integrated into the processor 180. In the present application, the processor 180 may run an operating system, an application program, a user interface display, and a touch response, and the processing method described in the embodiments of the present application. Further, the processor 180 is coupled to the display unit 130.

And the bluetooth module 181 is configured to perform information interaction with other bluetooth devices having a bluetooth module through a bluetooth protocol. For example, the terminal device 100 may establish a bluetooth connection with a wearable electronic device (e.g., a smart watch) having a bluetooth module via the bluetooth module 181, so as to perform data interaction.

The terminal device 100 also includes a power supply 190 (such as a battery) to power the various components. The power supply may be logically coupled to the processor 180 through a power management system to manage charging, discharging, and power consumption functions through the power management system. The terminal device 100 may also be configured with power buttons for powering the terminal device on and off, and for locking the screen.

Fig. 2 is a block diagram of a software configuration of the terminal device 100 according to the embodiment of the present application.

The layered architecture divides the software into several layers, each layer having a clear role and division of labor. The layers communicate with each other through a software interface. In some embodiments, the Android system may be divided into four layers, an application layer, an application framework layer, an Android runtime (Android runtime) and system libraries, and a kernel layer, from top to bottom.

The application layer may include a series of application packages.

As shown in fig. 2, the application package may include applications such as camera, gallery, calendar, phone call, map, navigation, WLAN, bluetooth, music, video, short message, etc.

The application framework layer provides an Application Programming Interface (API) and a programming framework for the application program of the application layer. The application framework layer includes a number of predefined functions.

As shown in FIG. 2, the application framework layers may include a window manager, content provider, view system, phone manager, resource manager, notification manager, and the like.

The window manager is used for managing window programs. The window manager can obtain the size of the display screen, judge whether a status bar exists, lock the screen, intercept the screen and the like.

The content provider is used to store and retrieve data and make it accessible to applications. The data may include video, images, audio, dialed and answered calls, browsing history and bookmarks, phone books, short messages, etc.

The view system includes visual controls such as controls to display text, controls to display pictures, and the like. The view system may be used to build applications. The display interface may be composed of one or more views. For example, the display interface including the short message notification icon may include a view for displaying text and a view for displaying a picture.

The phone manager is used to provide the communication function of the terminal device 100. Such as management of call status (including on, off, etc.).

The resource manager provides various resources, such as localized strings, icons, pictures, layout files, video files, etc., to the application.

The notification manager enables the application to display notification information (e.g., message digest of a short message, message content) in the status bar, can be used to convey notification-type messages, and can disappear automatically after a short dwell, without user interaction. Such as a notification manager used to inform download completion, message alerts, etc. The notification manager may also be a notification that appears in the form of a chart or scroll bar text at the top status bar of the system, such as a notification of a background running application, or a notification that appears on the screen in the form of a dialog window. For example, text information is prompted in the status bar, a prompt tone is given, the terminal device vibrates, an indicator light flickers, and the like.

The Android Runtime comprises a core library and a virtual machine. The Android runtime is responsible for scheduling and managing an Android system.

The core library comprises two parts: one part is a function which needs to be called by java language, and the other part is a core library of android.

The application layer and the application framework layer run in a virtual machine. And executing java files of the application program layer and the application program framework layer into a binary file by the virtual machine. The virtual machine is used for performing the functions of object life cycle management, stack management, thread management, safety and exception management, garbage collection and the like.

The system library may include a plurality of functional modules. For example: surface managers (surface managers), media Libraries (Media Libraries), three-dimensional graphics processing Libraries (e.g., openGL ES), 2D graphics engines (e.g., SGL), and the like.

The surface manager is used to manage the display subsystem and provide a fusion of the 2D and 3D layers for multiple applications.

The media library supports a variety of commonly used audio, video format playback and recording, and still image files, among others. The media library may support a variety of audio-video encoding formats such as MPEG4, h.264, MP3, AAC, AMR, JPG, PNG, etc.

The three-dimensional graphic processing library is used for realizing three-dimensional graphic drawing, image rendering, synthesis, layer processing and the like.

A 2D (one way of animation) graphics engine is a drawing engine for 2D drawing.

The kernel layer is a layer between hardware and software. The inner core layer at least comprises a display driver, a camera driver, an audio driver and a sensor driver.

The following describes exemplary workflow of the terminal device 100 software and hardware in connection with capturing a photo scene.

When the touch screen 131 receives a touch operation, a corresponding hardware interrupt is issued to the kernel layer. The kernel layer processes the touch operation into an original input event (including touch coordinates, a time stamp of the touch operation, and other information). The raw input events are stored at the kernel layer. And the application program framework layer acquires the original input event from the kernel layer and identifies the control corresponding to the input event. Taking the touch operation as a touch click operation, and taking a control corresponding to the click operation as a control of a camera application icon as an example, the camera application calls an interface of an application framework layer, starts the camera application, further starts a camera drive by calling a kernel layer, and captures a still image or a video through the camera 140.

The ink screen has the characteristics of eye protection, power saving and the like, so the ink screen is more and more widely applied to various terminal devices, such as an ink screen mobile phone, an ink screen flat plate and the like. But the refreshing principle of the ink screen is limited, 8 times of voltage needs to be applied to the ink screen for refreshing an image, and the average time consumption can reach 100ms-120ms. Because the ink screen is refreshed at a low speed, a user can feel that the user feels stuck when watching videos and performing handwriting by adopting an electronic product with the ink screen, and the user experience is influenced. For example, in a handwriting scene, the user has finished drawing, and the drawing track of the user may not be displayed on the ink screen, so that the handedness and smoothness of the handwriting effect cannot be guaranteed.

The reason why the existing ink screen handwriting refreshing is slow is that each handwriting track image drawn by the application is displayed through a pre-installed display system, namely, the system refreshing display. The display system preassembled by the ink screen product can be an android display system, an ios display system, a Linux display system or a windows display system and the like. Taking an android display system as an example, the android display system avoids displaying ghosts when a user watches videos and other behaviors, so that parallel refreshing display of multiple images is not supported. The term "displaying ghost" means displaying one image and the content of the previous image in the display device. Fig. 3a is a schematic flow chart of a system refresh display in the prior art.

The method comprises the following steps:

step 301, after detecting a touch operation of a user in real time, a touch screen acquires point reporting data according to the touch operation, and transmits the point reporting data to an application.

The touch screen in the embodiment of the invention mainly refers to a touch screen of an ink screen.

And 302, drawing a handwriting track according to the report point data by the application, and transmitting the layer data containing the handwriting track to an android display system.

And 303, synthesizing by the android display system to obtain a synthesized image.

And 304, the decoding module acquires a composite image A through the android display system, and the composite image A is disassembled into 8 frames, so that the composite image A is displayed in the display screen by applying voltage for 8 times. After the composite image A is displayed, the decoding module acquires the next composite image B, breaks the composite image B into 8 frames, and displays the composite image B in a display screen by applying voltages for 8 times \8230, and so on. Figure 3b shows a schematic diagram of a prior art system refresh. It can be seen that the refresh mode of system refresh display can refresh the next image only after one image is refreshed, so that the handwriting refresh speed is very slow, and the handedness and smoothness of the handwriting effect cannot be ensured.

The embodiment of the invention provides a possible handwriting refreshing method for an ink screen, and when technicians in the industry produce ink screen products, individually adaptive applications, such as handwriting notepad software, are pre-installed in the ink screen products. Therefore, the system refreshing display based on the android display system is avoided, and a parallel refreshing display method is adopted. Therefore, refreshing time is reduced to below 50ms, and handwriting is smooth.

FIG. 4a shows a flow diagram of one possible ink screen handwriting refresh. The method comprises the following steps:

step 401, a user touches and presses the handwriting notepad software, the touch screen detects the touch and press position (i.e. the report point data), and the report point data is transmitted to the android display system.

Step 402, the android display system determines that the handwriting notepad software is touched and pressed through processing, so that the display screen is controlled to display the page of the handwriting notepad software, and a user can perform handwriting operation on the page of the handwriting notepad software.

Step 403, in the process that the user writes through the handwriting notepad, the touch screen acquires the report point data in real time and transmits the report point data to the handwriting notepad software.

And step 404, drawing a handwriting track by the handwriting notepad software according to the report data, and obtaining a composite image based on the handwriting track. Because the handwriting notepad software is independently researched and developed by manufacturers of the ink screen products and is independently adaptive to the ink screen products, the handwriting notepad software cannot send the synthetic image to an android display system for system refreshing and displaying. But rather the composite image is sent directly to the decoding module of the ink screen.

Step 405, the decoding module performs parallel refreshing display based on the composite image sent by the handwriting notepad software.

The parallel refreshing display method is that if each image needs k frames to be displayed, at least m frames of two adjacent images are refreshed simultaneously, and m is more than or equal to 1 and less than k. For example, the handwriting notepad software sends the composite image a to the decoding module, and the decoding module divides the composite image a into 8 frames for frame-by-frame display. When the composite image a finishes displaying the 1 st frame, that is, after the 1 st voltage is applied, the decoding module receives the next composite image B of the composite image a, and then the composite image a and the composite image B can perform parallel refreshing display: the 2 nd frame of composite image a is refreshed with the 1 st frame of composite image B. When the 2 nd frame of the composite image A is displayed and the 1 st frame of the composite image B is displayed, the decoding module receives the next composite image C of the composite image B, and then the composite image A, the composite image B and the composite image C can be refreshed and displayed in parallel: frame 3 of composite image a, frame 2 of composite image B, and frame 1 of composite image C are refreshed together 82308230, 8230, and so on. Figure 4b shows a schematic diagram of one possible parallel refresh.

Therefore, the refresh rate of the parallel refresh in the ink screen is much faster than that of the refresh based on the android display system in the ink screen.

However, in the above method, it is necessary to adapt the application individually, for example, the handwriting notepad software is adapted individually, and the handwriting notepad software will send the synthesized image directly to the decoding module of the ink screen. In this manner, it may be determined that the user opens the individually adapted application when the individually adapted application is touched by the user. And the parallel refreshing display can be carried out only in the application of independent adaptation, so that the handwriting is improved. If the user opens other unadapted third-party applications for handwriting, the third-party applications draw handwriting tracks according to the report point data, and the layer data containing the handwriting tracks are transmitted to the android display system. The android display system obtains the synthetic image, transmits the synthetic image to the decoding module, so that the decoding module can only adopt a system refreshing display mode to display, the display speed is low, and the following chirality is poor. Because the third-party application is not independently adapted, the third-party application cannot cross an android display system, and the decoding module cannot perform parallel refreshing display.

If the third-party application is independently adapted, negotiation with a development company of the third-party application is needed, and if the development company of the third-party application does not agree, the handwriting refreshing speed of the third-party application in an ink screen product cannot be achieved, and the user experience is poor. Even if a development company of the third-party application agrees to perform independent adaptation on the ink screen product, the workload of performing independent adaptation one by one is very huge due to the huge number of the third-party applications, and a large amount of manpower and material resources are consumed.

An embodiment of the present invention provides another possible handwriting refresh method for an ink screen, as shown in fig. 5, including:

step 501, sending the layer data drawn by the application based on the touch pressing operation to a hardware abstraction layer for synthesis to obtain a first synthetic image.

Step 502, determining a first change area based on the first composite image and the second composite image; if the first change area is located in a preset range near the touch pressing operation, determining that the ink screen is in a handwriting state; the second composite image is obtained before the touch and press operation.

Step 503, after the ink screen is determined to be in the handwriting state, acquiring each composite image from the hardware abstraction layer for preprocessing; and refreshing and displaying the preprocessed composite images in parallel through the ink screen.

In step 501, a user performs a touch press on a touch screen, and the touch screen detects a position of the touch press (i.e., touch point data), and transmits the touch point data to an application. And drawing the handwriting track based on the report point data to obtain the layer data. The image Layer data is sent to a Hardware Abstraction Layer (HAL) of a display system preinstalled with an ink screen product by application, and the Hardware abstraction Layer carries out synthesis according to the image Layer data to obtain a first synthetic image.

The display system for pre-installing the ink screen product can be an android display system, an ios display system, a Linux system or a windows display system, and the like. And if the pre-installed display system is an android display system, the hardware abstraction layer is the Hwcomposer.

The application of the embodiment of the invention does not need to be adapted independently and can be various third-party applications.

For example, a user performs a touch-and-press operation on a touch screen, and the touch screen detects two trace data, a (1, 1) and b (2, 2). And sending the two report point data to an application displayed by a current display screen, for example, if the application displayed by the current display screen is a Youkou video, sending the two report point data to the Youkou video application. And drawing the handwriting track according to the two report point data to obtain the layer data. And the application sends the layer data to a hardware abstraction layer, namely the Hwcomposer, of the android display system. And synthesizing the hardware abstraction layer to obtain a first synthesized image. Fig. 6a shows a schematic view of one possible first composite image. The first composite image includes a handwritten trace drawn by the application.

In step 502, a second synthetic image before the touch pressing operation is obtained in the hardware abstraction layer, and the second synthetic image synthesized by the hardware abstraction layer is also based on the layer data drawn by the application.

The first variation region is determined based on the first composite image and the second composite image. The first composite image and the second composite image referred to in the embodiments of the present invention are composite images of a whole screen. The first variation region is a rectangle surrounding a portion of the second composite image that varies from the first composite image.

If the user performs a handwriting operation on the ink screen, the difference between the first composite image and the second composite image obtained before and after the touch-and-press operation is not so large, and should be concentrated near the touch-and-press operation. If the user performs a non-handwriting operation on the ink screen, for example, although the user also has a touch-and-press operation, the touch-and-press operation is to slide and browse the currently displayed content, the first composite image and the second composite image may be very different from each other, and may be completely different from each other with a high probability. Therefore, whether the user is performing handwriting operation, that is, whether the ink screen is in a handwriting state, can be determined by the first change area of the first composite image and the second composite image.

And if the first change area is located in a preset range near the touch pressing operation, determining that the ink screen is in a handwriting state. For example, in the above example, the first variation area should be as shown in fig. 6b, and the touch-and-press operation has the touch-and-press operation trace data of a (1, 1) and b (2, 2), respectively. The first variation area is located within a preset range near a (1, 1) and b (2, 2), and thus it is determined that the ink screen is in a handwriting state.

The preset range can be freely set according to the requirements of the technicians in the field, and the embodiment of the invention is not limited.

If the first change area is not located within the preset range near the touch pressing operation, as shown in fig. 6c, it indicates that the ink screen is not in the handwriting state. For example, the user may be a swipe browsing of videos in a youth video application.

And if the ink screen is judged not to be in the handwriting state, the subsequent parallel refreshing display method is not needed for displaying, and the composite image is obtained from the hardware abstraction layer for the system display refreshing of the ink screen. If the parallel refresh display is performed in the non-handwritten state, "display ghost" occurs.

The display ghost image can also appear when the handwriting state is performed, and only because a plurality of drawing actions of a user need to be displayed under the handwriting scene, a plurality of images are displayed on a display screen at the same time, and the discomfort of the user can not be caused.

In the previous embodiment, after the user opens the individually adapted application, the processor may determine that the current handwriting scene is in the handwriting scene, so that a parallel refresh manner is adopted to increase the display speed, that is, the application also needs to be individually adapted for determining the handwriting scene that needs to be refreshed in parallel. In this embodiment, no matter what application is opened by the user, as long as the touch and press operation is detected, the first composite image and the second composite image are compared, and whether the user is in the handwriting scene currently is determined according to the first change area obtained through comparison. No separate adaptation of the application is required.

In step 503, after it is determined that the ink screen is in the handwriting state, acquiring each composite image from the hardware abstraction layer for preprocessing; and refreshing and displaying the preprocessed composite images in parallel through the ink screen.

When a user carries out handwriting, the user firstly touches and presses, then touches and moves, and finally touches and lifts, so that a complete pen is formed. Therefore, the touch screen can acquire the report point data in real time and upload the report point data to the application aiming at the following touch moving operation of the user. Drawing a handwriting track according to the report point data by the application, drawing the layer data based on the handwriting track, and sequentially transmitting the layer data to a pre-installed display system, such as an android display system, by the application. The hardware abstraction layer of the android display system forms a composite image based on the layer data. For example, during the touch movement of the user, the hardware abstraction layer forms 3 synthetic images based on the layer data, and the 3 synthetic images are respectively shown in fig. 6 c.

The processor takes out each composite image from the hardware abstraction layer and preprocesses each composite image. The embodiment of the present invention does not limit the pretreatment method. The following are 2 exemplary pretreatment methods.

In a first mode

And taking out one composite image from the hardware abstraction layer, and refreshing the composite image directly and parallelly with the previous image without any processing.

For example, when composite image 1 in FIG. 6c is fetched from the hardware abstraction layer, composite image 1 is refreshed in parallel with the first composite image in FIG. 6 a. When the composite image 2 in FIG. 6c is fetched from the hardware abstraction layer, the composite image 2 is refreshed in parallel with the composite image 1, the first composite image in FIG. 6a 82308230, and so on.

Mode two

Each composite image is fetched from the hardware abstraction layer and compared with the previous image fetched from the hardware abstraction layer. And performing parallel refreshing display on the compared difference parts.

Specifically, before the touch-up operation is detected, a second change area is determined according to the third composite image and the fourth composite image; the third composite image and the fourth composite image are any two adjacent composite images obtained from the hardware abstraction layer. And then, performing parallel refreshing display on each second change area through the ink screen. The second variation region is a rectangle that encloses the variation portions of any two adjacent composite images.

For example, after obtaining the composite image 1, comparing the composite image 1 with the first composite image in fig. 6a to obtain a second variation region h, as shown in fig. 6 d; after obtaining the composite image 2, comparing the composite image 2 with the composite image 1 to obtain a second change area i; after the composite image 3 is obtained, the composite image 3 is compared with the composite image 2 to obtain a second variation region j.

And performing parallel refreshing display on each second change area. For example, when the second variation region h is obtained, the second variation region h is refreshed in parallel with the first variation region in fig. 6 b. When the second variation region i is obtained, the second variation region i and the second variation region h, the first variation region in fig. 6b are refreshed in parallel, 82308230, and so on.

Only partial refreshing is carried out on each changed area, so that the calculation amount is reduced, and the refreshing speed is further accelerated.

Because each composite image is taken out from the hardware abstract layer of the pre-installed display system, the composite image is only matched and compatible with the pre-installed display system according to the ink screen product, and the matching of each third-party application is not needed, so that manpower, material resources and time are saved.

In some embodiments, after determining that the ink screen is in the handwriting state, the method further includes: and stopping acquiring the composite image from the hardware abstraction layer to perform system refreshing display of the ink screen.

Specifically, the processor sends an instruction for stopping system refresh display to the hardware abstraction layer, so as to suspend the system refresh display process. So that the decoder does not perform the system refresh display any more, but only performs the parallel refresh display. The display speed is improved, and meanwhile, the computing resources are saved.

In some embodiments, after performing parallel refreshing display on the preprocessed composite images through the ink screen, the method further includes: and after the touch lifting operation is detected, recovering to acquire the composite image from the hardware abstraction layer to perform system refreshing display of the ink screen, and stopping acquiring the composite image from the hardware abstraction layer to perform preprocessing.

And when the user finishes drawing a stroke, the touch screen detects the touch lifting operation, and the processor can recover to obtain the composite image from the hardware abstraction layer to perform the step of refreshing and displaying the system of the ink screen. The hardware abstraction layer buffers a plurality of composite images, such as 3 composite images as shown in fig. 6c, where composite image 3 is the last one. And after the system is restored to refresh and display, the last image of the hardware abstraction layer is refreshed and displayed in the display screen through the system. The image is also displayed by 8 voltages.

Since the last image (i.e., the composite image 3) is displayed with 8 times of voltage, the display speed is fast; and because the user finishes the drawing of the stroke, the user does not pay much attention to the display condition of the finished stroke on the display screen, and the user experience is not influenced. In the prior art, after the composite image 1 is subjected to voltage display for 8 times, the composite image 2 is subjected to voltage display for 8 times; after the composite image 2 is voltage-displayed 8 times, the composite image 3 is voltage-displayed 8 times. The display speed is slow, and the display is performed in the process that the user draws a stroke, so that the user experience is greatly influenced.

After a stroke is finished, the handwriting displayed by the method for refreshing and displaying the system is used for replacing the handwriting displayed by the method for refreshing and displaying the system in parallel, because: 1. the system refreshing display is a conventional mode of refreshing an ink screen, and if a user exits from a third-party application after drawing a stroke, or executes a sliding browsing operation instead of a handwriting operation on the third-party application, the system refreshing display mode is adopted to refresh in time; 2. the system refreshing display has a storage function, and each drawn pen is stored. If the system refreshing display is not switched, if the parallel refreshing display is still adopted, the handwriting of the previous stroke can not be displayed when the next stroke is drawn. Therefore, after the user finishes drawing a stroke, the system is switched to refresh the display.

When the composite images are taken out from the hardware abstraction layer of the pre-installed display system during the parallel refreshing display, and the composite images in the hardware abstraction layer of the pre-installed display system are also obtained based on the layer data drawn by the application, so that the handwriting track obtained by the parallel refreshing based on the composite images is completely consistent with the handwriting track obtained by the system display refreshing based on the pre-installed display system. The user has no visual difference.

If the processor only automatically fits the handwriting track based on the report data and displays the handwriting track in the display screen by a parallel refreshing display method, the automatically fitted handwriting track may not be consistent with the handwriting track obtained by performing system display refreshing based on a pre-installed display system. Affecting the user experience.

In some embodiments, the performing parallel refreshing display on the preprocessed composite images through the ink screen includes: and performing soft decoding on the preprocessed composite images, and performing parallel refreshing display on the decoded image information through the ink screen.

Specifically, for any one of the synthesized images, the one synthesized image is split into 8 frames, and m frames of the 8 frames and m frames of the previous synthesized image are refreshed simultaneously.

Fig. 7 is a diagram illustrating a comparison between the handwriting refresh method provided by the embodiment of the present invention and the handwriting refresh method provided by the previous embodiment, and the handwriting refresh method of the prior art. The difference of several handwriting refreshing methods can be seen visually.

In order to better explain the embodiment of the present invention, the flow of the handwriting refresh of the ink screen will be described below in a specific implementation scenario, as shown in fig. 8.

Step 801, a touch screen detects a touch pressing operation of a user, acquires point reporting data, and transmits the point reporting data to an application and preprocessing module.

The preprocessing module also needs to acquire the report point data, and then can judge whether the first change area is located in a preset range near the touch pressing operation.

And 802, drawing the layer data according to the report point data by the application, and transmitting the layer data to a hardware abstract layer of the android display system.

And step 803, synthesizing the hardware abstraction layer to obtain a first synthesized image.

In step 804, the preprocessing module extracts the composite image from the hardware abstraction layer, and determines a first change area by comparing the first composite image with a previous composite image (a second composite image).

In step 805, the preprocessing module determines whether the first change area is within a preset range near the touch pressing operation, if so, the process goes to step 806, and if not, the process goes to step 807.

If so, the current ink screen is in a handwriting state, and the refreshing mode needs to be switched. If not, the current ink screen is not in a handwriting state, and the refreshing mode does not need to be switched.

In step 806, the preprocessing module issues a pause instruction to the hardware abstraction layer to pause the display through the android display system.

That is, the acquisition of the composite image from the hardware abstraction layer is stopped for a system refresh display of the ink screen.

In step 807, the preprocessing module does not obtain the synthesized image from the hardware abstraction layer any more, and the hardware abstraction layer of the android display system continues to send the synthesized image to the decoding module for system refresh display.

Step 808, before the touch-up operation is detected, the preprocessing module continuously obtains each composite image from the hardware abstraction layer.

In step 809, the preprocessing module determines a second variation region according to the two adjacent composite images.

In step 810, the preprocessing module continuously transmits each second variation region to the decoding module.

And 811, receiving each second change area from the preprocessing module by the decoding module, and performing parallel refreshing display on each second change area through the ink screen.

And 812, after the touch screen detects the touch lifting operation, stopping the preprocessing module from acquiring the synthetic image from the hardware abstraction layer, and sending a recovery instruction to the hardware abstraction layer of the android display system by the preprocessing module so that the hardware abstraction layer continuously sends the synthetic image to the decoding module for system refreshing display. Based on the same technical concept, fig. 9 exemplarily shows a structure of an ink screen handwriting refreshing apparatus according to an embodiment of the present invention, where the structure can perform a flow of ink screen handwriting refreshing.

As shown in fig. 9, the apparatus specifically includes:

a processing module 901 configured to:

a preprocessing module 902 configured to:

the processing module 901 is further configured to:

In some embodiments of the present application, the preprocessing module 902 is further configured to:

In some embodiments of the present application, the preprocessing module 902 is specifically configured to:

in some embodiments of the present application, the processing module 901 is specifically configured to:

and after the touch lifting operation is detected, recovering to acquire the composite image from the hardware abstraction layer for system refreshing display of the ink screen, and stopping acquiring the composite image from the hardware abstraction layer for preprocessing.

and performing soft decoding on the preprocessed composite images, and performing parallel refreshing display on the decoded image information through the ink screen.

In some embodiments of the present application, the processing module 901 is further configured to:

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. A handwriting refreshing method of an ink screen is characterized by comprising the following steps:

2. The method of claim 1, wherein determining that the ink screen is in a handwritten state further comprises:

3. The method of claim 1, wherein obtaining each composite image from the hardware abstraction layer for pre-processing comprises:

4. The method of claim 3, wherein the second variation region is a rectangle that encloses the variation portions of any two adjacent composite images.

5. The method of claim 2, wherein after refreshing the preprocessed composite images for display in parallel through the ink screen, further comprising:

6. The method of claim 1, wherein refreshing the preprocessed composite images for display in parallel through the ink screen comprises:

7. The method of any one of claims 1-6, further comprising:

8. An ink screen handwriting refreshing device, comprising:

a processing module to:

a pre-processing module to:

the processing module is further configured to:

9. A terminal device, characterized in that the terminal device is equipped with at least one application; the terminal device includes:

a memory for storing a computer program;

the processor, for calling a computer program stored in the memory, for executing the method of any one of claims 1 to 7 according to the obtained program.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer-executable program for causing a computer to execute the method of any one of claims 1 to 7.