CN118229858A - Image processing method, device, terminal and storage medium - Google Patents

Abstract

The disclosure provides an image processing method and device, a terminal and a storage medium. The image processing method comprises the following steps: determining an image to be displayed; drawing an image to be displayed to obtain a drawing image; identifying the drawn image; when the preset elements are identified in the drawing image, removing the preset elements in the drawing image to obtain a processed image; rendering the processed image to obtain a rendered image. The image processing method provided by the disclosure can allow the application to normally run on the low-end equipment under the condition that the basic function is ensured to be normally available, and the user experience is improved.

Description

Image processing method, device, terminal and storage medium

Technical Field

The disclosure relates to the field of information technology, and in particular, to an image processing method and device, a terminal and a storage medium.

Background

Performance optimization of third party applications is a very difficult task, especially in some low-end devices (e.g., TV devices) for some large mobile applications. These large applications are very heavy in their own resources in order to achieve the experience. Although the application has a complete set of performance optimization strategies, it is difficult to consider each chip platform, especially TV devices. Most of the TV devices on the market at present adopt a processor with a lower end, have poor experience when running large-scale applications, are relatively card-like to use, and cannot even be used.

Disclosure of Invention

In order to solve the existing problems, the present disclosure provides an image processing method and apparatus, a terminal, and a storage medium.

The present disclosure adopts the following technical solutions.

An embodiment of the present disclosure provides an image processing method including: determining an image to be displayed; drawing the image to be displayed to obtain a drawing image; identifying the drawn image; when a preset element is identified in the drawing image, removing the preset element in the drawing image to obtain a processed image; and rendering the processed image to obtain a rendered image.

Another embodiment of the present disclosure provides an image processing apparatus including: a determining module configured to determine an image to be displayed; the drawing module is configured to draw the image to be displayed to obtain a drawn image; an identification module configured to identify the drawn image; the processing module is configured to remove preset elements in the drawing image when the preset elements are identified in the drawing image, so as to obtain a processing image; and the rendering module is configured to render the processed image to obtain a rendered image.

In some embodiments, the present disclosure provides a terminal comprising: at least one memory and at least one processor; the memory is used for storing program codes, and the processor is used for calling the program codes stored in the memory to execute the image processing method.

In some embodiments, the present disclosure provides a storage medium for storing program code for performing the above-described image processing method.

According to the embodiment of the disclosure, the drawing image is identified, when the preset element is identified in the drawing image, the preset element in the drawing image is removed, and the processing image is obtained, so that the application can be allowed to normally run on the low-end equipment under the condition that the basic function is ensured to be normally available, and the user experience is improved.

Drawings

The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale.

Fig. 1 shows a general flow of the conventional image display.

Fig. 2 is a flowchart of an image processing method of an embodiment of the present disclosure.

Fig. 3 shows a schematic flow chart of an image processing method of an embodiment of the present disclosure.

Fig. 4 is a partial block diagram for an image processing apparatus according to another embodiment of the present disclosure.

Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure have been shown in the accompanying drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.

It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in and/or in parallel. Furthermore, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.

The term "including" and variations thereof as used herein are intended to be open-ended, i.e., including, but not limited to. The term "based on" is based at least in part on. The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments. Related definitions of other terms will be given in the description below.

It should be noted that the terms "first," "second," and the like in this disclosure are merely used to distinguish between different devices, modules, or units and are not used to define an order or interdependence of functions performed by the devices, modules, or units.

It should be noted that references to "a" and "an" in this disclosure are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be construed as "one or more" unless the context clearly indicates otherwise.

The names of messages or information interacted between the various devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

Performance optimization of third party applications for some low-end devices often requires pushing third party modification applications. However, the lower the end equipment, the greater the optimization difficulty, the less the volume of shipment, and the overall input to output ratio is not high for third party applications. Therefore, on the premise of not modifying the third party application, the problem of blocking of the large-scale third party application is expected to be solved, and the experience effect is improved. The optimization scheme can not destroy the main flow of the original service, for example, the original service can not be caused to frequently respond without response, crash and the like. In addition, the code intrusion of the optimization scheme is expected to be low, the whole system flow is not destroyed, and other unpredictable problems are not caused.

The running of the application mainly involves image processing. Fig. 1 shows a general flow of image display on a TV device, for example. For mobile applications, in step 10, the drawing of the image to be displayed is first performed, for example, by a Window Management Service (WMS) of a frame work to achieve drawing of a view or image. In step 11, the central processing unit CPU sends a rendering instruction to the image processor GPU to render the drawn image. In step 12, the data stream of the rendered image is GPU or hardware hybrid renderer (HWC) synthesized and finally displayed by the HWC. In some embodiments, the data stream of the rendered image is GPU or HWC synthesized, for example, by SrufaceFlinger.

Fig. 2 provides a flowchart of an image processing method of an embodiment of the present disclosure. The image processing method of the present disclosure may include step S101 of determining an image to be displayed. In some embodiments, before an image or image frame is displayed, the image or image frame to be displayed is first determined, and then the image can be rendered and used for display.

In some embodiments, the method of the present disclosure may further include step S102 of rendering an image to be displayed, resulting in a rendered image. As also shown in fig. 1, the drawing of images is typically accomplished through a frame work Window Management Service (WMS).

In some embodiments, the method of the present disclosure may further include step S103 of identifying the drawn image. In some embodiments, in order to normally run large applications in low-end devices, some processing is required to improve the stability and smoothness of application running without affecting the implementation of the functions of these applications. In some embodiments of the present disclosure, by identifying the drawn image, some elements that can be omitted may be identified for the following processing.

In some embodiments, the method of the present disclosure may further include step S104, when a preset element is identified in the drawn image, removing the preset element in the drawn image to obtain a processed image. In some embodiments, if no preset element is identified in the drawn image, no processing is performed, and subsequent rendering may be performed directly. When the preset elements are identified in the drawing image, the elements which need to be rendered subsequently can be reduced by removing the preset elements in the drawing image, so that a CPU (Central processing Unit) sends rendering instructions to the GPU, the GPU utilization rate is reduced, and therefore, some applications with higher requirements can be ensured to run in low-end equipment.

In some embodiments, the method of the present disclosure may further include step S105, rendering the processed image to obtain a rendered image. After the processed image is obtained, normal rendering of the image is performed.

By identifying and removing preset elements in the image, elements with smaller influence on the use of the applications can be identified and removed, the use of rendering of the GPU is reduced, and the flow and stability of the applications running on low-end devices (such as TV devices with poorer processors) are improved on the basis of ensuring the basic functions of the applications.

In some embodiments, the methods of the present disclosure may further include displaying the rendered image. As described above, after the rendered image is obtained, the data stream of the rendered image may be GPU or hardware hybrid renderer (HWC) synthesized and finally displayed by the HWC. In some embodiments, the data stream of the rendered image is GPU or HWC synthesized, for example, by SrufaceFlinger.

In some embodiments, identifying the rendered image includes: the drawn image is identified by an image processor rendering program. In some embodiments, GPU rendering programs are mainly OpenGL, skiaGL, skiaVulkan, most of which are currently SkiaGL. Thus, the method of the present disclosure is implemented without adding any new program.

In some embodiments, identifying, by the image processor rendering program, the drawn image includes: and identifying the scene corresponding to the drawn image through an image processor rendering program. In some embodiments, different applications have different scenes, and by identifying the scenes of the image of the corresponding application, it can be known which elements have less influence on the normal use of the application, and further identify the elements having less influence on the normal use of the application.

In some embodiments, the preset elements include special effects elements. In some embodiments, some special effects elements have less effect on the implementation of the basic functions of the application, e.g., caption drift special effects, some stones in the game, grass special effects, etc. In some embodiments, the preset elements may be different for different applications. In some embodiments, a mapping table of applications and preset elements may be established, so that it may be easily determined which elements of the corresponding applications may be omitted or removed, which facilitates quick and accurate identification of the preset elements.

In some embodiments, removing preset elements in the rendered image comprises: and removing preset elements in the drawn image in a blocking or clipping mode. In some embodiments, interception or clipping of a preset element in the drawn image may be achieved by the GPU rendering program such that the element is not embodied in the image to be rendered.

In some embodiments, after obtaining the rendered image, the method of the present disclosure further comprises: the data streams of the rendered image are synthesized by an image processor or a hardware hybrid renderer and displayed by the hardware hybrid renderer.

According to the method and the device, through identification of the preset elements, some elements which do not affect user experience are filtered or removed, GPU drawing and synthesis are reduced, so that GPU utilization rate is reduced, and occupation of a CPU is further reduced. In addition, the CPU can be reduced to send rendering instructions to the GPU, and the utilization rate of the GPU is reduced. In the synthesis stage, no matter GPU synthesis or HWC synthesis, the cost of the part required by the preset elements can be saved, so that the utilization rate of the CPU and the GPU is reduced, and the performance of the large-scale mobile application, especially the stability and fluency of running on low-end equipment, is improved.

The following description is provided in connection with fig. 3 to better understand the present disclosure. For the sake of simplicity, the same steps as in fig. 1 are not described in detail in fig. 3. In step 30, the scene of the application may be identified by Skia to identify those elements or scene information that have less impact on the normal use of the application. In step 31, after the specified scene information or the preset element is identified, the specified scene information or the preset element is intercepted, so that the elements required to be rendered subsequently are reduced, the utilization rate of the CPU and the GPU is reduced, the stability and the fluency of the running of the application are improved, and the user experience is improved.

The method of the disclosure avoids code invasion, and if the bottom code is modified, the main business flow of the application may not be available; in addition, the method can ensure that the application can normally run in the low-end equipment and ensure that basic functions are available.

Embodiments of the present disclosure also provide an image processing apparatus 400. The image processing apparatus 400 includes a determination module 401, a drawing module 402, an identification module 403, a processing module 404, and a rendering module 405. In some embodiments, the determination module 401 is configured to determine an image to be displayed. In some embodiments, the drawing module 402 is configured to draw the image to be displayed, resulting in a drawn image. In some embodiments, the identification module 403 is configured to identify the drawn image. In some embodiments, the processing module 404 is configured to remove a preset element in the drawn image when the preset element is identified in the drawn image, resulting in a processed image. In some embodiments, the rendering module 405 is configured to render the processed image to obtain a rendered image.

It should be understood that the descriptions regarding the image processing method also apply to the image processing apparatus 400 herein, and for the sake of simplicity, will not be described in detail herein.

In some embodiments, the image processing apparatus further includes: and a display module configured to display the rendered image. In some embodiments, identifying the rendered image includes: and identifying the drawn image through an image processor rendering program. In some embodiments, identifying the drawn image by an image processor rendering program includes: and identifying the scene corresponding to the drawn image through an image processor rendering program. In some embodiments, the preset elements include special effects elements. In some embodiments, removing the preset elements in the rendered image comprises: and removing the preset elements in the drawn image in a blocking or clipping mode. In some embodiments, the display module is further configured to, after obtaining the rendered image, render the data stream of the rendered image for image processor or hardware hybrid renderer synthesis and display by the hardware hybrid renderer.

In addition, the present disclosure also provides a terminal, including: at least one memory and at least one processor; the memory is used for storing program codes, and the processor is used for calling the program codes stored in the memory to execute the image processing method.

Further, the present disclosure also provides a computer storage medium storing a program code for executing the above-described image processing method.

The image processing method and apparatus of the present disclosure are described above based on the embodiments and application. In addition, the present disclosure also provides a terminal and a storage medium, which are described below.

Referring now to fig. 5, a schematic diagram of an electronic device (e.g., a terminal device or server) 500 suitable for use in implementing embodiments of the present disclosure is shown. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and stationary terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 5 is merely an example and should not be construed to limit the functionality and scope of use of the disclosed embodiments.

As shown in fig. 5, the electronic device 500 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 501, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 502 or a program loaded from a storage means 508 into a Random Access Memory (RAM) 503. In the RAM503, various programs and data required for the operation of the electronic apparatus 500 are also stored. The processing device 501, the ROM 502, and the RAM503 are connected to each other via a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.

In general, the following devices may be connected to the I/O interface 505: input devices 506 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 507 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 508 including, for example, magnetic tape, hard disk, etc.; and communication means 509. The communication means 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. While fig. 5 shows an electronic device 500 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead.

In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 509, or from the storage means 508, or from the ROM 502. The above-described functions defined in the methods of the embodiments of the present disclosure are performed when the computer program is executed by the processing device 501.

It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.

In some embodiments, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.

The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to perform the methods of the present disclosure described above.

Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units involved in the embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.

The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

According to one or more embodiments of the present disclosure, there is provided an image processing method including: determining an image to be displayed; drawing the image to be displayed to obtain a drawing image; identifying the drawn image; when a preset element is identified in the drawing image, removing the preset element in the drawing image to obtain a processed image; and rendering the processed image to obtain a rendered image.

According to one or more embodiments of the present disclosure, the image processing method further includes: and displaying the rendered image.

In accordance with one or more embodiments of the present disclosure, identifying the rendered image includes: and identifying the drawn image through an image processor rendering program.

In accordance with one or more embodiments of the present disclosure, identifying the drawn image by an image processor rendering program includes: and identifying the scene corresponding to the drawn image through an image processor rendering program.

According to one or more embodiments of the present disclosure, the preset elements include special effects elements.

According to one or more embodiments of the present disclosure, removing the preset elements in the drawn image includes: and removing the preset elements in the drawn image in a blocking or clipping mode.

According to one or more embodiments of the present disclosure, after obtaining the rendered image, further comprising: the data stream of the rendered image is synthesized by an image processor or a hardware hybrid renderer and displayed by the hardware hybrid renderer.

According to one or more embodiments of the present disclosure, there is provided an image processing apparatus including: a determining module configured to determine an image to be displayed; the drawing module is configured to draw the image to be displayed to obtain a drawn image; an identification module configured to identify the drawn image; the processing module is configured to remove preset elements in the drawing image when the preset elements are identified in the drawing image, so as to obtain a processing image; and the rendering module is configured to render the processed image to obtain a rendered image.

According to one or more embodiments of the present disclosure, there is provided a terminal including: at least one memory and at least one processor; wherein the at least one memory is configured to store program code, and the at least one processor is configured to invoke the program code stored by the at least one memory to perform any of the methods described above.

According to one or more embodiments of the present disclosure, there is provided a storage medium for storing program code for performing the above-described method.

The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by persons skilled in the art that the scope of the disclosure referred to in this disclosure is not limited to the specific combinations of features described above, but also covers other embodiments which may be formed by any combination of features described above or equivalents thereof without departing from the spirit of the disclosure. Such as those described above, are mutually substituted with the technical features having similar functions disclosed in the present disclosure (but not limited thereto).

Moreover, although operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are example forms of implementing the claims.

Claims (10)

1. An image processing method, characterized in that the image processing method comprises:

Determining an image to be displayed;

drawing the image to be displayed to obtain a drawing image;

Identifying the drawn image;

when a preset element is identified in the drawing image, removing the preset element in the drawing image to obtain a processed image;

and rendering the processed image to obtain a rendered image.

2. The image processing method according to claim 1, characterized by further comprising:

And displaying the rendered image.

3. The image processing method according to claim 1, wherein identifying the drawn image includes: and identifying the drawn image through an image processor rendering program.

4. The image processing method according to claim 3, wherein the identifying of the drawn image by an image processor rendering program includes:

and identifying the scene corresponding to the drawn image through an image processor rendering program.

5. The image processing method according to claim 1, wherein the preset element includes a special effect element.

6. The image processing method according to claim 1, wherein removing the preset element in the drawn image includes: and removing the preset elements in the drawn image in a blocking or clipping mode.

7. The image processing method according to claim 1, characterized by further comprising, after obtaining the rendered image: the data stream of the rendered image is synthesized by an image processor or a hardware hybrid renderer and displayed by the hardware hybrid renderer.

8. An image processing apparatus, characterized in that the image processing apparatus comprises:

a determining module configured to determine an image to be displayed;

the drawing module is configured to draw the image to be displayed to obtain a drawn image;

An identification module configured to identify the drawn image;

The processing module is configured to remove preset elements in the drawing image when the preset elements are identified in the drawing image, so as to obtain a processing image;

And the rendering module is configured to render the processed image to obtain a rendered image.

9. A terminal, comprising:

At least one memory and at least one processor;

Wherein the at least one memory is configured to store program code, and the at least one processor is configured to invoke the program code stored in the at least one memory to perform the image processing method of any of claims 1 to 7.

10. A storage medium storing program code for executing the image processing method according to any one of claims 1 to 7.