WO2021008418A1 - 图层合成方法、装置、电子设备及存储介质 - Google Patents

图层合成方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2021008418A1
WO2021008418A1 PCT/CN2020/100852 CN2020100852W WO2021008418A1 WO 2021008418 A1 WO2021008418 A1 WO 2021008418A1 CN 2020100852 W CN2020100852 W CN 2020100852W WO 2021008418 A1 WO2021008418 A1 WO 2021008418A1
Authority
WO
WIPO (PCT)
Prior art keywords
layer
bits
displayed
interface
preset
Prior art date
Application number
PCT/CN2020/100852
Other languages
English (en)
French (fr)
Inventor
胡杰
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Priority to EP20839629.1A priority Critical patent/EP4002281A4/en
Publication of WO2021008418A1 publication Critical patent/WO2021008418A1/zh
Priority to US17/576,640 priority patent/US20220139017A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/60—Editing figures and text; Combining figures or text
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/363—Graphics controllers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/37—Details of the operation on graphic patterns
    • G09G5/377—Details of the operation on graphic patterns for mixing or overlaying two or more graphic patterns
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/39—Control of the bit-mapped memory
    • G09G5/395—Arrangements specially adapted for transferring the contents of the bit-mapped memory to the screen
    • G09G5/397—Arrangements specially adapted for transferring the contents of two or more bit-mapped memories to the screen simultaneously, e.g. for mixing or overlay
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2330/00—Aspects of power supply; Aspects of display protection and defect management
    • G09G2330/02—Details of power systems and of start or stop of display operation
    • G09G2330/021—Power management, e.g. power saving
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/10—Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/12—Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2360/00—Aspects of the architecture of display systems
    • G09G2360/06—Use of more than one graphics processor to process data before displaying to one or more screens
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2360/00—Aspects of the architecture of display systems
    • G09G2360/08—Power processing, i.e. workload management for processors involved in display operations, such as CPUs or GPUs
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • This application relates to the field of image processing technology, and more specifically, to a layer synthesis method, device, electronic equipment, and storage medium.
  • the display interface of electronic devices usually has multiple display layers, which are displayed by combining multiple display layers. On electronic devices. However, when the electronic device synthesizes the layer, the power consumption is high.
  • this application proposes a layer synthesis method, device, electronic equipment, and storage medium to improve the above problems.
  • an embodiment of the present application provides a layer composition method to obtain the number of bits of each of the multiple layers of the interface to be displayed; the layer with the number of bits less than the preset number of bits is used as the first image Layer, the layer with the number of bits greater than or equal to the preset number of bits is used as the second layer; the first layer is synthesized by a graphics processing unit (GPU, Graphics Processing Unit) to obtain an intermediate layer; A processor (MDP, Multimedia Display Processor) synthesizes the second layer and the intermediate layer to obtain an interface to be displayed for display.
  • GPU Graphics Processing Unit
  • a processor MDP, Multimedia Display Processor
  • an embodiment of the present application provides a layer composition device, a bit number obtaining module, used to obtain the bit number of each of the multiple layers of the to-be-displayed interface;
  • the layer classification module used to The layer with the number of bits less than the preset number of bits is used as the first layer, and the layer with the number of bits greater than or equal to the preset number of bits is used as the second layer;
  • the first synthesis module is used to pass graphics processing units (GPU, Graphics). Processing Unit) synthesizes the first layer to obtain an intermediate layer;
  • a second synthesis module is used to combine the second layer and the intermediate layer through a multimedia display processor (MDP, Multimedia Display Processor) Perform synthesis to obtain the to-be-displayed interface for display.
  • MDP Multimedia Display Processor
  • an embodiment of the present application provides an electronic device, including a memory, a processor, a graphics processing unit (GPU, Graphics Processing Unit), and a multimedia display processor (MDP, Multimedia Display Processor), the memory, GPU, and The MDP is coupled to the processor, and the memory stores instructions.
  • the processor executes the aforementioned method
  • the GPU and the MDP execute the layers in the aforementioned method. synthesis. .
  • an embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores program code, and the program code can be invoked by a processor to execute the above-mentioned method.
  • Fig. 1 shows a logical framework diagram of image processing provided by an embodiment of the present application.
  • Fig. 2 shows a schematic diagram of a display interface provided by an embodiment of the present application.
  • FIG. 3 shows a schematic diagram of layers corresponding to the display interface shown in FIG. 2.
  • Fig. 4 shows a flow chart of the layer composition method provided by the embodiment of the present application.
  • FIG. 5 shows another flowchart of the layer composition method provided by the embodiment of the present application.
  • FIG. 6 shows another flowchart of the layer composition method provided by the embodiment of the present application.
  • Fig. 7 shows a functional block diagram of a layer composition device provided by an embodiment of the present application.
  • Fig. 8 shows a structural block diagram of an electronic device provided by an embodiment of the present application.
  • Fig. 9 is a storage medium for storing or carrying program codes for implementing the layer composition method according to the embodiment of the present application.
  • An application can create a window through the Windows manager.
  • the window manager creates a drawing surface for each window to draw various elements that need to be displayed on it, and each surface corresponds to one Layer, that is, a layer can be drawn on each surface.
  • each layer is rendered before composition. That is, each layer is drawn on the corresponding surface, that is, the corresponding layer is drawn on each surface.
  • the canvas provided by the surface can be provided by the hardware accelerated renderer (HWUI) and/or the Skia graphics library. ) Perform layer drawing.
  • the system then uses the layer transfer module (Surface Flinger) service to synthesize each surface, that is, synthesize each layer.
  • the SurfaceFlinger service runs in the system process and is used to manage the frame buffer (FrameBuffer) of the system.
  • SurfaceFlinger obtains all the layers, and can use the graphics processor (GPU, Graphics Processing Unit) to synthesize the layers and combine The result is saved to the frame buffer area.
  • the GPU can synthesize all or part of the layers.
  • the hardware layer mixer HWC, Hardware Composer
  • HWC Hardware Composer
  • HWC can call the Multimedia Display Processor (MDP, Multimedia Display Processor) to synthesize the layer obtained after the GPU synthesis in the frame buffer and other uncomposited layers, and finally form the BufferQueue A Buffer, and then under the action of the display driver, the image synthesized in the Buffer is used for display.
  • MDP Multimedia Display Processor
  • the display screen of the electronic device can display the display interface, and the display interface may include multiple layers, that is, the display interface is composed of multiple layers.
  • each layer is composed of many pixels, and each layer is superimposed to form the entire image to form the display interface of the electronic device.
  • Layers can accurately locate elements on the page. Text, pictures, tables, plug-ins can be added to the layer, and layers can also be nested inside.
  • a layer is like a film containing elements such as text or graphics.
  • the layers are stacked in a specified order to form the final image. The effect is the image formed by the combination of layers, such as the display interface of an electronic device.
  • the desktop display interface 100 of the electronic device shown in FIG. 2 includes multiple layers, including a status bar 101, a navigation bar 102, a wallpaper 103, and an icon layer 104.
  • the status bar 101, the navigation bar 102, the wallpaper 103, and the icon layer 104 shown in FIG. 3 are synthesized into the display interface 100 shown in FIG. 2.
  • GPU is a general-purpose graphics processor, it is more powerful than MDP in graphics processing.
  • 2D image processing In addition to 2D image processing, it can also do 3D image processing, special effects, etc., and can superimpose multiple layers at once.
  • GPU When only using the commonly used layer synthesis method GPU for layer synthesis, the number of layers that can be synthesized is basically unlimited, but the power consumption is relatively high.
  • the MDP is a dedicated display image processing unit that can do conventional 2-dimensional image processing, mainly used for composite overlay of multiple layers, and has low power consumption.
  • the number of layers that MDP can synthesize at a time is limited. Specifically, the number of layers synthesized by MDP performance at one time corresponds to the number of FIFO pipes in MDP. The more layers that MDP can superimpose at one time, the more FIFO pipes are needed inside. These are more expensive circuits. Therefore, The more layers that MDP can synthesize, the higher the cost. In the high-end platforms of some manufacturers, the MDP has only 8 FIFO pipes at most, and can synthesize and superimpose up to 8 layers at a time.
  • an MDP may only have 4 FIFO pipes, and it can only be used at a time. Overlay 4 layers. Therefore, if only MDP is used for layer synthesis, since MDP can synthesize a limited number of layers at one time, MDP cannot handle some display interfaces with many layers to be displayed, or it needs to be synthesized multiple times, which affects the synthesis. speed. Therefore, the inventor proposes the layer composition method, device, electronic device, and storage medium of the embodiments of the present application to perform layer composition through the cooperation of GPU and MDP, which overcomes the limitation of MDP on the number of composite layers and reduces the power as much as possible. Consumption.
  • the layers with a relatively large number of bits are synthesized by MDP, which has lower power consumption than using GPU for synthesis; due to the limited number of layers that can be synthesized by MDP at one time, other layers with a smaller number of bits GPU synthesis is used to achieve the lowest possible power consumption during the layer synthesis process.
  • the layer composition method of the embodiment of the present application will be described in detail below.
  • FIG. 4 shows a flowchart of a layer composition method provided by an embodiment of the present application, and the layer composition method is applied to an electronic device.
  • the layer composition method includes:
  • Step S110 Obtain the number of bits of each of the multiple layers of the interface to be displayed.
  • the display interface is displayed on the display screen.
  • the display interface displayed on the display screen is used as the interface to be displayed, and includes multiple layers. Among them, the number of bits in each layer is different. The larger the number of bits, the greater the power consumption during the layer synthesis process.
  • the layers can be divided into Two types, so that the layer with a larger number of bits is synthesized by MDP, so that the power consumption in the layer synthesis process is as low as possible; the other type of layer with a smaller number of bits is synthesized by GPU to overcome the existence of MDP The number of synthesis is limited, and the synthesis speed is increased.
  • the number of bits of each layer in the interface to be displayed can be obtained.
  • the number of bits of a layer indicates the data size of the layer, or how much storage capacity the layer needs, such as 16 bits, 32 bits, etc.
  • the embodiment of the present application may also use the number of bytes or other representation methods to indicate the data size of the layer. The larger the data, the larger the corresponding bit.
  • the embodiment of the present application takes the number of bits as an example for description.
  • the manner of obtaining the size of the bit number of the layer is not limited in the embodiment of the present application.
  • the number of pixels of a layer can be read, and the number of bits of the layer can be calculated by multiplying the number of pixels by the number of bits occupied by each pixel according to the number of bits occupied by each pixel.
  • a layer includes one or more elements, and the sum of the number of bits of various elements in each layer can be obtained as the number of bits of the layer.
  • the elements in the layer are the data that the layer is used for display and need to occupy storage space, such as text, pictures, etc.
  • the element in the layer is the picture
  • the number of bits in the layer is the number of bits in the picture.
  • the number of bits in the icon layer 104 is each The sum of the bits of the icon.
  • the elements in the layer are the text displayed in the layer, and the number of bits in the layer can be the sum of the number of bits in the text in the layer.
  • the calculation method of the number of bits of various elements in the layer is not limited. If yes, for any element, calculate the size of each element according to the number of the smallest component unit of the element and the number of bits occupied by each smallest component unit; or, in the relevant data of an element, the bit of the element is stored Number information, you can directly read the bit number information to determine the bit number of the element.
  • the way to calculate the number of bits in the picture can be to read the number of pixels in the picture, and according to the number of bits occupied by each pixel, multiply the number of pixels by the number of bits occupied by each pixel Calculate the number of bits of the picture; or it can be that the relevant information of the number of bits of the picture is stored in the data information of the picture, and the relevant information of the number of bits is directly read to obtain the number of bits of the picture.
  • Step S120 Use a layer with a number of bits less than a preset number of bits as the first layer, and use a layer with a number of bits greater than or equal to the preset number of bits as the second layer.
  • the number of bits of each layer is compared with the preset number of bits, the layer with the number of bits less than the preset number of bits is taken as the first layer, and the layer with the number of bits greater than or equal to the preset number of bits is taken as the second image Floor.
  • the specific value of the preset number of bits is not limited in the embodiment of this application, and can be set according to actual needs.
  • the value of the preset number of bits can be set according to the number of layers and the number of bits normally synthesized by the electronic device, so that the number of layers larger than the preset number of bits is less than or equal to the maximum number of layers synthesized by the MDP at one time.
  • Step S130 Synthesize the first layer through a graphics processing unit (GPU, Graphics Processing Unit) to obtain an intermediate layer.
  • a graphics processing unit GPU, Graphics Processing Unit
  • Step S140 Synthesize the second layer and the intermediate layer by a multimedia display processor (MDP, Multimedia Display Processor) to obtain an interface to be displayed for display.
  • MDP Multimedia Display Processor
  • the first layer and the second layer are synthesized by GPU and MDP.
  • MDP is limited to the number of layers for synthesis, not the number of bits of the layer, and in the same synthesis method, the layer with a larger number of bits consumes more power than a layer with a smaller number of bits. high. Therefore, the first layer with a smaller number of bits is synthesized by the GPU, so that the GPU not only shares part of the layers for the MDP, but also synthesizes with a smaller number of bits to reduce the synthesis power consumption.
  • the second layer with the larger number of bits is synthesized by MDP.
  • the GPU composites the first layer and defines the composite result of the first layer as an intermediate layer.
  • MDP combines the middle layer and the second layer. Obtain the interface to be displayed for display.
  • the multiple layers in the desktop display interface 100 shown in FIG. 2 are the status bar 101, the navigation bar 102, the wallpaper 103, and the icon layer 104. If the number of bits in the status bar 101 and the navigation bar 102 is less than the preset number of bits. If the number of bits of the wallpaper 103 and the icon layer 104 is greater than or equal to the preset number of bits, the status bar 101 and the navigation bar 102 are used as the first layer; the wallpaper 103 and the icon layer 104 are used as the second layer.
  • the GPU synthesizes the status bar 101 and the navigation bar 102, and the MDP synthesizes the synthesis result of the GPU, the wallpaper 103, and the icon layer 104 to obtain the final synthesized interface to be displayed.
  • the first layer after the first layer is synthesized to obtain the intermediate layer, it can be stored in the frame buffer. If all the layers included in the intermediate layer are not updated during the update process of the interface to be displayed on the screen, the frame buffer The intermediate layer in the area remains unchanged, and the GPU does not need to synthesize the intermediate layer again, reducing the synthesis frequency of the GPU.
  • the MDP can read the middle layer in the frame buffer and use it together with the second layer as the final image for display. That is, when the interface to be displayed is updated, the multimedia display processor can read the intermediate layer of the frame buffer, and synthesize the intermediate layer with other layers of the new interface to be displayed.
  • the MDP can synthesize the second layer at the same time, so that the synthesis of the first layer and the synthesis of the second layer can be performed simultaneously, which can reduce the synthesis power consumption , And can increase the synthesis speed.
  • the layer obtained after the MDP synthesizes the second layer may be defined as the layer to be synthesized. MDP synthesizes the layer to be synthesized and the intermediate layer to obtain the final interface to be displayed.
  • the intermediate layer may be saved in the frame buffer, and after the MDP synthesizes the second layer into the layer to be synthesized, the intermediate layer of the frame buffer is read, and the intermediate layer is combined with the The layers to be synthesized are synthesized into the interface to be displayed. Or if the MDP is in the process of compositing the second layer and the middle layer has been combined, the MDP can read the middle layer of the frame buffer and combine the middle layer and the second layer together.
  • the layer with the number of bits less than the preset number of bits is synthesized by the GPU as the intermediate layer, and the intermediate layer and the layer with the number of bits greater than or equal to the preset number of bits are synthesized by the MDP to obtain the final layer
  • the synthesis result is used for display, thus, through the combination of MDP and GPU, the power consumption of the electronic device in the layer synthesis process is as low as possible.
  • the size of the preset number of bits used to determine the first layer and the second layer can also be dynamically adjusted, so that the number of the first layer and the second layer The number of is in a reasonable range, which not only reduces the power consumption of layer synthesis but also increases the speed of synthesis.
  • the method includes:
  • Step S210 Obtain the number of bits of each of the multiple layers of the interface to be displayed.
  • Step S220 dynamically adjust the size of the preset number of bits.
  • Step S230 Use a layer with a number of bits less than the preset number of bits as the first layer, and a layer with a number of bits greater than or equal to the preset number of bits as the second layer.
  • the preset number of bits is the standard for dividing the first layer and the second layer.
  • different electronic devices have different user usage tendencies, so the number of layers on the interface displayed on the electronic device and the structure of the number of layer bits Different, in addition, watching different content at different times, the number of layers in the display interface may be quite different.
  • the preset number of bits is set too large or too small, it may cause too much of the first layer or too much of the second layer, which exceeds the processing capacity of MDP or MDP synthesis. Too few layers will affect the layer composition effect.
  • the embodiment of the present application can dynamically adjust the size of the preset number of bits.
  • the content browsed by the user in a period of time may be relatively close, and the total number of bits of the layers between the interfaces to be displayed is relatively close.
  • the layers included in the display interface may be all In the video layer where the video content is located and the background layer other than the video content, the number of bits and the number of layers included in each display interface are relatively close. Therefore, the preset number of bits can be dynamically adjusted according to the historical display record of the electronic device.
  • the preset number of bits may be adjusted once every preset time, and the adjustment is made according to the number of bits in the display interface within the preset time interval.
  • the average value of the number of bits of all layers used for the interface to be displayed within the preset time range before the current time is determined at every preset time when the screen is in the display state;
  • the preset number of bits is determined at every preset time when the screen is in the display state;
  • the preset number of bits the number of bits of each layer in each display interface can be recorded.
  • calculate the average value by dividing the total number of bits in the latest preset time range by the total number of layers.
  • the adjustment ladder is not limited in the embodiments of the present application. It can be a preset smaller value, or it can be the number of bits of the layer with the smallest number of bits within the preset time range. Try to avoid the preset number of bits. Too much adjustment at one time leads to too small number of second layers, which is far from the processing power of MDP. Among them, the maximum number of layers that the MDP can process at one time indicates the MDP synthesis capability.
  • the preset number of bits as the classification standard is relatively close, and the stepwise adjustment of the preset number of bits can only be based on the average value. Set the preset number of bits and do it once, instead of multiple adjustments within the preset time range, reducing the amount of data processing.
  • the number of the first layer and the second layer is too large to adjust the value of the preset number of bits.
  • the number of the first layer and the second layer in each interface to be displayed can be recorded; it is determined whether the number of the first layer is larger than the number of the second layer in the continuous preset number of interfaces to be displayed. Set the difference.
  • the number of the first layer in the interface to be displayed is larger than the number of the second layer by the preset difference, it indicates that the number of the first layer is too much, and the number of the second layer is too small, then the The value of the preset number of bits is reduced to increase the number of second layers and reduce the number of first layers during subsequent classification.
  • the continuous preset number of to-be-displayed interfaces does not meet the preset difference between the number of the first layer and the second layer, then judge whether the continuous preset number of to-be-displayed interfaces meet the number of the second layer The preset difference is larger than the number of the first layer. If the continuous preset number of to-be-displayed interfaces meets the requirement that the number of the second layer is greater than the number of the first layer by the preset difference, it indicates that the number of the second layer is too large, and the number of the first layer is too small, and the The value of the preset number of bits is increased to increase the number of the first layer and reduce the number of the second layer during subsequent classification.
  • the execution timing of dynamically adjusting the size of the preset number of bits may not be restricted by the execution timing of other steps.
  • the layers of the interface to be displayed are divided into the first layer and the second layer.
  • the current preset number of bits is used as the classification basis, that is, during classification, if the preset number of bits is adjusted, the adjusted number will be used as the classification basis; if the preset number of bits has not been adjusted, the The adjusted preset number of bits is used as the classification basis.
  • the number of second layers in the interface to be displayed is generally not too many, when the preset number of bits is used as the classification criterion, the number of second layers usually does not exceed the limit of MDP synthesis capability . Assuming that there are too many second layers in a certain time, which exceeds the synthesis capability of the MDP, the layer with the smallest number of bits exceeding the synthesis capability of the MDP can be allocated to the GPU for synthesis. That is, if the number of second layers exceeds the MDP synthesis capability, the second layer with the smallest number of bits exceeding the MDP synthesis capability is allocated to the GPU as the first layer for GPU synthesis. Or, the MDP is synthesized in multiple times, that is, a part of the MDP is synthesized and stored, then another part is synthesized, and all the intermediate synthesis results are synthesized.
  • Step S240 Synthesize the first layer through a graphics processing unit (GPU, Graphics Processing Unit) to obtain an intermediate layer.
  • a graphics processing unit GPU, Graphics Processing Unit
  • Step S250 Synthesize the second layer and the intermediate layer by a Multimedia Display Processor (MDP) to obtain an interface to be displayed for display.
  • MDP Multimedia Display Processor
  • step S240 and step S250 reference may be made to the foregoing embodiment, which will not be repeated here.
  • the number of layers synthesized by MDP can be less than or equal to the maximum number of layers that MDP can synthesize at a time, that is The number of layers synthesized by MDP can be less than or equal to the number of FIFOs of MDP.
  • the second layer synthesized by MDP needs to be less than or equal to the maximum number of layers that can be synthesized by the MDP at one time minus one, and the number of second layers synthesized by the MDP can be less than or equal to the number of FIFOs of the MDP minus one.
  • the intermediate layer obtained by GPU synthesis is also used as a layer, occupying the synthesis capability of MDP, that is, when the intermediate layer synthesized by GPU is synthesized from MDP to the interface to be displayed, it also occupies the FIFO pipeline of MDP.
  • the total number of layers including the intermediate layer does not exceed the maximum number of layers that can be synthesized by the MDP at one time, that is, it does not exceed the number of FIFOs of the MDP.
  • the size of the preset number of bits is dynamically adjusted, so that the preset number of bits fits the actual usage of the electronic device, and the first layer will not be caused by the preset number of bits being too large
  • the number of is too large, and the number of second layers will not be too small because the preset number of bits is too small, so that the number of layers synthesized by GPU and MDP is reasonable when the GPU and MDP are combined. Can be synthesized faster.
  • the layer composition method is applied in a resolution adjustment scene. Specifically, when resolution adjustment is required, the resolution adjustment is performed on the layer of the interface to be displayed. Specifically, please refer to Figure 6, the method includes:
  • Step S310 Obtain the resolution of each layer.
  • Step S320 Determine whether the resolution of each layer matches the screen resolution.
  • Step S330 For a layer whose resolution does not match the screen resolution, adjust the resolution of the layer to match the screen resolution.
  • the display screen of the electronic device has a corresponding screen resolution
  • the image used for display on the display screen also has a corresponding image resolution.
  • the resolution of the screen with 1080 pixels is 1960*1080.
  • vertical pixels*horizontal pixels represent the image resolution.
  • the resolution of an image with 1960 pixels in the vertical direction and 1080 pixels in the horizontal direction is 1960*1080.
  • the image display will be abnormal, which may be manifested in that the actual display physical size of the image is different from the physical size when the image is designed.
  • the physical size of the image is defined by device independent pixels (dp, device independent pixels), and the screen resolution and image resolution are adapted to keep the dp of the image unchanged. For example, if the size of a startup icon is 48x48dp, then Regardless of the screen resolution of the screen, the startup icon should be 48x48dp, and the icon can be adapted to the screen resolution.
  • the resolution of each layer Before the layer is rendered, the resolution of each layer can be obtained.
  • the way of obtaining the resolution of the layer may be to obtain the relevant description information of the pictures in each layer, and obtain the resolution of each picture from the description information, so as to obtain the resolution of each layer.
  • each picture is an image.
  • the method of obtaining the screen resolution is not limited in the embodiment of the present application.
  • it may be obtained through a window manager, obtained through resource data (Resources), obtained through a function of obtaining device performance parameters, and so on.
  • the judgment process can be, for example, obtaining the dp value of the picture, determining the screen density according to the screen resolution, and determining whether the corresponding resolution of the dp value of the picture at the screen density is consistent with the current actual resolution of the picture, if Inconsistent, the resolution of the picture does not match the screen resolution.
  • the judgment can be made according to the sp (scale-independent pixel) of the text.
  • the resolution of a layer does not match the screen resolution
  • adjust the resolution of the layer Specifically, if a picture in a layer does not match the screen resolution, the resolution of the picture is adjusted to match the screen resolution. For example, in the desktop display interface 100 shown in FIG. 2, if the icons in the icon layer 104 do not fit the current resolution of the screen, the resolution of each icon in the icon layer is adjusted to fit the screen resolution.
  • the specific adjustment process is not limited in the embodiment of the present application. In combination with the foregoing description, for example, the actual resolution of the picture may be adjusted to the resolution that the dp value of the picture should achieve under the current screen density.
  • the layers may also include acquiring each layer of the interface to be displayed.
  • the acquisition may include rendering of the layer.
  • each picture in the layer is rendered with the adjusted resolution. If the resolution of the layer matches the screen resolution, rendering will be performed at the current resolution.
  • the layer corresponding to the navigation bar defines a layer whose display content does not change on the screen as a fixed layer.
  • the electronic device can store it after one rendering.
  • the fixed layer can not be re-rendered, and the rendered fixed layer can be directly read for synthesis to reduce the image Processing time. That is, if the multiple layers of the interface to be displayed include a fixed layer, the fixed layer included in the multiple layers of the interface to be displayed is used as the target layer.
  • the way to obtain each layer of the interface to be displayed may be to read the target layer in the stored fixed layer, and to render the images other than the target layer in the multiple layers. Layer, obtain multiple layers of the interface to be displayed for synthesis.
  • some electronic devices have a resolution adjustment function, that is, the user can adjust the display screen of the electronic device from one screen resolution to another screen resolution, or the electronic device can change the display screen from one screen to another according to the current remaining power. Adjust one screen resolution to another screen resolution. After the screen resolution of the display screen is adjusted, the fixed layer should also be adjusted to fit the new screen resolution. Therefore, optionally, in the embodiment of the present application, when it is detected that the screen resolution has changed, a fixed layer for display is acquired, and the resolution of the fixed layer is adjusted to match the screen resolution And save.
  • detecting that the screen resolution has changed can be a situation where the screen resolution adjustment operation is monitored and the resolution adjustment is performed in response to the adjustment operation; or, the obtained screen resolution is inconsistent with the previously obtained screen resolution , Or through other means.
  • adjusting the resolution of the fixed layer to the screen resolution and saving it may be that the fixed layer is rendered at a resolution that adapts to the screen resolution, and the rendered layer is saved.
  • each layer includes a fixed layer, because the resolution of the fixed layer has been changed when the screen resolution is adjusted, the rendering is saved .
  • the display effect will change from before the adjustment. .
  • the display of the same picture or pictures of the same resolution will be reduced; if the screen resolution is adjusted lower, the display of the same picture or pictures of the same resolution will increase, and the screen will be adjusted on the same screen
  • the display of the same picture presents different display effects, which is not conducive to the development of the user's viewing habit, and brings discomfort to the user's viewing and affects the user's experience.
  • the pixel size of the picture should be increased by a multiple corresponding to the increase in screen density; for the same display, when the screen resolution is reduced, the pixel density of the screen ( dpi, dots per inch). If you want the picture to be adapted to the screen resolution, keep the dp of the picture unchanged.
  • the pixel size of the picture is the resolution The rate is reduced by a factor corresponding to the decrease in screen density.
  • the resolution of the picture itself does not adapt to the resolution before or after the adjustment, the resolution of the picture is directly adjusted according to the dp of the picture and the current screen density.
  • Step S340 Obtain the number of bits of each of the multiple layers of the interface to be displayed.
  • Step S350 Use a layer with a number of bits less than the preset number of bits as the first layer, and a layer with a number of bits greater than or equal to the preset number of bits as the second layer.
  • the number of bits of the layer may be determined before layer synthesis, and may be performed before layer rendering, during layer rendering, or after layer rendering is completed, which is not limited in the embodiment of the present application. Among them, if the determination of the number of bits and the classification of the layer are completed before the layer rendering is completed, the processing speed can be improved.
  • the classification of the layer is to determine whether the layer belongs to the first layer or the second layer according to the number of bits of the layer.
  • the number of bits of the layer can be determined before or after the resolution is adjusted. Among them, if the resolution of the layer needs to be adjusted, if it is determined that the number of bits of the layer is before the resolution adjustment, the number of bits of the layer can be determined according to the new resolution after the resolution to which the layer needs to be adjusted is determined.
  • the specific method for determining the number of layer bits is not limited in the embodiment of the present application. For details, reference may be made to the foregoing description, which will not be repeated here.
  • Step S360 Synthesize the first layer through a graphics processing unit (GPU, Graphics Processing Unit) to obtain an intermediate layer.
  • a graphics processing unit GPU, Graphics Processing Unit
  • Step S370 Synthesize the second layer and the intermediate layer by a Multimedia Display Processor (MDP) to obtain an interface to be displayed for display.
  • MDP Multimedia Display Processor
  • the adjustment of the screen to be displayed on the screen is realized through layer-based adjustment.
  • adjust each layer in the display interface to adapt each layer to the screen resolution, improve the processing effect of resolution adjustment, and reduce the time required for resolution adjustment and rendering as much as possible, and then adjust the The layers are combined to further reduce the power consumption of electronic devices.
  • the embodiment of the present application also provides a layer composition device 400.
  • the device 400 includes: a bit number obtaining module 410, configured to obtain the bit number of each of the multiple layers of the to-be-displayed interface; a layer classification module 420, configured to calculate the bit number The layer with less than the preset number of bits is taken as the first layer, and the layer with the number of bits greater than or equal to the preset number of bits is taken as the second layer; the first composition module 430 is used for graphics processing (GPU, Graphics Processing).
  • GPU Graphics Processing
  • the second synthesis module 440 is configured to combine the second layer and the intermediate layer with a multimedia display processor (MDP, Multimedia Display Processor) Perform synthesis to obtain the to-be-displayed interface for display.
  • MDP Multimedia Display Processor
  • the device 400 may further include a resolution adjustment module.
  • the resolution adjustment module may include a resolution acquisition unit for acquiring the resolution of each layer; an adaptation judging unit for judging whether the resolution of each layer is adapted to the screen resolution; an adaptation unit, For a layer whose resolution does not match the screen resolution, adjust the resolution of the layer to match the screen resolution.
  • the resolution adjustment module may further include: a layer unit, which is used to obtain a fixed layer for display when it is monitored that the screen resolution changes, and the fixed layer means that the display content on the screen does not change Layers.
  • the adapting unit is also used to adjust the resolution of the fixed layer to adapt to the screen resolution and save it.
  • the adaptation judging unit is configured to, if each layer included in the interface to be displayed includes the fixed layer, judge whether other layers than the fixed layer are consistent with the screen resolution.
  • the device 400 may further include a bit number adjustment module for dynamically adjusting the size of the preset bit number.
  • the bit number adjustment module can be used to determine the average value of the number of bits of all layers used for the interface to be displayed within the preset time range before the current time every preset time when the screen is in the display state; The average value is used as the preset number of bits.
  • the bit number adjustment module can be used to record the number of the first layer and the second layer in each interface to be displayed; to determine whether the continuous preset number of interfaces to be displayed meets the number ratio of the first layer The number of the second layer is large by the preset difference; if the continuous preset number of to-be-displayed interfaces meets the requirement that the number of the first layer is greater than the number of the second layer by the preset difference, the preset number of bits is The value decreases; if the continuous preset number of to-be-displayed interfaces are not all satisfying that the number of the first layer is larger than the second layer by the preset difference, it is determined whether the continuous preset number of to-be-displayed interfaces all satisfy the second image The number of layers is greater than the number of the first layer by a preset difference; if the continuous preset number of to-be-displayed interfaces all satisfy the preset difference that the number of the second layer is greater than the number of the first layer, the preset Set the value of the number of bits
  • the device may also include a cache module for saving the intermediate layer in the frame buffer.
  • the second synthesis module 440 may be used for the MDP to synthesize the second layer into the layer to be synthesized; the MDP reads the intermediate layer of the frame buffer, and combines the intermediate layer with the image to be synthesized The layers are combined into the interface to be displayed.
  • the coupling between the modules may be electrical, mechanical or other forms of coupling.
  • each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module.
  • the above-mentioned integrated modules can be implemented in the form of hardware or software functional modules.
  • Various implementation modes in the embodiments of the present application may be implemented by corresponding modules, and the corresponding descriptions are not repeated in the embodiments of the present application.
  • FIG. 8 shows a structural block diagram of an electronic device 700 provided by an embodiment of the present application.
  • the electronic device 700 may be an electronic device such as a mobile phone, a tablet computer, or an e-book.
  • the electronic device 700 includes a memory 710, a processor 720, a GPU 730, and an MDP 740.
  • the memory 710, GPU 730, and MDP 740 are coupled to the processor, and the memory 710 stores instructions.
  • the processor 720 executes one or more of the foregoing embodiments
  • GPU 730 and MDP 740 correspond to the layer composition process in the execution method.
  • the processor 710 may include one or more processing cores.
  • the processor 710 uses various interfaces and lines to connect various parts of the entire electronic device 700, and executes by running or executing instructions, programs, code sets, or instruction sets stored in the memory 720, and calling data stored in the memory 720.
  • the processor 710 may use at least one of digital signal processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA).
  • DSP Digital Signal Processing
  • FPGA Field-Programmable Gate Array
  • PLA Programmable Logic Array
  • the processor 710 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a modem, and the like.
  • the CPU mainly processes the operating system, user interface, and application programs; the GPU is used for rendering and drawing of display content; the modem is used for processing wireless communication. It is understandable that the above-mentioned modem may not be integrated into the processor 710, but may be implemented by a communication chip alone.
  • the memory 720 may include random access memory (RAM) or read-only memory (Read-Only Memory).
  • the memory 720 may be used to store instructions, programs, codes, code sets or instruction sets, such as instructions or code sets used to implement the layer synthesis method provided in the embodiments of the present application.
  • the memory 720 may include a storage program area and a storage data area, where the storage program area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing each of the foregoing method embodiments, and the like.
  • the storage data area can also be data created by the electronic device in use (such as audio and video data, chat record data), etc.
  • FIG. 9 shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application.
  • the computer-readable storage medium 800 stores program code, and the program code can be invoked by a processor to execute the method described in the foregoing method embodiment.
  • the computer-readable storage medium 800 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM.
  • the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium.
  • the computer-readable storage medium 800 has a storage space for the program code 810 for executing any method steps in the above-mentioned methods. These program codes can be read out from or written into one or more computer program products.
  • the program code 810 may be compressed in a suitable form, for example.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Computer Graphics (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

本申请公开了一种图层合成方法、装置、电子设备及存储介质。所述方法包括：获取待显示界面的多个图层中的各个图层的比特数；以比特数小于预设比特数的图层作为第一图层，以比特数大于或等于预设比特数的图层作为第二图层；通过图形处理器对所述第一图层进行合成，获得中间图层；通过多媒体显示处理器将所述第二图层以及所述中间图层进行合成，获得待显示界面用于显示，从而降低电子设备的图层合成功耗。

Description

图层合成方法、装置、电子设备及存储介质
本申请要求于2019年7月17日提交的申请号为201910647561.3的中国专利申请的优先权，在此通过引用将其全部内容并入本文。
技术领域
本申请涉及图像处理技术领域，更具体地，涉及一种图层合成方法、装置、电子设备及存储介质。
背景技术
随着电子技术的发展，具有图像显示功能的电子设备(如手机或平板电脑等)越来越多，电子设备的显示界面通常有多个显示图层，通过对多个显示图层进行合成显示在电子设备上。但是，电子设备在合成图层时，功耗较高。
发明内容
鉴于上述问题，本申请提出了一种图层合成方法、装置、电子设备及存储介质，以改善上述问题。
第一方面，本申请实施例提供了一种图层合成方法，获取待显示界面的多个图层中的各个图层的比特数；以比特数小于预设比特数的图层作为第一图层，以比特数大于或等于预设比特数的图层作为第二图层；通过图形处理器(GPU，Graphics Processing Unit)对所述第一图层进行合成，获得中间图层；通过多媒体显示处理器(MDP，Multimedia Display Processor)将所述第二图层以及所述中间图层进行合成，获得待显示界面用于显示。
第二方面，本申请实施例提供了一种图层合成装置，比特数获取模块，用于获取待显示界面的多个图层中的各个图层的比特数；图层分类模块，用于以比特数小于预设比特数的图层作为第一图层，以比特数大于或等于预设比特数的图层作为第二图层；第一合成模块，用于通过图形处理器(GPU，Graphics Processing Unit)对所述第一图层进行合成，获得中间图层；第二合成模块，用于通过多媒体显示处理器(MDP，Multimedia Display Processor)将所述第二图层以及所述中间图层进行合成，获得待显示界面 用于显示。
第三方面，本申请实施例提供了一种电子设备，包括存储器、处理器、图形处理器(GPU，Graphics Processing Unit)以及多媒体显示处理器(MDP，Multimedia Display Processor)，所述存储器、GPU以及MDP耦接到所述处理器，所述存储器存储指令，当所述指令由所述处理器执行时，所述处理器执行上述的方法，所述GPU和所述MDP执行上述方法中的图层合成。。
第四方面，本申请实施例提供了一种计算机可读取存储介质，所述计算机可读取存储介质中存储有程序代码，所述程序代码可被处理器调用执行上述的方法。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示出了本申请实施例提供的图像处理的逻辑框架图。
图2示出了本申请实施例提供的一种显示界面示意图。
图3示出了图2所示的显示界面对应的图层示意图。
图4示出了本申请实施例提供的图层合成方法的一种流程图。
图5示出了本申请实施例提供的图层合成方法的另一种流程图。
图6示出了本申请实施例提供的图层合成方法的又一流程图。
图7示出了本申请实施例提供的图层合成装置的功能模块图。
图8示出了本申请实施例提供的电子设备的结构框图。
图9是本申请实施例的用于保存或者携带实现根据本申请实施例的图层合成方法的程序代码的存储介质。
具体实施方式
为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。
应用程序(application)可以通过窗口管理器(Windows manager)创建窗 口(window)，窗口管理器为每一个窗口创建绘图表面(Surface)用于在上面绘制各种需要显示的元素，每个surface对应一个图层，即可以在每个surface绘制一个图层。
在显示界面(如Activity)中，可以包括多个图层，如导航栏、状态栏、程序界面等。每个图层在合成之前进行渲染。即，每个图层在相应的surface进行绘制，即在每个surface进行对应的图层的绘制，具体可以通过硬件加速渲染器(HWUI)和/或Skia图形库等在surface提供的画布(cavas)进行图层绘制。
系统(System)再使用图层传递模块(Surface Flinger)服务将各个surface进行合成，即将各个图层进行合成。其中，SurfaceFlinger服务运行在系统进程中，用来统一管理系统的帧缓冲区(FrameBuffer)，SurfaceFlinger获取所有的图层，可以使用图像处理器(GPU,Graphics Processing Unit)对图层进行合成，将合成的结果保存到帧缓存区。在本申请实施例中，GPU可以对所有或者部分图层进行合成。其中，当GPU对一个待显示的显示界面中的部分图层进行合成时，硬件图层混合器(HWC,Hardware composer)可以将SurfaceFlinger通过GPU合成的结果与其他图层一起合成，具体的，如图1所示，HWC可以调用多媒体显示处理器(MDP，Multimedia Display Processor)，对帧缓冲区中GPU合成后获得的图层与其他尚未合成的图层进行图层的合成，最终形成BufferQueue中的一个Buffer，再在显示驱动的作用下，将Buffer中合成的图像用于显示。
也就是说，电子设备的显示屏可以进行显示界面的显示，在显示界面中可能包括多个图层，即显示界面由多个图层合成。其中，每一个图层都是由许多像素组成的，各个图层通过叠加的方式组成整个图像，形成电子设备的显示界面。图层可以将页面上的元素精确定位，图层中可以加入文本、图片、表格、插件，也可以在里面再嵌套图层。对于图层以及图层合成的图像，可以通俗地理解为，图层就像是含有文字或图形等元素的胶片，一张张图层按指定的顺序叠放在一起，组合起来形成的最终图像效果，就是由图层组合形成的图像，例如电子设备的显示界面。当然，可以理解的是，图层并不是真的胶片，上述描述只是为了便于理解的比喻。例如图2所示的电子设备的桌面显示界面100，包括的多个图层，分别为状态栏101、导航栏102、墙纸103以及图标层104。
为了使显示界面在显示屏进行显示，需要将显示界面的各个图层进行合成。例如将图3所示的状态栏101、导航栏102、墙纸103以及图标层104合成为图2所示的显示界面100。发明人发现，由于GPU是一个通用的图形处 理器，在图形处理上比MDP功能强，除了能够做2维图像处理，还能做3维图像处理、特效等，一次性可以叠加多个图层，仅通过常用的图层合成方式GPU进行图层合成时，可以合成的图层数量基本不受限制，但是功耗相对较高。
而MDP是一个专用的显示图像处理单元，能够做常规的2维图像处理，主要用于做多个图层的合成叠加，且功耗低。但是MDP一次性能合成的图层数量有限。具体的，MDP一次性能合成的图层数量与MDP中FIFO管道数量对应，MDP一次性能够叠加的图层越多，在其内部需要更多的FIFO管道，这都是比较昂贵的电路，因此，MDP可以合成的图层数量越多，成本越高。在某些厂商的高端平台中，MDP也最多只有8个FIFO管道，一次性最多能够合成叠加8个图层，而在中低端平台，一个MDP可能只有4个FIFO管道，一次性最多只能够叠加4个图层。因此，若仅通过MDP进行图层合成，由于MDP一次性可以合成图层的数量有限，对于某些图层较多的待显示的显示界面，MDP无法处理，或者需要分多次合成，影响合成速度。因此，发明人提出了本申请实施例的图层合成方法、装置、电子设备及存储介质，通过GPU以及MDP相互协作进行图层合成，克服MDP对合成图层数量的限制且尽可能地降低功耗。其中，对比特数比较大的图层，通过MDP进行合成，从而相对于使用GPU进行合成功耗更低；由于MDP一次可合成的图层数量受限，则其他的比特数比较小的图层则通过GPU合成，以实现图层合成过程中产生尽可能低的功耗。下面对本申请实施例的图层合成方法进行详细介绍。
图4示出了本申请实施例提供的图层合成方法的流程图，该图层合成方法应用于电子设备。具体的，该图层合成方法包括：
步骤S110：获取待显示界面的多个图层中的各个图层的比特数。
电子设备在显示过程中，在显示屏进行显示界面的显示。显示屏显示的显示界面作为待显示界面，包括多个图层。其中，各个图层的比特数不同，比特数越大，图层合成过程中产生的功耗越大，为了尽可能降低电子设备进行图层合成的功耗，可以通过比特数将图层分为两类，以使比特数较大的一类图层由MDP合成，使图层合成过程中产生的功耗尽可能低；另一类比特数较小的图层通过GPU进行合成，克服MDP存在的合成数量限制，并提高合成速度。
因此，可以获取待显示界面中的各个图层的比特数。其中，图层的比特数表示了图层的数据大小，或者说图层所需存储容量的多少，如16比特，32比特等。当然，本申请实施例也可以用字节数或其他的表示方法表示图层的数据大小，数据越大对应的比特越大，本申请实施例以比特数为例进行说明。
在本申请实施例中，获取图层比特数大小的方式在本申请实施例中并不限制。例如，在一种实施方式中，可以读取图层的像素数量，并根据每个像素所占比特数，通过像素数量乘以每个像素所占比特数计算图层的比特数。
又如，在另一种实施方式中，图层中包括一种或多种元素，可以获取每个图层中各种元素的比特数之和，作为该图层的比特数。其中，图层中的元素为该图层用于显示的、需要占据存储空间的数据，如文本、图片等。例如，某个图层用于显示某图片，则该图层中的元素为该图片，该图层的比特数为该图片的比特数，例如图2所示，图标层104的比特数为各个图标的比特数之和。又如，某图层用于显示文字，则该图层中的元素为该图层用于显示的文字，图层的比特数可以是该图层中文字的比特数之和。
在该实施方式中，图层中各种元素的比特数计算方式并不限定。如可以是，对于任意一个元素，根据元素的最小组成单位的数量以及每个最小组成单位所占比特数，计算各元素的大小；或者是，某元素的相关数据中，存储有该元素的比特数信息，则可以直接读取该比特数信息确定该元素的比特数。例如，若图层中的元素包括图片，计算图片的比特数的方式可以是，读取图片的像素数量，并根据每个像素所占比特数，通过像素数量乘以每个像素所占比特数计算图片的比特数；或者可以是，在图片的数据信息中存储该图片比特数的相关信息，直接读取该比特数的相关信息，获得图片的比特数。
步骤S120：以比特数小于预设比特数的图层作为第一图层，以比特数大于或等于预设比特数的图层作为第二图层。
将各个图层的比特数与预设比特数进行比对，将其中比特数小于预设比特数的图层作为第一图层，将其中大于或等于预设比特数的图层作为第二图层。
其中，该预设比特数的具体值在本申请实施例中并不限定，可以根据实际需要设置。例如，可以根据电子设备通常合成的图层的数量以及比特数，设置预设比特数的值为，使比预设比特数大的图层数量小于或等于MDP一次性能合成的最大图层数量。
步骤S130：通过图形处理器(GPU，Graphics Processing Unit)对所述第一图层进行合成，获得中间图层。
步骤S140：通过多媒体显示处理器(MDP，Multimedia Display Processor)将所述第二图层以及所述中间图层进行合成，获得待显示界面用于显示。
通过GPU以及MDP对第一图层以及第二图层进行合成。其中，由于MDP进行合成所限制的为图层的数量，而非图层的比特数，而在同一种合成方式进行合成时，比特数大的图层相对于比特数小的图层功耗更高。因此，将比特数 较小的第一图层由GPU进行合成，使GPU既为MDP分担了部分图层，又能以较小的比特数进行合成以降低合成功耗。将其中比特数较大的第二图层由MDP合成。
其中，所有的图层合成以获得最终的待显示界面可以由MDP实现，以使电子设备的图层合成过程中功耗尽可能地低。即GPU将第一图层合成，定义第一图层的合成结果为中间图层。MDP将中间图层以及第二图层进行合成。获得待显示界面用于显示。
例如，图2所示的桌面显示界面100中的多个图层分别为状态栏101、导航栏102、墙纸103以及图标层104，若其中状态栏101以及导航栏102的比特数小于预设比特数，墙纸103以及图标层104的比特数大于或等于预设比特数，则将状态栏101以及导航栏102作为第一图层；将墙纸103以及图标层104作为第二图层。GPU将状态栏101以及导航栏102合成，MDP将GPU的合成结果、墙纸103以及图标层104合成，得到最终合成的待显示界面。
可选的，第一图层合成获得中间图层后，可以存储于帧缓冲区中，若显示于屏幕的待显示界面更新过程中，中间图层包括的各图层都没有更新，则帧缓冲区中的该中间图层保持不变，GPU不需要再次合成该中间图层，降低GPU的合成频率。在待显示界面刷新时，MDP可以读取帧缓冲区中的中间图层，用于与第二图层合称为最终的用于显示的图像。即在待显示界面更新时，多媒体显示处理器可以读取帧缓冲区的中间图层，将该中间图层与新的待显示界面的其他图层合成。
可选的，在GPU对第一图层进行合成时，MDP可以同时对第二图层进行合成，使第一图层的合成与第二图层的合成过程同步进行，既能降低合成功耗，又能提高合成速度。本申请实施例中，MDP将第二图层合成后获得的图层可以定义为待合成图层。MDP将待合成图层和中间图层进行合成，获得最终的待显示界面。其中，可以将所述中间图层保存到帧缓冲区，MDP将所述第二图层合成为待合成图层后，读取帧缓冲区的中间图层，将所述中间图层与所述待合成图层合成为待显示界面。或者若MDP在将第二图层合成的过程中，中间图层已经合成完成，MDP可以读取帧缓冲区的中间图层，将中间图层与第二图层进行共同合成。
在本申请实施例中，通过GPU合成比特数小于预设比特数的图层作为中间图层，通过MDP合成中间图层以及比特数大于或等于预设比特数的图层，获得最终的图层合成结果用于显示，从而，通过MDP和GPU的结合，使电子设备在图层合成过程中的功耗尽可能地低。
在本申请另一实施例提供的方法中，还可以对用于确定第一图层以及第二图层的预设比特数的大小进行动态调整，以使第一图层数量以及第二图层的数量处于合理的范围内，既使MDP以及GPU的共同合成降低图层合成功耗，又能提高合成速度。具体的，请参见图5，该方法包括：
步骤S210：获取待显示界面的多个图层中的各个图层的比特数。
该步骤可以参见前述实施例的对应描述，在此不再进行赘述。
步骤S220：动态调整预设比特数的大小。
步骤S230：以比特数小于预设比特数的图层作为第一图层，以比特数大于或等于预设比特数的图层作为第二图层。
预设比特数为划分第一图层以及第二图层的标准，但是，不同电子设备，其对应的用户使用倾向不同，则在电子设备显示的界面的图层数量以及图层比特数的结构不同，另外，在不同时刻观看不同的内容，显示界面中图层数量可能差别较大。对于某一待显示界面而言，预设比特数设置偏大或者偏小，都可能导致第一图层的数量过多或者第二图层的数量过多，超出MDP的处理能力或者MDP合成的图层过少，影响图层合成效果。为了使第一图层的数量以及第二图层的数量处于合理的状态，本申请实施例可以动态调整预设比特数的大小。
具体的，用户在一段时间内浏览的内容可能比较接近，则各个待显示界面之间的图层总比特数比较接近，例如，用户在观看视频过程中，显示界面中包括的图层可能都是视频内容所在的视频层以及视频内容以外的背景层，各个显示界面之间比特数比较接近以及包括的图层数量比较接近。因此，可以根据电子设备的历史显示记录动态调整预设比特数。
作为一种实施方式，可以每隔预设时间调整一次预设比特数，并且，根据间隔的预设时间内的显示界面中的比特数进行调整。具体的，可以是，在屏幕处于显示状态下每隔预设时间，确定当前时间之前预设时间范围内所有用于待显示界面的图层的比特数的平均值；以所述平均值作为所述预设比特数。其中，可以记录每个显示界面中各个图层的比特数。确定平均值时，将最近的预设时间范围内总的比特数除以总的图层数量计算平均值。
可选的，在该实施方式中，在以新的预设比特数作为分类标准，将大于预设比特数的图层作为第二图层时，若第二图层的数量大于或等于MDP一次性可以处理的图层数量的最大值，可以将预设比特数再阶梯性调大，直到根据预设比特数进行图层分类时，第二图层的数量小于MDP一次性可以处理的图层数量。其中，调整阶梯在本申请实施例中并不限定，可以是预先设置的一个较 小的值，也可以是在预设时间范围内比特数最小的图层的比特数，尽量避免预设比特数一次调整过多导致第二图层数量过小，远远不足MDP的处理能力。其中，MDP一次性可以处理的图层数量的最大值，表示MDP的合成能力。
其中，由于在显示屏处于显示状态的预设时间范围内，显示屏显示的内容比较接近，则作为分类标准的预设比特数比较接近，则阶梯性调整预设比特数可以只在根据平均值设置预设比特数后进行一次，而不在预设时间范围内多次调整，降低数据处理量。
作为另一种实施方式，可以在根据预设比特数进行图层分类后，第一图层和第二图层的数量差别过大而调整预设比特数的值。具体的，可以记录每个待显示界面中第一图层以及第二图层的数量；判断是否连续预设数量的待显示界面中，第一图层的数量比第二图层的数量大预设差值。
若连续预设数量的待显示界面中第一图层的数量比第二图层的数量大预设差值，表明第一图层数量过多，而第二图层数量过少，则将所述预设比特数的值减小，以增加后续分类时第二图层的数量，减少第一图层的数量。
若连续预设数量的待显示界面不是都满足第一图层的数量比第二图层的数量大预设差值，再判断是否连续预设数量的待显示界面都满足第二图层的数量比第一图层的数量大预设差值。若连续预设数量的待显示界面都满足第二图层的数量比第一图层的数量大预设差值，表明第二图层数量过多，而第一图层数量过少，将所述预设比特数的值增大，以增加后续分类时第一图层的数量，减少第二图层的数量。
若连续预设数量的待显示界面不是都满足第二图层的数量比第一图层的数量大预设差值，也不是都满足第一图层的数量比第二图层的数量大预设差值，表示当前预设比特数设置较为合理，可以保持所述预设比特数的值不变。
在本申请实施例中，动态调整预设比特数的大小的执行时序可以不受其他步骤的执行时序限制，在根据预设比特数将待显示界面的图层分为第一图层和第二图层时，以当前预设比特数作为分类依据，即在进行分类时，若预设比特数被调整，则以调整后的作为分类依据；若预设比特数未被调整，则以该未被调整的预设比特数作为分类依据。
可选的，在本申请实施例中，由于待显示界面中图层数量一般不会过多，从而以预设比特数作为分类标准时，第二图层的数量通常不会超出MDP的合成能力限制。假设在某次第二图层数量过多，超出MDP的合成能力，可以将该次超出MDP合成能力的比特数最小的图层分给GPU进行合成。即若第二图层数量超出MDP的合成能力，将超出MDP合成能力的比特数最小的第二 图层作为第一图层分配给GPU，用于GPU进行合成。或者是，MDP分多次合成，即MDP合成一部分存储，再合成另一部分，再将所有中间的合成结果进行合成。
步骤S240：通过图形处理器(GPU，Graphics Processing Unit)对所述第一图层进行合成，获得中间图层。
步骤S250：通过多媒体显示处理器(MDP，Multimedia Display Processor)将所述第二图层以及所述中间图层进行合成，获得待显示界面用于显示。
步骤S240和步骤S250的描述可以参见前述实施例，在此不再赘述。
可以理解的，在本申请实施例中，考虑MDP的合成能力限制时，若仅仅通过MDP进行合成，由MDP合成的图层数量可以是小于或等于MDP一次性可以合成的最多图层数量，即由MDP合成的图层数量可以是小于或等于MDP的FIFO数量。若由MDP和GPU同时合成，即MDP不仅需要合成分配给其的第二图层，还需要将GPU合成的中间图层与第二图层共同合成为待显示界面，则由MDP合成的第二图层的数量需要小于或等于MDP一次性可以合成的最多图层数量减一，由MDP合成的第二图层的数量可以是小于或等于MDP的FIFO数量减一。总之，GPU合成获得的中间图层也作为一种图层，占用MDP的合成能力，即GPU合成的中间图层由MDP合成为待显示界面时也占用MDP的FIFO管道，MDP一次可以合成的包括中间图层在内的图层总数量不超过MDP一次性可以合成的最多图层数量，即不超过MDP的FIFO数量。
在本申请实施例提供的图层合成方法中，动态调整预设比特数的大小，使预设比特数契合于电子设备的实际使用情况，不会因为预设比特数过大导致第一图层的数量过大，也不会因为预设比特数过小导致第二图层数量过多，从而使GPU和MDP混合合成时各自进行合成的图层数量合理，在节省功耗的情况下，也可以合成速度更快。
本申请实施例还提供了一种实施例。在该实施例中，该图层合成的方法应用于分辨率调整的场景下。具体的，在需要进行分辨率调整的情况下，对待显示界面的图层进行分辨率调整。具体的，请参见图6，该方法包括：
步骤S310：获取各个图层的分辨率。
步骤S320：判断各个图层的分辨率与屏幕分辨率是否适配。
步骤S330：对于分辨率与屏幕分辨率不适配的图层，调整该图层的分辨率到和所述屏幕分辨率适配。
电子设备的显示屏具有相应的屏幕分辨率，用于在显示屏显示的图像也具有相应的图像分辨率。其中，屏幕分辨率是指在屏幕横纵向上的像素点数， 单位是px，1px＝1个像素点，例如，若以纵向像素*横向像素表示屏幕分辨率，纵向上有1960个像素点、横向上有1080个像素点的屏幕的分辨率为1960*1080。本申请实施例中，为了便于理解，对应屏幕分辨率的表示方式定义图像分辨率，即定义图像分辨率为图像的像素尺寸，即图像在横纵向上的像素点数，单位是px，1px＝1个像素点，如以纵向像素*横向像素表示图像分辨率，纵向上有1960个像素点、横向上有1080个像素点的图像的分辨率为1960*1080。
若电子设备的屏幕分辨率与图像的图像分辨率不适配，则图像显示则会异常，具体可能表现为图像的实际显示物理尺寸与设计该图像时的物理尺寸不同。其中，图像的物理尺寸通过设备独立像素(dp，device independent pixels)进行限定，屏幕分辨率与图像分辨率的适配为使图像的dp保持不变，例如，一个启动图标的尺寸为48x48dp，则不管在何种屏幕分辨率的屏幕中，该启动图标都应当为48x48dp，该图标才与屏幕分辨率适配。
在图层进行渲染之前，可以获取各个图层的分辨率。其中，获取图层分辨率的方式可以是，获取各个图层中图片的相关描述信息，从描述信息中获得各个图片的分辨率，从而获得各个图层的分辨率。其中，每个图片即为一个图像。
另外，还可以获取屏幕的分辨率。获取屏幕分辨率的方式在本申请实施例中并不限定，例如可以是，通过窗口管理器获取，通过资源数据(Resources)获取、通过获取设备性能参数的函数获取等。
再判断各个图层的分辨率与屏幕分辨率是否适配，具体可以为判断各个图层中各元素的分辨率与屏幕分辨率是否适配，本申请实施例以图片为例进行说明。图片的分辨率与屏幕分辨率是否适配，即判断在该屏幕分辨率下，图片在当前分辨率的情况下是否能以图片所要显示的dp进行显示。结合前述描述，该判断过程例如可以是，获取图片的dp数值，根据屏幕分辨率确定屏幕密度，确定图片的dp数值在该屏幕密度下对应的分辨率与图片当前实际的分辨率是否一致，若不一致，则图片的分辨率与屏幕分辨率不适配。可选的，若判定文字分辨率是否适配，则可以根据文字的sp(scale-independent pixel)进行判断。
若有图层的分辨率与屏幕分辨率不适配，则对该图层的分辨率进行调整。具体的，有图层中的图片与屏幕分辨率不适配，则调整该图片的分辨率至与屏幕的分辨率适配。例如图2所示的桌面显示界面100中，若图标层104中的图标不适配屏幕当前的分辨率，则图标层中各个图标的分辨率进行调整以适配屏幕分辨率。具体调整过程在本申请实施例中并不限定。结合前述描述，例如可以是，将图片的实际分辨率调整为图片的dp数值在当前屏幕密度下应当达到 的分辨率。
可选的，在本申请实施例中，在对图层进行合成之前，还可以包括获取待显示界面的各个图层。该获取可以包括对图层的渲染。
可选的，在渲染图层时，以调整后的分辨率对图层中各个图片进行渲染。若图层的分辨率与屏幕分辨率适配，则以当前分辨率进行渲染。
另外，存在某些图层在屏幕中显示内容不变，在不同的显示界面中都是一致的，例如导航栏对应的图层，定义屏幕中的显示内容不变的图层为固定图层。对于固定图层，电子设备在一次渲染后可以将其存储，在待显示界面中有固定图层时，可以不再重新渲染固定图层，直接读取渲染好的固定图层进行合成，减少图像处理时间。也就是说，若待显示界面的多个图层中包括固定图层，以待显示界面的多个图层中包括的固定图层作为目标图层。在需要对图层进行合成时，获取待显示界面的各个图层的方式可以是，读取存储的固定图层中的目标图层，以及渲染所述多个图层中目标图层以外的图层，获得待显示界面的多个图层用于进行合成。
可选的，某些电子设备具有分辨率调整功能，即用户可以将电子设备的显示屏由一种屏幕分辨率调整到另一种屏幕分辨率，或者电子设备根据当前剩余电量将显示屏的从一种屏幕分辨率调整到另一种屏幕分辨率。由于显示屏的屏幕分辨率调整后，固定图层也应当调整为与新的屏幕分辨率适配。因此，可选的，在本申请实施例中，当监测到屏幕分辨率发生改变，获取用于显示的固定图层，将所述固定图层的分辨率调整到与所述屏幕分辨率适配并保存。其中，检测到屏幕分辨率发生改变，可以是监测到屏幕分辨率调整操作，并响应该调整操作进行分辨率调整的情况；或者是，获取到屏幕的分辨率与前一次获取的屏幕分辨率不一致，或者通过其他方式监测到。另外，将固定图层的分辨率调整到屏幕分辨率适配并保存可以是，将固定图层以适配屏幕分辨率的分辨率进行渲染，将渲染完成的图层保存。
从而，在判断各个图层的分辨率与屏幕分辨率是否适配时，若各个图层中包括固定图层，由于固定图层的分辨率已经在监测到屏幕分辨率调整时已经更改并渲染保存，则可以不再判断固定图层的分辨率是否与屏幕适配，而是判断固定图层以外的其他图层与所述屏幕分辨率是否适配，即判断固定图层以外的其他图层与所述屏幕分辨率是否一致，从而减少处理时间以及处理流程。若固定图层以外的其他图层中存在不适配屏幕分辨率的图层，对不适配屏幕分辨率的图层调整至适配屏幕分辨率。
另外，对于可以进行分辨率调整的电子设备，若电子设备的屏幕分辨率被 调整，则同一图片，如电子设备中的某图标，或者对于同一分辨率的图片，显示效果相对于调整前发生变化。其中，若屏幕分辨率被调高，则同一图片或者同一分辨率的图片的显示缩小；若屏幕分辨率被调低，则同一图片或者同一分辨率的图片的显示增大，在同一屏幕调整屏幕分辨率后，对同一图片的显示呈现不同的显示效果，不利于养成用户的观看习惯，并且给用户的查看带来不适，影响用户的体验。具体可以理解为，如果屏幕的像素密度为160dpi(dots per inch)，1dp＝1px；如果屏幕的像素密度为320dpi，1dp＝2px；如果屏幕的像素密度为480dpi，1dp＝3px，以此类推，随着屏幕的像素密度成倍改变，1dp对应的像素数以对应的倍数改变。对于同一显示屏，当屏幕分辨率被调高，屏幕的像素密度(dpi，dots per inch)增大，若要图片适配于屏幕的分辨率，保持图片的dp不变，在图片的分辨率适配调整前的屏幕分辨率的情况下，图片的像素尺寸，即分辨率，应以对应屏幕密度增加的倍数增大；对于同一显示屏，当屏幕分辨率被调低，屏幕的像素密度(dpi，dots per inch)降低，若要图片适配于屏幕的分辨率，保持图片的dp不变，在图片的分辨率适配调整前的屏幕分辨率的情况下则图片的像素尺寸，即分辨率，以对应屏幕密度降低的倍数降低。当然，若图片的分辨率本身并不适配调整前或者调整后的分辨率，则直接根据图片的dp以及当前屏幕密度调整图片的分辨率。
步骤S340：获取待显示界面的多个图层中的各个图层的比特数。
步骤S350：以比特数小于预设比特数的图层作为第一图层，以比特数大于或等于预设比特数的图层作为第二图层。
对待显示界面的各个图层的比特数进行确定。
可选的，确定图层的比特数可以在图层合成前进行，可以在图层渲染前、图层渲染过程中或者图层渲染结束后，在本申请实施例中并不限定。其中，若在图层渲染结束前完成对比特数的确定以及对图层的分类，可以提高处理速度。该对图层的分类即根据图层的比特数确定图层属于第一图层还是第二图层。
可选的，确定图层的比特数可以在调整分辨率前或者调整分辨率后。其中，若图层的分辨率需要调整，若确定图层的比特数在分辨率调整前，可以在确定图层需要调整到的分辨率后，根据新的分辨率确定图层的比特数。具体确定图层比特数的方式在本申请实施例中并不限定，具体可以参照前述描述，在此不再赘述。
步骤S360：通过图形处理器(GPU，Graphics Processing Unit)对所述第一图层进行合成，获得中间图层。
步骤S370：通过多媒体显示处理器(MDP，Multimedia Display Processor)将所述第二图层以及所述中间图层进行合成，获得待显示界面用于显示。
对分辨率调整后的各个图层进行合成，获得适配屏幕的待显示界面进行显示。具体描述可以参见前述实施例，在此不再进行赘述。
在本申请实施例中，对待显示界面适配屏幕的调整，通过基于图层的调整实现。也就是说，对待显示界面中各个图层进行调整，使各个图层与屏幕分辨率适配，提高分辨率调整的处理效果，尽可能降低分辨率调整以及渲染所需要的时间，再对调整后的各图层进行合成，进一步降低电子设备的功耗。
本申请实施例还提供了一种图层合成装置400。具体的，请参见图7，该装置400包括：比特数获取模块410，用于获取待显示界面的多个图层中的各个图层的比特数；图层分类模块420，用于以比特数小于预设比特数的图层作为第一图层，以比特数大于或等于预设比特数的图层作为第二图层；第一合成模块430，用于通过图形处理器(GPU，Graphics Processing Unit)对所述第一图层进行合成，获得中间图层；第二合成模块440，用于通过多媒体显示处理器(MDP，Multimedia Display Processor)将所述第二图层以及所述中间图层进行合成，获得待显示界面用于显示。
可选的，该装置400还可以包括分辨率调整模块。具体的，分辨率调整模块可以包括分辨率获取单元，用于获取各个图层的分辨率；适配判断单元，用于判断各个图层的分辨率与屏幕分辨率是否适配；适配单元，对于分辨率与屏幕分辨率不适配的图层，调整该图层的分辨率到和所述屏幕分辨率适配。
可选的，分辨率调整模块还可以包括：图层单元，用于当监测到屏幕分辨率发生改变，获取用于显示的固定图层，所述固定图层为在屏幕中的显示内容不变的图层。适配单元还用于将所述固定图层的分辨率调整到与所述屏幕分辨率适配并保存。适配判断单元用于，若待显示界面包括的各个图层中包括所述固定图层，判断固定图层以外的其他图层与所述屏幕分辨率是否一致。
可选的，该装置400还可以包括比特数调整模块，用于动态调整所述预设比特数的大小。
可选的，该比特数调整模块可以用于在屏幕处于显示状态下每隔预设时间，确定当前时间之前预设时间范围内所有用于待显示界面的图层的比特数的平均值；以所述平均值作为所述预设比特数。
可选的，该比特数调整模块可以用于记录每个待显示界面中第一图层以及第二图层的数量；判断是否连续预设数量的待显示界面都满足第一图层的数量比第二图层的数量大预设差值；若连续预设数量的待显示界面都满足第一图层 的数量比第二图层的数量大预设差值，将所述预设比特数的值减小；若连续预设数量的待显示界面不是都满足第一图层的数量比第二图层的数量大预设差值，判断是否连续预设数量的待显示界面都满足第二图层的数量比第一图层的数量大预设差值；若连续预设数量的待显示界面都满足第二图层的数量比第一图层的数量大预设差值，将所述预设比特数的值增大；若连续预设数量的待显示界面不是都满足第二图层的数量比第一图层的数量大预设差值，保持所述预设比特数的值不变。
可选的，该装置还可以包括缓存模块，用于将所述中间图层保存到帧缓冲区。第二合成模块440可以用于所述MDP将所述第二图层合成为待合成图层；所述MDP读取帧缓冲区的中间图层，将所述中间图层与所述待合成图层合成为待显示界面。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述装置和模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，模块相互之间的耦合可以是电性，机械或其它形式的耦合。
另外，在本申请各个实施例中的各功能模块可以集成在一个处理模块中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。本申请实施例中各种实施方式都可以有对应的模块进行实现，本申请实施例中不再一一进行对应赘述。
请参考图8，其示出了本申请实施例提供的一种电子设备700的结构框图。该电子设备700可以是手机、平板电脑、电子书等电子设备。该电子设备700包括存储器710、处理器720、、GPU 730以及MDP 740。所述存储器710、GPU 730以及MDP 740耦接到所述处理器，所述存储器710存储指令，当所述指令由所述处理器720执行时，所述处理器执行上述一个或多个实施例所描述的方法，GPU 730以及MDP 740对应执行方法中的图层合成过程。
处理器710可以包括一个或者多个处理核。处理器710利用各种接口和线路连接整个电子设备700内的各个部分，通过运行或执行存储在存储器720内的指令、程序、代码集或指令集，以及调用存储在存储器720内的数据，执行电子设备700的各种功能和处理数据。可选地，处理器710可以采用数字信号处理(Digital Signal Processing，DSP)、现场可编程门 阵列(Field－Programmable Gate Array，FPGA)、可编程逻辑阵列(Programmable Logic Array，PLA)中的至少一种硬件形式来实现。处理器710可集成中央处理器(Central Processing Unit，CPU)、图像处理器(Graphics Processing Unit，GPU)和调制解调器等中的一种或几种的组合。其中，CPU主要处理操作系统、用户界面和应用程序等；GPU用于负责显示内容的渲染和绘制；调制解调器用于处理无线通信。可以理解的是，上述调制解调器也可以不集成到处理器710中，单独通过一块通信芯片进行实现。
存储器720可以包括随机存储器(Random Access Memory，RAM)，也可以包括只读存储器(Read-Only Memory)。存储器720可用于存储指令、程序、代码、代码集或指令集，如用于实现本申请实施例提供的图层合成方法的指令或代码集。存储器720可包括存储程序区和存储数据区，其中，存储程序区可存储用于实现操作系统的指令、用于实现至少一个功能的指令、用于实现上述各个方法实施例的指令等。存储数据区还可以电子设备在使用中所创建的数据(比如音视频数据、聊天记录数据)等。
请参考图9，其示出了本申请实施例提供的一种计算机可读存储介质的结构框图。该计算机可读存储介质800中存储有程序代码，所述程序代码可被处理器调用执行上述方法实施例中所描述的方法。
计算机可读存储介质800可以是诸如闪存、EEPROM(电可擦除可编程只读存储器)、EPROM、硬盘或者ROM之类的电子存储器。可选地，计算机可读存储介质800包括非易失性计算机可读介质(non-transitory computer-readable storage medium)。计算机可读存储介质800具有执行上述方法中的任何方法步骤的程序代码810的存储空间。这些程序代码可以从一个或者多个计算机程序产品中读出或者写入到这一个或者多个计算机程序产品中。程序代码810可以例如以适当形式进行压缩。
最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不驱使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。

Claims (20)

  1. 一种图层合成方法，其特征在于，所述方法包括：
    获取待显示界面的多个图层中的各个图层的比特数；
    以比特数小于预设比特数的图层作为第一图层，以比特数大于或等于预设比特数的图层作为第二图层；
    通过图形处理器对所述第一图层进行合成，获得中间图层；
    通过多媒体显示处理器将所述第二图层以及所述中间图层进行合成，获得待显示界面用于显示。
  2. 根据权利要求1所述的方法，其特征在于，所述获取待显示界面的多个图层中的各个图层的比特数之前，还包括：
    获取各个图层的分辨率；
    判断各个图层的分辨率与屏幕分辨率是否适配；
    对于分辨率与屏幕分辨率不适配的图层，调整该图层的分辨率到和所述屏幕分辨率适配。
  3. 根据权利要求2所述的方法，其特征在于，所述方法还包括：
    当监测到屏幕分辨率发生改变，获取用于显示的固定图层，所述固定图层为在屏幕中的显示内容不变的图层；
    将所述固定图层的分辨率调整到与所述屏幕分辨率适配并保存；
    所述判断各个图层的分辨率与屏幕分辨率是否适配包括：
    若各个图层中包括所述固定图层，判断固定图层以外的其他图层与所述屏幕分辨率是否一致。
  4. 根据权利要求1至3任一项所述的方法，其特征在于，所述方法还包括：
    动态调整所述预设比特数的大小。
  5. 根据权利要求4所述的方法，其特征在于，所述动态调整所述预设比特数的大小包括：
    在屏幕处于显示状态下每隔预设时间，确定当前时间之前预设时间范围内所有用于待显示界面的图层的比特数的平均值；
    以所述平均值作为所述预设比特数。
  6. 根据权利要求4所述的方法，其特征在于，所述动态调整所述预设比特数的大小包括：
    记录每个待显示界面中第一图层以及第二图层的数量；
    判断是否连续预设数量的待显示界面都满足第一图层的数量比第二图层的数量大预设差值；
    若连续预设数量的待显示界面都满足第一图层的数量比第二图层的数量大预设差值，将所述预设比特数的值减小。
  7. 根据权利要求6所述的方法，其特征在于，所述方法还包括：若连续预设数量的待显示界面不是都满足第一图层的数量比第二图层的数量大预设差值，判断是否连续预设数量的待显示界面都满足第二图层的数量比第一图层的数量大预设差值；
    若连续预设数量的待显示界面都满足第二图层的数量比第一图层的数量大预设差值，将所述预设比特数的值增大。
  8. 根据权利要求7所述的方法，其特征在于，所述方法还包括：
    若连续预设数量的待显示界面不是都满足第二图层的数量比第一图层的数量大预设差值，也不是都满足第一图层的数量比第二图层的数量大预设差值，保持所述预设比特数的值不变。
  9. 根据权利要求4至8任一项所述的方法，其特征在于，将大于预设比特数的图层作为第二图层时，若所述第二图层的数量大于多媒体显示处理器的合成能力，将所述预设比特数阶梯性调大，直到根据预设比特数进行图层分类时，第二图层的数量小于多媒体显示处理器的合成能力。
  10. 根据权利要求1至9任一项所述的方法，其特征在于，所述获取待显示界面的多个图层中的各个图层的比特数，包括：
    读取图层的像素数量；
    根据每个像素所占比特数，通过像素数量乘以每个像素所占比特数计算图层的比特数。
  11. 根据权利要求1至10任一项所述的方法，其特征在于，所述获取待显示界面的多个图层中的各个图层的比特数，包括：
    获取每个图层中各种元素的比特数之和，作为该图层的比特数，图层中的元素为该图层用于显示的、需要占据存储空间的数据。
  12. 根据权利要求1至11任一项所述的方法，其特征在于，所述方法还包括：
    将所述中间图层保存到帧缓冲区，
    所述通过多媒体显示处理器将所述第二图层以及所述中间图层进行合成，包括：
    所述多媒体显示处理器将所述第二图层合成为待合成图层；
    所述多媒体显示处理器读取帧缓冲区的中间图层，将所述中间图层与所述待合成图层合成为待显示界面。
  13. 根据权利要求1至12任一项所述的方法，其特征在于，所述方法还包括：
    将所述中间图层保存到帧缓冲区，
    在待显示界面更新时，若中间图层包括的各图层都没有更新，多媒体显示处理器读取帧缓冲区的中间图层；
    将所述中间图层与新的待显示界面的其他图层合成为待显示界面。
  14. 根据权利要求1至13任一项所述的方法，其特征在于，所述通过图形处理器对所述第一图层进行合成，获得中间图层之前，还包括：
    若第二图层数量超出多媒体显示处理器的合成能力，将超出多媒体显示处理器合成能力的比特数最小的第二图层作为第一图层，用于图形处理器进行合成。
  15. 根据权利要求1至14任一项所述的方法，其特征在于，所述通过图形处理器对所述第一图层进行合成，获得中间图层之前，还包括：获取待显示界面的各个图层。
  16. 根据权利要求15所述的方法，其特征在于，所述获取待显示界面的各个图层，包括：
    对待显示界面的各个图层进行渲染。
  17. 根据权利要求15所述的方法，其特征在于，将在屏幕中的显示内容不变的图层作为固定图层存储，所述获取待显示界面的各个图层，包括：
    读取存储的固定图层中的目标图层，所述目标图层为待显示界面的多个图层中包括的固定图层；
    渲染待显示界面的多个图层中目标图层以外的图层。
  18. 一种图层合成装置，其特征在于，所述装置包括：
    比特数获取模块，用于获取待显示界面的多个图层中的各个图层的比特数；
    图层分类模块，用于以比特数小于预设比特数的图层作为第一图层，以比特数大于或等于预设比特数的图层作为第二图层；
    第一合成模块，用于通过图形处理器对所述第一图层进行合成，获得中间图层；
    第二合成模块，用于通过多媒体显示处理器将所述第二图层以及所述中间图层进行合成，获得待显示界面用于显示。
  19. 一种电子设备，其特征在于，包括存储器、处理器、图形处理器以及多媒体显示处理器，所述存储器、图形处理器以及多媒体显示处理器耦接到所述处理器，所述存储器存储指令，当所述指令由所述处理器执行时，所述处理器执行如权利要求1-17任一项所述的方法，所述图形处理器以及所述多媒体显示处理器执行如权利要求1-17中的图层合成。
  20. 一种计算机可读取存储介质，其特征在于，所述计算机可读取存储介质中存储有程序代码，所述程序代码可被处理器调用执行如权利要求1-7任一项所述的方法。
PCT/CN2020/100852 2019-07-17 2020-07-08 图层合成方法、装置、电子设备及存储介质 WO2021008418A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP20839629.1A EP4002281A4 (en) 2019-07-17 2020-07-08 METHOD AND DEVICE FOR LAYER COMPOSITION, ELECTRONIC DEVICE AND STORAGE MEDIA
US17/576,640 US20220139017A1 (en) 2019-07-17 2022-01-14 Layer composition method, electronic device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910647561.3A CN110363831B (zh) 2019-07-17 2019-07-17 图层合成方法、装置、电子设备及存储介质
CN201910647561.3 2019-07-17

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/576,640 Continuation US20220139017A1 (en) 2019-07-17 2022-01-14 Layer composition method, electronic device, and storage medium

Publications (1)

Publication Number Publication Date
WO2021008418A1 true WO2021008418A1 (zh) 2021-01-21

Family

ID=68220149

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/100852 WO2021008418A1 (zh) 2019-07-17 2020-07-08 图层合成方法、装置、电子设备及存储介质

Country Status (4)

Country Link
US (1) US20220139017A1 (zh)
EP (1) EP4002281A4 (zh)
CN (1) CN110363831B (zh)
WO (1) WO2021008418A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110363831B (zh) * 2019-07-17 2023-04-07 Oppo广东移动通信有限公司 图层合成方法、装置、电子设备及存储介质
CN110377263B (zh) * 2019-07-17 2021-08-17 Oppo广东移动通信有限公司 图像合成方法、装置、电子设备及存储介质
CN110992447B (zh) * 2019-12-05 2023-05-05 北京中网易企秀科技有限公司 一种图文适配方法、装置、存储介质及设备
CN112767231B (zh) * 2021-04-02 2021-06-22 荣耀终端有限公司 图层合成方法和设备
US11705091B2 (en) * 2021-09-30 2023-07-18 Qualcomm Incorporated Parallelization of GPU composition with DPU topology selection
CN116795197A (zh) * 2022-03-17 2023-09-22 华为技术有限公司 图层处理方法和电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103839533A (zh) * 2014-01-23 2014-06-04 华为技术有限公司 一种移动终端图像的显示方法和移动终端
CN105741819A (zh) * 2016-03-01 2016-07-06 晨星半导体股份有限公司 一种图层处理方法及装置
US20180189922A1 (en) * 2016-12-31 2018-07-05 Intel IP Corporation Smart composition of output layers
US10257487B1 (en) * 2018-01-16 2019-04-09 Qualcomm Incorporated Power efficient video playback based on display hardware feedback
CN110363831A (zh) * 2019-07-17 2019-10-22 Oppo广东移动通信有限公司 图层合成方法、装置、电子设备及存储介质

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8018472B2 (en) * 2006-06-08 2011-09-13 Qualcomm Incorporated Blending multiple display layers
JP5474887B2 (ja) * 2011-08-01 2014-04-16 株式会社ソニー・コンピュータエンタテインメント 動画データ生成装置、動画像表示装置、動画データ生成方法、動画像表示方法、および動画像ファイルのデータ構造
GB2544333B (en) * 2015-11-13 2018-02-21 Advanced Risc Mach Ltd Display controller
CN107615770B (zh) * 2016-03-31 2020-02-21 华为技术有限公司 应用程序的处理方法及终端设备
WO2017173576A1 (zh) * 2016-04-05 2017-10-12 华为技术有限公司 一种显示方法及终端
KR102488333B1 (ko) * 2016-04-27 2023-01-13 삼성전자주식회사 그래픽 데이터를 합성하는 전자 장치 및 방법
US10748235B2 (en) * 2018-08-08 2020-08-18 Qualcomm Incorporated Method and system for dim layer power optimization in display processing
CN109871192B (zh) * 2019-03-04 2021-12-31 京东方科技集团股份有限公司 一种显示方法、装置、电子设备及计算机可读存储介质
CN109992347B (zh) * 2019-04-10 2022-03-25 Oppo广东移动通信有限公司 界面显示方法、装置、终端及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103839533A (zh) * 2014-01-23 2014-06-04 华为技术有限公司 一种移动终端图像的显示方法和移动终端
CN105741819A (zh) * 2016-03-01 2016-07-06 晨星半导体股份有限公司 一种图层处理方法及装置
US20180189922A1 (en) * 2016-12-31 2018-07-05 Intel IP Corporation Smart composition of output layers
US10257487B1 (en) * 2018-01-16 2019-04-09 Qualcomm Incorporated Power efficient video playback based on display hardware feedback
CN110363831A (zh) * 2019-07-17 2019-10-22 Oppo广东移动通信有限公司 图层合成方法、装置、电子设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4002281A4

Also Published As

Publication number Publication date
EP4002281A4 (en) 2022-07-13
EP4002281A1 (en) 2022-05-25
CN110363831B (zh) 2023-04-07
CN110363831A (zh) 2019-10-22
US20220139017A1 (en) 2022-05-05

Similar Documents

Publication Publication Date Title
WO2021008420A1 (zh) 图层合成方法、装置、电子设备及存储介质
WO2021008418A1 (zh) 图层合成方法、装置、电子设备及存储介质
CN110377264B (zh) 图层合成方法、装置、电子设备及存储介质
CN110377263B (zh) 图像合成方法、装置、电子设备及存储介质
WO2021008373A1 (zh) 显示方法、装置、电子设备及计算机可读介质
CN109379627B (zh) 视频处理方法、装置、电子设备以及存储介质
WO2021008427A1 (zh) 图像合成方法、装置、电子设备及存储介质
US8803896B2 (en) Providing a coherent user interface across multiple output devices
JP2013126185A (ja) 情報処理装置、画像送信方法及び画像送信プログラム
WO2017202170A1 (zh) 一种图像压缩方法、装置及电子设备
AU2017317839B2 (en) Panoramic video compression method and device
CN111209422A (zh) 图像显示方法、装置、电子设备及存储介质
WO2017202175A1 (zh) 一种视频压缩方法、装置及电子设备
CN112399123B (zh) 视频清晰度的调整方法、装置、电子设备及存储介质
JP6003049B2 (ja) 情報処理装置、画像送信方法及び画像送信プログラム
CN112596843A (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
US8780120B2 (en) GPU self throttling
CN109587555B (zh) 视频处理方法、装置、电子设备及存储介质
CN109587561B (zh) 视频处理方法、装置、电子设备及存储介质
CN109688462B (zh) 降低设备功耗的方法、装置、电子设备及存储介质
CN114697555B (zh) 一种图像处理方法、装置、设备及存储介质
CN111475665B (zh) 图片播放方法、装置、终端及存储介质
US20230056599A1 (en) System and method for rendering a design including a dynamic design element
CN113986168A (zh) 一种图像显示方法、装置、设备及可读存储介质
CN118138784A (zh) 视频分割压缩方法、装置、设备以及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20839629

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020839629

Country of ref document: EP

Effective date: 20220217