WO2021136011A1

WO2021136011A1 - Display method and device

Info

Publication number: WO2021136011A1
Application number: PCT/CN2020/138297
Authority: WO
Inventors: 汤中峰; 刘中志
Original assignee: 华为技术有限公司
Priority date: 2019-12-31
Filing date: 2020-12-22
Publication date: 2021-07-08
Also published as: CN111208966A; CN111208966B

Abstract

A display method and device. The method comprises: encoding contents of multiple display regions, and generating a first buffer (S201), the contents of the multiple display regions being contents to be displayed in different regions of a frame to be displayed by a display assembly of a terminal apparatus, and the first buffer containing information of the multiple display regions arranged in series; transmitting the first buffer to the display assembly (S202); and controlling, according to the first buffer, the display assembly to display the contents of the multiple display regions in the frame (S203). In the method, information of multiple display regions are arranged in series in a first buffer, thereby preventing inefficient use of a DDR memory as a result of transmitting information of inactive display regions, reducing a data volume in a transmission process, and correspondingly reducing power consumption of a terminal apparatus.

Description

Display method and device

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office, the application number is 201911416443.8, and the application name is "Display Method and Apparatus" on December 31, 2019, the entire content of which is incorporated into this application by reference.

Technical field

The embodiments of the present application relate to the field of display, and in particular, to a display method and device.

Background technique

The display of terminal equipment mainly includes a hardware combination abstract component (hwcomposer, HWC) method and an OpenGL ES (OpenGL for Embedded Systems) method. Taking HWC as an example, when a picture of a certain frame needs to be displayed, it is usually necessary to synthesize multiple layers of the frame through the HWC, and determine the multiple actual display areas that need to be displayed from the synthesized picture.

In the prior art, when the information of the display area is sent to the display component, a single display method is often used. Therefore, when calculating the size of the display area before the display is performed, multiple actual display areas are often taken. The union of is used as the final display area, and the final display area is sent to the display component for display.

However, the final display area determined by the union method will cause double data rate (DDR) memory waste when transmitting the information in the display area, thereby increasing the amount of data in the transmission process. And correspondingly increased the power consumption of the terminal equipment.

Summary of the invention

The embodiments of the present application provide a display method and device to solve the problem of waste of DDR memory caused by the transmission of information in the display area in the prior art.

The first aspect provides a display method, which can be applied to a display device, and can also be applied to a chip of the display device, wherein the display device can be any terminal device with a display component. The following describes the application to a display device as an example. In this method, the display device encodes the content of multiple display areas to generate a first buffer, and the content of the multiple display areas is the display component of the terminal device. For the content of the frame to be displayed in different areas, the first buffer is serially arranged with the information of the multiple display areas; after the first buffer is transmitted to the display component, according to the The first buffer controls the display component to display the content of the multiple display areas in the frame to be displayed.

Through the display method provided in the first aspect, the information of multiple display areas can be serially transmitted to the display component, thereby avoiding the waste of DDR memory caused by the transmission of invalid display areas, reducing the amount of data in the transmission process, and correspondingly , Which reduces the power consumption of terminal equipment.

In a possible implementation manner, the information of the display area includes location information of the display area and content information of the display area.

With the display method provided by this possible implementation, since the first buffer includes the location information of the display area and the content information of the display area, the display component can directly obtain the location and content of the display area.

In a possible implementation manner, the first buffer stores location information and content information of the multiple display areas, and the location information is stored before the content information.

With the display method provided by this possible implementation, since the order of the information in the first buffer is preset, after the display component extracts a group of information in the display area, there is no need to process the information and directly send the information. The position of the display area displays the content of the display area, which simplifies the display steps.

In a possible implementation manner, the controlling the display component to display the content of the multiple display areas in the frame to be displayed according to the first buffer includes: controlling the decoding method of the display component The first buffer acquires position information of a plurality of display areas and content information of a plurality of display areas; controls the display components to display the corresponding display areas at the positions of the display areas respectively The content of the area.

With the display method provided by this possible implementation, by displaying the content of the corresponding display area at the position of the display area, the content of the invalid area is avoided to be displayed, and the power consumption of the display device is reduced.

In a possible implementation manner, before the encoding the content of the multiple display areas to generate the first buffer, the method further includes: synthesizing the multiple display components to be displayed in the frame to be displayed. Layer, generating a preset picture to be displayed in the frame to be displayed; extracting the multiple display areas from the preset picture.

With the display method provided by this possible implementation manner, the multiple display areas can be determined and extracted from multiple layers to be displayed in the frame to be displayed.

In a possible implementation, the display area is a rectangular area.

With the display method provided by this possible implementation manner, since the display component is usually rectangular, the rectangular display area can be better adapted to the display component, and display is more convenient.

In a possible implementation manner, the multiple display areas do not overlap.

Through the display method provided by this possible implementation, multiple display areas do not overlap, which can avoid repeated display, thereby further avoiding the waste of DDR memory caused by the transmission of invalid display areas, and reducing the data in the transmission process. Therefore, the power consumption of the terminal equipment is reduced accordingly.

A second aspect provides a display device, including: a processing module, a sending module, and a storage module;

Wherein, the storage module is used to couple with the processing module to store necessary program instructions and data;

Wherein, the storage module is coupled with the processing module and the sending module, and is used to store executable instructions of the display device.

The processing module is configured to call executable instructions of the display device, so that the display device encodes the content of multiple display areas to generate a first buffer, and the content of the multiple display areas is a display The display component of the device displays the content in different areas in the frame to be displayed, and the first buffer has the information of the multiple display areas arranged in series; according to the first buffer, the display component is controlled to be displayed in Displaying the content of the multiple display areas in the frame to be displayed;

The sending module is configured to call an executable instruction of the display device, so that the display device transmits the first buffer to the display component.

In a possible implementation manner, the processing module is specifically configured to invoke executable instructions of the display device, so that the display device controls the display component to decode the first buffer and obtain a plurality of the Position information of the sending display area and content information of a plurality of the sending display areas; controlling the display component to respectively display the content of the corresponding sending display area at the position of the sending display area.

In a possible implementation manner, the processing module is further configured to call executable instructions of the display device, so that the display device synthesizes the multiple images to be displayed by the display component in the frame to be displayed. Layer, generating a preset picture to be displayed in the frame to be displayed; extracting the multiple display areas from the preset picture.

In a possible implementation, the display area is a rectangular area.

In a possible implementation manner, the multiple display areas do not overlap.

On the third aspect, an embodiment of the present application provides a chip, including: a memory and a processor, the memory is used to store program instructions, and the processor is used to call the program instructions in the memory to execute the display method described in the first aspect and its alternatives .

A fourth aspect provides a program, which when executed by a processor is used to execute the display method described in the above first aspect and its alternative solutions.

The fifth aspect provides a program product, such as a computer-readable storage medium, including the program of the fourth aspect.

A sixth aspect provides a computer-readable storage medium. The computer-readable storage medium stores instructions that, when run on a computer, cause the computer to execute the display method described in the first aspect and its alternative solutions.

The display method and device provided by the present application generate a first buffer by encoding the contents of multiple display areas. The contents of the multiple display areas are the contents displayed in different areas by the display components of the terminal device in the frame to be displayed. , The first buffer is serially arranged with multiple display area information; then the first buffer is transmitted to the display component, and according to the first buffer, the display component is controlled to display multiple display areas in the frame to be displayed content. Through the above method, the information of multiple display areas is serially arranged in the first buffer, thereby avoiding the waste of DDR memory caused by the transmission of invalid display areas, reducing the amount of data in the transmission process, and correspondingly reducing The power consumption of the terminal device.

Description of the drawings

Figure 1 is an existing schematic diagram of determining the final display area;

Figure 2 is another existing schematic diagram of determining the final display area;

FIG. 3 is a schematic structural diagram of a terminal device provided by an embodiment of this application;

FIG. 4 is a schematic flowchart of a display method provided by an embodiment of the application;

FIG. 5 is a schematic diagram of a display area provided by an embodiment of the application;

Fig. 6 is a schematic diagram of another display area provided by an embodiment of the application;

FIG. 7 is a schematic diagram of yet another display area provided by an embodiment of the application;

FIG. 8 is a schematic diagram of encoding and decoding provided by an embodiment of the application;

FIG. 9 is a schematic flowchart of another display method provided by an embodiment of the application;

FIG. 10 is a schematic flowchart of yet another display method provided by an embodiment of this application;

FIG. 11 is a schematic diagram of a display principle according to an embodiment of the application;

Figure 12 is a signaling interaction diagram of a display method provided by an embodiment of the application

FIG. 13 is a schematic structural diagram of a display device provided by an embodiment of the application;

FIG. 14 is a schematic structural diagram of another terminal device provided by an embodiment of this application;

FIG. 15 is a structural block diagram when the terminal device provided by an embodiment of the application is a mobile phone.

Detailed ways

In the terminal device, when it is necessary to display a picture of a certain frame, it is usually necessary to synthesize multiple layers of the frame through HWC or OpenGL ES, and determine the multiple actual display areas that need to be displayed from the synthesized picture , And take the union of the actual display area to determine the final display area.

Fig. 1 is a schematic diagram of an existing method for determining the final display area. As shown in Figure 1, in the prior art, if there are two actual display areas, area A and area B, and area A and area B partially overlap, a single area is usually used for display when calculating the final display area. Take the union C of multiple actual display areas of the current frame as the final display area.

Figure 2 is another conventional schematic diagram of determining the final display area. As shown in Figure 2, in the prior art, if there are two actual display areas, area D and area E, area D is located at the upper left corner of the screen, area E is located at the lower right corner of the screen, and there is no space between area D and area E. Overlap, in calculating the final display area, since the union of area D and area E is a complete screen area, the entire screen area needs to be regarded as the final display area.

It can be seen from Figure 1 and Figure 2 that the final display area determined by HWC, especially the final display area shown in Figure 2, is often larger than the actual display area, which will result in double rate when transmitting the information in the display area. The waste of (double data rate, DDR) memory further increases the amount of data in the transmission process and correspondingly increases the power consumption of the terminal equipment.

In order to solve the above-mentioned problem, the embodiment of the present application provides a display method, thereby avoiding the waste of DDR memory caused by transmitting an invalid display area, and reducing the amount of data in the transmission process.

The display method provided by the embodiment of the present application can be applied to any terminal device. The terminal device may be a wireless terminal, which may be a device that provides voice and/or other service data connectivity to a user, a handheld device with a wireless connection function, or other processing devices connected to a wireless modem. A wireless terminal can communicate with one or more core networks via a radio access network (RAN). The wireless terminal can be a mobile terminal, such as a mobile phone (or “cellular” phone) and a computer with a mobile terminal. For example, they can be portable, pocket-sized, handheld, computer-built or vehicle-mounted mobile devices that exchange language and/or data with the wireless access network. For example, personal communication service (PCS) phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants, PDA) and other equipment. A wireless terminal can also be called a system, a subscriber unit, a subscriber station, a mobile station, a mobile station, a remote station, and a remote terminal. The access terminal (access terminal), user terminal (user terminal), and user agent (user agent) are not limited here.

FIG. 3 is a schematic structural diagram of a terminal device provided by an embodiment of the application. As shown in FIG. 3, the terminal device may include: a processor 101 and a display component 102. Optionally, the terminal may further include a communication module 103, a radio frequency (RF) circuit 104, a power supply 105, and a memory 106.

Among them, the processor 101 is the control center of the terminal. It uses various interfaces and lines to connect the various parts of the entire terminal device, runs or executes the software programs and/or modules stored in the memory 106, and calls the data stored in the memory 106. Data, perform various functions of the terminal and process data, so as to monitor the terminal as a whole.

The display component 102 can be used to display information transmitted by the processor 101 in a specific area. The display component 102 may include a display panel. Optionally, the display panel may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc.

The RF circuit 104 is connected to the communication module 103 and can be used for receiving and sending signals during information transmission or communication. For example, after receiving the downlink information of the base station, it is processed by the processor 101; in addition, the uplink data is sent to the base station.

The memory 106 may be used to store software programs and modules. The processor 101 executes various functional applications and data processing of the terminal by running the software programs and modules stored in the memory 106.

The terminal also includes a power source 105 (such as a battery) for supplying power to various components. Optionally, the power source can be logically connected to the processor 101 through a power management system, so that functions such as charging, discharging, and power consumption management can be managed through the power management system.

It can be understood that the display method provided in the embodiments of the present application may be implemented by a display device, and the display device may be part or all of a certain device, for example, the above-mentioned terminal device.

Where instructions are needed, the display method provided in the embodiments of this application can not only be used in the display of HWC, but also can be used in the display of OpenGL ES, which is not limited in this application.

The following takes a terminal device integrated or installed with relevant execution codes as an example, and specific embodiments are used to describe the technical solutions of the embodiments of the present application in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

FIG. 4 is a schematic flowchart of a display method provided by an embodiment of the application. The embodiment of the present application relates to the specific process of how the terminal device displays the display area. The execution subject of this embodiment is a terminal device. As shown in Figure 4, the display method includes:

S201. Encode the content of multiple display areas to generate a first buffer, where the content of the multiple display areas is the content displayed in different areas by the display component of the terminal device in the frame to be displayed, and the first buffer is serialized There are multiple display areas of information in the arrangement.

In this step, when the terminal device determines the multiple display areas of the frame to be displayed, it no longer takes the union of the multiple display areas, but directly encodes the content of the multiple display areas, and the multiple display areas The area information is stored in a first buffer (buffer).

First, the display area in this application will be described.

The display area is the area where the display component of the terminal device needs to refresh and display the content in the frame to be displayed, where the frame to be displayed can be any frame that the terminal device transmits and displays.

The display method provided in the embodiment of the present application is applicable to the situation where there are multiple display areas in the frame to be displayed. The embodiment of the present application does not limit the number of display areas, and for example, it may be two, three, four, etc. In an alternative embodiment, the display area may be a rectangular area. Since the display component is usually a rectangle, the rectangular display area can be better adapted to the display component, and display is more convenient.

FIG. 5 is a schematic diagram of a display area provided by an embodiment of the application. FIG. 6 is a schematic diagram of another display area provided by an embodiment of the application. The multiple display areas may not overlap each other as shown in FIG. 5, or may partially overlap as shown in FIG. 6, which is not limited in the embodiment of the present application.

There is a possible way. In some embodiments, multiple display areas do not overlap, so as to avoid overlapping display areas of the display area, thereby further avoiding the waste of DDR memory caused by the transmission of invalid display areas. The amount of data in the transmission process is reduced, and the power consumption of the terminal equipment is correspondingly reduced.

FIG. 7 is a schematic diagram of another display area provided by an embodiment of the application. In some embodiments, as shown in FIG. 7, the partially overlapping display area can be re-divided, for example, as shown in FIG. 6. The shown display sending area A and display sending area B can be divided into non-overlapping display sending areas A, B, and C, so as to avoid repeated display sending.

The encoding process in this application will be described below.

Among them, the first buffer (buffer) is usually used in the process of data transmission between the terminal device and the peripheral device of the terminal device, and is used to temporarily store the data sent by the terminal device to the peripheral device.

The embodiments of this application do not limit how to encode the contents of multiple display areas. In some embodiments, the contents of multiple display areas can be character-encoded one by one, and then the character-encoded data are stored one by one in In the first buffer. Among them, the embodiment of the present application does not limit the encoding form. Exemplary, character encoding forms such as ascii, utf8, utf16le, ucs2, base64, etc. may be used.

After completing the encoding of the contents of the multiple display areas, the processor serially arranges the encoded data in the first buffer. Among them, the serial arrangement is in the form of serial communication, in which the information of multiple display areas are arranged end to end in the first buffer.

Compared with the buffer generated by the union of the display area encoding in the prior art, the first buffer in this application does not contain the data of the invalid display area, thereby avoiding the DDR caused by the transmission of the invalid display area. Waste of memory.

S202: Transmit the first buffer to the display component.

In this step, after the terminal device encodes the content of the multiple display areas to generate the first buffer, the first buffer can be transmitted to the display component, so that the display component can display.

Among them, the embodiment of the present application does not show how to transmit the first buffer, and a commonly used data transmission protocol may be used for transmission.

S203: According to the first buffer, control the display component to display the content of multiple display areas in the frame to be displayed.

In this step, after the display component receives the first buffer, the information of the multiple display areas in the first buffer can be used to control the display component to display the content of the multiple display areas in the frame to be displayed.

In some embodiments, the information of multiple display areas may be extracted from the first buffer at a time, and then the information of the display area may be parsed to the display component for display. It should be noted that when the contents of multiple display areas are displayed on the display component, they need to be displayed in the same frame.

The display method provided by the embodiments of the present application generates a first buffer by encoding the content of multiple display areas, and the content of the multiple display areas is the content displayed in different areas by the display component of the terminal device in the frame to be displayed. , The first buffer is serially arranged with multiple display area information; then the first buffer is transmitted to the display component, and according to the first buffer, the display component is controlled to display multiple display areas in the frame to be displayed content. Through the above method, the information of multiple display areas is serially arranged in the first buffer, thereby avoiding the waste of DDR memory caused by the transmission of invalid display areas, reducing the amount of data in the transmission process, and correspondingly reducing The power consumption of the terminal device.

On the basis of the foregoing embodiment, the information of the display area stored in the first buffer will be described below. In some alternative implementations, the information of the display area includes the location information of the display area and the content information of the display area. Correspondingly, the first buffer stores the location information and content information of multiple display areas. , The location information is stored before the content information.

FIG. 8 is a schematic diagram of encoding and decoding provided by an embodiment of the application. As shown in Fig. 8, after the terminal device determines that the display area in the HWC includes area 1 and area 2, the content of area 1 and area 2 is encoded to generate a first buffer. The first buffer sequentially stores the screen position pos1 corresponding to area 1, the display data buf1 of area 1, the screen position pos2 corresponding to area 2, and the display data buf2 of area 2, forming a single buffer. Among them, the area is not limited to two.

After the first buffer is generated, the first buffer can be transmitted to the display component via DDR, and the display component sequentially decodes the screen position pos1 of area 1, the display data buf1 of area 1, and the screen of area 2 from the first buffer. The display data buf2 of the position pos2 and the area 2, and the display data of the area 1 is displayed at the screen position of the area 1, and the display data of the area 2 is displayed at the screen position of the area 2.

FIG. 9 is a schematic flowchart of another display method provided by an embodiment of the application. The embodiment of the present application relates to the specific process of how the terminal device displays the display area. On the basis of the foregoing embodiment, as shown in FIG. 9, the display method includes:

S301. Encode the content of multiple display areas to generate a first buffer, where the content of the multiple display areas is the content displayed in different areas by the display component of the terminal device in the frame to be displayed, and the first buffer is serialized The information of multiple display areas is arranged; the information of the display area includes the position information of the display area and the content information of the display area.

S302: Transmit the first buffer to the display component.

The technical terms, technical effects, technical features, and alternative implementations of steps S301-S302 can be understood with reference to steps S201-S202 shown in FIG. 4, and the repeated content will not be repeated here.

S303. Control the display component to decode the first buffer, and obtain the position information of the multiple display areas and the content information of the multiple display areas.

In this step, after the display component receives the first buffer, it can control the display component to parse the first buffer to obtain the position information of the multiple display areas and the content information of the multiple display areas.

In this application, there is no restriction on how to decode the first buffer. In some embodiments, the first buffer can be decoded in a decoding manner corresponding to the encoding method used to generate the first buffer.

S304. Control the display component to display the content of the corresponding display area at the position of the display area respectively.

In this step, since the position information of the multiple display areas and the content information of the multiple display areas are stored in the first buffer, the processor can control the display components to respectively display the corresponding display areas at the positions of the display areas. Content.

Exemplarily, the first buffer stores location information and content information of area A, and location information and content information of area B. Since the information in the first buffer is arranged in series, the extracted information is the location information of area A, the content information of area A, the location information of area B, and the content information of area B in order. The processor determines the location of area A on the display component, displays the content of area A at the location of area A, determines the location of area B on the display component, and displays the content of area B at the location of area B.

The display method provided by the embodiments of the present application generates a first buffer by encoding the content of multiple display areas, and the content of the multiple display areas is the content displayed in different areas by the display component of the terminal device in the frame to be displayed. In the first buffer, there are serially arranged information of multiple display areas; the information of the display area includes the position information of the display area and the content information of the display area, and then the first buffer is transmitted to the display component, And control the display component to decode the first buffer, obtain the position information of the multiple display areas and the content information of the multiple display areas, and control the display components to display the content of the corresponding display area at the positions of the display areas. In this way, the content of the corresponding display area can be displayed at the position of the display area, which avoids displaying the content of the invalid area and reduces the power consumption of the display device.

On the basis of the foregoing embodiment, the following describes how to determine multiple display areas. FIG. 10 is a schematic flowchart of yet another display method provided by an embodiment of the application. The embodiment of the present application relates to the specific process of how the terminal device displays the display area. As shown in Figure 10, the display method includes:

S401: Generate a preset picture to be displayed in the frame to be displayed by synthesizing multiple layers to be displayed in the frame to be displayed by the display component.

In this step, when the content of the frame to be displayed needs to be displayed, multiple layers to be displayed in the frame to be displayed can be obtained, and the multiple layers to be displayed in the frame to be displayed are synthesized to generate the displayed frame to be displayed Preset screen.

In some embodiments, the multiple layers to be displayed in the frame to be displayed may be drawn by a graphics processing unit (GPU), respectively. The composition of the above-mentioned layers can be completed by the HWC module or the OpenGL ES module, and the specific process of composition is not limited in this application.

S402: Extract multiple display areas from the preset screen.

In this step, after the preset picture to be displayed in the frame to be displayed is generated, multiple display areas can be extracted from the preset picture.

In some embodiments, the actual display area in the preset picture to be displayed in the frame to be displayed can be determined, and the display area can be divided from the actual display area. In other embodiments, the preset picture to be displayed in the frame to be displayed may be compared with the picture to be displayed in the previous frame, and the changed area is used as the display area.

In some optional implementation manners, when determining the display area, the display area may be divided into rectangular areas, so that it can be better adapted to the display component and more convenient for display.

S403. Encode the content of the multiple display areas to generate a first buffer, where the content of the multiple display areas is the content displayed in different areas by the display component of the terminal device in the frame to be displayed, and serially in the first buffer There are multiple display areas of information in the arrangement.

S404: Transmit the first buffer to the display component.

S405. According to the first buffer, control the display component to display the content of the multiple display areas in the frame to be displayed.

The technical terms, technical effects, technical features, and alternative implementations of steps S403-S405 can be understood with reference to steps S201-S203 shown in FIG. 4, and the repeated content will not be repeated here.

The display method provided by the embodiments of the present application generates a preset screen to be displayed in the frame to be displayed by synthesizing multiple layers to be displayed by the display component in the frame to be displayed; and extracts multiple display areas from the preset screen. In this way, multiple display areas can be determined and extracted from multiple layers to be displayed in the frame to be displayed, and the information of the multiple display areas can be serially arranged in the first buffer, thereby avoiding invalid transmission. The waste of DDR memory caused by the display area reduces the amount of data in the transmission process and correspondingly reduces the power consumption of the terminal device.

FIG. 11 is a schematic diagram of a display principle according to an embodiment of the application. As shown in Figure 11, the application (APP) in the terminal device may correspond to one or more graphical interfaces. When the display is required, the APP first generates the display data according to the graphical interface, and then, through the system service The layer (system server) sends the display data to SurfaceFinger. SurfaceFinger is a system service, which is mainly used to divide multiple layers from the display data and determine the overlapping display areas of multiple layers. Subsequently, SurfaceFinger sends the information of the display area to the HWC in the hardware abstraction layer. The HWC encodes the contents of multiple display areas through the Display Controller to generate a first buffer, where the first buffer The processor can be the FrameBuffer of the core layer. The liquid crystal display (LCD) displays the contents of multiple display areas according to the FrameBuffer.

Among them, the embodiment of the present application does not limit the display type, and it may be online synthesis or offline synthesis. Online composition can be displayed layer by layer in real time, and offline composition is to store the layers in FrameBuffer after composition, and then display them by the display component.

FIG. 12 is a signaling interaction diagram of a display method provided by an embodiment of this application. This embodiment mainly describes the signaling interaction between the various layers of the terminal device during display, so as to show the flow of each signaling of the terminal device during actual display. As shown in Figure 12, the display method includes:

S501. The SurfaceFinger generates a preset picture to be displayed in the frame to be displayed by synthesizing multiple layers to be displayed in the frame to be displayed by the display component.

S502. SurfaceFinger extracts multiple display areas from the preset screen.

S503. The SurfaceFinger sends information of multiple display areas to the HWC.

S504. The HWC encodes the content of the multiple display areas, and generates a first buffer in the FrameBuffer.

S505. The FrameBuffer transmits the first buffer to the display component.

S506: The display component displays the content of the multiple display areas in the frame to be displayed according to the first buffer.

A person of ordinary skill in the art can understand that all or part of the steps in the above method embodiments can be implemented by a program instructing relevant hardware. The foregoing program can be stored in a computer readable storage medium. When the program is executed, it is executed. Including the steps of the foregoing method embodiment; and the foregoing storage medium includes: ROM, RAM, magnetic disk, or optical disk and other media that can store program codes.

FIG. 13 is a schematic structural diagram of a display device provided by an embodiment of the application. The display device can be used to execute the display method in the foregoing embodiment, and may be the terminal device in the foregoing embodiment. As shown in FIG. 13, the display device includes: a processing module 601, a sending module 602, and a storage module 603; wherein, The storage module 603 is coupled with the processing module 601 and the sending module 602, and is used to store executable instructions of the display device.

The processing module 601 is configured to call the executable instructions of the display device, so that the display device encodes the content of the multiple display areas to generate a first buffer. The content of the multiple display areas is the display component of the display device that is to be displayed. The content of the frame displayed in different areas, the first buffer is serially arranged with the information of multiple display areas; according to the first buffer, the display component is controlled to display the content of the multiple display areas in the frame to be displayed;

The sending module 602 is configured to call an executable instruction of the display device, so that the display device transmits the first buffer to the display component.

In an optional implementation manner, the information of the display area includes location information of the display area and content information of the display area.

In an optional implementation manner, the first buffer stores location information and content information of multiple display areas, and the location information is stored before the content information.

In an alternative embodiment, the processing module 601 is specifically configured to call executable instructions of the display device, so that the display device controls the display component to decode the first buffer, and obtain the position information of multiple display areas and multiple display areas. The content information of the area; control the display components to respectively display the content of the corresponding display area in the position of the display area.

In an optional implementation manner, the processing module 601 is further configured to call executable instructions of the display device, so that the display device generates multiple layers to be displayed in the frame to be displayed by synthesizing the display component in the frame to be displayed. The preset screen; extract multiple display areas from the preset screen.

In an alternative embodiment, the display area is a rectangular area.

In an optional implementation manner, the multiple display areas do not overlap.

The display device provided in the present application can execute the actions of the terminal device in the foregoing method embodiments, and its implementation principles and technical effects are similar, and will not be repeated here.

FIG. 14 is a schematic structural diagram of another terminal device provided by an embodiment of this application. As shown in FIG. 14, the terminal device may include: a processor 701 (for example, a CPU), a memory 702, a receiver 703, and a transmitter 704; the receiver 703 and the transmitter 704 are coupled to the processor 701, and the processor 701 controls the receiver Transceiving actions between 703 and transmitter 704; memory 702 may include high-speed random-access memory (RAM) or non-volatile memory (NVM), such as at least two disks The memory. The memory 702 can store various instructions for completing various processing functions and implementing the method steps of the present application. In an implementable manner, the terminal device involved in the present application may further include: a power supply 705, a communication bus 706, and a communication port 707. The receiver 703 and the transmitter 704 may be integrated in the transceiver of the test equipment, or may be independent transceiver antennas on the test equipment. The communication bus 706 is used to implement communication connections between components. The above-mentioned communication port 707 is used to realize connection and communication between the test equipment and other peripherals.

In the embodiment of the present application, the above-mentioned memory 702 is used to store computer executable program code, and the program code includes instructions; when the processor 701 executes the instructions, the instructions cause the processor 701 of the terminal device to execute the processing of the terminal device in the above method embodiment. The action is to cause the receiver 703 and the transmitter 704 to perform the receiving and sending actions of the terminal device in the foregoing method embodiment. The implementation principles and technical effects are similar, and will not be repeated here.

Just like the above embodiment, the terminal device involved in the embodiment of this application may be a wireless device such as a mobile phone, a tablet computer, etc. Therefore, taking the electronic device as a mobile phone as an example: FIG. 15 is a structural block diagram of the terminal device provided by the embodiment of the application as a mobile phone . Referring to FIG. 15, the mobile phone may include: a radio frequency (RF) circuit 1110, a memory 1120, an input unit 1130, a display unit 1140, a sensor 1150, an audio circuit 1160, a wireless fidelity (WiFi) module 1170, a processing The device 1180, the power supply 1190, the communication unit 1210 and other components. Those skilled in the art can understand that the structure of the mobile phone shown in FIG. 13 does not constitute a limitation on the mobile phone, and may include more or fewer components than those shown in the figure, or a combination of certain components, or different component arrangements.

The following describes the components of the mobile phone in detail with reference to Figure 15:

The RF circuit 1110 can be used for receiving and sending signals during the process of sending and receiving information or talking. For example, after receiving the downlink information of the base station, it is processed by the processor 1180; in addition, the uplink data is sent to the base station. Generally, the RF circuit includes, but is not limited to, an antenna, at least two amplifiers, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, and so on. In addition, the RF circuit 1110 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (Code Division) Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE)), Email, Short Messaging Service (SMS), etc.

The memory 1120 may be used to store software programs and modules. The processor 1180 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1120. The memory 1120 may mainly include a program storage area and a data storage area, where the program storage area may store an operating system, application programs required by at least two functions (such as sound playback function, image playback function, etc.), etc.; the storage data area may store Data created based on the use of mobile phones (such as audio data, phone book, etc.), etc. In addition, the memory 1120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least two magnetic disk storage devices, flash memory devices, or other volatile solid-state storage devices.

The input unit 1130 may be used to receive inputted digital or character information, and generate key signal input related to user settings and function control of the mobile phone. Specifically, the input unit 1130 may include a touch panel 1131 and other input devices 1132. The touch panel 1131, also called a touch screen, can collect user touch operations on or near it (for example, the user uses any suitable objects or accessories such as fingers, stylus, etc.) on the touch panel 1131 or near the touch panel 1131. Operation), and drive the corresponding connection device according to the preset program. Optionally, the touch panel 1131 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch position, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it To the processor 1180, and can receive and execute the commands sent by the processor 1180. In addition, the touch panel 1131 can be implemented in multiple types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1131, the input unit 1130 may also include other input devices 1132. Specifically, other input devices 1132 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control buttons, switch buttons, etc.), trackball, mouse, and joystick.

The display unit 1140 may be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1140 may include a display panel 1141. Optionally, the display panel 1141 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 1131 can cover the display panel 1141. When the touch panel 1131 detects a touch operation on or near it, it transmits it to the processor 1180 to determine the type of the touch event, and then the processor 1180 according to The type of touch event provides corresponding visual output on the display panel 1141. Although in FIG. 15, the touch panel 1131 and the display panel 1141 are used as two independent components to realize the input and input functions of the mobile phone, but in some embodiments, the touch panel 1131 and the display panel 1141 can be integrated. Realize the input and output functions of the mobile phone.

The mobile phone may also include at least one sensor 1150, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor can include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1141 according to the brightness of the ambient light. The light sensor can close the display panel 1141 and/or when the mobile phone is moved to the ear. Or backlight. As a kind of motion sensor, the acceleration sensor can detect the magnitude of acceleration in various directions (usually three-axis), and can detect the magnitude and direction of gravity when it is stationary. It can be used to identify the application of mobile phone gestures (such as horizontal and vertical screen switching, related games). , Magnetometer attitude calibration), vibration recognition related functions (such as pedometer, percussion), etc.; as for other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that can be configured in mobile phones, I won’t repeat them here. .

The audio circuit 1160, the speaker 1161, and the microphone 1162 can provide an audio interface between the user and the mobile phone. The audio circuit 1160 can transmit the electrical signal converted from the received audio data to the speaker 1161, which is converted into a sound signal by the speaker 1161 for output; on the other hand, the microphone 1162 converts the collected sound signal into an electrical signal, and the audio circuit 1160 After being received, the audio data is converted into audio data, and then processed by the audio data output processor 1180, and sent to, for example, another mobile phone via the RF circuit 1110, or the audio data is output to the memory 1120 for further processing.

WiFi is a short-distance wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages, and access streaming media through the WiFi module 1170. It provides users with wireless broadband Internet access. Although FIG. 15 shows the WiFi module 1170, it is understandable that it is not a necessary component of the mobile phone, and can be omitted as needed without changing the essence of the embodiments of the present application.

The processor 1180 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes by running or executing software programs and/or modules stored in the memory 1120, and calling data stored in the memory 1120. Various functions and processing data of the mobile phone can be used to monitor the mobile phone as a whole. Optionally, the processor 1180 may include one or more processing units; for example, the processor 1180 may integrate an application processor and a modem processor, where the application processor mainly processes an operating system, a user interface, and application programs, etc. The modem processor mainly deals with wireless communication. It can be understood that the foregoing modem processor may not be integrated into the processor 1180.

The mobile phone also includes a power supply 1190 (such as a battery) for supplying power to various components. Optionally, the power supply can be logically connected to the processor 1180 through a power management system, so that functions such as charging, discharging, and power consumption management can be managed through the power management system.

The mobile phone may also include a camera 1200, which may be a front camera or a rear camera. Although not shown, the mobile phone may also include a Bluetooth module, a GPS module, etc., which will not be repeated here.

In the embodiment of the present application, the processor 1180 included in the mobile phone may be used to execute the foregoing embodiment of the data transmission method, and its implementation principles and technical effects are similar, and will not be repeated here.

The embodiment of the present application also provides a chip including a processor and an interface. The interface is used to input and output data or instructions processed by the processor. The processor is used to execute the method provided in the above method embodiment. The chip can be used in terminal equipment.

The embodiment of the present application also provides a program, which is used to execute the method provided in the above method embodiment when the program is executed by the processor.

The embodiment of the present application also provides a program product, such as a computer-readable storage medium, in which instructions are stored, and when the program product runs on a computer, the computer executes the method provided in the foregoing method embodiment.

In the several embodiments provided in this application, it should be understood that the disclosed device and method may be implemented in other ways. For example, the device embodiments described above are merely illustrative, for example, the division of units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or integrated. To another system, or some features can be ignored, or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware, or may be implemented in the form of hardware plus software functional units.

It should be noted that the division of modules in the embodiments of the present application is illustrative, and is only a logical function division, and there may be other division methods in actual implementation. The functional modules in the embodiments of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or software functional modules.

If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disks or optical disks and other media that can store program codes. .

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, it can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. Computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions may be transmitted from a website, computer, server, or data center through a cable (such as Coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) to transmit to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media. The usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).

Claims

A display method, characterized in that it comprises:

The content of the multiple display areas is encoded to generate a first buffer, where the content of the multiple display areas is the content displayed in different areas by the display component of the terminal device in the frame to be displayed, in the first buffer The information of the multiple display areas are arranged serially;

Transmitting the first buffer to the display component;

According to the first buffer, the display component is controlled to display the content of the multiple display areas in the frame to be displayed.
The method according to claim 1, wherein the information of the display area includes location information of the display area and content information of the display area.
The method according to claim 2, wherein the first buffer stores location information and content information of the plurality of display areas, and the location information is stored before the content information.
The method according to claim 2, wherein the controlling the display component to display the content of the multiple display areas in the frame to be displayed according to the first buffer comprises:

Controlling the display component to decode the first buffer to obtain position information of a plurality of display areas and content information of a plurality of display areas;

The display components are controlled to respectively display the content of the corresponding display area at the position of the display area.
The method according to claim 1, characterized in that, before said encoding the content of the multiple display areas to generate the first buffer, the method further comprises:

Generating a preset picture to be displayed in the frame to be displayed by synthesizing the multiple layers to be displayed by the display component in the frame to be displayed;

Extracting the multiple display areas from the preset screen.
The method according to any one of claims 1 to 5, wherein the display area is a rectangular area.
The method according to any one of claims 1 to 5, wherein the multiple display areas do not overlap.
A display device, characterized by comprising: a processing module, a sending module, and a storage module;

Wherein, the storage module is coupled with the processing module and the sending module, and is used to store executable instructions of the display device;

The processing module is configured to call executable instructions of the display device, so that the display device encodes the content of multiple display areas to generate a first buffer, and the content of the multiple display areas is a display The display component of the device displays the content in different areas in the frame to be displayed, and the first buffer has the information of the multiple display areas arranged in series; according to the first buffer, the display component is controlled to be displayed in Displaying the content of the multiple display areas in the frame to be displayed;

The sending module is configured to call an executable instruction of the display device, so that the display device transmits the first buffer to the display component.
The device according to claim 8, wherein the information of the display area includes location information of the display area and content information of the display area.
The device according to claim 9, wherein the first buffer stores location information and content information of the plurality of display areas, and the location information is stored before the content information.
The device according to claim 9, wherein the processing module is specifically configured to invoke executable instructions of the display device, so that the display device controls the display component to decode the first buffer and obtain Multiple location information of the display area and multiple content information of the display area; controlling the display components to respectively display the content of the corresponding display area at the location of the display area.
8. The device according to claim 8, wherein the processing module is further configured to invoke executable instructions of the display device, so that the display device synthesizes the display component to display in the frame to be displayed Generating a preset picture to be displayed in the frame to be displayed; extracting the plurality of display areas from the preset picture.
The device according to any one of claims 8-12, wherein the display area is a rectangular area.
The device according to any one of claims 8-12, wherein the multiple display areas do not overlap.
A program product, characterized in that the program product includes a computer program, the computer program is stored in a readable storage medium, and at least one processor of a communication device can read the computer program from the readable storage medium The execution of the computer program by the at least one processor causes the communication device to implement the method according to any one of claims 1-7.