CN110187858B

CN110187858B - Image display method and system

Info

Publication number: CN110187858B
Application number: CN201910368406.8A
Authority: CN
Inventors: 陈先东
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2016-09-28
Filing date: 2016-09-28
Publication date: 2022-10-18
Anticipated expiration: 2036-09-28
Also published as: CN106469039A; CN106469039B; CN110187858A

Abstract

The invention discloses an image display method and system, and belongs to the technical field of electronics. The method comprises the following steps: the acquisition module acquires data to be displayed; the acquisition module decodes data to be displayed to obtain YUV data; the obtaining module sends the YUV data to the VO unit; the VO unit processes the YUV data to obtain target display data; the VO unit sends the target display data to the display module; and the display module controls the terminal to display the image according to the data received by the display module. The invention solves the problem that the image displayed by the terminal according to the damaged data has deviation with the original image, reduces the deviation between the image displayed by the terminal and the original image, and is used for image display.

Description

Image display method and system

Technical Field

The present invention relates to the field of electronic technologies, and in particular, to an image display method and system.

Background

With the development of electronic technology, more and more user terminals (such as electronic devices like mobile phones or tablet computers) with an image display function are provided, specifically, an operating system of the terminal may be an Android (english: android) system, the terminal may be installed with an Application (APP) with the image display function, and a user may control the APP and the image display system on the terminal to control the terminal to display an image.

Specifically, the image display system includes: the device comprises an acquisition module, a processing module and a display module. The acquisition module can acquire compressed image data according to the data address sent by the APP, decode the compressed image data to obtain YUV (color coding mode) data, and send the YUV data to the processing module. The processing module can perform superposition processing and color gamut space conversion processing on the received YUV data to obtain RGB (a color coding mode) data, and then sends the obtained RGB data to a Display (DISP) module. The display module can control the terminal to display a corresponding image according to the received RGB data, and the processing module may include at least one of an Open Graphics Library for Embedded Systems (OpenGL ES) unit and a hardware composition (HWC) unit of an Embedded system.

Because the image data acquired by the acquisition module is compressed image data, and both the YUV data obtained by decoding the image data and the RGB data sent to the display module are damaged data, the image displayed by the terminal according to the damaged data has a deviation from the original image.

Disclosure of Invention

The invention provides an image display method and system, aiming at solving the problem that an image displayed by a terminal according to damaged data is deviated from an original image. The technical scheme is as follows:

in a first aspect, an image display method is provided for an image display system on a terminal, the image display system including: the device comprises an acquisition module, a processing module and a display module, wherein the processing module comprises: a video processing output VO unit, the method comprising: the acquisition module acquires data to be displayed; the acquisition module decodes the data to be displayed to obtain YUV data; the acquiring module sends the YUV data to the VO unit; the VO unit processes the YUV data to obtain target display data; the VO unit sends the target display data to the display module; and the display module controls the terminal to display images according to the data received by the display module.

Because the processing module comprises the VO unit, the VO unit not only has a color gamut space conversion processing function, but also has a quality improvement processing function, after the YUV data obtained by decoding is sent to the VO unit by the obtaining module, the VO unit can carry out color gamut space conversion processing and quality improvement processing on the YUV data, so that the YUV data is converted into more complete target display data, namely, the VO unit also repairs damaged parts in the YUV data, thereby improving the quality of the obtained target display data, and therefore, the deviation between the image displayed by the data control terminal sent by the VO unit and the original image by the display module is reduced.

Optionally, the processing module further includes: the management unit is used for decoding the data to be displayed by the acquisition module to obtain YUV data, and comprises the following steps: the acquisition module decodes the data to be displayed to obtain decoded data; the acquisition module adds a preset identifier to the decoded data to obtain the YUV data; the acquiring module sends the YUV data to the VO unit, and the acquiring module comprises: the acquiring module sends the YUV data to the management unit; the management unit judges whether the data received by the management unit contains the preset identification or not; when the data received by the management unit contains the preset identification, the management unit determines that the data received by the management unit is the YUV data; and the management unit sends the YUV data to the VO unit.

For example, the data received by the management unit may be processed by the underlying playback unit in the acquisition module and sent to the management unit via the Graphic Buffer, or may be sent to the management unit without being processed by the underlying playback unit. After the unit receives the data, the management unit can search the received data to judge whether the data received by the management unit contains a preset identifier or not to determine whether the data received by the management unit contains YUV data, wherein the data processed by the bottom layer playing unit is YUV data, and the YUV data contains the preset identifier; the data directly sent to the management unit does not contain a preset identification. The management unit can store a preset identifier, and when the management unit manages the YUV data.

Optionally, the processing module further includes: the auxiliary processing unit is used for judging whether the data received by the management unit contains the preset identification or not before the management unit, and the method further comprises the following steps: the management unit receives first RGB data; after the management unit judges whether the data received by the management unit contains the preset identification, the method further comprises the following steps: when the data received by the management unit does not contain the preset identifier, the management unit determines that the data received by the management unit is the first RGB data; the management unit sends the first RGB data to the auxiliary processing unit; the auxiliary processing unit is used for carrying out superposition processing on the first RGB data to obtain auxiliary display data; the auxiliary processing unit sends the auxiliary display data to the display module.

Optionally, the VO unit includes: a first hardware synthesizing HWC subunit, a virtual HWC subunit, and a VO subunit, the management unit sending the YUV data to the VO unit, comprising: the management unit sending the YUV data to the first HWC subunit; the first HWC subunit sends the YUV data to the VO subunit through the virtual HWC subunit; the virtual HWC subunit obtains a transmission frequency of the YUV data when the YUV data is sent to the VO subunit by the first HWC subunit; the virtual HWC subunit sends the transmission frequency of the YUV data to the VO subunit; the VO unit processes the YUV data to obtain target display data, and the method comprises the following steps: the VO subunit performs color gamut space conversion processing and quality improvement processing on the YUV data to obtain initial display data; and the VO subunit performs FRC (frequency-fast recovery) uniform processing on the initial display data according to the transmission frequency of the YUV data and the display frequency of the display module to obtain the target display data, wherein the transmission frequency of the target display data is equal to the display frequency of the display module.

For example, if the transmission frequency of the YUV data is 30 hz, the YUV data is transmitted 30 times in one second, the display frequency of the display module is 60 hz, and the display data is displayed 60 times in one second, at this time, the transmission frequency of the initial display data is also 30 hz, the display data is transmitted 30 times in one second, and when the VO subunit performs FRC uniform processing on the initial display data, each frame of data in the initial display data may be repeatedly transmitted once to obtain the target display data, so that the transmission rate of the target display data is changed to 60 hz, that is, the target display data is transmitted 60 times in one second, so that the transmission frequency of the target display data finally input to the display module by the VO subunit is the same as the display frequency of the display module, and smoothness of image display is improved. If the transmission frequency of the YUV data is 60 hz, the YUV data is transmitted 60 times in one second, the display frequency of the display module is 30 hz, and the display data is displayed 30 times in one second, at this time, the transmission frequency of the initial display data is also 60 hz, and the display data is transmitted 60 times in one second, and when the VO subunit performs FRC uniform processing on the initial display data, every other frame of data in the initial display data may be deleted to obtain the target display data, so that the transmission rate of the target display data becomes 30 hz, that is, the target display data is transmitted 30 times in one second, so that the transmission frequency of the target display data finally input to the display module by the VO subunit is the same as the display frequency of the display module, and smoothness of image display is improved.

Optionally, the auxiliary processing unit includes a second HWC subunit, an open graphics library OpenGL ES subunit of the embedded system, a hardware abstraction layer FB HAL subunit of a frame buffer, and a driving layer FB DRV subunit of the frame buffer, and the management unit sends the first RGB data to the auxiliary processing unit, where the auxiliary processing unit includes: the management unit judges whether the terminal has a hardware synthesis processing function; the management unit selects one subunit from the second HWC subunit or the OpenGL ES subunit as an auxiliary processing subunit according to a judgment result; the management unit sends the first RGB data to the auxiliary processing subunit; the auxiliary processing unit performs superposition processing on the first RGB data to obtain auxiliary display data, and the method comprises the following steps: the auxiliary processing subunit performs superposition processing on the first RGB data to obtain the auxiliary display data; the auxiliary processing unit sending the auxiliary display data to the display module, including: the auxiliary processing subunit sends the auxiliary display data to the display module sequentially through the FB HAL subunit and the FB DRV subunit.

Because the auxiliary processing sub-module (the second HWC sub-unit or the OpenGL ES sub-unit) only processes the first RGB data, and the processing of YUV data is shared on the VO, the load of the auxiliary processing sub-module is greatly reduced, and the image display efficiency of the whole image display system is improved.

Optionally, the selecting, by the management unit according to the determination result, one subunit from the second HWC subunit or the OpenGL ES subunit as an auxiliary processing subunit includes: when it is determined that the terminal has a hardware synthesis processing function, the management unit takes the second HWC subunit as the auxiliary processing subunit; when it is determined that the terminal does not have a hardware composition processing function, the management unit takes the OpenGL ES sub-unit as the auxiliary processing sub-unit.

Optionally, the image display system further includes: the acquiring module sends the YUV data to the management unit, and the synchronizing module comprises: the acquiring module sends the YUV data to a graphic buffer area; the management unit reads the YUV data from the graphic buffer area; when the acquisition module sends YUV data to the graphics buffer, the synchronization module prohibits the management unit from reading the YUV data from the graphics buffer; and when the management unit reads YUV data from the graphic buffer area, the synchronization module prohibits the acquisition module from sending the YUV data to the graphic buffer area.

That is, due to the existence of the synchronization module, the management unit does not perform an action of reading data from the graphics buffer when the acquisition module transmits data to the graphics buffer, thereby enabling the orderly operation of the image display system.

Optionally, the obtaining module includes: native play unit and bottom play unit, the acquisition module acquires the data of treating the display, includes: the native playing unit receives a data address sent by a preset application program APP; the native playing unit sends the data address to the bottom playing unit; and the bottom layer playing unit acquires data to be displayed according to the data address.

Optionally, the controlling, by the display module, the terminal to display an image according to the data received by the display module includes: the display module performs superposition processing on the target display data and the auxiliary display data to obtain target data; and the display module controls the terminal to display images according to the target data.

In a second aspect, there is provided a data display system provided on a terminal, the data display system including: at least one module to implement: the image display method according to the first aspect or any one of the possible implementation manners of the first aspect.

The technical effect obtained by the second aspect is similar to the technical effect obtained by the corresponding technical means in the first aspect, and the description of the present invention is omitted here.

In summary, the present invention provides an image display method and system, in which the processing module includes a VO unit, and the VO unit not only has a color gamut space conversion processing function, but also has a quality improvement processing function, and after the obtaining module sends the YUV data obtained by decoding to the VO unit, the VO unit can perform color gamut space conversion processing and quality improvement processing on the YUV data, so that the YUV data is converted into more complete target display data, that is, the VO unit also repairs damaged portions in the YUV data, thereby improving the quality of the obtained target display data, and thus reducing a deviation between an image displayed by the display module according to the data control terminal sent by the VO unit and an original image.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic structural diagram of an image display system provided in the related art;

fig. 2 is a schematic structural diagram of an image display system according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of another image display system according to an embodiment of the present invention;

FIG. 4 is a flowchart of a method of displaying an image according to an embodiment of the present invention;

FIG. 5 is a flowchart of another method for displaying an image according to an embodiment of the present invention;

fig. 6 is a flowchart of a method for acquiring data to be displayed according to an embodiment of the present invention;

fig. 7 is a flowchart of a method for processing data to be displayed according to an embodiment of the present invention;

fig. 8 is a schematic view of a display interface of a terminal according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Fig. 1 is a schematic structural diagram of an image display system 0 provided in the related art, and as shown in fig. 1, the image display system 0 may include: the device comprises an acquisition module 01, a processing module 02 and a display module 03.

Specifically, the obtaining module 01 may include: the Media Player is in communication connection with a preset APP with an image display function, and when a user needs to control a terminal to display an image, the user can operate on the preset APP to trigger the preset APP to send a data address, such as an address of data on the terminal, or an Uniform Resource Locator (URL) address of the data to the Media Player. The Media Player can send the data address received by the Media Player management unit to the Player, and the Player obtains the corresponding data to be displayed according to the data address, where it should be noted that the obtained data to be displayed are compressed data. The Player can decode the compressed data, that is, the Player can firstly analyze the information (for example, information such as time length, decoding format, resolution, etc.) of the code stream of the compressed data and report the information to the MediaPlayer, and then the Player configures a corresponding decoder according to the decoding format and resolution to decode the compressed data, and decodes the compressed data to obtain displayable YUV data (for example, YUV420 data). Furthermore, when the Player decodes the YUV data, the Player can also write the obtained YUV data into a graphics Buffer (english: graphics Buffer) which is in communication connection with the Player.

For example, the processing module may include a management unit (also referred to as a Surface flag unit), an Open Graphics Library for Embedded Systems (OpenGL ES) unit, a hardware composition (HWC) unit, a frame buffer hardware abstraction layer (FB HAL) unit, and a frame buffer driver (FB DRV) unit. After the Player writes the YUV data in the Graphic Buffer, the Graphic Buffer automatically sends the YUV data to the Surface flunger unit in the processing module. The Surface flanger can inquire whether the terminal has a hardware processing function or not according to the received YUV data, if the terminal has the hardware processing function, the Surface flanger sends the YUV data to the HWC unit, and if the terminal does not have the hardware processing function, the Surface flanger sends the YUV data to the OpenGL ES unit. After receiving the YUV data, the HWC unit or the OpenGL ES unit may perform color gamut space conversion processing on the YUV data to obtain RGB data, and send the obtained RGB data to the FB unit.

The FB DRV unit can be in communication connection with the display module, and the FB HAL unit can send the received RGB data to the display module through the FB DRV unit, so that the display module can control the terminal to display corresponding images according to the data received by the display module. In the related art, the data acquired by the acquisition module is compressed data, and both the YUV data obtained by decoding the data and the RGB data sent to the display module are damaged data, so that an image displayed by the terminal according to the damaged data has a deviation from an original image.

Fig. 2 is a schematic structural diagram of an image display system 1 according to an embodiment of the present invention, and for example, the image display system 1 is disposed on a terminal (not shown in fig. 2), as shown in fig. 2, the image display system 1 may include: an obtaining module 11, a processing module 12 and a display module 13, wherein the processing module 12 may include: and a video processing output (VO) unit 121.

The obtaining module 11 is configured to obtain data to be displayed;

the obtaining module 11 is further configured to decode data to be displayed to obtain YUV data;

the obtaining module 11 is further configured to send the YUV data to the VO unit 121;

the VO unit 121 is configured to process the YUV data to obtain target display data;

the VO unit 121 is further configured to send target display data to the display module 13;

the display module 13 is configured to control the terminal to display an image according to the data received by the display module 13.

In summary, in the image display system provided in the embodiment of the present invention, the processing module includes the VO unit, and the VO unit not only has a color gamut space conversion processing function, but also has a quality improvement processing function, and after the obtaining module sends the YUV data obtained by decoding to the VO unit, the VO unit can perform color gamut space conversion processing and quality improvement processing on the YUV data, so that the YUV data is converted into more complete target display data, that is, the VO unit also repairs a damaged portion in the YUV data, thereby improving the quality of the obtained target display data, and therefore, the deviation between the image displayed by the terminal and the original image is controlled by the display module according to the data sent by the VO unit is reduced.

Optionally, fig. 3 is a schematic structural diagram of another image display system 1 according to an embodiment of the present invention, as shown in fig. 3, on the basis of fig. 2, the processing module 12 may further include: a management unit 122.

The obtaining module 11 may further be configured to: decoding data to be displayed to obtain decoded data, adding a preset identifier to the decoded data to obtain YUV data, and sending the YUV data to the management unit 122; the management unit 122 may be configured to determine whether the data received by the management unit includes a preset identifier, determine, when the data received by the management unit includes the preset identifier, that the data received by the management unit 122 is YUV data, and send the YUV data to the VO unit 121.

Optionally, the processing module 12 may further include: an auxiliary processing unit 123. The management unit 122 is further configured to receive the first RGB data; the management unit 122 is further configured to determine that the data received by the management unit is the first RGB data when the data received by the management unit does not include the preset identifier; the management unit 122 is further configured to send the first RGB data to the auxiliary processing unit 123; the auxiliary processing unit 123 is configured to perform superposition processing on the first RGB data to obtain auxiliary display data; the auxiliary processing unit 123 is also configured to send auxiliary display data to the display module 13.

Specifically, VO unit 121 may include: a first HWC subunit 1211, a virtual HWC subunit 1212 and a VO subunit 1213, the management unit 122 further being configured to send YUV data to the first HWC subunit 1211; the first HWC subunit 1211 is configured to send the YUV data to the VO subunit 1213 through the virtual HWC subunit 1212; the virtual HWC subunit 1212 is configured to obtain a transmission frequency of the YUV data when the first HWC subunit 1211 transmits the YUV data to the VO subunit 1213 through the virtual HWC subunit 1212; the virtual HWC subunit 1212 is further configured to send the transmission frequency of YUV data to the VO subunit 1213; the VO subunit 1213 is configured to perform color gamut space conversion processing and quality improvement processing on the YUV data to obtain initial display data; the VO subunit 1213 is further configured to perform Frame Rate Conversion (FRC) on the initial display data according to the transmission frequency of the YUV data and the display frequency of the display module to obtain target display data, where the transmission frequency of the target display data is equal to the display frequency of the display module.

Optionally, the auxiliary processing unit 123 may include: a second HWC subunit 1231, an OpenGL ES subunit 1232, a FB HAL subunit 1233, and a FB DRV subunit 1234. The management unit 122 is further configured to determine whether the terminal has a hardware synthesis processing function; the management unit 122 is further configured to select one sub-unit from the second HWC sub-unit 1231 or the OpenGL ES sub-unit 1232 as an auxiliary processing sub-unit according to the determination result; the management unit 122 is further configured to send the first RGB data to the auxiliary processing subunit; the auxiliary processing subunit is further configured to perform superposition processing on the first RGB data to obtain auxiliary display data; the auxiliary processing subunit is further configured to send auxiliary display data to the display module via the FB HAL subunit 1233 and the FB DRV subunit 1234, respectively.

The management unit 122 is further configured to take the second HWC subunit 1231 as an auxiliary processing subunit when it is determined that the terminal has the hardware synthesis processing function; the management unit 122 is also configured to take the OpenGL ES subunit 1232 as an auxiliary processing subunit when it is determined that the terminal does not have a hardware synthesis processing function.

Alternatively, the image display system 1 may further include: a synchronization module (not shown in fig. 3), the obtaining module 11 is further configured to send the YUV data to a graphics buffer; the management unit 122 is further configured to read YUV data from the graphics buffer; the synchronization module is configured to prohibit the management unit 122 from reading the YUV data from the graphics buffer when the obtaining module 11 sends the YUV data to the graphics buffer; the synchronization module is further configured to prohibit the obtaining module 11 from sending the YUV data to the graphics buffer when the management unit 122 reads the YUV data from the graphics buffer.

Optionally, the obtaining module 11 includes: the system comprises a native playing unit 111 and a bottom layer playing unit 112, wherein the native playing unit 111 is used for receiving a data address sent by a preset application program APP; the native play unit 111 is configured to send the data address to the underlying play unit 112; the bottom layer playing unit 112 is configured to obtain data to be displayed according to the data address.

Optionally, the display module 13 is further configured to perform superposition processing on the target display data and the auxiliary display data to obtain target data; the display module 13 is further configured to control the terminal to display an image according to the target data.

Optionally, the target display data, the initial display data, and the auxiliary display data in the embodiment of the present invention may be data in RGB format at the same time, or may also be data in YUV format at the same time, which is not limited in the embodiment of the present invention.

The image display system provided by the embodiment of the present invention may be applied to the image display method described below, and the workflow and the working principle of each module or unit in the embodiment of the present invention may be referred to the description in the method embodiment below.

As shown in fig. 4, an embodiment of the present invention provides an image display method, which may be used in the image display system 1 shown in fig. 2 or fig. 3, where the image display system 1 may include: obtain module, processing module and display module, processing module includes: VO unit, the image display method may include:

step 401, the obtaining module obtains data to be displayed.

And 402, decoding the data to be displayed by an acquisition module to obtain YUV data.

And step 403, the acquiring module sends the YUV data to the VO unit.

And step 404, the VO unit processes the YUV data to obtain target display data.

Step 405, the VO unit sends the target display data to the display module.

And step 406, the display module controls the terminal to display the image according to the data received by the display module.

In summary, in the image display method provided in the embodiment of the present invention, after the obtaining module sends the decoded YUV data to the VO unit, the VO unit can perform color gamut space conversion processing and quality improvement processing on the YUV data, so that the YUV data is converted into more complete target display data, that is, the VO unit also repairs a damaged portion of the YUV data, thereby improving the quality of the obtained target display data, and therefore, the deviation between the image displayed by the terminal and the original image is controlled by the display module according to the data sent by the VO unit.

As shown in fig. 5, another image display method provided by the embodiment of the present invention may be used in the image display system 1 shown in fig. 3, where the image display system 1 may include: obtain module, processing module and display module, processing module includes: VO unit, the image display method may include:

step 501, the obtaining module obtains data to be displayed. Step 502 is performed.

Specifically, the obtaining module may include a native playing unit and a bottom playing unit, as shown in fig. 6, step 501 may include:

in step 5011, the native play unit receives a data address sent by the preset APP.

Optionally, the preset APP may be an APP with a video playing function, and the native playing unit may establish a communication connection with the preset AAP. For example, a communication interface connected to a preset APP may be provided on the native playing unit, and when a user needs to watch a certain video on the terminal, the user may trigger the preset APP to send a data address to the native playing unit by operating in the preset APP, where, for example, the data address is an address where compressed data is located, the compressed data may be stored in the terminal, and at this time, the data address may be a storage address of the compressed data in the terminal; optionally, the compressed data may also be located on the internet, and in this case, the data address may be a URL of the compressed data.

Optionally, the native playback unit in step 5011 may also be replaced by an original media decoding unit (english: mediacodec), or other playback units having the same function as the native playback unit. The embodiment of the present invention is not limited thereto.

In step 5012, the native playback unit sends the data address to the bottom playback unit.

The native play unit can be connected with the bottom layer play unit, and after the native play unit receives the data address sent by the preset APP, the native play unit can send the received data address to the bottom layer play unit.

In step 5013, the bottom layer playing unit obtains the data to be displayed according to the data address.

After receiving the data address, the bottom layer playing unit may obtain the data to be displayed stored at the data address according to the received data address.

Step 502, the obtaining module processes data to be displayed to obtain YUV data containing a preset identifier. Step 503 is performed.

As shown in fig. 7, step 502 may include:

step 5021, the bottom layer playing unit decodes the data to be displayed to obtain decoded data.

For example, after the bottom layer playing unit acquires the data to be displayed, the bottom layer playing unit may further analyze the data to be displayed to obtain code stream information of the data to be displayed, such as information of a duration, a decoding format, a resolution, and the like of the data to be displayed. Then, the bottom layer playing unit can also configure a corresponding decoder according to the decoded format and resolution obtained by parsing, and decode the data to be displayed to obtain displayable YUV data. For example, in the YUV color coding mode, the color of a pixel point may be labeled as (Y, U, V), where "Y" represents brightness, "U" is chroma, and "V" is density.

Step 5022, the bottom layer playing unit adds a preset identifier to the decoded data to obtain YUV data.

Specifically, the bottom layer playing unit may store a preset identifier, and after the decoded data is obtained, the bottom layer playing unit may add the preset identifier to the decoded data to obtain the YUV data.

Step 503, the obtaining module sends the YUV data to the management unit. Step 504 is performed.

For example, after obtaining the YUV data, the bottom layer playing unit in the obtaining module may send the YUV data to the Graphic Buffer, and optionally, the Graphic Buffer may also cache the YUV data. When the YUV data in the Graphic Buffer is more, the Graphic Buffer can send the YUV data to the management unit, namely the management unit reads the YUV data from the Graphic Buffer; or the management unit can periodically and actively read the YUV data from the Graphic Buffer, so that the bottom layer playing unit sends the YUV data to the management unit.

It should be noted that the image display system may further include: the synchronization module (also called an enter module) prohibits the management unit from reading the YUV data from the graphics buffer area when the acquisition module sends the YUV data to the graphics buffer area; and when the management unit reads the YUV data from the graphic buffer area, the synchronization module prohibits the acquisition module from sending the YUV data to the graphic buffer area. That is, due to the existence of the synchronization module, the management unit does not perform the action of reading data from the graphics buffer when the acquisition module transmits data to the graphics buffer, thereby enabling the image display system to operate orderly.

Step 504, the management unit receives the first RGB data. Step 505 is performed.

It should be noted that the management unit may also directly establish a communication connection with a part of modules on the terminal, and any one of the part of modules may send the first RGB data to the management unit as needed. For example, the default APP may be directly connected to the management unit, and the default APP may directly send the first RGB data to the management unit. For example, in the RGB color coding method, the color of a pixel point may be labeled as (R, G, B), where three letters represent Red (english: red), green (english: green), and Blue (english: blue), respectively.

Step 505, the management unit determines whether the received data contains a preset identifier. If the received data contains the preset identifier, go to step 506; if the received data does not include the predetermined identifier, go to step 510.

For example, the data received by the management unit may be processed by the underlying playback unit in the acquisition module and sent to the management unit via the Graphic Buffer, or may be sent to the management unit without being processed by the underlying playback unit. The data processed by the bottom layer playing unit is YUV data, and the YUV data comprises a preset identifier; the data directly sent to the management unit is first RGB data, and the first RGB data does not contain a preset identifier. The management unit may store a preset identifier, and when the management unit receives the data, the management unit may search the received data to determine whether the data received by the management unit includes the preset identifier.

It should be noted that the management unit in the embodiment of the present invention may be a Surface flag having a function of determining whether data has a preset identifier. That is, the Surface Flinger in the related art does not store the preset identifier, and the Surface Flinger does not have the function of judging whether the data has the preset identifier.

In step 506, the management unit determines that the received data is YUV data. Step 507 is performed.

If the data received by the management unit contains the preset identifier, it can be indicated that the data received by the management unit is YUV data processed by the bottom-layer playing unit.

And step 507, the management unit sends the YUV data to the VO unit. Step 508 is performed.

For example, the VO unit may include: the management unit can directly send YUV data to the first HWC subunit, and the first HWC subunit can send the YUV data to the VO subunit through the virtual HWC subunit after receiving the YUV data, that is, the first HWC unit can send the YUV data to the virtual HWC unit, and the YUV data is forwarded to the VO subunit by the virtual HWC unit. Specifically, when the first HWC subunit sends the YUV data to the VO subunit through the virtual HWC subunit, that is, when the virtual HWC forwards the YUV data, the virtual HWC subunit may obtain the transmission frequency of the YUV data, and send the obtained transmission frequency of the YUV data to the VO subunit.

And step 508, the VO unit processes the YUV data to obtain target display data. Step 509 is performed.

After receiving YUV data, a VO subunit in the VO unit can perform color gamut space conversion processing and quality improvement processing on the YUV data to obtain initial display data; then, the VO subunit may perform FRC uniform processing on the initial display data according to the transmission frequency of the YUV data and the display frequency of the display module to obtain target display data, where the transmission frequency of the target display data is equal to the display frequency of the display module, and it should be noted that the VO subunit may store the display frequency of the display module, or before step 508, the VO subunit may further obtain the display frequency of the display module from the display module.

For example, if the transmission frequency of the YUV data is 30 hz, the YUV data is transmitted 30 times in one second, the display frequency of the display module is 60 hz, and the display data is displayed 60 times in one second, at this time, the transmission frequency of the initial display data is also 30 hz, and the display data is transmitted 30 times in one second, when the VO subunit performs FRC uniform processing on the initial display data, each frame of data in the initial display data may be repeatedly transmitted once to obtain the target display data, so that the transmission rate of the target display data is changed to 60 hz, that is, the transmission frequency is transmitted 60 times in one second, so that the number of times of repeated transmission of each frame of data in the initial display data is the same, and the transmission frequency of the target display data finally input to the display module by the VO subunit is the same as the display frequency of the display module, thereby improving smoothness of image display. If the transmission frequency of the YUV data is 60 hz, the YUV data is transmitted 60 times in one second, the display frequency of the display module is 30 hz, and the display data is displayed 30 times in one second, at this time, the transmission frequency of the initial display data is also 60 hz, and the display data is transmitted 60 times in one second, and when the VO subunit performs FRC uniform processing on the initial display data, every other frame of data in the initial display data may be deleted to obtain the target display data, so that the transmission rate of the target display data becomes 30 hz, that is, the target display data is transmitted 30 times in one second, so that the transmission frequency of the target display data finally input to the display module by the VO subunit is the same as the display frequency of the display module, and smoothness of image display is improved.

In addition, the VO subunit in the embodiment of the present invention may have a function of performing color gamut space conversion on YUV data and performing quality improvement processing on the YUV data, and after performing quality improvement processing on the YUV data, damage to the YUV data when being compressed and decoded is repaired, so that target display data is relatively complete.

In step 509, the VO unit sends the target display data to the display module. Step 514 is performed.

The VO subunit in the VO unit can be in communication connection with the display module, and after obtaining the target display data, the VO subunit in the VO unit can directly send the obtained target display data to the display module.

Step 510, the management unit determines that the received data is the first RGB data. Step 511 is performed.

If the data received by the management unit does not include the preset identifier, it may be indicated that the data received by the management unit is the first RGB data that has not been processed by the bottom layer playing unit.

Step 511, the management unit sends the first RGB data to the auxiliary processing unit. Step 512 is performed.

For example, the auxiliary processing unit may include a second HWC subunit, an OpenGL ES subunit, an FB HAL subunit, and an FB DRV subunit, and in step 511, the management unit may first determine whether the terminal has a hardware synthesis processing function, and then the management unit may select one of the second HWC subunit or the OpenGL ES subunit as the auxiliary processing subunit according to the determination result, and send the first RGB data to the auxiliary processing subunit. Specifically, when it is determined that the terminal has the hardware synthesis processing function, the management unit may use the second HWC subunit as an auxiliary processing subunit; the management unit may take the OpenGL ES subunit as an auxiliary processing subunit when it is determined that the terminal does not have the hardware composition processing function.

And step 512, the auxiliary processing unit performs superposition processing on the first RGB data to obtain auxiliary display data. Step 513 is performed.

In step 511, after the first RGB data is sent to the corresponding auxiliary processing subunit, the corresponding auxiliary processing subunit (the second HWC subunit or the OpenGL ES subunit) may perform superposition processing on the first RGB data to obtain auxiliary display data, and a specific superposition processing process may refer to related technologies, which is not described herein in detail in this embodiment of the present invention.

Step 513, the auxiliary processing unit sends the auxiliary display data to the display module. Step 514 is performed.

After the auxiliary processing subunit obtains the auxiliary display data, the auxiliary processing subunit may sequentially send to the display module via the FB HAL subunit and the FB DRV subunit.

And 514, controlling the terminal to display the image by the display module according to the received data.

When the display module receives the target display data and the auxiliary display data at the same time, the display module may convert the received data into Composite Video Broadcast Signal (CVBS) data, high Definition Multimedia Interface (HIDM) data, or Low-Voltage Differential Signaling (LVDS) data. And performing superposition processing on the target display data and the auxiliary display data to obtain target data, as shown in fig. 8, the display module may also control the terminal to simultaneously display an image a corresponding to the target display data and an image B corresponding to the auxiliary display data according to the target data. Optionally, the target display data, the initial display data, and the auxiliary display data in the embodiment of the present invention may be data in RGB format at the same time, or may also be data in YUV format at the same time, which is not limited in the embodiment of the present invention.

Because the auxiliary processing sub-module (the second HWC sub-unit or the OpenGL ES sub-unit) in the embodiment of the present invention processes only the first RGB data, and the processing of the YUV data is shared on the VO, the load of the auxiliary processing sub-module is greatly reduced, and the efficiency of displaying images by the whole image display system is improved.

Further, when other modules on the terminal do not continuously send the first RGB data to the management unit, the Open GL subunit, the second HWC subunit, the FB HAL subunit, and the FB DRV subunit, which are used for processing the first RGB data, may all stop working, and at this time, the power consumption of the terminal may be greatly reduced.

It should be noted that, the order of the steps of the image display method provided in the embodiment of the present invention may be appropriately adjusted, and the steps may also be increased or decreased according to the circumstances, and any method that can be easily conceived by those skilled in the art within the technical scope of the present invention should be included in the protection scope of the present invention, and therefore, the detailed description is omitted.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and should not be taken as limiting the scope of the present invention, which is intended to cover any modifications, equivalents, improvements, etc. within the spirit and scope of the present invention.

Claims

1. An image display apparatus, characterized in that the apparatus comprises: the device comprises an acquisition module and a processing module, wherein the processing module comprises: the video processing system comprises a management unit, a video processing output VO unit and an auxiliary processing unit, wherein an acquisition module comprises a native Media playing unit Media Player and a bottom layer playing unit Player, the acquisition module is in communication connection with a preset application program APP, and the preset application program APP has an image display function;

responding to the operation of a user on the preset application program APP, and acquiring to-be-displayed data by the acquisition module; the acquisition module is also used for decoding the data to be displayed to obtain YUV data;

the management unit is used for sending the YUV data to the video processing output VO unit;

the management unit is also used for sending the RGB data to the auxiliary processing unit;

the video processing output VO unit is used for processing the YUV data to obtain target display data, and the processing comprises quality improvement processing;

the auxiliary processing unit is used for performing superposition processing on the RGB data to obtain auxiliary display data;

the auxiliary processing unit includes: the hardware synthesizes the HWC subunit and the OpenGL ES subunit, and the management unit is specifically used for:

and sending the RGB data to one of the hardware synthesis HWC subunit or the OpenGL ES subunit, so that the hardware synthesis HWC subunit or the OpenGL ES subunit performs superposition processing on the RGB data to obtain the auxiliary display data.

2. The image display device according to claim 1, further comprising: and the display module is used for controlling the terminal to display images according to the target display data and the auxiliary display data.

3. The image display device according to claim 2, wherein the display module is specifically configured to:

superposing the target display data and the auxiliary display data to obtain target data;

and controlling the terminal to display the image according to the target data.

4. The image display device according to any one of claims 1 to 3, wherein the management unit is specifically configured to:

when the received data is determined to be YUV data containing a preset identifier, sending the YUV data to the video processing output VO unit;

and when the received data is determined to be the RGB data not containing the preset identification, sending the RGB data not containing the preset identification to the auxiliary processing unit.

5. The image display device according to claim 4, wherein the obtaining module is specifically configured to: and decoding the data to be displayed and adding the preset identification to obtain the YUV data.

6. The image display device according to any one of claims 1 to 3, wherein the video processing output VO unit includes: a first hardware synthesis HWC subunit, a virtual hardware synthesis HWC subunit, and a video processing output VO subunit,

the management unit is specifically configured to send the YUV data to the first hardware synthesis HWC subunit;

the first hardware synthesis HWC subunit is used for sending the YUV data to the video processing output VO subunit through the virtual hardware synthesis HWC subunit;

and the VO subunit is used for processing the YUV data to obtain the target display data.

7. The image display device according to claim 6, wherein the processing further comprises: frame Rate Conversion (FRC) uniform processing;

the virtual hardware synthesis HWC subunit is also used for acquiring the transmission frequency of the YUV data when the first hardware synthesis HWC subunit sends the YUV data to the VO subunit through the virtual hardware synthesis HWC subunit;

the video processing output VO subunit is specifically configured to:

performing the quality improvement processing on the YUV data to obtain initial display data;

and performing Frame Rate Conversion (FRC) uniform processing on the initial display data according to the transmission frequency of the YUV data and the display frequency of a display module to obtain the target display data, wherein the transmission frequency of the target display data is equal to the display frequency of the display module.

8. The image display device according to any one of claims 1 to 3, characterized in that the image display device further comprises: the synchronization module is used for synchronizing the data of the data transmission system,

the acquiring module is also used for sending the YUV data to a graphic buffer area;

the management unit is also used for reading the YUV data from the graphic buffer area;

the synchronization module is used for forbidding the management unit to read YUV data from the graphic buffer when the acquisition module sends the YUV data to the graphic buffer;

the synchronization module is further configured to prohibit the obtaining module from sending YUV data to the graphics buffer when the management unit reads the YUV data from the graphics buffer.

9. An image display method, characterized in that the method comprises:

the method comprises the steps that in response to the operation of a user on a preset application program APP, an acquisition module acquires data to be displayed; the acquisition module comprises a native Media playing unit Media Player and a bottom layer playing unit Player, the acquisition module is in communication connection with the preset application program APP, and the preset application program APP has an image display function;

decoding the data to be displayed to obtain YUV data;

sending the YUV data to a video processing output VO unit for processing to obtain target display data, wherein the processing comprises quality improvement processing;

sending the RGB data to an auxiliary processing unit for superposition processing to obtain auxiliary display data; the auxiliary processing unit includes: the hardware synthesis HWC subunit and the OpenGL ES subunit, where the RGB data is sent to the auxiliary processing unit for superposition processing to obtain auxiliary display data, specifically including:

10. The method of claim 9, further comprising:

and controlling a terminal to display an image according to the target display data and the auxiliary display data.

11. The method according to claim 10, wherein the controlling a terminal to display an image according to the target display data and the auxiliary display data specifically comprises:

and controlling the terminal to display the image according to the target data.

12. The method according to any one of claims 9 to 11, wherein the sending the YUV data to a video processing output VO unit for processing comprises:

when the received data is determined to be YUV data containing a preset identifier, sending the YUV data to the video processing output VO unit for processing;

sending the RGB data to the auxiliary processing unit for superposition processing specifically includes:

and when the received data is determined to be the RGB data not containing the preset identification, sending the RGB data not containing the preset identification to the auxiliary processing unit for carrying out the superposition processing.

13. The method according to claim 12, wherein the decoding the data to be displayed to obtain YUV data comprises:

and decoding the data to be displayed and adding the preset identification to obtain the YUV data.

14. The method of any of claims 9 to 11, wherein the video processing outputting the VO unit comprises: the virtual hardware synthesis HWC subunit and a video processing output VO subunit, wherein the YUV data is sent to the video processing output VO subunit to be processed to obtain target display data, and the method comprises the following steps:

sending the YUV data to the VO subunit through a virtual hardware synthesis (HWC) subunit;

and the VO subunit for video processing output performs the processing on the YUV data to obtain the target display data.

15. The method of claim 14, wherein the processing further comprises: frame Rate Conversion (FRC) uniform processing;

the method further comprises the following steps:

when the YUV data is sent to the VO subunit for video processing output through the virtual hardware synthesis HWC subunit, obtaining the transmission frequency of the YUV data;

the video processing output VO subunit performs the processing on the YUV data to obtain the target display data, and specifically includes:

the VO subunit for video processing output performs the quality improvement processing on the YUV data to obtain initial display data;

and the VO subunit performs Frame Rate Conversion (FRC) uniform processing on the initial display data according to the transmission frequency of the YUV data and the display frequency of a display module to obtain target display data, wherein the transmission frequency of the target display data is equal to the display frequency of the display module.

16. The method according to any one of claims 9 to 11, wherein after said decoding the data to be displayed to obtain YUV data, the method further comprises:

sending the YUV data to a graphic buffer area;

when YUV data are sent to the graphic buffer area, the YUV data are forbidden to be read from the graphic buffer area;

and when the YUV data is read from the graphics buffer area, the YUV data is forbidden to be sent to the graphics buffer area.

17. An image display apparatus, characterized in that the apparatus comprises: the device comprises an acquisition module, a processing module and a display module, wherein the processing module comprises: the video processing system comprises a management unit and a video processing output VO unit, wherein an acquisition module comprises a native Media playing unit Media Player and a bottom layer playing unit Player, the acquisition module is in communication connection with a preset application program APP, and the preset application program APP has an image display function;

responding to the operation of a user on the preset application program APP, and acquiring to-be-displayed data by the acquisition module;

the acquisition module is also used for decoding the data to be displayed to obtain YUV data;

the acquiring module is also used for sending the YUV data to the management unit;

the VO unit is used for processing the YUV data to obtain target display data, and the processing comprises quality improvement processing;

the display module is used for controlling a terminal to display images according to the data received by the display module, and the data received by the display module comprises the target display data; wherein the video processing output VO unit includes: a first hardware synthesis HWC subunit, a virtual hardware synthesis HWC subunit, and a video processing output VO subunit,

the first hardware synthesis HWC subunit is used for sending the YUV data to the VO subunit through the virtual hardware synthesis HWC subunit;

18. The image display device according to claim 17, wherein the processing module further comprises: an auxiliary processing unit for processing the image data,

the management unit is further configured to:

sending RGB data to the auxiliary processing unit;

the auxiliary processing unit is used for performing superposition processing on the RGB data to obtain auxiliary display data; the data received by the display module further comprises the auxiliary display data.

19. The image display device according to claim 18, wherein the display module is specifically configured to:

and controlling the terminal to display an image according to the target data.

20. The image display device according to claim 17, wherein the processing further comprises: frame Rate Conversion (FRC) uniform processing;

the virtual hardware synthesis HWC subunit is further configured to obtain a transmission frequency of the YUV data when the YUV data is sent to the VO subunit by the virtual hardware synthesis HWC subunit by the first hardware synthesis HWC subunit;

the video processing output VO subunit is specifically configured to:

and performing Frame Rate Conversion (FRC) uniform processing on the initial display data according to the transmission frequency of the YUV data and the display frequency of the display module to obtain the target display data, wherein the transmission frequency of the target display data is equal to the display frequency of the display module.

21. The image display device according to any one of claims 17 to 20, further comprising: the synchronization module is used for synchronizing the data of the data transmission system,

the management unit is also used for reading the YUV data from the graphics buffer;

the synchronization module is used for forbidding the management unit to read YUV data from the graphic buffer area when the acquisition module sends the YUV data to the graphic buffer area;

the synchronization module is further used for forbidding the acquisition module to send the YUV data to the graphics buffer when the management unit reads the YUV data from the graphics buffer.

22. An image display method, characterized in that the method comprises:

decoding the data to be displayed to obtain YUV data;

sending the YUV data to a management unit;

the management unit sends the YUV data to a video processing output VO unit for processing to obtain target display data, wherein the processing comprises quality improvement processing;

controlling a terminal to display an image according to data received by a display module, wherein the data received by the display module comprises the target display data; wherein the video processing output VO unit includes: the virtual hardware synthesis HWC subunit and the video processing output VO subunit, wherein the step of sending the YUV data to the video processing output VO subunit for processing to obtain target display data comprises the following steps:

23. The method of claim 22, further comprising:

sending the RGB data to an auxiliary processing unit;

the auxiliary processing unit performs superposition processing on the RGB data to obtain auxiliary display data, and the data received by the display module further comprises the auxiliary display data.

24. The method according to claim 23, wherein the controlling the terminal to display an image according to the data received by the display module specifically comprises:

and controlling the terminal to display an image according to the target data.

25. The method of claim 22, wherein the processing further comprises: frame Rate Conversion (FRC) uniform processing;

the method further comprises the following steps:

when the YUV data is sent to the VO subunit through the virtual hardware synthesis HWC subunit, acquiring the transmission frequency of the YUV data; the video processing output VO subunit performs the processing on the YUV data to obtain the target display data, and specifically includes:

the VO subunit performs the quality improvement processing on the YUV data to obtain initial display data;

and the VO subunit performs Frame Rate Conversion (FRC) uniform processing on the initial display data according to the transmission frequency of the YUV data and the display frequency of the display module to obtain target display data, wherein the transmission frequency of the target display data is equal to the display frequency of the display module.

26. The method according to any one of claims 22 to 25, wherein after said decoding the data to be displayed to obtain YUV data, the method further comprises:

sending the YUV data to a graphic buffer area;

when YUV data is sent to the graphics buffer area, the YUV data is prohibited from being read from the graphics buffer area;