WO2018176734A1

WO2018176734A1 - Data processing method and terminal

Info

Publication number: WO2018176734A1
Application number: PCT/CN2017/100080
Authority: WO
Inventors: 仇建斌
Original assignee: 华为技术有限公司
Priority date: 2017-03-27
Filing date: 2017-08-31
Publication date: 2018-10-04
Also published as: CN109196865B; CN109196865A

Abstract

Provided are a data processing method and a terminal, relating to the technical field of terminals and able to save the electric quantity of terminals. The specific solution involves: a terminal comprises a central processing unit (CPU), an image processing unit and a display unit, and when a first format corresponding to decoded video frame data is not a pre-set type format, the CPU converts the first format into a second format, wherein the video frame data is used for representing a video layer picture, the second format is the pre-set type format, and the pre-set type format is a format that can be recognized by the image processing unit; the image processing unit converts the second format into a target format, wherein the target format is a data frame format for display by the display unit; and the image processing unit zooms a video layer picture represented by video frame data of the target format, so that the size of the zoomed video layer picture matches the size of the display unit. The embodiment of the present application is used for playing video.

Description

Data processing method and terminal

The present application claims priority to Chinese Patent Application No. PCT Application No. No. No. No. No. No. No. No. No. No. No. No. No. in.

Technical field

The present application relates to the field of terminal technologies, and in particular, to a data processing method and a terminal.

Background technique

In mobile Android, most video playback applications support common video formats such as MP4, 3GP, RMVB and AVI. For these common video formats, the Android system software and hardware are supported by default. Therefore, the system decoder of the Android system can decode the video source data of the general format, and then send the decoded video frame data to the image processing unit such as MDP for scaling. And then sent to the display unit for display.

However, due to different video sources, different supported platforms, and historical issues, some video playback applications may encode video source data using other video formats than the common video format. Correspondingly, for video source data of these non-universal video formats, the video playing application decodes through a decoder other than the system decoder, and the format of the decoded video frame data is usually not matched with the mobile phone hardware, and thus cannot be directly sent. The image processing unit of the mobile phone and the display unit perform processing. Referring to FIG. 1 , a solution adopted by the prior art is that a graphics processing unit (GPU) converts the format of the video frame data into a target format that can be directly used for display, and displays a video layer image of the video frame data. After being scaled to the size matching the display unit, it is sent to the image processing unit and the display unit for processing.

Wherein, when the video is played, the format of the video frame data is converted into a target format that can be directly used for display by the GPU, and the video layer picture is scaled to a size matching the display unit, which consumes more power of the mobile phone. Reduce the user experience.

Summary of the invention

The embodiment of the present invention provides a data processing method and a terminal, which can reduce power consumption during video playback and improve user experience.

To achieve the above objective, the embodiment of the present application adopts the following technical solutions:

In a first aspect, the embodiment of the present application provides a data processing method, which is applied to a terminal, where the terminal includes a central processing unit (CPU), an image processing unit, and a display unit, and the method includes: when the decoded video When the first format corresponding to the frame data is not the preset type format, the CPU converts the first format into the second format. The video frame data is used to represent the video layer picture, the second format is a preset type format, and the preset class format is a format that the image processing unit can recognize. The image processing unit converts the second format into a target format, which is a data frame format used by the display unit for display. The image processing unit scales the video layer picture represented by the video frame data of the target format such that the size of the scaled video layer picture matches the size of the display unit Match.

In this way, a smaller, simpler image processing unit can be used instead of the GPU to perform a part of the format conversion operation and the zoom operation, thereby consuming less power and improving the user experience.

In conjunction with the first aspect, in a possible implementation manner, the method further includes: the display unit displaying the scaled video layer picture.

That is, the display unit can display the video layer screen after the image processing unit is scaled.

With reference to the first aspect and the foregoing possible implementation manner, in another possible implementation manner, the first format is corresponding to the first video playing application, and the video frame data of the first format is a decoder corresponding to the first video playing application. Data obtained by decoding video source data.

That is to say, the first format processed by the CPU is the format of the video frame data obtained by decoding the decoder corresponding to the first video playing application.

In combination with the first aspect and the foregoing possible implementation manner, in another possible implementation manner, the CPU converting the first format into the second format includes: the CPU, according to the first video playback application corresponding to the first format, the first The format is converted to the second format.

Since the first format is the format of the video frame data obtained by decoding by the decoder corresponding to the first video playing application, the CPU may convert the first format into the second format according to the first video playing application.

With reference to the first aspect and the foregoing possible implementation manner, in another possible implementation manner, the terminal saves a correspondence between the first video playing application and the third format, where the third format is a preset type format, and the second format is A third format corresponding to the first video playback application.

In this way, the CPU may convert the first format corresponding to the first video playing application into the third format according to the correspondence between the first video playback saved in advance and the third format.

In combination with the first aspect and the foregoing possible implementation manners, in another possible implementation manner, the video layer is displayed through a surface view surfaceview, and before the CPU converts the first format into the second format, the method further includes: the CPU is in the video. A new video layer is created on the upper layer of the corresponding position of the original video layer corresponding to the frame data, the new video layer is used to cover the original video layer, and the new video layer corresponds to the new surface view new surface view. The CPU acquires video frame data through an open graphics library (opengl) interface. The CPU fills in the new surface view new surfaceview with the video frame data acquired through the opengl interface.

Since the original video layer and the corresponding original surface view new surfaceview are created by the video playing application, the terminal side is not easy to modify and control, so the terminal can replace the original surface view by creating a new surface view new surfaceview and a new video layer. And the original video layer to operate on the new surface view new surfaceview and the new video layer.

Moreover, by creating a new video layer on the upper layer of the corresponding position of the original video layer instead of the original video layer, the user does not perceive, and the resolution, the frame rate, the playback speed, and the display content are not affected, and thus Will not reduce the user experience.

With reference to the first aspect and the foregoing possible implementation manner, in another possible implementation manner, the CPU converting the first format into the second format includes: the first format corresponding to the video frame data in the new surface view new surface view of the CPU Convert to the second format.

In this way, the CPU can convert the first format corresponding to the video frame data in the new surface view new surface view corresponding to the new video layer into the second format.

With reference to the first aspect and the foregoing possible implementation manners, in another possible implementation manner, the CPU converts the first format corresponding to the video frame data in the new surface view new surface view into the second format, including: the CPU passes the new surface view. The undersurface module nativesurface of the new surfaceview converts the first format corresponding to the video frame data in the new surface view new surfaceview into the second format.

The CPU specifically converts the first format into the second format by using the underlying module.

With reference to the first aspect and the foregoing possible implementation manner, in another possible implementation manner, the image processing unit converting the second format into the target format includes: the image processing unit corresponding to the video frame data in the new surface view new surface view The second format is converted to the target format. The image processing unit scaling the video layer picture represented by the video frame data of the target format includes: the image processing unit scales the new video layer picture represented by the video frame data converted into the target format in the new surface view new surfaceview.

That is, the image processing unit may specifically convert the second format corresponding to the video frame data into the target format in the new surface view new surface view, and convert the video image data converted into the target format in the new surface view new surfaceview. The new video layer picture is scaled.

With reference to the first aspect and the foregoing possible implementation manner, in another possible implementation manner, the original video layer corresponds to an original surface view, and the method further includes: the display unit stops displaying the original video layer corresponding to the original surface view original surface view Picture. The CPU inserts the new surface view new surfaceview into the view hierarchy viewhieiarchy in which the original surface view is located. The display unit displays the scaled video layer picture including: the display unit displays the scaled target video layer picture, and the target video layer picture is a new video layer picture in the new surface view new surfaceview that has been converted into the target format video frame data.

In this way, the terminal can no longer display the original video layer picture, but replace it by displaying a new video layer picture.

In combination with the first aspect and the above possible implementation manners, in another possible implementation manner, the second format is a YUV type format.

In this way, the CPU can convert the first format to the second format of the YUV type.

With reference to the first aspect and the foregoing possible implementation manner, in another possible implementation manner, the CPU creates a new video layer on a higher layer of a corresponding position of the original video layer corresponding to the video frame data, including: the CPU corresponding to setting a new video layer Zorder creates a new video layer on the upper layer of the corresponding position of the original video layer of the video frame.

In this way, the CPU can set the upper and lower layers of the new video layer and the original video layer through zorder.

With reference to the first aspect and the foregoing possible implementation manner, in another possible implementation manner, when the first format corresponding to the decoded video frame data is a preset class format, the method further includes: the image processing unit is configured to: The video layer picture represented by the frame data is scaled such that the size of the scaled video layer picture matches the size of the display unit, and the display unit displays the scaled video layer picture.

In this way, when the first format corresponding to the decoded video frame data is a preset type format that can be recognized by the image processing unit, the image processing unit can directly process the video frame data of the first format.

In a second aspect, an embodiment of the present application provides a terminal, including at least one processor, an image processing unit, a screen, a memory, and a bus. The processor, the image processing unit, the screen, and the memory are connected by a bus. The memory is used to store instructions. The processor is configured to execute an instruction to: when the first format corresponding to the decoded video frame data is not a preset type format, convert the first format into a second format, where the video frame data is used to represent the video layer picture, and the second format For the preset class format, the preset class format is a format that the image processing unit can recognize. The image processing unit is configured to execute an instruction to: convert the second format into a target format, and the target format is a display unit for The displayed data frame format scales the video layer picture represented by the video frame data of the target format such that the size of the scaled video layer picture matches the size of the display unit.

In conjunction with the second aspect, in a possible implementation, the display unit is configured to execute an instruction to perform: displaying the scaled video layer picture.

With reference to the second aspect and the foregoing possible implementation manner, in another possible implementation manner, the first format corresponds to the first video playing application, and the video frame data of the first format is the decoding corresponding to the first video playing application. The data obtained by decoding the video source data.

With reference to the second aspect and the foregoing possible implementation manner, in another possible implementation, the processor is specifically configured to: convert the first format into the second format according to the first video playing application corresponding to the first format.

With reference to the second aspect and the foregoing possible implementation manner, in another possible implementation manner, the terminal saves a correspondence between the first video playing application and the third format, where the third format is a preset type format, and the second format is A third format corresponding to the first video playback application.

With reference to the second aspect and the foregoing possible implementation manners, in another possible implementation manner, the video layer is displayed by a surface view surfaceview, and the processor is further configured to: before converting the first format into the second format, in the video frame A new video layer is created on the upper layer of the corresponding position of the original video layer corresponding to the data, the new video layer is used to cover the original video layer, and the new video layer corresponds to the new surface view new surface view. Obtain video frame data through the opengl interface. Fill in the new surface view new surfaceview with the video frame data obtained through the opengl interface.

With reference to the second aspect and the foregoing possible implementation manners, in another possible implementation manner, the processor is specifically configured to: convert the first format corresponding to the video frame data in the new surface view new surface view into the second format.

With reference to the second aspect and the foregoing possible implementation manners, in another possible implementation manner, the processor is specifically configured to: corresponding to the video frame data in the new surface view new surface view by using the bottom surface module nativesurface of the new surface view new surface view The first format is converted to the second format.

With reference to the second aspect and the foregoing possible implementation manners, in another possible implementation manner, the image processing unit is specifically configured to: convert the second format corresponding to the video frame data into the target format in the new surface view new surface view. The new video layer picture represented by the video frame data that has been converted into the target format is scaled in the new surface view new surfaceview.

With reference to the second aspect and the foregoing possible implementation manners, in another possible implementation manner, the original video layer corresponds to the original surface view original surface view, and the display unit is configured to stop displaying the original video layer picture corresponding to the original surface view original surface view. The processor is further configured to insert the new surface view new surfaceview into the view hierarchy viewhieiarchy where the original surface view is located. The display unit is specifically configured to: display the scaled target video layer picture, and the target video layer picture is a new video layer picture that has been converted into a video frame data representation of the target format in the new surface view new surfaceview.

In combination with the second aspect and the above possible implementation manners, in another possible implementation manner, the second format is a YUV type format.

In a third aspect, an embodiment of the present application provides a device, which is in the form of a product of a chip. The device includes a processor and a memory, and the memory is coupled to the processor to save necessary program instructions of the device. And data, the processor for executing program instructions stored in the memory such that the apparatus performs the functions of data processing in the above method.

In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, where the storage medium includes an instruction, When it is running on the terminal, the terminal is caused to perform the data processing method according to any of the first aspect and the first aspect described above.

In a fifth aspect, the embodiment of the present application provides a computer program product including instructions, when the terminal is running on the terminal, causing the terminal to perform the data processing method in any one of the foregoing first aspect and the first aspect.

For the beneficial effects of the foregoing second to fifth aspects, reference may be made to the corresponding description in the above first aspect, and details are not described herein again.

DRAWINGS

1 is a flow chart of a data processing method provided in the prior art;

2 is a schematic structural diagram of a mobile phone according to an embodiment of the present application;

FIG. 3 is a flowchart of a data processing method according to an embodiment of the present application;

4 is a flowchart of another data processing method according to an embodiment of the present application;

FIG. 5 is a flowchart of another data processing method according to an embodiment of the present application;

FIG. 6 is a flowchart of another data processing method according to an embodiment of the present application;

FIG. 7 is a flowchart of another data processing method according to an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a terminal according to an embodiment of the present application;

FIG. 9 is a schematic structural diagram of another terminal according to an embodiment of the present application.

detailed description

Since the prior art shown in FIG. 1 performs video playback, the format of the video frame data is converted by the GPU into a target format that can be directly used for display of the display unit, and the video layer picture is scaled to a size matching the display unit. This consumes more power and reduces the user experience. The data processing method provided by the embodiment of the present application can reduce the power consumption during video playback. The main principle is: replacing the GPU with a part of the format conversion operation and the zoom operation by using a simpler and less power-consuming image processing unit.

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise stated, "/" means the meaning of or, for example, A/B may represent A or B; "and/or" herein is merely a description of the associated object. The association relationship indicates that there may be three relationships, for example, A and/or B, which may indicate that there are three cases in which A exists separately, A and B exist at the same time, and B exists separately. In addition, in the description of the embodiments of the present application, "a plurality" means two or more than two.

The data processing method provided by the embodiment of the present application can be applied to a terminal capable of video playback. Exemplarily, the terminal here may be a mobile phone, a tablet computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), or the like.

Specifically, the data processing method provided by the embodiment of the present application is introduced by using the terminal as a mobile phone as an example. The components of the mobile phone 100 will be specifically described below with reference to FIG. 2:

As shown in FIG. 2, the mobile phone 100 may include: a screen 101, a processor 102, a memory 103, a power source 104, a radio frequency (RF) circuit 105, a gravity sensor 106, an audio circuit 107, a video card 108, a GPU 109, and image processing. Units such as unit 110, which may be connected by a bus or directly connected. Those skilled in the art can understand that the structure of the mobile phone shown in FIG. 1 does not constitute a limitation on the mobile phone, and It includes more components than the illustrations, or some components, or different component arrangements.

The screen 101 may specifically be a touch display screen or a non-touch display screen, and may be used for interface display, video play, picture browsing, and the like.

The processor 102 is the control center of the handset 100, which connects various portions of the entire handset using various interfaces and lines, by running or executing software programs and/or modules stored in the memory 103, and recalling data stored in the memory 103, The various functions and processing data of the mobile phone 100 are executed to perform overall monitoring of the mobile phone 100. In a particular implementation, as an embodiment, processor 102 can include one or more processing units; processor 102 can integrate an application processor and a modem processor. The application processor mainly processes an operating system, a user interface, an application, and the like, and the modem processor mainly processes wireless communication. It will be appreciated that the above described modem processor may also not be integrated into the processor 102.

The memory 103 can be used to store data, software programs, and modules, and can be a volatile memory, such as a random-access memory (RAM), or a non-volatile memory. For example, a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); or a combination of the above types of memories. Specifically, the program code is stored in the memory 103, and the program code is used to enable the processor 102 to execute the data processing method provided by the embodiment of the present application by executing the program code.

The power source 104, which can be a battery, is logically coupled to the processor 102 through a power management system to manage functions such as charging, discharging, and power management through the power management system.

The RF circuit 105 can be used for transmitting and receiving information or during a call, receiving and transmitting signals, and in particular, processing the received information to the processor 102; in addition, transmitting the signals generated by the processor 102. Generally, RF circuits include, but are not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, and the like. In addition, RF circuit 105 can also communicate with the network and other devices via wireless communication.

The gravity sensor 106 can collect the acceleration of the mobile phone in various directions (usually three axes), and can collect the magnitude and direction of gravity when stationary, and can be used to identify the gesture of the mobile phone (such as horizontal and vertical screen switching, related Game, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tapping). It should be noted that the mobile phone 100 may further include other sensors, such as a pressure sensor, a light sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, and details are not described herein.

The audio circuit 107 is used to provide an audio interface between the user and the handset 100.

The video card 108 is a device for digital-to-analog signal conversion of the mobile phone, and has image processing capability, which can assist the completion of the work of the processor 102 and improve the overall running speed of the mobile phone 100.

The GPU 109 is a processor of a graphics card, which is a complex arithmetic unit, and includes a large number of logic arrays for converting and driving display information required by the mobile phone 100, and providing a line scan signal to the screen 101 to control the correctness of the screen 101. Display, you can perform complex mathematical and geometric calculations required for graphics rendering.

The image processing unit 110 is a simple hardware logic unit that contains fewer logic arrays and is small in size and low in power consumption, and is generally used to scale an image to a size that matches the size of the screen 101, such as an MDP unit. (Qualcomm) or DSS unit (His) and so on. In one implementation, image processing unit 110 may also be integrated into processor 110 or other chips of handset 100.

The audio circuit 107, the video card 108, the GPU 109, and the image processing unit 110 can be used to perform video playback in conjunction with the screen 101 and the processor 102.

Although not shown, the mobile phone 100 may further include a wireless fidelity (WiFi) module, a Bluetooth module, a camera, and the like, and details are not described herein.

The data processing method provided by the embodiment of the present application is described in detail below with reference to the specific components in the mobile phone 100 shown in FIG. 2 in order to make the purpose, the technical solution and the advantages of the embodiment of the present application more clear. Among them, the steps shown below can also be performed in any terminal other than the mobile phone shown in FIG. 2. In addition, although the logical order of the data processing methods provided by the embodiments of the present application is shown in the method flowchart, in some cases, the steps shown or described may be performed in an order different from that herein.

Referring to FIG. 3, an embodiment of the present application provides a data processing method, which may be applied to a terminal, where the terminal may include a central processing unit CPU, an image processing unit, and a display unit. The method may include:

201. When the first format corresponding to the decoded video frame data is a non-preset type format, the CPU converts the first format into a second format, where the video frame data is used to represent the video layer picture, and the second format is a preset class. The format, the preset class format is a format that the image processing unit can recognize.

The video refers to a continuous picture whose display speed exceeds 24 frames per second, and is specifically composed of video data of one frame and one frame. The video frame data is video data obtained after decoding the video source data, and may be referred to as rawdata data, and each frame of data may correspond to one video layer picture.

The preset class format here is a video frame format that the image processing unit can recognize and directly process, and is usually a general standard video frame format, for example, a format such as YUV, CMYK, YCBCR or RGB, and each type is specifically Can include multiple formats. For example, some of the preset video frame formats can be found in some common standard video formats listed in Table 1 below.

Table 1

When the first format corresponding to the decoded video frame data is a non-preset type format, that is, the first format cannot be recognized by the image processing unit, and thus cannot be directly processed by the image processing unit, the CPU may convert the first format. A second format that can be processed by the image processing unit such that the video frame data can be processed by the image processing single. The first format is usually close to the preset type format, and is generally a simple variant of the preset type format. For example, the first format adds, changes, or deletes some characters in the video data stream corresponding to the format of the preset class format YUV type. Moreover, the CPU converts the first format into the second format at a faster speed, and the delay is small, for example, may be several milliseconds.

The converting the first format to the second format by the CPU specifically means that the CPU converts the video frame data of the first format into the video frame data of the second format.

Specifically, the CPU may determine, by the framework framework layer module, whether the first format can be recognized by the image processing unit.

202. The image processing unit converts the second format into a target format, where the target format is a data frame format that the display unit can use for display.

A description of the image processing unit herein can be found in the description of the relevant portion of FIG. 2 above. The target format is a data frame format that the display unit can directly use for display, and can usually be an RGB type format. The image processing unit may specifically convert the second format into a target format that is adapted to the characteristics of the display unit according to the characteristics of the display unit. Exemplarily, when the display unit is a true color screen, the target format may be the RGB888 format; when the display unit is an enhanced color screen, the target format may be the RGB555 format.

203. The image processing unit scales the video layer picture represented by the video frame data of the target format, so that the size of the scaled video layer picture matches the size of the display unit.

After the image processing unit converts the second format corresponding to the video frame data into the target format, the size of the video layer picture represented by the video frame data may also be scaled so that the size of the video layer picture matches the size of the display unit. In order for the display unit to display the scaled video layer screen.

In the embodiment of the present application, when the first format is a format that the image processing unit cannot recognize, the CPU may convert the first format into a second format that is close to the format of the first format, and the image processing unit may convert the second format. The target format can be directly used for display, and the video layer picture is scaled to the screen size to facilitate display of the display unit.

When adopting the solution in the prior art, the terminal will convert the first format into the target format through the GPU. Since the first format and the target format usually differ greatly, the first format is usually not a simple deformation of the target format, and thus the GPU usually needs to perform complex processing to convert the GPU into a target format. Also, the GPU needs to scale the video layer to the screen size. Since the GPU is a complex arithmetic unit and internally contains a large number of logic arrays, the operation of the GPU will consume more power.

In the embodiment of the present application, the process of converting the first format into the target format may be divided into two parts, that is, the CPU converts the first format into the second format, and the image processing unit converts the second format into the target format. In the first part, compared with the GPU, although the CPU is also a complex arithmetic unit, the CPU is only used to convert the first format into a second format similar to the first format, so that the CPU can be realized by simple processing and processing. Fast speed and small delay. In the second part, the power consumption can be saved by running the image processing unit to convert the second format to the target format compared to running the GPU. It can be seen that the method for converting the first format into the second format by the CPU+image processing unit provided by the embodiment of the present application can save power. For the process of scaling a video layer picture to a screen size, the prior art is implemented by running a complex arithmetic unit GPU, and the present application is realized by running a simpler image processing unit, thereby saving power.

That is to say, the embodiment of the present application replaces the GPU by a smaller and simpler image processing unit to perform a part of the format conversion operation and the zoom operation, thereby reducing the power consumption and improving the user experience.

In addition, it should be noted that, in the embodiment of the present application, the image processing unit is configured to scale the video layer image to a screen size, that is, the size of the video layer picture entering the image processing unit is the original size; In the prior art, the GPU is used to scale the video layer picture to the screen size, that is, the video layer picture that enters the image processing unit from the GPU has been scaled to the screen size, and the image processing unit only performs 1:1 scaling (it can also be understood as No scaling).

In addition, the flowchart of the data processing method provided by the foregoing embodiment of the present application can also refer to FIG. 4 .

Further, referring to FIG. 5, after the step 203, the method provided by the embodiment of the present application may further include:

204. The display unit displays the scaled video layer screen.

The video frame data corresponding to the video layer picture herein is a target format. The display unit here may be a screen of the terminal, such as a liquid crystal display (LCD) screen or an organic light emitting display (OLED) screen. In the embodiment of the present application, unless otherwise stated, the display unit and the screen are used to indicate the same meaning, and both can be used universally.

After the image processing unit scales the size of the video layer picture, the display unit can display the scaled video layer picture. And, the display unit displays the video layer screen at a frequency greater than 24 frames/second for video playback.

The process of scaling the size of the video layer picture to match the size of the display unit is referred to as scaling the video layer picture to the screen size or scaling the video layer picture to the matching size.

In the embodiment of the present application, the first format in step 201 may correspond to the first video playing application, and the video frame data in the first format is data obtained by decoding the video source data by using a decoder corresponding to the first video playing application. .

Wherein, the video frame data here is the video data obtained by decoding by the decoder, where the decoder is a decoder corresponding to the video playing application, not a system decoder of the Android system, and specifically may include a software decoder or a hardware decoder. The decoders corresponding to different video playbacks may be different. The format of the video frame data obtained by decoding corresponding to different video playing applications may also be different, that is, the format of the decoded video frame data corresponds to the video playing application. For example, the video playback application 1 corresponds to a video frame format 1 (first format), and the video playback application 2 corresponds to a video frame format 2 (first format).

When the first format corresponds to the first video playing application, in the foregoing step 201, the CPU converting the first format into the second format may include:

In 2011, the CPU converts the first format into the second format according to the first video playing application corresponding to the first format.

When the first format corresponds to the first video playing application, the first format is a format of the video data obtained by decoding by the decoder corresponding to the first video playing application, and thus the CPU may use the first video playing application corresponding to the first format, Convert the first format to the second format.

Specifically, the terminal may save the correspondence between the first video playing application and the third format, where the third format is a preset type format, and the second format is a third format corresponding to the first video playing application. Therefore, the step 2011 may specifically include: the CPU converting the first format into a third format corresponding to the first video playing application, the third format That is the second format.

For example, the video playback application 1 corresponds to video frame format 3 (third format), and the video playback application 2 corresponds to video frame format 4 (third format). When the first format is the video frame format 2, the first video playing application corresponding to the video frame format 2 is the video playing application 2, and the third format corresponding to the video playing application 2 is the video frame format 4, and thus, the CPU can Video frame format 2 (first format) is converted into video frame format 4 (second format).

It is noted that, in addition to the correspondence between the first video frame application and the third format, the terminal may also store a correspondence between other video playback applications and video frame formats in the preset video, and the terminal may also pass The software upgrade and the like update and modify the correspondence between the video playback application and the video frame format saved in the terminal.

The correspondence between the first video playing application and the third format may be determined in a testing phase. For example, the format in the preset class format may be referred to as a fourth format. In an implementation manner, the first format of the video frame data obtained by decoding by the decoder corresponding to the first video playing application may be converted into a preset type format. a fourth format, and then the image processing unit can convert the fourth format into a target format, and the display unit can display the video layer picture represented by the video frame data of the target format, if the screen display condition is met, It can be determined that the fourth format belongs to the preset type format, and the third format is determined to be the format in which the video layer data represented by the corresponding video frame data has the highest resolution in the preset format; or, the third format can be determined. The format is any of the preset class formats. Furthermore, the correspondence between the first video playing application corresponding to the first format and the determined third format may be saved.

Among them, the screen display conditions can be set according to actual needs. For example, a pixel on the display unit that can normally display an image is greater than or equal to a preset value.

In another implementation manner, the first format may be converted into a fourth format in the preset class format, and if the video frame data corresponding to the fourth format converted to this time satisfies the screen display condition, the current format is determined. The fourth format converted into the third format; if the video frame data corresponding to the fourth format converted to the current format does not satisfy the screen display condition, the first format is converted into another fourth format in the preset class format. Until the video frame data corresponding to the converted fourth format satisfies the screen display condition.

In addition, if the video playback process is abnormal, for example, the video playback application exits abnormally, the corresponding relationship between the second format corresponding to the video playback application and the first format may be incorrect, and the video playback application may be deleted at this time. The correspondence of the third format.

Specifically, in the embodiment of the present application, the video layer may be displayed through a surface view surfaceview. Among them, surfaceview has a separate drawing surface, which does not share the same drawing surface with its host window. With a separate drawing surface, the surfaceview interface can be drawn in a separate thread without taking up the main thread resources, enabling a complex and efficient interface.

In the embodiment of the present application, referring to FIG. 6, when the first format cannot be recognized by the image processing unit, before the CPU converts the first format to the second format in step 201, the method may further include:

205. The CPU creates a new video layer on the upper layer of the corresponding position of the original video layer corresponding to the video frame data, where the new video layer is used to cover the original video layer, and the new video layer corresponds to the new surface view new surface view.

The CPU may create a new video layer at a corresponding position of the original video layer by setting the width wide and the height height corresponding to the new video layer to be the same as the width width and the cover height corresponding to the original video layer. Moreover, the CPU can create a new video on the upper layer of the original video layer of the video frame by setting the zorder corresponding to the new video layer. Floor. In this way, the new video layer set can completely cover the original video layer.

206. The CPU acquires video frame data through an open graphics library opengl interface.

207. The CPU fills the new surface view new surfaceview with the video frame data obtained through the open graphics library opengl interface.

In this way, the video frame data in the new surface view is the video frame data corresponding to the new video layer, and the video frame data corresponding to the new video layer is the video frame data corresponding to the original video layer.

Since the original video layer and the original surface view new surfaceview corresponding to the original video layer are created by the video playing application, the terminal side is not easy to modify and control them, so the CPU on the terminal side can create a new surface view new surfaceview and new The video layer replaces the original surface view new surfaceview and the original video layer, so that the terminal components can be formatted and scaled by controlling the original surface view new surfaceview and the original video layer to achieve the purpose of reducing power consumption.

On the basis of the foregoing steps 205-207, the CPU may convert the first format into the second format, which may include:

In 2010, the CPU converts the first format corresponding to the video frame data in the new surface view new surfaceview into the second format.

That is to say, after the new surface view new surfaceview replaces the original surface view new surfaceview, after the new video layer replaces the original video layer, the CPU can correspond to the new video layer corresponding to the first format corresponding to the video frame data in the new surface view new surfaceview. Convert to the second format.

The converting, by the CPU, the first format corresponding to the video frame data in the new surface view in the new surface view to the second format may include:

The CPU converts the first format corresponding to the video frame data in the new surface view new surfaceview into the second format through the underlying module nativesurface of the new surface view new surfaceview.

Specifically, the underlying module nativesurface is used to support the function of the new surfaceview. The CPU may first obtain video frame data (ie, data capture data catch) from the opengl interface through the nativesurface, and then copy the acquired video frame data (ie, data sharing data share), thereby converting the first format corresponding to the video frame into the second format. (ie data adjustment data adjust).

Based on step 2010, the image processing unit can process the video data in the new surface view new surfaceview. The foregoing step 202 may specifically include:

2020. The image processing unit converts the second format corresponding to the video frame data into the target format in the new surface view new surface view.

The foregoing step 203 may specifically include:

2030. The image processing unit scales a new video layer image represented by the video frame data that has been converted into the target format in the new surface view new surfaceview.

In the embodiment of the present application, the original video layer may correspond to the original surface view. Before the display unit displays the scaled video layer screen in step 204, the method may further include:

208. The display unit stops displaying the original video layer picture corresponding to the original surface view of the original surface view.

When the new surface view new surface view and the new video layer are not created, according to the flow in the prior art, the CPU sends the video frame data corresponding to the original video layer to the GPU, the image processing unit, and the display unit for processing. In the embodiment of the present application, when the new surface view new surface view and the new video layer are created instead of the original surface view original surface view and the original video layer, the original video layer picture does not need to be displayed again. Specifically, the CPU may stop sending the video frame data in the original surface view to the GPU, the image processing unit, and the display unit for processing, and the display unit may stop displaying the original video layer image, and display the new surface view new surfaceview new Video layer picture.

Specifically, the CPU can stop displaying the original video layer by setting the transparency of the new video layer to be non-transparent. When the transparency of the new video layer is non-transparent, the original video layer is completely occluded, and the terminal will not continue to perform the display processing flow on the original video layer, and the display unit will no longer display the original video layer picture. Among them, it is worth noting that for some target formats such as the new video layer of the RGB888 format, it is itself opaque, so the CPU does not need to set the transparency of the new video layer to be non-transparent.

209. The CPU inserts a new surface view new surfaceview into the view hierarchy viewhieiarchy where the original surfaceview is located.

Viewhieiarchy is a tree-tree structure of the view view, including all views and attributes of all views, such as the wide coordinate information of the view, the height coordinate information, and the z-axis coordinate information. Based on this information, the layout of the entire screen can be obtained. Viewhieiarchy can correspond to video playback applications.

After creating a new surface view new surfaceview, the CPU can insert the new surface view new surfaceview into the view hierarchy viewhieiarchy where the original surfaceview is located, so that the new surface view new surfaceview can be merged with other views in the screen, thus in the new surface view. When the new surfaceview replaces the original surface view corresponding to the original video layer, it does not affect the overall layout of the screen, and does not change the normal display of the screen.

It should be noted that the method flow shown in FIG. 6 is merely an exemplary description, and step 209 may be performed after step 205 and before step 2040.

The above step 204 may specifically include:

2040. The display unit displays the scaled target video layer picture, and the target video layer picture is a new video layer picture represented by the video frame data of the target format in the new surface view new surfaceview.

In addition, the flowchart of the data processing method provided by the foregoing embodiment of the present application can also refer to FIG. 7.

In addition, when the first format corresponding to the decoded video frame data is a preset type format, the method may further include:

210. The image processing unit scales the video layer picture represented by the video frame data, so that the size of the scaled video layer picture matches the size of the display unit.

211. The display unit displays the scaled video layer screen.

That is, when the first format corresponding to the decoded video frame data is a preset type format that can be recognized by the image processing unit, the image processing unit can directly process the video frame data of the first format.

In addition, in the data processing method provided by the embodiment of the present application, the video frame data obtained by decoding by the system decoder may be directly sent to the image processing unit for processing.

Further, when performing video playback, a video layer, a logo layer, and the like may be superimposed on the video layer. Thus the method can also include:

212. The display unit displays a bullet layer and/or a logo logo layer.

The solution provided by the embodiment of the present application is mainly introduced from the perspective of the terminal. It can be understood that, in order to implement the above functions, the terminal includes corresponding hardware structures and/or software modules for performing various functions. Those skilled in the art will readily appreciate that the present application can be implemented in a combination of hardware or hardware and computer software in combination with the algorithmic steps of the various examples described in the embodiments disclosed herein. Whether a function is implemented in hardware or computer software to drive hardware depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods to implement the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present application.

The embodiment of the present application may divide the function module into the terminal according to the foregoing method example. For example, each function module may be divided according to each function, or two or more functions may be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of the module in the embodiment of the present application is schematic, and is only a logical function division, and the actual implementation may have another division manner.

FIG. 8 is a schematic diagram showing a possible composition of the terminal 30 involved in the foregoing embodiment. As shown in FIG. 8, the terminal 30 may include: at least one processor. 31. Image processing unit 32, screen 33, memory 34, and bus 35. The processor 31, the image processing unit 32, the screen 33, and the memory 34 are connected by a bus 35.

Among them, the memory 34 is used to store instructions. The processor 31 is configured to execute an instruction to: convert the first format into the second format when the first format corresponding to the decoded video frame data is not the preset class format. The video frame data is used to represent the video layer picture, the second format is a preset type format, and the preset class format is a format that the image processing unit can recognize. The image processing unit 32 is configured to execute an instruction to: convert the second format into a target format, the target format is a data frame format used by the display unit for display, and scale the video layer image represented by the video frame data of the target format, so that The size of the scaled video layer picture matches the size of the display unit.

Further, the screen 33 is for supporting the terminal to execute step 204 in the data processing method shown in FIG. 5, step 208 and step 2040 in FIG.

The processor 31 is further configured to support the terminal to perform steps 205-207, step 2010, and step 209 in the data processing method shown in FIG. 6.

The image processing unit 32 can also be used to support the terminal to perform

steps

2020 and 2030 in the data processing method shown in FIG. 6.

It should be noted that all the related content of the steps involved in the foregoing method embodiments may be referred to the functional descriptions of the corresponding functional modules, and details are not described herein again.

The terminal provided in the embodiment of the present application is configured to execute the foregoing data processing method, so that the same effect as the data processing method described above can be achieved.

In the case of employing an integrated unit, FIG. 9 shows another possible composition diagram of the terminal 40 involved in the above embodiment. As shown in FIG. 9, the terminal 40 includes a processing module 41 and a communication module 42.

The processing module 41 is configured to control and manage the actions of the terminal 40. For example, the processing module 41 is configured to support the terminal 40 to perform the operations of the processor 31, the image processing unit 32, and the screen 33 shown in FIG. 8, and/or for the purposes of this document. Other processes of the described techniques. The communication module 42 is used to support communication between the terminal and other network entities. Terminal also A storage module 43 may be included for performing the operations of the memory 34 shown in FIG. 8, storing the program code and data of the terminal 40.

The processing module 41 can be a processor or a controller. It is possible to implement or carry out the various illustrative logical blocks, modules and circuits described in connection with the present disclosure. The processor can also be a combination of computing functions, for example, including one or more microprocessor combinations, a combination of a digital signal processor (DSP) and a microprocessor, and the like. The communication module 42 can be a transceiver, a transceiver circuit, a communication interface, or the like.

Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of the description, only the division of the above functional modules is illustrated. In practical applications, the above functions can be allocated according to needs. It is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or It can be integrated into another device, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.

The units described as separate components may or may not be physically separated, and the components displayed as units may be one physical unit or multiple physical units, that is, may be located in one place, or may be distributed to a plurality of different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

An integrated unit can be stored in a readable storage medium if it is implemented as a software functional unit and sold or used as a standalone product. Based on such understanding, the technical solution of the embodiments of the present application may be embodied in the form of a software product in the form of a software product in essence or in the form of a contribution to the prior art, and the software product is stored in a storage medium. A number of instructions are included to cause a device (which may be a microcontroller, chip, etc.) or a processor to perform all or part of the steps of the various embodiments of the present application. The foregoing storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

The foregoing is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto, and any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application. It should be covered by the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the claims.

Claims

A data processing method is applied to a terminal, where the terminal includes a central processing unit CPU, an image processing unit, and a display unit, wherein the method includes:

When the first format corresponding to the decoded video frame data is not a preset type format, the CPU converts the first format into a second format, where the video frame data is used to represent a video layer picture, and the second The format is a preset class format, and the preset class format is a format that the image processing unit can recognize;

The image processing unit converts the second format into a target format, where the target format is a data frame format used by the display unit for display;

The image processing unit scales a video layer picture represented by the video frame data of the target format such that the size of the scaled video layer picture matches the size of the display unit.
The method of claim 1 further comprising:

The display unit displays the scaled video layer picture.
The method according to claim 1 or 2, wherein the first format corresponds to a first video playing application, and the video frame data of the first format is a decoder corresponding to the first video playing application. Data obtained by decoding video source data.
The method according to claim 3, wherein the converting the first format to the second format by the CPU comprises:

The CPU converts the first format into a second format according to the first video playing application corresponding to the first format.
The method according to claim 4, wherein the terminal saves a correspondence between the first video playing application and a third format, the third format is a preset type format, and the second format is The third format corresponding to the first video playback application.
The method according to any one of claims 2 to 5, wherein the video layer is displayed by a surface view surfaceview, and before the CPU converts the first format into a second format, the method further comprises :

The CPU creates a new video layer on the upper layer of the corresponding position of the original video layer corresponding to the video frame data, where the new video layer is used to cover the original video layer, and the new video layer corresponds to a new surface view new surface view;

The CPU acquires the video frame data by using an open graphics library opengl interface;

The CPU fills the new surface view new surfaceview with the video frame data acquired through the open graphics library opengl interface.
The method according to claim 6, wherein the converting the first format to the second format by the CPU comprises:

The CPU converts a first format corresponding to the video frame data in the new surface view new surfaceview into the second format.
The method according to claim 7, wherein the converting, by the CPU, the first format corresponding to the video frame data in the new surface view new surfaceview into the second format comprises:

The CPU converts a first format corresponding to the video frame data in the new surface view new surface view into the second format by using an underlying module nativesurface of the new surface view new surface view.
The method according to any one of claims 6-8, wherein the converting the second format to the target format by the image processing unit comprises:

The image processing unit converts a second format corresponding to the video frame data into the target format in the new surface view new surface view;

The image processing unit scaling the video layer picture represented by the video frame data of the target format includes:

The image processing unit scales a new video layer picture represented by the video frame data that has been converted into the target format in the new surface view new surface view.
The method according to claim 9, wherein the original video layer corresponds to an original surface view, and the method further comprises:

The display unit stops displaying the original video layer picture corresponding to the original surface view original surface view;

The CPU inserts the new surface view new surfaceview into the view hierarchy structure viewhieiarchy where the original surface view original surfaceview is located;

Displaying, by the display unit, the scaled video layer screen includes:

The display unit displays the scaled target video layer picture, where the target video layer picture is a new video layer picture that has been converted into the video frame data representation of the target format in the new surface view new surfaceview.
The method according to any one of claims 1 to 10, wherein the second format is a YUV type format.
A terminal, comprising: at least one processor, an image processing unit, a screen, a memory, and a bus;

The processor, the image processing unit, the screen, and the memory are connected by a bus;

The memory is for storing instructions;

The processor is configured to execute the instructions to perform:

When the first format corresponding to the decoded video frame data is not a preset type format, the first format is converted into a second format, where the video frame data is used to represent a video layer picture, and the second format is a pre- a class format, where the preset class format is a format that the image processing unit can recognize;

The image processing unit is configured to execute the instructions to perform:

Converting the second format to a target format, where the target format is a data frame format used by the display unit for display;

The video layer picture represented by the video frame data of the target format is scaled such that the size of the scaled video layer picture matches the size of the display unit.
The terminal according to claim 12, wherein said display unit is configured to execute said instruction to perform:

The scaled video layer screen is displayed.
The terminal according to claim 12 or 13, wherein the first format corresponds to a first video playing application, and the video frame data of the first format is a decoding corresponding to the first video playing application. The data obtained by decoding the video source data.
The terminal according to claim 14, wherein the processor is specifically configured to:

Converting the first format to a second format according to the first video playing application corresponding to the first format.
The terminal according to claim 15, wherein the terminal saves the first video broadcast Corresponding to the third format, the third format is a preset type format, and the second format is the third format corresponding to the first video playing application.
The terminal according to any one of claims 13 to 16, wherein the video layer is displayed by a surface view surfaceview, and the processor is further configured to:

Before converting the first format to the second format, creating a new video layer on an upper layer of a corresponding position of the original video layer corresponding to the video frame data, where the new video layer is used to cover the original video layer. The new video layer corresponds to a new surface view new surfaceview;

Obtaining the video frame data through an open graphics library opengl interface;

Filling the new surface view new surfaceview with the video frame data obtained through the open graphics library opengl interface.
The terminal according to claim 17, wherein the processor is specifically configured to:

Converting a first format corresponding to the video frame data in the new surface view new surfaceview to the second format.
The terminal according to claim 18, wherein the processor is specifically configured to:

Converting, by the underlying module nativesurface of the new surface view new surfaceview, a first format corresponding to the video frame data in the new surface view new surfaceview to the second format.
The terminal according to any one of claims 17 to 19, wherein the image processing unit is specifically configured to:

Converting, in the new surface view new surfaceview, a second format corresponding to the video frame data into the target format;

The new video layer picture represented by the video frame data that has been converted into the target format is scaled in the new surface view new surfaceview.
The terminal according to claim 20, wherein the original video layer corresponds to an original surface view, and the display unit is further configured to stop displaying an original video layer corresponding to the original surface view;

The processor is further configured to insert the new surface view new surfaceview into the view hierarchy structure viewhieiarchy where the original surface view original surface view is located;

The display unit is specifically configured to:

Displaying the scaled target video layer picture, which is a new video layer picture that has been converted into the video frame data representation of the target format in the new surface view new surfaceview.
The terminal according to any one of claims 12 to 21, wherein the second format is a YUV type format.
A computer readable storage medium, comprising instructions that, when run on a terminal, cause the terminal to perform the data processing method of any of claims 1-11.
A computer program product comprising instructions, wherein when executed on a terminal, the terminal is caused to perform the data processing method according to any one of claims 1-11.