CN110958459A

CN110958459A - Data processing method and device

Info

Publication number: CN110958459A
Application number: CN201811124587.1A
Authority: CN
Inventors: 李正光; 王立军; 储开颜
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2018-09-26
Filing date: 2018-09-26
Publication date: 2020-04-03
Anticipated expiration: 2038-09-26
Also published as: CN110958459B

Abstract

The application discloses a data processing method and a device thereof, wherein the method comprises the following steps: acquiring a first syntax element corresponding to original data; judging whether the first syntax element meets a preset condition or not; and if the preset condition is met, encoding the original data by utilizing the first syntax element. By adopting the method and the device, the coding speed of data coding is high, the power consumption is low and the compression rate is high.

Description

Data processing method and device

Technical Field

The present application relates to the field of communications, and in particular, to a data processing method and apparatus.

Background

At present, a hardware encoder is arranged in an electronic device, the hardware encoder can encode data based on an h264 protocol, then, the electronic device transmits the encoded data, although the hardware encoder has a high encoding speed and low power consumption, the compression ratio is low, which results in a large amount of encoded data and requires more network bandwidth and storage space, or the electronic device can encode by using a software encoder, which can encode the data by using an encoding algorithm, and then, the electronic device transmits the encoded data, the compression ratio of the software encoder is high, the flexibility is high, but the encoding speed is low, the power consumption is high, and the requirement on a processor is high.

Therefore, when we need to use the encoder to implement some application functions, the encoder is usually selected according to the usage scenario of the application. That is, in the prior art, an application may select a software encoder or a hardware encoder according to the operating status of the electronic device and the requirement index of data, for example, in the case of low requirement on resolution and frame rate, the software encoder is selected, but in the case of high requirement on resolution Drung nationality and high requirement on frame rate, the hardware encoder is selected. However, the prior art cannot meet application scenarios with high requirements on indexes such as compression rate, encoding speed and power consumption. In summary, there is a need in the art for a technical solution with fast encoding speed, low power consumption and high compression rate.

The above information is presented merely as background information to aid in understanding the present disclosure. No determination has been made, nor has a statement been made, as to whether any of the above information is applicable as prior art against the present disclosure.

Disclosure of Invention

The present application mainly aims to provide a data processing method and a device thereof, and aims to solve the above-mentioned technical problems.

An exemplary embodiment of the present application provides a data processing method including obtaining a first syntax element corresponding to original data; judging whether the first syntax element meets a preset condition or not; and if the preset condition is met, encoding the original data by utilizing the first syntax element.

Another exemplary embodiment of the present application provides a computer-readable storage medium having stored thereon computer instructions, wherein the instructions, when executed, implement the above-described method.

Another exemplary embodiment of the present application provides a data processing apparatus, comprising a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to: acquiring a first syntax element corresponding to original data; judging whether the first syntax element meets a preset condition or not; and if the preset condition is met, encoding the original data by utilizing the first syntax element.

Another exemplary embodiment of the present application provides a data processing method including encoding original data into first encoded data using a first type encoder; parsing a first syntax element from the first encoded data; encoding is performed on the original data with a second-type encoder based on the first syntax element.

Another exemplary embodiment of the present application provides a data processing apparatus, comprising a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to: encoding the original data into first encoded data using a first type of encoder; parsing a first syntax element from the first encoded data; encoding is performed on the original data with a second-type encoder based on the first syntax element.

The above-mentioned at least one technical scheme that this application example embodiment adopted can reach following beneficial effect:

the data processing method of the exemplary embodiment of the application can encode the original data under the condition that whether the first syntax element meets the requirement is judged, so that the encoded data can better meet the requirement of a user.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 is a diagram of an overall structure of a content providing system for providing content;

fig. 2 is a schematic diagram of a structure of syntax elements in the H264 protocol according to an exemplary embodiment of the present application;

FIG. 3 is a flow chart of a data processing method according to an exemplary embodiment of the present application;

fig. 4 is a block diagram of a data processing apparatus of an exemplary embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Before describing exemplary embodiments of the present application, terms referred to in the present application will be explained first to facilitate better understanding of the present application by those skilled in the art.

The encoding protocol refers to a protocol adopted in the process of compressing data. Existing encoding protocols include, but are not limited to, H264, H265, and VP 9.

The compression ratio is a ratio of the size of the original data to the size of the data after the original data is compressed, and the higher the compression ratio is, the smaller the size of the compressed data is, the lower the bandwidth occupied during transmission is, and therefore, the compression ratio is an important index for measuring the performance of the encoder.

The motion search refers to searching for content similar to the current content and encoding using the similarity of the content to reduce the amount of computation for encoding.

The code stream refers to the data flow used by a video file in unit time, also called code rate, and is the most important part in picture quality control in video coding. At the same resolution, the larger the code stream of the video file, the smaller the compression ratio, and the better the picture quality.

Syntax element (syntax element) refers to data generated by an encoder according to an encoding protocol, and in a code stream output by the encoder, the basic unit of the data is the syntax element, each syntax element is composed of a plurality of bits, and it represents a specific physical meaning, for example: macroblock type, quantization parameter, etc. Syntax characterizes the organizational structure of the syntax elements, and semantics describes the specific meaning of the syntax elements. All video coding standards specify the workflow of coding by defining syntax and semantics. The syntax elements generated in the encoding process will be described in detail below with reference to fig. 2.

The residual is the accurate degree of the motion search, and the more accurate the search is, the smaller the residual is for the same video.

The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

Fig. 1 shows a diagram of an overall structure of a content supply system 11000 for providing content. As shown in fig. 1, devices within the content supply system 11000 can perform data transmission through the

wireless base stations

11700, 11800, 11900, and 12000.

The content supply system 11000 includes a plurality of independent devices. A plurality of independent devices such as a computer 12100, a Personal Digital Assistant (PDA)12200, a video camera 12300, and a mobile phone 12500 are connected to the internet 11100 via an internet service provider 11200, a communication network 11400, and

wireless base stations

11700, 11800, 11900, and 12000.

However, the content supply system 11000 is not limited to that shown in fig. 1, and furthermore, these devices may be selectively connected to the content supply system 11000. A plurality of independent devices may be directly connected to the communication network 11400 without via the

wireless base stations

11700, 11800, 11900, and 12000.

The video camera 12300 is an imaging device, such as a digital video camera, capable of capturing video images. The mobile phone 12500 may utilize at least one communication method of various protocols (e.g., Personal Digital Communication (PDC), Code Division Multiple Access (CDMA), wideband code division multiple access (W-CDMA), global system for mobile communications (GSM), and Personal Handyphone System (PHS)).

The video camera 12300 may be connected to the streaming server 11300 via a wireless base station 11900 and a communication network 11400. The streaming server 11300 allows content received from a user via the video camera 12300 to be streamed via real-time broadcast. The content received from the video camera 12300 may be encoded using the video camera 12300 or the streaming server 11300. Video data captured by the video camera 12300 can be sent to the streaming server 11300 via computer 12100.

Video data captured by the camera 12600 can also be sent to the streaming server 11300 via the computer 12100. Like a digital camera, the camera 12600 is an imaging device capable of capturing both still images and video images. Video data captured by camera 12600 may be encoded using camera 12600 or computer 12100. The software that performs encoding and decoding on the video may be stored in a computer-readable recording medium (e.g., a CD-ROM disk, a floppy disk, a hard drive, an SSD, or a memory card) that is accessible by the computer 12100.

Video data may be received from the mobile phone 12500 if captured by a camera built into the mobile phone 12500. Video data may also be encoded by a large scale integrated circuit (LSI) system installed in the video camera 12300, the mobile phone 12500, or the camera 12600.

The content provision system 1100 may encode content data (e.g., content recorded during a concert) recorded by a user using the video camera 12300, the camera 12600, the mobile phone 12500, or another imaging device, and transmit the encoded content data to the streaming server 11300. The streaming server 11300 can transmit the encoded content data of the streaming content type to other clients that request the content data.

The client is a device capable of decoding encoded content data, for example, a computer 12100, a PDA12200, a video camera 12300 or a mobile phone 12500. Accordingly, the content supply system 11000 allows a client to receive and reproduce encoded content data. Further, the content supply system 11000 allows the client to receive encoded content data in real time and decode and reproduce the encoded content data, thereby enabling personal broadcasting.

The encoding operations of the plurality of independent devices included in the content provision system 11000 may be similar to that of the video encoding device according to an embodiment of the present application. An encoding operation of the video encoding apparatus according to an embodiment of the present application will be described in detail below with reference to fig. 2 to 4.

Fig. 2 illustrates a structural diagram of syntax elements in the H264 protocol according to an exemplary embodiment of the present application.

In an image encoding method such as Moving Picture Experts Group (MPEG) -1, MPEG-2, and MPEG-4h.264/MPEG-4AVC (advanced video coding), an image is divided into blocks each having a predetermined size, and residual data of each block is obtained by using an inter prediction or intra prediction process. The residual data is compressed by processes such as transform, quantization, scanning, run-length coding, and entropy coding. The syntax elements are organized into hierarchical structures and respectively describe information of each hierarchy.

As shown in fig. 2, encoding video actually encodes a video sequence, that is, each frame of image in the video sequence is encoded, and then each frame of image is divided into slices, and each slice is divided into macroblocks, and each macroblock is divided into a plurality of sub-blocks. Thus, in H264, the syntax elements are co-organized into five levels of sequence, picture, slice, macroblock, sub-macroblock.

The hierarchical structure of syntax elements helps to save codestreams more efficiently. For example, in an image, there are often the same data between slices, and if each slice carries the data at the same time, the waste of the code stream is inevitable. It is more efficient to extract the common information of the picture to form picture-level syntax elements, and to carry only the syntax elements unique to the slice itself at the slice level and to encode these unique syntax elements. When encoding is performed on each slice, each slice may be divided into macroblocks. A prediction mode is then determined for each macroblock, and encoding is performed on each macroblock based on the determined prediction mode. For example, it is determined that a certain macroblock adopts an inter prediction mode or an intra prediction mode. When a certain macroblock is determined to be inter prediction, it is determined that the attribute of the macroblock is determined to be motion search, and if a certain macroblock is determined to be intra prediction, it is further determined that the prediction attribute, that is, one of the above-mentioned transform, quantization, scanning, run coding, and entropy coding, is determined for each subblock within the macroblock. It can thus be seen that the syntax elements include not only the hierarchical structure of the video sequence but also the prediction properties.

Based on the fact that obtaining the syntax element corresponding to each video requires a high computational capability of a processor and generates a large power consumption, exemplary embodiments of the present application may determine the syntax element of the video before encoding the video, and then encode the video using the obtained syntax element, thereby achieving technical effects of high compression rate, fast encoding speed, and low power consumption, which will be described in detail below with reference to fig. 3.

Fig. 3 is a flowchart of a data processing method according to an exemplary embodiment of the present application. As shown in fig. 3, at step S310, a first syntax element corresponding to original data, which may include images and/or videos acquired via a plurality of independent devices as described in fig. 1, is acquired. Specifically, the first syntax element corresponding to the original data may be acquired by a hardware device, that is, the step S310 may be performed using a chip dedicated to acquiring the syntax element, wherein the chip may employ an ASIC chip. By using hardware to perform this step, the processing speed can be significantly increased.

Since the electronic device is usually provided with a hardware encoder, the step S310 can be executed by using the hardware encoder, so that the components in the electronic device can be reasonably utilized, and the purpose can be achieved without additionally adding components. In the case of using the hardware encoder, the hardware encoder may be used to encode the original data into first encoded data, where the first encoded data corresponds to a data stream that includes not only the first syntax element but also data obtained by encoding the original data information based on the first syntax element. The first syntax element is then parsed from the first encoded data using a software decoder. For example, FFMPEG (an open source computer program that can be used to perform decoding) can be modified to a software encoder that satisfies the functionality of extracting only the first syntax element in the present application. That is, the software decoder need not fully encode the first encoded data, but only extract the first syntax element from the first encoded data.

Subsequently, in step S320, it is determined whether the first syntax element satisfies a preset condition, wherein the preset condition may be a condition for weighing/evaluating the first syntax element. That is, it may be determined whether the first syntax element satisfies a user requirement, and if the user requirement is satisfied, step S330 may be performed to encode the original data directly using the first syntax element, and if the user requirement is not satisfied, the second syntax element may be generated after performing an optimization operation on the first syntax element, and then encoding the original data using the second syntax element.

In the case where the first syntax element is parsed from the first encoded data using a software decoder, the preset condition may be an index of the first encoded data, for example, the preset condition may be a compression rate of the first encoded data. The first syntax element does not satisfy a preset condition if a compression rate of the first encoded data does not satisfy the preset condition, for example, the compression rate of the first encoded data is too low. In this case, the second syntax element is generated by upgrading the first syntax element using a preset encoding manner.

For example, a plurality of schemes for prediction attributes of the macroblock may be preset, and if the first syntax element does not satisfy a preset condition, one scheme may be selected from the plurality of schemes, and the first syntax element may be optimized using the scheme to generate the second syntax element. The original data may then be encoded into third encoded data using a software encoder based on the second syntax element. For example, OpenH264 (an open source computer program that can be used to perform the encoding) can be modified to meet the effect of performing the encoding using the second syntax element already in the present application. Therefore, the software encoder does not need to determine the hierarchical structure and the prediction attribute of the original data in the encoding process, a large amount of operations are saved, and the encoding speed is high, the power consumption is low and the compression rate is high.

Alternatively, the first syntax element may be converted into the second syntax element according to a protocol, that is, when it is determined that the first syntax element does not satisfy the second protocol after the first syntax element corresponding to the original data is acquired according to the first protocol (e.g., H264) in step S310, the first syntax element may be modified into the second syntax element corresponding to the second protocol according to the second protocol (e.g., H265). The original data may then be encoded according to the second syntax element.

An exemplary embodiment of the present application provides a data processing method, including: encoding the original data into first encoded data using a first type of encoder; parsing a first syntax element from the first encoded data; encoding is performed on the original data with a second-type encoder based on the first syntax element.

Optionally, the first type of encoder is a hardware encoder; the second type of encoder is a software encoder.

Optionally, parsing the first syntax element from the first encoded data comprises: a first syntax element is parsed from the first encoded data using a software decoder. In summary, the data processing method according to the exemplary embodiment of the present application may encode the original data when determining whether the first syntax element meets the requirement, so that the encoded data better meets the user requirement. Further, hardware may be utilized to obtain the first syntax element, which may significantly increase processing speed. Further, the first syntax element can be obtained by using a built-in hardware encoder and a software decoder, and the purpose of the present application can be achieved without adding additional elements. Furthermore, the first syntax element can be upgraded to a second syntax element, and then the second syntax element is used for encoding the original data, so that the encoded data more conforms to the requirements of users.

In order to more clearly understand the inventive concept of the exemplary embodiment of the present application, a block diagram of a data processing apparatus of the exemplary embodiment of the present application will be described below with reference to fig. 4. Those of ordinary skill in the art will understand that: the apparatus in fig. 4 shows only components related to the present exemplary embodiment, and common components other than those shown in fig. 4 are also included in the apparatus.

Fig. 4 shows a block diagram of a data processing device of an exemplary embodiment of the present application. Referring to fig. 4, the apparatus includes, at a hardware level, a processor, an internal bus, and a computer-readable storage medium, wherein the computer-readable storage medium includes volatile memory and non-volatile memory. The processor reads the corresponding computer program from the non-volatile memory and then runs it. Of course, besides the software implementation, the present application does not exclude other implementations, such as logic devices or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, and may also be hardware or logic devices.

Specifically, the processor performs the following operations: acquiring a first syntax element corresponding to original data; judging whether the first syntax element meets a preset condition or not; and if the preset condition is met, encoding the original data by utilizing the first syntax element.

Optionally, the processor, in the step of implementing, obtaining a first syntax element corresponding to the original data includes: a first syntax element corresponding to original data is obtained with a hardware device.

Optionally, the hardware device comprises a hardware encoder.

Optionally, the processor, in implementing the step, obtaining, with the hardware device, a first syntax element corresponding to the original data includes: encoding original data into first encoded data by using a hardware encoder; a first syntax element is parsed from the first encoded data using a software decoder.

Optionally, the processor performing, in the implementing step, encoding of the original data using the first syntax element includes: the original data is encoded into second encoded data using a software encoder based on the first syntax element.

Optionally, the processor may further implement the steps of: and if the first syntax element is determined not to meet the preset condition, upgrading the first syntax element to a second syntax element.

Optionally, the processor in implementing step upgrading the first syntax element to the second syntax element comprises: and upgrading the first syntax element by using a preset coding mode to generate a second syntax element.

Optionally, the processor, after implementing the upgrading of the first syntax element into the second syntax element, further includes: the original data is encoded into third encoded data using a software encoder based on the second syntax element.

According to an exemplary embodiment of the present application, there is provided a data processing apparatus including: a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to: encoding the original data into first encoded data using a first type of encoder; parsing a first syntax element from the first encoded data; encoding is performed on the original data with a second-type encoder based on the first syntax element.

Optionally, the parsing, by the processor, the first syntax element from the first encoded data in the implementing step includes: a first syntax element is parsed from the first encoded data using a software decoder.

In summary, the data processing apparatus according to the exemplary embodiment of the present application may encode the original data when determining whether the first syntax element meets the requirement, so that the encoded data better meets the user requirement. Further, hardware may be utilized to obtain the first syntax element, which may significantly increase processing speed. Further, the first syntax element can be obtained by using a built-in hardware encoder and a software decoder, and the purpose of the present application can be achieved without adding additional elements. Furthermore, the first syntax element can be upgraded to a second syntax element, and then the second syntax element is used for encoding the original data, so that the encoded data more conforms to the requirements of users.

It should be noted that the execution subjects of the steps of the method provided in embodiment 1 may be the same device, or different devices may be used as the execution subjects of the method. For example, the execution subject of steps 21 and 22 may be device 1, and the execution subject of step 23 may be device 2; for another example, the execution subject of step 21 may be device 1, and the execution subjects of steps 22 and 23 may be device 2; and so on.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A data processing method, comprising:

acquiring a first syntax element corresponding to original data;

judging whether the first syntax element meets a preset condition or not;

and if the preset condition is met, encoding the original data by utilizing the first syntax element.

2. The method of claim 1, wherein obtaining a first syntax element corresponding to original data comprises: a first syntax element corresponding to original data is obtained with a hardware device.

3. The method of claim 2, wherein the hardware device comprises a hardware encoder.

4. The method of claim 3, wherein obtaining, with a hardware device, a first syntax element corresponding to original data comprises:

encoding original data into first encoded data by using a hardware encoder;

a first syntax element is parsed from the first encoded data using a software decoder.

5. The method of claim 1, wherein performing encoding on the original data using the first syntax element comprises: the original data is encoded into second encoded data using a software encoder based on the first syntax element.

6. The method of claim 1, further comprising:

and if the first syntax element is determined not to meet the preset condition, upgrading the first syntax element to a second syntax element.

7. The method of claim 6, wherein promoting the first syntax element to the second syntax element comprises:

and upgrading the first syntax element by using a preset coding mode to generate a second syntax element.

8. The method of claim 6, wherein after upgrading the first syntax element to the second syntax element, further comprising:

the original data is encoded into third encoded data using a software encoder based on the second syntax element.

9. A computer readable storage medium having computer instructions stored thereon that, when executed, implement the method of any of claims 1 to 8.

10. A data processing apparatus, comprising:

a processor; and

a memory arranged to store computer executable instructions that, when executed, cause the processor to perform the method of any of claims 1 to 8.

11. A data processing method, comprising:

encoding the original data into first encoded data using a first type of encoder;

parsing a first syntax element from the first encoded data;

encoding is performed on the original data with a second-type encoder based on the first syntax element.

12. The method of claim 11, wherein the first type of encoder is a hardware encoder; the second type of encoder is a software encoder.

13. The method of claim 11, wherein parsing the first syntax element from the first encoded data comprises: a first syntax element is parsed from the first encoded data using a software decoder.

14. A computer readable storage medium having computer instructions stored thereon which, when executed, implement the method of any of claims 11 to 13.

15. A data processing apparatus, comprising:

a processor; and

a memory arranged to store computer executable instructions that, when executed, cause the processor to perform the method of any of claims 11 to 13.