CN114071145A

CN114071145A - Image frame encoding method, object searching method, computer device, and storage medium

Info

Publication number: CN114071145A
Application number: CN202010768892.5A
Authority: CN
Inventors: 黄玄; 杨名远; 叶琰
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2020-08-03
Filing date: 2020-08-03
Publication date: 2022-02-18
Also published as: WO2022028283A1

Abstract

The embodiment of the application discloses an image frame coding method, an object searching method, computer equipment and a storage medium. The image frame encoding method includes: dividing a target image frame into a plurality of first data blocks; inputting the first data blocks into a plurality of encoding processing units which are executed in sequence in batches; at least two encoding processing units synchronously execute processing in a part of time period of the processing executed by the plurality of encoding processing units; and coding the target image frame according to the coding parameters respectively corresponding to the plurality of first data blocks. Compared with the mode of sequentially executing by taking the video frame as a unit, the method can greatly shorten the time consumed by determining the encoding parameters, improve the efficiency of video encoding and simultaneously reduce the processing delay of the data block. Because the hardware unit is subjected to resource multiplexing in a mode of parallel operation of the multiple coding units, the chip area can be reduced, the hardware overhead can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

Description

Image frame encoding method, object searching method, computer device, and storage medium

Technical Field

The present application relates to the field of data processing technologies, and in particular, to an image frame encoding method, an image searching method, an object searching method, a computer device, and a computer-readable storage medium.

Background

The current video technology is rapidly developed and widely applied, and when the video is transmitted in a network, video images are coded and then used in various video application scenes based on the purposes of bandwidth limitation and flow saving.

Video coding refers to converting a file in an original video format into a file in another video format by performing discrete cosine transform, quantization, entropy coding and other processing on video image frames. The method aims to improve the compression rate of the video as much as possible and reduce the volume of the video file on the premise of ensuring the encoding quality.

Therefore, through video pre-analysis, selecting proper video has important effects on improving video quality and controlling video compression rate.

Disclosure of Invention

In view of the above, the present application is made to provide an image processing method, an image searching method, an object searching method, and a computer device, computer-readable storage medium that overcome or at least partially solve the above problems.

According to an aspect of the present application, there is provided an image frame encoding method including:

dividing a target image frame into a plurality of first data blocks;

inputting the first data blocks into a plurality of encoding processing units which are executed in sequence in batches; after the coding processing unit finishes processing the first data blocks of the previous batch, processing the first data blocks of the next batch, wherein the coding processing unit correspondingly executes one or more steps in the determination process of the coding parameters of the target image frame, and at least two coding processing units synchronously execute processing in partial time periods of the processing executed by the coding processing units;

and coding the target image frame according to the coding parameters respectively corresponding to the plurality of first data blocks.

According to an aspect of the present application, there is provided a video encoding method including:

dividing a target image frame in a video into a plurality of first data blocks;

inputting the first data blocks into a plurality of encoding processing units which are executed in sequence in batches; after the coding processing unit finishes processing a previous batch of first data blocks, processing a next batch of first data blocks, wherein the coding processing unit correspondingly executes one or more steps in the process of determining coding parameters of an image frame, and at least two coding processing units synchronously execute processing in partial time periods of processing executed by the coding processing units;

coding the target image frame according to coding parameters respectively corresponding to the plurality of first data blocks;

and obtaining the coding result of the video according to the coding result of the image frame.

dividing a specific area of a target image frame in a video into a plurality of first data blocks, wherein the target area is an area where the content of the target image frame changes relative to a reference image frame;

inputting the plurality of first data blocks into four encoding processing units which are executed in sequence in batches; the four coding processing units respectively execute quantization parameter calculation in the process of determining coding parameters of image frames according to a reading step, an information quantity obtaining step, a quantization parameter and information quantity calculation step and a quantization parameter and information quantity writing step, and at least two coding processing units synchronously execute processing in partial time periods of the processing executed by the four coding processing units;

coding the target image frame according to coding parameters respectively corresponding to the four first data blocks;

According to an aspect of the present application, there is provided an image frame processing method including:

dividing a target image frame into a plurality of first data blocks;

inputting the first data blocks into a plurality of processing units which are executed in sequence in batches; and after the processing unit finishes the processing of the first data blocks of the previous batch, the processing unit executes the processing of the first data blocks of the next batch, the processing unit correspondingly executes one or more steps in the processing process of the target image frame, and at least two encoding processing units synchronously execute the processing in partial time periods of the processing executed by the encoding processing units.

According to an aspect of the present application, there is provided an image frame processing method applied to a software service platform, including:

acquiring at least one target video submitted by a software service client;

dividing an image frame of a target video into a plurality of first data blocks;

obtaining the coding result of the target video according to the coding results of the plurality of image frames;

providing an encoding result of the target video based on the software service client.

According to an aspect of the present application, there is provided an image frame encoding system, including a frame dividing module, a plurality of encoding processing units and an encoding module, which are executed in sequence;

the frame dividing unit is used for dividing a target image frame into a plurality of first data blocks and inputting the first data blocks into a plurality of coding processing units in batches;

the encoding processing unit is used for correspondingly executing one or more steps in the process of determining the encoding parameters of the target image frame, executing the processing of the first data blocks of the next batch after the processing of the first data blocks of the previous batch is executed, and synchronously executing the processing by at least two encoding processing units in a part of time periods of the processing executed by the encoding processing units;

and the encoding unit is used for encoding the target image frame according to the encoding parameters respectively corresponding to the plurality of first data blocks.

According to one aspect of the application, a chip based on a field programmable gate array is provided, which comprises a frame dividing module, a plurality of coding processing units and a coding module, wherein the coding processing units and the coding module are executed in sequence;

In accordance with an aspect of the present application, there is provided an electronic device including: a processor; and

a memory having executable code stored thereon, which when executed, causes the processor to perform a method as in any one of the above.

According to an aspect of the application, there is provided one or more machine-readable media having stored thereon executable code that, when executed, causes a processor to perform a method as any one of the above.

According to the embodiment of the application, a plurality of coding processing units are adopted to execute the determining process of the coding parameters, a single coding processing unit executes one or more steps in the determining process of the coding parameters, when the coding parameters are calculated for an image frame, the image frame is divided into data blocks, the plurality of coding processing units which are executed in sequence are input in batches, the coding parameters can be obtained, and then the image frame is coded by utilizing the coding parameters. Because the coding processing units can execute the processing of the next data block after processing the previous data block, at least two coding processing units synchronously execute the processing in a part of time period of the processing executed by the coding processing units, thereby simultaneously executing the processing of the data blocks. Because the hardware unit is subjected to resource multiplexing in a mode of parallel operation of the multiple coding units, the chip area can be reduced, the hardware overhead can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

Moreover, the processing scheme of the multi-coding processing unit in the embodiment of the application has strong universality, can be copied to different coding systems for use, and can configure the coding processing unit to be compatible with other processing procedures, so that the optimization processing of a plurality of processing procedures is realized, and the processing efficiency of the system is improved.

The foregoing description is only an overview of the technical solutions of the present application, and the present application can be implemented according to the content of the description in order to make the technical means of the present application more clearly understood, and the following detailed description of the present application is given in order to make the above and other objects, features, and advantages of the present application more clearly understandable.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:

fig. 1 is a schematic diagram illustrating an FPGA-based video encoding process according to an embodiment of the present application;

fig. 2 shows a flow chart of an embodiment of an image frame encoding method according to a first embodiment of the present application;

fig. 3 shows a flow chart of an embodiment of an image frame encoding method according to the second embodiment of the present application;

fig. 4 shows a flow chart of an embodiment of an image frame encoding method according to a third embodiment of the present application;

fig. 5 shows a flow chart of an embodiment of a video encoding method according to the fourth embodiment of the present application;

fig. 6 shows a flow chart of an embodiment of a video encoding method according to the fifth embodiment of the present application;

fig. 7 shows a flowchart of an embodiment of an image frame processing method according to the sixth embodiment of the present application;

fig. 8 is a block diagram illustrating an embodiment of an image frame encoding apparatus according to a seventh embodiment of the present application;

fig. 9 is a block diagram illustrating an embodiment of a video encoding apparatus according to an eighth embodiment of the present application;

FIG. 10 is a block diagram of an embodiment of a video encoding apparatus according to the ninth embodiment of the present application;

fig. 11 is a block diagram illustrating a structure of an embodiment of an image frame processing apparatus according to a tenth embodiment of the present application;

FIG. 12 is a block diagram of an embodiment of an image frame encoding system according to the eleventh embodiment of the present application;

FIG. 13 is a block diagram illustrating an embodiment of a field programmable gate array based chip according to a twelfth embodiment of the present application;

FIG. 14 is a schematic diagram illustrating an image frame encoding flow;

fig. 15 illustrates an exemplary system that can be used to implement various embodiments described in this disclosure.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

To better understand the solution of the present application, the following explains the concept involved in the present application:

the encoding parameters are used to encode the video data, and may specifically include data such as a frame type and a Quantization Parameter (QP). The quantization parameter has an important influence on the video quality, and for each encoding object, the smaller the quantization parameter is, the higher the quantization precision is, and the better the video quality is, but at the same time, the bandwidth cost of the encoder will also increase. Therefore, the quantization parameters are reasonably distributed, so that the video quality is ensured, and meanwhile, the bandwidth cost is saved.

Based on the importance of the encoding parameters, a proper group of encoding parameters are selected for an encoder by pre-analyzing the video to be encoded, so that the video quality is improved, and the compression rate of encoding is controlled.

The method and the device adopt a Curree (block tree) algorithm to calculate the coding parameters, the Curree algorithm has the basic principle that reasonable coding parameters are distributed to each block by analyzing the relative information quantity of each block and other image frames of the current image frame, and the Curree algorithm greatly improves the video quality.

Generally, when the curree algorithm is used, the relative relationship between the current image frame and the reference image frame needs to be calculated, so that the iteration times are too many, the calculation amount is too large, the processing speed is slow when the image frames are calculated one by one, the high-resolution video cannot be processed in real time, and large resource consumption and high cost are caused when hardware is implemented.

In view of the above problems, in the processing procedure of the curree algorithm, a plurality of coding processing unit stages are adopted to execute a determination procedure of a coding parameter, and a single coding processing unit executes one or more steps in the determination procedure of the coding parameter. The coding processing unit is a unit main body for processing different steps, can be understood as a pipeline for executing corresponding functions of each step from an application level, and distributes the determination process of the coding parameters to a plurality of pipelines with a sequence for execution.

In an alternative embodiment, the process of determining the encoding parameters may be divided into four steps, namely a parameter reading step, an information amount obtaining step, a data calculating step, and a data interacting step. The parameter reading step and the information quantity obtaining step are used for determining the used data when the coding parameters are calculated, the data calculating step completes the calculation of the coding parameters, and the data interaction step stores the coding parameters for calling when the coding is carried out subsequently. In a specific implementation, the encoding processing units are respectively used to execute the above steps, for example, in an implementation manner, four encoding processing units may be arranged to respectively execute the above four steps. It can be understood that, in a specific implementation, different step division manners may be provided for the determination process of the encoding parameter, and the number of the encoding processing units may also be set according to actual requirements.

When the coding parameters are calculated for the image frame to be processed, the image frame is divided into data blocks, and the image frame to be processed is corresponding to the first data block and the reference image frame is corresponding to the second data block because the data blocks corresponding to the image frame to be processed and the reference image frame are related. The first data blocks are input into a plurality of coding processing units which are executed in sequence in batches, coding parameters corresponding to the data blocks can be obtained after the processing of the plurality of coding processing units, and then the image frames are coded by utilizing the coding parameters.

In the embodiment of the application, the encoding processing unit processes the data blocks entering the same batch at the same time, and before one data block is processed by the plurality of encoding processing units, the processing of the next data block in the encoding processing unit is started, so that the processing of the data blocks in the plurality of batches can be simultaneously carried out. In order to achieve higher efficiency of simultaneous processing of a plurality of data blocks, the coding processing units are designed to execute the processing of the next data block after processing the previous data block, and at least two coding processing units synchronously execute the processing in a part of the time period in which the plurality of coding processing units execute the processing. It can be understood that, in this processing mode, the processing time of the multiple data blocks corresponding to the target image frame to be processed in the encoding processing unit is the sum of the processing time of the first data block in the first order in all the encoding processing units and the processing time of the first data block in the last encoding processing unit, which greatly improves the processing efficiency compared with the frame-by-frame processing mode.

Therefore, compared with a mode of sequentially executing by taking a video frame as a unit, the method and the device for determining the coding parameters can greatly shorten the time consumed by determining the coding parameters, improve the efficiency of video coding (improve the display by several times compared with the prior art), and reduce the processing delay of the data blocks. Because the hardware unit is subjected to resource multiplexing in a mode of parallel operation of the multiple coding units, the chip area can be reduced, the hardware overhead can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

Due to the fact that motion correlation exists between image frames in the video, namely, partial change content and partial same content exist between the image frames, the target image frame can be represented as an image content difference (which can be represented as a transfer information amount) between a certain image frame serving as a reference image frame, and conversely, the target image frame is obtained according to the image content difference and the reference image frame. Therefore, when the target image frame is divided into data blocks, a specific area of the target image frame can be divided into a plurality of first data blocks, wherein the target area is an area where the content of the target image frame is changed relative to the reference image frame.

The above-mentioned inputting the first data block into the coding processing unit in batches may be a plurality of data blocks as one batch, or may be a data block as one batch, and all of them may achieve the effect of executing processing on data blocks of different batches at the same time. When a plurality of data blocks enter the coding processing unit for processing, because the data blocks processed earlier need to wait for the data blocks processed later, time is wasted, therefore, one data block is used as a batch, a plurality of first data blocks are input into the data processing unit one by one for data processing, and the processing efficiency can be improved better.

Compared with the scheme without the Curree technology, the video quality can be obviously improved, and the coding code rate is reduced. Through statistics, the coding system adopting the structure can averagely save 10-15% of coding code rate on the premise of equal video quality.

The above describes the determination process of the encoding parameters by a plurality of encoding processing units, and the video encoding process needs to update the total information amount of the reference image frame in real time in addition to the encoding parameters, so as to ensure that the information amount of the reference image frame used by each encoding unit is the latest accurate data. Correspondingly, the processing procedure of the curree algorithm is divided into two parts, one of which is to determine a coding parameter (mainly a quantization parameter) and calculate the quantization parameter of the first data block according to the information amount and the related information of the current first data block. The second is superposition information amount, which is to determine the transfer information amount of a first data block, and superimpose the transfer information amount of the first data block to the initial information amount of a second data block corresponding to a reference image frame according to a motion vector to obtain the total information amount of the second data block, which may also be referred to as curree information amount, different from the information amount in the conventional sense, if a block is superimposed more times, the information amount is larger, the correspondingly determined quantization parameter is smaller, which means that the important degree of the block is high, and high-quality coding needs to be adopted.

Therefore, the encoding processing unit may further perform one or more steps in the process of determining the total information amount of the reference image frame of the target image frame, the parameter reading step and the information amount obtaining step may further determine the used data corresponding to the information of the read reference image frame, and the data interaction step may further be configured to store the reference real information amount for calling during encoding.

The calculation of the total information amount of the reference frame may be performed before the determination of the encoding parameters, and the execution sequence may be set according to actual needs.

In a specific implementation, other calculation processes may also be implemented by using the execution mode of the multi-coding processing unit of the present application, and a specific implementation process is similar to the execution process described above and is not described herein again.

The embodiment of the present application may be deployed on a processing chip, for example, an FPGA (Field Programmable Gate Array) chip, where the chip may be further deployed on a video encoding device, the video encoding device has a certain computing capability, and the video encoding device may be a user side terminal, a conventional server, a cloud host, a virtual center, and the like. The video coded by the chip can be further transmitted through a network and is used for terminal playing, video processing and the like of video application.

Referring to fig. 1, a schematic diagram of implementing a video encoding process based on an FPGA according to an embodiment of the present application is shown. Four encoding processing units, namely a first encoding processing unit, a second encoding processing unit, a third encoding processing unit and a fourth encoding processing unit, are deployed on the FPGA chip and respectively and correspondingly execute a parameter reading step, an information quantity obtaining step, a data calculating step and a data interaction step in the determining process of encoding parameters.

The target image frame input to the FPGA chip is first divided into a plurality of first data blocks, and the video frame is divided into five first data blocks, namely CU0, CU1, CU2, CU3 and CU 4. And inputting the five first data blocks into a plurality of coding processing units in batches, and executing the processing of the next batch of first data blocks after the processing of the first data blocks of the previous batch. As shown in fig. 1, the first data block CU0 with the top of the sequence enters the first coding processing unit to perform the parameter reading step, and no processing task exists in any other coding processing unit at this time, the CU0 continues to enter the second coding processing unit for processing after the first coding processing unit finishes processing, and the idle first coding processing unit continues to process the CU 1. And for each CU, sequentially processing each CU in each coding processing unit, and continuously processing the next CU after processing the previous CU. After all the encoding processing units are processed, the CU obtains the corresponding encoding parameters of each pair, and the encoding result of the image frame can be further obtained according to the encoding result of each CU.

In a partial time period in which the plurality of encoding processing units perform processing, at least two encoding processing units perform processing synchronously, and as can be seen from fig. 1, processing procedures of the respective encoding units for different data blocks are simultaneously performed except for processing of CU0 in the first encoding processing unit and processing of CU4 in the fourth encoding unit, such as processing of CU4 in the first encoding processing unit, processing of CU3 in the second encoding processing unit, processing of CU2 in the third encoding unit, and processing of CU1 in the fourth encoding unit. The total processing time is the sum of: the processing time of the CU0 at the four encoding processing units, and the sum of the processing times of the CU1, the CU2, the CU3, and the CU4 at the fourth encoding processing unit. Compared with the mode that each data block is processed in sequence, the processing time is saved. Assuming that the processing time of each data block in the encoding processing unit is the same and is X duration, this example takes 8X duration, and the frame-by-frame processing takes 20X duration.

It should be noted that the above application examples are provided for the purpose of facilitating understanding of the method provided by the embodiments of the present application, and are not intended to limit the method.

Various embodiments of the present application are described next.

Referring to fig. 2, a flowchart of an embodiment of an image frame encoding method according to a first embodiment of the present application is shown, where the method may specifically include the following steps:

step 101, dividing a target image frame into a plurality of first data blocks.

A block CU is a smaller coding unit divided from the image frame. The dividing effect may be set according to actual needs, including the size, shape, area, number of data blocks, and the like of each data block, and for example, the data blocks may be divided into 8 × 8 or 16 × 16 data blocks. And the larger data block can be divided into smaller data blocks. The specific division method may be selected according to actual requirements, for example, a quadtree division method may be adopted.

To better accommodate image details, image regions of complex details may be partitioned into smaller data block sizes, while image regions of simple parts may be partitioned into larger data block sizes.

It can also be decided with reference to rate-distortion optimization during the division of the data block. For a larger data block, it can be divided into smaller data blocks, in the process, the sum of the rate-distortion cost of the larger data block and the rate-distortion cost of the smaller data block is calculated, if the latter is small, the division of the smaller data block will be performed.

Step 102, inputting the plurality of first data blocks in batches into a plurality of encoding processing units which are executed in sequence; and after the coding processing unit finishes processing the first data blocks of the previous batch, processing the first data blocks of the next batch, wherein the coding processing unit correspondingly executes one or more steps in the determination process of the coding parameters of the target image frame, and at least two coding processing units synchronously execute processing in partial time periods of the processing executed by the coding processing units.

The encoding processing unit processes the data blocks entering the same batch at the same time, and before one data block is processed by the plurality of encoding processing units, the next data block is started to be processed in the encoding processing unit, that is, the processing of the plurality of batches of data blocks can be simultaneously carried out.

The first data block is input to the encoding processing unit in batches, and a plurality of data blocks may be used as one batch, or one data block may be used as one batch.

In order to achieve higher efficiency of simultaneous processing of a plurality of batches of data blocks, the coding processing unit is designed to execute the processing of the next batch of data blocks after the previous batch of data blocks are processed. Compared with a frame-by-frame processing mode, the processing efficiency is greatly improved.

In a specific implementation, the control signal can also ensure that the coding processing units work independently and uninterruptedly.

And 103, coding the target image frame according to the coding parameters respectively corresponding to the plurality of first data blocks.

Each first data block may be encoded by using the encoding parameter, and the encoding result of the target image frame may be obtained according to the encoding result of each data block.

Referring to fig. 3, a flowchart of an embodiment of an image frame encoding method according to the second embodiment of the present application is shown, where the method specifically may include the following steps:

step 201, dividing a specific area of the target image frame into a plurality of first data blocks, wherein the target area is an area where the content of the target image frame changes relative to a reference image frame.

Based on the existence of motion correlation between image frames in the video, a target image frame can be represented as an image content difference between a certain image frame as a reference image frame, and accordingly, when the target image frame is divided into data blocks, an area where the content of the target image frame changes relative to the reference image frame can be determined, and the changed area is further divided into a plurality of first data blocks.

Step 202, inputting the plurality of first data blocks into a data processing unit one by one for data processing; and after the coding processing unit finishes processing the first data blocks of the previous batch, processing the first data blocks of the next batch, wherein the coding processing unit correspondingly executes one or more steps in the determination process of the coding parameters of the target image frame, and at least two coding processing units synchronously execute processing in partial time periods of the processing executed by the coding processing units.

Using one data block as one batch can improve the processing efficiency better than using a plurality of data blocks as one batch.

And 203, coding the target image frame according to the coding parameters respectively corresponding to the plurality of first data blocks.

Referring to fig. 4, a flowchart of an embodiment of an image frame encoding method according to a third embodiment of the present application is shown, where the method may specifically include the following steps:

in step 301, a target image frame is divided into a plurality of first data blocks.

Step 302, inputting the plurality of first data blocks in batches into a plurality of encoding processing units which are executed in sequence; and after the coding processing unit finishes the processing of the first data blocks of the previous batch, the processing of the first data blocks of the next batch is executed, the data processing unit correspondingly executes a parameter reading step, an information quantity acquisition step, a data calculation step and a data interaction step in the process of determining the coding parameters of the target image frame, and at least two coding processing units synchronously execute the processing in partial time periods of the processing executed by the four coding processing units.

Step 303, encoding the target image frame according to the encoding parameters respectively corresponding to the plurality of first data blocks.

In this embodiment, the processing content of each encoding processing unit is specifically determined, and the determination process of the encoding parameters is divided into a parameter reading step, an information amount obtaining step, a data calculating step, and a data interaction step. The number of the encoding processing units and the specific processing content can be set according to actual needs, and one encoding processing unit can process one or more steps. For example, three encoding processing units are designed, a first encoding processing unit processes the parameter reading step and the information amount obtaining step, a second encoding processing unit processes the data calculating step, and a third encoding processing unit processes the data interacting step.

In an optional embodiment of the present application, the parameter reading step includes: and reading the prediction parameters of the first data block, wherein the prediction parameters comprise the prediction direction and the prediction cost in the target image frame encoding process. The prediction direction may be intra-frame prediction or inter-frame prediction, or intra-frame prediction and inter-frame prediction may be combined, and the prediction cost measures deviation data caused in the encoding prediction process.

The prediction of an image frame is a compression process adopted in the encoding process, and can be divided into intra-frame prediction and inter-frame prediction, which can also be called as inter-frame coding and intra-frame coding. Intra-frame prediction is based on the fact that due to strong correlation between pixels in the same image, compression coding can be performed by utilizing the correlation between pixels. For example, a certain pixel is used as a reference pixel, and the difference value between other pixels and the reference pixel is substituted for the other pixels, so that the compression effect on the other pixels can be achieved. For example, when data is transmitted, the actual pixel X (current value) and the reference pixel (predicted value) are subtracted from each other, a difference value is transmitted, and the receiving side obtains the actual pixel from the sum of the difference value and the reference pixel. The inter-frame prediction is based on the fact that there is strong correlation between two previous and next image frames in the video, for example, the reference image frame may be an image frame closest to the target image frame in a certain direction, and the target image frame is expressed by using the difference value between the target image frame and the reference image frame, so as to play a role in compressing the image frame.

In an optional embodiment of the present application, the information amount obtaining step includes: according to the information quantity storage address of the first data block, acquiring the initial information quantity of the first data block from a corresponding memory, and according to the prediction cost and the coding frame rate of the first data block, acquiring the intra-frame information quantity of the first data block; and determining the total information quantity of the first data block according to the intra-frame information quantity and the initial information quantity. For example, the prediction cost may be multiplied by the encoding frame rate to obtain the intra information amount of the first data block, and then the total information amount of the first data block may be obtained by adding the initial information amount to the intra information amount.

The initial information amount of the image frame is used for calculating the total information amount, and if other frames refer to the current image frame, the information amount of the other frames is superposed on the information amount of the image frame, so that the information amount of each frame is continuously increased in the process of Curree calculation until the superposition process is finished, and the total information amount is obtained and used for deriving the quantization parameter.

The initial information amount of the image frame may be calculated in advance by the number of pixels of the image and the number of distinguishable colors.

In an optional embodiment of the present application, the data calculating step includes: and determining the corresponding coding parameters for the first data block according to the read total information amount, the prediction direction and the prediction cost of the first data block. Taking the prediction direction as intra-frame prediction and the coding parameter as quantization parameter as examples, the prediction cost is intra-frame prediction cost, the total information amount of the first data block is multiplied by the coding frame rate coefficient, the coding frame rate coefficient is divided by the intra-frame prediction cost coefficient of the first data block, and log logarithm is taken for the result, so that the quantization parameter can be obtained.

In an optional embodiment of the present application, the parameter reading step includes: and acquiring the information quantity storage address of the first data block according to the position of the first data block in the image. The position of the data block in the image is firstly determined, and the information content storage address of the first data block can be determined according to the mapping relation between the position and the information content storage address.

Since the process of video encoding requires information amount according to the reference image frame in addition to using the encoding parameters, the determination process of the encoding parameters can also be realized by a plurality of encoding processing units.

Correspondingly, the information amount obtaining step may further include: and determining the amount of transfer information between the target image frame and the reference image frame according to the total information amount of the first data block. The information amount obtaining step may further obtain one of the bases for calculating the total information amount of the reference frame, that is, the total information amount of the first data block of the target image frame, and further determine the transfer information amount between the target image frame and the reference image frame, that is, the difference information amount therebetween, based on the total information amount of the first data block.

Correspondingly, the data calculation step further comprises: acquiring initial information quantity of a second data block in a reference image frame; and superposing the transfer information quantity of the first data block to the initial information quantity of a second data block in a reference image frame to obtain the total information quantity of the second data block in the reference image frame. Since the transfer information amount is also the difference information amount between the target image frame and the reference image frame, the transfer information amount can be superimposed on the initial information amount of the reference image frame, and the total information amount of the second data block in the reference image frame is obtained.

In an optional embodiment of the present application, the parameter reading step includes: and reading the prediction cost coefficient. The prediction cost coefficient is a coefficient for a prediction cost, that is, a prediction deviation or a coding deviation, and may be preset, or the prediction cost coefficient set corresponding to the range may be determined according to a numerical range of the prediction cost.

Accordingly, in an optional embodiment of the present application, the determining the amount of transfer information between the target image frame and the reference image frame according to the initial amount of information of the first data block includes: obtaining the intra-frame information quantity of the first data block according to the prediction cost and the coding frame rate of the first data block; determining the total information quantity of the first data block according to the intra-frame information quantity and the initial information quantity; and determining the transfer information quantity of the first data block according to the total information quantity and the prediction cost coefficient. The specific calculation formula can be set according to actual requirements, and the calculation process of the data is a known technology in the art, which is not described herein again.

In an optional embodiment of the present application, the parameter reading step further includes: and determining the position of a second data block of which the information amount is to be superposed in the reference image frame according to the position of the first data block in the target image frame and the motion vector. Here, the Motion Vector (MV) represents a relative displacement between the first data block and the second data block with the highest matching degree in the reference image frame image during the inter-frame coding process, and therefore, according to the position of the first data block in the target image and the relative displacement represented by the motion vector, the position of the second data block corresponding to the first data block in the reference image frame can be determined.

In an optional embodiment of the present application, the data interaction step includes: and storing the coding parameters to corresponding storage units for calling in the subsequent coding process.

In an optional embodiment of the present application, the data interaction step includes: and transmitting the total information quantity of the second data block in the reference image frame to other related coding processing units, and updating the data used by the other coding processing units in time, so that the data used by the other coding processing units is ensured to be the latest accurate data, and the problem that the obtained coding processing units are stopped due to untimely updating of the data of the memory when the total information quantity of the second data block in the reference image frame is accessed from the storage position is avoided.

Referring to fig. 5, a flowchart of an embodiment of a video encoding method according to the fourth embodiment of the present application is shown, where the method specifically includes the following steps:

step 401, dividing a target image frame in a video into a plurality of first data blocks.

Step 402, inputting the plurality of first data blocks in batches into a plurality of encoding processing units which are executed in sequence; and the coding processing unit executes the processing of the next batch of first data blocks after executing the processing of the previous batch of first data blocks, and the coding processing unit correspondingly executes one or more steps in the determination process of the coding parameters of the image frame.

And 403, encoding the target image frame according to the encoding parameters respectively corresponding to the plurality of first data blocks.

And 404, obtaining a video coding result according to the coding result of the target image frame.

Referring to fig. 6, a flowchart of an embodiment of a video encoding method according to the fifth embodiment of the present application is shown, where the method specifically includes the following steps:

step 501, dividing a specific area of a target image frame in a video into a plurality of first data blocks, wherein the target area is an area where the content of the target image frame changes relative to a reference image frame.

Step 502, inputting the plurality of first data blocks in batches into four encoding processing units which are executed in sequence; the four coding processing units respectively execute the quantization parameter calculation, the information quantity acquisition, the quantization parameter and information quantity calculation and the quantization parameter and information quantity writing in the determination process of the coding parameters of the image frame.

Step 503, encoding the target image frame according to the encoding parameters respectively corresponding to the four first data blocks.

And step 504, obtaining the coding result of the video according to the coding result of the target image frame.

Referring to fig. 7, a flowchart of an embodiment of an image frame processing method according to the sixth embodiment of the present application is shown, where the method specifically includes the following steps:

step 601, dividing a target image frame into a plurality of first data blocks.

Step 602, inputting the plurality of first data blocks in batches into a plurality of processing units which execute in sequence. And after the processing unit finishes the processing of the first data blocks of the previous batch, the processing unit executes the processing of the first data blocks of the next batch, and the processing unit correspondingly executes one or more steps in the processing process of the target image frame.

According to the embodiment of the application, a plurality of processing units are adopted to execute the processing process of the target image frame, a single processing unit executes one or more steps in the processing process, the image frame is divided into data blocks, and a plurality of processing units which are executed in sequence are input in batches, so that the processing result can be obtained. The processing unit can execute the processing of the next data block after processing the previous data block, at least two coding processing units synchronously execute the processing in a part of time period for executing the processing by the coding processing units, so that the processing of the data blocks can be simultaneously performed.

Moreover, the processing scheme of the multi-processing unit in the embodiment of the application has strong universality, can be copied to different processing systems for use, and can configure the processing unit to be compatible with other processing processes, so that the optimization processing of a plurality of processing processes is realized, and the processing efficiency of the system is improved.

The image frame processing method in the embodiment of the application can be applied to video processing, can be further applied to a Software-as-a-Service (SaaS) platform, and is provided as a video processing Service oriented to various Software.

The SaaS provides software services through a network, the platform provider uniformly deploys application software on its own server, and a platform client (e.g., a service provider providing software) can order a required software service from the platform through the internet according to actual needs, and further obtain the service provided by the platform through the internet.

The specific implementation process may include:

step 1, at least one target video submitted by a software service client is obtained.

And 2, dividing the image frame of the target video into a plurality of first data blocks.

And 3, inputting the plurality of first data blocks in batches into a plurality of encoding processing units which are executed in sequence. And after the coding processing unit finishes processing the first data blocks of the previous batch, processing the first data blocks of the next batch, wherein the coding processing unit correspondingly executes one or more steps in the determination process of the coding parameters of the target image frame, and at least two coding processing units synchronously execute processing in partial time periods of the processing executed by the coding processing units.

And 4, coding the target image frame according to the coding parameters respectively corresponding to the plurality of first data blocks.

And 5, obtaining the coding result of the target video according to the coding results of the plurality of image frames.

And 6, providing an encoding result of the target video based on the software service client.

In order to make the present application better understood by those skilled in the art, an image frame encoding method of the present application is described below by way of specific examples, in which each stage pipeline is an encoding processing unit executed in sequence. Referring to fig. 14, a schematic diagram of an image frame encoding flow is shown, which specifically includes the following steps:

1. and the Curree control unit starts a first-stage pipeline, and reads out the information of the current CU from the CU parameter memory according to the current CU position of the target image frame, wherein the information comprises a motion vector, a prediction type, intra-frame prediction cost, an inter-frame cost coefficient and the like.

2. The first-stage pipeline calculates the information quantity storage address of the current CU in the memory according to the position of the current CU in the image; and calculating the information quantity storage address of the CU corresponding to the current CU in each reference image frame according to the position of the current CU in the image and the motion vector of the current CU.

3. After the first stage pipeline operation is finished, processing of the next CU is started, and meanwhile the control unit starts the second stage pipeline.

4. The second-stage pipeline reads the initial information quantity of the reference image frame from the information quantity memory of the reference image frame according to the information quantity memory address of the CU in each reference frame; and reading the information volume of the current CU from the information volume memory of the current CU according to the information volume storage address of the current CU.

5. And the second-stage pipeline multiplies the intra-frame prediction cost of the CU by the coding frame rate to obtain the intra-frame information quantity of the CU, and adds the intra-frame information quantity of the CU to the information quantity of the current CU read from the information quantity storage of the current frame to obtain the total information quantity of the current CU.

6. And the second-stage pipeline multiplies the total of the current CU information quantity by the inter-frame cost coefficient of the CU to obtain the transmission information quantity of the current CU, and the control unit starts the third-stage pipeline after the calculation is finished.

7. And the third-stage pipeline obtains the initial information quantity of the 4 CUs of each reference frame from the reference frame information quantity storage, and superposes the delivery information quantity of the current CU calculated by the second-stage pipeline on the initial information quantity of the 4 CUs in each reference frame.

8. And the third-stage pipeline divides the intra-frame prediction cost of the current CU by the total information amount of the current CU, and takes log logarithm of the result to obtain the quantization parameter.

9. And the fourth-stage pipeline writes the quantization parameter QP into a quantization parameter QP memory and writes the total information volume of the CU in each reference frame into a reference frame information volume memory after the third-stage pipeline is updated.

10. If the total information amount of the CU in the reference frame required by the third-stage pipeline is from the fourth-stage pipeline, the fourth-stage pipeline can feed back the result of the total information amount of the CU in the reference frame to the third-stage pipeline, so that the CU information amount required by the third-stage pipeline is ensured to be the latest result, and the pipeline pause caused by untimely memory data updating when different pipelines access the same address is avoided.

One specific example of a calculation is provided as follows:

f9, each data block in each image frame corresponds to a CUTREE information amount, and the starting information amount (which can be set according to actual needs) of CUTREE of each block is stored in a memory.

In determining the quantization parameter, when F1 is calculated, for each data block in F1, the initial information amount a (set to 0) of the block in the memory is read out, and the intra information amount B is calculated according to the intra prediction cost and the encoding frame rate of the block, and then the total information amount of the data block is obtained by adding a to B.

And multiplying the total information quantity by the interframe cost coefficient C to obtain a result (A + B) × C, and transmitting the total information quantity for the current block.

Assuming that the reference frame of F1 is F0, four blocks in F0 are found according to the motion vector, the information amount of the four blocks, for example, K0, K1, K2, and K3, are read from the memory, and (a + B) × C is divided into four parts, which are superimposed on K0, K1, K2, and K3, to obtain the total information amount corresponding to each of the four data blocks of F0, which is then updated to the memory of F0 frame.

Then, F2 is calculated, if F2 refers to F0 and F1 at the same time, then F2 is calculated, and the information amount of each block in F0 and F1 is updated. And by analogy, F0-F9 are calculated.

The calculation of the quantization parameter for each block in F0 to F9 may be started after the total information amount of each data block is calculated.

For example, the quantization parameter of one block in F0 is calculated, the total information amount is read from the memory (after the F1.. F9 superposition), the total information amount is multiplied by the encoding frame rate coefficient, then divided by the intra cost coefficient of the block, and finally the LOG is taken to obtain the quantization parameter.

Referring to fig. 8, a block diagram illustrating a structure of an embodiment of an image frame encoding apparatus according to a seventh embodiment of the present application is shown, which may specifically include:

a data block segmentation unit 701 configured to divide a target image frame into a plurality of first data blocks, and input the plurality of first data blocks in batches to a plurality of encoding processing units that are sequentially executed;

an encoding processing unit 702, configured to correspondingly execute one or more steps in the determination process of the encoding parameters of the target image frame, and after the processing of the first data blocks of the previous batch is executed, execute the processing of the first data blocks of the next batch;

the encoding unit 703 is configured to encode the target image frame according to encoding parameters respectively corresponding to the plurality of first data blocks.

In an optional embodiment of the present application, the data block segmentation unit is specifically configured to divide a specific area of the target image frame into a plurality of first data blocks, where the target area is an area where content of the target image frame changes with respect to a reference image frame.

In an optional embodiment of the present application, the data block segmentation unit is specifically configured to: and inputting the plurality of first data blocks into a data processing unit one by one for data processing.

In an optional embodiment of the present application, the plurality of data processing units include a parameter reading unit, an information amount obtaining unit, a data calculating unit, and a data interacting unit.

In an optional embodiment of the present application, the parameter reading unit is specifically configured to read a prediction parameter of the first data block; the prediction parameters comprise a prediction direction and a prediction cost in the target image frame encoding process.

In an optional embodiment of the present application, the information amount obtaining unit is specifically configured to obtain, according to an information amount storage address of the first data block, an initial information amount of the first data block from a corresponding memory; obtaining the intra-frame information quantity of the first data block according to the prediction cost and the coding frame rate of the first data block; and determining the total information quantity of the first data block according to the intra-frame information quantity and the initial information quantity.

In an optional embodiment of the present application, the data calculation unit is specifically configured to determine, according to the total information amount, the prediction direction, and the prediction cost of the read first data block, a corresponding encoding parameter for the first data block.

In an optional embodiment of the present application, the parameter obtaining unit is specifically configured to obtain an information amount storage address of the first data block according to a position of the first data block in an image.

In an optional embodiment of the present application, the encoding processing unit is further configured to correspondingly perform one or more steps in a process of determining a total information amount of a reference image frame of the target image frame, and the information amount obtaining unit is further configured to determine a transfer information amount between the target image frame and the reference image frame according to the total information amount of the first data block;

the data calculation unit is further used for acquiring the initial information amount of a second data block in the reference image frame; and superposing the transfer information quantity of the first data block to the initial information quantity of a second data block in a reference image frame to obtain the total information quantity of the second data block in the reference image frame.

In an optional embodiment of the present application, the parameter reading unit is configured to read a prediction cost coefficient;

and the transfer information amount determining subunit is used for determining the transfer information amount of the first data block according to the total information amount and the prediction cost coefficient of the first data block.

In an optional embodiment of the present application, the parameter reading unit is further configured to determine, according to the position of the first data block in the target image frame and the motion vector, a position of a second data block on which an information amount is to be superimposed in the reference image frame.

In an optional embodiment of the present application, the data interaction unit is configured to store the encoding parameter to a corresponding storage unit, so as to be called during subsequent encoding.

In an alternative embodiment of the present application, the data interaction unit is configured to transfer the total information amount of the second data block in the updated reference image frame to the other encoding processing unit used in association with the second data block.

Referring to fig. 9, a block diagram illustrating a structure of an embodiment of a video encoding apparatus according to an eighth embodiment of the present application may specifically include:

a data block segmentation unit 801, configured to divide a target image frame in a video into a plurality of first data blocks, and input the plurality of first data blocks in batches into a plurality of encoding processing units that are executed in sequence;

an encoding processing unit 802, configured to correspondingly execute one or more steps in the determination process of the encoding parameters of the target image frame, and after the processing of the previous batch of first data blocks is executed, execute the processing of the next batch of first data blocks;

an image frame encoding unit 803, configured to encode the target image frame according to encoding parameters respectively corresponding to the plurality of first data blocks;

and the video coding unit 804 is configured to obtain a coding result of the video according to the coding result of the target image frame.

Referring to fig. 10, a block diagram illustrating a structure of an embodiment of a video encoding apparatus according to the ninth embodiment of the present application may specifically include:

a data block segmentation unit 901, configured to divide a specific area of a target image frame in a video into a plurality of first data blocks, and input the plurality of first data blocks in batches into four encoding processing units that are sequentially executed, where the target area is an area where a content of the target image frame changes with respect to a reference image frame;

four encoding processing units 902, configured to perform a quantization parameter calculation basis reading step, an information amount obtaining step, a quantization parameter and information amount calculation step, and a quantization parameter and information amount writing step in the determination process of the encoding parameters of the image frame, respectively, where after the encoding processing units perform processing of the previous batch of first data blocks, the encoding processing units perform processing of the next batch of first data blocks;

an image frame encoding unit 903, configured to encode the target image frame according to encoding parameters corresponding to the four first data blocks respectively;

and a video encoding unit 904, configured to obtain an encoding result of the video according to an encoding result of the target image frame.

According to the embodiment of the application, a plurality of coding processing units are adopted to execute the determining process of the coding parameters, a single coding processing unit executes one or more steps in the determining process of the coding parameters, when the coding parameters are calculated for an image frame, the image frame is divided into data blocks, the plurality of coding processing units which are executed in sequence are input in batches, the coding parameters can be obtained, and then the image frame is coded by utilizing the coding parameters. Because the encoding processing units can execute the processing of the next data block after processing the previous data block, at least two encoding processing units synchronously execute the processing in a partial time period of the processing executed by the four encoding processing units, the processing of a plurality of data blocks can be simultaneously executed, compared with a mode of sequentially executing by taking a video frame as a unit, the method can greatly shorten the time consumed by determining the encoding parameters, improve the efficiency of video encoding and simultaneously reduce the processing delay of the data blocks. Because the hardware unit is subjected to resource multiplexing in a mode of parallel operation of the multiple coding units, the chip area can be reduced, the hardware overhead can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

Referring to fig. 11, a block diagram illustrating a structure of an embodiment of an image frame processing apparatus according to a tenth embodiment of the present application is shown, which may specifically include:

a block segmentation unit 1001 configured to divide a target image frame into a plurality of first data blocks, and input the plurality of first data blocks in batches to a plurality of processing units that execute in sequence;

the processing unit 1002 is configured to correspondingly execute one or more steps in the processing process of the target image frame and obtain a processing result, and after the processing of the first data blocks of the previous batch is completed, the processing unit executes the processing of the first data blocks of the next batch.

Referring to fig. 12, a block diagram of an embodiment of an image frame encoding system according to the eleventh embodiment of the present application is shown, including a frame dividing unit, a plurality of encoding processing units and an encoding unit, which are executed in sequence;

the frame dividing unit 1101 is configured to divide a target image frame into a plurality of first data blocks, and input the plurality of first data blocks into a plurality of encoding processing units in batches;

the encoding processing unit 1102 is configured to correspondingly execute one or more steps in the determination process of the encoding parameters of the target image frame, and execute processing of a next batch of first data blocks after executing processing of a previous batch of first data blocks;

the image frame encoding unit 1103 is configured to encode the target image frame according to encoding parameters respectively corresponding to the plurality of first data blocks.

Referring to fig. 13, a block diagram of an embodiment of a chip based on a field programmable gate array according to a twelfth embodiment of the present application is shown, where the chip includes a frame dividing unit, a plurality of coding processing units and coding units, which are executed in sequence;

the frame dividing unit 1201 is configured to divide a target image frame into a plurality of first data blocks, and input the plurality of first data blocks into a plurality of encoding processing units in batches;

the encoding processing unit 1202 is configured to correspondingly execute one or more steps in the determination process of the encoding parameters of the target image frame, and execute processing of the next batch of first data blocks after executing processing of the previous batch of first data blocks;

the image frame encoding unit 1203 is configured to encode the target image frame according to encoding parameters respectively corresponding to the plurality of first data blocks.

In the FPGA implementation scheme adopting the multiplexing structure, the memory and the computing unit are subjected to fine granularity multiplexing, and the corresponding occupied chip area is only 1800 LUTs, so that the hardware implementation is facilitated.

For the apparatus and system embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference may be made to some descriptions of the method embodiments for relevant points.

Embodiments of the disclosure may be implemented as a system using any suitable hardware, firmware, software, or any combination thereof, in a desired configuration. Fig. 15 schematically illustrates an example system (or apparatus) 1300 that can be used to implement various embodiments described in this disclosure.

For one embodiment, fig. 15 illustrates an exemplary system 1300 having one or more processors 1302, a system control module (chipset) 1304 coupled to at least one of the processor(s) 1302, system memory 1306 coupled to the system control module 1304, non-volatile memory (NVM)/storage 1308 coupled to the system control module 1304, one or more input/output devices 1310 coupled to the system control module 1304, and a network interface 1312 coupled to the system control module 1306.

Processor 1302 may include one or more single-core or multi-core processors, and processor 1302 may include any combination of general-purpose or special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, the system 1300 can function as a browser as described in embodiments herein.

In some embodiments, system 1300 may include one or more computer-readable media (e.g., system memory 1306 or NVM/storage 1308) having instructions and one or more processors 1302, which in conjunction with the one or more computer-readable media, are configured to execute the instructions to implement modules to perform the actions described in this disclosure.

For one embodiment, the system control module 804 may include any suitable interface controllers to provide any suitable interface to at least one of the processor(s) 1302 and/or any suitable device or component in communication with the system control module 1304.

The system control module 1304 may include a memory controller module to provide an interface to the system memory 1306. The memory controller module may be a hardware module, a software module, and/or a firmware module.

System memory 1306 may be used, for example, to load and store data and/or instructions for system 1300. For one embodiment, system memory 1306 may include any suitable volatile memory, such as suitable DRAM. In some embodiments, the system memory 1306 may include a double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

For one embodiment, system control module 1304 may include one or more input/output controllers to provide an interface to NVM/storage 1308 and input/output device(s) 1310.

For example, NVM/storage 1308 may be used to store data and/or instructions. NVM/storage 1308 may include any suitable non-volatile memory (e.g., flash memory) and/or may include any suitable non-volatile storage device(s) (e.g., one or more Hard Disk Drives (HDDs), one or more Compact Disc (CD) drives, and/or one or more Digital Versatile Disc (DVD) drives).

NVM/storage 1308 may include storage resources that are physically part of the device on which system 1300 is installed or may be accessed by the device and not necessarily part of the device. For example, NVM/storage 1308 may be accessible over a network via input/output device(s) 1310.

Input/output device(s) 1310 may provide an interface for system 1300 to communicate with any other suitable device, input/output device(s) 1310 may include communication components, audio components, sensor components, and so forth. Network interface 1312 may provide an interface for system 1300 to communicate over one or more networks, and system 1300 may communicate wirelessly with one or more components of a wireless network according to any of one or more wireless network standards and/or protocols, such as access to a communication standard-based wireless network, such as WiFi, 2G, 3G, 4G, or 5G, or a combination thereof.

For one embodiment, at least one of the processor(s) 1302 may be packaged together with logic for one or more controllers (e.g., memory controller modules) of the system control module 1304. For one embodiment, at least one of the processor(s) 1302 may be packaged together with logic for one or more controllers of the system control module 1304 to form a System In Package (SiP). For one embodiment, at least one of the processor(s) 1302 may be integrated on the same die with logic for one or more controller(s) of the system control module 1304. For one embodiment, at least one of the processor(s) 1302 may be integrated on the same die with logic of one or more controllers of the system control module 1304 to form a system on chip (SoC).

In various embodiments, system 1300 may be, but is not limited to being: a browser, a workstation, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet, a netbook, etc.). In various embodiments, system 1300 may have more or fewer components and/or different architectures. For example, in some embodiments, system 1300 includes one or more cameras, a keyboard, a Liquid Crystal Display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an Application Specific Integrated Circuit (ASIC), and speakers.

Wherein, if the display includes a touch panel, the display screen may be implemented as a touch screen display to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also identify the duration and pressure associated with the touch or slide operation.

The present application further provides a non-volatile readable storage medium, where one or more modules (programs) are stored in the storage medium, and when the one or more modules are applied to a terminal device, the one or more modules may cause the terminal device to execute instructions (instructions) of method steps in the present application.

In one example, a computer device is provided, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method according to the embodiments of the present application when executing the computer program.

There is also provided in one example a computer readable storage medium having stored thereon a computer program, characterized in that the program, when executed by a processor, implements a method as one or more of the embodiments of the application.

According to an aspect of an embodiment of the present application, there is provided an image frame encoding method, example 1 including:

dividing a target image frame into a plurality of first data blocks;

Example 2 includes the method of example 1, the dividing the target image frame into a plurality of first data blocks comprising:

dividing a specific area of the target image frame into a plurality of first data blocks, wherein the target area is an area where the content of the target image frame changes relative to a reference image frame.

Example 3 includes the method of example 1, wherein batching the plurality of first data chunks into the data processing unit for data processing comprises:

and inputting the plurality of first data blocks into a data processing unit one by one for data processing.

Example 4 includes the method of example 1, wherein the data processing unit is to perform a parameter reading step, an information amount obtaining step, a data calculating step, and a data interacting step in the determination of the encoding parameter of the target image frame.

Example 5 includes the method of example 1, the information amount obtaining step comprising:

acquiring the initial information quantity of the first data block from a corresponding memory according to the information quantity storage address of the first data block;

obtaining the intra-frame information quantity of the first data block according to the prediction cost and the coding frame rate of the first data block;

and determining the total information quantity of the first data block according to the intra-frame information quantity and the initial information quantity.

Example 6 includes the method of example 5, the parameter reading step comprising:

reading the prediction parameters of the first data block; the prediction parameters comprise a prediction direction and a prediction cost in the target image frame encoding process.

Example 7 includes the method of example 6, the data calculating step comprising:

and determining the corresponding coding parameters for the first data block according to the read total information amount, the prediction direction and the prediction cost of the first data block.

Example 8 includes the method of example 5, the parameter reading step comprising:

and acquiring the information quantity storage address of the first data block according to the position of the first data block in the image.

Example 9 includes the method of example 6, the encoding processing unit to further perform one or more steps in a total information amount determination process for a reference image frame of the target image frame;

the information amount obtaining step further includes:

determining the amount of transfer information between the target image frame and a reference image frame according to the total information amount of the first data block;

the data calculating step further comprises:

acquiring initial information quantity of a second data block in a reference image frame;

and superposing the transfer information quantity of the first data block to the initial information quantity of a second data block in a reference image frame to obtain the total information quantity of the second data block in the reference image frame.

Example 10 includes the method of example 9, the parameter reading step further comprising:

reading a prediction cost coefficient;

the determining an amount of transfer information between the target image frame and a reference image frame from a total amount of information of the first data block comprises:

and determining the transfer information quantity of the first data block according to the total information quantity and the prediction cost coefficient of the first data block.

Example 11 includes the method of example 9, the parameter reading step further comprising:

and determining the position of a second data block of which the information amount is to be superposed in the reference image frame according to the position of the first data block in the target image frame and the motion vector.

Example 12 includes the method of example 4, the data interaction step comprising:

and storing the coding parameters to corresponding storage units for calling in the subsequent coding process.

Example 13 includes the method of example 9, the data interaction step comprising:

the total information amount of the second data block in the reference image frame is transferred to the other coding processing units used in association.

According to an aspect of an embodiment of the present application, there is provided a video encoding method, example 14 including:

dividing a target image frame in a video into a plurality of first data blocks;

According to an aspect of an embodiment of the present application, there is provided a video encoding method, example 15 including:

According to an aspect of an embodiment of the present application, there is provided an image frame processing method, example 16 including:

dividing a target image frame into a plurality of first data blocks;

According to an aspect of an embodiment of the present application, there is provided an image frame processing method applied to a software service platform, where example 17 includes:

acquiring at least one target video submitted by a software service client;

According to an aspect of an embodiment of the present application, there is provided an image frame encoding system, example 18 including a frame dividing module, a plurality of encoding processing units that are executed in sequence, and an encoding module;

According to an aspect of an embodiment of the present application, there is provided a chip based on a field programmable gate array, example 19 includes a frame dividing module, a plurality of encoding processing units executing in sequence, and an encoding module;

In accordance with an aspect of an embodiment of the present application, there is provided an electronic device, example 20 including: a processor; and

a memory having executable code stored thereon that, when executed, causes the processor to perform the method of any of examples 1-17.

In accordance with an aspect of embodiments of the present application, there is provided one or more machine-readable media having executable code stored thereon that, when executed, causes a processor to perform the method of any of examples 1-17.

Although certain examples have been illustrated and described for purposes of description, a wide variety of alternate and/or equivalent implementations, or calculations, may be made to achieve the same objectives without departing from the scope of practice of the present application. This application is intended to cover any adaptations or variations of the embodiments discussed herein. Therefore, it is manifestly intended that the embodiments described herein be limited only by the claims and the equivalents thereof.

Claims

1. An image frame encoding method, comprising:

dividing a target image frame into a plurality of first data blocks;

2. The method of claim 1, wherein the dividing the target image frame into the plurality of first data blocks comprises:

3. The method of claim 1, wherein batching the first plurality of data chunks into a data processing unit for data processing comprises:

4. The method according to claim 1, wherein the data processing unit correspondingly performs the parameter reading step, the information amount obtaining step, the data calculating step and the data interacting step in the determination process of the encoding parameter of the target image frame.

5. The method according to claim 1, wherein the information amount obtaining step includes:

6. The method of claim 5, wherein the parameter reading step comprises:

7. The method of claim 6, wherein the data calculating step comprises:

8. The method of claim 5, wherein the parameter reading step comprises:

9. The method according to claim 6, wherein the encoding processing unit further performs one or more steps of a total information amount determination process for a reference image frame of the target image frame;

the information amount obtaining step further includes:

the data calculating step further comprises:

10. The method of claim 9, wherein the parameter reading step further comprises:

reading a prediction cost coefficient;

11. The method of claim 9, wherein the parameter reading step further comprises:

12. The method of claim 4, wherein the data interaction step comprises:

13. The method of claim 9, wherein the data interaction step comprises:

14. A video encoding method, comprising:

dividing a target image frame in a video into a plurality of first data blocks;

15. A video encoding method, comprising:

16. An image frame processing method, comprising:

dividing a target image frame into a plurality of first data blocks;

17. An image frame processing method is applied to a software service platform and comprises the following steps:

acquiring at least one target video submitted by a software service client;

18. An image frame coding system is characterized by comprising a frame dividing module, a plurality of coding processing units and a coding module, wherein the coding processing units and the coding module are executed in sequence;

19. A chip based on field programmable gate array is characterized by comprising a frame dividing module, a plurality of coding processing units and a coding module, wherein the coding processing units and the coding module are executed in sequence;

20. An electronic device, comprising: a processor; and

a memory having executable code stored thereon that, when executed, causes the processor to perform the method of any of claims 1-17.

21. One or more machine-readable media having executable code stored thereon that, when executed, causes a processor to perform the method of any of claims 1-17.