WO2022028283A1

WO2022028283A1 - Image frame coding method, object search method, computer device, and storage medium

Info

Publication number: WO2022028283A1
Application number: PCT/CN2021/108842
Authority: WO
Inventors: 黄玄; 杨名远; 叶琰
Original assignee: 阿里巴巴集团控股有限公司
Priority date: 2020-08-03
Filing date: 2021-07-28
Publication date: 2022-02-10
Also published as: CN114071145A

Abstract

Disclosed by embodiments of the present application are an image frame coding method, an object search method, a computer device, and a storage medium. The image frame coding method comprises: dividing a target image frame into a plurality of first data blocks; splitting the plurality of first data blocks into batches and inputting said blocks into a plurality of coding units which operate in sequence; at least two coding units synchronously perform processing within a portion of time segments when the plurality of coding units are performing processing; and performing coding on the target image frame according to coding parameters respectively corresponding to the plurality of first data blocks. Compared to a manner of sequential operation using video frames as units, the present method can greatly reduce time spent on coding parameter determination, improve video coding efficiency, and also reduce data block processing delay. Due to the manner in which the plurality of coding units run in parallel allowing for resource reuse on a hardware unit, chip size can be reduced, hardware overhead can be lowered, hardware costs can be reduced, and hardware implementation is facilitated.

Description

Image frame encoding method, object search method, computer device, storage medium

This application claims the priority of the Chinese patent application filed on August 3, 2020 with the application number of 202010768892.5 and the title of the invention as "image frame encoding method, object search method, computer equipment, and storage medium", the entire contents of which are incorporated herein by reference Applying.

technical field

The present application relates to the technical field of data processing, and in particular, to an image frame encoding method, an image search method, an object search method, a computer device, and a computer-readable storage medium.

Background technique

At present, video technology is developing rapidly and is widely used. When video is transmitted over the network, for the purpose of bandwidth limitation and traffic saving, video images are encoded and then used in various video application scenarios.

Video coding refers to converting the original video format file into another video format file by performing discrete cosine transform, quantization and entropy coding on the video image frame. Its purpose is to improve the compression rate of the video as much as possible and reduce the volume of the video file under the premise of ensuring the encoding quality.

Therefore, through video pre-analysis, selecting the appropriate one plays an important role in improving the video quality and controlling the video compression rate.

SUMMARY OF THE INVENTION

In view of the above problems, the present application is made to provide an image processing method, an image search method, an object search method, a computer device, and a computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

According to one aspect of the present application, an image frame encoding method is provided, comprising:

dividing the target image frame into a plurality of first data blocks;

Inputting the plurality of first data blocks in batches into a plurality of encoding processing units that are executed in sequence; after the encoding processing unit has performed the processing of the previous batch of the first data blocks, executes the processing of the next batch of the first data blocks , the encoding processing unit correspondingly executes one or more steps in the process of determining the encoding parameters of the target image frame, and in a part of the time period during which the multiple encoding processing units perform processing, at least two encoding processing units are synchronized perform processing;

The target image frame is encoded according to encoding parameters corresponding to the plurality of first data blocks respectively.

According to one aspect of the present application, a video encoding method is provided, comprising:

dividing the target image frame in the video into a plurality of first data blocks;

Inputting the plurality of first data blocks in batches into a plurality of encoding processing units that are executed in sequence; after the encoding processing unit has performed the processing of the previous batch of the first data blocks, executes the processing of the next batch of the first data blocks , the encoding processing unit correspondingly executes one or more steps in the process of determining the encoding parameters of the image frame, and in a partial time period during which the multiple encoding processing units perform processing, at least two encoding processing units perform processing synchronously;

encoding the target image frame according to encoding parameters corresponding to the plurality of first data blocks respectively;

The encoding result of the video is obtained according to the encoding result of the image frame.

Dividing a specific area of the target image frame in the video into a plurality of first data blocks, and the target area is an area where the content of the target image frame changes relative to the reference image frame;

The plurality of first data blocks are input in batches into four encoding processing units that are executed in sequence; after the encoding processing unit has performed the processing of the previous batch of the first data blocks, the processing of the next batch of the first data blocks is performed. , the four encoding processing units respectively perform the quantization parameter calculation in the determination process of the encoding parameter of the image frame according to the reading step, the information acquisition step, the quantization parameter and the information calculation step, and the quantization parameter and the information write step, in During a part of the time period when the four encoding processing units perform processing, at least two encoding processing units perform processing synchronously;

encoding the target image frame according to encoding parameters corresponding to the four first data blocks respectively;

According to an aspect of the present application, an image frame processing method is provided, comprising:

dividing the target image frame into a plurality of first data blocks;

The multiple first data blocks are input in batches into multiple processing units that are executed in sequence; after the processing unit performs the processing of the previous batch of the first data blocks, the processing of the next batch of the first data blocks is performed, so The processing unit correspondingly executes one or more steps in the processing process of the target image frame, and in a partial time period during which the multiple encoding processing units perform processing, at least two encoding processing units perform processing synchronously.

According to one aspect of the present application, an image frame processing method is provided, applied to a software service platform, including:

Obtain at least one target video submitted through the software service client;

dividing the image frame of the target video into a plurality of first data blocks;

Obtain the encoding result of the target video according to the encoding results of a plurality of image frames;

The encoding result of the target video is provided based on the software service client.

According to an aspect of the present application, an image frame encoding system is provided, including a frame division module, a plurality of encoding processing units and encoding modules executed in sequence;

the frame dividing unit, configured to divide the target image frame into a plurality of first data blocks, and input the plurality of first data blocks into a plurality of encoding processing units in batches;

The encoding processing unit is configured to correspondingly execute one or more steps in the process of determining the encoding parameters of the target image frame, and execute the next batch of first data blocks after the processing of the previous batch of first data blocks is completed. For the processing of the data block, in a part of the time period when the plurality of encoding processing units perform processing, at least two encoding processing units perform processing synchronously;

The encoding unit is configured to encode the target image frame according to encoding parameters corresponding to the plurality of first data blocks respectively.

According to one aspect of the present application, a chip based on a field programmable logic gate array is provided, comprising a frame division module, a plurality of encoding processing units and encoding modules executed in sequence;

According to one aspect of the present application, there is provided an electronic device, comprising: a processor; and

A memory having executable code stored thereon which, when executed, causes the processor to perform a method as described in any of the above.

According to one aspect of the present application, there is provided one or more machine-readable media having executable code stored thereon which, when executed, causes a processor to perform a method as described in any of the above.

According to the embodiment of the present application, a plurality of encoding processing units are used to perform the encoding parameter determination process, and a single encoding processing unit performs one or more steps in the encoding parameter determination process. When calculating encoding parameters for an image frame, the image frame is divided into data. The block is input in batches to multiple encoding processing units executed in sequence, and encoding parameters can be obtained, and then the image frames are encoded by using the encoding parameters. Since the encoding processing unit can execute the processing of the next batch of data blocks after processing the previous batch of data blocks, in a part of the time period when multiple encoding processing units perform processing, at least two encoding processing units perform processing synchronously, so that The processing of multiple batches of data blocks can be performed at the same time. Compared with the sequential execution of video frames, the time spent for determining coding parameters can be greatly shortened, the efficiency of video coding is improved, and the processing delay of data blocks is reduced. Since the hardware units are multiplexed in a way of running multiple coding units in parallel, the chip area can be reduced, the hardware overhead and the hardware cost can be reduced, and the hardware implementation can be facilitated.

In addition, the multi-coding processing unit processing solution of the embodiment of the present application is highly versatile, and can be copied to different coding systems for use, and the coding processing unit can be configured to be compatible with other processing processes, thereby realizing the optimization of multiple processing processes. The processing efficiency of the system is improved.

The above description is only an overview of the technical solution of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the description, and in order to make the above-mentioned and other purposes, features and advantages of the present application more obvious and easy to understand , and the specific embodiments of the present application are listed below.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for purposes of illustrating preferred embodiments only and are not to be considered limiting of the application. Also, the same components are denoted by the same reference numerals throughout the drawings. In the attached image:

1 shows a schematic diagram of a video encoding process implemented based on an FPGA in an embodiment of the present application;

FIG. 2 shows a flowchart of an embodiment of an image frame encoding method according to Embodiment 1 of the present application;

FIG. 3 shows a flowchart of an embodiment of an image frame encoding method according to Embodiment 2 of the present application;

FIG. 4 shows a flowchart of an embodiment of an image frame encoding method according to Embodiment 3 of the present application;

FIG. 5 shows a flowchart of an embodiment of a video coding method according to Embodiment 4 of the present application;

FIG. 6 shows a flowchart of an embodiment of a video coding method according to Embodiment 5 of the present application;

FIG. 7 shows a flowchart of an embodiment of an image frame processing method according to Embodiment 6 of the present application;

8 shows a structural block diagram of an embodiment of an image frame encoding apparatus according to Embodiment 7 of the present application;

FIG. 9 shows a structural block diagram showing an embodiment of a video encoding apparatus according to Embodiment 8 of the present application;

FIG. 10 shows a structural block diagram of an embodiment of a video encoding apparatus according to Embodiment 9 of the present application;

FIG. 11 shows a structural block diagram of an embodiment of an image frame processing apparatus according to Embodiment 10 of the present application;

12 shows a structural block diagram of an embodiment of an image frame encoding system according to Embodiment 11 of the present application;

13 shows a structural block diagram of a chip embodiment based on a field programmable logic gate array according to Embodiment 12 of the present application;

14 shows a schematic diagram of an image frame encoding process;

15 illustrates an exemplary system that may be used to implement various embodiments described in this disclosure.

detailed description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly understood, and will fully convey the scope of the present disclosure to those skilled in the art.

In order to make those skilled in the art better understand the solutions of the present application, the concepts involved in the present application are explained as follows:

The encoding parameter is used to encode the video data, and may specifically include data such as frame type and quantization parameter (Quantization Parameter, QP). Among them, the quantization parameter has an important influence on the video quality. For each coding object, the smaller the quantization parameter is, the higher the quantization accuracy is, and the video quality will be correspondingly better, but at the same time, the bandwidth cost of the encoder will also increase. big. Therefore, by reasonably allocating quantization parameters, bandwidth costs can be saved while ensuring video quality. For example, important regions in the video can be allocated smaller quantization parameters, and unimportant regions can be allocated larger quantization parameters.

Based on the importance of the coding parameters, pre-analysis of the video to be coded, selecting a suitable set of coding parameters for the encoder plays an important role in improving the video quality and controlling the compression rate of the coding.

The embodiment of the present application adopts the CUTREE (block tree) algorithm to calculate the coding parameters. The basic principle of the CUTREE algorithm is to allocate a reasonable amount of information to each block by analyzing the relative information amount of each block of the current image frame and other image frames. The coding parameters of the CUTREE algorithm can greatly improve the video quality.

Usually when using the CUTREE algorithm, it is necessary to calculate the relative relationship between the current image frame and the reference image frame, resulting in too many iterations, too much computation, slow processing speed when calculating one image frame by one, and unable to process high-resolution video in real time , and the hardware implementation will cause relatively large resource consumption and high cost.

In view of the above problems, the embodiments of the present application use multiple encoding processing units to perform the encoding parameter determination process in the processing process of the CUTREE algorithm, and a single encoding processing unit performs one or more steps in the encoding parameter determination process. The encoding processing unit is the main body of each unit that processes different steps. From the application level, it can be understood as a pipeline that performs functions corresponding to each step, and the process of determining encoding parameters is allocated to multiple pipelines with sequential execution.

In an optional embodiment, the process of determining the encoding parameters may be divided into four steps: a parameter reading step, an information amount acquisition step, a data calculation step, and a data exchange step. The parameter reading step and the information amount obtaining step are used to determine the basis data used in the calculation of encoding parameters, the data calculation step completes the calculation of the encoding parameters, and the data exchange step saves the encoding parameters for subsequent encoding calls. In a specific implementation, encoding processing units are respectively used to perform the above steps. For example, in an implementation manner, four encoding processing units may be set to perform the above four steps respectively. It can be understood that, in the specific implementation, there may be different step division methods for the process of determining the encoding parameters, and the number of encoding processing units may also be set according to actual requirements.

When calculating the encoding parameters for the image frame to be processed, the image frame is divided into data blocks. Since the data blocks corresponding to the currently processed image frame and the reference image frame are involved, the current image frame to be processed corresponds to the first data block. block, which corresponds the reference image frame to the second data block. The first data blocks are input in batches into multiple encoding processing units executed in sequence, and after processing by the multiple encoding processing units, encoding parameters corresponding to the data blocks can be obtained, and then the image frames are encoded by using the encoding parameters.

In the embodiment of the present application, the encoding processing unit processes the data blocks entered in the same batch at the same time, and before a data block is processed by multiple encoding processing units, the processing of the next data block in the encoding processing unit is started, so that the The processing of multiple batches of data blocks can be performed simultaneously. In order to make the processing of multiple batches of data blocks more efficient, the coding processing unit is designed to execute the processing of the next batch of data blocks after processing the previous batch of data blocks, and execute the processing part in multiple coding processing units. During the time period, at least two encoding processing units perform processing synchronously. It can be understood that in this processing mode, the processing time of multiple data blocks corresponding to the target image frame to be processed in the encoding processing unit is, the processing time of the first data block in all encoding processing units is the same as the processing time of the other encoding processing units. The summation of the sum of the processing time of the first data block in the last coding processing unit greatly improves the processing efficiency compared with the frame-by-frame processing method.

It can be seen that, compared with the sequential execution method in units of video frames, the embodiment of the present application can greatly shorten the time spent for determining the encoding parameters, and improve the efficiency of video encoding (compared with the prior art, the comparison shows that the time is increased by several times) ), while reducing the processing delay of the data block. Since the hardware units are multiplexed in a way of running multiple coding units in parallel, the chip area can be reduced, the hardware overhead and the hardware cost can be reduced, and the hardware implementation can be facilitated.

Since there is motion correlation between the image frames in the video, that is, there are partially changed contents and partially the same contents among the image frames, the target image frame can be expressed as the difference between the target image frame and a certain image frame as the reference image frame. The difference of the image content between the two (which can be expressed as the amount of information transmitted), on the contrary, the target image frame is obtained according to the difference of the image content and the reference image frame. Therefore, when the target image frame is divided into data blocks, a specific area of the target image frame can be divided into a plurality of first data blocks, wherein the target area is the content of the target image frame that changes relative to the reference image frame. area.

The above-mentioned inputting the first data blocks into the encoding processing unit in batches can be multiple data blocks as a batch, or a data block as a batch, which can achieve the effect of simultaneously processing data blocks of different batches . When multiple data blocks enter the encoding processing unit for processing, since the previously processed data blocks need to wait for the later processed data blocks, time is wasted. Therefore, one data block is used as a batch, and multiple data blocks are The first data block is input to the data processing unit one by one for data processing, which can better improve processing efficiency.

Compared with the solution without CUTREE technology, it can significantly improve the video quality and reduce the coding rate. According to statistics, the coding system using this structure can save 10%-15% of the coding bit rate on average under the premise of the same video quality.

The above describes the process of determining the encoding parameters through multiple encoding processing units. In addition to using the encoding parameters, the video encoding process also needs to update the total amount of information of the reference image frame in real time to ensure the information of the reference image frame used by each encoding unit. The quantity is the latest accurate data. Correspondingly, the processing process of the CUTREE algorithm is divided into two parts. One is to determine the coding parameters (mainly quantization parameters), and calculate the quantization parameters of the first data block according to the information amount and related information of the current first data block. The second is the amount of superimposed information. It is necessary to determine the amount of information transmitted in the first data block, and according to the motion vector, superimpose the amount of transmitted information in the first data block to the initial amount of information in the second data block corresponding to the reference image frame. The total amount of information of the second data block can also be called the amount of CUTREE information, which is different from the amount of information in the traditional sense. The more times a block is superimposed, the greater the amount of information. The smaller the value, the higher the importance of this block, and the need for high-quality encoding.

Therefore, the above encoding processing unit may also correspondingly execute one or more steps in the process of determining the total information content of the reference image frame of the target image frame, and the parameter reading step and the information content obtaining step may also correspond to reading the reference image frame. The information determines the basis data used, and the data interaction step is also used to save the reference information for invoking during encoding.

Wherein, the calculation of the total information amount of the reference frame can be performed before the determination of the coding parameters, and the execution order can also be set according to actual needs.

In specific implementation, other calculation processes may also be implemented by using the execution mode of the above-mentioned multi-coding processing unit of the present application, and the specific implementation process is similar to the above-mentioned execution process, which will not be repeated here.

The embodiments of the present application can be deployed on a processing chip, for example, an FPGA (Field Programmable Gate Array, programmable logic device) chip, and the chip can be further deployed on a video encoding device. The device has certain computing capabilities, and the device can It is a user-side terminal, conventional server, cloud server, cloud host, virtual center, etc. The video encoded by the chip can be further transmitted over the network and used for terminal playback and video processing of video applications.

The following description will be given with reference to a specific application example. Referring to FIG. 1 , a schematic diagram of implementing a video encoding process based on an FPGA in an embodiment of the present application is shown. Four encoding processing units are deployed on the FPGA chip, namely the first encoding processing unit, the second encoding processing unit, the third encoding processing unit and the fourth encoding processing unit, respectively corresponding to the parameter reading in the process of determining the encoding parameters steps, information acquisition steps, data calculation steps, and data interaction steps.

The target image frame input to the FPGA chip is first divided into multiple first data blocks. As shown in the figure, the video frame is divided into five first data blocks, namely CU0, CU1, CU2, CU3 and CU4. The five first data blocks are input into multiple encoding processing units in batches, and after the processing of the previous batch of the first data blocks, the processing of the next batch of the first data blocks is performed. As shown in FIG. 1 , the first data block CU0 ranked first enters the first encoding processing unit to perform the parameter reading step. At this time, other encoding processing units have no processing tasks. After CU0 is processed by the first encoding processing unit, Continue to enter the second encoding processing unit for processing, and the idle first encoding processing unit continues to process CU1. For each CU, the processing procedure is to sequentially enter each encoding processing unit for processing, and for each encoding processing unit, the processing procedure is to sequentially process each CU, and continue processing the next CU after processing the previous CU. After all the encoding processing units have finished processing, the CU obtains the encoding parameters corresponding to the respective pairs, and further obtains the encoding result of the image frame according to the encoding result of each CU according to the encoding parameter.

During a part of the time period when multiple encoding processing units perform processing, at least two encoding processing units perform processing synchronously. As can be seen from FIG. 1 , after excluding the processing of CU0 in the first encoding processing unit and CU4 in the fourth encoding unit In addition, each coding unit processes different data blocks at the same time. For example, CU4 is processed in the first coding processing unit, CU3 is processed in the second coding processing unit, CU2 is processed in the third coding unit, and CU1 is processed in the fourth coding unit. deal with. The total processing time is the sum of the following two: the processing time of CU0 in the four encoding processing units, and the sum of the processing time of CU1, CU2, CU3 and CU4 in the fourth encoding processing unit. Compared with the method of processing each data block sequentially, the processing time is saved. Assuming that the processing time of each data block in the encoding processing unit is the same, which is X duration, the example takes 8X duration, while frame-by-frame processing takes 20X duration.

It should be noted that the purpose of providing the above application examples is to facilitate the understanding of the methods provided in the embodiments of the present application, but is not intended to limit the methods.

Next, various embodiments of the present application are described.

Referring to FIG. 2, a flowchart of an embodiment of an image frame encoding method according to Embodiment 1 of the present application is shown, and the method may specifically include the following steps:

Step 101: Divide the target image frame into a plurality of first data blocks.

A data block CU is a smaller coding unit divided from an image frame. The division effect can be set according to actual needs, including the size, shape, area, and number of data blocks of each data block. For example, it can be divided into 8x8 or 16x16 data blocks. It is also possible to first divide a larger data block, and further divide the larger data block into smaller data blocks. The specific division method can be selected according to actual requirements, for example, a quad-tree division method can be used.

In order to better adapt to the image details, the image area of complex details can be divided into smaller data block sizes, while the image areas of simple parts can be divided into larger data block sizes.

In the process of dividing the data block, it can also be determined with reference to rate-distortion optimization. For a larger data block, it can be divided into smaller data blocks. In this process, the rate-distortion cost of the larger data block and the sum of the rate-distortion cost of the smaller data block will be calculated. If the latter is small, it will be Divide into smaller data blocks.

Step 102: Input the plurality of first data blocks in batches into a plurality of encoding processing units that are executed in sequence; after the encoding processing unit executes the processing of the previous batch of first data blocks, executes the next batch of first data blocks Block processing, the encoding processing unit correspondingly performs one or more steps in the process of determining the encoding parameters of the target image frame. The processing units execute processing synchronously.

The encoding processing unit processes the data blocks entered in the same batch at the same time. Before a data block is processed by multiple encoding processing units, the processing of the next data block in the encoding processing unit is started, that is, the processing of multiple batches of data blocks. Processing can be performed simultaneously.

The first data blocks are input into the encoding processing unit in batches, which may be multiple data blocks as one batch, or one data block as one batch.

In order to achieve higher efficiency in the simultaneous processing of multiple batches of data blocks, the coding processing unit is designed to process the next batch of data blocks after processing the previous batch of data blocks. Compared with the frame-by-frame processing method, the processing efficiency is greatly improved.

In specific implementation, control signals can also be used to ensure that the encoding processing units are independent of each other and work uninterruptedly.

Step 103: Encode the target image frame according to encoding parameters corresponding to the plurality of first data blocks respectively.

Each first data block may be encoded by using an encoding parameter, and an encoding result of the target image frame may be obtained according to the encoding result of each data block.

Referring to FIG. 3, a flowchart of an embodiment of an image frame encoding method according to Embodiment 2 of the present application is shown, and the method may specifically include the following steps:

Step 201: Divide a specific area of the target image frame into a plurality of first data blocks, where the target area is an area where the content of the target image frame changes relative to the reference image frame.

Based on the motion correlation between the image frames in the video, the target image frame can be represented as the image content difference between the target image frame and a certain image frame as the reference image frame. Correspondingly, the target image frame is divided into data blocks. When the content of the target image frame changes relative to the reference image frame, it can be determined, and the changed area is further divided into a plurality of first data blocks.

Step 202, inputting the plurality of first data blocks into a data processing unit one by one for data processing; after the encoding processing unit has performed the processing of the previous batch of the first data blocks, performs the processing of the next batch of the first data blocks, The encoding processing unit correspondingly executes one or more steps in the process of determining the encoding parameters of the target image frame, and in a part of the time period during which the multiple encoding processing units perform processing, at least two encoding processing units execute synchronously. deal with.

Using one data block as a batch can improve processing efficiency better than multiple data blocks as a batch.

Step 203: Encode the target image frame according to encoding parameters corresponding to the plurality of first data blocks respectively.

Referring to FIG. 4 , a flowchart of an embodiment of an image frame encoding method according to Embodiment 3 of the present application is shown, and the method may specifically include the following steps:

Step 301: Divide the target image frame into a plurality of first data blocks.

Step 302: Input the multiple first data blocks in batches into multiple encoding processing units that are executed in sequence; after the encoding processing unit executes the processing of the previous batch of first data blocks, executes the next batch of first data Block processing, the data processing unit corresponds to the parameter reading step, the information amount acquisition step, the data calculation step, and the data exchange step in the process of determining the encoding parameters of the target image frame. During a part of the time period during which the unit performs processing, at least two encoding processing units perform processing synchronously.

Step 303: Encode the target image frame according to encoding parameters corresponding to the plurality of first data blocks respectively.

In this embodiment, the processing content of each encoding processing unit is specifically determined, and the encoding parameter determination process is divided into a parameter reading step, an information amount acquisition step, a data calculation step, and a data exchange step. The number of encoding processing units and the specific processing content can be set according to actual needs, and one encoding processing unit can process one or more of the steps. For example, three encoding processing units are designed, the first encoding processing unit handles the parameter reading step and the information amount acquisition step, the second encoding processing unit handles the data calculation step, and the third encoding processing unit handles the data interaction step.

In an optional embodiment of the present application, the parameter reading step includes: reading a prediction parameter of the first data block, where the prediction parameter includes a prediction direction and a prediction cost in the encoding process of the target image frame. The prediction direction may be intra-frame prediction or inter-frame prediction, or may be intra-frame prediction combined with inter-frame prediction, and the prediction cost measures the deviation data caused in the encoding prediction process.

Prediction of image frames is a compression process used in the encoding process, which can be divided into intra-frame prediction and inter-frame prediction, which can also be called inter-frame coding and intra-frame coding. Intra-frame prediction is based on the fact that due to the strong correlation between pixels in the same image, the correlation between pixels can be used for compression coding. For example, a certain pixel is used as a reference pixel, and the difference between other pixels and the reference pixel is used to express other pixels, so as to compress other pixels. For example, when sending data, the actual pixel X (current value) and the reference pixel (predicted value) are subtracted, the difference value is transmitted, and the receiving end obtains the actual pixel according to the sum of the difference value and the reference pixel. Inter-frame prediction is based on the strong correlation between the two image frames before and after the video. For example, the reference image frame can be an image frame that is closest to the target image frame in a certain direction. The difference value of the image frame is used to express the target image frame, so as to compress the image frame.

In an optional embodiment of the present application, the step of obtaining the information amount includes: according to the information amount storage address of the first data block, obtaining the initial information amount of the first data block from a corresponding memory according to the first data block. The prediction cost and the encoding frame rate of a data block are used to obtain the intra-frame information amount of the first data block; the total information amount of the first data block is determined according to the intra-frame information amount and the initial information amount. For example, the prediction cost can be multiplied by the encoding frame rate to obtain the intra-frame information amount of the first data block, and then the intra-frame information amount is added to the initial information amount to obtain the total information amount of the first data block.

Among them, the initial information amount of the image frame is used to calculate the total information amount. If other frames refer to the current image frame, they will be superimposed on the information amount of this frame. Therefore, in the process of CUTREE calculation, the information amount of each frame is It keeps increasing until the superposition process ends, and the total amount of information is obtained, which is used to derive the quantization parameters.

The initial information amount of the image frame can be calculated in advance by the number of pixels and the number of distinguishable colors in the image.

In an optional embodiment of the present application, the data calculation step includes: determining a code corresponding to the first data block according to the read total information amount, prediction direction and prediction cost of the first data block parameter. Taking the prediction direction as intra-frame prediction and the encoding parameter as the quantization parameter as an example, the prediction cost is the intra-frame prediction cost. The quantization parameters can be obtained by predicting the cost coefficient and taking the logarithm of the result.

In an optional embodiment of the present application, the parameter reading step includes: acquiring an information storage address of the first data block according to the position of the first data block in the image. First, the position of the data block in the image is determined, and the information storage address of the first data block can be determined according to the mapping relationship between the position and the information storage address.

Since the video coding process needs to use the coding parameters as well as the information amount of the reference image frame, the coding parameter determination process can also be implemented by a plurality of coding processing units.

Correspondingly, the step of obtaining the information amount may further include: determining the amount of information transferred between the target image frame and the reference image frame according to the total information amount of the first data block. The information content acquisition step can also obtain one of the basis for calculating the total information content of the reference frame, that is, the total information content of the first data block of the target image frame, and further determine the target image according to the total information content of the first data block. The amount of information transferred between the frame and the reference image frame, that is, the amount of difference information between the two.

Correspondingly, the data calculation step further includes: acquiring the initial information amount of the second data block in the reference image frame; superimposing the transfer information amount of the first data block to the initial information amount of the second data block in the reference image frame , to obtain the total information amount of the second data block in the reference image frame. Since the transfer information amount is also the difference information amount between the target image frame and the reference image frame, the transfer information amount can be superimposed on the initial information amount of the reference image frame to obtain the information of the second data block in the reference image frame. total amount of information.

In an optional embodiment of the present application, the parameter reading step includes: reading a prediction cost coefficient. The prediction cost coefficient is a coefficient for the prediction cost, that is, the prediction deviation or the coding deviation, which can be preset, or the prediction cost coefficient set corresponding to the range can be determined according to the numerical range of the prediction cost.

Correspondingly, in an optional embodiment of the present application, the determining the amount of information transferred between the target image frame and the reference image frame according to the initial information amount of the first data block includes: according to the first data block The prediction cost of the data block and the encoding frame rate are used to obtain the intra-frame information amount of the first data block; the total information amount of the first data block is determined according to the intra-frame information amount and the initial information amount; The amount of information and the prediction cost coefficient determine the amount of conveyed information for the first data block. The specific calculation formula can be set according to actual needs, and the calculation process of the data is a well-known technology in the art, which will not be repeated here.

In an optional embodiment of the present application, the parameter reading step further includes: determining the amount of information to be superimposed in the reference image frame according to the position of the first data block in the target image frame and the motion vector The location of the second data block. The motion vector (MotionVector, MV) here represents the relative displacement between the first data block and the second data block with the highest matching degree in the reference image frame during the inter-frame coding process. Therefore, according to the first data block in the The position in the target image is combined with the relative displacement represented by the motion vector to determine the position of the second data block corresponding to the first data block in the reference image frame.

In an optional embodiment of the present application, the data interaction step includes: storing the encoding parameters in a corresponding storage unit for subsequent invocation during encoding.

In an optional embodiment of the present application, the data exchange step includes: transferring the total amount of information of the second data block in the reference image frame to other encoding processing units used in association, and updating other encoding processing units in time The data used ensures that the data used by other encoding processing units is the latest and accurate data, and avoids that when the total information amount of the second data block in the reference image frame is accessed from the storage location, the data in the memory is not updated in time and the encoding processing unit is stopped.

Referring to FIG. 5 , a flowchart of an embodiment of a video coding method according to Embodiment 4 of the present application is shown, and the method may specifically include the following steps:

Step 401: Divide the target image frame in the video into a plurality of first data blocks.

Step 402: Input the multiple first data blocks into multiple encoding processing units that are executed in sequence; after the encoding processing unit performs the processing of the previous batch of first data blocks, executes the next batch of first data blocks Block processing, the encoding processing unit correspondingly executes one or more steps in the process of determining the encoding parameters of the image frame.

Step 403: Encode the target image frame according to encoding parameters corresponding to the plurality of first data blocks respectively.

Step 404: Obtain the encoding result of the video according to the encoding result of the target image frame.

Referring to FIG. 6 , a flowchart of an embodiment of a video coding method according to Embodiment 5 of the present application is shown, and the method may specifically include the following steps:

Step 501: Divide a specific area of a target image frame in the video into a plurality of first data blocks, where the target area is an area where the content of the target image frame changes relative to the reference image frame.

Step 502: Input the multiple first data blocks in batches into four encoding processing units that are executed in sequence; after the encoding processing unit performs the processing of the previous batch of first data blocks, executes the next batch of first data The processing of the block, the four encoding processing units respectively perform the calculation of the quantization parameter in the process of determining the encoding parameter of the image frame according to the reading step, the information content acquisition step, the quantization parameter and the information content calculation step, and the quantization parameter and the information content write. step.

Step 503: Encode the target image frame according to the encoding parameters corresponding to the four first data blocks respectively.

Step 504: Obtain the encoding result of the video according to the encoding result of the target image frame.

Referring to FIG. 7 , a flowchart of an embodiment of an image frame processing method according to Embodiment 6 of the present application is shown, and the method may specifically include the following steps:

Step 601: Divide the target image frame into a plurality of first data blocks.

Step 602: Input the plurality of first data blocks in batches into a plurality of processing units that are executed in sequence. The processing unit executes the processing of the next batch of first data blocks after processing the previous batch of the first data blocks, and the processing unit correspondingly executes one or more steps in the processing process of the target image frame.

According to the embodiment of the present application, a plurality of processing units are used to perform the processing process of the target image frame, a single processing unit performs one or more steps in the processing process, the image frame is divided into data blocks, and multiple processing units that are executed sequentially are input in batches. The processing unit can obtain processing results. Since the processing unit can perform the processing of the next batch of data blocks after processing the previous batch of data blocks, in a part of the time period when multiple encoding processing units perform processing, at least two encoding processing units perform processing synchronously, so that multiple encoding processing units perform processing synchronously. The processing of batch data blocks can be carried out at the same time. Compared with the sequential execution of video frames, the processing time can be greatly shortened and the processing efficiency of video frames can be improved. Resource multiplexing is performed, which can reduce chip area, reduce hardware overhead, reduce hardware cost, and facilitate hardware implementation.

In addition, the multi-processing unit processing solution of the embodiment of the present application is highly versatile, and can be copied to different processing systems for use, and the processing unit can be configured to be compatible with other processing procedures, thereby realizing the optimization of multiple processing procedures and improving the performance of the processing. The processing efficiency of the system.

The image frame processing method in the embodiment of the present application can be applied to video processing, and further can be applied to a software service SaaS (Software-as-a-Service) platform, which is provided as a video processing service oriented to various software.

SaaS provides software services through the network, platform providers deploy application software on their own servers, platform customers (such as software service providers) can order required software services from the platform through the Internet according to actual needs, and further through the Internet Access to the services provided by the platform.

The specific implementation process can include:

Step 1. Obtain at least one target video submitted through the software service client.

Step 2: Divide the image frame of the target video into a plurality of first data blocks.

Step 3: Input the plurality of first data blocks in batches into a plurality of encoding processing units that are executed in sequence. After the encoding processing unit performs the processing of the previous batch of first data blocks, the processing of the next batch of first data blocks is performed, and the encoding processing unit correspondingly performs one of the processes of determining the encoding parameters of the target image frame. or in a plurality of steps, in a partial period of time during which the plurality of encoding processing units perform processing, at least two encoding processing units perform processing synchronously.

Step 4: Encode the target image frame according to encoding parameters corresponding to the plurality of first data blocks respectively.

Step 5: Obtain an encoding result of the target video according to encoding results of multiple image frames.

Step 6: Provide the encoding result of the target video based on the software service client.

In order for those skilled in the art to better understand the present application, an image frame encoding method of the present application is described below with specific examples, wherein the pipelines at all levels are encoding processing units executed in sequence. Referring to FIG. 14, a schematic diagram of an image frame encoding process is shown, which specifically includes the following steps:

1. The CUTREE control unit starts the first-stage pipeline, and reads the information of the current CU from the CU parameter memory according to the current CU position of the target image frame, including motion vector, prediction type, intra-frame prediction cost, inter-frame cost coefficient, etc.

2. The first-level pipeline calculates the information storage address of the current CU in the memory according to the position of the current CU in the image; according to the position of the current CU in the image and the motion vector of the current CU, calculates the difference between the current CU and the current CU in the image frame. The information storage address of the CU corresponding to the current CU.

3. After the first-stage pipeline operation is completed, the processing of the next CU starts, and the control unit starts the second-stage pipeline at the same time.

4. The second-stage pipeline reads the initial information amount of the reference image frame from the information amount memory of the reference image frame according to the information amount storage address of the CU in each reference frame; according to the information amount storage address of the current CU, from the current The information amount memory of the CU reads the information amount of the current CU.

5. The second-stage pipeline multiplies the CU intra-frame prediction cost by the encoding frame rate to obtain the CU intra-frame information amount, and adds the CU intra-frame information amount to the current CU information amount read from the information amount memory of the current frame. , get the sum of the current CU information.

6. The second-stage pipeline multiplies the sum of the current CU information amount by the CU's inter-frame cost coefficient to obtain the current CU's transfer information amount. After the calculation is completed, the control unit starts the third-stage pipeline.

7. The third-stage pipeline obtains the initial information volume of 4 CUs of each reference frame from the reference frame information volume memory, and superimposes the transfer information volume of the current CU calculated by the second-stage pipeline to 4 in each reference frame. On the initial amount of information of the CU.

8. The third-stage pipeline divides the total information amount of the current CU by the intra-frame prediction cost of the current CU, and takes the logarithm of the result to obtain the quantization parameter.

9. The fourth-stage pipeline writes the quantization parameter QP into the quantization parameter QP memory, and writes the total information amount of the CU in each reference frame after the third-stage pipeline is updated into the reference frame information amount memory.

10. If the total amount of information of the CU in the reference frame required by the third-level pipeline comes from the fourth-level pipeline, the fourth-level pipeline can feed back the result of the total information amount of the reference frame CU to the third-level pipeline, so as to ensure the third-level pipeline. The amount of CU information required by the stage pipeline is the latest result, so as to avoid the pipeline stall caused by the untimely update of memory data when different pipelines access the same address.

A specific calculation example is provided as follows:

Assuming that there are 10 frames of images, assuming F0....F9, each data block in each image frame corresponds to a CUTREE amount of information, and the initial amount of CUTREE information of each block (can be set according to actual needs), stored in in memory.

In the process of determining the quantization parameter, when calculating F1, for each data block in F1, read the initial information amount A of this block in the memory (set to 0), and according to the intra-frame prediction cost of this block and the coded frame rate, calculate the amount of information B in the frame, and then add A to B to get the total amount of information in the data block.

Multiply the total amount of information by the inter-frame cost coefficient C to obtain the result (A+B)×C, which is the total amount of information transmitted in the current block.

Assuming that the reference frame of F1 is F0, find the four blocks in F0 according to the motion vector, and read the information amount of these four blocks from the memory, such as K0, K1, K2, K3, set (A+B)× C is divided into four parts, superimposed on K0, K1, K2, K3, to obtain the total amount of information corresponding to the four data blocks of F0, and then update it to the memory of the F0 frame.

Then start to calculate F2. If F2 refers to F0 and F1 at the same time, then the calculation of F2 is completed, and the information amount of each block in F0 and F1 will also be updated. And so on, when F0-F9 are all calculated.

After calculating the total information amount of each data block, the quantization parameter of each block in F0 to F9 can be calculated.

For example, to start calculating the quantization parameter of a block in F0, first read the total amount of information from the memory (after the superposition of F1...F9), multiply the total amount of information by the encoding frame rate coefficient, and divide by the block's Intra-frame cost coefficient, and finally take the logarithm LOG to get the quantization parameter.

Referring to FIG. 8 , a structural block diagram of an embodiment of an image frame encoding apparatus according to Embodiment 7 of the present application is shown, which may specifically include:

a data block dividing unit 701, configured to divide the target image frame into a plurality of first data blocks, and input the plurality of first data blocks in batches to a plurality of encoding processing units that are executed in sequence;

The encoding processing unit 702 is configured to perform one or more steps in the process of determining the encoding parameters of the target image frame, and execute the next batch of first data blocks after performing the processing of the previous batch of first data blocks. processing of blocks;

The encoding unit 703 is configured to encode the target image frame according to encoding parameters corresponding to the plurality of first data blocks respectively.

In an optional embodiment of the present application, the data block dividing unit is specifically configured to divide a specific area of the target image frame into a plurality of first data blocks, and the target area is the target image The area where the content of the frame changes relative to the reference image frame.

In an optional embodiment of the present application, the data block dividing unit is specifically configured to: input the plurality of first data blocks one by one into a data processing unit to perform data processing.

In an optional embodiment of the present application, the plurality of data processing units include a parameter reading unit, an information amount acquisition unit, a data calculation unit, and a data interaction unit.

In an optional embodiment of the present application, the parameter reading unit is specifically configured to read the prediction parameter of the first data block; the prediction parameter includes the prediction direction and prediction in the encoding process of the target image frame cost.

In an optional embodiment of the present application, the information amount obtaining unit is specifically configured to obtain the initial information amount of the first data block from the corresponding memory according to the information amount storage address of the first data block; and, according to the prediction cost and the encoding frame rate of the first data block, obtain the intra-frame information amount of the first data block; determine the total amount of the first data block according to the intra-frame information amount and the initial information amount amount of information.

In an optional embodiment of the present application, the data calculation unit is specifically configured to determine, according to the read total information amount, prediction direction and prediction cost of the first data block, the data for the first data block the corresponding encoding parameters.

In an optional embodiment of the present application, the parameter obtaining unit is specifically configured to obtain the information storage address of the first data block according to the position of the first data block in the image.

In an optional embodiment of the present application, the encoding processing unit is further configured to perform one or more steps in the process of determining the total information content of the reference image frame of the target image frame. unit, further configured to determine the amount of information transferred between the target image frame and the reference image frame according to the total amount of information of the first data block;

The data calculation unit is further configured to obtain the initial information amount of the second data block in the reference image frame; and superimpose the transfer information amount of the first data block to the initial information amount of the second data block in the reference image frame to obtain The total amount of information of the second data block in the reference image frame.

In an optional embodiment of the present application, the parameter reading unit is configured to read the prediction cost coefficient;

The transfer information amount determination subunit is configured to determine the transfer information amount of the first data block according to the total information amount of the first data block and the prediction cost coefficient.

In an optional embodiment of the present application, the parameter reading unit is further configured to determine the information to be superimposed in the reference image frame according to the position of the first data block in the target image frame and the motion vector The position of the second data block of the amount.

In an optional embodiment of the present application, the data interaction unit is configured to store the encoding parameter in a corresponding storage unit for subsequent invocation during encoding.

In an optional embodiment of the present application, the data exchange unit is configured to transmit the total amount of information of the second data block in the updated reference image frame to other encoding processing units used in association.

Referring to FIG. 9 , a structural block diagram of an embodiment of a video encoding apparatus according to Embodiment 8 of the present application is shown, which may specifically include:

a data block dividing unit 801, configured to divide the target image frame in the video into a plurality of first data blocks, and input the plurality of first data blocks in batches into a plurality of encoding processing units executed in sequence;

The encoding processing unit 802 is configured to perform one or more steps in the process of determining the encoding parameters of the target image frame, and execute the next batch of first data blocks after performing the processing of the previous batch of first data blocks. processing of blocks;

An image frame encoding unit 803, configured to encode the target image frame according to encoding parameters corresponding to the plurality of first data blocks respectively;

The video encoding unit 804 is configured to obtain the encoding result of the video according to the encoding result of the target image frame.

Referring to FIG. 10 , a structural block diagram of an embodiment of a video encoding apparatus according to Embodiment 9 of the present application is shown, which may specifically include:

The data block dividing unit 901 is used to divide a specific area of the target image frame in the video into a plurality of first data blocks, and input the plurality of first data blocks in batches into four encoding processing units that are executed in sequence , the target area is the area where the content of the target image frame changes relative to the reference image frame;

Four encoding processing units 902 are used to respectively perform the quantization parameter calculation in the process of determining the encoding parameters of the image frame according to the reading step, the information content acquisition step, the quantization parameter and the information content calculation step, and the quantization parameter and the information content write. Step, the encoding processing unit performs the processing of the first data block of the next batch after the processing of the first data block of the previous batch is performed;

An image frame encoding unit 903, configured to encode the target image frame according to encoding parameters corresponding to the four first data blocks respectively;

The video encoding unit 904 is configured to obtain the encoding result of the video according to the encoding result of the target image frame.

According to the embodiment of the present application, a plurality of encoding processing units are used to perform the encoding parameter determination process, and a single encoding processing unit performs one or more steps in the encoding parameter determination process. When calculating encoding parameters for an image frame, the image frame is divided into data. The block is input in batches to multiple encoding processing units executed in sequence, and encoding parameters can be obtained, and then the image frames are encoded by using the encoding parameters. Since the encoding processing unit can perform the processing of the next batch of data blocks after processing the previous batch of data blocks, in a part of the time period when the four encoding processing units perform processing, at least two encoding processing units perform processing synchronously, so that the The processing of multiple batches of data blocks can be performed at the same time. Compared with the sequential execution of video frames, the time spent for determining coding parameters can be greatly shortened, the efficiency of video coding is improved, and the processing delay of data blocks is reduced. Since the hardware units are multiplexed in a way of running multiple coding units in parallel, the chip area can be reduced, the hardware overhead and the hardware cost can be reduced, and the hardware implementation can be facilitated.

Referring to FIG. 11 , a structural block diagram of an embodiment of an image frame processing apparatus according to Embodiment 10 of the present application is shown, which may specifically include:

a data block dividing unit 1001, configured to divide the target image frame into a plurality of first data blocks, and input the plurality of first data blocks into a plurality of processing units that are executed in sequence;

The processing unit 1002 is configured to execute one or more steps in the processing process of the target image frame, and obtain the processing result. After the processing unit has executed the processing of the first data blocks of the previous batch, execute the next batch of data blocks. Processing of the first data block.

12 , a structural block diagram of an embodiment of an image frame encoding system according to Embodiment 11 of the present application is shown, including a frame dividing unit 1101, a plurality of encoding processing units 1102 and encoding units 1103 executed in sequence;

The frame dividing unit 1101 is configured to divide the target image frame into multiple first data blocks, and input the multiple first data blocks into multiple encoding processing units in batches;

The encoding processing unit 1102 is configured to perform one or more steps in the process of determining the encoding parameters of the target image frame, and execute the next batch of first data blocks after the processing of the previous batch of first data blocks is completed. processing of a data block;

The image frame encoding unit 1103 is configured to encode the target image frame according to encoding parameters corresponding to the plurality of first data blocks respectively.

Referring to FIG. 13, it shows a structural block diagram of a chip embodiment based on a field programmable logic gate array according to the twelfth embodiment of the present application, including a frame dividing unit 1201, a plurality of encoding processing units 1202 executed in sequence, and an image Frame encoding unit 1203;

The frame dividing unit 1201 is configured to divide the target image frame into multiple first data blocks, and input the multiple first data blocks into multiple encoding processing units in batches;

The encoding processing unit 1202 is configured to perform one or more steps in the process of determining the encoding parameters of the target image frame, and execute the next batch of first data blocks after processing the previous batch of first data blocks. processing of a data block;

The image frame encoding unit 1203 is configured to encode the target image frame according to encoding parameters corresponding to the plurality of first data blocks respectively.

In the FPGA implementation scheme using this multiplexing structure, due to the fine-grained multiplexing of the memory and the computing unit, the corresponding occupied chip area is only 1800 LUTs, which is convenient for hardware implementation.

As for the apparatus and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and reference may be made to some descriptions of the method embodiments for related parts.

Embodiments of the present disclosure may be implemented as a system in a desired configuration using any suitable hardware, firmware, software, or any combination thereof. FIG. 15 schematically illustrates an exemplary system (or apparatus) 1300 that may be used to implement various embodiments described in this disclosure.

For one embodiment, FIG. 15 illustrates an exemplary system 1300 having one or more processors 1302, a system control module (chipset) coupled to at least one of the processor(s) 1302 1304 , system memory 1306 coupled to system control module 1304 , non-volatile memory (NVM)/storage 1308 coupled to system control module 1304 , one or more inputs/outputs coupled to system control module 1304 device 1310, and a network interface 1312 coupled to the system control module 1306.

The processor 1302 may include one or more single-core or multi-core processors, and the processor 1302 may include any combination of general-purpose processors or special-purpose processors (eg, graphics processors, application processors, baseband processors, etc.). In some embodiments, the system 1300 can function as a browser as described in the embodiments of the present application.

In some embodiments, system 1300 may include one or more computer-readable media having instructions (eg, system memory 1306 or NVM/storage device 1308 ) and in combination with the one or more computer-readable media configured to One or more processors 1302 that execute instructions to implement modules to perform the actions described in this disclosure.

For one embodiment, the system control module 804 may include any suitable interface controller to provide at least one of the processor(s) 1302 and/or any suitable device or component in communication with the system control module 1304 any appropriate interface.

System control module 1304 may include a memory controller module to provide an interface to system memory 1306 . The memory controller modules may be hardware modules, software modules, and/or firmware modules.

System memory 1306 may be used, for example, to load and store data and/or instructions for system 1300 . For one embodiment, system memory 1306 may include any suitable volatile memory, eg, suitable DRAM. In some embodiments, system memory 1306 may include double data rate type quad synchronous dynamic random access memory (DDR4 SDRAM).

For one embodiment, system control module 1304 may include one or more input/output controllers to provide interfaces to NVM/storage device 1308 and input/output device(s) 1310 .

For example, NVM/storage 1308 may be used to store data and/or instructions. NVM/storage 1308 may include any suitable non-volatile memory (eg, flash memory) and/or may include any suitable non-volatile storage device(s) (eg, one or more hard drives ( HDD), one or more compact disc (CD) drives and/or one or more digital versatile disc (DVD) drives).

NVM/storage device 1308 may include storage resources that are physically part of the device on which system 1300 is installed, or it may be accessed by the device without necessarily being part of the device. For example, NVM/storage device 1308 may be accessed via input/output device(s) 1310 over a network.

Input/output device(s) 1310 may provide an interface for system 1300 to communicate with any other suitable device, and input/output device(s) 1310 may include communication components, audio components, sensor components, and the like. Network interface 1312 may provide an interface for system 1300 to communicate over one or more networks, system 1300 may interface with one or more of one or more wireless network standards and/or protocols in accordance with any of one or more wireless network standards and/or protocols. Components communicate wirelessly, eg, by accessing a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, or 5G, or a combination thereof.

For one embodiment, at least one of the processor(s) 1302 may be packaged with the logic of one or more controllers (eg, memory controller modules) of the system control module 1304 . For one embodiment, at least one of the processor(s) 1302 may be packaged with logic of one or more controllers of the system control module 1304 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 1302 may be integrated on the same die with the logic of one or more controllers of the system control module 1304 . For one embodiment, at least one of the processor(s) 1302 may be integrated on the same die with logic of one or more controllers of the system control module 1304 to form a system-on-chip (SoC).

In various embodiments, system 1300 may be, but is not limited to, a browser, workstation, desktop computing device, or mobile computing device (eg, laptop computing device, handheld computing device, tablet computer, netbook, etc.). In various embodiments, system 1300 may have more or fewer components and/or different architectures. For example, in some embodiments, system 1300 includes one or more cameras, keyboards, liquid crystal display (LCD) screens (including touchscreen displays), non-volatile memory ports, multiple antennas, graphics chips, application specific integrated circuits ( ASIC) and speakers.

Wherein, if the display includes a touch panel, the display screen may be implemented as a touch screen display to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or swipe action, but also identify the duration and pressure associated with the touch or swipe action.

Embodiments of the present application further provide a non-volatile readable storage medium, where one or more modules (programs) are stored in the storage medium. When the one or more modules are applied to a terminal device, the terminal device can make the terminal The device executes the instructions (instructions) of each method step in the embodiment of the present application.

In one example, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the method according to the embodiments of the present application when the processor executes the computer program .

In one example, a computer-readable storage medium is also provided, on which a computer program is stored, and when the program is executed by a processor, implements one or more of the methods according to the embodiments of the present application.

According to an aspect of the embodiments of the present application, an image frame encoding method is provided, and example 1 includes:

dividing the target image frame into a plurality of first data blocks;

Example 2 includes the method of Example 1, wherein the dividing the target image frame into a plurality of first data blocks includes:

A specific area of the target image frame is divided into a plurality of first data blocks, and the target area is an area where the content of the target image frame changes relative to the reference image frame.

Example 3 includes the method of Example 1, wherein the inputting the plurality of first data blocks into the data processing unit in batches for data processing includes:

The plurality of first data blocks are input into the data processing unit one by one for data processing.

Example 4 includes the method described in Example 1, wherein the data processing unit correspondingly executes the parameter reading step, the information amount acquisition step, the data calculation step, and the data exchange step in the process of determining the encoding parameters of the target image frame.

Example 5 includes the method of Example 1, wherein the information amount obtaining step includes:

According to the information amount storage address of the first data block, obtain the initial information amount of the first data block from the corresponding memory;

According to the prediction cost and the encoding frame rate of the first data block, obtain the intra-frame information amount of the first data block;

The total information amount of the first data block is determined according to the intra-frame information amount and the initial information amount.

Example 6 includes the method of Example 5, the parameter reading step comprising:

Read the prediction parameters of the first data block; the prediction parameters include the prediction direction and the prediction cost in the encoding process of the target image frame.

Example 7 includes the method of Example 6, the data computing step comprising:

The coding parameter corresponding to the first data block is determined according to the read total information amount, prediction direction and prediction cost of the first data block.

Example 8 includes the method of Example 5, the parameter reading step comprising:

The information storage address of the first data block is obtained according to the position of the first data block in the image.

Example 9 includes the method of Example 6, wherein the encoding processing unit further correspondingly performs one or more steps in the process of determining the total information content of the reference image frame of the target image frame;

The step of obtaining the amount of information further includes:

Determine the amount of information transferred between the target image frame and the reference image frame according to the total information amount of the first data block;

The data calculation step also includes:

obtaining the initial amount of information of the second data block in the reference image frame;

The transfer information amount of the first data block is superimposed on the initial information amount of the second data block in the reference image frame to obtain the total information amount of the second data block in the reference image frame.

Example 10 includes the method of Example 9, the parameter reading step further comprising:

Read the prediction cost coefficient;

The determining the amount of information transferred between the target image frame and the reference image frame according to the total information amount of the first data block includes:

The transfer information amount of the first data block is determined according to the total information amount of the first data block and the prediction cost coefficient.

Example 11 includes the method of Example 9, the parameter reading step further comprising:

According to the position of the first data block in the target image frame and the motion vector, the position of the second data block of the amount of information to be superimposed in the reference image frame is determined.

Example 12 includes the method of Example 4, wherein the data interaction step includes:

The encoding parameters are stored in the corresponding storage unit for subsequent encoding.

Example 13 includes the method of Example 9, wherein the data interaction step includes:

The total amount of information of the second data block in the reference image frame is passed to other encoding processing units used in association.

According to an aspect of the embodiments of the present application, a video encoding method is provided, and Example 14 includes:

According to an aspect of the embodiments of the present application, a video encoding method is provided, and Example 15 includes:

The plurality of first data blocks are input in batches into four encoding processing units that are executed in sequence; after the encoding processing unit has performed the processing of the previous batch of the first data blocks, the processing of the next batch of the first data blocks is performed. , the four encoding processing units respectively perform the quantization parameter calculation in the determination process of the encoding parameter of the image frame according to the reading step, the information content acquisition step, the quantization parameter and the information content calculation step, and the quantization parameter and the information content write step, in During a part of the time period when the four encoding processing units perform processing, at least two encoding processing units perform processing synchronously;

According to an aspect of the embodiments of the present application, an image frame processing method is provided. Example 16 includes:

dividing the target image frame into a plurality of first data blocks;

According to an aspect of the embodiments of the present application, an image frame processing method is provided, which is applied to a software service platform. Example 17 includes:

Obtain at least one target video submitted through the software service client;

According to an aspect of the embodiments of the present application, an image frame encoding system is provided, and Example 18 includes a frame division module, a plurality of encoding processing units and encoding modules executed in sequence;

According to an aspect of the embodiments of the present application, a chip based on a field programmable logic gate array is provided, and Example 19 includes a frame division module, a plurality of encoding processing units executed in sequence, and an encoding module;

According to an aspect of the embodiments of the present application, an electronic device is provided, and example 20 includes: a processor; and

A memory having executable code stored thereon which, when executed, causes the processor to perform the method of any of Examples 1-17.

According to an aspect of the embodiments of the present application, one or more machine-readable media are provided, on which executable codes are stored, and when the executable codes are executed, cause the processor to execute any one of Examples 1-17. method described in item.

While certain examples have been presented for purposes of illustration and description, various alternative, and/or, equivalent embodiments, or calculations may be used to achieve the same purposes as the implementation of the examples shown and described, without departing from this application. scope of implementation. This application is intended to cover any adaptations or variations of the embodiments discussed herein. Therefore, it is manifestly intended that the embodiments described herein be limited only by the claims and their equivalents.

Claims

An image frame encoding method, comprising:

dividing the target image frame into a plurality of first data blocks;

Inputting the plurality of first data blocks in batches into a plurality of encoding processing units that are executed in sequence; after the encoding processing unit has performed the processing of the previous batch of the first data blocks, executes the processing of the next batch of the first data blocks , the encoding processing unit correspondingly executes one or more steps in the process of determining the encoding parameters of the target image frame, and in a part of the time period during which the multiple encoding processing units perform processing, at least two encoding processing units are synchronized perform processing;

The target image frame is encoded according to encoding parameters corresponding to the plurality of first data blocks respectively.
The method according to claim 1, wherein the dividing the target image frame into a plurality of first data blocks comprises:

A specific area of the target image frame is divided into a plurality of first data blocks, and the target area is an area where the content of the target image frame changes relative to the reference image frame.
The method according to claim 1, wherein the inputting the plurality of first data blocks into a data processing unit in batches for data processing comprises:

The plurality of first data blocks are input into the data processing unit one by one for data processing.
The method according to claim 1, wherein the data processing unit corresponds to the parameter reading step, the information amount acquisition step, the data calculation step, and the data interaction in the process of determining the encoding parameters of the target image frame. step.
The method according to claim 4, wherein the step of obtaining the amount of information comprises:

According to the information amount storage address of the first data block, obtain the initial information amount of the first data block from the corresponding memory;

According to the prediction cost and the encoding frame rate of the first data block, obtain the intra-frame information amount of the first data block;

The total information amount of the first data block is determined according to the intra-frame information amount and the initial information amount.
The method according to claim 5, wherein the parameter reading step comprises:

Read the prediction parameters of the first data block; the prediction parameters include the prediction direction and the prediction cost in the encoding process of the target image frame.
The method according to claim 6, wherein the data calculation step comprises:

The coding parameter corresponding to the first data block is determined according to the read total information amount, prediction direction and prediction cost of the first data block.
The method according to claim 5, wherein the parameter reading step comprises:

The information storage address of the first data block is obtained according to the position of the first data block in the image.
The method according to claim 6, wherein the encoding processing unit further executes one or more steps in the process of determining the total amount of information of the reference image frame of the target image frame;

The step of obtaining the amount of information further includes:

Determine the amount of information transferred between the target image frame and the reference image frame according to the total information amount of the first data block;

The data calculation step also includes:

obtaining the initial amount of information of the second data block in the reference image frame;

The transfer information amount of the first data block is superimposed on the initial information amount of the second data block in the reference image frame to obtain the total information amount of the second data block in the reference image frame.
The method according to claim 9, wherein the parameter reading step further comprises:

Read the prediction cost coefficient;

The determining the amount of information transferred between the target image frame and the reference image frame according to the total information amount of the first data block includes:

The transfer information amount of the first data block is determined according to the total information amount of the first data block and the prediction cost coefficient.
The method according to claim 9, wherein the parameter reading step further comprises:

According to the position of the first data block in the target image frame and the motion vector, the position of the second data block of the amount of information to be superimposed in the reference image frame is determined.
The method according to claim 4, wherein the data interaction step comprises:

The encoding parameters are stored in the corresponding storage unit for subsequent encoding.
The method according to claim 9, wherein the data interaction step comprises:

The total amount of information of the second data block in the reference image frame is passed to other encoding processing units used in association.
A video coding method, comprising:

dividing the target image frame in the video into a plurality of first data blocks;

Inputting the plurality of first data blocks in batches into a plurality of encoding processing units that are executed in sequence; after the encoding processing unit has performed the processing of the previous batch of the first data blocks, executes the processing of the next batch of the first data blocks , the encoding processing unit correspondingly executes one or more steps in the process of determining the encoding parameters of the image frame, and in a partial time period during which the multiple encoding processing units perform processing, at least two encoding processing units perform processing synchronously;

encoding the target image frame according to encoding parameters corresponding to the plurality of first data blocks respectively;

The encoding result of the video is obtained according to the encoding result of the image frame.
A video coding method, comprising:

Dividing a specific area of the target image frame in the video into a plurality of first data blocks, and the target area is an area where the content of the target image frame changes relative to the reference image frame;

The plurality of first data blocks are input in batches into four encoding processing units that are executed in sequence; after the encoding processing unit has performed the processing of the previous batch of the first data blocks, the processing of the next batch of the first data blocks is performed. , the four encoding processing units respectively perform the quantization parameter calculation in the determination process of the encoding parameter of the image frame according to the reading step, the information content acquisition step, the quantization parameter and the information content calculation step, and the quantization parameter and the information content write step, in During a part of the time period when the four encoding processing units perform processing, at least two encoding processing units perform processing synchronously;

encoding the target image frame according to encoding parameters corresponding to the four first data blocks respectively;

The encoding result of the video is obtained according to the encoding result of the image frame.
An image frame processing method, comprising:

dividing the target image frame into a plurality of first data blocks;

The multiple first data blocks are input in batches into multiple processing units that are executed in sequence; after the processing unit performs the processing of the previous batch of the first data blocks, the processing of the next batch of the first data blocks is performed, so The processing unit correspondingly executes one or more steps in the processing process of the target image frame, and in a partial time period during which the multiple encoding processing units perform processing, at least two encoding processing units perform processing synchronously.
An image frame processing method, characterized in that, applied to a software service platform, comprising:

Obtain at least one target video submitted through the software service client;

dividing the image frame of the target video into a plurality of first data blocks;

Inputting the plurality of first data blocks in batches into a plurality of encoding processing units that are executed in sequence; after the encoding processing unit has performed the processing of the previous batch of the first data blocks, executes the processing of the next batch of the first data blocks , the encoding processing unit correspondingly executes one or more steps in the process of determining the encoding parameters of the target image frame, and in a part of the time period during which the multiple encoding processing units perform processing, at least two encoding processing units are synchronized perform processing;

encoding the target image frame according to encoding parameters corresponding to the plurality of first data blocks respectively;

Obtain the encoding result of the target video according to the encoding results of a plurality of image frames;

The encoding result of the target video is provided based on the software service client.
An image frame encoding system, comprising a frame division module, a plurality of encoding processing units and encoding modules executed in sequence;

the frame dividing unit, configured to divide the target image frame into a plurality of first data blocks, and input the plurality of first data blocks into a plurality of encoding processing units in batches;

The encoding processing unit is configured to correspondingly execute one or more steps in the process of determining the encoding parameters of the target image frame, and execute the next batch of first data blocks after the processing of the previous batch of first data blocks is completed. For the processing of the data block, in a part of the time period when the plurality of encoding processing units perform processing, at least two encoding processing units perform processing synchronously;

The encoding unit is configured to encode the target image frame according to encoding parameters corresponding to the plurality of first data blocks respectively.
A chip based on a field programmable logic gate array, characterized in that it comprises a frame division module, a plurality of encoding processing units and encoding modules executed in sequence;

the frame dividing unit, configured to divide the target image frame into a plurality of first data blocks, and input the plurality of first data blocks into a plurality of encoding processing units in batches;

The encoding processing unit is configured to correspondingly execute one or more steps in the process of determining the encoding parameters of the target image frame, and execute the next batch of first data blocks after the processing of the previous batch of first data blocks is completed. For the processing of the data block, in a part of the time period when the plurality of encoding processing units perform processing, at least two encoding processing units perform processing synchronously;

The encoding unit is configured to encode the target image frame according to encoding parameters corresponding to the plurality of first data blocks respectively.
An electronic device, comprising: a processor; and

A memory having executable code stored thereon which, when executed, causes the processor to perform the method of any of claims 1-17.
One or more machine-readable media, characterized in that executable code is stored thereon, which, when executed, causes a processor to perform the method of any one of claims 1-17.