CN113038179A - Video encoding method, video decoding method, video encoding device, video decoding device and electronic equipment - Google Patents

Abstract

The present application discloses a video encoding method, a video decoding method, an apparatus and an electronic device, which belong to the technical field of communication. The method includes: obtaining a first video frame of a video to be processed; performing luminance-chroma YUV encoding on the first video frame; obtaining a second video frame of the video to be processed, wherein the second video frame is the same as the Any video frame among the N video frames associated with the first video frame, where N is a positive integer; only luminance Y encoding is performed on the second video frame. The video coding method provided by the present application does not need to reduce the resolution of each video frame to adapt to the bandwidth in the case of limited bandwidth, thereby ensuring the clarity of video playback.

Description

Video encoding method, video decoding method, device and electronic device

technical field

The present application belongs to the field of communication technologies, and in particular relates to a video encoding method, a video decoding method, an apparatus, and an electronic device.

Background technique

With the rapid development of communication technology, electronic devices such as smart phones and tablet computers are becoming more and more popular and become indispensable tools in people's daily life. Among them, the video recording function, as one of the main functions of the electronic device, can be used by the user to dynamically record everything that happens around him, so as to improve the user experience of the electronic device.

At present, in the process of video frame encoding, the bandwidth of the video frame data obtained after transmission encoding may be limited, or in order to meet the video bit rate requirements for real-time or smooth video playback, or in order to save space. , usually by reducing the resolution of the video frame during the encoding process to reduce the occupied space of the video frame data, thereby reducing the definition of the video playback.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present application is to provide a video encoding method, a video decoding method, an apparatus, and an electronic device, which can solve the problem that the current electronic device reduces the video frame resolution due to limited bandwidth, resulting in low resolution of video playback.

In order to solve the above technical problems, this application is implemented as follows:

In a first aspect, an embodiment of the present application provides a video encoding method, including:

Obtain the first video frame of the video to be processed;

performing luminance-chrominance YUV encoding on the first video frame;

Acquiring a second video frame of the video to be processed, wherein the second video frame is any video frame in N video frames associated with the first video frame, and N is a positive integer;

Only luminance Y encoding is performed on the second video frame.

In a second aspect, an embodiment of the present application provides a video decoding method, including:

Obtain the first video frame data and the second video frame data of the video to be processed;

Decode the first video frame data in the to-be-processed video to obtain the first YUV information of the first video frame, wherein the first video frame data is after performing YUV encoding on the first video frame generated data;

Decoding the second video frame data in the to-be-processed video to obtain Y information of the second video frame, wherein the second video frame data is generated after performing Y encoding on the second video frame data;

Based on the first YUV information and the Y information of the second video frame, color transfer processing is performed on the second video frame to obtain a second video frame with color.

In a third aspect, an embodiment of the present application provides a video encoding apparatus, including:

a first video frame acquisition module, used to acquire the first video frame of the video to be processed;

YUV encoding module, for performing luminance-chrominance YUV encoding on the first video frame;

A second video frame obtaining module, configured to obtain a second video frame of the to-be-processed video, wherein the second video frame is any video frame among N video frames associated with the first video frame, The N is a positive integer;

A Y encoding module, configured to perform only luminance Y encoding on the second video frame.

In a fourth aspect, an embodiment of the present application provides a video decoding apparatus, including:

a video frame data acquisition module for acquiring the first video frame data and the second video frame data of the video to be processed;

A first decoding module, configured to decode the first video frame data in the to-be-processed video, and obtain first YUV information of the first video frame, wherein the first video frame data is the data for the first video frame. Data generated after a video frame is YUV encoded;

A second decoding module, configured to decode the second video frame data in the to-be-processed video to obtain Y information of the second video frame, wherein the second video frame data is the data for the second video frame The data generated after the frame is Y-encoded;

A color transfer module, configured to perform color transfer processing on the second video frame based on the first YUV information and the Y information of the second video frame to obtain a second video frame with color.

In a fifth aspect, an embodiment of the present application provides an electronic device, the electronic device includes a processor, a memory, and a program or instruction stored on the memory and executable on the processor, the program or instruction being The processor implements the steps of the method according to the first aspect or the second aspect when executed.

In a sixth aspect, an embodiment of the present application provides a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the implementation is as described in the first aspect or the second aspect steps of the method.

In a seventh aspect, an embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or an instruction, and implement the first aspect Or the method described in the second aspect.

In this embodiment of the present application, YUV encoding is performed on the first video frame of the video to be processed, and only Y encoding is performed on the second video frame associated with the first video frame. In this way, the video encoding method in the embodiment of the present application is used. , because only Y-coding is performed on the second video frame, the data occupied space of the encoded second video frame can be effectively reduced; in the case of ensuring the video clarity, space can be saved and the encoding and decoding speed of the video frame can be improved, It can also ensure real-time playback during video recording; in the case of limited bandwidth, there is no need to reduce the resolution of each video frame to adapt to the bandwidth, thereby ensuring the clarity of video playback.

Description of drawings

1 is a schematic flowchart of a video encoding method provided by an embodiment of the present application;

2 is a schematic diagram of encoding a video frame based on H.265 provided by an embodiment of the present application;

3 is a schematic diagram of an encoding process of a video frame provided by an embodiment of the present application;

4 is a schematic flowchart of a video decoding method provided by an embodiment of the present application;

5 is a schematic diagram of a decoding process of a video frame provided by an embodiment of the present application;

6 is a schematic diagram of a process for calculating RGB provided by an embodiment of the present application;

7 is a schematic structural diagram of a video encoding apparatus provided by an embodiment of the present application;

8 is a schematic structural diagram of a video decoding apparatus provided by an embodiment of the present application;

9 is one of the schematic diagrams of the hardware structure of the electronic device provided by the embodiment of the present application;

FIG. 10 is the second schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

The terms "first", "second" and the like in the description and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the present application can be practiced in sequences other than those illustrated or described herein, and distinguish between "first", "second", etc. The objects are usually of one type, and the number of objects is not limited. For example, the first object may be one or more than one. In addition, "and/or" in the description and claims indicates at least one of the connected objects, and the character "/" generally indicates that the associated objects are in an "or" relationship.

The video coding method provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings through specific embodiments and application scenarios.

Please refer to FIG. 1, which is a schematic flowchart of a video encoding method provided by an embodiment of the present application, which is applied to an electronic device. As shown in FIG. 1, the above video encoding method includes the following steps:

Step 101, obtaining the first video frame of the video to be processed;

Step 102, performing luminance-color YUV encoding on the first video frame;

Step 103, obtaining the second video frame of the video to be processed;

Step 104: Perform only luminance Y encoding on the second video frame.

Based on this, YUV encoding is performed on the first video frame of the video to be processed, and only Y encoding is performed on the second video frame associated with the first video frame. Thus, with the video encoding method in the embodiment of the present application, since only the The second video frame is Y-coded, so that the data occupied space of the encoded second video frame can be effectively reduced; in the case of ensuring the video clarity, space can be saved, the encoding and decoding speed of the video frame can be improved, and the video frame can also be guaranteed. Real-time playback during recording; in turn, the clarity of video playback can be guaranteed.

In the above step 101, during the transmission process of the video to be processed, the electronic device may acquire the first video frame of the video to be processed.

Wherein, the above-mentioned video to be processed may be a video that has been recorded and stored in the electronic device, then, the above-mentioned acquisition of the first video frame of the to-be-processed video may be to extract a frame of video frame in the to-be-processed video as the above-mentioned first video frame. Alternatively, the video to be processed may also be a video being recorded. Then, obtaining the first video frame of the video to be processed may include: collecting the first video frame during recording of the video to be processed.

In addition, the above-mentioned first video frame may be any video frame in the above-mentioned to-be-processed video. For example, when the video to be processed is a video being recorded, the first video frame may be the current video frame of the recorded video; or, it may be any video frame extracted from the video stored by the electronic device.

Of course, the above-mentioned first video frame may also be a video frame that satisfies a preset condition. Specifically, the above-mentioned first video frame may be the first frame of video frame collected in the M-th video frame collection period, and the electronic device collects N+1 frames of video frames in each of the above-mentioned video frame collection periods, M and N All are positive integers.

Alternatively, the above-mentioned first video frame may also be a video frame determined according to the chromaticity of the video frame. More specifically, the above-mentioned obtaining the first video frame may include: between the k-th video frame and the k-th video frame of the video to be processed In the case where the chrominance difference value between 1 frame of video frames exceeds the preset chromaticity range, the kth video frame is determined as the first video frame, wherein the k is an integer greater than 1, that is, the same as the above. A video frame whose chrominance difference value of one frame of video frame exceeds the preset chrominance range is determined as the first video frame that needs to be YUV encoded.

It should be noted that the video frame of the video to be processed may be a video frame using any YUV format, and specifically, the YUV format of the video frame of the video to be processed may be 4:4:4, 4:2:2, Either of 4:2:0 and 4:1:1, etc.

In the above-mentioned step 102, after the electronic device obtains the above-mentioned first video frame, the electronic device can perform a luminance-chrominance (YUV, "Y"" on the above-mentioned first video frame to indicate the brightness, that is, the gray scale value; "U " and "V" represent chromaticity, which is used to describe the color and saturation of the image, and is used to specify the color of the pixel) encoding, that is, to encode the color image information of the first video frame, and the encoded color image information Including luminance information and chrominance information of the image in the first video frame.

In this embodiment of the present application, the YUV format of the video frame in the video to be processed may be any one of 4:4:4, 4:2:2, 4:2:0, and 4:1:1, etc. .

In addition, the above-mentioned encoding of the color image information of the first video frame may be to use any video compression technology to encode the color image information of the video frame, specifically, the video encoding standard H.264 or the video encoding standard H. .265 and other video compression techniques for encoding.

Taking H.265 as an example, H.265 divides an image into coding tree blocks (coding tree blocks, CTUs) instead of 16×16 macroblocks like H.264. According to different coding settings, the size of the coding tree block can be set to 64×64 or a limited 32×32 or 16×16. Among them, larger coding tree blocks can provide higher compression efficiency (and also require higher coding speed). Each coding tree block can be recursively divided, such as using a quad-tree structure to divide into 32×32, 16×16, 8×8 sub-regions, as shown in Figure 2, which is a partition example of a 64×64 coding tree block (It should be noted that in FIG. 2, it is actually a color image). Each picture is further divided into special coding tree blocks called Slices and Tiles. The coding tree unit is the basic coding unit of H.265, just like the macroblock of H.264. A coding tree unit may be partitioned down a Coding Unit (CU), a Prediction Unit (PU), and a Transform Unit (TU).

Each coding tree unit contains 1 luma and 2 chroma coding tree blocks, and syntax elements for recording additional information. It is assumed that the video frame is compressed with YUV 4:2:0 color sampling, so take a 16x16 coding tree unit as an example, which will contain a 16x16 luma coding tree block and two 8x8 chroma coding tree blocks. When using H26.5 to encode the above-mentioned first video frame, after encoding, the luma coding tree block and the chroma coding tree block in the coding tree unit will be retained (that is, the encoded color image information includes luminance information and chrominance information) .

It should be noted that after the electronic device performs YUV encoding on the first video frame, video frame data of the first video frame can be obtained, and the electronic device can send or cache the video frame data.

For example, in the process of recording the video, the electronic device may buffer the first video frame data, so that when the electronic device subsequently receives an instruction to play the recorded video, the electronic device decodes the first video frame data to display the first video frame data. A video frame.

In the foregoing step 103, the electronic device may further acquire the second video frame of the foregoing video to be processed.

It should be noted that the above-mentioned acquiring of the second video frame may be performed after acquiring the above-mentioned first video frame, for example, during the video recording process, the second video frame is a video frame collected after the first video frame; or, The above-mentioned acquisition of the second video frame may also be performed at the same time as the above-mentioned acquisition of the first video frame. For example, in the case where the above-mentioned video to be processed is a stored video, the electronic device can extract multiple video frames of the video to be processed each time. The video frame includes the above-mentioned first video frame and the above-mentioned second video frame, which is not limited herein.

In this embodiment of the present application, the second video frame is any one of the N video frames, and the N video frames are at least one video frame associated with the first video frame.

The above-mentioned first video frame is associated with the above-mentioned N frames of video frames, which may be a preset association relationship between the first video frame and the N-frame video frames, and specifically, may be each video frame in the above-mentioned N-frame video frames The time interval between the first video frame and the first video frame is less than or equal to the preset duration, that is, the N video frames and the first video frame are video frames collected within the preset duration.

It should be noted that, the above-mentioned preset duration may be a duration set according to actual needs. For example, the above-mentioned preset duration may be 0.5s, 1s or 2s, and so on.

As mentioned above, the above-mentioned first video frame may be a video frame collected during the video recording process, then any video frame in the above-mentioned N video frames may also be collected during the video recording process and is the same as the above-mentioned first video frame. The video frame associated with the video frame; as mentioned above, the time interval between each video frame in the above-mentioned N video frames and the above-mentioned first video frame is less than or equal to the preset duration, then the above-mentioned second video frame can also be is collected before or after the first video frame, and the time interval between the first video frame and the first video frame is less than or equal to the preset duration, that is:

The above step 101 may include:

in the process of recording the video to be processed, collecting the first video frame;

Wherein, the second video frame is collected before or after the first video frame, and the time interval between the second video frame and the first video frame is less than or equal to a preset duration.

Based on this, during the video recording process, the image difference between the video frames collected within the preset duration is relatively small, so by taking the video frame collected within the preset duration from the first video frame as the above-mentioned second video frame , which can improve the video encoding quality.

In this embodiment of the present application, the second video frame may be a video frame collected before the first video frame, for example, the second video frame may be a video frame preceding the first video frame, or the above The first video frame is the last video frame within the above-mentioned preset duration, and so on.

Alternatively, the above-mentioned second video frame may also be a video frame collected after the above-mentioned first video frame, for example, the above-mentioned second video frame may be a video frame after the above-mentioned first video frame, or the above-mentioned first video frame The frame is the first video frame within the above preset duration, and so on.

Specifically, the first video frame is the first video frame collected in the Mth video frame collection period, and the electronic device collects N+1 video frames in each of the video frame collection periods, and M is positive integer;

The acquiring the second video frame of the video to be processed may include:

The i+1 th video frame collected in the M th video frame collection period is determined as the second video frame, where the i is an integer less than or equal to the N.

Based on this, the first video frame in each acquisition period in the video recording process is taken as the first video frame, and the other video frames in the acquisition period (that is, the video frames in the above N video frames) are determined as the above-mentioned first video frame. The second video frame, that is, only the first video frame is subjected to YUV encoding in each acquisition period, while the other video frames are all subjected to Y encoding, which can further reduce the number of video frames subjected to YUV encoding; in the case of ensuring video clarity , which can save space, improve the encoding and decoding speed of video frames, and ensure real-time playback during video recording; in the case of limited transmission bandwidth, it can ensure the clarity of video playback; The above encoding process is performed on the video frame of the video frame, and the video encoding quality can also be improved.

It should be noted that the M th video frame collection period may be any video frame collection period in the video recording process, and the electronic device may collect N+1 video frames in each video frame collection period.

For example, as shown in FIG. 3, in the case where N=3, that is, the M-th video frame collection period includes the first video frame, the second video frame, the third video frame, and the fourth video frame, the electronic device The first video frame may be regarded as the first video frame, and the second video frame, the third video frame, and the fourth video frame may be regarded as the second video frame, respectively.

Of course, the above-mentioned N video frames may also be the above-mentioned first video frame or any video frame that satisfies the preset condition during the video recording process, and the second video frame does not satisfy the above-mentioned second video frame after the second video frame. A video frame with preset conditions and associated with the second video frame.

Specifically, performing YUV encoding on the first video frame above may include:

Under the condition that the luminance value of the first video frame is detected to be greater than or equal to the preset luminance value, YUV encoding is performed on the first video frame;

In the above, when it is determined that the second video frame is associated with the first video frame, only Y-coding is performed on the second video frame, including:

It is determined that the first luminance value of the luminance value of the second video frame is smaller than the preset luminance value, and there is no luminance value greater than or equal to the predetermined luminance value between the second video frame and the first video frame. In the case of a video frame with a luminance value, only Y encoding is performed on the second video frame.

Based on this, YUV encoding is performed on the second video frame whose brightness value is greater than or equal to the preset brightness value during the video recording process, and each frame of video that is collected after the second video frame and whose brightness value is less than the preset brightness value is subjected to YUV encoding. The frame (ie, the first video frame) is Y-coded, so that not only can the occupied space of the recorded video be further reduced, but also the coding modes in the video coding process can be more flexible and diverse.

For example, assuming that during the video recording process, the brightness value of the i-th video frame collected is greater than the preset brightness value, and i is a positive integer, YUV encoding is performed on the i-th video frame; if the i+1-th video frame is captured If the brightness values of the frame and the i+2th video frame are both smaller than the preset brightness value, Y-encoding is performed on the i+1th video frame and the i+2th video frame; if the i+3th video frame is collected and the luminance value of the i+4th video frame is greater than the preset luminance value, YUV encoding is performed on the i+3th video frame and the i+4th video frame; if the i+5th video frame is collected If the brightness value is less than the preset brightness value, Y-coding is performed on the i+5th video frame, ..., and so on.

It should be noted that after the electronic device encodes the second video frame, the video frame data of the second video frame can be obtained, and the electronic device can cache the video frame data of the second video frame, so that the electronic device can cache the video frame data of the second video frame. The second video frame may be played back; or, video frame data of the second video frame may also be sent to other devices, so that other electronic devices can view the second video frame, which is not limited herein.

In addition, it should be noted that the above step 103 may be executed after step 102, or before step 102, or concurrently with step 101, or concurrently with step 102, and this application is for the above video In the description of the encoding method, only the case where step 103 is executed after step 102 is shown in FIG. 1 , which is not limited here.

In the above step 104, after acquiring the second video frame, the electronic device may only perform Y encoding on the second video frame, that is, encode the grayscale image information of the second video frame, and the encoded grayscale image The information includes the luminance information of the image in the second video frame, but does not include the chrominance information, so that the space occupied by the video frame data of the encoded second video frame can be reduced.

Please refer to FIG. 4 , which is a schematic flowchart of a video decoding method provided by an embodiment of the present application, which is applied to an electronic device. As shown in FIG. 4 , the above video encoding method includes the following steps:

Step 401, obtaining the first video frame data and the second video frame data of the video to be processed;

Step 402: Decode the first video frame data in the to-be-processed video to obtain the first YUV information of the first video frame, wherein the first video frame data is performed on the first video frame. Data generated after YUV encoding;

Step 403: Decode the second video frame data in the to-be-processed video to obtain Y information of the second video frame, where the second video frame data is Y-encoding the second video frame post-generated data;

Step 404: Based on the first YUV information and the Y information of the second video frame, perform color transfer processing on the second video frame to obtain a second video frame with color.

Based on this, in the case of playing the video to be processed, by performing the YUV information of the first video frame of YUV encoding, the second video frame associated with the first video frame is decoded to restore the color image, so as to achieve The second video frame can be displayed as a color image to ensure the quality of video playback.

In the above step 401, during the process of playing the above video to be processed by the electronic device, the electronic device may acquire first video frame data and second video frame data of the video to be processed.

In the embodiment of the present application, the above-mentioned first video frame data is video frame data obtained after YUV encoding is performed on the first video frame, and the above-mentioned second video frame data is video frame data obtained by only Y-encoding the second video frame , and there is an association relationship between the first video frame corresponding to the first video frame data and the second video frame corresponding to the second video frame data.

Wherein, the obtaining of the first video frame data and the second video frame data may be that the electronic device reads the first video frame data and the second video frame data in the cache; or, it may also be receiving real-time transmission from other electronic devices. The first video frame data and the second video frame data.

In addition, it should be noted that the electronic device for executing the video decoding method and the electronic device for executing the video encoding method in FIG. 1 may be the same electronic device. For example, in the process of recording the above-mentioned video to be processed by the electronic device, the recorded video frame is encoded to obtain video frame data, and the video frame data is buffered, and in the process of playing back the above-mentioned video to be processed by the electronic device, the electronic device records the buffered data. The video frame data is decoded.

Alternatively, the electronic device that executes the video decoding method and the electronic device that executes the video encoding method in FIG. 1 may be different electronic devices. For example, in the process of recording the video to be processed, the electronic device 1 encodes the recorded video frame to obtain video frame data, and transmits the video frame data to the electronic device 2, and the electronic device 2 receives the above-mentioned transmission from the electronic device 1. After the video frame data is obtained, the video frame data can be decoded, so as to play the above-mentioned to-be-processed video.

In the above-mentioned step 402, after the electronic device obtains the above-mentioned first video frame data, the electronic device can decode the first video frame data to obtain the YUV information of the above-mentioned first video frame (that is, the first YUV information); Similarly, In the foregoing step 403, after the electronic device obtains the second video frame data, the electronic device may decode the second video frame data to obtain the Y information of the second video frame.

It should be noted that the decoding of the first video frame data and the decoding of the second video frame data may be performed respectively after the operations of acquiring the first video frame data and the second video frame data are completed, and the first video frame data and the second video frame data are respectively performed. The decoding of one video frame data may be performed before decoding the second video frame data, as shown in FIG. 4 , or, the decoding of the first video frame data may be performed or the decoding of the second video frame data may be performed. It is performed after that, or, the decoding of the first video frame data and the decoding of the second video frame data can also be performed simultaneously.

Alternatively, the decoding of the first video frame data may be performed while decoding the second video frame data, or may be performed after the electronic device obtains the first video frame data, that is, the decoding of the first video frame data is performed, that is, there is no need to wait for obtaining the first video frame data. Two video frame data; after the electronic device obtains the second video frame data, decoding the second video frame data, and obtaining the second video frame data may be performed before the decoding of the first video frame data is completed, or may be performed in It is executed after the decoding of the first video frame data is completed, which is not limited herein.

The above-mentioned decoding of the first video frame data and the second video frame data may be to decode each video frame data by a preset decoding method, that is, there is a chrominance coding unit in the first video frame data, and after decoding The color image information of the first video frame is obtained; there is only a luminance coding unit in the data center of the second video frame, and no chrominance coding unit exists, and the grayscale image information of the second video frame can be obtained after decoding.

In addition, the above-mentioned first YUV information may include luminance information and chrominance information of the first video frame; and the above-mentioned Y information may include only luminance information of the second video frame.

In the above step 404, when the electronic device obtains the first YUV information of the first video frame and the Y information of the second video frame, the electronic device can obtain the YUV information of the above-mentioned second video frame through a preset method by calculation. Or color display information associated with YUV information, etc.

For example, the electronic device may directly use the chrominance information in the first YUV information of the first video frame as the chrominance information of the second video frame associated with the first video frame, so as to restore the color image of the second video frame The YUV information includes the above-mentioned Y information (ie, luminance information) of the second video frame and chrominance information of the first video frame.

Among them, because in the image display process, the YUV information of the image is usually converted into the red (Red) green (Green) blue (Blue) (referred to as "RGB") information of each pixel to realize the display of color images. .

Specifically, the above-mentioned color transfer processing on the second video frame may include:

Based on a preset deep neural network model, the red, green, and blue RGB information of the second video frame is obtained by calculation.

Based on this, the RGB information of the second video frame is calculated through the neural network model, so that the color image of the second video frame after color transfer is closer to the actual collected color image, and the image quality during video playback is further improved.

It should be noted that the above-mentioned deep neural network model may be a model preset in an electronic device before performing color transfer processing on the above-mentioned second video frame, and the deep neural network model may be a model obtained by training a large amount of video frame data. , the training process may include: inputting the samples of the training set into the initial deep neural network model, and continuously iteratively updating the weights of the initial deep neural network model until the loss of the model obtained after the iterative update tends to remain unchanged, and finally Obtain the above-mentioned deep neural network model, wherein, each sample in the above-mentioned training set may include at least one third video frame with YUV information, a fourth video frame with only Y information and associated with each third video frame, and each third video frame. Labeling data of four video frames, the labeling data may be actual YUV information of the fourth video frame.

In the embodiment of the present application, the above-mentioned calculation based on the preset deep neural network model to obtain the RGB information of the second video frame may include:

Inputting the first YUV information and the Y information of the second video frame into the deep neural network model, and calculating the RGB information of the second video frame; or,

Input the second YUV information and the Y information of the second video frame into the deep neural network model, and calculate the RGB information of the second video frame, wherein the second YUV information is in the first YUV information of the N-1th video frame collected after the video frame, and the YUV information of the first video frame is calculated based on the first YUV information and the Y information of the first video frame.

Based on this, in the process of performing color transfer processing on the second video frame, the YUV information of the first video frame may be directly used for color transfer; or, at least one frame of video is spaced between the second video frame and the first video frame. In the case of frames, the YUV information of the previous video frame can also be used for color transfer, so that the color transfer processing method is more flexible and diverse.

Taking the decoding process shown in FIG. 5 as an example, in the case where the electronic device decodes the video frames collected in the Mth video frame collection period, it is assumed that the Mth video frame collection period includes the first video frame, the first video frame, the The 2nd video frame, the 3rd video frame and the 4th video frame, and the 1st video frame is the video frame of YUV encoding, and the 2nd video frame, the 3rd video frame and the 4th video frame are all video frames. In order to perform Y-coded video frames, then, when the electronic device displays the second video frame, the third video frame and the fourth video frame, the second video frame, the third video frame and the fourth video frame are respectively displayed. The Y decoding of the frame can be performed by inputting the YUV information of the first video frame and the Y information of any one of the second to fourth video frames into the above-mentioned deep neural network model, and calculating to obtain the above-mentioned second to fourth video frames in the video frame. Corresponding to the RGB information of the video frame, realize the color transfer processing of each video frame in the 2nd to 4th video frames, and display the video frame after the color transfer process;

Alternatively, the YUV information of the first frame of video frame and the Y information of the second frame of video frame can also be input into the deep neural network model, and the RGB information of the second frame of video frame is obtained by calculating; The YUV information corresponding to the RGB information and the Y information of the third video frame are input into the deep neural network model, and the RGB information of the third video frame is calculated; The Y information of the four video frames is input to the deep neural network model, and the RGB information of the fourth video frame is calculated to realize the color transfer processing of each video frame that is Y-coded.

In addition, the above-mentioned calculation based on the preset deep neural network model to obtain the RGB information of the second video frame may be by converting the decoded YUV information of the first video frame or the previous video frame into RGB components by color, and then Combine the Y channel of the second video frame (such as through the Concat function) into a 4-channel input, and output an RGB component image after calculation through the neural network (Network) model, and the output image corresponds to the second video frame. The rendered image (ie, the second video frame with color) is shown in Figure 6.

It should be noted that, in the video encoding method provided by the embodiments of the present application, the execution body may be a video encoding device, or a control module in the video encoding device for the video encoding method. In the embodiments of the present application, the video encoding device provided by the embodiments of the present application is described by taking the video encoding method performed by the video encoding device as an example.

Referring to FIG. 7 , an embodiment of the present application provides a video encoding apparatus. As shown in FIG. 7 , the video encoding apparatus 700 includes:

A first video frame obtaining module 701, configured to obtain a first video frame of a video to be processed;

YUV encoding module 702, configured to perform luminance-chrominance YUV encoding on the first video frame;

A second video frame obtaining module 703, configured to obtain a second video frame of the to-be-processed video, where the second video frame is any video frame among N video frames associated with the first video frame , the N is a positive integer;

The Y encoding module 704 is configured to perform only luminance Y encoding on the second video frame.

Based on this, YUV encoding is performed on the first video frame of the video to be processed, and only Y encoding is performed on the second video frame associated with the first video frame. Thus, with the video encoding method in the embodiment of the present application, since only the The second video frame is Y-coded, so that the data occupied space of the encoded second video frame can be effectively reduced. Therefore, when the bandwidth is limited, it is not necessary to reduce the resolution of each video frame to adapt to the bandwidth. In turn, the clarity of the video frame can be guaranteed.

Optionally, the first video frame acquisition module 701 is specifically used for:

in the process of recording the video to be processed, collecting the first video frame;

Wherein, the second video frame is collected before or after the first video frame, and the time interval between the second video frame and the first video frame is less than or equal to a preset duration.

Based on this, during the video recording process, the image difference between the video frames collected within the preset duration is relatively small, so by taking the video frame collected within the preset duration from the first video frame as the above-mentioned second video frame , which can improve the video encoding quality.

Optionally, the first video frame is the first video frame collected in the Mth video frame collection period, and the electronic device collects N+1 video frames in each of the video frame collection periods;

The second video frame acquisition module 703 is specifically used for:

The i+1 th video frame collected in the M th video frame collection period is determined as the second video frame, where the i is an integer less than or equal to the N.

Based on this, the first video frame in each capture period in the video recording process is determined as the above-mentioned first video frame, and the other video frames in the capture period are determined as the above-mentioned second video frame, that is, in each capture period, only The first video frame is YUV encoded, and other video frames are Y encoded, which can further reduce the number of video frames for YUV encoding; in the case of ensuring video clarity, it can save space and improve the encoding and decoding speed of video frames. , it can also ensure real-time playback during video recording; in the case of limited transmission bandwidth, it can ensure the clarity of video playback; in addition, periodically performing the above encoding process on the video frames of the video to be processed can also improve the video quality. encoding quality.

Optionally, the first video frame acquisition module 701 is specifically used for:

In the case that the chrominance difference value between the kth video frame of the video to be processed and the k-1th video frame exceeds the preset chrominance range, determine the kth video frame as the first video frame, wherein the k is an integer greater than 1;

The second video frame acquisition module 703 is specifically used for:

It is detected that the chrominance difference value between the i-th video frame of the video to be processed and the k-th video frame is within the preset chromaticity range, and the i-th video frame and the In the case where there is no target video frame between the k video frames, the i-th video frame is determined as the second video frame, wherein the target video frame is: the video frame between the k-th video frame and the k-th video frame The video frame whose chrominance difference value exceeds the preset chrominance range; the i is an integer greater than the k.

Based on this, YUV encoding is performed on the second video frame whose brightness value is greater than or equal to the preset brightness value during the video recording process, and each frame of video that is collected after the second video frame and whose brightness value is less than the preset brightness value is subjected to YUV encoding. The frame (ie, the first video frame) is Y-coded, so that not only can the occupied space of the recorded video be further reduced, but also the coding modes in the video coding process can be more flexible and diverse.

It should be noted that, in the video decoding method provided by the embodiments of the present application, the execution body may be a video decoding device, or a control module used in the video decoding method in the video decoding device. In the embodiments of the present application, a video decoding method performed by a video decoding device is used as an example to describe the video decoding device provided by the embodiments of the present application.

Referring to FIG. 8 , an embodiment of the present application provides a video decoding apparatus. As shown in FIG. 8 , the video decoding apparatus 800 includes:

A video frame data acquisition module 801, configured to acquire the first video frame data and the second video frame data of the video to be processed;

The first decoding module 802 is configured to decode the first video frame data in the to-be-processed video to obtain the first YUV information of the first video frame, wherein the first video frame data is for the Data generated after the first video frame is YUV encoded;

The second decoding module 803 is configured to decode the second video frame data in the to-be-processed video to obtain Y information of the second video frame, wherein the second video frame data is the second video frame data for the second video frame. The data generated after the video frame is Y-encoded;

The color transfer module 804 is configured to perform color transfer processing on the second video frame based on the first YUV information and the Y information of the second video frame to obtain a second video frame with color.

Based on this, in the case of playing the video to be processed, by performing the YUV information of the first video frame of YUV encoding, the second video frame associated with the first video frame is decoded to restore the color image, so as to achieve The second video frame can be displayed as a color image to ensure the quality of video playback.

Optionally, the color transfer module 804 is specifically configured to:

Based on a preset deep neural network model, the red, green, and blue RGB information of the second video frame is obtained by calculation.

Based on this, the RGB information of the second video frame is calculated through the neural network model, so that the color image of the second video frame after color transfer is closer to the actual collected color image, and the image quality during video playback is further improved.

Optionally, the N is greater than 1;

The color transfer module 804 is specifically used for:

Inputting the first YUV information and the Y information of the second video frame into the deep neural network model, and calculating the RGB information of the second video frame; or,

Input the second YUV information and the Y information of the second video frame into the deep neural network model, and calculate the RGB information of the second video frame, wherein the second YUV information is in the first YUV information of the N-1th video frame collected after the video frame, and the YUV information of the first video frame is calculated based on the first YUV information and the Y information of the first video frame.

Based on this, in the process of performing color transfer processing on the second video frame, the YUV information of the first video frame may be directly used for color transfer; or, at least one frame of video is spaced between the second video frame and the first video frame. In the case of frames, the YUV information of the previous video frame can also be used for color transfer, so that the color transfer processing method is more flexible and diverse.

The video encoding apparatus and the video decoding apparatus in the embodiments of the present application may be apparatuses, and may also be components, integrated circuits, or chips in a terminal. The apparatus may be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a palmtop computer, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (personal digital assistant). assistant, PDA), etc., the non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc. This application Examples are not specifically limited.

The video encoding apparatus and the video decoding apparatus in the embodiments of the present application may be apparatuses having an operating system. The operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, which are not specifically limited in the embodiments of the present application.

The video encoding apparatus provided in the embodiment of the present application can implement each process implemented by the method embodiment in FIG. 1 , and the video decoding apparatus provided in the embodiment of the present application can implement each process implemented in the method embodiment in FIG. 4 . To avoid repetition, I won't go into details here.

Optionally, as shown in FIG. 9 , an embodiment of the present application further provides an electronic device 900 . The electronic device 900 includes a processor 901 , a memory 902 , and a program stored in the memory 902 and running on the processor 901 . or instruction, when the program or instruction is executed by the processor 901, each process of the foregoing video encoding method embodiment or the foregoing video decoding method embodiment can be achieved, and the same technical effect can be achieved. To avoid repetition, details are not repeated here.

It should be noted that the electronic devices in the embodiments of the present application include the aforementioned mobile electronic devices and non-mobile electronic devices.

FIG. 10 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.

The electronic device 1000 includes but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010, etc. part.

Those skilled in the art can understand that the electronic device 1000 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 1010 through a power management system, so that the power management system can manage charging, discharging, and power functions. consumption management and other functions. The structure of the electronic device shown in FIG. 10 does not constitute a limitation on the electronic device, and the electronic device may include more or less components than the one shown, or combine some components, or arrange different components, which will not be repeated here. .

The processor 1010 is used for:

Obtain the first video frame of the video to be processed;

performing luminance-chrominance YUV encoding on the first video frame;

Acquiring a second video frame of the video to be processed, wherein the second video frame is any video frame in N video frames associated with the first video frame, and N is a positive integer;

Only luminance Y encoding is performed on the second video frame.

Based on this, YUV encoding is performed on the first video frame of the video to be processed, and only Y encoding is performed on the second video frame associated with the first video frame. Thus, with the video encoding method in the embodiment of the present application, since only the The second video frame is Y-coded, so that the data occupied space of the encoded second video frame can be effectively reduced. Therefore, when the bandwidth is limited, it is not necessary to reduce the resolution of each video frame to adapt to the bandwidth. In turn, the clarity of the video frame can be guaranteed.

Optionally, the processor 1010 is specifically configured to:

in the process of recording the video to be processed, collecting the first video frame;

Wherein, the second video frame is collected before or after the first video frame, and the time interval between the second video frame and the first video frame is less than or equal to a preset duration.

Based on this, during the video recording process, the image difference between the video frames collected within the preset duration is relatively small, so by taking the video frame collected within the preset duration from the first video frame as the above-mentioned second video frame , which can improve the video encoding quality.

Optionally, the first video frame is the first video frame collected in the Mth video frame collection period, and the electronic device collects N+1 video frames in each of the video frame collection periods;

The processor 1010 is specifically used for:

The i+1 th video frame collected in the M th video frame collection period is determined as the second video frame, where the i is an integer less than or equal to the N.

Based on this, the first video frame in each capture period in the video recording process is determined as the above-mentioned first video frame, and the other video frames in the capture period are determined as the above-mentioned second video frame, that is, in each capture period, only The first video frame is YUV encoded, and other video frames are Y encoded, which can further reduce the number of video frames for YUV encoding; in the case of ensuring video clarity, it can save space and improve the encoding and decoding speed of video frames. , it can also ensure real-time playback during video recording; in the case of limited transmission bandwidth, it can ensure the clarity of video playback; in addition, periodically performing the above encoding process on the video frames of the video to be processed can also improve the video quality. encoding quality.

Optionally, the processor 1010 is specifically configured to:

In the case that the chrominance difference value between the kth video frame of the video to be processed and the k-1th video frame exceeds the preset chrominance range, determine the kth video frame as the first video frame, wherein the k is an integer greater than 1;

It is detected that the chrominance difference value between the i-th video frame of the video to be processed and the k-th video frame is within the preset chromaticity range, and the i-th video frame and the In the case where there is no target video frame between the k video frames, the i-th video frame is determined as the second video frame, wherein the target video frame is: the video frame between the k-th video frame and the k-th video frame The video frame whose chrominance difference value exceeds the preset chrominance range; the i is an integer greater than the k.

Based on this, YUV encoding is performed on the second video frame whose brightness value is greater than or equal to the preset brightness value during the video recording process, and each frame of video that is collected after the second video frame and whose brightness value is less than the preset brightness value is subjected to YUV encoding. The frame (ie, the first video frame) is Y-coded, so that not only can the occupied space of the recorded video be further reduced, but also the coding modes in the video coding process can be more flexible and diverse.

Alternatively, processor 1010 for:

Obtain the first video frame data and the second video frame data of the video to be processed;

Decode the first video frame data in the to-be-processed video to obtain the first YUV information of the first video frame, wherein the first video frame data is after performing YUV encoding on the first video frame generated data;

Decoding the second video frame data in the to-be-processed video to obtain Y information of the second video frame, wherein the second video frame data is generated after performing Y encoding on the second video frame data;

Based on the first YUV information and the Y information of the second video frame, color transfer processing is performed on the second video frame to obtain a second video frame with color.

Based on this, in the case of playing the video to be processed, by performing the YUV information of the first video frame of YUV encoding, the second video frame associated with the first video frame is decoded to restore the color image, so as to achieve The second video frame can be displayed as a color image to ensure the quality of video playback.

Optionally, the processor 1010 is specifically configured to:

Based on a preset deep neural network model, the red, green, and blue RGB information of the second video frame is obtained by calculation.

Based on this, the RGB information of the second video frame is calculated through the neural network model, so that the color image of the second video frame after color transfer is closer to the actual collected color image, and the image quality during video playback is further improved.

Optionally, the N is greater than 1;

The processor 1010 is specifically used for:

Inputting the first YUV information and the Y information of the second video frame into the deep neural network model, and calculating the RGB information of the second video frame; or,

Input the second YUV information and the Y information of the second video frame into the deep neural network model, and calculate the RGB information of the second video frame, wherein the second YUV information is in the first YUV information of the N-1th video frame collected after the video frame, and the YUV information of the first video frame is calculated based on the first YUV information and the Y information of the first video frame.

Based on this, in the process of performing color transfer processing on the second video frame, the YUV information of the first video frame may be directly used for color transfer; or, at least one frame of video is spaced between the second video frame and the first video frame. In the case of frames, the YUV information of the previous video frame can also be used for color transfer, so that the color transfer processing method is more flexible and diverse.

It should be understood that, in this embodiment of the present application, the input unit 1004 may include a graphics processor (Graphics Processing Unit, GPU) 10041 and a microphone 10042. camera) to process the image data of still pictures or videos. The display unit 1006 may include a display panel 10061, which may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 1007 includes a touch panel 10071 and other input devices 10072 . The touch panel 10071 is also called a touch screen. The touch panel 10071 may include two parts, a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which will not be repeated here. Memory 1009 may be used to store software programs as well as various data, including but not limited to application programs and operating systems. The processor 1010 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the like, and the modem processor mainly processes wireless communication. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 1010.

An embodiment of the present application further provides a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, each process of the foregoing video encoding method or the foregoing video decoding method embodiment is implemented , and can achieve the same technical effect, in order to avoid repetition, it is not repeated here.

Wherein, the processor is the processor in the electronic device described in the foregoing embodiments. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like.

An embodiment of the present application further provides a chip, where the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or an instruction to implement the above video encoding method or the above video Each process of the decoding method embodiment can achieve the same technical effect, and to avoid repetition, it will not be repeated here.

It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, a system-on-chip, a system-on-a-chip, or a system-on-a-chip, or the like.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in the reverse order depending on the functions involved. To perform functions, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to some examples may be combined in other examples.

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or in a part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, CD-ROM), including several instructions to make a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of this application.

The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific embodiments, which are merely illustrative rather than restrictive. Under the inspiration of this application, without departing from the scope of protection of the purpose of this application and the claims, many forms can be made, which all fall within the protection of this application.

Claims (10)

1. A video encoding method, comprising:

acquiring a first video frame of a video to be processed;

performing luminance-chrominance YUV encoding on the first video frame;

acquiring a second video frame of the video to be processed, wherein the second video frame is any one of N video frames associated with the first video frame, and N is a positive integer;

only luma Y-coding is performed on the second video frame.

2. The method of claim 1, wherein obtaining the first video frame of the video to be processed comprises:

collecting the first video frame in the process of recording the video to be processed;

the second video frame is acquired before or after the first video frame, and the time interval between the second video frame and the first video frame is less than or equal to a preset time length.

3. The method according to claim 2, wherein the first video frame is a 1 st video frame captured in an mth video frame capture period, and the electronic device captures N +1 video frames in each of the video frame capture periods;

the acquiring a second video frame of the video to be processed includes:

determining an i +1 frame video frame acquired in the Mth video frame acquisition period as a second video frame, wherein i is an integer less than or equal to N.

4. The method of claim 1, wherein obtaining the first video frame of the video to be processed comprises:

determining a kth frame video frame of the video to be processed as a first video frame under the condition that a chromaticity difference value between the kth frame video frame and a kth-1 frame video frame of the video to be processed exceeds a preset chromaticity range, wherein k is an integer greater than 1;

the acquiring a second video frame of the video to be processed includes:

determining an ith frame video frame of the video to be processed as a second video frame when detecting that a chrominance difference value between the ith frame video frame and the kth frame video frame is in the preset chrominance range and a target video frame does not exist between the ith frame video frame and the kth frame video frame, wherein the target video frame is: the video frames with the chrominance difference value exceeding the preset chrominance range with the k frame video frames; and i is an integer greater than k.

5. A video decoding method, comprising:

acquiring first video frame data and second video frame data of a video to be processed;

decoding the first video frame data in the video to be processed to obtain first YUV information of a first video frame, wherein the first video frame data is generated after YUV coding is performed on the first video frame;

decoding second video frame data in the video to be processed to obtain Y information of a second video frame, wherein the second video frame data is generated after Y coding is performed on the second video frame;

and carrying out color transfer processing on the second video frame based on the first YUV information and the Y information of the second video frame to obtain a second video frame with color.

6. The method of claim 5, wherein the color-transferring the second video frame comprises:

and calculating to obtain the red, green and blue (RGB) information of the second video frame based on a preset deep neural network model.

7. The method of claim 6, wherein N is greater than 1;

the calculating to obtain the RGB information of the second video frame based on the preset deep neural network model includes:

inputting the first YUV information and the Y information of the second video frame into the deep neural network model, and calculating to obtain RGB information of the second video frame; or,

inputting second YUV information and Y information of the second video frame into the deep neural network model, and calculating to obtain RGB information of the second video frame, wherein the second YUV information is YUV information of an N-1 frame video frame collected after the first video frame, and the YUV information of the 1 frame video frame is calculated based on the first YUV information and the Y information of the 1 frame video frame.

8. A video encoding apparatus, comprising:

the first video frame acquisition module is used for acquiring a first video frame of a video to be processed;

a YUV encoding module for performing luminance-chrominance YUV encoding on the first video frame;

a second video frame obtaining module, configured to obtain a second video frame of the video to be processed, where the second video frame is any one of N video frames associated with the first video frame, and N is a positive integer;

and the Y coding module is used for only carrying out brightness Y coding on the second video frame.

9. A video decoding apparatus, comprising:

the video frame data acquisition module is used for acquiring first video frame data and second video frame data of a video to be processed;

the first decoding module is used for decoding the first video frame data in the video to be processed to obtain first YUV information of a first video frame, wherein the first video frame data is generated after YUV coding is performed on the first video frame;

the second decoding module is configured to decode second video frame data in the video to be processed to obtain Y information of a second video frame, where the second video frame data is data generated by Y-coding the second video frame;

and the color transfer module is used for carrying out color transfer processing on the second video frame based on the first YUV information and the Y information of the second video frame to obtain a second video frame with color.

10. An electronic device comprising a processor, a memory, and a program or instructions stored on the memory and executable on the processor, which when executed by the processor, implement the steps of the video encoding method of any of claims 1 to 4 or the video decoding method of any of claims 5 to 7.

Images