CN111327898A

CN111327898A - Video coding method and device, electronic equipment and storage medium

Info

Publication number: CN111327898A
Application number: CN201811535610.6A
Authority: CN
Inventors: 黄翊凇
Original assignee: China Mobile Communications Group Co Ltd; China Mobile Group Guangxi Co Ltd
Current assignee: China Mobile Communications Group Co Ltd; China Mobile Group Guangxi Co Ltd
Priority date: 2018-12-14
Filing date: 2018-12-14
Publication date: 2020-06-23
Anticipated expiration: 2038-12-14
Also published as: CN111327898B

Abstract

The invention discloses a video coding method, a video coding device, electronic equipment and a storage medium, wherein the video coding method comprises the following steps: determining each key frame and a corresponding common frame in a video file to be coded; aiming at any key frame and a corresponding common frame thereof, dividing the first key frame and the corresponding common frame thereof into a plurality of image macro blocks according to pixel values of pixel points contained in the key frame and the corresponding common frame thereof; aiming at each image macro block, determining a target variation rate of the pixel value of the image macro block according to the pixel value of each pixel point in the image macro block in the key frame and the corresponding common frame, determining a target frame rate corresponding to the target variation rate according to the preset corresponding relation between the variation rate and the frame rate, adjusting the frame number corresponding to the image macro block by adopting the target frame rate, and coding each frame corresponding to the image macro block, thereby further improving the quality of video coding.

Description

Video coding method and device, electronic equipment and storage medium

Technical Field

The present invention relates to the field of video processing technologies, and in particular, to a method and an apparatus for video encoding, an electronic device, and a storage medium.

Background

At present, high-definition videos are more and more popular, but the requirements of transmission broadband and storage are limited, the high-definition videos are often required to be compressed, coded and transmitted or compressed, coded and stored, and the coding quality of the high-definition videos is guaranteed as far as possible under the condition that a high video compression ratio is guaranteed.

However, the conventional video coding method generally adopts a bisection method, and simply divides two pictures, i.e., a static picture and a dynamic picture, and codes the divided static picture and dynamic picture, respectively, and the coding quality of the coding method is low. In addition, the prior art also discloses a variable frame rate coding method, which discloses that if a current coding macro block is located in a static area, a first preset coding mode is used for coding the current coding macro block, and if the current coding macro block is located in a motion area, a second preset coding mode is used for coding the current macro block.

Disclosure of Invention

The embodiment of the invention provides a video coding method, a video coding device, electronic equipment and a storage medium, which are used for solving the problem of low quality of video coding in the prior art.

The embodiment of the invention provides a video coding method, which comprises the following steps:

determining each key frame and a corresponding common frame in a video file to be coded;

aiming at any key frame and a corresponding common frame thereof, dividing the first key frame and the corresponding common frame thereof into a plurality of image macro blocks according to pixel values of pixel points contained in the key frame and the corresponding common frame thereof;

and aiming at each image macro block, determining a target variation rate of the pixel value of the image macro block according to the pixel value of each pixel point in the image macro block in the key frame and the corresponding common frame, determining a target frame rate corresponding to the target variation rate according to a preset corresponding relation between the variation rate and the frame rate, adjusting the frame number corresponding to the image macro block by adopting the target frame rate, and coding each frame corresponding to the image macro block.

Further, the determining the target variation rate of the pixel value of the image macro block according to the pixel value of each pixel point in the image macro block in the key frame and the corresponding common frame thereof includes:

aiming at each pixel point, judging whether the pixel point changes in the corresponding pixel value of the key frame and the corresponding common frame; if yes, recording the pixel point;

and determining the target variation rate of the pixel value of the image macro block according to the number of the recorded pixel points and the total number of the pixel points in the image macro block.

Further, the adjusting the frame number corresponding to the image macro block by using the target frame number rate includes:

if the current frame rate of the image macro block is equal to the target frame rate, no adjustment is performed;

if the current frame number rate of the image macro block is smaller than the target frame number rate, determining a first difference value between the target frame number rate and the current frame number rate, and adding an image frame of the first difference value in the image frame corresponding to the image macro block between the key frame and the corresponding common frame;

and if the current frame number rate of the image macro block is greater than the target frame number rate, determining a second difference value between the current frame number rate and the target frame number rate, and subtracting the image frame of the second difference value from the image frame of the image macro block corresponding to the key frame and the corresponding common frame.

Further, the image frames, in which the image macro blocks add the first difference value in the image frames corresponding to the key frame and the corresponding common frame, include:

and adding the image frame of the first difference value in the image frame corresponding to the key frame and the corresponding common frame of the image macro block by utilizing an interpolation method.

Further, the image frame obtained by subtracting the second difference value from the image frame corresponding to the image macro block between the key frame and the corresponding common frame comprises:

and subtracting the image frame of the second difference value from the image frame corresponding to the key frame and the corresponding common frame of the image macro block by using a frame extraction method.

Further, after the encoding of each frame corresponding to the image macro block, the method further includes:

and packaging and integrating the first video file corresponding to each encoded image macro block into a second video file.

The invention provides a video coding device, which comprises:

the device comprises a determining module, a coding module and a decoding module, wherein the determining module is used for determining each key frame and a corresponding common frame in a video file to be coded;

the dividing module is used for dividing any key frame and the corresponding common frame into a plurality of image macro blocks according to pixel values of pixel points contained in the key frame and the corresponding common frame;

and the coding module is used for determining a target variable rate of the pixel value of each image macro block according to the pixel value of each pixel point in the image macro block in the key frame and the corresponding common frame, determining a target frame rate corresponding to the target variable rate according to the preset corresponding relation between the variable rate and the frame rate, adjusting the frame number corresponding to the image macro block by adopting the target frame rate, and coding each frame corresponding to the image macro block.

The embodiment of the invention provides electronic equipment, which comprises a memory and a processor;

the processor is used for reading the program in the memory and executing the following processes: determining each key frame and a corresponding common frame in a video file to be coded; aiming at any key frame and a corresponding common frame thereof, dividing the first key frame and the corresponding common frame thereof into a plurality of image macro blocks according to pixel values of pixel points contained in the key frame and the corresponding common frame thereof; and aiming at each image macro block, determining a target variation rate of the pixel value of the image macro block according to the pixel value of each pixel point in the image macro block in the key frame and the corresponding common frame, determining a target frame rate corresponding to the target variation rate according to a preset corresponding relation between the variation rate and the frame rate, adjusting the frame number corresponding to the image macro block by adopting the target frame rate, and coding each frame corresponding to the image macro block.

Further, the processor is specifically configured to determine, for each pixel point, whether a pixel value of the pixel point in the key frame and the corresponding common frame changes; if yes, recording the pixel point; and determining the target variation rate of the pixel value of the image macro block according to the number of the recorded pixel points and the total number of the pixel points in the image macro block.

Further, the processor is specifically configured to not adjust the current frame rate of the image macroblock if the current frame rate is equal to the target frame rate; if the current frame number rate of the image macro block is smaller than the target frame number rate, determining a first difference value between the target frame number rate and the current frame number rate, and adding an image frame of the first difference value in the image frame corresponding to the image macro block between the key frame and the corresponding common frame; and if the current frame number rate of the image macro block is greater than the target frame number rate, determining a second difference value between the current frame number rate and the target frame number rate, and subtracting the image frame of the second difference value from the image frame of the image macro block corresponding to the key frame and the corresponding common frame.

Further, the processor is further configured to encapsulate and integrate the first video file corresponding to each encoded image macro block into a second video file after each frame corresponding to the image macro block is encoded.

The embodiment of the invention provides electronic equipment, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory finish mutual communication through the communication bus;

the memory has stored therein a computer program which, when executed by the processor, causes the processor to perform the steps of any of the methods described above.

An embodiment of the present invention provides a computer-readable storage medium, which stores a computer program executable by an electronic device, and when the program runs on the electronic device, the program causes the electronic device to execute the steps of any one of the above methods.

The embodiment of the invention provides a video coding method, a video coding device, electronic equipment and a storage medium, wherein the video coding method comprises the following steps: determining each key frame and a corresponding common frame in a video file to be coded; aiming at any key frame and a corresponding common frame thereof, dividing the first key frame and the corresponding common frame thereof into a plurality of image macro blocks according to pixel values of pixel points contained in the key frame and the corresponding common frame thereof; and aiming at each image macro block, determining a target variation rate of the pixel value of the image macro block according to the pixel value of each pixel point in the image macro block in the key frame and the corresponding common frame, determining a target frame rate corresponding to the target variation rate according to a preset corresponding relation between the variation rate and the frame rate, adjusting the frame number corresponding to the image macro block by adopting the target frame rate, and coding each frame corresponding to the image macro block.

In the embodiment of the invention, the key frame and the common frame are subjected to image macro block division, and the target variation rate of the pixel value of the image macro block is determined according to whether the pixel value of each pixel point in the image macro block in the key frame and the corresponding common frame is changed, so that the change in the image can be effectively identified, the target frame number rate corresponding to the target variation rate is determined, the frame number corresponding to the image macro block is adjusted, namely the frame number rate corresponding to the image macro block is determined according to the change of the image, and the quality of video coding is effectively improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a flowchart of a video encoding method according to embodiment 1 of the present invention;

fig. 2 is a schematic view of a video clip corresponding to a first key frame according to embodiment 2 of the present invention;

fig. 3 is a schematic diagram of a video segment before image macroblocks are cut according to embodiment 2 of the present invention;

fig. 4 is a schematic diagram illustrating a video segment after image macro blocks are cut according to embodiment 2 of the present invention;

fig. 5 is a schematic diagram of a target frame rate corresponding to each image macroblock according to embodiment 3 of the present invention;

fig. 6 is a schematic diagram illustrating an architecture of a video coding system according to embodiment 4 of the present invention;

fig. 7 is a flowchart of a video encoding method according to embodiment 4 of the present invention;

fig. 8 is a flowchart of a video encoding method according to embodiment 4 of the present invention;

fig. 9 is a schematic structural diagram of a video encoding apparatus according to embodiment 5 of the present invention;

fig. 10 is a schematic structural diagram of an electronic device according to embodiment 6 of the present invention;

fig. 11 is a schematic structural diagram of an electronic device according to embodiment 7 of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the attached drawings, and it should be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Example 1:

fig. 1 is a flowchart of a video encoding method according to an embodiment of the present invention, where the method includes:

s101: and determining each key frame and a corresponding common frame in the video file aiming at the video file to be coded.

The video input end receives a video file to be coded, wherein the video file to be coded can be a third video file which is obtained by firstly carrying out preliminary analysis on an original digital video file, extracting the corresponding video time length, the number of frames per second, the resolution ratio and the like, and then decoding and converting the original digital video file into a lossless uncompressed video file with the highest information degree. The original digital video file may be a video file recorded by a camera, or may be a video file from another place, for example, a video file uploaded by another user.

Then, according to the extracted information of the number of frames per second, the resolution and the like, a plurality of common frames between two key frames can be determined, wherein the key frames in the video file to be coded are periodic by time or fixed frames. Each key frame and the corresponding normal frame image before the next key frame are considered as a video clip (vedio (x)).

Specifically, the determination method of the key frame and the common frame is the prior art, and is not described herein again.

S102: aiming at any key frame and the corresponding common frame thereof, dividing the first key frame and the corresponding common frame thereof into a plurality of image macro blocks according to the pixel values of the pixel points contained in the key frame and the corresponding common frame thereof.

After each key frame and its corresponding common frame, i.e. each video segment (video) (x)), are determined, corresponding analysis is performed on any video segment, and corresponding analysis is performed on the pixel values of the pixels included in the video segment (video) (x)), where the specific analysis content may be the change in the pixel value of each pixel in the video segment and the motion range of the target object.

Then, according to the analysis result, the change of each pixel point in the video segment between the corresponding key frame and the common frame and the pixel value of the pixel point can be determined, then the change corresponding to each pixel point is compared with the pixel value of the pixel point, the change frequency trend of the maximum pixel similarity of the target object on the time line is determined according to the comparison result, then the video segment is cut into a plurality of image macro blocks from the angle of the pixel plane, and the key frame corresponding to the video segment and all the common frames are cut in the same cutting mode.

Specifically, techniques such as convolution integration, h.265 vision, Discrete Cosine Transform (DCT) algorithm technique, and neural network may be used for the cutting.

S103: and aiming at each image macro block, determining a target variation rate of the pixel value of the image macro block according to the pixel value of each pixel point in the image macro block in the key frame and the corresponding common frame, determining a target frame rate corresponding to the target variation rate according to a preset corresponding relation between the variation rate and the frame rate, adjusting the frame number corresponding to the image macro block by adopting the target frame rate, and coding each frame corresponding to the image macro block.

In order to further improve the compression quality of the video file to be encoded, before each frame of image corresponding to the image macro block is compressed and encoded, each image macro block needs to be independently adjusted in frame number, and a specific adjustment mode can be set according to actual conditions.

For each image macro block, according to the pixel value of each pixel point in the image macro block in the key frame and the corresponding common frame, the overall change rate of the pixel value of the image macro block, that is, the target change rate, is determined to reflect the motion and change conditions of the image corresponding to the image macro block.

If the target variation rate corresponding to the image macro block is larger, it indicates that the image motion corresponding to the image macro block is changed more severely, at this time, more image frames are needed to show such a severe action so as to make the sub-video corresponding to the image macro block more smooth, and therefore, an appropriate image frame needs to be added to the video segment corresponding to the image macro block to achieve an appropriate frame number rate.

If the target change rate corresponding to the image macro block is smaller, it indicates that the image motion corresponding to the image macro block is changed more slowly, at this time, it is not necessary to adopt as many image frames to show such slow motion, and therefore, it is necessary to delete an appropriate number of image frames from the video segment corresponding to the image macro block, so as to save the storage space of the video file under the condition of ensuring that the smoothness of the sub-video stream corresponding to the image macro block is not affected.

In order to obtain the suitable frame rate corresponding to each image macro block more accurately, a frame rate threshold list may be preset to record the corresponding relationship between different variation rates and frame rates, where the greater the variation rate, the greater the frame rate, the smaller the variation rate, and the smaller the frame rate. That is, the more rapid the picture movement and change in the video, i.e., the larger the change rate, the higher the frame rate to be set, and conversely, the more gradual the video picture change, i.e., the smaller the change rate, the lower the frame rate to be set.

For example, after any image macroblock is analyzed, it is found that the variation rate of pixel values in each frame of image of all frames corresponding to the image macroblock is 10% -20%, the corresponding frame rate is 12 frames/second, if the variation rate is 20-30%, the corresponding frame rate is 15 frames/second, if the variation rate is 85-100%, the corresponding frame rate is 90 frames/second, and the like.

According to the correspondence in the list and the target variation rate of the pixel values of the image macro block, a target frame number rate corresponding to the target variation rate may be determined, and then the frame number corresponding to the image macro block may be adjusted according to the target frame number rate, and a specific adjustment method is not limited. Then, according to a preset compression coding format, picture compression is performed on the sub-video corresponding to each image macro block of which the frame number is adjusted, wherein the specific compression coding format includes, but is not limited to, various existing video coding formats such as memg, mpeg4, h.245 and the like.

If a multi-core processor is adopted to process each image macro block, the multi-thread technology can be adopted to simultaneously and parallelly carry out the operation on the plurality of image macro blocks, and the speed and the efficiency of video coding can be improved.

In the embodiment of the invention, the target change rate of the pixel value of the image macro block is determined according to the pixel value of each pixel point in the image macro block in the key frame and the corresponding common frame thereof, the target frame number rate corresponding to the target change rate of the pixel value of the image macro block is determined according to the preset relation between the change rate and the frame number rate, the frame number corresponding to the image macro block is adjusted according to the target frame number rate, and each frame corresponding to the image macro block is coded, so that the quality of video coding can be further improved.

Example 2:

on the basis of the above embodiments, in order to better understand the video encoding method described in embodiment 1, the following examples are given:

in order to improve the quality of video coding, the following performs video coding on the third video file with the highest information degree without loss and compression, and the specific manner of obtaining the third video file has been described in the above embodiments, and is not described herein again. Since the process of obtaining the third video file is equivalent to formatting (formatted) the original digital video file, the third video file can be represented by video (formatted). In the embodiment of the present invention, the Original frame rate of the third video file (video (formatted)) can be represented by Original _ fps.

Assuming that the third video file is a video file with 48 frames per second and a total of 10 minutes, i.e. the third video file has a total of 48 × 10 × 60 28800 frames of images, assuming 1 second as one period, that is, every 1 second (i.e., every 48 frames) is set as a key frame, the first frame is used as a key frame, and then every 48 frames is used as a key frame image, the video file has 600 key frames, the 47 frames between two adjacent key frames are normal frame images, each key frame and the 47 frames after the key frame are understood as a video file with 10 minutes of a video clip and 48 frames per second, if a key frame is set every 1 second (i.e. every 48 frames), the video has 48 x 10 x 60-28800 frames of images in total, the first frame is taken as a key frame, then every 48 frames are taken as a key frame image, 600 frames of key frames are summed, and 47 frames between two adjacent key frames are common frames. Each key frame and its following 47 normal frames are considered as a video segment (video (X)), where X is a positive integer not less than 1, in this example X ═ 1.

Fig. 2 is a schematic diagram of a video clip corresponding to a first key frame according to an embodiment of the present invention.

After the video segment corresponding to the first key frame is obtained, the video segment can be divided into a plurality of image macro blocks according to the pixel values of the pixel points included in the video segment, so that each image macro block can be subjected to subsequent operation. Fig. 3 is a schematic diagram before image macro blocks are cut into the video segment according to the embodiment of the present invention, and it can be seen from the diagram that the video segment shows that in a pavilion in rainy days, a child plays a ball, and a car is driving slowly outside the pavilion, and branches are swaying in rain.

The video segment (VEDIO (1)) is analyzed accordingly, and the specific analysis process has been explained in the above embodiments, and is not described herein again. According to the analysis result, from the angle of a pixel plane, cutting the key frame and the common frame corresponding to the video clip into N (1) image macro blocks, wherein the cutting method of the key frame and the common frame is the same, wherein 1 represents that the video clip is a first video clip of a third video file, and the video clip comprises a first key frame; n (1) represents the number of image macroblocks cut by the first video segment.

As shown in fig. 4, 10 image macro blocks are cut out of each frame in the video segment, and meanwhile, for convenience of subsequent operations, the information of the 10 image macro blocks may be stored in a list, specifically, because each frame in the video segment obtains a plurality of image macro blocks by using the same cutting method, for convenience of each image macro block on the 1 st key frame, the coordinates of the upper left-corner pixel point and the coordinates of the lower right-corner pixel point of the image macro block on the 1 st key frame are recorded, so as to form an image macro block information list for the video segment.

The method for cutting image macro blocks is the prior art, and is not described herein again.

Example 3:

in order to determine the target variation rate of the pixel values of the image macro block more accurately, on the basis of the above embodiment, the determining the target variation rate of the pixel values of the image macro block according to the pixel values of each pixel point in the image macro block in the key frame and the corresponding common frame includes:

In order to more accurately determine the target variation rate of the pixel values of the image macro blocks, it may be determined, for each pixel point in any image macro block, whether the pixel value of the pixel point in the key frame and the corresponding common frame changes, and if so, it is determined that the pixel point changes, and therefore, the pixel point needs to be recorded. After all the pixel points in the image macro block are judged and all the changed pixel points in the image macro block are recorded, the first number of the recorded pixel points and the total number of the pixel points in the image macro block are counted, the first number is divided by the total number, and the obtained quotient is the target change rate of the pixel value of the image macro block.

For example, the total number of pixels of an image macroblock in the corresponding key frame and the corresponding normal frame is 1000, where 400 pixels with changed pixel values are generated, and the target change rate corresponding to the image macroblock is 40%.

In order to facilitate the determination of the target frame rate corresponding to the target variation rate, a corresponding relationship between the frame rate and the variation rate may be preset, and after the target variation rate is determined, the corresponding target frame rate may be determined. The target frame rate is a frame rate suitable for the image macro block, and if the current frame rate corresponding to the image macro block is not consistent with the target frame rate, it indicates that the frame rate of the image macro block cannot more accurately display the corresponding image action, and needs to be adjusted, specifically, adjusting the frame number corresponding to the image macro block by using the target frame rate includes:

Because the same dividing manner is adopted for the key frame and each normal frame in the same video segment to obtain each image macro block, that is, for each image macro block, the number of image frames corresponding to the image macro block in the video segment is the same as the number of the key frame plus the normal frame, as described in the above example, every 48 frames are divided into one video segment, one video segment includes one key frame and 47 normal frames, and after the key frame and the normal frame in the video segment are divided into a plurality of image macro blocks in the same dividing manner, for each image macro block, the number of image frames corresponding to the image macro block in the video segment is 48 frames.

If the current frame rate of the image macro block is equal to the target frame rate, it indicates that the current frame rate can well reflect the image motion shown by the image macro block, therefore, frame rate adjustment is not required to be performed on the image macro block. For example, the current frame rate of the image macroblock is 12, and the target frame rate is also 12, and the image macroblock does not need to adjust the frame rate.

If the current frame number rate of the image macro block is less than the target frame number rate, it is indicated that the image motion corresponding to the image macro block is severe, and a larger frame number rate is required to more accurately display the image motion corresponding to the image macro block, so that an image frame needs to be properly added to the image frame corresponding to the image macro block between the key frame and the corresponding common frame, and further, the sub-video corresponding to the image macro block is more smooth. Specifically, the number of the added image frames may be determined according to a first difference between a target frame rate and a current frame rate, for example, the current frame rate of the image macro block is 12, the target frame rate is 24, and the first difference is 12, so that 12 image frames need to be added to the image macro block in the image frame corresponding to the key frame and the corresponding common frame.

If the current frame rate of the image macro block is greater than the target frame rate, it means that the image motion corresponding to the image macro block is relatively slow, and it is not necessary to use such a large frame rate to display the image motion corresponding to the image macro block, and therefore, it is necessary to properly subtract the image frame from the image frame corresponding to the image macro block between the key frame and the corresponding common frame. Specifically, the number of image frames subtracted from the image frame corresponding to the image macroblock between the key frame and the corresponding common frame may be determined according to a second difference between the current frame rate and the target frame rate. For example, the current frame rate of the image macroblock is 12, the target frame rate is 4, and the second difference value is 8, so that 8 image frames need to be subtracted from the image frame corresponding to the image macroblock between the key frame and the corresponding common frame.

The image frame of the image macro block, in which the first difference value is added to the image frame corresponding to the key frame and the corresponding common frame, includes:

The image frame obtained by subtracting the second difference value from the image frame corresponding to the image macro block between the key frame and the corresponding common frame comprises:

In order to improve the quality of video coding, in the embodiment of the present invention, preferably, the image frame of the first difference is added between the image frames corresponding to the image macro blocks by using an interpolation method, where the image frame of the first difference is generated by using an image calculation technology and is inserted between the corresponding key frame and the common frame by using the interpolation method, and a specific process of generating the image frame and a process of adding the image frame by using the interpolation method are both related to the prior art and are not described herein again.

In order to improve the quality of video coding, the embodiment of the present invention preferably adds the image frame of the second difference value between the corresponding key frame and the normal frame by a frame decimation method. The specific process of subtracting the image frame by using the frame extraction method is the prior art, and is not described herein again.

Based on the above embodiment, the correspondence relationship between the variation rate and the frame rate is recorded in table 1, and specifically, the correspondence relationship between the variation rate range and the frame rate is recorded.

TABLE 1

Rate of change	Frame rate (Scheduled _ fps)/(frame/sec)
		0 to 10 percent	2
11 to 25 percent	4
		25 to 50 percent	12
50 to 75 percent	24
		75 to 90 percent	48
90 to 100 percent	90

In table 1, the correspondence between the variation rate and the frame rate is recorded, and specifically, the correspondence between the variation rate range and the frame rate is recorded. For any image macro block, after determining the target variation rate of the image macro block, the target variation rate of the image macro block may be compared with each variation rate range set in table 1, a target variation rate range to which the target variation rate belongs is determined, and the frame number rate corresponding to the target variation rate range is used as the target frame number rate of the image macro block.

Taking the image macro blocks in embodiment 2 as an example, fig. 5 shows the target frame rate corresponding to each image macro block, as shown in fig. 5, the frame rate schedule 1 is slow in frame fluctuation, the target frame rate schedule _ fps (1, 1) is 12 frames/s, the frame rate schedule 2 is severe in frame fluctuation, the frame rate schedule _ fps (1, 2) is 24 frames/s, the frame rate schedule 3 is slow in frame fluctuation, the frame rate schedule _ fps (1, 3) is 12 frames/s, the frame rate schedule 4 is severe in frame fluctuation, the frame rate schedule _ fps (1, 4) is 24 frames/s, the frame rate schedule 5 is severe in frame fluctuation, the frame rate schedule _ fps (1, 5) is 24 frames/s, the frame rate schedule 6 is slow in frame fluctuation, the frame rate schedule _ fps (1, 6) is 4 frames/s, the frame rate schedule _ fps (7) is slow in frame fluctuation, the frame rate schedule _ fps (1, 4) is 4 frames/s, the frame 8 has slow motion, the target frame rate Scheduled _ fps (1, 8) is 4 frames/s, the frame 9 has strong motion, the target frame rate Scheduled _ fps (1, 9) is 90 frames/s, the frame 10 has slow motion, and the target frame rate Scheduled _ fps (1, 10) is 4 frames/s.

Therefore, after the target frame rate corresponding to each image macro block in fig. 4 is determined, for each image macro block, frame rate adjustment may be performed on the image macro block according to the target frame rate corresponding to the image macro block in fig. 5 and the current frame rate, so as to improve the encoding quality of each frame of image corresponding to the image macro block.

In the embodiment of the invention, aiming at each pixel point, whether the pixel point changes in the key frame and the corresponding common frame or not is judged; if yes, recording the pixel point; and determining the target change rate of the pixel value of the image macro block according to the number of the recorded pixel points and the total number of the pixel points in the image macro block, so that the target change rate of the pixel value of the image macro block can be more accurately determined.

Example 4:

on the basis of the foregoing embodiments, in order to obtain a complete video encoding file corresponding to a video file to be encoded, after encoding each frame corresponding to the image macroblock, the method further includes:

According to the method described in the foregoing embodiment, the encoded file of each image macro block corresponding to each key frame and its corresponding common frame, that is, the first video file corresponding to each encoded image macro block, which are scattered encoded video files, may be obtained. Since the video file to be encoded has many video segments, in order to obtain the final complete video encoded file, it is necessary to integrate and package the second video files corresponding to the video segments, so as to obtain the final complete video encoded file.

The specific package integration and integration packaging processes are not described in detail herein for the prior art.

For example, after the frame number of each image macroblock in fig. 4 is adjusted, each image macroblock after adjustment is packaged and integrated into a new video coding file (Vedio _ Pack (1)), and then, according to the above method, the above steps are repeated for a video segment (Vedio (X +1)) corresponding to the next, i.e., the X +1 th key frame, so as to obtain a video coding file (Vedio _ Pack (X +1)) corresponding to the video segment (Vedio (X +1)) until all video segments corresponding to the key frames obtain corresponding video coding files.

And integrating and packaging all video coding files (Vedio _ pack (X)) into a final complete video coding file (Vedio (Final)), namely the coded video file.

Based on the above embodiments, fig. 6 is a schematic structural diagram of a video coding system according to an embodiment of the present invention, and as shown in fig. 6, the following components in the system architecture and their corresponding functions are introduced:

the video input end is used for receiving an input video file to be coded; the storage component is used for storing data; the operation component is a hardware layer of the global calculation of the method, can realize multi-core and multi-thread task processing, namely, the multi-core performance of the hardware is utilized, and a method of multi-thread parallel coding of different image macro blocks is adopted, so that after a main program sets different target frame rates of each image macro block, each thread can call the video coding method to carry out video coding on the corresponding image macro block, the time can be saved, and the performance of the system can be fully utilized. The operation components include, but are not limited to, a Central Processing Unit (CPU) for scheduling tasks to perform data budgeting, an image processor (GPU) for calculating image data of video files, and a coding and decoding chip (DSP) for coding and calculating images after video files are cut into blocks.

The application modules based on the hardware layer at least comprise but are not limited to a source video file decoding module, a time frame sequence cutting module, a key frame image cutting module, a cutting compression coding module and an encapsulation module. And each application module is used for executing corresponding operation according to the calling of the operation component.

Based on the foregoing system architecture and the foregoing embodiments, fig. 7 is a flowchart of a method for video encoding according to an embodiment of the present invention, where the method may include:

firstly, preliminarily analyzing and extracting information such as video duration, frame per second, resolution and the like corresponding to an original digital video file (video (input)); the video input end stores the received information of the original digital video file into the storage component and informs the operation component of processing; the operation component calls a source file decoding module to perform decoding conversion on an Original digital video file (video (input)) and convert the Original digital video file (video (input)) into a lossless uncompressed video file with the highest information degree to be coded, namely a third video file (video (formatted)), wherein the frame number per second of the third video file (video (formatted)) is Original _ fps; the operation component calls a time frame sequence cutting module to determine a key frame and a plurality of common frames between two key frames in a third video file (video (formatted)) which takes time or a fixed frame number as periodicity, and takes each key frame and the corresponding common frame image before the next key frame as a video segment (video (x)); the operation component calls a key frame image block cutting module to perform image macro block processing on any key frame and a corresponding common frame thereof, namely, the key frame and the corresponding common frame thereof are divided into N (X) image macro blocks according to pixel values of pixel points contained in the key frame and the corresponding common frame thereof, wherein N (X) represents the number of image macro blocks of a video segment corresponding to the X key frame cut from a pixel plane; and the operation component calls a block-cutting compression coding module to determine the target change rate of the pixel value of the image macro block according to the pixel value of each pixel point in the image macro block in the key frame and the corresponding common frame.

Based on the foregoing system architecture and the foregoing embodiments, fig. 8 is a flowchart of a method for video encoding according to an embodiment of the present invention, where the method may include:

for each image macro block, determining the Original frame rate Original _ fps of a sub video (X)) corresponding to the image macro block; setting a frame rate threshold list, wherein the list records the corresponding relation between the variation rate and the frame rate, and specifically records the corresponding relation between the variation rate range and the frame rate; the block compression coding module calculates a target variation rate of pixel values of each image macro block, determines a target variation rate range to which the target variation rate belongs, and takes a frame number rate corresponding to the target variation rate range as a target frame rate of the image macro block (Scheduled _ fps (x, n)).

If the Scheduled _ fps (x, n) is Original _ fps, the block compression coding module does not adjust the frame number of the image macro block; if the Scheduled _ fps (x, n) > Original _ fps, the block cutting compression coding module inserts the image frame obtained by utilizing the image calculation technology between the corresponding key frame and the common frame through an interpolation method so as to improve the image fluency of the sub-video corresponding to the image macro block; if the Scheduled _ fps (x, n) < Original _ fps, the block compression encoding module deletes a corresponding number of image frames from the image frames between the corresponding key frame and the normal frame by the frame decimation method.

And the block cutting compression coding module performs picture compression coding on the sub-video corresponding to each image macro block after the frame number rate is adjusted according to a preset compression coding format.

The operation component calls the encapsulation module to encapsulate and integrate the encoded file corresponding to each encoded image macro block into a new video encoded file (Vedio _ pack (X)), judges X as max (X), namely judges whether the Xth key frame is the last key frame, if so, shows that the video clips corresponding to all the key frames obtain the corresponding video encoded file, so the operation component calls the encapsulation module to integrate and pack all the new video encoded files (Vedio _ pack (X)) into a final complete video encoded file corresponding to the video file to be encoded; if not, the operation component returns the video segment (video (X +1)) corresponding to the next key frame, namely the X +1 th key frame, to the operation component to call the key frame image block-cutting module, performs image macro block-cutting processing on any key frame and the corresponding common frame, and performs subsequent steps, and makes X equal to X + 1.

The method can also adopt a multithreading technology to process the compressed video in parallel, realize more dynamic compression of the video, and improve the video coding ratio and better video visual impression under the same code rate.

In the embodiment of the invention, the first video file corresponding to each encoded image macro block is encapsulated and integrated into the second video file, so that a more complete compressed video file can be obtained.

Example 5:

based on the foregoing embodiments, fig. 9 is a schematic structural diagram of a video encoding apparatus according to an embodiment of the present invention, where the apparatus includes:

a determining module 901, configured to determine, for a video file to be encoded, each key frame and a corresponding common frame in the video file;

a dividing module 902, configured to divide, for any key frame and its corresponding ordinary frame, the first key frame and its corresponding ordinary frame into a plurality of image macro blocks according to pixel values of pixel points included in the key frame and its corresponding ordinary frame;

the encoding module 903 is configured to determine, for each image macroblock, a target variation rate of the pixel value of the image macroblock according to the pixel value of each pixel point in the image macroblock in the key frame and the corresponding common frame of the key frame, determine a target frame rate corresponding to the target variation rate according to a preset correspondence between the variation rate and the frame rate, adjust the number of frames corresponding to the image macroblock by using the target frame rate, and encode each frame corresponding to the image macroblock.

Further, the encoding module 903 is specifically configured to determine, for each pixel point, whether a pixel value of the pixel point in the key frame and the corresponding normal frame changes; if yes, recording the pixel point; and determining the target variation rate of the pixel value of the image macro block according to the number of the recorded pixel points and the total number of the pixel points in the image macro block.

Further, the encoding module 903 is specifically configured to not adjust the current frame rate of the image macroblock if the current frame rate is equal to the target frame rate; if the current frame number rate of the image macro block is smaller than the target frame number rate, determining a first difference value between the target frame number rate and the current frame number rate, and adding an image frame of the first difference value in the image frame corresponding to the image macro block between the key frame and the corresponding common frame; and if the current frame number rate of the image macro block is greater than the target frame number rate, determining a second difference value between the current frame number rate and the target frame number rate, and subtracting the image frame of the second difference value from the image frame of the image macro block corresponding to the key frame and the corresponding common frame.

Further, the encoding module 903 is specifically configured to add the image frame with the first difference value to the image frame corresponding to the image macroblock between the key frame and the corresponding common frame by using an interpolation method.

Further, the encoding module 903 is specifically configured to subtract the image frame of the second difference value from the image frame corresponding to the image macroblock between the key frame and the corresponding common frame by using a frame extraction method.

Further, the encoding module 903 is further configured to encapsulate and integrate the first video file corresponding to each encoded image macro block into a second video file after each frame corresponding to the image macro block is encoded.

The determining module 901 in the embodiment of the present invention is configured to determine, for a video file to be encoded, each key frame and a corresponding common frame in the video file; a dividing module 902, configured to divide, for any key frame and its corresponding ordinary frame, the first key frame and its corresponding ordinary frame into a plurality of image macro blocks according to pixel values of pixel points included in the key frame and its corresponding ordinary frame; the encoding module 903 is configured to determine, for each image macroblock, a target variation rate of the pixel value of the image macroblock according to the pixel value of each pixel point in the image macroblock in the key frame and the corresponding common frame of the key frame, determine a target frame rate corresponding to the target variation rate according to a preset correspondence between the variation rate and the frame rate, adjust the number of frames corresponding to the image macroblock by using the target frame rate, and encode each frame corresponding to the image macroblock, so that the quality of video encoding can be further improved.

Example 6:

on the basis of the foregoing embodiments, fig. 10 is an electronic device provided in an embodiment of the present invention, where the electronic device includes a memory 1001 and a processor 1002;

the processor 1002 is configured to read the program in the memory 1001, and execute the following processes:

determining each key frame and a corresponding common frame in a video file to be coded; aiming at any key frame and a corresponding common frame thereof, dividing the first key frame and the corresponding common frame thereof into a plurality of image macro blocks according to pixel values of pixel points contained in the key frame and the corresponding common frame thereof; and aiming at each image macro block, determining a target variation rate of the pixel value of the image macro block according to the pixel value of each pixel point in the image macro block in the key frame and the corresponding common frame, determining a target frame rate corresponding to the target variation rate according to a preset corresponding relation between the variation rate and the frame rate, adjusting the frame number corresponding to the image macro block by adopting the target frame rate, and coding each frame corresponding to the image macro block.

Based on the same inventive concept, the embodiment of the present invention further provides an electronic device, and as the principle of the electronic device for solving the problem is similar to the video encoding method, the implementation of the electronic device may refer to the implementation of the method, and repeated details are not repeated.

In fig. 10, the bus architecture may include any number of interconnected buses and bridges, with one or more processors, represented by the processor 1001, and various circuits, represented by the memory 1002, being linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. The processor 1001 is responsible for managing the bus architecture and general processing, and the memory 1002 may store data used by the processor 1001 in performing operations.

Alternatively, the processor 1001 may be a CPU (central processing unit), an ASIC (Application specific integrated Circuit), an FPGA (Field Programmable Gate Array), or a CPLD (Complex Programmable Logic Device).

The processor 1002 is specifically configured to determine, for each pixel point, whether a pixel value of the pixel point in the key frame and the corresponding normal frame changes; if yes, recording the pixel point; and determining the target variation rate of the pixel value of the image macro block according to the number of the recorded pixel points and the total number of the pixel points in the image macro block.

The processor 1002 is specifically configured to not perform adjustment if the current frame rate of the image macroblock is equal to the target frame rate; if the current frame number rate of the image macro block is smaller than the target frame number rate, determining a first difference value between the target frame number rate and the current frame number rate, and adding an image frame of the first difference value in the image frame corresponding to the image macro block between the key frame and the corresponding common frame; and if the current frame number rate of the image macro block is greater than the target frame number rate, determining a second difference value between the current frame number rate and the target frame number rate, and subtracting the image frame of the second difference value from the image frame of the image macro block corresponding to the key frame and the corresponding common frame.

The processor 1002 is further configured to encapsulate and integrate the first video file corresponding to each encoded image macro block into a second video file after each frame corresponding to the image macro block is encoded.

Example 7:

on the basis of the foregoing embodiments, an embodiment of the present invention provides an electronic device, as shown in fig. 11, the electronic device includes a processor 1101, a communication interface 1102, a memory 1103, and a communication bus 1104, where the processor 1101, the communication interface 1102, and the memory 1103 complete communication with each other through the communication bus 1104;

the memory 1103 has stored therein a computer program that, when executed by the processor 1101, causes the processor 1101 to perform the steps of:

The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface 1102 is used for communication between the electronic apparatus and other apparatuses.

The Memory 1103 may include a Random Access Memory (RAM) and a Non-Volatile Memory (NVM), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the processor.

The processor 1101 may be a general-purpose processor, and includes a central processing unit (cpu), a Network Processor (NP), and the like; but may also be a Digital instruction processor (DSP), an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like.

Further, the processor 1101 is specifically configured to determine, for each pixel point, whether a pixel value of the pixel point in the key frame and the corresponding common frame changes; if yes, recording the pixel point; and determining the target variation rate of the pixel value of the image macro block according to the number of the recorded pixel points and the total number of the pixel points in the image macro block.

Further, the processor 1101 is specifically configured to not adjust the current frame rate of the image macroblock if the current frame rate is equal to the target frame rate; if the current frame number rate of the image macro block is smaller than the target frame number rate, determining a first difference value between the target frame number rate and the current frame number rate, and adding an image frame of the first difference value in the image frame corresponding to the image macro block between the key frame and the corresponding common frame; and if the current frame number rate of the image macro block is greater than the target frame number rate, determining a second difference value between the current frame number rate and the target frame number rate, and subtracting the image frame of the second difference value from the image frame of the image macro block corresponding to the key frame and the corresponding common frame.

Further, the processor 1001 is specifically configured to add, by using an interpolation method, the image frame of the first difference value to the image frame corresponding to the image macroblock between the key frame and the corresponding common frame.

Further, the processor 1001 is specifically configured to subtract the image frame of the second difference value from the image frame corresponding to the image macroblock between the key frame and the corresponding common frame by using a frame extraction method.

Further, the processor 1101 is further configured to encapsulate and integrate the first video file corresponding to each encoded image macro block into a second video file after each frame corresponding to the image macro block is encoded.

Example 8:

on the basis of the foregoing embodiments, an embodiment of the present invention further provides a computer-readable storage medium storing a computer program executable by an electronic device, where the computer program, when executed on the electronic device, causes the electronic device to perform the following steps:

Further, for each pixel point, judging whether the pixel value of the pixel point in the key frame and the corresponding common frame changes; if yes, recording the pixel point;

The computer readable storage medium may be any available medium or data storage device that can be accessed by a processor in an electronic device, including but not limited to magnetic memory such as floppy disks, hard disks, magnetic tape, magneto-optical disks (MO), etc., optical memory such as CDs, DVDs, BDs, HVDs, etc., and semiconductor memory such as ROMs, EPROMs, EEPROMs, nonvolatile memories (NANDFLASH), Solid State Disks (SSDs), etc.

For the system/apparatus embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference may be made to some descriptions of the method embodiments for relevant points.

It is to be noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or operation from another entity or operation without necessarily requiring or implying any actual such relationship or order between such entities or operations.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely application embodiment, or an embodiment combining application and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.

Claims

1. A method of video encoding, the method comprising:

2. The method as claimed in claim 1, wherein said determining the target variation rate of the pixel values of the image macroblock according to the pixel values of each pixel point in the image macroblock in the key frame and its corresponding normal frame comprises:

3. The method of claim 1 or 2, wherein said adjusting the number of frames corresponding to the image macroblock using the target frame rate comprises:

4. The method as claimed in claim 3, wherein said adding the image frame of the first difference value to the image frame corresponding to the image macroblock between the key frame and the corresponding normal frame comprises:

5. The method as claimed in claim 3, wherein said subtracting said second difference value from said image frame corresponding to said image macroblock between said key frame and said corresponding normal frame comprises:

6. The method of claim 1, wherein after said encoding each frame corresponding to the image macroblock, the method further comprises:

7. An apparatus for video encoding, the apparatus comprising:

8. An electronic device, comprising a memory and a processor;

9. The electronic device according to claim 8, wherein the processor is specifically configured to determine, for each pixel point, whether a pixel value of the pixel point in the key frame and the corresponding normal frame changes; if yes, recording the pixel point; and determining the target variation rate of the pixel value of the image macro block according to the number of the recorded pixel points and the total number of the pixel points in the image macro block.

10. The electronic device according to claim 8 or 9, wherein the processor is configured to, if the current frame rate of the image macroblock is equal to the target frame rate, not perform an adjustment; if the current frame number rate of the image macro block is smaller than the target frame number rate, determining a first difference value between the target frame number rate and the current frame number rate, and adding an image frame of the first difference value in the image frame corresponding to the image macro block between the key frame and the corresponding common frame; and if the current frame number rate of the image macro block is greater than the target frame number rate, determining a second difference value between the current frame number rate and the target frame number rate, and subtracting the image frame of the second difference value from the image frame of the image macro block corresponding to the key frame and the corresponding common frame.

11. The electronic device of claim 8, wherein the processor is further configured to encapsulate and integrate the first video file corresponding to each encoded image macroblock into the second video file after each frame corresponding to the image macroblock is encoded.

12. An electronic device, characterized in that the electronic device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;

the memory has stored therein a computer program which, when executed by the processor, causes the processor to carry out the steps of the method of any one of claims 1 to 6.

13. A computer-readable storage medium, having stored thereon a computer program executable by an electronic device, for causing the electronic device to perform the steps of the method of any one of claims 1 to 6, when the program is run on the electronic device.