WO2024078106A1

WO2024078106A1 - Image frame sequence processing method and apparatus, computer device, and storage medium

Info

Publication number: WO2024078106A1
Application number: PCT/CN2023/111458
Authority: WO
Inventors: 雷明
Original assignee: 深圳市Tcl云创科技有限公司
Priority date: 2022-10-10
Filing date: 2023-08-07
Publication date: 2024-04-18
Also published as: CN117132669A

Abstract

The present application provides an image frame sequence processing method and apparatus, a computer device, and a storage medium. The method comprises: updating a first differential block on the basis of a target differential pixel while calculating a differential pixel between a target image frame and a reference image frame, calculating the distance between the updated first differential block and another second differential block, and merging the first differential block and the second differential block in real time if the distance is less than a preset distance threshold. Thus, the efficiency with which the target image frame is subsequently rendered is increased.

Description

Image frame sequence processing method, device, computer equipment and storage medium

This application claims priority to a Chinese patent application filed with the Chinese Patent Office on October 10, 2022, with application number 202211237073.3 and application name “Image frame sequence processing method, device, computer equipment and storage medium”, all contents of which are incorporated by reference in this application.

Technical Field

The present application relates to the field of artificial intelligence technology, and in particular to an image frame sequence processing method, apparatus, computer equipment, and computer-readable storage medium (storage medium for short).

Background technique

In the existing storage technology for image frame sequence processing, the difference area between the image frames is usually found by comparing the previous and next image frames. When saving the image frame sequence, only the image blocks corresponding to the difference area between the previous image frame and the next image frame can be saved to reduce the file size of the image frame sequence.

However, when the image frame sequence is played, the image blocks of each differential area of the subsequent image frame need to be decoded one by one in order to render and overlay them on the previous image frame. When the number of differential areas is large, the decoding and rendering of the differential areas requires a lot of resources and time, resulting in a significant reduction in drawing efficiency.

Summary of the invention

Based on this, it is necessary to provide an image frame sequence processing method, device, computer equipment and storage medium to improve the size recognition accuracy of the target object in response to the above technical problems.

In a first aspect, the present application provides a method for processing an image frame sequence, the method comprising:

Acquire a target image frame in an image frame sequence and a reference image frame corresponding to the target image frame;

Acquire differential pixels between a target image frame and a reference image frame in a differential step order, and update a first differential block in the target image frame based on a target differential pixel of a current differential step;

If the distance value between the first difference block and the second difference block is less than a preset distance threshold, a target difference block is generated based on the first difference block and the second difference block; wherein the second difference block is a difference block in the target image frame except the first difference block;

When the target differential pixel point of the current differential step is the last differential pixel point, the image frame sequence is saved based on the target differential block.

In some embodiments of the present application, the step of updating the first differential block in the target image frame based on the target differential pixel point of the current differential step includes:

Obtaining historical difference blocks in a target image frame;

If the target differential pixel point is not in any differential block in the historical differential blocks, the historical differential block whose distance to the target differential pixel point is less than a preset distance threshold is determined as the first differential block, and the first differential block is updated based on the target differential pixel point of the current differential step.

In some embodiments of the present application, after the step of obtaining the historical difference block in the target image frame, the method further includes:

If the target differential pixel point is not in any differential block in the historical differential blocks, and the distance value between any differential block in the historical differential blocks and the target differential pixel point is greater than or equal to the preset distance threshold, a first differential block is generated based on the target differential pixel point.

In some embodiments of the present application, after the step of generating a target differential block based on the first differential block and the second differential block, the method further includes:

The target differential block is determined as a new first differential block, and if the distance between the first differential block and the second differential block is less than a preset distance threshold, the target differential block is generated based on the first differential block and the second differential block. step.

In some embodiments of the present application, the step of acquiring differential pixels between a target image frame and a reference image frame in a differential step order includes:

Acquire the first pixel data of the target image frame at the current coordinate point and the second pixel data of the reference image frame at the current coordinate point in a differential step order;

If the first pixel data is different from the second pixel data, the pixel point of the current coordinate point is determined as a differential pixel point.

Acquire a first transparency parameter of the target image frame at the current coordinate point and a second transparency parameter of the reference image frame at the current coordinate point in a differential step order;

If one of the first transparency parameter and the second transparency parameter is greater than the preset transparency parameter and the other is less than or equal to the preset transparency parameter, determining the pixel point of the current coordinate point as the differential pixel point;

If the first transparency parameter and the second transparency parameter are both greater than the preset transparency parameter, obtaining a first RGB parameter of the target image frame at the current coordinate point and a second RGB parameter of the reference image frame at the current pixel point;

If the first RGB parameter is different from the second RGB parameter, the pixel point at the current coordinate point is determined as a differential pixel point.

In some embodiments of the present application, before the step of acquiring the target image frame in the image frame sequence and the reference image frame corresponding to the target image frame, the step further includes:

Get the image width and image height of the image frame sequence;

The maximum value of the image width and the image height is determined, and a preset distance threshold is determined based on the maximum value.

In some embodiments of the present application, the step of saving the image frame sequence based on the target difference block includes:

Generate differential image information according to the region position and region size of the target differential block in the target image frame;

The image frame sequence is saved based on the differential image information and the reference image frame.

In a second aspect, the present application provides an image frame sequence processing device, the device comprising:

An image frame acquisition module, used to acquire a target image frame in an image frame sequence and a reference image frame corresponding to the target image frame;

A differential pixel acquisition module, used to acquire differential pixel points between a target image frame and a reference image frame in a differential step order, and update a first differential block in the target image frame based on a target differential pixel point of a current differential step;

A difference block merging module, configured to generate a target difference block based on the first difference block and the second difference block when the distance value between the first difference block and the second difference block is less than a preset distance threshold; wherein the second difference block is a difference block in the target image frame except the first difference block;

The frame sequence saving module is used to save the image frame sequence based on the target differential block when the target differential pixel point of the current differential step is the last differential pixel point.

In a third aspect, the present application further provides a computer device, the computer device comprising:

one or more processors;

Memory; and

One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the image frame sequence processing method.

In a fourth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is loaded by a processor to execute the steps in the image frame sequence processing method.

The above-mentioned image frame sequence processing method, device, computer equipment and storage medium obtain a target image frame in the image frame sequence and a reference image frame corresponding to the target image frame; obtain the differential pixel points between the target image frame and the reference image frame in a differential step order, and update the first differential area in the target image frame based on the target differential pixel point of the current differential step. block; if the distance value between the first differential block and the second differential block is less than the preset distance threshold, a target differential block is generated based on the first differential block and the second differential block; wherein the second differential block is a differential block in the target image frame except the first differential block; when the target differential pixel point of the current differential step is the last differential pixel point, the image frame sequence is saved based on the target differential block. The method updates the first differential block based on the target differential pixel point of the current differential step while calculating the differential pixel point between the target image frame and the reference image frame, and simultaneously calculates the distance between the first differential block updated by the target differential pixel point and the other second differential blocks. If the distance between the first differential block and the other second differential block is less than the preset distance threshold, the first differential block is merged with the second differential block in real time to reduce the time consumption of the differential block in the target image frame, and effectively control the number of differential blocks in the target image frame, so as to avoid the consumption of resources and time for decoding and rendering of the differential area due to too many differential blocks in the target image frame, and improve the efficiency of subsequent drawing of the target image frame.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

FIG1A is a schematic diagram of a differential block in an image frame in the prior art;

FIG1B is a schematic diagram of drawing an image frame based on differential blocks in the prior art;

FIG2 is a schematic diagram of a flow chart of an image frame sequence processing method in an embodiment of the present application;

FIG3 is a schematic diagram of a differential block in a target image frame in an embodiment of the present application;

FIG4 is another schematic flow chart of the method for processing an image frame sequence in an embodiment of the present application;

FIG5 is a schematic diagram of a differential block in an image frame in an embodiment of the present application;

FIG6 is a schematic diagram of drawing an image frame based on differential blocks in an embodiment of the present application;

FIG7 is a schematic diagram of the structure of an image frame sequence processing device in an embodiment of the present application;

FIG. 8 is a schematic diagram of the structure of a computer device in an embodiment of the present application.

Detailed ways

The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

In the description of the present application, the word "for example" is used to mean "used as an example, illustration or description". Any embodiment described as "for example" in the present application is not necessarily to be construed as being more preferred or advantageous than other embodiments. In order to enable any technician in the field to implement and use the present application, the following description is given. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

In the prior art, in the storage technology for processing image frame sequences, usually the difference area between the image frames is found by comparing the previous and next image frames. When saving the image frame sequence, only the image blocks corresponding to the difference area between the previous image frame and the next image frame can be saved to reduce the file size of the image frame sequence. is the t-th image frame in the image frame sequence, and image frame 120 is the (t+1)-th image frame in the image frame sequence. By comparing the two preceding and succeeding image frames, a plurality of differential regions between the image frames are found (only some of the differential regions are marked in the figure). It can be seen from FIG1A that, compared with image frame 120, the differential region obtained by differential is smaller. When the image frame sequence is saved based on the differential region, the file size of the image frame sequence is greatly reduced.

However, assuming that the image frame sequence is an animation frame sequence similar to a starlight twinkling effect, there are often a large number of pixels with varying brightness between the two image frames, which correspondingly constitute a large number of independent differential regions. When the image frame sequence is saved based on the differential regions, the file size of the image frame sequence will increase instead.

In addition, in the process of rendering image frame 120, the image blocks corresponding to the differential areas need to be decoded one by one, and drawn one by one and overlaid on image frame 110. When there are many differential areas, a large number of decoding commands and drawing commands need to be called, as shown in 1B, resulting in a significant reduction in drawing efficiency.

Based on the above problems, an embodiment of the present application provides an image frame sequence processing method, which updates a first differential block based on the target differential pixel of the current differential step while calculating the differential pixel between the target differential pixel, and synchronously calculates the distance between the first differential block updated by the target differential pixel and the other second differential blocks. If the distance between the first differential block and the other second differential block is less than a preset distance threshold, the first differential block is merged with the second differential block in real time to reduce the time consumption of the differential block in the target image frame, and effectively control the number of differential blocks in the target image frame to avoid too many differential blocks in the target image frame, which leads to the consumption of resources and time for decoding and rendering of the differential area, thereby improving the efficiency of subsequent drawing of the target image frame.

The image frame sequence processing method provided in the embodiment of the present application can be run on a terminal device or a server. The terminal device can be a local terminal device. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

Referring to FIG. 2 , an embodiment of the present application provides a method for processing an image frame sequence, which is mainly illustrated by applying the method to a server. The method includes steps S210 to S230, which are specifically as follows:

Step S210 , obtaining a target image frame in an image frame sequence and a reference image frame corresponding to the target image frame.

The image frame sequence may specifically be video data, animation data, etc., and the image frame sequence includes multiple image frames that are continuous in time sequence. The target image frame refers to an image frame to be processed in the image frame sequence, and the reference image frame is an image frame that is the previous frame in time sequence of the target image frame and is adjacent to the target image frame in time sequence. For example, the target image frame may be an image frame at the t-th moment in the target frame sequence, and the reference image frame is an image frame at the (t-1)th moment in the target frame sequence.

Specifically, after the image frame sequence to be processed is determined, each image frame is determined as a target image frame in turn according to the order of the image frames in the image frame sequence, and a reference image frame corresponding to the target image frame is obtained.

Step S220, obtaining differential pixels between the target image frame and the reference image frame in a differential step order, and updating a first differential block in the target image frame based on the target differential pixel of the current differential step.

Among them, in a differential step, the terminal performs differential processing on the pixel point at the same coordinate point on the target image frame and the reference image frame to determine whether the pixel point at the coordinate point is a differential pixel point; the differential step order can be determined according to the arrangement order of the pixels in the target image frame (or reference image frame). For example, referring to Figure 3, Figure 3 shows a schematic diagram of a target image frame including 6 rows and 6 columns of pixels. The pixel point F can be used as the starting coordinate point to take points row by row, and a coordinate point is taken as the target coordinate point of the current differential step in each differential step. The pixel data of the target image frame and the reference image frame at the target coordinate point are compared to determine whether the pixel point corresponding to the target coordinate point is a differential pixel point; for example, the current differential step is to perform differential processing on the pixel point A in the target image frame and the pixel point in the first row and fifth column in the reference image frame to determine whether the pixel point on the pixel point in the first row and fifth column is a differential pixel point, and the next differential step is to perform differential processing on the pixel point in the first row and sixth column in the target image frame (i.e., pixel point B) and the pixel point in the first row and sixth column in the reference image frame. The pixel points in the sixth column of a row are subjected to differential processing to determine whether the pixel points in the sixth column of the first row are differential pixels, and so on.

The difference block refers to a rectangular area where there are difference pixels in the target image frame compared to the reference image frame; continuing to take Figure 3 as an example, the pixels A, B, C and D in the target image frame of Figure 3 are difference pixels, and a difference block 310 is formed in the target image frame based on the pixels A, B and C, and a difference block 320 is formed in the target image frame based on the pixel D. It can be understood that, compared to the reference image frame, the target image frame has one or more difference blocks; wherein the first difference block refers to the difference block to be included in the target difference pixel obtained by the current difference step.

After determining that the pixel point corresponding to the coordinate point processed by the current differential step is a differential pixel point, the pixel point corresponding to the coordinate point can be determined as the target differential pixel point, and the first differential block corresponding to the target differential pixel point can be determined in the existing historical differential blocks in the target image frame, or a new differential block can be generated based on the coordinate position of the differential pixel point to obtain the first differential block.

Furthermore, after determining the first differential block corresponding to the target differential pixel point, the coordinate position of the differential pixel point and the area position of the first differential block can be compared to determine whether the coordinate position of the differential pixel point is within the area range of the first differential block; if the coordinate position of the differential pixel point is within the area range of the first differential block, the area range of the first differential block remains unchanged after the differential pixel point is included in the first differential block; if the coordinate position of the differential pixel point is outside the area range of the first differential block, the area range of the first differential block changes after the differential pixel point is included in the first differential block. At this time, the area range of the first differential block is updated based on the coordinate position of the target differential pixel point.

Continuing to refer to FIG. 3 , for example, the pixel point D in FIG. 3 is the target differential pixel point obtained in the current differential step, and a new differential block 320 can be generated based on the coordinate position of the pixel point D to obtain a first differential block; in the next differential step, it is determined whether the pixel point E is a differential pixel point. When the pixel point E is a differential pixel point, the first differential block can be determined as the differential area 320 based on the pixel point E, and the pixel point E is included in the differential area 320 to update the area range of the differential area 320. The area range of the differential area 320 changes, and the changed area range is shown in the differential area 320'.

Step S230 : if the distance between the first difference block and the second difference block is less than a preset distance threshold, a target difference block is generated based on the first difference block and the second difference block; wherein the second difference block is a difference block in the target image frame except the first difference block.

Compared with the reference image frame, the target image frame has one or more difference blocks; after obtaining the updated first difference block, the distance value between the first difference block and the second difference block in the target image frame other than the first difference block can be calculated. Specifically, for any second difference block, the center coordinates of the first difference block and the center coordinates of the second difference block can be obtained, and based on the distance between the center coordinates of the first difference block and the center coordinates of the second difference block, the distance value between the first difference block and the second difference block is determined; or the shortest distance value between the boundary of the first difference block and the boundary of the second difference block is obtained, and the shortest distance value is determined as the distance value between the first difference block and the second difference block.

After obtaining the distance values between the first difference block and the other difference blocks, the distance value is compared with the preset distance threshold. If the distance value between the first difference block and a second difference block is less than the preset distance threshold, the first difference block and the second difference block can be merged to obtain the target difference block.

Furthermore, in one embodiment, after the step of generating a target difference block based on the first difference block and the second difference block, the method further includes: determining the target difference block as a new first difference block, and continuing to execute the step of generating the target difference block based on the first difference block and the second difference block if the distance value between the first difference block and the second difference block is less than a preset distance threshold.

The area range of the target difference block obtained by merging the first difference block and the second difference block must change, that is, the area range of the target difference block is expanded. At this time, the target difference block is determined as the new first difference block. The method further comprises: calculating a distance value between the first difference block and the second difference blocks other than the first difference block in the target image frame, and then determining, among the other second difference blocks, a second difference block whose distance value with the first difference block is less than a preset distance threshold, and merging the first difference block with the second difference block again to update the first difference block again, until the distance values between the first difference block and the other difference blocks other than the first difference block in the target image frame are all greater than or equal to the preset distance threshold.

It can be understood that if the distance values between the first differential block and any second differential block in the target image frame are greater than or equal to the preset distance threshold, then the differential pixels of the next differential step can be obtained according to the differential step order, and the above steps can be repeated, that is, the differential block in the target image frame is updated again based on the differential pixel points of the next differential step, and if the distance value between the differential block updated by the differential pixel points corresponding to the next differential step and other differential blocks in the target image frame is less than the preset distance threshold, the differential block updated by the differential pixel points corresponding to the next differential step and the differential block are used to generate a target differential block, until the distance values between any two differential blocks in the target image frame are greater than or equal to the preset distance threshold.

Step S240, when the target differential pixel point of the current differential step is the last differential pixel point, the image frame sequence is saved based on the target differential block.

Among them, the target difference block is the change area between the target image frame and the reference image frame. When the target difference pixel point of the current difference step is the last difference pixel point, the image frame sequence can be saved by saving the target difference block and the reference image frame.

In the above-mentioned image frame sequence processing method, a target image frame in the image frame sequence and a reference image frame corresponding to the target image frame are obtained; differential pixel points between the target image frame and the reference image frame are obtained in a differential step order, and a first differential block in the target image frame is updated based on the target differential pixel point of the current differential step; if the distance value between the first differential block and the second differential block is less than a preset distance threshold, a target differential block is generated based on the first differential block and the second differential block; wherein the second differential block is a differential block in the target image frame except the first differential block; when the target differential pixel point of the current differential step is the last differential pixel point, the image frame sequence is saved based on the target differential block. While calculating the differential pixel points between the target image frame and the reference image frame, the first differential block is updated based on the target differential pixel points of the current differential step, and the distance between the first differential block updated by the target differential pixel points and other second differential blocks is synchronously calculated. If the distance between the first differential block and the other second differential block is less than a preset distance threshold, the first differential block is merged with the second differential block in real time to reduce the time consumption of the differential block in the target image frame, and at the same time effectively control the number of differential blocks in the target image frame, avoid too many differential blocks in the target image frame, which leads to the consumption of resources and time for decoding and rendering of the differential area, and improve the efficiency of subsequent drawing of the target image frame.

In one embodiment, the step of updating the first differential block in the target image frame based on the target differential pixel point of the current differential step includes: obtaining the historical differential blocks in the target image frame; if the target differential pixel point is not in any differential block in the historical differential blocks, determining the historical differential block whose distance to the target differential pixel point is less than a preset distance threshold as the first differential block, and updating the first differential block based on the target differential pixel point of the current differential step.

The historical differential block refers to the existing differential block in the target image frame. When it is determined that the pixel point processed by the current differential step is a differential pixel point, the coordinate position of the target differential pixel point corresponding to the current differential step and the regional position of the historical differential block can be compared to determine whether the target differential pixel point is in a differential block in the historical differential block. If the target differential pixel point is in a differential block in the historical differential block, the differential block can be determined as the first differential block corresponding to the target differential pixel point of the current differential step; it can be understood that since the target differential block is in the first differential block, the first differential area is updated based on the target differential block, and the regional range of the first differential block remains unchanged. Continuing to refer to FIG. 3 , for example, the pixel point G in FIG. 3 is the target differential pixel point obtained by the current differential step. The coordinate position of the pixel point G corresponding to the current differential step and the regional position of the differential block 310 can be compared to determine whether the pixel point G is in the differential block 310. Since the pixel point G is in the differential block 310, the pixel point G is divided into the differential block 310, and the differential block 310 remains unchanged.

If the target differential pixel is not in any differential block in the historical differential block, the target differential can be calculated. The distance value between the pixel point and each historical differential block is determined, and then based on the distance value between the target differential pixel point and each historical differential block, the differential block corresponding to the target differential pixel point is determined to obtain the first differential block. Specifically, after determining the distance value between the differential pixel point and each historical differential block, if there is a differential block in the historical differential block, and the distance value between the differential block and the target differential pixel point is less than the preset distance threshold, the differential block is determined as the first differential block corresponding to the target differential pixel point, and then the first differential block is updated based on the target differential pixel point; it can be understood that since the target differential block is in the first differential block, the first differential area is updated based on the target differential block, and the area range of the first differential block becomes larger.

Furthermore, in one embodiment, after the step of obtaining the historical difference blocks in the target image frame, it also includes: if the target differential pixel point is not in any difference block in the historical difference blocks, and the distance value between any difference block in the historical difference blocks and the target differential pixel point is greater than or equal to a preset distance threshold, a first difference block is generated based on the target differential pixel point.

If there is no difference block in the historical difference blocks whose distance value to the target difference pixel point is less than the preset distance threshold, the coordinate position of the target difference pixel point is obtained, and a new difference block is generated based on the coordinate position, and the difference block is used as the first difference block corresponding to the target difference pixel point.

By judging whether the target differential pixel point processed by the current decoding step is in the existing historical differential blocks, when the target differential pixel is not in any differential block in the historical differential blocks, the differential block updated by the target differential pixel point is determined based on the distance value between the target differential pixel point and each historical differential block, so that while calculating the differential pixel point between the target image frame and the reference image frame, the differential block in the target image frame is updated in real time based on the target differential pixel point of the current differential step, so as to facilitate the subsequent distance-based merging of the differential blocks in the target image frame, so as to reduce the time consumption of the differential blocks in the target image frame.

In one embodiment, the step of acquiring differential pixel points between a target image frame and a reference image frame in a differential step order includes: acquiring first pixel data of the target image frame at a current coordinate point and second pixel data of the reference image frame at the current coordinate point in a differential step order; if the first pixel data is different from the second pixel data, determining the pixel point of the current coordinate point as a differential pixel point.

Among them, pixel data can refer to the color value of a pixel point corresponding to a certain coordinate point; specifically, pixel data can refer to a value in a color space, for example, it can be a value corresponding to an RGB color space, i.e., an RGB color value, or it can be a value in a YUV color space, i.e., a YUV color value.

The current target coordinate point refers to the target coordinate point processed in the current difference step.

Since the target image frame and the reference image frame are sequential image frames, the differential pixel points of the target image frame compared to the reference image frame are obtained based on the pixel data at the same coordinate point of the target image frame and the reference image frame. Specifically, the target coordinate point processed by the current differential step is determined, and the first pixel data at the current coordinate point is obtained from the target image frame, and the second pixel data at the current coordinate point is obtained from the reference image frame; then, the pixel data at the target coordinate point of the target image frame and the reference image frame are compared, and if the first pixel data and the second pixel data at the target coordinate point are not equal, the pixel point corresponding to the target coordinate point can be determined as a differential pixel.

It can be understood that if the pixel data at the target coordinate point is equal, the pixel point corresponding to the target coordinate point is a non-differential pixel point. At this time, the coordinate point processed by the next differential step can be determined, and the pixel data of the target image frame and the reference image frame at the new coordinate point can be compared to determine whether the pixel point corresponding to the next coordinate point is a differential pixel point.

By comparing the pixel data at the same coordinate points to quickly obtain the differential pixel points between the target image frame and the reference image frame, the generation of differential blocks is effectively improved, and the preservation efficiency of the image frame sequence is improved.

Furthermore, the pixel data generally includes a transparency parameter and an RGB parameter. Therefore, in one embodiment, the step of obtaining the differential pixel points between the target image frame and the reference image frame in a differential step order includes: obtaining the first transparency parameter of the target image frame at the current coordinate point and the first transparency parameter of the reference image frame at the current coordinate point in the differential step order. a second transparency parameter of the reference image frame; if one of the first transparency parameter and the second transparency parameter is greater than the preset transparency parameter and the other is less than or equal to the preset transparency parameter, the pixel point at the current coordinate point is determined as a differential pixel point; if both the first transparency parameter and the second transparency parameter are greater than the preset transparency parameter, the first RGB parameter of the target image frame at the current coordinate point and the second RGB parameter of the reference image frame at the current pixel point are obtained; if the first RGB parameter is different from the second RGB parameter, the pixel point at the current coordinate point is determined as a differential pixel point.

The pixel data may refer to the RGB-A value of the pixel corresponding to a certain coordinate point, that is, the pixel data includes the RGB parameters on the RGB channel and the transparency parameters on the alpha channel. It can be understood that when the transparency parameter of the alpha channel is 0%, the pixel corresponding to the target coordinate point is a completely transparent pixel, and generally displays white regardless of the value of the RGB parameter on the RGB channel; when the transparency parameter of the alpha channel is 100%, the pixel corresponding to the target coordinate point is an opaque pixel, and the pixel corresponding to the target coordinate point displays the color represented by the RGB parameter.

Specifically, the target coordinate point processed by the current difference step is determined, and a first transparency parameter at the current coordinate point is obtained from the target image frame, and a second transparency parameter at the current coordinate point is obtained from the reference image frame; then, based on the first transparency parameter and the second transparency parameter, the transparency at the corresponding target coordinate point is determined, which may specifically include the following three situations:

Case 1: When the first transparency parameter and the second transparency parameter are both less than or equal to the preset transparency parameter, at this time, the pixel points at the target coordinate point in both the target image frame and the reference image frame are transparent pixel points, and it can be considered that the pixel points at the target coordinate point in both the target image frame and the reference image frame display the same color, and the pixel points at the target coordinate point are non-differential pixel points;

Case 2: When one of the first transparency parameter and the second transparency parameter is greater than the preset transparency parameter and the other is less than or equal to the preset transparency parameter, at this time, among the pixels at the target coordinate point of the target image frame and the reference image frame, one is a transparent pixel point and the other is an opaque pixel point. At this time, the pixel point corresponding to the target coordinate point can be determined as a differential pixel point;

Case 3: When the first transparency parameter and the second transparency parameter are both greater than the preset transparency parameter, the pixel point at the target coordinate point in either the target image frame or the reference image frame is not a transparent pixel point. At this time, the RGB parameters at the target coordinate point can be further compared. If the first RGB parameter and the second RGB parameter at the target coordinate point are not equal, the pixel point corresponding to the target coordinate point can be determined as a differential pixel.

Among them, the preset transparency parameter is used to determine whether the pixel at the current coordinate point is a transparent pixel. When the transparency parameter of the pixel at the current coordinate point is less than or equal to the preset transparency parameter, the pixel at the current coordinate point can be considered to be a transparent pixel. Specifically, the preset transparency parameter can take a value of 0; in addition, considering the interference of noise, the preset transparency parameter can also be set to a smaller value, such as 5. By first comparing the transparency parameters of the pixel corresponding to the current coordinate point, and then performing further RGB parameter comparison based on the transparency of the pixel corresponding to the current coordinate point, the influence of transparent pixels on the judgment of differential pixels is taken into consideration, the accuracy of differential pixel judgment is improved, and the situation where transparent pixels are mistakenly identified as differential pixels, resulting in the enlargement of differential blocks, is effectively eliminated.

In one embodiment, before the step of obtaining the target image frame in the image frame sequence and the reference image frame corresponding to the target image frame, it also includes: obtaining the image width and image height of the image frame sequence; determining the maximum value of the image width and the image height, and determining a preset distance threshold based on the maximum value.

The image width and image height are used to identify the image size of the image frame in the image frame sequence. After obtaining the image width and image height in the image frame sequence, the image width and image height are compared, the maximum value of the two values is taken, and then the preset distance threshold is calculated based on the maximum value.

Specifically, the preset distance threshold can be a value obtained by dividing the maximum value of the image width and the image height by n, where n is an integer whose value range is [5,10]. The preset distance threshold is set based on the image width and image height of the image frame sequence to avoid the preset distance threshold being too large, resulting in the merging of the various difference blocks in the target image frame into one difference block. It also avoids the situation where the preset distance threshold is too small and the difference blocks in the target image frame cannot be merged, or there are too many difference blocks.

In one embodiment, the step of saving the image frame sequence based on the target difference block includes: generating difference image information according to the region position and region size of the target difference block in the target image frame; and saving the image frame sequence based on the difference image information and the reference image frame.

Specifically, when the differential processing of all pixels is completed, the differential area in the target image frame is also obtained. At this time, the area position and area size of the target differential block in the target image frame can be obtained to generate differential image information; then, based on the differential image information and the reference image frame, the image frame sequence is saved.

It can be understood that when rendering the target image frame, the image information of the differential area can be obtained, and the rendering position of the image information can be determined based on the area position and area size of the differential area, and then the image information of the differential area can be rendered at the corresponding rendering position of the reference image frame to obtain the target image frame.

The image frame sequence processing method is further described below in conjunction with a specific application scenario. Specifically, referring to FIG4 , the image frame sequence processing method includes:

Step S401 , obtaining a target image frame in an image frame sequence and a reference image frame corresponding to the target image frame.

Step S402, obtaining differential pixel points between the target image frame and the reference image frame according to the differential step order.

Step S403, obtaining the historical difference block in the target image frame.

Step S404, determine whether the target differential pixel point of the current differential step is in a differential block in the historical differential blocks; if so, jump to step S402; if not, jump to step S405.

Step S405, obtain the distance value between any differential block in the historical differential block and the target differential pixel point; if the distance value between any differential block in the historical differential block and the target differential pixel point is greater than or equal to the preset distance threshold, jump to step 406; if there is a distance value between the differential block and the target differential pixel point in the historical differential block that is less than the preset distance threshold, jump to 407.

Step S406, generating a new differential block based on the target differential pixel point, and jumping to step S402.

Step S407: determine the historical differential block whose distance to the target differential pixel point is less than a preset distance threshold as the first differential block, and update the first differential block based on the target differential pixel point of the current differential step.

Step S408, determining whether the distance between the first differential block and the second differential block is less than a preset distance value; if so, jumping to step S409; otherwise, jumping to step S402.

The second difference block is a difference block in the target image frame except the first difference block.

Step S409: generating a target differential block based on the first differential block and the second differential block.

Step S410: determine the target differential block as a new first differential block, and jump to execute step S480.

Continuing with the image frame 110 and the image frame 120 shown in FIG. 1 as an example, the image frame 110 and the image frame 120 are subjected to differential processing based on the above-mentioned image frame sequence processing method, and the obtained differential blocks are shown in FIG. 5 , which effectively controls the number of blocks. For example, in the previously mentioned starlight animation, a large number of tiny differential areas can be merged into one area. Another benefit brought by the image frame sequence processing method is that when drawing the image frame 120, the number of differential blocks is reduced, and the decoding instructions and drawing instructions called during drawing are also greatly reduced, as shown in FIG. 6 , which greatly improves the drawing efficiency.

In order to better implement the image frame sequence processing method provided in the embodiment of the present application, on the basis of the image frame sequence processing method provided in the embodiment of the present application, an image frame sequence processing device is further provided in the embodiment of the present application. As shown in FIG. 7 , the image frame sequence processing device 700 includes:

An image frame acquisition module 710 is used to acquire a target image frame in an image frame sequence and a reference image frame corresponding to the target image frame;

The differential pixel acquisition module 720 is used to acquire the difference between the target image frame and the reference image frame in a differential step order. Sub-pixel point, updating the first differential block in the target image frame based on the target differential pixel point of the current differential step;

The difference block merging module 730 is used to generate a target difference block based on the first difference block and the second difference block when the distance value between the first difference block and the second difference block is less than a preset distance threshold; wherein the second difference block is a difference block in the target image frame except the first difference block;

The frame sequence saving module 740 is used to save the image frame sequence based on the target differential block when the target differential pixel point of the current differential step is the last differential pixel point.

In some embodiments of the present application, the differential pixel acquisition module is specifically used to acquire the historical differential blocks in the target image frame; if the target differential pixel point is not in any of the historical differential blocks, the historical differential block whose distance to the target differential pixel point is less than a preset distance threshold is determined as the first differential block, and the first differential block is updated based on the target differential pixel point of the current differential step.

In some embodiments of the present application, the differential pixel acquisition module is specifically used to generate a first differential block based on the target differential pixel point if the target differential pixel point is not in any differential block in the historical differential blocks, and the distance value between any differential block in the historical differential blocks and the target differential pixel point is greater than or equal to a preset distance threshold.

In some embodiments of the present application, the difference block merging module is further configured to determine the target difference block as a new first difference block, and continue to execute the step of generating the target difference block based on the first difference block and the second difference block if the distance value between the first difference block and the second difference block is less than a preset distance threshold.

In some embodiments of the present application, a differential pixel acquisition module is used to acquire first pixel data of a target image frame at a current coordinate point and second pixel data of a reference image frame at the current coordinate point in a differential step order; if the first pixel data is different from the second pixel data, the pixel point of the current coordinate point is determined as a differential pixel point.

In some embodiments of the present application, a differential pixel acquisition module is used to acquire a first transparency parameter of a target image frame at a current coordinate point and a second transparency parameter of a reference image frame at the current coordinate point in a differential step order; if one of the first transparency parameter and the second transparency parameter is greater than a preset transparency parameter and the other is less than or equal to the preset transparency parameter, the pixel point at the current coordinate point is determined as a differential pixel point; if both the first transparency parameter and the second transparency parameter are greater than the preset transparency parameter, the first RGB parameter of the target image frame at the current coordinate point and the second RGB parameter of the reference image frame at the current pixel point are acquired; if the first RGB parameter is different from the second RGB parameter, the pixel point at the current coordinate point is determined as a differential pixel point.

In some embodiments of the present application, the image frame acquisition module is specifically used to obtain the image width and image height of the image frame sequence; determine the maximum value of the image width and the image height, and determine a preset distance threshold based on the maximum value.

In some embodiments of the present application, a frame sequence saving module is used to generate differential image information according to the regional position and regional size of the target differential block in the target image frame; and save the image frame sequence based on the differential image information and the reference image frame.

For the specific definition of the image frame sequence processing device, please refer to the definition of the image frame sequence processing method above, which will not be repeated here. Each module in the above-mentioned image frame sequence processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

In some embodiments of the present application, the image frame sequence processing device 700 can be implemented in the form of a computer program, and the computer program can be run on a computer device as shown in FIG8. The memory of the computer device can store various program modules constituting the image frame sequence processing device 700, such as the image frame acquisition module 710, the differential pixel acquisition module 720, the differential block merging 730, and the frame sequence saving module 740 shown in FIG7. The computer program composed of various program modules enables the processor to execute the steps of the image frame sequence processing method of various embodiments of the present application described in this specification.

For example, the computer device shown in FIG8 can be used in the image frame sequence processing device 700 shown in FIG7 The image frame acquisition module 710 performs step S210. The computer device can perform step S220 through the differential pixel acquisition module 720. The computer device can perform step S230 through the differential block merging 730. The computer device can perform step S240 through the differential block merging 740. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external computer device through a network connection. When the computer program is executed by the processor, an image frame sequence processing method is implemented.

Those skilled in the art will understand that the structure shown in FIG. 8 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

In some embodiments of the present application, a computer device is provided, comprising one or more processors; a memory; and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor in the steps of the above-mentioned image frame sequence processing method. The steps of the image frame sequence processing method here may be the steps of the image frame sequence processing method in each of the above-mentioned embodiments.

In some embodiments of the present application, a computer-readable storage medium is provided, which stores a computer program, and the computer program is loaded by a processor, so that the processor executes the steps of the above-mentioned image frame sequence processing method. The steps of the image frame sequence processing method here can be the steps of the image frame sequence processing method in each of the above-mentioned embodiments.

Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

The above is a detailed introduction to an image frame sequence processing method, device, computer equipment and storage medium provided in an embodiment of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

A method for processing an image frame sequence, wherein the method comprises:

Acquire a target image frame in an image frame sequence and a reference image frame corresponding to the target image frame;

Acquire differential pixels between the target image frame and the reference image frame in a differential step order, and update a first differential block in the target image frame based on a target differential pixel of a current differential step;

If the distance value between the first difference block and the second difference block is less than a preset distance threshold, generating a target difference block based on the first difference block and the second difference block; wherein the second difference block is a difference block in the target image frame except the first difference block;

When the target differential pixel point of the current differential step is the last differential pixel point, the image frame sequence is saved based on the target differential block.
The method according to claim 1, wherein the step of updating the first differential block in the target image frame based on the target differential pixel point of the current differential step comprises:

Acquire a historical difference block in the target image frame;

If the target differential pixel point is not in any differential block in the historical differential blocks, the historical differential block whose distance to the target differential pixel point is less than a preset distance threshold is determined as the first differential block, and the first differential block is updated based on the target differential pixel point of the current differential step.
The method according to claim 2, wherein after the step of obtaining the historical difference block in the target image frame, the method further comprises:

If the target differential pixel point is not in any differential block in the historical differential blocks, and the distance value between any differential block in the historical differential blocks and the target differential pixel point is greater than or equal to the preset distance threshold, a first differential block is generated based on the target differential pixel point.
The method according to claim 1, wherein after the step of generating a target differential block based on the first differential block and the second differential block, the method further comprises:

The target differential block is determined as a new first differential block, and if the distance value between the first differential block and the second differential block is less than the preset distance threshold, the step of generating a target differential block based on the first differential block and the second differential block is continued.
The method according to claim 1, wherein the step of acquiring the differential pixel points between the target image frame and the reference image frame in a differential step order comprises:

Acquire the first pixel data of the target image frame at the current coordinate point and the second pixel data of the reference image frame at the current coordinate point in a differential step order;

If the first pixel data is different from the second pixel data, the pixel point of the current coordinate point is determined as a differential pixel point.
The method according to claim 1, wherein the step of acquiring the differential pixel points between the target image frame and the reference image frame in a differential step order comprises:

Acquire a first transparency parameter of the target image frame at the current coordinate point and a second transparency parameter of the reference image frame at the current coordinate point in a differential step order;

If one of the first transparency parameter and the second transparency parameter is greater than a preset transparency parameter and the other is less than or equal to the preset transparency parameter, determining the pixel point of the current coordinate point as a differential pixel point;

If the first transparency parameter and the second transparency parameter are both greater than a preset transparency parameter, obtaining a first RGB parameter of the target image frame at the current coordinate point and a second RGB parameter of the reference image frame at the current pixel point;

If the first RGB parameter is different from the second RGB parameter, the pixel point of the current coordinate point is determined as a differential pixel point.
The method according to claim 1, wherein the target image frame in the acquired image frame sequence and the Before the step of obtaining a reference image frame corresponding to the target image frame, the method further includes:

Obtaining the image width and image height of the image frame sequence;

A maximum value between the image width and the image height is determined, and a preset distance threshold is determined based on the maximum value.
The method according to claim 1, wherein the step of saving the image frame sequence based on the target difference block comprises:

Generate differential image information according to the area position and area size of the target differential block in the target image frame;

The image frame sequence is saved based on the differential image information and the reference image frame.
According to the method of claim 1, wherein the target image frame refers to an image frame to be processed in the image frame sequence; the reference image frame is an image frame that is previous to the target image frame in time sequence and is adjacent to the target image frame in time sequence.
The method according to claim 2, wherein the method further comprises:

If the target differential pixel point is within a differential block in the historical differential blocks, the differential block is determined as a first differential block corresponding to the target differential pixel point in the current differential step.
The method according to claim 7, wherein determining a preset distance threshold based on the maximum value comprises:

The value obtained by dividing the maximum value by n is used as the preset distance threshold, where n is an integer and the value range of n is [5, 10].
The method according to claim 1, wherein the distance value between the first difference block and the second difference block is determined in the following manner:

Acquire the center coordinates of the first difference block, and acquire the center coordinates of the second difference block;

Based on the distance between the center coordinates of the first differential block and the center coordinates of the second differential block, a distance value between the first differential block and the second differential block is determined.
The method according to claim 1, wherein the distance value between the first difference block and the second difference block is determined in the following manner:

Obtaining the shortest distance between the boundary of the first difference block and the boundary of the second difference block;

The shortest distance value is determined as a distance value between the first difference block and the second difference block.
An image frame sequence processing device, wherein the device comprises:

An image frame acquisition module, used to acquire a target image frame in an image frame sequence and a reference image frame corresponding to the target image frame;

A differential pixel acquisition module, used for acquiring differential pixel points between the target image frame and the reference image frame in a differential step order, and updating a first differential block in the target image frame based on a target differential pixel point of a current differential step;

a difference block merging module, configured to generate a target difference block based on the first difference block and the second difference block when the distance value between the first difference block and the second difference block is less than a preset distance threshold; wherein the second difference block is a difference block in the target image frame except the first difference block;

The frame sequence saving module is used to save the image frame sequence based on the target differential block when the target differential pixel point of the current differential step is the last differential pixel point.
The device according to claim 14, wherein the differential pixel acquisition module is specifically used to: acquire a historical differential block in the target image frame; if the target differential pixel point is not in any differential block in the historical differential blocks, determine the historical differential block whose distance to the target differential pixel point is less than a preset distance threshold as the first differential block, and update the first differential block based on the target differential pixel point of the current differential step.
According to the device according to claim 15, the differential pixel acquisition module is specifically used to: if the target differential pixel point is not in any differential block in the historical differential blocks, and the distance value between any differential block in the historical differential blocks and the target differential pixel point is greater than or equal to a preset distance threshold, generate a first differential block based on the target differential pixel point.
The device according to claim 14, wherein the difference block merging module is further configured to: determine the target difference block as a new first difference block, and continue to execute the step of generating the target difference block based on the first difference block and the second difference block if the distance value between the first difference block and the second difference block is less than a preset distance threshold.
According to the device according to claim 15, the differential pixel acquisition module is used to: acquire the first pixel data of the target image frame at the current coordinate point and the second pixel data of the reference image frame at the current coordinate point in a differential step order; if the first pixel data is different from the second pixel data, the pixel point of the current coordinate point is determined as the differential pixel point.
A computer device, wherein the computer device comprises:

one or more processors;

Memory; and

One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by the processor to implement the image frame sequence processing method according to any one of claims 1 to 13.
A computer-readable storage medium, wherein a computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the image frame sequence processing method according to any one of claims 1 to 13.