CN113808157B

CN113808157B - Image processing method and device and computer equipment

Info

Publication number: CN113808157B
Application number: CN202111366266.4A
Authority: CN
Inventors: 焦华龙
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-11-18
Filing date: 2021-11-18
Publication date: 2022-02-22
Anticipated expiration: 2041-11-18
Also published as: CN113808157A

Abstract

The application provides an image processing method, an image processing device and computer equipment. The method can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic, auxiliary driving and the like. The method comprises the following steps: acquiring a target image frame; determining a motion estimation strategy of each first image block according to the image content of the target image frame; if the whole block estimation starting strategy is adopted, motion compensation is respectively carried out on each first image block, and a motion compensation result of the target image frame is determined according to the motion compensation results of the N first image blocks; if the whole block estimation closing strategy is adopted, dividing each first image block into M unit image blocks, respectively performing motion compensation on each unit image block, and determining a motion compensation result of the target image frame according to motion compensation results of the N × M unit image blocks; and filtering the target image frame according to the motion compensation result of the target image frame to obtain a filtering result of the target image frame. The accuracy of the filtering result and the filtering efficiency can be improved.

Description

Image processing method and device and computer equipment

Technical Field

The present invention relates to the field of internet technologies, and in particular, to an image processing method, an image processing apparatus, and a computer device.

Background

With the rapid development of computer technology, the development of video coding technology is greatly promoted. In the technical field of video coding, video acquisition equipment is often influenced by factors such as self defects, angles and light rays during shooting, and noise occurs in played video pictures. Therefore, denoising video becomes an important research point in the video field.

At present, a Temporal Filtering (TF) processing technology is generally used to perform noise reduction processing on a video, and an adopted method generally includes dividing an image frame into image blocks with preset sizes, and then directly performing Motion Estimation (ME) on each obtained image block, so as to determine a Motion Compensation (MC) result of the image frame, which is a single processing mode. Therefore, how to enrich the filtering processing manner of the image frames in the video is a technical problem to be solved urgently at present.

Disclosure of Invention

The embodiment of the application provides an image processing method, an image processing device, computer equipment and a storage medium, which can determine a motion compensation result of a target image frame according to a motion estimation strategy, so that the accuracy of a filtering result and the filtering efficiency are improved.

In one aspect, an embodiment of the present application provides an image processing method, where the method includes:

acquiring a target image frame to be coded, wherein the target image frame comprises N first image blocks, and N is a positive integer;

determining a motion estimation strategy of each first image block according to the image content of the target image frame, wherein the motion estimation strategy comprises an entire block estimation opening strategy and an entire block estimation closing strategy;

if the motion estimation strategy of each first image block is an entire block estimation starting strategy, respectively performing motion compensation on each first image block, and determining a motion compensation result of the target image frame according to motion compensation results of the N first image blocks;

if the motion estimation strategy of each first image block is an integral block estimation closing strategy, dividing each first image block into M unit image blocks, respectively performing motion compensation on each unit image block, and determining a motion compensation result of a target image frame according to the motion compensation results of the N × M unit image blocks, wherein M is a positive integer;

and filtering the target image frame according to the motion compensation result of the target image frame to obtain a filtering result of the target image frame.

In one aspect, an embodiment of the present application provides an image processing apparatus, including:

the device comprises an acquisition unit, a coding unit and a decoding unit, wherein the acquisition unit is used for acquiring a target image frame to be coded, the target image frame comprises N first image blocks, and N is a positive integer;

the determining unit is used for determining a motion estimation strategy of each first image block according to the image content of the target image frame, wherein the motion estimation strategy comprises an entire block estimation opening strategy and an entire block estimation closing strategy;

the processing unit is used for respectively performing motion compensation on each first image block if the motion estimation strategy of each first image block is an entire block estimation starting strategy, and determining a motion compensation result of the target image frame according to motion compensation results of the N first image blocks;

the processing unit is further configured to divide each first image block into M unit image blocks if the motion estimation policy of each first image block is an entire block estimation closing policy, perform motion compensation on each unit image block, and determine a motion compensation result of the target image frame according to motion compensation results of the N × M unit image blocks, where M is a positive integer;

and the processing unit is also used for carrying out filtering processing on the target image frame according to the motion compensation result of the target image frame to obtain a filtering result of the target image frame.

In one aspect, the present application provides a computer device, which includes a memory and a processor, where the memory stores a computer program, and the computer program, when executed by the processor, causes the processor to execute the image processing method described above.

In one aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is read and executed by a processor of a computer device, the computer device is caused to execute the image processing method.

In one aspect, embodiments of the present application provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the image processing method described above.

In the embodiment of the present application, whether to use the motion compensation result of the current image block as the motion compensation result of the target image frame or to divide the current image block into smaller image blocks and then use the motion compensation result of the smaller image blocks as the motion compensation result of the image frame may be determined according to the video content (e.g., content subject, resolution, etc.) of the target image frame. Wherein, the image block is smaller, the determined motion compensation result is more accurate, and the filtering effect is better; the larger the image block, the less the amount of computation. Therefore, the embodiment of the application enriches the processing mode of motion compensation, and in addition, the motion estimation strategy provided by the application can balance between the calculated amount and the filtering effect, so that a better filtering effect is ensured, and a smaller calculated amount is also ensured, thereby improving the filtering efficiency and accuracy.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic architecture diagram of an image processing system according to an embodiment of the present application;

fig. 2 is a schematic flowchart of an image processing method according to an embodiment of the present application;

fig. 3a is a schematic flowchart of a process of dividing a first image block according to an embodiment of the present application;

fig. 3b is a schematic flowchart of another process for dividing a first image block according to an embodiment of the present application;

fig. 3c is a schematic flowchart of another process for dividing a first image block according to an embodiment of the present application;

FIG. 4 is a schematic flowchart of another image processing method provided in the embodiments of the present application;

fig. 5 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

The embodiment of the application provides an image processing scheme, which can be applied to various fields such as cloud technology, artificial intelligence, block chains, intelligent traffic, auxiliary driving and the like, and particularly can reduce the calculation amount of motion compensation on a target image frame to be coded, so that the coding efficiency is improved. Wherein, the general flow of the scheme can comprise: firstly, a target image frame to be coded can be obtained, wherein the target image frame comprises N first image blocks; then determining a motion estimation strategy of each first image block according to the image content of the target image frame, wherein the motion estimation strategy comprises an entire block estimation opening strategy and an entire block estimation closing strategy; if the motion estimation strategy of each first image block is an entire block estimation starting strategy, respectively performing motion compensation on each first image block, and determining a motion compensation result of the target image frame according to motion estimation results of the N first image blocks; if the motion estimation strategy of each first image block is an integral block estimation closing strategy, dividing each first image block into M unit image blocks, respectively performing motion compensation on each unit image block, and determining a motion compensation result of a target image frame according to the motion compensation results of the N × M unit image blocks, wherein M is a positive integer; and finally, filtering the target image frame according to the motion compensation result of the target image frame to obtain a filtering result of the target image frame.

Therefore, in the embodiment of the present application, it may be determined whether to use the motion compensation result of the current image block as the motion compensation result of the target image frame or to divide the current image block into smaller image blocks according to the video content (e.g., content subject, resolution, etc.) of the target image frame, and then use the motion compensation result of the smaller image block as the motion compensation result of the image frame. Wherein, the image block is smaller, the determined motion compensation result is more accurate, and the filtering effect is better; the larger the image block, the less the amount of computation. Therefore, the embodiment of the application enriches the processing mode of motion estimation, and in addition, the motion estimation strategy provided by the application can balance between the calculated amount and the filtering effect, so that a better filtering effect is ensured, and a smaller calculated amount is also ensured, thereby improving the filtering efficiency and accuracy.

Next, the image processing scheme mentioned above is described in connection with the technical terms related to the present application:

firstly, artificial intelligence:

artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. With the research and progress of artificial intelligence technology, the artificial intelligence technology is developed and applied in a plurality of fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, unmanned aerial vehicles, robots, smart medical care, smart customer service, smart games, smart traffic, and the like.

In a possible implementation manner, the image processing scheme provided in the embodiment of the present application belongs to a Computer Vision technology (CV) belonging to the field of artificial intelligence, and can be applied to the fields of video encoding, video processing, video playing (e.g., video on demand, video live broadcast), and the like, so that the image quality in the video playing process can be improved, the bandwidth of video transmission can be saved, and the like. The computer vision technology is a scientific technology for researching how to make a machine see, and in particular, it refers to that a camera and a computer are used to replace human eyes to perform machine vision such as identification, tracking and measurement on a target, and further to perform graphic processing, so that the computer processing becomes an image more suitable for human eyes to observe or is transmitted to an instrument to detect. As a scientific discipline, computer vision research-related theories and techniques attempt to build artificial intelligence systems that can capture information from images or multidimensional data. Computer vision technologies generally include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content/behavior recognition, three-dimensional object reconstruction, 3D technologies, virtual reality, augmented reality, synchronous positioning, map construction, and other technologies, and also include common biometric technologies such as face recognition and fingerprint recognition.

Secondly, cloud technology:

cloud technology (Cloud technology) is based on a general term of network technology, information technology, integration technology, management platform technology, application technology and the like applied in a Cloud computing business model, can form a resource pool, is used as required, and is flexible and convenient. Cloud computing technology will become an important support. Background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture-like websites and more web portals. With the high development and application of the internet industry, each article may have its own identification mark and needs to be transmitted to a background system for logic processing, data in different levels are processed separately, and various industrial data need strong system background support and can only be realized through cloud computing.

In a possible implementation manner, the image processing scheme provided in the embodiment of the present application may be combined with a cloud technology, and specifically, the processes of "performing motion compensation on each first image block", performing motion compensation on each unit image block ", and the like in the present application relate to large-scale computing, which requires huge computing power and storage space. Among them, cloud computing (cloud computing) is a computing mode that distributes computing tasks over a resource pool formed by a large number of computers, so that various application systems can acquire computing power, storage space, and information services as needed. The network that provides the resources is referred to as the "cloud". Resources in the "cloud" appear to the user as being infinitely expandable and available at any time, available on demand, expandable at any time, and paid for on-demand.

Thirdly, block chaining:

the Blockchain (Blockchain) is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. The blockchain is essentially a decentralized database, which is a string of data blocks associated by using cryptography, each data block contains information of a batch of network transactions, and the information is used for verifying the validity (anti-counterfeiting) of the information and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, and an application services layer.

In a possible implementation manner, the image processing scheme provided in the embodiment of the present application may be combined with a block chain technique, for example, data such as a motion estimation result of N first image blocks, a motion estimation result of N × M unit image blocks, a motion compensation result of a target image frame, and a filtering result may be uploaded to a block chain for storage, so as to ensure that data on the block chain is not easily tampered.

Fourthly, intelligent traffic:

an Intelligent Transportation System (ITS), also called Intelligent Transportation System (Intelligent Transportation System), is a comprehensive Transportation System which effectively and comprehensively applies advanced scientific technologies (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operational research, artificial intelligence and the like) to Transportation, service control and vehicle manufacturing, strengthens the relation among vehicles, roads and users, and thus forms a safety-guaranteeing, efficiency-improving, environment-improving and energy-saving comprehensive Transportation System. Or;

an Intelligent Vehicle Infrastructure Cooperative System (IVICS), referred to as a Vehicle Infrastructure Cooperative system for short, is a development direction of an Intelligent Transportation System (ITS). The vehicle-road cooperative system adopts the advanced wireless communication, new generation internet and other technologies, implements vehicle-vehicle and vehicle-road dynamic real-time information interaction in all directions, develops vehicle active safety control and road cooperative management on the basis of full-time dynamic traffic information acquisition and fusion, fully realizes effective cooperation of human and vehicle roads, ensures traffic safety, improves traffic efficiency, and thus forms a safe, efficient and environment-friendly road traffic system.

In a possible implementation manner, the image processing scheme provided by the embodiment of the application can be applied to the fields of intelligent traffic, driving assistance and the like, for example, in the driving process of a target vehicle, a vehicle-mounted terminal in the target vehicle can acquire video information of a road around the target vehicle in real time, and then acquire a target image frame from the acquired video information. Subsequently, the determined filtering image frame can be sent to the automobile data recorder, and the filtering image frame is displayed in the automobile data recorder for a driver to watch, so that the effect of improving the image quality is achieved.

Referring to fig. 1, fig. 1 is a schematic diagram illustrating an architecture of an image processing system according to an embodiment of the present disclosure. The system architecture diagram for image processing may include: a terminal device 110 and a server 120, wherein the number of terminal devices in the system architecture diagram shown in fig. 1 is only for example and does not constitute a limitation on the system architecture diagram. In addition, the terminal device 110 and the server 120 may be directly or indirectly connected through wired or wireless communication, and the application is not limited herein.

The server 120 shown in fig. 1 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a CDN (Content Delivery Network), a big data and artificial intelligence platform, and the like.

The terminal device 110 shown in fig. 1 may be a mobile phone, a tablet computer, a notebook computer, a palm computer, a Mobile Internet Device (MID), a vehicle, an in-vehicle device, a roadside device, an aircraft, a wearable device, an intelligent voice interaction device, an intelligent appliance, and the like, for example, an intelligent watch, an intelligent bracelet, a pedometer, and the like, and an intelligent device having an image processing function.

In a possible implementation manner, the terminal device 110 may obtain a target image frame to be encoded, where the target image frame includes N first image blocks; the terminal device 110 then transmits the target image frame to the server 120. Then, after acquiring the target image frame, the server 120 determines a motion estimation policy for each first image block according to the image content of the target image frame, where the motion estimation policy includes an entire block estimation on policy and an entire block estimation off policy. If the motion estimation strategy of each first image block is the whole block estimation starting strategy, the server 120 performs motion compensation on each first image block respectively, and determines a motion compensation result of the target image frame according to motion estimation results of the N first image blocks. If the motion estimation strategy of each first image block is an entire block estimation closing strategy, the server 120 divides each first image block into M unit image blocks, performs motion compensation on each unit image block, and determines a motion compensation result of the target image frame according to motion estimation results of the N × M unit image blocks. Finally, the server 120 transmits the motion compensation result of the target image frame to the terminal device 110. Subsequently, the terminal device 110 may perform filtering processing on the target image frame according to the motion compensation result of the target image frame to obtain a filtering result of the target image frame.

Certainly, motion compensation is respectively carried out on each first image block, and a motion compensation result of the target image frame is determined according to motion estimation results of the N first image blocks; and respectively performing motion compensation on each first image block, and determining a motion compensation result of the target image frame according to the motion estimation results of the N first image blocks. The above-described process is not necessarily performed by the server 120, but may be performed by the terminal device 110 or any other computer device in the terminal device cluster. The filtering process performed on the target image frame based on the motion compensation result of the target image frame to obtain the filtering result of the target image frame may not necessarily be performed by the terminal device 110, but may be performed by the server 120.

In a possible implementation manner, the image processing system provided in the embodiment of the present application may be deployed at a node of a blockchain, for example, the server 120 and the terminal device 110 may both be used as node devices of the blockchain to jointly form a blockchain network. Therefore, the image processing flow of the target image frame to be coded can be executed on the block chain, so that the fairness of the image processing flow can be guaranteed, the image processing flow can have traceability, and the safety of the image processing flow is improved.

It is to be understood that the system architecture diagram described in the embodiment of the present application is for more clearly illustrating the technical solution of the embodiment of the present application, and does not constitute a limitation to the technical solution provided in the embodiment of the present application, and as a person having ordinary skill in the art knows that along with the evolution of the system architecture and the appearance of a new service scenario, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

Next, an image processing method provided by an embodiment of the present application is described in detail with reference to the drawings. Referring to fig. 2, fig. 2 is a schematic flowchart of an image processing method according to an embodiment of the present disclosure, where the image processing method may be executed by a computer device, where the computer device may be a terminal device, and the computer device may also be a server. As shown in fig. 2, the image processing method may include S201 to S205, in which:

s201: and acquiring a target image frame to be coded, wherein the target image frame comprises N first image blocks, and N is a positive integer.

In this embodiment of the present application, the target image frame may be an image frame that needs to be temporally filtered in a video stream to be encoded. Specifically, the target image frame may be any image frame in the video stream to be encoded, or the target image frame may also be an important frame or a designated frame in the video stream to be encoded. For example, the video stream to be encoded is a video stream related to human introduction, the target image frame may be an image frame including a human image in the video stream related to human introduction, the target image frame may also be an xth frame specified by a user in the video stream related to human introduction, and so on, where X is a positive integer. Further, the so-called important frames can be determined comprehensively according to the noise intensity of the image frames, the quantization factor for video compression of the video stream to be encoded, the consumed computing resources, and the factors of not crossing scenes. For example, an image frame that satisfies one or more conditions of noise intensity being less than a noise intensity threshold, quantization factor being less than a quantization factor threshold, computational resources needing to be consumed being less than a computational resource threshold, and not crossing a scene may be selected from the video stream to be encoded as the target image frame. In addition, for each target image frame to be encoded, the computer device may further acquire a reference image frame of the target image frame from the video stream to be encoded, wherein the number of the reference image frames of the target image frame may be one or more.

In one possible implementation, the target image frame may be divided into N first image blocks, where the N first image blocks are all the same size. For example, the size of the first image block may specifically be 64 × 64. That is, the target image frame may be divided into N first image blocks each having a size of 64 × 64.

In a possible implementation manner, if the size of the first image block is not greater than the size threshold, performing partial motion estimation processing on the target image frame to obtain a partial motion estimation processing result of the target image frame, and performing motion compensation on the target image frame according to the partial motion processing result of the target image frame to obtain a motion compensation result of the target image frame. The partial Motion processing result of the target image frame may include a Motion Vector (MV) of the target image frame, the Motion compensation result of the target image frame is a predicted image frame corresponding to the target image frame, and the size of the predicted image frame is the same as that of the target image frame. That is, the computer device may predict the target image frame based on the determined motion vector of the target image frame, thereby obtaining a predicted image frame of the target image frame. In another possible implementation manner, if the size of the first image block is larger than the size threshold, the step described in S202 is triggered to be executed.

S202: and determining a motion estimation strategy of each first image block according to the image content of the target image frame, wherein the motion estimation strategy comprises an overall estimation opening strategy and an overall estimation closing strategy.

In the embodiment of the present application, the image content of the target image frame may include, but is not limited to: the content subject and resolution, e.g., the content subject of the target image frame, can be a food subject, a fashion subject, an educational subject, a health subject, and the like. In one possible implementation, the computer device determines the motion estimation strategy of each first image block according to the image content of the target image frame, which may include the following cases:

if the content subject of the target image frame comprises a preset content subject and the resolution of the target image frame is smaller than a resolution threshold value, determining that the motion estimation strategy is an integral estimation starting strategy.

And if the content subject of the target image frame does not comprise the preset content subject and the resolution of the target image frame is greater than or equal to the resolution threshold, determining that the motion estimation strategy is an integral estimation closing strategy.

And thirdly, if the content subject of the target image frame comprises a preset content subject and the resolution of the target image frame is greater than or equal to a resolution threshold, determining the motion estimation strategy as an integral estimation closing strategy.

S203: and if the motion estimation strategy of each first image block is an entire block estimation starting strategy, respectively performing motion compensation on each first image block, and determining a motion compensation result of the target image frame according to motion compensation results of the N first image blocks.

In one possible implementation, the target first image block is any one of N first image blocks. If the motion estimation policy of the target first image block is an entire block estimation starting policy, the process of performing, by the computer device, motion compensation on the target first image block to obtain a motion compensation result of the target first image block may include: firstly, if the motion estimation strategy of the target first image block is an entire block estimation starting strategy, the computer device performs block division processing on the target first image block to obtain P second image blocks, wherein P is a positive integer, and the sizes of the P second image blocks are the same. For example, please refer to fig. 3a, where fig. 3a is a schematic flowchart of a process of dividing a first image block according to an embodiment of the present application, and as shown in fig. 3a, a target first image block is a 64 × 64 first image block, and a second image block obtained by dividing the target first image block is a 32 × 32 second image block, P = 4. If the size of the second image block is not larger than the size threshold (for example, the size threshold is 8 × 8), performing sub-motion estimation processing on the target first image block to obtain a first sub-motion estimation processing result of the target first image block, and performing motion compensation on the target first image block according to the first sub-motion estimation processing result to obtain a motion compensation result of the target first image block. And if the size of the second image block is larger than the size threshold, performing sub-motion estimation processing on each second image block respectively to obtain a second sub-motion estimation processing result of each second image block, and performing motion compensation on the target first image block according to the first sub-motion estimation processing result and the P second sub-motion estimation processing results to obtain a motion compensation result of the target first image block. Wherein, the first sub-motion estimation processing result may include a motion vector of the first image block; each of the P second sub motion estimation processing results may also include a motion vector of the corresponding second image block. Specifically, the computer device may perform motion compensation on the first image block based on the motion vector of the first image block to obtain a predicted image block corresponding to the first image block; similarly, the computer device may perform motion compensation on the second image block based on the motion vector of the second image block to obtain a predicted image block corresponding to the second image block.

In one possible implementation manner, the determining, by the computer device, a motion compensation result of the target first image block according to the first sub motion estimation processing result and the P second sub motion estimation processing results may include: firstly, the computer equipment determines a decision strategy of a target first image block according to a first sub-motion estimation processing result and P second sub-motion estimation processing results, wherein the decision strategy of the target first image block comprises a non-blocking strategy and a blocking strategy. And if the decision strategy of the target first image block is a non-blocking strategy, the computer equipment determines the motion compensation result of the target first image block according to the first sub-motion estimation processing result. And if the decision strategy of the target first image block is a blocking strategy, the computer equipment respectively performs motion compensation on each second image block, and combines the motion compensation result of each second image block into the motion compensation result of the target first image block.

It is understood that the detailed process of determining the motion compensation result of each first image block can refer to the detailed process of determining the motion compensation result of the target first image block, and thus the motion compensation result of each of the N first image blocks can be determined. Subsequently, the computer device may combine the determined motion compensation results of the N first image blocks into a motion compensation result of the target image frame.

Next, for the case that the decision policy of the target first image block is a blocking policy, the computer device performs motion compensation on each second image block in the target first image block to obtain a motion compensation result of each second image block, and combines the motion compensation results of each second image block into a motion compensation result of the target first image block. Specifically, the target second image block is any one of P second image blocks included in the target first image block; the process of performing, by the computer device, motion compensation on the target second image block to obtain a motion compensation result of the target second image block may include: first, the computer device performs block division processing on a target second image block to obtain Q third image blocks, where Q is a positive integer, for example, please refer to fig. 3b, and fig. 3b is a schematic flow chart of dividing a first image block according to an embodiment of the present application, as shown in fig. 3b, the target first image block is a 64 × 64 first image block, and a second image block obtained by dividing the target first image block is a 32 × 32 second image block, where P = 4; next, if the division processing is continued for any one of the 4 32 × 32 second image blocks to obtain a 16 × 16 third image block, Q = 4. It is to be understood that the 64x64 first image block may be constituted by 4 32x32 second image blocks, or the 64x64 first image block may be constituted by 16 x 16 third image blocks. And if the size of the third image block is not larger than the size threshold, performing motion compensation on the target second image block according to the second sub-motion estimation processing result of the target second image block to obtain a motion compensation result of the target second image block. And if the size of the third image block is larger than the size threshold, performing fractional motion estimation processing on each third image block respectively to obtain a third fractional motion estimation processing result of each third image block, and performing motion compensation on the target second image block according to the second fractional motion estimation processing result of the target second image block and the Q third fractional motion estimation processing results to obtain a motion compensation result of the target second image block. The size threshold is used to determine whether a blocking stop condition is satisfied, that is, if a divided block is small enough (smaller than or equal to a preset size threshold), the blocking motion estimation process for a smaller image block is stopped.

In a possible implementation manner, the performing, by the computer device, motion compensation on the target second image block according to the second fractional motion estimation processing result and the Q third fractional motion estimation processing results of the target second image block to obtain a motion compensation result of the target second image block may include: firstly, the computer equipment determines a decision strategy of the target second image block according to the second fractional motion estimation processing result and the Q third fractional motion estimation processing results of the target second image block. And if the decision strategy of the target second image block is a non-blocking strategy, the computer equipment estimates the processing result according to the second sub-motion of the target second image block. And if the decision strategy of the target second image block is a blocking strategy, the computer equipment respectively performs motion compensation on each third image block, and combines the motion compensation result of each third image block into the motion compensation result of the target second image block.

It is to be understood that subsequently, the division of the smaller blocks may also be continued for the 16 × 16 third image block, for example, the 16 × 16 third image block is divided into 4 8 × 8 fourth image blocks. According to the logic, the fractional motion estimation is performed on each 16 × 16 third image block, and a third fractional motion estimation result of the 16 × 16 third image block is determined. And in the same way, performing fractional motion estimation on 4 8 × 8 fourth image blocks constituting the 16 × 16 third image block respectively to obtain a third fractional motion estimation result of each 8 × 8 fourth image block, and so on. And circularly executing the processes until the size of the divided image block is smaller than or equal to the size threshold, and stopping dividing the image block.

By the method, whether the image block is divided continuously or not can be judged according to the decision strategy of the image block, and the motion compensation result of the image block before division is determined according to the motion compensation result obtained after motion estimation is carried out on the divided image block.

S204: if the motion estimation strategy of each first image block is an overall block estimation closing strategy, dividing each first image block into M unit image blocks, respectively performing motion compensation on each unit image block, and determining a motion compensation result of the target image frame according to motion estimation results of the N × M unit image blocks, wherein M is a positive integer.

In this embodiment of the present application, if the motion estimation policy of each first image block is an entire block estimation closing policy, the first image block may be divided into M unit image blocks, where the sizes of the M unit image blocks are the same, and in addition, the size of the second image block may be the same as or different from the size of the unit image block, that is, P and M may be the same or different. For example, the size of the first image block may be 64 × 64, and the size of the unit image block may be 16 × 16, then M = 16. Referring to fig. 3c, fig. 3c is a schematic view of another flow chart for dividing the first image block according to an embodiment of the present application. As shown in fig. 3c, the 64 × 64 first image block may be divided into 16 unit image blocks of 16 × 16.

In a possible implementation manner, if the size of the unit image block is not greater than the size threshold, the computer device performs motion compensation on each first image block respectively, and determines a motion compensation result of the target image frame according to motion compensation results of the N first image blocks. The process of performing motion compensation on each first image block in the N first image blocks by the computer device is the same as the step performed by the computer device for performing motion compensation on the target image block in step S203, and details of the embodiment of the present application are not repeated herein.

In another possible implementation manner, if the size of the unit image block is greater than the size threshold, the unit image blocks are respectively subjected to motion estimation processing to obtain a unit motion estimation processing result of each unit image block, and the target image frame is subjected to motion compensation according to the N × M unit motion estimation processing results to obtain a motion compensation result of the target image frame. Specifically, the computer device may combine the motion compensation results of the N × M unit image blocks into the motion compensation result of the target image frame.

In this way, after determining that the motion estimation strategy of each first image block is an entire block estimation closing strategy, the computer device may perform division processing on each first image block to obtain a plurality of unit image blocks, and perform motion compensation on each unit image block, so as to determine a motion compensation result of the target image frame according to a unit division motion estimation processing result of each unit image block. Since the fine-grained fast division processing is performed, the accuracy of the motion compensation result of the target image frame can be improved.

S205: and filtering the target image frame according to the motion compensation result of the target image frame to obtain a filtering result of the target image frame.

It should be noted that the number of motion compensation results of the target image frame determined in the embodiment of the present application may be one or multiple. If the number of the motion compensation results of the target image frame is multiple, the multiple motion compensation results may be subjected to weighted average operation to obtain a motion compensation result after the weighted average operation. And then the computer equipment carries out filtering processing on the target image frame according to the motion compensation result after the weighted average operation to obtain a filtering result of the target image frame. And the filtering result of the target image frame is the filtering image frame of the target image frame.

In a possible implementation manner, the target image frame is one image frame in a video stream to be encoded, and in the embodiment of the application, a certain number of reference image frames may be selected from the video stream to be encoded for the target image frame that needs to be currently processed. According to each reference image frame, the motion compensation result of the corresponding target image frame can be obtained according to the method of the embodiment of the application, and then the determined motion compensation result of each target image frame is subjected to weighted average operation, so that the filtering image frame (i.e. the filtering result) corresponding to the target image frame can be obtained. Because the filtering image frame removes noise by using the information of a plurality of reference image frames, and meanwhile, the filtering image frame is more accurate for reversely referencing the filtering image frame for the reference image frame used for referencing before the subsequent encoding, the code rate can be reduced, and the compression rate can be improved. And finally, the computer equipment can also carry out coding processing on the filtering image frame corresponding to the target image frame to obtain a coding code stream corresponding to the target image frame.

In the embodiment of the present application, whether to use the motion compensation result of the current image block as the motion compensation result of the target image frame or to divide the current image block into smaller image blocks and then use the motion compensation result of the smaller image blocks as the motion compensation result of the image frame may be determined according to the video content (e.g., content subject, resolution, etc.) of the target image frame. Wherein, the image block is smaller, the determined motion compensation result is more accurate, and the filtering effect is better; the larger the image block, the less the amount of computation. Therefore, the embodiment of the application enriches the processing mode of motion estimation, and in addition, the motion estimation strategy provided by the application can balance between the calculated amount and the filtering effect, so that a better filtering effect is ensured, a smaller calculated amount is also ensured, the filtering efficiency and the accuracy can be improved, and the video compression performance is improved.

Next, referring to fig. 4, fig. 4 is a flowchart illustrating another image processing method provided in the embodiment of the present application, where the image processing method may be executed by a computer device, where the computer device may be a terminal device, and the computer device may also be a server, and specifically, the embodiment of fig. 4 is a specific embodiment of step S203 in the embodiment of fig. 2. As shown in fig. 4, the image processing method may include S401-S403, in which:

s401: and carrying out block division processing on the target first image block to obtain P second image blocks, wherein P is a positive integer.

For example, please refer to fig. 3a, where fig. 3a is a schematic flowchart of a process of dividing a first image block according to an embodiment of the present application, and as shown in fig. 3a, a target first image block is a 64 × 64 first image block, and a second image block obtained by dividing the target first image block is a 32 × 32 second image block, P = 4. The sizes of the P second image blocks are all the same, that is, the sizes are all 32 × 32.

S402: if the size of the second image block is not larger than the size threshold, performing sub-motion estimation processing on the target first image block to obtain a first sub-motion estimation processing result of the target first image block, and performing motion compensation on the target first image block according to the first sub-motion estimation processing result to obtain a motion compensation result of the target first image block.

For example, if the size threshold can be 32 × 32, the size of the second tile is 32 × 32. The computer device performs a sub-motion estimation process on the target first image block to obtain a first sub-motion estimation process result of the target first image block. Then, the computer device may perform motion compensation on the target first image block according to the first sub-motion estimation processing result to obtain a motion compensation result of the target first image block.

In one possible implementation, the first partial motion estimation process result includes a first Absolute Difference (SAD). If the target first image block is a 64 × 64 first image block, the first absolute value error of the target first image block may be represented as SAD64 × 64, and the specific process of the computer device for determining the first absolute value error of the target first image block SAD64 × 64 may include: firstly, computer equipment acquires a reference image frame of a target image frame from a video stream to be coded, and searches a matching image block matched with a first image block of a target from the reference image frame; the computer device then determines the first absolute value error SAD64 x64 of the target first image block from a matching image block matching the target first image block. It should be noted that, in this embodiment of the present application, the number of the reference image frames may be one or more, if the number of the reference image frames is one, a motion compensation result of the target image frame may be determined according to one reference image frame, and if the number of the reference image frames is k, and k is a positive integer, the number of the motion compensation results of the target image frame determined according to the embodiment of the present application may also be k. For convenience of explanation, the number of reference image frames is taken as an example in the embodiments of the present application to illustrate correspondingly. For example, the computer device may find a matching image block that matches the target first image block from a plurality of image blocks to be matched included in the video stream to be encoded based on a block matching algorithm. Specifically, the computer device may perform full-pixel-precision search on the target first image block within a specified search range, and then determine a matching image block that matches the target first image block from among a plurality of image blocks to be matched included in the video stream to be encoded. More specifically, the target first image block includes m × m original pixel points, and any one to-be-matched image block of the multiple to-be-matched image blocks included in the to-be-coded video stream includes m × m to-be-matched pixel points. Firstly, respectively calculating the sum of absolute errors between the pixel value of each original pixel point in m × m original pixel points and the pixel value of each pixel point to be matched in m × m pixel points of each image block to be matched; then, the computer device determines the image block to be matched with the minimum sum of absolute values of the calculated errors as a matching image block matched with the target first image block.

In one possible implementation, the first partial motion estimation process result includes a first Sum of Squares Error (SSE). If the target first image block is a first image block of 64 × 64, the first sum of squares error of the target first image block may be represented as SSE64 × 64, and then a specific procedure for the computer device to determine the first sum of squares error of the target first image block SSE64 × 64 may include: the computer device performs a split-pixel motion estimation on the target first image block and determines a first sum-of-squares error SSE64 x64 for the target first image block. Specifically, after the computer device performs full search with integer-pixel precision on the target first image block in the above steps, further, the computer device may perform sub-pixel motion estimation on the target first image block, so as to obtain a sub-pixel motion estimation result of the target first image block, i.e. a first sum-of-squares error (which may be represented as SSE64 × 64) of the target first image block.

S403: and if the size of the second image block is larger than the size threshold, performing sub-motion estimation processing on each second image block respectively to obtain a second sub-motion estimation processing result of each second image block, and performing motion compensation on the target first image block according to the first sub-motion estimation processing result and the P second sub-motion estimation processing results to obtain a motion compensation result of the target first image block.

In one possible implementation, the second partial motion estimation process result includes a second absolute value error and a second sum of squares error. For example, if the size of the target first image block is 64 × 64, the size of the second image block obtained by performing the block division process on the target first image block may be 32 × 32. Then, the detailed step of determining the second absolute value error (which may be represented as SAD32 × 32) of each of the P second image blocks by the computer device may specifically refer to the detailed step of determining the first absolute value error (SAD 64 × 64) of the target first image block by the computer device, which is not described herein again in this embodiment of the present application. Similarly, the detailed step of determining the second sum of squares error (which may be represented as SSE32 × 32) of each second image block in the P second image blocks by the computer device may specifically refer to the detailed step of determining the first sum of squares error (SSE 64 × 64) of the target first image block by the computer device, which is not repeated again in this embodiment of the present application.

In one possible implementation, the decision strategy for the computer device to determine the target first image block according to the first sub-motion estimation processing result (including the first absolute value error and the first sum of squares error) and the P second sub-motion estimation processing results (including the second absolute value error and the second sum of squares error) may include: calculating a first absolute value matching error between the first absolute value error and the sum of the P second absolute value errors; a first sum-of-squares match error is calculated between the first sum-of-squares error and the sum of the P second sum-of-squares errors. And if the first absolute value matching error is smaller than the first absolute value matching error threshold, and the first square sum matching error is smaller than the first square sum matching error threshold, determining the decision strategy of the target first image block as a non-blocking strategy. For example, please refer to fig. 3a, wherein fig. 3a is a schematic flowchart illustrating a process of dividing a first image block according to an embodiment of the present disclosure. The target first image block is a 64 × 64 first image block, and a second image block obtained by dividing the target first image block is a 32 × 32 second image block, if P = 4; the first absolute value match error calculated by the computer device from SAD64 × 64 and 4 SAD32 × 32 may be: SAD _ DIFF _1= SAD64 × 64-4 × SAD32 × 32, where the first absolute value match error threshold may be expressed as: SadTHr64x64 (typically 0< SadTHr64x64 ≦ 512). As another example, the first square sum matching error calculated by the computer device from the SSE64 × 64 and the 4 SSEs 32 × 32 may be: SSE _ DIFF _1= SSE64 × 64-4 × SSE32 × 32, the first sum-of-squares match error threshold may be expressed as: SseThr64x64 (generally 0< SseThr64x64 ≦ 12288). Finally, if SAD64 × 64-4 × SAD32 × 32 < SadTHr64x64, and SSE64 × 64-4 × SSE32 × 32 < SseThr64x64, the decision policy for the target first image block is a no-blocking policy.

In another possible implementation manner, the determining, by the computer device, the decision strategy for the target first image block according to the first absolute value error, the first sum of squares error, the P second absolute value errors, and the P second sum of squares errors may further include: and if the first absolute value matching error is larger than or equal to the first absolute value matching error threshold, determining the decision strategy of the target first image block as a blocking strategy. And if the first absolute value matching error is smaller than the first absolute value matching error threshold and the first sum of squares matching error is larger than or equal to the first sum of squares matching error threshold, determining the decision strategy of the target first image block as a blocking strategy. For example, if the target first image block is a 64 × 64 first image block and the second image block is a 32 × 32 second image block, P = 4. As described above, if SAD64 × 64-4 × SAD32 × 32 ≧ SadThr64x64, the computer device determines the decision policy for the target first image block to be the blocking policy. For another example, if SAD64 × 64-4 × SAD32 × 32 < SadTHr64x64 and SSE64 × 64-4 × SSE32 × 32 ≧ SseThr64x64, the computer device determines the decision policy for the target first image block to be the blocking policy.

Further, for the decision policy of the target first image block being the blocking policy, the computer device performs motion compensation on each second image block in the target first image block to obtain a motion compensation result of each second image block, and combines the motion compensation results of each second image block into a motion compensation result of the target first image block. Specifically, the target second image block is any one of P second image blocks included in the target first image block; the process of performing, by the computer device, motion compensation on the target second image block to obtain a motion compensation result of the target second image block may include: firstly, the computer device performs block division processing on the target second image block to obtain Q third image blocks. And if the size of the third image block is not larger than the size threshold, performing motion compensation on the target second image block according to the second sub-motion estimation processing result of the target second image block to obtain a motion compensation result of the target second image block. And if the size of the third image block is larger than the size threshold, performing fractional motion estimation processing on each third image block respectively to obtain a third fractional motion estimation processing result of each third image block, and performing motion compensation on the target second image block according to the second fractional motion estimation processing result of the target second image block and the Q third fractional motion estimation processing results to obtain a motion compensation result of the target second image block. Wherein, the first sub-motion estimation processing result may include a motion vector of the first image block; each of the P second sub motion estimation processing results may also include a motion vector of a corresponding second image block; each of the Q third motion estimation processing results may also include a motion vector of the corresponding third image block. Specifically, the computer device may perform motion compensation on the first image block based on the motion vector of the first image block to obtain a predicted image block corresponding to the first image block; similarly, the computer device may perform motion compensation on the second image block based on the motion vector of the second image block to obtain a predicted image block corresponding to the second image block; and the computer device may perform motion compensation on the third image block based on the motion vector of the third image block to obtain a predicted image block corresponding to the third image block.

In one possible implementation, the determining, by the computer device, the decision policy of the target second image block according to the second partial motion estimation processing result (including the second absolute value error and the second sum of squares error) and the Q third partial motion estimation processing results (including the third absolute value error and the third sum of squares error) of the target second image block may include: calculating a second absolute value matching error between the second absolute value error and the sum of the P second absolute value errors; a second sum of squares match error is calculated between the second sum of squares error and the sum of the Q third sum of squares errors. And if the second absolute value matching error is smaller than the second absolute value matching error threshold, and the second square sum matching error is smaller than the second square sum matching error threshold, determining the decision strategy of the target second image block as a non-blocking strategy. And if the second absolute value matching error is greater than or equal to the second absolute value matching error threshold, determining the decision strategy of the target second image block as a blocking strategy. And if the second absolute value matching error is smaller than the second absolute value matching error threshold and the second sum of squares matching error is greater than or equal to the second sum of squares matching error threshold, determining the decision strategy of the target second image block as a blocking strategy.

Specifically, please refer to fig. 3b, where fig. 3b is a schematic flowchart of another process for dividing the first image block according to an embodiment of the present application. The target first image block is a 64 × 64 first image block, and a second image block obtained by dividing the target first image block is a 32 × 32 second image block, if P = 4; next, if the division processing is continued for any one of the 4 32 × 32 second image blocks to obtain a 16 × 16 third image block, Q = 4. It is to be understood that the 64x64 first image block may be constituted by 4 32x32 second image blocks, or the 64x64 first image block may be constituted by 16 x 16 third image blocks. It should be noted that, for the detailed step of determining the third absolute value error (which may be represented as SAD16 × 16) of each of the Q third image blocks by the computer device, reference may be specifically made to the detailed step of determining the first absolute value error (SAD 64 × 64) of the target first image block by the computer device, which is not described herein again in this embodiment of the present application. Similarly, the detailed step of determining, by the computer device, the third sum of squares error (which may be represented as SSE16 × 16) of each of the Q third image blocks may specifically refer to the detailed step of determining, by the computer device, the first sum of squares error (SSE 64 × 64) of the target first image block, and this is not repeated again in this embodiment of the present application.

For example, according to the above description, it can be determined that the second absolute value matching error is: SAD _ DIFF _2= SAD32 × 32-4 × SAD16 × 16, where the second absolute value match error threshold may be expressed as: SadTHr32x32 (typically 0< SadTHr32x32 ≦ 1024). And the second sum-of-squares match error is: SSE _ DIFF _2= SSE32 × 32-4 × SSE16 × 16, where the second sum of squares match error threshold may be expressed as: SseThr32x32 (generally 0< SseThr32x32 ≦ 3072). If SAD32 × 32-4 × SAD16 × 16 < SadTHr32x32, and SSE32 × 32-4 × SSE16 × 16 < SseThr32x32, the decision strategy for the target second image block is a no-blocking strategy. And if the SAD32 is multiplied by 32-4 by SAD16 is multiplied by 16 is larger than or equal to SadTHr32x32, determining the decision strategy of the target second image block as the blocking strategy. If SAD32 × 32-4 × SAD16 × 16 < SadTHr32x32 and SSE32 × 32-4 × SSE16 × 16 ≧ SseThr32x32, the computer device determines the decision strategy for the target second image block to be the blocking strategy.

By analogy, the motion compensation result of each second image block can be determined by referring to the motion compensation result of the target second image block to obtain the motion compensation result of each second image block. And finally, combining the motion compensation result of each second image block into the motion compensation result of the target first image block. Specifically, the size of the target first image block is 64 × 64, the size of each second image block is 32 × 32, and the size of each third image block is 16 × 16. For example, if the motion compensation result of the 1 st second image block is: specifically, the motion compensation method includes splicing and combining prediction image blocks corresponding to the 4 16 × 16 third image blocks into a motion compensation result of the 1 st second image block (i.e., a prediction image block of the 1 st second image block) according to the determination of the 4 16 × 16 third image blocks. The motion compensation result of the 2 nd second image block is: the motion compensation method is determined according to 1 32 × 32 second image block, and specifically, a predicted image block corresponding to 1 32 × 32 second image block is used as a motion compensation result of the 2 nd second image block (i.e., a predicted image block of the 2 nd second image block). The motion compensation result of the 3 rd second image block is: the motion compensation method is determined according to 1 32 × 32 second image block, and specifically, a predicted image block corresponding to 1 32 × 32 second image block is used as a motion compensation result of the 3 rd second image block (i.e., a predicted image block of the 3 rd second image block). The motion compensation result of the 4 th second image block is: the motion compensation method is determined according to 1 32 × 32 second image block, and specifically, a predicted image block corresponding to 1 32 × 32 second image block is used as a motion compensation result of the 3 rd second image block (i.e., a predicted image block of the 3 rd second image block). Then, the motion compensation result of the target first image block is: and combining the predicted image blocks corresponding to the 4 16 × 16 third image blocks, the 1 predicted image blocks corresponding to the 32 × 32 second image blocks and the 1 predicted image blocks corresponding to the 32 × 32 second image blocks into the predicted image block of the target first image block.

In the embodiment of the application, on the basis of obtaining N first image blocks after dividing the target image frame, the target first image block may be continuously divided to obtain P second image blocks. Then, a decision strategy of the target first image block may be determined, and if the decision strategy of the target first image block is a blocking strategy, motion compensation is continuously performed on each second image block. In the process of performing motion compensation on the target second image block, the target second image block may be continuously divided to obtain Q third image blocks. By analogy, whether the blocking processing is continued or not can be determined according to the decision strategy of the corresponding image block. And stopping the blocking processing until the size of the divided image block is smaller than or equal to the size threshold. By the method, the motion compensation result of the target image frame is determined based on the motion estimation result obtained after motion estimation is carried out on the image block with finer granularity obtained after division, so that the accuracy of the motion compensation result can be improved, the accuracy of coding processing on the target image frame can be improved, and the video compression performance is improved.

Referring to fig. 5, fig. 5 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present disclosure. The image processing apparatus 500 is applicable to the computer device in the foregoing method embodiment, and the computer device may be a terminal device or a server. The image processing apparatus 500 may be a computer program (including program code) running on a computer device, for example, the image processing apparatus 500 is an application software; the apparatus may be used to perform the corresponding steps in the methods provided by the embodiments of the present application. The image processing apparatus 500 may include:

an obtaining unit 501, configured to obtain a target image frame to be encoded, where the target image frame includes N first image blocks, and N is a positive integer;

a determining unit 502, configured to determine a motion estimation policy of each first image block according to image content of the target image frame, where the motion estimation policy includes an entire block estimation on policy and an entire block estimation off policy;

the processing unit 503 is configured to perform motion compensation on each first image block if the motion estimation policy of each first image block is an entire block estimation starting policy, and determine a motion compensation result of the target image frame according to motion estimation results of the N first image blocks;

the processing unit 503 is further configured to, if the motion estimation policy of each first image block is an entire block estimation closing policy, divide each first image block into M unit image blocks, perform motion compensation on each unit image block, and determine a motion compensation result of the target image frame according to motion estimation results of the N × M unit image blocks, where M is a positive integer;

the processing unit 503 is further configured to perform filtering processing on the target image frame according to the motion compensation result of the target image frame, so as to obtain a filtering result of the target image frame.

In one possible implementation, the target first image block is any one of N first image blocks; if the motion estimation policy of the target first image block is an entire block estimation start policy, the process of performing motion compensation on the target first image block by the processing unit 503 to obtain a motion compensation result of the target first image block includes:

if the motion estimation strategy of the target first image block is a whole block estimation starting strategy, performing block division processing on the target first image block to obtain P second image blocks, wherein P is a positive integer;

if the size of the second image block is not larger than the size threshold, performing sub-motion estimation processing on the target first image block to obtain a first sub-motion estimation processing result of the target first image block, and performing motion compensation on the target first image block according to the first sub-motion estimation processing result to obtain a motion compensation result of the target first image block;

and if the size of the second image block is larger than the size threshold, performing sub-motion estimation processing on each second image block respectively to obtain a second sub-motion estimation processing result of each second image block, and performing motion compensation on the target first image block according to the first sub-motion estimation processing result and the P second sub-motion estimation processing results to obtain a motion compensation result of the target first image block.

In one possible implementation manner, the determining, by the processing unit 503, a motion compensation result of the target first image block according to the first sub motion estimation processing result and the P second sub motion estimation processing results includes:

determining a decision strategy of a target first image block according to the first sub-motion estimation processing result and the P second sub-motion estimation processing results, wherein the decision strategy of the target first image block comprises a non-blocking strategy and a blocking strategy;

if the decision strategy of the target first image block is a non-blocking strategy, determining a motion compensation result of the target first image block according to a first sub-motion estimation processing result;

and if the decision strategy of the target first image block is a blocking strategy, respectively performing motion compensation on each second image block, and combining the motion estimation result of each second image block into the motion compensation result of the target first image block.

In one possible implementation, the first partial motion estimation processing result includes a first absolute value error and a first sum of squares error; the target image frame is an image frame in the video stream to be coded; the processing unit 503 performs a fractional motion estimation process on the target first image block to obtain a first fractional motion estimation processing result of the target first image block, including:

acquiring a reference image frame of a target image frame from a video stream to be coded, and searching a matching image block matched with a first image block of the target from the reference image frame;

determining a first absolute value error of the target first image block according to the matched image block matched with the target first image block;

and performing pixel-by-pixel motion estimation on the target first image block, and determining a first square sum error of the target first image block.

In one possible implementation, the second partial motion estimation processing result includes a second absolute value error and a second sum of squares error; the determining, by the processing unit 503, a decision policy of the target first image block according to the first sub motion estimation processing result and the P second sub motion estimation processing results includes:

calculating a first absolute value matching error between the first absolute value error and the sum of the P second absolute value errors;

calculating a first sum-of-squares match error between the first sum-of-squares error and the sum of the P second sum-of-squares errors;

and if the first absolute value matching error is smaller than the first absolute value matching error threshold, and the first square sum matching error is smaller than the first square sum matching error threshold, determining the decision strategy of the target first image block as a non-blocking strategy.

In one possible implementation, the processing unit 503 is further configured to perform the following operations:

if the first absolute value matching error is larger than or equal to the first absolute value matching error threshold, determining the decision strategy of the target first image block as a blocking strategy;

and if the first absolute value matching error is smaller than the first absolute value matching error threshold and the first sum of squares matching error is larger than or equal to the first sum of squares matching error threshold, determining the decision strategy of the target first image block as a blocking strategy.

In a possible implementation manner, the target second image block is any one of P second image blocks included in the target first image block; the process of performing motion compensation on the target second image block by the processing unit 503 to obtain a motion compensation result of the target second image block includes:

performing block division processing on the target second image block to obtain Q third image blocks, wherein Q is a positive integer;

if the size of the third image block is not larger than the size threshold, performing motion compensation on the target second image block according to a second sub-motion estimation processing result of the target second image block to obtain a motion compensation result of the target second image block;

and if the size of the third image block is larger than the size threshold, performing fractional motion estimation processing on each third image block respectively to obtain a third fractional motion estimation processing result of each third image block, and performing motion compensation on the target second image block according to the second fractional motion estimation processing result of the target second image block and the Q third fractional motion estimation processing results to obtain a motion compensation result of the target second image block.

In one possible implementation, the image content of the target image frame includes a resolution and a content subject; the determining unit 502 determines a motion estimation strategy for each first image block according to the image content of the target image frame, including:

if the content subject of the target image frame comprises a preset content subject and the resolution of the target image frame is smaller than a resolution threshold, determining that the motion estimation strategy is an integral estimation starting strategy;

and if the content subject of the target image frame does not comprise a preset content subject, or the resolution of the target image frame is greater than or equal to the resolution threshold, determining that the motion estimation strategy is an overall estimation closing strategy.

In the embodiment of the present application, whether to use the motion compensation result of the current image block as the motion compensation result of the target image frame or to divide the current image block into smaller image blocks and then use the motion compensation result of the smaller image blocks as the motion compensation result of the image frame may be determined according to the video content (e.g., content subject, resolution, etc.) of the target image frame. Wherein, the image block is smaller, the determined motion compensation result is more accurate, and the filtering effect is better; the larger the image block, the less the amount of computation. Therefore, the embodiment of the application enriches the processing mode of motion estimation, and in addition, the motion estimation strategy provided by the application can balance between the calculated amount and the filtering effect, so that a better filtering effect is ensured, and a smaller calculated amount is also ensured, thereby improving the filtering efficiency and accuracy.

Referring to fig. 6, fig. 6 is a schematic structural diagram of a computer device according to an embodiment of the present application. The computer device 600 is configured to perform the steps performed by the computer device in the foregoing method embodiments, and the computer device 600 includes: one or more processors 610; one or more input devices 620, one or more output devices 630, and memory 640. The processor 610, input device 620, output device 630, and memory 640 are connected by a bus 650. The memory 640 is used for storing a computer program comprising program instructions, and the processor 610 is used for calling the program instructions stored in the memory 640 to execute the following operations:

if the motion estimation strategy of each first image block is an entire block estimation starting strategy, respectively performing motion compensation on each first image block, and determining a motion compensation result of the target image frame according to motion estimation results of the N first image blocks;

if the motion estimation strategy of each first image block is an integral block estimation closing strategy, dividing each first image block into M unit image blocks, respectively performing motion compensation on each unit image block, and determining a motion compensation result of a target image frame according to motion estimation results of the N × M unit image blocks, wherein M is a positive integer;

In one possible implementation, the target first image block is any one of N first image blocks; if the motion estimation strategy of the target first image block is an entire block estimation starting strategy, the process of performing motion compensation on the target first image block by the processor 610 to obtain a motion compensation result of the target first image block includes:

In one possible implementation manner, the determining, by the processor 610, a motion compensation result of the target first image block according to the first sub motion estimation processing result and the P second sub motion estimation processing results includes:

In one possible implementation, the first partial motion estimation processing result includes a first absolute value error and a first sum of squares error; the target image frame is an image frame in the video stream to be coded; the processor 610 performs a sub-motion estimation process on the target first image block to obtain a first sub-motion estimation process result of the target first image block, including:

In one possible implementation, the second partial motion estimation processing result includes a second absolute value error and a second sum of squares error; the processor 610 determines a decision strategy of the target first image block according to the first sub motion estimation processing result and the P second sub motion estimation processing results, including:

In one possible implementation, the processor 610 is further configured to perform the following operations:

In a possible implementation manner, the target second image block is any one of P second image blocks included in the target first image block; the process of the processor 610 performing motion compensation on the target second image block to obtain a motion compensation result of the target second image block includes:

In one possible implementation, the image content of the target image frame includes a resolution and a content subject; the processor 610 determines a motion estimation policy for each first image block according to the image content of the target image frame, including:

Further, here, it is to be noted that: an embodiment of the present application further provides a computer storage medium, where a computer program is stored in the computer storage medium, and the computer program includes program instructions, and when the processor executes the program instructions, the method in the foregoing corresponding embodiment can be executed, and therefore, details will not be described here again. For technical details not disclosed in the embodiments of the computer storage medium referred to in the present application, reference is made to the description of the embodiments of the method of the present application. By way of example, program instructions may be deployed to be executed on one computer device or on multiple computer devices at one site or distributed across multiple sites and interconnected by a communication network.

According to an aspect of the application, a computer program product or computer program is provided, comprising computer instructions, the computer instructions being stored in a computer readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and executes the computer instruction, so that the computer device can perform the method in the foregoing embodiments, and therefore, the detailed description is omitted here.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.

The above disclosure is only for the purpose of illustrating the preferred embodiments of the present application and is not to be construed as limiting the scope of the present application, so that the present application is not limited thereto, and all equivalent variations and modifications can be made to the present application.

Claims

1. An image processing method, characterized in that the method comprises:

acquiring a target image frame to be encoded, wherein the target image frame comprises N first image blocks, and N is a positive integer;

determining a motion estimation strategy of each first image block according to the image content of the target image frame, wherein the motion estimation strategy comprises an entire block estimation opening strategy and an entire block estimation closing strategy; the image content of the target image frame comprises a resolution and a content subject;

if the motion estimation strategy of each first image block is the whole block estimation starting strategy, respectively performing motion compensation on each first image block, and determining a motion compensation result of the target image frame according to motion compensation results of the N first image blocks;

if the motion estimation strategy of each first image block is the whole block estimation closing strategy, dividing each first image block into M unit image blocks, respectively performing motion compensation on each unit image block, and determining a motion compensation result of the target image frame according to the motion compensation results of the N × M unit image blocks, wherein M is a positive integer;

and carrying out filtering processing on the target image frame according to the motion compensation result of the target image frame to obtain a filtering result of the target image frame.

2. The method of claim 1, wherein the target first image block is any one of N first image blocks; if the motion estimation strategy of the target first image block is an entire block estimation starting strategy, the process of performing motion compensation on the target first image block to obtain a motion compensation result of the target first image block comprises the following steps:

3. The method according to claim 2, wherein the performing motion compensation on the target first image block according to the first sub motion estimation processing result and P second sub motion estimation processing results to obtain a motion compensation result of the target first image block includes:

determining a decision strategy of the target first image block according to the first sub-motion estimation processing result and the P second sub-motion estimation processing results, wherein the decision strategy of the target first image block comprises a non-blocking strategy and a blocking strategy;

if the decision strategy of the target first image block is a non-blocking strategy, determining a motion compensation result of the target first image block according to the first sub-motion estimation processing result;

and if the decision strategy of the target first image block is a blocking strategy, respectively performing motion compensation on each second image block, and combining the motion compensation result of each second image block into the motion compensation result of the target first image block.

4. The method of claim 3 wherein said first fractional motion estimation process result comprises a first absolute value error and a first sum of squares error; the target image frame is an image frame in a video stream to be coded; the performing sub-motion estimation processing on the target first image block to obtain a first sub-motion estimation processing result of the target first image block includes:

acquiring a reference image frame of the target image frame from the video stream to be coded, and searching a matching image block matched with the first image block of the target from the reference image frame;

and performing pixel-by-pixel motion estimation on the target first image block, and determining a first sum of squares error of the target first image block.

5. The method of claim 4, wherein the second fractional motion estimation process result comprises a second absolute value error and a second sum of squares error; the determining a decision strategy for the target first image block according to the first sub motion estimation processing result and the P second sub motion estimation processing results includes:

calculating a first absolute value match error between the first absolute value error and the sum of the P second absolute value errors;

and if the first absolute value matching error is smaller than a first absolute value matching error threshold and the first sum of squares matching error is smaller than a first sum of squares matching error threshold, determining that the decision strategy of the target first image block is a non-blocking strategy.

6. The method of claim 5, wherein the method further comprises:

if the first absolute value matching error is greater than or equal to the first absolute value matching error threshold, determining that the decision strategy of the target first image block is a blocking strategy;

and if the first absolute value matching error is smaller than the first absolute value matching error threshold and the first sum of squares matching error is greater than or equal to the first sum of squares matching error threshold, determining that the decision strategy of the target first image block is a blocking strategy.

7. The method according to claim 6, wherein the target second image block is any one of P second image blocks included in the target first image block; the process of performing motion compensation on the target second image block to obtain a motion compensation result of the target second image block includes:

and if the size of the third image block is larger than the size threshold, performing fractional motion estimation processing on each third image block respectively to obtain a fractional motion estimation processing result of each third image block, and performing motion compensation on the target second image block according to the second fractional motion estimation processing result of the target second image block and the Q fractional motion estimation processing results to obtain a motion compensation result of the target second image block.

8. The method according to any one of claims 1 to 7, wherein the determining a motion estimation strategy for each first image block according to the image content of the target image frame comprises:

if the content subject of the target image frame comprises a preset content subject and the resolution of the target image frame is smaller than a resolution threshold, determining that the motion estimation strategy is an entire block estimation starting strategy;

9. An image processing apparatus, characterized in that the apparatus comprises:

the determining unit is used for determining a motion estimation strategy of each first image block according to the image content of the target image frame, wherein the motion estimation strategy comprises an entire block estimation opening strategy and an entire block estimation closing strategy; the image content of the target image frame comprises a resolution and a content subject;

the processing unit is used for respectively performing motion compensation on each first image block if the motion estimation strategy of each first image block is the whole block estimation starting strategy, and determining the motion compensation result of the target image frame according to the motion compensation results of the N first image blocks;

the processing unit is further configured to, if the motion estimation policy of each first image block is the whole block estimation closing policy, divide each first image block into M unit image blocks, perform motion compensation on each unit image block, and determine a motion compensation result of the target image frame according to motion compensation results of the N × M unit image blocks, where M is a positive integer;

the processing unit is further configured to perform filtering processing on the target image frame according to the motion compensation result of the target image frame to obtain a filtering result of the target image frame.

10. A computer device, comprising: a storage device and a processor;

a memory having a computer program stored therein;

a processor for loading the computer program to implement the image processing method of any one of claims 1 to 8.

11. A computer storage medium, characterized in that the computer storage medium stores a computer program comprising program instructions which, when executed by a processor, cause a computer device having the processor to perform the steps of the method of any one of claims 1-8.