CN112565791A - Motion estimation method and device - Google Patents
Motion estimation method and device Download PDFInfo
- Publication number
- CN112565791A CN112565791A CN202011358918.5A CN202011358918A CN112565791A CN 112565791 A CN112565791 A CN 112565791A CN 202011358918 A CN202011358918 A CN 202011358918A CN 112565791 A CN112565791 A CN 112565791A
- Authority
- CN
- China
- Prior art keywords
- motion vector
- target
- image block
- determining
- vector set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 239000013598 vector Substances 0.000 claims abstract description 369
- 238000011156 evaluation Methods 0.000 claims description 74
- 238000012216 screening Methods 0.000 claims description 28
- 238000006073 displacement reaction Methods 0.000 claims description 18
- 230000006870 function Effects 0.000 claims description 15
- 238000010586 diagram Methods 0.000 description 12
- 238000012545 processing Methods 0.000 description 9
- 238000004590 computer program Methods 0.000 description 8
- 238000010845 search algorithm Methods 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000000737 periodic effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/513—Processing of motion vectors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/56—Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a motion estimation method and equipment, which are used for improving the accuracy of motion estimation. The method comprises the following steps: aiming at any target image block in a previous frame of image, determining a plurality of reference image blocks corresponding to the target image block, wherein the target image block is an image block currently subjected to motion estimation, and the reference image block is an image block subjected to motion estimation after the target image block; determining a motion vector set of the target image block and the plurality of reference image blocks in the previous frame of image, wherein the motion vector set comprises the target motion vector set of the target image block and the reference motion vector set of the reference image block; and determining the position of the target image block in the current frame image according to the frequency of the target motion vector in the target motion vector set appearing in the plurality of reference motion vector sets.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a motion estimation method and device.
Background
Motion Estimation and Motion Compensation (MEMC) is a technology widely used for frame rate conversion at present, and a method of interpolating an intermediate image between two frame images by estimating a Motion trajectory of an object in a continuous Motion image improves a video frame rate and solves problems of jitter and tailing during video playing and the like.
Motion estimation based on image block matching is a common method in MEMC, and the basic principle is to divide an image into a plurality of non-overlapping sub-blocks, each sub-block can be regarded as a moving object, and the position of an image block most similar to a target image block in a previous image is found from the image blocks divided by the current image.
However, the motion estimation is prone to be wrong due to the complicated and intricate image content, and particularly in a scene without details and periodicity, since the similarity of areas in an image is high, the matching degree of a current image block and surrounding image blocks in the same image is very high, the position of an image block similar to a target image block is prone to be estimated by mistake, so that a wrong matching result is generated, a broken picture is caused during image compensation, and the image quality is seriously affected.
Disclosure of Invention
The invention provides a motion estimation method and device, which are used for improving the accuracy of motion estimation, improving the video frame rate, improving the problems of jitter trailing and the like during video playing.
In a first aspect, the present invention provides a motion estimation method, including:
aiming at any target image block in a previous frame of image, determining a plurality of reference image blocks corresponding to the target image block, wherein the target image block is an image block currently subjected to motion estimation, and the reference image block is an image block subjected to motion estimation after the target image block;
determining a motion vector set of the target image block and the plurality of reference image blocks in the previous frame image, wherein the motion vector set comprises a target motion vector set of the target image block and a reference motion vector set of the reference image block, and motion vectors in the motion vector set are used for representing displacement generated when the target image block in the previous frame image moves into the current frame image;
and determining the position of the target image block in the current frame image according to the frequency of the target motion vector in the target motion vector set appearing in the plurality of reference motion vector sets.
After the target motion vector set of the target image block in the previous frame image is determined, the optimal target motion vector is not determined from the target motion vector set as the position of the target image block in the current frame image, but a plurality of reference image blocks corresponding to the target image block are referred, and the frequency of the target motion vector appears in the reference motion vector set of the plurality of reference image blocks, so that the position of the target image block in the current frame image is finally determined, the wrong motion estimation caused by similarity of image areas is reduced or avoided through the motion consistency of the reference areas, and the accuracy of the motion estimation is effectively improved.
As an alternative embodiment, the motion vector set is obtained by screening as follows:
determining a target evaluation index of a target motion vector in the target motion vector set and a reference evaluation index of a reference motion vector in the reference motion vector set through an evaluation function;
and screening target motion vectors corresponding to a preset number of minimum target evaluation indexes from the target motion vector set, and screening reference motion vectors corresponding to a preset number of minimum reference evaluation indexes from the reference motion vector set.
The motion vector in the motion vector set determined in this embodiment is obtained by screening, and the motion vector corresponding to the minimum target evaluation index is obtained by screening through an evaluation function, and the motion vector corresponding to the minimum target evaluation index is used for representing that the motion vector is the optimal motion vector.
As an optional implementation manner, determining, according to the frequency of occurrence of a target motion vector in the target motion vector set in a plurality of reference motion vector sets, a position of a target image block in a current frame image includes:
and determining the position of the target image block in the current frame image according to the frequency of the target motion vectors in the target motion vector set except the target motion vector corresponding to the minimum target evaluation index appearing in the multiple reference motion vector sets.
As an optional implementation manner, determining, according to the frequency of occurrence of a target motion vector in the target motion vector set in a plurality of reference motion vector sets, a position of a target image block in a current frame image includes:
and if the frequency of the target motion vector appearing in the plurality of reference motion vector sets is greater than a threshold value, determining the position of the target image block in the current frame image according to the target motion vector.
In this embodiment, the result of motion estimation is finally determined according to the frequency of the target motion vector appearing in the multiple reference motion vector sets, and if the frequency is greater than the threshold, it indicates that the motion vector conforms to most of the image blocks, and the region consistency is higher, so that erroneous motion estimation caused by similarity of image regions is reduced or avoided.
As an optional implementation manner, determining, according to the frequency of occurrence of a target motion vector in the target motion vector set in a plurality of reference motion vector sets, a position of a target image block in a current frame image includes:
and if the frequency of the target motion vector appearing in the plurality of reference motion vector sets is not greater than a threshold value, determining the position of the target image block in the current frame image according to the target motion vector corresponding to the minimum target evaluation index in the target motion vector sets.
In a second aspect, the present invention provides a motion estimation apparatus, comprising: a processor and a memory, wherein the memory stores program code that, when executed by the processor, causes the processor to perform the steps of:
aiming at any target image block in a previous frame of image, determining a plurality of reference image blocks corresponding to the target image block, wherein the target image block is an image block currently subjected to motion estimation, and the reference image block is an image block subjected to motion estimation after the target image block;
determining a motion vector set of the target image block and the plurality of reference image blocks in the previous frame image, wherein the motion vector set comprises a target motion vector set of the target image block and a reference motion vector set of the reference image block, and motion vectors in the motion vector set are used for representing displacement generated when the target image block in the previous frame image moves into the current frame image;
and determining the position of the target image block in the current frame image according to the frequency of the target motion vector in the target motion vector set appearing in the plurality of reference motion vector sets.
As an alternative embodiment, the motion vector set is obtained by screening as follows:
determining a target evaluation index of a target motion vector in the target motion vector set and a reference evaluation index of a reference motion vector in the reference motion vector set through an evaluation function;
and screening target motion vectors corresponding to a preset number of minimum target evaluation indexes from the target motion vector set, and screening reference motion vectors corresponding to a preset number of minimum reference evaluation indexes from the reference motion vector set.
As an alternative embodiment, the processor is configured to perform:
and determining the position of the target image block in the current frame image according to the frequency of the target motion vectors in the target motion vector set except the target motion vector corresponding to the minimum target evaluation index appearing in the multiple reference motion vector sets.
As an alternative embodiment, the processor is configured to perform:
and if the frequency of the target motion vector appearing in the plurality of reference motion vector sets is greater than a threshold value, determining the position of the target image block in the current frame image according to the target motion vector.
As an alternative embodiment, the processor is configured to perform:
and if the frequency of the target motion vector appearing in the plurality of reference motion vector sets is not greater than a threshold value, determining the position of the target image block in the current frame image according to the target motion vector corresponding to the minimum target evaluation index in the target motion vector sets.
In a third aspect, the present invention provides a motion estimation apparatus, comprising:
the image processing device comprises a determining reference unit, a calculating unit and a processing unit, wherein the determining reference unit is used for determining a plurality of reference image blocks corresponding to a target image block aiming at any target image block in a previous frame image, the target image block is an image block currently subjected to motion estimation, and the reference image block is an image block subjected to motion estimation after the target image block;
a determining vector unit, configured to determine a set of motion vectors of the target image block and the plurality of reference image blocks in a previous frame image, where the set of motion vectors includes a target motion vector set of the target image block and a reference motion vector set of the reference image block, and a motion vector in the set of motion vectors is used to represent a displacement generated when the target image block in the previous frame image moves into the current frame image;
and the position determining unit is used for determining the position of the target image block in the current frame image according to the frequency of the target motion vector in the target motion vector set appearing in the plurality of reference motion vector sets.
As an alternative embodiment, the motion vector set is obtained by screening as follows:
determining a target evaluation index of a target motion vector in the target motion vector set and a reference evaluation index of a reference motion vector in the reference motion vector set through an evaluation function;
and screening target motion vectors corresponding to a preset number of minimum target evaluation indexes from the target motion vector set, and screening reference motion vectors corresponding to a preset number of minimum reference evaluation indexes from the reference motion vector set.
As an optional implementation manner, the location determining unit is specifically configured to:
and determining the position of the target image block in the current frame image according to the frequency of the target motion vectors in the target motion vector set except the target motion vector corresponding to the minimum target evaluation index appearing in the multiple reference motion vector sets.
As an optional implementation manner, the location determining unit is specifically configured to:
and if the frequency of the target motion vector appearing in the plurality of reference motion vector sets is greater than a threshold value, determining the position of the target image block in the current frame image according to the target motion vector.
As an optional implementation manner, the location determining unit is specifically configured to:
and if the frequency of the target motion vector appearing in the plurality of reference motion vector sets is not greater than a threshold value, determining the position of the target image block in the current frame image according to the target motion vector corresponding to the minimum target evaluation index in the target motion vector sets.
In a fourth aspect, the present invention provides a computer storage medium having stored thereon a computer program which, when executed by a processing unit, performs the steps of the method of the first aspect.
In addition, for technical effects brought by any one implementation manner of the second aspect to the fourth aspect, reference may be made to technical effects brought by different implementation manners of the first aspect, and details are not described here.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a flowchart illustrating an implementation of a motion estimation method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a partial region of an image after a frame of image is equally divided into a plurality of parts according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a motion vector set provided in an embodiment of the present invention;
FIG. 4 is a flowchart illustrating an embodiment of motion estimation;
fig. 5 is a schematic diagram of a motion estimation apparatus according to an embodiment of the present invention;
fig. 6 is a schematic diagram of a motion estimation apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The term "and/or" in the embodiments of the present invention describes an association relationship of associated objects, and indicates that three relationships may exist, for example, a and/or B may indicate: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
The application scenario described in the embodiment of the present invention is for more clearly illustrating the technical solution of the embodiment of the present invention, and does not form a limitation on the technical solution provided in the embodiment of the present invention, and it can be known by a person skilled in the art that with the occurrence of a new application scenario, the technical solution provided in the embodiment of the present invention is also applicable to similar technical problems. In the description of the present invention, the term "plurality" means two or more unless otherwise specified.
Example 1
The frame rate of the current mainstream movie is 24fps, the frame rate of the television program is generally 25fps, and the refresh frequency of the current mainstream display equipment is 60Hz, so that when people watch the television movie, the picture can increase the repeated frames in a pull up mode. However, the method does not add new picture content to the video, the quality of the moving picture is not really changed, and the situation of 'smear' or 'blur' occurs when a high-speed moving scene (such as a ball game scene) is encountered. The MEMC intelligently calculates the motion direction of the picture in the process according to the relation between the front picture and the back picture, so that the motion is smoother, and the conditions of 'smear' or 'blur' and the like caused by the motion of people in sports events such as a ball game or a violent motion shot can be well avoided.
The motion estimation algorithm is one of the core algorithms of frame rate conversion. High quality motion estimation algorithms are a prerequisite and basis for efficient frame rate conversion. Among them, the Block matching method (BMA) is widely used in motion estimation because of its simple Algorithm and easy hardware implementation. The basic idea of the block matching method is to divide the image into many sub-blocks, then to find out the matching block of the current block in the adjacent frame according to a certain matching criterion for each block in the current frame, and thus to obtain the relative displacement between the two, i.e. the motion vector of the current block. In the search algorithm of the h.264 standard, the current frame of an image sequence is divided into subblocks of 16 × 16 sizes that do not overlap each other, each subblock can be divided into smaller subblocks, and the current subblock finds the best matching block within a certain search range of the corresponding position in the adjacent frame according to a certain block matching criterion, thereby obtaining a motion vector and a matching error. The estimation accuracy and computational complexity of motion estimation depend on the search strategy and block matching criteria.
Due to the complicated image content, especially the similarity of the scene areas without details and periodicity is high, the matching degree of the current image block and the surrounding image blocks is high, the motion estimation is easy to make mistakes, and the wrong matching result causes the broken picture during the image compensation, thereby seriously affecting the image quality.
Aiming at the problem that image block matching errors are easy to occur in processing periodic and non-detailed image scenes by the current motion estimation and motion compensation technologies, the embodiment of the invention improves the current motion estimation method, after the motion vector of a target image block is determined, the position of the target image block in a current image is not directly determined according to the optimal motion vector, but a plurality of motion vectors are selected, and the motion vector with the highest frequency is screened out from the motion vectors of reference image blocks behind the target image block, so that the motion vector is considered to accord with the motion displacement of most image blocks, thereby effectively solving the problem that the estimation result is more reliable due to higher similarity between the image blocks of the periodic scene and the reference of the displacement condition of the image blocks.
As shown in fig. 1, an implementation flow of a motion estimation method provided by an embodiment of the present invention is as follows:
the plurality of reference image blocks determined in the present embodiment are determined with respect to the current target image block, and since when performing motion estimation on an image, each image block in the image is typically subjected to motion estimation from left to right and from top to bottom, the reference image block corresponding to the target image block in the present embodiment may be a plurality of reference image blocks arranged in order after the target image block, as shown in fig. 2, a partial region of a frame of the image is equally divided into a plurality of parts of the image, the resolution of the image is 3840 × 2160, and the image is divided into 240 × 135 image blocks, then the size of each image block is 16 × 16, a represents the target image block, and B, C, D, E represents the reference image block corresponding to the target image block.
It should be noted that, in the embodiment of the present invention, the target image block and the plurality of reference image blocks corresponding to the target image block are image blocks in the same frame image.
the motion vector set in this embodiment includes a target motion vector set and a reference motion vector set, where the target motion vector set includes a plurality of target motion vectors, and the reference motion vector set includes a plurality of reference motion vectors.
It is easily understood that the motion estimation based on the image block matching algorithm divides the image into a plurality of sub-blocks which do not overlap with each other, and considers that the displacement of all pixels in the sub-blocks is the same, which means that each sub-block in the image is regarded as a moving object. Suppose that in the image sequence, time t corresponds to the k-th frame image and time t-1 corresponds to the k-1 th frame image. For a sub-block in the k-1 frame, the sub-block most similar to the sub-block in the k frame is found in the k frame, the process is called finding a matching block, and the position of the matching block in the k-1 frame is considered to be the position before the sub-block displacement of the k frame, and the change of the position is represented by a motion vector.
A motion vector is understood to be a motion displacement determined before motion estimation, and a set of motion vectors is a set of displacement sets (usually more than ten) obtained by referring to motion information of previous image blocks, and the set of motion vectors is unique for each image block and is not identical between image blocks.
And 102, determining the position of the target image block in the current frame image according to the frequency of the target motion vector in the target motion vector set appearing in the plurality of reference motion vector sets.
The target motion vectors determined by the embodiment of the invention are multiple, and the target motion vectors with high area consistency are determined from the multiple target motion vectors, so that the estimation accuracy is improved, and the wrong motion estimation caused by the similarity of image areas is reduced or avoided.
It should be noted that, in the embodiment of the present invention, when performing position estimation (motion estimation) on a target image block, multiple reference image blocks subsequent to the target image block are referred to, so that according to the number of times that a motion vector obtained by performing motion estimation on the target image block appears in motion vectors of the multiple reference image blocks, if the number of times that the motion vector appears is high, it is indicated that the motion vector conforms to a motion trend of most image blocks, and the reliability is higher.
In the implementation, each frame of image is equally divided into several image blocks (which may be understood as pixel blocks), and for each frame of image, the position of any target image block in the previous frame of image can be determined according to the method in the present embodiment. In this embodiment, any target image block is an image block in which a plurality of reference image blocks exist after the target image block in an image, that is, the any target image block does not include an image block at an edge of the image.
The method for performing motion estimation in this embodiment may use an image block matching algorithm, but is not limited to use this method, and this embodiment does not excessively limit this method. The performance of image block matching is mainly determined by three factors, namely, the matching criteria, the search algorithm and the search area, and the complexity is also determined by the above three factors. The image block matching criteria in this embodiment include, but are not limited to Sum of Absolute Differences (SAD), Search area indicates how large a motion Search is performed, and Search algorithms include, but are not limited to, Full Search Method (FS), Three-Step Search algorithm (TSS), and the like.
As an optional implementation manner, the motion vector set in the embodiment of the present invention is obtained by screening in the following manner:
1) determining a target evaluation index of a target motion vector in the target motion vector set and a reference evaluation index of a reference motion vector in the reference motion vector set through an evaluation function;
2) and screening target motion vectors corresponding to a preset number of minimum target evaluation indexes from the target motion vector set, and screening reference motion vectors corresponding to a preset number of minimum reference evaluation indexes from the reference motion vector set.
The evaluation function in this embodiment means that when the optimal motion vector is screened for each image block, all reference vectors obtain an evaluation index according to a preset algorithm, and the optimal motion vector is usually selected according to the evaluation criterion. The evaluation index in this embodiment may be, but is not limited to, SAD, that is, the pixel difference between the corresponding positions of the two frames of images calculated according to the motion vector, and the evaluation index in this embodiment may be any index for evaluating the quality of the vector, and is not limited to this.
Fig. 3 shows a motion vector set in the present embodiment, wherein the target motion vector set a includes a plurality of target motion vectors, which are a1, a2, a3, a4, and a 5; the reference motion vector set B comprises a plurality of target motion vectors B1, B2, B3, B4 and B5; the reference motion vector set C includes a plurality of target motion vectors C1, C2, C3, C4, C5; the reference motion vector set D comprises a plurality of target motion vectors D1, D2, D3, D4 and D5; the reference motion vector set E includes a plurality of target motion vectors E1, E2, E3, E4, and E5, respectively.
As an optional implementation manner, determining, according to the frequency of occurrence of a target motion vector in the target motion vector set in a plurality of reference motion vector sets, a position of a target image block in a current frame image includes:
and determining the position of the target image block in the current frame image according to the frequency of the target motion vectors in the target motion vector set except the target motion vector corresponding to the minimum target evaluation index appearing in the multiple reference motion vector sets.
It is easy to understand that, since the target motion vector corresponding to the minimum target evaluation index is the displacement of the image block that best matches the target image block, but based on the higher similarity between image blocks in the image, in order to prevent erroneous determination, reference is made to the target motion vectors in the target motion vector set except for the target motion vector corresponding to the minimum target evaluation index, so that the motion estimation is more reliable.
As an optional implementation manner, determining, according to the frequency of occurrence of a target motion vector in the target motion vector set in a plurality of reference motion vector sets, a position of a target image block in a current frame image includes:
and if the frequency of the target motion vector appearing in the plurality of reference motion vector sets is greater than a threshold value, determining the position of the target image block in the current frame image according to the target motion vector.
In this embodiment, when performing motion estimation on an image block, the frequency of a target motion vector appearing in a plurality of reference motion vector sets is referred to, and if the frequency is greater than a threshold, it is indicated that the motion displacement vector conforms to most of the image blocks, and the result is more reliable.
As an optional implementation manner, determining, according to the frequency of occurrence of a target motion vector in the target motion vector set in a plurality of reference motion vector sets, a position of a target image block in a current frame image includes:
and if the frequency of the target motion vector appearing in the plurality of reference motion vector sets is not greater than a threshold value, determining the position of the target image block in the current frame image according to the target motion vector corresponding to the minimum target evaluation index in the target motion vector sets.
In practice, if the target motion vector set a includes a1, a2, a3, a4, a 5; the reference motion vector set B includes B1, B2, B3, B4, B5; the reference motion vector set C includes C1, C2, C3, C4, C5; the reference motion vector set D includes D1, D2, D3, D4, D5; the reference motion vector set E includes E1, E2, E3, E4, E5. Counting the frequency of a2 in the B, C, D, E set, counting the frequency of a3 in the B, C, D, E set, counting the frequency of a4 in the B, C, D, E set, counting the frequency of a5 in the B, C, D, E set, and determining the maximum frequency; if the maximum frequency is greater than the threshold, determining the position of the target image block in the current frame image according to the target motion vector corresponding to the maximum frequency, and if the maximum frequency is not greater than the threshold, determining the position of the target image block in the current frame image according to the minimum target motion vector in the target motion vectors. Because the similarity among the image blocks of the periodic scene is high, the judgment only by using the current image block has inaccurate condition, so the result of referring to a plurality of image blocks for motion estimation is more reliable.
As shown in fig. 4, an embodiment of the present invention further provides a specific implementation flow of motion estimation, which is as follows:
the target image block is an image block currently performing motion estimation, and the reference image block is an image block performing motion estimation after the target image block.
Example 2
Based on the same inventive concept, an embodiment of the present invention further provides a motion estimation device, and since the device is a device in the method in the embodiment of the present invention, and the principle of the device to solve the problem is similar to the method, the implementation of the device may refer to the implementation of the method, and repeated details are not repeated.
As shown in fig. 5, the apparatus comprises a processor 500 and a memory 501, wherein the memory stores program code which, when executed by the processor, causes the processor to perform the steps of:
aiming at any target image block in a previous frame of image, determining a plurality of reference image blocks corresponding to the target image block, wherein the target image block is an image block currently subjected to motion estimation, and the reference image block is an image block subjected to motion estimation after the target image block;
determining a motion vector set of the target image block and the plurality of reference image blocks in the previous frame image, wherein the motion vector set comprises a target motion vector set of the target image block and a reference motion vector set of the reference image block, and motion vectors in the motion vector set are used for representing displacement generated when the target image block in the previous frame image moves into the current frame image;
and determining the position of the target image block in the current frame image according to the frequency of the target motion vector in the target motion vector set appearing in the plurality of reference motion vector sets.
As an alternative embodiment, the motion vector set is obtained by screening as follows:
determining a target evaluation index of a target motion vector in the target motion vector set and a reference evaluation index of a reference motion vector in the reference motion vector set through an evaluation function;
and screening target motion vectors corresponding to a preset number of minimum target evaluation indexes from the target motion vector set, and screening reference motion vectors corresponding to a preset number of minimum reference evaluation indexes from the reference motion vector set.
As an alternative embodiment, the processor is configured to perform:
and determining the position of the target image block in the current frame image according to the frequency of the target motion vectors in the target motion vector set except the target motion vector corresponding to the minimum target evaluation index appearing in the multiple reference motion vector sets.
As an alternative embodiment, the processor is configured to perform:
and if the frequency of the target motion vector appearing in the plurality of reference motion vector sets is greater than a threshold value, determining the position of the target image block in the current frame image according to the target motion vector.
As an alternative embodiment, the processor is configured to perform:
and if the frequency of the target motion vector appearing in the plurality of reference motion vector sets is not greater than a threshold value, determining the position of the target image block in the current frame image according to the target motion vector corresponding to the minimum target evaluation index in the target motion vector sets.
Example 3
Based on the same inventive concept, the embodiment of the present invention further provides a motion estimation apparatus, and since the apparatus is an apparatus in the method in the embodiment of the present invention, and the principle of the apparatus to solve the problem is similar to the method, the implementation of the apparatus may refer to the implementation of the method, and repeated details are not repeated.
As shown in fig. 6, the apparatus includes:
a determining reference unit 600, configured to determine, for any target image block in an image of a previous frame, a plurality of reference image blocks corresponding to the target image block, where the target image block is an image block currently undergoing motion estimation, and the reference image block is an image block after the target image block undergoing motion estimation;
a determining vector unit 601, configured to determine a set of motion vectors of the target image block and the plurality of reference image blocks in the previous frame image, where the set of motion vectors includes a target motion vector set of the target image block and a reference motion vector set of the reference image block, and a motion vector in the set of motion vectors is used to represent a displacement generated by a target image block in the previous frame image moving into the current frame image;
a position determining unit 602, configured to determine a position of the target image block in the current frame image according to a frequency of occurrence of the target motion vector in the target motion vector set in the multiple reference motion vector sets.
As an alternative embodiment, the motion vector set is obtained by screening as follows:
determining a target evaluation index of a target motion vector in the target motion vector set and a reference evaluation index of a reference motion vector in the reference motion vector set through an evaluation function;
and screening target motion vectors corresponding to a preset number of minimum target evaluation indexes from the target motion vector set, and screening reference motion vectors corresponding to a preset number of minimum reference evaluation indexes from the reference motion vector set.
As an optional implementation manner, the location determining unit is specifically configured to:
and determining the position of the target image block in the current frame image according to the frequency of the target motion vectors in the target motion vector set except the target motion vector corresponding to the minimum target evaluation index appearing in the multiple reference motion vector sets.
As an optional implementation manner, the location determining unit is specifically configured to:
and if the frequency of the target motion vector appearing in the plurality of reference motion vector sets is greater than a threshold value, determining the position of the target image block in the current frame image according to the target motion vector.
As an optional implementation manner, the location determining unit is specifically configured to:
and if the frequency of the target motion vector appearing in the plurality of reference motion vector sets is not greater than a threshold value, determining the position of the target image block in the current frame image according to the target motion vector corresponding to the minimum target evaluation index in the target motion vector sets.
An embodiment of the present invention further provides a computer-readable non-volatile storage medium, which includes program code, and when the program code runs on a computing terminal, the program code is configured to enable the computing terminal to execute the following steps:
aiming at any target image block in a previous frame of image, determining a plurality of reference image blocks corresponding to the target image block, wherein the target image block is an image block currently subjected to motion estimation, and the reference image block is an image block subjected to motion estimation after the target image block;
determining a motion vector set of the target image block and the plurality of reference image blocks in the previous frame image, wherein the motion vector set comprises a target motion vector set of the target image block and a reference motion vector set of the reference image block, and motion vectors in the motion vector set are used for representing displacement generated when the target image block in the previous frame image moves into the current frame image;
and determining the position of the target image block in the current frame image according to the frequency of the target motion vector in the target motion vector set appearing in the plurality of reference motion vector sets.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (10)
1. A method of motion estimation, the method comprising:
aiming at any target image block in a previous frame of image, determining a plurality of reference image blocks corresponding to the target image block, wherein the target image block is an image block currently subjected to motion estimation, and the reference image block is an image block subjected to motion estimation after the target image block;
determining a motion vector set of the target image block and the plurality of reference image blocks in the previous frame image, wherein the motion vector set comprises a target motion vector set of the target image block and a reference motion vector set of the reference image block, and motion vectors in the motion vector set are used for representing displacement generated when the target image block in the previous frame image moves into the current frame image;
and determining the position of the target image block in the current frame image according to the frequency of the target motion vector in the target motion vector set appearing in the plurality of reference motion vector sets.
2. The method of claim 1, wherein the set of motion vectors is obtained by screening:
determining a target evaluation index of a target motion vector in the target motion vector set and a reference evaluation index of a reference motion vector in the reference motion vector set through an evaluation function;
and screening target motion vectors corresponding to a preset number of minimum target evaluation indexes from the target motion vector set, and screening reference motion vectors corresponding to a preset number of minimum reference evaluation indexes from the reference motion vector set.
3. The method according to claim 2, wherein determining the position of the target image block in the current frame image according to the frequency of the target motion vector in the target motion vector set appearing in the plurality of reference motion vector sets comprises:
and determining the position of the target image block in the current frame image according to the frequency of the target motion vectors in the target motion vector set except the target motion vector corresponding to the minimum target evaluation index appearing in the multiple reference motion vector sets.
4. The method according to any one of claims 1 to 3, wherein determining the position of the target image block in the current frame image according to the frequency of the target motion vector in the target motion vector set appearing in the plurality of reference motion vector sets comprises:
and if the frequency of the target motion vector appearing in the plurality of reference motion vector sets is greater than a threshold value, determining the position of the target image block in the current frame image according to the target motion vector.
5. The method according to any one of claims 1 to 3, wherein determining the position of the target image block in the current frame image according to the frequency of the target motion vector in the target motion vector set appearing in the plurality of reference motion vector sets comprises:
and if the frequency of the target motion vector appearing in the plurality of reference motion vector sets is not greater than a threshold value, determining the position of the target image block in the current frame image according to the target motion vector corresponding to the minimum target evaluation index in the target motion vector sets.
6. A motion estimation device, characterized in that the device comprises: a processor and a memory, wherein the memory stores program code that, when executed by the processor, causes the processor to perform the steps of:
aiming at any target image block in a previous frame of image, determining a plurality of reference image blocks corresponding to the target image block, wherein the target image block is an image block currently subjected to motion estimation, and the reference image block is an image block subjected to motion estimation after the target image block;
determining a motion vector set of the target image block and the plurality of reference image blocks in the previous frame image, wherein the motion vector set comprises a target motion vector set of the target image block and a reference motion vector set of the reference image block, and motion vectors in the motion vector set are used for representing displacement generated when the target image block in the previous frame image moves into the current frame image;
and determining the position of the target image block in the current frame image according to the frequency of the target motion vector in the target motion vector set appearing in the plurality of reference motion vector sets.
7. The apparatus of claim 6, wherein the set of motion vectors is obtained by screening as follows:
determining a target evaluation index of a target motion vector in the target motion vector set and a reference evaluation index of a reference motion vector in the reference motion vector set through an evaluation function;
and screening target motion vectors corresponding to a preset number of minimum target evaluation indexes from the target motion vector set, and screening reference motion vectors corresponding to a preset number of minimum reference evaluation indexes from the reference motion vector set.
8. The apparatus of claim 7, wherein the processor is configured to perform:
and determining the position of the target image block in the current frame image according to the frequency of the target motion vectors in the target motion vector set except the target motion vector corresponding to the minimum target evaluation index appearing in the multiple reference motion vector sets.
9. The apparatus of any of claims 6 to 8, wherein the treatment device is configured to perform:
and if the frequency of the target motion vector appearing in the plurality of reference motion vector sets is greater than a threshold value, determining the position of the target image block in the current frame image according to the target motion vector.
10. The apparatus of any of claims 6 to 8, wherein the treatment device is configured to perform:
and if the frequency of the target motion vector appearing in the plurality of reference motion vector sets is not greater than a threshold value, determining the position of the target image block in the current frame image according to the target motion vector corresponding to the minimum target evaluation index in the target motion vector sets.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011358918.5A CN112565791A (en) | 2020-11-27 | 2020-11-27 | Motion estimation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011358918.5A CN112565791A (en) | 2020-11-27 | 2020-11-27 | Motion estimation method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112565791A true CN112565791A (en) | 2021-03-26 |
Family
ID=75046398
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011358918.5A Pending CN112565791A (en) | 2020-11-27 | 2020-11-27 | Motion estimation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112565791A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1578469A (en) * | 2003-07-29 | 2005-02-09 | 三星电子株式会社 | Apparatus for estimating motion considering correlation between blocks and method thereof |
| JP2005318106A (en) * | 2004-04-27 | 2005-11-10 | Mitsubishi Electric Corp | Moving image encoding device and program |
| US20090195698A1 (en) * | 2008-02-04 | 2009-08-06 | Chung-Yi Chen | Video processing apparatus and related method to determine motion vector |
| US20090244388A1 (en) * | 2008-03-27 | 2009-10-01 | Siou-Shen Lin | Motion estimation method and related apparatus for determining target motion vector according to motion of neighboring image blocks |
| CN102088589A (en) * | 2009-12-08 | 2011-06-08 | 英特尔公司 | Frame rate conversion using bi-directional, local and global motion estimation |
| CN103338377A (en) * | 2013-07-11 | 2013-10-02 | 青岛海信信芯科技有限公司 | Method for confirming optimal motion vector in motion estimation |
| CN111754429A (en) * | 2020-06-16 | 2020-10-09 | Oppo广东移动通信有限公司 | Motion vector post-processing method and device, electronic device and storage medium |
-
2020
- 2020-11-27 CN CN202011358918.5A patent/CN112565791A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1578469A (en) * | 2003-07-29 | 2005-02-09 | 三星电子株式会社 | Apparatus for estimating motion considering correlation between blocks and method thereof |
| JP2005318106A (en) * | 2004-04-27 | 2005-11-10 | Mitsubishi Electric Corp | Moving image encoding device and program |
| US20090195698A1 (en) * | 2008-02-04 | 2009-08-06 | Chung-Yi Chen | Video processing apparatus and related method to determine motion vector |
| US20090244388A1 (en) * | 2008-03-27 | 2009-10-01 | Siou-Shen Lin | Motion estimation method and related apparatus for determining target motion vector according to motion of neighboring image blocks |
| CN102088589A (en) * | 2009-12-08 | 2011-06-08 | 英特尔公司 | Frame rate conversion using bi-directional, local and global motion estimation |
| CN103338377A (en) * | 2013-07-11 | 2013-10-02 | 青岛海信信芯科技有限公司 | Method for confirming optimal motion vector in motion estimation |
| CN111754429A (en) * | 2020-06-16 | 2020-10-09 | Oppo广东移动通信有限公司 | Motion vector post-processing method and device, electronic device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230077355A1 (en) | Tracker assisted image capture | |
| EP2180695B1 (en) | Apparatus and method for improving frame rate using motion trajectory | |
| EP3488388B1 (en) | Video processing method and apparatus | |
| US10154230B2 (en) | Generating an output frame for inclusion in a video sequence | |
| JP4991101B2 (en) | Temporary motion vector filtering | |
| CN107968946B (en) | Video frame rate improving method and device | |
| CN110378348B (en) | Video instance segmentation method, apparatus and computer-readable storage medium | |
| US8773595B2 (en) | Image processing | |
| US8804834B2 (en) | Image processing apparatus, image processing method and image processing program | |
| EP3149940B1 (en) | Block-based static region detection for video processing | |
| KR20090006068A (en) | Method and apparatus for modifying a moving image sequence | |
| JP2003533800A (en) | Motion estimator to reduce halo in MC upconversion | |
| CN106791279B (en) | Motion compensation method and system based on occlusion detection | |
| CN111629215B (en) | Method for detecting video static identification, electronic equipment and storage medium | |
| CN112800850A (en) | Video processing method and device, electronic equipment and storage medium | |
| CN114302226B (en) | Intelligent cutting method for video picture | |
| US9357161B1 (en) | Motion vector interpolation for motion compensation | |
| CN112565791A (en) | Motion estimation method and device | |
| JP2006521740A (en) | Motion vector determination method | |
| JP2006215655A (en) | Method, apparatus, program and program storage medium for detecting motion vector | |
| CN109788297B (en) | Video frame rate up-conversion method based on cellular automaton | |
| CN112837349A (en) | Target tracking method, target tracking equipment and computer-readable storage medium | |
| JP2980810B2 (en) | Motion vector search method and apparatus | |
| JP5824937B2 (en) | Motion vector deriving apparatus and method | |
| KR102105766B1 (en) | Apparatus and method for motion compensated frame interpolation suitable for both linear and nolinear motion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210326 |