WO2017020393A1

WO2017020393A1 - Motion estimation method and motion estimation system based on block matching, and application thereof

Info

Publication number: WO2017020393A1
Application number: PCT/CN2015/088947
Authority: WO
Inventors: 李志�; 张光烈; 李文荣
Original assignee: 深圳大学
Priority date: 2015-08-05
Filing date: 2015-09-06
Publication date: 2017-02-09
Also published as: CN105243673A; CN105243673B

Abstract

The present invention provides a motion estimation method and system based on block matching, and an application thereof. The method comprises the following steps: obtaining a rotation motion image sequence of a small object; performing translational motion compensation on a target small object by using a customized block algorithm that is based on blocking matching; and performing pixel block related analysis on the target small object on which the translational motion compensation has performed, to calculate the number of rotations of the target small object, a round rotation template being used in the customized algorithm that is based on block matching. The method is used for rotation motion tracking of a target, and uses a round rotation template, so that the tracking efficiency of rotation motion of the target object by using a block matching algorithm is effectively improved. In view of this, it is proposed that the method is applied to, based on the above matching algorithm, cell rotation speed estimation, and has far-reaching significance for the analysis of the dynamics characteristics of a cell.

Description

A Motion Estimation Method Based on Block Matching, Motion Estimation System and Its Application

Technical field

The invention relates to the technical field of image processing, in particular to an image motion estimation method based on block matching algorithm, a motion estimation system and an application thereof in cell dynamics analysis.

Background technique

Computer vision-based motion estimation is a research project with broad application prospects, such as target tracking in the military field, dynamic monitoring of industrial processes, video data compression in the commercial field, video data analysis, and cardiac motion research in the medical field. Virtual reality and other aspects.

In the video sequence changing with time, there is a large spatial redundancy between the frame and the frame. The goal of motion estimation is to achieve the following goal: if the scene and the camera are both stationary, the position of the scene in the current frame It should be the same as the position in the next frame. If there is still a moving object in the still scene, the estimated optimal motion position for a pixel on the moving object in the current frame should be the position of the pixel in the next frame.

Therefore, motion estimation can effectively remove redundancy and preserve valid information between frames, which is very important for image sequence (including video) data compression and transmission.

Motion estimation algorithms are diverse and can be roughly divided into four categories: block matching, recursive estimation, Bayesian estimation, and optical flow. Among them, the block matching algorithm is the most simple and effective, and is widely used. The typical algorithms in the block matching algorithm include: full search algorithm (calculate all pixels in the search window (M+2w) x (N+2w) to find the smallest error The best match block. For the search of the motion vector of a block to be matched in the current frame, calculate (2w+1)x(2w+1) times error value); the three-step search algorithm (the three-step search process is: (1) step by w/2 Long, test eight points centered on the origin; (2) center on the minimum matching error point, halve the step size, test the new eight points; (3) repeat step 2 to get the final motion vector. TSS algorithm for each block The test points are fixed (988) 25. When the displacement size is w<7, the acceleration factor of the three-step search relative to the full search algorithm is 9) and the two-dimensional logarithmic search (to track the direction of the minimum mean square error) The main idea is to initialize five points, one point is the origin, the other four points are (±w/2, ±w/2); then with the same step size, the smallest point searched in the above step is the center test point; then, step Repeat the above steps for half lengths until the step size becomes 1 stop) and so on.

However, in the existing block matching algorithm, the full search algorithm has high accuracy, but the speed is too slow; while other fast search methods reduce the computational complexity by limiting the number of search positions, but it is not conducive to estimating small motion and Search is easy to fall into local optimum; and the matching template of the rectangle is used, only the panning operation can be performed, and the tracking efficiency is too low when the target performs the rotating motion.

Recursive Estimation Although the recursive estimation algorithm converges faster when the moving target is small, there is also the ability to use past information for estimation. However, when the frame sequence changes before and after the image sequence is relatively large, it is difficult to obtain correct results by this method, and the absolute value of the frame difference of the point shift often has multiple minimum values, and the result of the recursive estimation algorithm is sometimes a local minimum. Not a global minimum.

The Bayesian estimation method is too computationally intensive. The iterative speed of the optical flow field method is relatively fast, but the obtained optical flow field is only an approximation of the velocity field. The brightness of the image is abrupt and transported At the discontinuity, the assumptions that the algorithm relies on are not true, and the resulting error is also large.

Summary of the invention

In view of the above-mentioned deficiencies of the prior art, the object of the present invention is to provide a motion estimation method based on block matching, a motion estimation system and an application thereof, aiming at solving the problem that the existing block matching algorithm is too low in tracking the rotational motion. Take into account the speed and accuracy of the calculation.

In order to achieve the above object, the present invention adopts the following technical solutions:

A motion estimation method based on block matching, wherein the method comprises the steps of: acquiring a rotational motion image sequence of a small object; and performing translation compensation on the target small object by a block matching based custom block algorithm; The target micro object is then subjected to pixel block correlation analysis to calculate the number of rotations of the target micro object; the block matching based custom algorithm uses a circular rotation template.

The motion estimation method, wherein the method further comprises: performing noise reduction on a rotated moving image sequence of the minute object by using a Gaussian low-pass filter and enhancing contrast of the target minute object and the background by histogram equalization.

The motion estimation method, wherein the step of performing translation compensation on a target micro object by a block matching-based custom block algorithm specifically includes: generating a circular rotation template; performing block matching by using the circular rotation template, Estimating the trajectory of the target minute object.

The motion estimation method, wherein the step of generating a circular rotation template specifically includes: converting a target image into a grayscale image; converting the grayscale image into a binary image by an adaptive threshold algorithm; The center and radius of the circular rotating template, The circular rotating template is formed.

The motion estimation method, wherein the block matching step specifically includes: calculating, by using a preset matching criterion, a best matching block as a reference block in the search window; and calculating a motion vector of the reference block to a current position of the macroblock.

The motion estimation method, wherein the matching criterion is specifically represented by the following formula:

Where (x, y) ∈ S, (i, j) ∈ M, S is a search window, and M is a circular rotation template.

The motion estimation method, wherein an absolute difference and a minimum motion vector in a search window corresponding to the motion vector are specifically calculated by the following formula:

The motion estimation method, wherein the pixel correlation analysis specifically includes: calculating a correlation coefficient between a template and a sequential image block by using Equation 1; searching for a local maximum value of the correlation coefficient to track a peak point; The index calculation obtains the number of revolutions of the minute object; the formula 1 is:

Where t(·) is a template and f(·) is a sequential image block.

with

They are the average of t(·) and f(·), respectively.

A method for analyzing a cell dynamics characteristic, wherein a rotational velocity analysis is performed on a cell in a dielectrophoretic force field using a motion estimation method as described above.

A motion estimation system based on block matching, wherein the system comprises: an image acquisition module for acquiring a sequence of rotational motion images of a small object; and a translation compensation module for targeting the target by a block matching based block algorithm Small objects are compensated for translation; And a correlation coefficient calculation module, configured to perform pixel block correlation analysis on the target micro object after the translation compensation to calculate a rotation number of the target micro object; and the block matching based custom algorithm uses a circle Rotate the template.

Advantageous Effects: The present invention provides a motion estimation method based on block matching, a motion estimation system and an application thereof, and uses a circular rotation template for target rotational motion tracking, thereby effectively improving the rotation of the block matching algorithm for the target object. The tracking efficiency of the motion greatly improves the efficiency of the block matching algorithm while maintaining accuracy. Based on this, it is proposed that the above-mentioned matching algorithm is applied to the estimation of cell rotation velocity, which has far-reaching significance for the analysis of cell dynamics characteristics, and can provide great convenience for cell dynamics analysis.

DRAWINGS

1 is a schematic diagram of a system of a light-induced dielectrophoresis platform according to a specific embodiment of the present invention.

2 is a flow chart of a method for a block matching based motion estimation method according to an embodiment of the present invention.

3 is a schematic diagram of a step of generating a circular rotation template in a motion estimation method according to an embodiment of the present invention.

4 is a schematic diagram of a block matching step in a motion estimation method according to an embodiment of the present invention.

FIG. 5 is a flowchart of a method for generating a circular rotation template in a motion estimation method according to an embodiment of the present invention.

detailed description

The invention provides a motion estimation method based on block matching, a motion estimation system and an application thereof. The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

In a specific embodiment of the invention, a sequence of stained cell images that are rotated in a dielectrophoretic field is used as an example. It should be understood that the block matching based motion estimation method and the estimation system thereof according to the present invention are particularly suitable for image sequence analysis of rotating motions of other small objects having similar image characteristics, and may also be applied to the image sequence including the dyed cells. Image processing is performed in any other suitable image sequence or video stream.

As shown in Figure 1, a specific embodiment of a system for acquiring the sequence of cell images.

The system includes an ODEP chip 100, an optical microscope 200, a high resolution projector 300, a programmable high precision ODEP chip driver (not shown), and a computer 500. (ie a complete light-induced dielectrophoresis platform)

The computer 500 can employ any suitable electronic computing device or platform capable of satisfying the required computing power, such as a personal computer, a laptop, a cloud host, and the like.

ODEP refers to optically induced-electrophoresis (optically-induced dielectrophoresis). The ODEP chip may specifically be composed of a three-layer structure: wherein the substrate 110 is an ITO glass coated with a 1 micron thick hydrogenated amorphous silicon (a-Si:H) coating 11, and the uppermost layer is a common ITO glass 120. A 100 micron high microfluidic channel 130 is encapsulated between the substrate and the uppermost layer using PDMS or double sided tape.

The computer is connected to the projector through the data line 10, and the lens of the projector is connected to the incident optical path of the microscope through the switching device (as shown by A1 in FIG. 1). The ODEP chip is placed on a microscope stage (wherein 200 is specifically an objective lens of the microscope).

The use of the system is:

First, the cells are cultured and the cells and matrix are placed in an ODEP chip. The cells may specifically be any type of cells, depending on the needs of the actual research, such as tumor cells, immunocompetent cells (ICC), and the like.

Then, a pattern is generated on the computer and projected by the projector onto the ODEP chip on the microscope stage to form a pattern as shown in FIG. 1 (ie, the rectangular frame B1 in FIG. 1). The chip driver 400 adds alternating current to the ODEP chip, changes the frequency and size of the alternating current signal, and cooperates with the corresponding projection pattern to rotate the cells 13 in the ODEP chip.

Finally, the image sequence of cell movement was recorded by a high speed CCD of the microscope.

As shown in FIG. 2, in order to analyze the image sequence of the cell motion recorded by the high-speed CCD of the above microscope using the block matching-based motion estimation method according to the present invention, a flow chart of a method for calculating the number of rotations of the cells is shown.

The method includes the following steps:

S1. Acquire a sequence of rotational motion images of a small object. As described above, the image sequence to be analyzed is obtained here as an image sequence of cell motion recorded by a high-speed CCD of the microscope.

S2, performing translation compensation on the target small object by a block matching based custom block algorithm. Wherein, the block matching based custom algorithm uses a circular rotation template.

S3. Perform pixel block correlation analysis on the target micro object after the translation compensation to calculate the number of rotations of the target micro object.

In general, the rotation motion of the cells in the dielectrophoretic force field can be decomposed into two motions, translational and rotational. In order to accurately analyze the rotation of the cells, firstly, the translation compensation is performed by the step S2, and then the rotation analysis of the compensated image is performed by the step S3.

Preferably, the image may be pre-processed to improve the efficiency and accuracy of subsequent processing. The pre-processing method includes: performing noise reduction on a rotating motion image sequence of the minute object by using a Gaussian low-pass filter and enhancing contrast of the target minute object and the background by histogram equalization.

Regarding noise reduction preprocessing: In general, in the image sequence obtained by the above system, the main source of noise of the image frame is the camera. Because the image-to-charge conversion is done by the CCD in the camera, the CCD randomly generates some electrons that form noise in the signal. Since these noises are randomly distributed, a Gaussian low-pass filter can be used to effectively filter out noise.

The core formula of the Gaussian filter is as follows:

Regarding contrast preprocessing: The histogram equalization is one of the commonly used methods of image contrast enhancement. The specific principle is that the gray histogram of the original image is changed from a certain gray interval in the comparison set to a uniform distribution in the entire gray range. The modified pixel value conversion function of the grayscale image can be expressed by the following formula:

f(x)=X ₀ +(X _L-1 -X ₀ )cdf(x)

Among them, the grayscale image {x} contains L discrete gray levels, expressed as {Xi}. when However, it is also possible to use other suitable methods or a combination of different methods to perform contrast enhancement (eg, histogram stretching) on the image frames in the sequence of images or to improve image quality.

Specifically, the step of performing translation compensation on the target micro object by using a block matching-based custom block algorithm (ie, S2) specifically includes: generating a circular rotation template; performing block matching by using the circular rotation template, and estimating the The trajectory of the target tiny object. In the conventional block matching algorithm, a rectangular template is used. Since the cells are rotational in the sequence of images, the tracking efficiency using rectangular templates will be very low. The use of a circular rotation template for matching can very well achieve tracking of the rotation motion, greatly improving the efficiency and accuracy of the algorithm.

In a specific embodiment of the present invention, as shown in FIG. 3 and FIG. 5, the step of generating a circular rotation template may specifically include:

S100. Convert the target image (A image in FIG. 3) into a grayscale image (B image in FIG. 3).

S200. Convert the grayscale image into a binary image (C image in FIG. 3) by an adaptive threshold algorithm.

S300. Calculate a center and a radius of the circular rotating template to form the circular rotating template (D image in FIG. 3).

The circular rotating template can be rotated at a certain angle for matching (E image in FIG. 3), that is, an E image obtained by rotating the D image at a certain angle. Thus, the term "circular rotation template" is used to denote this re-customized matching template.

Specifically, the block matching step may include (as shown in FIG. 4, a matching process between the reference image frame in the image sequence and the current operation image frame):

First, the best matching block is obtained as the reference block 10 according to the preset matching criterion in the search window. Wherein, the matching criterion is specifically represented by the following formula:

Where (x, y) ∈ S, (i, j) ∈ M, S is a search window, and M is a circular rotation template (ie, an image mask). The SAD is a main operation form in the operation estimation, and the specific operation manner is well known to those skilled in the art, and will not be described herein.

Then, the motion vector 30 of the reference block 10 to the current position of the macroblock 20 is calculated (as shown in FIG. 4). "Macroblock" is a basic concept in video coding. That is, in video coding, an image frame is usually composed of several macroblocks. Wherein, the absolute difference and the smallest motion vector in the corresponding search window can be expressed by the following formula:

The minimum absolute difference and calculation process based on the reference block, which is a circular rotation template, can be represented by the following pseudo code:

1.for(x,y)in search window S

2.{for(θ=0to 2π)

3.{calculate SAD(x,y,θ)

4.incrementθby a stepΔ

5.}

6.increment x,y by 1

The above step of "blocking-based custom block algorithm" uses a circular rotation template to match to track the translation of the target cell, effectively improving the tracking accuracy of the block matching algorithm for rotating objects, while at the same time Computational efficiency, with good Application prospects.

After the translation compensation is completed, pixel correlation analysis can be used to estimate the number of cell revolutions. Specifically, the pixel correlation analysis may include the following steps:

For two grayscale image blocks, the correlation coefficient between the two is calculated by Equation 1.

A local maximum of the correlation coefficient is sought to track the peak point.

The number of rotations of the template cells is obtained based on the index of the peak points.

Wherein the formula 1 is:

Where t(·) is the selected template block (that is, the matching template used above), and f(·) is the sequential image block.

with

They are the average of t(·) and f(·), respectively.

The invention also provides a method for analyzing cell dynamics characteristics. The analysis method applies a motion estimation method as described above to perform rotational velocity analysis on cells in the dielectrophoretic force field. That is, by obtaining the number of rotations of the target cells, the rotational speed of the target is estimated for further analysis.

The present invention still further provides a motion estimation system based on block matching. As shown in FIG. 3, the system specifically includes: an image acquisition module 100, configured to acquire a sequence of rotational motion images of a small object; and a translation compensation module 200 configured to perform target microscopic objects through a block-based custom block algorithm. The translation compensation module 300 is configured to perform pixel block correlation analysis on the target micro object after the translational compensation to calculate the number of rotations of the target micro object. As described above, the image acquisition module can implement image sequence acquisition using a suitable system according to actual conditions, for example, the system shown in FIG. 1 acquires a moving image sequence of cells. The translation compensation module 200 and the correlation coefficient meter The calculation module 300 can be executed on any suitable electronic computing platform or integrated into a system as a component of the motion estimation function.

Of course, the image sequence to be analyzed can also be directly input to the translation compensation module 200 and the correlation coefficient calculation module 300 for analysis and calculation without passing through the image acquisition module 100.

It is to be understood that those skilled in the art can make equivalent substitutions or changes to the present invention and the present invention. All such changes and substitutions are intended to fall within the scope of the appended claims.

Claims

A motion estimation method based on block matching, characterized in that the method comprises the steps of: acquiring a sequence of rotational motion images of a small object;

Performing translation compensation of the target small object by a block matching based custom block algorithm;

Performing a pixel block correlation analysis on the target minute object after the translation compensation to calculate the number of rotations of the target minute object;

The block matching based custom algorithm uses a circular rotation template.
The motion estimation method according to claim 1, wherein the method further comprises: performing noise reduction on the rotational motion image sequence of the minute object by using a Gaussian low-pass filter

as well as

Enhance the contrast of the target tiny object and the background by histogram equalization.
The motion estimation method according to claim 1, wherein the step of performing translation compensation on the target small object by using a block matching-based custom block algorithm specifically includes:

Generating a circular rotation template;

Block matching is performed using the circular rotation template to estimate a motion trajectory of the target minute object.
The motion estimation method according to claim 3, wherein the step of generating a circular rotation template specifically comprises:

Converting the target image to a grayscale image;

Converting the grayscale image into a binary image by an adaptive threshold algorithm;

A center and a radius of the circular rotating template are calculated to form the circular rotating template.
The motion estimation method according to claim 3, wherein the block matching step specifically comprises:

Calculating the best matching block as a reference block according to a preset matching criterion in the search window;

A motion vector of the reference block to the current position of the macroblock is calculated.
The motion estimation method according to claim 5, wherein the matching criterion is specifically expressed by the following formula:

Where (x, y) ∈ S, (i, j) ∈ M, S is a search window, and M is a circular rotation template.
The motion estimation method according to claim 5, wherein the absolute difference and the minimum motion vector in the search window corresponding to the motion vector are specifically calculated by the following equation:
The motion estimation method according to claim 1, wherein the pixel correlation analysis specifically comprises:

Calculating the correlation coefficient between the template and the sequential image block by Equation 1;

Finding a local maximum of the correlation coefficient to track the peak point;

Obtaining a number of rotations of the minute object according to an index of the peak point;

The formula 1 is:

Where t(·) is a template and f(·) is a sequential image block.
with
They are the average of t(·) and f(·), respectively.
A method for analyzing cell dynamics characteristics, characterized in that the motion estimation method according to any one of claims 1-8 is used to perform rotational velocity analysis on cells in a dielectrophoretic force field.
A motion estimation system based on block matching, characterized in that: the system comprises: an image acquisition module, configured to acquire a sequence of rotational motion images of a small object;

a translation compensation module for performing translation compensation on a target small object by a block matching-based custom block algorithm; and a correlation coefficient calculation module for performing pixel block correlation analysis on the target small object after translation compensation, Calculating a number of revolutions of the target micro object; the block matching based custom algorithm uses a circular rotation template.