CN106815861A - A kind of optical flow computation method and apparatus of compact - Google Patents

Abstract

Big for current optical flow computation equipment size, the application and existing optical flow algorithm for being not suitable for the smaller occasion of size calculate complicated shortcoming, and the present invention proposes a kind of optical flow computation method and apparatus of compact.The optical flow computation equipment its be a kind of by imageing sensor and the integrated compact optical stream calculation equipment on a single die of central processing unit, its size for reducing optical flow computation equipment.And optical flow computation method is that characteristic point is quick on the basis of LK optical flow algorithms to calculate light stream by extracting, its more traditional dense optical flow algorithm calculates simpler, and real-time is good, can meet engineering demand.

Description

A kind of optical flow computation method and apparatus of compact

Technical field

The present invention relates to computer vision field, a kind of optical flow computation method and apparatus of compact is refered in particular to.

Background technology

Optical flow computation method and apparatus is one of computer vision field major issue to be solved.The concept of light stream is earliest Proposed by Gibson in nineteen fifty.Light stream is the instantaneous velocity of pixel motion of the space movement target on observation imaging surface.

Optical flow method can provide target motion and structural information, and computational accuracy is high, can accurately detect sub-pix and displacement, Meanwhile, the relative complex motion such as optical flow method can apply to translate, rotate, scales.Optical flow method is led in target detection, tracking etc. There is important application in domain.The visual movement measurement in robot field of optical flow computation equipment has important answering with relative motion perceptible aspect With.In recent years, one that flight stability controls also to be studied into SUAV aircraft with avoidance is carried out using optic flow technique Hot issue.Optical flow computation equipment traditional at present is as shown in figure 1, be to pass through simultaneously mouth line by imageing sensor and central processing unit It is formed by connecting, its volume is larger, is not suitable for the minute vehicle for having strict demand to size and weight, and it is existing most Number optical flow algorithm is computationally intensive, calculates time-consuming, hinders the application of light stream.

Existing optical flow computation method has a lot, can be largely classified into four classes, be respectively based on differential method, based on area The matching method in domain, the method based on energy and the method based on phase.1981, Horn and Schunck were according to gray consistency It is assumed that deriving optical flow constraint equation, optical flow algorithm is allowed to be developed.1981, Horn and Schunck was in optical flow constraint equation On the basis of addition of optical flow field the overall situation smoothness constraint, be deduced HS optical flow algorithms.Same year, Lucas and Kanade two Scholar proposes a kind of image matching method of iteration, and the algorithm can be used to estimate light stream, be referred to as LK algorithms.H-S algorithms It is two most classical computational methods with L-K algorithms, is all based on the optical flow computation method of gradient.Barron is in 1994 from being based on 9 are have chosen in the method for differential, the matching method based on region, the method based on energy and the method based on phase these four methods In the method commonly used, done quantitative comparison from precision, reliability etc. in terms of, discovery LK optical flow algorithm good reliabilitys.Lk algorithms are only Need the local message of the wicket around point-of-interest just to can obtain sparse optical flow, the time overhead of optical flow computation process compared with It is low, it is adapted to calculate in real time.

The content of the invention

It is big for current optical flow computation equipment size, be not suitable for the application of the smaller occasion of size and existing optical flow algorithm meter Complicated shortcoming is calculated, the present invention proposes a kind of optical flow computation method and apparatus of compact.A kind of compact optical stream calculation sets Standby its be it is a kind of by imageing sensor with the integrated compact optical stream calculation equipment on a single die of central processing unit, it reduces The size of optical flow computation equipment.The invention allows for a kind of new optical flow computation method, by extracting characteristic point in LK light streams It is quick on the basis of algorithm to calculate light stream.The present invention by first carrying out feature extraction then right on the basis of LK light streams to image The characteristic point for extracting carries out LK and calculates light stream, further increases the computational efficiency of light stream, and improve the accurate of light stream Degree, wherein before feature extraction, the enhancing of smooth and texture is first carried out to original image, improves the accuracy rate for extracting feature.

A kind of optical flow computation equipment of compact, including camera lens, cmos image sensor and central processing unit, and by CMOS Imageing sensor, central processing unit are integrated on a chip, cmos image sensor collection image information, and by image information Being transferred directly to central processing unit carries out optical flow computation, and optical flow data is exported finally by serial ports.Cmos image sensor due to Integrated level is high, small volume, light weight the characteristics of, be suitable in the optical flow computation equipment of small size, cmos pixel array is one Individual two-dimentional addressable sensor array, each row of sensor are connected with a bit line, and row is allowed in the line selected row of permission Each sensing unit output signal is sent on the bit line corresponding to it, and bit line end is MUX, independent according to each row Row addressing selected.Cmos image sensing instead of the ccd image sensor in traditional camera, be more suitable for small size Occasion.

A kind of optical flow computation method, comprises the following steps：

(1) using gauss low frequency filter to initial pictures I₀Smothing filtering is carried out, the image I of the noise that is eliminated₁。

(2) with Laplace filter to image I₁Processed, obtained image I₂。

(3) the image I that will be obtained after smothing filtering₁Subtract the image I after Laplce's filtering process₂, obtain texture enhancing Image I₃。

(4) by image I₃The block of 8 × 8 pixels is divided into, the entropy of each block is calculated, entropy ranks the block q of the first half_n (n=1,2,3 ..., n) as feature point extraction block.

(5) to feature point extraction block q_n(n=1,2,3 ..., n) are calculated with Fast Corner Detection Algorithms, are filtered out The pixel point set P that Fast angle points are constituted_n1。

(6) to pixel point set P_n1Postsearch screening is carried out with Shi-Tomasi, the pixel of Shi-Tomasi angle points composition is obtained Point set P_n2, and by point set P_n2As block q_n(n=1,2,3 ..., n) final characteristic point.

(7) with L-K optical flow algorithms to feature point set P_n2(n=1,2,3 ..., n) calculate light stream, and export.

Optical flow computation equipment proposed by the present invention is more compact, is connected by parallel port with a traditional image sensor chip Optical flow computation equipment onto a microprocessor for operation optical flow algorithm, dimensionally small, application scenario is more flexible；

The more traditional dense optical flow algorithm of other optical flow algorithm proposed by the present invention calculates simpler, and real-time is good, can be full Sufficient engineering demand.

Brief description of the drawings

Fig. 1 is traditional optical flow computation equipment schematic diagram；

Fig. 2 is optical flow computation equipment schematic diagram of the invention；

Fig. 3 is the flow chart of optical flow computation method proposed by the present invention；

In above-mentioned accompanying drawing：

1- cameras, 2- microprocessors, 3- outputs, 4- camera lenses, 5-CMOS imageing sensors, 6- central processing units, 7- is defeated Go out optical flow data.

Specific embodiment

The invention will be further described with reference to the accompanying drawings and detailed description.

Traditional optical flow computation equipment as shown in figure 1, traditional light flow device is to obtain view data by camera 1, and By parallel port, camera 1 is connected with the microprocessor 2 of operation optical flow algorithm, view data is sent to micro- place by parallel port Reason device 2 is processed, and calculates light stream and output 3.Because this optical flow computation equipment image collection module and optical flow computation mould Block is to separate, therefore size is larger.

As shown in Fig. 2 a kind of optical flow computation equipment of compact proposed by the present invention, including camera lens 4, cmos image sensing Device 5 and central processing unit 6, and cmos image sensor 5, central processing unit 6 are integrated on a chip, cmos image sensing Being directly transmitted to central processing unit 6 after the view data that the reception camera lens 4 of device 5 is gathered and transmitted carries out optical flow computation, and passes through Serial ports exports optical flow data 7.

Traditional camera is to receive image information by ccd image sensor, and the charge information of CCD storages need to be Synchronizing signal control next bit reads after one implementing transfer, and charge information transfer and reading output need clock control electricity The road power supply different with three groups is engaged, and whole circuit is complex, and speed is slower, is not suitable for size small and real-time Optical flow computation equipment.Cmos image sensor is because integrated level is high, small volume, light weight the characteristics of, be suitable for small size In optical flow computation equipment, cmos pixel array is a two-dimentional addressable sensor array, each row of sensor and a position Line is connected, and row allows each sensing unit output signal in the line selected row of permission to send on the bit line corresponding to it, position Line end is MUX, is selected according to the independent row addressing of each row.Cmos image sensing instead of traditional camera In ccd image sensor, be more suitable for small size occasion.

Optical flow computation method is operated on central processing unit, reference picture 3, for a kind of optical flow computation method that the present invention is provided Flow chart, from camera lens obtain initial picture needed by following steps to optical flow data is obtained：

(1) first it is image pretreatment operation, smothing filtering is carried out to initial pictures first with gauss low frequency filter, obtains To image f (x, y) for eliminating noise.

Gaussian filter is the linear smoothing filter that a class selects weights according to the shape of Gaussian function.Gaussian smoothing Wave filter is highly effective for suppressing the noise of Normal Distribution.One-dimensional zero-mean gaussian function is：G (x)=exp (- x²/ (2δ²)), wherein δ determines the width of Gaussian function.Gaussian filter is highly desirable low pass filter.It is low by Gauss The image of bandpass filter has preferably filtered noise, but also result in the fuzzy of image texture simultaneously, so to Gauss low pass Filtered image needs further to carry out texture enhancing using Laplace filter.

(2) with Laplace filter to image I₁Processed, obtained image I₂。

Image after gauss low frequency filter smothing filtering is f (x, y), and the Laplace operator of f (x, y) is defined as：

The discrete form that its second order is led is：

Can be obtained more than：▽²F (x, y)=[f (and x+1, y)+f (x-1, y)+f (x, y+1)+f (x, y-1)] -4f (x, y).

(3) the image I that will be obtained after smothing filtering₁Subtract the image I after Laplce's filtering process₂, obtain texture enhancing Image I₃。

From step (2) as can be seen that Laplace filter is that image border is extracted, the pixel beyond edge Pixel value can be changed into 0, this be not the present invention need effect, the present invention it is desirable that can strengthen texture while can The other image of image gray levels is kept, therefore the present invention is poor by the image after being processed with Laplce with f (x, y), obtains Required texture enhancing image, makes the enhanced image of texture for g (x, y), then have：

G (x, y)=f (x, y)-▽²f(x,y)。

(4) the enhanced image g (x, y) of texture is divided into the block of 8 × 8 pixels, the entropy of each block, entropy is calculated Rank the block q of the first half_n(n=1,2,3 ..., n) as feature point extraction block.

In conventional determination image characteristics extraction block method, extracting characteristic point is extracted by image overall scope , so extracting characteristic point needs to calculate each pixel, and operand is larger.And it is a discovery of the invention that spy in image Levy and a little mainly concentrate on the region of texture-rich, can be by first finding out the region of texture-rich, then for texture-rich Region carry out feature extraction, reduce amount of calculation.The present invention uses the entropy of image as texture measure.The entropy of image is a kind of right The statistics of feature, can reflect average information in image number, an entropy of image represents the poly- of intensity profile in image Collect the information content that feature is included, expression formula is：Wherein z_iRepresent a random change of brightness Amount, p (z) represents the histogram of the gray level in a region, and L represents the grey exponent number in the region.

Determine that the specific method that feature extraction block is operated is：(1) the enhanced image g (x, y) of texture is divided into 8 × 8 The block of pixel；(2) entropy of each block is calculated；(3) entropy is ranked up；(4) the block conduct that entropy ranks the first half is taken Feature point extraction block.

(5) to feature point extraction block q_n(n=1,2,3 ..., n) are calculated with Fast Corner Detection Algorithms, are filtered out The pixel point set P that Fast angle points are constituted_n1。

The basic thought of Fast angle point algorithms is：Selection feature point extraction block q_nIn a pixel p, if its gray value It is I (p), with p as the center of circle, makees to justify by 3 pixels of radius, it is considered to 16 pixels on circumference, gives a threshold value t, when There are 12 continuous pixel grey scale pixel values on 16 pixels more than I (p)+t or less than I (p)-t, it is determined that the point is one Individual Fast angle points.

(6) to pixel point set P_n1Postsearch screening is carried out with Shi-Tomasi angle points algorithm, Shi-Tomasi angle point structures are obtained Into pixel point set P_n2, and by point set P_n2As block q_n(n=1,2,3 ..., n) final characteristic point.

The basic thought of Shi-Tomasi angle point algorithms is：If gray scale of the image at point p (x, y) place is I (x, y), with p points Centered on set up window Ω, w (x, y) of nn for window function, window is translated into [Δ x, Δ y], then grey scale change E [Δ x, Δ y] is：

For small translation, by I, (x+ Δs x, y+ Δ y) carries out Taylor expansion and ignores second order and the above, brings into Shi Ke get：

Wherein, I_x、I_yRepresent that gradation of image, in the partial derivative in x and y directions, the form that above formula is write as matrix is had respectively：

Wherein M is 2 × 2 matrix：

For pixel p (x, y), if in two characteristic values of M less one be more than given threshold value, i.e. λ₁≥λ₂And λ₂ ≥k·λ_2max(λ_2maxIt is the maximum in all pixels smaller characteristic value of point).So point p (x, y) is Shi-Tomasi angle points.

(7) with L-K optical flow algorithms to feature point set P_n2(n=1,2,3 ..., n) calculate light stream, and export.

The basic thought of L-K optical flow algorithms assumes that in a small neighbourhood light stream vector keeps constant, with a most young waiter in a wineshop or an inn Multiplication obtains light stream.It is 5 × 5 that neighborhood window is taken in L-K algorithms, according to optical flow constraint equation I_x·u+I_y·v+I_t=0, wherein I_x、I_y、I_tGradation of image space-to-space and the partial derivative of time are represented respectively, and u, v represent horizontal and vertical point of light stream respectively Amount, can set up 25 equations：

Minimum is solved by formula (6) | | Ad-b | |², i.e.,

(A^TA) d=A^Tb (6)

Linear algebra knowledge understands, as (AA^T) can the inverse time, solution of equations is as follows：

As (A^TA when) full rank, i.e. order are 2, (A^TA) there are two larger characteristic vectors, (A^TA it is) reversible.When the line in image Reason can just meet the condition when comprising at least two gradient directions, and light stream is calculated by asking for non trivial solution.

Claims (7)

1. the optical flow computation equipment of a kind of compact, it is characterised in that：Including camera lens, cmos image sensor and center treatment Device, and cmos image sensor, central processing unit are integrated on a chip, cmos image sensor collection image information, And image information is transferred directly to central processing unit carries out optical flow computation, export optical flow data finally by serial ports.

2. a kind of optical flow computation method, it is characterised in that comprise the following steps：

(1) using gauss low frequency filter to initial pictures I₀Smothing filtering is carried out, the image I of the noise that is eliminated₁；

(2) with Laplace filter to image I₁Processed, obtained image I₂；

(3) the image I that will be obtained after smothing filtering₁Subtract the image I after Laplce's filtering process₂, obtain texture enhancing image I₃；

(4) by image I₃The block of 8 × 8 pixels is divided into, the entropy of each block is calculated, entropy ranks the block q in the first half_nMake It is characterized an extraction block, wherein n=1,2,3 ..., n；

(5) to feature point extraction block q_nCalculated with Fast Corner Detection Algorithms, filtered out the pixel point set of Fast angle points composition P_n1；

(6) to pixel point set P_n1Postsearch screening is carried out with Shi-Tomasi, the pixel point set of Shi-Tomasi angle points composition is obtained P_n2, and by point set P_n2As block q_nFinal characteristic point；

(7) with L-K optical flow algorithms to feature point set P_n2Light stream is calculated, and is exported.

3. optical flow computation method according to claim 2, it is characterised in that：In step (2), if through step (1) Gauss Image I after low pass filter smothing filtering₁It is f (x, y) that the Laplace operator of f (x, y) is defined as：

${\▿}^{2}f(x,y)=\frac{{\∂}^{2}f(x,y)}{\∂x^2}+\frac{{\∂}^{2}f(x,y)}{\∂y^2}$

The discrete form that its second order is led is：

$\frac{{\∂}^{2}f(x,y)}{\∂x^2}=f(x+1,y)+f(x-1,y)-2f(x,y)$

$\frac{{\∂}^{2}f(x,y)}{\∂y^2}=f(x,y+1)+f(x,y-1)-2f(x,y)$

Image I after being processed through Laplace filter more than₂For：

${\▿}^{2}f(x,y)=\[f(x+1,y)+f(x-1,y)+f(x,y+1)+f(x,y-1)\]-4f(x,y).$

4. optical flow computation method according to claim 3, it is characterised in that：In step (3), texture is made to strengthen image I₃It is g (x, y), then have：

$g(x,y)=f(x,y)-{\▿}^{2}f(x,y).$

5. optical flow computation method according to claim 4, it is characterised in that：In step (5), Fast Corner Detection Algorithms For：Selection feature point extraction block q_nIn a pixel p, if its gray value be I (p), be 3 pictures with radius with p as the center of circle Element is made to justify, it is considered to 16 pixels on circumference, a threshold value t is given, when there is 12 continuous pixels on 16 pixels Grey scale pixel value is more than I (p)+t or less than I (p)-t, it is determined that the point is a Fast angle point.

6. optical flow computation method according to claim 4, it is characterised in that：In step (6), Shi-Tomasi angle point algorithms For：If image pixel p (x, y) place gray scale be I (x, y), set up centered on p points window a Ω, w of nn (x, Y) it is window function, window is translated into [Δ x, Δ y], then grey scale change E [Δ x, Δ y] is：

$E\[\mathrm{\Δ}x,\mathrm{\Δ}y\]=\underset{(x,y)\∈\mathrm{\Ω}}{\mathrm{\Σ}}w(x,y){\[I(x+\mathrm{\Δ}x,y+\mathrm{\Δ}y)-I(x,y)\]}^2$

For small translation, by I, (x+ Δs x, y+ Δ y) carries out Taylor expansion and ignores second order and the above, and bringing above formula into can ：

$E\[\mathrm{\Δ}x,\mathrm{\Δ}y\]={\mathrm{\Δx}}^2\underset{(x,y)\∈\mathrm{\Ω}}{\Σ}w(x,y)I_x^2+2\mathrm{\Δ}x\mathrm{\Δ}y\underset{(x,y)\∈\mathrm{\Ω}}{\Σ}w(x,y)I_x{I}_y+{\mathrm{\Δy}}^2\underset{(x,y)\∈\mathrm{\Ω}}{\Σ}w(x,y)I_y^2$

Wherein, I_x、I_yRepresent that gradation of image, in the partial derivative in x and y directions, the form that above formula is write as matrix is had respectively：

$E(\mathrm{\Δ}x,\mathrm{\Δ}y)=\[\mathrm{\Δ}x,\mathrm{\Δ}y\]M\left[\begin{array}{c}\mathrm{\Δ}x\\ \mathrm{\Δ}y\end{array}\right]$

Wherein M is 2 × 2 matrix：

$M=\left[\begin{array}{cc}\underset{\left(x,y\right)\∈\mathrm{\Ω}}{\mathrm{\Σ}}w\left(x,y\right)I_x^2& \underset{\left(x,y\right)\∈\mathrm{\Ω}}{\mathrm{\Σ}}w\left(x,y\right)I_x{I}_y\\ \underset{\left(x,y\right)\∈\mathrm{\Ω}}{\mathrm{\Σ}}w\left(x,y\right)I_x{I}_y& \underset{\left(x,y\right)\∈\mathrm{\Ω}}{\mathrm{\Σ}}w\left(x,y\right)I_y^2\end{array}\right]$

For pixel p (x, y), if in two characteristic values of M less one be more than given threshold value, i.e. λ₁≥λ₂And λ₂≥k· λ_2max, wherein λ_2maxIt is the maximum in all pixels smaller characteristic value of point；So point p (x, y) is Shi-Tomasi angle points.

7. the optical flow computation method according to claim 5 or 6, it is characterised in that：In step (7), a small neighbourhood is located at Interior, light stream vector keeps constant, and light stream is obtained with least square method, and method is：It is 5 × 5 that neighborhood window is taken in L-K algorithms, According to optical flow constraint equation I_x·u+I_y·v+I_t=0, wherein I_x、I_y、I_tGradation of image space-to-space and time are represented respectively Partial derivative, u, v represent the horizontal and vertical component of light stream, can set up 25 equations respectively：

Minimum is solved by formula (6) | | Ad-b | |², i.e.,

(A^TA) d=A^Tb (6)

Linear algebra knowledge understands, as (AA^T) can the inverse time, solution of equations is as follows：

$\left[\begin{array}{c}u\\ v\end{array}\right]=\left(A^{T}A\right)A^{T}b---\left(7\right)$

As (A^TA when) full rank, i.e. order are 2, (A^TA) there are two larger characteristic vectors, (A^TA it is) reversible；When the texture in image extremely Can just meet the condition when including two gradient directions less, light stream is calculated by asking for non trivial solution.