CN107292850B - A kind of light stream parallel acceleration method based on Nearest Neighbor Search - Google Patents
A kind of light stream parallel acceleration method based on Nearest Neighbor Search Download PDFInfo
- Publication number
- CN107292850B CN107292850B CN201710532775.7A CN201710532775A CN107292850B CN 107292850 B CN107292850 B CN 107292850B CN 201710532775 A CN201710532775 A CN 201710532775A CN 107292850 B CN107292850 B CN 107292850B
- Authority
- CN
- China
- Prior art keywords
- light stream
- image
- optical flow
- flow field
- tomographic image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 230000001133 acceleration Effects 0.000 title claims abstract description 10
- 230000003287 optical effect Effects 0.000 claims abstract description 31
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 10
- 238000011478 gradient descent method Methods 0.000 claims abstract description 8
- 230000006854 communication Effects 0.000 claims abstract description 5
- 239000013598 vector Substances 0.000 claims description 17
- 230000001902 propagating effect Effects 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 5
- 230000003321 amplification Effects 0.000 claims description 3
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 1
- 239000010931 gold Substances 0.000 description 1
- 229910052737 gold Inorganic materials 0.000 description 1
- 238000012067 mathematical method Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000002560 therapeutic procedure Methods 0.000 description 1
- 230000017105 transposition Effects 0.000 description 1
Classifications
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4007—Interpolation-based scaling, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
Abstract
The invention discloses a kind of light stream parallel acceleration method based on Nearest Neighbor Search, modifies to the light stream correction algorithm of Barnes k-nearest neighbor, corrects light stream using gradient descent method, improves efficiency；It modifies simultaneously to the communication process of light stream, makes it easier to realize concurrent operation on GPU to be accelerated.The present invention is used for optical flow computation, arithmetic speed can be significantly improved, while guaranteeing certain accuracy.
Description
Technical field
The invention belongs to technical field of image processing, and in particular to a kind of light stream based on the Nearest Neighbor Search side of acceleration parallel
Method.
Background technique
The concept of light stream (Optical Flow) is that Gibson is proposed first in nineteen fifty, it is by camera and scene phase
To movement generate, expression be each instantaneous relative motion of pixel in image speed conditions.Optical flow field refers to image grayscale mould
The apparent motion of formula.Optical flow computation is one of the hot spot of computer nowadays vision, and optical flow algorithm is widely used in moving target
The fields such as detection and tracking, robot navigation, three-dimensional reconstruction.The initial stage eighties, Horn and Schunck propose foundation in light stream
Dense optical flow algorithm on the basis of smoothness assumption has served founder for the development of optical flow computation, and scholars are successive later
Propose a variety of algorithms for calculating light stream.According to mathematical method and corresponding theoretical basis, the method for calculating light stream, which is divided into, to be based on
The method of gradient, the method based on Block- matching, the method based on energy and the method based on phase.
Compared with other calculate the method for light stream, the optical flow method based on Block- matching is fast with arithmetic speed, is easy to hardware reality
Existing advantage.Block- matching is a kind of problem very common in image procossing, is applied in the estimation especially in Video coding
Extensively, for many years, the algorithm of many Block- matchings, such as three-step approach, four step rule, rhombus therapy has been proposed in scholars.In order to
Arithmetic speed is further increased, scholars propose approximate block matching algorithm, such as local search, dimension etc. is reduced, although these
The result that method obtains is accurate matched approximation, but due to being widely used in various height which greatly reduce operand
In tomographic image processing.
Light stream is a kind of expression way of simple and practical image motion, is defined as Geometrical change and the radiation of dynamic image
Spend comprehensive expression of variation.The research of light stream be using the pixel intensity data in image sequence time domain variation and correlation come
It determines " movement " of respective location of pixels, i.e. object structures and its movement in the variation and scene of research image grayscale in time
Relationship.
The basic principle of optical flow method detection moving object is: assigning a speed arrow to each of image pixel
Amount, which forms an image motion fields, the point one on point and three-dimension object in a particular moment of movement, on image
One is corresponding, and this corresponding relationship can be obtained by projection relation, according to the velocity vector feature of each pixel, can to image into
Mobile state analysis.If not having moving object in image, light stream vector is consecutive variations in whole image region.Work as image
In when having moving object, there are relative motion, moving object is formed by velocity vector certainty and neighborhood for target and image background
Background velocity vector is different, to detect moving object and position.
Most of optical flow computation method is considerably complicated at present, and calculation amount is huge, and time-consuming, real-time and practicability all compared with
Difference.The present invention carries out parallel acceleration method to the calculating of light stream, can improve arithmetic speed while guaranteeing precision.
Summary of the invention
Technical solution of the invention: to overcome the shortcomings of current optical flow computation, time-consuming, provides a kind of based on most
The parallel acceleration optical flow approach of proximity search, this method can be protected by carrying out acceleration calculating using GPU parallelization calculating
Demonstrate,prove certain accuracy.
The technical solution adopted by the present invention are as follows: a kind of light stream parallel acceleration method based on Nearest Neighbor Search, including it is following
Step:
Step 1: input two field pictures are converted into grayscale image, and carry out gaussian filtering, to reduce picture noise；
Step 2: being respectively that filtered two images carry out pyramid layering, according to the sequence of resolution ratio from small to large
Arrangement, and optical flow field being uniformly distributed between (0,1) of the image of resolution ratio smallest tier is randomly initialized, and by its light stream
Field picture is as current tomographic image；
Step 3: calculating the gradient value of current tomographic image, including x direction gradient value and y direction gradient value, and to obtaining
Gradient value carries out gaussian filtering；
Step 4: utilize Barnes algorithm, to the optical flow field of current tomographic image, using it is improved propagate modified method into
Row repeatedly improved propagations amendment, when propagating every time, the communication process of every row or each column be it is parallel, adopted in propagation makeover process
It uses gradient value as matching criterior, light stream is corrected using gradient descent method, obtains revised light stream field picture；
Step 5: obtained optical flow field is amplified to the size of the adjacent tomographic image in image pyramid, amplification process
It is middle to use bicubic interpolation；
Step 6: using amplified light stream field picture as current tomographic image, step 3,4 and 5 are repeated, until optical flow field figure
As size reaches original image size.
It is in step 4, improved to propagate modified method are as follows:
Motion vector when error function minimum in order to obtain, i.e., real light stream vector, using normal in optimization algorithm
Gradient descent method.Assuming that certain point light stream vector is (u, v) after propagating, error function is E (u, v), then amendment light stream
Vector are as follows:
Wherein, △ u and △ v is respectively the differential of transverse and longitudinal coordinate, and taking 0.001, α is step-length, and the present invention uses dynamic step length
Strategy, step-length α can be set to the function of the pyramid number of plies:
Wherein l is the current pyramid number of plies, and L is the total number of plies of image pyramid.
It is in step 4, improved to propagate modified method with parallel computation are as follows:
To the optical flow field of each tomographic image, the propagation amendment of four different directions is all carried out, direction is followed successively by from left to right,
From top to bottom, from right to left, from top to bottom.When propagating amendment, the calculating of current block only relies upon an adjacent block, simultaneously
Good result is also only broadcast to an adjacent block, so that the propagation of every a line or each column becomes independent, is easy
The upper Parallel Implementation of GPU.
In conclusion by adopting the above-described technical solution, the invention has the benefit that the present invention to optical flow field into
The mode propagated is used only during row iteration is modified, it is asynchronous in different resolution Gradient descent method by being arranged in
It is long, improve the modified efficiency of light stream.Each layer of optical flow field is propagated, using four single directions, so that the process can be with
Concurrent operation is advantageously implemented the accelerator using GPU parallel computation.
Detailed description of the invention
Fig. 1 is the method for the present invention implementation flow chart；
Communication process schematic diagram in Fig. 2 present invention.
Specific embodiment
To keep the purpose of the present invention clearer with technical solution, below with reference to embodiment and schematic diagram, to the present invention
It is described in further detail.
As shown in Figure 1, the method is specifically implemented by the following steps:
Step 1: reading in two field pictures to be processed, be converted into grayscale image, and carry out gaussian filtering, made an uproar with reducing
Sound.
Step 2: being respectively that filtered two images carry out pyramid layering, the size of every tomographic image is upper one layer
0.9 times, the size depending on original image is divided into 20 to 30 layers.
Step 3: the optical flow field of random initializtion bottom image makes being uniformly distributed between its (0,1).
Step 4: calculating separately the direction x of the tomographic image and the gradient value in the direction y, and Gauss filter is carried out to calculated result
Wave.
Step 5: to the optical flow field of the tomographic image, carry out the propagation amendment of four different directions respectively, direction be respectively from
Left-to-right, from top to bottom, from right to left, from top to bottom, when propagating every time, each row or the propagation respectively arranged carry out parallel.It passes
Using gradient value as matching criterior during broadcasting, light stream is corrected using gradient descent method.As shown in Fig. 2, for example carrying out from upper
When propagation under, the propagation of each column in 3 column image blocks be all it is independent, it is unrelated with other column, thus can be concurrently
Calculate the communication process of this three column.
Step 6: using the matched criterion of gradient value are as follows:
The gray scale of two pixels in image is expressed as the functional form with time, spatial position, it may be assumed that
G1(x, y)=f (x, y, t)
G0(x+u, y+v)=f (x+u, y+v, t+ △ t)
It will be unfolded using Taylor's formula, and ignore higher-order shear deformation item and think that the time is substantially equal to 0, obtain:
It is obtained after transposition:
If the size of image block is M × N, then the gradient motion vector (u, v) of image block is defined as:
It is required that the minimum value of above formula, can be obtained with least square method about (u, v)TNonhomogeneous equation group, then
To new matching criterior:
Wherein m (i, j) represents window weight, influence of each pixel to matching result in block is illustrated, in practical meter
In calculation, generally replaced to ρ by the Gaussian function of radius, and calculate the volume between Gaussian function and image sequence gradient difference value
Product.
Step 7: correcting the gradient descent method of light stream are as follows:
For some block, matching error can regard the function of its motion vector, true light stream vector as
Exactly make motion vector when error function minimum.Assuming that after propagating certain point light stream vector be (u, v), error function be E (u,
V), then amendment light stream vector are as follows:
Wherein, it is step-length that △ u and △ v, which takes 0.001, α, and the present invention uses the strategy of dynamic step length, close to image gold word
When tower top layer, error is generally large, step-length is arranged larger；When close to pyramid bottom, step-length setting is smaller.Step-length
α is set as the function of the pyramid number of plies:
Wherein l is the current pyramid number of plies, and L is the total number of plies of image pyramid.
Step 8: obtained optical flow field is amplified to the size of a upper tomographic image, uses bicubic interpolation in amplification process,
Value of the function f at point (x, y) is obtained by the weighted average of 16 points nearest in rectangular mesh, its calculation formula is:
Step 9: repeat step 4~8, until optical flow field reach the size of original image to get arrive entire image light stream
?.Optical flow algorithm terminates.
Above embodiments are provided just for the sake of the description purpose of the present invention, and are not intended to limit the scope of the invention.This
The range of invention is defined by the following claims.It does not depart from spirit and principles of the present invention and the various equivalent replacements made and repairs
Change, should all cover within the scope of the present invention.
Claims (2)
1. a kind of light stream parallel acceleration method based on Nearest Neighbor Search, which comprises the following steps:
Step 1: input two field pictures are converted into grayscale image, and carry out gaussian filtering, to reduce picture noise；
Step 2: being respectively that filtered two images carry out pyramid layering, arranged according to the sequence of resolution ratio from small to large
Column, and optical flow field being uniformly distributed between (0,1) of the image of resolution ratio smallest tier is randomly initialized, and by its optical flow field
Image is as current tomographic image；
Step 3: calculating the gradient value of current tomographic image, including x direction gradient value and y direction gradient value, and to the gradient obtained
Value carries out gaussian filtering；
Step 4: utilizing Barnes algorithm, to the optical flow field of current tomographic image, carried out using the improved modified method of propagation more
Secondary improved propagation amendment, when propagating every time, the communication process of every row or each column be it is parallel, propagate in makeover process using ladder
Angle value corrects optical flow field as matching criterior, using gradient descent method, obtains revised light stream field picture；Improved propagation is repaired
Positive method are as follows: to the optical flow field of each tomographic image, all carry out the propagation amendment of four different directions, direction be followed successively by from a left side to
The right side, from top to bottom, from right to left, from top to bottom, when propagating amendment, the calculating of current block only relies upon an adjacent block,
Good result is also only broadcast to an adjacent block simultaneously, so that the propagation of every a line or each column becomes independent, is held
The Parallel Implementation easily on GPU；Step 5: the size that obtained optical flow field is amplified to the adjacent tomographic image in image pyramid is big
It is small, bicubic interpolation is used in amplification process；
Step 6: using amplified light stream field picture as current tomographic image, step 3,4 and 5 are repeated, until light stream field picture ruler
It is very little to reach original image size.
2. a kind of light stream parallel acceleration method based on Nearest Neighbor Search according to claim 1, it is characterised in that: in step
In rapid 4, gradient descent method corrects the process of optical flow field are as follows:
Certain point light stream vector is (u, v) after propagating, and error function is E (u, v), then corrects light stream vector are as follows:
Wherein, u, v are light stream vectors, respectively indicate transverse and longitudinal coordinate；Δ u and Δ v is respectively the differential of transverse and longitudinal coordinate, takes 0.001,
α is step-length, and using the strategy of dynamic step length, step-length α is set as the function of the pyramid number of plies:
Wherein l is the current pyramid number of plies, and L is the total number of plies of image pyramid.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710532775.7A CN107292850B (en) | 2017-07-03 | 2017-07-03 | A kind of light stream parallel acceleration method based on Nearest Neighbor Search |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710532775.7A CN107292850B (en) | 2017-07-03 | 2017-07-03 | A kind of light stream parallel acceleration method based on Nearest Neighbor Search |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107292850A CN107292850A (en) | 2017-10-24 |
| CN107292850B true CN107292850B (en) | 2019-08-02 |
Family
ID=60098281
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710532775.7A Active CN107292850B (en) | 2017-07-03 | 2017-07-03 | A kind of light stream parallel acceleration method based on Nearest Neighbor Search |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107292850B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111353948B (en) * | 2018-12-24 | 2023-06-27 | Tcl科技集团股份有限公司 | Image noise reduction method, device and equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101620730A (en) * | 2008-06-30 | 2010-01-06 | 英特尔公司 | Computing higher resolution images from multiple lower resolution images |
| CN101923718A (en) * | 2009-06-12 | 2010-12-22 | 新奥特（北京）视频技术有限公司 | Optimization method of visual target tracking method based on particle filtering and optical flow vector |
| CN103761749A (en) * | 2014-02-12 | 2014-04-30 | 北京工业大学 | Optical flow field rapid estimation method based on non-linear multigrid method |
| CN103871056A (en) * | 2014-03-11 | 2014-06-18 | 南京信息工程大学 | Anisotropic optical flow field and deskew field-based brain MR (magnetic resonance) image registration method |
| CN105261042A (en) * | 2015-10-19 | 2016-01-20 | 华为技术有限公司 | Optical flow estimation method and apparatus |
| CN106803265A (en) * | 2017-01-06 | 2017-06-06 | 重庆邮电大学 | Multi-object tracking method based on optical flow method and Kalman filtering |
| CN106803072A (en) * | 2016-12-30 | 2017-06-06 | 中国计量大学 | Variable visual angle gait recognition method based on the fusion of quiet behavioral characteristics |
-
2017
- 2017-07-03 CN CN201710532775.7A patent/CN107292850B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101620730A (en) * | 2008-06-30 | 2010-01-06 | 英特尔公司 | Computing higher resolution images from multiple lower resolution images |
| CN101923718A (en) * | 2009-06-12 | 2010-12-22 | 新奥特（北京）视频技术有限公司 | Optimization method of visual target tracking method based on particle filtering and optical flow vector |
| CN103761749A (en) * | 2014-02-12 | 2014-04-30 | 北京工业大学 | Optical flow field rapid estimation method based on non-linear multigrid method |
| CN103871056A (en) * | 2014-03-11 | 2014-06-18 | 南京信息工程大学 | Anisotropic optical flow field and deskew field-based brain MR (magnetic resonance) image registration method |
| CN105261042A (en) * | 2015-10-19 | 2016-01-20 | 华为技术有限公司 | Optical flow estimation method and apparatus |
| CN106803072A (en) * | 2016-12-30 | 2017-06-06 | 中国计量大学 | Variable visual angle gait recognition method based on the fusion of quiet behavioral characteristics |
| CN106803265A (en) * | 2017-01-06 | 2017-06-06 | 重庆邮电大学 | Multi-object tracking method based on optical flow method and Kalman filtering |
Non-Patent Citations (4)
| Title |
|---|
| Dense optical Flow Computation for 2D and 3D images via multigrid on a parallel machine;El Mostafa Kalmoun etc.;《 2007 International Symposium on Computational Intelligence and Intelligent Informatics》;20071231;全文 |
| 一种基于众核架构的稠密光流并行计算方法;喻 津 等;《计算机工程与应用》;20161231;全文 |
| 基于光照辐射模型的光流场估计;廖彬 等;《华 南 理 工 大 学 学 报 ( 自 然 科 学 版 )》;20110531;第39卷(第5期);全文 |
| 移动视点视频中的在线阴影检测与跟踪;陈祥祥 等;《现代计算机》;20170331;全文 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107292850A (en) | 2017-10-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2959456B1 (en) | Real time stereo matching | |
| CN109272537B (en) | Panoramic point cloud registration method based on structured light | |
| Henson et al. | Attitude-trajectory estimation for forward-looking multibeam sonar based on acoustic image registration | |
| CN110637461A (en) | Densified optical flow processing in computer vision systems | |
| JP5809575B2 (en) | Image processing apparatus and method, distortion correction map creating apparatus and method, and semiconductor measurement apparatus | |
| Xu et al. | Mononerd: Nerf-like representations for monocular 3d object detection | |
| Tang et al. | An improved spatiotemporal correlation method for high-accuracy random speckle 3D reconstruction | |
| CN107292850B (en) | A kind of light stream parallel acceleration method based on Nearest Neighbor Search | |
| CN110651475B (en) | Hierarchical data organization for compact optical streaming | |
| Almanza-Medina et al. | Deep learning architectures for navigation using forward looking sonar images | |
| Yan et al. | An efficient real-time accelerator for high-accuracy DNN-based optical flow estimation in FPGA | |
| CN106097277A (en) | The rope material point-tracking method that a kind of view-based access control model is measured | |
| CN109215118A (en) | A kind of increment type motion structure recovery and optimization method based on image sequence | |
| Han et al. | A PTV-based feature-point matching algorithm for binocular stereo photogrammetry | |
| CN117351333A (en) | Quick star image extraction method of star sensor | |
| CN108197613B (en) | Face detection optimization method based on deep convolution cascade network | |
| CN112101113A (en) | Lightweight unmanned aerial vehicle image small target detection method | |
| Ling et al. | Flowacc: Real-time high-accuracy dnn-based optical flow accelerator in fpga | |
| Yang et al. | A light CNN based method for hand detection and orientation estimation | |
| Yuan et al. | Mfusenet: Robust depth estimation with learned multiscopic fusion | |
| JP6315542B2 (en) | Image generating apparatus and image generating method | |
| Ren et al. | T-UNet: A novel TC-based point cloud super-resolution model for mechanical lidar | |
| Liang et al. | Real-time stereo image depth estimation network with group-wise L1 distance for edge devices towards autonomous driving | |
| CN110414337B (en) | Target attitude detection system and detection method thereof | |
| Pinchukov et al. | Comparison of the Neural Networks with Crosscorrelation Algorithm for the Displacements on Images Estimation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |