CN104240217B - Binocular camera image depth information acquisition methods and device - Google Patents
Binocular camera image depth information acquisition methods and device Download PDFInfo
- Publication number
- CN104240217B CN104240217B CN201310229171.7A CN201310229171A CN104240217B CN 104240217 B CN104240217 B CN 104240217B CN 201310229171 A CN201310229171 A CN 201310229171A CN 104240217 B CN104240217 B CN 104240217B
- Authority
- CN
- China
- Prior art keywords
- matching
- row
- binocular camera
- image
- cached
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Image Processing (AREA)
Abstract
The invention provides a kind of binocular camera image depth information acquisition methods and device.Methods described includes step：1）Image rectification is carried out to two width destination image datas of input according to the binocular camera calibration information being obtained ahead of time, so that pixel corresponding on target image is located in same horizontal line；2）Pixel matching energy is calculated according to the information of image rectification, and matching is constructed to every side right weight in directed graph and setting connection figure according to pixel neighbouring relations；3）The max-flow of connection figure is solved using graph theory maximum-flow algorithm using row matching way, matching result is converted to and is cached；4）Post processing is carried out to the matching result cached and obtains post processing result；5）Target image pixel depth is calculated according to binocular camera calibration information and post processing result, depth information is obtained.Meeting reasonable hardware spending, low-power consumption and performance requirement simultaneously, carrying out binocular camera depth recovery and obtain depth information.
Description
Technical field
It is particularly a kind of to recover depth letter using binocular camera the present invention relates to information gathering and processing technology field
Breath, meets the binocular camera image depth information acquisition methods and device of reasonable hardware spending, low-power consumption and performance requirement.
Background technology
It is that a pair of cameras for being similar to eyes using relative position are caught that simulation mankind binocular vision, which perceives scenery depth,
The image caught is to recovering the depth information of scenery.It is an important ring for computer vision and artificial intelligence, be computer/
Robot understands real world using visible optical information automatic sensing, and shows certain intelligent behavior based on this and established base
Plinth.In recent years, increasing researcher is directed to technique to be applied to natural human-machine interaction, computer intelligence and imitative
It is raw, automatically control, multiple application fields such as unmanned image information processing instructed.
Although considerable research has been carried out to binocular camera depth recovery, and some algorithms reached it is at a relatively high
Precision, but its distance applications also very long distance.This is primarily due to the amount of calculation for the better algorithm being currently known all
It is very huge, even if so that most strong CPU can not be handled in real time（>=30 frames/second）Low resolution input picture.And it is sharp
Huge power consumption is needed with CPU/GPU computation capability, and its performance is also constrained by memory read-write bandwidth.This with
The main trend of current electronic product Portable energy-saving runs in the opposite direction.On the other hand, in terms of human intelligence angle, depth perception still belongs to
In rudimentary intelligence.Bionics teaches that this kind of work should be completed by the specialized hardware similar to visual centre, and CPU/GPU
Operational capability should be primarily used to it is senior control and reasoning from logic in terms of, this is the mode of most efficient energy-saving.
The core of binocular camera depth recovery is the parallax for calculating left and right camera, can mathematically be classified as a class
Function minimization problem, but it, which is solved, but has NP difficulty.In addition to disparity computation, binocular camera depth recovery is carried out
Also need to be corrected input picture, calculate matching energy and post-processed.At present due to the meter of acquisition depth information
Calculation amount is larger, realizes that difficulty is larger, cost is very high to the hardware of this complex process, it is impossible to meet reasonable hardware spending, low
Power consumption and performance requirement.
The content of the invention
The purpose of the present invention is that a kind of binocular camera image depth information acquisition methods are provided to solve the above problems
And device, while reasonable hardware spending, low-power consumption and performance requirement is met, carry out binocular camera depth recovery acquisition
Image depth information.
To achieve these goals, the invention provides binocular camera image depth information acquisition methods, including it is following
Step：（1）Image calibration is carried out to two width destination image datas of input according to the binocular camera calibration information being obtained ahead of time
Just, so that pixel corresponding on all target images is located in same horizontal line；（2）Corrected according to described image
Information calculates pixel matching energy, and constructs matching to every in directed graph and setting connection figure according to pixel neighbouring relations
Side right weight；（3）The max-flow of constructed connection figure is solved using graph theory maximum-flow algorithm using row matching way, is converted to
With result and cached；（4）The matching result cached is post-processed, post processing result is obtained；（5）According to described double
Mesh camera calibration information and post processing result calculate all target image pixel depths, obtain all target figures
As depth information.
Further, step（1）Further comprise before：The inside and outside parameter of each group of binocular camera is precalculated, and
The form of target image of binocular camera output is set as RGB or YUV.
Further, step（3）Described in row matching way further comprise to each target image multirow pixel
Every trade matching is clicked through, every a line matching result is cached respectively.
Further, single file matching comprises the following steps in the row matching way：（301）Initialization matching is to directed connection
Figure；（302）Using graph theory maximum-flow algorithm to bonding strength of the image after correction to real-time calculate node, and it is strong according to connection
Spend parallel-expansion source node and sink nodes collection；（303）Search Growth Route is concentrated in source node and sink nodes, residual graph is calculated；
（304）Repeat step 301-303, untill without Growth Route, obtains an Optimum Matching, and matching result is cached.
Further, the row matching way further comprises taking parallel node to expand single file, comprises the following steps：
（311）Initialization matching is to directed graph and caches；（312）The initialization data that obtaining step 311 is cached, is used
Graph theory maximum-flow algorithm to the image after correction to calculating the bonding strength of each node in multiple nodes in real time, and according to corresponding
The source node and sink nodes collection of bonding strength parallel-expansion respective nodes are simultaneously cached；（313）What obtaining step 312 was cached
Source node and sink nodes collection data, concentrate search Growth Route in source node and sink nodes, calculate residual graph；（314）Repeat to walk
Rapid 311-313, untill without Growth Route, obtains an Optimum Matching, and matching result is cached.
To achieve these goals, present invention also offers a kind of binocular camera image depth information acquisition device, bag
Include：Image correction module, for according to two width destination image datas of the binocular camera calibration information being obtained ahead of time to input
Image rectification is carried out, so that pixel corresponding in all described images is located in same horizontal line, and image rectification is believed
Breath is exported to delaying one-row module；Connection figure constructing module, for calculating pixel matching energy, and constructs matching to directed connection
Weigh and export to the row cache module per side right in figure and setting connection figure；Row matching module, for using row matching way
The max-flow of constructed connection figure is solved using graph theory maximum-flow algorithm, matching result is converted to and caches to row caching
Module；Post-processing module, the matching result for reading the row cache module is post-processed, and obtains post processing result；It is deep
Data obtaining module is spent, based on the post processing result according to the binocular camera calibration information and the post-processing module
All target image pixel depths are calculated, all target image depth informations are obtained.
Further, described device further comprises calibration information acquisition module, for precalculating each group of binocular camera shooting
The inside and outside parameter of head, and the form for the target image that binocular camera is exported is set as RGB or YUV.
Further, the row matching module further comprises multiple row matched sub-blocks, each row matched sub-block
Interacted with the row cache module, click through every trade matching for the one-row pixels to each target image and will go
Matching result is cached to the row cache module.
Further, the row matched sub-block includes：Initialization unit, for initializing matching to directed graph；Section
Node expansion unit, for using graph theory maximum-flow algorithm to bonding strength of the image after correction to real-time calculate node, and root
According to bonding strength parallel-expansion source node and sink nodes collection；Processing unit, for concentrating search to increase in source node and sink nodes
Path, calculates residual graph；Matching result acquiring unit, for repeat call the initialization unit, dilated node unit and
Processing unit, untill without Growth Route, obtains an Optimum Matching, and matching result is cached.
Further, the row matched sub-block includes：Initialization unit, for initializing matching to directed graph and delaying
Deposit to the row cache module；Multiple dilated node units, each dilated node unit is used to obtain the row caching mould
The initialization data that block is cached, the connection using graph theory maximum-flow algorithm to the image after correction to calculating a node in real time is strong
Degree, and source node and sink nodes collection according to corresponding bonding strength parallel-expansion respective nodes and cache to the row and cache mould
Block；Processing unit, for obtaining source node and the sink nodes collection data that the row cache module is cached, in source node and the section that converges
Point concentrates search Growth Route, calculates residual graph；Matching result acquiring unit, for repeating to call the initialization unit, every
Dilated node unit and processing unit described in one, untill without Growth Route, obtain an Optimum Matching, and by matching result
Cache to the row cache module.
It is an advantage of the current invention that using energy function feature dynamic construction figure, reduce in piece the size of internal memory and keep away
Exempt from Installed System Memory read-write, realize based on capable energy match computing and post processing operations.By caching all intermediate data, keep away
Installed System Memory read-write expense is exempted from；Algorithm is improved, it is carried out parallel node expansion, and parallel-by-bit matching；
The flexibility ratio of system is effectively improved, reasonable hardware spending, low-power consumption and performance requirement is met.
Brief description of the drawings
Fig. 1, the flow chart of binocular camera image depth information acquisition methods of the present invention；
Fig. 2, binocular camera image depth information acquisition device structural representation of the present invention；
Fig. 3, the embodiment configuration diagram of single file processing scheme of the present invention；
Fig. 4, parallel node of the present invention expands the configuration diagram of embodiment；
Fig. 5, parallel-by-bit of the present invention matches the configuration diagram of embodiment.
Embodiment
The binocular camera image depth information acquisition methods and device provided below in conjunction with the accompanying drawings the present invention are done in detail
Explanation.
The embodiment of binocular camera image depth information acquisition methods of the present invention is provided first.
The core of binocular camera depth recovery is the parallax for calculating left and right camera, can mathematically be classified as a class
Function minimization problem, the best way is to utilize the maximum-flow algorithm in graph theory in theory at present, namely minimal cut set algorithm,
Such issues that to solve, the solution so drawn and real optimal solution only differ an invariant.Many experiments show, this side
Method and such as adaptive support, other methods such as confidence spread are compared, its precision highest, and amount of calculation is relatively small, and energy
Used cooperatively with a variety of methods.
It is the flow chart of binocular camera image depth information acquisition methods described in present embodiment shown in accompanying drawing 1,
Next the step shown in accompanying drawing 1 is elaborated.
S110：Image is carried out to two width destination image datas of input according to the binocular camera calibration information being obtained ahead of time
Correction, so that pixel corresponding on all target images is located in same horizontal line.
Further comprise S100 before step S110：Precalculate the inside and outside parameter of each group of binocular camera, Yi Jishe
The form for determining the target image of binocular camera output is RGB or YUV.Then, to each line of input of two width target images, profit
Corrected with the inside and outside parameter of binocular camera by projective transformation.Corresponding pixel position on two width target images after correction
In in same horizontal line so that, can be with multirow Parallel implementation during the follow-up solution using graph theory maximum-flow algorithm.
S120：The information corrected according to described image calculates pixel matching energy, and is constructed according to pixel neighbouring relations
Matching is to every side right weight in directed graph and setting connection figure.Pixel matching energy can be calculated using any measuring similarity
Amount.Utilize energy function feature dynamic construction figure, it is possible to reduce in piece the size of internal memory and avoid Installed System Memory read and write expense.
S130：The max-flow of constructed connection figure, conversion are solved using graph theory maximum-flow algorithm using row matching way
For matching result and cached.Classical graph theory maximum-flow algorithm, such as Ford-Fulkerson algorithms, Edmonds-Karp is calculated
Method and its mutation all can be used to solve, and can select corresponding algorithm according to different application fields.Solve constructed connection figure
Max-flow, thus obtained max-flow correspond to target image a minimal cut set, the cut set may be interpreted as one it is optimal
Matching.
Corresponding pixel is located in same horizontal line on two width target images after due to being corrected by step S110,
Therefore the row matching way further comprises entering the multirow pixel of each target image every trade matching, to every a line
Cached respectively with result.Namely using parallel-by-bit processing can be carried out the characteristics of the connection figure constructed, can effectively improve
Speed.
To each line of input, single file matching is completed using following steps：1）Initialization matching is to directed graph；2）Using
Graph theory maximum-flow algorithm is to bonding strength of the image after correction to real-time calculate node, and according to bonding strength parallel-expansion source
Node and sink nodes collection；3）Search Growth Route is concentrated in source node and sink nodes, residual graph is calculated；4）Repeat step 1-3, directly
Untill without Growth Route, thus obtained max-flow corresponds to a minimal cut set of figure, and the cut set may be interpreted as one most
Excellent matching, so as to obtain matching result and be cached to matching result.
In single file processing, simultaneously multiple nodes can be expanded to improve dilated node speed.Now, initialization
Pairing directed graph is simultaneously cached, rather than directly carries out graph theory maximum-flow algorithm.Specifically include following steps：11）Just
Beginningization matching is to directed graph and caches；12）The initialization data that obtaining step 11 is cached, using graph theory max-flow
Algorithm to the image after correction to calculating the bonding strength of each node in multiple nodes in real time, and according to corresponding bonding strength simultaneously
The source node and sink nodes collection of row extension respective nodes are simultaneously cached；13）Source node and the section that converges that obtaining step 12 is cached
Point set data, concentrate search Growth Route in source node and sink nodes, calculate residual graph；14）Repeat step 11-13, Zhi Daowu
Untill Growth Route, thus obtained max-flow corresponds to a minimal cut set of figure, and the cut set may be interpreted as one optimal
Match somebody with somebody, so as to obtain matching result and matching result is cached.
S140：The matching result cached is post-processed, post processing result is obtained.Can be using any based on window
Image processing method carries out window post processing to eliminate pseudo- matching.Post processing comprises the following steps：A. determine block pixel and
Unique parallax；B. interpolation calculation subpixel accuracy parallax（Optional step）；C. medium filtering；D. Speckle Filter（Optional step
Suddenly）.
S150：Target image pixel depth is calculated according to the binocular camera calibration information and post processing result, obtained
Take target image depth information.Obtain target image depth information and be specifically divided into two steps：A. binocular camera inside and outside parameter is utilized
And gained parallax, calculated pixel-by-pixel or sub-pixel depth with Triangulation Algorithm；B. depth map is smoothed, reduced
The influence that trigonometric ratio error is brought.
By caching all intermediate data, it is to avoid Installed System Memory read-write expense；Algorithm is improved, enters it
Row parallel node is expanded.The processing method of parallel-by-bit matching can improve the flexibility ratio of system, with hardware spending and performance it
Between find optimal balance point, to meet reasonable hardware spending, low-power consumption and performance requirement.
Next the specific implementation of binocular camera image depth information acquisition device of the present invention is provided with reference to accompanying drawing
Mode.
It is the structural representation of binocular camera image depth information acquisition device described in present embodiment shown in accompanying drawing 2
Figure, the binocular camera image depth information acquisition device includes calibration information acquisition module 20, image correction module 21, connected
Map interlinking constructing module 22, row matching module 23, post-processing module 24, Depth Information Acquistion module 25 and row cache module 26.
The calibration information acquisition module 20, the inside and outside parameter for precalculating each group of binocular camera, Yi Jishe
The form for determining the target image of binocular camera output is RGB or YUV.
Described image correction module 21 is connected with the calibration information acquisition module 20, for being obtained according to the calibration information
Modulus block 20 precalculates obtained binocular camera calibration information and carries out image rectification to two width destination image datas of input,
So that pixel corresponding on all target images is located in same horizontal line, and image correction information is exported to one
Row cache module 26.To each line of input of each target image, projective transformation is passed through using the inside and outside parameter of binocular camera
Correction.Corresponding pixel is located in same horizontal line on all target images after correction so that follow-up using figure
, can be with multirow Parallel implementation when being solved by maximum-flow algorithm.
The connection figure constructing module 22 is connected with described image correction module 21, for according to described image correction module
Target image control information after 21 corrections calculates pixel matching energy, and according to the construction matching of pixel neighbouring relations to oriented
Weigh and export to the row cache module per side right in connection figure and setting connection figure.It can be calculated using any measuring similarity
Pixel matching energy.Utilize energy function feature dynamic construction figure, it is possible to reduce the size of internal memory and avoided in piece in system
Deposit read-write expense.
The row matching module 23 is connected with the connection figure constructing module 22, for utilizing graph theory using row matching way
Maximum-flow algorithm solves the max-flow for the connection figure that the connection figure constructing module 22 is constructed, and is converted to matching result and caches
To the row cache module 26.
Corresponding pixel position on all target images after due to being corrected by described image correction module 21
In in same horizontal line, thus constructed using the connection figure constructing module 22 connection figure the characteristics of can carry out parallel-by-bit
Processing, can effectively improve speed.As preferred embodiment, the row matching module 23 further comprises multiple row matching
Module 231, each row matched sub-block 231 is interacted with the row cache module 26, for each target figure
The one-row pixels of picture click through every trade and match and cache row matching result to the row cache module 26.
The row matched sub-block 231 completes single file matching to each line of input.It includes：Initialization unit, for first
Beginningization is matched to directed graph；Dilated node unit, for using graph theory maximum-flow algorithm to the image after correction to real-time
The bonding strength of calculate node, and according to bonding strength parallel-expansion source node and sink nodes collection；Processing unit, for being saved in source
Point and sink nodes concentrate search Growth Route, calculate residual graph；Matching result acquiring unit, for repeating to call the initialization
Unit, dilated node unit and processing unit, untill without Growth Route, thus obtained max-flow corresponds to the one of figure
Individual minimal cut set, the cut set may be interpreted as an Optimum Matching, so as to obtain matching result and cache to the row cache module.
In single file processing, the row matched sub-block 231 can be expanded to improve node expansion to multiple nodes simultaneously
Zhang Sudu.From unlike single-unit node expansion, initialization unit initialization matching to being cached after directed graph, rather than
Directly carry out graph theory maximum-flow algorithm.Accordingly, the row matched sub-block 231 includes：Initialization unit, for initialization
Match directed graph and cache to the row cache module 26；Multiple dilated node units, each dilated node unit
For obtaining the initialization data that the row cache module 26 is cached, using graph theory maximum-flow algorithm to the image pair after correction
The bonding strength of a node is calculated in real time, and according to the source node and sink nodes collection of corresponding bonding strength parallel-expansion respective nodes
And cache to the row cache module 26；Processing unit, for obtaining source node and the remittance that the row cache module 26 is cached
Set of node data, concentrate search Growth Route in source node and sink nodes, calculate residual graph；Matching result acquiring unit, is used for
The initialization unit, each the dilated node unit and processing unit are called in repetition, untill without Growth Route, by
This obtained max-flow corresponds to a minimal cut set of figure, and the cut set may be interpreted as an Optimum Matching, so as to obtain matching
As a result and cache to the row cache module 26.
The post-processing module 24 is connected with the row cache module 26, the matching for reading the row cache module 26
As a result post-processed, obtain post processing result.Window post processing can be carried out using any image processing method based on window
To eliminate pseudo- matching.
The Depth Information Acquistion module 25 respectively with the post-processing module 24 and the calibration information acquisition module
20 are connected, for calculating target according to the post processing result of the binocular camera calibration information and the post-processing module 24
Image pixel depth, obtains target image depth information.Specially：Using binocular camera inside and outside parameter and gained parallax,
Calculated pixel-by-pixel or sub-pixel depth with Triangulation Algorithm；Depth map is smoothed, reduces trigonometric ratio error and is brought
Influence.
Using energy function feature dynamic construction figure, reduce in piece the size of internal memory and avoid Installed System Memory from reading and writing, it is real
Now based on capable energy match computing and post processing operations.Pass through row cache module and cache all intermediate data, it is to avoid be
System memory read-write expense；Algorithm is improved, it is carried out parallel node expansion.The processing method of parallel-by-bit matching can
It is low to meet reasonable hardware spending to find optimal balance point between hardware spending and performance to improve the flexibility ratio of system
Power consumption and performance requirement.
Next the preferred embodiment of above-mentioned technical proposal is provided with reference to accompanying drawing.The reality of single file processing scheme shown in accompanying drawing 3
Apply a configuration diagram.Next single file handling process is illustrated with reference to accompanying drawing 3：
Step 1：To each group of binocular camera, its precalculated inside and outside parameter obtains binocular camera calibration information；
Step 2：Binocular camera output format is RGB or YUV image；
Step 3：The view data inputted using binocular camera calibration information to binocular camera carries out image rectification, school
Data deposit row cache module after just；
Step 4：Initialization；Wherein the implementation order of step 4, can be real after step 3 without compulsive requirement
Apply, can also be with step 1 synchronization implementation；
Step 5：Row matching logic calling figure initialization logic carries out initialization figure and exported to maximum-flow algorithm logic；
Step 6：Maximum-flow algorithm logic calls local join-strength calculation logic, by the image after correction to reading in node
Logic is expanded, bonding strength is calculated in real time to the figure after initialization；
Step 7：According to the bonding strength calculated, the source node expansion logical sum sink nodes in dilated node logic are expanded
Logic parallel-expansion source node and sink nodes collection, and result is exported to Incremental Route search and residual graph algorithm logic；
Step 8：Incremental Route is searched for and residual graph algorithm logic concentrates search Growth Route, meter in source node and sink nodes
Calculate residual graph；
Step 9：The repeat step 5-8 under the control of control logic, untill without Growth Route, output matching result is arrived
Row cache module；
Step 10：Post processing logic is read in matching result from row cache module and post-processed, and will post-process result
Input depth calculation logic；
Step 11：It is deep that depth calculation logic calculates image pixel using binocular camera calibration information and post processing result
Degree, and write the result into Installed System Memory.
When single file is handled, repeat multiple dilated node logics to improve dilated node speed.Accompanying drawing 4 is shown parallel
The configuration diagram of dilated node embodiment, it is with attached embodiment illustrated in fig. 3 difference：Figure initialization logic will be initial
View data writing line cache module after change, is rather than directly to maximum-flow algorithm logic.Each dilated node logical AND
Between Incremental Route search and residual graph algorithm logic data are exchanged also by row cache module.
In order to further improve speed, multiple row matching logics can be instantiated.Accompanying drawing 5 show parallel-by-bit matching embodiment
Configuration diagram, it is with attached embodiment illustrated in fig. 3 difference：Multiple row matching logics to the multirow of view data simultaneously
Row processing, each row matching logic is swapped with row cache module, and corresponding control logic will also carry out the same time control of multirow
System.
Described above is only the preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
Member, under the premise without departing from the principles of the invention, can also make some improvements and modifications, these improvements and modifications also should be regarded as
Protection scope of the present invention.
Claims (10)
1. a kind of binocular camera image depth information acquisition methods, it is characterised in that comprise the following steps：
(1) image rectification is carried out to two width destination image datas of input according to the binocular camera calibration information being obtained ahead of time,
So that pixel corresponding on all target images is located in same horizontal line；
(2) information corrected according to described image calculates pixel matching energy, and according to the construction matching pair of pixel neighbouring relations
Per side right weight in directed graph and setting connection figure；
(3) max-flow of constructed connection figure is solved using graph theory maximum-flow algorithm using row matching way, matching is converted to
As a result and cached；
(4) matching result cached is post-processed, obtains post processing result；
(5) all target image pixel depths are calculated according to the binocular camera calibration information and post processing result,
Obtain all target image depth informations.
2. binocular camera image depth information acquisition methods according to claim 1, it is characterised in that step (1) it
Take a step forward including：Precalculate the inside and outside parameter of each group of binocular camera, and the target that setting binocular camera is exported
The form of image is RGB or YUV.
3. binocular camera image depth information acquisition methods according to claim 1, it is characterised in that in step (3)
The row matching way further comprises entering each target image multirow pixel every trade matching, every a line is matched
As a result cached respectively.
4. binocular camera image depth information acquisition methods according to claim 3, it is characterised in that the row matching
Single file matching comprises the following steps in mode：(301) initialization matching is to directed graph；(302) calculated using graph theory max-flow
Method is to bonding strength of the image after correction to real-time calculate node, and according to bonding strength parallel-expansion source node and sink nodes
Collection；(303) search Growth Route is concentrated in source node and sink nodes, calculates residual graph；(304) repeat step (301)-(303),
Untill without Growth Route, an Optimum Matching is obtained, and matching result is cached.
5. binocular camera image depth information acquisition methods according to claim 3, it is characterised in that the row matching
Mode further comprises taking parallel node to expand single file, comprised the following steps：(311) initialization matching is to directed connection
Scheme and cached；(312) initialization data that obtaining step (311) is cached, using graph theory maximum-flow algorithm to correction after
Image to calculating the bonding strength of each node in multiple nodes in real time, and accordingly saved according to corresponding bonding strength parallel-expansion
The source node and sink nodes collection of point are simultaneously cached；(313) obtaining step (312) is cached source node and sink nodes collection number
According to, concentrated in source node and sink nodes and search for Growth Route, calculating residual graph；(314) repeat step (311)-(313), until
Untill without Growth Route, an Optimum Matching is obtained, and matching result is cached.
6. a kind of binocular camera image depth information acquisition device, it is characterised in that including image correction module, for root
Image rectification is carried out to two width destination image datas of input according to the binocular camera calibration information being obtained ahead of time, so that all institutes
Pixel corresponding on image is stated in same horizontal line, and image correction information is exported to delaying one-row module；Even
Map interlinking constructing module, for calculating pixel matching energy, and constructs matching to every side right in directed graph and setting connection figure
Weigh and export to the row cache module；Row matching module, for being solved using row matching way using graph theory maximum-flow algorithm
The max-flow of the connection figure constructed, is converted to matching result and caches to the row cache module；Post-processing module, for reading
Take the matching result of the row cache module to be post-processed, obtain post processing result；Depth Information Acquistion module, for basis
The post processing result of the binocular camera calibration information and the post-processing module calculates all target image pixels
Depth, obtains all target image depth informations.
7. binocular camera image depth information acquisition device according to claim 6, it is characterised in that described device is entered
One step includes calibration information acquisition module, the inside and outside parameter for precalculating each group of binocular camera, and setting binocular
The form of the target image of camera output is RGB or YUV.
8. binocular camera image depth information acquisition device according to claim 6, it is characterised in that the row matching
Module further comprises multiple row matched sub-blocks, and each row matched sub-block is handed over the row cache module
Mutually, click through every trade matching for the one-row pixels to each target image and cache row matching result to the row and cache
Module.
9. binocular camera image depth information acquisition device according to claim 8, it is characterised in that the row matching
Submodule includes：Initialization unit, for initializing matching to directed graph；Dilated node unit, for using graph theory most
Big flow algorithm to bonding strength of the image after correction to real-time calculate node, and according to bonding strength parallel-expansion source node and
Sink nodes collection；Processing unit, for concentrating search Growth Route in source node and sink nodes, calculates residual graph；Matching result is obtained
Unit is taken, for repeating to call the initialization unit, dilated node unit and processing unit, until being without Growth Route
Only, an Optimum Matching is obtained, and matching result is cached.
10. binocular camera image depth information acquisition device according to claim 8, it is characterised in that the row
Sub-module includes：Initialization unit, for initializing matching to directed graph and caching to the row cache module；It is multiple
Dilated node unit, each dilated node unit is used to obtain the initialization data that the row cache module is cached, and adopts
With graph theory maximum-flow algorithm to bonding strength of the image after correction to one node of calculating in real time, and according to corresponding bonding strength simultaneously
The source node and sink nodes collection of row extension respective nodes are simultaneously cached to the row cache module；Processing unit, it is described for obtaining
Source node and sink nodes collection data that row cache module is cached, concentrate search Growth Route in source node and sink nodes, calculate
Residual graph；Matching result acquiring unit, for repeating to call the initialization unit, each dilated node unit and place
Unit is managed, untill without Growth Route, an Optimum Matching is obtained, and matching result is cached to the row cache module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310229171.7A CN104240217B (en) | 2013-06-09 | 2013-06-09 | Binocular camera image depth information acquisition methods and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310229171.7A CN104240217B (en) | 2013-06-09 | 2013-06-09 | Binocular camera image depth information acquisition methods and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104240217A CN104240217A (en) | 2014-12-24 |
| CN104240217B true CN104240217B (en) | 2017-08-11 |
Family
ID=52228226
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310229171.7A Active CN104240217B (en) | 2013-06-09 | 2013-06-09 | Binocular camera image depth information acquisition methods and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104240217B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106228608A (en) * | 2016-08-24 | 2016-12-14 | 集美大学 | A kind of hand-held device for object quick reconfiguration based on binocular and method |
| CN107330930B (en) * | 2017-06-27 | 2020-11-03 | 晋江市潮波光电科技有限公司 | Three-dimensional image depth information extraction method |
| CN108961328A (en) * | 2017-11-29 | 2018-12-07 | 北京猎户星空科技有限公司 | Singly take the photograph depth of field model generating method, generating means and electronic equipment |
| CN108564536B (en) * | 2017-12-22 | 2020-11-24 | 洛阳中科众创空间科技有限公司 | Global optimization method of depth map |
| CN111277902B (en) * | 2020-02-17 | 2022-03-25 | 北京达佳互联信息技术有限公司 | Video matching method, device and equipment |
| CN111489313B (en) * | 2020-04-13 | 2023-10-31 | 湖南国科微电子股份有限公司 | CFA image demosaicing method and device |
| CN113963163A (en) * | 2021-12-21 | 2022-01-21 | 北京中科慧眼科技有限公司 | Obstacle extraction method and system based on image segmentation and intelligent terminal |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101272511A (en) * | 2007-03-19 | 2008-09-24 | 华为技术有限公司 | Method and device for acquiring image depth information and image pixel information |
| CN101625761A (en) * | 2009-08-06 | 2010-01-13 | 浙江工业大学 | Computer binocular vision matching method based on global and local algorithms |
| CN103106651A (en) * | 2012-07-16 | 2013-05-15 | 清华大学深圳研究生院 | Method for obtaining parallax error plane based on three-dimensional hough |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8736672B2 (en) * | 2006-08-24 | 2014-05-27 | Reald Inc. | Algorithmic interaxial reduction |
-
2013
- 2013-06-09 CN CN201310229171.7A patent/CN104240217B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101272511A (en) * | 2007-03-19 | 2008-09-24 | 华为技术有限公司 | Method and device for acquiring image depth information and image pixel information |
| CN101625761A (en) * | 2009-08-06 | 2010-01-13 | 浙江工业大学 | Computer binocular vision matching method based on global and local algorithms |
| CN103106651A (en) * | 2012-07-16 | 2013-05-15 | 清华大学深圳研究生院 | Method for obtaining parallax error plane based on three-dimensional hough |
Non-Patent Citations (3)
| Title |
|---|
| Computing Visual Correspondence with Occlusion via Graph Cuts;Vladimir Kolmogorov et al;《IEEE International Conference on Computer Vision》;20010707;第508-515页 * |
| 基于图割理论的图像匹配问题研究;王兰柱;《中国优秀硕士学位论文全文数据库 信息科技辑》;20120615(第6期);第I138-1812页 * |
| 基于网络最大流的立体匹配算法;黄海赟 等;《上海交通大学学报》;20010228;第35卷(第2期);第168-172页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104240217A (en) | 2014-12-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104240217B (en) | Binocular camera image depth information acquisition methods and device | |
| WO2021035807A1 (en) | Target tracking method and device fusing optical flow information and siamese framework | |
| Zou et al. | Df-net: Unsupervised joint learning of depth and flow using cross-task consistency | |
| CN108398139B (en) | Dynamic environment vision mileometer method fusing fisheye image and depth image | |
| CN104317391B (en) | A kind of three-dimensional palm gesture recognition exchange method and system based on stereoscopic vision | |
| CN105809712B (en) | A kind of efficient big displacement light stream method of estimation | |
| CN109377530A (en) | A kind of binocular depth estimation method based on deep neural network | |
| CN107590827A (en) | A kind of indoor mobile robot vision SLAM methods based on Kinect | |
| CN113205595B (en) | Construction method and application of 3D human body posture estimation model | |
| CN103106688A (en) | Indoor three-dimensional scene rebuilding method based on double-layer rectification method | |
| CN114036969B (en) | 3D human body action recognition algorithm under multi-view condition | |
| WO2021051526A1 (en) | Multi-view 3d human pose estimation method and related apparatus | |
| JP2019149142A (en) | Object marking system and object marking method | |
| CN106934351A (en) | Gesture identification method, device and electronic equipment | |
| CN103870843A (en) | Head posture estimation method based on multi-feature-point set active shape model (ASM) | |
| Zhai et al. | Optical flow estimation using channel attention mechanism and dilated convolutional neural networks | |
| CN110443849A (en) | A kind of object localization method of the double-current convolutional neural networks recurrence learning based on depth image | |
| CN107065866A (en) | A kind of Mobile Robotics Navigation method based on improvement optical flow algorithm | |
| CN111882602A (en) | Visual odometer implementation method based on ORB feature points and GMS matching filter | |
| CN112183506A (en) | Human body posture generation method and system | |
| CN116030498A (en) | Virtual garment running and showing oriented three-dimensional human body posture estimation method | |
| CN111914639A (en) | Driving action recognition method of lightweight convolution space-time simple cycle unit model | |
| CN114689038A (en) | Fruit detection positioning and orchard map construction method based on machine vision | |
| CN112634305B (en) | Infrared visual odometer implementation method based on edge feature matching | |
| Li et al. | Scale-aware monocular SLAM based on convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| ASS | Succession or assignment of patent right |
Owner name: SHANGHAI XINGXIN MICRO-ELECTRONIC TECHNOLOGY CO., Free format text: FORMER OWNER: ZHOU YU Effective date: 20150710 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20150710 Address after: 201999 4403-K room, No. 1325, Mudanjiang Road, Shanghai, Baoshan District Applicant after: SHANGHAI X-CHIP MICROELECTRONIC TECHNOLOGY CO., LTD. Address before: 200441, room 39, No. 388, Lane 202, Chang Yi Road, Shanghai, Baoshan District Applicant before: Zhou Yu |
|
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |