CN105447885B - The method and apparatus for calculating parallax - Google Patents
The method and apparatus for calculating parallax Download PDFInfo
- Publication number
- CN105447885B CN105447885B CN201410331089.XA CN201410331089A CN105447885B CN 105447885 B CN105447885 B CN 105447885B CN 201410331089 A CN201410331089 A CN 201410331089A CN 105447885 B CN105447885 B CN 105447885B
- Authority
- CN
- China
- Prior art keywords
- pixel
- reliable
- parallax
- selected pixels
- image frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The embodiment provides a kind of method and apparatus calculating parallax.The method according to the ... of the embodiment of the present invention for calculating parallax, including：According to the disparity map for the first picture frame that stereoscopic camera is shot in previous moment, its parallax reliably reliable pixel is selected in the first picture frame；Each reliable corresponding pixel points of the pixel in stereoscopic camera in the second picture frame that current time shoots of prediction；Determine that the reliable pixel of at least part is selected pixels point；For each selected pixels point, the disparity computation range of the corresponding pixel points of the selected pixels point is determined in the second picture frame of current time shooting；And the disparity computation range according to each selected pixels point, calculate parallax of the selected pixels o'clock in the second picture frame.
Description
Technical field
The present invention relates to the method and apparatus for calculating parallax, more particularly it relates to based on the parallax previously obtained
Method and apparatus to calculate current parallax.
Background technology
With the development of the technologies such as three-dimensional reconstruction, virtual reality, distance between camera and captured special object is indicated
Parallax information drive etc. in such as auxiliary and be widely used in applications.By Stereo image matching technology, can about
Matched pixel pair is found in the benchmark image and reference image of identical three-dimensional scenic, each pixel coordinate pair is calculated and answers
Parallax.And then the three-dimensional spatial information of scene can accurately be described using parallax.
Disparity computation is individually carried out for each picture frame, accurate result of calculation can be obtained, but in entire image
In the range of parallax calculate need to take a significant amount of time, while also will produce unnecessary noise.In order to save parallax
Calculate the time, it has been proposed that parallax inheritance method, in the parallax inheritance method, by making the pixel in a later frame inherit
The parallaxes of corresponding pixel points of the pixel in former frame obtains the parallax of the pixel in a later frame.Fig. 1 is to show
The exemplary disparity map 100 that is obtained by parallax inheritance method.As shown in Figure 1, since the method receives former frame completely
Parallax is as a result, can be mixed into a large amount of noise and wrong parallax in the parallax result of present frame.For example, on vehicle 110 and wall 120
There are significant noises.
Invention content
The embodiment of the present invention is designed to provide a kind of method and apparatus calculating parallax, to solve the above problems.
An embodiment provides a kind of methods calculating parallax, including：According to stereoscopic camera when previous
The disparity map for carving the first picture frame of shooting, selects its parallax reliably reliable pixel in described first image frame；Prediction
Each reliable corresponding pixel points of the pixel in stereoscopic camera in the second picture frame that current time shoots；Determine at least one
It is selected pixels point to divide reliable pixel；For each selected pixels point, determined in the second picture frame of current time shooting
The disparity computation range of the corresponding pixel points of the selected pixels point；And the disparity computation range according to each selected pixels point,
Calculate parallax of the selected pixels o'clock in the second picture frame.
Another embodiment of the present invention provides a kind of device calculating parallax, including：Pixel selecting unit, configuration come
According to the disparity map for the first picture frame that stereoscopic camera is shot in previous moment, select its parallax can in described first image frame
The reliable pixel leaned on；Pixel predicting unit configures to predict that each reliable pixel is clapped in stereoscopic camera at current time
Corresponding pixel points in the second picture frame taken the photograph；First pixel determination unit configures to determine the reliable pixel of at least part
Point is selected pixels point；Range determination unit, configuration comes for each selected pixels point, in the second image of current time shooting
The disparity computation range of the corresponding pixel points of the selected pixels point is determined in frame；And disparity computation unit, it configures and comes according to every
The disparity computation range of a selected pixels point, calculates parallax of the selected pixels o'clock in the second picture frame.
In the method and apparatus for the calculating parallax that foregoing invention embodiment provides, the figure shot in previous moment may be selected
As the pixel with reliable parallax in frame, determination is corresponding with the pixel being selected in the picture frame of current time shooting
Pixel, and the parallax of these corresponding pixels is calculated within the scope of specific disparity computation, so as to more quickly
Ground obtains accurate parallax.
Description of the drawings
Fig. 1 is to show the exemplary disparity map obtained by parallax inheritance method.
Fig. 2 depicts the flow chart of the method according to an embodiment of the invention for calculating parallax.
Fig. 3 is depicted according to an embodiment of the present invention, and the method for selected pixels point is selected from reliable pixel
Flow chart.
Fig. 4 is the exemplary parallax that the method for showing the disparity computation provided by the embodiments of the present invention obtains
Figure.
Fig. 5 is the exemplary block diagram for showing the device according to an embodiment of the invention for calculating parallax.
Fig. 6 is the demonstrative structure frame for showing the first pixel determination unit according to an embodiment of the invention
Figure.
Fig. 7 is the general hardware block diagram for the hardware system for showing the calculating parallax according to the embodiment of the present invention.
Specific implementation mode
In order to make those skilled in the art more fully understand the present invention, the preferred reality of the present invention will be described in detail with reference to the attached drawings
Apply example.Note that in the present description and drawings, there is substantially the same step and element to be denoted by the same reference numerals,
And the repetition of explanation of these steps and element will be omitted.
Fig. 2 depicts the flow chart of the method 200 according to an embodiment of the invention for calculating parallax.In the following, will ginseng
The method 200 according to the ... of the embodiment of the present invention for calculating parallax is described according to Fig. 2.As shown in Fig. 2, in step s 201, according to vertical
The disparity map for the first picture frame that body camera is shot in previous moment selects its parallax reliably reliable picture in the first picture frame
Vegetarian refreshments.Preferably, the first picture frame shot in previous moment stereoscopic camera can be calibrated first, after then using calibration
Image calculate parallax.
In step s 201, any of parallax calculation method can be used.For example, half global registration algorithm can be used
(SGM) parallax is calculated.In SGM algorithms, the search range [0, d of the parallax for calculating pixelmax], wherein dmaxIt is pre-
The max-thresholds of the search parallax first determined.For example, d can be determined according to the size of the picture of camera shootingmax.For each
Pixel is in entire scope [0, dmax] in scan for, and by the minimum point of Matching power flow curve it is corresponding value be used as the picture
The final parallax of vegetarian refreshments.Then it can determine whether the parallax of each pixel in the first picture frame is reliable.For example, can by with
Whether lower formula (1) is reliable to determine the parallax of the pixel P (t-1) in the first picture frame：
Cost(P(t-1),dmin)<<Cost(P(t-1),dsecmin) ……(1)
Wherein dminFor the corresponding parallax of minimum point of Matching power flow curve, dsecminFor the second from the bottom of Matching power flow curve
The corresponding parallax of low spot.When meeting formula (1), determine that pixel P (t-1) is reliable pixel.
Then, in step S202, the second figure that each reliable pixel is shot in stereoscopic camera at current time is predicted
As the corresponding pixel points in frame.In step S202, it is every to predict that any of computational methods such as optical flow method can be used
Corresponding pixel points of a reliable pixel in stereoscopic camera in the second picture frame that current time shoots.
In addition in step S203, it may be determined that the reliable pixel of at least part is selected pixels point.It should be noted that in Fig. 2
Shown in each step of method sequentially executed not necessarily in shown in.It can overturn or be performed in parallel certain steps.According to
The example of the present invention can be by all reliable pixels obtained in step S201 all as selection picture in step S203
Vegetarian refreshments.In the case, selected in the first picture frame its parallax reliably reliable pixel (step S201) and determine at least
A part of reliable pixel, which is selected pixels point (step S203), to be performed simultaneously.And it can first determine at least one
It is that selected pixels point (step S203) predicts that each reliable pixel is clapped in stereoscopic camera at current time again to divide reliable pixel
Corresponding pixel points (step S202) in the second picture frame taken the photograph.
Alternatively, another example according to the present invention can be according to movable information from step S201 in step S203
Selected pixels point is selected in the reliable pixel obtained.Fig. 3 is depicted according to an embodiment of the present invention, from reliable pixel
The flow chart of the method 300 of middle selection selected pixels point.Each reliable pixel in first picture frame can be executed respectively
Method 300 shown in Fig. 3.
As shown in figure 3, in step S301, the movable information of a reliable pixel is obtained.One according to the present invention is shown
Example, movable information can be when the first picture frame and the second picture frame to be placed in same coordinate system, from a reliable pixel
To the displacement of the corresponding pixel points of the reliable pixel.
In step s 302, determine calculated movable information whether within the scope of predetermined information.And works as and calculated
When movable information is within the scope of predetermined information, the reliable pixel is determined as selected pixels point in step S303.For example,
In the case of displacement of the movable information for a reliable pixel to the corresponding pixel points of the reliable pixel, in step s 302,
It can determine calculated displacement whether within the scope of predetermined displacement.Preferably, can pre-set predetermined displacement range does not include
Amplitude is less than the displacement of first threshold, and to remove noise, and predetermined displacement range does not include the second threshold that amplitude is more than
Displacement moves excessive pixel with removal.And when the displacement calculated is within the scope of predetermined information, in step S303
In, which is determined as selected pixels point.
Furthermore according to another example of the present invention, stereoscopic camera can be the binocular phase for including two image acquisition units
Machine.In the case, the first picture frame captured by stereoscopic camera may include the first benchmark image frame and the first reference image frame,
And the second picture frame may include the second benchmark image frame and the second reference image frame.In step s 201, in the first picture frame
Reliably reliable pixel may include selecting its parallax reliably reliable pixel from the first benchmark image frame middle its parallax of selection
Point, and obtained and the one-to-one reference image vegetarian refreshments of reliable pixel from the first reference image frame.In addition, according to fig. 3
Shown in from reliable pixel select selected pixels point method 300 can be according to the first benchmark image frame in step S301
The benchmark image movable information of the reliable pixel is obtained with the second benchmark image frame, and according to the first reference image frame and
Two reference image frames obtain the reference picture movable information of reference image vegetarian refreshments corresponding with the reliable pixel.In step S302
In, it can determine benchmark image movable information and reference picture movable information whether within the scope of predetermined information respectively.In addition, working as base
It, can be by the reliable picture in step S303 when quasi- image motion information and reference picture movable information are all within the scope of predetermined information
Vegetarian refreshments is determined as selected pixels point.
For example, can according to this in the position coordinates and the second benchmark image frame of reliable pixel in the first benchmark image frame
By the position coordinates of the corresponding pixel points of pixel, calculating exists from the corresponding pixel points of reliable pixel to the reliable pixel
Displacement (Vx, Vy) in horizontal direction and in vertical direction, and by the amplitude mag of the displacementvIt moves and believes as benchmark image
Breath；Furthermore according to the reference image vegetarian refreshments in the position coordinates and the second reference image frame of reference image vegetarian refreshments in the first reference image frame
Corresponding pixel points position coordinates, calculate corresponding pixel points from the reference image vegetarian refreshments to the reference image vegetarian refreshments in the horizontal direction
Displacement (Ux, Uy) in upper and vertical direction, and by the amplitude mag of the displacementuAs with reference to image motion information.It can pass through
Following formula (2) calculates magv：
And mag can be calculated by following formula (3)u：
Amplitude mag can be determined respectivelyvAnd maguWhether within the scope of predetermined displacement.And when amplitude magv and magu are pre-
When determining in displacement range, which is determined as selected pixels point.
Fig. 2 is returned to, in step S204, for each selected pixels point, in the second picture frame of current time shooting
Determine the disparity computation range of the corresponding pixel points of the selected pixels point.It according to an example of the present invention, can be according to selection picture
Position of the corresponding pixel points of the position coordinates and the selected pixels point in the first picture frame of vegetarian refreshments in the second picture frame
Coordinate determines disparity computation range.For example, it is assumed that stereoscopic camera be binocular camera, a selected pixels point in the first benchmark image
Coordinate in the horizontal direction in frame is Px (t-1), and the reference image vegetarian refreshments which selects is in the first reference image frame
Coordinate in the horizontal direction be Qx (t-1), the corresponding pixel points of the selected pixels point in the second benchmark image frame
Coordinate in the horizontal direction is Px (t), and the corresponding pixel points of the selected pixels point are corresponding in the second reference image frame
The coordinate in the horizontal direction of position is Qx (t).Disparity computation model can be determined according to the coordinate of pixel in the horizontal direction
It encloses [d (t-1), d (t)], wherein d (t-1)=| Px (t-1)-Qx (t-1) |, and d (t)=| Px (t)-Qx (t) |.In addition excellent
Selection of land, disparity computation range can be [d (t-1)-Δ, d (t)+Δ], and wherein Δ is noise margin.
Then, it in step S205, according to the disparity computation range of each selected pixels point, calculates the selected pixels point and exists
Parallax in second picture frame.Pixel in the second picture frame can be calculated using above-mentioned SGM methods according to disparity computation range
Parallax.
In addition, for each residual pixel other than the corresponding pixel points of selected pixels point in the second picture frame
Point can calculate parallax of the residual pixel o'clock in the second picture frame, wherein each corresponding pixel points according to predetermined disparity range
Disparity computation range it is smaller than predetermined disparity range.Such as described above, in SGM methods, predetermined disparity range be [0,
dmax], and the disparity computation ranging from [d (t-1), d (t)] determined according to embodiments of the present invention.It determines according to embodiments of the present invention
Coordinate determine that disparity computation range is obviously smaller than predetermined disparity range.
In the method for the calculating parallax that the embodiments of the present invention provide, the picture frame shot in previous moment may be selected
In the pixel with reliable parallax, corresponding with the pixel being selected pixel is determined in the picture frame of current time shooting
Point, and calculate within the scope of specific disparity computation the parallax of these corresponding pixels, so as to more rapidly
To accurate parallax.
Fig. 4 is the exemplary disparity map that the method for showing the disparity computation provided by the embodiments of the present invention obtains
400.Disparity map 400 in Fig. 4 is directed to similar scene with the disparity map 100 in Fig. 1.With in disparity map 100 shown in FIG. 1
Vehicle 110 is compared with wall 120, in exemplary the regarding of the method acquisition of the disparity computation provided by the embodiments of the present invention
In poor Figure 40 0, the noise on vehicle 410 and wall 420 considerably reduces.That is, compared with conventional parallax preparation method,
The method of the disparity computation provided by the embodiments of the present invention can rapidly obtain more accurate parallax.
The method of the calculating parallax provided according to above-described embodiment can use in the video image being continuously shot.Specifically
Ground, can in video image in addition to first picture frame other than each picture frame respectively use above-described embodiment provide meter
The method for calculating parallax, to obtain the disparity map corresponding to each picture frame, and with the increase of number of use, disparity computation is fast
Degree and accuracy are continuously improved.
In the following, illustrating the device of the calculating parallax of the embodiment of the present invention with reference to Fig. 5.Fig. 5 is shown according to the present invention
One embodiment calculating parallax device 500 exemplary block diagram.As shown in Figure 5, the calculating of the present embodiment regards
The device 500 of difference includes pixel selecting unit 510, pixel predicting unit 520, the first pixel determination unit 530, range
Determination unit 540 and disparity computation unit 550.Calculating each unit of the device 500 of parallax can respectively execute in above-mentioned Fig. 2
Calculate each step/function of the method 200 of parallax.Therefore, the critical piece of the device 500 only to calculating parallax carries out below
Description, and it is omitted the above detail content for having been combined Fig. 2 and describing.
For example, the parallax for the first picture frame that pixel selecting unit 510 can be shot according to stereoscopic camera in previous moment
Figure, selects its parallax reliably reliable pixel in the first picture frame.It preferably, can be first in previous moment stereoscopic camera
First picture frame of shooting is calibrated, and then calculates parallax using the image after calibration.
Any of parallax calculation method can be used in pixel selecting unit 510.For example, pixel selecting unit 510
Half global registration algorithm (SGM) can be used to calculate parallax.In SGM algorithms, the search model of the parallax for calculating pixel
Enclose [0, dmax], wherein dmaxIt is predetermined the max-thresholds of search parallax.For example, the ruler for the picture that can be shot according to camera
It is very little to determine dmax.For each pixel in entire scope [0, dmax] in scan for, and most by Matching power flow curve
The corresponding value of the low spot parallax final as the pixel.Then can determine whether the parallax of each pixel in the first picture frame is
It is no reliable.For example, can determine whether the parallax of the pixel P (t-1) in the first picture frame is reliable by above-mentioned formula (1).
Then, the predictable each reliable pixel of pixel predicting unit 520 is shot in stereoscopic camera at current time
Corresponding pixel points in second picture frame.Any of calculating side such as optical flow method can be used in pixel predicting unit 520
Method predicts corresponding pixel points of each reliable pixel in stereoscopic camera in the second picture frame that current time shoots.
First pixel determination unit 530 can determine that the reliable pixel of at least part is selected pixels point.According to this hair
A bright example, the reliable pixel that the first pixel determination unit 530 can obtain all pixels point selecting unit 510 is all
As selected pixels point.In addition, another example according to the present invention, the first pixel determination unit 530 can be according to movable information
Selected pixels point is selected in the reliable pixel obtained from pixel selecting unit 510.
Fig. 6 is the demonstrative structure for showing the first pixel determination unit 530 according to an embodiment of the invention
Block diagram.As shown in Figure 6, the first pixel determination unit 530 of the present embodiment includes data obtaining module 610, information determination
Module 620 and the first pixel determining module 630.The modules of first pixel determination unit 530 can execute above-mentioned respectively
Each step/function of the method 300 that selected pixels point is selected from reliable pixel in Fig. 3.Therefore, only the first picture below
The critical piece of vegetarian refreshments determination unit 530 is described, and the above detail content for having been combined Fig. 3 and describing is omitted.
As shown in fig. 6, data obtaining module 610 can obtain the movable information of a reliable pixel.According to the present invention one
A example, movable information can be in same coordinate system, the corresponding pixel points of a reliable pixel to the reliable pixel
Displacement.
Whether information determination module 620 can determine calculated movable information within the scope of predetermined information.When what is calculated
When movable information is within the scope of predetermined information, which can be determined as choosing picture by the first pixel determining module 630
Vegetarian refreshments.For example, in the case where movable information is displacement of the reliable pixel to the corresponding pixel points of the reliable pixel, letter
Whether breath determining module 620 can determine calculated displacement within the scope of predetermined displacement.For example, predetermined displacement can be pre-set
Range does not include the displacement that amplitude is less than first threshold, and to remove noise, and predetermined displacement range does not include what amplitude was more than
The displacement of second threshold moves excessive pixel with removal.And when the displacement calculated is within the scope of predetermined information,
The reliable pixel can be determined as selected pixels point by the first pixel determining module 630.
Another example according to the present invention, stereoscopic camera can be the binocular cameras for including two image acquisition units.
In this case, the first picture frame captured by stereoscopic camera may include the first benchmark image frame and the first reference image frame, and
Second picture frame may include the second benchmark image frame and the second reference image frame.In the case, pixel selecting unit 510 can
Including reliable pixel point selection module and reference pixel point selection module.Reliable pixel point selection module can be from the first benchmark image
Its parallax reliably reliable pixel is selected in frame, and reference pixel point selection module can be obtained from the first reference image frame
With the one-to-one reference image vegetarian refreshments of reliable pixel.In addition, in the first pixel determination unit 530 shown in Fig. 6, information
Acquisition module 610 can obtain the benchmark image movement of the reliable pixel according to the first benchmark image frame and the second benchmark image frame
Information, and reference image vegetarian refreshments corresponding with the reliable pixel is obtained according to the first reference image frame and the second reference image frame
Reference picture movable information.Information determination module 620 can determine benchmark image movable information and reference picture movement letter respectively
Whether breath is within the scope of predetermined information.In addition, when benchmark image movable information and reference picture movable information are all in predetermined information
When in range, which can be determined as selected pixels point by the first pixel determining module 630.For example, acquisition of information
Module 610 can be according to the reliable pixel in the position coordinates and the second benchmark image frame of reliable pixel in the first benchmark image frame
The position coordinates of the corresponding pixel points of point calculate the corresponding pixel points from the reliable pixel to the reliable pixel in level side
Upwards and the displacement (Vx, Vy) in vertical direction, and by the amplitude mag of the displacementvAs benchmark image movable information；And
According to the correspondence of the reference image vegetarian refreshments in the position coordinates of reference image vegetarian refreshments in the first reference image frame and the second reference image frame
The position coordinates of pixel calculate the corresponding pixel points from the reference image vegetarian refreshments to the reference image vegetarian refreshments in the horizontal direction and hang down
The upward displacement of histogram (Ux, Uy), and by the amplitude mag of the displacementuAs with reference to image motion information.Above-mentioned public affairs can be passed through
Formula 2 calculates magvAnd mag is calculated by above-mentioned formula 3u.Information determination module 620 can determine amplitude mag respectivelyvWith
maguWhether within the scope of predetermined displacement.And work as amplitude magvAnd maguWhen within the scope of predetermined displacement, the first pixel determines
The reliable pixel is determined as selected pixels point by module 630.
Fig. 5 is returned to, range determination unit 540 can be for each selected pixels point, in the second image of current time shooting
The disparity computation range of the corresponding pixel points of the selected pixels point is determined in frame.According to an example of the present invention, range determines
Unit 540 can exist according to the corresponding pixel points of the position coordinates and the selected pixels point in the first picture frame of selected pixels point
Position coordinates in second picture frame determine disparity computation range.For example, it is assumed that stereoscopic camera is binocular camera, a selected pixels
The coordinate in the horizontal direction in the first benchmark image frame of point is Px (t-1), the reference image vegetarian refreshments which selects
Coordinate in the horizontal direction in the first reference image frame is Qx (t-1), the corresponding pixel points of the selected pixels point
Coordinate in the horizontal direction in second benchmark image frame is Px (t), and the corresponding pixel points of the selected pixels point are the
The coordinate in the horizontal direction of corresponding position is Qx (t) in two reference image frames.Range determination unit 540 can be according to pixel
Coordinate in the horizontal direction determines disparity computation range [d (t-1), d (t)], wherein d (t-1)=| Px (t-1)-Qx (t-1)
|, and d (t)=| Px (t)-Qx (t) |.Further preferably, disparity computation range can be [d (t-1)-Δ, d (t)+Δ],
Middle Δ is noise margin.
Then, disparity computation unit 550 can calculate the selected pixels according to the disparity computation range of each selected pixels point
Parallax o'clock in the second picture frame.Pixel in the second picture frame can be calculated using above-mentioned SGM methods according to disparity computation range
The parallax of point.
In addition, the device 500 for calculating parallax may also include the second pixel determination unit, to be determined in the second picture frame
Residual pixel point other than the corresponding pixel points of selected pixels point.For in the second picture frame in addition to selected pixels point
Corresponding pixel points other than each residual pixel point, disparity computation unit 550 can calculate the residue according to predetermined disparity range
Parallax of the pixel in the second picture frame, wherein the disparity computation range of each corresponding pixel points is smaller than predetermined disparity range.
Such as described above, in SGM methods, predetermined disparity range is [0, dmax], and the parallaxometer determined according to embodiments of the present invention
It calculates ranging from [d (t-1), d (t)].It is predetermined to determine that disparity computation range is obviously smaller than for determining coordinate according to embodiments of the present invention
Disparity range.
In the device for the calculating parallax that the embodiments of the present invention provide, the picture frame shot in previous moment may be selected
In the pixel with reliable parallax, corresponding with the pixel being selected pixel is determined in the picture frame of current time shooting
Point, and calculate within the scope of specific disparity computation the parallax of these corresponding pixels, so as to more rapidly
To accurate parallax.
The device of the calculating parallax provided according to above-described embodiment can use in the video image being continuously shot.Specifically
Ground, can in video image in addition to first picture frame other than each picture frame respectively use above-described embodiment provide meter
The device of parallax is calculated, to obtain the disparity map corresponding to each picture frame, and with the increase of number of use, disparity computation is fast
Degree and accuracy are continuously improved.
In addition, another example according to the present invention, the present invention can also be by a kind of hardware system calculating parallax come real
It applies.Fig. 7 is the general hardware block diagram for the hardware system 700 for showing the calculating parallax according to the embodiment of the present invention.As shown in fig. 7,
Calculate parallax hardware system 700 may include：Input equipment 710 is used for from the related image of external input or information, such as takes the photograph
The left image and right image of camera shooting, the parameter of video camera or initial parallax figure etc., for example, may include keyboard, Genius mouse,
And communication network and its remote input equipment etc. that is connected；Processing equipment 720, it is above-mentioned according to the present invention for implementing
The method of the calculating parallax of embodiment, such as may include the central processing unit of computer or other cores with processing capacity
Piece etc., in addition, processing equipment 720 may be also connected to the network (not shown) of such as internet, according to the need of processing procedure
It will be to teletransmission treated image etc.；Output equipment 730 implements above-mentioned disparity map improvement process for being output to the outside
Gained as a result, such as may include display, printer and communication network and its remote output devices connected
Deng；And storage device 740, for being stored in a manner of volatile and nonvolatile involved by above-mentioned disparity map improvement process such as
The data such as the disparity map after initial parallax figure, similarity matrix, graph model, improvement, such as may include random access memory
(RAM), the various volatile and nonvolatile property memories of read-only memory (ROM), hard disk or semiconductor memory etc..
Those skilled in the art will appreciate that the present invention can be implemented as system, device, method or computer program
Product.Therefore, the present invention can be with specific implementation is as follows, i.e.,：It can be complete hardware, can also be complete software
(including firmware, resident software, microcode etc.), can also be hardware and software combine form, referred to generally herein as " component,
" module ", " device " or " system ".In addition, in some embodiments, the present invention is also implemented as calculating in one or more
The form of computer program product in machine readable medium includes computer-readable program generation in the computer-readable medium
Code.
The arbitrary combination of one or more computer-readable mediums may be used.Computer-readable medium can be computer
Readable signal medium or computer readable storage medium.Computer readable storage medium can for example be but not limited to electricity, magnetic,
Optical, electromagnetic, the system of infrared ray or semiconductor, device or device, or the arbitrary above combination.Computer-readable storage medium
The more specific example (non exhaustive list) of matter includes：Electrical connection, portable computer magnetic with one or more conducting wires
Disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM or sudden strain of a muscle
Deposit), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device or above-mentioned appoint
The suitable combination of meaning.In this document, can be any include computer readable storage medium or the tangible medium of storage program,
The program can be commanded the either device use or in connection of execution system, device.
Computer-readable signal media may include in a base band or as the data-signal that a carrier wave part is propagated,
Wherein carry computer-readable program code.Diversified forms may be used in the data-signal of this propagation, including but unlimited
In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By instruction execution system, device either device use or program in connection.
The program code for including on computer-readable medium can transmit with any suitable medium, including but not limited to without
Line, electric wire, optical cable, RF etc. or above-mentioned any appropriate combination.
It can be write with one or more programming languages or combinations thereof for executing the computer that operates of the present invention
Program code, described program design language include object oriented program language-such as Java, Smalltalk, C++,
Further include conventional procedural programming language-such as " C " language or similar programming language.Program code can be with
It fully executes, partly execute on the user computer on the user computer, being executed as an independent software package, portion
Divide and partly executes or executed on a remote computer or server completely on the remote computer on the user computer.
It is related in the situation of remote computer, remote computer by the network-of any kind can include LAN (LAN) or wide
Domain net (WAN)-be connected to subscriber computer, or, it may be connected to outer computer (such as provided using Internet service
Quotient is connected by internet).
Above with reference to the method, apparatus (system) of the embodiment of the present invention and the flow chart and/or frame of computer program product
Figure describes the present invention.It should be appreciated that each box in each box and flowchart and or block diagram of flowchart and or block diagram
Combination, can be realized by computer program instructions.These computer program instructions can be supplied to all-purpose computer, special
The processor of computer or other programmable data processing units, to produce a kind of machine, these computer program instructions
It is executed by computer or other programmable data processing units, produces and advised in the box in implementation flow chart and/or block diagram
The device of fixed function/operation.
These computer program instructions can also be stored in can be so that computer or other programmable data processing units
In computer-readable medium operate in a specific manner, in this way, the instruction of storage in computer-readable medium just produces one
Command device (the instruction of function/operation specified in a box including in implementation flow chart and/or block diagram
Means manufacture (manufacture)).
Computer program instructions can also be loaded into computer, other programmable data processing units or miscellaneous equipment
On so that series of operation steps are executed on computer, other programmable data processing units or miscellaneous equipment, in terms of generating
The process that calculation machine is realized, so that the instruction executed on a computer or other programmable device is capable of providing implementation flow chart
And/or the process of function/operation specified in the box in block diagram.
Flow chart and block diagram in attached drawing show the system, method and computer journey of multiple embodiments according to the present invention
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part for a part for one module, section or code of table, the module, section or code includes one or more uses
The executable instruction of the logic function as defined in realization.It is also noted that each box in block diagram and or flow chart, with
And the combination of the box in block diagram and or flow chart, the dedicated hardware based of functions or operations as defined in executing can be used
System is realized, or can be realized using a combination of dedicated hardware and computer instructions.
Various embodiments of the present invention are described above, above description is exemplary, and non-exclusive, and
It is not limited to disclosed each embodiment.Without departing from the scope and spirit of illustrated each embodiment, for this skill
Many modifications and changes will be apparent from for the those of ordinary skill in art field.The selection of term used herein, purport
In the principle, practical application or improvement to the technology in market for best explaining each embodiment, or make the art
Other those of ordinary skill can understand each embodiment disclosed herein.
Claims (10)
1. a kind of method calculating parallax, including：
According to the disparity map for the first picture frame that stereoscopic camera is shot in previous moment, it is selected to regard in described first image frame
Poor reliable reliable pixel；
Each reliable corresponding pixel points of the pixel in the stereoscopic camera in the second picture frame that current time shoots of prediction；
Determine that at least part reliable pixel is selected pixels point；
For each selected pixels point, the correspondence of the selected pixels point is determined in the second picture frame of current time shooting
The disparity computation range of pixel；And
According to the disparity computation range of the corresponding pixel points of each selected pixels point, the corresponding pixel points of the selected pixels point are calculated
Parallax in second picture frame.
2. the method as described in claim 1, wherein determining at least part reliable pixel is selected pixels point
Including：For each reliable pixel
Obtain the movable information of the reliable pixel；
Determine calculated movable information whether within the scope of predetermined information；
When the movable information calculated is within the scope of predetermined information, which is determined as selected pixels point.
3. method as claimed in claim 2, wherein
It is described obtain the reliable pixel movable information include：
Calculate the displacement from the reliable pixel to the corresponding pixel points of the reliable pixel；And
The movable information that the determination is calculated whether within the scope of predetermined information include：
Determine calculated displacement whether within the scope of predetermined displacement.
4. method as claimed in claim 2 or claim 3, wherein
Described first image frame includes the first benchmark image frame and the first reference image frame；
Second picture frame includes the second benchmark image frame and the second reference image frame；
The disparity map of first picture frame shot in previous moment according to stereoscopic camera, selects in described first image frame
Reliably reliable pixel includes its parallax：
Its parallax reliably reliable pixel is selected from the first benchmark image frame,
It is obtained from first reference image frame and the reliable one-to-one reference image vegetarian refreshments of pixel；The acquisition should
Reliably the movable information of pixel includes：
The benchmark image movable information of the reliable pixel is obtained according to the first benchmark image frame and the second benchmark image frame；
The ginseng of reference image vegetarian refreshments corresponding with the reliable pixel is obtained according to the first reference image frame and the second reference image frame
Examine image motion information；
The movable information that the determination is calculated whether within the scope of predetermined information include：
Determine the benchmark image movable information and the reference picture movable information whether within the scope of predetermined information respectively；With
And
It is described when the movable information calculated is within the scope of predetermined information, which is determined as selected pixels point packet
It includes：
It is when the benchmark image movable information and the reference picture movable information are all within the scope of predetermined information, this is reliable
Pixel is determined as selected pixels point.
5. method as claimed in claim 1 or 2, further includes：
The residual pixel point other than the corresponding pixel points of selected pixels point is determined in second picture frame；And
For each residual pixel point, which is calculated according to predetermined disparity range
Difference, wherein
The disparity computation range of each corresponding pixel points is smaller than the predetermined disparity range.
6. a kind of device calculating parallax, including：
Pixel selecting unit, configuration carrys out the disparity map of the first picture frame shot in previous moment according to stereoscopic camera, in institute
It states and selects its parallax reliably reliable pixel in the first picture frame；
Pixel predicting unit, configure predict each reliable pixel the stereoscopic camera current time shoot second
Corresponding pixel points in picture frame；
First pixel determination unit, to determine, the reliable pixel is selected pixels point at least partially for configuration；
Range determination unit, configuration come for each selected pixels point, in the second picture frame of current time shooting really
The disparity computation range of the corresponding pixel points of the fixed selected pixels point；And
Disparity computation unit, configuration carry out the disparity computation range of the corresponding pixel points according to each selected pixels point, calculate the choosing
Parallax of the corresponding pixel points of capture vegetarian refreshments in second picture frame.
7. device as claimed in claim 6, wherein the first pixel determination unit includes：
Data obtaining module configures to obtain the movable information of each reliable pixel；
Whether information determination module configures to determine calculated movable information within the scope of predetermined information；And
First pixel determining module configures when the movable information calculated is within the scope of predetermined information, which to be believed
It ceases corresponding reliable pixel and is determined as selected pixels point.
8. device as claimed in claim 7, wherein
Described information acquisition module obtains displacement of each reliable pixel to the corresponding pixel points of the reliable pixel；
Whether described information determining module determines calculated displacement within the scope of predetermined displacement.
9. device as claimed in claim 7 or 8, wherein
Described first image frame includes the first benchmark image frame and the first reference image frame；
Second picture frame includes the second benchmark image frame and the second reference image frame；
Pixel selecting unit includes
Reliable pixel point selection module, configures to select its parallax reliably reliable pixel from the first benchmark image frame
Point,
Reference pixel point selection module, configuration are a pair of with the reliable pixel one to be obtained from first reference image frame
The reference image vegetarian refreshments answered；
Described information acquisition module obtains the base of each reliable pixel according to the first benchmark image frame and the second benchmark image frame
Quasi- image motion information, and it is corresponding with each reliable pixel according to the first reference image frame and the acquisition of the second reference image frame
Reference image vegetarian refreshments reference picture movable information；
Described information determining module determine respectively the benchmark image movable information and the reference picture movable information whether
Within the scope of predetermined information；And
When the benchmark image movable information of a reliable pixel and the reference picture movable information are all within the scope of predetermined information
When, which is determined as selected pixels point by described information determining module.
10. device as claimed in claims 6 or 7, further includes：
Second pixel determination unit, configuration to determine the corresponding pixel points in addition to selected pixels point in second picture frame
Residual pixel point in addition, wherein
The disparity computation unit is further configured to, for each residual pixel point, according to predetermined disparity range, calculate the residue picture
Parallax of the vegetarian refreshments in second picture frame, and
The disparity computation range of each corresponding pixel points is smaller than the predetermined disparity range.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410331089.XA CN105447885B (en) | 2014-07-11 | 2014-07-11 | The method and apparatus for calculating parallax |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410331089.XA CN105447885B (en) | 2014-07-11 | 2014-07-11 | The method and apparatus for calculating parallax |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105447885A CN105447885A (en) | 2016-03-30 |
| CN105447885B true CN105447885B (en) | 2018-09-25 |
Family
ID=55558017
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410331089.XA Active CN105447885B (en) | 2014-07-11 | 2014-07-11 | The method and apparatus for calculating parallax |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105447885B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108171739A (en) * | 2017-12-25 | 2018-06-15 | 清华大学苏州汽车研究院（吴江） | Binocular vision solid matching method based on movable information |
| CN110728710B (en) * | 2018-07-16 | 2023-10-27 | 株式会社理光 | Visual mileage calculation method, device and computer readable storage medium |
| CN113298869B (en) * | 2021-04-23 | 2023-08-04 | 南方电网数字电网科技(广东)有限公司 | Distance measuring method, distance measuring device, computer device, and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102026013A (en) * | 2010-12-18 | 2011-04-20 | 浙江大学 | Stereo video matching method based on affine transformation |
| CN103810690A (en) * | 2012-11-07 | 2014-05-21 | 富士通株式会社 | Stereo matching method and device thereof |
-
2014
- 2014-07-11 CN CN201410331089.XA patent/CN105447885B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102026013A (en) * | 2010-12-18 | 2011-04-20 | 浙江大学 | Stereo video matching method based on affine transformation |
| CN103810690A (en) * | 2012-11-07 | 2014-05-21 | 富士通株式会社 | Stereo matching method and device thereof |
Non-Patent Citations (3)
| Title |
|---|
| Disparity search range estimation：enforcing temporal consistency;Dongbo Min et al;《IEEE International Conference on Acoustics Speech & Signal Processing》;20101231;第2366-2369页 * |
| 一种基于自适应窗口和图切割的快速立体匹配算法;尹传历 等;《光学 精密工程》;20080630;第16卷(第6期);第1117-1121页 * |
| 基于双摄像机的多视点立体成像系统;武国梁;《万方学位论文》;20130731;正文第33页第1段，第51-52页第4.3.2.2节、第56页图4-11 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105447885A (en) | 2016-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA3035298C (en) | Predicting depth from image data using a statistical model | |
| CN106033601B (en) | The method and apparatus for detecting abnormal case | |
| CN109816769A (en) | Scene based on depth camera ground drawing generating method, device and equipment | |
| JP6669063B2 (en) | Image processing apparatus and method | |
| EP2402906A2 (en) | Apparatus and method for providing 3D augmented reality | |
| JP6837279B2 (en) | Equipment and methods for 3D display | |
| CN106327454B (en) | The method and apparatus of composograph | |
| CN111179329B (en) | Three-dimensional target detection method and device and electronic equipment | |
| KR20100119559A (en) | Method and system for converting 2d image data to stereoscopic image data | |
| EP4307233A1 (en) | Data processing method and apparatus, and electronic device and computer-readable storage medium | |
| CN105530502B (en) | According to the method and apparatus for the picture frame generation disparity map that stereoscopic camera is shot | |
| CN108986197A (en) | 3D skeleton line construction method and device | |
| JP4892113B2 (en) | Image processing method and apparatus | |
| CN105447885B (en) | The method and apparatus for calculating parallax | |
| CN105335984B (en) | The method and apparatus for tracking object | |
| CN115272575B (en) | Image generation method and device, storage medium and electronic equipment | |
| CN104915941B (en) | The method and apparatus for calculating parallax | |
| CN111179331A (en) | Depth estimation method, depth estimation device, electronic equipment and computer-readable storage medium | |
| Shidanshidi et al. | A quantitative approach for comparison and evaluation of light field rendering techniques | |
| CN105100768B (en) | Method for Stereo matching and the method for up-sampling | |
| KR20220169472A (en) | Sensor calibration method and device, electronic device and storage medium | |
| KR20220149717A (en) | Full skeletal 3D pose recovery from monocular camera | |
| CN113537194A (en) | Illumination estimation method, illumination estimation device, storage medium, and electronic apparatus | |
| Peng et al. | Projective reconstruction with occlusions | |
| JP2013255172A (en) | Image processing apparatus and image processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |