CN108596012A - A kind of barrier frame merging method, device and terminal - Google Patents
A kind of barrier frame merging method, device and terminal Download PDFInfo
- Publication number
- CN108596012A CN108596012A CN201810052738.0A CN201810052738A CN108596012A CN 108596012 A CN108596012 A CN 108596012A CN 201810052738 A CN201810052738 A CN 201810052738A CN 108596012 A CN108596012 A CN 108596012A
- Authority
- CN
- China
- Prior art keywords
- barrier
- barrier frame
- frame
- intermediate region
- parallax
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
A kind of barrier frame merging method of present invention offer, device and terminal, it is related to assisting driving technology field, this method is to meeting two barrier frames of pre-determined distance threshold value in space coordinates, the quantity of abnormal parallax point based on the intermediate region between barrier frame in disparity map, it is determined whether barrier frame is merged.If the quantity of the abnormal parallax point of intermediate region is more, determine that two barrier frames are not belonging to same barrier, without merging；Conversely, if the negligible amounts of the abnormal parallax point of intermediate region, determine that two barrier frames belong to same barrier, are merged to two barrier frames, obtain target obstacle frame, to promote the accuracy of barrier frame merging.
Description
Technical field
The present invention relates to auxiliary driving technology field more particularly to a kind of barrier frame merging method, device and terminals.
Background technology
When carrying out detection of obstacles to the disparity map based on side, since the effective of barrier regards in the disparity map based on side
Difference is concentrated mainly on the larger object edge pixel of gradient, and row is usually invalid parallax to the smaller pixel of gradient, is led
Cause effective parallax of barrier in disparity map very little, capable discontinuous to parallax.It is less based on effective parallax and parallax is discontinuous
Side disparity map, may by same detection of obstacles be multiple barrier frames, multiple barriers that therefore, it is necessary to go out to Preliminary detection
Hinder object frame to merge processing, the multiple barrier frames for originally belonging to same barrier is made to merge into a barrier frame, to
Obtain accurate detection of obstacles result.
When carrying out the merging of barrier frame, the distance threshold in Z-direction and X-direction is preset respectively first, if two obstacles
The difference of Z-direction distance between object frame and the difference of X-direction distance are respectively less than the distance threshold on respective direction, then by two
Barrier frame merges, and still, the setting of distance threshold is difficult to accurately hold, if setting is too small, can lead to the obstacle that should merge
Object frame does not merge, and setting is excessive, and the barrier frame that merge can be caused to be merged, and therefore, barrier frame merges accurate
Degree is not high.
Invention content
The present invention is not high in order to solve the problems, such as existing barrier frame merging accuracy, provides a kind of barrier frame merging side
Method, device and terminal, to promote the accuracy of barrier frame merging.
For achieving the above object, the present invention provides following technical solutions：
In a first aspect, the present invention provides a kind of barrier frame merging method, the method includes：
With the first barrier frame row to overlapping and arrange to the second misaligned barrier frame in search disparity map；
If the first barrier frame described in space coordinates is exhausted with the difference of the Z-direction distance of the second barrier frame
Pre-determined distance threshold value on respective direction is respectively less than to the absolute value of the difference of value and X-direction distance, is determined in the disparity map
Intermediate region between the first barrier frame and the second barrier frame, and count abnormal in the intermediate region
The quantity of parallax point；
If the quantity of the exception parallax point is less than number of parallaxes threshold value, merge the first barrier frame and described second
Barrier frame obtains target obstacle frame.
Optionally, further include before the quantity of abnormal parallax point in the statistics intermediate region：
Based in the corresponding disparity range of the first barrier in the first barrier frame and the second barrier frame
The corresponding disparity range of two barriers, determines the disparity range of the intermediate region；
If the parallax value of the pixel of the intermediate region does not fall within the disparity range of the intermediate region, the pixel is determined
Point is abnormal parallax point.
Optionally, the quantity for counting abnormal parallax point in the intermediate region, including：
By the intermediate region along arranging to being divided into preset quantity sub-regions；
Count the quantity of abnormal parallax point in the subregion；
If the quantity of the exception parallax point is less than number of parallaxes threshold value, merge the first barrier frame and described
Second barrier frame obtains target obstacle frame, including：
If the quantity of abnormal parallax point is less than the number of parallaxes threshold value of the subregion in the subregion, the son is determined
Region is the normal subregion of parallax；
If the quantity of the normal subregion of parallax reaches preset normal-sub region quantity threshold value in the intermediate region, merge
The first barrier frame and the second barrier frame obtain target obstacle frame.
Optionally, the number of parallaxes threshold value and the abnormal parallax point affiliated area are in the quantity for arranging upward pixel
Correlation.
Optionally, the central point of the first barrier frame is located in the range of the second barrier frame is expert at, and/
Or, the central point of the second barrier frame is located in the range of the first barrier frame is expert at.
Second aspect, the present invention provide a kind of barrier frame merging device, and described device includes：
Search unit, for searching in disparity map with the first barrier frame row to overlapping and arrange to the second misaligned obstacle
Object frame；
Statistic unit, if the Z-direction for the first barrier frame and the second barrier frame described in space coordinates
The absolute value of the difference of distance and the absolute value of the difference of X-direction distance are respectively less than the pre-determined distance threshold value on respective direction, really
Intermediate region in the fixed disparity map between the first barrier frame and the second barrier frame, and described in statistics
The quantity of abnormal parallax point in intermediate region；
Combining unit merges first obstacle if the quantity for the abnormal parallax point is less than number of parallaxes threshold value
Object frame and the second barrier frame obtain target obstacle frame.
The third aspect, the present invention provide a kind of barrier frame merging terminal, including CCD camera assembly, processor and machine
Readable storage medium storing program for executing, the machine readable storage medium are stored with the machine-executable instruction that can be executed by the processor,
The processor is promoted by the machine-executable instruction：Realize a kind of any of the above-described barrier frame merging method.
Fourth aspect, the present invention provide a kind of machine readable storage medium, are stored in the machine readable storage medium
Machine-executable instruction, the machine-executable instruction realize a kind of any of the above-described barrier frame when being executed by processor
Merging method.
By the present invention it can be seen from above description to meeting two barriers of pre-determined distance threshold value in space coordinates
Frame, the quantity of the abnormal parallax point based on the intermediate region between barrier frame in disparity map, it is determined whether to barrier frame into
Row merges.If the quantity of the abnormal parallax point (for example, the parallax value generated based on background) of intermediate region is more, two barriers are determined
Object frame is hindered to be not belonging to same barrier, without merging；Conversely, if the negligible amounts of the abnormal parallax point of intermediate region, determine
Two barrier frames belong to same barrier, are merged to barrier frame, to promote the accuracy of barrier frame merging.
Description of the drawings
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for
For those of ordinary skill in the art, without creative efforts, other are can also be obtained according to these attached drawings
Attached drawing.
Fig. 1 is a kind of disparity map based on side of application scenarios shown in the embodiment of the present invention；
Fig. 2 is the schematic diagram of the initial obstacle object frame detected based on the disparity map based on side shown in Fig. 1；
Fig. 3 is the schematic diagram of the barrier frame obtained after being merged to the frame of initial obstacle object shown in Fig. 2 using the prior art；
Fig. 4 is a kind of barrier frame merging method flow chart shown in the embodiment of the present invention；
Fig. 5 is the schematic diagram for the barrier frame that a kind of disparity map based on side shown in the embodiment of the present invention detects；
Fig. 6 is the schematic diagram of barrier frame A and barrier frame D in a kind of space coordinates shown in the embodiment of the present invention；
Fig. 7 is the schematic diagram of barrier frame A and barrier frame D in a kind of disparity map shown in the embodiment of the present invention；
It includes barrier frame A, barrier frame D and centre that Fig. 8, which is under a kind of application scenarios shown in the embodiment of the present invention,
The disparity map of frame AD；
Fig. 9 be under another application scenarios shown in the embodiment of the present invention comprising barrier frame A, barrier frame D and in
Between frame AD disparity map；
Figure 10 is the barrier frame schematic diagram after barrier frame A and barrier frame D merges under application scenarios shown in Fig. 9；
Figure 11 is a kind of sub-zone dividing schematic diagram shown in the embodiment of the present invention；
Figure 12 is the schematic diagram of the barrier frame after the embodiment of the present invention is merged based on barrier frame shown in Fig. 7；
Figure 13 is the structural schematic diagram of the barrier frame merging terminal shown in the embodiment of the present invention；
Figure 14 is a kind of structural schematic diagram of barrier frame merging logic shown in the embodiment of the present invention.
Specific implementation mode
Example embodiments are described in detail here, and the example is illustrated in the accompanying drawings.Following description is related to
When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment
Described in embodiment do not represent and the consistent all embodiments of the present invention.On the contrary, they be only with it is such as appended
The example of the consistent device and method of some aspects being described in detail in claims, of the invention.
It is the purpose only merely for description specific embodiment in terminology used in the present invention, is not intended to limit the invention.
It is also intended to including majority in the present invention and "an" of singulative used in the attached claims, " described " and "the"
Form, unless context clearly shows that other meanings.It is also understood that term "and/or" used herein refers to and wraps
Containing one or more associated list items purposes, any or all may be combined.
It will be appreciated that though various information, but this may be described using term first, second, third, etc. in the present invention
A little information should not necessarily be limited by these terms.These terms are only used for same type of information being distinguished from each other out.For example, not departing from
In the case of the scope of the invention, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as
One information.Depending on context, word as used in this " if " can be construed to " ... when " or " when ...
When " or " in response to determination ".
It is a kind of disparity map based on side of application scenarios shown in the embodiment of the present invention, from this based on side referring to Fig. 1
As can be seen that effectively parallax value is concentrated mainly on the larger object edge pixel of gradient in disparity map, for example, people in disparity map
Profile on pixel, and row to the parallax value of the smaller pixel of gradient be usually invalid parallax value, for example, remote vehicle
The parallax value of the upward pixel of row be substantially invalid parallax value.If the pixel of effective parallax value of barrier in disparity map
Point is very little, and the pixel of row to invalid parallax value is more, then, can be by same obstacle quality testing when carrying out the detection of barrier frame
It is multiple barrier frames to survey, and therefore, it is necessary to the barrier frames gone out to the disparity map Preliminary detection based on side to merge processing.
Referring to Fig. 2, for the signal of the initial obstacle object frame detected on the basis of the disparity map shown in Fig. 1 based on side
Figure.The prior art is determined whether based on Z-direction distance and X-direction distance of the initial obstacle object frame in Fig. 2 in space coordinates
Merge barrier frame, if for example, the difference of the Z-direction distance of two barrier frames and the difference of X-direction distance are small among Fig. 2
In preset distance threshold, then the two barrier frames are merged, obtains the barrier frame after merging as shown in Figure 3
Schematic diagram.But figure it is seen that two intermediate barrier frames should not be merged, that is to say, that be based only upon Z-direction
The accuracy that distance and X-direction distance carry out barrier frame merging is not high.
Merge the not high problem of accuracy for existing barrier frame, the present invention proposes a kind of barrier frame merging side
Method is one embodiment flow chart of barrier frame merging method of the present invention referring to Fig. 4, which merges barrier frame
Process be described.
Step 401, it searches in disparity map with the first barrier frame row to overlapping and arrange to the second misaligned barrier frame.
Current barrier frame to be combined is denoted as the first barrier frame by the present invention.
In the disparity map based on side, effective parallax value of barrier is concentrated mainly on row to the larger object edge of gradient
Edge, it is largely invalid parallax value to go to the parallax value of the smaller pixel of gradient, that is to say, that barrier has row to parallax
Value is continuous, goes to the discontinuous feature of parallax value, therefore, may merge with the first barrier frame in searching for disparity map second
When barrier frame, which should be expert at the first barrier frame overlaps and upward misaligned in row upwards.
Optionally, the present invention couple the second barrier frame and the registration of the first barrier frame limit, specifically, second
The central point of barrier frame is located in the range of the first barrier frame is expert at, and/or, the central point of the first barrier frame is located at
In the range of second barrier frame is expert at, to increase the possibility that two barrier frames are same barrier, merging is promoted
Accuracy.
It is the barrier that a kind of disparity map based on side shown in the embodiment of the present invention detects for example, referring to Fig. 5
The schematic diagram of frame, the schematic diagram include barrier frame A~barrier frame D, point centered on the black dot in barrier frame, if barrier
It is the first barrier frame to hinder object frame A, is denoted as rectA (x, y, width, height), searched barrier frame is denoted as rectX
(x, y, width, height), wherein x is the abscissa of barrier frame (left upper apex) x-axis in disparity map；Y is barrier
The ordinate of frame (left upper apex) y-axis in disparity map；Width is the width value of barrier frame, and unit is number of pixels；
Height is the height value of barrier frame, and unit is number of pixels.
For searching for the right, the barrier frame on the right side of barrier frame A meets following formula：
RectA.x+rectA.width ＜ rectX.x formula (1)
Wherein, rectA.x represents the abscissa of barrier frame A；RectA.width represents the width value of barrier frame A；
RectX.x represents the abscissa of barrier frame X.
If the registration of barrier frame A and barrier frame X meet following formula (2) and/or formula (3), barrier frame X
For the second barrier frame searched.Specifically,
Wherein, rectA.y represents the ordinate of barrier frame A；RectA.height represents the height value of barrier frame A；
RectX.y represents the ordinate of barrier frame X；RectX.height represents the height value of barrier frame X.Formula (2) indicates barrier
The central point of object frame A is hindered to be located in the range of barrier frame X is expert at；Formula (3) indicates that the central point of barrier frame X is located at barrier
In the range of hindering object frame A to be expert at.
Based on above-mentioned formula it is found that barrier frame D and barrier frame A meet registration requirement, therefore, barrier in Fig. 5
Frame D is the second barrier frame.
Step 402, if the first barrier frame described in space coordinates and the Z-direction distance of the second barrier frame
The absolute value of difference and the absolute value of the difference of X-direction distance are respectively less than the pre-determined distance threshold value on respective direction, determine described in
Intermediate region in disparity map between the first barrier frame and the second barrier frame, and count the middle area
The quantity of abnormal parallax point in domain.
Disparity range based on the first barrier in the first barrier frame (carries out initial obstacle object in the disparity map based on side
When frame detects, barrier in the information of the barrier frame detected, including the coordinate of barrier frame, barrier frame has been got
Disparity range), calculate Z-direction distance of the first barrier frame in space coordinates；Based on second in the second barrier frame
The disparity range of barrier calculates Z-direction distance of the second barrier frame in space coordinates, and the space coordinates are with camera
For coordinate origin, Z-direction distance is the usually signified depth of field, that is, in z-direction apart from the actual range of camera.
If the absolute value of the difference of the Z-direction distance between the first barrier frame and the second barrier frame is less than preset Z
Distance threshold on direction, then illustrate the first barrier frame and the second barrier frame in z-direction at a distance from it is closer, be same barrier
Hinder the possibility of object larger.
The X-direction distance of the first barrier frame and the second barrier frame is calculated, for details, reference can be made to following formula：
Wherein, ZqRepresent the coordinate (i.e. Z-direction distance) of q points Z-direction in space coordinates in space；F is camera
Focal length；P is the actual size of a pixel；PixelqFor the abscissa of q points x-axis in disparity map；x0For disparity map central point
In the abscissa of x-axis；XqRepresent the coordinate (i.e. X-direction distance) of q points X-direction in space coordinates in space.
The X-direction distance that the first barrier frame and the second barrier frame cross can be calculated by formula (4), if the first obstacle
The absolute value of the difference of X-direction distance between object frame and the second barrier frame is less than the distance threshold in preset X-direction, then
It is closer at a distance from illustrating the first barrier frame and the second barrier frame in the X direction, it is that the possibility of same barrier is larger.
For example, with reference to Fig. 6, be in the space coordinates (Z-direction is not shown) shown in the embodiment of the present invention barrier frame A and
The absolute value of the schematic diagram of barrier frame D, the difference of the X-direction distance between barrier frame A and barrier frame D can be by as follows
Formula indicates：
Δ X=| Xa-Xd | formula (5)
Wherein, XaFor the X-direction distance of the right margin of barrier frame A；XdFor barrier frame D left margin X-direction away from
From；Δ X is the absolute value of the difference of the X-direction distance of barrier frame A and barrier frame D.If Δ X is less than in preset X-direction
Distance threshold, then illustrate barrier frame A and barrier frame D in the X direction distance it is closer, may be same barrier.
In the present invention, if the absolute value and X-direction of the difference of the Z-direction distance of the first barrier frame and the second barrier frame
The absolute value of the difference of distance is respectively less than the distance threshold on respective direction, then preliminary judgement the first barrier frame and the second obstacle
Object frame may be same barrier, that is, belong to distance of the two barrier frames of same barrier in Z-direction and X-direction not
It can differ greatly.
After preliminary judgement the first barrier frame and the second barrier frame may be same barrier, position in disparity map is determined
Intermediate region between the first barrier frame and the second barrier frame.Still by taking barrier frame A and barrier frame D as an example, referring to
Fig. 7 is the schematic diagram of barrier frame A and barrier frame D in the disparity map shown in the embodiment of the present invention, wherein be located at barrier
Shadow region between frame A and barrier frame D is intermediate region, is denoted as middle boxes AD, the coordinate rectAD of middle boxes AD (x, y,
Width, height) it can be obtained by following formula：
Wherein, rectA.x represents the abscissa of barrier frame A；RectA.width represents the width value of barrier frame A；
RectA.y represents the ordinate of barrier frame A；RectD.y represents the ordinate of barrier frame D；RectD.x represents barrier frame
The abscissa of D；RectA.height represents the height value of barrier frame A；RectD.height represents the height of barrier frame D
Value；RectAD.x represents the abscissa of middle boxes AD；RectAD.y represents the ordinate of middle boxes AD；RectAD.width is represented
The width value of middle boxes AD；RectAD.height represents the height value of middle boxes AD.
After determining the intermediate region between the first barrier frame and the second barrier frame, count abnormal in intermediate region
The quantity of parallax point.Specifically, first, based on the corresponding disparity range of the first barrier and the second obstacle in the first barrier frame
The corresponding disparity range of second barrier in object frame, determines the disparity range of intermediate region.Still it is with the middle boxes AD in Fig. 7
Example, if the barrier in barrier and barrier frame D in barrier frame A is same barrier, then the parallax of middle boxes AD
Range should be：
Wherein, A.minDisp represents the minimum parallax value of barrier in barrier frame A；D.minDisp represents barrier frame
The minimum parallax value of barrier in D；A.maxDisp represents the maximum disparity value of barrier in barrier frame A；D.maxDisp generations
The maximum disparity value of barrier in table barrier frame D；AD.minDisp represents the minimum parallax in the disparity range of middle boxes AD
Value；AD.maxDisp represents the maximum disparity value in the disparity range of middle boxes AD.
If the parallax value of the pixel of intermediate region does not fall within the disparity range of the above-mentioned intermediate region determined, it is determined that
The pixel is abnormal parallax point, counts the quantity of abnormal parallax point in intermediate region.
Referring to Fig. 8, be under a kind of application scenarios shown in the embodiment of the present invention comprising barrier frame A, barrier frame D and
The disparity map of middle boxes AD, the obstacle from the barrier and barrier frame D that can be seen that in the disparity map in barrier frame A
Object is not belonging to same barrier, the middle boxes AD between barrier frame A and barrier frame D, includes not only invalid parallax value
(black represents invalid parallax value) also includes the parallax value largely generated by background, for example, the parallax value of zebra line position, this
The disparity range of a little barriers being both not belonging in barrier frame A or the disparity range for the barrier being not belonging in barrier frame B
The corresponding pixel of parallax value be abnormal parallax point, that is to say, that two barrier frames for being not belonging to same barrier it
Between generally comprise a large amount of abnormal parallax points.
Referring to Fig. 9, be under another application scenarios shown in the embodiment of the present invention comprising barrier frame A, barrier frame D with
And the disparity map of middle boxes AD, the barrier from the barrier and barrier frame D that can be seen that in the disparity map in barrier frame A
Object is hindered to belong to same barrier (vehicle), the middle boxes AD between barrier frame A and barrier frame D is substantially not present different
Normal parallax point, that is to say, that belong between two barrier frames of same barrier and do not include usually or comprising minimal amount of different
Normal parallax point.
Step 403, if the quantity of the exception parallax point is less than number of parallaxes threshold value, merge the first barrier frame and
The second barrier frame obtains target obstacle frame.
The number of parallaxes threshold value of the predeterminable intermediate region of the present invention.
Optionally, due to barrier have the characteristics that it is near big and far smaller, the intermediate region between corresponding barrier frame
Also have the characteristics that near big and far smaller, i.e., the negligible amounts of pixel, the closer middle area of distance in the intermediate region of distance farther out
The quantity of pixel is more in domain, and since the pixel with effective parallax value in the disparity map based on side is less, this
The quantity correlation for arranging upward pixel of the number of parallaxes threshold value and intermediate region of invention setting intermediate region, tool
Body, the number of parallaxes threshold value that intermediate region can be arranged is the half that intermediate region arranges upward pixel quantity.
If the quantity of abnormal parallax point is less than the number of parallaxes threshold value of intermediate region in intermediate region, illustrate the first barrier
Abnormal parallax point (may lead to abnormal parallax point occur due to matching error) between frame and the second barrier frame seldom, first
The first barrier in barrier frame and the second barrier in the second barrier frame are very big for the possibility of same barrier, because
This, merges the first barrier frame and the second barrier frame obtains target obstacle frame.For example, in middle boxes AD shown in Fig. 9
Abnormal parallax point is few, therefore, should merge barrier frame A and barrier frame D, obtain the barrier after merging as shown in Figure 10
Frame.
If the quantity of abnormal parallax point is greater than or equal to the number of parallaxes threshold value of intermediate region in intermediate region, illustrate first
There is a large amount of abnormal parallax points, the first barrier and second in the first barrier frame between barrier frame and the second barrier frame
The second barrier in barrier frame is not belonging to same barrier, forbids merging the first barrier frame and the second barrier frame.Example
Such as, the abnormal parallax point in middle boxes AD shown in Fig. 8 is more, and therefore, barrier frame A and barrier frame D cannot merge.
Optionally, the present invention can to intermediate region carry out sub-zone dividing, based on subregion judge the first barrier frame with
Whether the second barrier frame needs to merge.Specifically, by intermediate region along arranging to preset quantity sub-regions are divided into, statistics is sub
The quantity of abnormal parallax point in region, if the quantity of abnormal parallax point is less than the number of parallaxes threshold value of the subregion in subregion
(optional, the number of parallaxes threshold value of subregion and the quantity correlation for arranging upward pixel of subregion specifically may be used
The half of upward pixel quantity is arranged for subregion), it is determined that the subregion is the normal subregion of parallax；It counts in intermediate region
The quantity of the normal subregion of parallax is closed if the quantity of the normal subregion of parallax reaches preset normal-sub region quantity threshold value
And first barrier frame and the second barrier frame obtain target obstacle frame；Otherwise, forbid merging the first barrier frame and second
Barrier frame.
Referring to Figure 11, it is a kind of sub-zone dividing schematic diagram shown in the embodiment of the present invention, passes through middle boxes AD in figure
Dotted line is divided into 3 sub-regions, respectively subregion 1, subregion 2 and subregion 3, counts and is regarded extremely in each sub-regions
Not good enough quantity, as shown in table 1.
Table 1
The number of parallaxes threshold value of default subregion is 50, and it is 2 to preset normal-sub region quantity threshold value.As shown in Table 1,
The quantity of abnormal parallax point (100) is more than number of parallaxes threshold value (50) in subregion 1, and therefore, subregion 1 is that parallax is abnormal
Subregion；The quantity of abnormal parallax point (40) is less than number of parallaxes threshold value (50) in subregion 2, therefore, subregion 2 be regarding
The normal subregion of difference；The quantity of abnormal parallax point (60) is more than number of parallaxes threshold value (50), therefore, sub-district in subregion 3
Domain 3 is parallax exception subregion.The quantity for counting the normal subregion of parallax in middle boxes AD is 1, is less than preset normal-sub
Region quantity threshold value (2), accordingly, it is determined that the barrier in barrier and barrier frame D in barrier frame A be not belonging to it is same
Barrier cannot merge.
The present invention concentrates the case where appearing in a certain regional area to reject by sub-zone dividing, by abnormal parallax point, from
And two barrier frames are merged based on the normal subregion of most of parallax is accounted for.
In addition, it can be seen from figure 11 that due to the posture of barrier, it is not all barrier in barrier frame, also wraps
Background containing part, the disparity range of the parallax value of the corresponding pixel of these backgrounds due to being not belonging to barrier, background pair
The pixel answered, which is abnormal parallax point, can count the abnormal parallax point in barrier frame when carrying out abnormal parallax point statistics
Wherein, specifically, from boundary intermediate region (middle boxes AD) adjacent with barrier frame into barrier frame point pixel-by-pixel line by line
Scanning, if the parallax value of pixel does not fall within barrier disparity range, is counted and (adds 1), until scanning is fallen into first
The parallax value of barrier disparity range stops current line scanning.It, can by counting wherein the abnormal parallax point in barrier frame
Promoted judge two barrier frames whether the accuracy that can merge.
By taking the A of barrier frame shown in Fig. 7 and barrier frame D as an example, if barrier frame A and barrier frame D can merge, close
And rear new barrier frame Obs is as shown in figure 12, wherein the dotted line frame in barrier frame Obs respectively represents the barrier before merging
Hinder object frame A and barrier frame D, the coordinate of barrier frame Obs, which is rectObs (x, y, width, height), can pass through following formula
It obtains：
Wherein, rectA.x represents the abscissa of barrier frame A；RectD.x represents the abscissa of barrier frame D；
RectA.y represents the ordinate of barrier frame A；RectD.y represents the ordinate of barrier frame D；RectA.width represents obstacle
The width value of object frame A；RectD.width represents the width value of barrier frame D；RectA.height represents the height of barrier frame A
Angle value；RectD.height represents the height value of barrier frame D；RectObs.x represents the abscissa of barrier frame Obs；
RectObs.y represents the ordinate of barrier frame Obs；RectObs.width represents the width value of barrier frame Obs；
RectObs.height represents the height value of barrier frame Obs.
It can be seen from the above description that the present invention is to meeting two barrier frames of distance threshold, base in space coordinates
The quantity of the abnormal parallax point of intermediate region in disparity map between barrier frame, it is determined whether barrier frame is closed
And if the quantity of the abnormal parallax point of intermediate region is more, determine that two barrier frames are not belonging to same barrier, without closing
And；Conversely, if the negligible amounts of the abnormal parallax point of intermediate region, determine that two barrier frames belong to same barrier, to barrier
Object frame is hindered to merge, to promote the accuracy of barrier frame merging.
The disparity map based on side under a kind of application scenarios with reference to shown in figure 1 carries out obstacle to the disparity map based on side
Object frame detects, and initial obstacle object frame as shown in Figure 2 can be obtained.Larger distance threshold is set according to the prior art, then may be used
Obtain the barrier frame after merging as shown in Figure 3, that is, intermediate two barrier frames are identified as due to meeting distance threshold
Same barrier and be merged.But it is clear from figure 2 that two barrier frames are not belonging to same barrier, do not answer
This is merged, and therefore, the accuracy that existing barrier frame merges is not high.
But with reference to shown in figure 8, Figure 11, the present invention is based on two obstacles to meeting two barrier frames of distance threshold
The quantity of the abnormal parallax point of intermediate region (middle boxes AD) statistics between object frame, if the negligible amounts of abnormal parallax point, recognize
Belong to same barrier for two barrier frames, merges；If the quantity of abnormal parallax point is more, illustrate two barrier frames
Between there are a large amount of background information (for example, zebra stripes in Fig. 8, Figure 11 between barrier frame A and barrier frame D), therefore,
Assert that two barrier frames are not belonging to same barrier, without merging, to promote the accuracy of barrier frame merging.
Figure 13 is the hardware architecture diagram that a kind of barrier frame provided by the invention merges terminal.The terminal 13 includes place
Manage device 1301, the machine readable storage medium 1302 for being stored with machine-executable instruction, CCD camera assembly 1304.Wherein, it handles
Device 1301 can be communicated with machine readable storage medium 1302 via system bus 1303.Also, it can by reading and executing machine
Machine-executable instruction corresponding with barrier frame merging logic in storage medium 1302 is read, processor 1301 is executable to be retouched above
The barrier frame merging method stated.
Machine readable storage medium 1302 referred to herein can be that any electronics, magnetism, optics or other physics are deposited
Storage device can include or store information, such as executable instruction, data, etc..For example, machine readable storage medium can be：
RAM (Radom Access Memory, random access memory), volatile memory, nonvolatile memory, flash memory, storage are driven
Dynamic device (such as hard disk drive), solid state disk, any kind of storage dish (such as CD, dvd) or similar storage are situated between
Matter or combination thereof.
For CCD camera assembly 1304 for acquiring image, which may include at least two cameras, should
At least two cameras can be respectively the left camera of binocular camera and right camera.
As shown in figure 14, it functionally divides, it may include search unit 1401, statistics that above-mentioned barrier frame, which merges logic,
Unit 1402 and combining unit 1403, wherein：
Search unit 1401, for searching in disparity map with the first barrier frame row to overlapping and arrange to misaligned second
Barrier frame；
Statistic unit 1402, if the Z for the first barrier frame and the second barrier frame described in space coordinates
The absolute value of the difference of direction distance and the absolute value of the difference of X-direction distance are respectively less than the pre-determined distance threshold on respective direction
Value, determines the intermediate region between the first barrier frame and the second barrier frame in the disparity map, and unite
Count the quantity of abnormal parallax point in the intermediate region；
Combining unit 1403 merges described first if the quantity for the abnormal parallax point is less than number of parallaxes threshold value
Barrier frame and the second barrier frame obtain target obstacle frame.
Optionally, the barrier frame merging logic further includes：
Determination unit, for based on the corresponding disparity range of the first barrier and described second in the first barrier frame
The corresponding disparity range of second barrier in barrier frame, determines the disparity range of the intermediate region；If the intermediate region
The parallax value of pixel do not fall within the disparity range of the intermediate region, determine that the pixel is abnormal parallax point.
Optionally,
The statistic unit 1402, specifically for by the intermediate region along arranging to being divided into preset quantity sub-regions；
Count the quantity of abnormal parallax point in the subregion；
The combining unit 1403, if being less than the subregion specifically for the quantity of abnormal parallax point in the subregion
Number of parallaxes threshold value, determine the subregion be the normal subregion of parallax；If the normal subregion of parallax in the intermediate region
Quantity reach preset normal-sub region quantity threshold value, merge the first barrier frame and the second barrier frame obtain
Target obstacle frame.
Optionally, the number of parallaxes threshold value and the abnormal parallax point affiliated area are in the quantity for arranging upward pixel
Correlation.
Optionally, the central point of the first barrier frame is located in the range of the second barrier frame is expert at, and/
Or, the central point of the second barrier frame is located in the range of the first barrier frame is expert at.
The present invention also provides a kind of machine readable storage mediums including machine-executable instruction, such as the machine in Figure 13
Readable storage medium storing program for executing 1302, the machine-executable instruction can be executed by the processor 1301 in vehicle identification terminal, to realize
Barrier frame merging method described above.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all essences in the present invention
With within principle, any modification, equivalent substitution, improvement and etc. done should be included within the scope of protection of the invention god.
Claims (10)
1. a kind of barrier frame merging method, which is characterized in that the method includes：
With the first barrier frame row to overlapping and arrange to the second misaligned barrier frame in search disparity map；
If the absolute value of the first barrier frame described in space coordinates and the difference of the Z-direction distance of the second barrier frame
It is respectively less than the pre-determined distance threshold value on respective direction with the absolute value of the difference of X-direction distance, determines and is located in the disparity map
Intermediate region between the first barrier frame and the second barrier frame, and count abnormal parallax in the intermediate region
The quantity of point；
If the quantity of the exception parallax point is less than number of parallaxes threshold value, merge the first barrier frame and second obstacle
Object frame obtains target obstacle frame.
2. according to the method described in claim 1, it is characterized in that, the number for counting abnormal parallax point in the intermediate region
Before amount, further include：
Based on the second barrier in the corresponding disparity range of the first barrier in the first barrier frame and the second barrier frame
Hinder the corresponding disparity range of object, determines the disparity range of the intermediate region；
If the parallax value of the pixel of the intermediate region does not fall within the disparity range of the intermediate region, determine that the pixel is
Abnormal parallax point.
3. according to the method described in claim 1, it is characterized in that, the number for counting abnormal parallax point in the intermediate region
Amount, including：
By the intermediate region along arranging to being divided into preset quantity sub-regions；
Count the quantity of abnormal parallax point in the subregion；
If the quantity of the exception parallax point is less than number of parallaxes threshold value, merge the first barrier frame and described second
Barrier frame obtains target obstacle frame, including：
If the quantity of abnormal parallax point is less than the number of parallaxes threshold value of the subregion in the subregion, the subregion is determined
For the normal subregion of parallax；
If the quantity of the normal subregion of parallax reaches preset normal-sub region quantity threshold value in the intermediate region, described in merging
First barrier frame and the second barrier frame obtain target obstacle frame.
4. method according to any one of claims 1 to 3, which is characterized in that the number of parallaxes threshold value is regarded with the exception
Not good enough affiliated area is in the quantity correlation for arranging upward pixel.
5. according to the method described in claim 1, it is characterized in that, the central point of the first barrier frame is located at described second
In the range of barrier frame is expert at, and/or, the central point of the second barrier frame is located at where the first barrier frame
In capable range.
6. a kind of barrier frame merges device, which is characterized in that described device includes：
Search unit, for searching in disparity map with the first barrier frame row to overlapping and arrange to the second misaligned barrier
Frame；
Statistic unit, if the Z-direction distance for the first barrier frame and the second barrier frame described in space coordinates
The absolute value of difference and the absolute value of difference of X-direction distance be respectively less than the pre-determined distance threshold value on respective direction, determine institute
The intermediate region between the first barrier frame and the second barrier frame in disparity map is stated, and counts the centre
The quantity of abnormal parallax point in region；
Combining unit merges the first barrier frame if the quantity for the abnormal parallax point is less than number of parallaxes threshold value
Target obstacle frame is obtained with the second barrier frame.
7. device according to claim 6, which is characterized in that described device further includes：
Determination unit, for based on the corresponding disparity range of the first barrier and second obstacle in the first barrier frame
The corresponding disparity range of second barrier in object frame, determines the disparity range of the intermediate region；If the picture of the intermediate region
The parallax value of vegetarian refreshments does not fall within the disparity range of the intermediate region, determines that the pixel is abnormal parallax point.
8. device according to claim 6, it is characterised in that：
The statistic unit, specifically for by the intermediate region along arranging to being divided into preset quantity sub-regions；Described in statistics
The quantity of abnormal parallax point in subregion；
The combining unit, if being less than the parallax numbers of the subregion specifically for the quantity of abnormal parallax point in the subregion
Threshold value is measured, determines that the subregion is the normal subregion of parallax；If the quantity of the normal subregion of parallax reaches in the intermediate region
To preset normal-sub region quantity threshold value, merges the first barrier frame and the second barrier frame obtains target disorders
Object frame.
9. a kind of barrier frame merges terminal, which is characterized in that be situated between including CCD camera assembly, processor and machine readable storage
Matter, the machine readable storage medium are stored with the machine-executable instruction that can be executed by the processor, the processor
Promoted by the machine-executable instruction：Realize any method and steps of claim 1-5.
10. a kind of machine readable storage medium, which is characterized in that it is executable to be stored with machine in the machine readable storage medium
Claim 1-5 any method and steps are realized in instruction when the machine-executable instruction is executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810052738.0A CN108596012B (en) | 2018-01-19 | 2018-01-19 | Barrier frame combining method, device and terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810052738.0A CN108596012B (en) | 2018-01-19 | 2018-01-19 | Barrier frame combining method, device and terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108596012A true CN108596012A (en) | 2018-09-28 |
| CN108596012B CN108596012B (en) | 2022-07-15 |
Family
ID=63608500
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810052738.0A Active CN108596012B (en) | 2018-01-19 | 2018-01-19 | Barrier frame combining method, device and terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108596012B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020244414A1 (en) * | 2019-06-03 | 2020-12-10 | 杭州海康机器人技术有限公司 | Obstacle detection method, device, storage medium, and mobile robot |
| CN113378628A (en) * | 2021-04-27 | 2021-09-10 | 阿里云计算有限公司 | Road obstacle area detection method |
| WO2021184663A1 (en) * | 2020-03-19 | 2021-09-23 | 苏州科瓴精密机械科技有限公司 | Automatic working system, automatic walking device and control method therefor, and computer-readable storage medium |
| CN113448329A (en) * | 2020-03-25 | 2021-09-28 | 百度（美国）有限责任公司 | Method and system for reducing obstacles used for planning path of autonomous driving vehicle |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110130874A1 (en) * | 2009-11-27 | 2011-06-02 | Chuan-Ching Tsao | Moving devices and controlling methods therefor |
| CN103123722A (en) * | 2011-11-18 | 2013-05-29 | 株式会社理光 | Road object detection method and system |
| CN105513064A (en) * | 2015-12-03 | 2016-04-20 | 浙江万里学院 | Image segmentation and adaptive weighting-based stereo matching method |
| CN106650708A (en) * | 2017-01-19 | 2017-05-10 | 南京航空航天大学 | Visual detection method and system for automatic driving obstacles |
-
2018
- 2018-01-19 CN CN201810052738.0A patent/CN108596012B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110130874A1 (en) * | 2009-11-27 | 2011-06-02 | Chuan-Ching Tsao | Moving devices and controlling methods therefor |
| CN103123722A (en) * | 2011-11-18 | 2013-05-29 | 株式会社理光 | Road object detection method and system |
| CN105513064A (en) * | 2015-12-03 | 2016-04-20 | 浙江万里学院 | Image segmentation and adaptive weighting-based stereo matching method |
| CN106650708A (en) * | 2017-01-19 | 2017-05-10 | 南京航空航天大学 | Visual detection method and system for automatic driving obstacles |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020244414A1 (en) * | 2019-06-03 | 2020-12-10 | 杭州海康机器人技术有限公司 | Obstacle detection method, device, storage medium, and mobile robot |
| WO2021184663A1 (en) * | 2020-03-19 | 2021-09-23 | 苏州科瓴精密机械科技有限公司 | Automatic working system, automatic walking device and control method therefor, and computer-readable storage medium |
| CN113496146A (en) * | 2020-03-19 | 2021-10-12 | 苏州科瓴精密机械科技有限公司 | Automatic work system, automatic walking device, control method thereof, and computer-readable storage medium |
| CN113448329A (en) * | 2020-03-25 | 2021-09-28 | 百度（美国）有限责任公司 | Method and system for reducing obstacles used for planning path of autonomous driving vehicle |
| CN113448329B (en) * | 2020-03-25 | 2023-12-22 | 百度（美国）有限责任公司 | Method and system for reducing obstacles for planning a path of an autonomously driven vehicle |
| CN113378628A (en) * | 2021-04-27 | 2021-09-10 | 阿里云计算有限公司 | Road obstacle area detection method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108596012B (en) | 2022-07-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108596012A (en) | A kind of barrier frame merging method, device and terminal | |
| US11170466B2 (en) | Dense structure from motion | |
| US6873912B2 (en) | Vehicle tracking system | |
| CN100587538C (en) | Imaging apparatus and control method of imaging apparatus | |
| US10659762B2 (en) | Stereo camera | |
| US8369609B2 (en) | Reduced-complexity disparity map estimation | |
| KR20140000195A (en) | Autofocus for stereoscopic camera | |
| US7327442B1 (en) | Methods and systems of calculating the height of an object observed by a camera | |
| JP3729025B2 (en) | Pedestrian detection device | |
| TWI609807B (en) | Image evaluation method and electronic apparatus thereof | |
| US11889047B2 (en) | Image processing device and image processing method | |
| CN111784885A (en) | Passage control method and device, gate equipment and multi-gate system | |
| CN108182666B (en) | Parallax correction method, device and terminal | |
| CN107977649A (en) | A kind of obstacle recognition method, device and terminal | |
| CN108647579B (en) | Obstacle detection method and device and terminal | |
| CN113188509B (en) | Distance measurement method and device, electronic equipment and storage medium | |
| US9791599B2 (en) | Image processing method and imaging device | |
| CN108256510A (en) | A kind of road edge line detecting method, device and terminal | |
| US10810757B2 (en) | Vehicle exterior environment recognition apparatus | |
| CN108319910A (en) | A kind of vehicle identification method, device and terminal | |
| US9008413B2 (en) | Processing method for a pair of stereo images | |
| CN109472180A (en) | Road image processing method and road image processing unit | |
| CN114866758B (en) | Parallax image filling method and image processing device | |
| JP2020027057A (en) | Stereo camera device | |
| US11460582B2 (en) | Vehicle exterior environment monoscopic and stereoscopic based detection apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |